The Summer of Testing Research

We have kicked off our next round of research for the software testing tools market.  We are close to publication on a new Market Guide for API Testing and Service Virtualization and are going to continue expanding the notes we have to cover the overall market.  In the past we have published a Magic Quadrant for Integrated Software Quality Suites.  This document was designed to give a view of the enterprise ready providers of suites of tools designed to cover “all” of your testing needs.  It was initially focused on Test Management, Functional Test Automation, and Performance Testing.  Over time we included elements like SOA testing, code quality, unit testing, etc. into the elements of what makes up the Breadth of Vision axis.

Now, with both an expanded team and an expanding variety of software to test, we are going to expand the number of documents we produce as well as shifting and tightening the focus of the MQ.  The MQ will change from being Integrated Software Quality Suites to the MQ for Software Test Automation.  Thus is will focus on functional test automation and be targeted primarily at GUI layer test automation (API/Service level testing being covered in the soon to be released Market Guide).  We will also produce a critical capabilities document for this market and while still tuning the use cases are looking at: Package Application Testing (the back office), Responsive Web (the new desktop), and Mobile (field and customer).  We believe this will better address the inquiry flow that we typically have than the current document.

In conjunction with this we will also take the major pieces removed from the current MQ: Peformance/Load testing and test  management and move these into separate Market Guides.  Thus by the end of the year we will have four Market Guides:

1. Mobile/Device Management
2. API Testing and Service Virtualization
3. Performance Testing
4. Test Planning and Management

as well as the restructured Magic Quadrant for Software Test Automation and a Critical Capabilities for Test Automation.

We are currently in the kick-off phase updating our vendor/product lists, firming up the use cases and designing the surveys we will use for vendors and users.  We are also looking to produce updates and/or new documents on test automation, QA team structures (testing and bimodal IT and agile/devops), and coordinate with our security and UX teams around the broader view of software quality.  If there are specific themes or issues you would like to see addressed in this research please either leave responses to this doc, connect to us for a client inquiry or message me on twitter (@metamurph).

Note that our documents on tools are focused on commercial companies as Magic Quadrants illustrate the capabilities of different vendors to service the needs of enterprise IT organizations.  Thus open source will only appear in lists or where it has a commercial offering and we are evaluating the provider. Our goal is to complete the majority of this work by the end of Summer.