Gartner Blog Network

Behind the Scenes with the New BI & Analytics Magic Quadrant

by Cindi Howson  |  February 10, 2016  |  11 Comments

By Cindi Howson

The 2016 BI and Analytics Magic Quadrant published last week and it seems the twitter-blogosphere is on fire with some positives, negatives, and gasps over a dramatically different dot picture over previous years.  This also was my first end-to-end participation in the Magic Quadrant process so I wanted to share some of what surprised me.

1.      New Inclusion Criteria

The single biggest change for this year over previous years is in the definition of what constitutes a BI and analytics platform.  This year, only products considered modern as defined in this note were evaluated for this Magic Quadrant. Any BI and analytics platform not considered modern went into a Market Guide for IT-Centric Reporting Platforms (due out shortly in final editing). Truth be told, many of us authors would have liked to call this a Market Guide for traditional BI platforms, or a Market Guide for production reporting-adhoc query-IT authored dashboards-OLAP.  But between political correctness, clarity, and brevity, we landed on IT-centric Reporting.

So if you are wondering why product A, vendor B and so on are not on the Magic Quadrant, this change in inclusion criteria is the first thing to consider.

2.      Same Old Gating Criteria

Beyond that, we are constrained to 24 vendors above a certain revenue threshold. There are many more modern BI platforms we are tracking and listed in the Magic Quadrant but that didn’t warrant a dot position.

3.     “But Those Magic Quadrant Leaders Aren’t Market Share Leaders!”

Market share has always been a hard thing to define, depending on what you count. Revenues may be quantifiable but for what? For reporting tools? Modern BI tools? Net new buying? Maintenance? What about when a vendor bundles software licenses in with an ERP purchase and those products never actually get deployed?  Thankfully, this is something that Jim Hare, Alys Woodword, and Bhavish Sood track in excruciating detail. They are planning to try to split out the IT-centric revenues from the modern BI platform revenues in their 2015 market share analysis.

As we wrote in the Magic Quadrant note, a lot of the net new buying has been going to the Magic Quadrant leaders.  Some buying has been going to niche and visionary vendors, depending on customer requirements.  Leadership in the Magic Quadrant has less to do with market share and more to do with the combined scores on ability to execute and completeness of vision.

4. “But They Have the Best Product!”

This remains the biggest fallacy of the Magic Quadrant and misuse of the Magic Quadrant.  Best dot position does not equate to best product. Product is just one input among many.

As a new Gartner analyst, it was an interesting process for me to see the extensive scoring from both analyst opinion and customer opinion that feeds the Magic Quadrant model. We take all this data and see where the dots land, sometimes confirming what we knew intuitively, and in other cases, coming as a surprise to us analysts.

Through the whole process, it’s been helpful to me to keep a cheat sheet on my desk as to what really drives each axis:

Completeness of Vision Ability to Execute
•       Market Understanding*

•       Offering (Product) Strategy

•       Innovation

•       Marketing Strategy

•       Vertical/Industry Strategy

•       Sales Strategy

•       Geographic Strategy

•       Customer Experience*

•       Operations*

•       Product or Service* (Critical Capabilities)

•       Overall Viability

•       Sales Execution/Pricing*

•       Market Responsiveness and Track Record

The bold items with an asterisk are based on customer opinion. Analyst opinion and customer opinion sometimes differs. Rita has called happy customers as having a halo effect: they over rate the vendor on product abilities and tech support through a romantic lens.  We can’t think of a politically correct term for the unhappy customers whose anger over bad technical support and price gouging may lower scores on everything else.

So great product, with limited global support (geography) does not make a leader. Awesome product vision and poor customer experience also doesn’t make a leader.

If you really want to know who has the best product, then you need to look at the Critical Capabilities note, which focuses exclusively on product.  (The new Critical Capabilities is due out later this month, so here is a link to last year’s).

And by the way, for Gartner customers, both the Magic Quadrant and Critical Capabilities are interactive so you can customize your own weights. You could even set the Magic Quadrant criteria to be entirely based on product and see where the dots land. If you read either of these notes as a reprint, you are seeing a static version. I would love for us to figure out a way to change that.

5. “You Killed Vendor X!”

I know too well that vendors live and die by their dot placement. Kurt warned me that in joining Gartner, I would go to bed dreaming of dots. He was right. All six of us authors (and no doubt every Gartner analyst) feel a huge responsibility to get this right. We are happy for the ones that do well, and feel the pain of the ones who don’t. But it’s our jobs to inform, you, our customers, no matter the consequence on a vendor. This is just one tool to aid customers in buying decisions. For the vendors that are not happy with their placement, it’s not our fault. In a way, we are just the bearer of bad news, whether of dissatisfied customers or of changing market dynamics that can make a once good product now inferior.

It reminds me of a speech I recently heard from NY Times columnist Tom Friedman to incoming students at the University of Miami Students. “Average” is not enough to compete on a global basis. You have to be great. Jim Collins has a similar mantra:  good is the enemy of great.

As my colleague and lead Magic Quadrant author Josh Parenteau sums it up: “There are some people who think Blockbuster will make a comeback and taxis will prevail in the war with Uber.  The smarter IT folks will use the new MQ as a license to change for the better.”

So I hope if you have questions about this year’s Magic Quadrant and your BI and analytics investments, I hope you will schedule an inquiry with any one of us. We also have three BI and analytics summits coming up in the next month to help you navigate this area further (Sydney, London, Dallas). Join us as well this Thursday (2/11) for a real-time tweet chat on Twitter at 12 EST using hashtag BIAMQ.  We also are hosting a webinar on March 8. In short, we are here for you, any way you want to connect!






Additional Resources

100 Data and Analytics Predictions Through 2024

Gartner’s annual predictions disclose the varied importance of data and analytics across an ever-widening range of business and IT initiatives. Data and analytics leaders must consider these strategic planning assumptions for enhancing their vision and plans.

Read Free Gartner Research

Category: business-analytics  business-intelligence  data-and-analytics-strategies  

Cindi Howson
Research VP
1 years at Gartner
25 years IT Industry

Cindi Howson is a Research Vice President at Gartner, where she focuses on business intelligence (BI) and analytics. Her work includes writing about market trends, vendors and best practices and advising organizations on these subjects. Read Full Bio

Thoughts on Behind the Scenes with the New BI & Analytics Magic Quadrant

  1. […] to the Gartner 2016 Magic Quadrant for BI and Analytics report, I am delighted with our results. The process is extremely thorough. The annual RFI exercise started one month after our July 2015 GA release when it seemed like […]

  2. Hakan says:

    Thanks for your information….As a consultant I work with Qlikview, Qliksense, Tableau, Microsoft PowerBI and OBIEE. My clients works with OBIEE as a selfservice/enduser platform They change parameter, groups, comments and filters. Isnt OBIEE answer a superuser or enduser platform?

    • Cindi Howson says:

      Thank you, Hakan. OBIEE is indeed a ad hoc query platform – as are many others such as IBM Cognos and SAP BusinessObjects Web Intelligence. The primary difference between these tools and the ones in the MQ are that the former require IT to initially model a meta data layer. We like semantic models for re-usability and control, but in a modern BI and analytics world, we would expect them to be created automatically, intelligently, and optionally. This is explained in this blog and in the note I referenced above: I think this note is available as a reprint somewhere if you google around.

  3. Ted werner says:


    I thought you guys were fair on your evaluation. In my new role I have had the luxury of using many different BI products. What really makes it hard is each vendor is very good at certain things, but other items that are not their core competency they are ok with a “C”. What happens is many companies have taken a stance that truly there is a right tool for the job and they buy two or three tools. For large text indicators and dials a user may not pick tableau. For interactions a company may not pick Powerbi based on their choice for interaction models. I believe the magic quadrant is important, but what it comes down to is what are the 5-10 features an analyst will use daily. If a vendor can get most of the 10 items, then they should be considered for evaluation regardless of their position in the quadrant. I just hope people don’t eliminate contenders based on Gartner scores. There are contenders that might be well suited to get a job done despite not being in the leaders quadrant.

    • Cindi Howson says:

      Thank you, Ted. I agree with your comments and often advocate the “right tool for the right user and use case.” This is a theme of some of the sessions I am speaking on at the BI summits. Also, it is very clearly demonstrated in the Critical Capabilities report in which customers can set their own weights for how important a capability is. I LOVE this interactive version Gartner offers but unfortunately, I don’t think enough of our clients use it. Maybe there are just too many who are used to the static PDF view? If I find a video of how it works, I will post a link here.

  4. Bruno Aziza says:

    Nice job Cindi. I can’t imagine that it was an easy conversation with the vendors involved this time. Thanks for the clarifications you bring in this post.

    I also liked the section in the report that talks about the bi-modal procurement mode. What happens to a market when twice as many people participate? It grows!

    There is an incredible amount of potential in this market and that’s how I think your readers should internalize this report!

    Analytically Yours,
    Bruno Aziza

    • Cindi Howson says:

      Thanks, Bruno, the redefinition of the MQ was a multi-year effort and one not taken lightly but very much in response to customer demands and market dynamics. I wonder if the NFL went through this much angst in changing the 2-point conversion rule?

  5. […] Platform and a traditional, IT-centric Reporting and Analysis Platform”. I think this; ‘Behind the Scenes‘ gives some more background about how Gartner has put this Magic Quadrant for Business […]

  6. […] following is an excerpt from a recent post in the Gartner Blog Network, authored by Cindi Howson and published on February 10, […]

  7. […] which vendors offer the best solution for your specific use case. For more details, read this blog post by Gartner Research VP Cindi […]

Leave a Reply

Your email address will not be published. Required fields are marked *

Comments or opinions expressed on this blog are those of the individual contributors only, and do not necessarily represent the views of Gartner, Inc. or its management. Readers may copy and redistribute blog postings on other blogs, or otherwise for private, non-commercial or journalistic purposes, with attribution to Gartner. This content may not be used for any other purposes in any other formats or media. The content on this blog is provided on an "as-is" basis. Gartner shall not be liable for any damages whatsoever arising out of the content or use of this blog.