Gartner Blog Network

Oh, Michael — Your Rant ….

by French Caldwell  |  October 10, 2012  |  1 Comment

Analyst Having a Major Rant

Dear Michael —

Good to hear from you.  Thanks for sending me your latest blog post.  I have to say though that when you rant you really do go ballistic — you want to throw six months of my work on the compost pile — ooh, that hurts.  But I get it, it’s not me; you just don’t like magic quadrants.

Despite the negativity, you’ve got some interesting points — some valid, some inflated.  So let’s take a look at your points:

1 — “There is no transparency or clarity on how vendors are scored.”


There are 12 criteria in the Magic Quadrant for Enterprise GRC Platforms.  They are described in the research note, and the weightings are given as well.  A client could take these criteria and do their own magic quadrant if they wanted.  I’ve even had audience members follow the methodology during my presentations and score vendors in real time.  It’s amazing how close they come to the analysts’ scores.

For more on how magic quadrants are created, see: “Magic Quadrants and MarketScopes: How Gartner Evaluates Vendors Within a Market”

2 — “The current Magic Quadrant is a mile wide and an inch deep.”


My colleague John Wheeler and I worked for over six months with input from 19 vendors and 211 customer references to create an evaluation that is concise and readable.  When using the magic quadrant, analyst advice can help clients to extract the details for each enterprise’s particular requirements, and to develop a decision framework for choosing which vendors are most appropriate for a given enterprise to consider — including vendors who are not leaders or who may not be in the magic quadrant itself.

Based on your rant you must not know this, but for the last two years we’ve followed up the magic quadrant with critical capabilities notes providing more details on how vendors rank in core use cases like audit management, enterprise risk management, financial reporting integrity compliance, and policy management.   However, the number of use cases is growing rapidly — case management, regulatory change management, vendor risk management, anti-bribery compliance, privacy compliance, sustainability reporting, IT risk management, etc., are all use cases that we commonly see, and there are more.   So this time, instead of critical capabilities notes, we’re following up with additional notes on audit management, vendor risk management, transactional controls monitoring, ERM applications and many more.

To meet the demand for the breadth and depth of risk management and compliance analysis, Gartner has enlarged our risk management team and our industries and supply chain teams have expanded their risk management and compliance research.

To see some of the new focus on audit, take a look at Khushbu Pratap‘s research, and for enterprise risk management take a look at what Paul Proctor is doing with risk value management.  We’re also expanding into related legal IT areas — besides e-discovery where we’ve had a strong research base for sometime thanks to Debra Logan and Shelia Childs John Wheeler is addressing Enterprise Legal Management.  And we have analysts leading research communities for privacy, Carsten Casper, and BCM, Roberta Witty.

Oh, gosh, I shouldn’t forget that Jacqueline Heng and John Wheeler are tracking the consultancies that support GRC services and have a Marketscope for Global ERM Consulting Services.  Man — so much stuff ….

I could go on — but you get the point, right Michael — there’s an ocean of depth under that inch of magic quadrant.

3 — “Gartner has a script and gives a vendor a short time period to demo their GRC product to Gartner.  They do not allow you to go off script – I have heard this from multiple vendors frustrated with the process. “


For two hours of one of 365 days in the year, we hold the vendors to a demonstration script.  The vendors have the other 364 days of the year to communicate to us all the breadth and depth of the capabilities that they have and those are considered in the magic quadrant.

A vendor does not have to be a Gartner client to do a vendor briefing.  They talk, they show us what they do, and we listen to them the  rest of the year — but for two hours of one day, they follow our script.  Michael, is that really unreasonable?

When establishing a baseline of capabilities for evaluation, we compare apples to apples.  But the demo script is fairly open — any vendor who can’t find a place in it to demonstrate their best differentiators is slipping up.

By the way, we’ll share that script with Gartner clients who want to build their own customized script.  We tell them to have the vendors follow their customized script first, but then to give the vendors plenty of time afterwards to demonstrate additional capabilities that were not in the script — just as we encourage the vendors to demonstrate their capabilities throughout the year — most take advantage of that open door, some don’t.

Everything we learn about the vendors over the course of the year is considered in the magic quadrant evaluation — not just the two hour demo, or the vendor questionnaire.

4 — “I also take issue with how Garter defines and presents the GRC market.  While they give lip service to a lot of areas of GRC throughout the document they assume that an EGRC platform is comprised of only the four categories of risk management, audit management, compliance and policy management, and regulatory change management. “

Heavily inflated

On one hand, Michael, you’re mixing markets with the core functions on the platform, and on the other hand with broad categories of related markets — what we call a marketplace.  The Enterprise GRC Platform is just one of many GRC-related markets.

The Gartner “Hype Cycle for GRC Technologies” lists over three dozen technology markets.  And then there’s the “Hype Cycle for Legal and Regulatory Information Governance” that lists over two dozen technology markets.  For more background on how we view the broad GRC marketplace, take a look at “A GRC Marketplace Comparison Model, 2011 – 2013.”

You’ll find, Michael, that you and I are not so different in our points of view on the elements of the GRC marketplace — but with dozens of contributing analysts, Gartner just has loads of depth.

5 — “Gartner states that there are many businesses implementing a single EGRC platform.”


Michael  — where’s the integrity, the transparency.  That’s selective quoting.  Here’s the full sentence from the magic quadrant (italics added): “Instead of acquiring separate solutions for finance, IT and other business units, many enterprises are choosing to use a single EGRC platform and, when necessary, integrating the many point and functional solutions to satisfy specific GRC needs.

So yes, there are often many solutions for specific compliance and risk management needs, but lots of enterprises are using the platform to get that holistic top level enterprise view for the board and senior executives.  Otherwise, it’s not really enterprise, is it?

Okay, Michael — I hope this helps.  I haven’t seen you in a few months, and I look forward to seeing you soon.  I’d like to talk to you about my new GRC 4G concepts.

Cordially — French

PS — Remember the time in Rio when our hotel was taken over by an armed drug gang.  Who’d have put that in their risk assessments?  Fortunately we had skipped town!  See ya on the circuit, buddy.




Additional Resources

View Free, Relevant Gartner Research

Gartner's research helps you cut through the complexity and deliver the knowledge you need to make the right decisions quickly, and with confidence.

Read Free Gartner Research

Category: applications  compliance  grc  risk-management  

Tags: compliance  grc  risk-management  

French Caldwell
VP and Gartner Fellow
15 years at Gartner
19 years IT industry

French Caldwell is a vice president and Gartner Fellow in Gartner Research, where he leads governance, risk and compliance research. Mr. Caldwell also writes and presents on knowledge management. His research includes analysis of the impact… Read Full Bio

Thoughts on Oh, Michael — Your Rant ….

  1. […] a blogged response to Rasmussen’s post, Gartner Fellow and vice president French Caldwell says he considers […]

Comments are closed

Comments or opinions expressed on this blog are those of the individual contributors only, and do not necessarily represent the views of Gartner, Inc. or its management. Readers may copy and redistribute blog postings on other blogs, or otherwise for private, non-commercial or journalistic purposes, with attribution to Gartner. This content may not be used for any other purposes in any other formats or media. The content on this blog is provided on an "as-is" basis. Gartner shall not be liable for any damages whatsoever arising out of the content or use of this blog.