Jonah KowallWe have published the inclusion criteria, and the detailed evaluation criteria here : http://www.gartner.com/resId=2328015
The evaluation criteria have been enhanced in terms of the detail we published this year to provide better visibility of how the products are graded.
The note outlines reduced interest and investment in synthetic end user experience monitoring tools, as real user monitoring tools continue to become more mainstream and well adopted. This is a trend which has happened over the last few years, but is increasing as the options and cost of real user monitoring has decreased dramatically. The one area this has not happened is mobile APM, where synthetic is the current state of the art. This will be changing throughout 2013, as this year real user and real device mobile APM is adopted by customers. This is a similar transition web applications have gone through over the past several years.
Some areas of note within the document is the increased importance of analytic technologies, these are not the tools which allow you to build reports and filter data, but actually analyze and provide suggestions and insight into what is being collected. In my discussions with vendors every company seems to do Cloud, Big Data Analytics, and SDN. These terms are washed out to the point of being meaningless without context. Please read Will Cappelli’s research and insight in analytics.
Category: Analytics APM Big Data Monitoring Tags:
9 responses so far ↓
1 Simon Levin March 1, 2013 at 5:54 pm
Jonah,
Per our blog on this matter http://wp.me/p2cCRk-tU this is NOT a level playing field so long as the criteria details are only available for clients. The position right now implies a clear bias by Gartner to clients when it comes to assessments.
2 Jonah Kowall March 1, 2013 at 6:37 pm
Thanks for posting about that, I enjoy dispelling rumors which other firms spread about the pay to play that many of them have. We include non-paying vendors all the time in research including in this very magic quadrant. Additionally vendors who are hostile to us, and upset for whatever reason, may refuse to speak with us at our request are also included. It’s not the vendors decision to be dropped from research, its the analysts decision. If the vendor is not a client, I was quite clear in my blog post, that they contact me via twitter, or email if there are questions. I will reply to everyone who contacts me, it’s a minimum courtesy I believe strongly in. I am happy to share the inclusion criteria if they feel they qualify.
In response to the changes in the research from your blog, we removed the AA-NPM products which can in fact accomplish 3 out of 5 dimensions in the model. We built a vendor landscape last year for those products, which normally have a different buyer than APM tools. I still often recommend AA-NPM products for APM use cases since they are often easier to deploy, although more limited. Those more limited products are also not comparable to the market leading APM technologies, thus moving them to other research was a better course of action. Look for additional branded research in the network space in 2014.
If a vendor has contacted you who feels wronged and has the fact behind them they should contact the analyst directly, or if that fails use the Ombudsman’s office, which is a impartial way to deal with incorrect research.
Thanks for the commentary and the blog post, even if it’s factually inaccurate
3 Analysts in the Press, 2/26/2013 | Thomas Ward Lynch March 2, 2013 at 2:58 am
[...] 2013 – Application Performance Monitoring (APM) – Gartner Blog … Jonah Kowall is a Gartner research director in the IT Operations area. He focuses on application performance management, runbook automation, event … blogs.gartner.com/…/2013-application-performance-monitorin… [...]
4 Oops, Gartner Tilted the Playing Field Again « The Skills Connection March 4, 2013 at 8:21 am
[...] Well, part of the answer lies here, in Jonah’s blog entry from 22 February. [...]
5 Jonah Kowall March 4, 2013 at 1:27 pm
I have been contacted already by non-vendor clients, but not due to your blog post Simon. They were already in touch since the questionnaires went out to possible participant companies on Friday (3/1). I am very clear on twitter in regards to non-vendors being in research, which happens very regularly in what I publish.
The only issue I see is that non-clients don’t brief or talk with analysts as often. We also tend to see them mis-aligned with what client demands are; hence they don’t do as well in terms of market execution (being on the client short lists).
I’ll keep this updated if anyone new contacts me, thanks for the correction on the wording Simon.
6 Simon Levin March 4, 2013 at 4:43 pm
Glad to be of help.
In my 10 years at Gartner we didn’t have published notes of criteria which could disrupt impartiality between clients and non-clients. These are clearly pieces of real value – but they must always be available to all relevant suppliers if they are published before the assessment cycle starts.
The minute anyone at Gartner starts offering better quality information to clients to prepare than non-clients then that is the day the company’s reputation for impartiality goes forever. This is an incredibly important core value which you are risking or protecting by your actions.
7 Jonah Kowall March 8, 2013 at 7:29 pm
Well the prior way it was handled was that nothing was published prior to a magic quadrant. This method is an improvement. I’m working with the methodology department on figuring out a way to fully correct visibility with regards to kicking off a magic quadrant and making criteria public early in the cycle.
8 Eyal March 27, 2013 at 11:53 am
Is there a planned date for the release of Magic Quadrant for APM 2013?
Thanks in advance,
Eyal
9 Jonah Kowall March 27, 2013 at 1:11 pm
These are published on our website: http://www.gartner.com/technology/research/methodologies/magicQuadrants.jsp
The APM MQ is scheduled for Q3 this year (and every year).
Leave a Comment