Gartner Blog Network


A Rant – My Integrity as an Analyst

by Tom Bittman  |  October 8, 2009  |  56 Comments

I need to rant a little.

As an analyst at Gartner, I can’t describe how angry I get when I read bloggers spouting as “fact” their opinion that I and my teammates have no integrity. That we can be “bought.”

In my 14+ years at Gartner, I have never, ever allowed a vendor to influence my opinion with anything but facts. Period. They have certainly tried to influence me with non-facts. I can say this definitively – it has never worked.

I don’t think there is a single vendor that I have dealt with who has not been very angry with me at some point. Tough. I’ve been yelled at by many IT executives – including the CEOs of Microsoft and HP, and many other firms. I can’t think of one of those cases when I changed my analysis one bit. I can’t speak for other firms, but at Gartner, getting yelled at by a CEO is a badge of honor. Being proven right as time goes on – priceless

I certainly spend time helping vendors with their strategies and their marketing messages – and I enjoy doing it. Frankly, the ones who yell at us the most seem to respect our opinion the most. We can spot holes a mile away, and engaged early enough, we can help vendors fill those holes with real product offerings – that not only help the vendors, but help our end user clients. And my primary business is helping end users.

I worked at IBM for 11 years. During the last few years, I was doing some technical evangelizing to analyst firms – and hiring these firms to write white papers. There were plenty available for hire – and still are. There was only one firm that wouldn’t allow us to edit their work – Gartner. Only one. We tried, but they wouldn’t budge. They could not be bought. It was unusual, and I completely respected that. When the opportunity came to join Gartner, I jumped. I wouldn’t have even considered anyone else.

Since I’ve joined the firm, my appreciation for the integrity of the firm and its analysts has only increased. If I felt that the firm’s integrity was bending, first I would fight it with all my might, and then I would be outta there. I’m not gone yet!

So I understand the impression in the marketplace that analyst firms can be bought. But that’s not where I work. My integrity is very important to me. I’m sure we’ll continue to make enemies of vendors, and bloggers who have a vested interest in one thing or another. Badge of honor! But my goal is to provide value to my clients, and to be proven right over time – priceless!

End of rant.

Additional Resources

View Free, Relevant Gartner Research

Gartner's research helps you cut through the complexity and deliver the knowledge you need to make the right decisions quickly, and with confidence.

Read Free Gartner Research

Category: industry-analyst  

Tags: analyst  

Thomas J. Bittman
VP Distinguished Analyst
20 years at Gartner
31 years IT industry

Thomas Bittman is a vice president and distinguished analyst with Gartner Research. Mr. Bittman has led the industry in areas such as private cloud computing and virtualization. Mr. Bittman invented the term "real-time infrastructure," which has been adopted by major vendors and many… Read Full Bio


Thoughts on A Rant – My Integrity as an Analyst


  1. […] This post was mentioned on Twitter by Erik Kingham and Avnet SolutionsPath. Avnet SolutionsPath said: Gatner A Rant – My Integrity as an Analyst: I need to rant a little. As an analyst at Gartner, I c.. http://bit.ly/1BBVZI […]

  2. Anthony Bradley says:

    Here. Here. People disparaging Gartner on being “bought” need to realize they aren’t talking about the logo. They are talking about us, the analysts. I immediately wonder about their motives and why they are so angry?

    I also love the part about us not having “touched code in ages.” I’ve written a bunch of code in my day, some pretty recently. You know what I never got from writing code? A broad view of technology. Why would I write code to evaluate a technology when I can talk to 100 clients who are doing it and get the benefit of their experiences. It is a much more leveraged model. And that is what we provide.

    Sure, we listen to a bunch of vendors and clients and advise a bunch of vendors and clients, with a boat load of investigation and analysis in between. Remind me again what the problem is here? If you think it is easy to ingest an immense and continuous stream of information, make sense of it, spot patterns, draw conclusions, present findings and defend them in at least 20 client meetings a week then you better keep to writing code because being an analyst would be a rude awakening.

    OK, now I’m done ranting.

  3. Rick Brusuelas says:

    Tom, many kudos speaking up. I have worked with a number of Gartner analysts over many many years and each and every one demonstrated the utmost integrity. The advice, feedback, and challenges that Gartner analysts have shared with me and the vendors I have worked with have tested and improved the strategies and offerings. I have been fortunately to have worked directly with Tom on numerous occasions over many years and his counsel has been invaluable.

    At the same time, Tom’s rant speaks up for many other analyst firms and individual analysts that also share Tom’s passion for helping vendors better meet (and exceed) end-user requirements.

  4. I worked for Gartner for 10 years.
    Many times vendors wanted to influence or change the benchmarking reports I wrote regarding price and SLA on outsourcing contracts, and I was very lucky to have top management support me, backing me up, so that I could maintain my findings and recommendations to renegotiate contracts.
    Today I am a client, I pay for Gartner services because integrity matters!
    Integrity in my asset, it’s not for sale, and it’s a Gartner value.

  5. Like to echo my sentiments here. The problem only occurs when a few bad apples spoil what I consider to be one of the most trusted industries going. There is a reason why the Edelman Trust Barometer consistently puts analysts as the highest profession when it comes to trust.

    However, successful analyst houses will only remain so provided they remain impartial. Where the ‘very few’ firms have fallen foul of that – I am delighted to see that they very quickly fall off the map – afterall it is your independence that counts.

  6. Tom, as for your former colleague and now a blogger I guess I see both sides. If you think it is just bloggers raising the issues, you are not reading the market, including your customers well.

    The issue is not at an individual analyst level. It is at the firm level. You know it and I know it that vendor sourced revenues have been growing nicely as percentage of Gartner revenues. You and I know the majority of top 25 Gartner revenues are vendors like IBM – and individually they dwarf what even the biggest user entities like GE or BP pay you.

    It would help if Gartner was transparent about that, rather than avoid talking about it, and if Gartner aggressively showcased how those vendor interests and subtle and not so subtle pressure from them is balanced with user inteersts. The answer usually is the Gartner Ombudsman blog does that. Seriously?

    The other thing is 2 of Gartner’s best known tools – the MQ and the Hype Cycle are skewed towards larger vendors and against startups. The MQ rewards market share and viability or tenure in a market. By its very definition the Hype cycle warns people about newer stuff. So Gartner comes across as pro-establishment with more skew towards larger, incumbent vendors

    In the field I see time and time again vendors using your metrics where it suits them. I see economics that are dated. In a consulting assignment last year, the CIO asked me fairly loudly during a presentation when a vendor put some of your metrics “so is that 30 or 40% over priced?”

    Finally, when I joined Gartner in 1995 I was told of an episode when David Stein proactively called a CIO – I think of an insurance co and pulled him out of a meeting with IBM and telling him not to sign a contract because pricing would go down significantly in a couple of weeks.

    My question is how many Gartner analysts do that today? A proactive, “customer is God” POV?

    Most CIOs would love to see that Gartner again. Then you cold tell the pesky bloggers to go pound sand!

  7. Lydia Leong says:

    Fantastic post, Tom.

    Vinnie,

    I’m currently an analyst at Gartner, and I certainly do proactively contact clients with whom I’ve had recent contact, and where a change in the market or a change in the information I have available means that I ought to change the advice that I’ve given them.

  8. Tom Bittman says:

    Vinnie, I respect the concern about how Gartner balances vendor interests with user interests. But this is basic logic. The reason we have vendor clients is because we have user clients who trust us. If we did anything to change that trust, we lose both. It is in our best interest as a business to maintain our integrity. It’s a fact that we work even harder at it now than when you were with Gartner.

    All that really matters is where the rubber meets the road. As I said in my post, if I ever felt management trying to bias me for vendor clients, I would first fight it, then leave. I’m telling you that I am not under any kind of pressure to be biased for vendor clients.

    Your statement about hype cycles and MQs is not accurate. Hype cycles are all about calling out new technologies, and almost all of them come from start-ups. Do we caution on new technologies? Of course. But, we also elevate new stuff that we think is important. Specific example – when VMware was a 40-person company in 2001 I wrote that they had a killer app that would change the market. Microsoft was furious that I devoted a slide in a presentation about Windows Futures to VMware beginning in 2002. Didn’t matter what Microsoft thought – I saw this as important to our clients.

    On MQs, we have worked even harder. In your day, MQs were developed by smart analysts placing dots. Today, we have a much more rigorous mathematical model – we don’t just place dots. It isn’t perfect, but it’s better. “Ability to execute” includes viability, which will tend to favor large, incumbent vendors. But “vision” actually favors the start-up. When we execute the MQ well, you clearly see the visionaries, the incumbents, etc. It is in our best interest to show that variety in an MQ.

    I don’t speak for Gartner, I speak for myself. That’s what this post was all about, but it’s also, as I said, where the rubber meets the road. I look forward to being challenged on specifics – I’ll defend my credibility and integrity any day.

  9. Tom, like I said the issue is not as much with individual analysts. It is at the firm level.

    So let me reiterate a few points I think would help Gartner’s independence image

    Show how much revenue from vendors, the top 5, 10,.25. Many companies do that in their 10-K. I mean we live in a world of transparency, SaaS trust sites with SLAs for all to see etc why will Gartner not show how much IBM, HP, Oracle etc pay and how that does not influence what you as analyst say about IBM, HP, Oracle? No body can be as pure as Consumer Reports which does not take a dime from companeis whose products it tests, but transparency would go a long way.

    Gartner has rules around how vendors can use your published research. But there appears to be little rules around what your consulting, metrics etc groups issue and how that is marketed by vendors. I have seen at multiple clients where vendors attribute dated metrics to Gartner and then argue that is the market benchmark. That shades your credibility when compared against emerging cloud or other disruptive economics. You have ways to enforce vendor usage policies. When you don’t it looks like you are on their side.

    On MQ’s perception is reality. Go look at the 40 to 50 MQs you issue and see how few smaller, innovative vendors show up on top right. you have the data points – but with so many markets going through rapid change it rewards established vendors far longer than it should because they are more “viable” . Time to question the MQ methodology. I would say.

    I have been in your shoes, and know how hard analysts try to stay objective, but at the firm level Gartner looks like a mouthpiece for larger vendors. pro-status quo etc. You need to keep doing your solid work, but the firm could do many many things to help recreate the “voice of the market” image of yesteryear. Without that you will be tempted to shoot the messengers – bloggers and others – but not fundamentally change the truism in the market today

    “In the 70s CIOs turned to IBM for advice
    In the 80s to Accenture
    In the 90s to Gartner
    In this decade they rely on each other – unbiased peer input”

  10. John says:

    Surely you’re kidding.

    As a software vendor, we are told first hand by Gartner’s salespeople that to enter the Magic Quadrant for our own market, we must pay between 30 and 50 K$.

    How can you say you can’t be bought then! This is just ludicrous.

    J. (Not my real name, witholding the name of my company out of fear of retribution).

  11. […] VP and distinguished analyst Tom Bittman wrote a passionate rant on his blog yesterday defending his and his coworkers’ integrity as analysts. In my 14+ years at Gartner, I have never, ever allowed a vendor to influence my opinion with […]

  12. Tom Bittman says:

    John, first I don’t believe it, but there are always bad apples and if this was true, and we knew about it, this apple would be gone (Gartner SPA – 0.9 probability).

    Nancy Erskine’s job (as ombudsman) is to solve these. Nancy’s great. Would you be willing to give her enough information to deal with it? If not, emails at Gartner are first.last@gartner.com, and I’m listening.

  13. To eliminate any concerns about vendor bias, how about Gartner eliminate consulting contracts and payments from vendors?

    I assume this is a naive question.

    However, such a move would provide the strongest possible financial incentivements and align with end users interest.

  14. Val Sribar says:

    Those of you interested in smaller vendors might want to look at the Cool Vendors Special Report that Gartner publishes every year dedicated to finding innovative no companies: http://www.gartner.com/DisplayDocument?doc_cd=16766

  15. Val, that’s nice but how many of those cool vendors show up in the top right – or at all – in your MQs?

    and given the 50+ categories on the annual Hype Cycle, should there not be 10x more cool vendors each year?

    And how about on the other side of the bell cycle – uncool vendors – where they are in lock-in, milk product mode and many customers call in to complain of their tactics. You and I could list 25 categories of such spend in about 10 minutes…

  16. Tom Bittman says:

    Vinnie, let me explain a bit more about MQs.

    1) MQs describe existing markets that have 8+ competitors. Cool vendors often create new markets and don’t fit in “old” MQs That’s why we write about cool vendors – because they are shaking things up, and eventually might create a new market and MQ. When you were at Gartner, MQs might have been more static. Today new markets are being formed much faster, and we are creating more MQs to describe them as they mature.

    2) If a cool vendor emerges in an existing market covered by an MQ, they would be high on visionary, but it is unlikely they would be strong on ability to execute, and I hope that’s not a surprise to you. It’s a rare cool vendor that becomes an “established” vendor over time without encountering competition from fast-following big vendors, or being acquired. And our purpose in an MQ is not to raise a vendor on a pedestal, it’s to give good advice to end users. If an end user is leading edge and wants to push risks, go for the visionaries.

    3) I’m not sure how to interpret your last para. Are there uncool vendors, and do we talk about them, their lock-in tactics, etc.? Of course, as you know (because you worked at Gartner), we have always helped our clients navigate the licensing, pricing, support, etc. tactics of the vendors who thrive on lock-in. This is why Gartner was formed. I know this is your business now, but it was also what you did at Gartner, right? Our business is to help clients move forward while some of their vendors are holding them back.

    You imply we aren’t motivated to point out new and exciting vendors who are going to shatter existing markets and loosen the stranglehold that existing vendors have on our clients. I absolutely don’t understand how that would make business sense for us. This is the reason we spend so much time in vendor briefings (I get at least a dozen vendor requests a week, and have to pick and choose). And this why we have cool vendor reports (many – and introduced after you left) and hype cycles (many more than when you worked at Gartner).

  17. @tom: I get complaints pretty much every week of the ‘pay to play’ argument so whether you believe it or not is immaterial. It goes back to what @vinnie says about firm level issues and the corporate emphasis on aggressive selling – or as one of your major clients puts it to me: tin cupping.

    I appreciate and respect that individual analysts try maintain their independence and I see your examples. I know several of your specialist analysts in my specialist space and I would not doubt their personal integrity.

    But the same holds true for people like @vinnie and I. We get the calls when we say something vendors don’t like. I’ve had the veiled threats of withholding access and the ‘I wish you hadn’t said that…’ emails. I’ve been banned by one company for talking about customers and accused of shilling for competition. It goes with the turf but then I can point to thousands of pieces I’ve written that consistently express my position AND show full disclosures of any affinities or what not. As a blogger, I have that luxury but also the necessity.

    For other data points, check what Carter Lusher has said in the past about Gartner missing massive opportunities in the end user market. Solve that problem, be transparent and much of this critique will have been answered.

    But then I read what was said in the media about @vinnie, Ray Wang, Ollie Marks, Frank Scavo and my wee venture and I ask myself: why did a bunch of journalists and others say the buy side is badly under-represented if it isn’t at least perceptually true?

  18. Tom Bittman says:

    Dennis, my feeling is these aren’t firm level issues. As a firm, this kind of selling only hurts us. These are issues with individuals, and aggressive selling. Regardless of what a salesperson may imply or outright say, sales people have no influence on our research. I know I’ve had my own conversation with vendors and prospects and sales people to emphasize this, but I’m sure this kind of problem will continue. Regardless, as a firm, we do work hard to avoid it. What bothers me is when I hear people claiming we are doing this as policy. Not true, and not smart business. Maybe just a sales person trying to close a deal. And if we are aware of it, they will feel the wrath.

    I need to understand more about the start-up issue, and I’ll read up to understand it. In my own sphere of analyst influence, it doesn’t exist. That doesn’t mean there aren’t issues in other areas, and I’d like to know where they are.

    Anyway, appreciate your comments.

  19. @tom – you’re seeing things from the ‘inside’ – always difficult to be objective even with the best of intentions. As one who’s been on both sides of that fence, I believe that’s a fair statement.

    FWIW – I would not repeat what I am told if it was one off or obvious sour grapes but I can say that some vendors I’ve spoken with see ‘pay to play’ (and not just Gartner but the analyst community as a whole) as an irritant to the point where I can immediately think of at least a handful that have voted with their wallets and said ‘no more’ after many years of engagement.

    Gartner might not see these companies but as I learned a long time ago via Ghandi (and slightly mangled): ‘You learn most from those with which you disagree.’

    In the meantime, I trust you understand these comments are NOT a personal attack on yours or anyone’s integrity but about the fracture that exists between well intentioned individuals and the firm’s management different agendas.

  20. Jonathan Yarmis says:

    I’ve been on all sides of this argument (Gartner analyst, non-Gartner analyst, AR professional) and to argue that vendor money does not influence perceptions and outcomes is hugely naive. My standard spiel as an AR practitioner was that money did not necessarily equate to influence. I’ve seen people spend a lot of money to no apparent benefit. I’ve seen people spend no money and get great outcomes. But money well spent equates to more positive outcomes.

    If you think all that money vendors are spending is purely because of your insights, again you’re being naive. There’s an influence component and an insight component. Why do you think they spend so much more with Gartner? Are your insights really that much keener? Or are the vendors more concerned with your market reach and therefore the need to more strongly influence you compared with everyone else?

    So, the vendors are spending because they believe they’re influencing you. You’re claiming you can’t be influenced by this. Either you’re wrong, and there is an influence element at play, or they’re wrong and they shouldn’t be spending nearly as much money with you. Which is it?

  21. Tom, first of all appreciate your conversing…this is healthy.

    When I say uncool…I know Gartner will at times put out alerts when a vendor product is really crappy…how about when margins on some products approach 95%, when roaming charges approach $ 3 a minute, when printer ink approach $ 5,000 a gallon, when some on-premise storage is 15X cloud pricing etc that is beyond crappy and I I don;t see Gartner saying it loud enough – the opportunity has been there for years now.

    Rightly or wrongly the perception is it’s your biggest vendor customers which are doing it so you are soft about it. .

  22. […] This post was mentioned on Twitter by Peter Carr. Peter Carr said: What do Aussie Tech vendors think? Gartner analyst rant…something to read over your morning coffee http://bit.ly/19UFAI […]

  23. Ben Dominici says:

    Tom,
    your comment at 4.42pm is fascinating.

    You are saying that Gartner sales people are not representative of the rest of the firm and that your analysis is independent of any sales offer. Hmmm, not sure how this works?? Gartner’s pitch is “Our sales folks are unethical but don’t worry, our Analysts are untouchable.”

    I would never expect that any AR firm will have a written policy of cash for comment i.e. give me your money and I will give you good research. But by your comments, you know that Gartner sales people offer this and so long as it does not affect the research, this will be OK. Sounds like a classic over-sell and under-deliver approach.

    There must be great rapport between sales and analysis at Gartner if you will write like this about your co-workers…

    Try some transparency on for size. You demand it of your vendors but do not deliver it yourself. Who are your top 25 and how much do they pay you?

  24. Tom Bittman says:

    Ben, I’m sorry you didn’t understand what I said, or at least it didn’t fit your worldview, which is far off base. Either accept what I say or don’t, but don’t tell me what I really meant – I said what I really meant. Re-reading my words might help, but I doubt it, so I’ll leave it there.

  25. […] You don’t have to be a Gartner client to get a good “dot” on the Magic Quadrant Posted on October 13, 2009 by sagecircle One of the continuing myths in the IT industry is that Gartner demands payment from vendors for placement on its research. This even came up in a comment – anonymously posted of course – on a blog post written by Gartner VP and Distinguished Analyst Tom Bittman (bio, blog, Twitter) called A Rant – My Integrity as an Analyst. […]

  26. Ben Dominici says:

    Hi Tom,

    Took your advice and read your comment again.

    To quote you, “Regardless of what a salesperson may imply or outright say, sales people have no influence on our research.”
    You also say “I’m sure this kind of problem will continue…Maybe just a sales person trying to close a deal.”.

    From my dim worldview, I read these comments from you and draw the conclusion that you know sometimes your sales people will offer clients more than what the research part of your organisation is prepared to deliver, in an effort to close the deal.

    It seems that many of the comments on this blog also agree with you. Your sales people are promising more than the research department are prepared to deliver. Because of this behaviour, you are then forced to write this defence of your own integrity.

    Good luck to you!

  27. […] The Gartner bias issue has popped up again – where are the aliens? CarterLusher has leaped to the defence of Gartner following a post by Tom Bittman. […]

  28. […] Gartner VP and Distinguished Analyst Tom Bittman (bio, blog, Twitter) has addressed the issue in A Rant – My Integrity as an Analyst along with Gartner Client Ombudsman Nancy Erskine who posted It’s Still True: Gartner Opinion is […]

  29. One way around the perception issue and to ensure that sales representatives do not imply that vendors have to pay-to-play is for Gartner, and other firms, to adopt a “Bill of Rights for Vendors.” This Bill of Rights for Vendors would clearly state that vendors do not have to be client to brief the analysts or pay to be included in research. The analyst firm sales representative would be required to discuss the Bill of Rights with every vendor they meet with and provide a copy personally signed and dated by the sales representative.

    To ensure the credibility of a Bill of Rights for Vendors, analyst firms would have to deal harshly with any employees that violates the policy.

    We are starting a process to collect analyst ecosystem on creating a standard “Bill of Rights for Vendors” that all analyst firms and vendors can use. To contribute to the development of this standard please visit:

    http://sagecircle.wordpress.com/2009/10/13/bill-of-rights-for-industry-analyst-vendor-prospects/

  30. Tom Bittman says:

    No, Ben, in 14 years, I’ve never encountered this. Never. The quote you missed was “I don’t believe it” (from which you concluded that I must KNOW it). I better understand why some people think Elvis is alive, Area 51 has aliens, the world will end in 2012, etc.

    I love the bill of rights idea. Now I’m going back to work. Sheesh.

  31. James says:

    I have noticed that many of your analysts have gone out of their way to not even mention in “print” non-vendor solutions such as OWASP in the security space. An obvious example of this behavior was the release of the static analysis (SAST) magic quadrant where it is well-known that everyone uses the OWASP WebGoat project as the benchmark.

    As an end-customer, I think the whole pay for play has another dimension that is not frequently discussed. I tell others, the KKK doesn’t discriminate against minorities, they simply choose criteria where the outcome is the same.

    In this light, does Gartner choose criteria that gives non-commercial organizations a second class status?

  32. Neil MacDonald says:

    @James,

    WebGoat isn’t really a tool to perform static analysis, so I’m not sure it belongs in the quadrant.

    I do agree that clients need to be able to compare the capabilities of SAST tools. WebGoat and similar freely available offerings give them a way to do this by providing an application with known vulnerabilities. However, SAST tool vendors can adjust for the test cases and skew the results. My advice would be to expand the test cases to include applications that your organization has deployed in the past that were found to be vulnerable after the fact and run the tool against the unpatched version to see if these issues (and perhaps others yet undiscovered) could have been proactively identified.

    For SAST, the few open source alternatives fall well short of commercially available offerings. For dynamic web application security testing (a different market), I’ll direct clients to OWASP WebScarab, Nikto and others as solid alternatives to commercial products.

  33. James Dixon says:

    The metrics and criteria that analysts use are critical in staying relevant. By using biased metrics analysts favor proprietary solutions over open source ones, and produce analysis that is not complete.

    The problem is that in most domains, the current metrics are biased. Until the point where analysts understand why their metrics are biased and change them their analysis becomes increasing irrelevant.

    James Dixon, CTO, Pentaho

  34. […] regard as an incestuous ‘pay-to-play’ arrangement with vendors. Earlier in the month, Gartner analyst Tom Bittman hit out at those who argue ‘pay to play:’ As an analyst at Gartner, I can’t describe how angry I get when I read bloggers spouting as […]

  35. Lydia Leong says:

    There are absolutely salespeople who do, in fact, either imply or directly state pay-to-play. But they’re breaking Gartner’s policies when they do; they’re flat-out lying, and they know it.

    I know that every time I’ve reported an incident of this sort to my management, prompt action has been taken to deal with the offending individual.

  36. […] Blog my Tom Bittman from Gartner – A Rant – My Integrity as an Analyst […]

  37. […] Blog my Tom Bittman from Gartner – A Rant – My Integrity as an Analyst […]

  38. Tom

    Great piece you have my full support.

    Only one thing to add, is that good vendors are good clients too. A vendor briefing only relationship is one way traffic, but a value added vendor is one that uses us like a client not as a vendor and we get involved with there ultra NDA’s not just product and project related.

    We both know we do this, but not all vendors do or see us as adding value. Product marketing/managment relationships with analysts are key, not just VB AR

    regds
    phild

  39. […] Thomas Bittman addressed the issue of analyst integrity in a blog entry appropriately titled: A Rant – My Integrity as an Analyst. Bittman, a vice president and distinguished analyst, has been with Gartner for more than 14 years. […]

  40. Tom Bittman says:

    James, it is not true that in most domains the current metrics are biased. I just looked in several and don’t see a issues – lots of small vendors, lots of open source, lots of non-profits getting fairly included and evaluated. However, I did find one that I thought was questionable – the issue was an inclusion criterion that required a certain minimum revenue amount to be considered. The rationale for inclusion criteria in general is to draw a line somewhere so we don’t have to cover unimportant players. But if an inclusion criterion leaves out an important player that just happens to have a different business model, or be a non-profit, that’s not right.

    Anyway, I’m pursuing this with our Senior Research Board (they actually discuss this next week). The question is what guidelines should we put in place to ensure balanced inclusion, and also how we police that to ensure individual analysts don’t exclude vendors/projects for the wrong reason.

    I would also recommend that this is exactly the kind of feedback that would be ideally handled by our Ombudsman, when it’s occurring, and not much later.

  41. […] Thomas Bittman addressed the issue of analyst integrity in a blog entry appropriately titled: A Rant – My Integrity as an Analyst. Bittman, a vice president and distinguished analyst, has been with Gartner for more than 14 years. […]

  42. Scott Feuless says:

    When I was a CTO at a startup back in 2000 we had a service that we thought Gartner would be interested in looking at, so that when it commented on our emerging market they’d know about what we were doing and, hopefully, mention us. We were told that Gartner was absolutely interested in doing that.

    The next call we received was from someone wanting to sell us a research subscription. We didn’t think we needed research, we _were_ the research, and we didn’t have a ton of cash lying around to buy it with anyway. The calls stopped. We called back – same thing – “yes, we’re very interested, how much research do you want to buy?”

    This is the kind of thing that analysts such as yourself don’t see at all. It never gets to you. But you can certainly see where the impression of “pay to play” comes from…

  43. […] Erst vor kurzem verteidigte sich Gartner-Mitarbeiter Tom Bittman dagegen in einem Blogposting vehement. Das Thema steht aber diesmal gar nicht auf der Tagesordnung. Gartner wird vielmehr vorgehalten, […]

  44. Tom Bittman says:

    Scott, understand. All I can say is we have a channel to get to analysts, and we have a sales channel that will look to sell you stuff (and hopefully, they would forward appropriately if it wasn’t a sales opportunity). This is true with any company in the world, and some are better at it than others.

    If a vendor wants to get info to Gartner analysts, the route they use is vendor.briefings@gartner.com. That doesn’t go to sales, it goes to a team inside of Gartner that determines which analysts might be interested, and asks them if they would like the briefing. At that point, it all comes down to analyst triage – do we have time, does it sound interesting, why should our clients care, etc. I get about half a dozen vendor briefing requests a week, and accept 1-2.

    The same would be true with any vendor in the world – in fact, I know I’ve talked with companies trying to find a way “in” at IBM, Microsoft, HP, etc., etc. They didn’t want to buy stuff, they wanted to create a partnership, or somesuch.

    Anyway, for Gartner, if you want direct connection to the analysts, you send an email through vendor.briefings@gartner.com. Can’t say that the analysts will make time (although if you make the request INTERESTING enough, they just might!), but I can say that no sales person will see that request!

  45. […] Blog my Tom Bittman from Gartner – A Rant – My Integrity as an Analyst […]

  46. […] tags: ECM, Forrester, Gartner by Pie There has been a lot of talk the last few months about the integrity, and completeness, of Gartner’s Magic Quadrant reports.  While the lawsuit against […]

  47. […] I have no problem with people disagreeing or challenging judgements, but I am as insulted as Tom is by offhand, unsupported assumptions that question my integrity. Luckily, I have enough customers […]

  48. […] Gartner VP and Distinguished Analyst Tom Bittman (bio, blog, Twitter) has addressed the issue in A Rant – My Integrity as an Analyst along with Gartner Client Ombudsman Nancy Erskine who posted It’s Still True: Gartner Opinion is […]

  49. Tech co says:

    We’re all flawed human beings, even Gartner analysts. We all have our egos, biases, filters, blinders, whatever you want to call them.

    I’m sure many Gartner analysts do a great job of being as objective as they can be. However, I also know that at least one that I’m dealing with is being completely unreasonable. Some of the research is completely 180 degrees from what companies are telling us. It’s incredibly frustrating knowing that there’s little that we can do, since challenging that will just make us seem bitter, pretty much guaranteeing that a biased analyst will never write anything positive about the company. So I bite my tongue.

    We could theoretically talk to the Ombudsman, but how do we know it won’t just have more of the same effect?

    Meanwhile, the analyst effect is a self-reinforcing loop. Company A is rated highly, and gets more customers. Gartner rates bigger companies better, Company A gets more customers. Company B loses opportunities to Company A because of Gartner ratings, and stays the same size. This makes it easier for Gartner to say “Company B, you’re just not big enough.”

  50. […] his own glass edifices. There is no question that individual analysts there have a great deal of integrity, but would you jump off a Gov 2.0 bridge if Gartner told you […]



Leave a Reply

Your email address will not be published. Required fields are marked *

Comments or opinions expressed on this blog are those of the individual contributors only, and do not necessarily represent the views of Gartner, Inc. or its management. Readers may copy and redistribute blog postings on other blogs, or otherwise for private, non-commercial or journalistic purposes, with attribution to Gartner. This content may not be used for any other purposes in any other formats or media. The content on this blog is provided on an "as-is" basis. Gartner shall not be liable for any damages whatsoever arising out of the content or use of this blog.