Thomas Bittman

A member of the Gartner Blog Network

Thomas J. Bittman
VP Distinguished Analyst
18 years at Gartner
29 years IT industry

Thomas Bittman is a vice president and distinguished analyst with Gartner Research. Mr. Bittman has led the industry in areas such as private cloud computing and virtualization. Mr. Bittman invented the term "real-time infrastructure," which has been adopted by major vendors and many… Read Full Bio

Coverage Areas:

A Rant – My Integrity as an Analyst

by Tom Bittman  |  October 8, 2009  |  55 Comments

I need to rant a little.

As an analyst at Gartner, I can’t describe how angry I get when I read bloggers spouting as “fact” their opinion that I and my teammates have no integrity. That we can be “bought.”

In my 14+ years at Gartner, I have never, ever allowed a vendor to influence my opinion with anything but facts. Period. They have certainly tried to influence me with non-facts. I can say this definitively – it has never worked.

I don’t think there is a single vendor that I have dealt with who has not been very angry with me at some point. Tough. I’ve been yelled at by many IT executives – including the CEOs of Microsoft and HP, and many other firms. I can’t think of one of those cases when I changed my analysis one bit. I can’t speak for other firms, but at Gartner, getting yelled at by a CEO is a badge of honor. Being proven right as time goes on – priceless

I certainly spend time helping vendors with their strategies and their marketing messages – and I enjoy doing it. Frankly, the ones who yell at us the most seem to respect our opinion the most. We can spot holes a mile away, and engaged early enough, we can help vendors fill those holes with real product offerings – that not only help the vendors, but help our end user clients. And my primary business is helping end users.

I worked at IBM for 11 years. During the last few years, I was doing some technical evangelizing to analyst firms – and hiring these firms to write white papers. There were plenty available for hire – and still are. There was only one firm that wouldn’t allow us to edit their work – Gartner. Only one. We tried, but they wouldn’t budge. They could not be bought. It was unusual, and I completely respected that. When the opportunity came to join Gartner, I jumped. I wouldn’t have even considered anyone else.

Since I’ve joined the firm, my appreciation for the integrity of the firm and its analysts has only increased. If I felt that the firm’s integrity was bending, first I would fight it with all my might, and then I would be outta there. I’m not gone yet!

So I understand the impression in the marketplace that analyst firms can be bought. But that’s not where I work. My integrity is very important to me. I’m sure we’ll continue to make enemies of vendors, and bloggers who have a vested interest in one thing or another. Badge of honor! But my goal is to provide value to my clients, and to be proven right over time – priceless!

End of rant.

55 Comments »

Category: Industry Analyst     Tags:

55 responses so far ↓

  • 1 Tweets that mention A Rant – My Integrity as an Analyst -- Topsy.com   October 8, 2009 at 10:21 pm

    [...] This post was mentioned on Twitter by Erik Kingham and Avnet SolutionsPath. Avnet SolutionsPath said: Gatner A Rant – My Integrity as an Analyst: I need to rant a little. As an analyst at Gartner, I c.. http://bit.ly/1BBVZI [...]

  • 2 Anthony Bradley   October 8, 2009 at 11:17 pm

    Here. Here. People disparaging Gartner on being “bought” need to realize they aren’t talking about the logo. They are talking about us, the analysts. I immediately wonder about their motives and why they are so angry?

    I also love the part about us not having “touched code in ages.” I’ve written a bunch of code in my day, some pretty recently. You know what I never got from writing code? A broad view of technology. Why would I write code to evaluate a technology when I can talk to 100 clients who are doing it and get the benefit of their experiences. It is a much more leveraged model. And that is what we provide.

    Sure, we listen to a bunch of vendors and clients and advise a bunch of vendors and clients, with a boat load of investigation and analysis in between. Remind me again what the problem is here? If you think it is easy to ingest an immense and continuous stream of information, make sense of it, spot patterns, draw conclusions, present findings and defend them in at least 20 client meetings a week then you better keep to writing code because being an analyst would be a rude awakening.

    OK, now I’m done ranting.

  • 3 Rick Brusuelas   October 9, 2009 at 7:17 am

    Tom, many kudos speaking up. I have worked with a number of Gartner analysts over many many years and each and every one demonstrated the utmost integrity. The advice, feedback, and challenges that Gartner analysts have shared with me and the vendors I have worked with have tested and improved the strategies and offerings. I have been fortunately to have worked directly with Tom on numerous occasions over many years and his counsel has been invaluable.

    At the same time, Tom’s rant speaks up for many other analyst firms and individual analysts that also share Tom’s passion for helping vendors better meet (and exceed) end-user requirements.

  • 4 Pedro L B Maschio   October 9, 2009 at 9:18 am

    I worked for Gartner for 10 years.
    Many times vendors wanted to influence or change the benchmarking reports I wrote regarding price and SLA on outsourcing contracts, and I was very lucky to have top management support me, backing me up, so that I could maintain my findings and recommendations to renegotiate contracts.
    Today I am a client, I pay for Gartner services because integrity matters!
    Integrity in my asset, it’s not for sale, and it’s a Gartner value.

  • 5 Jonny Bentwood   October 9, 2009 at 9:35 am

    Like to echo my sentiments here. The problem only occurs when a few bad apples spoil what I consider to be one of the most trusted industries going. There is a reason why the Edelman Trust Barometer consistently puts analysts as the highest profession when it comes to trust.

    However, successful analyst houses will only remain so provided they remain impartial. Where the ‘very few’ firms have fallen foul of that – I am delighted to see that they very quickly fall off the map – afterall it is your independence that counts.

  • 6 vinnie mirchandani   October 9, 2009 at 10:05 am

    Tom, as for your former colleague and now a blogger I guess I see both sides. If you think it is just bloggers raising the issues, you are not reading the market, including your customers well.

    The issue is not at an individual analyst level. It is at the firm level. You know it and I know it that vendor sourced revenues have been growing nicely as percentage of Gartner revenues. You and I know the majority of top 25 Gartner revenues are vendors like IBM – and individually they dwarf what even the biggest user entities like GE or BP pay you.

    It would help if Gartner was transparent about that, rather than avoid talking about it, and if Gartner aggressively showcased how those vendor interests and subtle and not so subtle pressure from them is balanced with user inteersts. The answer usually is the Gartner Ombudsman blog does that. Seriously?

    The other thing is 2 of Gartner’s best known tools – the MQ and the Hype Cycle are skewed towards larger vendors and against startups. The MQ rewards market share and viability or tenure in a market. By its very definition the Hype cycle warns people about newer stuff. So Gartner comes across as pro-establishment with more skew towards larger, incumbent vendors

    In the field I see time and time again vendors using your metrics where it suits them. I see economics that are dated. In a consulting assignment last year, the CIO asked me fairly loudly during a presentation when a vendor put some of your metrics “so is that 30 or 40% over priced?”

    Finally, when I joined Gartner in 1995 I was told of an episode when David Stein proactively called a CIO – I think of an insurance co and pulled him out of a meeting with IBM and telling him not to sign a contract because pricing would go down significantly in a couple of weeks.

    My question is how many Gartner analysts do that today? A proactive, “customer is God” POV?

    Most CIOs would love to see that Gartner again. Then you cold tell the pesky bloggers to go pound sand!

  • 7 Lydia Leong   October 9, 2009 at 10:25 am

    Fantastic post, Tom.

    Vinnie,

    I’m currently an analyst at Gartner, and I certainly do proactively contact clients with whom I’ve had recent contact, and where a change in the market or a change in the information I have available means that I ought to change the advice that I’ve given them.

  • 8 Tom Bittman   October 9, 2009 at 11:39 am

    Vinnie, I respect the concern about how Gartner balances vendor interests with user interests. But this is basic logic. The reason we have vendor clients is because we have user clients who trust us. If we did anything to change that trust, we lose both. It is in our best interest as a business to maintain our integrity. It’s a fact that we work even harder at it now than when you were with Gartner.

    All that really matters is where the rubber meets the road. As I said in my post, if I ever felt management trying to bias me for vendor clients, I would first fight it, then leave. I’m telling you that I am not under any kind of pressure to be biased for vendor clients.

    Your statement about hype cycles and MQs is not accurate. Hype cycles are all about calling out new technologies, and almost all of them come from start-ups. Do we caution on new technologies? Of course. But, we also elevate new stuff that we think is important. Specific example – when VMware was a 40-person company in 2001 I wrote that they had a killer app that would change the market. Microsoft was furious that I devoted a slide in a presentation about Windows Futures to VMware beginning in 2002. Didn’t matter what Microsoft thought – I saw this as important to our clients.

    On MQs, we have worked even harder. In your day, MQs were developed by smart analysts placing dots. Today, we have a much more rigorous mathematical model – we don’t just place dots. It isn’t perfect, but it’s better. “Ability to execute” includes viability, which will tend to favor large, incumbent vendors. But “vision” actually favors the start-up. When we execute the MQ well, you clearly see the visionaries, the incumbents, etc. It is in our best interest to show that variety in an MQ.

    I don’t speak for Gartner, I speak for myself. That’s what this post was all about, but it’s also, as I said, where the rubber meets the road. I look forward to being challenged on specifics – I’ll defend my credibility and integrity any day.

  • 9 vinnie mirchandani   October 9, 2009 at 12:44 pm

    Tom, like I said the issue is not as much with individual analysts. It is at the firm level.

    So let me reiterate a few points I think would help Gartner’s independence image

    Show how much revenue from vendors, the top 5, 10,.25. Many companies do that in their 10-K. I mean we live in a world of transparency, SaaS trust sites with SLAs for all to see etc why will Gartner not show how much IBM, HP, Oracle etc pay and how that does not influence what you as analyst say about IBM, HP, Oracle? No body can be as pure as Consumer Reports which does not take a dime from companeis whose products it tests, but transparency would go a long way.

    Gartner has rules around how vendors can use your published research. But there appears to be little rules around what your consulting, metrics etc groups issue and how that is marketed by vendors. I have seen at multiple clients where vendors attribute dated metrics to Gartner and then argue that is the market benchmark. That shades your credibility when compared against emerging cloud or other disruptive economics. You have ways to enforce vendor usage policies. When you don’t it looks like you are on their side.

    On MQ’s perception is reality. Go look at the 40 to 50 MQs you issue and see how few smaller, innovative vendors show up on top right. you have the data points – but with so many markets going through rapid change it rewards established vendors far longer than it should because they are more “viable” . Time to question the MQ methodology. I would say.

    I have been in your shoes, and know how hard analysts try to stay objective, but at the firm level Gartner looks like a mouthpiece for larger vendors. pro-status quo etc. You need to keep doing your solid work, but the firm could do many many things to help recreate the “voice of the market” image of yesteryear. Without that you will be tempted to shoot the messengers – bloggers and others – but not fundamentally change the truism in the market today

    “In the 70s CIOs turned to IBM for advice
    In the 80s to Accenture
    In the 90s to Gartner
    In this decade they rely on each other – unbiased peer input”

  • 10 John   October 9, 2009 at 4:32 pm

    Surely you’re kidding.

    As a software vendor, we are told first hand by Gartner’s salespeople that to enter the Magic Quadrant for our own market, we must pay between 30 and 50 K$.

    How can you say you can’t be bought then! This is just ludicrous.

    J. (Not my real name, witholding the name of my company out of fear of retribution).

  • 11 Industry analysts: A problem of integrity - The Network Hub   October 9, 2009 at 4:38 pm

    [...] VP and distinguished analyst Tom Bittman wrote a passionate rant on his blog yesterday defending his and his coworkers’ integrity as analysts. In my 14+ years at Gartner, I have never, ever allowed a vendor to influence my opinion with [...]

  • 12 Tom Bittman   October 9, 2009 at 4:44 pm

    John, first I don’t believe it, but there are always bad apples and if this was true, and we knew about it, this apple would be gone (Gartner SPA – 0.9 probability).

    Nancy Erskine’s job (as ombudsman) is to solve these. Nancy’s great. Would you be willing to give her enough information to deal with it? If not, emails at Gartner are first.last@gartner.com, and I’m listening.

  • 13 John Honovich   October 9, 2009 at 6:23 pm

    To eliminate any concerns about vendor bias, how about Gartner eliminate consulting contracts and payments from vendors?

    I assume this is a naive question.

    However, such a move would provide the strongest possible financial incentivements and align with end users interest.

  • 14 Val Sribar   October 9, 2009 at 6:36 pm

    Those of you interested in smaller vendors might want to look at the Cool Vendors Special Report that Gartner publishes every year dedicated to finding innovative no companies: http://www.gartner.com/DisplayDocument?doc_cd=16766

  • 15 vinnie mirchandani   October 10, 2009 at 5:49 am

    Val, that’s nice but how many of those cool vendors show up in the top right – or at all – in your MQs?

    and given the 50+ categories on the annual Hype Cycle, should there not be 10x more cool vendors each year?

    And how about on the other side of the bell cycle – uncool vendors – where they are in lock-in, milk product mode and many customers call in to complain of their tactics. You and I could list 25 categories of such spend in about 10 minutes…

  • 16 Tom Bittman   October 10, 2009 at 11:55 am

    Vinnie, let me explain a bit more about MQs.

    1) MQs describe existing markets that have 8+ competitors. Cool vendors often create new markets and don’t fit in “old” MQs That’s why we write about cool vendors – because they are shaking things up, and eventually might create a new market and MQ. When you were at Gartner, MQs might have been more static. Today new markets are being formed much faster, and we are creating more MQs to describe them as they mature.

    2) If a cool vendor emerges in an existing market covered by an MQ, they would be high on visionary, but it is unlikely they would be strong on ability to execute, and I hope that’s not a surprise to you. It’s a rare cool vendor that becomes an “established” vendor over time without encountering competition from fast-following big vendors, or being acquired. And our purpose in an MQ is not to raise a vendor on a pedestal, it’s to give good advice to end users. If an end user is leading edge and wants to push risks, go for the visionaries.

    3) I’m not sure how to interpret your last para. Are there uncool vendors, and do we talk about them, their lock-in tactics, etc.? Of course, as you know (because you worked at Gartner), we have always helped our clients navigate the licensing, pricing, support, etc. tactics of the vendors who thrive on lock-in. This is why Gartner was formed. I know this is your business now, but it was also what you did at Gartner, right? Our business is to help clients move forward while some of their vendors are holding them back.

    You imply we aren’t motivated to point out new and exciting vendors who are going to shatter existing markets and loosen the stranglehold that existing vendors have on our clients. I absolutely don’t understand how that would make business sense for us. This is the reason we spend so much time in vendor briefings (I get at least a dozen vendor requests a week, and have to pick and choose). And this why we have cool vendor reports (many – and introduced after you left) and hype cycles (many more than when you worked at Gartner).

  • 17 Dennis Howlett   October 10, 2009 at 12:25 pm

    @tom: I get complaints pretty much every week of the ‘pay to play’ argument so whether you believe it or not is immaterial. It goes back to what @vinnie says about firm level issues and the corporate emphasis on aggressive selling – or as one of your major clients puts it to me: tin cupping.

    I appreciate and respect that individual analysts try maintain their independence and I see your examples. I know several of your specialist analysts in my specialist space and I would not doubt their personal integrity.

    But the same holds true for people like @vinnie and I. We get the calls when we say something vendors don’t like. I’ve had the veiled threats of withholding access and the ‘I wish you hadn’t said that…’ emails. I’ve been banned by one company for talking about customers and accused of shilling for competition. It goes with the turf but then I can point to thousands of pieces I’ve written that consistently express my position AND show full disclosures of any affinities or what not. As a blogger, I have that luxury but also the necessity.

    For other data points, check what Carter Lusher has said in the past about Gartner missing massive opportunities in the end user market. Solve that problem, be transparent and much of this critique will have been answered.

    But then I read what was said in the media about @vinnie, Ray Wang, Ollie Marks, Frank Scavo and my wee venture and I ask myself: why did a bunch of journalists and others say the buy side is badly under-represented if it isn’t at least perceptually true?

  • 18 Tom Bittman   October 10, 2009 at 4:42 pm

    Dennis, my feeling is these aren’t firm level issues. As a firm, this kind of selling only hurts us. These are issues with individuals, and aggressive selling. Regardless of what a salesperson may imply or outright say, sales people have no influence on our research. I know I’ve had my own conversation with vendors and prospects and sales people to emphasize this, but I’m sure this kind of problem will continue. Regardless, as a firm, we do work hard to avoid it. What bothers me is when I hear people claiming we are doing this as policy. Not true, and not smart business. Maybe just a sales person trying to close a deal. And if we are aware of it, they will feel the wrath.

    I need to understand more about the start-up issue, and I’ll read up to understand it. In my own sphere of analyst influence, it doesn’t exist. That doesn’t mean there aren’t issues in other areas, and I’d like to know where they are.

    Anyway, appreciate your comments.

  • 19 Dennis Howlett   October 10, 2009 at 6:45 pm

    @tom – you’re seeing things from the ‘inside’ – always difficult to be objective even with the best of intentions. As one who’s been on both sides of that fence, I believe that’s a fair statement.

    FWIW – I would not repeat what I am told if it was one off or obvious sour grapes but I can say that some vendors I’ve spoken with see ‘pay to play’ (and not just Gartner but the analyst community as a whole) as an irritant to the point where I can immediately think of at least a handful that have voted with their wallets and said ‘no more’ after many years of engagement.

    Gartner might not see these companies but as I learned a long time ago via Ghandi (and slightly mangled): ‘You learn most from those with which you disagree.’

    In the meantime, I trust you understand these comments are NOT a personal attack on yours or anyone’s integrity but about the fracture that exists between well intentioned individuals and the firm’s management different agendas.

  • 20 Jonathan Yarmis   October 10, 2009 at 6:59 pm

    I’ve been on all sides of this argument (Gartner analyst, non-Gartner analyst, AR professional) and to argue that vendor money does not influence perceptions and outcomes is hugely naive. My standard spiel as an AR practitioner was that money did not necessarily equate to influence. I’ve seen people spend a lot of money to no apparent benefit. I’ve seen people spend no money and get great outcomes. But money well spent equates to more positive outcomes.

    If you think all that money vendors are spending is purely because of your insights, again you’re being naive. There’s an influence component and an insight component. Why do you think they spend so much more with Gartner? Are your insights really that much keener? Or are the vendors more concerned with your market reach and therefore the need to more strongly influence you compared with everyone else?

    So, the vendors are spending because they believe they’re influencing you. You’re claiming you can’t be influenced by this. Either you’re wrong, and there is an influence element at play, or they’re wrong and they shouldn’t be spending nearly as much money with you. Which is it?

  • 21 vinnie mirchandani   October 10, 2009 at 9:47 pm

    Tom, first of all appreciate your conversing…this is healthy.

    When I say uncool…I know Gartner will at times put out alerts when a vendor product is really crappy…how about when margins on some products approach 95%, when roaming charges approach $ 3 a minute, when printer ink approach $ 5,000 a gallon, when some on-premise storage is 15X cloud pricing etc that is beyond crappy and I I don;t see Gartner saying it loud enough – the opportunity has been there for years now.

    Rightly or wrongly the perception is it’s your biggest vendor customers which are doing it so you are soft about it. .

  • 22 Tweets that mention What do Aussie Tech vendors think? Gartner analyst rant...something to read over your morning coffee -- Topsy.com   October 12, 2009 at 8:36 pm

    [...] This post was mentioned on Twitter by Peter Carr. Peter Carr said: What do Aussie Tech vendors think? Gartner analyst rant…something to read over your morning coffee http://bit.ly/19UFAI [...]

  • 23 Ben Dominici   October 12, 2009 at 9:14 pm

    Tom,
    your comment at 4.42pm is fascinating.

    You are saying that Gartner sales people are not representative of the rest of the firm and that your analysis is independent of any sales offer. Hmmm, not sure how this works?? Gartner’s pitch is “Our sales folks are unethical but don’t worry, our Analysts are untouchable.”

    I would never expect that any AR firm will have a written policy of cash for comment i.e. give me your money and I will give you good research. But by your comments, you know that Gartner sales people offer this and so long as it does not affect the research, this will be OK. Sounds like a classic over-sell and under-deliver approach.

    There must be great rapport between sales and analysis at Gartner if you will write like this about your co-workers…

    Try some transparency on for size. You demand it of your vendors but do not deliver it yourself. Who are your top 25 and how much do they pay you?

  • 24 Tom Bittman   October 12, 2009 at 11:45 pm

    Ben, I’m sorry you didn’t understand what I said, or at least it didn’t fit your worldview, which is far off base. Either accept what I say or don’t, but don’t tell me what I really meant – I said what I really meant. Re-reading my words might help, but I doubt it, so I’ll leave it there.

  • 25 You don’t have to be a Gartner client to get a good “dot” on the Magic Quadrant « SageCircle Blog   October 13, 2009 at 8:07 am

    [...] You don’t have to be a Gartner client to get a good “dot” on the Magic Quadrant Posted on October 13, 2009 by sagecircle One of the continuing myths in the IT industry is that Gartner demands payment from vendors for placement on its research. This even came up in a comment – anonymously posted of course – on a blog post written by Gartner VP and Distinguished Analyst Tom Bittman (bio, blog, Twitter) called A Rant – My Integrity as an Analyst. [...]

  • 26 Ben Dominici   October 13, 2009 at 8:40 am

    Hi Tom,

    Took your advice and read your comment again.

    To quote you, “Regardless of what a salesperson may imply or outright say, sales people have no influence on our research.”
    You also say “I’m sure this kind of problem will continue…Maybe just a sales person trying to close a deal.”.

    From my dim worldview, I read these comments from you and draw the conclusion that you know sometimes your sales people will offer clients more than what the research part of your organisation is prepared to deliver, in an effort to close the deal.

    It seems that many of the comments on this blog also agree with you. Your sales people are promising more than the research department are prepared to deliver. Because of this behaviour, you are then forced to write this defence of your own integrity.

    Good luck to you!

  • 27 The Gartner bias issue has popped up again – where are the aliens? « Analystanalyst’s Weblog   October 13, 2009 at 9:04 am

    [...] The Gartner bias issue has popped up again – where are the aliens? CarterLusher has leaped to the defence of Gartner following a post by Tom Bittman. [...]

  • 28 Bill of Rights for industry analyst vendor prospects « SageCircle Blog   October 13, 2009 at 12:19 pm

    [...] Gartner VP and Distinguished Analyst Tom Bittman (bio, blog, Twitter) has addressed the issue in A Rant – My Integrity as an Analyst along with Gartner Client Ombudsman Nancy Erskine who posted It’s Still True: Gartner Opinion is [...]

  • 29 Carter Lusher   October 13, 2009 at 12:35 pm

    One way around the perception issue and to ensure that sales representatives do not imply that vendors have to pay-to-play is for Gartner, and other firms, to adopt a “Bill of Rights for Vendors.” This Bill of Rights for Vendors would clearly state that vendors do not have to be client to brief the analysts or pay to be included in research. The analyst firm sales representative would be required to discuss the Bill of Rights with every vendor they meet with and provide a copy personally signed and dated by the sales representative.

    To ensure the credibility of a Bill of Rights for Vendors, analyst firms would have to deal harshly with any employees that violates the policy.

    We are starting a process to collect analyst ecosystem on creating a standard “Bill of Rights for Vendors” that all analyst firms and vendors can use. To contribute to the development of this standard please visit:

    http://sagecircle.wordpress.com/2009/10/13/bill-of-rights-for-industry-analyst-vendor-prospects/

  • 30 Tom Bittman   October 13, 2009 at 1:55 pm

    No, Ben, in 14 years, I’ve never encountered this. Never. The quote you missed was “I don’t believe it” (from which you concluded that I must KNOW it). I better understand why some people think Elvis is alive, Area 51 has aliens, the world will end in 2012, etc.

    I love the bill of rights idea. Now I’m going back to work. Sheesh.

  • 31 James   October 14, 2009 at 8:39 am

    I have noticed that many of your analysts have gone out of their way to not even mention in “print” non-vendor solutions such as OWASP in the security space. An obvious example of this behavior was the release of the static analysis (SAST) magic quadrant where it is well-known that everyone uses the OWASP WebGoat project as the benchmark.

    As an end-customer, I think the whole pay for play has another dimension that is not frequently discussed. I tell others, the KKK doesn’t discriminate against minorities, they simply choose criteria where the outcome is the same.

    In this light, does Gartner choose criteria that gives non-commercial organizations a second class status?

  • 32 Neil MacDonald   October 14, 2009 at 9:44 am

    @James,

    WebGoat isn’t really a tool to perform static analysis, so I’m not sure it belongs in the quadrant.

    I do agree that clients need to be able to compare the capabilities of SAST tools. WebGoat and similar freely available offerings give them a way to do this by providing an application with known vulnerabilities. However, SAST tool vendors can adjust for the test cases and skew the results. My advice would be to expand the test cases to include applications that your organization has deployed in the past that were found to be vulnerable after the fact and run the tool against the unpatched version to see if these issues (and perhaps others yet undiscovered) could have been proactively identified.

    For SAST, the few open source alternatives fall well short of commercially available offerings. For dynamic web application security testing (a different market), I’ll direct clients to OWASP WebScarab, Nikto and others as solid alternatives to commercial products.

  • 33 James Dixon   October 19, 2009 at 8:50 pm

    The metrics and criteria that analysts use are critical in staying relevant. By using biased metrics analysts favor proprietary solutions over open source ones, and produce analysis that is not complete.

    The problem is that in most domains, the current metrics are biased. Until the point where analysts understand why their metrics are biased and change them their analysis becomes increasing irrelevant.

    James Dixon, CTO, Pentaho

  • 34 Gartner in the dock over Magic Quadrant | Irregular Enterprise | ZDNet.com   October 20, 2009 at 12:00 pm

    [...] regard as an incestuous ‘pay-to-play’ arrangement with vendors. Earlier in the month, Gartner analyst Tom Bittman hit out at those who argue ‘pay to play:’ As an analyst at Gartner, I can’t describe how angry I get when I read bloggers spouting as [...]

  • 35 Lydia Leong   October 20, 2009 at 11:46 pm

    There are absolutely salespeople who do, in fact, either imply or directly state pay-to-play. But they’re breaking Gartner’s policies when they do; they’re flat-out lying, and they know it.

    I know that every time I’ve reported an incident of this sort to my management, prompt action has been taken to deal with the offending individual.

  • 36 Contentious conversations in analyst relations « Technobabble 2.0   October 21, 2009 at 11:22 am

    [...] Blog my Tom Bittman from Gartner – A Rant – My Integrity as an Analyst [...]

  • 37 Contentious conversation 1 – integrity of analysts and the future of AR « The Naked Pheasant   October 21, 2009 at 12:30 pm

    [...] Blog my Tom Bittman from Gartner – A Rant – My Integrity as an Analyst [...]

  • 38 Philip Dawson   October 22, 2009 at 8:11 am

    Tom

    Great piece you have my full support.

    Only one thing to add, is that good vendors are good clients too. A vendor briefing only relationship is one way traffic, but a value added vendor is one that uses us like a client not as a vendor and we get involved with there ultra NDA’s not just product and project related.

    We both know we do this, but not all vendors do or see us as adding value. Product marketing/managment relationships with analysts are key, not just VB AR

    regds
    phild

  • 39 Email archiving vendor sues Gartner, doesn’t see magic in quadrant - Security Bytes   October 22, 2009 at 11:30 am

    [...] Thomas Bittman addressed the issue of analyst integrity in a blog entry appropriately titled: A Rant – My Integrity as an Analyst. Bittman, a vice president and distinguished analyst, has been with Gartner for more than 14 years. [...]

  • 40 Tom Bittman   October 22, 2009 at 11:33 am

    James, it is not true that in most domains the current metrics are biased. I just looked in several and don’t see a issues – lots of small vendors, lots of open source, lots of non-profits getting fairly included and evaluated. However, I did find one that I thought was questionable – the issue was an inclusion criterion that required a certain minimum revenue amount to be considered. The rationale for inclusion criteria in general is to draw a line somewhere so we don’t have to cover unimportant players. But if an inclusion criterion leaves out an important player that just happens to have a different business model, or be a non-profit, that’s not right.

    Anyway, I’m pursuing this with our Senior Research Board (they actually discuss this next week). The question is what guidelines should we put in place to ensure balanced inclusion, and also how we police that to ensure individual analysts don’t exclude vendors/projects for the wrong reason.

    I would also recommend that this is exactly the kind of feedback that would be ideally handled by our Ombudsman, when it’s occurring, and not much later.

  • 41 The TopOfMemory Security Feed » Blog Archive » Email archiving vendor sues Gartner, doesn’t see magic in quadrant   October 22, 2009 at 12:03 pm

    [...] Thomas Bittman addressed the issue of analyst integrity in a blog entry appropriately titled: A Rant – My Integrity as an Analyst. Bittman, a vice president and distinguished analyst, has been with Gartner for more than 14 years. [...]

  • 42 Scott Feuless   October 23, 2009 at 9:25 am

    When I was a CTO at a startup back in 2000 we had a service that we thought Gartner would be interested in looking at, so that when it commented on our emerging market they’d know about what we were doing and, hopefully, mention us. We were told that Gartner was absolutely interested in doing that.

    The next call we received was from someone wanting to sell us a research subscription. We didn’t think we needed research, we _were_ the research, and we didn’t have a ton of cash lying around to buy it with anyway. The calls stopped. We called back – same thing – “yes, we’re very interested, how much research do you want to buy?”

    This is the kind of thing that analysts such as yourself don’t see at all. It never gets to you. But you can certainly see where the impression of “pay to play” comes from…

  • 43 Marktforschung: alles nur fauler Zauber? - Business Talk | IT Business | ZDNet.de   October 23, 2009 at 12:05 pm

    [...] Erst vor kurzem verteidigte sich Gartner-Mitarbeiter Tom Bittman dagegen in einem Blogposting vehement. Das Thema steht aber diesmal gar nicht auf der Tagesordnung. Gartner wird vielmehr vorgehalten, [...]

  • 44 Tom Bittman   October 23, 2009 at 12:06 pm

    Scott, understand. All I can say is we have a channel to get to analysts, and we have a sales channel that will look to sell you stuff (and hopefully, they would forward appropriately if it wasn’t a sales opportunity). This is true with any company in the world, and some are better at it than others.

    If a vendor wants to get info to Gartner analysts, the route they use is vendor.briefings@gartner.com. That doesn’t go to sales, it goes to a team inside of Gartner that determines which analysts might be interested, and asks them if they would like the briefing. At that point, it all comes down to analyst triage – do we have time, does it sound interesting, why should our clients care, etc. I get about half a dozen vendor briefing requests a week, and accept 1-2.

    The same would be true with any vendor in the world – in fact, I know I’ve talked with companies trying to find a way “in” at IBM, Microsoft, HP, etc., etc. They didn’t want to buy stuff, they wanted to create a partnership, or somesuch.

    Anyway, for Gartner, if you want direct connection to the analysts, you send an email through vendor.briefings@gartner.com. Can’t say that the analysts will make time (although if you make the request INTERESTING enough, they just might!), but I can say that no sales person will see that request!

  • 45 Is shooting on the referee productive? « The IIAR Blog   October 23, 2009 at 3:06 pm

    [...] Blog my Tom Bittman from Gartner – A Rant – My Integrity as an Analyst [...]

  • 46 What Can We Expect from ECM Analysts? « Word of Pie   December 1, 2009 at 11:41 pm

    [...] tags: ECM, Forrester, Gartner by Pie There has been a lot of talk the last few months about the integrity, and completeness, of Gartner’s Magic Quadrant reports.  While the lawsuit against [...]

  • 47 My New Year’s Anti-Resolutions for 2010   December 30, 2009 at 12:10 pm

    [...] I have no problem with people disagreeing or challenging judgements, but I am as insulted as Tom is by offhand, unsupported assumptions that question my integrity. Luckily, I have enough customers [...]

  • 48 Bill of Rights for vendor prospects of industry analyst firms » SageCircle Blog   May 15, 2010 at 9:43 am

    [...] Gartner VP and Distinguished Analyst Tom Bittman (bio, blog, Twitter) has addressed the issue in A Rant – My Integrity as an Analyst along with Gartner Client Ombudsman Nancy Erskine who posted It’s Still True: Gartner Opinion is [...]

  • 49 Tech co   August 8, 2010 at 11:11 am

    We’re all flawed human beings, even Gartner analysts. We all have our egos, biases, filters, blinders, whatever you want to call them.

    I’m sure many Gartner analysts do a great job of being as objective as they can be. However, I also know that at least one that I’m dealing with is being completely unreasonable. Some of the research is completely 180 degrees from what companies are telling us. It’s incredibly frustrating knowing that there’s little that we can do, since challenging that will just make us seem bitter, pretty much guaranteeing that a biased analyst will never write anything positive about the company. So I bite my tongue.

    We could theoretically talk to the Ombudsman, but how do we know it won’t just have more of the same effect?

    Meanwhile, the analyst effect is a self-reinforcing loop. Company A is rated highly, and gets more customers. Gartner rates bigger companies better, Company A gets more customers. Company B loses opportunities to Company A because of Gartner ratings, and stays the same size. This makes it easier for Gartner to say “Company B, you’re just not big enough.”

  • 50 What does Gov 2.0 have to do with cloud computing? | Gov 2.0   September 16, 2010 at 3:46 pm

    [...] his own glass edifices. There is no question that individual analysts there have a great deal of integrity, but would you jump off a Gov 2.0 bridge if Gartner told you [...]

  • 51 a   March 15, 2011 at 5:20 pm

    Bottom line: Analysts conclusions are “Subjective” despite the claim of “facts” and “science”.
    - In many cases the analysts might even have a different definition of what the product they are evaluating is (different than what some of the vendor’s definition).
    - Biasing the evaluation by market share, size of marketing budget, etc. rather than pure viability of a technical solution is never a good “scientific” evaluation of a product.
    - to claim that the money spent by big “names” don;t influence the evaluations is simply not believable.

  • 52 Jim   June 3, 2011 at 10:48 pm

    We all know that some analysts at Gartner are extremely bias particularly avivah litan !..give me a break, avivah litan reports about fraud are ridiculous and dishonest (the winner is my friend and we hare the same ?.)

  • 53 Jim   June 5, 2011 at 3:22 pm

    We all know that some analysts at Gartner are extremely bias particularly avivah litan !..give me a break, avivah litan reports about fraud are ridiculous and dishonest (the winner is my friend and we share the same ?.)

  • 54 Tim   August 2, 2011 at 7:17 pm

    Avivah Lithan is more than bias…. selecting ac….. is a joke.
    BTW, ac…is known for using …

  • 55 JHis Biased   November 3, 2011 at 6:49 pm

    John Honovich is so hypocritical! This from a guy who was the Milestone Keynote Speaker in 2009! Have you seen how high Milestone ranks in his tests? Did you do the keynote for free John? You paid for your own hotel room and meals while at the Red Rock Casino?
    ————————————
    John Honovich October 9, 2009 at 6:23 pm

    To eliminate any concerns about vendor bias, how about Gartner eliminate consulting contracts and payments from vendors?

    I assume this is a naive question.

    However, such a move would provide the strongest possible financial incentivements and align with end users interest.

Leave a Comment