Office of the Ombudsman

Yesterday a vendor tweeted, “New Question, how many Software Vendors in the Wave and Quadrant are not clients of Gartner and Forrester?”

We can’t speak for anyone else, but Gartner’s answer is that usually most, but not all, vendors in Magic Quadrants are Gartner clients in one way or another – such as exhibiting at our events or accessing analysts for advice to help them run a more successful business. But client status (or even the amount a client spends) is not a factor in determining inclusion or positioning in research reports. Research analysts cover vendors that, based on end-user requirements, are significant players in the product and service sectors we cover. Gartner analysts determine if vendors are relevant to the clients in their area of coverage; the vendors don’t decide if they want to be covered or not. All Gartner analysts are strictly governed by our Code of Ethics to ensure our research is independent and objective, and it’s my job to enforce it.

Do some vendors buy Gartner services expecting to be included in Gartner research or to improve their position in Magic Quadrants or Marketscopes? If they do, they quickly find that becoming a client doesn’t guarantee either of these. The main reason vendors buy Gartner services is to get access to our analysts – to bounce ideas off them, learn from what they see happening in the market, or get specific reactions to potential product or service plans. While any vendor can brief our analysts on new products or services, regardless of whether or not they are a client, the key missing ingredient is feedback, which only clients get. 

A related question vendors sometimes ask is, “Why doesn’t Gartner disclose which vendors are clients in Magic Quadrants or Marketscopes?” The answer is doing so would violate the confidentiality that Gartner affords its clients.

So what? It’s obvious from the fact that Gartner even has an Office of the Ombudsman that we take integrity and independence very seriously. The Office was put in place to help ensure that Gartner associates play by the rules – but also are able to act free of undue influence.

Recently, a vendor questioned the integrity of a piece of research, claiming that Gartner hired one of the contributing analysts from his competitor. We investigated the issue and didn’t find any evidence of bias, but it is reasonable to ask, “How does Gartner ensure that a new analyst hired from a vendor is not biased?” A related question is “How does Gartner protect the interest of vendors when an analyst leaves Gartner to go to one of their competitors?”

Gartner analysts typically have 10 to 20 years’ experience in their field of expertise so it’s common for Gartner to hire them from a vendor in the space. To ensure that the new analyst is able to deliver objective, balanced advice to Gartner clients, several safeguards are in place:

1. Analyst candidates go through an extensive interview process which includes individual interviews with other analysts and Research managers. There is also a grueling group interview where the candidate is required to prepare and defend a position. In the course of these interviews, the Gartner associates are looking for signs that the candidate is able to think broadly, has a balanced view of the marketplace, and can see both sides of an issue. A candidate who rants, derides or disparages vendors is not likely to get an offer.
2. New analysts don’t take on a full workload day one. It takes time (some say a year) to do research and gain perspective on current activities and trends in a particular market. New analysts follow a detailed on-boarding roadmap that teaches them about Gartner’s methodologies and processes, and introduces them to analyst activities in a measured, step-wise fashion. The roadmap takes the analyst through the first six months, and during that time the analyst’s manager and select peers help the analyst learn the ropes and become increasingly self-sufficient. So you won’t see a new analyst, freshly hired from a vendor, working on a Magic Quadrant in a substantial way without strong supervision.
3. As I covered in an earlier post, Gartner analysts have to abide by a strict set of ethical behavior rules. These prohibit the analyst and the analyst’s family from financial investments in vendors in their coverage area. They also include written acknowledgement that the analyst will not divulge trade-secret or other confidential information that is owned by previous employers.

On the back end, when an analyst leaves to go to a vendor in their coverage area, a specific set of actions takes place immediately, which includes:

1. Stopping the analyst from taking client inquiry calls, fulfilling vendor briefings or consulting related activities, or participating in any related e-mail distribution lists;
2. Reminding the analyst of client confidentiality obligations; and
3. Informing other vendors in the space that the analyst is leaving.

So what? While there’s no doubt that Gartner analysts can have very strong opinions, Gartner’s business processes strive to ensure that those opinions are the result of observation, detailed research, and collaboration — not the result of bias, experiences at a past employer, or some other purely personal interaction.

A vendor asked if we had a formal procedure that analysts follow to check user references provided by vendors and was specifically interested in whether analysts are obligated to contact the companies that the vendors provide as references.

Analysts are not obligated to ask for references when creating any type of document, but they often do because they believe that feedback will enrich their research findings and advice. When references are requested, Gartner has an established policy around checking them. Here are a few key highlights from the policy that were shared with the vendor:

If an analyst requests references from a vendor, that analyst is required to contact those references or advise the vendor which of the provided references will not be contacted. References are considered “contacted” once the analyst attempts to reach them (usually via phone, web survey or email), whether they respond or not. Analysts are encouraged to make more than one attempt to contact a reference if they don’t hear back, because anyone can forget to respond to a call or email.

If one or more references for a vendor does not respond, this would reflect negatively on the evaluation of the vendor and there is sufficient time, analysts generally give the vendor the opportunity to provide additional references. Note: The analysts are not required to reveal which references did not respond.

Here’s another relevant question we sometimes get: Why doesn’t Gartner contact references once a year, and in a more general way, so the answers can be shared between research teams? While that would certainly be easier for both parties, here’s the hard part: analysts strive to ask references questions that will give strong insight into how a product or service “really” works – or not. You can’t get at this information by asking generic questions, so the more specific the questions are, the more relevant the answers are to the research.

So what? Providing a list of high-quality references is hard work for vendors; it takes effort, skill and finesse. Knowing this, Gartner Research crafted its reference policies so these efforts are not wasted (and maintains them to ensure this, as well). It isn’t perfect, of course. How can it be improved? Let us know what you think. We’ll be sure to share your ideas with Research, who manages these policies.

An analyst called the other day and told us that a vendor client called her asking some questions about a specific end-user client — did she know the CIO, was she aware of their IT strategy, did she know whose equipment they had installed. Immediately this analyst became suspicious that the vendor was fishing for information because they were bidding on a deal with that end-user client.

She told the vendor she could respond to questions about the end-user client’s industry and the trends in end-user implementations, but could provide nothing more specific than that, as this could compromise confidential client discussions.

So what? Specific details of inquiry discussions may not be shared with anyone outside of Gartner. Clients know this and therefore feel comfortable coming to Gartner with even highly sensitive questions, because they know they will be held close.

We got a call from a vendor client who complained that an analyst who authored a report recently published by Gartner was obviously “against” this vendor. The client questioned why we would allow him to get away with this.

The answer is he didn’t get away with it. Gartner published opinions are just that: Gartner opinions. They are published after extensive peer review, management approval and copy editing for clarity.

Contrast this published opinion with the content posted by Gartner analysts on their blogs, which is clearly analysts’ personal opinion and not fully vetted. There’s a big difference, though I think there’s a good place for both. Frankly, I think there may be some confusion about this as analysts post more and more opinions on their blogs, but the alternative (not posting) would not be in keeping with Gartner’s position as a thought leader.

So what? Clients pay Gartner for fully vetted, forthright opinions. When it’s published Gartner research, it isn’t just one analyst’s opinion. And if you believe it is, that’s when you call the Office of the Ombudsman.

An analyst called and said she’d been asked by a vendor client to attend its “user group” meeting and present an award to the end user that this vendor had determined had the most advanced implementation of its product.

My team declined this request, as it could have looked like an endorsement by that analyst and therefore Gartner of this vendor – even if she hadn’t said anything about the vendor or its product in this presentation. The mere act of presenting an award “on behalf of” a vendor aligns Gartner too closely with that vendor. We’d give the same response if asked whether an analyst could participate in a press conference alongside a vendor.

Then why, you may ask, is it acceptable for a Gartner analyst to make any presentation at all at a vendor’s user group meeting? For one, there’s tremendous value for those users. And Gartner has got some pretty strict guidelines around what our analysts can present at these meetings: their materials must be based completely on content that has already been presented at a Gartner conference, and it must be industry-focused (that is, not focused on specific vendors or products). They can’t even answer questions from the audience that are specific to a products, vendors or competitors.

So what? Sometimes even the appearance of possible impropriety is enough for us to draw hard lines – even if no one intends anything inappropriate. We often invoke the “Wall Street Journal” test, where we ask whether we would feel comfortable if something we are considering doing or saying appeared on the front page of the Journal. If our answer is “Maybe not” or “No,” we decline.

An analyst joined Gartner from an IT Services company but his “coverage area” was not at all related to IT Services. About a year later, his coverage area changed and he is now covering IT Services. He asked me if he had to sell his stock in his former employer’s company.

The answer is yes. Gartner has a policy against analysts owning stock in any companies within their coverage areas. Even if this stock was granted during years of service and has been owned for a long time, that analyst had to divest himself of the shares in a timely manner (if his spouse owns the same kind of stock, that has to be sold, too).

So what? Stock ownership could create an enormous conflict of interest. It’s important for clients to know that analysts don’t have any vested financial interest in the companies they analyze or those companies’ competitors.

While I’m at Symposium, I thought it might be helpful to share the rules that Gartner has in place to ensure that Gartner’s research positions and event activities remain separate and distinct from vendor marketing activities. At Gartner events more than anywhere else, analysts and vendors intermix, so it’s imperative that all Gartner clients understand the nature of those relationships, and also that Gartner carefully monitors vendor activities. Client trust and analyst integrity the very cornerstones of Gartner’s business, so ensuring that there are no perceptions of impropriety is essential. Here are some examples:

1) Vendors are restricted in both the types and content of materials they can distribute during the Gartner event. For example, marketing materials require Gartner Vendor Relations approval and may not include language that appears to endorse a particular vendor or product.

2) Analysts may not attend pricey exclusive events sponsored by vendors. While we realize time constraints may require a business lunch or dinner, it is important that those interactions avoid even the appearance that an analyst is accepting gifts from vendors.

3) Analyst participation in Solution Provider Sessions is only permitted within established Gartner Sponsor guidelines. For example, in these presentations the analyst may only cover industry-general content – nothing vendor or product-specific.

4) While the costs of putting on an event as large as Symposium are offset somewhat by sponsor participation, sponsors have no input into the research process or material presented during the conference.

So what? Gartner analysts are independent and objective and work hard to earn and keep your trust. The rules we have in place help avoid even the appearance that anything else is true.

A vendor client of Gartner had a full-day session with an analyst where the vendor presented its detailed upcoming product plans and got lots of feedback from this analyst (Gartner calls it “SAS” – or Strategic Advisory Services). Two weeks later, the analyst announced that he was leaving Gartner to join a major competitor of this vendor.

When the vendor who had paid for the SAS found out the analyst was going to its competitor, they contacted us, concerned that the non-disclosure (NDA) information shared with him would now be shared with their arch-enemy — and what was Gartner going to do about it?

Well, one thing was done before that analyst walked through the doors on his first day of work: he signed an agreement that bound him not to tell anyone outside Gartner any confidential information he learned while working at Gartner. That obligation to maintain the confidentiality of this information doesn’t end when his employment does. When he gave notice, he was also reminded by his manager that he signed this agreement and that it’s legally binding.

So what? Protecting confidential client information is one a cornerstone of the Gartner Principles of Ethical Conduct. That obligation to keep confidential information confidential doesn’t end when employment with Gartner does.

Shortly after Gartner first launched the Office of the Ombudsman, the Office also started a blog, with the idea of being completely transparent about the issues of objectivity, completeness and accuracy that cross our desks every day. It sounded good “on paper,” but many issues turned out to be so sensitive and potentially explosive in the wrong context that the blog saw few posts, and then none. We simply didn’t want to run the risk that we would compromise the very ability to maintain the confidentiality that Gartner is known for.

Recently I started thinking that it would be valuable to share some aspects of the issues that come our way (about 100 a month, as it’s turned out), because it will get to the heart of why Gartner has the rules it has, the processes it follows, and perhaps even its reputation.

I’m hoping that you’ll read these posts and share your reactions and suggestions, because it’s always good to look hard at what you do every day and make sure it’s still absolutely the right thing to do – and if it’s not, make it better. In the true spirit of blogging, I’ve decided to make this a personal blog and not an “Office” one, so you feel like you’re reading an actual person’s words and, when you respond, which I hope you do, you’ll know I am reading your response.