Office of the Ombudsman

I’ve noticed lately that a number of vendors have taken draft documents they were sent during the Fact Check process and shared them (sometimes liberally) with people inside and outside of their organizations. We don’t allow this and push back strongly when we find out about these – I think for good reason.

A draft is just what that word implies — not ready for publication. It may change dramatically before it’s published: the opinions may change, even the backup data may change. Indeed, that’s why we ask vendors to review documents that include information about them: to make sure we’ve got all the facts right. And if you’ve read enough of these drafts you know they’re not copy-edited before we send them. So even if the content doesn’t change when they’re published, the wording and even structure might.

So what? You can probably tell we’re really sensitive to ownership of our IP — I’ve blogged about it, and we have a department of folks who work hard to ensure the rules are followed. But distributing drafts is worse than just breaking our external-use rules because the content isn’t even ready for the “external” world yet. So please, even if sorely tempted, resist the temptation to share until it’s been published… then follow these rules: http://tinyurl.com/lnubxs!

Gartner published research is copyrighted material. The Gartner Copyright and Quote Policy exists to protect this material when referenced in the public domain. We don’t allow indiscriminate, unauthorized distribution of research, and we also do our best to prevent anyone from misrepresenting or misusing Gartner opinions by taking them out of context. For example, we don’t allow vendors to say, “Gartner says we’re number one!” And we don’t let them disparage their competition by saying, “Gartner says they’re going out of business!” Most vendors respect our copyright and external use policies, as they wouldn’t appreciate companies abusing their intellectual property.
 
Riverbed Technologies has ignored multiple requests to respect our policy and our intellectual property by allowing their sales associates to distribute a Gartner published report in its entirety on more than one occasion, and by quoting excerpts from the same report to disparage their competition. Despite our repeated requests to stop, Riverbed continues to violate our external-use policies.
 
So what? We’re naming names to let everyone know that we do actively monitor use of our research, and will not ignore the unauthorized use of our name and redistribution of documents.

Yesterday a vendor tweeted, “New Question, how many Software Vendors in the Wave and Quadrant are not clients of Gartner and Forrester?”

We can’t speak for anyone else, but Gartner’s answer is that usually most, but not all, vendors in Magic Quadrants are Gartner clients in one way or another – such as exhibiting at our events or accessing analysts for advice to help them run a more successful business. But client status (or even the amount a client spends) is not a factor in determining inclusion or positioning in research reports. Research analysts cover vendors that, based on end-user requirements, are significant players in the product and service sectors we cover. Gartner analysts determine if vendors are relevant to the clients in their area of coverage; the vendors don’t decide if they want to be covered or not. All Gartner analysts are strictly governed by our Code of Ethics to ensure our research is independent and objective, and it’s my job to enforce it.

Do some vendors buy Gartner services expecting to be included in Gartner research or to improve their position in Magic Quadrants or Marketscopes? If they do, they quickly find that becoming a client doesn’t guarantee either of these. The main reason vendors buy Gartner services is to get access to our analysts – to bounce ideas off them, learn from what they see happening in the market, or get specific reactions to potential product or service plans. While any vendor can brief our analysts on new products or services, regardless of whether or not they are a client, the key missing ingredient is feedback, which only clients get. 

A related question vendors sometimes ask is, “Why doesn’t Gartner disclose which vendors are clients in Magic Quadrants or Marketscopes?” The answer is doing so would violate the confidentiality that Gartner affords its clients.

So what? It’s obvious from the fact that Gartner even has an Office of the Ombudsman that we take integrity and independence very seriously. The Office was put in place to help ensure that Gartner associates play by the rules – but also are able to act free of undue influence.

Recently, a vendor questioned the integrity of a piece of research, claiming that Gartner hired one of the contributing analysts from his competitor. We investigated the issue and didn’t find any evidence of bias, but it is reasonable to ask, “How does Gartner ensure that a new analyst hired from a vendor is not biased?” A related question is “How does Gartner protect the interest of vendors when an analyst leaves Gartner to go to one of their competitors?”

Gartner analysts typically have 10 to 20 years’ experience in their field of expertise so it’s common for Gartner to hire them from a vendor in the space. To ensure that the new analyst is able to deliver objective, balanced advice to Gartner clients, several safeguards are in place:

1. Analyst candidates go through an extensive interview process which includes individual interviews with other analysts and Research managers. There is also a grueling group interview where the candidate is required to prepare and defend a position. In the course of these interviews, the Gartner associates are looking for signs that the candidate is able to think broadly, has a balanced view of the marketplace, and can see both sides of an issue. A candidate who rants, derides or disparages vendors is not likely to get an offer.
2. New analysts don’t take on a full workload day one. It takes time (some say a year) to do research and gain perspective on current activities and trends in a particular market. New analysts follow a detailed on-boarding roadmap that teaches them about Gartner’s methodologies and processes, and introduces them to analyst activities in a measured, step-wise fashion. The roadmap takes the analyst through the first six months, and during that time the analyst’s manager and select peers help the analyst learn the ropes and become increasingly self-sufficient. So you won’t see a new analyst, freshly hired from a vendor, working on a Magic Quadrant in a substantial way without strong supervision.
3. As I covered in an earlier post, Gartner analysts have to abide by a strict set of ethical behavior rules. These prohibit the analyst and the analyst’s family from financial investments in vendors in their coverage area. They also include written acknowledgement that the analyst will not divulge trade-secret or other confidential information that is owned by previous employers.

On the back end, when an analyst leaves to go to a vendor in their coverage area, a specific set of actions takes place immediately, which includes:

1. Stopping the analyst from taking client inquiry calls, fulfilling vendor briefings or consulting related activities, or participating in any related e-mail distribution lists;
2. Reminding the analyst of client confidentiality obligations; and
3. Informing other vendors in the space that the analyst is leaving.

So what? While there’s no doubt that Gartner analysts can have very strong opinions, Gartner’s business processes strive to ensure that those opinions are the result of observation, detailed research, and collaboration — not the result of bias, experiences at a past employer, or some other purely personal interaction.

A vendor asked if we had a formal procedure that analysts follow to check user references provided by vendors and was specifically interested in whether analysts are obligated to contact the companies that the vendors provide as references.

Analysts are not obligated to ask for references when creating any type of document, but they often do because they believe that feedback will enrich their research findings and advice. When references are requested, Gartner has an established policy around checking them. Here are a few key highlights from the policy that were shared with the vendor:

If an analyst requests references from a vendor, that analyst is required to contact those references or advise the vendor which of the provided references will not be contacted. References are considered “contacted” once the analyst attempts to reach them (usually via phone, web survey or email), whether they respond or not. Analysts are encouraged to make more than one attempt to contact a reference if they don’t hear back, because anyone can forget to respond to a call or email.

If one or more references for a vendor does not respond, this would reflect negatively on the evaluation of the vendor and there is sufficient time, analysts generally give the vendor the opportunity to provide additional references. Note: The analysts are not required to reveal which references did not respond.

Here’s another relevant question we sometimes get: Why doesn’t Gartner contact references once a year, and in a more general way, so the answers can be shared between research teams? While that would certainly be easier for both parties, here’s the hard part: analysts strive to ask references questions that will give strong insight into how a product or service “really” works – or not. You can’t get at this information by asking generic questions, so the more specific the questions are, the more relevant the answers are to the research.

So what? Providing a list of high-quality references is hard work for vendors; it takes effort, skill and finesse. Knowing this, Gartner Research crafted its reference policies so these efforts are not wasted (and maintains them to ensure this, as well). It isn’t perfect, of course. How can it be improved? Let us know what you think. We’ll be sure to share your ideas with Research, who manages these policies.

An analyst called the other day and told us that a vendor client called her asking some questions about a specific end-user client — did she know the CIO, was she aware of their IT strategy, did she know whose equipment they had installed. Immediately this analyst became suspicious that the vendor was fishing for information because they were bidding on a deal with that end-user client.

She told the vendor she could respond to questions about the end-user client’s industry and the trends in end-user implementations, but could provide nothing more specific than that, as this could compromise confidential client discussions.

So what? Specific details of inquiry discussions may not be shared with anyone outside of Gartner. Clients know this and therefore feel comfortable coming to Gartner with even highly sensitive questions, because they know they will be held close.

We got a call from a vendor client who complained that an analyst who authored a report recently published by Gartner was obviously “against” this vendor. The client questioned why we would allow him to get away with this.

The answer is he didn’t get away with it. Gartner published opinions are just that: Gartner opinions. They are published after extensive peer review, management approval and copy editing for clarity.

Contrast this published opinion with the content posted by Gartner analysts on their blogs, which is clearly analysts’ personal opinion and not fully vetted. There’s a big difference, though I think there’s a good place for both. Frankly, I think there may be some confusion about this as analysts post more and more opinions on their blogs, but the alternative (not posting) would not be in keeping with Gartner’s position as a thought leader.

So what? Clients pay Gartner for fully vetted, forthright opinions. When it’s published Gartner research, it isn’t just one analyst’s opinion. And if you believe it is, that’s when you call the Office of the Ombudsman.

An analyst called and said she’d been asked by a vendor client to attend its “user group” meeting and present an award to the end user that this vendor had determined had the most advanced implementation of its product.

My team declined this request, as it could have looked like an endorsement by that analyst and therefore Gartner of this vendor – even if she hadn’t said anything about the vendor or its product in this presentation. The mere act of presenting an award “on behalf of” a vendor aligns Gartner too closely with that vendor. We’d give the same response if asked whether an analyst could participate in a press conference alongside a vendor.

Then why, you may ask, is it acceptable for a Gartner analyst to make any presentation at all at a vendor’s user group meeting? For one, there’s tremendous value for those users. And Gartner has got some pretty strict guidelines around what our analysts can present at these meetings: their materials must be based completely on content that has already been presented at a Gartner conference, and it must be industry-focused (that is, not focused on specific vendors or products). They can’t even answer questions from the audience that are specific to a products, vendors or competitors.

So what? Sometimes even the appearance of possible impropriety is enough for us to draw hard lines – even if no one intends anything inappropriate. We often invoke the “Wall Street Journal” test, where we ask whether we would feel comfortable if something we are considering doing or saying appeared on the front page of the Journal. If our answer is “Maybe not” or “No,” we decline.

An analyst joined Gartner from an IT Services company but his “coverage area” was not at all related to IT Services. About a year later, his coverage area changed and he is now covering IT Services. He asked me if he had to sell his stock in his former employer’s company.

The answer is yes. Gartner has a policy against analysts owning stock in any companies within their coverage areas. Even if this stock was granted during years of service and has been owned for a long time, that analyst had to divest himself of the shares in a timely manner (if his spouse owns the same kind of stock, that has to be sold, too).

So what? Stock ownership could create an enormous conflict of interest. It’s important for clients to know that analysts don’t have any vested financial interest in the companies they analyze or those companies’ competitors.

While I’m at Symposium, I thought it might be helpful to share the rules that Gartner has in place to ensure that Gartner’s research positions and event activities remain separate and distinct from vendor marketing activities. At Gartner events more than anywhere else, analysts and vendors intermix, so it’s imperative that all Gartner clients understand the nature of those relationships, and also that Gartner carefully monitors vendor activities. Client trust and analyst integrity the very cornerstones of Gartner’s business, so ensuring that there are no perceptions of impropriety is essential. Here are some examples:

1) Vendors are restricted in both the types and content of materials they can distribute during the Gartner event. For example, marketing materials require Gartner Vendor Relations approval and may not include language that appears to endorse a particular vendor or product.

2) Analysts may not attend pricey exclusive events sponsored by vendors. While we realize time constraints may require a business lunch or dinner, it is important that those interactions avoid even the appearance that an analyst is accepting gifts from vendors.

3) Analyst participation in Solution Provider Sessions is only permitted within established Gartner Sponsor guidelines. For example, in these presentations the analyst may only cover industry-general content – nothing vendor or product-specific.

4) While the costs of putting on an event as large as Symposium are offset somewhat by sponsor participation, sponsors have no input into the research process or material presented during the conference.

So what? Gartner analysts are independent and objective and work hard to earn and keep your trust. The rules we have in place help avoid even the appearance that anything else is true.