October 11th, 2019 by Nancy Erskine · No Comments
In this post, we’ll share a timely case study on how the Office of the Ombudsman receives, investigates and responds to complaints raised by technology providers, and why we always value getting our research right more than publishing to deadlines. This particular case study is based on the Gartner Magic Quadrant for Data Center Backup and Recovery Solutions, which was significantly delayed following a complaint made by one of the vendors included in the research.
First, why was this particular Magic Quadrant delayed? The reason is that we received a complaint, following Fact Review, from a vendor called Rubrik, claiming that our evaluation was inaccurate. As we investigated its claims, Rubrik later disclosed that one of the document’s authors had unsuccessfully applied for a job at its company about a year before the analyst had been assigned to the team for this particular Magic Quadrant. Rubrik claimed that this analyst was therefore biased against it.
We immediately investigated these concerns as well as the process the analysts followed to create the draft Magic Quadrant. We conducted an extensive review of all the interactions of the analyst in question, examined the veracity of peer and vendor review throughout the Magic Quadrant research process, and examined the data informing the underlying analysis. We also involved a senior analyst from another team, with extensive experience, to do a deep review of the analysis; he found nothing improper in the data, research process, analysis, the analyst’s interactions with Rubrik, the advice provided by this analyst to more than 100 end-user clients regarding this vendor, or the conclusions presented in the draft Magic Quadrant.
Following our investigation, we informed Rubrik of our detailed findings. We also explained that every piece of Gartner research represents more than one analyst’s opinion. The documented Gartner research process mandates collaboration, with both co-authors and other analysts within Research & Advisory, who provide input into the research. This is followed by rigorous peer review by analysts who are not directly involved in the specific research document but have relevant content knowledge. The draft is then shared with vendors for factual review. After this, research documents require final approval by management—all designed to create research that represents a consensus Gartner point of view—not a single analyst’s point of view.
Despite our investigation showing no evidence of bias for or against Rubrik, or any other provider included in the Magic Quadrant, out of an abundance of caution we made the decision to remove the analyst from the report. We assigned another analyst, with extensive knowledge of the market and the technologies in it, to conduct a full review of all original data and briefing materials, the scoring of all vendors involved, and the strengths and cautions for each company. He, too, agreed with the team’s conclusions and helped to finalize the Magic Quadrant. The original lead analyst played no role in this process.
Independence and objectivity are paramount attributes of Gartner research, so even the perception of a conflict of interest requires careful examination by the Office of the Ombudsman. Rubrik absolutely did the right thing to contact us and voice its concerns. In this case, it took a considerable amount of time to thoroughly investigate every single complaint raised by Rubrik, ultimately resulting in the assignment of a new lead analyst for this research. We are fully satisfied that Gartner’s rigorous research methodologies, combined with the actions taken by the Research & Advisory leadership team throughout this process, ensures all the vendors in this market segment are accurately—and fairly—evaluated relative to their competitors in the final Magic Quadrant.
Unfortunately, Rubrik does not agree with Gartner’s point of view expressed in the Magic Quadrant, but we respect the company’s right to voice its opinion. We believe Gartner’s opinion on vendor capabilities in this market is accurately expressed in the Magic Quadrant, a rigorous, independent analysis that helps buyers navigate technology purchase decisions.
Office of the Ombudsman
September 27th, 2019 by Nancy Erskine · Comments Off on What does it cost to be included in Gartner research? The answer is simple: nothing.
Guest Post from Frank DiPerna, Senior Ombudsman Specialist, Gartner
We’ve heard it before. Someone might say, “If you want to be included in Gartner research or improve your position, you have to spend at least [insert very high dollar figure here].” Not only is this untrue, but for Gartner to allow such a practice would destroy the foundation that our business is based on: providing truly independent, objective research to our clients. Status as a client has no bearing on research position—never has, never will. To get more specific, these claims often relate to the Magic Quadrant, part of a set of research written to help Gartner’s end-user clients make better purchasing decisions. The Research analysts who author these reports care only about providing relevant advice to end-user clients; anything beyond this would be antithetical to Gartner’s values.
So why does this mistaken notion persist in some circles? These false rumors may be spread by someone who disagrees with a research position, it could be the result of a misunderstanding, or perhaps a company seeks to grow its business by making outrageous claims against its competitors or Gartner to get attention. No matter the reason, whenever challenged on this issue, we respond assertively and confidently by referring to Gartner’s Ethical Principles, the Gartner research process and methodologies, and our Code of Conduct. We ask to speak directly with whomever has these concerns to set the record straight.
How can we so boldly proclaim this level of independence? Because all vendors, client or not, have the ability to engage with Gartner analysts through the Vendor Briefing process. In 2018 alone, more than 12,000 unique briefings were conducted with Gartner analysts. When considering whether to accept briefing requests, analysts consider only whether the information they will learn contributes to their research agenda—never whether the vendor is a client. Some types of research, such as the Magic Quadrant, also have strict inclusion and evaluation criteria that are applied to all vendors equally, and each vendor is evaluated with a consistent, rigorous methodology. If a vendor asks to be included in or excluded from a piece of research, we explain that it’s not up to them; only Gartner analysts, following our methodologies, decide which vendors will, or won’t, be included in our research.
A logical question is, why would a vendor want to become a client if it doesn’t somehow influence the outcome of our research? In a word: strategy. Gartner analysts speak to thousands of end-user clients every year about their unmet expectations and future needs. That provides Gartner analysts with unique insight on market trends that—when shared in aggregate because we never share confidential client information—can help vendors evolve their products and services. And who ultimately benefits? End users, because vendors are better able to meet their changing needs.
Want to drill even deeper? That’s why the Office of the Ombudsman exists. Contact us if you have any questions about independence, accuracy, or timeliness of any Gartner deliverable.
How to contact us:
In APAC: APACombudsman@gartner.com
In EMEA: EMEAombudsman@gartner.com
In LATAM: LATAMombudsman@gartner.com
Rest of World: email@example.com or +1(203)316-3334.
May 1st, 2018 by Nancy Erskine · Comments Off on What Happens When Analysts Leave Gartner? Hint: Planning for that day begins before the day they start.
Guest Post from Frank DiPerna, Ombudsman Specialist, Gartner
When research analysts leave Gartner for any reason (a well-deserved retirement, to start a completely new career, or to join an IT end-user or vendor organization), our office at times gets outreaches, usually from IT vendor organizations with which that analyst has interacted as clients. Here are a few things they ask:
- What happens to all of the upcoming client interactions scheduled?
- Are various stakeholders notified of the analyst’s departure?
- What steps does Gartner take to ensure client-confidential information is protected?
All valid questions, which I will answer. Before I do that, I’m going to start before an analyst—or any other Gartner associate—is hired. Prior to joining Gartner all associates must agree to, among other things, protect all client-confidential information they learn while at Gartner and even after they leave the company. There is no time limit on this obligation, and Gartner takes it very seriously. You see, our business is built on trust. The free flow of information between client and analyst is essential for clients to get the most out of their relationship with Gartner, which is why Gartner goes to great lengths to ensure analysts understand their obligations regarding confidentiality. And, beyond the policies and the training that occurs when associates join Gartner, there are yearly updates to the training and annual affirmation programs to ensure that our analysts understand and comply with those obligations while at Gartner and after they leave.
Even with all of that training, you might wonder, “How can someone just ‘unlearn’ or ignore confidential information they have acquired?” That is a tricky question – and one addressed in a previous blog post. Understanding what we reasonably can and can’t expect of departing employees has lead Gartner to the conclusion that consistent training and continual reinforcement makes it clear to every employee that we expect nothing short of total compliance. It is part of our culture.
Some vendors have asked why Gartner doesn’t take it a step further by not allowing analysts to join IT vendors when they leave Gartner. Remember, we don’t only receive confidential information from vendor clients. We have clients from all industries. That would essentially mean having to stop analysts from going just about anywhere. As you can see, this approach may sound ironclad, but it would make it all but impossible for an analyst to work anywhere after they leave Gartner.
Now back to the matter at hand: an analyst has given notice. Then what happens? First and foremost, that analyst is reminded of his or her post-employment confidentiality obligations with a letter from his or her manager. Then, the analyst’s manager shifts into “knowledge transfer” mode, identifying the most likely person to take over and bringing that individual up to speed. All upcoming interactions on the departing analyst’s schedule are transferred to others (including client inquiry and vendor briefings). The analyst is removed from all active and planned research projects. If the analyst covers an IT sector, the relevant vendors in that market are alerted that the analyst is leaving Gartner. All of this is designed to ensure that the process of delivering world-class insight to our clients proceeds as smoothly as possible, while removing the analyst from any new interactions in which he or she would be exposed to client-confidential information. Finally, Gartner’s Employment Counsel sends an additional letter to the analyst just before his or her last day, reinforcing the confidentiality obligations initially outlined by the analyst’s manager.
I hope this bit of detail was helpful. As always, if you have any questions, please don’t hesitate to contact us at firstname.lastname@example.org or +1(203) 316-3334.
February 28th, 2018 by Nancy Erskine · Comments Off on Words Matter: Gartner Takes Its Independence Very Seriously
Guest Post from Dean Daniels, MVP Ombudsman, Gartner
We were recently alerted to a press release, whose contents many who contacted us challenged. They said, “This doesn’t look like something Gartner would have approved.” They were right. The headline read:
Gartner Inc. will strengthen [Provider’s*] position as a global player in [market] (redacted)
We immediately contacted the client and demanded that the press release be removed. It was.
The release had been submitted to the Quote Requests team for review, but none of the changes required by the Quote Request Specialist were made before it was posted. The document described a “partnership” in which Gartner would “assist and support” the technology provider to “become a global player” in its market. It also contained a quote from a Gartner associate that had never been said. And it referenced outdated research excerpts.
Aside from the fact that this press release was not approved for publication, what is more concerning is that this language could be interpreted to mean that Gartner endorses this provider over others or that Gartner will do whatever it takes to make it successful, intimating that client status would help achieve this.
We do want our technology provider clients to succeed, but the way we do it is to arm them with the market insight they need to be successful in their businesses. We help them make decisions about strategy, product, marketing and sales decisions, every day. However, we do not provide a service that helps technology providers assess their applicability for inclusion in our research. And we definitely don’t partner with technology providers to advocate their products or services.
One other thing: We do provide support for press releases. Often, technology providers (client and non-client) want to leverage Gartner Research excerpts in press releases and other documents. That is perfectly fine, as long as the provider contacts our Quote Request Team (email email@example.com) to ensure that what the provider wants to say, and how it wants to say it, adheres to Gartner’s Copyright & Quote Policy.
*It is important to understand that Gartner does not disclose the names of its clients, which is why the name and industry has been redacted from the headline here. We take client confidentiality very seriously, and as a practice, we do not reveal anything about our clients without their express permission.
February 7th, 2018 by Nancy Erskine · Comments Off on UPDATE: Treatment of Company and Product Names in Gartner Research
Guest post by Jackie Ryan, VP of Content Production, Gartner
Providers sometimes ask why Gartner has rendered their company or product names differently from how the providers represent these names themselves. For example, our style is to change “Verint Systems Inc.” to be “Verint Systems” instead.
Gartner does strive to reproduce company and product names exactly, but we have long shied away from reproducing “textual effects” — such as backward letters, bold or italic type, and varying point sizes and fonts — and we will continue to do so. The foundation of our style remains the Associated Press (AP) Stylebook (see “company names”). However, we have decided to break with AP on the treatment of company and product names in Gartner Research in an effort to ensure that our research represents what our clients see in the marketplace.
What You Will See:
- Company names will be presented in full on first reference, as they appear on the “About the Company” page of the company’s website. Capitalization, spelling, spacing and punctuation will be reproduced exactly (unless the name uses hard-to-render characters, such as backward letters). When this page gives contradictory guidance, we will always go by what is used in the text blurb about the company, rather than the company name as presented in logos, addresses, copyright lines and elsewhere. If there is a mixture of long and short versions of the name in the text blurb, we will favor the long version.
- Product names will be treated the same as company names, and the product landing page will be considered the gold standard for determining the proper presentation.
- Ampersands will be used only if they are part of the company’s formal name, but not otherwise in place of “and.”
- A lowercase “the” will be used unless it is part of the company’s formal name.
- “Inc.,” “Corp.,” “GmbH” and similar abbreviations will be omitted, as will their expanded forms (e.g., “Incorporated”).
- Company and product names will be rendered as current at the time of publication. Gartner will not update published research after company rebranding efforts, mergers, or acquisitions.
Our editors and writers will begin implementing this updated guidance immediately — including content that is already in progress.
September 12th, 2017 by Nancy Erskine · Comments Off on Judge Grants Gartner Motion to Dismiss NetScout Law Suit
NetScout Systems, Inc. filed a lawsuit against Gartner in 2014, stemming from its unhappiness with its placement in a Gartner Magic Quadrant. We defended ourselves against the claims in the pre-trial processes, filing a motion for a summary judgment to dismiss the case earlier this year. Yesterday, our request to dismiss the case was granted in its entirety.
We are pleased with this result and continue to stand behind the research findings that were the subject of the technology provider’s complaint.
We take the independence and objectivity of our research very seriously. Without it, we wouldn’t have a business. Gartner is not “pay for play.” Influence over research content or the amount of research coverage focused on any vendor, sector or topic is not, and has never been, for sale by Gartner. Period.
Gartner has rigorous research methodologies that ensure our clients can trust the insight and advice we provide. Each piece of Gartner research is subject to a peer-review process by members of the worldwide analyst community, and review by research management is required prior to publication. This process is designed to surface any inconsistencies in research methodology, data collection and conclusions, as well as to fully use Gartner collective expertise on any research topic. Anyone can review our research methodologies in more detail here.
As always, we are committed to providing our clients with independent research and advice about the products and services that we cover and upon which they have relied for decades.
August 30th, 2017 by Nancy Erskine · Comments Off on Gartner Opinion Is Not For Sale
A company recently issued a press release that caught our attention. In the press release, the company stated that it had entered into an agreement with Gartner to provide paid research and analysis, assess one of its products regarding its applicability to the Magic Quadrant, and that Gartner will partner with this company to advocate its products. It caught our attention because none of it is correct.
The client did seek to become a client but Gartner was obliged, first and foremost, to correct the company’s misunderstanding and misportrayal. Gartner does not produce custom research, nor do we provide a service that helps technology providers assess their applicability for inclusion in our research methodologies. And we definitely don’t partner with technology providers to advocate their services. Never have, never will.
Gartner demanded that this post be removed. The company complied and acknowledged to us that its contents were inappropriate.
November 10th, 2015 by Nancy Erskine · Comments Off on Gartner Peer Insights: External sharing rules added to the Copyright and Quote Policy
Note: I updated this post on 13 Nov., 2015 to reflect the actual wording of the Copyright & Quote policy, which has been updated.
You are probably familiar with Gartner Peer Insights, a new offering by Gartner that allows IT end users to share their ratings and reviews of IT solutions. Based on discussions at Orlando Symposium, I know many of you are excited to begin sharing the ratings and reviews that end users have written about your own products and services. With this in mind, my team has been busy updating the Copyright & Quote Policy to accommodate your requests. We expect these updates to go live during the week of 16 November. In the meantime, this post includes the ground rules that will appear as “Section 3.7” in the Policy. Feel free to start operating by these guidelines right away to share excerpts from reviews and display your ratings:
- You may quote or display your company’s current Overall Peer Insights Ratings for markets that have been evaluated, as well as excerpt review comments about your company or its products. You may not quote or display the ratings or excerpt review comments made about any other provider, or use Gartner Peer Insights ratings or reviews to criticize a competitor.
- When excerpting from a Gartner Peer Insights review, the excerpt must also include the date of the review (for example, “On 23 Oct. 2015, a reviewer wrote, ‘[My Company] has the best product I’ve ever used.’”). Given that the nature of these reviews is so different from quoting Gartner research, when submitting review-excerpt requests you need only include the date the review was written and the product it was about (you do not need to provide further context about the reviewer).
- All excerpted reviews must be less than 12 months old.
- To quote ratings:
- At this point, you may quote only your Overall Peer Insights Rating; you may not share any sub-ratings (for example, you may not display your Integration and Deployment score – only your Overall Rating).
- When sharing an Overall Rating for your product, you must include the “as of” date and number of reviews on which the overall rating is based (for example, “As of 23 Oct., 2015, the overall Gartner Peer Insights rating for [Your Company] in the Advanced Analytics Platforms market is 4.7 out of 5, based on 53 reviews.”).
- When displaying the Overall Rating using the existing 5-star model and/or a histogram of the overall rating distribution, you must include the “as of” date, number of reviews on which the overall rating is based and provide a visual of how these will appear within your marketing material(s).
- Any references to the Gartner Peer Insights logo, ratings, charts and review-excerpts must include a link to the corresponding Gartner Peer Insights pages (Note: registration is required to see the details of individual reviews).
You MUST include this disclaimer with any Gartner Peer Insight content reference(s):
Gartner Peer Insights reviews constitute the subjective opinions of individual end users based on their own experiences and do not represent the views of Gartner or its affiliates.
Our branding team is also working on a logo and for now Gartner will be providing temporary branding. The logo parameters below will apply to the temporary branding:
Gartner Peer Insights Logo
The Gartner Peer Insights Logo is a trademark and service mark of Gartner Inc. and/or its affiliates and may only be used after securing express, written permission.
- You may use the Gartner Peer Insights Logo provided the legal disclaimer appears with the logo (see Section 3.7 for the disclaimer).
- The following legal trademark disclaimer must always appear with use of the Gartner Peer Insights Logo:The Gartner Peer Insights Logo is a trademark and service mark of Gartner, Inc., and/or its affiliates, and is used herein with permission. All rights reserved.
- The Gartner Peer Insights Logo may not be used to imply that your company has been given an award.
- Your logo must be the primary image (relative to both size and positioning) in your promotional materials (as shown in all examples in the Gartner Logo Usage Guidelines).
- The Gartner Peer Insights logo must be secondary in position to your logo and at least 10% smaller in size. Why?
- The Gartner Peer Insights logo may not appear in your press release, email signature stamps or securities offering documents (including annual reports). Why?
Please reference: Gartner Logo Usage Guidelines.
If you have any questions, feel free to contact my team: firstname.lastname@example.org. Happy quoting!
October 12th, 2015 by Nancy Erskine · Comments Off on Important Updates to Gartner Copyright & Quote Policy
We’re always looking for ways to improve the Copyright and Quote Policy, and a recent review showed a few areas that warranted an update — all with an eye toward reinforcing Gartner’s core principles of independence and objectivity.
- NEW – Magic Quadrant graphic parameters for homepage promotions (plus expanded guidance on quoting from MQ documents – Section 3.4a);
- Clarified rules around quoting material in limited audience documents (LADs — Section 2.1), securities offerings, and reports to shareholders (Section 2.5);
- New sections for:
- Gartner event booth signage (Section 2.6)
- Videos (Section 2.7)
- IT Key Metrics Data (Section 3.5)
- Toolkits (Section 3.6 )
- Gartner Trademarks/Logos (Section 2.9);
- Increased transparency regarding violations and an important update to the “assessment” guide (Section 6.4);
- Revamped Policy Essentials section to increase clarity (Section 1);
- Added supporting examples and explanations in several policy areas.
Here’s the detail behind the changes…
Magic Quadrant Guidance
- NEW! Clients may now display Magic Quadrant graphics on homepage banners, provided – of course – that licensing rights to the full report have been secured through Gartner Reprints
- Also new to Policy Section 3.4a are: detail around use of Magic Quadrant research, including common “pitfall” language” (for example, mistakenly calling positions “rankings” or “ratings”); examples of appropriate wording to describe placement and words to avoid; and sample documents with further details.
- We often get questions about what we view as a Limited Audience Documents (LADs). To quickly determine whether your material needs to be submitted for review, we have provided more detail on the four types of documents that do not require pre-approval in Section 2.1.
- To make the process easier to follow, we have restructured the Securities Offerings and Reports to Shareholders usage parameters with detailed processes and outlined conditions in Section 2.4.
- The Policy now provides guidance on IT Key Metrics Data in Section 3.5, which are for internal non-commercial use by licensed users only and not available for external quoting.
- To help you quickly identify whether a Toolkit is eligible for external usage (Section 3.6), we identified what to look for.
- We have added specific pre-production guidelines for including Gartner references in Gartner event booth signage and videos (see Sections 2.6 and 7, respectively).
- We have dedicated a section on the use of Gartner trademarks and logos under Section 2.9. We distinguished the differences between the Gartner Logo and the Gartner Cool Vendor Logo, included appropriate disclaimer text for each, and clarified proper usage for both.
Gartner takes violations of our Policy very seriously. While we have long had sanctions against those who externally share draft Magic Quadrant content, we have identified a second scenario that warrants an immediate, three-month quote and reprint blackout. Effective January 1, 2016: We will impose these bans whenever we discover that Gartner published research or references have been included in an earnings call without being pre-approved by Gartner Quote Requests (see Section 6.4).
We have reorganized, clarified and numbered the parameters outlined in the Policy Essentials (see Section 1). The content is not new, but it’s now easier to understand and identify.
Last year we introduced several clarifying samples of Do’s and Don’ts to key policy parameters that had appeared to be confusing to clients. These documents provided additional explanations, along with visual or textual examples of what would or would not be allowed. We received very positive feedback on these so we added new documents throughout to provide greater context and examples. Click on any “Why?” “Examples” or “here” links for this information.
Where to go with questions: If you have questions about the Policy, contact email@example.com. Please continue to send all quote requests (including reports of potential Policy violations) to firstname.lastname@example.org.
February 19th, 2015 by Nancy Erskine · Comments Off on Magic Quadrant Closed-Period for SAS
As many of you know, it has been a common practice for Gartner Research Managers to direct analysts on their teams who are authoring Magic Quadrants (MQs) not to participate, during the heart of any MQ process, in internal or remote SAS for vendors in the MQ.
Research leadership sees this as a best practice and thus has decided to make it a general policy, effective April 1, 2015. Here is the “MQ Closed-Period Policy”:
During the complete calendar month of planned publication for a particular MQ, and for the two complete preceding calendar months — that is, for a total period of three months — MQ authors will not participate in any internal or remote SAS with a vendor being evaluated in the particular MQ.
If you have any questions, feel free to contact your Account Executive or my team: email@example.com.