Office of the Ombuds

In this post, we’ll share a timely case study on how the Office of the Ombudsman receives, investigates and responds to complaints raised by technology providers, and why we always value getting our research right more than publishing to deadlines. This particular case study is based on the Gartner Magic Quadrant for Data Center Backup and Recovery Solutions, which was significantly delayed following a complaint made by one of the vendors included in the research.

First, why was this particular Magic Quadrant delayed? The reason is that we received a complaint, following Fact Review, from a vendor called Rubrik, claiming that our evaluation was inaccurate. As we investigated its claims, Rubrik later disclosed that one of the document’s authors had unsuccessfully applied for a job at its company about a year before the analyst had been assigned to the team for this particular Magic Quadrant. Rubrik claimed that this analyst was therefore biased against it.

We immediately investigated these concerns as well as the process the analysts followed to create the draft Magic Quadrant. We conducted an extensive review of all the interactions of the analyst in question, examined the veracity of peer and vendor review throughout the Magic Quadrant research process, and examined the data informing the underlying analysis. We also involved a senior analyst from another team, with extensive experience, to do a deep review of the analysis; he found nothing improper in the data, research process, analysis, the analyst’s interactions with Rubrik, the advice provided by this analyst to more than 100 end-user clients regarding this vendor, or the conclusions presented in the draft Magic Quadrant.

Following our investigation, we informed Rubrik of our detailed findings. We also explained that every piece of Gartner research represents more than one analyst’s opinion. The documented Gartner research process mandates collaboration, with both co-authors and other analysts within Research & Advisory, who provide input into the research. This is followed by rigorous peer review by analysts who are not directly involved in the specific research document but have relevant content knowledge. The draft is then shared with vendors for factual review. After this, research documents require final approval by management—all designed to create research that represents a consensus Gartner point of view—not a single analyst’s point of view.

Despite our investigation showing no evidence of bias for or against Rubrik, or any other provider included in the Magic Quadrant, out of an abundance of caution we made the decision to remove the analyst from the report. We assigned another analyst, with extensive knowledge of the market and the technologies in it, to conduct a full review of all original data and briefing materials, the scoring of all vendors involved, and the strengths and cautions for each company. He, too, agreed with the team’s conclusions and helped to finalize the Magic Quadrant. The original lead analyst played no role in this process.

Independence and objectivity are paramount attributes of Gartner research, so even the perception of a conflict of interest requires careful examination by the Office of the Ombudsman. Rubrik absolutely did the right thing to contact us and voice its concerns. In this case, it took a considerable amount of time to thoroughly investigate every single complaint raised by Rubrik, ultimately resulting in the assignment of a new lead analyst for this research. We are fully satisfied that Gartner’s rigorous research methodologies, combined with the actions taken by the Research & Advisory leadership team throughout this process, ensures all the vendors in this market segment are accurately—and fairly—evaluated relative to their competitors in the final Magic Quadrant.

Unfortunately, Rubrik does not agree with Gartner’s point of view expressed in the Magic Quadrant, but we respect the company’s right to voice its opinion. We believe Gartner’s opinion on vendor capabilities in this market is accurately expressed in the Magic Quadrant, a rigorous, independent analysis that helps buyers navigate technology purchase decisions.

Office of the Ombudsman