Crowd sourcing has become the norm from everything from hotel reviews to ride sharing to vacation stays, and now to BI software— with Gartner’s just launched Peer Insights.
I’ll confess, I have been a skeptic of some of the other crowd-sourced review sites. If I don’t know who’s providing the review, what their reference point is, and how the website prevents a stuffing of the ballot box, I am wary. While Peer Insights has been in stealth mode, reviewers were restricted to Gartner customers, giving a degree of quality control. As of Friday, Gartner has opened up access to the public. Reviewers must use a company email address, and the Peer Insights team continues to check the validity of each reviewer.
So what does this mean for you?
First, if you are a BI expert or user, give Peer Insights a try and rate your vendor and product.
Second, if you are a BI buyer, use Peer Insights as just one more data point in your evaluation process. Crowd-sourced sites are a great source for qualitative points such as customer satisfaction, ease of use, or effectiveness of technical support. When looking at product capabilities, be sure to consider if the reviewer disclosed what version of the product they were on.
Third, use the Magic Quadrant and Critical Capabilities reports as additional data points that combines expert analyst opinion with reference customer opinion. Keep these factors in mind when reviewing peer ratings:
- Pay attention to the disconnects or where there seems to be a difference of opinion between Peer Insights and the analyst opinion. We DO expect this to happen,and it’s in the understanding of the differences that may reveal a product’s biggest gotcha or a recent major enhancement. For example, in this year’s Critical Capabilities, we saw that many SAP customers rated their mobile BI capabilities lower than we analysts did. This is both because customers were on older versions and because the vendor was not communicating the rapid changes as well to their customers. Likewise, in working on a research note for Domo (due out soon!), customers complained that the product lacked row level security. We found it – but indeed, it may not have been enabled for all customer instances.
- Check the reviewer demographics. One reason I trust analyst opinion more than customer surveys is because I know the MQ team is working from a consistent definition of a capability. The same is not necessarily true of customers. For example, if I ask an American who has the best chocolate, they may say Hersheys, an Englishman Cadburys, and a Swiss, the truffes du jour, made only in Switzerland with fresh cream and a three day shelf life. And trust me, once you’ve had those truffles, it’s hard to go back to chocolate by the pound! So in checking Peer Insights, be sure to know the experience and perspective of the reviewer.
- Watch for the halo effect. As I wrote in this blog, customers who are happy with their vendor and product seem to rate product capabilities higher, sometimes too high. Conversely, customers angry about poor tech support may be overly critical about capabilities. This is not to say that the extremes don’t matter; you just need to be aware of when emotions are skewing the results.
I am also most excited that Peer Insights will provide some additional insights into smaller vendors where we just don’t have enough dots on the MQ to cover them.
Let me know what you think!
View Free, Relevant Gartner Research
Gartner's research helps you cut through the complexity and deliver the knowledge you need to make the right decisions quickly, and with confidence.Read Free Gartner Research
Comments or opinions expressed on this blog are those of the individual contributors only, and do not necessarily represent the views of Gartner, Inc. or its management. Readers may copy and redistribute blog postings on other blogs, or otherwise for private, non-commercial or journalistic purposes, with attribution to Gartner. This content may not be used for any other purposes in any other formats or media. The content on this blog is provided on an "as-is" basis. Gartner shall not be liable for any damages whatsoever arising out of the content or use of this blog.