We developed the PI research note as we heard consistently from end-users that they need a better shortlisting tool in a format they could easily share with their investment committees. The research note synthesizes the key insights from all the information available on the site using our proprietary methodology.
Although the data is available on site, the images that synthesize the information is not available. For example, the User Interest and Adoption axis in the main quadrant (Figure 1) is only available through this document.
It includes only a smaller set of eligible vendors, and shows their ratings over a fixed period of time (last 18 months) vs. the PI site which is continuously changing as reviews come in.
Many vendors who license the reprint of this report, often use it for lead generation. You cannot do that with Peer Insights site.
Purchasing a reprints license gives you access to a dedicated program manager who will handle all of the legwork for quote request approvals and help you to track the reprint impact
Methodology related (including CCD)
Which VOC methodology is currently used? Will the methodology change from year to year?
Our goal is to be analytically sound and transparent. To meet these goals, we expect our methodology will evolve over time as we learn from the process. We are committed to being completely transparent about our criteria for this distinction. View latest methodology and historical list of all methodologies. The new methodology will go into effect for all Voice of the Customer documents published in May 2021 and later.
What are the inclusion criteria in a VOC?
To be included in the “Voice of the Customer” report for a Gartner Peer Insights market, a vendor must have a product that aligns to that market as determined by Gartner Peer Insights and have 20 or more eligible reviews during the specified 18-month submission period. Please refer VOC methodology for full details.
How and when is market average calculated?
The average market rating is determined by all eligible reviews submitted in the 18-month review period and therefore can change until the end of the review period as reviews are moderated and published. Vendors can view whether they are above, below, or in-range of market average for both axes on the Standings Page (updated daily). For more information, see the detailed methodology.
Does the overall market rating on the Y-axis use the same calculations as what we will see with the new site scoring updates?
Voice of the Customer Y-axis calculations use a specified 18-month time period with a lower weight for the oldest 6 months of that period. The star rating displayed on the Gartner Peer Insights reviews page shows either past 12 months or an all time average, with a lower weight for earlier years. Thus, the time periods for the Voice of the Customer and the Gartner Peer Insights reviews page will typically be different. For more information, see the detailed methodology.
Is there a limit to the maximum number of vendors that can be listed in a Voice of the Customer document for a market?
Yes, there is a limit. The Voice of the Customer document was specifically created to help end-users shortlist vendors in their buying/selection process. Currently, Gartner Peer Insights limits the number of vendors included in the document to 25. If more than 25 vendors meet the inclusion criteria, then 25 vendors with the highest number of eligible published reviews will be included. If two or more qualified vendors with the same number of eligible reviews tie for the 25th position, all vendors in the tie will be included in the document.
For more details, please go to the Voice of the Customer Methodology page.
What are the inclusion criteria for Category Cuts?
A vendor’s inclusion in category cuts will remain independent of representation in the overall Voice of the Customer i.e, in a scenario where a vendor is excluded from ‘Voice of the Customer’ due to the maximum vendor limit, the vendor will continue to exist in category cuts if eligibility criteria for the category are met. See the category distinction methodology here.
What is User Interest & adoption?
Please refer to the Voice of the Customer methodology here for full details
How are vendors/products included in a VOC report?
A dedicated Peer Insights content team evaluates whether a technology provider’s products and/or services align with the Peer Insights markets based on a series of market guidelines defined by Gartner experts. The Peer Insights markets are periodically reviewed throughout the year to ensure alignment is maintained. For more details, refer to the Peer Insights listing guidelines.
Are reviews for products that are no longer sold included in the distinction analysis? What about reviews for products that are in beta test?
Reviews for products that are in production / general availability, along with beta products are included in calculations for the Voice of the Customer. Reviews for legacy products (that are no longer sold by the vendor) are not included in these calculations.
Why are you excluding reviews from companies that are less than 50M USD in annual revenue?
Gartner Peer Insights was developed with the global enterprise customer in mind. We define enterprise customers whose company sizes are greater than 50M USD in annual revenue.
For how long is the distinction current?
The Voice of the Customer document and Customers’ Choice distinction will be current for 15 months or until the next version of the Voice of the Customer for the market is published. Documents that are no longer current will remain visible on Gartner.com with an “Archived” tag. Our schedule for Voice of the Customer publication is determined based on the publish date of the corresponding MQ/MG and is subject to change based on the research calendar at the discretion of the Peer Insights team in conjunction with Research.
Right now the reviews come sourced by Gartner as well as from vendors’ sourcing links. Aren’t vendor-sourced reviews inherently biased because vendors select their best clients and offer them incentives to write reviews?
Our interviews with reviewers suggest that most of them are motivated by a desire to give back to the broader IT community. We’ve decided to include honest vendor-incentivized reviews as part of the relevant data set used to determine the distinctions. Our vetting process for reviews is extensive (see our FAQs for a summary), and we investigate any anomalies to ensure that users can trust the reviews in Gartner Peer Insights.
Can I see my status for eligibility towards Customers’ Choice in a specific market?
When a specific Peer Insights market is about 2 months out from a Voice of Customer sourcing deadline, the vendor Dashboard on the Technology Provider Tools portal will update to show eligibility towards inclusion in the Voice of the Customer and Customers’ Choice in that market. This information is dynamic, updating daily. Please note, the information shown in the Dashboard is not final and may change after sourcing periods close as part of our moderation and auditing process.
Are vendor ratings rounded/truncated in the VOC?
Yes. The market average Overall Rating is calculated by averaging all eligible vendors’ Overall Ratings. All vendor ratings and the market average Overall Rating are then truncated to the tenths place (e.g., ‘4.512’ becomes ‘4.5’). Vendors whose truncated scores meet or exceed the truncated market average are positioned in the upper-half quadrants, while vendors below the truncated market average are positioned in the bottom-half quadrants. These truncated scores are used to position vendors in the Voice of the Customer quadrants and are listed in Figure 2 in the Voice of the Customer document.
Are more recent reviews weighted more heavily in VOC calculations and scores?
Reviews from the first 6 months of the 18-month submission eligibility period will be given 50% weight in all calculations for both axes. This means that one review from the first 6 months will have half the impact of one review from the next 12 months in determining a vendor’s X-axis and Y-axis scores. The 18-month review period allows for a larger sample size of reviews and greater confidence in each vendor’s placement on the quadrant. The 50% weighting for reviews in the first 6 months allows more recent reviews to factor more heavily in determining a vendor’s placement.
How is sensitivity to review volume managed?
An asymptotic function is applied to reduce sensitivity to small changes in review count and review market coverage, especially for vendors with a large number of reviews. This means that for a vendor with a low review count (e.g., 20 reviews) who adds 10 more reviews will see a larger increase in review volume and review market coverage than a vendor with a high review count (e.g., 100 reviews).
What if I have more questions?
If you have any further questions, send us an email at PeerInsights@gartner.com and we will get back to you as soon as we can.