by Cory Sims
March 18, 2021
Gartner provided 2021 updates to its Peer Insights offering, including changes to its Voice of the Customer report methodology and review scoring calculations:
Gartner’s Peer Insights team hosted a January 2021 webinar to announce changes to its peer review site scoring calculations, VoC report methodology, and vendor engagement program. Webinar host and Peer Insights lead Anatoli Olkhovets focused the bulk of the presentation on updates (and upgrades) to the vendor engagement program and vendor-facing review management tools, though the meatiest updates for AR Pros came in updates to review scoring and VoC report methodology.
Gartner’s popular VoC reports, which feature the ‘Customer’s Choice’ distinction for each market’s top-rated offerings, will get a scoring overhaul starting in May 2021. To be featured as a Customer’s Choice vendor, companies:
There are now three equally-weighted elements to calculations for User Interest & Adoption scoring:
Gartner revised its methodology for these scores. Previously, the X-axis used to be calculated using the Product Adoption Review Coverage (PARC) Score, which focused only on review count and firmographics.
For VoC reports, product review scores only include reviews from the last 18 months. The scores give full weight to the reviews submitted (or updated) within the last 12 months and will give 50% weighting to those published in the first 6 months of the specified 18 month evaluation period.
Gartner is emphasizing scores for Willingness to Recommend, which stem from a review survey question asking reviewers if they would or would not recommend a product to a peer, mirroring the familiar NPS methodology with scores from 0 (not at all likely to recommend) to 10 (extremely likely to recommend). Previously, Gartner had used a modified Likert scale (yes, yes with reservations, I don’t know, and no) but will use the 10 point scale moving forward.
Gartner called out its ‘asymptotic function’ for User Interest & Adoption (X-axis) scoring, which it says will normalize scoring across each element regardless of total review count. For those meeting the minimum review count threshold (20 in the past 18 months), this ‘diminishing returns’ calculation will reward quality reviews over quantity of reviews.
Older reviews are still eligible to be updated. Reviewers are able to access their previous reviews and provide updates to text or scoring. When the review is accessed in this manner, it is removed from the site, and any changes go through the review moderation/approval process. It will then be republished with a new date, qualifying it for the 12-month cycle. Gartner says this is a very popular feature and the team is exploring ways to ‘streamline this process on the vendor side.’
Gartner also announced an update to its existing vendor engagement program, including a rebrand to ‘Customer First’ and an introduction of badge-based recognition.
Gartner said there will be no changes to the value provided to vendors participating in the Customer First program – each participating vendor will still get free access to readership analytics tools and advanced website capabilities.
Badges will be available for placement on vendor websites starting in March and will externally promote participation in the Customer First Peer Insights program. Badges will link directly to vendor profiles on the Peer Insights website.
If you’re looking for more information beyond these specific updates to Peer Insights, check out our Guide to Peer Review Sites. Included in this guide, we share an overview of the peer review sites landscape, best practices on management, as well as specific information on Gartner Peer Insights.
