Gartner Blog Network

Collective Intelligence Benchmarking

by Andrew Lerner  |  January 20, 2017  |  Submit a Comment

Traditional monitoring tools often struggle to correlate the myriad of datasets resulting from increasingly distributed applications and services beyond the control of enterprise IT, and this will only get harder…

To that end, Collective intelligence benchmarking (CIB) is an emerging and increasingly popular approach being adopted by monitoring tool vendors to ascertain a baseline of performance for a given service or application, based upon aggregated (often network) data from hundreds/thousands of end users.  The aggregated data can be internal or external, although shared datasets from third-party end users is anonymized.

This is a technology that networking professionals should keep an eye on. There are a variety of usage scenarios, but one that stands out to me is network configuration benchmarking. For example, CIB tools can alert when the configuration of network devices strays from best practices or when there are opportunities for performance and utilization to be improved.  In other words, it would be very cool if the network would send me an email telling me that my configurations were suboptimal and recommend specific changes – that’s where it’s at. There are a variety of vendors, with differing approaches including (but not limited to) Indeni, Lakeside Software,, Nyansa, ThousandEyes, and Unify Square. We/Gartner (specifically Vivek Bhalla and Will Cappelli) just published research on the topic:

Innovation Insight for Collective Intelligence Benchmarking,

Summary: Demand for benchmarking of monitoring KPIs and metrics is on the rise. While monitoring tools look to deliver more consistent and standardized means of comparison, approaches vary. I&O leaders must determine key criteria and considerations when identifying tools to meet this growing requirement.

regards, Andrew

Category: just-published  networking  

Tags: cib  indeni  lakeside-software  logz-io  nyansa  thousandeyes  unify-square  

Andrew Lerner
Research Vice President
4 years at Gartner
19 years IT Industry

Andrew Lerner is a Vice President in Gartner Research. He covers enterprise networking, including data center, campus and WAN with a focus on emerging technologies (SDN, SD-WAN, and Intent-based networking). Read Full Bio

Leave a Reply

Your email address will not be published. Required fields are marked *

Comments or opinions expressed on this blog are those of the individual contributors only, and do not necessarily represent the views of Gartner, Inc. or its management. Readers may copy and redistribute blog postings on other blogs, or otherwise for private, non-commercial or journalistic purposes, with attribution to Gartner. This content may not be used for any other purposes in any other formats or media. The content on this blog is provided on an "as-is" basis. Gartner shall not be liable for any damages whatsoever arising out of the content or use of this blog.