Benchmarking is a hot topic among CIOs and IT leaders as they look to evaluate and justify their operations by answering the question — ‘how do we compare with others?”
Increasing change creates demand for increasing benchmarking as companies value peer comparisons over their own experience and history. This bias makes it surprisingly easy for executives to misuse benchmarking in making these decisions.
There are two types of benchmarking: Peer and Historical. Peer benchmarking is what we normally think about when we think benchmarking against our industry. Historical benchmarking compares your company performance over time.
Peer Benchmarking helps you ‘mark to the market’, but not drive strategy and commitments.
Peer benchmarking helps you answer questions concerning how do we compare to others in our industry? Are we above or below industry peers? How will or relative position make or break our decisions and strategies?
This information is helpful in understanding where you stand. Peer benchmarking advocates will point to the fact that the only way to be ‘world class’ is to know what the rest of the world is doing and then beat it. A problem is that where you stand is not a powerful enough answer to drive your strategy, to make major decisions, to deliver commitments on value/performance/cost etc. Companies that look to others for direction have an issue with cerebral non-viability, aka ‘brain dead.’
Peer benchmarks are less powerful for the simple reason that executives use peer benchmark information to do one thing — keep IT Cost/Spend low because the logic of the peer benchmark drives decisions in a single direction.
- Organizations rarely recognize the legitimate reasons why your IT costs may be higher than the industry peer benchmark average. They see the gap between your high costs and the lower industry average as a source of future savings. Peer data leads to a mandate to get back within industry averages because the peers are assumed to be right.
- If peer metrics were an intellectually honest metric, then situations where you underspend your peers would support increasing the IT budget. Now if you just smiled at that sentence you know that is not the case. If your spending is below peers, the answer all to often is that others are foolish and we are the ones getting the best ‘value for money.”
I can see where executives would ask for a peer based benchmark as they either feel that they are getting relatively poor service/capability or that IT spending levels are out of whack. Peer information is helpful but cannot determine or even drive your strategy because you have no choice when comparing yourself to peers based on the reasoning discussed above. Historical based benchmarks do not have this issue.
Historical Benchmarking defines performance support making commitments
A historical benchmark compares current performance against your past. “Marking yourself to yourself’ may sound self serving and a way of blinding yourself to knowing market performance. They are right, but here is the counterpoint.
Comparing yourself to peers only tells you how different you are from everyone else. That can be helpful to know in situations where you have no idea of market performance. Knowing the gap does not create value. Value comes form changing your performance and demonstrating that change is the domain of a history based benchmark.
Historical based benchmarking gives your management systems information they can use. Tools like control charts or statistical process control (SPC) use changes in actual performance to drive decisions, actions and change all things that cannot be supported through peer benchmarking.
Benchmarking is big enough to support peer and historically based comparison. The different benchmarks answer different questions. One looking outside to performance beyond your company and the other inside into the actual improvement in what you do and how you do it. Recognizing the difference between the two and how best to use them is essential to building your goals (peer) and to making the improvements that demonstrate value (history).
Which is the more powerful peers or the past? Well only one feeds the management decisions that drive value as the only way you get better is by being better than you were in the past regardless of how that stacks up in the broader market.
Comments or opinions expressed on this blog are those of the individual contributors only, and do not necessarily represent the views of Gartner, Inc. or its management. Readers may copy and redistribute blog postings on other blogs, or otherwise for private, non-commercial or journalistic purposes, with attribution to Gartner. This content may not be used for any other purposes in any other formats or media. The content on this blog is provided on an "as-is" basis. Gartner shall not be liable for any damages whatsoever arising out of the content or use of this blog.