Gartner Blog Network

Metrics, metrics who has the right metrics – well we have some data and five observations.

by Mark P. McDonald  |  August 22, 2012  |  3 Comments

In the 2012 Gartner Executive Programs CIO survey we asked CIOs several questions about the metrics they use, how effective the metrics were and how effective they were at demonstrating the business value of IT.  We shared the data with CIOs over the past year and some interesting insights came up that are the subject of this post.

Now, with 2013 planning underway it may help to review the metrics that CIOs see as the most effective in helping to set your metrics for 2013.

We asked CIOs which types of metrics they use most often and how effective their metrics were at demonstrating the business value of IT.  The chart below summarizes this information by looking at the percentage of IT organizations always using or making each type of metrics mandatory in their organization.

Anyone can measure something once, but the chart looks at the metrics that sit at the core of IT.  It’s a dense chart, so apologies ahead of time.

Here are some observations and supporting discussion based on a summary graphic that bring this into perspective.

#1 The more effective the IT organization is, the more effective its ability to demonstrate the business value of IT

It is common sense.  The more effective IT shops should be better able to demonstrate that effectiveness.  Effective IT in this case can be summarized into a single term – “we meet the commitments we make.”  This observation is more important in the opposite as no amount of metrics and reporting will hide poor performance.

CIOs spend too much effort trying to find a metrics to prove that we are not as bad as the business thinks we are – if only they understood or if only we could give them the right metric.  If you face this situation, take the management time and refocus it on raising IT effectiveness rather than providing a paper justification for your performance.

#2 Everyone measures scope, cost, schedule and budget, so these measures mean little in showing the business value of IT.

Scope, schedule, cost and budget reflect the core of IT measurement with these metrics mandatory at most organizations.  These metrics were once critical to demonstrating the business value of IT, particularly 25 or more years ago when the only thing the business could judge was how IT spent money.  That has all changed as IT’s value has gone public , the definition of IT success has gone more business of focused.

There is no business value in showing on time, on budget, on-scope, etc.  These metrics prove that you are doing your job.  In terms of value, all these metrics prove is that IT is not wasting the company’s money – a rather weak source of business value.

That is the logic behind the idea of IT budget as a percent of revenues , which happens to be a dubious metric at best.   If you want to consider how the business really thinks of IT costs, then take a look at IT budget as a percent of margin.  Yes margin at the OPEX level as people think of IT as something they can live without so its cost is cost to the bottom line as IT savings lead directly into increased profits – at least in the short term.   That is why people are obsessed with IT costs.

#3 Few IT organizations measure benefits realization, business results or technology adoption. 

The majority of CIOs believe that they can demonstrate the business value of IT without requiring IT to measure its business impact!  Here is the data based on looking at the percentage of respondents that say they use these metrics always or they are mandatory for IT.

  • 22% Metrics that measure IT’s contribution to business value or outcome, 78% do not regularly measure business value.
  • 25% Metrics that measure how IT reduces risk frequency and/or impact, 75% have no repeatable way of showing IT’s contribution to reducing risk.
  • 17% Metrics that compare IT against others, benchmarking, 83% do not regularly mark themselves to the market.
  • 10% Metrics that look at the adoption or deployment rates of new technology, 90% of organizations just put the technology out there and hope for the best?

It is no wonder proving the business value of IT is an issue – we simply do not measure business value.

Creating business value, reducing risk, being more effective than the competition and using technology all capture the levers that drive business value from technology.   If we do not measure it, give it visibility, then how can we expect it to happen on a regular basis?

Who uses externally oriented metrics on a regular basis to demonstrate the business value of IT?

#4 The more effective the organization, the more likely it is to use external metrics and focus on business value.

CIOs creating business value use business value based metrics.  The data shows a connection between the two, not causality so we can say that they go together not that one leads to the other.   Top performers in this case are those who believe that their organization is effective at demonstrating the business value of IT.

The information and conversations with CIOs indicates that the two go hand in hand.  They measure the business impact of IT because they want to improve it and as they improve it the measures become part of their standard operating model.

While there is no single metric to demonstrate value we know of one that destroys value:  IT budget as a percent of revenue.  Instead look for a single metric that reflects IT’s revenue impact and leverage – EBITDA/IT FTE

#5  Metrics are at the sole discretion of the CIO and IT leadership team.

You are what you measure and you determine the measures you use.  So we in IT are part of the problem as well as the source of the solution.  Creating metrics that challenge the organization, expose its weaknesses and lead it to change demonstrate CIO leadership.  You cannot improve what you cannot see or decide not to see.

Some suggestions in this area:

1)   Start measuring business value and outcomes for every aspect of IT.  Do not get caught up in false precision or trying to prove causality.  Simply raise the visibility of business results within your leadership team.

2)   Make business value a topic of discussion within your leadership team.  Look at improving IT’s ability to deliver on its ‘production function‘ rather than its budget and plan.

3)   Recognize that the value of IT accrues over time not at any one point in time. It takes a plan, focused attention and concentrated effort to change IT’s ability to create business value.

4)   Think small about IT and IT value as too often attention follows issues and size seems to matter, when in fact we deliver more value through focus and short sharp actions.

5)   Do not be afraid to show the need for improvement.  Setting IT on a business value/ business impact footing requires confronting reality and some uncomfortable facts.  It is better to show where you stand, how far you have come than allow perception, rumor and sentiment provide answers to this most important question.

A final thought:  If you found this analysis helpful, then I invite you to participate in the 2013 CIO Survey when it launches in early September.  Please look out for a post on this blog and other materials at that time with more details. 


Category: 2013  amplifying-the-enterprise  management  

Tags: 2013-planning  finance  it-management  metrics  

Mark P. McDonald
8 years at Gartner
24 years IT industry

Mark McDonald, Ph.D., is a former group vice president and head of research in Gartner Executive Programs. He is the co-author of The Social Organization with Anthony Bradley. Read Full Bio

Thoughts on Metrics, metrics who has the right metrics – well we have some data and five observations.

  1. Shaleen Shah says:

    And they say you have to measure so you can improve things for your business. Intriguing findings you have here – some folks may argue that creation of business value is outside the scope of IT.. When it comes to showing the need for improvement, you often encounter resistance to change and frustrations set in not because business goals are misaligned, but there is simply a lack of change. I love that you highlighted transparency here – to show facts even if you wouldn’t want to share it otherwise.

  2. TomTom says:

    On a micro level, the more complex enterprise environments become, the more important performance monitoring tools will become. Gaining reliable metrics on the performance of transactions across your entire IT infrastructure should be a chief goal as well.

  3. Mark P. McDonald says:


    Thanks for your comments and I want to address them specifically. The people who say that business value is outside the scope of IT are putting IT in a box labeled ‘administrative’, ‘overhead’, ‘minimize’, and ‘tolerate.’ You cannot say that IT is strategy without saying that It has a role in creating business value. Its simply counter-intuitive. It also gives IT a scapegoat, change management, which you mention.

    The transparency of metrics, incorporating them into the way you manage and make decisions, driving out fear and recrimination are all part of the solution. Retaining old beliefs that IT is an enabler, not responsibility, not contributing and subject to the whims of change management do not move IT or organizations forward. As I mentioned in the post, it takes courage to confront reality, realize that its your responsibility and take action.

    That is what the higher performers are doing, the ones whose metrics best demonstrate the business value of IT. That requires making a change in the way we see IT, evaluate IT and lead IT.

    Thanks again for your comments.


Comments are closed

Comments or opinions expressed on this blog are those of the individual contributors only, and do not necessarily represent the views of Gartner, Inc. or its management. Readers may copy and redistribute blog postings on other blogs, or otherwise for private, non-commercial or journalistic purposes, with attribution to Gartner. This content may not be used for any other purposes in any other formats or media. The content on this blog is provided on an "as-is" basis. Gartner shall not be liable for any damages whatsoever arising out of the content or use of this blog.