How Are We Doing Compared To Peers?

I learned something new the other day (yes, I love my job a lot for that reason). A high percentage of people I take inquiries from (called “dialogs” in our team due to its Burton roots) ask me: how are we doing compared to our peers? The first time I was asked that, it took me a bit by surprise. Sure, it is nice to know it, but how important is it really? It turned out it is really, really important for many organizations, and I hear that question very often.

Some want to know what organizations of their size are doing, others mostly care about what their peers in the industry are doing, etc. Once I’ve heard something to the effect of “our RISK is that, unless we know it, we’d spend more than our peers on security and thus lose our competitive edge” (by the way, this is a fictitious quote). Moreover, some organizations are not just “very interested” in this – they are literally obsessed with it. In fact, they won’t make any security decision, unless they know that their peers are doing it too.

As I result, I did some thinking about it and this is what came out:

• There are useful peer comparisons and then there are useless ones (example: % of IT budget spent on infosec is known to be low at both negligent AND efficient organizations)
• Donn Parker and his “diligence-based” security makes heavy use of “what others do successfully” for making decisions –  as with many Donn’s security insights, it probably comes out of 1970s
• Compliance is sort of a way to sidestep that question as “everybody should be doing the same” so instead of asking the peers, just go read the document. In reality, compliance turned this a bit on its head so that now people ask “what others do to become compliant?”, not “what problem does the regulation really intend to solve?”
• People want to know what worked/failed for others; not necessarily how exactly others solved their problems as there are key differences in IT environments. In fact, “what others are doing?” approach seems to over-emphasize the similarities between organizations and downplay the differences (“typical large enterprise”…. yeah right )
• It seems that the most desirable position is in the front side of the main pack – not with the leaders, not with the laggards and not in the true middle. People want to be doing a bit better than the average, but not much better. How peculiar is that?
• Security metrics and benchmarks are useful, but their massive value will be realized when they are shared at large scale across the organizations. And this has direct ties to security data sharing challenges.

Finally, Gartner has a few useful tools to connect to that information:

• ITScore Framework
• Gartner PeerConnect
• A note called “How to Determine If Your Organization Practices Due-Care Security”