How Should We Measure Our SOC?

There is a growing number of organizations out there, both big and small, that want to make an investment in a Security Operations Centre (SOC). Most organizations don’t really plan for the significant spending and the internal effort involved in doing so properly. Many go ahead and throw money at the problem without carefully thinking about how to measure the long term success. Whilst the business case that won the funding was valid, future investment, continued growth, and maintenance fall by the wayside. Our SOCs often get worse over time.

Security (and the SOC) exists to protect our business objectives.

The first thing we need to think about is how will we represent, track and communicate successes. Most of our organizations are not security businesses, so it doesn’t make sense to track security things. No CEO wants to hear about how many phishing attempts there were, no matter how funny the email was. We need to start thinking a little more about how security reporting is contextualized to our own business drivers.

For most businesses, security means protecting efficiency, intellectual property, and availability of key systems. Therefore it makes sense that this is the first place we go when we think about tracking SOC metrics. How do we show we helped maintain efficiency? How do we recognize that we have protected our intellectual property? Are we stopping a negative impact on the availability of key systems?

When we present these metrics to the business, we need to do so in a way that:

• Shows that we have combatted business risk, or are combatting issues that jeopardize those things we need to protect.
• Shows that we are getting better over time, creating headroom to do more.
• Relates to a quantifiable measure, such as a $ value or a time-saving for the business.

Pyramid metrics are so 90’s

No one ever cared about how many billions of events your toolset processed, so forget these types of easy metrics. We need to focus on the ‘needs of the business’ and not the need for security. The metrics we have been focussing on reporting not only mean nothing to our organization but in many cases, mean nothing to the security staff as we have no effective way of improving them.

We must focus on measuring the very top of the pyramid. Use everything below this to inform how to configure/improve the toolset and the processes, but not to measure security. “The Stuff That Needs Fixing” should be categorized in-line with specific business risk. It should have a well-developed response process associated with it. To have an impact, we must measure and report on how we helped maintain or improved the productivity of our organization as a whole, not just how well security is doing.

SOC performance is different from Security Metrics

We, of course, need to measure our own performance within the SOC and we can do that in three ways:

• Track metrics in line with that time, compare what happened last month, with what happened this month (and so on).
• Track the time spent in each phase of a security incident, using a defined workflow and good ticket management.
• Directionally adjust our services so each of these efficiency metrics improves over time, which generally means, do things faster.

So, in summary: don’t focus on the metrics that come with your SOC toolset or standard with your Managed Security Service. Show your organization the benefits that security brings – through reporting about how it affects productivity, protecting; availability, intellectual property, brand, and efficiency. Report in $ values and in time-saved, not in volumes of logs processed, incidents written or some technical jargon about the latest ransomware attack. Security needs to be relative to the organizational objectives to be appreciated as valuable.