Gartner Blog Network

Threat Detection Is A Multi-Stage Process

by Augusto Barros  |  December 4, 2017  |  7 Comments

We are currently working on our SOAR research, as Anton has extensively blogged about. SOAR tools have been used to help organizations  triage and respond to the deluge of alerts coming from tools such as SIEM and UEBA. Although this is sometimes seen as the earlier stages of incident response, I’ve been increasingly seeing it as a way to implement “multi-stage threat detection”.

Let’s look at a basic use case of SOAR tools. Before the tool coming into play, there could be a playbook like this:

The SIEM performs basic correlation between a threat intelligence feed and firewall logs, generating an alert for every match (I know, many will argue it’s a bad use case example, but many orgs are actually doing it exactly like that). The SOC analyst would triage each of those events by identifying the internal workstation responsible for that traffic, checking it with an EDR tool, extracting some additional indicators related to that network traffic (the binary file that initiated the connection request, for example) and submitting them to external validation services or sandboxes. If the result is positive, they would use the EDR tool to kill the process, remove the files from the endpoint and also search for the existence of the same indicators on other systems.

With the SOAR tool in place, the organization can automate almost everything performed by the analyst, effectively moving from minutes to seconds to execute all the actions above. The tool starts the playbook when an alert from the SIEM arrives, integrating with the EDR tool and the validation services. We could expand it even further to make it add the new identified indicators to blacklists and firewall rules. Of course, corrective measures would be executed only after the analyst authorizes them.

Now, let’s think about an alternative, hypothetical world:

Your SIEM is immensely powerful and fast. So, you send all the detailed endpoint telemetry collected from the EDR tool to it. You also download all the databases of the external validation services into it. Then, you build a monster correlation rule that will cross the TI feed, the EDR data (linking connection requests to processes and binaries) to that huge database of known malicious processes and binaries. Now you’re doing almost everything from that playbook above on the SIEM, in just one shot (ok, I’m cheating, the sandbox validation still needs a step apart…although the SIEM could have sandbox capabilities embedded; it is immensely powerful, remember?).  No need for the playbook, or the SOAR too, at all!

Unfortunately there’s no such thing as a SIEM like that. That’s why we end up having this single detection use case implemented in multiple steps. if you think about it this way, you’ll see that the SIEM alert is not meant to be a final detection, subject to “false positives”. It’s just the first part of a multi-stage process, each stage looking at a smaller universe of “threat candidates”.

Thinking about detection as multi-stage process unlocks interesting use cases that wouldn’t be able to be implemented as an “atomic decision model”. Any interesting detection use cases that would be discarded because of high false positive rates could be a good fit for a multi-stage process.

But multi-stage detection is not effective if done manually. Score based correlation, as done by UEBA and some SIEM tools, can help linking multiple atomic detection items, but those situations where you need to query external systems (such as sandboxes), external services or big reference sets are still problematic. But SOAR comes to rescue! Now you can have an automated pipeline that takes those initial detection cases (or even entities that hit a certain score threshold) and put them through whatever validation and adhoc queries you might need to turn them into “confirmed detection”, full contextualized alerts.

Most of us would think about advanced automated response use cases, dynamically patching or removing things from the network, as the main way to get value from SOAR.  Not necessarily. Making detection smarter is probably where most organizations will find the value for those tools.





Category: incident-response  siem-and-log-management  threat-detection  threat-intelligence  

Tags: siem  soar  threat-detection  ueba  

Augusto Barros
Research VP
3 years at Gartner
21 years IT Industry

Augusto Barros is Research Director in the Gartner for Technical Professionals (GTP) Security and Risk Management group. Read Full Bio

Thoughts on Threat Detection Is A Multi-Stage Process

  1. Danny Lancashire says:

    Short comment from me: how do we think the Azure ATP / Defender ATP functions – when orchestrated via Azure Security Center – can contribute to the overall solution?

  2. […] to ensure we are also covering technologies such as UEBA, EDR and even SOAR. If we consider that detection can often be implemented as multi-stage process, that’s a natural […]

  3. […] SIEM tools are often used to aggregate multiple sources of information, but are limited in their ability to query additional data sources and verification services after an initial set of conditions are met. The usual approach is to do as much as possible with that set of conditions and then provide the alert to an analyst for triage, where those additional queries take place. However, when the initial conditions set (whether via rules or algorithms, such as machine learning) generate too many alerts, the use case can be infeasible due to the high cost of the manual steps analysts require for triage. The ability to automate postalert queries, such as submitting indicators of compromise (IOCs) to TI services or even artifacts to external sandboxes, allows organizations to implement more threat detection use cases with a high number of initial alerts. (Some of the noisy detection use cases actually deliver valuable insights for as long as they can be quickly triaged.) The automated triage by SOAR effectively acts as the remaining stages of the multistage detection process. […]

  4. […] Threat Detection Is A Multi-Stage Process (by Augusto) […]

Leave a Reply

Your email address will not be published. Required fields are marked *

Comments or opinions expressed on this blog are those of the individual contributors only, and do not necessarily represent the views of Gartner, Inc. or its management. Readers may copy and redistribute blog postings on other blogs, or otherwise for private, non-commercial or journalistic purposes, with attribution to Gartner. This content may not be used for any other purposes in any other formats or media. The content on this blog is provided on an "as-is" basis. Gartner shall not be liable for any damages whatsoever arising out of the content or use of this blog.