A long time ago, in a galaxy far far away … at the very dawn of my security career I attended a presentation by somebody who is now a notable incident response expert. Well … who am I kidding? He was a notable IR expert back in 2000, way…way before IR was cool and way before the word “APT” entered common usage. In any case, I don’t recall much from the presentation apart from one point he made: he has never seen a significant intrusion detected by an intrusion detection system (IDS) [another example of the same kind can be found here]. That line has been burned into my brain since that day…
We routinely talk about prevention/detection/response mantra, [which, some people, for some strange reason, hear as prevention/prevention/prevention as if the room is noisy or something …but I digress], but industry research often reminds that that we really suck at detection [BTW, I find calls for “more prevention” to solve this problem to be sheer idiocy].
Still, “deploying a SIEM – as with any detection technology – will result in things being detected. After things are detected then someone will need to respond to it to investigate it.” (source) This post includes a structured look at SIEM detection methods and approaches. By the way, this post explicitly talks about the THREAT DETECTION, which implies near-real-time observation, as opposed to THREAT DISCOVERY, which involves digging out traces of threats that persist in your environment. Threat discovery is a very fun topic, and we can talk about it again later.
First, I have to repeat something I think I mentioned a few times over the years: SIEM is not an old-style HIPS that matches vendor-provided character sequences to logs. Well, you can use it as such, for sure. But SIEM’s ability to normalize, enrich with context (users, assets, vulnerabilities, etc), correlate across log sources, apply algorithms to streams and “pools” of data, and visualize the data for exploration makes it a different technology – and one with much more difficult mission than a 1997 HIPS.
Here is my quick summary of SIEM detection methods in use today, with select pros/cons of each [NOT a comprehensive list – a longer table may show up in a future paper of mine].
|SIEM Detection Method||Pros||Cons|
|Human analyst event stream review||An analyst observes a filtered stream of events in the console||
|Simple log matching rules||“HIPS mode”: if I see string X123 in logs, alert||
|Vendor-provided cross-device correlation rules||Vendor-provided / default/ OOB correlation rules||
|Matching events to threat intelligence feed||Match incoming events to collected threat intel data such as “bad” IPs, domains, etc||
|Log to context matching via rules||Match incoming events to context such as user role (user with role X should never do Y, etc)||
|Custom-written stateful correlation rules||The ultimate in SIEM detection for years, custom correlation rule enable many scenarios and use cases||
|Real-time event scoring||Algorithms to assess event attributes (source, type, time, other metadata) to highlight events of interest||
|Statistical algorithms on stream data||Statistics such as average, standard deviation, skew and kurtosis [yes, really!]||
|Baseline comparisons||Compare event streams to historical baselines and metrics; related to stat methods, but uses stored historical baselines||
Note that this is not about the data sources, but about the methods themselves – they can apply to many/all data source combinations. Also, the use of context data (users, assets, application, data, vulnerabilities, etc) is useful to enrich many detection methods as well as improve their accuracy. Next I suspect I need to talk about the data sources enabling various types of detection…
As with other functionality, there is probably a maturity curve here somewhere (here!). Who will know how to create statistical models if he never created basic SIEM rules?
P.S. All of these method, separately and together, will fail once in a while. Two choices you have then:
- Wait for the threat to manifest visibly – then go to security incident response.
- Go and dig for threats; do threat discovery.
Select recent SIEM blog posts:
- “Stop The Pain” Thinking vs the Use Case Thinking
- More on SIEM Maturity – And Request for Feedback!
- My Evaluation Criteria for Security Information and Event Management Publishes
- On SIEM Tool and Operation Metrics
- SIEM Analytics Histories and Lessons
- How to Use Threat Intelligence with Your SIEM?
- Popular SIEM Starter Use Cases
- Detailed SIEM Use Case Example
- On “Output-driven” SIEM
- On SIEM Deployment Evolution
- On People Running SIEM
- On SIEM Processes/Practices
- On Large-scale SIEM Architecture
- All posts tagged SIEM
The Gartner Blog Network provides an opportunity for Gartner analysts to test ideas and move research forward. Because the content posted by Gartner analysts on this site does not undergo our standard editorial review, all comments or opinions expressed hereunder are those of the individual contributors and do not represent the views of Gartner, Inc. or its management.
Comments are closed