Anton Chuvakin

A member of the Gartner Blog Network

Anton Chuvakin
Research VP
2+ years with Gartner
14 years IT industry

Anton Chuvakin is a research VP at Gartner's GTP Security and Risk Management group. Before Mr. Chuvakin joined Gartner, his job responsibilities included security product management, evangelist… Read Full Bio

SIEM and Badness Detection

by Anton Chuvakin  |  July 24, 2014  |  2 Comments

A long time ago, in a galaxy far far away … at the very dawn of my security career I attended a presentation by somebody who is now a notable incident response expert. Well … who am I kidding? He was a notable IR expert back in 2000, way…way before IR was cool and way before the word “APT” entered common usage. In any case, I don’t recall much from the presentation apart from one point he made: he has never seen a significant intrusion detected by an intrusion detection system (IDS) [another example of the same kind can be found here]. That line has been burned into my brain since that day…

We routinely talk about prevention/detection/response mantra, [which, some people, for some strange reason, hear as prevention/prevention/prevention as if the room is noisy or something …but I digress], but industry research often reminds that that we really suck at detection [BTW, I find calls for “more prevention” to solve this problem to be sheer idiocy].

Still, “deploying a SIEM – as with any detection technology – will result in things being detected. After things are detected then someone will need to respond to it to investigate it.” (source) This post includes a structured look at SIEM detection methods and approaches. By the way, this post explicitly talks about the THREAT DETECTION, which implies near-real-time observation, as opposed to THREAT DISCOVERY, which involves digging out traces of threats that persist in your environment. Threat discovery is a very fun topic, and we can talk about it again later.

First, I have to repeat something I think I mentioned a few times over the years: SIEM is not an old-style HIPS that matches vendor-provided character sequences to logs. Well, you can use it as such, for sure. But SIEM’s ability to normalize, enrich with context (users, assets, vulnerabilities, etc), correlate across log sources, apply algorithms to streams and “pools” of data, and visualize the data for exploration makes it a different technology – and one with much more difficult mission than a 1997 HIPS.

Here is my quick summary of SIEM detection methods in use today, with select pros/cons of each [NOT a comprehensive list – a longer table may show up in a future paper of mine].

SIEM Detection Method Pros Cons
Human analyst event stream review An analyst observes a filtered stream of events in the console
  • None :–)
  • Does not scale
  • Skilled analyst required
Simple log matching rules “HIPS mode”: if I see string X123 in logs, alert
  • Simple
  • Specific
  • Light on SIEM resources
  • Need to know what to match
  • Useless for advanced, multi-stage attacks
Vendor-provided cross-device correlation rules Vendor-provided / default/ OOB correlation rules
  • Cross-device correlation
  • No need to write rules
  • Relevance to customer use cases may be lacking
  • Need to tune the rules
Matching events to threat intelligence feed Match incoming events to collected threat intel data such as “bad” IPs, domains, etc
  • Useful detection with minimal tuning
  • Low FPs [given quality TI]
  • Requires high-quality TI data
  • Timing: TI data needs to be loaded before the event
Log to context matching via rules Match incoming events to context such as user role (user with role X should never do Y, etc)
  • Easy policy alerts
  • Site-specific content
  • Need a clear policy
  • Context data needs to be loaded and be current in SIEM
Custom-written stateful correlation rules The ultimate in SIEM detection for years, custom correlation rule enable many scenarios and use cases
  • Targeted to what the organization needs
  • Refine and adapt over time
  • Rules need to be written and refined by a SIEM content expert
  • Errors in rule logic often not obvious
Real-time event scoring Algorithms to assess event attributes (source, type, time, other metadata) to highlight events of interest
  • Easy way to highlight potentially interesting events
  • Prioritization may not match your priorities
  • “Potentially” interesting
Statistical algorithms on stream data Statistics such as average, standard deviation, skew and kurtosis [yes, really!]
  • Useful complement to rules
  • Can be used with rules to look beyond single events
  • Choosing meaningful stats is often harder than writing rules
  • FPs are common
Baseline comparisons Compare event streams to historical baselines and metrics; related to stat methods, but uses stored historical baselines
  • Useful complement to rules
  • Can be used with rules to look beyond single events
  • Fails to detect when baseline includes badness, or attack traffic is not anomalous
  • FPs are common

Note that this is not about the data sources, but about the methods themselves – they can apply to many/all data source combinations. Also, the use of context data (users, assets, application, data, vulnerabilities, etc) is useful to enrich many detection methods as well as improve their accuracy. Next I suspect I need to talk about the data sources enabling various types of detection…

As with other functionality, there is probably a maturity curve here somewhere (here!). Who will know how to create statistical models if he never created basic SIEM rules?

P.S. All of these method, separately and together, will fail once in a while. Two choices you have then:

  1. Wait for the threat to manifest visibly – then go to security incident response.
  2. Go and dig for threats; do threat discovery.

Select recent SIEM blog posts:

2 Comments »

Category: analytics security SIEM     Tags:

My Blueprint for Designing a SIEM Deployment Publishes

by Anton Chuvakin  |  July 22, 2014  |  4 Comments

Another new document on SIEM that I wrote just published: Blueprint for Designing a SIEM Deployment. “Planning a distributed enterprise SIEM deployment is challenging for information security teams at many organizations. This Blueprint shows the architecture and timeline for an enterprise security information and event management deployment and highlights key tasks for each stage. “ This is another new Gartner GTP document type called “an architectural blueprint”, and it has distinctly non-Burton’ian length: 2 pages (!), with one taken by a picture. GTP Blueprints make perfect gifts for your favorite IT architect :-)

For reference, here are my other SIEM research papers [access requires Gartner GTP subscription]:

For those without a GTP subscription, some fun SIEM blog posts:

4 Comments »

Category: announcement security SIEM     Tags:

“Stop The Pain” Thinking vs the Use Case Thinking

by Anton Chuvakin  |  July 17, 2014  |  2 Comments

“Hello, I am your anti-virus program. Which specific viruses would you like me to kill today? Enter names here: [……..]” While I don’t recall the exact state of the art of anti-virus back in the late 1980s, I do not remember any anti-virus program ever asking such a question. The technology originated in response to a definite threat – malware [collectively called “viruses” at the time]. At no point in this technology evolution, was the user supposed to steer it towards particular targets. It just “did it.”

OK, Anton, and your point is?

SIEM use cases, naturally (example use case, list of common SIEM starter use cases). I’ve met a few folks who loudly wondered “why SIEM can’t just DO IT.” Here is how they think: anti-virus just does it, firewalls just do it (well, after you write the rules), even NIPS just does it [well, in their minds it does…]

Why oh why can’t SIEM just do it? When the enlightened SIEM vendor offers them a tool and adds “now you need to tell us what use cases you’d like to focus on first”, they complain that the vendor is shifting the burden to then; why can’t their SIEM tool “just do it”?

OK, the enlightened readers of this blog will start to snicker – or even ROFL – just about now. However, let’s scrutinize this delusion.

Back in my SIEM vendor days, we had a situation when a field engineer was asking a customer “what use cases do you want to start with?” with the customer countering with “so, what use cases should I start with?” and this ping-pong going on for a while with both parties getting increasing frustrated (all the way to “so…wait a second here…you just paid us $740K for a SIEM and you don’t know what you want with it?! WTH!” — “Whaat?! You just sold us a $740K pile of stuff that does not even DO anything” and so on). In the end, they simply said “we want to do with our SIEM what most other people want to do with theirs” — and left it at that…

Intuitively, people feel that SIEM technology as well as some other technologies are inherently different from others, but they are unable to spell out how (i.e. SIEM is not like anti-virus). For once, monitoring technologies require an open-ended commitment from the organization wanting to utilize them. Also, successful monitoring nowadays MUST be mission-specific; you are unlikely to succeed if you want to generate a critical alert if “something bad” happens. You have BAD, next-morning BAD, end-of-the-week BAD and of course that scary “wake me up at 3AM BAD” — with the exact priorities depending on YOUR BUSINESS. Not the vendor default correlation rules, now some “security intelligence”, not what Gartner thinks – your business [BTW, DLP is even more so this way]. Contrast this with “I don’t like viruses – please kill them all” seen through the anti-virus lens….

To summarize, a lot of security gear is bought to “plug a hole” (be it an audit finding or a new threat such as malware). However, SIEM is most explicitly NOT of this kind. As I’ve written many times, SIEM is a “force multiplier”, but this definition implies that you have something to multiply. If you have 0 capabilities, a purchase of a SIEM tool will still leave you at – you guessed it!—0. SIEM will make YOUR security monitoring problem-solving better/faster, but it won’t “plug any hole” for you.

And if you somehow cannot transcend “see a hole-buy a box” thinking about security, some expensive education is available

Related blog posts:

Select recent SIEM blog posts:

2 Comments »

Category: philosophy security SIEM     Tags:

More on SIEM Maturity – And Request for Feedback!

by Anton Chuvakin  |  July 14, 2014  |  11 Comments

During my original SIEM architecture and operational practices research (see the paper here and a presentation here), I looked at the topic of SIEM operation maturity. Organizations that purchase and deploy SIEM technologies are at different stages of their IT and information security maturity (such as when measured by Gartner ITScore for Security). Certain security monitoring goals are extremely hard to achieve at lower maturity stages (such as “hunting” when you can barely collect data); they are also frequently unachievable unless the organization climbs all the steps in the maturity ladder to get to that step [so, no jumping stages].

The key purpose of this maturity scale is to evolve an SIEM deployment toward getting more value out of it at higher stages of the scale. Also, SIEM team members can use it to make sure that specific operational processes are in place as SIEM deployment evolves from stage to stage. For example, enabling alerts without having an alert triage process and incident response process is usually counterproductive and ends in frustration. Still, all the processes from lower stages must remain in place as SIEM deployment maturity grows.

Here is the current version of the table:

Table 7. SIEM Maturity Scale

Stage No. Maturity Stage Key Processes That Must Be in Place

(inclusive of previous stages)

1 SIEM deployed and collecting some log data SIEM infrastructure monitoring process

Log collection monitoring process

2 Periodic SIEM usage, dashboard/report review Incident response process

Report review process

3 SIEM alerts and correlation rules enabled Alert triage process
4 SIEM tuned with customized filters, rules, alerts and reports Real-time alert triage process

Content tuning process

5 Advanced monitoring use cases, custom SIEM content, niche use cases (such as fraud or threat discovery) Threat intelligence process

Content research and development

Source: Gartner (January 2013)

SIEM team members may also choose to add a Stage 0 (“tool deployed, no process”) and possibly higher stages, which are sometimes seen at security-mature, “Type A” organizations (with such exciting activities as data modeling process, visual data exploration process, use-case discovery process and so on).

At this point, I’d like to ask for your feedback and improvement suggestions?

Should I add dimensions to the maturity table, such as essential personnel skills, typical tool components deployed and utilized, use cases common at each stage?

In any case, feel free to suggest it below in comments, via email or via whatever social media venue you happen to frequent.

Previous version of the maturity table:

Select recent SIEM blog posts:

11 Comments »

Category: monitoring security SIEM     Tags:

Why No Security Analytics Market?

by Anton Chuvakin  |  July 8, 2014  |  15 Comments

So, occasionally I get this call from somebody (vendor, end-user, investor, etc) inquiring about “the size of the security analytics market.” They are usually shocked at our answer: since there is no such market, there is no size to report.

If you recall, we [as well as myself] don’t really believe there is such a market at this time and find any discussion of its size “premature” (at least). Let’s explore this in detail – and hopefully save some of my time for loftier pursuits.

In essence, if you are in the market for a car, you are very unlikely to buy a toilet bowl or a jet plane instead. Everybody knows what is a car, what it does, how it functions [well, at some level] and how much it costs. Sure, there is a difference between a Kia and a Maserati, but such variances are easily understood by the customers. While market definition in general is hard, industrial organization (IO) economics have made a lot of practical advances towards that goal (for example, some use “the smallest area within which it is possible to be a viable competitor”). Close to home in our infosec (“cyber security”?) realm, if you need DLP, you go and buy DLP. If you need a WAF, you go get that. Even with SIEM, there is relative clarity in terms of features, benefits and prices.

Do we see ANYTHING of this sort when “security analytics” is mentioned?

No, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no! :-)

There is no common feature set, no critical/core capabilities, no jointly understood need, no buyer-seller agreement on anything, no clear competitive dynamics ….

As we say in our paper “defining “security analytics” at this point simply involves looking up the words in the dictionary. There is no “security analytics market” or dedicated and purchasable “security analytics tools”; security analytics is a concept that an organization can practice, but can’t buy. Many different tools — from network intrusion prevention system (NIPS) to DLP and SIEM — use various algorithms to analyze data, thus performing analytics. Thus, if security-relevant data is subjected to analytic algorithms, security analytics is being practiced.” Along the same line, one enterprise I spoke with defined it as “ability to analyze lot of security data over long periods of time, find threats and create models” [not too specific – but hitting a few interesting things such as long term analysis, threat discovery, models, etc]

In fact, I can give you a handy analytical tool to create your very own “security analytics” vendor – right here, right now! FREE!!

Here is how it works – pick one or more from each item 1.-4. below:

  1. Pick a problem to solve (sadly, some vendors have skipped this step altogether; others chose really hard, fuzzy problems like insider threat or “advanced” threat)
  2. Collect some data (some logs, network flows, session metadata, full packets, threat intelligence, process execution records, whatever – the more, the merrier!)
  3. Analyze it in some way (ideally not by using rules, but any algorithm will suffice – think various types of ML [supervised or unsupervised], clustering, deep anything, forensics something, text mining, etc]
  4. Present the results in some way (ideally visualize, but – if you are adventurous – also act automatically, reconfigure, etc)

That’s it! In your mind, you are now a player in a burgeoning [in your mind] “security analytics market”…

BTW, if you want to hear me ramble about it even more, check out this podcast [MP3]

Related blog posts:

15 Comments »

Category: analytics philosophy security     Tags:

My Evaluation Criteria for Security Information and Event Management Publishes

by Anton Chuvakin  |  July 2, 2014  |  4 Comments

It is with tremendous excitement that I am announcing the publication of my “Evaluation Criteria for Security Information and Event Management” document and SIEM selection tool (download link inside the document).

Love the “Magic Quadrant for Security Information and Event Management” and “Critical Capabilities for Security Information and Event Management” but want more details? [and I mean MORE DETAILS!!] Use our SIEM evaluation criteria!!!

There are numerous use cases for this essential document/tool, such as:

  • Figure out what to look for in a SIEM product
  • Create your very own set of SIEM selection criteria
  • Evaluate a SIEM product based on a set of criteria
  • Compare two or more SIEM products and choose the product that fits better
  • Build an RFP/RFI for SIEM
  • Understand the vendor materials and map vague claims to specific, measurable features
  • Impress your friends with knowledge of esoteric SIEM features such as “Agents and collectors should be able to operate within low-bandwidth requirements and throttle the data based on predefined rules and requirements” or “The ability to group assets, users, log sources and so forth automatically and/or based on external information.”

Without further ado, enjoy the …

Evaluation Criteria for Security Information and Event Management

30 June 2014 | G00262712

Analyst(s): Anton Chuvakin

SIEM is a pivotal and widely used security technology, and a deep understanding of SIEM technology is critical for success in acquiring the right SIEM product. This evaluation criteria document helps define and refine SIEM buying criteria.

And don’t tell me I didn’t warn you about the details :-)

Related announcement posts:

Select recent SIEM blog posts:

4 Comments »

Category: announcement security SIEM     Tags:

My Top 7 Popular Gartner Blog Posts for June

by Anton Chuvakin  |  July 1, 2014  |  1 Comment

Most popular blog posts from my Gartner blog during the past month were:

  1. SIEM Magic Quadrant 2014 Is Out! (announcements)
  2. SIEM Analytics Histories and Lessons (SIEM research)
  3. On SIEM Tool and Operation Metrics (SIEM research)
  4. Detailed SIEM Use Case Example (SIEM research)
  5. Popular SIEM Starter Use Cases (SIEM research)
  6. Security Essentials? Basics? Fundamentals? Bare Minimum? (misc fun posts)
  7. On Comparing Threat Intelligence Feeds (threat intelligence research)

Enjoy!

Past top posts:

1 Comment »

Category: popular     Tags:

SIEM Magic Quadrant 2014 Is Out!

by Anton Chuvakin  |  June 26, 2014  |  7 Comments

SIEM Magic Quadrant and SIEM Critical Capabilities documents have just been published [Gartner subscription require for access – at least until some vendor republishes the content…]

Some fun quotes from this year’s documents:

  • “Broad adoption of SIEM technology is being driven by the need to detect threats and breaches, as well as by compliance needs.” and “Breach detection is the primary driver, and compliance remains a secondary driver.” [note the order – A.C]
  • “SIEM is a $1.5 billion market that grew 16% during 2013 — with an expected growth rate of 12.4% during 2014.” and “During this period [past year], the number of Gartner inquiry calls from end-user clients with funded SIEM projects increased by 12% over the previous 12 months” [so, NO, SIEM is not doing away! – A.C.]
  • “Analytics are an important [SIEM] capability to support the early detection of targeted attacks and breaches. […] Initial deployments of the “separate analytics back store” approach have been implemented by a small number of Type A companies.” [further confirming what I’ve been saying here and here – A.C]
  • “The SIEM market is mature and very competitive. […] The greatest area of unmet need is effective targeted attack and breach detection. […]The situation can be improved with stronger threat intelligence, the addition of behavior profiling and better analytics. ” [please use what you have first, then think of another box to buy. Remember: the more you spend on boxes, the less you have for people who will use them! – A.C.]

Enjoy!

P.S. My add-on effort, a detailed SIEM Evaluation Guide is coming out shortly as well!!

7 Comments »

Category: announcement security SIEM     Tags:

Security Essentials? Basics? Fundamentals? Bare Minimum?

by Anton Chuvakin  |  June 20, 2014  |  15 Comments

Let’s think together – what technologies and practices constitute information security essentials? The question is actually bitchingly hard – so think before answering! One way to think of this is to imagine somebody describing his security capabilities to you, and when they miss something from that list you go “ZOMG!!! No way you missed X! What are you, stupid or something?! STOP talking to me and go deploy/do/buy that now!!!”

First, it is probably NOT just “firewalls and SSL” (and anti-virus) – endpoint security and data security should be represented, as well as possibly cloud security. Also, some practices will span both information security and IT operations, such as asset management, change management, patch management, etc. For example, PCI DSS is not a bad list of basics, as some would say.

So, let me try my hand at this admittedly thankless task (we may do a formal research project on this latter…)

This first batch are the “unquestionables” (IMHO):

  • Security policy
  • Firewalls/network segmentation
  • Transit data encryption
  • Authentication
  • Anti-malware
  • Vulnerability management (including remediation such as patch management)
  • Incident response

This batch is strong contenders:

  • Security awareness
  • [Some] risk assessment
  • Stored data encryption
  • Log analysis and monitoring

And this batch starts to cover what can be called debatable:

  • NIPS
  • WAF
  • Lots of other stuff that I am too lazy to mention – this last section can be long!

(as you noticed, the list mixes tools and practices, but favors tools slightly; this is not intentional)

And, of course THE primary risk is that the list is BOTH “too long” and “too short” – at the same time (a complaint frequently tossed in the direction of PCI DSS)

Here are some attempts to ponder this question or come up with specific lists of “cyber security” things to DO and BUY:

BTW, Ben, I am not stealing your thunder – I just wanted to start this discussion :-)

Loosely related blog posts:

15 Comments »

Category: philosophy security     Tags:

On SIEM Tool and Operation Metrics

by Anton Chuvakin  |  June 17, 2014  |  19 Comments

While some people whine that “their SIEM deployment has failed”, how the hell do they know? I’ve met some folks who spent 8 digits (that’s EIGHT digits!) on SIEM and they are as happy as pigs in clover. They think that SIEM is the best security investment they’ve ever made, for realz.

Measuring SIEM health and operations is still an emerging art, and there is no set of accepted SIEM metrics. The core SIEM team has to define success criteria at the planning stage and periodically check for progress in regard to these criteria (source). However, collecting logged event numbers, correlated event numbers, the related rules enabled, the number of incidents handled and even the number of changes implemented as a result of SIEM monitoring has proven useful for many organizations. These do allow for basic measuring of SIEM tool and program performance. For example, if the volume of collected and correlated logs have decreased dramatically, maybe the tool usage is waning.

Measuring SIEM impact on incident recovery time (similar to the operational mean time to repair [MTTR] metric) and on incident severity also present great evidence of more strategic SIEM success. Even better, a reduced incident discovery window, if observed, can provide a great boost to an SIEM program. The number of cases open for investigation as a result of SIEM and the potential incidents resolved at early stages are useful metrics as well. The number of alerts handled for each analyst allows the organization to track team performance and not just tool performance.

Select SIEM tool metrics:

  1. Event collection rate, EPS (average, maximum – per log source, per type, etc)
  2. Event processing/analysis rate, EPS (average, maximum)
  3. Total log storage, GB (in SIEM, log management)
  4. Log source count (by type, region, log volume, etc)
  5. Alerts triggered count (per time unit, by target, by type, by rule, etc)
  6. SIEM resource usage (CPU, RAM, disk)

Select SIEM operation metrics:

  1. Alerts handled (per analyst, per rule, per target, etc)
  2. Alert response timing [such as time from triggering to review, then to first action, then to closure or escalation (by alert type, by target, by analyst, etc)] <- some call this metric “the only one that matters”
  3. Incidents opened based on SIEM alerts (by time unit, by analyst, by target, etc)

[note the word usage above, these are “select”, not “top” metrics – I feel that I don’t know enough at this stage to proclaim knowledge of the best or top metrics!]

Care to suggest more? Which ones you find the most useful?

Select recent blog posts related to SIEM:

19 Comments »

Category: security SIEM     Tags: