Blog post

Security Analytics: Projects vs Boxes (Build vs Buy)?

By Anton Chuvakin | February 03, 2015 | 6 Comments


This is going to be a sad one. This is going to include lines like “Even if you only spend $1m on security data scientists per year, you can …” and “Our ML-based appliance can detect 68% of attacks that utilize DNS covert channel for exfiltrating RAR files, but only if …” and such.

If you recall, I am trying to bring order to the chaos of security analytics this quarter (BTW, what do you think of a paper title like “De-mystifying Security Analytics: Data Sources, Methods, Use Cases”?).

One of the emerging themes that started a few years back but has become more visible now relates to security analytics tools and technologies. So, back in in 2012, BUILD was the only choice: if you wanted to make use of big data tools and approaches for security, the choice was “BUILD or … BUILD.” Sure, you can utilize open source components (Hadoop, Mongo, Cassandra and, of course, R all come to mind), but ultimately your system will be your own, hand-built by your engineers and used by your security analysts and statisticians / data scientists. This is how the real innovators in the field did it, this is how some others are doing it now.

However, today there are dozens of companies promising advanced analysis of user, system and network data, flows, logs, packets, binary execution records, etc, rambling about unsupervised learning, K-means clustering, distance functions, PCA (not to be confused with PCI :-)), random forests and Bayes like it is 1761 (and then there is this hyper-dimensional stuff – presumably coming from another dimension [and you naively thought alien cyber security is cool … now try this stuff from another dimension! :-)]). So, it may seem that there is now a BUILD vs BUY choice…

… but is there?!

Surprisingly, the organizations that have built their own security analytics operations and that keep looking at the market for a chance to “go commercial” report that the choice is just not there, at least not for them. The set of problems solved – and solved REALLY well, as I was informed – by their in-house analytics teams seems to vastly exceed what any one commercial product or even a small set of products [that don’t talk to each other] can do. Now, I am old enough to remember the time when many leading organizations had custom-written SIEM tools (like big banks in 1997-2002), but most if not all were abandoned in favor of commercial SIEM over time. This doesn’t seem to be happening in the analytics domain [yet?]

On the other hand, we now have boxes that claim to do analytics for “user access hijacking, but not insiders”, “insiders that abuse access to files”, “traffic anomalies inside the network”, “traffic anomalies on the egress”, “improving threat intelligence feeds”, “lateral movement detection that does not use valid credentials”, “exfiltration detection but not all egress problems” and so on. I am not joking here; I have spoken with vendors whose analytics appliances solve incredibly narrow (if important!) problems by applying advanced statistical algorithms to various security data [of course, the enlightened readers who studied the materials on adversarial machine learning will argue that it is not that simple, but that is a separate point to make].

Still, do you want to live in the future where every little problem requires its own analytics appliance or SaaS service? Where all the thinking is outsourced to the vendors and where – think about this one! – any new data-related security problem WILL REMAIN UNSOLVED until there is a box shipped [or account provisioned, in case of SaaS] from the appropriate analytics vendor. On top of this, given that each one of such boxes costs in 6-digits, do you think owning a dozen of different ones will be cheap? So, if you don’t like it, work on the analytic mindset before you work on your shopping list….

In light of this, here is what is emerging in my mind:

  1. If you ONLY have a problem that vendors tout (e.g. detecting attackers that hacked in and now move laterally using stolen credentials), go shop in the UBA supermarket ; use the same approach if you somehow believe that your problems of the next year will be solved by your vendor or if you are comfortable with only solving the problems that your analytics box vendor will solve well [box approach]
  2. If you have problems that can be probably solved by a general purpose analytic tool (such as SAS or Palantir) AND consulting, you may go with one of those consulting-heavy vendors, but do check if they ever solved infosec [cyber in newspeak] problems for clients similar to your organization; expect to pay for each subsequent problem or learn to use the tools well yourself [service approach]
  3. If you have your own problem set and want to be prepared for solving future problems using the data-driven approach, you are back in the BUILD camp … sorry! [build a capability approach]

Another way to think about it is: is your security analytics more security or more analytics? If you think “more security”, you may start to gravitate towards “oh, that needs an appliance.” If you think “more analytics”, then maybe you will realize: “oh, that needs analysts!” In essence, “security > analytics” thinking leads to money going to boxes (sadly), while “security < analytics” leads to resources going to people with data science skills.

Gartner GTP has published a lot of excellent materials on advanced analytics usage for business, and nearly all of them emphasize the need fort skilled analysts and analytics culture/mindset – way before tools. For example, “A final, consistent message from the field research [on business, not security, advanced analytics – A.C.] reiterated the fundamental need for a pervasive analytical culture. An analytical culture is one where people value information and are fact-based decision makers as opposed to making decisions relying solely on gut feel and personal experience.“ (“Why Business Analytics Projects Succeed: Voices From the Field”). So, why do you think that for security analytics you just need a box?!

So, my preliminary conclusion that may upset some: if you want security analytics that solves your specific problem set well, you are going to BUILD – and BUILD both the culture/mindset and the tooling.

Now, a useful question to ask next is: is this inherent (as some claim, and with good evidence) or “better boxes” will emerge in the coming years, that will reduce the need to build?

Blog posts on the security analytics topic:

The Gartner Blog Network provides an opportunity for Gartner analysts to test ideas and move research forward. Because the content posted by Gartner analysts on this site does not undergo our standard editorial review, all comments or opinions expressed hereunder are those of the individual contributors and do not represent the views of Gartner, Inc. or its management.

Comments are closed


  • Chris Grant says:

    Where does a player like Splunk fit into this paradigm? It seems like there is a combination of buy and build that sits in between these options that is a platform for security data management and analytics which isn’t a straight “buy” or appliance option, nor is it an entirely hand-crafted, BUILD platform.

  • @Chris Thanks for the comment — indeed, splunk fits with both *general purpose analysis tools* and (with some add-ons like prelert) as *boxed analytics* as well.

    Indeed, a BUY in this area is BUY-THEN-BUILD [well, it IS if you actually plan for some success :-)]

  • Mustafa Rassiwala says:

    Anton – you bring up a very important topic specially when you discuss is security analytics – security > analytics or analytics > security

    Is security analytics the classic chicken or egg problem? What comes first – security domain data sets and use cases OR analytics – specialized infrastructure to analyze data that happens to be security data in this case?

    Culture of Security Swagger versus Meek Toe the Line

    I think the answer will depend on the fundamentals – whats the culture and mindset of the security organization – Is it my security people are my best line of defense and lead with a people drive product and process decision? Or is still the attitude that security spending is corporate tax – just invest in some pre-canned security appliances which help me check the box and I am done.

    Security Analysts are the ones conducting security investigations and making decisions on root cause analysis. They are in the best position to analyze data specific to your organization and relevant to your business. Analytics on security and your organization specific data helps them model threats and responses to your organization. In that situation this is an analytics problem – does your security analyst have access to the right tools to analyze the data – the security analysts will provide the security domain expertise.

    The push for pre-canned algorithms is a result of the hangover from the intoxication of preventive real-time security. The mindset was security products need to be completely prescriptive i.e. firewalls with 1000s of rules, IPS with 1000s of signatures, web and email gateways pre-loaded with 100s of security policies, endpoint security with 1000s of signatures and polices and the list goes on. While that works for preventive – black and white approach in real-time, as security organizations move to detection-oriented approach they need a different mindset.

    Detection-oriented is a much more detective/crime scene investigation approach. Security Analysts are looking for clues, investigating small incidents to unravel deeper mysteries and pivot on different attributes i.e. user, device, IP, type of information etc as they go about their detective work. What security analysts need is tools and analytics infrastructure that matches their expertise and intuition capabilities. As the popular adage goes – Give a man (or woman) a fish and you feed him/her for a day; teach them to fish and provide the tools and you feed them for a lifetime.

    Having worked on security analytics in some form or the other at several security vendors my opinion is that this is first a big data infrastructure and analytics problem. Let the security analysts drive the projects and over time you build a library of proven, time-tested analytics for specific narrow use cases all on a single platform.

  • @Mustafa Thanks for your super-insightful post. Indeed, culture and sec maturity seem to play the most important role, and “box buyers” will … more boxes, while the leaders will develop strong internal capabilities, using the tools needed by the teams.

    >The push for pre-canned algorithms is a result of the hangover from
    >the intoxication of preventive real-time security.

    Very very true!! They had “preventative boxes”, and now that they need to think, …guess what… they need a box that thinks for them 🙁

  • Harvey S says:

    The BUILD camp is a very hard place to be in. It’s incredibly difficult (read impossible) to find good people with data science, technical and security knowledge… It’s also hard to let different people with one specific knowledge area function as a effective team.

  • @Harvey Very, very true. It is a very hard place where “sudden” success follows “years and millions of $”

    However, in other domain people have built a lot of this, such as:
    infosec + statistics team/pair is hard to make work, sure, but so is marketing + statistics. Yet, orgs have figured it out.