Security Analytics: Projects vs Boxes (Build vs Buy)?

This is going to be a sad one. This is going to include lines like “Even if you only spend $1m on security data scientists per year, you can …” and “Our ML-based appliance can detect 68% of attacks that utilize DNS covert channel for exfiltrating RAR files, but only if …” and such.

If you recall, I am trying to bring order to the chaos of security analytics this quarter (BTW, what do you think of a paper title like “De-mystifying Security Analytics: Data Sources, Methods, Use Cases”?).

One of the emerging themes that started a few years back but has become more visible now relates to security analytics tools and technologies. So, back in in 2012, BUILD was the only choice: if you wanted to make use of big data tools and approaches for security, the choice was “BUILD or … BUILD.” Sure, you can utilize open source components (Hadoop, Mongo, Cassandra and, of course, R all come to mind), but ultimately your system will be your own, hand-built by your engineers and used by your security analysts and statisticians / data scientists. This is how the real innovators in the field did it, this is how some others are doing it now.

However, today there are dozens of companies promising advanced analysis of user, system and network data, flows, logs, packets, binary execution records, etc, rambling about unsupervised learning, K-means clustering, distance functions, PCA (not to be confused with PCI :-)), random forests and Bayes like it is 1761 (and then there is this hyper-dimensional stuff – presumably coming from another dimension [and you naively thought alien cyber security is cool … now try this stuff from another dimension! :-)]). So, it may seem that there is now a BUILD vs BUY choice…

… but is there?!

Surprisingly, the organizations that have built their own security analytics operations and that keep looking at the market for a chance to “go commercial” report that the choice is just not there, at least not for them. The set of problems solved – and solved REALLY well, as I was informed – by their in-house analytics teams seems to vastly exceed what any one commercial product or even a small set of products [that don’t talk to each other] can do. Now, I am old enough to remember the time when many leading organizations had custom-written SIEM tools (like big banks in 1997-2002), but most if not all were abandoned in favor of commercial SIEM over time. This doesn’t seem to be happening in the analytics domain [yet?]

On the other hand, we now have boxes that claim to do analytics for “user access hijacking, but not insiders”, “insiders that abuse access to files”, “traffic anomalies inside the network”, “traffic anomalies on the egress”, “improving threat intelligence feeds”, “lateral movement detection that does not use valid credentials”, “exfiltration detection but not all egress problems” and so on. I am not joking here; I have spoken with vendors whose analytics appliances solve incredibly narrow (if important!) problems by applying advanced statistical algorithms to various security data [of course, the enlightened readers who studied the materials on adversarial machine learning will argue that it is not that simple, but that is a separate point to make].

Still, do you want to live in the future where every little problem requires its own analytics appliance or SaaS service? Where all the thinking is outsourced to the vendors and where – think about this one! – any new data-related security problem WILL REMAIN UNSOLVED until there is a box shipped [or account provisioned, in case of SaaS] from the appropriate analytics vendor. On top of this, given that each one of such boxes costs in 6-digits, do you think owning a dozen of different ones will be cheap? So, if you don’t like it, work on the analytic mindset before you work on your shopping list….

In light of this, here is what is emerging in my mind:

1. If you ONLY have a problem that vendors tout (e.g. detecting attackers that hacked in and now move laterally using stolen credentials), go shop in the UBA supermarket ; use the same approach if you somehow believe that your problems of the next year will be solved by your vendor or if you are comfortable with only solving the problems that your analytics box vendor will solve well [box approach]
2. If you have problems that can be probably solved by a general purpose analytic tool (such as SAS or Palantir) AND consulting, you may go with one of those consulting-heavy vendors, but do check if they ever solved infosec [cyber in newspeak] problems for clients similar to your organization; expect to pay for each subsequent problem or learn to use the tools well yourself [service approach]
3. If you have your own problem set and want to be prepared for solving future problems using the data-driven approach, you are back in the BUILD camp … sorry! [build a capability approach]

Another way to think about it is: is your security analytics more security or more analytics? If you think “more security”, you may start to gravitate towards “oh, that needs an appliance.” If you think “more analytics”, then maybe you will realize: “oh, that needs analysts!” In essence, “security > analytics” thinking leads to money going to boxes (sadly), while “security < analytics” leads to resources going to people with data science skills.

Gartner GTP has published a lot of excellent materials on advanced analytics usage for business, and nearly all of them emphasize the need fort skilled analysts and analytics culture/mindset – way before tools. For example, “A final, consistent message from the field research [on business, not security, advanced analytics – A.C.] reiterated the fundamental need for a pervasive analytical culture. An analytical culture is one where people value information and are fact-based decision makers as opposed to making decisions relying solely on gut feel and personal experience.“ (“Why Business Analytics Projects Succeed: Voices From the Field”). So, why do you think that for security analytics you just need a box?!

So, my preliminary conclusion that may upset some: if you want security analytics that solves your specific problem set well, you are going to BUILD – and BUILD both the culture/mindset and the tooling.

Now, a useful question to ask next is: is this inherent (as some claim, and with good evidence) or “better boxes” will emerge in the coming years, that will reduce the need to build?

Blog posts on the security analytics topic:

• Do You Want “Security Analytics” Or Do You Just Hate Your SIEM?
• Security Analytics – Finally Emerging For Real?
• Why No Security Analytics Market?
• SIEM Real-time and Historical Analytics Collide?
• SIEM Analytics Histories and Lessons
• Big Data for Security Realities – Case 4: Big But Narrowly Used Data
• Big Data Analytics Mindset – What Is It?
• Big Data Analytics for Security: Having a Goal + Exploring
• More On Big Data Security Analytics Readiness
• Broadening Big Data Definition Leads to Security Idiotics!
• 9 Reasons Why Building A Big Data Security Analytics Tool Is Like Building a Flying Car
• “Big Analytics” for Security: A Harbinger or An Outlier?