Gartner Blog Network

Sad Hilarity of Predictive Analytics in Security?

by Anton Chuvakin  |  March 31, 2016  |  14 Comments

After spending a week in Siberia, I am ready for more fun blogging – and of course for more drama that is our industry (GO CYBER DRAMA!). In any case, the topic is PREDICTIVE ANALYTICS in SECURITY: What is it? Can we have it?

What is it?

First, I’ve encountered a few “false predictive” examples like “if you don’t patch your Windows for 3 years, I *predict* that you will be hacked” (common sense predictive) and “if I see this attack first, I can tell you and thus you can *predict* it in your environment” (my past is your future predictive). Frankly, I don’t think this is real prediction, like predicting the weather. Finally, some people treat other types of analytics as predictive: for example, they treat some UBA / UEBA tool output “user jsmith maliciousness score is 73%” as 73% that this user will go bad… To me, even guessing the likelihood of some event isn’t truly prediction, since the event will either happen or it won’t [update: OK, maybe I was too harsh here, but simply saying some known outcome can happen with X% probability is often unimpressive predicting].

What is real prediction then? I’d rely on Gartner definition of predictive analysts that “create models that anticipate future behavior, or estimate unknown outcomes” (here; also see this). In other words, it better be about goddamn future! (“Prediction is very difficult, especially about the future.” – maybe Niels Bohr)

Can we have it?

Depending on the definition, the traditional reason for why not (= because there is a smart, evil-intentioned and possibly psychopathic human on the other end of the wire) may not apply since predictions of customer behavior (e.g. see this) has been done for a while. Still, if you predict the future activity based on the past record and some context, you are bound to be wrong in some cases, especially when a) you don’t have enough information about the subject of that activity – the adversary and b) said subject is actively trying to avoid being seen.

Frankly, I’ve seen idiots and their little idiot AIs make judgments like “because 100.0% of DDoS you were hit with so far was ICMP, we predict that your next DDoS will be ICMP.” And then – BAMM! – a massive DNS flood blows them away…. And this example does not even start to take into account any active deception and misdirection by the attacker. On the other hand, the past clearly tells us something about the future, and thus maybe there is some hope here [“Organizations that use traditional predictive analytics [tools] for security? You mean ‘both of them?’” (source)]

There you have it – what do you think? Can we have real [clearly defined] prediction in information security?

P.S. Now back to our EDR research … much excitement is coming! BTW, our updated IR paper is in editing already, but if you have any fun last second comments on this, please send them over.

Category: analytics  future  philosophy  security  

Anton Chuvakin
Research VP
5+ years with Gartner
16 years IT industry

Anton Chuvakin is a research VP at Gartner's GTP Security and Risk Management group. Before Mr. Chuvakin joined Gartner, his job responsibilities included security product management, evangelist… Read Full Bio

Thoughts on Sad Hilarity of Predictive Analytics in Security?

  1. Seth Hall says:

    I think you are right and the root of the problem is that ultimately all predictive approaches are doomed to fail when faced with a person or people that are driven by unknown motivations and mountains of creativity.

    If there is some biological process or natural process, it can be modeled and predicted with varying degrees of accuracy. Until someone figures out how to model psychopathic humans, it’s just not going to work so well. 😉

    • Thanks for the comment, Seth. So, to summarize, you are in the camp of “we can’t predict threats, the attackers are unpredictable by design”, right?

  2. […] Anton Chuvakin After spending a week in Siberia, I am ready for more fun blogging – and of course for more drama […]

  3. Bassam Khan says:

    Why is there so much hype about whiz bang technologies, and companies, that are literally dying to be proven in the real world when so many enterprises have yet to put into place even the most basic safety hyegene; patching, application control, least privilege and training the users not to do stupid things!

    It’s like a Rube Goldberg machine to ring a bell when all you have to do is to pick up the little hammer and swing. I guess that’s not very flashy. Never mind, I answered my question.

  4. Alex Loffler says:

    Hi Anton, excellent post! We have been looking at applying predictive analytics to the victims/targets as opposed to the attackers. The assumption being that victims/targets are not actively trying to evade detection, hence it may be easier to predict/identify high risk behaviours/targets. Have you seen this approach applied successfully elsewhere?

  5. Hoping you weren’t sent away to Siberia! But I guess since you returned you went under your own free will. To the topic…Instead of getting overly fancy to predict the future, how about we improve and speed our detection of recently risky past activities first. I would rather know that yesterday I probably was hacked versus tomorrow I might be hacked.

  6. Alexei Suvorov says:

    I understand that the real value of prediction in the security analytics is not the prediction per se. The prediction is used for comparison with what is being actually observed, to detect an anomaly. E.g. if we predict a certain DNS traffic patterns based on the past data, however the new DNS data is very different from the prediction, then an anomaly (potential threat) is detected (vs. predicted).

    • Sure, but some vendors seem to tout the actual prediction, no just “note that the future is different from the past” and then flag this as an anomaly

  7. Dori Fisher says:

    If you do not have all the data. you cannot create effective analytics. And we have a small part of the picture. This is why intelligence and sharing have value in assisting to learn from other experience, but again, their experience is also lacking and limited to observed, external phenomenons.

  8. Joe Bonnell says:

    Siberia? For fun? In the WINTER??

    If I had a predictive analytics engine that could *reliably* predict human behavior and was primarily motivated by financial gain, I’d be applying that to the futures markets rather than taking a product to market.


    • Hey Joe…thanks a lot for the comment. Indeed, if you had one of those “human behavior predictor engines”, you can profit in many others areas before you apply it to infosec :-)

Comments are closed

Comments or opinions expressed on this blog are those of the individual contributors only, and do not necessarily represent the views of Gartner, Inc. or its management. Readers may copy and redistribute blog postings on other blogs, or otherwise for private, non-commercial or journalistic purposes, with attribution to Gartner. This content may not be used for any other purposes in any other formats or media. The content on this blog is provided on an "as-is" basis. Gartner shall not be liable for any damages whatsoever arising out of the content or use of this blog.