Blog post

Weird Hunting Analogy and Machines vs Humans

By John Collins | July 30, 2020 | 3 Comments

The more people I talk with about threat hunting, the more I realize everyone has their own opinion and interpretation of what it is. Unfortunately, I believe some entities misrepresent what it is to fit their agenda which matches a lot of human behavior throughout history, but I digress. There are calls within the security industry for standards or a framework for threat hunting, but in my opinion this threatens the innovative nature of threat hunting. The last thing defenders need is yet another standard and control to force them into a cumbersome process, while threat actors act with immunity and iterate at light speed. Once standards or framework guardrails are put around threat hunting, the most interested party in learning and circumventing it will be adversaries. Rest assured threat actors are avid readers of reports and frameworks detailing their TTPs and security control framework documentation. Don’t misunderstand me, there is a need for standards and frameworks to provide minimum viable security (MVS), but threat hunting ain’t the place to do it.

One question I hear a lot is, how is threat hunting different from detection and incident response (IR)? Great question, and I’ve put together a rudimentary graphic to illustrate the overlap in Figure 1. I don’t believe they are mutually exclusive and complement each other. You have to detect something to respond to it, unless a third party sends you a pwnage notice letter. You leverage detection to form a hypothesis to go hunt, otherwise you are just walking around in the dark. A hunt turns into a IR engagement to evict. All three form the Pinnacle of SecOps Capability. This is where everyone wants to be, but few are actually doing it.

Figure 1: Threat Hunting, Detection and IR Relationships

Once you have a repeatable process in SecOps it should be promoted to the relevant stage of the predict, prevent, detect, respond (PPDR) model. Scanning for indicators on a recurring basis is not threat hunting, that is detection, especially if you have outsourced monitoring to a provider. Threat hunting often begins with detection processes like scanning for known indicators or TTPs of threat actors, which can and should be formalized. However, intuition and drawing conclusions from facts that cannot be scanned for to generate a hypothesis is still very much a human art form. Thankfully humans still provide value in security! Yes, there are some data science models that can aid in speeding up this process, but IMHO you still need someone with questioner and slight rebel tendencies (look up Gretchen Rubin) to really peak over the horizon into the unknown. A machine definitely aids a threat hunter in his/her decisions and efforts, but the human is usually modifying the machine on the fly to improve the outcome of their hunt.

What is this unknown and how does it relate to SecOps?  I created the graphic in Figure 2 for a research note Gorka Sadowski and I wrote on threat hunting, titled Are You Getting What You Thought with Outsourced Threat Hunting? I don’t think anyone liked the graphic, except me! It didn’t make the cut for a published note so it made it to the blog. I know I’m not a graphics expert so I realize the obtuse nature of the drawing. The concept of threat hunting and what it provides to the overall SecOps process is what I was trying to explain.  It is a feedback loop and there is a state of unknown which is really hard to define and detect.

Threat Hunting in the SecOps Lifecycle

The Pyramid of Pain created by David Bianco in 2013 inspired this monstrous graphic. However, many security operators know it and it conveys the point of threat hunting in the SecOps process. Well, it does to me. I used the PPDR model at the bottom to illustrate the foundational concepts in order from easiest to hardest to accomplish:

  • Prevent as much as possible
  • Detection as a best effort
  • Respond when needed
  • Predict if possible

Everyone reaches a point when prediction has been exceeded and your pucker factor is maxed out, this is the unknown. Think of this like the singularity of a black hole in space, we have no idea what is there and you have to travel past the event horizon to see it. You enter the realm of unknown problems with unknown means to detect. Remember those points about innovation and freedom to hypothesis and test?  This is where you find a way to look beyond the security event horizon to see into the singularity which can present threat actor presence and history. I!

But wait, there’s more!  Now you encounter the known concepts of detection which lead to response.  There is indicator and intelligence collection, which bigger brains than mine argue about what an indicator is. This leads to IR work to trap the adversary and evict them from the environment in a methodical manner. Adversaries leave nasty messages for third party hunters on servers to get out of their environment and hunters shutdown shells in response. It really is a digital version of cloak and dagger in the espionage world.

Will threat hunting ever be fully automated to detect every unknown? Maybe, but it won’t be any time soon. Human analysis and reasoning still plays a critical role in the art of threat hunting and I’m not aware of an autonomous system that is a direct equivalent to a threat hunter’s brain to hypothesis and reason about the possibility of alternate threat actor methods never seen before.  When that system/solution does arrive, I’ll be one of the first to acknowledge and cheer it’s capability…with skepticism!

The Gartner Blog Network provides an opportunity for Gartner analysts to test ideas and move research forward. Because the content posted by Gartner analysts on this site does not undergo our standard editorial review, all comments or opinions expressed hereunder are those of the individual contributors and do not represent the views of Gartner, Inc. or its management.

Comments are closed


  • Carolyn Reuss says:

    Hey John! Love the graphics. Didn’t get the reference to Gretchen Rubin (have read some of the Happiness Project, what am I missing?!)
    Thanks for the post, Hunting does get tossed around too much.

    • John Collins says:

      Gretchen created the Four Tendencies concept of Upholder, Questioner, Obliger and Rebel to categorize human behaviors and personalities. It is similar to a Myers-Briggs assessment.

      I was introduced to it a leadership workshop and everyone thought it was pretty accurate not only about themselves, but also about their peers based on working relationships. Her book goes into great detail about how to work with and manage the other tendency types. The vast majority of humans are mix of two tendency types. I was a Rebel/Questioner, which explains why I couldn’t stay in the military until retirement!

  • It’s worth keeping in mind that different types of A.I. used in commercial cyber defense systems have different types of reasoning.

    Non-symbolic AI used for Machine Learning / Deep Learning uses Inductive Reasoning which supports inductive statistical inference. The results are probable and the results are normally black box.

    Symbolic AI used for Machine Understanding thanks to the description logics knowledge representation & reasoning standards uses Deductive Reasoning which supports deductive logical inference. The results are certain and the results are fully transparent and explainable.

    Deductive reasoning (“top-down logic”) contrasts with inductive reasoning (“bottom-up logic”) in the following way; in deductive reasoning, a conclusion is reached deductively by applying general rules which hold over the entirety of a closed domain of discourse, narrowing the range under consideration until only the conclusion(s) is left (there is no epistemic uncertainty; i.e. unrecognized parts of the currently available set; all parts of the currently available set are available and recognized). In inductive reasoning, the conclusion is reached by generalizing or extrapolating from specific cases to general rules, i.e., there is epistemic uncertainty (unrecognized parts of the currently available set).

    Inductive reasoning is distinct from deductive reasoning. While the conclusion of a deductive argument is certain, the truth of the conclusion of an inductive argument is probable, based upon the evidence given.

    Humans use both inductive and deductive reasoning daily. The scientific method is made up of both inductive and deductive reasoning. Inductive reasoning is normally used in support of Discovery Science approaches while Deductive reasoning is normally used in support of Hypothesis-based Science.