Gartner Blog Network

Advanced Persistent Threats: Finding the Needle in a Haystack

by Neil MacDonald  |  April 14, 2011  |  4 Comments

Whether or not you agree with the use of the term “Advanced Persistent Threat” (APT), we can agree that there is a very real threat from advanced intrusions which persist undetected in our systems.

By definition, these intrusions are advanced so our traditional (and increasingly ineffective) protection mechanisms such as firewalls and antivirus don’t catch these APTs.

Think of the APT as a “needle in the haystack” that we need to find. To make things worse, we aren’t sure what the needle looks like (no signatures). So how do we find them?

The answer may seem counterintuitive and is rooted in a whitelisting paradigm: Remove the hay that you know is good (“high assurance hay”). When you are done, the needles remain.You don’t have to know what needles (APTs) look like, you only need to know what high assurance hay looks like.

This simple analogy illustrates the foundational importance of “whitelisting” based approaches in information security across the entire stack — whether it is session flows on the network, applications that a system is allowed to execute or transactions on the back-end.

Of course, this example assumes you know what high assurance hay looks like. In the application control / whitelisting space that I research there are a number of providers that are stepping up to build high assurance databases of application executables including Bit9, CA, CoreTrace, Harris (acquired SignaCert), Lumension, McAfee, Symantec, Trend and others.

There is no silver bullet in information security, but applying a whitelisting-based mentality to all of our information security solutions should be foundational to keep the bad guys from gaining the upper hand – and to keep the needles out of our hay.

Additional Resources

Five Board Questions That Security and Risk Leaders Must Be Prepared to Answer

As board members realize how critical security and risk management is, they are asking leaders more complex and nuanced questions. This research helps security and risk management leaders decipher five categories of questions they must be prepared to answer at any board or executive meeting.

Read Free Gartner Research

Category: beyond-anti-virus  next-generation-security-infrastructure  security-of-applications-and-data  

Tags: beyond-anti-virus  defense-in-depth  endpoint-protection-platform  information-security  security-summit-na  whitelisting  

Neil MacDonald
VP & Gartner Fellow
15 years at Gartner
25 years IT industry

Neil MacDonald is a vice president, distinguished analyst and Gartner Fellow in Gartner Research. Mr. MacDonald is a member of Gartner's information security and privacy research team, focusing on operating system and application-level security strategies. Specific research areas include Windows security…Read Full Bio

Thoughts on Advanced Persistent Threats: Finding the Needle in a Haystack

  1. Paul Zimski says:

    Very concise analogy that distills the whitelisting paradigm cleanly – I like it. I tend to think about the “haystack” from two different angles – I need to find the needles already in it (because they are definitely there and I want to remove them) and I also want to avoid having any new needles added moving forward. Do you think focusing on the “high assurance hay” solves both of these issues?

  2. Neil MacDonald says:


    Agree. If you have a signature, use it! Much easier than all that hay shuffling 🙂

    For devices that directly support end-users, application whitelisting alone won’t cut it. See these links

    Also, there are lots of ways that whitelisting can make blacklisting approaches more effecient and effective.

    High assurance workloads (desktop or server) are foundational, but if the workload directly supports end-users that are able to download, install and execute arbitrary code then blacklisting approaches will continue to be needed.


  3. Kevin Rowney says:

    At Symantec, we see very few enterprises successfully implementing pure white listing. End-users really resist having one blanket policy of allowable white-listed applications. The CFO or CEO may not care that much (since they don’t tend to use edgier applications) but the software development and marketing teams often resent the policy. Bottom line, one policy with one approved whitelist of applications is now highly unpopular at most enterprises.

    They key is to identify the volumes of applications that are in the wild and assign reputation scores to them. If an application is incredibly rare and unknown (relative to a comprehensive knowledge of known-good software now in circulation) it would be scored accordingly to help enterprises rate and rank the risks.

    This approach is better at finding APTs, and enterprises don’t resist this kind of policy because it is adjustable. You can assign a reputation response threshold dependent upon the user that varies with their job function. For example, give the CFO a very constraining threshold of reputation so that very few applications of any kind of level of edgy risk ever reach his/her desktop, and give your software devs more permission to do the edgy things they need to do.

  4. Neil MacDonald says:


    Agree partially. Your point about a pure whitelisting approach for end-user desktops is accurate. However, this is why my Gartner research refers to this as “Application Control” and not “whitelisting”. There are many shades of grey in the real world between things that we have a high assurance are good (whitelist) and those that we have a high assurance are bad (blacklist).

    So, reputation services will be a key enabler to making application control work in the enterprise for end-users. Bit9, Symantec, Trend, McAfee and others are ALL building reputation databases.

    On the other hand, servers and embedded devices are great candidates for a “pure” whitelisting / application control approach.

    Ideally vendors would provide a foundational whitelist of high assurance applications and supplment this with the reputation database. Think of this as a continuum — for example, a score of a +10 if the application is known, digitially signed and received directly from the vendor. A score of -10 if the applicaiton is known to be malware… and everywhere in between for those shades of grey.

    The future is a combination of whitelists, blacklists and greylists working together as a system – thus the term “endpoint protection platform” in my Gartner research on the future of endpoint security:


Comments are closed

Comments or opinions expressed on this blog are those of the individual contributors only, and do not necessarily represent the views of Gartner, Inc. or its management. Readers may copy and redistribute blog postings on other blogs, or otherwise for private, non-commercial or journalistic purposes, with attribution to Gartner. This content may not be used for any other purposes in any other formats or media. The content on this blog is provided on an "as-is" basis. Gartner shall not be liable for any damages whatsoever arising out of the content or use of this blog.