Neil MacDonald

A member of the Gartner Blog Network

Neil MacDonald
VP & Gartner Fellow
15 years at Gartner
25 years IT industry

Neil MacDonald is a vice president, distinguished analyst and Gartner Fellow in Gartner Research. Mr. MacDonald is a member of Gartner's information security and privacy research team, focusing on operating system and application-level security strategies. Specific research areas include Windows security…Read Full Bio

Coverage Areas:

Application Security: A Tool Cannot Solve What Fundamentally is a Process Problem

by Neil MacDonald  |  March 7, 2009  |  12 Comments

One of the areas I research is application security – not only how to develop applications that are more secure, but also how applications should be architected to consume security services. The former is increasingly important as the bad guys move “up the stack” to target applications and information. Secure application development is a priority for nearly every organization I talk with. Whether they are developing their own applications in-house or outsourcing development to a third party, the concern is the same: how can I produce applications that are more secure?

My colleague, Joseph Feiman, and I research application security testing tools, methodologies and best practices. For example, we recently published our magic quadrant rating of static application security testing (SAST) tool vendors. The interest in these tools is remarkable. The market for SAST tools has doubled every year for the past several years. However, we’ve talked with a number of clients that purchased a tool which later becomes expensive “shelfware” or where the project was halted after delivering mixed results. What went wrong? Too much emphasis on a product and not enough on process. Since the lack of a SAST tool wasn’t the root of their problem, purchasing a tool couldn’t be the complete answer. A tool alone cannot solve what fundamentally is a development process problem.

This week, Fortify and Cigital released the results of their study (The “Building Security In Maturity Model”) of nine organizations which have implemented a secure application development process. The resulting study contains a multitude of common activities and best practices observed in these organizations. While a few of the activities are not practical for every organization (for example AM3.1 “Have a science team that develops new attack methods”), the value in the study is that it collected and documented which activities have worked in the real world. Rather than try and implement every activity listed, you pick and choose which practices make sense for your organization.

Out of the one hundred or so activities the study documents, the use of automated tools are a small fraction. I believe this makes my point exactly. Automated security testing tools? Great. But they represent only a small fraction of what we need to do to produce secure applications. To be successful, we absolutely cannot ignore the associated process and mindset changes necessary.


Category: Application Security     Tags: , , ,

12 responses so far ↓

  • 1 Gary McGraw   March 10, 2009 at 1:13 pm

    Well done Neil. Just posted a pointer to sc-l. Keep up the great work.


  • 2 Jack Danahy   March 12, 2009 at 10:55 am

    I support Neil’s conclusion entirely.

    In a recent trip to Dallas, I met with two very different organizations in this area. One had taken the time to first think about their process and their people, prior to looking for products to solve the problem. The second was completely focused on the product accquistion, allowing the urgency of their interest in the space to dominate their thinking, and causing them to delay their planning for the reality of the ultimate roll-out and return on investment. I hope that they can be successful, but it is far from assured.

    I hope that people take Neil’s recommendation to heart.

    For a supporting discussion of the same topic, take a look at my post “Five Rules to Saving Money by Avoiding Security Shelfware” at :


  • 3 Mandeep Khera   March 12, 2009 at 10:38 pm


    I couldn’t agree with you more. As we have talked before a few times , throwing just the technology at a problem is a bad idea. Without a sound process, all automation attempts can fail regardless of whether it’s ERP, App Security, or any other business process. Process defines the clear path to your goal, automation gets you there faster than manual efforts.


  • 4 Yet another big company with SQL Injection problems (British Telecom) | N-Stalker Web Security Community   March 13, 2009 at 6:59 pm

    […] good post that we read today really fit this problem: “Application Security: A Tool Cannot Solve What Fundamentally is a Process Problem“  that simply agrees with that. Companies aren’t making their home work: scanning […]

  • 5 Ashish Popli   March 16, 2009 at 12:08 pm

    This is a great post!

    After several years of working in application security domain and reading this post just by chance, I felt a lot of data points are nicely amalgamated.

    I am happy that we have come a long way in understanding the problem – the first step in solving it.

  • 6 Neil MacDonald   March 16, 2009 at 7:12 pm


    We’ve been talking about building more secure applications for years. And talking. And talking. But, there’s some hope. You are right – we’ve come a long way in understanding the problem. We’ve got several successful companies that we can point to to start to understand the best practices.

    That’s why I like the study cited in my post above from Fortify and Cigital. It provides at least a framework for starting the discussion. Even if you have started, you could use the study to compare to what you are doing and see what might be missing.

  • 7 Shrinking Budgets: Application Security Tools vs Process Tradeoff « noFUD - No Fear Uncertainty or Doubt   April 10, 2009 at 2:44 pm

    […] is given to how to integrate this in existing into development lifecycles. Application security is fundamentally a process problem and you can’t solve it just by using […]

  • 8 Beware New Malware in Web Apps | Computer Security   April 14, 2009 at 2:50 pm

    […] vendors aren’t ready either. It’s pretty simple. If we don’t proactively start efforts now to produce more secure applications and demand the same from our software providers, the bad guys will find the vulnerabilities for […]

  • 9 James   June 13, 2009 at 9:20 am

    I am surprised that the research didn’t contain additional guidance such as referring readers to organizations such as OWASP. Would be curious to know why this was left out?

  • 10 Byte Code Analysis is not the same as Binary Analysis   July 24, 2009 at 1:28 pm

    […] posted many times on the importance of application security. Recently, my colleague Joseph Feiman and I published a magic quadrant for static application […]

  • 11 Security Thought for Thursday: DLP Should be a Process, not a Product   September 10, 2009 at 5:28 pm

    […] one of the DLP vendors such as McAfee, Symantec, EMC/RSA and so on. Much like I talked about in this post on application security, a product cannot solve what first and foremost is a process problem. The […]

  • 12 More Application Security Goodness From OWASP   January 14, 2010 at 9:13 am

    […] the beginning. Many of the clients I talk with are highly focused on the ‘produce’ part – improving their development processes to ensure that more secure code is produced and that security testing is incorporated in the […]