Gartner Blog Network


Why SIEMs F*cked Up Application Log Analysis?

by Anton Chuvakin  |  January 13, 2017  |  13 Comments

This is going to be a short one: why do you think the SIEM vendors f*cked up application log analysis so badly?

Think about it, SIEM technology started roughly in 1997, so 20 years ago. 20 years is like 2 gazillion years in “IT years.” But even today I see a lot of people who equate SIEM with “IT infrastructure monitoring” (i.e. routers, servers, security appliances) or, worse, think that “SIEM is a network security technology.” I’ve long fought this view, but I essentially lost this battle …

So, now, I am sitting here listening to UEBA / UBA clients gush about how great their UEBA is with application log analysis and application security monitoring. They bring up all sorts of esoteric applications (machine parts management, medical research support, financial transaction processing, etc) and then wax poetic about how great their UEBA tool is for revealing insights from the log data and how it saved them so much dough, despite the fact that they paid $1,000,000 for their UEBA.

Why is that? What do UEBA tools know that SIEMs miss?

If you think, “but wait, UEBA has ML and magic AI stuff”, think again. Well, it does, but some of the use cases are very rule-based and do not extend beyond the “IF <this> AND <that> THEN <whatever>” correlations.

So, what is to blame here? Perhaps, some of these …

  • SIEM historical association with IT infrastructure, not business application, monitoring
  • Lack of schema support for business application log data
  • The very requirement to shove the data into a standard schema
  • Obvious lack of out-of-the-box content (rules, alerts, dashboards, etc) for custom application security monitoring
  • Heavy role of identity and entitlement data in application monitoring
  • Requirements for inclusion of non-IT data in analysis
  • Or even … customers never pushing SIEM vendors to “do applications”?

… or even something completely different. All in all, it feels like the SIEM guys blew this one as well… and now need to catch up with the UEBAs.

Related blog posts about security analytics:

Category: application  monitoring  security  siem  ueba  

Anton Chuvakin
Research VP and Distinguished Analyst
5+ years with Gartner
17 years IT industry

Anton Chuvakin is a Research VP and Distinguished Analyst at Gartner's GTP Security and Risk Management group. Before Mr. Chuvakin joined Gartner, his job responsibilities included security product management, evangelist… Read Full Bio


Thoughts on Why SIEMs F*cked Up Application Log Analysis?


  1. Avi Chesla says:

    Couldn’t agree more. I believe that SIEM were also supposed to “translate” security logs into a unified language of security intent. allowing to make correlation much simpler
    Avi

  2. Matt says:

    I have never considered SIEM to be very useful for analysis other than simple “if you see X, let me know”. X is supposed to be some normalized attribute applied across all sources, but SIEMs parse so inconsistently that X usually = this specific event from this specific type of device. Or worse, this specific keyword in the raw events from this specific type of device. And this is just for the infrastructure events! Don’t get me started on how long it takes the SIEM vendors to update parsers to support new versions of supposedly supported infrastructure. Throwing custom application logs into this mess is asking a lot.

    I think one big problem is that we have no meaningful standards, at least none being consistently followed, for application security logs (and I’m not just talking about internally developed apps). Large enterprises have thousands of app log sources. Acquisition is difficult enough, never mind creating the custom parsers/schema mappings. I pick C, “the very requirement to shove the data into a standard schema”, as a major contributor.

    • Thanks for the comment.

      >analysis other than simple “if you see X, let me know”.

      Sure, of course, but also “the above and only for the infrastructure logs”

      Also, good point re: lack of standard AND bad normalization are worse for app logs, these things are better for the infra..

  3. Jeff Towle says:

    Pulling log data from applications and transforming it into a data lake can be useful. The problem I’ve witnessed is that finding actual anomalous activity can take a lot of human interaction to evaluate and flag false positive for training the platform to report reasonable (actionable) events over time. So the algorithms and automation are immature and its very tough to justify investing in that sort of engineering. UBA was born from the IDM world, so there’s a tendency to focus on user identity access/attribute/activity aspects of risk, which is easier for them to correlate, visualize and score for risk. We are a ways off from true AI with unsupervised learning and automation of risk event patterns IMHO.

    • THanks for the comment Jeff. Indeed, ” take a lot of human interaction ” seems to the central issue since for security appliances and tools the SIEM vendor can build reports/alerts/analysis and just ship them while for apps it is often about a lengthy human discussion….

  4. Jeff Hall says:

    I think it comes down to the fact that application log data is so all over the board in not only format but what it contains. Not only that, but application developers are notorious for not documenting EVERY message that could be produced. As a result, how do perform an analysis on something that is incomplete?

    • Thanks for the comment Jeff — I think app logs are harder to make sense of, indeed, and requires the SIEM/UEBA people interact with “pesky humans” ™ at a customer site to gather those insights… But somehow UEBAs vendors seem to be doing a better job here compared to SIEMs…

  5. I’ve found that most development teams (“DevOps” buzzword officially required) simply couldn’t wait 20 minutes or more for their SIEM to index and analyze their application logs because that would mean they don’t know if the latest push to production introduced errors and caused an outage. Every minute here can cost the business millions.

    Once they went elsewhere for solutions that could do analyze application logs in (near) real-time, security teams usually didn’t have access to these application logs in their SIEM, so they didn’t push the vendors to improve the analysis. Then again, this is just a theory and I may be seeing causation where there is simply coincidence.

  6. Hi Anton, I agree with you: bare SIEM without custom use cases is just expensive almost unused tool. And there is no motivation for customers and partners to share their created content (almost all cases in HPE, IBM an Splunk appstores are free, lite and poorly supported).

    That is why our company created multi-SIEM online Use Case Library where everyone can earn points for creating content and spend them – for downloading. Please, have a look at https://my.socprime.com/en/ucl/

  7. […] Why SIEMs F*cked Up Application Log Analysis? (UEBA / UBA research) […]



Comments are closed

Comments or opinions expressed on this blog are those of the individual contributors only, and do not necessarily represent the views of Gartner, Inc. or its management. Readers may copy and redistribute blog postings on other blogs, or otherwise for private, non-commercial or journalistic purposes, with attribution to Gartner. This content may not be used for any other purposes in any other formats or media. The content on this blog is provided on an "as-is" basis. Gartner shall not be liable for any damages whatsoever arising out of the content or use of this blog.