Gartner Blog Network


Big Data for Security Realities – Case 2 Variety Explosion

by Anton Chuvakin  |  October 17, 2013  |  Comments Off on Big Data for Security Realities – Case 2 Variety Explosion

Part of my research this quarter focuses on assessing the reality of using big data approaches for security and providing practical, GTP-style recommendations for enterprises. So, what else IS real?

One more case that occasionally (not as often as Case 1) shows up is “data variety explosion.”

Specifically, this scenario goes like this:

  1. The organization gets a SIEM tool and generally likes it (a good SIEM can be effectively used for both real-time and historical data analysis – typically in the 14-90 days range)
  2. Use cases are defined, logs and context data collected to support them
  3. Their use of SIEM expands, and the organization starts to think what else they can consume and analyze
  4. Some people suggest flows, others – packets; there is a group that wants email contents, IM conversations, application transaction records, surfed web pages, transmitted files, system configurations; then somebody suggests phone call records, social status feeds , voice recordings and eventually video from cameras …
  5. In essence, a dizzying array of timed records and loosely related context data is proposed to solve various problems (often related to not just infosec, but also non-IT compliance, fraud, safety, operational risk, etc)
  6. The organization looks beyond SIEM and discovers the wonderful world of Hadoop, Cassandra, MongoDB and other “big data-ish” technologies and starts their own big data project.

What is the purpose of this “high variety” pile of data? Linking and relating! As a result, the organization may [eventually] possess a central data hub, useful for wide array of projects, going much beyond infosec and IT. The system enables all sorts of analysis of data relationships, whether starting from logs/alerts or not.

Related posts on the topic of big data for security:

Category: analytics  big-data  security  siem  

Anton Chuvakin
Research VP and Distinguished Analyst
5+ years with Gartner
17 years IT industry

Anton Chuvakin is a Research VP and Distinguished Analyst at Gartner's GTP Security and Risk Management group. Before Mr. Chuvakin joined Gartner, his job responsibilities included security product management, evangelist… Read Full Bio




Comments are closed

Comments or opinions expressed on this blog are those of the individual contributors only, and do not necessarily represent the views of Gartner, Inc. or its management. Readers may copy and redistribute blog postings on other blogs, or otherwise for private, non-commercial or journalistic purposes, with attribution to Gartner. This content may not be used for any other purposes in any other formats or media. The content on this blog is provided on an "as-is" basis. Gartner shall not be liable for any damages whatsoever arising out of the content or use of this blog.