We have SO much security data, how do we make sense of it?
Now, leaving aside the question of how you ended up in this position (maybe somebody simply gifted you a 41-node Hadoop cluster choke full of security data, or you bought one on eBay and found it to contain logs …), let’s review some scenarios that are known to occur after the above phrase is uttered. So, WE HAVE SO MUCH DATA AND ….
- … what tool do we buy to make sense of it? See this discussion (and also this) to see what happens next. To summarize: often it doesn’t end well… (<Anton steps away to submit this as his entry to The Understatement of The Year contest>)
- … let’s get some people in who can help us. If this launches your security data analysis team – great! If this enriches consultants, and is forgotten in a year – FAIL.
- … what are the questions we can ask that data? This is one of the best scenarios; indeed “having a goal” and “data exploration” are two different approaches known to lead to success
- … what are the questions others are asking of their data? See the difference between this one and the one above? Notice something funny about it? Tired of questions? 🙂 In any case, popular questions are a little like popular items on the menu: if you are truly average, you may like them. If not, they may still be a useful starting point – as long as you continue.
- … let’s assess the data, profile it and see what it may be able to tell us? in all honesty, this doesn’t happen often – it does happen only after the #2 above already happened. This book, BTW, has some ideas on how to do it.
There you have it … just thinking aloud about what makes organizations succeed with security analytics and how to guide them towards that success…
P.S. I am ready to start writing my analytics paper, so don’t expect much blogging on this topic in the next month or so. When you have 40 page paper to write, blogging kinda loses its appear, for a while.
Blog posts on the security analytics topic:
- Who Validates Alerts Validated by Your Alert Validator Software?
- Killed by AI Much? A Rise of Non-deterministic Security!
- SIEM / DLP Add-on Brain
- Those Pesky Users: How To Catch Bad Usage of Good Accounts
- Security Analytics Lessons Learned — and Ignored!
- Security Analytics: Projects vs Boxes (Build vs Buy)?
- Do You Want “Security Analytics” Or Do You Just Hate Your SIEM?
- Security Analytics – Finally Emerging For Real?
- Why No Security Analytics Market?
- SIEM Real-time and Historical Analytics Collide?
- SIEM Analytics Histories and Lessons
- Big Data for Security Realities – Case 4: Big But Narrowly Used Data
- Big Data Analytics Mindset – What Is It?
- Big Data Analytics for Security: Having a Goal + Exploring
- More On Big Data Security Analytics Readiness
- Broadening Big Data Definition Leads to Security Idiotics!
- 9 Reasons Why Building A Big Data Security Analytics Tool Is Like Building a Flying Car
- “Big Analytics” for Security: A Harbinger or An Outlier?