One of the mysteries I am planning to explore in my research on using big data approaches for security is this: why so many surveys and media reports seem to show (no links here!) that 20%-40% of organizations utilize big data approaches for security today, while in reality this is not the case – by a long shot.
Let’s see. Here is the canonical definition of “big data”:
“Big data” is high-volume, -velocity and -variety information assets that demand cost-effective, innovative forms of information processing for enhanced insight and decision making. (source)
Notice something interesting: the 3Vs are described as volume, velocity AND variety! If you have a small pile of variable data, say, 10Mb of it, we are definitely not in a big data realm. A huge RDBMS of structured (not varied) records is not big data either. The idea is AND, not OR!
On the other hand, see how some other people define big data and “big data tools”:
Sorry, guys, but this is SECURITY IDIOTICS, not security analytics. A reality of using big data for security is much more rare – and much more precious….