Much of the unstructured data we collect from our infrastructure components has limited value. It’s often used to understand threats against our organizations in common SIEM type tools. Yet another set of technologies have evolved in the business intelligence space, as well as the web analytics space to make sense of this type of data. These tools started off being simplistic log analysis tools, but have evolved into full featured analysis platforms with an understanding of behavioral patterns.
This technology has started to be applied to operations, in assisting with the triage of production issues. There is a lack of research in this area, I’ve been working on defining a coverage area here at Gartner to address this gap. There are many vendors which have evolved to create value from these unstructured data sets. Look for the first part of my upcoming research to be published soon defining this area, and upcoming research based on your interest.