Machine learning automation is affecting all of enterprise software, but will completely transform how we build, analyze, and consume data and analytics.
Over the past 10 years or more, visual-based data discovery tools (e.g. Tableau, Qlik, Tibco Spotfire) have disrupted the traditional BI market (e.g. IBM Cognos, SAP BusinessObjects). Yet, as transformative as these tools have been, analytics is once again at a critical inflection point.
Across the analytics stack, tools have become easier to use and more agile, enabling greater access and self-service. And yet organizations’ processes for preparing data for analysis, analyzing data, building advanced analytics models, interpreting results and telling stories with data remain largely manual and prone to bias.
Data volumes are increasing and becoming more complex to optimize cross-functional digital business decisions. As a result, the number of variables driving an outcome or best action is growing to the point where exploring every possible pattern and determining the most relevant and actionable findings is either impossible or impractical using current manual approaches leaving business people and analysts increasingly prone to confirmation bias. They often resort to exploring their own biased hypotheses, miss key findings, and draw incorrect or incomplete conclusions, which adversely affects decisions and outcomes. Furthermore, data science modeling, which is also largely manual, requires specialist skills that are in short supply at time when insights from advanced analytics must be pervasive to fuel digital business transformation.
Augmented Analytics is the Future of Data and Analytics
A new paradigm — augmented analytics — has emerged, which we started writing about in 2015 (“Smart Data Discovery: Enabling a New Class of Citizen Data Scientists”). We have now updated this research including many case studies and an updated vendor table and changed the umbrella term from Smart Data Discovery to Augmented Analytics (see Augmented Analytics is the Future of Data and Analytics).
Central to this development is the use of machine-learning automation to augment human intelligence and contextual awareness across the entire data and analytics workflow — from data to insight, to action, to impact the entire data management, BI and analytics, and data science and machine learning analytic workflow. Augmented analytics will be crucial for delivering unbiased decisions and impartial contextual awareness. It will transform how users interact with data, and how they consume and act on insights.
Both small startups and large vendors now offer augmented analytics capabilities that could disrupt business intelligence (BI) and analytics, data science, data integration and embedded analytic application vendors. Many existing modern BI and analytics vendors (e.g. Tableau, Qlik, Sisense) have augmented analytics investments on the roadmap.
This is happening largely in response to innovations from startups such as BeyondCore (acquired by Salesforce in 2016 and rebranded Salesforce Einstein Discovery, a part of the Salesforce Einstein Analytics portfolio) – (see a Tale of Two Acquisitions), DataRobot, Endor, SparkBeyond, and others, as well as from traditional BI vendors like IBM (with IBM Watson Analytics). The same is happening to self-service data preparation platforms, where machine-learning augmented data preparation vendors such as Paxata, Trifacta and UniFi are driving innovation.
Augmented analytics will also be a key feature of conversational analytics. This is an emerging paradigm that enables business people to generate queries, explore data, and receive and act on insights in natural language (voice or text) via mobile devices and personal assistants.
Data and analytics leaders must therefore review their investments and plan for adoption alongside existing investments.
You may also find this new research from Shubhangi Vashisth and team: Machine Learning: FAQ From Clients very helpful to separate the hype from reality around AI and machine learning. Both of these research notes along with many others will be included in Gartner’s special report on AI to publish by mid-August.
Ten years ago, you would have been hard pressed to find a single business application driven by machine learning. Ten years from now, you won’t find one that isn’t. Data and Analytics will be a driving force of this change.