Human Interviewer: “Do you prefer dogs or cats?”
Randy the Robot, “Yes, I’m very familiar with their pixel patterns.”
In the words of former U.S. Secretary of Defense, Donald Rumsfeld, “There are known knowns, known unknowns and unknown unknowns.” He positioned unknown-unknowns as the most challenging situation. For AI, it is an impossible situation. AI operates best in the known-known situation. In other words, it is best to know exactly what you are looking to find.
Known-Knowns is Where AI Accuracy is Most Accurate
AI is taking the field of radiology by storm. In 2018, Stanford created the AI CheXNeXt algorithm trained with over 100,000 chest X-rays to identify 14 pathologies. For 10 of the diseases CheXNeXt performed on par with radiologists. On one it outperformed radiologists. With three maladies the radiologists outperformed CheXNeXt. This success is only possible because we know what normal and abnormal chest X-rays look like for these pathologies. Actually, the known-known scenario applies to a large number of AI computer vision detection scenarios from recognizing defects on a manufacturing line to identifying products in a shoppers grocery cart and even diagnosing illnesses by analyzing facial features. Computer vision is one of the fastest growing sectors of AI because we know what we are looking for and deep learning is getting better and better at image recognition.
Remember the sage words of Yogi Berra, “If you don’t know where you are going, you’ll end up somewhere else.”
A poorly defined result was one point of difficulty in Princeton’s Fragile Families Challenge that tasked 160 researcher teams with predicting the life trajectories of a set of children given a large observational data set. Pursuing something as nebulous as predicting “life outcomes” will rarely bear fruit with AI. In addition, they chose some variables from one wave as the “outcomes” from data provided in previous waves with no known causal relationship between the variables from one wave to another. It was a shot in the dark to see if there was some correlation.
Poorly Defined Outcomes + Lack of Transparency = The “Uncertainty Dilemma”
A poorly defined outcome is but half the real problem. Machine learning based AI, by its nature, suffers from a lack of transparency. Transparency is obtaining a clear understanding of how the AI came up with the answer it did. And the “deeper” the neural network (e.i., more layers) the less transparent.
A poor understanding of the desired result in addition to inherent transparency challenges creates an “Uncertainty Dilemma.” In the computing world, this is pretty unique to AI. Although artificial intelligence is math delivered by a computer, it is not a calculator. Business leaders should never accept AI results as if they were basic calculations. AI can suffer from a similar uncertainty to “gut feel.” This turns artificial intelligence (AI) into artificial gut feel (AGF).
Figure 1 depicts the “think you are correct” vs. “you are actually correct” matrix.
Artificial intelligence, especially deep neural network (DNN), results have a high potential to end up in the worst quadrant where you think you are right but you are not. This is the recipe for a bad business decision. Prudent business leaders will default to AI’s tendency towards this bottom left quadrant and guard against it. This tendency is why AI governance is needed and is a growing market.
One of my favorite aphorisms comes from astrophysicist Neil deGrasse Tyson where he states that the real challenge is “knowing enough to think you are right but not enough to know you are wrong.” So how do you, as a business leader, combat this challenge?
Know Enough to Know When You are Right
This means, have a very clear understanding of the desired outcome. Business leaders need a healthy skepticism when it comes to AI and should not abdicate decision making to the data scientists, mathematicians and developers. Nothing is black and white. So, business leadership is necessary to probe, uncover and manage the uncertainties. If the AI professionals can’t articulate, in clear business language, exactly what they are trying to solve, what the answer looks like and the basics of their solution then you are facing a serious business red flag.
Better yet, as a leader in the business you should drive a clear direction on what the AI needs to achieve and provide a clear description of what good looks like. This is what leads to valuable applications of AI to business.
The Gartner Blog Network provides an opportunity for Gartner analysts to test ideas and move research forward. Because the content posted by Gartner analysts on this site does not undergo our standard editorial review, all comments or opinions expressed hereunder are those of the individual contributors and do not represent the views of Gartner, Inc. or its management.