Algorithmic bias is often in the news and has become a bit of a hot button in ethical AI. The vilification of algorithmic bias is unwarranted and misplaced. A focus on bias in algorithms in this context is wasted effort.
Recently I have participated in a few long Gartner research threaded discussions on “explainable AI,” AI ethics and how to manage bias in particular. I’ve also recently attended a few discussions with machine learning luminaries such as Harry Shum, former EVP of AI and Research at Microsoft. In those discussions there was a lot of talk about bias in algorithms and how to recognize it and manage it. However, my position is that if you are primarily looking for bias in the algorithm you are looking in the wrong place. From these interactions, among other things, I believe there are 4 stages relevant to AI bias; real world bias, data bias, algorithm bias and business bias.
The “real world” bias involves actual bias in the real world. Or more accurately, what are the biases that people and systems impose on the relevant portion of the real world. So if we are examining home lending practices or real estate sales practices or K-12 teaching systems, this layer is about the inherent biases within those people, practices, systems, etc.
Bias in the data involves whether the data accurately reflects the real world. If the data is reflective of the real world it will also reflect the inherent biases in the real world.
Bias in the algorithm is a measure of how well the algorithm “fits” the data. If the algorithm is overfitted then the result is lots of false negatives. If it is underfitted the result is false positives. Algorithm bias is more of a data science mathematical measure than an ethical issue. ML algorithms seek out bias to classify data. Getting the algorithm bias right is key to getting a good result. “Good” in this case is not an ethical question but a matter of accuracy between underlying data and AI results.
And finally business bias represents how businesses act upon the data/AI for business benefit. Businesses discriminate all the time. It is critical to success. Ethical businesses will strive to discriminate ethically. They will discriminate based on relevant criteria that determines a good prospective customer, storefront geography, job candidate, product investment, etc. They will avoid discriminating based on gender, race, sexual preference or other unethical or immaterial criteria.
So what does all this mean? It means that:
- You don’t want unethical bias in the real world (but that may be beyond the control of many of us).
- You do want bias in the data because you want data that reflects the real world.
- You want best fit bias in the algorithm because you want the result to accurately reflect the bias in the data.
- You don’t want unethical bias in your business so it is critical that you act appropriately upon accurate results from AI algorithms.
At this point it should be pretty obvious that “business bias” or how a business chooses to act upon the AI results is the more appropriate place to look (and act) when it comes to ethical AI. AI machine learning algorithms provide an unprecedented capability to expose unethical bias. Organizations should actively look for the often hidden unethical biases in the data (that accurately reflects the real world) to make ethical business decisions. This goes beyond “explainable AI” and is the true core of ethical AI.
Harry Shum told a story about when they were building computer vision algorithms against ImageNet. They had hundreds of hidden layers in their deep neural network (DNN) with tens of nodes in each hidden layer. This means there were tens of thousands or more weights that were set dynamically. Explainable AI is near impossible with these extensive DNNs. He showed how, with these types of DNNs, instead of trying to achieve explainability they set out to find the unethical bias in the data. They listed out possible unethical biases in the data (i.e., descrimination based on gender, race, etc.) and actively used AI to find and map them using techniques like proximity and parallelism. This goes beyond explainable AI and gets closer to transparent AI which exposes all relevant bias in the data.
As has been true for eaons, business bais remains the key to ethical behavior (AI is just the current flavor). Businesses should actively use AI to discover both ethical and unethical bias and act responsibly upon those findings.
Mathematically speaking, ethical AI is the sum of transparent AI and ethical business bias 🙂
Some relevant Gartner content includes (available only to Gartner clients):
The Gartner Blog Network provides an opportunity for Gartner analysts to test ideas and move research forward. Because the content posted by Gartner analysts on this site does not undergo our standard editorial review, all comments or opinions expressed hereunder are those of the individual contributors and do not represent the views of Gartner, Inc. or its management.