Blog post

Bias Labeling Is Useful, but We Need a Better Approach

By Darin Stewart | July 28, 2021 | 0 Comments

Security and Risk Management LeadersTechnology, Information and Resilience Risk

I am a big advocate of fact checking. I’ve recommended that fact-checking plug-ins be standard extensions on every browser.  I’ve also cautioned that fact checkers should avoid value judgements and restrict themselves to the veracity of a factual assertions. But what about bias labeling?

Many fact check services also indicate the political bias of a resource. Isn’t categorizing content as liberal or conservative, left-leaning or right-leaning a form of value judgement?  If the designation is done arbitrarily, then yes. Bias labeling of this form is unquestionably problematic and should be rejected. To avoid this issue, most bias checking features depend on third parties, such as AllSides and Media Bias / Fact Check.

Media bias evaluation services claim to be neutral, explicitly chartered to assess bias without value judgement. They adhere to transparent and published methodologies to assign ratings.  Importantly, they are not intended to “warn, discourage, or prohibit” but rather to encourage viewpoint exploration. For example, AllSides’ declared mission is to “help you to easily identify different perspectives so you can get the full picture and think for yourself. Hidden bias misleads and divides us. By making the political leanings of hundreds of media sources transparent, AllSides frees people from one-sided filter bubbles so they can better understand the world — and each other.

Admirable, but also worthy of scrutiny. The most biased media sources usually present themselves as “fair and balanced” while being anything but. Mediators, fact checkers and bias raters need to be held to account; judged according to their own published methodologies and the highest standards of professionalism.  They don’t always measure up.

According to University of California Los Angeles Communication Professor Tim Groelingalmost all the claims of bias that are out there are exceptionally poorly supported.” For example, the Poynter Institute  has called out Media Bias / Fact Check as “a widely cited source for news stories and even studies about misinformation, despite the fact that its method is in no way scientific.

The problem is there is no standard methodology or rubric for identifying and quantifying bias. Poynter developed a code of principles for the International Fact-Checking Network (IFCN) but these are ethical guidelines not a procedural methodology.  Current bias-ratings amount to little more than good faith, best efforts by “armchair media analysts.” Most fact checkers are acting in good faith to try to highlight problematic content without suppressing it.  But if we are ever going to build a resilient, reality-based infosphere, we are all going to need to up our game.

Unscientific methodologies are not going to cut it in the disinformation wars, especially as we become more and more dependent on machine learning to make our judgements for us. Radical transparency and rigorous evaluation are necessary if we are to evaluate information without suppressing viewpoints.

Comments are closed