Are Media Bias Charts Reliable?

In an increasingly polarized world, more people are looking for unbiased information. One of the tools commonly used to find such information is media bias charts, the most popular ones being from Ad Fontes Media and AllSides. How reliable and useful are these charts?

Summary

The Good

1. Both Ad Fontes Media and AllSides are relatively transparent about their methodology, which makes it easier to trust their classifications of news sources.
2. Both organizations appear to be neutral from an ownership and funding perspective, without any major outside owners, suggesting that their products are genuinely intended for the public’s benefit.
3. Both charts are simple and easy to understand.

The Bad

1. Bias — as in a frame of reference, or viewpoint — is not necessarily a bad thing. Every journalist has a point of view when they write a story. Downgrading news sources because they have this viewpoint is often unfair and not of much use to readers.
2. Both charts use humans in their analysis. While they strive to be objective, humans evaluators have their own biases and it’s probable that this creeps into their ratings for a source.
3. Rating the bias of an entire source with one single grade is too coarse a metric to be useful. There is often a mix of good writing, and poor writing, at every news outlet. A single rating for an entire source means readers often miss out on the best writing and viewpoints.

It’s Human to Have Bias

The idea of unbiased news is alluring, but all humans have some bias — or frame of reference — so every article is likely to have some bias. A better approach is to accept that bias exists and get into the habit of reading multiple well-researched articles from different points of view so that you get a better understanding of the story. 

Indeed, Kelly McBride, senior vice-president at Poynter Institute — a news guidelines organization — is not sure “bias should be the focus of the charts at all. Other factors — accountability, reliability and resources — would offer better insight into what sources of news are best.”

A good example of this is The Economist. A well-respected publication that’s almost 200 years old, it’s endorsed by an impressive list of political figures, entrepreneurs, and cultural icons. But any longtime reader of The Economist will admit that the newspaper has a visible bias. 

The Economist is clear about its stance: fiscally conservative, socially liberal, and pro free trade and free markets. Elements of these stances can arguably be seen as moderately conservative, moderately liberal, or somewhere in between, which calls into question the ratings of media bias charts (Ad Fontes says they are a relatively neutral source while AllSides they are moderate left). It’s possible that they equate the lack of opinionated writing to being neutral but one can write in a neutral manner and still frame a story with a very specific viewpoint.

Regardless, the bias rating should not detract from the fact that The Economist has generally highly informative reporting. The main value of the bias rating should be for readers to understand how stories are likely to be framed and to compensate for that by reading different viewpoints across the political spectrum.

Adding Human Bias to Bias Ratings

Both Ad Fontes Media and AllSides use analysts, sometimes in combination with crowd-sourced votes, to determine a site’s bias rating. Since all humans have bias, and are particularly susceptible to confirmation bias, there is a risk that ratings will be inconsistent or unfair.

Ad Fontes Media, for example, says the following five sources are roughly similar in bias and credibility: New York Times, Vox, CNN, Vice, and The Guardian. There is likely considerable disagreement among readers of this point. 

Similarly, AllSides groups together the National Review and the American Spectator on the right. The former was generally anti-Trump, during his tenure, while the latter was unabashedly pro-Trump. 

These examples show how difficult it is to have a bias rating that is acceptable to a wide audience. Fundamentally, bias is relative to a baseline of neutrality and that itself has no accepted definition. Hence, there is always a degree of subjectivity in bias ratings.

Bias Ratings Are Too Coarse

Media bias charts offer a single rating for every source. This can be too coarse of a method given that most news outlets are composed of dozens of writers and editors of varying quality and perspectives. 

Some reporters research their work deeply and, by virtue of reporting on a topic consistently, often include rich context on issues they report on. However, not all reporters do this. And lumping the two together means you might miss out on great writing and perspectives offered by that source.

A good example is Reason Magazine, a prominent Libertarian news site. Ad Fontes rates them as “skews right” with “high variation in reliability.” So how does a reader use this information? Do they ignore Reason altogether? How can they tell which are their best articles?

Going Beyond Bias Ratings

For all the reasons above, bias ratings are not useful enough to find high-quality journalism. As Poynter’s Kelly McBride says, more useful is “accountability, reliability and resources.” 

This is the premise behind how The Factual rates individual news articles for credibility. The Factual looks at four factors to determine how informative and neutral each article is, including: (1) extent and diversity of sources, (2) the neutrality of writing tone, (3) author’s topical expertise based on previous publications, and (4) site reputation. And because The Factual does not penalize an outlet for having a bias, but rather provides this data so readers know how a story is framed, readers end up with a range of highly credible viewpoints from across the political spectrum. This approach helps readers combat any bias present and gets them closer to the full story.