Founded in March of 2008 by baseball analyst Nate Silver, FiveThirtyEight originally began as a polling aggregator, its name derived from the number of electors in the U.S. electoral college. Well known for its election predictions, it successfully predicted the outcomes of both presidential races in 2008 and 2012. Silver pioneered unique methods incorporating statistics and demographic data not used before. As the organization grew, it would go on to receive many awards as its focus broadened under the rubric of data journalism, but it received significant flak in 2016 and 2020 for projections that significantly diverged from electoral results. Today, over half of Americans indicate they are skeptical of polling. Given this mixed reputation, how reliable is FiveThirtyEight?
How Does The Factual Rate News Sources?
The Factual analyzes more than 10,000 news stories every day to help readers find the most informative, least-biased articles. Our news-rating algorithm scores each article along four metrics: (1) cited sources and quotes, (2) publication history, (3) writing tone, and (4) author expertise. These scores combine in a weighted average we call a Factual Grade, which ranges from 0–100%. (See our How It Works page to learn more about our algorithm.)
For this study, we analyzed ~1,000 articles each from 240 news sources. The average Factual Grade for the entire dataset was 62.5%. Based on these averages, we can compare the performance of news sites across the media ecosystem. The entire dataset can be explored in greater detail here.
How Factual Is FiveThirtyEight?
FiveThirtyEight scored an average Factual Grade of 76.2%, placing it in the 97th percentile of our dataset. In fact, the site achieved the seventh-highest score for all site that we measured.
These high scores are not much of a surprise given FiveThirtyEight’s strong commitment to data. The site relies heavily on data and uses extensive sourcing to substantiate claims, meaning most stories include numerous links to high-quality external sources. Likewise, stories are generally authored by experienced members of a dedicated team of journalists. These individuals routinely cover the same topics and produce high-scoring articles, receiving credit for strong author expertise.
Like any news source, scores for articles from FiveThirtyEight varied widely. For example, some scored above 90%, while others scored below 70%.
How Opinionated Is FiveThirtyEight?
One of the metrics The Factual uses is the Writing Tone, which measures how opinionated the writing is in an article. For this metric, the algorithm looks for signs of subjective commentary (e.g., first person pronouns and unnecessary adverbs), as well as the emotional nature of selected words, and sees how prevalent they are for a given length of text. More neutral text receives higher ratings, with “0” being the most opinionated and “1” being the most neutral.
FiveThirtyEight had an average Writing Tone score of 0.49, placing it in the 30th percentile in our dataset for this metric. This suggests that articles from FiveThirtyEight are often highly opinionated, meaning they are likely to use emotionally loaded language. Partially, this can be attributed to the sports journalism that the site largely focuses on, which involves articles that generally involve more opinion and argumentation, as seen through titles such as “New Hampshire Is Tiny And Pretty Weird. That Could Help Maggie Hassan.”
What Is FiveThirtyEight’s Political Bias?
The Factual classifies news sites by political bias as either Left, Moderate Left, Center, Moderate Right, or Right. This classification is derived from third-party assessments from media bias organizations such as AllSides and Media Bias/Fact Check (MBFC). Based on this data, The Factual assigns FiveThirtyEight a Moderate Left bias.
AllSides assigns a “Center” bias to FiveThirtyEight based on 8,266 community ratings and independent research. While AllSides notes this as an initial rating, it emphasizes a three-year partnership with the New York Times, which AllSides has designated as “Lean-Left” for the past three years. FiveThirtyEight’s coverage of the 2016 presidential election received criticism from both sides of the political spectrum, on one side for referring to Trump as not “a real candidate” and for downplaying Sander’s primary bid on the other.
MBFC assigns a “Left-Center” bias based on story selection that typically favors liberal causes. It notes a tendency to publish factual information that utilizes emotionally loaded headlines like “Is Joe Biden Toast If He Loses Pennsylvania?” and “What The Heck Is Going On With AstraZeneca’s COVID-19 Vaccine?” However, MBFC gives the site high scores for factual reporting, proper sourcing, and a clean fact-check record.
Who Owns FiveThirtyEight?
FiveThirtyEight had grown from a personal blog to a highly respected site with decorated journalists. After a three-year partnership (2010–2013) with the New York Times, Silver wanted to transform the site into a mature media organization. ESPN bought the blog in 2013, expanding the newsroom from two full-time journalists to a staff of 20. In April of 2018, ABC News acquired ESPN, making Disney the overarching owner of the FiveThirtyEight.
Disney is an immensely successful media conglomerate with interests in all corners of entertainment. It has an annual revenue of close to $67 billion — most of which comes from advertising. In 2019, during the launch of a slew of Disney media products, many critics ESPN network sports analysts for advertising the new Disney+ service. Many critics contended this degraded consumer confidence in the various networks’ news coverage. However, this does not appear to have implicated FiveThirtyEight. Overall, there is no evidence that ownership by ESPN, ABC News, or Disney has influenced the publication's output.
Please check your email for instructions to ensure that the newsletter arrives in your inbox tomorrow.
Why Does It Matter?
News articles are bound to have bias because all authors have some frame of reference within which they describe a story. Political bias ratings are helpful in understanding this framing. However, it can be more beneficial to know how factual an article is based on quantifiable metrics that can be seen across the media ecosystem, from cited evidence, to author expertise, to the writing tone. This is what The Factual ascertains.
Reading several, highly rated articles from across the political spectrum helps counter the bias of any news source or story. To have the day’s most factual news stories delivered to your inbox every morning, subscribe to our daily newsletter.
Article updated on September 15, 2022 to reflect new data.