Since the election on November 3, one question on many people’s minds was how the polling industry could have interpreted the desires of American voters so poorly. While many polls had shown large leads for President-elect Joe Biden in key swing states, the evident reality that played out over the next few days was that pollsters had once more failed to capture the level of support for President Donald Trump. Though this error has not proven as significant as 2016, given Biden’s victories in many key states, the results are much closer than predicted by most major election pollsters and forecasters. How did the polling industry once again misjudge Americans’ voting preferences by such a large margin?
Soul-searching in the industry began almost immediately. Pollsters and forecasters had worked hard to fix their reputation after 2016, but the polling errors this time around are even greater, leaving many looking for explanations. Among potential culprits are a shifting voter base, an overrepresentation of Biden supporters, the idea of “shy” Trump voters, and, perhaps most clearly, a failure to poll the full spectrum of Trump voters.
This week, The Factual used 17 highly credible articles from 15 outlets across the political spectrum to look at polling errors in the 2020 election to see where the polls went wrong and what this means for the future of the industry. This early post-mortem documents much of the discussion occurring in the media, but it should be noted that a full understanding of the issue will likely take months.
Please check your email for instructions to ensure that the newsletter arrives in your inbox tomorrow.
Just How Wrong Were the Polls?
While there were a number of calls that were rather accurate, like Georgia narrowly favoring Biden, these are seemingly the exception rather than the rule when looking at nationwide results. Florida, narrowly forecasted for Biden, ended up going to Trump by about 3 points. Ohio and Texas were thought to be in play for Biden, but went to Trump by roughly 8 and 6 points, respectively. Biden’s polling margins in the pivotal states of Wisconsin and Michigan put him ahead by about 8 points according to FiveThirtyEight, making a victory there seemingly assured before election day; in the AP’s most recent counts, Biden won both by 0.7 and 2.7 points, respectively. In Pennsylvania, arguably the most critical state in the election, the slow counting of mail-in ballots didn’t reveal a Biden victory for days — by about 40,000 votes, or 0.7%, a full 4.1 points off the pre-election polling averages, though these figures may change as the final votes are counted.*
Note: These numbers are projections based on the remaining votes to be counted.
Source: FiveThirtyEight
The same held true in down-ballot races, where Republicans thwarted Democrats’ efforts to win a Senate majority and even picked up some seats in the House, made possible by results like Susan Collins retaining her Senate seat in Maine against Sarah Gideon, an opponent who led every poll in the weeks running up to the election. In short, polling fell substantially short, locally and nationally.
Potential Explanations
Some, such as forecaster Nate Silver, have come to the defense of pollsters quickly to argue that such polling errors are fairly common. But for many, the polls are seen to have dropped the ball, sparking debate almost immediately about how the industry could have misrepresented voters preferences by such significant margins. Here are some of the most prominent explanations being discussed in the media and among pollsters.
Changes to the Voter Base: Record-setting turnout this election has served to complicate pollsters’ jobs because there is little precedence for such high turnout numbers, especially in recent history. This means assumptions about how these previously inactive voters will vote equates to a lot of guesswork. For example, high-turnout elections in recent history have tended to be associated with an advantage for Democrats, but the voting surge in this election seems to have benefited both sides, with record numbers of votes for both the president-elect and incumbent. At the same time, there are unforeseen shifts in voter preferences, such as Trump’s unexpectedly high favorability among Latinos. Since polls generally use past voting behavior to help project future voting behavior, they can be susceptible to missing significant or rapid shifts in voter preferences.
“In other words, Mr. Trump won several key states in the election in part because he changed the electorate, a possibility the pollsters had signaled but which the poll’s results didn’t fully reflect.” – Aaron Zitner, Wall Street Journal
Overrepresentation of Biden Voters: Another potential explanation comes from overrepresentation of Biden voters. As Nate Cohn posits for the New York Times, high levels of political engagement related to the Black Lives Matter movement or women’s rights likely drove more progressive voters to be politically active, resulting in more poll responses among that group. Similarly, the Covid-19 pandemic seems to have driven an uncharacteristically high response rate from Democratic voters, who were both likely to be abiding by stay-at-home guidance and to be seeking to voice political frustrations during an arguably mismanaged pandemic. And contrary to projections that Covid-19 would hurt Trump, Cohn notes that Trump “fared slightly better in places with high coronavirus cases than in places with lower coronavirus cases.”
“The theory is that the kind of people who answer polls are systematically different from the kind of people who refuse to answer polls — and that this has recently begun biasing the polls in a systematic way.” – Dylan Matthews, Vox
Source: New York Times
“Shy” Trump Voter Theory: This theory, a much-discussed explanation for polling errors in 2016, is basically that Trump voters will actively choose to tell pollsters that they favor Trump’s opponent in an effort to avoid being viewed as socially undesirable. Many pollsters would agree that their polls have failed to register support for Trump, but most cast doubt on the shy voter theory. Indeed, a report from the American Association of Public Opinion Research found no evidence of such voters in 2016, nor do polls in 2020. However, this hasn’t stopped proponents of this theory from putting it forward. The Trafalgar Group, a pollster which used this theory to project 2016 rather effectively, also made solid predictions this time around, including Wisconsin (+1 for Biden) and Florida (+2 for Trump), but they notably also projected Georgia going to Trump by 5 points, where Biden leads by just over 10,000 votes. This variability suggests that the theory about shy voters is probably not the explanation for inaccurate polling, or at least that there are other overriding factors at play beyond specific voters hiding their preferences, but the theory will surely receive further scrutiny.
Failure to Poll Trump Voters: Perhaps the most likely explanation for the polling errors is that the polls simply failed to create truly representative samples. The thinking here goes that Trump supporters are highly skeptical of authority — be it the media, science, or academia — and therefore are less likely to participate in polls generally. As part of this, moderate Trump supporters may be both more likely to participate in polls and more likely to defect politically given their more moderate viewpoints. As a result, pollsters found that they significantly overpredicted the number of former Trump voters that would support Biden and underpredicted the number of voters intending to vote for Trump. This is something that pollsters anticipate and try to correct for, for example, by adjusting for the level of education among respondents or by engaging with other forms of polling beyond traditional landline surveys (especially given the declining use of landline phones), such as in person, online, or through text messaging, but it seems that such measures this year were entirely insufficient.
“If it turns out that support for right-wing populism is independently correlated with survey response rates, then it could be impossible for pollsters to get an accurate count no matter how they weight the demographics.” – Gilad Edelman, Wired
Please check your email for instructions to ensure that the newsletter arrives in your inbox tomorrow.
The Future of Polling
Polling got a lot of this election wrong, and led to misdirected journalism as a consequence, but that doesn’t mean it’s time to throw in the towel on the whole industry. As pollsters adamantly argue, they are not meant to be predictors, merely a snapshot of attitudes that are subject to the complexity of the entire voting populace and shifting preferences. While they were off the mark this election cycle, they are essential to understanding the thoughts and desires of the people and did, to some degree, give us a sense of which direction the election was likely to trend.
“The inability to rely on polling also undermines a range of generic political choices. Many voters decide which candidate they favor in primary elections based in part on rational calculations about who is most likely to win a general election. Policy makers choose what ideas they support or oppose based on what they think the public supports. If poll data are not reliable, though, the candidates that triumph in primaries are more likely to be the loudest and best-funded, and the policies politicians back might run contrary to the wishes of the voters who elected them.” – David A. Graham, The Atlantic
Polls are also critical outside of elections, where favorability for and against specific policy choices indicates to politicians where the realm of possibility lies for lawmaking. For example, voters this election indicated their strongest priorities are the economy, racial inequality, and the Covid-19 pandemic. Absent of these signals, policymakers would be left flying blind. Flying with a rough estimate of where to go is better than having no sense of direction at all. It’s up to pollsters and forecasters to improve their methods and make that picture clearer once more.
* These margins of victory are preliminary until recounts are completed and any allegations of voter fraud, and the impact they had, are resolved. However, the final tallies are not expected to change materially, so this analysis uses the existing AP numbers.
Appendix
This appendix shows each of the articles used to inform the findings of this article, as well as how the articles scored according to The Factual’s credibility algorithm. To learn more, read our How It Works page.