Are the polls missing Republican voters?
By Nate Cohn
With polls showing Joe Biden holding a commanding lead, one question keeps popping up: Are these polls missing Trump voters?
Self-identified Democrats outnumber Republicans in most surveys, sometimes by a wide margin. This might simply mean there are more Democrats than Republicans. But to critics, the partisan makeup of most public polls is self-evidently out of step with a closely divided country.
There are many reasons the polls might ultimately be wrong in November, as many state polls were four years ago, but there’s no serious evidence that the polls are systematically missing Republican voters. There’s more evidence to the contrary — that the polls represent Republicans just fine, and President Donald Trump still trails.
Many insist otherwise. Trump campaign pollster John McLaughlin, for instance, cited the 2016 exit poll results to argue that public polls understate the number of Republicans by a clear margin. In 2016, Democrats outnumbered Republicans by 3 percentage points in the exit poll, and Republicans made up 33% of the electorate. By contrast, the CNN/SSRS poll that McLaughlin criticized found that Democrats outnumbered Republicans by 7 points among the adult population, with Republicans at just 25% of the sample.
But few serious pollsters rely on the exit polls to determine the demographic or political makeup of their samples, with good reason. When checked against census or voter file data, the exit polls are demonstrably inaccurate on many variables, like age and education. The errors are so large that it’s hard to trust their results on other measures. Even if the exit polls were more accurate on these demographics, they would still be just a snapshot of the electorate four years ago. And the partisan makeup of the electorate in the exit poll of a given election almost never matches that of the exit poll in the prior election.
In the absence of an authoritative benchmark for party identification, many public pollsters simply ignore the problem, doing nothing to ensure that the sample includes a certain number of self-identified Democrats or Republicans. It’s not that pollsters at various news media organizations are trying to bring about a significant partisan advantage for the Democrats, as their detractors claim. It’s just what their results show.
Of course, those results could still be wrong. Pollsters can ignore partisanship only on the assumption neither Democrats nor Republicans are likelier to respond to telephone surveys, controlling for their demographic characteristics. If it turned out that Democrats were far likelier to respond to telephone surveys than Republicans, the public polls could be systematically biased — and the critics would be vindicated at the ballot box.
Ultimately, there’s no way to be absolutely sure that the polls include the right number of self-identified Democrats or Republicans. Moreover, party identification is not a fixed characteristic.
Clues in the voter files
But there are other ways to consider the effect of partisanship on polling, such as party registration. Unlike party identification — how respondents respond to a question about their preferred party, which could potentially flip to independent and back with every news alert — party registration is a more or less fixed characteristic, and so is the partisan composition of a state. We know that registered Democrats outnumber registered Republicans in Florida by a few points, even if we don’t know whether those same voters would identify as Democrats or Republicans today.
Pollsters can take advantage of this if they use voter registration files, a large data set of every registered voter. Most private pollsters and a handful of high-quality public ones use voter registration files as the foundation of their polling, and these pollsters can use a respondent’s party registration to help ensure that their sample is representative. They can make sure that 37% of their respondents in Florida, for instance, are registered Democrats. In states where voters do not have the opportunity to register with a party, a pollster can use a voter’s primary participation history in a similar way.
If polls using partisan characteristics from voter registration files showed a fundamentally different race, this could be a sign that the other polls were biased on partisanship. But the recent surveys that are weighted by party registration or primary vote history offer nearly the same picture as polls that are not. Arguably, they offer a picture even worse for Republicans.
The Monmouth University poll of Pennsylvania this month, which showed Biden up by 13 points among registered voters, was arguably Trump’s worst poll of the cycle, considering the quality of the pollsters. The New York Times/Siena College polls last month showed Biden leading by 9 points across the battleground states, and 14 points nationwide. A CBS/YouGov battleground tracker, which uses online polling data matched to voter file records, found Biden up 9 points among likely voters nationwide.
Voter-file-based data lends credibility to polls showing a lopsided Democratic advantage in other ways. If all the various measures of partisanship — say, Republicans plus-two in Arizona or Democrats plus-two in Florida — are added together across the 94% of the nation with a measure of partisanship, Democrats outnumber Republicans by 6 points, 36% to 30%. This doesn’t prove that more voters identify as Democrats today, but the Democratic advantage in registration and recent primary participation nonetheless offers evidence consistent with the basic proposition that Democrats outnumber Republicans, and probably significantly.
Using this data, we found that self-identified Democrats outnumbered Republicans by 8 percentage points in the Times/Siena national poll (the unrounded figures are Democrats 34.5% to 26.4% for Republicans). Among those characterized as Democrats based on party registration or primary vote history, 69% identified as Democrats in the poll; similarly, 65% of those characterized as Republicans identified as Republicans. Those without a party, disproportionately young and nonwhite, split toward Democrats by a narrow margin of 24% to 19%. At the same time, 6% of registered Democrats identified as Republicans, while just 3% of registered Republicans now identified as Democrats.
The irrelevance of partisanship
The polls that rely on voter files can help in another way: The pollster can tell whether there was a difference between Republicans and Democrats in their willingness to complete interviews. Pollsters using the voter files know the party registration of the intended respondent before they place the call. As a result, it’s easy to tell whether members of one party or another were more or less likely to respond to a survey.
If registered Democrats were likelier to respond than registered Republicans, it would undermine a core assumption of most public pollsters. It would also raise the possibility that bias contaminates the survey in unobservable ways: For instance, if Democrats were far more likely to respond than Republicans, one would have to assume that Democratic-leaning independents were also likelier to respond compared with Republican-leaning independents.
But perhaps surprisingly, registered Republicans were actually more likely than registered Democrats to respond to the Times/Siena survey. Overall, telephone calls to registered Republicans or those who participated in a recent Republican primary were about 12% likelier to yield a completed interview than calls to Democrats were. This seemingly noteworthy difference can be explained by well-known demographic biases in polling: Older, rural and white voters are likelier than young, urban and nonwhite voters to respond to surveys. After these factors were controlled for, Republicans were no likelier than Democrats to respond to the survey. And if Republicans are just as likely to respond to surveys as Democrats, there’s little reason to believe that they’re vastly underrepresented in political surveys.
And there are reasons to doubt that Trump’s voters are being understated. In the last Times/Siena polls of six battleground states, for instance, respondents who voted in the 2016 election said they voted for Trump over Hillary Clinton by a margin of 2.5 points, which was even more than Trump’s actual margin of victory.
Trump’s problem wasn’t the number of people who said they voted for him last time: It was that only 86% of those who said they voted for him last time said they would do so again. Perhaps there’s a way the poll could have the right number of voters who said they voted for Trump last time, but not this time. It would have to be an awfully specific form of polling error.