Despite some glaring discrepancies between the initial vote counts in some states and the final public poll numbers, survey researchers initially wanted to wait to assess their 2020 performance. “Patience is necessary,” the American Association of Public Opinion Researchers said back in November.
Now, a group of Democratic pollsters say they’ve got enough data to make a hard conclusion: “[O]ur industry saw major errors and failed to live up to our own expectations.”
With just 43,000 votes across Wisconsin, Georgia, and Arizona separating Joe Biden and Donald Trump in November, the left’s top pollsters’ mistakes could have been more costly.
“The tremendous impact of such a small number of votes underscores the importance of continually questioning our assumptions and working to improve our methods to produce more accurate, reliable data,” said the leadership of ALG Research, Garin-Hart-Yang Research Group, GBAO Strategies, Global Strategy Group and Normington Petts.
Diverging from the post-mortem digital panel format, the top five Democratic polling shops published a memo Tuesday — in the works since mid-November — that detailed how their research got turnout and their measures of voters’ sentiments wrong and what to do about it. In particular, the pollsters singled out their survey work in “more Republican areas.” That’s where their “data were often wrong, sometimes egregiously so.”
When it comes to measurement of voter attitudes, the pollsters found “there is something systematically different about the people we reached, and the people we did not.
“This problem appears to have been amplified when Trump was on the ballot, and it is these particular voters who Trump activated that did not participate in polls.”
Republican voter non-response or errors in the number of conservative voters in a sample size from a Democratic pollster is something other researchers have already addressed.
During a panel hosted by TargetSmart last December, Jane Rayburn, a pollster at EMC Research, said pollsters may be getting the right number of Republicans, but not the right type of Republicans.
“We want to make sure that our sampling is inclusive,” said Rayburn.
Ben Lazarus, who leads polling at TargetSmart, put it this way: “There’s a very decent chance that these folks just won’t take a survey.”
In fact, ALG Research’s John Anzalone, the Biden campaign’s 2020 pollster, has also publicly acknowledged the problems that researchers had during pandemic polling. During a digital panel in February hosted by the University of Chicago Institute of Politics and the Cook Political Report, he said that Democrats were getting oversampled because they were at home during the pandemic.
“It actually turned out that you had a bigger miss in public polling and private polling because of COVID,” he said.
When it came to turnout last cycle, the five pollsters said this week they failed to account for low-propensity voters’ support for President Trump. Among those voters, “the Republican share of the electorate exceeded expectations at four times the rate of the Democratic share. This turnout error meant, at least in some places, we again underestimated relative turnout among rural and white non-college voters, who are overrepresented among low propensity Republicans,” the pollsters found.
The authors’ turnout modeling, which relies on “historical voting records for millions of individual voters,” has its critics. During the February post-mortem event, J. Ann Selzer, of Iowa-based, Selzer & Company, questioned why pollsters were even trying to model the electorate’s makeup in their surveys.
“Why not let your data show you the size and the shape of the coming electorate? I’m at a little bit of a loss for the details of how one would go about deciding, based on science, what a future electorate might look like,” Selzer said. “If you’re looking backward, you’re going to miss a freight train that might be coming right at you.”
Many of the solutions offered in the memo were, in fact, a look backwards. They included door-to-door polling to overcome not being able to reach voters on their phones, using paid incentives to recruit subjects for research and other “multi-modal research tools.”
Still, the pollsters warned the industry risks over-correcting for “an error that may be geared to one man who will, hopefully, never be on the ballot again.”