AAPOR Report: 2016 Polls Failed Because Of Weighting And Late-Deciders

Late-deciding voters and the less educated swath of the electorate attracted by Donald Trump’s unique candidacy made it difficult for pollsters to track his support last cycle, a group of survey researchers said Thursday. 

“There was a strong relationship between education and vote preference in 2016 and that was not addressed in many polls at the state level,” said Courtney Kennedy, director of survey research at the Pew Research Center. “Voters with more education were [Hillary] Clinton supporters.”

At the national level, pollsters “correct for an education imbalance in our weighting,” she said. That didn’t happen at the state level, where polls  failed to track Trump’s surge in support in the closing stages of the campaign. 

In the wake of last cycle’s polling misses, the American Association for Public Opinion Research (AAPOR), an industry trade group, assembled a committee of 13 researchers to dissect the methodology surveyors used. 

That committee, which released its findings Thursday, found that pollsters failed to account for real, late change in voter settlement while at the same time there was a deficiency in how polls were weighted. 

“Pollsters who sample from the voter file could adjust for education using some other source, such as the Current Population Survey, but most of them choose not to,” the report states. 

Kennedy, who was among the committee’s researchers present at an AAPOR press conference Thursday in D.C., added: “I think that the unusual contours of Trump as an individual and his campaign did make it harder to poll this year. We saw higher level of undecideds. It wasn’t off the charts, but it was higher than you saw in 2012 and 2008. That just introduced that much more uncertainty into the polling.”

Trump was an historically unpopular candidate, as was Clinton, which confounded voters and helped make polls taken in the earlier stages of the campaign less accurate, explained Kennedy.  “A sizable share of voters waited until the last week to decide who to vote for.”

At an event in New York on Tuesday, Clinton said she would have defeated Trump had the election been held on Oct. 27 — the day before the release of FBI Director James Comey’s infamous letter. 

But Kennedy said there was no evidence to suggest that the letter influenced Clinton’s level of support. “The decline in Clinton’s lead was definitely started before Oct. 28. The question is whether the letter made the decline more severe,” Kennedy said. “It’s not knowable with the data available to us.”  

Clinton’s drop in support didn’t happen nationally, but instead was concentrated in states in the upper midwest like Michigan and Wisconsin, as well as Florida, where there were 10-20 point swings among some voters shifting to Trump, she said.

Those swings tracked with the attention received from the Trump campaign to voters those states. The Republican campaigned aggressively in Michigan and Wisconsin in the closing stages of the race. Clinton famously didn’t visit Wisconsin after winning the Democratic nod and added stops in Michigan only in the final days of the race. 

Meanwhile, inconsistent voters, a group that usually splits between the two major parties, went decidedly for Trump. “That means that either they changed their minds after we spoke with them on the phone, or they shaded their true feelings when we spoke with them on the phone,” said Kennedy, who dismissed the idea of the shy Trump voter and differential non-response as being influential factors. 

“In the survey data, we didn’t see that pattern,” she said. 

The report concluded that national polls were largely accurate in 2016, as were state-level polls done during the primaries. But that accuracy may have fueled a false sense of confidence, according to Mark Blumenthal, SurveyMonkey’s head of election polling.

“The bottom line is that the polling was just wrong enough nationally and at the state level that people got the wrong impression of what would happen,” said Blumenthal, a member of the AAPOR committee. 

“What do you do about it next time?,” he asked.  “We shouldn’t be over confident.”