Pollsters must abandon their traditional methodology or risk survey inaccuracies like they had in 2012.
That’s according to Ole Forsberg and Mark Payton, professors in the department of statistics at Oklahoma State University who published a study Monday that explored how pre-Election Day battleground state polling in the 2012 cycle wound up favoring Mitt Romney over President Obama. The culprit, they said in their study published on the website of Statistics and Public Policy, is cellphones.
“The cellphone-only households don’t poll the same as the landlines, in general,” Payton told C&E. “If [pollsters] continue to do business like they did in 2012, I expect they’ll have some bias [this cycle].”
While state polling tended to be accurate in 2004 and 2008, last presidential cycle many public surveys of battleground states had Romney winning. A post-election analysis by Nate Silver noted pollsters “that called only landlines or took other methodological shortcuts, performed poorly and showed a more Republican-leaning electorate than the one that actually turned out.”
The professors followed up on Silver’s reporting and concluded that the 2012 inaccuracies stemmed from the growth of cellphone-only (CPO) households. In the 2012 cycle, 40 percent of all households had cell service but no landline — that was up from roughly 8 percent in 2006. That number has since risen to 58.8 percent of households, according to one industry estimate.
“We are completely ignoring or partially ignoring a population that votes differently from the segment of the population we are reaching and that’s getting worse as times goes by,” said Forsberg.
Political researchers say they’ve tried increasing the number of interviews being conducted via cells in their surveys, but blame FCC restrictions and higher costs for the underrepresentation of CPO households in polling.
Those households lean Democratic, for a variety of reasons, and their underrepresentation in survey respondents is skewing the results. Moreover, stratified sampling may not be enough to cor-rect the problem of the under-representation of CPO households.
“In addition to this inverse probability weighting, polling firms may perform additional weighting to adjust for other known factors, such as known biases in the sampling method,” the professors wrote. “The problem is that these estimates are only unbiased when the weights used accurately reflect the differences between the sampled population and the population they are trying to describe. … Thus, while those polling firms not using weights in their estimation process are at risk of producing biased estimates in the presence of cellphone-only households, those polling firms who use incorrect weights are also in danger.”
The duo recommended sampling methodology be adjusted “to reflect the new realities and account for a segment of the CPO voting population that tends to vote for Democratic candidates but currently is not included in the sampled population.”
Still, that could be easier said than done if the proposed FCC rules adjustment goes through. Part of the existing proposal penalizes surveyors for calling cellphones that have been reassigned to new users, which could further inflate the cost of polling CPO households. The FCC will vote on those new rules Thursday.