John Anzalone is a partner at Anzalone Liszt Grove Research, a Democratic polling firm. Glen Bolger is a partner at the Republican polling firm Public Opinion Strategies.
If the 2012 cycle revealed anything about the current state of polling, it’s that, overall, it is still incredibly accurate.
Based on Pollster.com’s averages, public polls predicted the correct winner in 11 of the 12 presidential battleground states. And in the 12th—Florida—they were less than one point of, showing a tie in a state that President Obama won by nine-tenths of a point.
Not only were public polls extremely effective at picking the winners, they were remarkably good at predicting the margin of victory as well. Once again, in 11 of 12 states, they projected a margin of victory within three points of the actual result. The one exception in this case was Colorado, where their margin was four points of.
But while polling remains very accurate, because of the increasing difficulty of reaching and keeping people on the phone, getting it right is becoming harder and harder to do. In 2012, pollsters had to dial an average of 10 numbers to get one complete interview. In 1997, they only had to dial three. This plunging response rate makes it harder and more expensive for pollsters to get the representative sample they need.
So given the challenges posed by falling response rates, what does it take for a pollster to get it right these days? The industry must continue adapting to change.
Cellphone Dialing
According to the Center for Disease Control and Prevention, by the end of 2012, 36.5 percent of Americans lived in a cellphone-only household, with that number increasing by 2-3 percent every six months. That means by the time voters cast their ballots for the 2014 elections, over 40 percent of Americans won’t have access to a landline.
At this point, dialing cellphones should be a must for any pollster, especially when conducting polls in areas with a significant Latino population given that over 50 percent of Hispanic adults are cellphone-only. But just dialing a few isn’t enough. We believe that 20 percent should be the minimum share of cellphone interviews in any poll, and the goal should be higher, especially in areas with greater diversity or more young people.
As is typically the case, the main obstacle to doing cellphone interviews is cost. Because federal law requires that cellphone numbers be dialed by hand rather than by an auto-dialer, calling cellphones makes polling more expensive. So instead of incurring the higher costs of cellphone dialing, some pollsters have instead opted to cut corners by increasing the number of young people and minorities in their polls without dialing cellphones.
But weighting up these groups will not solve the problem of not conducting cellphone dialing, as those who live in cellphone-only households hold distinctly different attitudes from those who do not. For this reason, not dialing cellphones can cause a campaign to make serious strategic miscalculations.
In Florida, President Obama’s advantage over Romney among 18-34-year-olds who were polled on cellphones was 28 points higher than his lead among 18-34-year-olds who were polled on landlines. A pollster who did not include cellphones in the sample would therefore have identified this age group as a significant problem area for the president, while those who did would recognize it was one of his most important groups to turn out.
Bilingual Dialing
Though cellphone dialing gets more attention, using bilingual dialers to call Hispanic households is another essential component of the modern pollster’s toolkit, and one that is too often overlooked. According to the Census, 35 million Latinos now speak Spanish at home, up from 25 million in 2000. As a result, pollsters who fail to use a sufficient number of bilingual dialers to capture this group will get a very inaccurate picture of where the Hispanic electorate stands. That’s because, like cellphone and landline phone users, the political attitudes of Spanish-speaking Hispanics differ significantly from the views of their English speaking brethren.
For instance, in Nevada in 2012, about a quarter of Hispanic poll respondents opted to take surveys in Spanish and they were far more supportive of Obama than the Nevada Latinos who took polls in English. While Obama’s support among English-speaking Latinos still exceeded 60 percent, he did a remarkable 24 points better among those who preferred to speak in Spanish.
This divide, and the inconsistent use of bilingual dialers by pollsters, may help explain why the public polls in states with large Hispanic populations like Colorado and Nevada tended to be a little further of the actual results than polls in other states.
Just like cellphones, those pollsters who do use bilingual dialing often cut corners when doing it, such as calling back Spanish respondents with a Spanish-speaking interviewer rather than having a sufficient number of bilingual dialers on hand. Having to schedule callbacks with Spanish speakers is a good way to ensure that your poll will under represent these voters.
Voting Universe
While getting the dialing end of polling right is critical, it won’t do you much good if you don’t have an accurate view of what the electorate will look like before dialing starts. Of course there is no way for a pollster to know exactly what a future electorate is going to look like, but the accurate ones base their projections on data from recent, comparable elections, and then make small adjustments based on census data and other outside factors.