The first shots have been fired in the 2020 polling wars. From now until November, expect a strong focus on the seeming minutiae of how polling samples are constructed, what methodology is employed for the data collection, and whether there are better ways of measuring public opinion in a digital and mobile age.
In anticipation of these hostilities, let’s review four areas likely to be discussed and hotly debated during the remainder of the 2020 election cycle.
Who is polled
What’s at the heart of the confusion about differing polling results is this fundamental question: Who was polled? Much of the polling we’re presented with is actually comprised of three entirely different audience segments – the American public, registered voters, and likely voters – those who are registered and report they’re likely to vote.
Obviously, for elections, only the latter category matters, but there are many public polls by reputable organizations conducted among registered voters – many of whom never actually vote on Election Day. This discrepancy was recently called out by the Trump campaign.
Demographic composition
Pollsters begin their work before they start polling by studying the voter file, voter turnout data, and where relevant, state or national exit polls.
They do this to ensure that the demographic composition of the survey is representatively sampling voters. For example, a national poll that mainly samples women given the current gender gap (“gender canyon”) is obviously inaccurate. But the question is, what’s the right number?
Should women account for 50 percent of the likely voters, 55 percent, or even higher? Examining the voter file gives us the answer. And indeed in some of the Democratic primary elections that our firm is working on now, women may actually account for more than 65 percent of primary voters if this coming election is consistent with prior primary elections. But again, all of this can be verified from the voter file.
What’s harder to measure are variables that generally don’t appear on the voter file. Education is a perfect example of this, and where there’s a pronounced and growing partisan divide based on whether you have a college education or not.
That data doesn’t exist on a voter file and thus cannot be verified so there will be a variation in how many non-college graduates there are in different polls that we see reported on.
One other variable where there’s frequent disagreement about is sampling for political party. Many voters originally register with one political party, but later in life and based on how the political winds are blowing change their “identification.” In fact, they say they’re affiliated with one party when you speak with them, even though they haven’t changed their party registration with their local Boards of Election.
As an example, a voter may be registered with their local Board of Elections as a Republican, but may consider themselves in the Age of Trump a political “Independent”.
So a poll may contain the proper proportion of registered Republicans, even as a certain portion of them may identify with another party. Indeed, Gallup has shown that the number who consider themselves Republican is down 8 points from February 2020 through early June.
And to make matters even more confusing, some polls ask only for a voter’s party registration, while others ask for their party identification or affiliation.
So stay tuned. The party and education composition will be at the center of many arguments and disagreements among political opponents — even within non-partisan polling circles.
Shy Trump Voters
The third issue to watch for is the so-called Shy Trump vote. The thinking goes that as the country becomes increasingly polarized, fewer want to talk politics with strangers, co-workers, or even closer family members.
And specifically, as popular opinion towards the president turns even more negative, fewer Trump voters will feel comfortable sharing publicly or to a pollster that they’re voting for Trump – even as millions confidently strut around wearing MAGA swag.
Increasingly as the president’s national numbers decline, in many social circles across the country it’s less and less socially acceptable to express any degree of enthusiasm or admiration for Trump – and obviously that makes polling much, much more difficult.
While the Bradley effect is the most commonly cited example of this phenomenon, there are other examples of this as well – and not just focused on race. In fact, men who were polled by female pollsters during the Monica Lewinsky scandal were more likely to say Clinton had done something wrong than when male pollsters called them and they were asked the same questions.
Methodological Changes
So what are we to do? What changes are needed to make polls more accurate and unbiased? This is complicated, but here are some of the easier fixes to help mitigate the current political polarization.
Automated IVR polls where voters press buttons and don’t actually talk with a live phone operator increases the confidentiality of the responses given and significantly reduces the number of socially desirable answers that pollsters collect. More campaigns need to use services like this. And this technique is also fast and very affordable.
Mobile surveys are another answer. As the Wall Street Journal recently reported, there’s new research from the Journal of Marketing about how surveys conducted exclusively on mobile devices gather more truthful answers even than online surveys. Assuming the online surveys aren’t conducted exclusively with young voters, this approach can be useful in reducing social bias.
Whether you’re a pollster or not, get ready to get in the weeds of how measuring public opinion is done and how it can be improved upon during this upcoming election season.
Bradley Honan is the CEO & President of the Honan Strategy Group, a Democratic polling and data analytics company.