Survey researchers have faced a host of challenges this cycle with technological shifts and changes in voter behavior making it harder to generate representative samples.
As part of our 2024 Election Briefing Series, C&E spoke to Betsy App, senior pollster at Change Research, and Brent Buchanan, president and founder of Cygnal, about the current state of survey research and how their firms are attacking the ongoing methodological challenges.
Difficulty reaching voters certainly isn’t a new problem for pollsters, but both App and Buchanan agree it’s one that constantly requires new adjustments on the part of researchers.
“I actually think it’s a really exciting time to be a pollster in this world that is morphing so quickly with so many new technologies,” said App. “But meeting voters where they’re at and being able to access them and recruit them into surveys in a representative way is really the key to getting samples that you don’t have to weight a ton on the backend.”
More traditional methods, like landlines, are becoming largely obsolete. And reaching voters via mobile has its own set of challenges. So in a world where the electorate has really fractured in the last decade when it comes to reach, there’s no longer “one right way to reach voters,” said App.
Sure, all voters are online, but “they’re not all online in the same places. And so that has for us has been the challenge — trying to find voters where they actually exist online.”
One major way Change Research recruits survey respondents into its polls is via Meta platforms: “Your grandma is on Facebook. Your niece might be on Facebook, too, but she’s only there for Facebook marketplace. She’s otherwise on Snapchat or WhatsApp. Younger men are on YouTube. And so this is the constant challenge.”
Cygnal’s Buchanan said his firm has found one of the most effective ways to get the sample right is utilizing text to web: “Young people, older people, people of color, evangelical whites — name any demographic group that somebody might say is challenging to get representative samples of, and you can do that really well with text to web.”
Another big point of emphasis from Buchanan: handling the sampling in house. “If your pollster is outsourcing the texting or the calling or the advertising or the online panel recruitment or whatever it may be, you then have so little control over what is the most important part of the polling process — getting a representative sample” he said. “We’ve got great partners who are on the outside doing those things for us, but we’re controlling the entire process internally because we believe that that’s the most important and hardest part of polling now.”
One of the big lessons following the 2016 election, noted App, was not accounting for some key cultural differences among voters. “And so now one proxy for how voters are different from one another would be whether or not they have a four year college degree,” said App. “Now we ask those questions in our polls. We ask education level, and then we are able to match our samples to what the expectation is for the likely electorate on that one dimension.”
Fast forward to 2024 and something Change has found critical in this cycle is ensuring surveys have the right breakdown of young men and young women voters under the age of 35.
“They’re behaving pretty differently than they have in the past,” App emphasized. “We see women under the age of 35 being very progressive … whereas we have data suggesting that men under the age of 35 are becoming a bit more conservative. So making sure that we have that right mix within our age buckets or the right mix of college educated women and college educated men. You need to look at all of these in-the-weeds things to make sure that my sample is looking like the likely electorate.”
