Traditional pollsters have recently been questioned by observers, myself included, for making erroneous election predictions. One of the reasons for their faulty numbers is that they don’t use the voter file to predict vote probabilities, which is the best way to determine who votes and who'll win an election.
But predicting election outcomes is only the first step. After all, they only tell you who’s probably going to win and how close it should be. In a word, the “what.”
What we’re all really interested in, though, is the “how.”
For instance, how to win a close election and maximize the margin of victory. Modeling “observational” data can’t tell you how to change the outcome, it can only tell you what the outcome will be. Modeling “experimental” data, on the other hand, can tell you how to change the outcome.
A classic example of how to collect and use this experimental data to alter an election outcome is the work my firm did last year for a pro-life effort called the Texas Gubernatorial Project, run by Joe Arlinghaus.
Greg Abbott was never in any danger of losing to Wendy Davis in the 2014 Texas gubernatorial election, but Democratic strategists and Progressives had already marked the state for serious effort to mainstream an aggressive pro-choice agenda.
Davis’s prominent pro-choice stance gave us a fortuitous backdrop for testing an aggressive wedge-issue campaign emphasizing her views on abortion. This was a great opportunity for us to cut the legs out from under the ongoing “turn Texas blue” effort.
Do the ads work?
Does pro-life messaging work as a wedge issue? To find out, we conducted a randomized-controlled experiment testing three different pro-life radio ads attacking Davis.
I want to belabor this point — we didn’t ask voters what they thought of the ads, which is what standard message testing does. People are terrible at introspection and self-prediction. Rather, we observed how the ad influenced their likelihood of supporting Abbott or Davis compared to the placebo-control group. We ran a randomized-controlled experiment, just like a blind clinical drug trial.
We found something remarkable. The best targets for pro-life messaging were Democratic-leaning women, young voters and Hispanic voters. Exposure to just one pro-life video ad shifted Democratic-leaning women by 10 points away from Davis and toward Abbott.
Moreover, voters aged 18 to 34 shifted about 8 points, and Hispanic voters shifted about 13 net points from Davis to Abbott. Those were staggering results for these demographic groups on this issue. Of course, it wasn’t all positive: these same ads caused a backlash among white men.
Using abortion as a wedge issue worked spectacularly—and counterintuitively—well with some voters, a finding that confirmed by similar experiments in the lab and field which we’ve conducted in five states, and nationally across a range of elections.
Experiment results spurred action
In Texas, these results were then used to guide the development and deployment of Hispanic-language radio ads on stations across the state in the final weeks of the campaign, as well as online ads targeting Hispanic voters. The effort helped push Abbott’s share of the Hispanic vote up to 44 percent. That beat outgoing Gov. Rick Perry’s previous haul of 40 percent.
The data from this experiment was also used to model the expected impact of the ads on the vote probabilities for each of the approximately 13 million voters in Texas.
In other words, with this information in hand, we could identify the best targets for political persuasion across all of Texas based on the impact of the ads on the survey response and their demographic, consumer and past voting data. We didn’t model what voters thought of the ads. We modeled how the ads would shift their probability of support for Davis or Abbott.
Now that’s seeing into the future: knowing accurately and in advance how an ad will affect a voter.
Adam B. Schaeffer, Ph.D., is the director of research and co-founder of Evolving Strategies, a data and analytics firm dedicated to understanding human behavior through the creative application of randomized-controlled experiments.