Virginia’s 2013 gubernatorial election was much closer than anyone, with the possible exception of Terry McAuliffe’s data team, expected. Rather than the 7-point drubbing the poll averages suggested, it came down to about 55,000 votes out of more than 2 million cast, a 2.5 percent margin.
This relatively close call has given rise to all manner of what if questions: What if Ken Cuccinelli had more money? What if Governor McDonnell hadn’t been embroiled in scandal? What if the shutdown didn’t happen?
But none of these questions get to the core problem with the Cuccinelli campaign. Indeed, the core problem with almost every Republican campaign to date: the absence of randomized-controlled experiments.
True experiments produce dynamic data, which tell us how to move an electorate and what to do to win an election. Experiments have been at the core of the progressive revolution in modeling, targeting, persuasion and turnout over the past decade.
In fact, the McAuliffe campaign plan—the messaging, targeting, and tactics—was built around extensive experimental findings. And yet it remains exceedingly rare to find anyone on the right who is aware of applied political experiments, let alone someone who understands and utilizes them.
In Virginia, many have argued the Obamacare debacle helped close the gap in the final weeks of the gubernatorial election, and that with more time and money to get Cuccinelli’s message out on that front, he might have won. But this is pure speculation.
Correlation doesn’t prove causation, and what we have here is a correlation between the media’s focus on problems with Obamcare and an unexpectedly close result in the gubernatorial election. Absent experimental data, we can’t say whether these events impacted the race or whether messaging tied to it would have shifted more votes toward Cuccinelli.
Evolving Strategies and the Middle Resolution PAC conducted experimental research that suggests an aggressive attack on McAuliffe for supporting ObamaCare was ineffective at best and counter-productive at worst. An attack on McAuliffe’s business record possibly helped, but was anemic.
What moved the voters most was an attack on McAuliffe’s positions on abortion; a single phone message emphasizing McAuliffe’s support for unrestricted, late-term, and taxpayer-funded abortions shifted support a net 13 to 15 points away from McAuliffe and toward Cuccinelli. The cost per vote here was a remarkably cheap $0.50 per additional vote, and even less expensive still when targeting the most persuadable segment of the electorate.
A topic declared radioactive by nearly everyone, locked away in secure storage behind a blazing Hazmat warning by the Cuccinelli campaign, appears to have been a powerful weapon for the Republican ticket that could have substantially closed the gap, and possibly even won Cuccinelli the election.
Here is the fundamental lesson: We do not know what works until we test it, repeatedly, using experiments. Randomized-controlled experiments allow us to block out all the other noise and pinpoint precisely how a message or tactic changes voter behavior. This is key: rigorous experiments randomly assign voters to receive a treatment, or to a control group that receives nothing, or a placebo.
Quasi-experimental approaches, such as match-control groups where a set of voters or precincts is matched on observable characteristics, fall far short of the certainty and precision we need. But a true experiment, with large enough numbers, allows us to statistically compare the outcomes in the treatment to those in the control group, and any difference between the two can be confidently attributed to the treatment alone.
Democrats and the broader collection of progressive organizations attack issues and elections as problems to understand and then solve. They conduct rigorous, aggressive research on the composition and disposition of the electorate well in advance of an election. The right still largely relies on gut instinct and guru-ism, where the argument goes to the most forceful personality and best storyteller. This is changing, but too slowly.
Earlier this year, a national grassroots organization, Evolving Strategies, and Middle Resolution worked together to conduct experiments in three Republican primary elections for the Virginia House of Delegates. Each race featured 20-year-plus incumbents with deep pockets and the full backing of the governor given their support of his controversial transportation bill and tax increase.
In each race, we backed a challenger with strong grassroots support, but fewer resources and little or no political experience. We conducted experiments testing the impact of yard signs, GOTV text messages, and GOTV survey scripts guided by voter psychology experiments and innovative hypotheses. Two out of the three incumbents we targeted lost their seats.
This year, Evolving Strategies has tested mail, texts, GOTV surveys, radio ads, and robocalls for persuasion and GOTV in New Jersey, Massachusetts, Colorado, and Minnesota. Each experiment adds to our base of knowledge and provides a basis for testing even more promising possibilities for turning out or persuading voters more efficiently.
Our winning survey script, for instance, boosted turnout for our candidates by 12 points over the control group, and we tested additional variations during the Virginia gubernatorial race. Our cold GOTV text messaging impacts range from -15 to +15 points depending on the message and the targeted demographic. We have found massive differences in the impact that a message has on particular kinds of voters.
In the absence of experiments, we can’t know whether we’re helping or hurting our cause. With experimental testing, however, we discover not only what’s effective, but what’s most efficient and with what kinds of voters. As we learn and test more, we reap returns from a ratchet effect, discarding what fails and improving on what succeeds.
To loosely paraphrase pioneering scientist Louis Pasteur, the left relies on experiments to determine strategy and tactics, and the right relies on speeches. The right largely remains mired in a pre-scientific age of just-so stories and persuasive conjecture, using theories that are the electoral equivalent of the four humors and tactics the equivalent of leeches. Things are beginning to change on the right, but not nearly as quickly or broadly as is necessary to start winning in races like Virginia in 2013.
That’s not to say we won’t win a lot of races next year. Campaign strategy and tactics can’t make an unpopular president or healthcare debacle magically disappear. After all, some patients sick with smallpox survived in spite of being bled with leeches, but far fewer than those in later years who would be vaccinated against the disease. We will lose seats that are winnable, and that might mean the difference in control of the Senate.
There’s no shortage of changes that Republicans and the right might make for the better, but there are none more important for winning minds and elections than a full embrace of true experiments and rigorous research.
Adam B. Schaeffer, Ph.D., is a co-founder and director of research at Evolving Strategies, LLC. Nancy Smith is Grassroots Coordinator for Middle Resolution PAC.