As a research consultant for nearly 20 years, I entered the 2016 campaign assuming I had a decent grasp of what to expect. I had lived through the missed calls of Brexit, Eric Cantor's House primary in 2014 and the 2012 presidential race. But last cycle was something altogether different.
In fact, three months later I still find myself in the position of defending what's considered by many a polling “failure.” Many people smarter than I have given more thorough, vigorous, compelling defenses of political and public polling. I’ll try to limit my very brief treatise to a simple proposition: professional pollsters and researchers (such as myself), while different than public, "horse race" pollsters, must recognize that these public polls are the most visible, and most narrowly-defined artifacts of our industry. And we must continue to differentiate and demonstrate the value of what we provide.
Public-release polls are a ubiquitous element of the modern campaign season. But as the demand to conduct and release national polls has grown ever more, it has come at the expense of state-level surveys, which have dramatically declined in the last few cycles. What gets lost in these national surveys, however, is the ability to focus on the unique, often vexing idiosyncrasies of statewide polling, and the harder-to-measure sentiment that drives that data – the “why” behind the “what.”
Sentiment analysis and qualitative research are often the first things dropped from “normal” research plans – they are costly, manpower intensive, and slow. And while certain pollsters do specialize (and are very good) at state polls, the vast preponderance of polls that capture the public’s attention are national polls, which has led in part to the rise of probability prognostication – predicting the likelihood of winning through the amalgamation and analysis of all of these public-release surveys.
People like Nate Silver and Nate Cohn have already weighed in on the delta between public horse race polls, probability predictions, and the election results. But what's important to note is that for researchers like me, these polls and probabilities only vaguely resemble the work we do for our political and corporate clients. For me, the horse race number is simply one of innumerable data points that comprise the work our industry produces, and often not the most important one.
But rarely does this custom work – a deep, thorough, and usually expensive approach – see the light of day. This isn’t research for show. It's research designed to answer key questions, the answers to which inform business, campaign, and brand strategies. Embedded in this research are some of the most advanced research techniques available, qualitative, quantitative, and increasingly, analytic. A hybrid approach to research is neither easy, nor fast, nor cheap. But it’s certainly the best approach to answering the biggest challenges our clients face. And, unfortunately, not very practical for public-facing political pollsters.
So as we depart this election cycle and focus on the next, the question remains: how do professional political pollsters manage the relationship that we have with public pollsters? How do we shape and repair the perception that many of our clients and potential clients have, which is that “polling is dead?”
First, we don’t throw out the baby with the bath water. The public polls were good, even really good, at the national level. So any support we can provide to help shift resources back to state polling will help us all.
Second, it’s time to listen, even more. Qualitative research has always been my favorite tool. Surveys show you the border and shape of the tapestry, but qualitative data gives you the color, the texture, the feel. These are the stories that should help inform how we look at survey research data. You hear enough anecdotes, and you’re left with anecdata.
And third, let’s not be afraid to embrace open and unstructured data – we have the tools today to aggregate, curate, and analyze so much information. In a world full of data, lets help our clients understand and differentiate the signal from the noise.
Jason Boxt is an executive vice president at PSB Research.