As we approach Election Day, hundreds of campaigns across the country are faced with incredibly difficult choices, with diminishing time and resources available to them. But the choice of whether and what kind of opinion research to employ in the service of a campaign is always a challenging one, and one not reserved just for political campaigns.
Companies, non-profits, and associations are always faced with the same choice: how do we allocate limited dollars for research to answer our most critical questions?
If you have already ruled out qualitative research (focus groups, one-on-one interviews, et cetera), and are facing a decision about the kind of quantitative research you want to do, here are a few questions that can be asked to offer clarity and guidance on the topic:
1. What do I need to know?
The more complex the questions you need answered, the bigger the research program (or survey instrument) must be. A simple question with a simple answer might be answered by throwing a question on an existing survey, but insights to drive an entire campaign strategy may be a bit more complicated to ascertain.
2. What’s my most critical audience?
Most omnibus surveys that are available — polls that are explicitly designed to offer questions to multiple buyers on a range of often unrelated topics — are conducted with very broadly defined audiences: adults, consumers, or voters. So if you need to do research with a specific audience, omnibus surveys may not be the right answer.
3. What is the geography I’m interested in?
A congressional candidate hardly needs to ask a question of 1,000 Americans. With an omnibus survey, you’re limited by what the geographic boundaries are. If you’re running a statewide campaign, you might be able to find a statewide omnibus, but local races usually need local solutions.
4. What is your timeline?
Generally speaking, omnibus surveys are run on a set timeline. If you have the luxury of time, that might work. But if you need an answer tomorrow, this limits your options.
5. What is your budget?
This is always the $64,000 question. If you only have a few thousand dollars to spend, that will shape your options. The more money you have, the more freedom you have to design a research strategy that will explicitly provide the answers you need, from the people, and geography, you need to learn about.
At the end of the day, stand-alone, or custom, research offers both pros and cons, as do omnibus surveys.
Omnibus survey PROs: Can be fairly cheap (~$1000 a question), can be run either online or with phones — though online more prevalent — aren’t terribly complicated, although you need to write the question well, and can get back relatively quick results. Moreover, it usually has all of the demos you might want built into the survey instrument.
CONs: It might not fit your audience or geography, often come without a pollster to provide guidance or analysis, usually has order bias in the survey results (bias that results from how your question is perceived based on what respondents saw or heard before your questions), and you have very little control over the fielding or weighting process.
Stand-alone survey PROs: Is designed to give you insights you need, often for complicated challenges, comes with a qualified pollster, can be fielded on a custom timeline, with your exact audience and geography. Can be run online or by phone, and you have complete control over the process.
CONs: It’s more expensive than an omnibus survey ($15K or more), and it comes with a pollster (kidding, sort of).
What I always recommend to my clients is this: if you think you need research, call me. You’ll help me understand what you want to learn, and then I’ll talk you through your options. A good pollster is never going to fleece his or her clients. She’ll give you choices, and recommend a path that’s most efficiently designed to give you the insights you need to make the best decisions about your strategy. Then you’ll have to decide whether the benefits of research — clarity, guidance, answers — are important enough to spend time and money on them.
There’s no easy answer. If there were, you wouldn’t need a pollster.
Jason Boxt is the founder and CEO of 3W Insights, a strategic research consultancy. Prior to 3WI, Jason led the DC office of PSB Research, and before that headed up research at the public affairs firm Glover Park Group. He got his start in politics at the Democratic polling firm Global Strategy Group.