Campaign practitioners have said for two cycles now: the Internet’s great for fundraising and field but not nearly as good for persuasion. The problem is that campaigns lack a social media tool that accurately measures the quality of conversations, and until they have one there’s no way to tell if online messages are moving the needle.Facebook has its Insights and Twitter its Analytics, but what metric is there for knowing where people stand on an issue and how to engage them? There’s not a good one—at least not yet. But social media experts are optimistic about the role sentiment analysis could play at the presidential level in 2016.
“It makes sense to me, given all of the data segmenting campaigns are doing online as it is,” says Beth Becker, founder of the Democratic social media firm Progressive PST. “To me this is just the next logical step.” There are whispers of larger NGOs dabbling in sentiment analysis, but Becker says she didn’t heard of a single campaign really spending much time on it last cycle—not even President Obama’s.
The problem is, up to this point, it’s never really been done in a manner campaigns view as statistically valid. Becker hopes to play a role in changing that, having teamed up with the VP of Lake Research Partners, Rick Johnson, to test sentiment analysis against traditional polling. During the 2012 cycle, their team looked at 500 active Twitter users in a particular district—verified through self-identification and cookies with 85 percent accuracy—and then used Twitter’s Firehose to read their tweets for a set period of time. A total of three experiments were performed in conjunction with robopolls, and both methods fell within the other’s margin of error. What’s more, the sentiment analysis was cheaper because no one needed to be paid to make calls for weeks on end. Becker estimates a campaign might pay $10,000-a-year to perform sentiment analysis once monthly, versus $40,000-per-traditional poll. Still, there are plenty of caveats. Sentiment analysis can still be costly for down-ballot campaigns.
And Soren Dayton, SVP at Prism Public Affairs, says only presidential races would have the signal-to-noise ratio for sentiment analysis to truly be meaningful. “I think the key thing here is that polling or sentiment analysis is valuable for checking models when you have good enough models,” he says. “All of this can be integrated as people try to build profiles.”
The online demographic also differs from the one represented in a traditional poll, so sentiment analysis shouldn’t be viewed as a polling replacement. Protected accounts on Twitter and Facebook are impossible to analyze, methods for determining a user’s location aren’t 100 percent accurate and automating sentiment analysis is impossible at this stage. Johnson gives the example of someone tweeting, “The Obama campaign is sick.” A program might code that as negative, even though “sick” often has a positive connotation among younger users.
There are a number of other phrases that algorithms tend to miss and miscode, though they’re getting better, Johnson says. “No,” he says, when asked if he intends sentiment analysis to replace traditional polling methods. “But it should be another arrow in the quiver when you’re trying to figure out what people think.”
Becker expects campaigns will always need humans in the mix when performing sentiment analysis— to perform the initial sampling and targeting, teach the program what a particular sentiment is and interpret the data once it’s crunched out the other end. Clarabridge is one popular program for analyzing sentiment in text. Radian6, Crimson Hexagon and DiscoverText are also heavily utilized. Not surprisingly, the private sector has been quicker to adopt these programs: corporations use them when releasing new products, gauging early consumer reactions on social media and tweaking their ad campaigns to incorporate the language used by early adopters. When it comes to influencing public opinion and promoting new products, Johnson says, “The people who are on social media tend to be more proactive.”