Polls, Polls Everywhere and Not A Consensus In Sight

Seemingly contradictory public opinion polls drive political observers nuts, and recent surveys of President Obama’s approval rating have been no different.According to a recent Rasmussen Reports survey, more Americans disapprove of Obama’s performance than approve of it. That stands at odds with a recent Gallup poll, which found 56 percent approving of Obama, while just 28 percent disapprove—a small gain for the president since Gallup’s last measure. Zogby’s latest results fell in the middle with 53 percent approving, 38 disapproving. Meanwhile, a Marist poll released Wednesday put Obama’s approval rating at 55 percent—pretty much where it has been for the last three months.Some variation in polling results is normal, but in the heat of this August recess with healthcare reform on the line, these polls get plenty of headlines and tend to send Obama’s supporters and detractors into a tizzy with each perceived shift.Politics asked some of the major pollsters why these variations are taking place and whether they are significant. Nearly all of them explained the differences the same way: The results depend on whom you ask and what you ask.Rasmussen Reports generally has the president’s lowest approval rating. Scott Rasmussen said that his trends typically match other polling firms like Gallup’s, but he uses a different universe which lowers the approval rating. “The difference between likely voters [which Rasmussen samples] and adults [which compose Gallup’s sample] is the difference between Gallup and us,” he said.Rasmussen said that one of the biggest groups where this difference is apparent is among voters under 30 years old. They tend to have a very high opinion of the president but don’t tend to vote, so they, in theory, fall outside of Rasmussen’s sample universe and into Gallup’s.Jeff Jones, the managing editor of the Gallup survey, agreed. The difference in universe, he said, “should definitely make his more Republican, which it is.””The whole point of likely voters is basically that there are more Democrats out in the population but when you get into a voting situation, the parties become closer,” he said. “In 90 percent of these cases or more, using likely voters is going to make the poll more Republican.”Jones also noted that Gallup samples cell phone users, which would also tip the results slightly toward Obama.Another key difference is how the approval question is asked. Rasmussen asks respondents how they would “rate the job Barack Obama has been doing as President… do you strongly approve, somewhat approve, somewhat disapprove, or strongly disapprove of the job he’s been doing?” Gallup, on the other hand, asks more directly: “Do you approve or disapprove of the job Barack Obama is doing as president?” There is no rating system in Gallup’s question.”Depending on what exactly you ask people, you are going to get a different answer,” said Jay Leve, the editor of SurveyUSA. “Rasmussen asks his question different than do other pollsters so his results are at odds with other pollsters. It doesn’t mean his answer is wrong, it’s just different.”John Zogby, the eponymous pollster, echoed Rasmussen’s thoughts on the difference between likely voters, registered voters and adults and emphasized that there is normal ebb and flow in poll numbers.”For the most part you have everyone within a range of three or four points,” he said. “That’s margin of error stuff. If I were doing a poll every day, not a tracking poll, but a brand new poll, I could easily get a three point fluctuation.”More than one polling expert noted that the current focus on polls should lead to a better explanation of what causes these differences. That way, there might not be a frenzy every time the president or any other lawmaker sees his or her approval rating shift three points.”Most of the time when I see these polling controversies, the differences are usually about a failure to appreciate random sampling, polling error, wording or something else,” said Mark Blumenthal of Pollster.com. “I do think that the bottom line is that a smart consumer of polling data is someone that understands the limitations of survey data, why they can be different, why they can have errors and what sampling error means.”Jeremy P. Jacobs is the staff writer for Politics magazine. He can be reached at jjacobs@politicsmagazine.com.