Posted by: Janine Balekdjian, Consular Intern
In theory, public opinion polls are so simple. Ask a question, record a response, and keep doing this enough times until you have a large enough sample to draw conclusions from it. In reality, though, polling can be extremely complicated and influenced by seemingly inconsequential factors, which explains why different polls on the same issue can end up with some wildly different results.
5. Polls have political affiliations.
They might not have an R or D (for Republican and Democrat) next to their name like elected officials, but often, polling firms have a trend of leaning particularly to the left or to the right. For example, Rasmussen Reports, which is often used by Fox News, favors Republicans relative to other polls, and Public Policy Polling sometimes leans liberal. This isn’t to say that the pollsters are intentionally biased – although that can be the case, such as in the Research 2000 poll scandal. Even when polls do systematically favor one party over the other, the difference from the average is typically small. Usually, different polling methodologies can explain overall partisan leanings, which brings us to…
4. People respond differently to robots and real people.
A frequent strategy of polling firms which want to get as large a sample size as possible is to use what are called “robopolls” – polling phone calls in which an automated recording asks questions and records responses based on what buttons the person on the other end of the phone pushes. As counterintuitive as this may seen, robopolls tend to lean Republican when compared with a poll conducted by a real person, even though it seems like the caller shouldn’t make a difference to one’s opinion. No one knows exactly why this is, or whether robopolls or human pollsters are more accurate. Some of the effect could be caused by other methodological practices which are correlated with robopolls, such as only calling landlines and excluding cell phones. A hypothesis which explains conservative leanings in robopolls on issues is that people being polled may be more likely to express a view they think can be perceived as biased (such as opposing gay marriage) to an automated recording than to a real person. However, this theory doesn’t explain why robopolls would favor Republican candidates over Democrats.
3. It’s harder to get a random sample than you think.
Political polling is usually conducted over the phone. The only reason that polls are informative is the assumption that their data are generalizable to the country as a whole. When Gallup reports that President Obama leads Mitt Romney by 6%, we assume that their data applies to the whole country, not just to the people who responded to their poll. Pollsters have developed a number of strategies to make sure that they collect random samples, such as random digit dialing. Pollsters also go through their data afterwards and “weight” it so it reflects the American public better. However, as technology changes, random sampling methods are starting to become obsolete. Traditionally, polls only call conventional landlines, but now 25% of American adults do not own a landline, and this demographic is only increasing. Consequently, polls which do not call cell phones display a significant Republican bias. People who own cell phones but not landlines are much more likely to be young, urban, and members of minority groups, all of which correlate with being liberal. Thus, polls which exclude the cell phone-only population are going to show a stronger conservative leaning than the population at large. Recently, some pollsters have made forays into online polling, where random sampling is still in the experimental stage.
2. Question order influences people’s opinions.
Since it can be difficult to get people on the phone willing to answer questions, many pollsters take the opportunity to ask not just one but several questions: on the economy, on social issues, on major legislative developments, and on their candidate preference, for example. However, people’s opinions on all these issues aren’t as fixed as you might think – studies have proven that the order of questions influences people’s answers. When “primed” with a question about the economy, voters will give different responses to presidential approval than if asked about the President first – approval increases if the economy is good, and decreases if the economy is bad. This principle applies to issues as well – when asked a question that includes a viewpoint significantly to the right or left of the mainstream, the rest of the respondent’s answers become more extreme as well, possibly to avoid coming off as inconsistent. Good pollsters try to minimize this effect either by randomizing the order of questions for each respondent or making sure that general questions are asked before specific ones.
1. People will be for and against the same thing, depending on how you ask it.
Want a different result? Ask the same question, but change the wording. Certain buzzwords evoke strong emotional reactions, and pollsters can either use or avoid those words depending on what kind of result they’re looking for. For example, Americans will have unfavorable reactions to a poll using the word “welfare,” but they support government aid to the poor, which is logically equivalent. This just confirms what political operatives have known for decades: messaging matters.
The world of polling can be confusing, so some of the best places to look are aggregate polling agencies. These take into account all recent credible polls on a candidate or an issue and present an average. Real Clear Politics is one such site, and FiveThirtyEight both averages poll data and blogs about polling and statistics in politics for those curious about the field.
Comments