It seems that I need to re-up my 2012 post on “why having unequal numbers of Democrats and Republicans in polls is OK”. I was hoping after the massive failure of the “Unskewing Guy” in 2012 I wouldn’t have to revisit this, but it seems I’m wrong. If the amount of abusive emails and tweets we get accusing us of intentionally biasing the polls is any indicator, even MORE people this year feel compelled to attempt to “unskew” the polls than they did in 2012.
I’m going to be a bit impolitic up front and admit that I have no time for this nonsense. Anyone with a working understanding of the American electorate and how polling functions comprehends why polls of the population of 18+ Americans do NOT (and should not) contain equal numbers of Democrats and Republicans. I find amateur attempts to slander the polls on the basis of this issue infuriating and unworthy of notice or response. But I am going to sit here and re-write this blog in the hopes that at the very least it reduces the volume of hate mail we get…
Ipsos has been getting a lot of questions lately about the political makeup of polls. This is normal towards the end of an election cycle – lots of people scrutinize the polls a lot more closely! In an effort to explain this clearly, we first need to make some distinctions between question types. My colleague Chris has already explained the difference between all voting age Americans, Registered Voters, and Likely Voters. Equally important in political polling is being clear about the different types of questions we use to look at the political balance of our samples. We’ve covered the basic ones below:
- VOTING INTENTION: This is the ‘head-to-head’ or ‘horserace’ question that asks how people will vote. Ipsos asks: “If the 2012 Presidential Election were being held today and the candidates were as below, for whom would you vote?” followed by the candidates in a random order, ‘other’, ‘undecided’, and ‘not voting’. Not all firms ask this in exactly the same way, but in general there is a broad consensus on approach to this question. It is this question which is filtered on Registered Voters, or on Likely Voters when pollsters are reporting on their published voting data. This is also the question that poll aggregator sites HuffPo Pollster, FiveThirtyEight, and RealClearPolitics focus on for the national race when they track our data and other pollsters’ data.
- PARTY IDENTIFICATION (ID): This is also referred to as ‘partisanship’ or ‘political affiliation’. This question asks which party people most identify with, and how strongly they identify with it. We have explained this in great detail in another blog entry, but the crux of it is that Ipsos uses a ‘squeeze’ on people who identify as Independents or with no party at all, in order to better understand their likely voting behavior. This ‘squeeze’ approach gives Ipsos a smaller number of Independent voters than some other pollsters, however we publish the full breakdown of our question and so it is easy for interested parties to simply add the ‘leaners’ to the Independents in order to get a number comparable to other organizations’ metrics. It is critical to bear in mind that this question is self-reported and that there is no ‘correct’ target for this question – it changes constantly with the political ‘winds’, mainly as Independent and unaffiliated voters shift their allegiances (although it is worth noting that many campaign pollsters’ approach to surveys is to sample from voter lists and they therefore make a specific decision about the number of Democrats and Republicans in their polls at the outset). On balance, however, when talking about polls carried out as representative samples of the American public age 18 and over, there are more people in the United States who identify as Democratic than who identify as Republican. This is balanced out, however, by the fact that those identifying as Democrats are less likely than those identifying as Republicans to vote – so when we apply filters by likelihood of voting, the party balance comes much closer. In recent years, we’ve also seen a steady increase in the proportion identifying as Independent.
- Liberal-Conservative spectrum: This question is not as central when it comes to reporting vote behavior, but Ipsos tracks it as a key political ‘demographic’ to look for shifts over the medium-to-long term. Ipsos asks the question in this way: “Thinking about politics and government, do you consider yourself…?”(Very conservative; Moderately conservative; Lean conservative; Lean liberal; Moderately liberal; Very liberal; Don’t know/Refuse). While this question may seem similar to the Party ID question, that is not the case at all: far more people identify as ‘Conservative’ (47%) than ‘Republican’ (31%), for example.
These three political metrics are all important to capture when carrying out political polling work, but they do not measure the same things. For example, while more Americans identify as Democrats than Republicans, more also identify as Conservative than Liberal. This is because, for example, there are lots of Democrats (26% in fact) who identify as Conservative (often called ‘Blue Dog Democrats’), and about 10% of Republicans identify as ‘liberal’.
So Ipsos’ data reflects the fact that there are more people who identify as Conservative than Liberal, by a fairly large margin. However, this cannot be directly mapped onto the Party ID question. I emphasize these points to underline how inter-related but not identical these metrics are.
Likely Voter Filters Move the Electorate More Republican (Usually)
As a key step in reporting on voting data, Ipsos – like many other pollsters – filters the data to more accurately capture the opinions of those who will actually vote on Election Day – via the use of our Likely Voter Model. To illustrate this, just look at how Clinton’s lead over Trump narrows as we move from all voters to likely voters.
For the Voting Intention question, Ipsos’ goal is to replicate as closely as possible what might happen on Election Day, which means we think that the ‘Likely Voter’ figure is the most accurate.
A final thing to consider in looking at political polls is whether you are looking at weighted or unweighted data. Most pollsters report sample sizes and subgroup sizes based on unweighted figures. This is considered best practice in the polling world because it reflects the actual number of interviews conducted (rather than a base size inflated or contracted by the effects of weighting). However, when assessing poll quality, the absolute number of interviews conducted with people who identify as Democrats and Republicans is less important than the way that the weighted percentages break down. This is because the process of weighting the data balances out any imperfections in the sample makeup: if too many people from the East Coast, for example, were included in the sample, the weighting process will balance this out and ‘down-weight’ those interviews. It is the final, weighted percentages that matter, rather than the raw and unweighted counts.
So: please stop sending me angry emails. Our polls are reflective of the makeup of the American population. We are not trying to include more or fewer Democrats or Republicans than occur naturally as we work to reflect an accurate, Census-driven picture of the American public. As a nonpartisan polling firm, we are judged SOLELY on our ability to accurately predict the election outcome. We have no incentive – in fact there is a disincentive – to skew or adjust the data to show one candidate more likely to win.
Honestly, we are just trying to get the damn thing right.