It’s election season. The time when a political junkie’s fancy turns to polls, polls, and more polls. With so many polls being released, many of the polls will show results that are contradictory from each other, which one do you believe? Should you believe any of them?
Conservatives have long been leery of polls, but that skepticism reached its height after Donald Trump’s surprise victory in 2016. Much of this skepticism lies in a misunderstanding of how to look at polls and what they represent. Some of this misunderstanding is fed by media outlets and politicians that misuse polls.
When considering polls, keep a few simple rules of thumb in mind:
Polls are historical, not predictive. They provide a snapshot of public opinion when they were taken. By themselves, they do not forecast the outcome of elections. Polls are lagging indicators that measure public opinion as it was on a given date.
Look at the trend of all the polls. To see which way public opinion is moving, don’t just look at one poll. Look at similar polls taken over a period of time and compare the results. For example, one poll on Donald Trump’s approval rating isn’t very useful, but if you look at the trend of all polls showing the presidential approval rating you can see whether it has improved or declined.
Real Clear Politics is a useful site that acts as a clearinghouse of polls. You can look up individual poll types such as President Trump’s approval, individual races such as Ted Cruz v. Beto O’Rourke, or generic ballots.
RCP shows all the polls of each particular type, which makes it easy to see trends. For instance, the page of Cruz-O’Rourke polls shows 10 polls going back to April. The trend shows a surge by O’Rourke in which Cruz moved from a double-digit lead to a statistical tie.
Discard the outliers. Politicians and the media often trumpet shocking poll results such as the recent Rasmussen poll that showed President Trump’s support among black voters at 36 percent. Polls that differ wildly from other polls are outliers and should be treated with suspicion.
One way of reducing the effect of outliers is to take an average of polls. Both Real Clear Politics and FiveThirtyEight have pages that show the average of polls asking about President Trump’s approval rating that go all the way back to his inauguration. These are useful tools for getting the big picture of the longterm trends in presidential approval.
Look at the nuts and bolts of the poll. Not all polls are created equal. You can get an idea of how reliable a poll is by looking at who it surveyed. The most accurate polls talk to likely voters. Polls of registered voters are less accurate while those that survey adults are the least accurate.
Pollsters also must make assumptions about who will show up to vote. These assumptions are used to weight the data from the poll, but there is no way to test their validity until Election Day when we find out who comes out to vote. Many polls give information about the weighting and the mix of Democrat and Republican respondents if you read the fine print.
Consider the margin of error. No poll is exact because they all represent only a sample of the population. The accuracy of the poll can be calculated and is usually disclosed as the “margin of error.” A large sample size is more accurate and reduces the poll’s margin of error.
The thing to remember is that the closer a poll is, the less it can be used to predict a specific outcome. For example, the most recent Cruz-O’Rourke poll that showed a one-point race had a 4.4-point margin of error. This is known as a statistical tie. The one-point difference is well within the margin of error so the key takeaway from the poll is that the race is currently too close to call, not that Ted Cruz would win the election by one point.
If a poll shows a large difference discrepancy between two viewpoints, you can be reasonably confident that the general breakdown is correct, even if the specific percentages are not. For example, when 72 percent opposed the Trump Administration policy of separating immigrant children from their parent and only 27 percent were in favor in a June CBS News poll, there was little doubt that Americans strongly opposed the policy.
Consider polling difficulties. The smaller the race, the tougher it is to get good polling. National polls are the most accurate, but state and district polls are more questionable. There might be no public polling at all in some House and Senate races.
The root cause of much of the 2016 polling problem was polling at the state level in a few Rust Belt states. By Election Day, national polls were showing a close race between Donald Trump and Hillary Clinton. The RCP average gave Hillary Clinton a 3.1 point edge, within the margin of error of most polls. In the final tally, Clinton won the popular vote by 2.1 points which was very close to what the polls showed.
Of course, the popular vote does not decide presidential elections and the predicted outcome of elections in key states turned the Electoral College results. An after-action report by the American Association for Public Opinion Research noted that “eight states with more than a third of the electoral votes needed to win the presidency had polls showing a lead of three points or less” and that “polls on average indicated that Trump was one state away from winning the election.”
The outcome in Pennsylvania, Michigan and Wisconsin was the surprise for forecasters that turned the election to Trump, yet the polling in both Michigan and Pennsylvania showed a close race. In both states, the last poll before the election favored Donald Trump. The polls were only substantially off in Wisconsin where the last poll favored Hillary by eight points, but Trump won by less than one point.
In the end, most polls did not capture a late surge by Donald Trump in the wake of FBI Director James Comey’s memo to Congress. The lagging indicators were too far behind to include the rapidly changing landscape but did reflect a very close race in most cases. The AAPOR after-action report also faults an overrepresentation of college graduates in many polls that would have favored Clinton.
Polling is not an exact science. Poll results shouldn’t be considered to be gospel, but neither should they be ignored. By looking beyond the headline, you can determine whether a poll is reliable and how it fits into the big picture of the election.
Originally published on The Resurgent
No comments:
Post a Comment