One of my favorite narratives as I follow any election is one of how the polls "got it right" or more often how the "polls got it wrong". My impression, from speaking with friends and family is that most people look at the top line numbers, but don't really bother to look at the margin or error in each poll, or any information about the sample. Even when folks rely on polling aggregators, such as FiveThirtyEight, I suspect that there is a tendency to look at the averages (what I usually refer to as means) without looking at the probability of any of a number of reasonable outcomes.
Nate Silver spends a good deal of time discussing what is okay and what is not okay with the polls at this moment in time. The short version is that the polls are generally a good marker of outcome in broad brushstrokes but are likely to be off if we went to be more specific. Nate Silver minces no words about the problems with non-response rates in current polling efforts (they're awful) and the challenges that pollsters will face in the future, assuming that the US population becomes more and more polarized. And yet, he's able to provide evidence that 2020 polling numbers were arguably no worse than 2016. State polling is worse than national polling, but that seems to be typical. The numbers, thus far, are about where we'd expect them, on average.
I tend to rely on polling aggregators much more than individual polls. Although I highly doubt that Silvers's FiveThirtyEight site is anywhere near perfect, it does a reasonable job. What I look at in particular are the range of probable outcomes based on the site's simulations, with regard to Presidential, Senatorial, and House outcomes. The challenge for readers is to understand that probabilities are just that - probabilities. A Presidential candidate who has a low probability for winning is not destined to lose. We saw how that worked out in 2016, and Silver was more than willing to alert his readers that Trump had a realistic probability of winning the Electoral College. This year, even with a lower probability of re-election, the polling data and simulations Silver's team ran were ones that urged caution against writing Trump off from winning a second term. The apparent electoral vote for Biden is well within the bounds of possibility, based on those simulations.
From what I recall of the simulations on Nate Silver's site, the Democratic and Republican parties performed within their expected margins. Outside the Presidency, the House was likely going to remain controlled by the Democratic Party. The only question was whether it would be a narrower or wider majority. There was no evidence I am aware of to suggest another 2018 style Blue Wave, given the strategy of focusing on suburban areas that had been, until recently held by Republicans. If anything, this year may have been something of a regression toward the mean. The most probable outcomes for the Senate were for the Democratic Party to hold between 48 and 55 seats. The latter would have required the Democratic Party to run the table in nominally competitive races in otherwise heavily red states. As it turns out, the worst case scenario is that the Democratic Party gains a seat, but still remains in the minority, with all that entails. We will know more once the Georgia run-off elections are held January 5th. Runoff elections, like special elections, tend to be low-turnout. So, whichever side manages to get their base motivated to go back to the polls is the one that wins. In Georgia, that has typically favored the Republicans. We are in unusual times, so nothing is a given, and polls indicate contests that are essentially dead heats.
At the end of the day, I'd probably tell my Democractic friends that they had an okay 2020. They won the White House back, held the House, and have something of a shot at breaking even in the Senate (though that is probably a long shot bet). I'd tell my Republican friends that they probably held the Senate, and were able to more than cut their losses in the House (although I'd warn them that some of their new House members may not help them nationally going forward). Those paying attention to the data analyses from this election are probably not particularly surprised.
I am not in the business of post-mortems and such from elections, nor do I intend to start now. I'll leave that to the political scientists. The polls and results were about as expected on average. My concern is more about accepting uncertainty when reading data and analyses from pollsters and from those who aggregate polls. I'm inevitably amazed at how well polls do given the non-response rate. I am impressed with the various factors aggregators weigh when assessing how to simulate what those polls are really telling us. As a consumer of this sort of data, I would only be concerned if the final results were outside the upper and lower bounds of what we would expect to see. That didn't happen in 2016, 2018, or 2020. For those hoping for something closer to the upper bounds, there is likely considerable disappointment. I'd counsel the opposite.
No comments:
Post a Comment