*Published with the generous permission of Simon Rosenberg. Read more of his important work on his Hopium substack.
By Simon Rosenberg
Many of you have asked about the Washington Post/ABC News poll, which came out on Sunday, and so I’ve put together some notes about how we can start thinking about polling and electoral analysis this cycle here at Hopium. Let me know if you find all this helpful.
Tom Bonier and I are working on a longer, more in-depth piece, so today will be something more like some loose notes than a big comprehensive memo…
Polls Don’t Predict, They Are A Snapshot in Time
Every poll matters and every poll can tell you something. What polls cannot do is predict the future. All they can do is tell you what is happening today. That’s why polls like Navigator, which track things over time, are so important. They can compare today to a few weeks ago, last year, etc., and it is that comparison of data over time that provides the best understanding of politics.
Also, think of polls as sketches, not a full painting. They give you a rough guide to where you are, but are not 100% accurate and never will be. So yes, all poll analysis has to be done with a great deal of humility, as there is significant uncertainty built into the process
Stick to Averages and Trends, Never Allow A Single Poll To Dictate Your Understanding
Just as we should look at data over time, we should look at data over as many polls as we can. Polls can get things wrong, are rough, as discussed above, which is why the more polls and data you have, the more accurate your insights are likely to be.
Despite its flaws, 538 is still the best poll aggregator out there. I check it first thing every morning. Here’s a link to its Biden approval rating. Note how wide the variance is - mid-30s up to the low 50s - with an average of 43%. 43% is about where the President was in the fall of last year when Democrats racked up big wins in many of the battlegrounds, which will determine the Presidential election next year. So while it isn’t high, it’s a number Democrats can live with right now.
Real Clear Politics, while once a serious site, has become a right-wing propaganda outfit and should be ignored, as should any analyst referring to its data and insights.
Polling Has Gotten Expensive and Hard
Polling is hard. We are a remarkably diverse country, and getting samples which accurately capture a state or district is really hard. It has gotten harder because response rates to traditional polling have declined, making it more expensive to get a good and accurate sample. That extra cost and difficulty in constructing a good sample can encourage pollsters to cut corners a bit, leading to lower-quality data.
The big NYTimes story on the red wave that never came does a good job of explaining this dynamic.
The Amount of Public Polling Being Done By Right Leaning Outfits Is Increasing
The recent proliferation of right-leaning public polls was at the core of my battle against the red wave in the final weeks of the election. It used to be that a majority of polling was done by independent pollsters, often media organizations or university projects. Last year we saw a significant increase in right-leaning partisan public polling. Historically partisan polling remains private, as candidates and party committees do not want the other side to see what they are seeing. But in the last year, there has been an increase in public, non-candidate/non-party polls by right-leaning outfits. In the 538 Biden approval average I linked to earlier, there are roughly as many right-leaning polls as independent media polls and just a few left-leaning outfits. This means right-leaning money is playing a much bigger role today in shaping our understanding of polls and politics (yes, the right is louder than we are), which is something we have to discuss in the coming months. It’s not sustainable, for Democrats cannot be expected to take poll averages seriously where 40-50% of the polling is being done by outfits on the right.
For it was a flood of these right-leaning polls that threw off the averages late in the last election, causing almost everyone to get the election wrong.
What About that WaPo/ABC Poll?
Okay, that takes us to the Washington Post/ABC News poll that came out Sunday. Simply, it wasn’t a good poll and probably shouldn’t have been published. First, it polled adults, not just registered voters. Given all the problems we’ve had with polling and accuracy in recent years, it’s my opinion that pollsters should only be polling registered voters. What value is there in asking someone who cannot vote who they are voting for? Polling adults saves money but leads to junkier data.
Next, and this is something both Tom Bonier and our friend Stephen Clermont at Change Research have very strong feelings about, some of the demographic breakouts of the top-line data are just hard to believe and raise serious questions about the integrity of the poll itself. The most obvious example is that the poll has Trump winning younger voters and Biden winning older voters. Based on the election results of the last three elections and lots of other polling this year, we know that these findings cannot be true. These kinds of things happen in polling, but when they do, pollsters should either 1) acknowledge the problems 2) not publish the findings. Given these problems and others in the poll, it is my view this poll should have not been published, particularly by such reputable news organizations (I once worked at ABC News and have very high regard for the news division).
So, a reminder - stick to the averages (and yes, they have too many R polls right now), watch trends over time, don’t freak out over a single poll either way, be patient and measured.
I haven’t really geared up my own electoral analysis machine yet because it is early, and we don’t yet know who will be running and what kind of shape they will be in next spring when it all really starts to matter. I don’t plan on writing about individual polls but will be writing about broader trends at this point - things like how unpopular the Republicans are, Biden’s underperformance on economic issues, etc. We’ve already had a fair amount of that so far at Hopium.
I will also be incorporating into my analysis this cycle the most important parts of the analysis Tom and I did last cycle, which looks at data outside of polling - voter registration figures, fundraising, election results in places like Wisconsin last month or Jacksonville next week, early vote performance - that help paint a clearer and more detailed picture than what polling alone can provide. As I have already discussed with many reporters this cycle, I would be very careful about drawing conclusions about the intensity of the Democratic family from current polling. It didn’t capture the intensity we saw in the battlegrounds last year, and we already saw a fired-up Democratic machine blow it out in Wisconsin last month. I have no doubt that the anti-MAGA majority, which has shown up in such big numbers in the last three elections, is going to do so again next year with either Trump or DeSantis (two sides of the MAGA coin) on the ballot next year.
The central reason so many analysts got the election so wrong last time is they underestimated how powerful a force the fear of MAGA is in our politics and how mature and muscular our grassroots have become. Analysts underestimated the intensity of Democrats last cycle and should be very, very careful about doing so this time too.
For those who want to read a bit more about how Tom and I got it right when so many got it so wrong last time, here are a few things to check out:
My Post-Election Memo - The Democratic Party Is Strong, the Rs Remain All MAGA
Still Waiting On The Red Wave - A Recap of Our Heralded 2022 Election Analysis
Our Most Comprehensive Public Discussion Of What We Did (Video)
The "'‘Red Wave’ Washout: How Skewed Polls Fed A False Election Narrative
My Interview With Ron Brownstein in The Atlantic
