The Volokh Conspiracy - How did the pollsters do in predicting the popular vote?

The popular vote totals are not final. On Wednesday afternoon, the margin was 6.2% and rising. The pollsters who nailed it by reporting 6-7% included Rasmussen Reports, Pew Research, FOX News, Ipsos/McClatchy, CNN/Opinion Research, and the poll aggregator 538.

The worst performers were Gallup and Zogby with an 11% spread, followed by Battleground with a 3.5% average of its two models, and Marist, CBS News, and ABC News/Wash Post a 9% spread.

Interestingly, CNN and Fox News both outperformed the oldline networks -- ABC, CBS, and NBC.

Predicted Spread Poll

2-5 Battleground (combined models)

5 Diageo/Hotline

6 Rasmussen Reports

6 Pew Research

7 FOX News

7 Ipsos/McClatchy

7 CNN/Opinion Research

8 IBD/TIPP

8 NBC News/Wall St. Jrnl

9 Marist

9 CBS News

9 ABC News/Wash Post

11 Reuters/C-SPAN/Zogby

11 Gallup

Overall, the polls did pretty well this year. If the Obama ground game weren't so incredibly good, they might have done a tad worse. But only 2 polls were too low on Obama's margin, while 7 were too high.

In aggregating them, one would have done fairly well by throwing out the most Democratic poll among every 4 polls.

One thing struck me as well, though this opinion is probably tainted by hindsight bias: since 1960, it might well be that the campaign staff that ran the better campaign won each election, though I don't remember a few of them (eg, 1984) well enough to be certain of that. Of course, a great candidate makes a campaign look good, especially in retrospect.

UPDATE: Even though Nate Silver at 538 was not in the poll list I used for the post, I reached out to give him his props for being essentially as good as others who predicted a 6-7% spread before Tuesday's election.

Several commenters point to 538's Tuesday afternoon prediction of a 6.1% spread, issued long after the polls opened and turnout info was filtering back.

Judging his prediction by the same standard as the pollsters on the list above — subtracting his pre-election McCain prediction (46.1%) from his pre-election Obama prediction (52%), his predicted pre-election spread would be 5.9%, very slightly farther from the current spread than a couple of the pollsters above.

538 Mon PM & Tues AM prediction

Yet even that is complicated because his last pre-election presidential prediction post (on Monday evening) put the predicted spread at 6.0%, which simply means that the 52 to 46.1% spread actually rounded up to 6.0%:

With fewer than six hours until voting begins in Dixville Notch, New Hampshire, the national polling picture has cleared up considerably. Barack Obama is on the verge of a victory, perhaps a decisive victory, in the race for the White House.

The national polls have all consolidated into a range of roughly Obama +7. That is right about where our model sees the race as well, giving Obama a 6.8 point advantage in its composite of state and national polling. Our model notes, however, that candidates with large leads in the polls have had some tendency to underperform marginally on election day, and so projects an Obama win of 6.0 points tomorrow.

Silver's performance this year has been terrific, clearly establishing himself as the most reliable of the poll-based aggregators /predictors. He has an intuitive feel for numbers and knows when to tweak his models. In part because he appears to be the best out there, I hope that next time he releases his "final" predictions BEFORE the election.

	[Jim Lindgren, November 5, 2008 at 6:01pm] Trackbacks How did the pollsters do in predicting the popular vote? The popular vote totals are not final. On Wednesday afternoon, the margin was 6.2% and rising. The pollsters who nailed it by reporting 6-7% included Rasmussen Reports, Pew Research, FOX News, Ipsos/McClatchy, CNN/Opinion Research, and the poll aggregator 538. The worst performers were Gallup and Zogby with an 11% spread, followed by Battleground with a 3.5% average of its two models, and Marist, CBS News, and ABC News/Wash Post a 9% spread. Interestingly, CNN and Fox News both outperformed the oldline networks -- ABC, CBS, and NBC. Predicted Spread Poll 2-5 Battleground (combined models) 5 Diageo/Hotline 6 Rasmussen Reports 6 Pew Research 7 FOX News 7 Ipsos/McClatchy 7 CNN/Opinion Research 8 IBD/TIPP 8 NBC News/Wall St. Jrnl 9 Marist 9 CBS News 9 ABC News/Wash Post 11 Reuters/C-SPAN/Zogby 11 Gallup Overall, the polls did pretty well this year. If the Obama ground game weren't so incredibly good, they might have done a tad worse. But only 2 polls were too low on Obama's margin, while 7 were too high. In aggregating them, one would have done fairly well by throwing out the most Democratic poll among every 4 polls. One thing struck me as well, though this opinion is probably tainted by hindsight bias: since 1960, it might well be that the campaign staff that ran the better campaign won each election, though I don't remember a few of them (eg, 1984) well enough to be certain of that. Of course, a great candidate makes a campaign look good, especially in retrospect. UPDATE: Even though Nate Silver at 538 was not in the poll list I used for the post, I reached out to give him his props for being essentially as good as others who predicted a 6-7% spread before Tuesday's election. Several commenters point to 538's Tuesday afternoon prediction of a 6.1% spread, issued long after the polls opened and turnout info was filtering back. Judging his prediction by the same standard as the pollsters on the list above — subtracting his pre-election McCain prediction (46.1%) from his pre-election Obama prediction (52%), his predicted pre-election spread would be 5.9%, very slightly farther from the current spread than a couple of the pollsters above. 538 Mon PM & Tues AM prediction Yet even that is complicated because his last pre-election presidential prediction post (on Monday evening) put the predicted spread at 6.0%, which simply means that the 52 to 46.1% spread actually rounded up to 6.0%: With fewer than six hours until voting begins in Dixville Notch, New Hampshire, the national polling picture has cleared up considerably. Barack Obama is on the verge of a victory, perhaps a decisive victory, in the race for the White House. The national polls have all consolidated into a range of roughly Obama +7. That is right about where our model sees the race as well, giving Obama a 6.8 point advantage in its composite of state and national polling. Our model notes, however, that candidates with large leads in the polls have had some tendency to underperform marginally on election day, and so projects an Obama win of 6.0 points tomorrow. Silver's performance this year has been terrific, clearly establishing himself as the most reliable of the poll-based aggregators /predictors. He has an intuitive feel for numbers and knows when to tweak his models. In part because he appears to be the best out there, I hope that next time he releases his "final" predictions BEFORE the election.