Reporting all the information about exit polls:

A Slate article argues that it was right to report the exit polls, on the theory that this way voters get to know what the journalists know. Maybe so; I have no fixed opinion on the matter.

But if you report 50-49 exit polls, and then report 51-49 numbers with subheads such as “Mucho flattering to Kerry,” wouldn’t it be more helpful to also report the following, from the National Election Pool page?

The margin of error for a 95% confidence interval is about . . . +/-4% for a typical state exit poll. . . . Other nonsampling factors may increase the total error.

You might also thrown in this from the Mystery Pollster (thanks to Douglas Johnson for the pointer), if you think he’s right — or come up with a better analysis yourself, if you think he isn’t quite right:

Even if comparable to the final numbers — which they are decidedly not — the mid-day leaked numbers would have much greater error, perhaps +/- 7% or more.

Even if you think 95% is a more demanding confidence interval, and you think 68% confidence is fine, you’d still have a margin of error half of those mentioned above. And this is the purely mathematical margin of error, i.e., accounting only for random variation, and not for the other problems that bedevil polls; you might also warn people of other possible causes of error, such as differences between early voters and late voters, or between those who talk to pollsters and those who don’t, or between the true vote and the vote reported to the pollster.

So if you’re really trying to inform your readers, it seems to me the story ought not be:

Updated Late Afternoon Numbers
Mucho flattering to Kerry; plus Nader makes an appearance.
. . .
Updated Tuesday, Nov. 2, 2004, at 4:28 PM PT

Florida
Kerry 51
Bush 49

Ohio
Kerry 51
Bush 49
. . .

Rather, they should be (using the Mystery Pollster’s 7% level, though if you prefer to use 6% the picture doesn’t look much different):

Updated Late Afternoon Numbers
Mucho flattering to Kerry; plus Nader makes an appearance.
. . .
Updated Tuesday, Nov. 2, 2004, at 4:28 PM PT

[Margin of error: +/-7% if you want to be 95% confident in the results, +/-3.5% if you want to be 68% confident; and note that the margin of error may be even greater if early voters differ from late voters, or if voters for one candidate are more likely to talk to exit pollsters than voters for the other.]

Florida
Kerry 51
Bush 49

Ohio
Kerry 51
Bush 49
. . .

And if you realize that, even at the 68% confidence level, the results are Kerry 47.5-54.5 vs. Bush 45.5-52.5 — a statistical tie even at this very low level of confidence — should the headline really be “Mucho flattering to Kerry”?

(Note that some of the other pro-Kerry predictions in the Slate-reported exit polls, such as those for Pennsylvania and New York, proved to be right as to the bottom line — not coincidentally, these were mostly those in which the gap was indeed pretty large, and not just a couple of percent.)

UPDATE: Martin Plissner, also in Slate, likewise has a criticism of paying much attention to exit polls that are within the margin of error.