Exit polls:

Slate Press Box — subtitled “media criticisms” — reports:

To hear [Joe] Lenski of Edison Media talk about it, the whole election brouhaha of 2004 can be blamed on the people who leaked the exit poll information and the outlets (Slate, drudgereport.com, wonkette.com, dailykos.com, mydd.com, et al.) that tossed the raw data out for consumption.

“I’m not designing polls for some blogger who doesn’t even understand how to read the data,” Lenski told the Los Angles Times yesterday. “It’s like if you were graded by your readers on the first draft of your article.”

Yet it is Lenski and the networks who are at fault for not telling viewers — and bloggers — the deeper meaning of exit poll data. The business of calling an election is as much an art as it is a science, and they’ve not been candid about that. . . .

That may well be right — but wouldn’t one also say that “it is those who post exit poll results, including Slate and in particular Pressbox, who are at fault for not telling readers the deeper meaning of exit poll data,” such as for instance

  1. the +/-4% announced mathematical margin of error (95% confidence interval, half that if you want 68% confidence) for state exit polls,

  2. the likelihood that the margin is higher earlier in the day, when the exit poll isn’t complete, and

  3. the possibility that the true of margin of error is higher still, because of different response rates among different people, and other factors?

Even if item 3 is obvious — and I suspect that many readers don’t find it obvious — the precise mathematical margin of error is generally not well-known to readers. Shouldn’t media outlets that report polls have a responsibility to give their readers the relevant information needed to figure out the poll’s limitations, just as the pollsters have a responsibility to give the media and the public the relevant information needed to figure out the poll’s limitations?

UPDATE: My friend Gil Milbauer points out that, at the bottom of their exit poll item, Slate did write:

These early exit-poll numbers do not divine the name of the winner. Instead, regard these numbers as a sportswriter does the line scores from the fourth inning of a baseball game. The leading team might win the game, but then again it might not. But having the early data in front of him helps the sportswriter plot the story he thinks he’ll need to write at game’s end.

But the analogy to reporting the fourth inning scores only goes so far — at least reports on the fourth inning scores are precise reports of the current score (though not the ultimate score), with no margin of error. Here, there are no precise results even as of the fourth inning; and 51-49% is not an accurate way of reporting even an intermediate result that’s actually 51-49% plus or minus 6%.

Comments are closed.

Powered by WordPress. Designed by Woo Themes