I just wanted to stress that the Tulane Law Review article incident isn’t just an interesting story of academic error — it’s also a story of law review embarrassment. I’m pretty sure that no law review likes to have to post on its front page,
Erratum
The Louisiana Supreme Court in Question: An Empirical Statistical Study of the Effects of Campaign Money on the Judicial Function published in Volume 82 of the Tulane Law Review at 1291 (2008), was based on empirical data coded by the authors, but the data contained numerous coding errors. Tulane Law Review learned of the coding errors after the publication. Necessarily, these errors call into question some or all of the conclusions in the study as published. The Law Review deeply regrets the errors.
I assume the law review will also have to publish a print correction. The incident also led the law school dean to have to feel obligated to publicly apologize for the errors in the article, and though the apology said the law review members did nothing wrong, the matter can’t have been great for relations between the dean and the journal. And I suspect the incident in some measure tarnished the law review’s brand with local employers, especially those who are friendly with the judges whom the article criticized based on inaccurate information (and an unsound confusion of causation and correlation).
Of course, law reviews must accept the risk of public hostility when they publish articles that criticize much-liked people and institutions. That’s part of law review editors’ responsibilities as participants in the scholarly publishing process. But the hostility is likely to be considerably higher when the criticisms prove to be based on error. And it’s one thing to incur unjustified hostility in the service of truth, and quite another to incur justified condemnation because one’s institution has been mistaken.
So it seems to me that there are three important lessons here:
1. When the author’s article rests on data that you can check, check it. Here, the data was information about who voted which way in certain cases, and who got what contributions from whom — something cite-checkers are amply competent to check; and checking the data for fewer than 200 cases is not a crushing burden.
If the data had been in footnotes or in an appendix, as it is in many articles, the law review would have checked it. That the data never made its way into a print article is no reason to skip checking it (as this incident illustrates). The printed article, after all, relied on the data, and errors in the data infected the information reported by the article. Had the law review done the cite-checking, they might have avoided the embarrassment to themselves, their dean, and (incidentally) the authors.
2. Look closely through the article’s description of what it’s saying, and watch out for self-contradiction (especially when the article is controversial enough that authors might be tempted into some self-contradictory self-protection). So when a footnote says,
It is worth observing that this Article does not claim that there is a cause and effect relationship between prior donations and judicial votes in favor of donors