The Volokh Conspiracy - Comparisons:

Correlation, we're correctly told, doesn't demonstrate causation. At best, it's a first step towards making causation more likely. But it's striking how some stories hint at causation without even showing correlation.

Here's an example, with thanks to reader Adam Sofen. Slate ran a piece last week on mining accidents, and included the following (emphasis added):

The real obstacle to safety reform is that miners no longer have a powerful union sticking up for them. History shows that when miners have: 1) been organized and angry; and 2) had the strong national leadership of the United Mine Workers of America backing them up, they've been able to push for the legislative changes necessary for lasting advances in safety conditions. Sadly, neither of those two factors exist today. In fact, mining in the United States is only safer today than it has ever been because organized mine workers pushed hard for reforms a generation ago—reforms that are still in effect. Whether those reforms are enough is now in question. The majority of mining deaths in the past few years have occurred in nonunion mines. Sago was not a unionized mine, and, according to public records, federal inspectors noted 46 alleged violations of federal mine health and safety rules at the Sago site during an 11-week review that ended in late December.

Perhaps I'm mistaken, but it seems to me that this suggests that the stated fact -- "The majority of mining deaths in the past few years have occurred in nonunion mines" -- at least tends to indirectly show causation -- lack of unionization decreases safety.

But here's another fact, noted at the end of the sixth paragraph following the quoted ones:

Today, according to the union's own optimistic estimates, only about 30 percent of all mines are organized.

So the majority of mining deaths have occurred in the about 70% of all mines that are nonunionized -- how does this even show a correlation between lack of unionization and lack of safety? And given this, how is the assertion that "The majority of mining deaths in the past few years have occurred in nonunion mines" remotely probative of anything?

Of course, it's possible that the author could show correlation between danger and nonunion status. For instance, if "majority of mining deaths" was 90% of all mining deaths, that would tend to show correlation. If the 30% of all mines that are unionized (aan "optimistic estimate[]") contained 80% of all miners, that would tend to show correlation. But the story gives no such data; the only directly relevant data it gives are the statements that "The majority of mining deaths in the past few years have occurred in nonunion mines," and, six paragraphs later, "only about 30 percent of all mines are organized." (I tried to find such data -- many thanks to UCLA Law School research librarian Jill Fukunaga for her help -- but the closest I could get is that according to the Bureau of Labor Statistics, in 2004 28.9% of coal mine employees were union-affiliated, which I realize may not indicate what fraction of miners were union-affiliated.)

So, an elaboration: Correlation doesn't prove causation. But a statement that "the majority of Xs occur at Ys" doesn't even show correlation. It's at best irrelevant, and at worst misleading. Yet I've seen this sort of "evidence" time and again; watch out for it.

	[Eugene Volokh, February 13, 2006 at 6:13pm] Trackbacks Comparisons: Correlation, we're correctly told, doesn't demonstrate causation. At best, it's a first step towards making causation more likely. But it's striking how some stories hint at causation without even showing correlation. Here's an example, with thanks to reader Adam Sofen. Slate ran a piece last week on mining accidents, and included the following (emphasis added): The real obstacle to safety reform is that miners no longer have a powerful union sticking up for them. History shows that when miners have: 1) been organized and angry; and 2) had the strong national leadership of the United Mine Workers of America backing them up, they've been able to push for the legislative changes necessary for lasting advances in safety conditions. Sadly, neither of those two factors exist today. In fact, mining in the United States is only safer today than it has ever been because organized mine workers pushed hard for reforms a generation ago—reforms that are still in effect. Whether those reforms are enough is now in question. The majority of mining deaths in the past few years have occurred in nonunion mines. Sago was not a unionized mine, and, according to public records, federal inspectors noted 46 alleged violations of federal mine health and safety rules at the Sago site during an 11-week review that ended in late December. Perhaps I'm mistaken, but it seems to me that this suggests that the stated fact -- "The majority of mining deaths in the past few years have occurred in nonunion mines" -- at least tends to indirectly show causation -- lack of unionization decreases safety. But here's another fact, noted at the end of the sixth paragraph following the quoted ones: Today, according to the union's own optimistic estimates, only about 30 percent of all mines are organized. So the majority of mining deaths have occurred in the about 70% of all mines that are nonunionized -- how does this even show a correlation between lack of unionization and lack of safety? And given this, how is the assertion that "The majority of mining deaths in the past few years have occurred in nonunion mines" remotely probative of anything? Of course, it's possible that the author could show correlation between danger and nonunion status. For instance, if "majority of mining deaths" was 90% of all mining deaths, that would tend to show correlation. If the 30% of all mines that are unionized (aan "optimistic estimate[]") contained 80% of all miners, that would tend to show correlation. But the story gives no such data; the only directly relevant data it gives are the statements that "The majority of mining deaths in the past few years have occurred in nonunion mines," and, six paragraphs later, "only about 30 percent of all mines are organized." (I tried to find such data -- many thanks to UCLA Law School research librarian Jill Fukunaga for her help -- but the closest I could get is that according to the Bureau of Labor Statistics, in 2004 28.9% of coal mine employees were union-affiliated, which I realize may not indicate what fraction of miners were union-affiliated.) So, an elaboration: Correlation doesn't prove causation. But a statement that "the majority of Xs occur at Ys" doesn't even show correlation. It's at best irrelevant, and at worst misleading. Yet I've seen this sort of "evidence" time and again; watch out for it.