The Volokh Conspiracy - ProPublica's dishonest defense its dishonest article:

My media column for the Saturday Rocky Mountain News, discussed gross misstatements of fact which had appeared in a November 13 article by ProPublica, an organization which supplies investigative articles for free to the mainstream media. A shorter version of that article had appeared in the November 17 Denver Post.

Today, ProPublica author Abrahm Lustgarten has written a defense of his article. He claims that my article is "indisputably misleading." Let's take a look at each of the three charges which I leveled at the ProPublica article.

1. I wrote:

The theme of the ProPublica article, headlined "Buried Secrets," is the natural gas industry's refusal to disclose a list of all chemicals which are injected into the ground in hydraulic fracturing. The article accurately characterizes the Colorado Oil and Gas Conservation Commission as the "most stringent" regulatory agency regarding hydraulic fracturing.

The COGCC promulgated its final draft rules on Nov. 7, before the Nov. 13 ProPublica article, and before its Nov. 17 appearance in the Post. The article misdescribes the new regulations, and, significantly, omits the fact that the commission's new disclosure rule is nearly identical to what the drilling company Halliburton proposed in its June testimony to the commission. Section 205 of the new regulations protects drillers' trade secrets about the precise chemical recipes, while mandating full disclosure when specifically needed by the state for health or environmental protection.

In the spring, the Colorado Oil and Gas Conservation Commission had proposed draft regulations which would have required natural gas drillers to disclose the exact recipes for the fluids which are injected into the ground in hydraulic fracturing. During summer hearings, the industry vehemently objected, and said that they would pull out of Colorado, rather than disclose their trade secrets. Lustgarten's article accurately describes this part of the story.

Then, according to Lustgarten:

In August, the industry struck a compromise by agreeing to reveal the chemicals in fracturing fluids to health officials and regulators — but the agreement applies only to chemicals stored in 50 gallon drums or larger. As a practical matter, drilling workers in Colorado and Wyoming said in interviews that the fluids are often kept in smaller quantities. That means at least some of the ingredients won't be disclosed.

"They'll never get it," says Bruce Baizel, a Colorado attorney with the Oil and Gas Accountability Project, about the states' quest for information. "Not unless they are willing to go through a lawsuit. When push comes to shove, Halliburton is there with its attorneys."

This is entirely wrong. Under section 205 of the the final draft rules, which were published on November, the reporting trigger is not 50 gallon drums, but whether an individual well site uses 500 more more pounds of a chemical product in a quarter. Significantly, fracking companies must disclose to the public the trade names of the fracking ingredients; moreover, whenever an environmental or health official needs information for a specific investigation, the companies must disclose the exact chemicals in their recipes, with the chemical list being treated a Confidential Business Information by the officials who receive the list. As a described in my article, the final rule is similar to what Halliburton proposed in its June 6, 2008 testimony.

Lustgarten's January 12 self-exculpation does not even mention his misdescription of the regulations, and does not attempt any rebuttal of my evidence that he falsely accused the natural gas industry of hiding "buried secrets" even though the industry had proposed a disclosure rule and the "most stringent" (Lustgarten's words) agency had adopted something very close to that rule.

2. A second issue is Lustgarten's bait and switch about data. The article includes an extensive discussion of a case in Sublette County, Wyoming, in which groundwater was alleged to have been contaminated by hydraulic fracturing, and in which the federal Bureau of Land Management determined that fracturing might be the cause. Lustgarten wrote:

The contamination in Sublette County is significant because it is the first to be documented by a federal agency, the U.S. Bureau of Land Management. But more than 1,000 other cases of contamination have been documented by courts and state and local governments in Colorado, New Mexico, Alabama, Ohio and Pennsylvania. In one case, a house exploded after hydraulic fracturing created underground passageways and methane seeped into the residential water supply. In other cases, the contamination occurred not from actual drilling below ground, but on the surface, where accidental spills and leaky tanks, trucks and waste pits allowed benzene and other chemicals to leach into streams, springs and water wells.

It is difficult to pinpoint the exact cause of each contamination, or measure its spread across the environment accurately, because the precise nature and concentrations of the chemicals used by industry are considered trade secrets.

In an e-mail to Lustgarten, I specifically asked him what he now calls "a precisely tailored question." I asked him for the documentation of his claim that there were over a thousand "documented" state and local cases of groundwater contamination from "hydraulic fracturing." ("where can the data be found which substantiate the fact about over a thousand documented cases of contamination from fracking in five states?") He responded by pointing to the Colorado and New Mexico agencies. ("The New Mexico Energy, Minerals and Natural Resources Department and the Colorado Oil and Gas Conservation Commission have together documented more than 1000 cases where water was contaminated by drilling activities.") I asked the Colorado and New Mexico agencies the same question I had asked Lustgarten, and the response was "zero" for Colorado; and that New Mexico has no such data.

Now, Lustgarten says that all along he was talking about any water contamination that resulted in any way from oil or gas drilling--such as leakage of chemicals from a waste pit on an oil-drilling site.

But that's not the question that I asked Lustgarten, and it's not what he wrote in his article. His article contrasts "the first to be documented by a federal agency" with "But more than 1,000 other cases of contamination have been documented by courts and state and local governments in Colorado, New Mexico, Alabama, Ohio and Pennsylvania." How many times do you think that any "federal agency" has "documented" groundwater contamination that resulted in any way from oil or natural gas drilling. If and only if the 2008 BLM case in Wyoming is the first and only case of such federal documentation can Lustgarten's defense of his article be true.

3. I also wrote that Lustgarten had falsely described a study by the Interstate Oil and Gas Compact Commission:

The Commission surveyed regulatory agencies in 28 states (including Colorado and the other four states where ProPublica claimed that there were more than 1,000 "documented" cases of contamination). The response covered the entire history of hydraulic fracturing in those states. Every single one of those 28 states reported that there had never been groundwater harm due to fracturing.

The ProPublica article did not report the evidence from that government study, but brusquely dismissed it as "an anecdotal survey done a decade ago." Actually, the 2002 study has no anecdotes, and with a dataset of almost a million wells, it cannot plausibly be considered "anecdotal."

There are three comprehensive studies about hydraulic fracturing: a 2004 study by the EPA; a second, earlier, study about the same subject as the EPA study (groundwater safety as it relates to about hydraulic fracturing in coal methane beds); and the third study, mentioned above, by the Interstate Commission. My article mentioned and discussed only the third study.

Lustgarten writes:

The drilling industry, echoed by Kopel, cites three documents when asserting the environmental safety of hydraulic fracturing. They are a 2004 EPA study (PDF), a 2002 survey of state agencies (PDF) by the Interstate Oil and Gas Compact Commission and a similar survey in 1998 by the Ground Water Protection Council (PDF).

In its Nov. 13 article, ProPublica detailed flaws in the EPA study and reported that the two surveys were "anecdotal," meaning that they included none of the basic data required to qualify as a scientific study.

To say the least, that's an extremely idiosyncratic meaning of "anecdotal." The dictionary defines "anecdotal" as "based on personal observation, case study reports, or random investigations rather than systematic scientific evaluation."

"Anecdotal" is an accurate description of Lustgarten's article, which examines a few cases of alleged contamination. There's nothing wrong with anecdotal news stories. "Anecdotal" is not an accurate description of the Interstate Commission study, which has no anecdotes, and which collected decades of data from 28 state regulatory agencies.

Now, we find that Lustgarten apparently has his own definition of "anecdotal"--that is, something is "anecdotal" if does not include "the basic data required to qualify as a scientific study." Perhaps at some time Lustgarten will explain what basic data he thinks were lacking from the two studies. As a media critic, I would not have criticized him for offering plausible critiques of the studies. His article, however, did not contain any argument about what data he thought were missing, and his characterization of the studies as "anecdotal" was false and misleading--at least for readers who understand the word to mean what the dictionary says it means.

Today I received this e-mail from Mark Thiesse, a Wyoming groundwater regulator who is quoted in Lustgarten's original article:

I'd like to thank you for your recent editorial on the ProPublica article. I was one of the folks (I'm with the WY Dept of Env Quality) interviewed for this article by Mr. Lustgarten. I spent several hours on the phone and around a dozen follow up emails to try and help him write a factual article. Unfortunately he seemed to have his own agenda. The one error that was most blatant from my perspective was the "20 mile long plume" that he mentions. I must have told him 5 times that it was individual impacts to separate water wells due to water well drilling practices -- not related to oil and gas drilling at all -- but that did not make it into his article that way.

If you had to make an important decision, would you rely on the factual information in a ProPublica article? I have only studied one article from ProPublica in detail, but the organization's implausible efforts to defend the validity of a wildly inaccurate article would make me hestitate to rely on anything from ProPublica.

	[David Kopel, January 12, 2009 at 7:58pm] Trackbacks ProPublica's dishonest defense its dishonest article: My media column for the Saturday Rocky Mountain News, discussed gross misstatements of fact which had appeared in a November 13 article by ProPublica, an organization which supplies investigative articles for free to the mainstream media. A shorter version of that article had appeared in the November 17 Denver Post. Today, ProPublica author Abrahm Lustgarten has written a defense of his article. He claims that my article is "indisputably misleading." Let's take a look at each of the three charges which I leveled at the ProPublica article. 1. I wrote: The theme of the ProPublica article, headlined "Buried Secrets," is the natural gas industry's refusal to disclose a list of all chemicals which are injected into the ground in hydraulic fracturing. The article accurately characterizes the Colorado Oil and Gas Conservation Commission as the "most stringent" regulatory agency regarding hydraulic fracturing. The COGCC promulgated its final draft rules on Nov. 7, before the Nov. 13 ProPublica article, and before its Nov. 17 appearance in the Post. The article misdescribes the new regulations, and, significantly, omits the fact that the commission's new disclosure rule is nearly identical to what the drilling company Halliburton proposed in its June testimony to the commission. Section 205 of the new regulations protects drillers' trade secrets about the precise chemical recipes, while mandating full disclosure when specifically needed by the state for health or environmental protection. In the spring, the Colorado Oil and Gas Conservation Commission had proposed draft regulations which would have required natural gas drillers to disclose the exact recipes for the fluids which are injected into the ground in hydraulic fracturing. During summer hearings, the industry vehemently objected, and said that they would pull out of Colorado, rather than disclose their trade secrets. Lustgarten's article accurately describes this part of the story. Then, according to Lustgarten: In August, the industry struck a compromise by agreeing to reveal the chemicals in fracturing fluids to health officials and regulators — but the agreement applies only to chemicals stored in 50 gallon drums or larger. As a practical matter, drilling workers in Colorado and Wyoming said in interviews that the fluids are often kept in smaller quantities. That means at least some of the ingredients won't be disclosed. "They'll never get it," says Bruce Baizel, a Colorado attorney with the Oil and Gas Accountability Project, about the states' quest for information. "Not unless they are willing to go through a lawsuit. When push comes to shove, Halliburton is there with its attorneys." This is entirely wrong. Under section 205 of the the final draft rules, which were published on November, the reporting trigger is not 50 gallon drums, but whether an individual well site uses 500 more more pounds of a chemical product in a quarter. Significantly, fracking companies must disclose to the public the trade names of the fracking ingredients; moreover, whenever an environmental or health official needs information for a specific investigation, the companies must disclose the exact chemicals in their recipes, with the chemical list being treated a Confidential Business Information by the officials who receive the list. As a described in my article, the final rule is similar to what Halliburton proposed in its June 6, 2008 testimony. Lustgarten's January 12 self-exculpation does not even mention his misdescription of the regulations, and does not attempt any rebuttal of my evidence that he falsely accused the natural gas industry of hiding "buried secrets" even though the industry had proposed a disclosure rule and the "most stringent" (Lustgarten's words) agency had adopted something very close to that rule. 2. A second issue is Lustgarten's bait and switch about data. The article includes an extensive discussion of a case in Sublette County, Wyoming, in which groundwater was alleged to have been contaminated by hydraulic fracturing, and in which the federal Bureau of Land Management determined that fracturing might be the cause. Lustgarten wrote: The contamination in Sublette County is significant because it is the first to be documented by a federal agency, the U.S. Bureau of Land Management. But more than 1,000 other cases of contamination have been documented by courts and state and local governments in Colorado, New Mexico, Alabama, Ohio and Pennsylvania. In one case, a house exploded after hydraulic fracturing created underground passageways and methane seeped into the residential water supply. In other cases, the contamination occurred not from actual drilling below ground, but on the surface, where accidental spills and leaky tanks, trucks and waste pits allowed benzene and other chemicals to leach into streams, springs and water wells. It is difficult to pinpoint the exact cause of each contamination, or measure its spread across the environment accurately, because the precise nature and concentrations of the chemicals used by industry are considered trade secrets. In an e-mail to Lustgarten, I specifically asked him what he now calls "a precisely tailored question." I asked him for the documentation of his claim that there were over a thousand "documented" state and local cases of groundwater contamination from "hydraulic fracturing." ("where can the data be found which substantiate the fact about over a thousand documented cases of contamination from fracking in five states?") He responded by pointing to the Colorado and New Mexico agencies. ("The New Mexico Energy, Minerals and Natural Resources Department and the Colorado Oil and Gas Conservation Commission have together documented more than 1000 cases where water was contaminated by drilling activities.") I asked the Colorado and New Mexico agencies the same question I had asked Lustgarten, and the response was "zero" for Colorado; and that New Mexico has no such data. Now, Lustgarten says that all along he was talking about any water contamination that resulted in any way from oil or gas drilling--such as leakage of chemicals from a waste pit on an oil-drilling site. But that's not the question that I asked Lustgarten, and it's not what he wrote in his article. His article contrasts "the first to be documented by a federal agency" with "But more than 1,000 other cases of contamination have been documented by courts and state and local governments in Colorado, New Mexico, Alabama, Ohio and Pennsylvania." How many times do you think that any "federal agency" has "documented" groundwater contamination that resulted in any way from oil or natural gas drilling. If and only if the 2008 BLM case in Wyoming is the first and only case of such federal documentation can Lustgarten's defense of his article be true. 3. I also wrote that Lustgarten had falsely described a study by the Interstate Oil and Gas Compact Commission: The Commission surveyed regulatory agencies in 28 states (including Colorado and the other four states where ProPublica claimed that there were more than 1,000 "documented" cases of contamination). The response covered the entire history of hydraulic fracturing in those states. Every single one of those 28 states reported that there had never been groundwater harm due to fracturing. The ProPublica article did not report the evidence from that government study, but brusquely dismissed it as "an anecdotal survey done a decade ago." Actually, the 2002 study has no anecdotes, and with a dataset of almost a million wells, it cannot plausibly be considered "anecdotal." There are three comprehensive studies about hydraulic fracturing: a 2004 study by the EPA; a second, earlier, study about the same subject as the EPA study (groundwater safety as it relates to about hydraulic fracturing in coal methane beds); and the third study, mentioned above, by the Interstate Commission. My article mentioned and discussed only the third study. Lustgarten writes: The drilling industry, echoed by Kopel, cites three documents when asserting the environmental safety of hydraulic fracturing. They are a 2004 EPA study (PDF), a 2002 survey of state agencies (PDF) by the Interstate Oil and Gas Compact Commission and a similar survey in 1998 by the Ground Water Protection Council (PDF). In its Nov. 13 article, ProPublica detailed flaws in the EPA study and reported that the two surveys were "anecdotal," meaning that they included none of the basic data required to qualify as a scientific study. To say the least, that's an extremely idiosyncratic meaning of "anecdotal." The dictionary defines "anecdotal" as "based on personal observation, case study reports, or random investigations rather than systematic scientific evaluation." "Anecdotal" is an accurate description of Lustgarten's article, which examines a few cases of alleged contamination. There's nothing wrong with anecdotal news stories. "Anecdotal" is not an accurate description of the Interstate Commission study, which has no anecdotes, and which collected decades of data from 28 state regulatory agencies. Now, we find that Lustgarten apparently has his own definition of "anecdotal"--that is, something is "anecdotal" if does not include "the basic data required to qualify as a scientific study." Perhaps at some time Lustgarten will explain what basic data he thinks were lacking from the two studies. As a media critic, I would not have criticized him for offering plausible critiques of the studies. His article, however, did not contain any argument about what data he thought were missing, and his characterization of the studies as "anecdotal" was false and misleading--at least for readers who understand the word to mean what the dictionary says it means. Today I received this e-mail from Mark Thiesse, a Wyoming groundwater regulator who is quoted in Lustgarten's original article: I'd like to thank you for your recent editorial on the ProPublica article. I was one of the folks (I'm with the WY Dept of Env Quality) interviewed for this article by Mr. Lustgarten. I spent several hours on the phone and around a dozen follow up emails to try and help him write a factual article. Unfortunately he seemed to have his own agenda. The one error that was most blatant from my perspective was the "20 mile long plume" that he mentions. I must have told him 5 times that it was individual impacts to separate water wells due to water well drilling practices -- not related to oil and gas drilling at all -- but that did not make it into his article that way. If you had to make an important decision, would you rely on the factual information in a ProPublica article? I have only studied one article from ProPublica in detail, but the organization's implausible efforts to defend the validity of a wildly inaccurate article would make me hestitate to rely on anything from ProPublica.