The Volokh Conspiracy - Global Warming and Hurricanes, Problems in Recent Research.--

After my post on ABC's coverage of the threat of global warming, I read through some interesting bits of the evidence on issues I raised. Several commenters pointed me to an interesting blog, Real Climate, which seemed to be a reasonable treatment of issues written by those who generally accept the orthodox view of man-made global warming. [In the comments, Bruce McCullough points me to a blog that disagrees with the prevailing orthodoxy, Climate Audit.]

Real Climate pointed to an interesting scholarly article on the hurricane debate: Curry, Webster, & Holland, "Mixing Politics and Science in Testing the Hypothesis That Greenhouse Warming Is Causing a Global Increase in Hurricane Intensity" in the current Bulletin of the American Meteorological Society (BAMS). Although I found the paper mostly persuasive that there had been a very large increase in category 4 hurricanes since 1970 (with drops or no change in the other categories of hurricanes), I see three problems with the paper.

Starting at 1970. First, the paper dismisses concerns that the choice of 1970 as a starting point may give a misleading account because of the evidence that there was global cooling from 1940 to 1970. They [combine] ~~treat~~ this legitimate concern as [with] a logical fallacy [and attack that fallacy], but they never ~~explain~~ [deal adequately with the implications of] ~~coherently what's fallacious about~~ potentially choosing start or cut-off dates that are unrepresentative of larger trends or that give misleading measurements of the strength of any overall trend. [The authors justify their choice of 1970 because of the pooorer data quality before that date, which is fine, but they do not fully recognize the possible implications of that choice for the generalizability of their results.]

Double Counting 1994 Hurricanes? Second, if the authors actually did what they report having done with their data, then the BAMS paper should never have been published. In two charts showing the main hurricane trends, they report the data in five year periods, except for 1994, which is included in two periods, 1990-94 and the six-year period 1994-99. This may be just a typographical error, but they make this error three times in the paper (in most of the most important charts). And exactly the same error appears in another paper they published in Science in 2005 using related data, so it may well not be a typo. If these are not merely typographical errors, and they did what the article reports that they did (i.e., double counted 1994 hurricanes), then the paper should never have been published. [Just to be crystal clear, if I had to bet, I'd bet that the authors just made repeated typographical errors. And even if the error is substantive, obviously it wouldn't have been intentional.]

False Statements to the Public About Category-5 Hurricanes. Third, although the article ended with a substantial discussion of responsible argumentation over the issue of hurricanes and global warming in the mainstream press, as an apparent model they pointed to their own public commentary:

In our AAAS press release . . ., given the recent devastation associated with Hurricane Katrina, the main public message that we wanted to communicate was

The key inference from our study [in Science released along with the press release] of relevance here is that storms like Katrina should not be regarded as a "once-in-a-lifetime" event in the coming decades, but may become more frequent. This suggests that risk assessment is needed for all coastal cities in the southern and southeastern U.S. . . . The southeastern U.S needs to begin planning to match the increased risk of category-5 hurricanes.

Just to remind people: Katrina was a category-5 hurricane at its peak, but it was a category-3 hurricane when it hit the Gulf Coast, and it was only a category-1 hurricane at New Orleans (95 mph), though it was just below the threshold for a category-2 hurricane. The damage at New Orleans probably occurred, not because it was such an unusual hurricane but because the levees were in appalling condition (whether you fully buy Wizbang's provocative account or not).

But the data presented in the BAMS paper show what looks to be a very small and statistically insignificant rise in category-5 hurricanes from 1970 through 2005 (these data include some Pacific as well as some Atlantic hurricanes). The big increase shown in the BAMS paper is almost a tripling of category-4 hurricanes; other classes of hurricanes seem to show significant drops or no significant changes.

A 2005 Science article co-authored by the same group as the BAMS paper--Webster, Holland, Curry, & Chang, "Changes in Tropical Cyclone Number, Duration, and Intensity in a Warming Environment"--does look at Northern Atlantic hurricanes 1970-2004 separately from Pacific ones, but lumps category-4 and category-5 storms together, showing an increase for the combination, not reporting anything on category-5 hurricanes alone. I went to the data source cited in the 2005 Science paper and this is what I found for 1960-2004 hurricanes (the Science study covered 1970-2004, excluding the first two rows below and the 4 category-5 hurricanes that occurred after the period of their data, in 2005):

Category 5 Hurricanes in the North Atlantic:
1960-64 . . 4
1965-69 . . 2
1970-74 . . 1
1975-79 . . 2
1980-84 . . 1
1985-89 . . 2
1990-94 . . 1
1995-99 . . 1
2000-04 . . 2

As you can see, in the data they claimed to have used in their Science article (as I counted the events), there is absolutely no trend in category-5 hurricanes in the period of their study: 1970-2004. Indeed, the 1990s showed insignificantly fewer hurricanes than either the 1970s or 1980s. Thus, all of the increase in the North Atlantic category 4-5 storms reported in the 2005 Science article must be due to an increase category-4, not category-5 storms.

Neither paper reports any data that would show a statistically significant increase in category-5 storms that would form the scientific basis for their public claim, made along with their release of the 2005 Science article: "The southeastern U.S needs to begin planning to match the increased risk of category-5 hurricanes."

What increased risk?

If they have the data to support that claim, they should make it public. Anyone reading that claim would think that their Science paper showed such a significant increase. But it didn't. Even after I added the 2005 data on category-5 hurricanes, which they did not use because the season wasn't over yet, the quick regressions I ran didn't show any statistically significant increase in category-5 storms.

Did they just fabricate this claim of "increased risk" of category-5 storms?

If they don't have such data—-and it appears that they don't—-then it's irresponsible for a scientist to imply a scientific basis for such a fear-inducing claim released along with a scientific paper. And it's particularly odd that the authors of the 2006 BAMS paper actually discuss and criticize the mainstream press for poor environmental reporting that gives too much weight to critics of the environmental orthodoxy. The authors tell us that their scrupulousness put themselves at a disadvantage in public debate because they restricted themselves to making claims that were supported by peer-reviewed articles and data. Yet their own peer-reviewed data would seem to me to show that they had no scientific basis for saying that "The southeastern U.S needs to begin planning to match the increased risk of category-5 hurricanes."

Bottom line:

1. The new BAMS article shows persuasive evidence of a huge jump in category-4 hurricanes 1970-2004, but declines or flat trends in the numbers of stronger and weaker hurricanes.

2. The BAMS article does not deal adequately with whether its choice of a relative cool period (1970) as a starting time influenced the results.

3. In both their 2005 Science article and their 2006 BAMS article, the authors appear to double count data from 1994, but it may just be the result of repeated typographical errors in both journals.

4. In the BAMS article, the authors criticize others for irresponsible public statements on global warming and praise their own caution, yet the press release they quote asserts an "increased risk" of category-5 hurricanes threatening the southeastern U.S., but neither their own two articles, nor the data they claim to have used, show any such statistically significant trend.

5. If the quality of peer review and editing in this field is only as careful as it seems to be on the BAMS paper, then I think it prudent for educated lay people to continue to be skeptical about the research and public assertions of climate experts, especially those who tell you to just trust them or who insist that they are just relying on what their data show. Wouldn't expert reviewers of the BAMS paper already know that there had been no increase in category-5 hurricanes in the North Atlantic, and thus that the public statements that the authors proudly trumpet were irresponsible? Certainly, this brief foray into the literature leads me to be less confident of the conclusions of climate researchers, no matter how fervently they are asserted.

UPDATE: I emailed Judith Curry, the first author on the BAMS paper, pointing out the double listing of 1994 and requesting her 1970-2005 data by year by basin by hurricane category, which should allow me to resolve some oustanding questions.

There is a lot of back-and-forth discussion in the comments, particularly here, here, and here.

2d UPDATE: Judith Curry responded at RealClimate here and here. I have submitted a response, which has yet to appear there.

Overall, my criticisms seem to be pretty well confirmed. After thanking Judith Curry for her prompt response, I review where the exchange stands.

1. I am happy that you have confirmed the "1994" errors I found in your two papers, and I am also happy that they are typographical, rather than substantive.

2. You justify your choice of 1970 as the starting date for your study. My criticism was not whether the starting date was the best one (it probably was), but rather how choosing a relative minimum as the start date should affect the interpretation and generalizability of your results. I think we have both said our piece on this issue; I doubt that further discussion will resolve things any further.

3. I was pleased that you acknowledged that, as I had pointed out, the data on cat-5 hurricanes shows no significant trend, an observation that was the main focus of my comments (at realClimate.org post #212). On this point, you wrote:

With regard to category 5 storms, there is no point to trying to identify a trend only in the category 5 storms, owing to the small numbers and the uncertainties in classifying storms with the strongest windspeeds.

Of course, statistical significance is determined not only by sample size, but by the strength of any relationship and the variability in the data. From eyeballing your data, it appears that, if the relationship for cat-5 storms worldwide had been as strong as the powerful relationship that you nicely established for cat-4 storms, then the sample size of cat-5 storms was probably large enough to show statistically significant results. So the fact that there is no significant trend in cat-5 storms is due to the absence of a strong relationship in the data, as much or more than to the relatively small sample sizes.

As for my criticizing your assertion of an increase in the risk of cat-5 storms, which you now acknowledge that your data do not show, you wrote:

In terms of our statements in the press release issued in sept 2005, specifically the Q&A from Science, we had NO IDEA that anyone would even pay much attention this, particularly a year later. We never anticipated this paper and subject to become such a big deal. We were as careful as we could be (given our inexperience with press releases and media in general, and specifically were very careful not to overplay the global warming issue). . . .

The key statement of contention was in our Q&A with Science, which was made in response to the following direct question:

Given that increased intensity, what are the possible ramifications for policy-makers, generally? Would your research have any bearing, at least potentially, on decisions regarding the rebuilding of New Orleans and Gulf Coast?
It is not a statement that we made in our Science paper, nor in the news release issued from Georgia Tech. Rather, it was a statement made on a short time fuse at a request from Science, to respond to policy makers and to the catatstrophe [sic] in New Orleans. In hindsight, I don't thing we would change a word that was said. What is said by a scientist in peer reviewed publications in done in a very different context from interacting with the media. Scientists are expected to respond to questions outside of their field that they are ill prepared to deal with, and do not directly follow from their scientific research. This is the so-called value gap between scientists and policy makers that was addressed in the BAMS article.

I did not rummage through your group's responses to the press to unearth something your co-authors said in dealing with the press back in 2005. In your 2006 BAMS article, which I was commenting on, you purported to quote just three sentences from your group's 2005 dealings with the press. These were the sentences that your new 2006 paper chose to emphasize. These were the three sentences that I quoted, taking issue with the last sentence:

"The southeastern U.S. needs to begin planning to manage the increased risk of category 5 hurricanes."

Since, as you now kindly acknowledge, your data show no increased incidence of cat-5 hurricanes--indeed, you argue that "there is no point to trying to identify a trend only in the category 5 storms"--I objected to your group's use of the phrase "the increased risk of category 5 hurricanes."

My criticism thus seems eminently sound, with one strange, but important caveat.

Judith, you write that the passage I criticize was said in response to a question posed by Science about precautions for hurricanes and you quote those questions. But the phrase that I objected to--"the increased risk of category 5 hurricanes" --does not appear in the interview you referenced. Webster referred instead to "the risk of category 5 hurricanes." In the interview, Webster later refers to "this increased risk," but in context he appears to be referring to a range of risks from cat-3 through cat-5 that the public needs to address, so there is nothing objectionable about such a reference.

Judith, can you point me to any place where Webster or anyone in your group actually uttered or wrote the words that you quote in your 2006 BAMS article?

Or did someone (mistakenly) rewrite Webster's answer to add a scientific assertion that is not supported by the data you present (i.e., "the increased risk of category 5 hurricanes")?

You write: "In hindsight, I don't thing [sic] we would change a word that was said." But if you just pointed us to the right source for the text you quoted, it appears that someone (mistakenly) did "change a word that was said"--and changed it to something unwarranted by the scientific data your group presents.

Is there any way to correct the online version of the 2006 BAMS paper so that the charts are labeled correctly and so that you don't make a claim of an "increased risk of category 5 hurricanes" if Webster didn't actually say that in the passage you purport to quote.

	[Jim Lindgren, September 1, 2006 at 8:48pm] Trackbacks Global Warming and Hurricanes, Problems in Recent Research.-- After my post on ABC's coverage of the threat of global warming, I read through some interesting bits of the evidence on issues I raised. Several commenters pointed me to an interesting blog, Real Climate, which seemed to be a reasonable treatment of issues written by those who generally accept the orthodox view of man-made global warming. [In the comments, Bruce McCullough points me to a blog that disagrees with the prevailing orthodoxy, Climate Audit.] Real Climate pointed to an interesting scholarly article on the hurricane debate: Curry, Webster, & Holland, "Mixing Politics and Science in Testing the Hypothesis That Greenhouse Warming Is Causing a Global Increase in Hurricane Intensity" in the current Bulletin of the American Meteorological Society (BAMS). Although I found the paper mostly persuasive that there had been a very large increase in category 4 hurricanes since 1970 (with drops or no change in the other categories of hurricanes), I see three problems with the paper. Starting at 1970. First, the paper dismisses concerns that the choice of 1970 as a starting point may give a misleading account because of the evidence that there was global cooling from 1940 to 1970. They [combine] ~~treat~~ this legitimate concern as [with] a logical fallacy [and attack that fallacy], but they never ~~explain~~ [deal adequately with the implications of] ~~coherently what's fallacious about~~ potentially choosing start or cut-off dates that are unrepresentative of larger trends or that give misleading measurements of the strength of any overall trend. [The authors justify their choice of 1970 because of the pooorer data quality before that date, which is fine, but they do not fully recognize the possible implications of that choice for the generalizability of their results.] Double Counting 1994 Hurricanes? Second, if the authors actually did what they report having done with their data, then the BAMS paper should never have been published. In two charts showing the main hurricane trends, they report the data in five year periods, except for 1994, which is included in two periods, 1990-94 and the six-year period 1994-99. This may be just a typographical error, but they make this error three times in the paper (in most of the most important charts). And exactly the same error appears in another paper they published in Science in 2005 using related data, so it may well not be a typo. If these are not merely typographical errors, and they did what the article reports that they did (i.e., double counted 1994 hurricanes), then the paper should never have been published. [Just to be crystal clear, if I had to bet, I'd bet that the authors just made repeated typographical errors. And even if the error is substantive, obviously it wouldn't have been intentional.] False Statements to the Public About Category-5 Hurricanes. Third, although the article ended with a substantial discussion of responsible argumentation over the issue of hurricanes and global warming in the mainstream press, as an apparent model they pointed to their own public commentary: In our AAAS press release . . ., given the recent devastation associated with Hurricane Katrina, the main public message that we wanted to communicate was The key inference from our study [in Science released along with the press release] of relevance here is that storms like Katrina should not be regarded as a "once-in-a-lifetime" event in the coming decades, but may become more frequent. This suggests that risk assessment is needed for all coastal cities in the southern and southeastern U.S. . . . The southeastern U.S needs to begin planning to match the increased risk of category-5 hurricanes. Just to remind people: Katrina was a category-5 hurricane at its peak, but it was a category-3 hurricane when it hit the Gulf Coast, and it was only a category-1 hurricane at New Orleans (95 mph), though it was just below the threshold for a category-2 hurricane. The damage at New Orleans probably occurred, not because it was such an unusual hurricane but because the levees were in appalling condition (whether you fully buy Wizbang's provocative account or not). But the data presented in the BAMS paper show what looks to be a very small and statistically insignificant rise in category-5 hurricanes from 1970 through 2005 (these data include some Pacific as well as some Atlantic hurricanes). The big increase shown in the BAMS paper is almost a tripling of category-4 hurricanes; other classes of hurricanes seem to show significant drops or no significant changes. A 2005 Science article co-authored by the same group as the BAMS paper--Webster, Holland, Curry, & Chang, "Changes in Tropical Cyclone Number, Duration, and Intensity in a Warming Environment"--does look at Northern Atlantic hurricanes 1970-2004 separately from Pacific ones, but lumps category-4 and category-5 storms together, showing an increase for the combination, not reporting anything on category-5 hurricanes alone. I went to the data source cited in the 2005 Science paper and this is what I found for 1960-2004 hurricanes (the Science study covered 1970-2004, excluding the first two rows below and the 4 category-5 hurricanes that occurred after the period of their data, in 2005): Category 5 Hurricanes in the North Atlantic: 1960-64 . . 4 1965-69 . . 2 1970-74 . . 1 1975-79 . . 2 1980-84 . . 1 1985-89 . . 2 1990-94 . . 1 1995-99 . . 1 2000-04 . . 2 As you can see, in the data they claimed to have used in their Science article (as I counted the events), there is absolutely no trend in category-5 hurricanes in the period of their study: 1970-2004. Indeed, the 1990s showed insignificantly fewer hurricanes than either the 1970s or 1980s. Thus, all of the increase in the North Atlantic category 4-5 storms reported in the 2005 Science article must be due to an increase category-4, not category-5 storms. Neither paper reports any data that would show a statistically significant increase in category-5 storms that would form the scientific basis for their public claim, made along with their release of the 2005 Science article: "The southeastern U.S needs to begin planning to match the increased risk of category-5 hurricanes." What increased risk? If they have the data to support that claim, they should make it public. Anyone reading that claim would think that their Science paper showed such a significant increase. But it didn't. Even after I added the 2005 data on category-5 hurricanes, which they did not use because the season wasn't over yet, the quick regressions I ran didn't show any statistically significant increase in category-5 storms. Did they just fabricate this claim of "increased risk" of category-5 storms? If they don't have such data—-and it appears that they don't—-then it's irresponsible for a scientist to imply a scientific basis for such a fear-inducing claim released along with a scientific paper. And it's particularly odd that the authors of the 2006 BAMS paper actually discuss and criticize the mainstream press for poor environmental reporting that gives too much weight to critics of the environmental orthodoxy. The authors tell us that their scrupulousness put themselves at a disadvantage in public debate because they restricted themselves to making claims that were supported by peer-reviewed articles and data. Yet their own peer-reviewed data would seem to me to show that they had no scientific basis for saying that "The southeastern U.S needs to begin planning to match the increased risk of category-5 hurricanes." Bottom line: 1. The new BAMS article shows persuasive evidence of a huge jump in category-4 hurricanes 1970-2004, but declines or flat trends in the numbers of stronger and weaker hurricanes. 2. The BAMS article does not deal adequately with whether its choice of a relative cool period (1970) as a starting time influenced the results. 3. In both their 2005 Science article and their 2006 BAMS article, the authors appear to double count data from 1994, but it may just be the result of repeated typographical errors in both journals. 4. In the BAMS article, the authors criticize others for irresponsible public statements on global warming and praise their own caution, yet the press release they quote asserts an "increased risk" of category-5 hurricanes threatening the southeastern U.S., but neither their own two articles, nor the data they claim to have used, show any such statistically significant trend. 5. If the quality of peer review and editing in this field is only as careful as it seems to be on the BAMS paper, then I think it prudent for educated lay people to continue to be skeptical about the research and public assertions of climate experts, especially those who tell you to just trust them or who insist that they are just relying on what their data show. Wouldn't expert reviewers of the BAMS paper already know that there had been no increase in category-5 hurricanes in the North Atlantic, and thus that the public statements that the authors proudly trumpet were irresponsible? Certainly, this brief foray into the literature leads me to be less confident of the conclusions of climate researchers, no matter how fervently they are asserted. UPDATE: I emailed Judith Curry, the first author on the BAMS paper, pointing out the double listing of 1994 and requesting her 1970-2005 data by year by basin by hurricane category, which should allow me to resolve some oustanding questions. There is a lot of back-and-forth discussion in the comments, particularly here, here, and here. 2d UPDATE: Judith Curry responded at RealClimate here and here. I have submitted a response, which has yet to appear there. Overall, my criticisms seem to be pretty well confirmed. After thanking Judith Curry for her prompt response, I review where the exchange stands. 1. I am happy that you have confirmed the "1994" errors I found in your two papers, and I am also happy that they are typographical, rather than substantive. 2. You justify your choice of 1970 as the starting date for your study. My criticism was not whether the starting date was the best one (it probably was), but rather how choosing a relative minimum as the start date should affect the interpretation and generalizability of your results. I think we have both said our piece on this issue; I doubt that further discussion will resolve things any further. 3. I was pleased that you acknowledged that, as I had pointed out, the data on cat-5 hurricanes shows no significant trend, an observation that was the main focus of my comments (at realClimate.org post #212). On this point, you wrote: With regard to category 5 storms, there is no point to trying to identify a trend only in the category 5 storms, owing to the small numbers and the uncertainties in classifying storms with the strongest windspeeds. Of course, statistical significance is determined not only by sample size, but by the strength of any relationship and the variability in the data. From eyeballing your data, it appears that, if the relationship for cat-5 storms worldwide had been as strong as the powerful relationship that you nicely established for cat-4 storms, then the sample size of cat-5 storms was probably large enough to show statistically significant results. So the fact that there is no significant trend in cat-5 storms is due to the absence of a strong relationship in the data, as much or more than to the relatively small sample sizes. As for my criticizing your assertion of an increase in the risk of cat-5 storms, which you now acknowledge that your data do not show, you wrote: In terms of our statements in the press release issued in sept 2005, specifically the Q&A from Science, we had NO IDEA that anyone would even pay much attention this, particularly a year later. We never anticipated this paper and subject to become such a big deal. We were as careful as we could be (given our inexperience with press releases and media in general, and specifically were very careful not to overplay the global warming issue). . . . The key statement of contention was in our Q&A with Science, which was made in response to the following direct question: Given that increased intensity, what are the possible ramifications for policy-makers, generally? Would your research have any bearing, at least potentially, on decisions regarding the rebuilding of New Orleans and Gulf Coast? It is not a statement that we made in our Science paper, nor in the news release issued from Georgia Tech. Rather, it was a statement made on a short time fuse at a request from Science, to respond to policy makers and to the catatstrophe [sic] in New Orleans. In hindsight, I don't thing we would change a word that was said. What is said by a scientist in peer reviewed publications in done in a very different context from interacting with the media. Scientists are expected to respond to questions outside of their field that they are ill prepared to deal with, and do not directly follow from their scientific research. This is the so-called value gap between scientists and policy makers that was addressed in the BAMS article. I did not rummage through your group's responses to the press to unearth something your co-authors said in dealing with the press back in 2005. In your 2006 BAMS article, which I was commenting on, you purported to quote just three sentences from your group's 2005 dealings with the press. These were the sentences that your new 2006 paper chose to emphasize. These were the three sentences that I quoted, taking issue with the last sentence: "The southeastern U.S. needs to begin planning to manage the increased risk* of category 5 hurricanes."* Since, as you now kindly acknowledge, your data show no increased incidence of cat-5 hurricanes--indeed, you argue that "there is no point to trying to identify a trend only in the category 5 storms"--I objected to your group's use of the phrase "the increased risk of category 5 hurricanes." My criticism thus seems eminently sound, with one strange, but important caveat. Judith, you write that the passage I criticize was said in response to a question posed by Science about precautions for hurricanes and you quote those questions. But the phrase that I objected to--"the increased risk of category 5 hurricanes" --does not appear in the interview you referenced. Webster referred instead to "the risk of category 5 hurricanes." In the interview, Webster later refers to "this increased risk," but in context he appears to be referring to a range of risks from cat-3 through cat-5 that the public needs to address, so there is nothing objectionable about such a reference. Judith, can you point me to any place where Webster or anyone in your group actually uttered or wrote the words that you quote in your 2006 BAMS article? Or did someone (mistakenly) rewrite Webster's answer to add a scientific assertion that is not supported by the data you present (i.e., "the increased risk of category 5 hurricanes")? You write: "In hindsight, I don't thing [sic] we would change a word that was said." But if you just pointed us to the right source for the text you quoted, it appears that someone (mistakenly) did "change a word that was said"--and changed it to something unwarranted by the scientific data your group presents. Is there any way to correct the online version of the 2006 BAMS paper so that the charts are labeled correctly and so that you don't make a claim of an "increased risk of category 5 hurricanes" if Webster didn't actually say that in the passage you purport to quote.