[Note partial correction below, noted in the UPDATE.]
I know next to nothing about climate change questions, which is why I almost never blog about them. But the recent discussion of the East Anglia CRU e-mails touches on a matter that I have studied a bit, though not much — standards related to the sharing of raw data by researchers.
As I understand, and please correct me if I’m wrong, the CRU researchers apparently refused to share the underlying raw data that was the basis of their research. Is that so? If so, is there a scientifically acceptable justification for this refusal, especially when the topic is as important as this one?
My inclination would be to say that data should nearly always be shared. If you share your data, this lets others check the conclusions you draw from the data, as well as verifying the accuracy of the data against other available sources. They might disprove your arguments, or lead you to improve your arguments, or, if they reproduce your results, they might help prove the validity of your arguments. But in either case, science progresses better, and the decisions made based on the science are more reliable, than if you keep the data secret.
I can see some possible exceptions, for instance where there are concerns about the privacy of research subjects, or where the data was gathered as part of a commercial endeavor that requires that the data be kept proprietary for the commercial project to be viable, or some such. Even there I stress that the exceptions are merely possible; perhaps on balance the data should be shared even then, and in any event, even if there’s a good reason for the data not to be shared, people should view the research skeptically because of the lack of sharing.
But in any event, I don’t quite see why any such possible exceptions would apply here. The explanations that I’ve sometimes heard, which are that (1) the data might be misused by people who would make misleading arguments based on them, and (2) the initial gatherers of the data didn’t authorize the CRU to release it, strike me as unsound. Argument 1 does not seem acceptable for scientific research, since the proper scientific way to fight bad arguments is by responding with good ones, and not denying everyone access to the important data that would let them check your work. And argument 2 strikes me as implausible on the facts, since it’s not clear why the initial gatherers of the data would see it as any sort of secret. [UPDATE: It seems, as I note in my follow-up post, that some government agencies that gather meteorological data may indeed have insisted that CRU not share the data, apparently because they make money by selling it. But, as I argue in that post, that insistence strikes me as unjustifiable, given that the countries in which the data is gathered likely have vastly more to gain from better science that stems from more data sharing than they have to lose from lost climate data licensing revenue.]
Am I missing some important justification for keeping the data confidential in this case? Or am I misunderstanding the situation, and the data was indeed made available to everyone? I am not looking for generalities about the alleged sins or virtues of various schools of thought on climate change, or even arguments about whether climate is indeed changing. My question is solely limited to the question of the scientific ethics related to data sharing; and it would be particularly useful to hear from scientists who have actually dealt with such questions in their own work.
Kevin says:
Failure to share the data, the adjustment techniques and he code used to generate the predictions is a huge red flag.
On a somewhat related note there have been claims that publishing the emails and data from the CRU were an invasion of privacy. What do you think? These people were working for a government agency, using government research funds to contact other government employees about their publicly funded work using government emails accounts etc. I think there is a pretty strong case that these emails and research data were public property and not private. Further there seem to have been at least some attempts by some of those asserting the privacy to avoid related FOI requests. Presumably there is some law against disclosing them without the proper authorization, but is it really breach of privacy?
Quote
November 27, 2009, 4:46 pmrpt says:
This is an excellent question, and one that I (not a scientist and not a partisan in this debate) have asked in the other posts from a different perspective. Perhaps we’ll get an answer to both now.
Now that the review of arguably credibility damaging internal emails—and suppressed raw data —is deemed essential if not dispositive in the evaluation of the respective sides of this debate, and I presume there are more than two, will other parties in the debate also share their internal communications and raw data, without claims of trade secrets, attorney-client privilege, proprietary information, and so on. Will other academics do so? What about trade and petro– and petro-chemical industry groups? Even damaging attorney-client communications can be, but are not always, suppressed if the disclosure was inadvertent. Can one presume that those who fail to disclose are hiding their own problems?
Can someone address this is a neutral way?
Quote
November 27, 2009, 4:47 pmrpt says:
“Failure to share the data, the adjustment techniques and he code used to generate the predictions is a huge red flag.”
Does this work both ways? Does it matter that the CRU documents are deemed public and anti-AGW partisans are, I believe, by and large private or private-funded? What about anti-AGW academics? Recipients of government grants?
Quote
November 27, 2009, 4:50 pmCurlyDave says:
I am a Ph.D. Engineer and worked in both the nuclear and defense industries for over 30 years.
While there are several legitimate reasons for not sharing data (U.S. Government classified, proprietary information, trade secret, etc.) none of them apply in this case. There is no excuse not to share the original data, and there is precious little excuse not to share the source code for the data reduction programs. Most real scientists would love to have a large number of other scientists look at their computer programs to ferret out all of the possible errors. Plus, considerable prestige attaches to being the original author of a widely used computer program.
You didn’t ask this question, but the nearly universal experience among scientists is that when someone claims that a particular theory is “settled science”, it is always for a political purpose and to discourage independent investigation.
While we need a way of weeding out the crackpots, all real science is always open for further investigation and discovery. Especially new areas which rely heavily on computer models.
Quote
November 27, 2009, 4:51 pmDavid Bernstein says:
I’ve been meaning to blog about the same issue, which is also what stood out for me about the controversy. I don’t think you’re missing anything, and, indeed, you are being far too easy on the offenders. People who withhold raw scientific data because they want to control the interpretation (and/or don’t want to be fact-checked) aren’t engaging in “science” at all, but are going through seemingly scientific motions to support a preconceived result.
Quote
November 27, 2009, 4:58 pmreasonable persons says:
You argue that justification (1) (the data might be misused by people who would make misleading arguments based on them) is not acceptable for scientific research since “the proper scientific way to fight bad arguments is by responding with good ones, and not denying everyone access to the important data that would let them check your work”.
The flaw in your agument is that the deniers like McIntyre are not interested in “scientific research” at all. The deniers are threatening the entire safety of the human race by refusing to accept the settled science. Deniers are not interested in “argument” — they just don’t understand science.
In fact, courageous and underappreciated scientists like Phil Jones, who have personally sacrificed so much to prove to us our impending doom unless we act NOW, are perfectly willing to share data with other qualified climate scientists. But the deniers are not real scientists — they are mainly either right-wingers, in the pay of big oil companies, or just are looking for rationalizations so they keep driving their Hummers. And if they are scientists, they are not climate scientists, so they do not even have the expertise to understand the data.
For example, there is no reason a physicist should have to share data with a flat-earthers that the earth is really round. It’s a waste of time. Similarly, should a biologist have to share data with a creationist?
Even when the data is shared, the clear history here is that deniers will confuse the populace by overemphasizing the importance of minor errors, and unfairly criticizing real climate scientists on blogs.
Thus, deniers and apologists for deniers should let the real climate scientists work in peace. We’re in a race for humanity’s survival here, and we don’t have the luxury of giving every crackpot a soapbox. There are only a few years to act — if we don’t cut carbon emissions drastically by then, the Earth could wind up looking like Venus. Do you want to sacrifice humanity on the alter of “scientific argument”?
Only climate scientists are qualified to interpret the raw data anyway, and to understand exactly what it represents and what corrections to make to it. The deniers aren’t even interested in the data per se, they just want to attack real climate science.
Quote
November 27, 2009, 5:02 pmDisintelligentsia says:
While many are disturbed that this incident gives a black eye to science and climatology in particular, I’m still hopeful that in the end it will create a benefit much greater than the current damage. Hopefully it will encourage a healthy introspection among those in science, foster an atmosphere of openness among researchers and a welcoming of unorthodox or oppositional views so that all issues can be thoroughly vetted without fear of demonization. Hopefully researchers will let the data speak for itself without preordaining an outcome which the data must be made to conform to.
Quote
November 27, 2009, 5:02 pmKevin says:
rpt says:
Yes, absolutely. If I tell you your research conclusions are wrong but I refuse to show you how I reached that conclusion, then my claim that you are wrong will carry very little weight. Back in grade school we all learned you have to show your work, this still applies even after you get a graduate degree and prestigious job.
Quote
November 27, 2009, 5:02 pmef says:
The most common explanation I have heard from the pro climate change researchers is that much of the data was provided to them under the condition that it not be shared further leaving them with no option but to withhold the data from public disclosure. That seems an odd condition to me given, as you mention, the importance of the subject. I would at least like them to publish those restrictive agreements they entered into so that skeptics could pursue access on their own, or pressure the sources to release the data for the sake of replicating the original work.
Quote
November 27, 2009, 5:05 pmKevin says:
reasonable persons says:
You forgot to close the sarcasm tag at the end of your post. The motives of those who attack (or promote) a scientific claim should not matter.
Also, given the sloppy statistics and other techniques of the ‘climate scientists’ it is good that people from outside that fraternity are looking at their work.
Quote
November 27, 2009, 5:11 pmSoronel Haetir says:
Except that they had absolutely no documentation of the confidentiality condition either.
When you start saying you’ll delete data rather than produce FOIA mandated material I hope you are treading into criminal territory.
Quote
November 27, 2009, 5:15 pmDerHahn says:
Though I’m generally not sympathetic to AGW proponents (and I’m delighted that evidence of the bias at CRU and other climate researchers made it into general circulation) I am inclined to think (2) the initial gatherers of the data didn’t authorize the CRU to release it... is plausible. CRU received data from a variety of sources, some national weather services and other researchers. It doesn’t seem unusual to think they provided it for CRU’s use without giving them the rights to release it to other parties.
Quote
November 27, 2009, 5:16 pmJohnF says:
Here’s a modification of the question: if one of these scientists were put on the stand to testify to his opinions about human-caused global warming, and he refused to disclose the data he was basing the opinion on, would he be allowed to testify under Daubert?
Quote
November 27, 2009, 5:18 pmef says:
Well, yeah, my guess is they made that part up too, but that remains (as of at least Wednesday afternoon) their primary excuse.
Quote
November 27, 2009, 5:21 pmAnon says:
I once asked a law professor for a copy of a data set he had created and used for some empirical research that ended up in a top law review. He said he was thinking about making it public but was not going to do so until he had written a couple more papers with it. I’m glad to see at least some law professors recognized that data should be made public.
Quote
November 27, 2009, 5:25 pmef says:
But would an agreement outside the normal framework of FOIA protected communications (attorney/client, contract negotiations) override the FOIA requirement to release the data? IOW, could a third party place extra-legal restrictions on the disclosure of the data and enforce those provisions on the (FOIA subject) party with whom they shared the information?
Quote
November 27, 2009, 5:26 pmAllan Walstad says:
There’s something to be said for being able to keep the data confidential for awhile, to confirm one’s results and perhaps “milk” the data for more publishable results. But if you are making a claim about public policy, surely the basis for the claim needs to be made public, short of, you know, CIA stuff. (And maybe there, too.) If you get data from people who have ownership and won’t let it be released, then do you really have anything to be taken seriously from a public policy (or even just vanilla scientific) standpoint?
Quote
November 27, 2009, 5:34 pmBuckland says:
reasonable persons says:
And I think that’s one misunderstood pieces of this puzzle. Most researchers are more than willing to share data, code and procedures with anyone who asks. But in fact nobody ever asks. When your research is on the mating habits of the Austrian tree frog the number of people who care are rare indeed. And those who do are part of the tribe that really cares about tree frog mating and will be constructive with comments. A researcher could easily stiff the local bait store owner that wants to see his tree frog data.
Believe it or not, climatology used to be such a sleepy science. Nobody cared or questioned papers written about long term trends. As someone who served in that field for 5 years in the 1980’s it was difficult to explain exactly what we did in that era. Most of the time I would change my title to weather forecaster instead of climatologist as it was better understood by those not in the field (and close enough for government work, literally). Indeed when the first of the global warming papers came out there was little interest. It wasn’t until Al Gore started pushing the issue that it became politicized. In the late 1990’s early 2000’s it was easy to put off guys that weren’t part of the club.
As happens often decisions made in an earlier era are coming back to haunt the makers. It was easy to stiff a Canadian businessman without the right background to assess the information. And once that pattern was set it’s really hard to back off. To release data after many refusals is an admission that the refusals were wrong. I think the importance of those early decisions are just now seeping into the inner sanctums of the anointed ones. And besides, great embarrassment comes when you can’t explain every jot and tittle in some really scrambled source code.
I don’t think the people involved in this planned for this to play out this way. They thought they were doing God’s work and any small lapses were made up for by the many things that they were doing right. Unfortunately the number of people who care about this type of research has mushroomed, literally from a few thousand to millions. People with far more expertise in computers and statistical measurements than the original researchers are checking the smallest detail. Which is a very good thing. The model that emerges will be much better than the ones now being used. It still may show a hockey stick, or it may not.
Jones and company would say they’re being pecked to death by a flock of geese. Another way of saying being slain by An Army of Davids
Quote
November 27, 2009, 5:36 pmSG says:
It’s been a while since I worked in an academic research environment, but my recollection was that while data was expected to be shared, there was some degree of professional courtesy extended to data requests. That is, someone might request data, but it wouldn’t necessarily expected to be released right away. The person/team that gathered the data (which was often time consuming and expensive) was allowed some exclusive time in order to wring all the publications they could from their data. This could last a year or more. But ultimately you were expected to release the data if someone requested it. Source code was another issue. That really was expected to be supplied upon request, and I was on both sides of that at different times.
All of this assumes you’ve published something.
Quote
November 27, 2009, 5:38 pmDavid Schwartz says:
I assume “reasonable persons” argument is satire, but the flaw in it is obvious. Without skeptics to point out the minor flaws, those minor flaws will never get corrected. Because the effect of AGW is so small compared to the error bars, one minor flaw is sufficient to flip the balance in any given data set or forecast.
But the argument is brilliant satire and points out how earnestly the advocates of AGW corrupt the scientific process with political considerations. At least the bad behavior of the AGW advocates can’t be laid at science’s door.
Failures of science, like Pons and Fleischman, are generally detected and corrected rapidly. It takes government and politics to create a massive screw up like CRU.
Keith Briffa, in an April 2007 email to Michael Mann.
Quote
November 27, 2009, 5:40 pmM.Gross says:
There are no scientific reasons not to disclose data.
There are only legal ones.
Most of the data in question was published in journals which require authors to make their data sets available. They should have either done so or never requested publication in the first place.
Quote
November 27, 2009, 5:40 pmpot meet kettle says:
this is not true.
there is a reason that has nothing to do with “controlling the interpretation” as db posits, or similar such conspiracy theories: if you have put in hard work to collect the data, and write the analysis scripts, it is purely the nature of the business that you don’t open up the crown jewels to late comers who get access to the data, or the code, or both for free, and a miniscule fraction of the effort and time that you put in. instead, you use the data to drive your paper publishing and similar record that you need to establish your reputation in the field.
this may not be pretty, but it is the way things often are in academia. this is nothing peculiar about climate research alone. in biology, you don’t put your mice cadavers out there for everybody if you have discovered a new hormonal pathway. you get all the papers you can out of it, and of course, it will take others, years to get to the point where they can compete with you in the field. it’s the basic competitive edge you need to maintain.
now, i don’t know the motives of the east anglia folks and whether they are as evil as db and others like to posit. it is possible, but it is hardly the only explanation. and not to excuse the east anglia folks, i bet that any presumed-private (a stupid assumption) email conversation about loaded topics where there is a lot of bad-faith public discourse will have incendiary sounding comments that might seem especially sinister, but it’s par for the course.
Quote
November 27, 2009, 5:41 pmForestGirl says:
The main reason scientists don’t like to share data is that they want to get as many publications out of the information as they can. Once the data are released, other people can begin to use them, potentially stealing your thunder and limiting the number of papers you can write.
Nevertheless, as a former government scientist, I can attest that in the United States, all data gathered with public money (i.e., those national weather stations) are technically in the public domain (and any publications written by govt scientists are immediately placed in the public domain, i.e., a copyright cannot be attached to them). There can be no legitimate refusal to make the data or models or source code public, but it is possible to drag out the process for a long time by forcing requesters to make use of the FOIA and by dragging your heels.
Quote
November 27, 2009, 5:42 pmLaura(southernxyl) says:
If you are going to assert something then you need to be able to back it up. I don’t care who you are.
If a creationist wants to see a biologist’s data, and the biologist declines on grounds that he’s about 1000 times smarter and the creationist just needs to quit being an idiot and take his word for it (which pretty much is what happens in a lot of cases) then no one should be surprised when the creationist doesn’t find the biologist’s arguments persuasive.
If the earth being round is going to impact the flat-earther’s way of life, make him drive lighter-weight and less safe cars, raise his taxes, drive the business that employs him out of work b/c they can’t afford the round-earth bills, then you are damn skippy the physicist should have to share data with the flat-earther.
What is with this arrogant attitude? It’s been my experience that people who know what they’re talking about and are confident about their data and their conclusions don’t act that way.
Quote
November 27, 2009, 5:42 pmHugh says:
On an even simpler level, would the opinions of the CRU team made analyzing this data be admissible as evidence? Would they even be able to establish the necessary foundation to get it admitted. I am pretty confident that if this were a situation involving business records I would be able to keep it out of evidence.
Darn, my volume of Evidentiary Foundations by Imwinkelreid is at the office. I love that book.
Quote
November 27, 2009, 5:43 pmNativeAmerican says:
Suffice to say that “politicized science” is exactly why Galileo was tortured and imprisoned. IMHO Scientist must hold themselves above the fray of politics. Holding out elitist ideologies that eliminate thorough scientific analysis is dangerous. If we lose our trust in science then civilization will begin to devolve.
I want to add, regardless of the truth of about manmade climate change; our environment deserves to be protected. Man and earth can live in harmony and still take advantage of modern conveniences. The fossil fuel dilemma is an old one, the world has built a great body of knowledge around sustainable technologies. Cap and Trade will not necessarily promote those to the forefront, nor will it stop companies from polluting. Often times well meaning legislation is short sighted and creates a larger problem than it solved.
Quote
November 27, 2009, 5:45 pmreasonable persons says:
No, you are wrong. The motives do matter. Here is why.
James Hansen in his testimony to Congress last year (reprinted in Andrew Revken’s article for The New York Times) stated that:
Hansen is the head of NASA’s climate change efforts. I suppose if you distrust Hansen, you are also distrusting NASA — do you think people who believe the moon landing was faked also need to be able to see all NASA’s data?
Hansen pointed out in his testimony, id., that climate change deniers — or CEOs of corrupt big oil companies who fund them — should be prosecuted for war crimes because
.
So the demand by climate change deniers that they want to “see more data” makes no sense. Every major scientific organization has already officially supported the official scientific pronouncements of climate scientists. There is nothing more to discuss, the science is settled. Deniers like McIntyre evade the peer review system, which is the gatekeeper of science, by going directly to the public, which lacks the climate science expertise of actual climate scientists, like Hansen.
As an analogy, when the Apollo 13 had an explosion, the astronauts famously called down for help to NASA, saying “Houston, we have a problem.” Those astronauts only had few days to get back to earth safely. Now, if a bunch of flat earthers and creationists had demanded the raw data from NASA at that time, that NASA prove the earth was not flat, that there really was a moon, all with the intent of distracting NASA, then NASA would have been right to refuse. NASA’s primary mission was rescuing the astronauts, not appeasing flat-earthers and deniers.
Likewise here. Earth — Gaia herself — is like a spaceship travelling through space. And the climate scientists have warned us “Houston, we have a problem”. Gaia has more time to be saved than Jim Lovell on Apollo 13 did — months, not days — but she does not have the luxury of endless argument, of demagoguery. The motives of the deniers — like Glenn Beck, Rush Limbaugh, Steve McIntyre, Richard Lindzen, big oil — are not to save Gaia, but to destroy her. Only the climate scientists can save her, and they must be allowed to act soon.
So yes, motives do matter.
Quote
November 27, 2009, 5:48 pmPeteP says:
“As I understand, and please correct me if I’m wrong, but the CRU researchers apparently refused to share the underlying raw data that was the basis of their research. Is that so?”
Yes, and in plain English, too.
” If so, is there a scientifically acceptable justification for this refusal, especially when the topic is as important as this one? ”
No — most espcialy when the data acquisition effort is publicly funded, and also subject to FOI ( British and American ) laws.
“My inclination would be to say that data should nearly always be shared.”
When said data was aqcuired via public funding, it is a legal REQUIREMENT that it be shared, as per FOIA’s. It is also the long standing fundamental basis of ‘good science’.
Quote
November 27, 2009, 5:51 pmD.R.M. says:
No.
No.
Arguments (1) and (2) in your post are the only arguments being made for keeping the data secret.
However, the arguments over whether it should be kept secret are all now moot. Because, you see, when the CRU was actually faced with an FOI (UK) request some time ago, it stated that the data had been “accidentally” destroyed. Accordingly, there is no data to share anymore. It was destroyed, just as the leaked email archive shows was planned.
Quote
November 27, 2009, 5:55 pmpot meet kettle says:
yes. collecting data is hard, especially in uncontrolled environments where it is hard to get a second stab at refining, polishing, or controlling the collection environment. oftentimes, even figuring out what data to collect is hard because of a variety of limitations. real life data is imperfect, with mistakes, errors and so on (for example, a sensor might go wonky and report garbage, or it might fail and not report data in some area, and so on). figuring out how to filter and analyze the data is hard. this is part of the intellectual capital a scientist builds while doing research, so to expect him or her to just give data is a bit naive.
if you are one of the econometrics folks who make their reputations by applying analysis to data sets and hoping for something with p-value less than 0.05 to pop out and voila, become a rock star (cough-cough Emily Oster cough-cough), that’s often different because most often the data is just there and it is a matter of applying some functions from S to it. but if you are a law and econ person who has to painstakingly tabulate material and collect data, that’s an entirely different matter. unless you want your contribution to be that you created a new dataset of X, Y or Z, whereas the big prize is often the conclusions you can keep mining from the data.
Quote
November 27, 2009, 5:57 pmHugh says:
reasonable person, I do not find your arguments persuasive.
Why do you keep waiving your hand? What are you, some kind of Jedi? Your mind tricks will not work on me!
Quote
November 27, 2009, 5:57 pmPeteP says:
RPT -
“Does this work both ways?”
Yes.
” Does it matter that the CRU documents are deemed public and anti-AGW partisans are, I believe, by and large private or private-funded?”
Yes.
“What about anti-AGW academics? Recipients of government grants?”
Apples and organges question(s).
Privately-funded research is not public property. Publicly funded data and research ARE public property. “Pro” or “Anti” does not enter into it.
Quote
November 27, 2009, 5:57 pmEli Rabett says:
The data comes from national meteorological organizations. Some of them maintain a proprietary interest (they sell the data or they want to be the ones who distribute it). Both are quite common in many areas besides weather data.
At this point, it is probably useful to print a comment that appeared at Real Climate from a scientist in Iceland, Halldór Björnsson
Last month’s furor about Keith Briffa not giving Steven McIntyre data that Briffa had from Russian collaborators is pretty much along the same lines. It turns out that a) the data was the property (intellectual and real) of the Russians, and not Briffa’s to share. b) Briffa had asked the Russians to share the data with McIntyre and they had, way back c) McIntyre was whining.
Almost all of the data used by the CRU IS available through NOAA.
The question of formal agreements is moot in a practical sense. If CRU broke even an informal agreement, they would not get future data.
Quote
November 27, 2009, 6:03 pmJimH says:
When the subject is addressed under the term “Peer Reviewed,” this can only be true and correct if the raw data, and the mathematic methods and calculations of how those conclusions were reached, are released. That is precisely what “Peer Review” is intended to convey; that both the data and the results can be verified and are replicable.
It surprises me to find that members of a “Law Blog” would even consider a proposition to the contrary. That would be similar to the announcement of a SCOTUS decision sans citation of the law addressed and the reasoning behind it. That would amount to a “Baseball scores” approach with the court simply saying “The Plaintiff won, 5 to 4; Court is Adjourned.”
Quote
November 27, 2009, 6:06 pmDavid Nieporent says:
I honestly can’t tell whether reasonable persons is a (a) satirist, (b) troll, or (c) nut.
Quote
November 27, 2009, 6:08 pmjccamp says:
Could anyone with the requisite background comment on the actual code and the much bandied-about “Harry Read Me” notes to himself from the last programmer (Is that correct?). Is it accurately described that entering almost any data set of random numbers — the ages of all the dogs on the block, numeric values for my great-grandparents’ middle names, etc — will still provide a solution that suggests fairly consistent global warming trends? Or is it more precise to say that no one really knows what the code might provide, since it is so unintelligible, but that it does deviate from computation and rely on unverified and untested tables for some questioned (and questionable) sub-sets of values (those pesky post-1960 things)?
I confess to a certain shattered arrogance about this. I was confident I would be able to follow the technical discussion, but was quickly disabused of that impression.
Addition/Edit: I neglected to add there may be more than my two suggested explanations. I would appreciate whatever is accurate.
Quote
November 27, 2009, 6:09 pmtamerlane says:
reasonable person says:
Your parody(?) is so perfect I’m unsure it is parody. Would you help me out here by posting whichever one of these sentences is truer: (1) Yes: I am parodying the AGW folks. (2) No: I am not parodying AGW folks. I really do believe that what I’ve written above are compelling arguments.
Quote
November 27, 2009, 6:15 pmconiston says:
Reasonable person (hmmm).
There is an error in every sentence you have written. Here are a few:
Jim Hansen is an astronomer not a climate scientist and not a statistician.
Steve McIntyre is not a denier. That is a provable falsehood which, if you had ever spent time on his blog, you would know was false. McIntyre has not evaded the peer reviewed system and has tried to get his work published for five years — without much success — in the big mainstream journals because, as we know now from the emails, Jones, Mann, Briffa, etc strong armed editors to get them to refuse to publish. But they were submitted and peer reviewed in minor journals — and the McIntyre McKitrick paper regarding the MBH hockey stick was was reviewed by Wegman (world class statistician) and he agreed with them that the hockey stick was not robust. (Stat speak for having NO validity). Steve releases all of his code. Richard Lindzen is one of our foremost climate scientists and also does the same. That’s what real scientists do.
Making wild and untrue claims about people with real accomplishments reflects badly on you.
Educate yourself. OK, I realise that the equations on the Climate Audit site will be way above your head, but if you stick with it — in a year or so you will understand the graphs. This is not a snarky comment. Without a background in maths it’s very tough — and the issues are very complex.
Last comment: okay, it’s snarky. Gaia??? Gaia?? Next you’ll be touting crystals.
Quote
November 27, 2009, 6:15 pmdanjchapman says:
“Reasonable persons” is a perfect example of why normal people don’t trust the AGW crowd. This guy is a nut. Why on earth would anyone listen to him or anyone associated with him?
And no, I don’t think it’s satire. Maybe I’m just too trusting, though.
Quote
November 27, 2009, 6:17 pmmariner says:
Objection (1) above was obviously correct.
To understand it better, consider the full-length form:
“The data might be misused by people who would make misleading arguments based on them, and we want to make sure they’re not caught at it.”
Quote
November 27, 2009, 6:22 pmLaura(southernxyl) says:
From one of the earlier threads:
Still ROFL.
Quote
November 27, 2009, 6:23 pmJones' Cell Mate says:
As Laura notes, Reasonable persons is being sarcastic– mocking the warmmongers if you will. And if I might, he’s a lot more interesting than our normal sarcastrians.
As to this, “[a]lmost all of the data used by the CRU is available through NOAA”- that’s great. It means that the conclusions of the CRU, as well as any derivative conclusions based on CRU data, are “almost” compatible with legitimate scientific inquiry.
So they got going for them, which is nice.
Quote
November 27, 2009, 6:28 pmDavid Bernstein says:
If material is FOIAable, and it’s being deleted rather than being turned over, yes, I think it’s reasonable to assume that the relevant “scientists” are hiding the data to control the interpretation, and not because they have a proprietary interest in the data (which they can no longer use since they have claimed that it’s been deleted).
Quote
November 27, 2009, 6:31 pmAllan Walstad says:
As I recall, neither the astronauts nor NASA were demanding massive interventions in the economy.
Quote
November 27, 2009, 6:37 pmEli Rabett says:
David, the term is FOI under British law, which is the applicable one and different in detail from the US FOIA.
Quote
November 27, 2009, 6:40 pmM.Gross says:
I’m not sure their claims about it being bound by private confidentiality agreements are any good. To begin with, most of the FOIA requests stem from papers published in journals which require archiving and sharing data requests. They already agreed to share the data there, they can’t very well hide behind the confidentiality agreements at this point.
Quote
November 27, 2009, 6:50 pmjuris imprudent says:
Eugene,
This is somewhat comparable to Kellermann’s “work” on gun control and his efforts to avoid releasing his data. Good scientific process is to let someone else check your work, in the CRU case both the data, and the reduction process for getting that data into the model, and finally the model itself.
It appears, from the programmer’s notes, that over the course of 3 years he could not replicate, internally, the original work. That tends to indicate that no one else would have much of a chance of doing so either. That means you have to take the original conclusion on faith, since you cannot independently validate that result based on re-running the data through the process/model. Now that in and of itself is embarrassing enough because it is very sloppy work. Depending on the extent of reliance on that result in subsequent works, a key chunk of the assumed validity of all the works is now thrown into question.
Again the parallel in the gun control research world is that you are left to wonder if we have left the realm of Kellermann or Wintemute (questionable work with debatable conclusions) and moved on to Bellesiles (flat out fraud).
Quote
November 27, 2009, 6:53 pmSG says:
If material is FOIAable, and it’s being deleted rather than being turned over, yes, I think it’s reasonable to assume that the relevant “scientists” are hiding the data to control the interpretation, and not because they have a proprietary interest in the data (which they can no longer use since they have claimed that it’s been deleted).
Furthermore, if they deleted it I think we’re justified in assuming the data would have been harmful to their case. It’s spoilation of evidence.
Quote
November 27, 2009, 6:56 pmDavid Bernstein says:
Oh, and to answer a previous question, absent exceptional circumstances, one couldn’t testify under Rule 702 (Daubert) if one refuses to reveal one’s underlying data. Rule 702 states:
If the expert refuses to reveal the underlying data, how is the judge to determine whether the data is “sufficient”, and/or whether the data has been applied “reliably to the facts of the facts” based on the underlying “principles and methods.”
Daubert is not the be all and end all, but since someone asked...
Quote
November 27, 2009, 7:09 pmpot meet kettle says:
first, no data has been lost or deleted, afaik.
as has already been pointed out by another commenter, the information is not releasable under foi (not foia) due to agreements with participating meteorological stations.
you might be well served to actually read up on the issues before attributing the worst intent to those whose opinions differ from yours, often for carefully considered reasons. for example, consider reading the relevant posts on real climate, as well as the responses to the comments.
honestly, these emails have been inflammatory, but there is absolutely nothing in them that talks about manufacturing evidence or a conspiracy by global warming fundamentalists. of course, that shouldn’t stand in the way of those who derive their opinions from soundbites.
Quote
November 27, 2009, 7:19 pmMark Buehner says:
Datasets that were processed and developed decades ago and that are now regarded as essential elements of the climate data record often contain elements whose raw data or metadata were not preserved (this appears to be the case with HADCRUT).
Professor Judy Curry School of Earth and Atmospheric Sciences Georgia Institute of Technology
link
Quote
November 27, 2009, 7:28 pmpot meet kettle says:
what does this have to do with the current discussion of scientists at the cru apparently deleting data subject to foi, or hiding data that they should be releasing?
btw, this is judith curry’s letter that you’re quoting from, just so people have context. i fully support her stance asking that people improve their standards and acknowledging the sad reality that one side in the debate has to be seen to be purer than caesar’s wife, so they can stand up to the barrage from the other side.
Quote
November 27, 2009, 7:42 pmJones' Cell Mate says:
This link, while an advocate piece, contains direct quotes regarding missing data. As well as a truly frightening quote from Phil Jones that was made publicly in 2004.
Quote
November 27, 2009, 7:53 pmDavid Bernstein says:
Here is how the article linked by JCM above starts:
Quote
November 27, 2009, 7:58 pmG says:
> (2) the initial gatherers of the data didn’t authorize the CRU to release it,
The horror, the HORROR I say, of date, time, place, and temperature released to the public! sheesh
Quote
November 27, 2009, 8:09 pmpot meet kettle says:
soundbites again to avoid actually seeing whether hadcru differs from gisstemp in any material way, i see :) or if conclusions made using hadcru have been disproved using gisstemp.
but whaddo i know, i am not a “professor” (just adopting your style of commenting and putting titles in quotes).
Quote
November 27, 2009, 8:16 pmSG says:
i fully support her stance asking that people improve their standards and acknowledging the sad reality that one side in the debate has to be seen to be purer than caesar’s wife, so they can stand up to the barrage from the other side.
I support her stance as well. But since it’s your side (or more accurately, you) who’s ignorant of the facts here (“first, no data has been lost or deleted, afaik.” Please, try to keep up...), I encourage you to heed the wise advice offered earlier in this thread:
Quote
November 27, 2009, 8:17 pmpot meet kettle says:
when you don’t have data, let the fact-free mudslinging begin! cheers, mate!
Quote
November 27, 2009, 8:18 pmrpt says:
This does not make sense to me. How then can anyone verify the conclusions of the privately funded researchers if htey are not re? Are they immune from the same kind of scrutiny? Please clarify.
Quote
November 27, 2009, 8:20 pmrpt says:
spelling and grammar correction...
“if they are not required to disclose their data”.......
Quote
November 27, 2009, 8:21 pmLeo Marvin says:
Mocking Reasonable Person is low hanging fruit. I’d appreciate someone who knows what they’re talking about explaining whether, and if so why, Eli Rabett’s comment, doesn’t answer EV’s question.
Quote
November 27, 2009, 8:21 pmrpt says:
Again, does this mean that no privately funded researcher will be qualified to testify?
Quote
November 27, 2009, 8:23 pmrc says:
“the initial gatherers of the data didn’t authorize the CRU to release it”
So... it’s ok to advocate civilization-crippling economic changes based on secret data, but we can’t check that data because someone might be out a coupla bucks in royalties?
I hereby declare imminent domain on all climate data that is used to advocate economic changes with costs in excess of $100 billion. I have spoken.
Quote
November 27, 2009, 8:30 pmpot meet kettle says:
can you please provide evidence that data under foi has been deleted? people in the know seem to believe that no crimes were committed, although there were exceedingly stupid and rash statements made. maybe you know differently?
Quote
November 27, 2009, 8:39 pmSG says:
when you don’t have data, let the fact-free mudslinging begin!
Lets have some data then. If you read up on the issue, there’s all kinds of information in these emails about how to avoid disclosing data, including a willingness to delete it, rather than let skeptics look at it.
Then this year, an FOIA request came to CRU for for the data. Here’s the timeline:
May 5 – FOI request
May 6 – CRU Acknowledgement
May 29 — “Can you delete any emails you may have had with Keith re AR4?” — Phil Jones
June 3 – CRU Refusal Notice
June 4 – Holland Appeal
June 20 – CRU Rejection of Appeal
c. August 12 — CRU announces that it no longer has the raw data, only the cooked data
So, data has been lost. At least emails were explicitly deleted under direction from Phil Jones. It is unproven, yet seems highly likely, that the loss of the raw data was not an accident.
So we’ve got a illegal conspiracy to destroy data (at least emails and quite likely the data itself) to avoid it being independently examined. What additional data do you think is germane?
Quote
November 27, 2009, 8:46 pmDavid Bernstein says:
Theoretically, a judge could take an expert’s word for it that the underlying data is sufficient, or accept an affidavit summarizing that data as enough to meet Rule 702, but given that the burden of persuasion re admissibility is on the proponent of the evidence, if I were the judge I’d say if you want to keep your data private, you have no business relying on it in court because I’d have no sound way of checking the reliability of the bases of your testimony.
[I’d add, though, that this is only true if the expert is relying on raw data. Once the expert is relying on published studies based on raw data, it’s very unlikely that a judge would demand to see the raw data, as opposed to assuming that if the study was published somewhere reputable, it has sufficient indicia of reliability to be relied upon in court. Interesting question whether this would apply in a climate-science related case, given the recent revelations, or whether the other side could successfully argue that the absence of the underlying data combined with intimations of a conspiracy to dominate the peer-review process means that one can’t simply rely on the published literature, taken at face value].
Quote
November 27, 2009, 8:49 pmBrett says:
So, assuming that B has agreed to a contractual restriction with A regarding the release of party A’s data, party B can release it if he thinks that A has no reason to believe that the data is secret?
Quote
November 27, 2009, 8:54 pmrc says:
David B: “if I were the judge I’d say if you want to keep your data private, you have no business relying on it in court because I’d have no sound way of checking the reliability of the bases of your testimony.”
In other words, if you can’t put up, then shut up.
I think this criteria should also be applied to the scientific debate surrounding climate change.
Quote
November 27, 2009, 9:00 pmpot meet kettle says:
My question and statement were very clear, so i will just repeat it since you seem to be answering something irrelevant.
can you please provide evidence that data under foi has been deleted? people in the know seem to believe that no crimes were committed, although there were exceedingly stupid and rash statements made.
thank you for admitting that your allegation about data is unproven. also, you have thrown in another allegation: that emails were explicitly deleted. do you have proof, especially when recipients of the email have denied either acknowledging or acting in response to it?
Quote
November 27, 2009, 9:04 pmGaussling says:
I understand your concern about a reluctance on the part of researchers to share data. As someone who generates research data on a regular though less cosmic basis, let me offer a bit of insight.
The form and quality of data may be subject to some type of alteration over the course of a research project. Raw data directly from an instrument often requires calibration and correction to afford greater linearity or to compensate for some kind of error. Data with low signal to noise ratio may require some form of workup to clean up the signal. Raw data at this phase may not be useful to outsiders unfamiliar with the equipment or the particulars of calibration and workup, or with the assumptions made in the design of the experiment.
Data that is of good quality and sufficient quantity may be in more or less continuous use by the workers who competed for the funds to obtain it and whose careers will rise or fall by it.
Here is an important point: Science is competitive. Scientists must compete for scarce research dollars. Science really is a publish or perish world. If you are busy trying to understand a phenomenon in competition with other groups, you do not help your fundability by making your data public before you submit a paper or papers. Resources are allocated largely on the basis of your past published research results.
Premature disclosure of your expensive and hard won data just allows others to steal your thunder. Nobody remembers the honorable sod who fell on his sword by disclosing his data into the public domain– we only remember the fellow who published first.
Certainly after the work is published one should be willing to disclose the data upon which their conclusions are based. And as alluded to, the refusal to release data for outside examination should be viewed with skepticism. But this should happen after the workers have had a fair chance to make their best case.
Quote
November 27, 2009, 9:06 pmJames Darby says:
This scientific dishonesty and manipulation of the temperature record destroys the entire basis of the EPA’s endangerment finding regarding carbon dioxide and other greenhouse gases, and obviously goes to the root of the IPCC reports relied on by it and court rulings upholding greenhouse gas nuisance lawsuits, as commentators like CBS News’ Declan McCullagh and Point of Law have noted.
As Point of Law observes,
“IPCC as well as the Environmental Protection Agency relied on the now beleaguered Climatic Research Unit. In a widely cited and excellent report, ‘Congress May Probe Leaked Global Warming E-Mails,’ at a CBS News blog, Declan McCullagh writes:
‘The leaked documents (see our previous coverage) come from the Climatic Research Unit of the University of East Anglia in eastern England. In global warming circles, the CRU wields outsize influence: it claims the world’s largest temperature data set, and its work and mathematical models were incorporated into the United Nations Intergovernmental Panel on Climate Change’s 2007 report. That report, in turn, is what the Environmental Protection Agency acknowledged it “relies on most heavily” when concluding that carbon dioxide emissions endanger public health and should be regulated.’”
Quote
November 27, 2009, 9:11 pmDishman says:
reasonable person wrote:
Hansen does not operate in accordance with NASA quality standards. If he did, he would have a “Software Assurance Classification Report” for both GCM Model E and GISTEMP. He does not.
Instead, he chooses to ignore the quality standards which made the moon landings possible.
Quote
November 27, 2009, 9:11 pmpot meet kettle says:
when recipients of the email have denied either acknowledging or acting in response to it?
i meant:
when recipients of the email have denied both acknowledging or acting in response to it?
Quote
November 27, 2009, 9:11 pmDavid Schwartz says:
If a private company wants to keep its data private, it likely can do so (until and unless legal process forces it open for some reason). But what that private organization cannot do is expect others not privy to the data to accept its conclusions. It is welcome to use that data for its own purposes which may give it a market advantage. However, it has no right to expect others to take their word for it when they make claims based on this secret knowledge.
They can, for example, make secondary arguments that they can back up with other sources of data. For example, a company can say “our secret stock purchasing timing algorithm results in our customers buying lower and selling higher”. In this case, they needn’t disclose the purchasing algorithm, but we’d be suspicious if they didn’t at least disclose comparison data of their customer’s buying and selling prices and industry averages.
And, of course, we’d be quite justified in being suspicious that their data and the way they processed it was heavily colored by their desire to get exactly that result. And the problem with AGW data is that there is plenty of space to control the inputs and the mechanics to get the desired results.
For example, the range of reasonable corrections (from highest justifiable to lowest justifiable) for the urban heat island effect alone is on the same order of magnitude as the total expected effect of AGW (according to even the largest predictions) on land surface temperatures today.
You would, however, be wise not to rely on the conclusions of others where those conclusions align with their self-interest. A barber will stretch the truth to claim you need a haircut.
Quote
November 27, 2009, 9:16 pmpot meet kettle says:
i guess there is some irony to be found in the completely baseless outraged protestations about dishonesty :)
it is a good time to be a denier, or should i be polite and say, skeptic? if you are a mediocre scientist or one far past your prime, it is certainly the better position to take rather than the far more plausible position of agw. it is surely a better guarantee of attention, and if you need it, funding.
Quote
November 27, 2009, 9:18 pmJohn Moore says:
I’m not going to bother to find the links. You can if you really care. In one case, in responding to the request for data, the researchers said that the original data had been lost because of limitations on data storage in the ‘80s (which, btw, sounds like total BS).
Second, if you look at the README notes, it is clear that whatever data they do have is in such a condition that they could not use it even under duress. If the meta-data necessary to interpret the data is lost, the data is as good as lost: it is useless.
Quote
November 27, 2009, 9:19 pmpot meet kettle says:
ok :) i guess we know what standards of evidence you have when it comes to yourself :)
so, they used the data, or didnt they?
the second problem with your soundbite based allegation is that harry read me has nothing to do with the hadcru data set. oops!
Quote
November 27, 2009, 9:23 pmzuch says:
Not really. Most scientists are not the world’s best computer science engineers, and don’t necessarily follow best coding practises. As long as what they wrote works, and they’re not being intentionally dishonest or making some serious misteak, I suspect they don’t have any particular desire to have CS students lecturing them on modular design, functional specifications, and best coding practises....
Witness, for example, the dorks lecturing them on what they wrote in the freakin’ comments in previous threads here....
Cheers,
Quote
November 27, 2009, 9:25 pmAaron Bergman says:
There’s plenty of data available. All the NASA data is available, for example, and all their code is, too. I believe this is the correct link. You could remove the entire workproduct of CRU from climate science, and very little would change. Just like you could complete remove Mann’s “hockey stick” from climate science and nothing would change. Read the IPCC reports. The case for climate change is vast, robust and includes replicated, independent analyses. The sort of picayune mau-mauing engaged in by McIntyre and his cohorts is akin to attacking an elephant with a pellet gun. It gets nowhere, and it wastes the elephant’s time.
Of course, it also has a tendency to piss off the elephant, and you can see a lot of that in the e-mails. It’s clear that Jones went too far when he advocated the deletion of things that could be covered under a FOI request (whether or not this deletion actually occurred is unknown as I understand it). On the other hand, take the papers he said he’d keep out of the IPCC report. They are in the final report. So, it ends up being a whole pile of nothing.
Scientists are, shockingly I’m sure, people. We don’t like having their integrity attacked. The idea that the entire edifice of climate science is some vast conspiracy to reduce us to a pre-industrial agrarian society is patently absurd. There is plenty of data available. The “decline” that was being “hidden” is, for example, a known problem with a particular subset of tree rings that has been discussed in the peer-reviewed literature for (I believe) over a decade. There are plenty of model codes available, too, that anyone can run. Given the absurdity of this position, then, the fallback position seems to be that scientists don’t question global warming because they’re afraid of losing grant money. This is the sort of critique that will cause the average scientist to laugh in your face. First of all, grants aren’t written about proving or disproving global warming; they’re instead written about studying a proxy, or improving code, or physics, or whatever. It’s the idea that’s important. But the really absurd thing is that nobody goes into science for the money. The pay sucks, the hours are long, you spend years in school and postdocs, you lose control over where you live, etc. etc.. Most scientists could be making better money in the private sector. People do science because they love doing science. Sure, some of them are dicks, some of them engage in bitter infighting and politics, and most of them are crap at communication, but you don’t do this unless you believe in the truth.
The unfortunate fact is that you could subject almost anyone’s personal correspondence to this sort of context-free parsing and obtain seeming breaches of ethics. That was the basis of the government’s case against Bear Stearns, for example. But the government lost that case, and the quote mining in these e-mails isn’t going to go anywhere either. Politics barely functions (if at all) being subjected to this sort of inane hyper-parsing. I worry about what will happen to science if scientists have to face those same pressures.
Quote
November 27, 2009, 9:31 pmsteve says:
In medical science it is not uncommon to not have access to basic data. You often get the data for clinical tests without ever seeing the basic science data. The research for some of our equipment is considered proprietary. The BIS monitor, which I use daily, has never had its methods of processing EEG signals published. We can only look at the data on its efficacy. Why not just call up a physics prof at your university if you want an answer?
Steve
Quote
November 27, 2009, 9:31 pmJohn says:
reasonable persons says:
Why not try Mr. McIntyre’s website http://www.climateaudit.org and see for yourself. Mr McIntype is scrupulous about posting both data and code to “show his work”.
Quote
November 27, 2009, 9:32 pmMark Buehner says:
Bottom line– why not just release the data, the metadata, etc, and lets see if Mann et als work can be reproduced. Isn’t that where we’re really at right now? Defenders are saying no data is ultimately missing, great, lets see it now and end the controversy.
On the other hand, if the goalposts start moving and data IS missing, I think we should establish right now that any studies based on unreproducible data is of no use.
If absolutely nothing else, the care for data and notation of metadata has proven to be god-awful. It appears the models themselves could be hopelessly cludged together. Lets take the opportunity to stop playing games and put everybodys cards on the table.
The idea that these scientists are just too important to deal with the needling of critics is total BS. If their science and methodology is sound, the only errors will be honest ones, and an honest scientist should be HAPPY to have anybody point out a flaw. Since when do scientists care about politics one way or the other? Truth is truth, no matter who points it out. Or isn’t it? Are the conclusions just too important to question?
Quote
November 27, 2009, 9:35 pmMark Buehner says:
Who cares? If Jack the Ripper pointed out an error in Einsteins calculations, he would thank him.
Quote
November 27, 2009, 9:37 pmpot meet kettle says:
he might be scrupulous about posting his data, but not scrupulous about actually using data properly. i don’t blame him. since he is a mediocrity, he has to massage the data to get anti global warming results so somebody will quote him and make him an expert.
mcintyre fail1
mcintyre fail2
mcintyre fail3
Quote
November 27, 2009, 9:37 pmzuch says:
That revanchist troglodyte scientist cabal is deliberately suppressing my alternative scribblings to Maxwell’s hokey equations. I knew it!!!! What we need are more two-handed scientists, able to appreciate the inventiveness, if not the accuracy, of my suggested replacements. Thanks, I really need the moral support in my lonely quest.
Cheers,
Quote
November 27, 2009, 9:40 pmpot meet kettle says:
psst.. don’t tell anybody, but mann’s work has been substantially reproduced using, shock, even different data sets, at least 20 times.
also, before getting too excited, let me repeat what i said before: hadcru is different from the data set represented by harry-read-me, which had the comments about the metadata.
Quote
November 27, 2009, 9:41 pmpot meet kettle says:
sorry, i meant: at least 10 times.
Quote
November 27, 2009, 9:44 pmKen Arromdee says:
There’s a difference between going into the field for the money, and doing unethical things for the money when you’re already within the field. Nobody goes into the retail store business for the money either, yet retail stores sometimes cheat customers.
At any rate, ideology is probably as big a motivator as money here.
Quote
November 27, 2009, 9:45 pmCalling it Black says:
A couple simple statements:
#1 — The “Climategate” files (both emails and programs) paint a clear picture of scientists who have left the field of science and entered politics.
#2 — If the climate scientists involved had operated in a transparent manner (publishing their data, metadata, methods, rational, and programs) climate change would be a scientific discussion rather than a political issue.
#3 — As a scientist, regarding research, if you can’t show me exactly how you got to your conclusion, you don’t have one.
#4 — “42” (The answer to the ultimate question of the universe and I’m not going to share my data or how I arrived at that conclusion. Maybe I can work with CRU? You are a “denier” if you dare question me or want to reproduce my results!)
Quote
November 27, 2009, 9:49 pmneurodoc says:
And that was clearly unreasonable on his part, less than entirely honest, somehow improper, or what?
Quote
November 27, 2009, 9:55 pmSG says:
can you please provide evidence that data under foi has been deleted? people in the know seem to believe that no crimes were committed, although there were exceedingly stupid and rash statements made.
This is the (as far as I know) uncontested sequence of events
1. Phil Jones says he’ll delete the data before releasing it under an FOI
2. An FOI arrives
3. Phil Jones sends email telling people to delete things
4. CRU announces the data is lost
Believe what you want, but denier sounds like a label that fits you pretty well. You can believe AGW to be true (I think it’s more likely true than not) without having to defend these people or their actions.
Quote
November 27, 2009, 9:56 pmJohn Moore says:
That’s an interesting queston now, isn’t it. The answer appears to be that they used it, once upon a time, and have been publishing based on their previous *results* rather than the data.
That they can’t find the data means that the science cannot be replicated.
QED
Oh? It was about CRU data. So which data do you think it was about?
Quote
November 27, 2009, 10:00 pmrc says:
In public and in politics, folks claim that AGW is settled, clear, established consensus science.
But when critics ask for the data, they’re stonewalled. Something about scientists needing to get their message out first... money to be made... prestige... etc.
So it boils down to this: “The world’s going to end, but I can’t show you why. So now you have to cripple your economy and give me a billion dollars. Haven’t you heard? This is the way science works!”
Quote
November 27, 2009, 10:01 pmAaron Bergman says:
Just coming in on the wires: The nice people at realclimate (Gavin Schmidt deserves a National Medal of Science for what’s he’s done over the past few days) have given us a nice repository of open datasets in climate science:
http://www.realclimate.org/index.php/data-sources/
I hope this allays whatever concerns our hosts might have.
Quote
November 27, 2009, 10:01 pmpot meet kettle says:
i have never defended these people or their actions. i have talked about the substance of the issue as opposed to the soundbite-based aggrieved protestations that seem to be par for the course. i do urge you to find evidence to the contrary :)
Quote
November 27, 2009, 10:02 pmAaron Bergman says:
At any rate, ideology is probably as big a motivator as money here.
Just so I understand what you’re saying here: except for maybe Richard Lindzen and some Pielke or another, they’re all green, lefty neo-luddites?
Quote
November 27, 2009, 10:04 pmMark Buehner says:
Substantially? Different data sets? How about simply reproducing exactly his own work using his original data and explaining his assumptions? In such a way that anyone able to crunch the numbers can do it?
Quote
November 27, 2009, 10:04 pmpot meet kettle says:
ROFLMAO!!!!!
i didn’t realize there was a “cru data” and there was “other data”. why mess with that unwieldy concept of “dataset” and all that, or bother trying to find out? and you feel qualified to have an opinion about gw :) and people are shocked by the tone of exasperation, frustration and irritation of the emails :)
Quote
November 27, 2009, 10:05 pm2¢ says:
For example, there is no reason a physicist should have to share data with a flat-earthers that the earth is really round.
But here we have flat Earthers and creationists refusing to share data that they say proves their case.
Quote
November 27, 2009, 10:05 pmCalling it Black says:
The scientific method relies upon reproducibility and even adversarial review. For this to function, data and methods must accompany conclusions. Without the data, methods, and rational you are publishing dogma, not science. In the scientific field the peer-reviewed journals are the medium for discussion.
(BTW — publishing in a peer-reviewed journal is just a beginning and just because a paper is published doesn’t make it “established science.” It makes it either a target or stepping stone in the scientific process.)
I am not an attorney. How does publishing in a law review journal relate to the advancement of law?
Quote
November 27, 2009, 10:06 pmpot meet kettle says:
again. please to read the papers before trotting out whatever blooming deliciousness mcintyre thought to excrete onto his blog :) the links in my mcintyre fail comment should be a good starting point. i’d be glad to talk to you after you have an informed viewpoint.
Quote
November 27, 2009, 10:06 pmSeamus says:
I trust that this is sarcasm.
Quote
November 27, 2009, 10:09 pmpot meet kettle says:
i agree and would go further. gavin schmidt’s patience and his excellent documentation over the years of the science and the bullshit is yeoman service.
Quote
November 27, 2009, 10:10 pmShelbyC says:
Reasonable Person, U DA MAN!!
Quote
November 27, 2009, 10:10 pmrc says:
pot meet kettle: “i do urge you to find evidence to the contrary :)”
I’ll add my $0.02 In science, the burden of proof is on the scientist. And when the scientist says ‘no I won’t release my data’ then ‘oops I deleted the data’ then ‘Bob developed a trick to hide the decline’... Things start sounding a little fishy.
Remember, this isn’t just baout fame or being able to compensate for research dollars spent– this is ‘consensus’ science that has long-ranging impact on the global economy.
Reading this thread, I have been surprised at the large amount of good reasons why data should be withheld... but none of them are persuasive in the face of the scope and consequences of the AGW debate.
The burden of proof is on the scientist.
Quote
November 27, 2009, 10:11 pmNick B says:
RE: pot meet kettle
So, the scientists themselves saying “please delete the original data” and “ok” are not sufficient proof that they deleted data? How many signed and notarized copies are required to constitute proof in your world? Possibly direct video footage of the deletion, or is that too easy to fake?
Nick
Quote
November 27, 2009, 10:13 pmElliot says:
We have known for years that climate researchers refused to release the data upon which they based their conclusions. Yet the House of Representatives passed a bill calling for a complete reordering of our economy based on these conclusions.
They didn’t want the data.
Sixty-five heads of state are going to Copenhagen hoping to enact a treaty greatly restricting economic activity.
They don’t want the data.
Obama is a leading cheerleader for climate treaties, cap and trade, punitive energy tax, and restricted fuel usage.
He doesn’t want the data.
The fact that the rest of us are not climate scientists doesn’t matter. We don’t have to be climate scientists to recognize a seriously flawed process.
Quote
November 27, 2009, 10:13 pmMark Buehner says:
Do you guys take classes together to learn dodging questions? I’m astonished, if I took you at your word it would make me extremely confused as to wy all these FOIA requests have been flying around, not to mention all these emails. Exactly what are the folks in these emails talking about hiding, since you assure me everything is freely available? Wow, what an unforced error risking jail time passing emails apparently conspiring to hide and even destroy evidence rather than release it, considering a simple email link or FTP site with the already available data would save all the trouble.
Or maybe what you are saying isn’t entirely true.
Quote
November 27, 2009, 10:15 pmCalling it Black says:
Quote
November 27, 2009, 10:16 pmJohn Moore says:
Re: Pot meets Kettle
If you mean to prove that the pro-AGW folks are arrogant fools, you are doing a good job by example.
If you mean to convince anyone of anything in the debate, you have failed utterly — providing nothing but sarcasm and arrogance.
You might try again.
Quote
November 27, 2009, 10:17 pmDan Hamilton says:
The only other policy argument where they refuse to share their data is gun control. The pro gun control people never seem to share their data. The anti gun control always seems to be willing to share.
Why does it seem that the side that is on the LEFT wants to hide their data? The pro-gun people know why. The data of the pro gun control people is always cherry picked, or down right BS. Looks like the AGW people are the same.
Why don’t people trust what the LEFT says their data shows? Because the LEFT believes the end justifies the means. If they have to fudge or even make things up that is alright because their cause is just. They want to Save the Earth or stop people getting killed by guns. The fact that what they want to do will NOT accomplise what they want doesn’t matter to them. Better to fail for the “Right” reasons then to succeed for the “Wrong” reasons.
If the AGW peopled realy believed in AGW they would be pushing Nuc Power like mad. Since they aren’t, they care more for their anti-Nuc policy then for stopping AGW.
When one side doesn’t want to share their data and the other side does I will always support the side willing to share the data.
When Lott shared his data on Concealed Carry and the Best the Left came up with was that there wasn’t a decline in crime with Cancealed Carry. Everyone KNEW that Lott was right.
If you will not share your data, you don;t trust your results OR you know your results are WRONG. It is as simple as that.
Quote
November 27, 2009, 10:17 pmAaron Bergman says:
Substantially? Different data sets? How about simply reproducing exactly his own work using his original data and explaining his assumptions? In such a way that anyone able to crunch the numbers can do it?
Yes. Quoting from the NRC Report p. 111:
They further say on p. 115
The IPCC AR4 (WG1, Chapter 6) is more up-to-date with an extensive discussion on tree rings and the like, but I went with the National Academies for some variety.
Quote
November 27, 2009, 10:20 pmpot meet kettle says:
fail. i provided links with detailed analysis of the evidence which i asked you to read. there is nothing ad hominem about that.
john moore, i assume this means that you will not humor my request to develop an informed opinion. that’s fine. i will note, however, that i have provided a variety of links with substantial data and analysis debunking the claims you tout, in addition to sarcasm and arrogance.
Quote
November 27, 2009, 10:20 pmneurodoc says:
I think this somewhat naive for a number of reasons. For one, it imagines that science is a selfless pursuit and the scientist with the data will want to share it all with the wider world as soon as possible to: i) allow whoever wants to scrutinize it and see whether the published conclusions stand up under scrutiny (something that journal reviewers are supposed to do): and, ii) promote the dissemination of knowledge. Well, the scientist may not be exceptionally selfless and may want to make exclusive use of the data for the time being. And it is very facile to say that good arguments must trump bad ones, so let everyone who wants to see the data see it.
All sorts of mischief is possible with scientific data, especially that collected through clinical trials of drugs and other medical interventions. And it can take an incredible amount of time and effort to combat misuse of the data and misinforming of the public about what the data shows or implies, with success by means guaranteed. The National Institutes of Health is very concerned about the quality of the research it supports, and takes steps to ensure it to the extent practical, but its grantees are not routinely required to disclose their data as a condition of their grant awards.
I can’t speak to how the climate data in question here has been handled, since I now next to nothing about the surrounding circumstances. I am speaking about medical research and the data generated is not shared like the financial data publically traded companies may be required to publish.
Quote
November 27, 2009, 10:23 pmShelbyC says:
pot, something about your tone tells me I probably won’t end up with an informed opinion after reading your links.
Quote
November 27, 2009, 10:29 pmMark Buehner says:
Again– different data, different studies that come to similar conclusions– that does NOT lend credibility to the inherent cohesion of a given study. Particularly when cherry picking data to meet (or set) the norm is in question.
Quote
November 27, 2009, 10:29 pmpot meet kettle says:
hilarious! i love how you arrived at an opinion without even looking at the links :)
Quote
November 27, 2009, 10:30 pmpot meet kettle says:
huh? where?
also for all those who have become free-the-data advocates, every supposedly controversial result has been replicated with multiple different data sources, not just one data source. the results have passed every reasonable test of reproducibility. there are good arguments for asking cru to release whatever data they can release legally or ethically without compromising their future access. that said, cru might also have good reasons not to release the data to protect their own edge in research.
but to somehow claim that the edifice of substantial research establishing agw stands or falls on the release of whatever data subsets are not available is pathetically misinformed or monumentally disingenuous, or as is often the case, both.
i understand this is not the claim ev is making, but it is what db as well as many of the commenters are gloating about, and it is a claim that is completely false.
Quote
November 27, 2009, 10:34 pmShelbyC says:
Glad you liked it, it was pretty easy. I simply read your comments and asked myself if this was a guy would give me the straight dish on the climate stuff.
Quote
November 27, 2009, 10:35 pmAaron Bergman says:
Again– different data, different studies that come to similar conclusions– that does NOT lend credibility to the inherent cohesion of a given study. Particularly when cherry picking data to meet (or set) the norm is in question.
This exactly misses the point. In a well-developed field, no one cares about any particular study. You have one you don’t like? Throw it out. The other studies will stand on their own. Different data and different studies coming to the same conclusion is the whole point of science.
So, you again fall back on conspiracy theories, saying that the scientists are cherry picking data. Most of the reconstructions have open data and the code is available. Nothing is being hidden here.
Quote
November 27, 2009, 10:36 pmpot meet kettle says:
good to know that ad hominem is the basis of your logical deduction process.
Quote
November 27, 2009, 10:37 pmJones' Cell Mate says:
What has been noticeably absent so far in the ClimateGate discussion is a public reaffirmation by climate researchers of our basic research values: the rigors of the scientific method (including reproducibility), research integrity and ethics, open minds, and critical thinking. Under no circumstances should we ever sacrifice any of these values; the CRU emails, however, appear to violate them.
In my opinion, there are two broader issues raised by these emails that are impeding the public credibility of climate research: lack of transparency in climate data, and “tribalism” in some segments of the climate research community that is impeding peer review and the assessment process.
But the broader issue is the need to increase the public credibility of climate science. This requires publicly available data and metadata, a rigorous peer review process, and responding to arguments raised by skeptics.
The proceeding quotations have been authored by Judith Curry, Chair, School of Earth and Atmospheric Sciences, Georgia Institute of Technology, subsequent to the release of the CRU material.
Quote
November 27, 2009, 10:41 pmJohn Moore says:
PMK — you assert, without any additional information, that the “harry” notes do not apply to the data used by CRU in their paleoclimate temperature reconstructions. Please provide more information. Oh, and if that isn’t what “harry” is talking about, just what data is it?
As to your assertion of numerous links, etc, you have posted a few links to realclimate articles, a climate blog of such high standards that it routinely and frequently deletes posts contradicting its claims, and one known for its strong advocacy of your position, rather than its scientific rigor.
Now, it is quite easy to assert superior knowledge, say to go read the 1000s of articles in the field, blah blah blah.
It’s a lot harder to explain away the CRU information that was leaked. You have failed to do so, reverting instead to sneers. You have failed to explain why scientists from a large number of fields have now commented that the information revealed by the CRU leak is unacceptable from a scientific institutional standpoint.
——
What you seem to be missing is the principle that extraordinary claims require extraordinary proof. The extraordinary claims that the world must dramatically alter its economies requires extraordinary proof of two factors:
1) The climate science behind that is correct at an extraordinary level (IPCC’s 90% confidence interval doesn’t suffice, and sloppy data handling and suppression of dissent don’t help)
2) Outside the climate field, that the measures proposed are possible and cost effective.
That the climate community would continue to accept the work of the quality shown by the CRU papers, and would fail to censure the CRU “scientists” shows that the community is not willing to provide that extraordinary proof.
If these folks were studying butterfly wing pattern variation with latitude, or something, it really wouldn’t matter. We could wait for nature to run its course.
But they are not. They are making strong claims, and many of them are making strong political claims that require action before the research can be verified to an extraordinary level.
That they should do so without high degrees of certainty is unacceptable.
BTW, #2 above renders all this moot anyway, but not before a lot of damage to the world economy (and by extension, the lives of billions) may occur.
Quote
November 27, 2009, 10:45 pmMichael H Schneider says:
“What has been noticeably absent so far in the ClimateGate discussion is a public reaffirmation by climate researchers of our basic research values: ...”
“The climate science community fully understands how important it is that data sources are made as open and transparent as possible, for research purposes as well as for other interested parties, and is actively working to increase accessibility and usability of the data. We encourage people to investigate the various graphical portals to get a feel for the data and what can be done with it....”
http://www.realclimate.org/index.php/archives/2009/11/wheres-the-data/
Quote
November 27, 2009, 10:47 pmpot meet kettle says:
from the very emails that you claim to have full knowledge of, i present... http://di2.nu/foia/1252090220.txt
it is the cru ts data, not hadcrut.
huh? lack of scientific rigor? this does take the cake. and i didnt realize i was suspect merely for quoting from a blog that provides a principled scientific analysis and one of the clearest expositions on the web of all the evidence in favor of agw.
again and again, you resort to ad hominem when i present you with posts which actually detail scientific analysis.
again, not once have i defended the tone of cru’s emails, or the content of emails urging deletion, just the substance of the allegations that are being thrown around. further, my responses have primarily been regarding those who somehow believe that this is a smoking gun that reveals that agw is a lie, and to ev’s question about why scientists might not choose to share data (in this case, cru has claimed agreements with met stations, which certainly seems plausible to me).
Quote
November 27, 2009, 10:55 pmDavid Bernstein says:
I really don’t have a very well-informed view on AGW. What I have an informed view on is how scientists go about their profession. A part of that view is that all results are tentative, subject to being disproven by new data, or corrections to old data and methodology. This is especially true with regard to a global, complex phenomenon such as global temperature change especially given the background of constant, natural background change in temperature, and even more especially given that the estimates of how much warming can be expected over X years have consistently change (and been reduced) already.
Unfortunately, the emails I’ve seen, not to mention the use of language like “deniers” to describe people who are unconvinced of some or all of the mainstream AGW claims (Holocaust “deniers” deny a known, historical, indisputable fact, the equivalent of “it snowed last Wednesday”; AGW skeptics “deny” a well, but not universally, accepted scientific theory about the past and its predictions of the future), suggest people who are no longer acting as proper scientific skeptics, as all scientists should, and have decided that their conclusions are not tentative, but set in stone. And they’ve done so with good-hearted reasons: they believe they are right, and that further debate will simply muddy the waters and inhibit the world from doing what needs to be done to reduce the damage from AGW. Unfortunately, once one becomes so wedded to a scientific theory that one starts trying to prevent the other sides’ papers from being published, tries to avoid sharing data with one’s “adversaries” lest it be “misused,” start celebrating the deaths of your critics, etc. etc., you are no longer to be trusted as a scientist, but instead should be treated as an advocate who has become impervious to debate/criticism. It may well be that these advocates are nevertheless ultimately going to be proven correct, but they are harming their own credibility by switching from the “there are no final answers” skepticism of science to the “we must hunker down and defend our current position” boosterism of political/ideological advocacy.
Quote
November 27, 2009, 10:55 pmDaniel says:
I think some kind of corollary to Godwin’s Law ought to apply to people who use intentionally loaded terms like “deniers.” We all know exactly what the users of that term are trying to invoke.
From the evidnece I’ve seen, I think climate change is real, and is to some degree—the exact degree being debatable—the result of human activity. However I am not so certain of my own opinion in this matter that I feel the need or the right to dismiss the other side with ugly terminology.
In any field, it’s entirely possible that people will abuse data in order to support faulty conclusions. The solution to this is not to conceal the data, but to point out the errors in the reasoning of the party abusing the data. Making the data available only to those who share your point of view and agree with your conclusions is not a formula for having a real scientific, logical debate about any matter.
If someone studying tree frogs concealed their data for fear some creationist loon would abuse the data and use it to argue for creationist ideas, I think we’d consider that a pretty poor reason for hiding the data. As we’ve seen with creationists, if they aren’t abusing your data to make their points, they’ll be abusing someone else’s, or simply making things up out of whole cloth. The way to handle this is not through secrecy, but through directly refuting the arguments of those you consider to be abusing data or making fallacious arguments.
Data being public brings with it many benefits, like the possibility that analysis by others will yield new and useful conclusions missed by the original researchers, the possibility that some overlooked anomaly in the data might lead to further research into something tangentially related, or that errors in the researchers’ work might be identified and brought to light, allowing faulty conclusions to be revised or abandoned. These are enormous benefits which certainly outweigh the possibility someone will use the data to make a fallacious argument of their own.
I realize that many people working in this field believe that they’re on a mission to save the human race. Perhaps they are. But when you feel so certain of your conclusions that you think the ends justify the means, you’re engaged in something other than science.
Quote
November 27, 2009, 11:00 pmHan Solo says:
If its not REPRODUCIBLE BY ANYONE, at anytime, anywhere following the same methods.
It’s NOT SCIENCE.
Am I the only one that learned this as a young person in school?
I have thought something was fishy since I started hearing stories about no one being able to reproduce the results of many of these claims for the last few years.
Quote
November 27, 2009, 11:03 pmRichard Aubrey says:
Quick question: If the warming–quoth Mann and others–is unprecedented in the last thousand years, what happened, say, 1100 years ago, and why?
And, to forestall some nonsense, why is it not relevant to today?
Ref Deniers and their data:
What data do I have to show to ask...show me yours?
What have I failed to disclose when the other party refuses?
Quote
November 27, 2009, 11:03 pmpot meet kettle says:
actually, here is a statement from somebody from the icelandic met station about data exchange.
also, here is a paper that details how the cru-ts (NOT hadcrut) data was generated, for those who are interested.
Quote
November 27, 2009, 11:04 pmJones' Cell Mate says:
and is actively working to increase accessibility and usability of the data.
So “they” admit it’s a problem, but they’re willing to increase accessibility to data that, either through incompetence or intent, they lost. Yet they haven’t turned over other data due to agreements they are unable to produce. Sounds pretty sciency.
But the best part of that link, and that which perfectly articulates the entire problem, is where the “source” presumes to speak for the entire “climate science community.” As if the entire relevant community was all on one e-mail chain or something... oh wait.
Quote
November 27, 2009, 11:09 pmAaron Bergman says:
David, I would urge you to ignore these personal issues. Scientists fail towards the ideals you present every day. It is only in the conglomeration of the work of many that science proceeds, and it is there that we should look. The climate science community has put together an incredible resource in the IPCC documents. Please set aside some time to at least give it a skim, and you can judge for yourself the body of work and not the people involved.
Quote
November 27, 2009, 11:13 pmagesilaus says:
I haven’t read thru every one of the comments so I don’t know if this was already brought up. However the major point is that this data was used to publish papers in major journals, those journal for the most part require (in theory) that all data used for papers be archived in a public archive. This was not done and when the climate realist group wrote the various journals to complain they mostly didn’t answer or said that they couldn’t force the author to comply. A strange claim. Science and Nature are two of the journals involved.
Furthermore this data was produced using public money, the authors are paid with public funds, and that would make this data and program code, public information. The authors should not be asking who wants the data it should be publicly archived an one of a number of climatological archives so that any interested party can examine it.
These warmist are trying to get governments to spend trillions of dollars on these schemes. Doesn’t it seem odd that they are asking the public to go along with this mad scheme based on “It’s true becasue I said it is and no you cannot look at my data”.
Quote
November 27, 2009, 11:24 pmNobody at All says:
Not entirely correct, Eugene.
From the Real Climate website, some of the scientists’ responses to comments are worth looking at. Selected examples (in addition to that linked to by Pot, above):
1) All of the GISTEMP data and codes is publicly available, as is NOAA raw data and ModelE code
and raw GHCN v.2 (NOAA) data;
2) The GISTEMP data correlates to approx. .97 with the CRU global mean data;
3) National Weather Services would rather sell their data than distribute it freely, and CRU is bound by such permissions.
(edited)
Quote
November 27, 2009, 11:35 pmTweets that mention The Volokh Conspiracy » Blog Archive » Data Sharing and Climate Change Research -- Topsy.com says:
[...] This post was mentioned on Twitter by PostRank – Economics and Bill Giltner, Eugene Volokh. Eugene Volokh said: Data Sharing and Climate Change Research: I know next to nothing about climate change questions, which is why I.. http://bit.ly/5WllYu [...]
random commenter says:
“The climate science community has put together an incredible resource in the IPCC documents.”
The East Anglia trove is inspiring a lot of people to suspect this. I know I stopped viewing Jim Hansen as a scientist when he called for his political opponents to be tried for “crimes against nature.” Actually, earlier. You could see the cheese starting to slip off Jim’s cracker quite a while ago. Advocates tend to be resistant to unwelcome ideas in proportion to their fervor. Like Hansen, Phil Jones and his collaborators appear to be true believers. Caveat lector. I know I will, a lot more than before.
Quote
November 27, 2009, 11:42 pmNobody at All says:
To echo Michael Schneider, Real Climate has posted a page of data sources.
Quote
November 27, 2009, 11:45 pmflyovertard says:
Its laughable to quote realclimate regarding transparency and data accessibility when they are the very folks involved in data obfuscation as explained by themselves in the leaked emails.
No all data isn’t available. The HADCRUT and GISSTEMP are THE most authoritative temperature records available. The code in the HARRY README files are used to create the temperature records based on the data from both measured and paleoclimate reconstruction — it NOT the paleoclimate reconstruction code.
HADCRUT and GISSTEMP use mostly the same temperature stations (as their authors have reported). HADCRUT is administered by Jones and GISSTEMP is administered by Hansen. NEITHER have released their algorithms for developing the data sets. Jones code is now out in HARRY README and its a mess — it can’t recreate the data set used as the basis for the 2007 IPCC report.
Hansen won’t release his code but the emails demonstrate that they work closely in developing their respective data sets.
So what does this tell us? Can’t rely on 2007 IPCC report because its based on the HADCRUT faulty code. Can’t rely on the GISSTEMP data cause its evidently related to the HADCRUT (as explained in the emails). NASA (GISSTEMP) won’t release their raw data or their algorithms. But you can get their value-added data at the links provided by our friends above.
And yes — I’ve read the “fail” links and frequented realclimate for several years now. I’ve also read the rebuttals to those articles. I know who is transparent and who is not.
Quote
November 27, 2009, 11:46 pmCurlyDave says:
zuch:
Not really.Most scientists are not the world’s best computer science engineers, and don’t necessarily follow best coding practises.As long as what they wrote works, and they’re not being intentionally dishonest or making some serious misteak, I suspect they don’t have any particular desire to have CS students lecturing them on modular design, functional specifications, and best coding practises....Witness, for example, the dorks lecturing them on what they wrote in the freakin’ comments in previous threads here....Cheers,
It is apparent that the people you refer to claim to be scientists, but are actually fraudulent, not what I consider real scientists. Of course scientists have egos, and want to bask in the glory of being the discoverer of something new. But, once I have developed a concept and shown it to be valuable, the prospect of CS students, or professors, voluntarily spending their time analyzing my code and improving it only inflates the value of my work.
The more sophisticated measures of a scientist’s status measure both the number of publications and the number of times his/her work is cited. I welcome suggestions from people in other fields, as it increases my prestige.
Only a man who is fearful of the validity of his results discourages review.
Quote
November 27, 2009, 11:49 pmlonetown says:
Actually its worse. They said all that remains is the modified data, the original data was destroyed. But if one has the modified data one can easily reconstruct the original if one knows how it was modified. Therefore they cannot even explain how they modified the data. ergo baloney.
Also its been pointed out that Manns “hockey stick” was already debunked once using manns raw data and this new data revitalized the theory.
This is clearly fraud.
Quote
November 27, 2009, 11:51 pmNobody at All says:
You haven’t been able to find GISTEMP (not GISSTEMP) data or code? Did you try the homepage?
Quote
November 28, 2009, 12:06 amCurlyDave says:
Instead of rambling like that, why don’t you give me three historical examples of scientific controversies where one side has claimed that their viewpoint was “settled science” and have been proven to be correct in the harsh illumination of history?
Quote
November 28, 2009, 12:11 amBrett says:
Why? This is what it is frequently — they’re using some data that they got from various organizations (particularly National Meteorological Services) that grant it to them on the condition that they not release the raw data themselves. Here’s a statement from Halldor Bjornsson of the Icelandic National Met. Office:
Quote
November 28, 2009, 1:34 amBS says:
The situation is that “The Team”, AKA “The Cabal”, controls the data. The skeptics are not well-enough funded to spend years collecting their own data; trips to Antartica to drill ice cores are expensive. Most (all?) of the thermometer and satellite data are generated by governments. The government collected data is given to The Team (the CRU in the UK, the NOAA in the US, and etc.) for analysis. The Cabal makes secret adjustments to the data and publishes the adjusted data. Many people believe that the adjustments are made to achieve a certain result.
One of the most famous skeptics is Steve McIntyre who runs http://www.climateaudit.org. He always clearly describes his techniques. He often, if not always, posts his data reduction scripts. Those scripts are written in “R” which is a descendant of the “S” statistics language. So, yes at least the heavyweight skeptics do make their programs available.
For example, scroll down to the first comment in this post about problems with Briffa’a Yamal data for a script to retrieve the Yamal data
McIntyre has allowed Briffa to respond to McIntyre’s criticism of Briffa’s work on McIntyre’s blog. That sort of thing doesn’t happen on RealClimate (this is confirmed by the emails)
Surfacestations.org is involved in a volunteer effort to audit the siting of every weather station in the US. They’ve found some good stations but many stations would be expected to have a warm bias, especially those sited in the exhaust from air conditioners. They have not released their data but I expect they will when the publish the results within the next few months. It seems reasonable that after all of that work, they get the credit for the results.
Most skeptics are mostly self-funded AFAIK. Some of them have received small amounts of money for their work.
Anti-AGW academics are few and far between in climate science. The skeptic claim is that AGW skeptics have been driven out of academia by The Cabal (the emails provide some evidence of this).
Roger Pielke Sr. is an AGW-leaning to neutral scientist, though he thinks that land use changes are important. He also seems to be a good scientist, IMO. Here is his annotation of a few of the leaked email threads that he was part of. His account reveals machinations on the IPCC report team. He resigned from the committee over them. Email exchange
There has long been suspicion that the reported temperatures have been massaged to reach a predetermined result. Here’s an example from New Zealand where there is no trend in the raw data but after the adjustments we have warming. This is unrelated to the leaked CRU documents. No Trend to Trend
From the CRU documents, here’s my understanging a widely reported adjustment; I’ve not analyzed the code, I’ve merely read the analyses of others. It is referred to in the leaked progam code as a “very artificial adjustment”. In the file briffa_Sep98_d.pro there is this program statement:
valadj=[0.,0.,0.,0.,0.,-0.1,-0.25,-0.3,0.,-0.1,0.3,0.8,1.2,1.7,2.5,2.6,2.6,2.6,2.6,2.6]*0.75 ; fudge factor
Ignoring the factor of 0.75 here are the adjustments made to the temperatures for each five year time period
1904–1909 0.
1910–1914 0.
1915–1919 0.
1920–1924 0.
1925–1929 0.
1930–1934 –0.1
1935–1939 –0.25
1940–1944 –0.3
1945–1949 0.
1950–1954 –0.1
1955–1959 0.3
1960–1964 0.8
1965–1969 1.2
1970–1974 1.7
1975–1979 2.5
1980–1984 2.6
1985–1989 2.6
1990–1994 2.6
1995–1999 2.6
2000–2004 2.6
Note that temperatures from the early half of the 20th century are mostly not adjusted or have negative adjustments; i.e., the adjusted temperatures are adjusted to be colder. Adjustments from 1955 on increase the reported temperature. That’s just the thing if you are trying to cook the books to show global warming. AFAIK, no one has claimed any justification for these adjustments.
There seem to be somewhere between tens and thousands of such adjustments to the data, each of which should be questioned.
Quote
November 28, 2009, 1:40 amMadHatChemist says:
I am a scientist that has published in peer reviewed journals and presented at national and international conferences.
If I make a claim, then I will explain how to reproduce my results and make available the data that supported that claim.
There is NO excuse to deny raw data and methodology for something that you have publicly claimed. If someone does not go the extra distance to prove something, even something in the peer-reviewed literature, I can and often do doubt it and am not afraid to do so.
Getting scouped is not an excuse. Someone taking your brilliant idea and going the distance is always a possibility, and also a way in which we can rapidly improve and refine our understanding.
If someone refused to offer the proof of their claim, then their claim should not be given any credence or even published. This is particularly disturbing when such claims can adversely affect billions of people and put great power in the hands of the few.
Quote
November 28, 2009, 2:13 amsitzpinkler says:
And, it turns out, “real climate scientists” is a term defined in practice as those inclined to agree with the conclusion of the scientist holding the data.
Quote
November 28, 2009, 2:16 amMadHatChemist says:
CurlyDave wrote:
Ptolemaic earth-centered universe.
Germ theory of disease overturning spontaneous generation ideas.
A plethora of theories on anatomy by Galen.
Quote
November 28, 2009, 2:26 amjuris imprudent says:
in favor of agw.
Anyone tell me why RealClimate is using ACC rather than AGW these days? I also wonder at the point raised earlier about the 1000 year cut-off point in describing Mann’s work. What has the climate been like during modern man’s era on the planet — shouldn’t that be the time-frame most relevant when discussing our interaction with the climate?
Quote
November 28, 2009, 2:46 amuberVU - social comments says:
Social comments and analytics for this post...
This post was mentioned on Twitter by VolokhConspirac: Data Sharing and Climate Change Research: I know next to nothing about climate change questions, which is why I.. http://bit.ly/5WllYu...
» “Past predictions of the Climatic Research Unit at University of East Anglia” and related posts - 935th Edition Www.composition4u.info says:
[...] Data Sharing and Climate Change Research - The Volokh Conspiracy [...]
Frank O'Dwyer says:
“Or am I misunderstanding the situation, and the data was indeed made available to everyone?”
This is closest to the truth.
Raw data is available to anyone who genuinely wants to ‘do the science’ and confirm the results. There is no curtain.
Quote
November 28, 2009, 6:24 amneurodoc says:
The scientific method, not surprisingly, pertains to science. Anon said nothing about science, and law reviews are not where scientific findings are presented and conclusions argued.
Quote
November 28, 2009, 8:05 amneurodoc says:
When you say “all data gathered with public money,” do you mean that to include data gathered by non-government scientists whose work has been supported by research grants from the federal government (e.g., NIH, NSF, etc.)? It matters not whether the data was generated in a federal facility by government employed scientists or in a non-federal facility by non-government scientists?
A decade ago, Senator Shelby added an amendment to a bill, his intention being to enlarge the reach of FOIA where scientific data was involved. The amendment passed, but I am not sure where things settled out in the end after OMB promulgated implementing regs. Do you know? I would be surprised to learn that all NIH grantees had to share their data with everyone who wanted to it.
http://www.aaas.org/spp/cstc/briefs/accesstodata/index.shtml#shelby
Quote
November 28, 2009, 8:39 amwb says:
Eugene,
I fully subscribe to the point you’ve made in this post. The thread also exposes the primary defense that is made for the secrecy of raw data, to wit ”
This defense is sophistry... no scientist is forced to respond to criticisms. No responsible scientist should fear having the interpreted data plus techniques and methods fully public. The argument is made that only specialists can can analyze and extract the “truth;” so hide the primary data from the non-believers, skeptics, etc. Climatologists are not high priests privy to secret knowledge. They are ordinary scientists and mathematicians — some good, some hacks — all subject to the same scrutiny as any other scientist engineer or politician.
Sunshine — full disclosure – is the best disinfectant for bad science as it is for bad government. I have my greatest confidence in those practitioners with nothing to hide.
Quote
November 28, 2009, 9:28 amzuch says:
That’s what you get when you take six by nine. Easy, and sorry you didn’t see it. Of course that’s in base 13 (decimal). Which is why the comment about “what do you get when you take 6 by 9″ showed up in the DEC disk driver code for a patch to fix flawed handling of disk unit 13....
Cheers,
Quote
November 28, 2009, 9:49 amLaura(southernxyl) says:
Neurodoc, it’s not just that they won grants, although I’d expect to see some provision for reporting what grant winners produce with government money.
It’s more like a drug company offering a new drug for human use, and declining to share any data about it. “We did studies — take our word for it, the drug is very effective and safe. No, we won’t share the data. Take us to court, but in the meantime, we’re selling the drug.”
Quote
November 28, 2009, 9:54 amflyovertard says:
Nobody at All says:
“You haven’t been able to find GISTEMP (not GISSTEMP) data or code? Did you try the homepage?”
Value-added data isn’t original data. The only code available is for global averaging calcs — not the code used for creating the value added data. Bottom line is critical review starting from raw temperature data to global average is not possible.
Quote
November 28, 2009, 9:59 amflyovertard says:
Frank O’Dwyer says:
“Raw data is available to anyone who genuinely wants to ‘do the science’ and confirm the results. There is no curtain.”
Yes anyone could contact any of the met stations and request the data. Many data sets are available online and I access and use them often in my work.
The point is that taxpayers have funded institutions to do this for them. The problem is full disclosure of the methodology used to produce the resulting value-added data sets is indeed “behind the curtain”.
Quote
November 28, 2009, 10:23 amkidneystones says:
I considered myself quite willing to accept the data that passed peer-review. However, I’ve since seen enough smoke to question the veracity of the entire ‘the earth is getting warmer’ argument. That surely can’t be the desired outcome.
Fire away if you feel my comment simply confirms that I’m unqualified to hold an opinion on the topic. I’d never have considered questioning the earth is getting warmer argument until this week.
I’m profoundly unimpressed by claims of ‘you wouldn’t understand’ and ‘why should we share the data with anyone?’ from scientists warning in the same breath that failure to take earth-warming seriously might result in the destruction of life as we know it.
If the crisis is a serious as you claim, how on earth do you defend your handling of the data supporting your arguments?
Why should anyone take anything climate scientists say seriously?
Quote
November 28, 2009, 10:31 amJEM says:
wb — further, it’s the Joneses and the Manns et al, the guys whose names are RIGHT THERE at the top of the lists of contributors, lead authors, and editors of all the pieces that make up the IPCC ARs, who have set themselves up in the position to decide what ‘climate science’ is and who is a ‘climate scientist’.
Quote
November 28, 2009, 10:36 amfsfsfsfsfsfsfs says:
Willis Eischenbach is quoted here. His claims are worth reading.
He walks through many of the emails regarding the Freedom of Information Act evasions. These quotations appear to negate the claim that data was withheld because of contractual restrictions on their use.
Unfortunately, due to the large number of quotations from the emails, it is difficult to summarize here, but here are a few excerpts.
In September of 2004, Warwick Hughes requests items like “the names of stations that you’ve compared” and certain specific data.
In response, Phil Jones wrote in 2005:
Here, the justification for the refusal does not appear to be contractually based, but rather based on a desire not to give the data to people who want to try to find something wrong with it.
Eischenbech then, in 2006, asks for a list of meteorological stations and the raw data used in the preparation of the CRU data published by Jones.
He got no response, filed a FOI request, which he believes may have led to the current disclosures.
Anyway, one of the disclosed emails by Jones states:
Again, this is not consistent with the claim that data was withheld due to contractual restrictions.
Michael Mann, another prominent global warming activist/scientist, writes to Jones:
This language does not seem consistent with the “contractual restriction” theory.
Phil Jones also writes
Even more interesting is a long log of Eischenbach’s repeated FOI attempts to obtain various climate data. It is quite interesting reading, a highlight is Phil Jones letter of 2007:
The claim here is that Phil Jones “managed to persuade UEA [University of East Anglia] to ignore all further FOIA requests if the people hae anything to do with Climate Audit”.
Again, this is entirely inconsistent with the claim of proprietary data being the reason for the lack of disclosures — if it the data being proprietary were the true reason for the lack of disclosure, then the requesters affiliation with Climate Audit would not justify its not being provided.
Skipping over a lot of machinations on several continents to avoid divulging the data to Climate Audit in the face of FOIA requests, Phil Jones makes the well-known request for deletion:
Although this pertains to email, and not data, the genesis of the request for emails was the refusal of CRU and Jones to provide the data.
Ben Santer later writes that:
Here, the refusal of data to McIntyre is not based on concerns about proprietary restrictions, but rather on the author’s opinion of the use to which McIntyre intends to put the data.
I will stop here, there are many, many more excerpts in the link cited at the top of this post. However, taken together, these emails are not consistent with the claim that data was withheld from outside researchers because the data was encumbered by proprietary restrictions. Rather, the justifications provided in the emails were that the scientists at CRU and in Australia did not like the uses to which to the requested data was being put, did not like their conclusions being attacked in ways they felt was unfair, and felt they were in general being mistreated. Whether those opinions were justified or not, contractual restrictions on the data was not the reason for the lack of sharing.
Quote
November 28, 2009, 10:39 amsteve says:
This is different than other research data. The researchers in global warming claim that their conclusions require us to make extremely costly and instrusive changes in our lives. Refusing to share their data means we must alter our lives simply because they say so.
The arguments against misuse of their data are anti-democratic. It would be like the government withholding unemployment statistics because the opposition party might use it to make misleading arguments. AGW is a public issue now that affects us all and we simply have a right to know.
Quote
November 28, 2009, 10:40 amBill Harshaw says:
I know nothing, but it’s obvious from the above back and forth that, unlike the pure, serene world of gentle(manly. womanly) lawyers debating legal theories, science is a contact sport, much like soccer, lots of hairpulling and hand balls. Lots of ego involvement, unlike law.
An interesting book in this regard is “Einstein’s Luck”. A quote from the Amazon descripton: “The first group of scientists — those right for the wrong reasons — includes Robert Millikan and the demonstration of the existence of electrons, Arthur S. Eddington and the proof of general relativity, and Louis Pasteur and the germ theory of disease. In each case the scientists had preconceived ideas, which they supported sometimes by means of obfuscation and deception and other times by manipulating their data or ignoring data that did not fit the ideas. Of course, if their ideas had been wrong, these scientists would have been ignored by posterity and long forgotten.”
I’d also throw out the idea that while transparency may be a norm in the ideal world, even disregarding personal or perverse motives, sharing data must have costs, costs which probably have been declining with technology, but which still are real. So there’s a conflict between adherence to a universal idealistic norm, and the pursuit of self-interest.
Quote
November 28, 2009, 10:41 amresearcher says:
Some of the key data behind the claims of the success of affirmative action at Michigan has never been released.
Quote
November 28, 2009, 10:45 amrc says:
There are many theories for why the scientists kept the data secret. Some of the proposed motives are ‘good,’ as in beneficial to future science, while some of them are ‘bad.’
But let’s consult the scientists’ explanations themselves. In 2004, a CRU scientist said, why should I share the data, when all you want to do is disprove it. Later, the leaked emails reveal that the scientists do not want data to get into the hands of sceptics.
These are not ‘good’ reasons for withholding evidence.
Quote
November 28, 2009, 10:47 amDonald Lacombe says:
The solution is relatively easy: Journals should only accept papers with the data and code files that are placed on the Journal’s website for others to download. The Journal of Applied Econometrics, as well as other journals, use this method and have about a 99% compliance rate.
Quote
November 28, 2009, 10:48 amRichard of Oregon says:
In areas of science that are still functional, if you don’t share both your data and method you can’t get published. The only reason for withholding that imformation is if you intend to apply for a patent. In appling for the patent you then must give full disclosure. Climate science has not been functional for decades. I first became aware that climate science was a fraud was about 25 years ago. Scientific American published an article on man-caused global warming that met the New Yorker magazine’s criteria for science but was not even close to SA’s standard. Over the next several years more and more of their articles failed to come close to their previous standards. A revival of rigorous standards in science is desperately needed.
Quote
November 28, 2009, 10:49 amJW says:
This is a red herring. The skeptics largely work without government or corporate funding of any kind. They may accept donations on their websites or ad revenue, but the funding discrepancy is truly huge. The best example of working on a shoestring is Anthony Watts’ volunteer project validating the surface temperature measuring stations in the U.S.
It clearly establishes that the U.S. surface temperature record is unreliable. I believe there has been or will be a “peer-reviewed” publication on that work.
Data and methods by which the study was done is widely available, and established beyond doubt with photographic evidence.
Quote
November 28, 2009, 10:50 amNobody at All says:
GHCN and USHCN raw data for GISTEMP are both freely available online.
Quote
November 28, 2009, 10:59 amwolfwalker says:
Way back uptopic, “reasonable persons” asked: For example, there is no reason a physicist should have to share data with a flat-earthers that the earth is really round. It’s a waste of time. Similarly, should a biologist have to share data with a creationist?
Yes, they should. And they do. In fact, this is what has left creationists flailing desperately for arguments that work, and not finding any: the raw data is all available, in countless museums and universities and field sites all around the planet. And the raw data says the creationists are wrong. This open-ness is why creationism loses every time it’s tested in a fair and open forum. By contrast, time after time, in forum after forum, the pro-AGW argument boils down to “trust me, I’m a scientist.” I’m sorry, that’s not the way good science works.
The data-gatherers have every right to keep control of their data until they’ve obtained proper compensation for it. In science, the currency of compensation is formal papers. After it’s been published, all comers should have access. I think it’s okay to enforce some standards for access, just to make sure the data-holders don’t get DDOS’d. But access should always be available to those who have a genuine need and have demonstrated their own trustworthiness.
Quote
November 28, 2009, 11:01 amDerHahn says:
I see from the comments that RC is now using the tactic of point to raw data stores to claim the data has always been available.
fs.. quoting Willis Eischenbach illuminates the problem with that tactic. AGW papers from CRU often made claims based on analysis of specific subsets of the raw data (re — the request for ‘CRU station data’) RC is referencing. Uless the requesters knew exactly which subset of data was studied, there would be no way for them to generate an apples to apples comparison of the CRU claims.
Quote
November 28, 2009, 11:05 amClaude Hopper says:
A thought experiment for Reasonable Persons:
An asteroid is aimed at the earth with a high likelihood of a catastrophic impact. Your government deliberates on two courses of action. One is to marshal all the earth’s technical resources to deflect the asteroid’s path away from the earth’s orbit. The other approach is to tax the citizens of the developed countries and transfer that wealth to the under developed countries. Which action would you choose? Yes, this is sort of stupid, but considering the target.....
Quote
November 28, 2009, 11:05 amJoe says:
Somewhere in the world a very tiny violin is playing for them.
Quote
November 28, 2009, 11:06 amJohn says:
The sourse code seems to be very problematic. If you don’t provide the source code for your simulations, who are you doing anything but saying that the “magic box” is giving results? I don’t see how any of these scientists’ claims can be properly evaluated without allowing other experts and programers to look at their source code.
Second, it appears from the e-mails, their models can’t account for known data. If the models are valid, you ought to be able to put in the temperature data from say date A, run the model and come up with an answer for the temperatures at date B that matches or closely matches the known data for date B. They can’t do that. They admit as much in the e-mails. Their answer is that it must be that the observed data is wrong. Maybe. But how do we know that? It is just as likely that the model is wrong. There is no way to tell. Given that fact, I can’t see how you can claim that the models are in anyway definitive or the predictions of future climate mean anything.
Quote
November 28, 2009, 11:40 amRHSwan says:
Neurodoc,
In the US, any data created or discovered by Federal Government funds (I’m not sure about the states, but it probably is similar) is by default public data and is available to anybody who wants it. They may have to provide money to duplicate the data, but other than that it is available. This can be modified by the contract the non-government entity has with the government but that is the result of negotiations. It is not the default position.
One example of science done with public funds but has a restriction based on the data. When a scientist takes a picture with the Hubble Space Telescope they have one year to research the data the picture provided. After that, anybody can get the data. They don’t have to be scientist, they can be a six year old kid from Boise, Idaho who wants to make a pretty picture. I’m not sure how “raw” the data available to the public is. There are some well known corrections to the cameras Hubble uses and the techniques used to come up with these corrections are widely known.
Quote
November 28, 2009, 11:41 amrpt says:
Is this really correct? No commercial or industrial interests are involved in the the climate debate?
Quote
November 28, 2009, 12:06 pmMDr says:
NASA is hip deep in this scandal.
“Today, the Competitive Enterprise Institute filed three Notices of Intent to File Suit against NASA and its Goddard Institute for Space Studies (GISS), for those bodies’ refusal — for nearly three years — to provide documents requested under the Freedom of Information Act.”
http://pajamasmedia.com/blog/competitive-enterprise-institute-sues-nasa-in-wake-of-climategate-scandal/
Quote
November 28, 2009, 12:09 pmbbbeard says:
Well, FWIW, I agree with CurlyDave. I am a computational physicist, and though I work in the space program now, when I was active in lattice gauge theory I routinely posted my codes and data on my web site for anyone to access — skeptic or not.
We seem to have at least of couple of “protectionist” lines of argument that are directly contradictory. On the one hand, some folks (“reasonable persons” [sic]) argue that we have no time to waste with skeptics. Others (“pot meet kettle”) defend the right of scientists to withhold data from peer review for the sake of their careers, to milk the data for as many papers as they can for, protecting their competitive advantage as long as they can. But if the problem is so pressing, we can only benefit by having more people crunching the numbers, not fewer — and it is immoral to slow down the process. Conversely, if the point of the enterprise is to fill library stacks with journal articles, enhancing the careers of scientists, then again, good public policy would seem to encourage data sharing to maximize the number of publishable articles.
Frankly I am incredulous that anyone would push the “proprietary intellectual property argument” given the fact that the revealed e-mails show this argument was explicitly constructed as a roadblock to FOIA. I am also incredulous that anyone would make arguments of quality from “peer review” because of the scandalous efforts by CRU and their peers to block the functioning of the peer review system.
I have a legal question, too: Phil Jones sent out an email calling for the destruction of documents explicitly to avoid revealing them in the FOIA datadump. Even if Jones was the only person to engage in direct criminality by destroying documents, are the recipients of that email liable in any way (as, say, “accessories to the evasion of a FOIA request”) for not reporting Jones’ call for criminal conspiracy to the authorities?
BBB
Quote
November 28, 2009, 12:22 pmSG says:
Is this really correct? No commercial or industrial interests are involved in the the climate debate?
You and others keep bringing this up, but it’s ultimately a red herring. Either they’re doing proper science or they’re not — their funding may be interesting, but it’s not determinative. And it’s clear now that the publically funded researchers aren’t angels either, we’re left to (properly) debating the various claims which means investigating the data and the analysis they used to reach those claims.
Commercial/industrial interests do research all the time. Some of it is junk (tobacco), some of it has changed the way we live (Bell Labs). Ultimately the research has to stand or fall on its own merits.
Quote
November 28, 2009, 12:26 pmrpt says:
SG:
CEI is, of course, a corporate funded organization.
SG: This debate has been very enlightening for me (I am not a scientist), but one of the points which did not ring true was the contention that the AGW side has all of the power and funding and the “skeptic” or “denier” side is a group of unaffiliated sincere truth-seekers with no other agenda than scientific truth and no connection to corporate interests which benefit mightily from the status quo (whatever it is). Funding and motive is always relevant.
Quote
November 28, 2009, 12:41 pmzuch says:
Writing some code to analyze data is not some great scientific invention ... well, maybe it would have been had I had a chance to implement some bootstrap statistics to analyze my old prof’s cell data rather than using his lame TI-45 T-square statistics and “it’s not real if you can’t see it” approach (which works well generally, but tends to have less power and be subject errors due to violations of assumptions).
The real work is the output, analysis, and interpretation, not the code. The code tends to be hack work suitable for grad students and RAs.
Cheers,
Quote
November 28, 2009, 12:44 pmrpt says:
FWIW, Hans Bader is the designated CEI agent to the VC.
Per wikipedia:
CEI’s Foundation Funders
Media Transparency lists CEI as receiving a total of $4,296,645 (unadjusted for inflation) in 123 grants from a range of foundations in the period 1985 through to 2004. [3]
* Armstrong Foundation
* Barre Seid Foundation
* Castle Rock Foundation
* Carthage Foundation Scaife Foundations
* Charles G. Koch Charitable Foundation (Koch Family Foundations)
* Claude R. Lambe Charitable Foundation (Koch Family Foundations)
* David H. Koch Charitable Foundation (Koch Family Foundations)
* Earhart Foundation
* Gordon and Mary Cain Foundation
* Jacqueline Hume Foundation
* JM Foundation
* John M. Olin Foundation
* John Templeton Foundation
* Lynde and Harry Bradley Foundation
* Philip M. McKenna Foundation, Inc.
* Randolph Foundation
* Rodney Fund
* Roe Foundation
* Sarah Scaife Foundation (Scaife Foundations)
* Scaife Family Foundations
* Shelby Cullom Davis Foundation
* William H. Donner Foundation
Other Funding Sources
The Capital Research Center (CRC) formerly had a profile on CEI in its database on non-profit groups which listed corporate foundations and other groups not identified by Media Transparency. [4] However, since its profile was linked to this page in 2004, the profile on CEI has been removed from the database.
CEI does not publish a list of its institutional donors. However, in a CEI report sent to Philip Morris, the think tank identified a range of companies and foundations as having given $10,000 or more. [5] Contributors included:
* Aequus Institute
* Amoco Foundation, Inc.
* Coca-Cola Company, contributions were $25,000 per annum for the period 1991–1995;
* E.L. Craig Foundation
* CSX Corporation
* Fieldstead and Co.
* FMC Foundation
* Ford Motor Company Fund
* Curtis and Edith Munson Foundation
* Philip Morris Companies, Inc.
* Pfizer Inc.
* Precision Valve Corporation
* Prince Foundation
* Sheldon Rose
* Texaco, Inc.
* Texaco Foundation
* Alex C. Walker Foundation
In a 2006 profile of CEI and other global warming skeptics, Washington Post reporter Joel Achenbach noted that “the most generous sponsors” of CEI’s 2005 annual dinner were “the Alliance of Automobile Manufacturers, Exxon Mobil, the Pharmaceutical Research and Manufacturers of America, and Pfizer. Other contributors included General Motors, the American Petroleum Institute, the American Plastics Council, the Chlorine Chemistry Council and Arch Coal.”
This sort of disclosure is also good. Both sides should disclose whatever is relevant.
Quote
November 28, 2009, 12:45 pmzuch says:
Newtonian mechanics (at rest or low enough velocities). Still works like a charm for most real-world applications. Beats the hell out of Aristotelean physics.
Quantum physics. The fine structure constant has been determined to accuracies of unimaginable precision.
The DNA theory of genetics.
The list goes on and on.....
Needless to say, each of these theories have some play around the edges (relativity, prions, reverse transcriptase, etc., but the vast corpus is unassailable and unarguable.
Cheers,
Quote
November 28, 2009, 12:50 pmepignosis says:
Mr. Volokh
AGW proponents are, in effect, yelling “fire” on a crowded planet. That the resulting stampede will unfold more slowly in no way diminishes the potential for social and economic injury. Therefore, having a little more time at our disposal, would we not want to examine carefully the claims of those who opine that it is just a reflection from a red metallic candy wrapper?
Buckland says it clearly:
Add a new entry to the Wikipedia article on “cognitive bias”. NASA has had some rather unpleasant experience resulting from very smart and talented people who have succumbed to cognitive bias (Challenger & Columbia accidents). One expression of that bias is when a group of scientist adhere to a belief so strongly that they are willing to ignore or rationalize all evidence to the contrary. Some refer to this as groupthink, though that does injustice to the scope of the danger in this particular case.
What permeates climate science regarding anthropogenic global warming (AGW) is properly called crusader arrogance. “I am intent on saving the planet, and I am one of the few people smart enough to know what is necessary, everyone who disagrees with me is not only wrong, but is a threat to humanity.” These true believers have forfeited intellectual honesty. Data has been manipulated, opponents have been ridiculed, castigated, impugned, and silenced. Contrary information has been suppressed. Decent scientists with integrity has lost their jobs.
These people have absolutely no case for refusing to release data that will withstand serious scrutiny. They did so because their crusade was just too damned important to allow opponents to derail the message.
And if there message is wrong? Ooops!
Quote
November 28, 2009, 12:58 pmRaven says:
Here is detailed explanation of the lenths that sceptics had to go through in order to get access to the data.
http //camirror.wordpress.com/2009/11/25/willis-eschenbachs-foi-request/
We know from the leaked emails that the reasons stated in the FOI refusals are just excuses. They decided in advance that they would refuse to release the data because ‘they just want to find something wrong with it’.
Quote
November 28, 2009, 1:03 pmMR says:
I’d like to clear up some confusion that has muddied the debate regarding raw data and temperature records.
The specific question is “What is happening to global temperatures?” To address this question, researchers must look at historical weather station data. However, the raw temperature readings over a long period of time must be normalized.
Over time the thermometers have changed, sites have been moved or closed, screens have been altered and the frequency and time of measurements has changed. Without adjusting for these variables the raw numbers are worthless for determining trends. The process of normalizing the data isn’t easy, even for the most scrupulous of scientists, and the numbers are constantly being adjusted.
As part of the process of measuring temperature trends, researchers must make difficult decisions about which stations to use and how to adjust for the changes at each of the stations. It is this specific information that critics and researchers ask for when the term “Raw Data” is batted about.
This information is critical in judging the accuracy of temperature series and anomalies because the selection of what stations are used affects the trends, as does the manual adjustments to the temperature records.
So when someone points out that the raw station data is available from other sources that is disingenuous at best. That ‘raw’ information is worthless without knowing which stations were used for a temperature series and how those measurements were adjusted/normalized for changes over time.
Also, it is disingenuous to point to GISS’s FTP folders and say that all the data is available there. It is not. What GISS provides is post-processed anomaly data for gridded areas. There is no specific information on station selection or individual adjustments. SOURCE: ftp://data.giss.nasa.gov/pub/gistemp/download/README.txt.
In my opinion it is critical that researchers release this information so that errors can be fixed (it is a very manual process), other researchers can better refine their processes, and so that the process can be trusted.
Quote
November 28, 2009, 1:04 pmvepxistqaosani says:
The Russian/Soviet connection
While I agree that the data should be shared — and all the procedures used to manipulate the data documented — it does not seem unreasonable that some of the data could have been embargoed.
The data goes back several decades and includes datasets from Russian (formerly Soviet) sources. It’s amazing, actually, that so traditionally paranoid a government as that allowed any data out of the country. You may recall that official Soviet maps were wrong, simply because they didn’t want invaders to have accurate information. Back in the day, the only accurate maps one could find were those produced by Western intelligence services.
Of course, in the current era of satellite surveillance, there’s no reason for the Russians not share geographical and cartological data. But it is by no means clear that they ever needed reasons.
And, finally, it’s not exactly clear from the CRU emails that the Russian data was the sticking point. But, hypothetically, I do find it plausible.
Quote
November 28, 2009, 1:06 pmNobody At All says:
The underlying GHCN and USHCN station-level raw data for GISTEMP were both previously linked to. Ex (USHCN) Source:
ftp://ftp.ncdc.noaa.gov/pub/data/ushcn/v2/monthly/readme.txt
Quote
November 28, 2009, 1:22 pmMR says:
Nobody, I apologize if I wasn’t clear earlier.
Yes, that is raw station data. However, it has no value in analyzing GISS’s work.
1. It is a superset of a portion of the data. GISTEMP does not use all of the USHSC stations. (Nor should they, as there is varying levels of quality to the data.)
2. It is does not contain all of the data sources, as USHCN only contains weather station data from the United States.
3. These files contain no infomation on what adjustments GISS made to the data they DID use from USHSC.
4. There is evidence that there is direct or indirect weighting of the station data, this information isn’t available in the USHSC files.
Quote
November 28, 2009, 1:42 pmstan says:
If data, methods, code, etc. are not readily available for others to check, the conclusions of a study should never, ever be used to formulate public policy. It doesn’t matter if the study was publicly financed or not. While the product of publicly financed research should be available, that is a completely different issue.
Secret science cannot be used to formulate policy. Period. Regardless of who paid for it. There may be legitimate reasons why someone chooses not to make the data, etc. public, but that disqualifies the study for public policy purposes. You can announce your findings, but if you don’t allow the study to be replicated or audited, your announcement doesn’t reach the threshold of science. If we can’t check it, we can’t rely on it for policy. No exceptions.
Quote
November 28, 2009, 1:48 pmNobody At All says:
So we agree: Raw station-level data is publicly available. Someone who wished to independently determine changes in measured temperature could, in fact, do so.
This is the crux of the criticism; it’s good to have it settled.
1. I cannot speak to this point, so I won’t.
2. Yes; nothing otherwise was claimed — USHCN was illustrative, not complete. GISTEMP’s raw data sources are, to my understanding, all publicly accessible.
3. Yes; the raw data rather than the normalizing procedures were called for by skeptics — supposedly so that they could engage in the science and independently verify measured temperature changes. Such data is available. If normalizing procedures, etc. are not publicly available, I agree, they should be in order to enable criticism of a particular time-series. But this is separate issue from whether skeptics were able to independently assess measured temperature changes.
4. I cannot speak to this point, so I won’t.
Quote
November 28, 2009, 2:02 pmMoneyrunner says:
One possible side effect of this scandal is the explosion of what I call the “Science Bubble.” It seems that every other story in the popular press includes the phrase “scientists say.” If the accusations are true and the entire global warming crusade is a scientific hoax, the blowback can be enormous.
The 21st Century has, in less than a decade, seen the explosion of three “bubbles.” In 2001 we saw the tech bubble explode, leaving in its wake the collapse of numerous fortunes and a renewed appreciation for value rather than promises of ever more extravagant high-tech dreams. Tech stocks were valued at hundreds if not thousand times their earnings, and even more if they had no earnings at all. And then it all came tumbling down and took with them venerable names like Bell Labs (Lucent).
Swearing that the stock market was rigged, people decided to put their money in houses. “Real estate never goes down” was the new mantra, and as evidence we saw houses “flipped” – on TV — for huge profits and real estate assessments climb 20, 30 or 40% per year, proving how to get rich by simply buying your own home. And then the real estate bubble burst and crushed people whose mortgages were larger, much larger than their homes’ values. And the people who owned these mortgages? They could not get rid of their worthless loans and venerable names like Bear Stearns and Lehman Brothers went out of business, whole governments throughout the world injected capital (read credibility) into the remaining lenders to prevent a total global financial collapse.
And now, out of the blue, we see something that the innocent bystanders never expected to see: a collapse of the “science bubble.” It’s early yet in this collapse. As in previous bubbles, the professionals, the press and the government tell us that it’s just a blip. Here are some echoes of the past:
“Buy Enron when it’s down 50%” was the advice; you’ll never see it this cheap again!
Here’s your chance to buy your dream house at low, low interest rates … no money down, no principal payment for 10 years! No credit, no job, no problem!
And now we’re told that the global warming science is still settled even though the “scientists” pushing global warming conspired to keep opponents from publishing, created computer models that were pure sludge, refused to share their data so that it could be checked, and claimed that they lost large parts of their original data. And the experts, press and politicians are telling us that it’s just a blip, a glitch, not to worry, they know what they’re doings and that they are going ahead with their plans to re-make the world.
The unforeseen outcome of this scandal goes way beyond the issue off global warming. Just as the collapse of the tech bubble did not just affect one or two companies or even just tech stocks, and the collapse of the housing bubble did not just affect homeowners. Scientists in all fields will be eyed with suspicion. The next time I read a news article that includes the phrase “scientists say,” my BS detectors will go off. Because scientists say a lot of things and it’s now apparent that a lot of it is a lie.
Quote
November 28, 2009, 2:06 pmgeokstr says:
Quote
November 28, 2009, 2:14 pmRobert Ayers says:
Aaron Bergman wrote: “All the NASA data is available, for example, and all their code is, too. I believe this is the correct link. http://data.giss.nasa.gov”
I visited the GISS three days ago to see if that was true. Based upon my trawling of GISS the site I believe that a) the original USA data, although not lost, is not obtainable at GISS or USHCN (see cdiac.ornl.gov); only “adjusted” data is available for download and b) algorithms and code is not available.
I wrote to the “contact” email address at USHCN on 26 November and asked them whether the raw station records and the algorithms/programs were available. We’ll see if they reply.
Quote
November 28, 2009, 2:20 pmTim says:
The real reason the raw data wasn’t released is the same reason that “real journalists” hate bloggers. It’s the same reason union workers hate non union workers. It’s the same reason “real teachers” hate charter schools. And it’s the same reason that “The Church” hated people who published the Bible in the vernacular so long ago. The “real scientists” who concocted this once novel theory set themselves up as an all-knowing priesthood. In the process they turned a scientific theory into a religion the mere questioning of which could get you sentenced to a Galileo style retreat. We think we’re so much wiser than our ancestors – alas we’re not.
Quote
November 28, 2009, 2:22 pmNobody At All says:
Quote
November 28, 2009, 2:26 pmRoger Sweeny says:
If 1) some journals require that the data used to reach the conclusions of an article be made public and archived, and
2) people who used CRU data couldn’t do this because of confidentiality conditions set by the national meteorological organizations that some of the data came from, then
3) wouldn’t it be unethical for someone using the CRU data to submit to any of these journals?
Some of the things I have read lead me to believe that many of the journals that have such policies don’t vigorously enforce them. Perhaps if they did, it would put pressure on scientists to put pressure on the national meteorological organizations to be more liberal where such important research is concerned.
Quote
November 28, 2009, 2:40 pmMR says:
Nothing...
We seem to be speaking past each other and I’m not sure how we can get past that. Let me put in a metaphor:
The scientific process includes being able to replicate results.
The argument that all the raw data station data is the equivalent of saying to a baker that all the raw ingredients to bake a cake are available in a grocery store. That statement would be correct, but not helpful.
However, you could not replicate the baker’s cake without knowing what ingredients were used to make the cake, what proportions were used, and how the cake was cooked.
Same with these temperature studies, we don’t know what stations were used, if enough stations were used in each grid to adequately replicate the varous microclimates, what adjustments were made, and how the data was baked into it’s finished product. (Yes, I avoided using the word ‘cooked’.)
On point 3:
Those who you’ve read say that the ‘skeptics’ just want the raw station data are wrong or misleading. Raw station data is ubiquitous, what they are saying is that they want to know WHICH subset of data was used (it’s important) and the specific process used to create the final results.
When the term ‘Raw Data’ is used, typically, it is in reference to the raw data that was USED by GISS, CRU & whoever else. It does NOT meant in reference to the total superset of raw data available in the world. To put another way, USHSC’s raw data is not the same thing as GISS’s or CRU’s raw data and that is a critical difference.
Horse. Beaten. Dead.
For the record, I could take the same raw station data, pick the stations I wish to use, adjust the numbers so they seem right to me, run my grid calculations, and “prove” the earth was cooling.
It would be a load of TOTAL BS, but without revealing what stations I chose, what adjustments I made, and what calculations I used...you couldn’t prove me wrong. Better still, the differences we’re talking about are miniscule in statistical terms so the final results would look very similar but with a slightly different slope.
And when pressed for details, I could just point to the larger superset of station data and say “my raw data is readily available from other sources”.
For the record, I’m not saying that GISS, or CRU, or anybody else’s calculations are wrong. I’m a big believer in the scientific process and believe that information should be made available so that other’s can verify the work. It’s too important not too.
Quote
November 28, 2009, 3:06 pmNick Temple says:
What seems to have been missed here is that:
Legitimate scientists (published climate researchers not McIntyre) sent in FOI requests for the data.
The CRU refused saying “we have non-release agreements”
When FOI requests were made for the agreements the CRU stated that they couldn’t find the agreements ( actually some claptrap about how we moved our offices, never standardized how differnt departments file them and it would take more than 18 hours to find the agreements. The CRU could not even state who the agreements are with.
When asked for all the data that was available that would not violate the agreements with parties that could not be enumerated the response was just not possible (probably because they couldn’t filter the not to be disclosed to third party data out because they did not know which data belonged to the non-disclosure set, and they couldn’t find the agreemnts to find out what was and was not allowable.....
This is all documented on Climateaudit.org under the FOI topics in the frame to the left, I think this is the link http://go2.wordpress.com/?id=725X1342&site=camirror.wordpress.com&url=http%3A%2F%2Fwww.climateaudit.org%2F%3Fcat%3D53 but that site is pretty overloaded so I can’t verify that
There is a good post on Willis Eschenbach’s FOI Request, on camirror.wordrpess.com
http://camirror.wordpress.com/2009/11/25/willis-eschenbachs-foi-request/
The last I checked Gavin Schmidt & company at RealClimate.org are still spewing “non-disclosure” agreements as the reason the data can’t be released, and RealClimate.org isn’t saying who the “non-disclosure” agreements are with. I imagine that if asked the response would be:
We can’t say who it is we have an agreement, or we can’t find those.
Quote
November 28, 2009, 3:09 pmconiston says:
Frank O’Dwyer — if the raw data and codes were available, why wouldn’t Jones et al have simply provided a link? Why hours and hours spent trying to evade FOI requests when it would take a few minutes to reveal it?
Quote
November 28, 2009, 3:44 pmNobody At All says:
We are talking past each another, I agree.
This is not the way that “Raw Data” is used in the context of this thread (nor in Adler’s most recent thread). The three comments preceding yours (excluding mine) all precede on the basis of unavailable station data, as does the comment right after yours.
But, as you say: raw station-level data is ubiquitous. If someone wished to independently participate in the science of determining changes in measured temperature, a lack of data availability does not prevent them.
We’re on the same page, here. Were I to quibble, I would point out that there is a large set of applicable literature by many authors, and that many authors do, in fact, defend their normalization, etc. procedures, and that this literature is largely in agreement as to the overall trend.
Quote
November 28, 2009, 3:45 pmCBI says:
A key aspect of science is not “consensus” but predictive value.
I recollect reading NASA gent James Hansen’s testimony to congress, and thought that he presented some valid concerns.
We now have had enough time to see how his predictions panned out. To put it bluntly: they did not. Of his various scenarios, the one which most closely follows actual CO2 levels since then (Scenario B) showed a 1 C increase in earth temperature. Actual increase: 0 C.
The scenario which most closely follows actual temperature (scenario C) was a “zero increase in CO2 emissions” scenario — something which obviously did not happen.
In the late 1980s, Hansen predicted a rise in sea level of several feet, heat increases which could be felt, ubiquitous high winds, etc.
Summary. The predictions made by Hansen have failed. His predictions have erred in the pro-AGW direction. Conclusion: his predictions are not to be trusted.
Quote
November 28, 2009, 3:56 pmSG says:
This debate has been very enlightening for me (I am not a scientist), but one of the points which did not ring true was the contention that the AGW side has all of the power and funding and the “skeptic” or “denier” side is a group of unaffiliated sincere truth-seekers with no other agenda than scientific truth and no connection to corporate interests which benefit mightily from the status quo (whatever it is). Funding and motive is always relevant.
You’re missing the beauty of the scientific method — funding and motive are not relevant. Reproducibility and falsifiability are what’s important. If you make a positive claim, you have to provide enough information for others to be able to independently reproduce it. If you put forth a hypothesis to explain your data, it should generate testable conditions. This holds independently of what your funding source is, what’s your motivation in looking at the problem, etc.
Note that on the these grounds, a lot of climate science fails to make the grade.
Quote
November 28, 2009, 5:16 pmLee Peters says:
Scientists Caught with Pants Down on Special Climate Change Episode of Cheaters TV Show
Police Round Up Notorious Gang “The Hockey Stick CRU” in International Sting Operation
Quote
November 28, 2009, 5:32 pmRobert Ayers says:
I thank the commenter “Nobody At All” for replying to my post about “raw data” and the GISS and USHCN. “Nobody” suggest that
Here’s GHCN data, here’s USHCN data, and here’s the GISTEMP source code; you may also be interested in the page of internet-accessible data sources compiled by the scientists.
I saw that UXHCN data before I posted. It is valuable. It is not as-observed data. The USHCN link that “Nobody” provides is to the FTP site ftp://ftp.ncdc.noaa.gov/pub/data/ushcn/v2/monthly. A typical file at that FTP location contains data “rows”. Each row contains twelve monthly values for one US station and year. Like this: 14585631961 312 374 422 476 591 707 769 761 623 557 372 246 517
The above says that the temperature at station 145856 in January 1961 was 31.2. Since I used a file with “raw-avg” in its name, the README tells me that the value is “the average of un-adjusted mean monthly maximum and minimum temperatures”. (I could instead have chosen a “tob-avg” file: “the average of mean monthly maximum and minimum temperatures adjusted only for the time of observation bias”)
That all sounds fairly “raw”. And it is certainly much better than data with all manner of “adjustments”. But real “raw” would be like “the observation at station 145856 on 3 January 1961 at 12:02pm was 38.8″.
The FTP site offers me the choice of a mean monthly “minimum” or “maximum” or (as I read it) (min+max)/2. For creating time-series, I would really like to deal with a sequence of observed values (like the temperature as-observed every noon) rather than extrema.
Quote
November 28, 2009, 6:26 pmNobody At All says:
That’s “real” raw data. I could be wrong (and happily retract, in advance, if I am so shown), but I don’t think that the lack of temperature data for every minute of January 1961 is due to scientists acting badly, but rather the technological limits of data storage in 1961.
Quote
November 28, 2009, 6:34 pmNobody At All says:
I think that I read Robert uncharitably, for which I apologize. You are asking for daily data, not minute-by-minute, right? This type of historical daily data (that day’s low/high/average) is probably available from NWSs (I think), though I don’t believe that minute-by-minute data is.
Quote
November 28, 2009, 6:40 pmXcontra says:
I am a scientist, Prof. Volokh. I think you are focused on the real nut, here: the CRU’s data set. Good.
Comments:
If you think about it that way, their stonewalling makes a lot of sense.
Their data set sounds like a nightmare, and it is normal fiduciary diligence for them to prevent any competitor to view it out in the sunlight, where all the makeup and cosmetic surgery is revealed in all its nastiness.
Just have a programmer at UCLA have a look through the HARRY_READ_ME.txt file. Every one of us number crunchers have toiled on complex programming projects and made our own pleading log files like the HARRY_READ_ME.txt log. It shows a mare’s nest of tinkering and finagling that cannot be unwound to get back to the initial data set. To a scientist, this is a severe fault, causing Harry’s heart to cry out in that log file. But CRU is not a small time egghead project — they are global players in this multi-trillion dollar Kyoto/Copenhagen scheme, spectacularly leveraged on this data set.
Over-leveraged, it looks like. That data set is now crumbling.
Quote
November 28, 2009, 6:57 pmRobert Ayers says:
“Nobody” asks “You are asking for daily data, not minute-by-minute, right?”
Yes. My belief is that an official weather station takes several readings per day at nominal times, like noon and midnight. (And the innocent “time of observation” correction is applied when the actual times do not quite match the desired nominal times: you arrived late and the “noon” observation was really taken at 12:15.)
For creating a time series, I’d settle for monthly averages of particular observation times: “January average of all 31 noon observations”. But that doesn’t seem to be on offer at the USHCN.
Quote
November 28, 2009, 6:57 pmNobody At All says:
You’re right, it isn’t. To my best of my knowledge, however, climate data does not rely upon noontime readings rather than average daily temperatures, nor is there is reason to believe that the former is more indicative of that day’s temperature than the latter. (e.g. (a) a storm passes overhead at 11 am, suppressing sunshine, but is gone by 1 pm; (b) a storm passes overhead at 1 pm, suppressing sunshine, but is gone by 3 pm. Unclear as to why a noontime temperature would be a more reliable indicator of that day’s temperature.)
In any case, if you wish to create a time-series, the data is available.
Quote
November 28, 2009, 7:19 pmRobert Ayers says:
To my best of my knowledge, however, climate data does not rely upon noontime readings rather than average daily temperatures, nor is there is reason to believe that the former is more indicative of that day’s temperature than the latter.
I agree. I used the example just due to my understanding about how standard weather stations historically operated.
If I could access data-sets of the form “temperature averaged over 24 hours” that would be fine too. (Athough the definition of “average” there is a bit tricky, since, as you noted, one doesn’t record the value every minute, and probably not even at uniform intervals.)
But, as I noted, the available values at USHCN seem to be a) average of January’s 31 minima (time-sampling to observe an extremum unknown); b) average of 31 maxima; c) average of 31 (min+max)/2.
Let me add here that I thank “Nobody” for his conversation. And this small discussion (which I started) is really a sideshow. I agree it is likely that I could get “real as observed” (noon, whatever) values from the US stations with some digging and begging. Much more difficult for result-replication will be all the data-dependent processing steps, like reproducing the CRU triangulation of the world’s reporting stations and myriad interpolations.
Quote
November 28, 2009, 7:45 pmNobody At All says:
Thanks, Robert. I enjoyed this as well.
Tangentially, a paper that I found interesting: Tim Mitchell & Phil Jones, “An Improved Method of Constructing a Database of Monthly Climate Observations and Associated High-Resolution Grids (2005)” (warning: PDF).
I do not know whether this methodology was used, e.g. in the Harry_Read_Me.txt file, but interesting nonetheless.
(edited link.)
Quote
November 28, 2009, 8:05 pmBig Al says:
Reasonable person says...“The flaw in your agument is that the deniers like McIntyre are not interested in “scientific research” at all. The deniers are threatening the entire safety of the human race by refusing to accept the settled science. Deniers are not interested in “argument” — they just don’t understand science.”
What a load of pure, unadulterated poppycock. “Safety of the human race” was never at question here, these guys were more worried about the “safety of their exhaulted position and assurance of their cash stream”
If these “climate scientists” are so concerned about the human condition then they should release their data, code and methodology for full public peer review (not a behind-closed-doors review by their buddies either). Good science isn’t something that a small group of “smart guys” hoards and dispenses to the the rest of the world because they “know better and have a higher calling”.
How about a little humility and a good dollop of “I could very well be wrong”? Their numbers didn’t fit the decline which means that they still didn’t understand the modalities at work! All of this “rush out and save the world” BS is exactly that, BS. Just a smokescreen to cover up their very human traits of dishonesty, arrogance and selfish greed.
Quote
November 28, 2009, 10:20 pmKing Lee says:
If a scientist writes a paper based on unpublished data, the paper has less credibility than one on which the data is published or can be recreated. Science is not based on faith or authority. It does not matter whether or not the reason not to release the data is legitimate or not.
Klee12
Quote
November 29, 2009, 1:00 amSamA says:
It’s customary in space science for the Primary Investigator (the prof or group who built a particular instrument on a spacecraft) to have exclusive access to the data from that instrument for a year after the data is collected. That gives the PI a head start, so that his papers will usually be the first published with that data. Those first papers usually contain the most surprising and publishable results, so they are more likely to be published in prominent journals. This (ideally) enhances the PI’s reputation, which is his “payment” for the effort building the instrument in the first place.
After the year is up, the data are supposed to be released to all comers.
IMHO, there’s no reason for a researcher to release (say) tree-ring data he’s collecting, until he’s analyzed it and written a paper on it. The data should be released simultaneously with the paper’s publication, however. In a scientific field that depends on data-gathering rather than experiment, the data is everything. Results are not reproducible without the original data. If it’s not reproducible, it’s not science.
Quote
November 29, 2009, 1:02 amneurodoc says:
Sorry, but I don’t understand what you are saying either as to federal research grants or a drug company with a new drug for human use.
Do you know it know it to be the case that grant recipients are obliged as a condition of their grants to make public all the data that they collect in the course of their sponsored research save for that data which might be exempted under a FOIA exception, e.g., privacy of individuals?
Drug companies that want approval to marked a new drug need the approval of the FDA to do so, and to get that approval, they must turn over to the FDA all the data that the FDA requires of them in order to determine safety and efficacy, to decide on labeling and advertising claims, to monitor experience with the drug after it is on the market, etc.
Quote
November 29, 2009, 1:25 amneurodoc says:
What authority do you have for that broad claim?
This document prepared in 2001 may be helpful here. Speaking of OMB A-110, a highly relevant document where federal research money is concerned, it says inter alia:
Quote
November 29, 2009, 1:41 amRM3 Frisker FTN says:
An English professional software developer has performed an impromptu multi-day code review of the leaked FOIA.zip source code. Link here ... http://di2.nu/blog.htm ... suffice it to say no one involved with FAA Avionics Software Development or FDA Life Support Software Development would give the climate source code a passing grade. Check it out!
Eric Raymond, noted open-source Linux guru and author of best selling “The Cathedral and the Bazaar”, makes a convincing argument that climate science MUST be ‘open-sourced’. Link here ... http://esr.ibiblio.org/?p=1436. In other postings on his blog, Eric is looking at the the leaked FOIA.zip source code, finding the emails a distraction from the underlying climate model programming. Check it out!
Quote
November 29, 2009, 2:27 amPaulD says:
Pot meet Kettle wrote:
“he might be scrupulous about posting his data, but not scrupulous about actually using data properly. i don’t blame him. since he is a mediocrity, he has to massage the data to get anti global warming results so somebody will quote him and make him an expert.
mcintyre fail1
mcintyre fail2
mcintyre fail3”
It is interesting to me that all three hyperlinks go to RealClimate.org, which is the website run by the persons whose work McIntrye is criticizing. It is also a website that routinely blocks comments by persons who disagree with their position. McIntrye consistently replies to threads from realclimate.org at his own website. He replies on his own website because his comments are blocked at Realclimate, even when they are discussing his work.
Realclimate is where I go If I want to read a one-sided viewpoint expressed by AGW partisans. It has little value for anything else.
Quote
November 29, 2009, 8:36 amLaura(southernxyl) says:
Neurodoc, my point is that it’s not just that the AGW folks have data we’d like to see, that we paid for them to gather.
What do you suppose they’d like to do in Copenhagen, and what they’re using to sell that to us? And what it’s going to cost us?
Quote
November 29, 2009, 9:30 amreasonable parsons says:
I don’t know why you people are giving pot meet kettle such a hard time. I mean, it’s only fair that exacting standards of evidence be used against Denier Enemies of Science and mediocrities like McIntyre who attempt to examine and question Real Scientists, but we should not question the infallable Science of Mann and other AGW promoters, who are doing God’s work.
Where they are concerned, pot meet kettle is right: Your job is to heed and obey! You are not entitled to see their data, and should not have access to it, and since you don’t have legitimate access to it, you have no evidence with which to question what they say!
Real Scientists are those who go along with the consensus, peer-review each other’s work to support the consensus! Anyone who dares to deny the Consensus is simply to be dismissed as a mediocrity, a crank in the service of corporations and an ideologue!
Real Scientists and decent people like pot meet kettle and I will speak power to truth until you Heed and obey! Gaia cries out!
Quote
November 29, 2009, 9:50 amreasonable parsons says:
I meant “speak truth to power”, of course.
Quote
November 29, 2009, 9:51 amLaura(southernxyl) says:
No, you had it right the first time.
Quote
November 29, 2009, 9:59 amXcontra says:
I would prefer to speak the truth to Harry.
Quote
November 29, 2009, 3:11 pmHarryEagar says:
Robert Ayers and Nobody. Not only do you need raw data, you need global raw data.
No such animal exists.
If you want to reconstruct from raw data what the climate was in the United States and western Europe, you probably could.
If you wanted to use that to determine what the climate of the whole globe was, though, you’d have to pull that out of your fundament, like Jones did.
Quote
November 30, 2009, 1:29 amRobert Ayers says:
HarryEagar points out that global raw data is unobtanium. I agree.
My “plan” is to work with a subset of the US surface stations and draw conclusions (e.g. essentially no net warming, other than urban heat island, since 1998 or indeed since 1936) and then state “If you think that these conclusions would be much different if one included the rest of the globe, feel free to try to collect the data and add to the science.”
Quote
November 30, 2009, 1:44 amVictoria Stodden says:
Some of the problem is that our publishing mechanisms for scientific results don’t take the use of the (relatively new) computational tools into account — so we end up in situations like this where reproducibility and verification of computational results is essentially impossible (as HARRY_README.txt showed). Computational science needs to adapt to sharing code and data so we can have some realistic notion of reproducibility. I wrote a blog post about this here: http://blog.stodden.net/2009/11/30/the-climate-modeling-leak-code-and-data-generating-published-results-must-be-open-and-facilitate-reproducibility/ .
Quote
December 3, 2009, 6:02 pm