Hacked E-mails from Climate Scientists released

[MKSheppard]

I think the best aftereffect of this is the New York Times suddenly discovering an ethical responsibility:

The documents appear to have been acquired illegally and contain all manner of private information and statements that were never intended for the public eye, so they won’t be posted here.

[MKSheppard]

Imagine being a scientist who is competing with others.

Isn't much of the work climate scientists do funded by governments and thus legally subject to FOIA/FOI acts in the US/UK?

[Bakustra]

Imagine being a scientist who is competing with others.

Isn't much of the work climate scientists do funded by governments and thus legally subject to FOIA/FOI acts in the US/UK?

On the other hand, shouldn't their work be subjected to protection from people trying to steal it? Seriously, does FOIA permit private citizens to request the tax returns of other people or the blueprints of military equipment? Those are considered private for a reason, so why should unpublished data not be?

[gizmojumpjet]

Imagine being a scientist who is competing with others.

Isn't much of the work climate scientists do funded by governments and thus legally subject to FOIA/FOI acts in the US/UK?

On the other hand, shouldn't their work be subjected to protection from people trying to steal it? Seriously, does FOIA permit private citizens to request the tax returns of other people or the blueprints of military equipment? Those are considered private for a reason, so why should unpublished data not be?

Requesting access to data derived using the public purse, and which is being used to argue for sweeping changes to public policy, cannot honestly be described as stealing.

Tax returns and military blueprints are not remotely analogous to weather data.

[Bakustra]

Isn't much of the work climate scientists do funded by governments and thus legally subject to FOIA/FOI acts in the US/UK?

On the other hand, shouldn't their work be subjected to protection from people trying to steal it? Seriously, does FOIA permit private citizens to request the tax returns of other people or the blueprints of military equipment? Those are considered private for a reason, so why should unpublished data not be?

Requesting access to data derived using the public purse, and which is being used to argue for sweeping changes to public policy, cannot honestly be described as stealing.

Tax returns and military blueprints are not remotely analogous to weather data.

The same could exactly be said for tax returns, for the purpose of statistical analysis of current tax policy, and the same for military blueprints and defense policy/spending. Those are also derived using the public purse. Further, the primary reason to prevent people from using FOIA to gain access to unpublished data is to prevent them from plagiarizing it. Or do you somehow think that there are no unscrupulous individuals in the scientific world?

Furthermore, why can't people wait for the data to be published, at which point you can use it freely? Please explain why scientific data must be made available (particularly weather data) immediately. Do you also fear depravity and nefarious intent (or whatever it is that motivates you to make this argument) when Hubble and other space telescope data is held and interpreted for a year after capture, during which it is off-limits?

[Simon_Jester]

The same could exactly be said for tax returns, for the purpose of statistical analysis of current tax policy, and the same for military blueprints and defense policy/spending. Those are also derived using the public purse.

Not sure I agree. For one, some governments will send you tax returns with the names and other identifying information blacked out. I think a professor I know got some returns from the US government precisely to do a statistical analysis of the American income distribution, for instance.

As for military blueprints, the problem is that the information has a specific, well defined, obvious nefarious use. Weather data does not.

[Bakustra]

The same could exactly be said for tax returns, for the purpose of statistical analysis of current tax policy, and the same for military blueprints and defense policy/spending. Those are also derived using the public purse.

Not sure I agree. For one, some governments will send you tax returns with the names and other identifying information blacked out. I think a professor I know got some returns from the US government precisely to do a statistical analysis of the American income distribution, for instance.

As for military blueprints, the problem is that the information has a specific, well defined, obvious nefarious use. Weather data does not.

Okay, but the point was that there are forms of data that are considered to be protected from FOIA requests. Why should unpublished scientific data (that is being processed by a research team) be freely available for anyone to plagiarize? Why should weather data be held to a different standard than astronomical data, or other scientific data that is explicitly (at least in the US) protected from all FOIA requests for a period of time? Of course, I am not necessarily talking about the data mentioned in the stolen e-mails, but rather in general terms, as the information within appears to date from the 1980s.

On another note, I wonder if zoologists'/evolutionary biologists' e-mails contain similar vitriol concerning creationists, or if theirs is even harsher in tone. I also find it saddening that every two-bit global warming denier is going to wave these e-mails in people's faces while screaming "conspiracy!"

[MKSheppard]

Seriously, does FOIA permit private citizens to request the tax returns of other people or the blueprints of military equipment?

I believe tax return information becomes public after the death of the person -- it's why there's a 75 year limit on raw data from the Census; e.g. we're coming up on the date that all the cards filled out in 1940 for the census will become available.

And yes; you can get lots of blueprints of military equipment. Just ask in the right places.

[gizmojumpjet]

Okay, but the point was that there are forms of data that are considered to be protected from FOIA requests. Why should unpublished scientific data (that is being processed by a research team) be freely available for anyone to plagiarize?

Requesting access to data, reviewing data, and criticizing data and the conclusions drawn using that data is not plagiarism. Publishing the data as one's own would be.

Why should weather data be held to a different standard than astronomical data, or other scientific data that is explicitly (at least in the US) protected from all FOIA requests for a period of time?

Of course, I am not necessarily talking about the data mentioned in the stolen e-mails, but rather in general terms, as the information within appears to date from the 1980s.

Thanks for talking in general terms when we're talking about something specific: data that dates back to the 80's.

On another note, I wonder if zoologists'/evolutionary biologists' e-mails contain similar vitriol concerning creationists, or if theirs is even harsher in tone. I also find it saddening that every two-bit global warming denier is going to wave these e-mails in people's faces while screaming "conspiracy!"

You're saddened that people are interested in reviewing the data and debating the findings? Why do you hate science so much?

[Guardsman Bass]

Imagine being a scientist who is competing with others.

Isn't much of the work climate scientists do funded by governments and thus legally subject to FOIA/FOI acts in the US/UK?

Assuming they are funded by the government in this case.

If I recall correctly, one of the reasons why they were wary to release the data was because they'd gotten a lot of it from a number of National Weather Service equivalents on the condition that they not release it themselves, since those agencies wanted to do that themselves (possibly in a form that costs money).

That said, remember that these guys aren't a monolithic block; Gavin Schmidt over at Realclimate was pretty critical of the guys who were threatening to destroy the data before releasing it.

[MKSheppard]

Assuming they are funded by the government in this case.

They are. I've downloaded the zip file and looked at the excels listing their grants. They got:

DEPT OF ENVIRONMENT £179,484.00
DEPT OF ENVIRONMENT £101,914.00
NATO - NORTH ATLANTIC TREATY ORGANISATION £33,678.00
EUROPEAN UNION - EU / CEC £59,446.00
EUROPEAN UNION - EU / CEC £39,833.00
NATIONAL RIVERS AUTHORITY £4,800.00
EUROPEAN UNION - EU / CEC £62,605.00
NERC £104,230.00
US DEPT OF ENERGY £128,000.00
EUROPEAN UNION - EU / CEC £42,464.00
MASSACHUSETTS UNIV. £20,953.00
NATIONAL RIVERS AUTHORITY £19,780.00
NERC £18,639.00
EUROPEAN UNION - EU / CEC £209,843.00
US DEPT OF ENERGY £132,000.00
UNIV CORP ATMOS RES £12,903.00
NERC £24,859.00
US DEPT OF ENERGY £125,000.00
EUROPEAN UNION - EU / CEC £183,333.00
EUROPEAN UNION - EU / CEC £71,333.00
EUROPEAN UNION - EU / CEC £33,333.00
US DEPT OF ENERGY £99,555.00
BRITISH SUGAR PLC/IACR - BROOM'S BARN £8,148.00
NERC £92,383.00
US DEPT OF ENERGY £102,752.00
UK WATER INDUSTRY RESEARCH LTD £5,000.00
US DEPT OF ENERGY £106,151.00
EUROPEAN UNION - EU / CEC £26,809.00
EUROPEAN UNION - EU / CEC £123,600.00
EUROPEAN UNION - EU / CEC £99,492.00
NERC £39,255.00
US DEPT OF ENERGY £212,500.00
NERC £34,660.00
HEFCE / JIF £6,608,541.00
MET OFFICE £45,052.00
EUROPEAN UNION - EU / CEC £178,812.00
SCOTTISH OFFICE £14,164.00
CEC £134,978.00
EPSRC £105,178.00
EPSRC £69,125.00
NERC £226,981.00
MET OFFICE £13,870.00
ENVIRONMENT AGENCY £9,000.00
EUROPEAN UNION - EU / CEC £105,499.00
US DEPT OF ENERGY £262,629.00
MET OFFICE £14,492.00
EUROPEAN UNION - EU / CEC £426,895.00
SCOTTISH ENVIRONMENT PROTECTION AGENCY £730.00
ENVIRONMENT AGENCY £26,516.00
ENVIRONMENT AGENCY £52,177.00
CEC £84,431.00
COUNCIL FOR THE CENTRAL LAB. OF THE RES. COUNCILS £50,000.00

Since 1990

[Razorgeist]

Yeah Limbaugh was going nuts over this whole thing today.

[Alyrium Denryle]

So you'd commit a crime rather than comply with the law and turn your data over to a FOIA request?

No. But I would sure as fuck bitch about it and try to find a way around it.

You're assuming the statement is truthful.

No real reason to assume it is not.

Considering the threat to delete data rather than submitting to the law and answering a FOIA request, the statement is dubious.

Not at all. The statement made on the matter is perfectly reasonable. Raw data, which is what was requested, is often very very useless until it is compiled, and put into a database, transformed into a model (like a structural equation model). Often times one does not even keep the original data file as a matter of course depending on what you are doing. The data was not deleted. It was moved, transformed and subsumed into other databases to make it useful. Why the fuck would a scientist delete his life's work exactly? If we dont publish we perish.

Requesting access to data derived using the public purse, and which is being used to argue for sweeping changes to public policy, cannot honestly be described as stealing.

All of my data has been collected so far under funding from public grants. Science is Science regardless of what its social implications are. I work with frogs. Do you think my data should be subject to being FOI Acted? What happens to my intellectual property rights? How do I assure that I do not get scooped by a lazy competitor? How exactly is the raw data requested even useful to someone who does not have the expertise to analyze it?

On another note, I wonder if zoologists'/evolutionary biologists' e-mails contain similar vitriol concerning creationists, or if theirs is even harsher in tone.

I can answer that for you.

Yes. It is actually worse. I know because I write the emails

Requesting access to data, reviewing data, and criticizing data and the conclusions drawn using that data is not plagiarism. Publishing the data as one's own would be.

Do you understand how criticizing data is pointless? The only reason to take the raw data through a FOIA request is to publish it yourself, or sling accusations of data falsification after publication, because the data after statistical transformations etc does not appear exactly as the raw data. You do not criticize the data points prior to publishing of the analysis. Why? You dont know how it was collected. You get a string of numbers. Were I to give you the data in a way you could do anything with it, I might as well hand you a road map to plagiarism.

The individuals filing FOIA requests I can almost guarantee were not scientists. We dont file those to get access to other people's data. If we want someone else's data we ask after publication, or with an offer of collaboration on a joint project.

This rules out reviewing and criticizing and leaves plagiarism or accusations of data falsification. Do you get why someone would be pissed?

Thanks for talking in general terms when we're talking about something specific: data that dates back to the 80's.

When data storage and database capacity and recall were very very limited and it would not have been prudent to keep raw data.

You're saddened that people are interested in reviewing the data and debating the findings? Why do you hate science so much?

Because it is not being done in good faith. Moron.

Assuming they are funded by the government in this case.

Even if they are that does not IIRC subject their data to FOIA requests. They have to be an actual government lab like the NIH lab, and I think even then their raw data is protected. I could be wrong there though.

As far as I am aware, the agencies are required to release documents such as reports and meeting minutes but not raw data.

[hongi]

Whatever some of these scientists may have done, it does not mean that the theory of global warming has gone kapoof - and yet that's what the skeptics are screaming at the top of their lungs. The sheer hysteria I'm reading is a bit breathtaking in light of the fact that we already know that scientists can do wrong, remember that Korean geneticist who faked his cloning research?

You're saddened that people are interested in reviewing the data and debating the findings? Why do you hate science so much?

This isn't science, this is a witch hunt. I don't like using that term, in fact I don't think I've so much as said it in years, but it's perfectly applicable here. You think millions of people on the internet are going to do 'science' using these emails?

[Bakustra]

Seriously, does FOIA permit private citizens to request the tax returns of other people or the blueprints of military equipment?

I believe tax return information becomes public after the death of the person -- it's why there's a 75 year limit on raw data from the Census; e.g. we're coming up on the date that all the cards filled out in 1940 for the census will become available.

And yes; you can get lots of blueprints of military equipment. Just ask in the right places.

And scientific data is generally released after a period of time, too. Of course, asking for the blueprints to the F-22, or the exact makeup of the M1's armor through FOIA, and you'd be laughed at. We also have to take context into account, too. Another member of the scientific community is more likely to ask politely or wait for the paper to be published than to try to go through FOIA.

Okay, but the point was that there are forms of data that are considered to be protected from FOIA requests. Why should unpublished scientific data (that is being processed by a research team) be freely available for anyone to plagiarize?

Requesting access to data, reviewing data, and criticizing data and the conclusions drawn using that data is not plagiarism. Publishing the data as one's own would be.

The point is that allowing FOIA requests on unpublished data would allow people to plagiarize that data for use in their own work far more easily. In fact, there is already a one-year protection on any data from Hubble, a government-owned space telescope, during which only the researcher who made use of the Hubble time can use it. Furthermore, you seem to be making a hasty assumption here, but I'll get to that with the rest of your post.

Why should weather data be held to a different standard than astronomical data, or other scientific data that is explicitly (at least in the US) protected from all FOIA requests for a period of time?

Of course, I am not necessarily talking about the data mentioned in the stolen e-mails, but rather in general terms, as the information within appears to date from the 1980s.

Thanks for talking in general terms when we're talking about something specific: data that dates back to the 80's.

So you agree that protections should be in place for scientific data. If the data has been published, then it is already freely available, in most of the cases. I will admit that data from the 1980s likely should be available. Of course, your presumption appears to be that these scientists are hiding some of the data, presumably to hide the truth about global warming from everybody, and thus only by looking through the raws can we find the truth. Lovely implications of scientific dishonesty on their part.

On another note, I wonder if zoologists'/evolutionary biologists' e-mails contain similar vitriol concerning creationists, or if theirs is even harsher in tone. I also find it saddening that every two-bit global warming denier is going to wave these e-mails in people's faces while screaming "conspiracy!"

You're saddened that people are interested in reviewing the data and debating the findings? Why do you hate science so much?

Why are you brain-dead? You seem to conclude several things here, not all of them unjustified, I admit.

1)Everybody has an equal ability to review the raw data and "debate the findings." Let me use myself, my sister, and a climatologist as examples. I am a first-year university student of Physics. My sister is a first-year middle-schooler, and the climatologist is a Doctor of Climatology or Doctor of Meteorology with years of studying and analysis of meteorological data under his belt. I'm absolutely sure that we all are equally able to evaluate the raw data. Absolutely so.

2)Well, as far as I can tell, point 2 is the cry of "conspiracy" I mentioned in my post. Hit a little close to home? You appear to state that the data should be made available once it's collected, simply because it's necessary to have more debate on the issue; AKA give the deniers more data to misinterpret. Of course, you also seem to think, judging from these implications and this previous point, that scientific papers do not provide the methodology used for statistical analysis, because otherwise you'd simply buy a copy of the International Journal of Climatology and tear right through dodgy papers.

3)There is some pressing need for the data. Will SPECTRE destroy the world if somebody doesn't hand over the observation of tree-ring growths, 1963-1990? Of course, I believe that scientific data should become freely available, but why not wait for the paper to be published, apart from conspiracy theories involving a grand coalition of climatologists, seeking to convince everybody to reduce the output of greenhouse gasses and slow global warming for some nefarious yet oddly ill-defined reason.

Whatever some of these scientists may have done, it does not mean that the theory of global warming has gone kapoof - and yet that's what the skeptics are screaming at the top of their lungs. The sheer hysteria I'm reading is a bit breathtaking in light of the fact that we already know that scientists can do wrong, remember that Korean geneticist who faked his cloning research?

You're saddened that people are interested in reviewing the data and debating the findings? Why do you hate science so much?

This isn't science, this is a witch hunt. I don't like using that term, in fact I don't think I've so much as said it in years, but it's perfectly applicable here. You think millions of people on the internet are going to do 'science' using these emails?

Actually, I think he's talking about a conspiracy to shut GW-denying scientists out of the loop somehow, but your interpretation is funnier and that's what I went with. Of course, said conspiracy would then need some method to stop the "rebels" from being able to read their papers. I guess they'll need a treehouse or something. Of course, we can't take the private e-mails of a number of individuals and generalize to the whole scientific population or assume that everybody involved has the exact same opinions on the subject matter.

[gizmojumpjet]

Not at all. The statement made on the matter is perfectly reasonable. Raw data, which is what was requested, is often very very useless until it is compiled, and put into a database, transformed into a model (like a structural equation model). Often times one does not even keep the original data file as a matter of course depending on what you are doing. The data was not deleted. It was moved, transformed and subsumed into other databases to make it useful. Why the fuck would a scientist delete his life's work exactly? If we dont publish we perish.

How can other researchers replicate the results if the raw data has gone down the rabbit hole? The raw data existed at one time, and now (apparently) does not. That is the definition of "deleted." It has been made to no longer exist.

All of my data has been collected so far under funding from public grants. Science is Science regardless of what its social implications are. I work with frogs. Do you think my data should be subject to being FOI Acted? What happens to my intellectual property rights? How do I assure that I do not get scooped by a lazy competitor? How exactly is the raw data requested even useful to someone who does not have the expertise to analyze it?

Why shouldn't data you collected using the people's money be available to the people? They paid for it. The product of the work I do while on the clock becomes my employer's property; what makes you any different?

Do you understand how criticizing data is pointless? The only reason to take the raw data through a FOIA request is to publish it yourself, or sling accusations of data falsification after publication, because the data after statistical transformations etc does not appear exactly as the raw data. You do not criticize the data points prior to publishing of the analysis. Why? You dont know how it was collected. You get a string of numbers. Were I to give you the data in a way you could do anything with it, I might as well hand you a road map to plagiarism.

How can anyone duplicate the results of the research if they can't get access to the data?

The individuals filing FOIA requests I can almost guarantee were not scientists.

It doesn't matter; also, can you support your assertion that no actual scientists submitted FOIA requests for climate data from CRU? Deleting data subject to a FOIA request is a crime regardless of who is submitting the request.

When data storage and database capacity and recall were very very limited and it would not have been prudent to keep raw data.

The 80's weren't the Dark Age of Computers. This mealy mouthed bullshit about data storage and retrieval being too hard and complicated sounds like a ruse especially when coming from a research university. If anyone should be on the cutting edge of data retrieval and storage it should be those folks in the business of generating large amounts of data. The fact that they did not or were not able to sounds to me like negligence or incompetence at the very least.

Because it is not being done in good faith. Moron.

Good faith? The content of the leaked emails does not instill a great sense in me that the CRU is operating in good faith, and has rather attempted to subvert the peer review process.

Even if they are that does not IIRC subject their data to FOIA requests. They have to be an actual government lab like the NIH lab, and I think even then their raw data is protected. I could be wrong there though.

Talking about deleting data rather than turn it over to a FOIA request makes it seem to me like we're talking about data that's actually subject to such a request. Maybe it wasn't, but if you're going to talk about deleting data rather than sharing it when required to by law, you've just tossed all your credibility out the window.

[gizmojumpjet]

The point is that allowing FOIA requests on unpublished data would allow people to plagiarize that data for use in their own work far more easily. In fact, there is already a one-year protection on any data from Hubble, a government-owned space telescope, during which only the researcher who made use of the Hubble time can use it. Furthermore, you seem to be making a hasty assumption here, but I'll get to that with the rest of your post.

We're talking about data upon which papers, many of them, have already been published.

So you agree that protections should be in place for scientific data. If the data has been published, then it is already freely available, in most of the cases. I will admit that data from the 1980s likely should be available. Of course, your presumption appears to be that these scientists are hiding some of the data, presumably to hide the truth about global warming from everybody, and thus only by looking through the raws can we find the truth. Lovely implications of scientific dishonesty on their part.

Science isn't founded on trust, its founded on data and results that can be replicated and reviewed. But we don't have the original data anymore, we only have the "value-added" data; how can another we determine if the "value-added" dataset is reliable when we cannot replicate the process by which it was arrived at?

Why are you brain-dead? You seem to conclude several things here, not all of them unjustified, I admit.

1)Everybody has an equal ability to review the raw data and "debate the findings." Let me use myself, my sister, and a climatologist as examples. I am a first-year university student of Physics. My sister is a first-year middle-schooler, and the climatologist is a Doctor of Climatology or Doctor of Meteorology with years of studying and analysis of meteorological data under his belt. I'm absolutely sure that we all are equally able to evaluate the raw data. Absolutely so.

I'm not making the argument that any Tom, Dick, or Harry can take the raw data and make valid conclusions based on it. There are people who could, however, if only they had access to it.

2)Well, as far as I can tell, point 2 is the cry of "conspiracy" I mentioned in my post. Hit a little close to home? You appear to state that the data should be made available once it's collected, simply because it's necessary to have more debate on the issue; AKA give the deniers more data to misinterpret. Of course, you also seem to think, judging from these implications and this previous point, that scientific papers do not provide the methodology used for statistical analysis, because otherwise you'd simply buy a copy of the International Journal of Climatology and tear right through dodgy papers.

I've never said data should be made available once it's collected, but data collected thirty years ago certainly should be, especially if published papers have been based on it, which they have.

Also, "deniers" can only "misinterpret" the data, huh?

3)There is some pressing need for the data. Will SPECTRE destroy the world if somebody doesn't hand over the observation of tree-ring growths, 1963-1990? Of course, I believe that scientific data should become freely available, but why not wait for the paper to be published, apart from conspiracy theories involving a grand coalition of climatologists, seeking to convince everybody to reduce the output of greenhouse gasses and slow global warming for some nefarious yet oddly ill-defined reason.

Papers already have been published based on the data in question.

[Coyote]

Well now that the secret is out, obviously we need to reprogram our Climate Change Doom Machine to directly target the right wing heartland.

More tornadoes for the trailer parks!

Actually, there is a paranoid fringe out there that sincerely believes that government weather-control satellites are used to make tornadoes to attack trailer parks and wipe out "the poor".

And they're serious.

[Bakustra]

The point is that allowing FOIA requests on unpublished data would allow people to plagiarize that data for use in their own work far more easily. In fact, there is already a one-year protection on any data from Hubble, a government-owned space telescope, during which only the researcher who made use of the Hubble time can use it. Furthermore, you seem to be making a hasty assumption here, but I'll get to that with the rest of your post.

We're talking about data upon which papers, many of them, have already been published.

So you agree that protections should be in place for scientific data. If the data has been published, then it is already freely available, in most of the cases. I will admit that data from the 1980s likely should be available. Of course, your presumption appears to be that these scientists are hiding some of the data, presumably to hide the truth about global warming from everybody, and thus only by looking through the raws can we find the truth. Lovely implications of scientific dishonesty on their part.

Science isn't founded on trust, its founded on data and results that can be replicated and reviewed. But we don't have the original data anymore, we only have the "value-added" data; how can another we determine if the "value-added" dataset is reliable when we cannot replicate the process by which it was arrived at?

Why are you brain-dead? You seem to conclude several things here, not all of them unjustified, I admit.

1)Everybody has an equal ability to review the raw data and "debate the findings." Let me use myself, my sister, and a climatologist as examples. I am a first-year university student of Physics. My sister is a first-year middle-schooler, and the climatologist is a Doctor of Climatology or Doctor of Meteorology with years of studying and analysis of meteorological data under his belt. I'm absolutely sure that we all are equally able to evaluate the raw data. Absolutely so.

I'm not making the argument that any Tom, Dick, or Harry can take the raw data and make valid conclusions based on it. There are people who could, however, if only they had access to it.

2)Well, as far as I can tell, point 2 is the cry of "conspiracy" I mentioned in my post. Hit a little close to home? You appear to state that the data should be made available once it's collected, simply because it's necessary to have more debate on the issue; AKA give the deniers more data to misinterpret. Of course, you also seem to think, judging from these implications and this previous point, that scientific papers do not provide the methodology used for statistical analysis, because otherwise you'd simply buy a copy of the International Journal of Climatology and tear right through dodgy papers.

I've never said data should be made available once it's collected, but data collected thirty years ago certainly should be, especially if published papers have been based on it, which they have.

Also, "deniers" can only "misinterpret" the data, huh?

3)There is some pressing need for the data. Will SPECTRE destroy the world if somebody doesn't hand over the observation of tree-ring growths, 1963-1990? Of course, I believe that scientific data should become freely available, but why not wait for the paper to be published, apart from conspiracy theories involving a grand coalition of climatologists, seeking to convince everybody to reduce the output of greenhouse gasses and slow global warming for some nefarious yet oddly ill-defined reason.

Papers already have been published based on the data in question.

I am talking about the general idea of whether scientific data should be protected from FOIA requests or not. You seem to be fixating on the idea that the data should be freely available, whereas I am arguing that the protection of scientific papers from plagiarism is more important. In other words, I support the idea of making data freely available after a certain point in general. Of course, you are avoiding the implications of your own statements: if FOIA applies to all scientific data, then what stops unscrupulous individuals from simply using FOIA requests to steal the material. I am not familiar with the means of climatology and the analysis of temperatures, but what you seem to be saying is that scientific papers are invalid if all the raw data used is not published with them.

Would you be willing to provide more context on what these "MMs" mentioned in the e-mail are? You seem to be claiming that these are scientists that aren't receiving any of the data, especially given that the person in question mentions sending station data to another individual in the same e-mail. Justify this, please. I also ask you to point me to the papers that have been published and ask you why you think that FOIA is an appropriate tool for ensuring scientific openness if you do not support the idea that unqualified persons are equally valid in their conclusions.

As for your remarks, yes, I think that global-warming deniers tend to misinterpret data to accommodate their pet theories, so I feel that it is in the public interest to not grant them special access to raw scientific data. I feel the same way about creationists as well.

[Razorgeist]

Well now that the secret is out, obviously we need to reprogram our Climate Change Doom Machine to directly target the right wing heartland.

More tornadoes for the trailer parks!

Actually, there is a paranoid fringe out there that sincerely believes that government weather-control satellites are used to make tornadoes to attack trailer parks and wipe out "the poor".

And they're serious.

Or that the government is using the Arecibo observatory to manipulate the weather and cause droughts and global warming....yeah I wish I was kidding.

[gizmojumpjet]

I am talking about the general idea of whether scientific data should be protected from FOIA requests or not. You seem to be fixating on the idea that the data should be freely available, whereas I am arguing that the protection of scientific papers from plagiarism is more important.

I wish you'd stop talking in generalities when we are discussing specific datasets. Why do you keep doing that? Stop trying to turn this into a discussion about what limits should be placed on FOIA requests. We're discussing specific data that apparently is/was subject to such a request. Whether or not you or I think it should have been subject to such a request is immaterial to the discussion at hand.

In other words, I support the idea of making data freely available after a certain point in general. Of course, you are avoiding the implications of your own statements: if FOIA applies to all scientific data, then what stops unscrupulous individuals from simply using FOIA requests to steal the material. I am not familiar with the means of climatology and the analysis of temperatures, but what you seem to be saying is that scientific papers are invalid if all the raw data used is not published with them.

Kindly refrain from putting words in my mouth. I never said all scientific data should be subject to FOIA requests.

Would you be willing to provide more context on what these "MMs" mentioned in the e-mail are? You seem to be claiming that these are scientists that aren't receiving any of the data, especially given that the person in question mentions sending station data to another individual in the same e-mail.

The MM's referenced are Stephen McIntyre and Ross McKitrick which anyone with half a brain could have determined for themselves.

Justify this, please. I also ask you to point me to the papers that have been published

http://www.cru.uea.ac.uk/cru/pubs/full/

and ask you why you think that FOIA is an appropriate tool for ensuring scientific openness if you do not support the idea that unqualified persons are equally valid in their conclusions.

If someone won't disclose publicly owned data so that it can be reviewed properly, FOIA provides a method by which they can be compelled to do so. That is unless they delete it, which is apparently what they did.

As for your remarks, yes, I think that global-warming deniers tend to misinterpret data to accommodate their pet theories, so I feel that it is in the public interest to not grant them special access to raw scientific data. I feel the same way about creationists as well.

It's not possible to misinterpret data that has been deleted or to which you otherwise have no access. It's also not possible to duplicate results if you don't have access to the data.

[PeZook]

You know, the hackers supposedly stole thousands of emails, yet all I see are endlessly circulated quotes from five or six messages, that are supposed to prove that all climatologists are part of a massive conspiracy to distort data and fake results. It reminds me all too well of moon hoaxers who point out NASA altered a few images from Mars, which obviously means that no moon landing ever happened, right?

Is it too harsh to demand an actual analysis of the material? It seems reasonable, what with cretins attempting to disprove a well established theory based on heaps of data from different sources.

[MKSheppard]

You know, the hackers supposedly stole thousands of emails

Want them? I can put them up on my website.

[Dalton]

You know, the hackers supposedly stole thousands of emails

Want them? I can put them up on my website.

I don't care what you put on your website, but do me a favor and don't talk about it, link to it or reference it here if you do.

[MKSheppard]

Link to CBS News

Congress May Probe Leaked Global Warming E-Mails
Font size Print E-mail Share 5 comments Posted by Declan McCullagh

(AP)A few days after leaked e-mail messages appeared on the Internet, the U.S. Congress may probe whether prominent scientists who are advocates of global warming theories misrepresented the truth about climate change.

Sen. James Inhofe, an Oklahoma Republican, said on Monday the leaked correspondence suggested researchers "cooked the science to make this thing look as if the science was settled, when all the time of course we knew it was not," according to a transcript of a radio interview posted on his Web site. Aides for Rep. Darrell Issa, a California Republican, are also looking into the disclosure.

The leaked documents (see our previous coverage) come from the Climatic Research Unit of the University of East Anglia in eastern England. In global warming circles, the CRU wields outsize influence: it claims the world's largest temperature data set, and its work and mathematical models were incorporated into the United Nations Intergovernmental Panel on Climate Change's 2007 report. That report, in turn, is what the Environmental Protection Agency acknowledged it "relies on most heavily" when concluding that carbon dioxide emissions endanger public health and should be regulated.

Last week's leaked e-mails range from innocuous to embarrassing and, critics believe, scandalous. They show that some of the field's most prominent scientists were so wedded to theories of man-made global warming that they ridiculed dissenters who asked for copies of their data ("have to respond to more crap criticisms from the idiots"), cheered the deaths of skeptical journalists, and plotted how to keep researchers who reached different conclusions from publishing in peer-reviewed journals.

One e-mail message, apparently from CRU director Phil Jones, references the U.K.'s Freedom of Information Act when asking another researcher to delete correspondence that might be disclosed in response to public records law: "Can you delete any emails you may have had with Keith re AR4? Keith will do likewise." Another, also apparently from Jones: global warming skeptics "have been after the CRU station data for years. If they ever hear there is a Freedom of Information Act now in the UK, I think I'll delete the file rather than send to anyone." (Jones was a contributing author to the chapter of the U.N.'s IPCC report titled "Detection of Climate Change and Attribution of Causes.")

In addition to e-mail messages, the roughly 3,600 leaked documents posted on sites including Wikileaks.org and EastAngliaEmails.com include computer code and a description of how an unfortunate programmer named "Harry" -- possibly the CRU's Ian "Harry" Harris -- was tasked with resuscitating and updating a key temperature database that proved to be problematic. Some excerpts from what appear to be his notes, emphasis added:

I am seriously worried that our flagship gridded data product is produced by Delaunay triangulation - apparently linear as well. As far as I can see, this renders the station counts totally meaningless. It also means that we cannot say exactly how the gridded data is arrived at from a statistical perspective - since we're using an off-the-shelf product that isn't documented sufficiently to say that. Why this wasn't coded up in Fortran I don't know - time pressures perhaps? Was too much effort expended on homogenisation, that there wasn't enough time to write a gridding procedure? Of course, it's too late for me to fix it too. Meh.

I am very sorry to report that the rest of the databases seem to be in nearly as poor a state as Australia was. There are hundreds if not thousands of pairs of dummy stations, one with no WMO and one with, usually overlapping and with the same station name and very similar coordinates. I know it could be old and new stations, but why such large overlaps if that's the case? Aarrggghhh! There truly is no end in sight... So, we can have a proper result, but only by including a load of garbage!

One thing that's unsettling is that many of the assigned WMo codes for Canadian stations do not return any hits with a web search. Usually the country's met office, or at least the Weather Underground, show up – but for these stations, nothing at all. Makes me wonder if these are long-discontinued, or were even invented somewhere other than Canada!

Knowing how long it takes to debug this suite - the experiment endeth here. The option (like all the anomdtb options) is totally undocumented so we'll never know what we lost. 22. Right, time to stop pussyfooting around the niceties of Tim's labyrinthine software suites - let's have a go at producing CRU TS 3.0! since failing to do that will be the definitive failure of the entire project.

Ulp! I am seriously close to giving up, again. The history of this is so complex that I can't get far enough into it before by head hurts and I have to stop. Each parameter has a tortuous history of manual and semi-automated interventions that I simply cannot just go back to early versions and run the update prog. I could be throwing away all kinds of corrections - to lat/lons, to WMOs (yes!), and more. So what the hell can I do about all these duplicate stations?...

As the leaked messages, and especially the HARRY_READ_ME.txt file, found their way around technical circles, two things happened: first, programmers unaffiliated with East Anglia started taking a close look at the quality of the CRU's code, and second, they began to feel sympathetic for anyone who had to spend three years (including working weekends) trying to make sense of code that appeared to be undocumented and buggy, while representing the core of CRU's climate model.

One programmer highlighted the error of relying on computer code that, if it generates an error message, continues as if nothing untoward ever occurred. Another debugged the code by pointing out why the output of a calculation that should always generate a positive number was incorrectly generating a negative one. A third concluded: "I feel for this guy. He's obviously spent years trying to get data from undocumented and completely messy sources."

Programmer-written comments inserted into CRU's Fortran code have drawn fire as well. The file briffa_sep98_d.pro says: "Apply a VERY ARTIFICAL correction for decline!!" and "APPLY ARTIFICIAL CORRECTION." Another, quantify_tsdcal.pro, says: "Low pass filtering at century and longer time scales never gets rid of the trend - so eventually I start to scale down the 120-yr low pass time series to mimic the effect of removing/adding longer time scales!"

It's not clear how the files were leaked. One theory says that a malicious hacker slipped into East Anglia's network and snatched thousands of documents. Another says that the files had already been assembled in response to a Freedom of Information request and, immediately after it was denied, a whistleblower decided to disclose them. (Lending credence to that theory is the fact that no personal e-mail messages unrelated to climate change appear to have been leaked.)

For its part, the University of East Anglia has posted a statement calling the disclosure "mischievous" and saying it is aiding the police in an investigation.

The statement also quotes Jones, CRU's director, explaining his November 1999 e-mail, which said: "I've just completed Mike's Nature trick of adding in the real temps to each series for the last 20 years (ie from 1981 onwards) amd from 1961 for Keith's to hide the decline." Jones said that the word trick was used "colloquially as in a clever thing to do" and that it "is ludicrous to suggest that it refers to anything untoward."

Also unclear is the ultimate impact of the leak, which came before next month's Copenhagen summit and Democratic plans for cap and trade legislation.

On one hand, over at RealClimate.org, Gavin Schmidt, a modeler for the NASA Goddard Institute for Space Studies, has been downplaying the leak. Schmidt wrote: "There is no evidence of any worldwide conspiracy, no mention of George Soros nefariously funding climate research ... no admission that global warming is a hoax, no evidence of the falsifying of data, and no 'marching orders' from our socialist/communist/vegetarian overlords."

On the other, groups like the free-market Competitive Enterprise Institute, the target of repeated derision in the leaked e-mails, have said: "We have argued for many years that much of the scientific case for global warming alarmism was weak and some of it was phony. It now looks like a lot of it may be phony."

ScienceMag.org published an article noting that deleting e-mail messages to hide them from a FOI request is a crime in the United Kingdom. George Monbiot, a U.K. activist and journalist who previously called for dramatic action to deal with global warming, wrote: "It's no use pretending that this isn't a major blow. The emails extracted by a hacker from the climatic research unit at the University of East Anglia could scarcely be more damaging."

Complicating matters for congressional Republicans who'd like to hold hearings is that East Anglia, of course, is a U.K. university. The GOP may intend to press the Obama administration for details on how the EPA came to rely on the CRU's predictions, and whether the recent disclosure will change the agency's position. Another approach lies in e-mail messages discussing grants from the U.S. Commerce Department's National Oceanic and Atmospheric Administration to East Anglia; one says: "We need to show some left to cover the costs of the trip Roger didn't make and also the fees/equipment/computer money we haven't spent otherwise NOAA will be suspicious."

The irony of this situation is that most of us expect science to be conducted in the open, without unpublished secret data, hidden agendas, and computer programs of dubious reliability. East Anglia's Climatic Research Unit might have avoided this snafu by publicly disclosing as much as possible at every step of the way.

-------------------------

I've read the HARRY_READ_ME.txt file and all I have to say is...that poor poor bastard.