
Open Science and climategate: follow CERN's lead - nice1
http://www.freesoftwaremagazine.com/columns/open_science_climategate_ipcc_cru_needs_take_leaf_out_cerns_book
======
yungchin
As for the "climategate" part of that uhm, rant, I'd say Ars' commentary
provided a more balanced, real-world view on matters:
[http://arstechnica.com/science/news/2009/11/uk-hack-puts-
cli...](http://arstechnica.com/science/news/2009/11/uk-hack-puts-climate-
scientists-personal-e-mails-on-display.ars) \- to me, like to the Ars editor,
the CRU emails (sadly) look like your average internal communication.

Of course, the guy is otherwise right: we'd all be better off if research was
done completely in the open. The problem that's holding that back is that the
research-world still only rewards authorship; there's no mechanism in place to
evaluate the "impact" of the linked data someone puts online.

CERN are in a special position: they have little problems protecting their
authorship, because few peers would be able to claim that they too have a LHC
in their backyards...

~~~
vixen99
Rewards for decent behaviour? What's holding 'that back' is an evident paucity
of scientists (in this small area anyway) with integrity and a desire to
discover the truth, come what may and irrespective of the views of their
paymasters, relating to their speciality.

You ask for a mechanism to 'evaluate "impact" of linked data someone puts
online'. I'm intrigued. What on earth do you have in mind? How would it work?
On the face of it (I trust I've got this about face) it sounds an appalling
idea. A load of value judgements delivered by whom?

~~~
yungchin
Just to be sure we're talking about the same thing: the article above is
advocating that we all publish every detail of our research proceedings
online. What I'm saying is, that doing that would jeopardize your position as
a researcher: the average labjournal is full of thoughts and ideas for further
work. Researchers however don't get rewarded for inspiring peers, they only
get rewarded for doing the work.

If we want to encourage people to disclose all their data, we need to get rid
of the system where grants, tenured posts, and awards are all based on
authorships only.

I'm not sure which idea you think is appalling, as I don't have any idea as to
how to do it! One way would be to establish something similar to PageRank: if
you publish a database of, say, world-temperature readings, an indicator of
its value might be how many online labjournals in the world draw data from it.
That's a very raw idea, and obviously not good enough, but calling it
appalling before I even came up with it?

------
cswetenham
"What is required is the wholesale adoption of the standards applied in the
pharmaceutical industry drug trials: double blind trials (aka peer reviewing)"

I think the gentleman who authored this article neglected to look up the terms
in question.

[http://en.wikipedia.org/wiki/Blind_experiment#Double-
blind_t...](http://en.wikipedia.org/wiki/Blind_experiment#Double-blind_trials)
<http://en.wikipedia.org/wiki/Peer_review>

~~~
lutorm
Rofl. I gave up on the article after the first paragraph, so I missed that
nugget.

Did he explain how you conduct a double blind trial on climate change? That's
like asking me (an astrophysicist) to conduct a double blind trial on the Big
Bang.

It's not possible to do trials on some systems, either because there is only
one or because it's impossible to manipulate it. In those cases, you are
unfortunately stuck with collecting data and building (physically reasonable)
models where you try to match those data and suggest new data which are
predicted by the theory that can be used for testing it.

It _is_ important to do the latter. Just because you can adjust a bunch of
parameters in the model to match existing data doesn't give a lot of
confidence in your model, because of the feedback loop that causes you to work
on the model until it fits the data. What you need to do is then say "given
this model, we predict x" and then go and look for x.

From what I know about the climate modeling, it would be nice to see more of
the latter. I know there are some things (development of regional patterns of
changes) that match the models, but I don't know whether those were pre- or
post-dictions.

