
Doing science in the open - robg
http://physicsworld.com/cws/article/print/38904
======
lutorm
There are many things that make the analogies made in the article not
applicable.

If I buy a Pokemon, I can crank out a review in probably 5 min after playing
with it for a while. Contrast that to peer review: to write a careful review
of an article takes me at least a full work day to make sure that I understand
what's done, think about possible pitfalls and weaknesses, and then write a
comprehensive and helpful review. No one will put in that amount of time to
write a comment on a web page, so the only people that can offer substantial
comments would be those that have taken the time to understand and think about
the paper for other reasons (usually because it directly concerns their own
work). The problem is then that even if these few people actually make
comments, they are likely to be totally drowned out by people who make useless
comments (either because they don't understand the paper or because they have
no incentive to make an effort).

Regarding the shoe store, that's also a false analogy. A shoe store selling
shoes is akin to a scientist publicizing their finished papers, they have
nothing to lose by revealing their design because they already have them in
the store. A better analogy would be to ask: if Nike's having some problem
figuring out how to manufacture a shoe, do they ask Reebok for advice? Since
Reebok works in the same field, they likely would have useful advice to give,
but they don't because the danger of Reebok taking the design and running with
it is too large. That's how it would be with scientists openly sharing their
works in progress.

I'm not debating that the progress of science would be helped with more open
sharing, but offering questionable analogies does not help in understanding
the problem.

However, the problem of conflicting incentives is a very real one. As long as
scientists are judged based on the numbers of published papers and numbers of
citations, there's an incentive to hoarding data to squeeze every last
possible paper out of them. Also, there really is little incentive to share
tools with the community apart from gaining recognition. This is compounded by
people being bad at citing papers describing tools, too, so you don't even get
that reward.

