

Fixing Science - Systems and Politics - DiabloD3
http://neuroskeptic.blogspot.com/2012/04/fixing-science-systems-and-politics.html

======
diminish
The notion of a central registry and government regulation for science is
horrific and backwards no matter what the initial intention is. Why not a
distributed, peer-to-peer approach for scientific publishing where all
articles are anonymously accessed and linked to each other?

~~~
pessimizer
Because that, in no way, will prevent researchers from trashing studies until
they get the results they want, or fishing for any significant correlation
after the fact as if that's what they were looking for all along.

Some sort of validation mark that a study announced itself before being
started would at least show me which ones I should ignore - because IMO this
type of thing is ruining medicine, esp. pharma.

At this point, there's very little evidence that some major drugs, classes of
drugs, and treatments _work at all_ , and that's both scary and a systemic
financial drain.

------
hobin
I think it's a rather misleading post. He cites a paper from PLoS Medicine,
and I suspect that those without a good background in research would rather
quickly jump to the conclusion that "it's in a science journal, so it must be
(very likely to be) true." However, it should be noted that this is an _essay_
, which PLoS Medicine defines as _opinion pieces on a topic of broad interest
to a general medical audience._

More importantly, lets look at this from a pragmatic point of view. Are there
things that can be done better in science? Absolutely. Does science somehow
'not work'? Not at all. Does it need fixing? No, why should it? Anyone who
suggests that science is fundamentally broken must've lived under a rock for
the past decades, or at least hasn't been keeping an eye on what science has
achieved.

I also disagree with the blog author's 'solution' to the problem. He (or is it
a she? I don't know) argues in favor of a centralized approach to science,
whereas one of the reasons science is so successful is because it's so
decentralized!

~~~
disgruntledphd2
Um, on your point about the paper being an opinion piece, and thus not
important, I would respectfully suggest that you are wrong. The author is John
Ionnadis and he has published a number of meta-analyses demonstrating this
problem.
[http://scholar.google.co.uk/scholar?hl=en&q=JPA+Ioannidi...](http://scholar.google.co.uk/scholar?hl=en&q=JPA+Ioannidis&btnG=Search&as_sdt=0%2C5&as_ylo=&as_vis=0)
Check out all the articles on the first page, he's been working in this area
for a long time.

The author of the piece also identifies a massive, massive problem with
science. The problem is essentially, tenure is hard to get, and many people
want it. Tenure is dependent on highly cited papers in good journals. These
journals have a bias toward significant results which pushes scientists to
data-dredge looking for any significant result.

His system of pre-registration (which could be done on the ArXiV model, no
government required) would allow us to have a much better idea of the things
we're trying to study.

My field (psychology, though I suspect the problem is just as bad in other
areas) has a massive problem in this area, as a counter-intuitive result gets
published in a good journal, while high quality replications which show no
significant effects either don't get published or get published in a much
lower ranking journal.

This kind of registration becomes even more important when there are
commercial interests riding on the outcome of a study, as in clinical trials.
This (and the scandals) is presumably why such a system exists for clinical
trials.

Finally, I don't see how registering studies and their designs before running
the tests creates a centralised approach to science. The research is all
planned beforehand (for ethics committees at the very least), so there's no
extra work involved. It also increases our trust in results, as the ones which
make it through were predicted in advance, while the weird findings can be
replicated.

On a personal level, I can see the pressures to hypothesise after the results
are known (HARK) and with tenure decisions looming, I can understand why
people do it. Its horribly wrong, so this system would at least cut down on
that behaviour.

Full disclosure: almost every time I carry out research, the results are
opposite to what I expect, so I am a somewhat biased participant in this
debate.

