

What is medicine's 5 sigma? [pdf] - jeffreyrogers
http://www.thelancet.com/pdfs/journals/lancet/PIIS0140-6736%2815%2960696-1.pdf

======
Gatsky
What a lot of nonsense. I would have hoped for more from a journal that
facilitated one of the greatest and most harmful scientific frauds of all time
when they published the Andrew Wakefield paper about autism and vaccination
[1]. The Lancet also doesn't publish data except what they can stuff in a
single PDF and sell their journal through the glorified academic extortion
racket Elsevier. So yes, thanks for pointing out 'nobody is ready to take the
first step to clean up the system.' Completely empty rhetoric.

[1]
[http://en.wikipedia.org/wiki/Andrew_Wakefield](http://en.wikipedia.org/wiki/Andrew_Wakefield)

------
minthd
Can we determine ,by looking at papers if a research is replicated ?

Can Google determine that ?

If so , adding a Google Scholar search filter for replicated, proven research
could be useful, and might even make verified works more citable, hence
improving incentives somewhat.

~~~
austinjp
Cited does not equal verified.

Negative results often go unpublished, yet may be just as important to the
working researchers.

Until financial rewards are divorced from citation rates, this will remain a
problem.

~~~
minthd
Sure , cited doesn't mean verified, you need to read more deeply than that.
But if a paper from a reputable source says - "We replicated X", or "X was
replicated in Y" , those are useful data points with regard to an article
being verified.

And if Google understand that and help find that, it's far easier to find
verified research, so it makes it more likely that i'll base my research on
that , and cite them.

And if it's more likely to be cited, maybe more verification work will be
done.

------
return0
Wow. Disappointing. This is a like a tragedy of the commons , nobody is
willing to take the first step. I think severely hard remedies measures are
needed, "better peer review" is not going to cut it. For example, make it a
requirement that a finding is reproduced in an independent lab before
publishing.

~~~
dnautics
such a journal exists.

[https://en.wikipedia.org/wiki/Organic_Syntheses](https://en.wikipedia.org/wiki/Organic_Syntheses)

~~~
analog31
Interestingly, synthetic chemistry is a good example of a field that has
policed itself well. I don't know if it's an official rule anywhere, but
synthetic chemists follow an rule of performing a pretty good number of
independent tests on any new chemical, such as IR, NMR, mass spectrometry,
crystallography, and elemental analysis. Plus they need to have a plausible
explanation of how the molecule was formed.

~~~
dnautics
Fairly well. There's still Corey yield. The sezen/sames crisis happened too. I
had a grad school friend who sent a sample to the crystallography service and
got the structure he was looking for when he submitted salt.

------
jmmcd
One thing is for sure, changing alpha is not the answer!

I thought the article was going to be about six sigma and Japanese theory of
manufacturing. For a while the article was going there, with a mention of
changing incentives and rewarding honest criticism.

~~~
gwern
Yeah, if there's anywhere I _don 't_ want to see using six-sigma alpha, it's
medicine. That's a death sentence for hundreds of millions of people, at a
minimum.

(Alpha is only part of the tradeoff with power and sample size; since we're
not going to get sample sizes boosted into the millions for medical trials,
six-sigma could only be achieved by trashing power to the point where they
never reach any positive result at all.)

------
ska
This article has at least one thing right, the practical incentives in much of
science run counter to what we might reasonably agree are the right outcomes
to target.

