

Why epidemiology refuses to reduce a ~80% false positive rate - gwern
http://lesswrong.com/r/discussion/lw/72f/why_epidemiology_will_not_correct_itself/

======
rlvesco7
While randomized trials are a good thing (they help separate causation from
correlation), correlations can be valuable too. Especially since we cannot
always run randomized trials.

For instance, the first studies on cigarette smoking and cancer were done in
1930's and they showed a relationship between the two. They were purely
correlational. They could of been spurious. Even if we could do a randomized
trial, it probably wouldn't be ethical. However, these initial studies paved
the way for more research. Now we know with a high probability that the two
are related. If we followed the author's advice where would we be?

There are also parallels to start ups. If we always followed the "hard
evidence" how many new innovations and insights might we have missed?

~~~
gwern
The smoking correlations were noticed because they were so high that the
actual studies were kind of merely confirmation of what some people were
suspecting. A strong correlation like that ought to survive Bonferroni
correction.

> Even if we could do a randomized trial, it probably wouldn't be ethical.

What is unethical is _not_ running a trial. We don't know anything and what we
do know is frequently falsified, no matter how confident we think we are. This
is the whole point of 'evidence-based medicine' - there are countless
treatments where people went 'oh it'd be unethical not to use this awesome
treatment' and made snide jokes about randomized testing of parachutes. But
nicotine has many benefits (<http://www.gwern.net/Nicotine#performance>) and
it's not clear to anyone who was not raised in the 20th or 21st century
without the benefits of indoctrination that tobacco is bad _on net_. This is
why we do trials. I am reminded of one quote on the topic:

> 'One day when I was a junior medical student, a very important Boston
> surgeon visited the school and delivered a great treatise on a large number
> of patients who had undergone successful operations for vascular
> reconstruction. At the end of the lecture, a young student at the back of
> the room timidly asked, “Do you have any controls?” Well, the great surgeon
> drew himself up to his full height, hit the desk, and said, “Do you mean did
> I not operate on half the patients?” The hall grew very quiet then. The
> voice at the back of the room very hesitantly replied, “Yes, that's what I
> had in mind.” Then the visitor's fist really came down as he thundered, “Of
> course not. That would have doomed half of them to their death.” God, it was
> quiet then, and one could scarcely hear the small voice ask, “Which half?”'

And besides, are you seriously trying to defend the last 70 years of bad
epidemiology with _one_ example which it is not clear even matters to the
statistical question?

> If we followed the author's advice where would we be?

We wouldn't have pro-smoking people pointing out the absence of randomized
trials and ambiguous animal results, a respected statistician like Fisher
pointing out the curious correlations undermining the case for causation, or
the smoking industry able to cast doubt and delay regulation for decades.
(Ethical my ass!)

