
Paul Meier - Statistican who saved millions of lives - ColinWright
http://www.significancemagazine.org/details/webexclusive/1315465/Paul-Meier---Statistican-who-saved-millions-of-lives.html
======
eumendies
This is one of those moments for me where you kinda assumed a technique was
used in the past, but the truth is it had to be invented. Then when you look
at the past with this knowledge, you think, "My god, how barbaric!". Sort of
how McCoy (a la Star Trek) acts when he goes back in time.

~~~
iwwr
Not all problems with trials have been ironed out. Only as of 2004, have some
medical journals been requiring pre-registration of clinical trials.
Previously, drug manufacturers could pretend null results didn't exist.

<http://en.wikipedia.org/wiki/Publication_bias>

~~~
briggsbio
First, I think this is a closed view of the issue. Second, I think publication
bias has more effect on basic research and translational research (read: early
animal pharmacology studies) than drug development.

Conduct of a clinical trial for the purposes of establishing the safety and
efficacy of a drug product intended for a specific clinical indication has the
goal of registration (NDA, BLA, PMA, 510(k)) of the drug so that the drug can
be marketed and sold. Publications are great, and are a component of that, but
not the primary goal.

To conduct a clinical study, the company must present all preclinical (FDA
calls it nonclinical) pharmacology and toxicology on the drug product to
determine whether the product is safe for testing in human trials. This is
done via an investigational new drug application (IND, big damn documents that
take months to prepare). But I digress...

To conduct a clinical study, the protocol has to be submitted and FDA given
time to comment (and they will, often and without much room to argue). That
means the patient population, randomization scheme, endpoints, power analysis,
etc. are vetted by FDA pretty thoroughly. Next, the same process occurs for
the study at local or national (central) institutional review boards (IRBs).
They get really nitpicky (especially local university IRBs for numerous
reasons). Their primary goal is to ensure the safety of patients, but they
won't hesitate to comment on the design of the study in most cases. And (not
sure exactly when it became a requirement) sponsors have to register their
clinical trial with a national registry (which you can search on
clinicaltrials.gov). The results of those studies MUST be submitted to the FDA
under the IND, to the IRB, and updated in the registry, or development stops
and the IND cancelled.

So unless I misunderstand your comment, the publication practices of journals
affects the validity and thoroughness of clinical trial data. This is not the
case. Publication bias may alter the available literature, but drugs are NOT
registered on publications and effected by "pre-registration." Drugs are
developed and brought to market through a seriously controlled and monitored
process, not at the whim of medical journals. Now, preclinical data, basic
science, "academic studies" not moving a drug toward registration, etc. are
affected by publication bias. It is a problem, and pre-registration is a
complex issue but beneficial.

One final note: Writing this comment made me think that an alternative form of
pre-registration would be really interesting. But not to medical journals, but
an open registry for "coming soon research science." Think of a registry of
"here's what we want to do" for researchers in basic science, all putting up
ideas (ones they are willing to do so openly) for preclinical or clinical
studies on the site, indexed by keyword. This would allow researchers to look
online and not see what things HAVE been done, but see what things are GOING
to be done. This could help researchers identify colleagues (communicate via
social features), collaborate on the research, and pool their funding efforts
to get multi-institutional grants (usually higher dollar and currently a
little less competitive and thus easier to get). Put your research field and
interest, and studies you would do "if you had the money" and then the
registry would send you recommendations of people to connect with to try and
move those research areas forward. If this does not currently exist (which it
does not, to my knowledge), it would be amazing to have. It may also be an
easier task to implement as a new feature of an existing research social
network or community. "Enhancing serendipity in research..." Interesting.
Thoughts?

~~~
killerswan
There have been corporations who suppressed evidence that their drugs didn't
work right. I'd describe that as publication bias, too, if not simply fraud...

~~~
briggsbio
I agree that there are long-term safety and efficacy studies that are
fraudulently suppressed by some larger players in industry. It is an insidious
problem that has horrible effects on patients. However, it is separate from my
argument.

My point was that the registration of drugs is not effected by publication
bias. The "suppressed evidence" typically comes in the form of post-marketing
studies in significant patient populations, some of which aren't even
randomized or placebo controlled (some are). NDA-directed studies are tightly
controlled and reported. Then the drug is registered, and the FDA says, "we'd
still like to see long-term safety data on this to see what things look like
over time. We'll let you put this drug out there, but keep sending us more
data."

I think the problem therein is related to the regs on post-marketing clinical
studies. Companies have too much latitude to conduct post-marketing studies
and then decide what to send FDA (or to publish), thus introducing the
publication bias. FDA should have more control over Phase IV studies. The
Prescription Drug User Fee Act (PDUFA, or pud-oof-a) was established to charge
companies over $1million to register a drug, funneling money back into FDA so
they could acquire more resources to review and grand NDAs. So the review
times for NDAs went down, FDA got more efficient, and industry was happier.
However, the act did not increase Phase IV controls as stringently as they
should have, and companies have a lot of latitude on "additional clinical
data" from studies conducted after the drug hits the market. That's why you
see articles like the following:

GSK Suppresses Paxil Suicide Data: <http://goo.gl/LCM94>

AstraZeneca Suppresses Data on Seroquel: <http://goo.gl/408fJ>

Pfizer Suppresses Data on Edronax: <http://goo.gl/fxLbW>

So yes, there is publication bias and possibly fraud at work here, but it
lies, in my view, early in the process (preclinical, basic science) and very
late in the process (Phase IV studies). The registration process is pretty
well controlled.

Note: My expertise is in preclinical pharmacology, toxicology, and NDA-
directed studies (Phase I-III) not post-marketing studies (Phase IV), so if
anyone has more experience here please speak up.

------
sigil
Hopefully obvious, but some of this stuff also applies to A/B testing.

 _Yet in the 1950s the usual technique was to give a new treatment to the
patients whom it was thought would most benefit from it. It frequently
happened that those were the patients whose chances of recovery were the best
in any case, under the existing treatments as well as the new one. The result
was all too often that new treatments were thought to be better than old ones
but in fact were not._

------
gwern
> ‘When I said “randomize” in breast cancer trials I was looked at with
> amazement by my clinical colleagues’ said Meier in an interview in 2004. ‘
> “Randomize? We know this treatment is better than that one” they said. I
> said “Not really…” ’

