
Is the FDA Too Conservative or Too Aggressive? Analysis of Clinical Trial Design - nkurz
http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2641547
======
mjfl
A case study on FDA from my father, a cardiologist: A recent advancement in
replacement heart valves reduces mortality rates from 10% in high risk
patients (old and very unhealthy) to 0.9%, from 5% in medium risk patients to
0.4%. This studies were confirmed through clinical trials run by very well
respected doctors. Nevertheless, the FDA still required their own set of
trials to confirm the safety of the procedure. This was 5 years ago. The FDA
first ran a set of trials, not on standard patients but only those that were
deemed "inoperable". They spent a few years working that out and found that
the device succeeded, but the approval was only given for patients that were
inoperable. Then the FDA covered patients that were high risk, spent a few
years and found again that the device was hugely beneficial in mortality rates
and the device was approved for high risk patients. We are currently in the
waiting for the FDA to approve the device for medium risk patients. In the
mean time, my father had a medium risk patient who needed a valve replacement.
He could not use the new device, so he sent the man to surgery for a standard
valve replacement with 5% mortality risk. The man died on the table, becoming
part of the 5%, and my father insists that he would have survived if they had
been able to use the new procedure. In Germany there are less regulations, and
so they have been using the new technique for 5 years with great success.
Because of things like this, you'll commonly see medical procedures in Europe
that are 5 years ahead of their counterparts in America, all because the FDA
insists on using such plodding methods to verify safety.

~~~
DennisP
I know an M.D. who travelled to Germany to get a medical implant (for himself)
that's not yet approved in the U.S., even though it meant paying $90K out of
pocket.

------
regressiveparty
My experience with the FDA:

I'm "congenitally limb deficient" below the elbow. Grew up in the 90s using
horrible myo electric prosthetic limbs that had 2 hour battery life and were
in all practicality a glorified claw machine. I ditched that as soon as I
realized they were actually a hindrance.

In 2006, there was a TED talk with Dean Kamen about a prosthetic arm developed
by DARPA. It blew my mind and got me finally interested in wearing a
prosthetic again. So much promise in a field that had been in a coma for the
last 20 years.

That arm never happened. Why? Because it took the FDA 7 years to approve it.
The project now is dead in water and the group responsible for building it is
trying to literally give away their designs. Why did the FDA have to approve a
prosthetic limb? Well, because it touches your skin. Yes, the silicon on the
back had to be tested because of skin contact.

I'm now the lucky recipient of BeBionic 3 arm, which is great. Worst part
about it? It uses radio frequency to control the hand and has a tendency to go
haywire. I asked why they didn't just use Bluetooth and he said that would
require another 2 years to get FAA approval. Ugh.

Anecdotal, I know.

~~~
Vexs
See, this just seems overly investigative. If they were physically fusing the
arm to your body, sure, that's a reasonable time for the FDA to investigate.
But just because it is related to your body in any way doesn't warrant that
sort of careful consideration. Similarily, if the radio was controlling a
pacemaker, I wouldn't want it using bluetooth, I'd want something FDA
approved. If it's an arm, I probably wouldn't want bluetooth (stupid pairing
issues....) but something like the xbee format would be great, not some
proprietary and less reliable RF protocol.

------
cryoshon
I've been on the science side of a number of FDA clinical trials for a few
years now.

The FDA is too conservative with many metrics, and too aggressive with others.
The clinical trial pipeline is tortuously slow for everyone involved-- the EU
equivalent agency has done a lot to accelerate the process fruitfully.
Eclipsing these FDA malregulations is the problem of transparency; if the FDA
forced clinical trials to publish 100% of their data rather than only the
positive results, we'd have a better medicine producing industry.

~~~
minthd
Is there any realistic possibility Americans could use drugs approved in
Europe ?

~~~
kirsebaer
Any American MD can ask the FDA for "expanded access" to use a medication not
approved by the FDA. 99% of the time the FDA grants expanded-access requests.

However, a recent review in NEJM explained how it is time consuming and
expensive to apply for expanded-access and often the pharmaceutical companies
refuse to distribute the drugs even after the FDA grants expanded-access.

[http://www.nejm.org/doi/full/10.1056/NEJMhle1409465](http://www.nejm.org/doi/full/10.1056/NEJMhle1409465)

"Compassionate use" is a separate, but related concept with different rules
than "expanded access". And in the EU expanded access is often called "named-
patient".

~~~
minthd
Thanks. You really know this stuff.

Why is there dependence on drug companies ? distributing drugs bought in
Europe or elsewhere is pretty easy.And as for drug companies being protected
from american litigation - if they don't sell it on u.s. soil, they are not
responsible in u.s. courts, right?

As for the process being complex and expensive, is there work being done to
solving this, with regards to EMA approved drugs ?

------
kesor
A family member of mine has tried to participate in a clinical trial. The
trial also forces him to forgo any existing medication, basically making his
illness intolerable. There is no indication in the trial if he is being served
with real medication or a placebo, but his life has been extremely affected
just by participation. Naturally he quit the trial. How would this statistical
analysis address these common behaviors where ill people would just not
participate in trials when certain illnesses are involved?

~~~
KingMob
There's a whole statistical literature on survival bias. The basic equations
all require "independent and identically distributed" data loss. As soon as
that's not the case (e.g., more placebo patients drop out than treatment
patients), you have to compensate for that.

That being said, I suspect this analysis is operating at a higher level where
survivor bias isn't as relevant, because it's looking across many studies at
once.

------
wcbeard10
A good summary and commentary here

[http://www.marginalrevolution.com/marginalrevolution/2015/08...](http://www.marginalrevolution.com/marginalrevolution/2015/08/is-
the-fda-too-conservative-or-too-aggressive.html)

------
transfire
TriCorder X-Prize? FDA has done nothing but hobbled it's development. DNA
screening for the masses? FDA shut down 23andMe. Terminally ill and willing to
try an experimental drug? FDA says, "too bad" (and the Supreme Court agrees).

~~~
rayiner
They didn't "shut down" 23andMe. They stopped them from marketing unproven
health diagnoses derived from the DNA tests.

~~~
fedups
I think it went beyond a marketing ban, unless 'marketing' extends to selling
and displaying health related information.

~~~
vitd
You can still use 23 and Me to get your results, but you have to go elsewhere
(like to your doctor) to interpret the results. They were basically giving
patients a diagnosis without going through the proper legal work to be able to
give diagnoses. It would be no different than some random person giving you a
medical diagnosis without being an actual licensed doctor.

This was not a case of over regulation. It was pretty simple, and the FDA
tried repeatedly to work with 23 and Me to bring them into compliance, but the
company basically ignored the FDA's requests for meetings and information.
They have nobody to blame but themselves, and what happened was good for
consumers.

~~~
fedups
This does highlight the effort to maintain the historic paradigm of health
professionals as gatekeepers in the face of wider and easier access to health
information.

Fortunately sources like snpedia.com still exist for those wanting to use
their 23andme data to do their own exploring without an intermediary, should
they choose.

~~~
rayiner
Nobody is gatekeeping access to health information. You can go online and try
and correlate your test results with the information that's out there.

What the FDA prevented was a company _profiting from selling a diagnostic
service_ that wasn't proven.

------
DennisP
So the FDA is too conservative for life-threatening conditions, and not
conservative enough for relatively mild conditions. That probably fits pretty
well with most people's intuitive sense of things; it's interesting to see the
math works out that way.

------
minthd
Here's another example:

There's oragenics, which developed a vaccine against dental carries - a
disease that affects 5 billion people. Something that i'm sure everybody would
really like to have.

And this is why we don't have it ? In their own words:

" Regulatory Status:

We initiated our first Phase 1 clinical trial in April 2005, but we found it
difficult to find subjects who fit the trial’s highly cautious inclusion and
exclusion criteria, particularly with respect to the subjects’ lack of
dentition. We concluded this trial early after enrolling only two of the 15
planned subjects. The FDA then recommended that we revise the protocol for the
evaluation of ten healthy male subjects, ranging from 18 to 30 years old and
with normal dentition, in an institutionalized setting. After we submitted
additional information, the FDA issued a clinical hold letter in June 2007 for
the proposed trial with the attenuated strain, citing the need for a plan with
respect to serious adverse effects; a plan for the eradication of the
attenuated strain in trial subjects’ offspring; and a required pregnancy test
for female partners of subjects. We submitted additional protocols in response
to the FDA’s concerns. In August 2007, the FDA issued a clinical hold letter
with required revisions to the protocol for offspring of subjects. We
submitted a response to the clinical hold letter in September 2007, and the
FDA removed the clinical hold for our Phase 1 trial in the attenuated strain
in October 2007.

While we commenced a Phase 1b clinical trial for SMaRT Replacement Therapy
during the first quarter of 2011, the very restrictive study enrollment
criteria required by the FDA made the enrollment of candidates meeting the
restrictive criteria difficult. Due to the enrollment difficulty we
encountered with our initial our Phase 1a clinical trial and now with our
phase 1b clinical trial, we determined to discontinue pursuit of our Phase 1b
clinical trial."

Such a shame.

[http://www.oragenics.com/?q=cavity-
prevention](http://www.oragenics.com/?q=cavity-prevention)

------
reasonattlm
Also: [http://fdareview.org/](http://fdareview.org/)

"Medical drugs and devices cannot be marketed in the United States unless the
U. S. Food and Drug Administration (FDA) grants specific approval. We argue
that FDA control over drugs and devices has large and often overlooked costs
that almost certainly exceed the benefits. We believe that FDA regulation of
the medical industry has suppressed and delayed new drugs and devices, and has
increased costs, with a net result of more morbidity and mortality. A large
body of academic research has investigated the FDA and with unusual consensus
has reached the same conclusion. Drawing on this body of research, we evaluate
the costs and benefits of FDA policy. We also present a detailed history of
the FDA, a review of the major plans for FDA reform, a glossary of terms, a
collection of quotes from economists who have studied the FDA, and a reference
section with many webbed links. A more detailed table of contents follows. We
are happy to receive comments and criticisms."

~~~
jcranmer
I'm wary of relying on economists to assess the utility of the FDA. The
alternative to the FDA they mention is basically self-policing. Except we
already have a drug-like industry that relies entirely on self-policing: the
"health supplements" industry, which the FDA is explicitly banned by
legislation from regulating. And the precedent of the health supplements
industry is extremely disconcerting--companies are basically legally lying
through their teeth (it's not illegal if someone else, e.g., Dr. Oz, says it,
cause _they_ aren't advertising, they're exercising free speech), and they
sometimes don't even bother to stuff their products with what they say the
active ingredient is.

The problem with drugs is the placebo effect. Evidencing efficacy beyond the
placebo effect requires large, controlled trials (the infamous Phase III
clinical trial) that end up being correspondingly expensive; nothing less is
sufficiently powerful. Because of placebos, you can sell expensive nothings,
and it will appear to work sufficiently well that reputation wouldn't suffer.

------
iskander
My small sample size of experience with the FDA: they seem to be very
receptive to novel therapies for terminal illness and it's surprisingly
straight-forward to get a Phase I trial approved.

There's no static rule uniformly enforcing the initiation of trials, rather
your IND (investigational new drug) application is reviewed by experts in the
relevant field to determine whether (1) is there a plausible reason to believe
your new therapy helps and (2) is it unlikely to kill or severely injure
patients.

I can't think of a better way for things to work for late-stage cancer or
other untreatable terminal illnesses.

------
AnimalMuppet
This is not an easy problem. If the FDA is too conservative, people will die
who could have lived. If the FDA is too aggressive, a different group of
people will die who could have lived. And no matter where the FDA is on the
spectrum (insane extremes excluded), there are people in both groups who die.
And there are people who cared for those in both groups, who scream about how
the FDA is doing it all wrong, and who have diametrically opposed views on how
the FDA should change.

Maybe the test should be whether the screaming is equally loud from both
sides...

~~~
crusso
Maybe if the FDA were only making recommendations then people could decide for
themselves whether to go with the doctor who advising an FDA-approved
treatment vs going with some other doctor who is advising something else?

------
frostirosti
There's so much conflict within the FDA that their opinion should carry no
real weight.

