
Too Many Medicines Simply Don’t Work - pseudolus
https://www.bloomberg.com/news/articles/2019-05-30/too-many-medicines-simply-don-t-work
======
ncmncm
Even to know (in the double-blind trials "gold standard" sense) whether a
medicine works, you need to diagnose reliably what it is supposed to treat.

But the DSM is full of illnesses that are really a half dozen completely
different conditions with the same (or just similar) apparent symptoms, each
with its own cause, and no reliable way to distinguish it from the others
except that maybe drug X helps one, and drugs Y, Z, W, and V are useless for
it. Likewise, the one that Y helps, and the one that responds to Z, and the
rest where nothing seems to work. In double-blind trials, V, W, X, Y, and Z
are all "shown" to be equally useless.

That does NOT mean V, W, X, Y, and Z don't work. The "gold standard" is as
subject to GIGO as everything else. You simply cannot tell anything about what
it means without looking at underlying facts.

~~~
LorenPechtel
Yeah. When evaluating a drug we should not only be looking overall, but if
there is a subset of the test population for which it behaves differently.
This is especially true for any diagnosis of exclusion.

Such a drug should be approved but with a big warning that it only works in
some cases.

~~~
WhiteSage
You have to fix the hypotheses to test before you begin the experiment, as
otherwise you will stumble upon a lot of spurious correlations. That your
birthday is the same day than someone's in your class/work group is a rare
event, that two people have the same birthday in your class/work group, not at
all uncommon. This is the birthday paradox. In the same spirit, with a drug
you'll always find groups where it works because of spurious correlations,
specially if you don't fix beforehand the group where you want to test it.

~~~
derefr
Run one trial to find correlations with specific groups, then run another
trial under the hypothesis that it works for that group.

~~~
nradov
Well sure but that costs a fortune. Drug development is already extremely
expensive.

------
WhompingWindows
1\. Drug candidates seeking approval must show non-inferiority to gold
standard, meaning they are not worse than the top choice for the same
treatment. This non-inferiority is quite easy to game, as we all know
statistics are fungible and may be manipulated. If all I have to show is no
significant difference, and I can use one of 10 different outcomes or
surrogates, well my job got a LOT easier than showing superiority on the
primary outcome of interest.

2\. Before approval, drugs are tested in non-representative samples, either
because they are too small in sample size, the disease is too rare, or because
there needs to be a VERY large (post-market) population to tease out rare side
effects.

3\. Doctors and geneticists should come together and provide genetic testing
to ensure that patients and their drugs are appropriately matched. This could
be done in cancer, for instance, as you can genotype the cancer and provide a
drug cocktail specifically for it. Another instance would be depression, as
some drugs do not work for people with certain neurotransmitter genotypes.
Physically, this means that each of us may have slightly differently shaped
receptors that different drugs bind to differently. This explains a lot of the
difference in drug responses, whether it be SSRIs or alcohol or cannabinoids,
people simply have different receptors/biochem.

4\. Medicine has always been a battle between doing nothing, providing the
placebo effect, providing beneficial treatment, and avoiding/minimizing harm.
In medieval times, exposure to medicine was tantamount towards increased risk
of death and complication (see leeching, blood letting, terrible hygiene in
surgery/childbirth, etc). Even now in modern times, going into a hospital is
risky: hospital-associated infection is a massive killer, never mind acute
psychological stress leading to delirium. We demand improvements in
healthcare, new drugs, innovations, and extending life, and yet that is a very
complex and challenging process that we will work on for centuries to come.

Simply put, doing medicine well has always been very hard. We have many legacy
treatments and practices which probably should be put on the back shelf, but
as with coding practices and legacy code, it is very hard to stop practices
with momentum and inertia, whether the field is energy (fossils dominate),
medicine, coding, or anything.

~~~
JamesBarney
> 1\. Drug candidates seeking approval must show non-inferiority to gold
> standard, meaning they are not worse than the top choice for the same
> treatment. This non-inferiority is quite easy to game, as we all know
> statistics are fungible and may be manipulated. If all I have to show is no
> significant difference, and I can use one of 10 different outcomes or
> surrogates, well my job got a LOT easier than showing superiority on the
> primary outcome of interest.

Before they to the test they have to declare a primary outcome to avoid this
issue. And most diseases just don't have that many outcomes relevant outcomes
you can test for.

> 3\. Doctors and geneticists should come together and provide genetic testing
> to ensure that patients and their drugs are appropriately matched. This
> could be done in cancer, for instance, as you can genotype the cancer and
> provide a drug cocktail specifically for it. Another instance would be
> depression, as some drugs do not work for people with certain
> neurotransmitter genotypes. Physically, this means that each of us may have
> slightly differently shaped receptors that different drugs bind to
> differently. This explains a lot of the difference in drug responses,
> whether it be SSRIs or alcohol or cannabinoids, people simply have different
> receptors/biochem.

They're trying, and they're failing hard. Precision medicine has been pretty
useless outside of cancer.

~~~
astazangasta
Even in cancer precision medicine is pretty crap. It works, but only
sometimes. Most of the time even a correctly targeted drug results in a
resistant tumor.

------
rjkennedy98
I think a big part of this is that statistical significance is taken as a
surrogate for clinical significance. They are not the same. Antidepressants
for instance may be statistically significant in meta-analyses, but the
average gain is less than 3 points on the Hamilton scale which is not even
detectable by most people or doctors. The same goes with these other measures
such as slight tumor shrinkage, ect. Its a difference that doesn't make a
difference clinically.

~~~
JamesBarney
The effect size of xanax is .3, antidepressants .3, and morphine .4. So if the
effect size of antidepressants aren't clinically significant than neither are
xanax and morphine.

~~~
all_blue_chucks
For anxiety at least, benzos like xanax are extremely effective. If they
weren't addictive, chronic anxiety would be pretty much cured.

~~~
JamesBarney
Yeah same with morphine

------
BadassFractal
I recently ran into this issue with statins. Doctors are eager to prescribe
them, but there seems to be conflicting evidence of them ultimately changing
outcomes:
[https://www.sciencedaily.com/releases/2015/02/150220110850.h...](https://www.sciencedaily.com/releases/2015/02/150220110850.htm)

~~~
ncmncm
Statins have only ever been shown to help people who have had a heart attack,
or were in imminent danger of one -- despite decades of effort by Big Pharma
to find other benefits. (Big Pharma wants every drug they sell to be needed
continuously from today until you drop dead. Statins would perfect because
they are also extremely cheap to make.)

What we do know about statins is that they have terrible long-term side
effects that might not put off somebody worried about dropping dead at any
moment, but should give everyone else the heebie-jeebies.

They interfere with the liver's production of cholesterol, but we (like every
other animal) need cholesterol to live.

~~~
hammock
Please share a link to this stuff that I can share with my parents

------
DanBC
This study doesn't mention "number needed to treat" \- if we give X people
this medication we'd expect to prevent Y occurances of this condition.
Sometimes X can be surprisingly high and Y worryingly low.

It also doesn't mention the drug company funding provided to some patient
campaign groups. It's pretty cynical: you know your medication is not
effective enough for the cost, so you give money to dying people and their
families to campign to get your med provided by single payer systems and
insurance companies.

~~~
amelius
Can you give this "number needed to treat" for some common medications?

~~~
DanBC
Here's an article that mentions a few of them:
[http://www.pulsetoday.co.uk/clinical/clinical-
specialties/pr...](http://www.pulsetoday.co.uk/clinical/clinical-
specialties/prescribing/nnt-how-many-is-too-many/20033549.article)

> For example, for every thousand low-risk patients prescribed statins for
> primary prevention, only a single stroke is prevented per year. Meanwhile,
> you would need to treat more than a thousand people with antihypertensive
> therapy per year to prevent one death. This is a far higher NNT than that of
> many interventions GPs are advised against prescribing due to lack of
> efficacy.

[https://www.bmj.com/content/348/bmj.g3458/rr/761453](https://www.bmj.com/content/348/bmj.g3458/rr/761453)

> Enas bases his numbers on a 2013 Cochrane review. Enas reports that NNT
> equals 167 for low risk (less than 1%) and 67 for intermediate risk (1-2%)
> individuals. Enas reinterprets this as meaning 1000 low-risk people would
> need to be treated for five years to prevent six major adverse
> cardiovascular events. Enas notes the increased risk of type II diabetes
> from statin use and also estimates that 100 of the thousand people treated
> would have myopathy (that the same might occur with placebo is irrelevant as
> I will not be taking a placebo). Enas judged that the benefits of treatment
> in low risk subjects far outweigh any possible hazards and concluded that
> early and aggressive statin therapy offers the greatest potential for
> reducing the continuing epidemic of coronary artery disease among Indians.

~~~
khuey
The accepted NNT for antihypertensives is around 1 in 100 depending on the
exact metric you use ([http://www.thennt.com/nnt/anti-hypertensives-to-
prevent-deat...](http://www.thennt.com/nnt/anti-hypertensives-to-prevent-
death-heart-attacks-and-strokes/)).

------
avsteele
Related, Recommended:

[https://www.amazon.com/Medical-Nihilism-Jacob-
Stegenga/dp/01...](https://www.amazon.com/Medical-Nihilism-Jacob-
Stegenga/dp/0198747047)

You can listen to a recent EconTalk podcast for a one hour summary of the
book.

------
icegreentea2
This is obviously a hard and serious problem to address. That said, I little
part of me smiled a bit at 5 withdrawals out of 93 drugs. It's bizarrely close
to what blindly applying 95% confidence intervals get you. I'm both terrified
and impressed that it's that close enough.

~~~
SamReidHughes
Passing a 95% significance test doesn't have any connection to the notion that
there is a 5% chance the drug is no good.

This is what you would expect from a blind application of a 95% confidence
interval only if 50% of tested drugs were good.

~~~
rifung
Care to clarify?

My understanding was that a 95% confidence interval implies the chance of a
false positive of 5%.

From
[http://www.mit.edu/~6.s085/notes/lecture2.pdf](http://www.mit.edu/~6.s085/notes/lecture2.pdf)

"In particular, we can ask for an interval around ˆp for any sample so that in
95% of samples, the true mean p will lie inside this interval. Such an
interval is called a confidence interval."

~~~
6gvONxR4sf7o
The false positive rate has nothing to do with positive examples (hugely
counterintuitively). A false positive happens when there's nothing there but
you think there is. The rate of these is compared to all incidents where
there's nothing there. So it's the fraction of _negative instances_ where you
think there's a positive. No actual positives have to come into the picture.

~~~
rifung
Ah you're right that makes a lot of sense thanks for the explanation

------
awalton
My favorite example of this was when the feds essentially replaced
pseudoephedrine (which works brilliantly, but now you're treated like a
criminal if you need to buy it) with phenylephrine (which, quite frankly,
doesn't do a damned thing:
[https://www.jwatch.org/na39054/2015/09/17/phenylephrine-
no-m...](https://www.jwatch.org/na39054/2015/09/17/phenylephrine-no-more-
effective-placebo-nasal-congestion) \- it's worse than a placebo in that it
has side-effects AND doesn't do anything for what it's supposed to treat!)

~~~
ddenisen
Anecdotally, I find phenylephrine to be definitely effective as a topical
decongestant (nasal spray); especially more so than its alternative
(oxymetazoline), so it's not a placebo effect. But I agree that it doesn't do
a thing as an oral drug.

------
village-idiot
Statins are very notorious in this regard. While they do indeed affect
cholesterol, for most members of the population it does nothing to change all
cause mortality.

------
sharadov
What about the placebo effect? They've done studies where the patients knew
that they were taking placebos, but still saw improvements...

~~~
danieltillett
The placebo effect causes all sorts of problems for drug trials in certain
areas where the control arm does much better than expected swamping the
statistics of the treatment arm (trials are powered with an assumption of how
well the placebo will work).

Interestingly, the unexpectly effective placebo effect seems to be limited to
the USA. Why nobody is sure.

------
vkhn
ctrl+f for phenylephrine.

That was my first thought.

~~~
frenchy
I ended up getting phenylephrine once because I was vacationing in the US.
Needless to say, I was not impressed. How do you guys treat colds?

~~~
mrosett
Get real Sudafed from behind the counter.

------
vecplane
Slightly off topic, but one medicine that really does work is Accutane
(Isotretinoin) for cystic acne, and I'm super grateful for it. If you still
have acne, I highly recommend trying it.

I came across it on reddit a while back and it's basically solved my acne
problem. You get a super dry face and chapped lips, but there are lots of
products to help mitigate that.

The Accutane subreddit is full of tips and before/after photos -
[https://www.reddit.com/r/Accutane/](https://www.reddit.com/r/Accutane/)

~~~
digianarchist
There was a story on the BBC site about accuntane recently.

[https://www.bbc.com/news/health-47952076?intlink_from_url=ht...](https://www.bbc.com/news/health-47952076?intlink_from_url=https://www.bbc.com/news/topics/c34vnl2xl5pt/medicine&#38;link_location=live-
reporting-story)

I approached my GP after suffering with acne who put me on tetracycline which
has some horrible side-effects such as staining your teeth and it didn't work
for me at all.

I bypassed my GP and bought accutane online. The side-effects are some of the
worst I've had from a medicine, dry skin and eyes, regular nose bleeds,
sensitivity to the sun etc. Nothing mental though. That said. It works.

Before I used accutane I had several acne spots turn into keloid scars which I
have been treating over the last decade with corticosteroid injections.

~~~
irq11
PSA to anyone reading this thread: Do not take Accutane without a
prescription. There are _very good_ reasons that doctors don’t want to
prescribe it, and if you can’t find someone willing to do it, it means that
the risks very likely don’t outweigh the benefits in your case. Doctors are
going to want you to try more conservative treatments _first_ , and you
shouldn’t second-guess that.

Accutane makes the “horrible side-effects” of Tetracycline look like a mild
headache. Aside from being a powerful teratogen (i.e. it causes nasty birth
defects), Accutane is linked to suicidal tendencies, liver damage, heart
damage, and more. It’s not a drug anyone should be taking without supervision.

~~~
anonthrowaway2
OTOH, it's often that the risks _to the doctor_ outweigh the benefits _to the
doctor_ of recommending you use it. Correlated with but far from the same as
the risks/benefits to you.

------
bbulkow
Other points not covered.

We need to end off label use. Saying a drug is approved for one diagnosis and
then allowing it for any other diagnosis is, in the modern age, simply crazy.
One might as well not have fda approval.

The problem of tainted and ineffective generics confounds the analysis. A
recent investigation shows the fda's proceedure of unannounced inspections
only in the us means that 75 percent of indian and Chinese generics makers
failed inspections during the few years when unannounced inspections were
done. Now they have moved back to announcing. It is not sure that each failure
meant there are bad drugs out there, but a majority of failures were 100
percent faked test data regarding the chemicals inside the drugs.

~~~
klipklop
As somebody that benefits from using a drug 'off-label' I strongly disagree.

There are many cases where off-label drug use gives a minority of people
tremendous benefit. Take a look at the people with various brain disorders
using Ambien off-label. It can literally bring people out of coma-like states
temporarily.

Other examples off-label use: * Propranolol for anxiety. * Guanfacine for
blood pressure issues, now used on label for ADHD.

~~~
DanBC
> Propranolol for anxiety.

Is that off-label?

[https://bnf.nice.org.uk/drug/propranolol-
hydrochloride.html](https://bnf.nice.org.uk/drug/propranolol-
hydrochloride.html)

~~~
awalton
As far as I'm aware there's no beta blocker that's approved for performance
anxiety in the US.

Paperwork for Propranolol in the US has on-label usages listed as:
Hypertension, Angina Pectoris, Atrial Fibrillation, Myocardial Infarction,
Migraine, Essential Tremor, Hypertrophic Subaortic Stenosis or
Pheochromocytosis.

------
fabiofzero
This is the sort of headline that fuels the likes of anti-vaxxers.

------
JoeAltmaier
tl;dr: sometimes medicines don't achieve their early promise. But what are you
gonna do about it?

~~~
jfengel
Especially in the studies they're talking about here, which are about cancer
drugs. It's life-and-death stuff, and it's not so simple as "Well, we haven't
proven this yet, so you just run off and die while we run a few more studies."

We have a hard enough time keeping desperate people from trying things that we
know for a fact are scams. Perhaps doctors could put the studies in clearer
context when prescribing, but they are always clear that these drugs aren't
perfectly effective and that it's kind of a crapshoot.

When you've got patients who will die anyway, it's no surprise that we end up
treating them as a final stage of clinical trials. You need to be honest about
that and get consent, but many will give it willingly.

~~~
JoeAltmaier
Actually, my sister's full-time job is explaining to folks at the end of their
cancer journey that there's actually one more thing they can try. It very
likely will not help them survive. Not for themselves, but for people that
come after. A drug that might not do anything, or it might do a little, or it
might be what we've been praying for.

And those Iowa farmers and shopkeepers and schoolteachers at the end of their
lives sign the form. An unreasonable population of them. For no reason but to
help.

------
naringas
I wonder how many of these are pushed as mandatory vaccines

~~~
dang
"Eschew flamebait. Don't introduce flamewar topics unless you have something
genuinely new to say. Avoid unrelated controversies and generic tangents."

[https://news.ycombinator.com/newsguidelines.html](https://news.ycombinator.com/newsguidelines.html)

