
Something Doesn't Add Up: John Ioannidis and medical statistics - jseliger
http://alumni.stanford.edu/get/page/magazine/article/?article_id=53345
======
ChuckMcM
This is a good read, it is hard to find things written about the issue of
medicine vs the business of medicine.

There is always the risk with science that you will "finish" which is to say
you'll figure something out and by and large be done with it. Nothing else to
see, etc. I read an article many years ago about the long lived efficacy of
Asprin, or acetylsalicylic acid. Here is a drug that has been around literally
since the stone age where folks chewed willow bark and it continues to be a
'wonder drug' whereas more modern drugs seem to arrive on the scene and fade
away. While their were obvious cases like antibiotics which became ineffective
against resistant bacteria, there was a number of drugs which seem to have
never been effective, rather they were marketed, pushed really hard, and then
faded. The paper observed that for all their lack of sophistication, witch
doctors and other lay healers had a vested interest in keeping things that
worked and not keeping things that didn't work. But that doesn't always seem
to be the case for modern medicine. Rather there is the 'new recommendation'
vs the 'old recommendation' but rarely any follow up on whether or not the old
recommendation or the new recommendation actually do anything positive toward
treatment.

~~~
elemeno
Ben Goldacre's Bad Pharma ( [http://www.amazon.com/Bad-Pharma-Companies-
Mislead-Patients/...](http://www.amazon.com/Bad-Pharma-Companies-Mislead-
Patients/dp/0865478007/) [http://www.amazon.co.uk/Bad-Pharma-companies-
mislead-patient...](http://www.amazon.co.uk/Bad-Pharma-companies-mislead-
patients/dp/0007350740/) ) covers some of the same ground.

He's a physician who writes mostly about bad science reporting, pseudo-science
and quackery and Bad Pharma is all about the tricks that pharmaceutical
companies get up to to ensure that trials with negative outcomes never see the
light of day and to try and spin their products in the best posible light -
like telling doctors that drug X is more effective than a placebo, but failing
to mention that it's no more effective than existing drugs on the market.
Ultimately, if the doctors who are prescribing the drug don't have the
complete picture, how are they supposed to make an informed choice about what
to give you?

I heard him talk a few weeks ago and while I might not call him 100% unbiased
(I think that some of his allegations are a touch exaggerated in terms of
their potential harm) he's definitely very interesting and eye-opening.

~~~
lostlogin
Buried inside this depressing article is a depressing story about a non-
needle-stick syringe that has been prevented from hitting the market. There
are a lot of innovation proof system and people within hospitals, good on you
if you break through it all.
[http://www.washingtonmonthly.com/features/2010/1007.blake.ht...](http://www.washingtonmonthly.com/features/2010/1007.blake.html)

------
larrys
Stanford people mentioned:

"Stanford orthopedic surgeon Eugene Carragee"

"One driving force is John P.A. Ioannidis, chief of the Stanford Prevention
Research Center."

"Stanford's Marcia L. Stefanick, PhD '82, a professor of medicine"

"Robert Tibshirani, a professor of health research and policy, and statistics
at Stanford"

"Dean of Medicine Philip Pizzo"

With the exception of "Muin J. Khoury, director of the Office of Public Health
Genomics at the Centers for Disease Control and Prevention, has worked with
Ioannidis" I can find nothing in this article that offers any counter to this
piece which, while it could be correct, is clearly slanted toward touting the
work that these people at Stanford are doing.

Not that being well balanced (presenting opposing viewpoints) is necessary in
a university magazine but there is no corroboration to what they are saying
here. This is not the same as saying they are not correct I am just pointing
out that I don't accept it on it's face and seems to be the type of thing that
would get repeated by the main stream media (the same way the studies that
they are complaining about get repeated).

Edit: Clarified some things.

~~~
archgoon
I don't think it would be overly cynical to claim that the entire point of
university magazines are to convince alumni to donate.

------
robbiep
Just going to clarify a point made about the WHI Study of HRT in post-
menopausal women.

The study did (relatively) conclusively (in a certain population of females
with long term Rx) show that HRT can increase cardiovascular risk factors.

This is useful information as women have lower CVD risks than men up to
menopause (Framingham study showed reduced CV risk for women still cycling
over those of same age with menopause) and it is thought that oestrogen was
the protective factor, therefore giving HRT would be protective.

The women studied were all post menopausal (many more than 1-2 years out) had
varying risk factors for disease. I.e they had not just reached menopause, and
thus represent a different population to that which would normally be
prescribed HRT. Besides increased heart disease, stroke and VTE the study
group also had reduced Colorectal cancer and Hip fractures that were
statistically significant.

Subsequent to this study, the KEEPS study (which may not have been published
yet) randomised women to placebo, transdermal HRT and oral HRT, and which
selected patients more appropriately as they were going through menopause, is
showing no significant difference in CV outcomes or stroke, blood pressure or
cancers between all groups.

Another study, the DOPS study, a 10y follow up of 1000 Women over 50 on either
HRT or placebo shows decreased CVD and no change in stroke or cancer risks.

This study overturned the notion that for the _general_ female population, HRT
would exert various cardiovascular benefits.

Nowadays it is established practice to only use HRT for severe symptoms of
menopause (severe, lifestyle crimping hot flushes, mood changes etc) which in
some women can last for many years. However treatment is meant to be reviewed
regularly and tapered down.

~~~
mikenuman
Thank you.

The HRT study mentioned in the article (WHI study) looked at the effects of
starting HRT, often years after menopause, often in women with existing
morbidity.

Those women who started it at the correct time ie healthy women aged around
50, lowered their all-cause mortality by 25%+.

Thus, the original analysis of HRT WAS correct. Many women have died young
since the WHI published with its erroneous conclusions, who would not had they
taken HRT.

------
alanctgardner2
As a serious question, do papers typically get published if they simply
disprove a promising idea?

As an example, my friend is doing honours research right now, and her
supervisor is very emphatic that she should do an experiment that will be
'successful' so she can publish it. My question is, doesn't performing an
'unsuccessful' experiment progress the state of the art as much as proving
something? If a reasonable researcher might have an idea, and I prove it won't
work, why wouldn't that be shared within the community as a negative result?

~~~
elemeno
Medical Journals tend not to be interested in publishing 'negative' result
papers. I can see why to an extent - they're not of particular value or
interest to the vast majority of the journal's readers. They are however
exceptionally valuable to anyone else running trials or doing research into
similar drugs in the future, so there should be, IMO, somewhere where those
papers can be made publically accesible.

~~~
archgoon
"Journal of Plausible, but Ultimately Unworkable Hypotheses"

Problem is, you don't want a high publication count in such a hypothetical
journal. That's the other side of publication bias.

~~~
alanctgardner2
But we have computers! This would be final step of proposing a project; check
that a negative, peer-reviewed version hasn't been published. Putting them in
separate journals actually solves a lot of the complaints in this thread

~~~
archgoon
Sorry for the confusion, that wasn't my point. The issue isn't that there
would be a ton of research to wade through, the issue is that scientists whose
research didn't pan now have a track record of failure. I can easily imagine a
academic position hiring conversation going like "This guy has a higher
failure to success ratio than average, and his successful projects aren't
above average."

EDIT: But perhaps I'm being too cynical (must be the New Year). Such a journal
has been constructed.

<http://www.jnrbm.com/>

With luck, it will be wildly (un)successful :).

------
rogerbinns
Sadly no mention of Dr Ben Goldacre who has also been beating the drum in this
area. Here is a good talk he did at TED (13 minutes):
[http://www.ted.com/talks/ben_goldacre_what_doctors_don_t_kno...](http://www.ted.com/talks/ben_goldacre_what_doctors_don_t_know_about_the_drugs_they_prescribe.html)

He has a website at <http://www.badscience.net>

He also wrote a book titled Bad Pharma. The Foreword contains me insight into
just how drugs are tested [http://www.badscience.net/2012/09/heres-the-intro-
to-my-new-...](http://www.badscience.net/2012/09/heres-the-intro-to-my-new-
book/)

------
dirtyaura
I remember reading "Why Most Published Research Findings Are False" when it
came out: it was both a great, bold article and the first PLoS paper that I
read. I'm glad to see that Ioannidis has continued to study the subject and
apparently is making at least some impact in medical research circles.

