
How a Reporter Pierced the Hype Behind Theranos - danso
http://www.propublica.org/podcast/item/how-a-reporter-pierced-the-hype-behind-theranos/
======
arcticfox
The point about needing a doctor's order to get a blood test seems pretty
irrelevant to the rest of the situation. I don't know why they included it.

I'd love to get a simple blood test without needing to talk to my doctor
(plenty of people would). I would _not_ love to get a blood test
misrepresenting its accuracy (nobody would).

~~~
timr
The reason is the same as why you require higher standards of statistical
evidence when you don't have a strictly defined hypothesis before an
experiment: if you go looking for something in noisy data, you'll probably
find it.

Lab tests have significant margins of error. Just by chance, if you go out and
get a bunch of lab tests, you'll probably find something "wrong". Unless
you're knowledgable about interpreting the results -- and unless you are a
practicing medical doctor, you are _highly unlikely_ to be knowledgable, even
if you think you are -- this can/does lead to catastrophically bad outcomes.

(And now, for an ironic aside: the median HN user is extremely quick to attack
valid science for the reason that "most results are wrong", while also
complaining loudly when the medical establishment wants to restrict access to
clinical tests with a large margin of error. Go figure.)

~~~
cheriot
For things like STD tests it raises the barrier and cost of frequent testing.
We need a more fine grained policy than all blood tests require a physician.

~~~
timr
They don't. You can get cheap/free STD tests from non-physician providers. You
can also buy STD tests at the drugstore, over the counter. But like it or not,
anyone who collects biological samples from strangers' is going to require
some training and licensing.

Moreover, the interpretation of an STD test is not as straightforward as you
might think. False positive rates can be high for the cheaper tests, and
there's natural variability in things like antibody titers. People also
neglect that timing matters -- if you test too soon after infection, even the
most sensitive tests won't work. Or they fixate on one particular scary
illness (e.g. HIV), and neglect testing for others that are less well-known,
but more common (e.g. Chlamydia). Some of these problems can be mitigated.
Others can't.

Regardless, having someone trained to interpret tests is a good idea for just
about everyone. Even the cheap/free/OTC tests come with that service, at a
minimum. It's a bad idea to just allow people to pick and interpret their own
tests, even for the "simple" ones, like STD tests.

------
goodcjw2
Lab test, or medical practice in general, is a well protected industry here in
States. The public wasn't educated well enough to understand lab tests at all
so that professional opinion IS REQUIRED to understand your own health.

I might be over cynical here, but I blame that both medical education and
health care industry are restricted and protected to cause a very inefficient
public health system in US. Thus normal people without a MD has no clue to
judge good product vs bad product. In the case of Theranos, I don't think most
of understand what's going on with the test. How the test works, how to judge
whether it's effective, how to compare pros and cons between different
products. We have no idea. I doubt whether the report himself understands the
real deal behind the scene, or even investors of Theranos understands it.

~~~
SilasX
I'm pretty sure it wasn't the "doctors get to be gatekeepers" problem that
held back Theranos here. Rather, it was the "they don't have an actual working
finger-prick test and have avoided numerous cheap ways to prove they have one"
problem.

In contrast, 23andme can legitimately claim to have been held up in that
respect. They did have a working product, which correctly identified the
patient's genes, and correctly reported the state of the literature on each
gene; no one claimed otherwise. It's just that the combination of those two
services counts as a medical device under current law, and they didn't get
regulatory approval first.

~~~
goodcjw2
I tend to agree with you on Theranos. I wasn't trying to say that Theranos is
doomed from the beginning because they are about to disrupt a well protected
industry. Instead, I feel the problem is that we as general public are not
educated well enough to make a reasonable judge on whether Theranos is
actually working.

------
danso
Someone asked about a transcript. There's no official transcript as far as I
can tell so I sent it up to IBM Watson:

[https://gist.github.com/dannguyen/71d49ff62e9f9eb51ac6](https://gist.github.com/dannguyen/71d49ff62e9f9eb51ac6)

First part (rather than paste the entire thing, you can just check out the
gist:
[https://gist.github.com/dannguyen/71d49ff62e9f9eb51ac6#raw-t...](https://gist.github.com/dannguyen/71d49ff62e9f9eb51ac6#raw-
txt-md)

and nbsp walking to the ProPublica podcast I'm Charlie Orenstein and I cover
healthcare here at ProPublica

today we're gonna be talking about laboratory testing

it's no secret to many that lab testing has been caught in the dark ages of
medicine

or you have to go to your doctor to get a prescription for a lab test

and then your doctor gets the results of the lab test if you're lucky will
share with you

and then you'll be back to going to your doctor again if you need a follow up

a company called their in house aim to change that

founded in two thousand and three by Elisabeth homes

a nineteen year old drop out of Stanford University this company and to really
shake up laboratory testing

it received fawning coverage from many mainstream media outlets including The
New York Times

but John Kerry Roo a reporter at the Wall Street journal took a more critical
look at their in house

and found that behind this glossy surface there were a lot of questions about
both its effectiveness and the science that underlie its main products

John is joining us here on the podcast John Markham hi thanks for having so
how did you decide to take a look at their house

well I had read Ken Auletta as profile of Elizabeth homes in the New Yorker in
which I think came out in December of two thousand fourteen and

I found it interesting there were some brief critical %HESITATION sections in
there that raised questions for me but I don't think all that much more of it
and then as the calendar year turned to %HESITATION two thousand fifteen a
couple weeks later mid two thousand fifteen I got a tip from someone not a
primary source who had any primary information but someone who is a relating
to me

some you know third hand information that that things might not be exactly as
they seemed at this company thoroughness

so it's interesting because I think a number of reporters got tips along the
way and and you were the one to pursue them I personally heard from somebody
who said that his lab results didn't match the lab results he got from a
different lab so it doesn't seem like this was a huge secret that there were
questions out

right and shortly after I began looking into the company in and looking into
this tip a Stanford on a medical school professor I believe %HESITATION put
out a critical opinion piece in the journal of the American medical
association

and he didn't really have any any information other than the say this is a
company that was

making very bold statements about its breakthroughs in the science of
laboratory testing saying that it could test for a number of conditions off
just a drop of blood ____from the finger and yet it hadn 't really done what
you usually do in medicine which is a peer review

and he was questioning %HESITATION this new trend of making all these
assertions about what you've invented without really proving it and having it
vetted by your peers and then there was another %HESITATION professor in
Toronto I believe a couple weeks or couple months later who who came out with
a critical medical journal and I editorial as well so people are beginning to
speak out in the scientific community as I was doing my reporting

so let's just take a step back and put the science aside for a second what is
the hope of fairness what it promised people

so the the fair knows invention as Elizabeth homes are

you know announced it in magazines and at conferences was that %HESITATION
with a tiny drop of blood ____from a finger with the lancet

they could run the full range of laboratory tests and get you back results on
all these tests within hours

and at a low cost to the latter part is true they charge very low prices and
then when you look at the assertion that they can do the full range you you
ask laboratory experts what that means and they say can mean anywhere from
several hundred to several thousand tests

so the claim was quite bold in this hadn't been done before from just a drop
of blood being able to run the full gamut of tests and get results back to the
patient's very quickly %HESITATION it did sound like a real scientific
breakthrough and third for listeners who may not be aware of what is it
usually take as far as blood to run that gamut of test what weirdest blood
come from and how much blood is needed

well if you get tested for say a a comprehensive metabolic panel which is a
typical panel that you might not get a prescription for your doctor for which
is about a half dozen tests

you'll go to request or a lab Corp or a hospital lab in they will draw your
blood with a needle odd that they put in your arm and they'll draw about five
tubes of blood so quite a bit of blood is typically %HESITATION required to to
run you know half dozen tests

so Liz with homes was incredibly successful I think getting it hundreds of
millions of dollars four hundred million dollars %HESITATION of invest more
money and more than us more than that so that value the company at nine
billion dollars or so

yeah we we went back and found some %HESITATION some regular

for filings and were able to calculate that they raised at least seven hundred
and fifty million dollars most of it more than six hundred million dollars was
raised in two thousand fourteen and that last fundraising round valued the
company at about nine billion dollars

which is a a huge of valuation that that's our moral less what quest and
LabCorp are each valued at and these companies have been around for a long
time and they have a huge revenues in huge profits

are in this laboratory upstart that was founded by a college drop out ten
years ago suddenly valued at at the

the same valuation as those two huge companies alright let's stop there
because I think listeners Arabic %HESITATION why like how did this unicorn

get so much money what did people see it

that isn't clear what people saw in it other than %HESITATION Elizabeth homes
as pitch

that she had made this scientific breakthrough is not clear because I've heard
that and and pretty much ascertained during my reporting that the company did
not offer really any information

about the science and about how the technology works about how it's

laboratory %HESITATION instrument worked

or about its financials so are investors who were putting up this money were
for the most part going in blind

but at a star studded board right

they did they had %HESITATION Henry Kissinger

and now George Shultz and Sam Nunn and %HESITATION

Phil frets they had a bunch of older statesman some exit some retired military
commanders

it was a %HESITATION heavy duty board a lot of big names

in %HESITATION the military and and former cabinet members

%HESITATION incident seemed impressive at first glance

so your first piece ran last year and you began raising questions what were
the key questions that you found

so the first thing was to

look at whether they were or were not running the full range of laboratory
tests

on other proprietary technology

and after a lot of reporting in talking to former employees who were in a
position to know exactly what the reality of that was

I was able to determine dead of the more than two hundred and forty blood
tests that they offered consumers at their blood draw centers in Walgreens
stores

that at the very most at the end of two thousand fourteen fifteen of them were
run on their

pride Terry lab instrument which by the way they called the Edison after the
inventor Thomas Edison pretty clever out why does it matter though that who
cares which instrument that tests were run

well

if you're asserting that you that you made a scientific breakthrough that it
enables you

to run the full range of laboratory tests off just

a drop of blood and it turns out that you can only do fifteen tests then

you're you're probably hyping where you are in and your ability and so then
the question arises whether you've told the truth to investors whether you've
told the truth to the public

but then there's the the other issue that I discussed in my first piece which
was published in October which is what whether or not

the testing for those fifteen tests and and that the testing for all the other
test

was accurate and produced good results and and I had ex employees telling me

that %HESITATION they questioned the accuracy of the Edison machine and that
their endless was also doing things like diluting small blood samples in order
to create a bigger volume to run them on commercial analyzers

that also created problems with accuracy

and I would travel to Arizona and talk to patients and doctors there and came
up with anecdotal evidence of

a test that didn't seem to square with comparative tests done at other
laboratories and so all of that made me realize that it wasn't just about
potentially company that it over hyped

~~~
autopov
I can forgive Watson for not understanding proper nouns, but not for confusing
it's and its (given plenty of context).

The insertion of %HESITATION to denote pauses of speech is an unexpected
delight.

A long time ago, as work-study, I transcribed several audio recordings of
descendants of the first non-Native settlers of Santa Clara Valley. There were
many ellipses in my transcriptions, and I wonder if future researchers will
question if they represent hesitation, ums or uhs, or periods where the
recording was unintelligible.

------
alanh
Hmm. Is there a transcript?

------
vonklaus
> ...Theranos also played a role in the passage of an Arizona law that allows
> people to get blood tests without a doctor’s order. Carreyrou: That was
> controversial, because there are many in the medical profession [...] who
> say, "Well, how is that progress?" [...] more likely than not, they're going
> to need a doctor's opinion to decipher them.

This is a good point. Diabetics usually go to the doctor 1-3 times a day to
have bloodwork done. Surprisingly, despite how open the medical field is from
a regulatory, technological and informational standpoint we have done almost
nothing to combat the complexity of tests taken regularly by most of the
population. I would have overlooked this and I am glad the author brought this
up, maybe it would be possible for users to be trained on some of the newer
technology being rolled out in the medical field like the Facsimile Machine
(often called FAX or sometimes by the synecdoche XeroX) to communicate with
highly skilled professionals, to interpret the data. It is likely that every
person would fall outside of a standard bell curve so it would be impossible
to teach anyone how to interpret the test, but we could set our sites on
leveraging FAX in the future.

> How a Reporter Pierced the Hype ...

This is great reporting and a perfectionist usecase for new media. The
journalist was able to quickly go on wikipedia to create a paragraph of color,
and then link to the interview so that readers would be able to immeadiately
hear how this reporter had broken the hype machine over a 20 minute interview
at an offsite location. Maybe, unlike everyone else, you actualy don't have 20
minutes to listen to this interview, perfect contingency here 3 bullet points.

Much like the author, I skipped around through the clip pulling random qoutes,
the reporter is asked:

 _A number of reports got tips and yoiu were the one to purse this. Other
reporters got tips as well but you pursued this?_

Not knowing much about how the Peer review system, I called my colleague G.
oog LeSearch Barr who confirmed peer review is basically the only important
thing about conducting science, and the system is renowned for it's
perfection.

The reporter did say 2 people in the community had spoken out, which to be
fair, is quite reasonable. He was pragmatic about waiting to fully engage with
the story until, and I am paraphrasing but editorializing much less than one
would think:

The 9 Billion in 2014 is a big valuation.

How did this unicorn get money?

What people saw other than Holme's pitch isn't clear. I have _heard_ the
company didn't offer anything about how the technology worked.

After a lot of reporting, I talked to a lot of employees who were in a
position to know, I determined that 350 blood samples 15 were run on their
machine.

\---

Now it is important to realize, Theranos didn't use the AMA standard bootstrap
medical breakthrough CDN and while not much is known about the internals of
the company, it is believed they didn't spin up a AWS Full Scale Research
Labratory, they may have not even _considered_ using GO.

Now, all that can be forgiven, right? But it is such a shame they didn't just
roll this out overnight. It is fucking trivial to build this, you can just use
Crimson on Beaker for the front arm and then persist your regularatory data to
the USFDADEAFBI?SOMESENATOR.gov/api and get approval.

750 million dollars to basically build a crud app seems like a lot of
investment. Obviously Holmes, much like her predecessor Alexander the great,
had to hire Michael R. Taylor the Head of the FDA to follow her around saying
"You are just a human, only a mortal". Notably, this actually should have
helped Theranos as Taylor is a renowned scientist who has won a nobel peace
prize. He graduated with a degree in political science doing pretty serious
research as a staff attorney at the FDA, and working with the private firm
that represents the worlds biggest provider of open source seeds, Monsanto.

As an aside, Taylor is a New York Times best selling author for his riveting
1988 article entitled "The De Minimis Interpretation of the Delaney Clause:
Legal and Policy Rationale", where his interpretation of a 1958 law about the
ppm of chemicals, is considered truly breathtaking when read.

\----

She may not succeed, she may (we DO NOT KNOW) have cheated, but what fucking
world do we live in if we can preach "Fail Fast" to the 40th
uberPetBNBPariscopeAOLtimewarnerTwitterBlockchain-#NOBACKDOOR-google-earth-
for-mars-but-with-instagram-filters

and tell the people who are fucking trying to push forward the most highly
regulated industry forward, which requires not only innovation but capex, and
a strong board (Kissinger/etc are there to lobby because you CAN"T EVEN
COMPETE without someone connected, much less succeed).

So you can question her ethics, her strategy, etc, and if she committed fraud
I certainly wouldn't defend that, but if it ends up she took a shot at
something big and missed, I vote we let the next guy or girl take a similar
shot.

------
lingben
posted yesterday:
[https://news.ycombinator.com/item?id=11128535](https://news.ycombinator.com/item?id=11128535)

