
Theranos Ran Tests Despite Quality Problems - Asparagirl
http://www.wsj.com/articles/theranos-ran-tests-despite-quality-problems-1457399479
======
Asparagirl
Key takeaways from this new article, for people blocked by the WSJ paywall:

"That letter cited “deficient practices” in five categories. In one of them,
hematology, the problems found by inspectors posed “immediate jeopardy to
patient health and safety,” regulators said.

The test that inspectors found Theranos kept doing despite the erratic
quality-control results was a hematology test that measures how long it takes
blood to clot, the people familiar with the matter said.

The test is an important part of the treatment regimen for patients who are at
risk of strokes or have blood disorders...

People who have seen the Theranos inspection report said it seems to indicate
the company ignored quality-control results for the blood-clotting test that
fell short of its own criteria.

The results produced in the quality-control checks repeatedly deviated from
the lab’s typical result by more than two standard deviations, those people
said...

At Theranos, there were seven quality-control failures in a single day, said a
person familiar with the report...

Dr. Hamill said any doctors who received prothrombin time test results from
the Theranos lab during the six-month period shouldn’t rely on them and should
have their patients retested as soon as possible. “Those results are not worth
anything,” he said...

The federal inspectors also concluded that the Theranos lab in California used
expired reagents, or substances added to blood samples to elicit chemical
reactions, for some prothrombin time tests, according to the people familiar
with the inspection report.

Those people said the report also found that the lab’s director didn’t have
responsibility for the lab’s quality-control program and the lab’s quality-
control manager was unqualified..."

EDITED TO ADD: Actually, the really key sentence is this one: "Theranos
spokeswoman Brooke Buchanan said: “The PT/INR issues identified by CMS related
to tests run on conventional equipment using venipuncture samples.” A CMS
spokeswoman declined to comment." Because that means that these quality
control problems were _not_ due to Theranos' "revolutionary" finger-prick
tests, they were from regular old venipuncture tests running on a standard
machine that were giving wildly incorrect results, expired reagents, no QC in
the lab, etc. Bad data is worse than no data. If they couldn't get old-
fashioned labs right, no wonder they were having trouble with the finger-prick
stuff.

~~~
cordite
That is rather scary. Quality control is a serious requirement to any
competent lab. With physicians making decisions by results labs transmit,
there is an inherent need of verifiable trustworthiness to the results.

A patient may be subjected to life altering medication or treatments, which is
not a trivial responsibility that can be shrugged off with erratic processes
or lack thereof.

A lab director's responsibility includes ownership of quality processes,
reviewing abnormal results after the fact to see if proper processes were
followed (such as communicating with the respective provider), reviewing
financial success, and much more. To shrug off quality control and operate as
if reputable is not just unethical, but incompetent. Such operations should
cease immediately.

Disclaimer: I have experience in the medical lab industry.

~~~
TazeTSchnitzel
It makes me wonder if Theranos's inaccurate results have killed anyone. It's
not unlikely.

------
Animats
At what point here do people go to jail? The FDA can and has done that for
faking lab tests.[1][2]

[1] [http://www.nytimes.com/1983/10/22/us/3-ex-officials-of-
major...](http://www.nytimes.com/1983/10/22/us/3-ex-officials-of-major-
laboratory-convicted-of-falsifying-drug-tests.html) [2]
[http://pubs.acs.org/subscribe/archive/tcaw/10/i11/html/11reg...](http://pubs.acs.org/subscribe/archive/tcaw/10/i11/html/11regs.html)

~~~
ageofwant
When they have been found guilty in a court of law ?

~~~
HarryHirsch
But who has standing to sue? Whoever suffered a stroke because their warfarin
dose was too high or too low is dead, they cannot sue any longer. Whose dose
was accidentally correct or whoever was lucky enough not to suffer a stroke
did not suffer harm, so they can't bring suit either.

And Holmes and the board believed in good faith that their prothrombin test
was accurate. No _mens rea_ , no culpability. No prison for anyone.

~~~
ageofwant
The federal agency tasked with that responsibility I would naively presume, or
do you prefer trial by media ? And why, exactly, would someone that acted in
good faith deserve prison ?

~~~
nkrisc
[https://en.wikipedia.org/wiki/Negligent_homicide](https://en.wikipedia.org/wiki/Negligent_homicide)

You don't have to mean for something bad to happen to get jail time. You're
still responsible for your actions, intentional or not.

I'm not lawyer, so bear with me, but my lay understanding is that if it can be
shown that they should have been aware of the quality and other issues
regarding their results, they could be criminally negligent. Feel free to
correct me.

~~~
ageofwant
I'm not a lawyer either. And being responsible for your actions certainly
seems reasonable to me. But I cannot see how you could have acted in good
faith if you were aware of quality and other issues regarding your result.
That would be acting in bad faith.

------
jorgecurio
This is speculation but...the company was founded in 2003....13 years ago and
no product out yet...I have an awful feeling Holmes felt desperate, took on
more than she could chew in order to compensate for the decade she lost, and
now all the shortcut taking is catching up to her....

It's starting to look very ugly for what started out as something
overreaching, and I think she will fight even harder to keep the house of
cards from falling but pretty much this unicorn will be the first to go in
2016.

------
ghshephard
The key takeway I have from all this is if you are relying on Blood Tests for
data that is critical to your health (as opposed to informative, educational,
or personal interests) - it's probably a good idea, if you can afford it, to
get results from three completely separate diagnostic systems - kind of like
time with ntpd.

~~~
nradov
Not practical for most patients in the real world. Insurers won't pay for
redundant lab tests. Getting blood drawn multiple times is painful and
logistically difficult. For patients with clotting problems just getting blood
drawn presents some risk. And there often won't be 3 independent labs in the
patient's area.

~~~
ghshephard
You could reduce your accuracy by doing three tests of the same blood sample
(of course, exposing yourself to the risk that the blood was taken
incorrectly, was stored in an incorrect vessel, was transported incorrectly,
etc...).

In terms of practicality/Insurance - this would mostly be a function of how
absolutely certain we are that the tests are accurate, and how damaging
incorrect results would be. If we are 99.99% certain the values are correct,
and incorrectly results wouldn't result in irreparable harm - then yes, not
worth the investment in redundant lab tests.

But, if there is, say, a 5% chance I might die on an incorrect test result
that has a 1% chance of happening - and that test is performed 12 times/year -
then you can pretty quickly calculate the net-present-value of my life versus
cost of test. (I.E. If I think my Life is worth $10mm, and the test normally
costs $500 each time, or $6000/year, and I need to have this test done for the
next 10 years, (assuming the value of my life doesn't drop over time, to make
this simpler) - then each time I get the test there is a 1% chance of failure
* 5% chance of death = 0.05% chance of dying as a result of each test.

The odds of me dying then, in the next 120 test = 100%-(chance-of-not-
dying-120-times-in-a-row) = 1-.9995 ^ 120) = 5.825% chance of dying in this
scenario using tests that cost $60,000. If I could reduce that to 1% (4.825%
reduction) chance by spending an extra $120,000 (perform the test in two extra
labs) - then my my investment of $120k would return 4.825%*10mm = $482,500.

You can work the numbers, assumptions, etc.. however appropriate - but for
some combination of test costs, frequency, potential harm, and value assigned
to life - there is quantifiable value that justifies performing the test
multiple times (and, likewise, clearly circumstances in which, even if the
test results are more likely to be inaccurate, not worth performing thee
test).

Not having access to multiple labs is a real problem though.

~~~
nradov
That's not how healthcare economics works in the real world. No one can
quantify the chance of death due to an incorrect test result precisely enough
to calculate when it's cost effective to perform multiple redundant tests. The
supporting data doesn't exist and no one is going to fund the research to
figure it out. And what does 'incorrect' even mean? There is always some
margin of error.

In real medical practice if a physician sees a lab result that is way out of
range, or significantly different from the patient's previous result, or
inconsistent with other aspects of the patient's condition, then she will
order a repeat of the same test in order to confirm the results.

~~~
ghshephard
It's certainly how healthcare economics works. It determines what, if any
tests should be run. It also determines how many parties/independent reviews
should take place for high stakes tests such as tumor identification in
carcinoma.

Other high-stakes scenarios in which the same tests are run multiple times are
for high-economic cost events such as drug abuse, either in the workplace, or
in sports.

You even note in your response, that, _In real medical practice if a physician
sees a lab result that is way out of range, or significantly different from
the patient 's previous result, or inconsistent with other aspects of the
patient's condition, then she will order a repeat of the same test in order to
confirm the results._ \- which is exactly what I'm suggesting people have done
- only with a (presumably acceptable) time delay between when the first, and
later tests are required. If the results of such tests were that clinical
activity had to take place quickly, then you might not have the luxury of a
delay, and in which case, multiple independent test vectors would be very
useful - if they were economically feasible.

What people sometimes fail to consider - if the test was worth doing _once_ \-
at, say, $600 - then why wouldn't it be worth doing (at presumably much higher
confidence), three times at $100? And, if people agree, "OK, for $600 and $100
that makes sense" \- they have to understand, those two numbers are completely
arbitrary, and can only be understood in relationship to the value of the
results. Whose to say there aren't a ton of tests today that are "$100" \-
that couldn't be done multiple times to be certain.

------
return0
This all sounds spectacular i just wonder why it's only WSJ that is doing all
the reporting for them.

------
nkrisc
Well they certainly managed to "disrupt" something, like trustworthy lab
results!

------
daveguy
Does anyone know if Theranos conducting tests outside of California? I thought
I heard something about Nevada and maybe one other state?

~~~
seehafer
Arizona.

------
yitchelle
Just starting to read up on this. Does anyone know if any of Theranos's
customers used the lab results for ongoing medical treatment and suffered as a
result of it?

------
thrillgore
I just want to know when the tell-all book will be released.

------
stevebmark
I've never seen so much scrutiny over a company before. Everyone is screaming
"death to Theranos" at every opportunity. How much of that is purely balking
over the high valuation? I have a feeling if they didn't raise so much money
then journalists wouldn't be swarming this at all.

All of the critiques I see don't seem even close to show stoppers. No public
peer reviewed studies? Some lab updates to match FDA regulations? Using some
"unauthorized" third party equipment? This is a new company, why should these
be the doomsday callings?

I have no idea if Theranos will succeed, but I can't shake the feeling that
this media response is inflated due to the high valuation, and journalists are
falling victim to the hype as much as the public, whereas this is a fairly
normal health startup otherwise. It probably was a mistake to accept such a
high valuation, so I don't think Theranos is totally innocent here.

It's also strange that a paywall article made it to the front page? I wouldn't
guess there's a big overlap of WSJ subscribers and HN readers (lol
conspiracy).

~~~
BinaryIdiot
> How much of that is purely balking over the high valuation?[...]this is a
> fairly normal health startup otherwise

Absolutely not. Start-ups are supposed to fail early and fail often but when
you're in the medical space you simply can't act entirely like a technology
start-up. People's lives depend on many of these tests; you can't just go
"well let's ignore the FDA and we'll take the slap on the wrist" like in other
industries. Here they might deliver inaccurate test results that are then used
to treat someone for a medical issue.

That can be deadly. Yeah many people are going crazy from the valuation and
secrecy but there is plenty of legitimate scrutiny here especially if you read
this article (I'm assuming you didn't because of your very generic Theranos
comment versus addressing the issues in the article itself which sound grossly
negligent, IMHO).

~~~
sbose78
True, if a startup in the medical didn't have the resources for quality
control they should not have started up in the first place.

