

IBM's Watson will be used to make decisions about cancer care in 14 hospitals - Brakenshire
http://www.bbc.co.uk/news/technology-32607688

======
nl
Everyone here has got the wrong idea about what Watson is doing here.

It maybe doing sequence analysis, but I've never seen anything about that
before.

The real strength is the journal article reading mentioned in passing. What it
does is read studies on new treatment options as they are released, and then
when a person presents symptoms it will do a diagnosis and suggest treatment
options along with supporting evidence.

The natural language understanding needed to do the journal understanding is
the real unique offering here.

There are videos on YouTube showing the system. I'm on mobile so I'm not
looking them up now.

~~~
grownseed

      Everyone here has got the wrong idea about what Watson is doing here.
    

That would be because the article is very misleading, and without some
background in the field, can you blame people for thinking that?

Take for instance this paragraph:

    
    
      However the process is very time-consuming - a single patient's 
      genome represents more than 100 gigabytes of data - and this needs 
      to be combined with other medical records, journal studies and 
      information about clinical trials.
      
      What would take a clinician weeks to analyse can be completed by 
      Watson in only a few minutes.
    

The first section seems to imply that the sequencing data and the analysis
data are the same thing, which they aren't.

As for the second section, it is simply not true. Clinicians only intervene at
certain points in the process, and their work for a single patient certainly
does not take "weeks", however the whole process does. But Watson would not
cover the whole process, despite the impression being given.

It is just shoddy reporting, taking shortcuts in the wrong places. What
saddens me is that the general public will take the article at face value and
expect their caretakers to live up to this fiction.

Also, please see my other comment for more details, in particular about the
job Watson is actually supposed to be taking care of:
[https://news.ycombinator.com/item?id=9501912](https://news.ycombinator.com/item?id=9501912)

~~~
nl
Yes, I agree the report was pretty bad.

------
madaxe_again
This is like using a sledgehammer to put up pictures. You simply do not need
the kind of parallel compute grunt something like watson can provide to do
correlation analysis as they describe. You could cobble this together in
{dirty secret scripting language of choice} and run it on your laptop.

This is a PR piece - not the article, the activity - to bring forth the idea
of computers making decisions about healthcare, based on metrics - not humans,
based on metrics _and compassion_.

The cynic in me sees this as setting a disquieting precedent in the direction
of healthcare being distributed according not only to patients prospects and
treatment needs, but other factors which a financially liable party, say an
insurer, would be interested in - such as the earning potential of the
patient.

I'm pretty sure there was a Star Trek episode about this.

~~~
micro_cam
I've worked in the area of clinical genomics using whole genome sequencing.
Your statement is unfortunately untrue though it perhaps could be if tools
were better written.

While it is easy enough to analyze a single genome on your laptop most current
popular analytical tools simply fall over when you start looking at hundreds
of genomes on even a large server. Even basic stuff like combining multiple
genomes into one file with consistent naming of variants can take entirely
ridiculous multi terabyte amounts of ram because the tools to do so just
weren't written with this scale in mind.

Most of these tools could (and should) be rewritten to do things without
loading the whole data set in memory and work natively in a cluster of
commodity clusters. There is some resistance to this of course because
scientists prefer to use the published and established methods and often feel
new methods need to be published and peer reviewed etc.

Until new tools are written and widely adopted to a large shared memory
machine is a bandaid many hospitals and research seam eager to adopt.

~~~
a8da6b0c91d
What hard evidence is there that genomics is relevant to cancer treatment, as
proven by survival rates? Color me skeptical.

~~~
Gatsky
You are right to say this. I treat breast cancer, and I'm doing a PhD on
breast cancer genomics, and there is no evidence that high throughput data of
any kind, whether it is genomics, transcriptomics, epigenomics, proteomics,
metabolomics etc-omics actually helps patients. At the moment, a small panel
of biomarkers using technology that is at least 20 years old is all we use to
make treatment decisions. Is it adequate? Certainly not, but there is a HUGE
amount of carefully collected data in many thousands of patients backing it
up.

Not sure who is downvoting you, but they seem to have swallowed the hype
wholesale. At the risk of sounding gratuitously negative, I find the
discussion of medicine on HN to be of very poor quality, markedly below the
general standard.

~~~
tom_b
I think there is a distinction to be made here in questioning patient outcomes
and questioning the relevance of genomic sequencing in treatment decisions.

Don't you think it is fair to say that high throughput data (whole genome
sequencing with variant calling) is still in a state of being evaluated to
measure its effectiveness in aiding the treatment decision process but that
_early_ results seems to lean towards it becoming part of the standard
diagnostic approach?

Genomic sequencing and patient outcomes is a thornier question. My non-
practitioner take is that it is too early to tell scientifically, but that
there will probably be some benefit to early identification of specific cancer
types and choosing treatment. But I think many people would have made a
similar statement about mammography and early detection, and absolute
mortality appears to not be reduced by adding mammography to the diagnostic
procedures, right?

The research value of genomic sequencing seems high enough to make it
worthwhile. At least, when I sit in on molecular tumor board reviews (the
oncologists at a table looking at called variant results for a specific
patient), I hear them commenting about possibly new and unknown variants being
of research value.

I am really looking forward to your reply - Internet message boards in general
have to be almost the worst way to discuss medicine, but having participation
from researchers and practioners like you is tremendously illuminating!

~~~
Gatsky
I define genomics as the unbiased interrogation of the genome using high
throughput technology. Sequencing one mutant locus using Sanger sequencing
does not fall under this definition - I don't think IBM's business model is
using Watson to interpret that. So when other people point out that HER2 is a
useful genomic marker they are missing the point - HER2 can be determined with
immunohistochemistry for example which has been around for 50 years.

I'm not sure what your question is... genomics has research value, for sure,
it's great. Is it worth trying to incorporate it into routine care? Yes,
probably, if you have enough cash. Should a hospital pay for a black box
machine learning algorithm to make recommendations from a highly polluted,
often erroneous and hugely incomplete literature corpus? The alternative put
forward by people actually doing the science is that we should try and develop
large open source databases/repositories about the significance of genomic
findings, and then collect the data about what happens to the patients.

------
socialist_coder
I have a daughter with Nephroblastomatosis, a super rare kind of kidney cancer
that often lasts for years before just naturally going away (no one knows
why), and dealing with the treatment options is agonizing.

For example, she's been on 3 different kinds of chemotherapy for 12+ months. 2
of the chemotherapy drugs don't seem to do anything. The 3rd kind seemed to
result in an immediate reduction of the nephroblastomatosis, but unfortunately
this chemo drug is very harmful to your body and there is a limit to how much
of the drug you can receive. Now, she is not on chemo anymore.

Every 1.5 months we go in for a checkup, alternating between an MRI and
sonogram.

2 times they have found "spots" on her kidneys. But, what to do about them?
Additional chemo? Surgery? Wait and see? This is what I want a computer to
tell me. What are the risks of surgery? What are the risks of losing more of
your kidney? What are the risks of additional chemo? What are the risks of
doing nothing? I would rather a computer had this information and gave its
recommendation, rather than a group of doctors and us, the parents. We would
be using the same logic in our decision, so it seems like the computer could
do it more accurately and with zero bias.

I welcome Watson into our health care system. Anything that takes some of the
agony out of very very difficult treatment decisions will be a good thing.

------
gkoberger
I wonder if the Jeopardy appearances helped Watson (by getting the name out)
or hindered it (by making people think it's just a parlor trick). Either way,
this is seriously fascinating: I'm excited to see what other applications it
has.

~~~
duaneb
People aren't stupid, and if it was a parlour trick it was an exceptionally
well-executed on. People, even those who don't have a grasp on the
computational complexity of language and reasoning, seem legitimately
impressed by the performance.

~~~
sgt101
Yes, and it was a real research collaboration - a brilliant insight - to see
that the Jeopardy language was constrained and tractable to analysis that
would allow a mapping to a solver.

I have not been involved in these efforts or the community closely enough to
say, but I think that things like controlled english

[http://en.wikipedia.org/wiki/Attempto_Controlled_English](http://en.wikipedia.org/wiki/Attempto_Controlled_English)

and

[http://web.eecs.umich.edu/~soar/sitemaker/docs/pubs/Interact...](http://web.eecs.umich.edu/~soar/sitemaker/docs/pubs/Interactive_task_learning_for_simple_games.pdf)

may have been inspired by the success of Watson in Jeopardy.

On the other hand I was running a question answering question when Watson
started, and the PI said to me "oh yeah, but that's cheating..."

And on the third hand we stopped and closed that project pretty well six
months later.

So - there's a lot to think about!

------
grownseed
The (very rough) process with next-gen sequencing is as follows:

    
    
      1. one or more samples are taken from the patient (tumor/normal, peripheral blood, bone marrow, etc.)
      2. the samples are sequenced, this can take different forms, involve different processes, but basically, specific sections of the genome are extracted, amplified, then turned into digital data
      3. the data is aligned to a reference genome
      4. the aligned data is scanned for variants (mutations), often in specific genes, and even specific sites of certain genes. This again involves different methods often used in combination
      5. the variants are partially filtered out automatically (poor quality, bad reads, spurious, synonymous, etc.)
      6. the remaining variants are then reviewed manually, relying on academic papers and historical data
      7. what remains is reviewed again, taking the patient's history into account, and potential targeted treatments are inferred
      8. the patient and his/her doctor make the final decision
    

So from what I understand, even in an optimal scenario, Watson would be taking
care of steps 6 and 7, which represent maybe 15% of the overall time. I
believe the article presented here is rather misleading in this regard, as it
seems to suggest that Watson would be taking care of the actual sequencing
too.

Additionally, I imagine that in most cases some humans would still need to
review Watson's data.

What the article doesn't mention either is access to the actual patient data,
which in itself is a political and operational problem, not so much a
technical one.

Finally, I imagine Watson would infer the individual variant diagnoses from
papers, which again, is far easier said than done (though I do believe that
would be the point where Watson shines most). Parsing papers reliably for
usable data is really, really hard, in no small part because the academic
system is antiquated, as are paper formats. Even if you manage that, you still
have to figure out whether the papers in question, or their authors, are
actually reliable sources. And even then, you will likely end up with
conflicting interpretations which will need to be sorted out.

So while a nice fluff piece for IBM, I have serious doubts about the actually
meaningful benefits Watson supposedly brings to the table.

To be clear, I do believe that AI/ML can and do really help, and contrarily to
what the article might want you to believe, people are already working with/on
it. However making Watson look like it will solve all the current problems
with cancer care is disingenuous at best.

~~~
dekhn
I agree with your guess that Watson covers 6/7\. From the clinical geneticists
I've talked to, variants of unknown significance (where a person differences
from the reference, the impact of the difference) are investigated manually
through literature searches- often just basic text matches on the variant
name.

Regarding paper parsing: there are companies that employ hundreds of PhDs that
read papers all day long and enter the data into a database based on ontology
(Ingenuity). IBM would license that database and use it as a prior.

------
kifler
Would be interesting to see what kind of liability this could open them up to.
It had thought that Toronto was a US city, for example.

~~~
duckmysick
I imagine there are some renowned physicians in Europe and Asia who also don't
know that Toronto is a Canadian city. Or other arbitrary pieces of trivia for
that matter. It doesn't stop them from being successful at their job.

Conversely, knowing which city has airports named after a WWII lieutenant and
a WWII battle doesn't protect from malpractice.

------
WalterBright
More accurately it will be used as a diagnostic advisor.

------
zkhalique
About this prediction:

 _" in the future, every decision mankind makes, every decision, is going to
be informed by a cognitive system like Watson and, as a result, our lives in
this world are going to be better for it."_

well if by "better" you mean "more machine-like and regulated like animals in
a zoo".

This is already somewhat true of consumer credit reports, and now this:
[http://www.ibtimes.com/china-use-big-data-rate-citizens-
new-...](http://www.ibtimes.com/china-use-big-data-rate-citizens-new-social-
credit-system-1898711)

~~~
Dewie3
Just train/program the AI to be more empathetic. :)

------
sjg007
This is more of a research PR piece than anything.

~~~
OscarCunningham
For certain tasks, AI has long been more accurate than doctors. But it isn't
used because doctors and patients don't trust it. So perhaps PR is exactly the
missing piece of the puzzle.

~~~
dudurocha
Not trying to doubt you, but can you tell me in what tasks Ai has been more
accurate than doctos?

~~~
Strilanc
The paper "Algorithm Aversion: People Erroneously Avoid Algorithms After
Seeing Them Err" [1] has references to examples.

> _Dawes subsequently gathered a large body of evidence showing that human
> experts did not perform as well as simple linear models at clinical
> diagnosis, forecasting graduate students’ success, and other prediction
> tasks (Dawes, 1979; Dawes, Faust, & Meehl, 1989)._

Given that a majority of people (and doctors) fail even the simplest
probabilistic reasoning tests, this is not _particularly_ surprising. [2]

> _We know from several studies that physicians, college students (Eddy,
> 1982), and staff at Harvard Medical School (Casscells, Schoenberger, &
> Grayboys, 1978) all have equally great difficulties with this and similar
> medical disease problems. For instance, Eddy (1982) reported that 95 out of
> 100 physicians estimated the posterior probability p(cancer|positive) to be
> between 70% and 80%, rather than 7.8%_

1:
[http://opim.wharton.upenn.edu/risk/library/WPAF201410-Algort...](http://opim.wharton.upenn.edu/risk/library/WPAF201410-AlgorthimAversion-
Dietvorst-Simmons-Massey.pdf)

2:
[http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.128...](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.128.3201&rep=rep1&type=pdf)

------
dba7dba
A doctor (internist in primary care) friend told me once how he was glad he
wasn't a heart surgeon (of some specialty that I don't recall). He said that
surgeon in that specialty simply does one type of surgery all the time, as
ordered by some other doctors. As far as he could tell, that surgeon did not
have to do much mental work, but was rather like a technician.

I wonder when work of doctor (diagnosing a disease) will be handled by a
computer. I'd say in 100 years?

~~~
visarga
We can do diagnosis now. All that is needed is a medical nurse to take
symptoms and run them through Watson to get a diagnosis. The idea is that
people in poor/remote regions could do this whole process themselves, with the
help of a mobile phone and some medical sensors. There are already regions
where it would be more beneficial to run with the current medial AI rather
than leave things as they are.

------
nhswatson2
Wouldn't work here in the UK. Watson: "I see you have cancer x - there are
three effective drugs for that, but NICE has decided not to pay for them"
Patient: "Fucking NHS bean counting cunts" Watson: "Indeed. Now, I have
calculated it will cost the NHS a few pennies more between now and your death,
what with the doctor, hospital, and terminal care, not to mention the endless
drugs and things we will spend to slowly let you die." Patient: "I see, it's
like the NHS doesn't have a scoody doo what they are actually doing, how
things are spent." Watson: "Indeed".

