
IBM Watson correctly diagnoses a form of leukemia - adamnemecek
http://siliconangle.com/blog/2016/08/05/watson-correctly-diagnoses-woman-after-doctors-were-stumped/
======
rdtsc
It is funny that so many are quick to dismiss this because it is IBM, so "it
must be just PR" (saw a few comments here). If Google did this or Elon Musk,
everyone would be singing praises and musing about a better, brighter future.

Which is funny because it is Google / Facebook / Microsoft which collect and
hold everyone's personal details and data. For better or for worse, IBM never
seemed to have gotten into that business (yet?) of recording and selling every
American's data for ads.

In general, I think healthcare is more important than self-driving Teslas or
robots making us dinner and scratching our backs. So at least they seem to
have focused on the right thing.

~~~
amelius
True, but I think it is sad that IBM uses the Watson name for everything
related to AI, because this way they implicitly pretend to have found a
unified approach to AI. I don't believe this is true. The AI doing the
diagnosing is not the same AI as the one that played Jeopardy. But please
correct me if I'm wrong.

~~~
krona
As someone who's worked on/with teams building various Watson
services/solutions over the past 3 years, I can confirm Watson is almost
entirely a branding exercise without (from the engineering perspective) common
leadership, coherent inter-team objectives, etc.

There isn't a 'unified approach' to anything here beyond the use of various
open source machine learning libraries with huge amounts of medical data
either licensed or acquired over the past 2 years.

~~~
sangnoir
Who exactly thinks its a good idea to hijack an existing brand for
tangentially related, poorer quality products? Microsoft also does this with
Skype and 'Skype' for business. Instead of a win-win scenario they hope for,
they instead burn up the goodwill generated by the original product to bear
the name. So instead of thinking "Watson, the amazing AI that won Jeorpardy",
I now think "Watson, the horrible mishmash of IBM APIs"

~~~
ethbro
IBM's primary sales channel is enterprise, where branding makes _all_ the
difference. Most of the time, the person authorizing the project has no idea
on the technical details (and may or may not have solicited internal technical
commentary).

These deals are honestly mostly about trust. Do they trust IBM / {vendor} has
the expertise to deliver the project? Given that it's mostly bespoke
integration, branding the entire IBM AI/ML area as "Watson" isn't too
disingenuous.

------
apathy
I can't get past the adblocker spam, but I'm going to guess this woman had APL
(acute promyelocytic leukemia, caused almost exclusively by a very specific
chromosomal translocation) and the doctors didn't think to look for Auer rods
and thus did not try FISHing for the translocation. APL is deadly (5%
survival) if treated incorrectly but curable (95%) if treated correctly.
However, something or other about that differential has been a standard
question on the hematology medical boards for, what, 3 decades now?

John Welch, Rick Wilson, Tim Ley and colleagues showed how to do this several
years ago:
[http://jama.jamanetwork.com/article.aspx?articleid=897152](http://jama.jamanetwork.com/article.aspx?articleid=897152)

I'm curious whether this case was confounded by cytogenetic complexity, which
was part of the problem in that case. Ley later diagnosed Lukas Wartman, a
fellow in his lab (or, more like, they worked together to figure out what to
do), too:

[http://oncology.wustl.edu/people/faculty/Wartman/Wartman_Bio...](http://oncology.wustl.edu/people/faculty/Wartman/Wartman_Bio.html)

Lukas did not look nearly as healthy as he does in that picture when I saw him
last. He is a great scientist and what he had (adult acute lymphoblastic
leukemia, ALL) is a nasty malignancy to treat. Children with ALL tend to do
well, but their disease typically seems to arise from different underlying
causes than adults, who do poorly.

(Not-so-ninja edit: Here is Lukas' own writeup of his experience. He was in a
much tighter spot than I recalled:
[http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4850894/](http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4850894/)
Even children in second relapse are viewed as bad news. Adults in second
relapse are generally considered dead men walking. Anyways, read the paper if
you're curious.)

If Watson caught that the woman had (say) Ph+-like ALL and suggested the right
TKI, that will impress me quite a bit, because I've seen someone die from a
wrong guess on the latter. On the other hand, if it was APL, they fucked up
the differential somehow and are probably kicking themselves.

(Double not-so-ninja edit: It wasn't APL-vs-AML, or ALL subtyping. Another
poster kindly pointed out the primary source for this news, and something
doesn't add up. _shrug_ )

~~~
jacquesm
It would definitely be impressive, but it doesn't mean much without knowing
how many times it mis-diagnosed. IBM is very good at PR and Watson is milked
for every little bit so I'm always a bit suspicious about it in triumphant
announcements like these.

After all, winning the lottery is a lot less impressive if you've bought a few
million tickets and this article says absolutely nothing about any kind of
controls or whether or not this was a one-off performance or if it would work
at scale.

~~~
apathy
Well, to play devil's advocate, if all the tickets are free, and the attending
gets to ignore all the ones that don't win, that could be useful.

FWIW, here's Lukas' writeup of his experience.

[http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4850894/](http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4850894/)

~~~
smallnamespace
> if all the tickets are free

They're definitely not free; doctors have to evaluate what Watson says, and if
Watson is wrong a significant fraction of the time, then doctors will spend a
lot of time and energy barking up the wrong tree.

~~~
jfoutz
Eh, it's a tool. Some are useful, some less so. I imagine this would be a tier
two kind of thing. It's an unusual case, what suggestions does the robot have?
Maybe something obscure and easy to forget.

------
airplane
Somewhat interesting and related is that software that performs medical
diagnoses better than doctors has existed since the 1970s (Mycin, an expert
system). It seems like nowadays that this kind of solution can possibly be
provided at scale at a consumer level (can process new medical knowledge by
itself, be installed on commodity hardware), not an expert myself though so I
don't know if totally true. What exciting times though.

[https://en.wikipedia.org/wiki/Mycin](https://en.wikipedia.org/wiki/Mycin)

 _" it proposed an acceptable therapy in about 69% of cases, which was better
than the performance of infectious disease experts who were judged using the
same criteria."_

------
rgoebel
I see two important parts to this dialog now: 1) machine support for sifting
through mountains of data, and 2) the increasing jeopardy of large
corporations controlling data. For part 1, the dialog confirms what some of us
already know: "Watson" is a branding exercise, and there is no overall
coherent plan for strategic AI. But general open source components for
literature-based discovery are being built. Part 2 is scarier, and as noted by
two speakers from NSF last Friday (Peter Arzberger, Chaitan Baru), "data IS
infrastructure," and so the increasing difficulty for public data sources to
compete with Google, Yahoo, Facebook, Baidu, etc. is perhaps the more serious
challenge for AI support of humans in the future. University research faculty
can't tackle problems that require the de novo or nearly de novo creation of
such data infrastructure, so we await public policy of funding agencies to
clarify the construction and access to such. (Americans in this forum please
note that you need to stop writing and thinking as if there is no other part
of the planet ... I would be most grateful if Google brokered access to only
American data ;-)

------
YeGoblynQueenne
>> The technology is certainly there for the eventual creation of an AI
version of House

"The technology" has "certainly been there" since at least 1970, with software
like MYCIN:

[https://en.wikipedia.org/wiki/Mycin](https://en.wikipedia.org/wiki/Mycin)

 _MYCIN was an early expert system that used artificial intelligence to
identify bacteria causing severe infections, such as bacteremia and
meningitis, and to recommend antibiotics_

It and its ilk (expert systems, with hand-crafted rules) (it was the '70s)
where commonly shown to outperform experts:

 _MYCIN was never actually used in practice but research indicated that it
proposed an acceptable therapy in about 69% of cases, which was better than
the performance of infectious disease experts who were judged using the same
criteria._

Yet, we still don't have (flying cars) AI that can help doctors make better
diagnoses- and note I'm by no means advocating replacing the experts with AI.
That would open a whole other can of worms (what do you train your AI on when
there's no more experts, because you replaced them all with AI?).

But- just having the tech doesn't auto-solve your problems as if by magic. You
gotta beat dumb politics first.

~~~
peter303
Each generation of computer power enables new AI techniques. Expert systems
require explicit distillation of "rules". That took a lot of human work. And
they are considered brittle when they are outside their expertise. But expert
systems could on the million times weaker computers of the 1980s.

Watson employes statistical searching of large knowledge bases. You dont have
to explicity ferret out all the rules and relationships. Google Translate does
this too. There is no preprogrammed language dictionary.

The next frontier is deep learning which requires powerful computing to
operate in real time.

All these techniques have their limitations. No magic bullet yet.

------
epmaybe
I have a slightly unrelated question, but there may be medically trained
individuals in this thread.

I'm a medical student, and it seems like most people in the field are rather
cavalier when it comes to talking about the job outlook for physicians in most
any specialty. Do you all think that some healthcare jobs will not be as vital
in the next decade thanks to improvements in computing and AI?

~~~
chae
Also a medical student with an interest in AI. Healthcare jobs that rely on
visual recognition (dermatology, radiology, some pathology) are probably the
most likely to benefit in the short term (see Enlitic). Presumably a lot of
other jobs require advances in Natural Language Processing/Understanding, as
one of the big problems in health is the mostly unstructured nature of the
data.

It is also possible that many healthcare jobs are essentially AI-complete
problems - in this scenario, subjective opinion is not really a reliable
marker, but lots of AI specialists give around a 90% chance of human-level
machine intelligence by 2070 (there's a table in Nick Bostrom's
Superintelligence with the actual figures).

------
yread
Ironically, the original story [http://www.ndtv.com/health/artificial-
intelligence-used-to-d...](http://www.ndtv.com/health/artificial-intelligence-
used-to-detect-rare-leukemia-type-in-japan-1440789) was also "written" by AI

> (This story has not been edited by NDTV staff and is auto-generated from a
> syndicated feed.)

------
vonnik
One correct diagnosis is simply not that significant, except for the woman
concerned. The real news will be A/B testing on much larger data contrasting
Watson's diagnoses with those of doctors sans Watson, and checking both
against a ground-truth value, if that is possible. Newsworthy would be a
marked bump in accuracy across many cases with Watson...

------
oh_wow
IBM's Watson scares me primarily because the source and data is not in the
public domain. Something this powerful, and this rare should be given to
humanity to better it.

That said, I'm unsure how doing something like that would provide the right
incentives for the research needed to create the next breakthrough.

~~~
etangent
1\. It is most likely that Watson is far less powerful than advertised. Given
enough trials, a pigeon could probably detect some forms of cancer that a
human expert can't. [1]

2\. I too would prefer that more open-source work (and published research)
would come out of Watson project than it has been the case so far. However,
there's a devil's advocate point to be made that closedness encourages
diversity of approaches and implementations. When a high-profile project
becomes open-source (as recently happened with TensorFlow), it exerts a lot of
pull on time/attention of other developers and researchers that could have
been focused on trying out entirely different approaches.

[1] [http://www.scientificamerican.com/article/using-pigeons-
to-d...](http://www.scientificamerican.com/article/using-pigeons-to-diagnose-
cancer/)

~~~
bitmapbrother
Of course, just like monkeys picking stocks with darts can outperform the best
traders on Wall street.

~~~
adventured
That's not even remotely true.

The Quantum Fund by Soros and others, returned upwards of 30% per year for
over three decades.

Steve Cohen averaged near 30% annual returns for two decades.

Buffett's investing track record is similarly off the charts.

And lastly, a monkey couldn't do what John Paulson (or Burry and Eisman) did
with the 'greatest trade ever,' producing a radical outcome from an extremely
intricate concentrated investment (some of which required them goading the
opportunity into existence to begin with).

If you had said monkeys with darts can sometimes outperform the bottom half of
traders on Wall Street, you might have been close.

~~~
bitmapbrother
30% returns per year for over three decades? Impressive. I wonder how much of
that was attributed to insider information.

~~~
hiou
How is this down voted? Steve Cohen is practically synonymous with insider
trading in the modern era.

------
yuvalkarmi
Now _that_ is f-ing cool.

------
seesomesense
Without more information, it is difficult to know if this is just another
Theranos.

~~~
TillE
It's not difficult to see how a computer - especially with a huge database -
could do very well at differential diagnosis. It would require the correct
inputs, but its advantages are very clear even with fairly simple algorithms.

Theranos hand-waved several huge problems with their approach by claiming
proprietary magic. What they tried to do is probably impossible. So, big
difference.

------
dheerthan
Well, IBM Watson will be surprised to find this little girl, Brittany Wenger
built a " Global Neural Network Cloud Service for Breast Cancer " from her
bedroom computer around 2012.

[http://www.qreoo.com/v/dheerthan/316](http://www.qreoo.com/v/dheerthan/316)

