
IBM halting sales of Watson AI tool for drug discovery - hprotagonist
https://www.statnews.com/2019/04/18/ibm-halting-sales-of-watson-for-drug-discovery/
======
imroot
I have a friend who is an OB/GYN Oncologist in Indianapolis. Her hospital had
IBM Watson Health on campus last year, and she mentioned to me in passing last
month that they finally had kicked IBM off of their (learning) university's
campus. When I asked her why, she said: "Often times, Watson would recommend
courses of treatment that would be completely incorrect, if not detrimental --
or even sometimes lethal -- to the patient. It became more of a hassle than a
learning tool."

~~~
tabtab
The purpose of such a system should be to give leads for further research. If
you treat them as leads and only leads, then bad leads don't matter much.

Unless, the leads are so bad on average that too much effort is spent on
vetting them that could be put to better use elsewhere.

~~~
arkitaip
I doubt that's how Watson is being marketed to customers, though. IBM tends to
oversell and under deliver and have grown increasingly desperate with their
shrinking market shares in both hardware and services.

~~~
tabtab
You can only live on spin for so long.

~~~
randcraw
Unless you work as a consultant. Or in Washington.

------
distant_hat
Slightly over a year back, we were contacted by IBM to try out Watson under
some program for startups. Starting out, they asked us to give them a try. We
gave them a NLP related task and they were confident they would be able to do
a good job of it. After multiple meetings over the next 2 months, they barely
produced any useful output. In the meantime, an intern with us got a pretty
decent solution using just Python. Our management did get invited to lunches
and meetings for selling them some more products. During one of such
invitations, since our CEO couldn't make it, I was sent. I was surprised by
how shoddy their presentation was given that I thought they might not be doing
a good job on the problem we gave them but they should be slick on the selling
side. Their presentation seemed like it was out of 2005 or something.

~~~
m463
You know, it would be very profitable for someone to profess an expertise in
X, reach out to various companies to help them use X to solve their problems,
then simultaneously gain experience in X while patenting the heck out of any
problems solved with X.

~~~
erikpukinskis
You’re describing a person who has a huge amount of expertise in sales.

But a person like that can very lucratively sell actual products that already
exist, so why would they want to sell this mysterious X?

~~~
throwawayjaksdk
Because the person is actually a business that:

a) employs enough scientists who know what they're doing and lawyers who know
what they're doing to file relevant and defensible patents; and

b) has enough entrenched relationships that they can pay for a sales force
what knows what they're doing; but

c) doesn't employ enough engineers who know what they're doing to develop
actual competitive products around those patentable ideas.

------
wpasc
I sincerely hope IBM's failures do not deter further progress in trying to
innovate in healthcare technology. I believe healthcare vitally needs
GOOD/USEFUL technology and AI applications because medical progress is not
advancing quickly enough by several metrics.

I hope IBM continues to fail and becomes a case study in the failure of
massive bureaucracy.

~~~
PedroBatista
IBM will not fail, Oracle will fail first than IBM and that's unlikely to
happen.

You see, they are NOT tech companies. They have something you would call
"tech", they sell something that looks like "tech" but they really solve
problems people with access to large amounts of money have.

In other words, they produce cushy and warm beds for C-level execs to have a
good night of sleep, and everyone needs some bed to sleep every night..

Technology is a risky and volatile business and IBM has more than 100 years in
business.

~~~
pm90
Spot on.

IBM really does capitalize on the adage of nobody being fired for buying IBM.
Their biggest strength has been in their marketing rather than actual
technology.

Note that, being a large org, they do have teams doing excellent work too. IBM
research, some of the cloud, design etc. teams seem to be doing good stuff.
But most of it is still overpromising and underdelivering while charging
clients a lot of cash.

~~~
exelius
This is less true than it used to be. Oracle/SAP are largely recognized as
multi-billion dollar technology quagmires these days; integration has become
so much easier that cobbling together a bunch of SaaS solutions to replace a
centralized ERP is a viable approach.

Likewise, IBM has become known as a vaporware vendor. They combine the worst
parts of management consulting with an offshore / outsourced development
process, so I guess it’s not a surprise they ended up like all the other
companies that sell business tech solutions (Infosys, Wipro, etc). They’re
often mentioned in the same breath these days.

~~~
scrooched_moose
IBM has been banned from our data center for several years. The decision was
before my time, but the answer I've been told is overly expensive, unreliable,
and poor support when compared to other vendors.

~~~
i_am_nomad
In my experience, IBM’s support is excellent, bordering on fanatical. It’s
just that their hardware (in this case, Spectrum Scale and Power9) are
preposterously overpriced. Same for their software, if not more so.

~~~
pickle-wizard
As a former IBMer I really like the POWER hardware. It is just a shame that it
is made by IBM. Like you said IBM stuff is just too damn expensive.

~~~
i_am_nomad
Right, I should have pointed that out: the Power hardware is truly incredible,
especially the fast I/O to GPUs. There’s a reason why the national labs and
Google have gone with that architectue.

------
batmansmk
Watson is more a marketing term than a technical connection between products.

I invite you to test "Watson Tone Analyzer" [https://tone-analyzer-
demo.ng.bluemix.net/](https://tone-analyzer-demo.ng.bluemix.net/) :

\- "I like this product." => "this is an analytical opinion with neutral
emotion."

\- "I like it" => "Tentative, 50% happy answer."

\- "It's not a bad product." => "analytical".

\- "It's not a bad tool" => "joyful answer".

~~~
darkpuma
\- "Spiders are not insects." => "fear"

Exactly what I expected, and also flat wrong.

\- "There's a monster under my bed." => "joy"

Fascinating.

~~~
chillacy
You'd think they'd pair it up with a naive bayes classifier at least.

------
foobiekr
Slowly, I am noticing that people are starting to question the abilities of
machine learning systems and, in particular, seeing a lot of the transparent
AI-washing that is now going on for systems that aren't AI at all. This is one
of the most positive changes in the tech industry, honestly.

~~~
Barrin92
Machine learning and neural net systems seem to be the inverse of the "uncanny
valley", call it the "miraculous hill" if you will.

They're just good enough to, with insane amount of computing resources, churn
out economically viable results, but in my opinion they're the biggest setback
in actually developing intelligence or cognition in a long time.

There's very little insight to be gained from what they pick up, and they
function purely in a stochastic sense. Even the best ML algorithm has no
ability to reason at a high level or produce counterfactuals, it works purely
by correlation and still, in those 1% edge cases, will be as helpless as
anything else.

That might be good enough when selling advertisements, but in automated cars
and healthcare treatment, this sort of failure is not an option.

~~~
hodgesrm
ML and neural nets work surprisingly well on a lot of boring but important use
cases. Optimizing online ad placement might be humdrum but it's the foundation
of a $100B market in the US alone. [0]

In my opinion the real hype is that ML has become so popular that new
practitioners tend to forget other analytic techniques like SQL data
warehouses. Interestingly these are starting to absorb ML capabilities like
logistic regression, which are now accessible through SQL and can benefit from
MPP and vectorwise query execute in DBMS types like ClickHouse, Vertica, and
Google BigQuery.

[1] [https://www.forbes.com/sites/danafeldman/2018/03/28/u-s-
tv-a...](https://www.forbes.com/sites/danafeldman/2018/03/28/u-s-tv-ad-spend-
drops-as-digital-ad-spend-climbs-to-107b-in-2018/#31c701a77aa6)

Disclaimer: I work on ClickHouse.

------
haolez
Slightly unrelated, but I hope IBM doesn't ruin Red Hat and CoreOS. I would
love to see the CoreOS tools to gain more adoption.

~~~
jniedrauer
I'm a long time Fedora user. I love the sane defaults. It's just a solid
workhorse. But I've been feeling pretty uneasy about using it lately...

~~~
jsight
Alternatively, a lot more IBMers will be able to use Fedora and it may improve
even more. I don't see them having a strong reason to do any damange there.

~~~
aquaticsunset
Maybe? IBM is mainly a Windows and Mac house for dev machines, with Ubuntu
being a very, very small portion of workstations.

I’m not sure the acquisition will create a mass exodus from the status quo
there.

------
opsiprogram
I work with a group using machine learning for drug discovery (I'm not a
biologist/chemist) but the bio people around me loved to talk poorly about
Watson's drug discovery tool.

Lots of focus on the algorithms in the comments here, but from what I could
glean generally they lacked domain experts when developing the datasets... we
spend 90% just finding the best data... and even then it's tricky. I think
they may have had lower standards for the input into the system... garbage in
garbage out.

------
tolstoy77
a lot of these AI companies products are really terrible. Has anyone ever
tried the AI API models from clarifai ? Just so unaccurate. It seems like a
scam. I've also had a really bad experience with watson's speech to text apis.

~~~
acdc4life
Deep learning and machine learning don’t work. Quantitative math will always
prevail, as it always has. Unfortunately, mathematical research isn’t there
yet. We don’t have models for vision, audition and linguistics. Neuroscience
and psychology are in their infancy, a good analogy would compare these fields
to where physics was pre-Newton, Galileo era of understanding. I suspect that
in the decades to come, these fields will influence mathematics the same way
physics influenced calculus. Physics historically had a huge influence on
math, in the coming century it will be neuroscience and psychology, in linking
brains to behavior, and the quantitative laws that allow brains to give rise
to minds.

~~~
bduerst
>Deep learning and machine learning don’t work. Quantitative math will always
prevail,

I have a neural net onboard my phone which automatically detects songs offline
and tells me what they are. Is that semantically 'quantitative math' and not
machine learning?

~~~
acdc4life
Quantitative math, or applied math isn't based on fitting data to an arbitrary
mathematical structure. It's looking at real life, and deriving the
mathematical laws that govern what you see. You could have a neural net
predict planetary motion. However, it doesn't know jack shit about physics.

>I have a neural net onboard my phone which automatically detects songs
offline and tells me what they are.

MP3 uses something called psycho acoustics, which is a quantitative model on
human perception, which is used to eliminate frequencies that can't be heard
based on this model.

Your neural network doesn't tell you what features make songs distinct, it's
not a quantitative model at all, but a black box heuristic on what the
important features are superficially. If actual mathematicians worked on this
problem, I guarantee you they'd do a better job, and their models would work
on a commadore64, with real time training. Moreover it would tell you things
like who is singing, if it's a live performance, which concert it was.

~~~
sonnyblarney
" If actual mathematicians worked on this problem, I guarantee you they'd do a
better job"

No, this is wrong.

Some of the most brilliant people in the world have been working on image
recognition, voice recognition etc. and AI is crushing all of their work.

"Your neural network doesn't tell you what features make songs distinct, it's
not a quantitative model at all" \- it doesn't matter at all if our objective
is detecting the song. Neither does the mp3 compression algorithm.

~~~
acdc4life
>Some of the most brilliant people in the world have been working on image
recognition, voice recognition etc. and AI is crushing all of their work.

This is very true. I take my stronger statements back, MAINSTREAM
mathematicians attempting this problem are all wrong, and have been wrong for
50 years. But you do need the right theory, and the right math that realizes
this theory.

"AI" is superficially beating the work in computer vision. Computer vision is
complete bogus. The gabor filters, fourier transfroms etc. are all wrong
conceptually. The known methods do abysmally on basic tasks like object
recognition, texture segmentation etc. But they keep trying it.

I would take this one step further: computer vision, audio and NLP researchers
have been stuck in a rut for the past 50 years. DL is beating THEIR math, but
this is because of data and computation speed, not because of any insights.
But DL is also wrong, and giving you an illusion of progress. Both of these
things are doomed to go the way of GOFAI.

I can go into great detail and carefully explain why MAINSTREAM contemporary
ideas in math for vision, audition and language are completely wrong, and have
been wrong for 50 years. What is the right model? Like I mentioned before, the
right ideas are emerging, neural networks will dominate, just not DL.

~~~
wetmore
Ok so who are the real, non-mainstream mathematicians who would do better?

------
throw2016
It seems just like crypto the 'ai' hype cycle is coming to an end with zero
achievements to show for itself beyond puffery and premature speculation about
radiologists, drivers and others losing their jobs.

And its not just hype, like crypto the ai hype stepped well beyond the line to
outright fraud and deception with tech folks trying to pass off backward
looking pattern matching as 'intelligence' and hope no one notices. Every
single commentator here knows there is nothing in computing or software
engineering today that will allow one to 'create' an 'ai' as the world
understands the term yet no one questioned pushing intentionally deceptive
communication.

This end result of dystopian scaremongering amounting to nothing is there is
now zero credibility and extreme suspicion of problematic in-built bias. At
the minimum there must be some standards for machine learning solutions to be
thoroughly transparent, open to verification and exhaustively tested for
racist and sexist bias before any rollout, for anyone who cares about the
impact of their work in the real world.

~~~
acdc4life
There is no "ai", it's the same bull crap we had in the 80s. We haven't had
true innovation in this space for a very long time, the current hype was
caused by GPU, cloud, and abundance of data for training.

------
ct520
How is Watson not the butt of jokes in the tech industry by now? Seriously,
never heard anything that didn’t end up being smoke and mirrors.

------
sambroner
I can’t read most of that article because it’s behind a paywall, but wow, is
Watson just the largest attempt to sell vaporware of all time?? They had a
super bowl ad!

A few friends from university were hired into the Watson team as hardly
technical PMs. I have never heard any of them describe what Watson is or does.

~~~
donkeyd
I've commented this before, but Watson is nothing but branding for anything
IBM does that is somewhat intelligent. There's pretty much no common
technology between the different Watson products, so there's no 'core Watson'
like many people believe.

Some products with Watson branding are great and industry leading. Others
suck. The latter tend to appear in headlines and severely impact the brand
they established with Jeopardy and their ads.

~~~
apohn
I suspect what happened was that Watson started out as a single product or
focused suite of products. I'm just guessing, but then as the marketing
induced hype started and growth/results were not on target , they realized
there was a lot of interest in Watson so IBM pivoted and started rolling
everything under the Watson brand - products, consulting, cloud stuff, etc.
Basically that way they could say Watson was a "success" and
internally/externally it would be opaque as to where exactly the profits and
losses were.

I say all this because I'm at a place now where this is happening. Hugh
company with a supposedly game changing product that is mediocre (at best).
Massive marketing campaign for "Product X" that is 100% buzzwords and 200% BS.
After constant missed revenue targets and product disappointments internally
and externally, company is clearly pivoting (but not saying so) to rolling
everything under "Brand X."

~~~
nintendo95
I worked there (Watson Health) a few years back. All that happened is IBM.
That's right. They bought us (a small start-up) and this is what they did in a
year to us: 1\. replaced managers with their own who didn't get anything, kind
of old date executives taken away from mainframe 2\. banned remote work (this
costed them a few brilliant engineers) 3\. opened one huge open space full of
noise and chatter, I mean sales team next to the development team, etc (this
cost them even more headcount among experienced developers) 4\. For months we
did _nothing_! There were whole teams doing nothing for months on an end. I
wrote zero lines of code in over 6 month period. Why? Because management
didn't know which direction should be taken... and they kept hiring too! They
kept hiring new developers when the ones already at place had absolutely
nothing to do!

This is from my (simple developer) perspective. Not sure how it looked in
sales, among executives, etc. But that was very weird environment.

I had great Manager who asked me to learn react and take courses in react
(mind you it was two years back!) As we "might want to do something in react
in the future". So I basically spent my last six months there learning React,
aka preparing to the job interviews... they even got us paid courses and all.
I mean... IBM.

And once they fired me (these were lay-offs, thousands affected, many of them
just hired in past year, like myself) I was paid severance pay too. I went
there worked a year, last 6 months was learning for job interviews...
fantastic pay too. IBM is crazy.

~~~
justwalt
What did you end up doing in a period of non-work work like that? Read books?

~~~
bastijn
Advise you to read the comment again :).

Spoiler: (s)he learned React.

~~~
justwalt
Not sure if I stopped reading early or just didn't comprehend what they had
written. Thanks, lol.

------
cromwellian
Basically, Triumph of Sales over Engineering.

------
bawana
And now Google is pushing DeepMind Healthcare in England?! Do they not learn
from these examples? Anyone who uses EMR knows very well the beauty of copy-
paste and this single little convenience is creating more disinformation than
the Mt Everest piles of illegible scrawl that used to be medical records. At
least in those days, a HUMAN had to write something. And how did everybody
forget this little gem from the 1960s: 'GIGO' = garbage in , garbage out. I
dont care how many layers your RNN has, it will NEVER get accurate data. Sick
people are not reliable sources of info and it takes a HUMAN to strip off all
the emotional distortion to get the facts.

------
StreamBright
No surprise here. IBM has stopped producing usable engineering solutions long
time ago.

------
notyourday
Watson's applications outside of a quick search of existing datasets are
vaporware. I'm amazed not a single customer has gone public with it when so
many of them are whispering about it on LinkedIn.

------
wessorh
Even Watson wasn't a general AI. Hearing all the anecdotes from folks that
tried to use it.. Narrow AI has the attribute that it can't be ported from one
field to another. It might win at Jepordy but sucks at Oncology and Drug
Discovery. All AI isn't the same. We are general AI and we can solve problems
in asymmetrically bad situations. Narrow AI is not portable

------
sonnyblarney
For reference, here is the IBM Watson VP hustling it [1] on Charlie Rose.
Funny in retrospect. Especially that haircut ...

[1]
[https://charlierose.com/videos/29530](https://charlierose.com/videos/29530)

------
duxup
There have been a couple articles that a lot of issues with Watson was how it
was sold. Watson was being sold as sort of a magic drop in solution that could
do oh so many things.

In reality Watson took a great deal of work to carefully structure and process
data, evaluation of the output, and more thoughtful of approaches to further
refine the information and evaluate outputs. It also required a fair amount of
involvement with the individual customer's staff. And there was always the
possibility that because each use case was different... it simply wouldn't
work out.

To some extent it seems like it should have been sold as a journey... not
cookie cuter solution for things that Watson had never encountered before.

How do you sell that, I don't know, if I knew I'd probabbly be a pretty good
salesman.

------
DebtDeflation
Good. Focus on the use cases that actually work, namely a chatbot for handling
customer service questions and an enterprise search engine.

~~~
aquaticsunset
Can’t tell if sarcasm, but IBM’s had massive headache and trouble in this
segment too. Good customer service agents are expensive. Bad ones are cheaper.
But both still have a far higher success rate than conversational bots do. And
a hybrid approach is, again, harder than one would initially assume (you
know... why it’s painful and infuriating to press fifteen numbers on the phone
to get to a real person? Same concept)

------
bilater
'And the trough is coming...' \- Gartner

