
AI Failures in 2017 - dark_archer
https://syncedreview.com/2017/12/23/2017-in-review-10-ai-failures/
======
chillee
Some of these are pretty eh.

"Facebook chatbot shut down" doesn't really show anything. It was a everyday
failure of training that got overblown in the media.

"Google AI looks at rifles and sees helicopters": This wasn't new at all.
Black box attacks have been around for a while. The paper purported to show a
much more efficient attack, but it's not really a huge advancement.

In general, most of these aren't really what I would call "failures" of AI.
Some of them are illuminating parts of ML that are problems (copying society's
bias, adversarial attacks), but most of the rest are simply things being
overhyped/regular design errors (anything to do with Alexa/google home).

~~~
innagadadavida
Nonsense. These types of problems have been “illuminating” researchers for the
last 5 decades. These are real shortcomings and will not be solved for another
5 decades or more. It’s unbeliebable how much in denial the AI folks are in.

~~~
chillee
Could you elaborate on what you consider these "problems" are? The way I would
break down these "problems" are:

Overhyped marketing (Face ID, Las Vegas self-driving bus, HSBC voice id, "AI
imagines a Bank Butt sunset")

Questionable design choices (Amazon Echo, Alexa)

Adversarial attacks ("Google AI looks at rifles", street sign hack)

Bias (Allo turban emoji)

wtf is this (Facebook chatbot)

ML does have real shortcomings, but the problems in this list does not
exemplify them well.

~~~
bsaul
Adversarial attacks are a bit more than just that : they reveal how much we
don’t have any intuitive knowledge on what the algorithm understands. In the
sense that we cannot build any intuition on what the 1% mistakes cognitive ML-
based algorithms will be triggered by.

Aka : If a gun could be mistaken for an helicopter just by altering a few
invisible pixels, then the behavior of that system is by definition not
similar at all to « vision » in its common sense. And so if it’s not, then
what is it ?

~~~
hanbura
They changed a lot of pixels in a way that led a human to classify the image
as not notably changed but led one AI to classify it as a different type of
object. Of course the same thing is possible with human and AI reversed, there
are images that are obvious to some AI but that would be misclassified by
humans.

Humans do have the advantage that our vision system can draw on a huge
repository of real-world knowledge and has been fed with decades of high
resolution training data.

~~~
bsaul
Humans also have the advantage to understand what world they’re living in, and
be capable of reasoning. The way you describe human cognition using ML terms
makes me think you’re confusing the map with the territory.

------
tehramz
When I read anything about “AI” that’s not just hype or some “futurist”
predicting that we’re on the cusp of something huge and it’s just around the
corner, it makes me think we still have a long way to go before it’s actually
“intelligent” and nothing more than PR/marketing.

~~~
chillee
We do have a long way to go, but AI has made huge strides over the past few
years, irrespective of the hype/marketing. Notably this year, AlphaZero,
advances in generating images
([https://github.com/tkarras/progressive_growing_of_gans](https://github.com/tkarras/progressive_growing_of_gans)),
voice synthesis with Tacotron, etc. were all very impressive results. Not to
mention all the recent advances in the theory of deep learning.

It's good to be on watch for the bullshit (the news stories about Facebook's
chatbots' "language", most futurist's predictions, any hype about general
artificial intelligence, replacing x% of jobs in y years), but don't throw the
baby out with the bathwater.

IMO, unlike blockchain, AI/ML is undoubtedly here to stay, and it will have a
huge effect on everything.

~~~
simsla
IMO even the blockchain is here to stay. But more in the toolbox of useful
datastructures than in the realm of "hip company ideas".

~~~
lagadu
I think blockchains are here to stay, and that's fine; they're extremely
interesting. I feel that at the moment they suffer from the problem that a few
technologies suffered: they're a solution looking for problems to solve. By
that I mean that currently there's a trend towards applying blockchains to
solve every problem regardless of a solution for it not requiring blockchain
at all. Do you want to make a transfer? Blockchain. Do you want to manage
inventory? Blockchain. Do you want to help starving children in Africa?
Blockchain. Do you want to transfer files? Blockchain. All of those are
problems and they're problems that don't need a blockchain to be solved and a
blockchain is arguably not the best way of handing them but because
blockchains can be used as part of a solution, because it's a trendy
technology, there's a huge effort towards applying it to solve them.

I don't think it's a bad thing to try and apply new solutions to old problems
but it just feels that we're trying to apply it too much to problems that are
otherwise already solved.

------
Houshalter
The most interesting thing about this is that it's not interesting. None of
these cases are very bad. The self driving car accident is the most severe, it
was a minor accident and the car wasn't even at fault.

~~~
tedeh
I mostly agree, however regarding the bus accident the problem is that AI out
of the box has to be way better than the human in order to gain widespread
acceptance. That means it needs to be able to make tough calls, and do it
better than the humans it is meant to replace (or complement). The description
of the incident in particular shows why self-driving AIs have such a long way
to go.

~~~
qohen
This may not be so. I expect many people would see the good in technology that
prevents accidents and deaths even if it isn't perfect in its first release
and that those who don't feel this way at the start might be brought around by
good arguments.

Speaking of which, there's a study by some RAND Corporation researchers
(described on RAND's blog [0] and here [1]) about how it is likely a good idea
to get the technology out even before it is perfected, not only to save lives
now, etc. but also to speed up the perfection of the technology -- the rubber
needs to hit the road, so to speak.

[0] [https://www.rand.org/blog/articles/2017/11/why-waiting-
for-p...](https://www.rand.org/blog/articles/2017/11/why-waiting-for-perfect-
autonomous-vehicles-may-cost-lives.html)

[1]
[https://www.rand.org/pubs/research_reports/RR2150.html](https://www.rand.org/pubs/research_reports/RR2150.html)

------
timhh
> Bkav simply scanned a test subject’s face, used a 3D printer to generate a
> face model, and affixed paper-cut eyes and mouth and a silicone nose.

Ah yes, simple as that!

> a BBC reporter’s twin brother was able to access his account by mimicking
> his voice. The experiment took seven tries.

That's actually extremely impressive!

------
sytelus
You can make similar lists about "Airline Industry Failures in 2017" or
"Software Failures in 2017" and so on. No one is stopped flying or using
software.

~~~
dontreact
It appears there were no crashes in the commercial Airline industry in the US
in 2017. There was 1 near miss.

[https://en.wikipedia.org/wiki/Category:Aviation_accidents_an...](https://en.wikipedia.org/wiki/Category:Aviation_accidents_and_incidents_in_the_United_States_in_2017)

AI and software engineering probably have a long, long way to go before they
are in the same category of dependability.

------
tambre
> passengers on the smart bus complained that it was not intelligent enough to
> move out of harm’s way as the truck slowly approached

Didn't someone point out in the thread for that, that backing up would've
technically been breaking the law?

~~~
tabs_masterrace
One thing you have to realize about the law, it doesn't translate to a rule
set, as understood by computers, at all. If that were the case we could just
implement the law in code, there would be no need for judges or lawyers, and
every case should be clear. But the law isn't just a bunch of logical
statements, that you can run any given situation against, instead it
constantly leads to edge cases or even contradictions. People practicing the
law are interpreters, that often have to argue about how a real world
situation even fits into this rule set.

It's weird if you think about it, but from a logical standpoint, whether or
not you broke the law is often undefined.

~~~
chrisseaton
Yes people in tech often have his misconception about the law, and they think
they can find clever loopholes or gotchas where a judge would really say
‘that’s clearly not a reasonable interpretation’.

------
ungzd
Meaningless, entertainment listicle.

------
wepple
was FaceID a failure? It was engineered to the same allowances as touchID: a
determined, relatively well resources adversary can bypass it.

~~~
folksinger
Do you think you could fool a human intelligence with some paper printouts
taped to your face?

~~~
rand0mbits
Bouncers are fooled by fake IDs all the time

~~~
folksinger
Bouncers are fooled by people wearing paper masks of the people depicted on a
photo ID?

When bouncers let people enter a bar with a photo ID that does not match the
person in question they are not failing to identify a human. They are failing
to give enough of a shit to carefully examine the picture in question.

------
LeoPanthera
The Alexa one is confusing. Alexa asks for a PIN when you buy something. Maybe
the little girl knew the PIN, but the newsreader's voice would not have
accidentally triggered any actual purchases.

~~~
mrtksn
How do you type the pin? Just say it? What if there are people in the room?
Tell them to get out?

~~~
LeoPanthera
You just say it. I guess the theory is that you trust anyone you invite into
your house.

------
ramgorur
Well I won't say the HSBC voice ID one was a failure, identical looking guy
with similar voice can fool any human being.

~~~
pimmen
Sure, but a trained human would probably pick up that something is not quite
right, especially if family situation has been revealed earlier.

My neighbor has an identical twin and one day his twin came visiting, but I
had no idea he was coming. He was standing in front of our intercom that’s
been broken for years looking bewildered when I got home from work, and since
I’d heard about my neighbor’s identical twin before it didn’t take me long to
piece together what was going on, even though I’d never met this guy before
and I had no idea he was going to be there.

~~~
ramgorur
"something is not quite right" is called intuition, machines can't have
intuition, once they have they won't be machines anymore.

~~~
fooker
What do you think a trained neural network classifying seemingly random noise
as an image of a panda is?

~~~
xkcd-sucks
Schizophrenia?

------
Agathos
Those paint color names are an AI success.

~~~
cardamomo
They certainly made me laugh out loud! It reminds me of the kinds of
unexpected language one finds in experimental poetry.

------
DrNuke
Democratisation and solving some local problems are the convincing bread and
butter of the most recent progress in the AI field. Generalization efforts are
nowhere near the hype and the mediatic bs yet but AI is dangerous enough
already if used by malicious parties. All in all, it is a case of mediatic
smoke because there is some fire already.

------
jsemrau
At least they did not include @Dropinin. Phew. I liked it's replies though
until it got shut down by Twitter.

[https://imgur.com/mCeAV1b](https://imgur.com/mCeAV1b)

------
juanmirocks
We as humans can try to make fun of robots and AI as much as we want. However,
the list of AI failures will shrink year after year.

~~~
codeman1181
I would guess there will be a lot more failures as AI is deployed more widely.
But there will also be way more successes.

~~~
collyw
More AI = more successes and more failures. Isn't that stating the obvious?

------
visarga
Haha, the last one is so stupid I think the newspaper should be better off
closed. Clearly non-AI people writing about what they don't understand.

------
mauritzio
Maybe the biggest failure is to call it 'intelligence'. It seams just a ball
trained to fall in a certain sink. Maybe call it artificial selection?
Experiences with 'selection' in human history didn't had the nicest outcome.
Curious what the artifical one will produce :o

~~~
quotemstr
"Selection" is responsible for all good outcomes, including the entire course
of natural history that led to your comment. Natural selection got us
mitochondria, eyes, legs, thumbs. Sexual selection gave us a brain capable of
reasoning about itself. Machine learning is just the next step in that
progression. It's amazing that the culmination of 3.7 billion years of
selection is a mind that can come up with elaborate ways to cast aspersions on
the selection mechanism that spawned it.

You know why I love machine learning? It's immune to anti-intellectual fads.
It speaks the same truth that children do. While you can _make_ an algorithm
spit out the answers you want, you have to explicitly train it to do so,
making the selective blindness and hypocrisy we demand crystal clear.

~~~
selestify
Machine learning is not immune to biased data collection. Poor data in, poor
models out.

I am really curious though, how did sexual selection help us evolve our
brains?

~~~
mythrwy
Smarter males were more able to get mates (perhaps through control of
resources among other things)?

Smarter mates were, or were able to make themselves appear, more attractive?

To me it's fairly evident that sexual selection played a large role in brain
development.

~~~
anonytrary
You are probably not wrong.

~~~
ralfd
He is likely right, because genes for intelligence are very often on the
X-chromosome:

[https://www.psychologytoday.com/articles/201109/the-
incredib...](https://www.psychologytoday.com/articles/201109/the-incredible-
expanding-adventures-the-x-chromosome)

------
c0deR3D
Formal Verification is truly needed especially on this.

~~~
aoeusnth1
It seems difficult to combine formal verification, which is rooted in symbol
manipulation, with deep learning, which is based on high dimensional stacked
distributed representations.

~~~
chillee
While his reply seems like a hammer in search of a nail, his general point has
merit, and is something that has been a topic of contention recently in deep
learning (with Ali Rahimi's NIPS talk).

What he calls formal verification is what we would call regular math in
machine learning. Deep learning is sorely lacking hard bounds for all sorts of
things (generalization, etc.). It's something that's gotten substantially
better over the past year, but is something that needs a lot of work.

~~~
aoeusnth1
Generalization bounds aren't even hard, though. They're in the form of
"probably approximately correct", - there's no way to guarantee that any
particular example won't be misclassified.

~~~
chillee
I think I'd still consider those bounds in the form x% chance that the result
is within epsilon of optimum as hard. Otherwise, you'd have to discount the
entire field of randomized algorithms as not having hard proofs.

~~~
aoeusnth1
With randomized algorithms you can get within an arbitrarily small epsilon
chance of getting a wrong answer, and it’s not unusual to get on the order of
machine error in practice.

PAC guarantees are very different. They generally rely on:

\- Data proportional to 1/eps^2

\- Only reduce the “variance” component of the “bias + variance” (you’ll never
fit a linear model perfectly to a nonlinear dataset, regardless of PAC)

\- Get worse as you decrease bias

\- Assume your data is IID and identically distributed to the test data. In
TFA, several of the examples of bad behavior come from data which is
distributed differently from training data (adversarial examples or automated
cars not moving out of the way)

~~~
chillee
I don't think we're really disagreeing here. Afaik, hard bounds aren't a well
defined mathematical concept, and it's fine to have different ones.

What I initially meant by "hard bounds" was any kind of mathematical proof
more rigorous than "well, dropout kinda makes your neural network not rely on
one feature so that's why it generalizes".

As for your points, I don't think they're really criticisms of PAC bounds.

I'm not familiar with the first point, but it'd be surprising to me if most
PAC bounds had that, considering PAC is a framework and not a specific
technique...

Your second point is irrelevant to generalization. You're looking for theory
about capacity I think? I think learnability also comes into play.

3rd: that is indeed what the bias variance trade-off would imply. It's also
why most classic PAC bounds are vacuous for neural networks.

4th: I think that's a fair assumption to make for any meaningful study of
generalization.

