
Facebook shuts AI program after bots create own language - werber
http://m.economictimes.com/tech/internet/facebook-shuts-ai-program-after-bots-create-own-language/bots-gone-rogue/slideshow/59847579.cms
======
joshuahedlund
[http://www.bbc.com/news/technology-40790258](http://www.bbc.com/news/technology-40790258)

This article takes a less alarmist tone, including excerpts of the alleged
"language" which it describes as "simply modified human language"

> Bob: "I can can I I everything else" > Alice: "Balls have zero to me to me
> to me to me to me to me to me to me to"

It also claims "it was shut down because it was doing something the team
wasn't interested in studying - not because they thought they had stumbled on
an existential threat to mankind."

(Epistemological status: I've only read this article and the submitted article
on the topic.)

------
samfriedman
This story has really gotten legs somehow. The research is related to work
with AI agents developing a trade/negotiation logic with shorthand language
[0], but the media seems to have spun it into a tale of rogue computers being
shut down in the nick of time. There have been some articles attempting to
roll back the sensationalism [1] [2], but most of the headlines are still
working the "scary AI needs to be shut down" angle. The actual paper in [0] is
a very interesting read, and recommended if you want to get some idea of what
the kernel of this story actually is.

[0] [https://arxiv.org/abs/1706.05125](https://arxiv.org/abs/1706.05125)

[1] [http://gizmodo.com/no-facebook-did-not-panic-and-shut-
down-a...](http://gizmodo.com/no-facebook-did-not-panic-and-shut-down-an-ai-
program-1797414922)

[2]
[http://www.bbc.com/news/technology-40790258](http://www.bbc.com/news/technology-40790258)

------
sp332
The source is this Tech Times article
[http://www.techtimes.com/articles/212124/20170730/facebook-a...](http://www.techtimes.com/articles/212124/20170730/facebook-
ai-invents-language-that-humans-cant-understand-system-shut-down-before-it-
evolves-into-skynet.htm) which doesn't seem to have any checkable sources. The
research published back in June shows that Facebook got the AI to speak in
normal English sentences.
[https://code.facebook.com/posts/1686672014972296/deal-or-
no-...](https://code.facebook.com/posts/1686672014972296/deal-or-no-deal-
training-ai-bots-to-negotiate/) So maybe this happened since then? Anyway,
here's a more level-headed article from Fast Co which points out that this
kind of thing happens on a pretty regular basis.
[https://www.fastcodesign.com/90132632/ai-is-inventing-its-
ow...](https://www.fastcodesign.com/90132632/ai-is-inventing-its-own-perfect-
languages-should-we-let-it)

------
deepnotderp
For anyone interested, this is a sad running joke of a headline in the DL
community.

All that happened was that the models created an intermediate representation
(think a latent space like an encoder decoder model) as intended by the
researcher and then for whatever reason shut it down, presumably because it
failed to get good results.

Then some jackass reporter decided to get some clicks by making a click bait
title.

------
incan1275
Such bad journalism. Creating one's own language is not even trivial, and it
seems Facebook did it in a flash.

For more perspective on what the engineers intended:
[https://code.facebook.com/posts/1686672014972296/deal-or-
no-...](https://code.facebook.com/posts/1686672014972296/deal-or-no-deal-
training-ai-bots-to-negotiate/)

------
castis
Of the hundreds of places this article was put online today, this website is
probably the worst presented form I've seen.

------
scardine
Lets not forget how much Facebook boosted PHP's reputation by sending patches
at all levels of the stack and developing excellent patterns of discipline,
code conventions and style that manage to avoid the worst attributes of the
language.

------
tshirtcollector
This whole situation is almost completely falsified and fabricated. It truly
is a great example of how misinformation can spread when a lack of
journalistic integrity is coupled with irrational fear and bias. There are
clear risks to using A.I. at the moment, but they all stem from pre-existing
prejudice in training data. To really show how misleading this story is,
here's the reaction of a respected and brilliant AI researcher:
[https://twitter.com/egrefen/status/891362804074033152](https://twitter.com/egrefen/status/891362804074033152)

------
6d6b73
Most likely it was a bug, and since they used DNNs they could not correctly
explain why it's happening. Since they could not fix it, they had to shut down
the service.

AI is not taking over - nothing to see, please move along.

------
triptych
That's how the Emoji Movie was created.

------
linsomniac
Fear what you do not understand. Which in the case of the singularity, is
probably good advice, come to think of it.

