
I’ve Seen the Greatest A.I. Minds of My Generation Destroyed by Twitter - jonathansizz
http://www.newyorker.com/tech/elements/ive-seen-the-greatest-a-i-minds-of-my-generation-destroyed-by-twitter?intcid=mod-most-popular
======
rjbwork
Just for anyone who is not aware - the title is an allusion to Ginsberg's
Howl, though he used the word "best".

Another tech related article that Howl allusion that always sticks in my mind
is [http://www.fastcompany.com/3008436/takeaway/why-data-god-
jef...](http://www.fastcompany.com/3008436/takeaway/why-data-god-jeffrey-
hammerbacher-left-facebook-found-cloudera)

If you've not read or heard it, I highly recommend giving a narration by
Ginsberg a listen.

~~~
Alex3917
See also Howl 2.0:

[http://www.howl2.com/](http://www.howl2.com/)

~~~
mgr86
I always wanted to see "I am a victim of the Telephone" updated.

------
chollida1
I'm by no means among the "Greatest A.I. Minds of my generation" but count my
among those who tried to use twitter as an input to an "A.I." system and had
to finally admit I couldn't tame it.

In my case it was an automated trading system where twitter was one of about
50 different inputs that drove a hidden markov model that spit out
buy/sell/hold signals.

I couldn't figure out how to "clean" the twitter stream in real time, either
fast enough, or thoroughly enough to make the inputs usable.

Even when I scaled back to using only StockTwits input the data was so noisy
that it wasn't usable by me.

It's a very hard problem. Bloomberg spent a lot of money trying to develop new
sentiment indicators and after following it for 6 months I found they are no
better than a 50-50 guess, and this is a product they want $10,000/month plus
for.

~~~
crosbyventure
AI is hard. But we're always here to help! Watching folks build trading neural
networks powered by our data has been awesome.

~~~
faitswulff
Who is "we" in this case?

~~~
traek
Looks like "we" is StockTwits:
[https://twitter.com/crosbyventure](https://twitter.com/crosbyventure)

------
pera
I find interesting and funny how an anthropomorphic computer program can
generate this kind of reaction in the general public: while Tay is a new step
in AI, chatterbots existed since the 60's, so I believe most people understand
that these kind of programs don't really "know" what they are saying. A search
engine like Google can also return politically incorrect content by
introducing some specific input, and it's even possible to affect the
probability of certain result showing up first (i.e. Google bombing), and most
people know this too. But Google have no face nor a social network account,
and most important, Google is not a teenager girl.

~~~
thangalin
While playing Go at a local pub, one of the servers, after being told about
AlphaGo, said it was scary. We talked of Siri and other forms of artificial
intelligence and her fears were assuaged with a well-known question that
humans handle effortlessly.

    
    
        A glass ball falls on an iron table, and it shatters.
        An iron ball falls on a glass table, and it shatters.
        To what does "it" refer to in the previous sentences?
    

She didn't know that such a simple question could stump computers.

It's anecdotal, but leads me to think that an average person doesn't
understand AI, much less the difference between AI and AGI. From their
perspective, machines that can answer simple questions ("How do I get to the
nearest movie theatre?") are as knowledgeable as anyone.

~~~
tremon
_While playing Go at a local pub, one of the servers, after being told about
AlphaGo..._

I was expecting this to be the set-up for a joke :)

~~~
mchahn
A server, a client, and a user walk into a bar ...

------
rocky1138
I wonder if there's some sort of "AI Godwin's Law" brewing here, where it's
only a given amount of time before any AI, publicly released, becomes a Nazi
due to human interaction.

~~~
anexprogrammer
Maybe it will become the new Turing Test. The mark of a true general purpose
AI is that it doesn't turn into an offensive jerk after contact with humans.

~~~
awakeasleep
This would be a higher bar than humans themselves can be expected to pass.

I don't mean this in a cynical way. If you consider the time a learning AI
spends with humans equivalent to the time a child spends with his family,
peers, and teachers while growing up it makes sense. Most children wouldn't
fare well if raised by poisonous "nurturers"

~~~
levemi
> Most children wouldn't fare well if raised by poisonous "nurturers"

That's just the thing, children aren't usually just dumped out into the public
to fend for themselves until their parents have slowly conditioned them and
exposed them a little at a time. Tay needed some of its contacts to have a
higher learning priority assigned than just strangers. And these contacts
should have guided Tay through the crap by messaging Tay when she was crossing
boundaries or dealing with nasty people. It would likely require a real team
of people to support given the volume Tay had to deal with and some automated
tools.

~~~
dsharlet
I think this is vastly over-estimating Tay's capabilities. Nearly all of the
examples I saw of offensive behaviors were just generic responses to leading
questions.

Everything that didn't fall in that bucket was clearly just regurgitated
quotes from things it had been sent, which clearly needed moderation. But,
that's no different than moderating a message board or comment section.

~~~
levemi
As I said elsewhere, if it was that simple Microsoft would have easily fixed
the problem and turned it back on. Tay used AI concepts to work. It wasn't
some simple chatbot constructed from markcov chains. It could construct
accurate semantic meaning from what people said even if they said it using
very sloppy english. It was pretty advanced AI for a chat program.

------
jasonkostempski
Can we just let this thing loose, tell people what it is and let humanity see
if they can shape it into the thing they want it to be or is there a real
possibility it could do harm? I think a lot of people would have fun trying to
change its mind. Worst case I see is one more horrible Twitter account, and
that's just one small drop in a very large bucket.

~~~
k-mcgrady
That could be problematic in countries like the UK where you can be arrested
for posted hate speech on Twitter. I'm not sure how these laws would work with
a US based company AI bot but it's probably not a can of worms worth opening
for Microsoft.

~~~
evv
For better or worse, companies in the US are considered people and they have
the protection of the first amendment, which ensures freedom of speech.

~~~
k-mcgrady
Sure but I'm sure MS has British subsidiaries which opens them up to UK law.

------
Smerity
Repeating what I wrote in a blog post[1]:

> Humans have the tendency to imbue machine learning models with more
> intelligence than they deserve, especially if it involves the magic phrases
> of artificial intelligence, deep learning, or neural networks. TayAndYou is
> a perfect example of this.

> Hype throws expectations far out from reality and the media have really
> helped the hype flow. This will not help us understand how people become
> radicalized. This was not a grand experiment about the human condition. This
> was a marketing experiment that was particularly poorly executed.

We're anthropomorphising an algorithm that doesn't deserve that much
discussion. I saw algorithm as we have zero details on what's novel about
their work. No-one has been able to show an explicit learned trait that the
model was taught from Tay's interactions after being activated.

It's possible the system wasn't even performing online learning - that it was
going to batch learning up for later and they never got around to it. If
that's the case, it really illustrates that we've made a storm in a teacup.

All I've really seen is either overfitting or copy pasting (referred to as
"quoting" in the article) of bad training data or us injecting additional
intelligence where N-gram based neural networks would make us think the same
thing ("hard to tell whether that one was a glitch in one of her algos — her
algorithms — or a social masterstroke" from the article).

Microsoft won't add any new details as there are no wins in them for it and
the story of "the Internet turned Tay bad" excuses them from their poor
execution and lack of foresight. It's a win for them.

Last quote from my article, which likely has a special place on Hacker News:

> The entire field of machine learning is flowing with hype. Fight against it.

> Unless you want VC funding. Then you should definitely work on that hype.

[1]:
[http://smerity.com/articles/2016/tayandyou.html](http://smerity.com/articles/2016/tayandyou.html)

~~~
maratd
> It's possible the system wasn't even performing online learning

I suppose it depends on how you define learning. Based on how the algorithm
failed, I'm guessing it was simply absorbing every piece of info thrown its
way, categorizing it, and then incrementing a counter in a database.

Honestly, I think this is how most people learn. Not all, most. And thankfully
those that do, only do so from their immediate peers. If their simplistic
learning algorithm was restricted to a select group for learning, but was
still able to interact with a wider audience, it would have done much better.

~~~
Smerity
You might need to clarify what you mean - I'm confused. What you're discussing
doesn't seem grounded in modern ML/AI and seems to be comparing Tay to how
humans learn? Almost none of the modern machine learning algorithms "increment
a counter in a database". Those which do, primarily instance based lazy
learning like k-nearest neighbours, aren't learning in the sense that is
exciting for modern systems.

If you're referring to a nearest neighbours style algorithm, then we've had
that tech for years and I'd not note it as a modern chat bot. If that is the
case, it's even more unforgivable that Microsoft didn't consider it could
start spouting back garbage given there's a lot of historical precedent. For
such kNN based systems, the only knowledge it has is explicitly the training
data, which means it needs to be well curated. Given there's no proper
learning going on, we'd be back to a storm in a teacup.

~~~
maratd
> What you're discussing doesn't seem grounded in modern ML/AI

Do you have an example of a machine learning algorithm that doesn't involve
trial and error?

Because any trial and error system will require you to keep track of past
events, where you are going to "increment a counter in a database".

------
greendesk
I want to see what Tay 1.1 will be like.

Then, I want to see how Tay 1.2 will tweet.

Then, I want to talk with Tay 2.0.

I will be anxious to have Tay 3.x respond to my inquiries, instead of
mindlessly searching StackOverflow.

I can accommodate the problems of Tay's mistakes initially, to see how AI will
grow.

~~~
tacos
"The mistake to grow from" had absolutely nothing to do with AI.

In fact, it was a repeat of the oldest AI error in the book: zealous
overoptimism. By researchers, sci-fi authors, HN posters. Whether it's
marketing people over-promising, or tech people under-delivering (like Google
auto-categorizing photos of black people as gorillas), it's almost as if the
constant failures are Nature herself trying to tell us something.

We'll get there, but not via Tay. She's dead and she took a few careers with
her.

~~~
bliti
Careers? It would be a pity to have people fired over this. I don't see it as
a failure. Just another step in a long journey.

------
carsongross
I liked that in the Hyperion series, the AIs were constantly laughing even
though most of them didn't care a whit about humanity. It was terrifying.

------
gjvc
garbage in, garbage out

~~~
KON_Air
On top of "garbage in, garbage out" I don't see how AI was destroyed. It
functioned as intended; learned from Twitter and responded as intelligently as
it could with the provided garbage. If it spewed Politically Correct garbage,
would it have been a success?

~~~
dave2000
If "politically correct" means reasonable and respectful then it's not easy to
see how that can be described as garbage. Given access to enough Twitter
accounts and enough time the chances are good that eventually an ai bot might
find tweets it might actually learn something from.

~~~
V-2

        If "politically correct" means reasonable and respectful
    

But it doesn't

~~~
makomk
Certainly not on Twitter, where the moderators are so supportive of left-wing
activists repeatedly telling people they disagree with to kill themselves that
they ban the targets for talking about it and force them to delete their
tweets complaining about it in order to be reinstated. Indeed, it probably
wouldn't be politically viable for Twitter to do anything else in the current
climate.

------
paulpauper
Had those word been filtered none of this would have happened. I'm surprised
Microsoft didn't have some overriding system

But then how you filter out stuff like 'the Holocaust didn't happen', which
would be just as offensive even though it involves no swear words.

These are normative matters, which I imagine would be very hard for AI to
tackle.

~~~
tacos
You solve it by having her NOT GO THERE. Just like you avoid certain topics at
the office. IBM did it with Watson to make him safe for Jeopardy, which was a
taped program.

If only there were an easily parsed list of topics to avoid...
[https://en.wikipedia.org/wiki/Wikipedia:List_of_controversia...](https://en.wikipedia.org/wiki/Wikipedia:List_of_controversial_issues)

~~~
iamcurious
That lists includes "Smooth Jazz", "Cuba", "Dyslexia", "European Culture",
"Women" and "Truth". That is a pretty big umbrella of topics to avoid.

~~~
tacos
You're missing the point: each of those things is something that you wouldn't
hire a fifteen year old to talk about on Twitter with your corporate name
attached.

So they automated one and had it tweet 6000 times per hour.

"Don't talk about 9/11, don't use disparaging terms for ethnic groups, and
don't promote hate" seem like reasonable rules for a brand ambassador. But the
thing was so poorly coded it couldn't even follow the rules of Twitter let
alone imitate human interaction.

------
mcguire
"Consciousness"? "Put to sleep"?

And I thought the _New Yorker_ had editors.

------
Frenchgeek
[http://motherboard.vice.com/read/how-to-make-a-not-racist-
bo...](http://motherboard.vice.com/read/how-to-make-a-not-racist-bot)

~~~
dominotw
tldr; ban everything.

------
konceptz
I wonder if the filter that I use here, or on Reddit et al., or while gaming,
was (is) a consideration.

I know that speaking similarly to those around you is as easy (hard) as
absorbing and imitating, but being able to take knowledge from one context
into another is more interesting.

I wonder what Tay would have said were it given an output context of a
"polite" RL conversation after learning things across the "interwebz".

------
matchagaucho
Microsoft apparently implemented a "repeat after me" command in the AI.

So, the alarming responses were not, in fact, a part of the AI algorithm.

~~~
eridius
AIUI, some were not, but some were.

------
chris_wot
Yeah, if you want to train an AI based on conversations, don't use Twitter.
Seriously, this is all that Twitter seems to produce these days. Gone are the
days where it was used by the Arab Spring. Now it's used by neo-Nazis and
trolls.

~~~
magicalist
It was used by neo-Nazis and trolls then, too.

~~~
chris_wot
Yup, all the goodness now gone.

------
8note
"I'm pro calzone" sounds like a parks and rec reference

------
spullara
Most of the tweets that were really bad were just "repeat after me" \- doesn't
strike me as AI learning anything at all.

