
Facebook M – The Anti-Turing Test - arik-so
https://medium.com/@arikaleph/facebook-m-the-anti-turing-test-74c5af19987c
======
username223
Can M wade through phone support menus and cancel Comcast subscriptions? I
look forward to a darkly-humorous future in which we pit poorly-paid third-
world citizens against each other in wars of call center attrition. Better,
once we equip them with those sound-board UIs that play pre-recorded answers
in native English-speaking voices (can't find the link), English can become a
transmission protocol that few people deal with directly.

~~~
Hortinstein
> I look forward to a darkly-humorous future in which we pit poorly-paid
> third-world citizens against each other in wars of call center attrition.

This sounds like a 10 page side story in a Neil Stephenson Book. I love it.

~~~
JadeNB
It's very nearly part of the strategy of one of the players in Stross's
Accelerando ([http://www.amazon.com/Accelerando-Singularity-Charles-
Stross...](http://www.amazon.com/Accelerando-Singularity-Charles-
Stross/dp/0441014151)).

~~~
templaedhel
For what it's worth this book (which is excellent) is also published as a free
ebook under a Creative Commons license ([http://www.antipope.org/charlie/blog-
static/fiction/accelera...](http://www.antipope.org/charlie/blog-
static/fiction/accelerando/accelerando-intro.html)).

The author is also a relatively prolific HN poster
([https://news.ycombinator.com/user?id=cstross](https://news.ycombinator.com/user?id=cstross)).

------
graeham
"I'm AI but humans help train me"

The implication to me is that the chat is with a human, who is using an AI
tool with the intention of training that tool. What better way to train a new
service than to launch it, then answer all the weird, unexpected questions
with humans? Gradually more of the questions get answered, the AI gets better
trained, and the human-AI becomes increasing more AI.

Further, as the AI gets better, the human working with it has to do less, so
they can roll out the service to more users without requiring more staff.
Perhaps eventually, no human is needed.

~~~
abritishguy
The AI can assist the human as well. It may not be confident that its response
is accurate and the human just clicks send.

~~~
PaulHoule
Great point -- if a system has a good probability estimator it can send
questionable answers for review to raise accuracy up to the commercially
useful level.

------
pjc50
This is a new angle on the app-outsourcing-to-low-paid-contractors
"technology": it's so dehumanising that you have to pretend to be a computer
while you work!

It's also strikingly similar to the original "mechanical turk".

~~~
provemewrong
>It's also strikingly similar to the original "mechanical turk".

That's what the M stands for.

~~~
personjerry
No, it's not. Unless you have a citation...?

~~~
provemewrong
It's a joke.

~~~
avalaunch
I still want a citation.

~~~
monochromatic
Cite:
[https://news.ycombinator.com/item?id=10538766](https://news.ycombinator.com/item?id=10538766)

------
argonaut
Why is this even a question? According to Facebook itself, it's mostly human-
driven.

[http://recode.net/2015/11/03/facebooks-virtual-assistant-
m-i...](http://recode.net/2015/11/03/facebooks-virtual-assistant-m-is-super-
smart-its-also-probably-a-human/)

~~~
BukhariH
> "The opinion is split as to whether or not it’s a real AI, and there seems
> to be no way of proving its nature on way or the other."

Clearly, the author didn't even do the most basic fact checking. Since,
Facebook clearly told everyone that M was going to be AI that was assisted by
humans.

It's literally in the announcement post:
[https://www.facebook.com/Davemarcus/posts/10156070660595195](https://www.facebook.com/Davemarcus/posts/10156070660595195)

> "It's powered by artificial intelligence that's trained and supervised by
> people."

~~~
icebraining
But that's exactly the point: _is_ it AI trained by people, or people aided by
AI tools? It's not the same.

~~~
ikeboy
See [http://www.wired.com/2015/08/how-facebook-m-
works/](http://www.wired.com/2015/08/how-facebook-m-works/)

~~~
acqq
Yes, the article is explicit: "For more complicated tasks, such as making a
driving test appointment at the DMV, the humans will do most of the heavy
lifting. They’ll actually place a call to the DMV."

------
fab1an
Facebook's strategy here is to build an AI brand before they have the actual
technology, which could make a lot of sense. At the same time the interactions
between M's team and its users will provide meaningful data to train the AI
on.

~~~
andreasvc
I've never heard vaporware characterized as something that "could make a lot
of sense"; how could it?

Plus I'm doubtful whether the data would be very meaningful. A bunch of people
adversarially trying to figure out whether the AI is real is not
representative or generally useful data.

~~~
drumdance
Microsoft Windows was vaporware for years. They famously did a "demo" that was
just a manipulation of graphics. But Bill Gates correctly grasped that the
future of the business rested on it and set about building the brand.

~~~
andreasvc
How does your N=1 anecdote show that vaporware makes a lot of sense? I'd say
they got away with it, but not that it was a crucial factor in their success.

~~~
drumdance
Because that N became the dominant computing company for a generation?

Apple was way ahead of Microsoft with windowing technology. Gates even offered
to help Apple port it to the x86 architecture. He was content with being the
dominant application developer, not the OS.

Jobs blew him off so he seized the opportunity.

Around the same time IBM was working on OS/2, which was a much better
technology than Windows. Microsoft worked with IBM on OS/2, but then
gaslighted them by "debuting" Windows 1.0

Edit: And don't forget how me VC pitches are about "what we're going to build"
vs. "what we've already built."

Vaporware is not an optimal strategy, but in many cases it works.

~~~
andreasvc
Look I can see that it worked out well for them in the end. What I'm
specifically disputing is whether the vaporware-thing was a causal factor
contributing positively. If they actually had the software already and didn't
need to fool anyone, things might've worked out even better. In that case
things worked out _despite_ the vaporware, not _because of_.

~~~
drumdance
I think if you have the foresight to see a multi-billion dollar opportunity,
and you're behind schedule, then you do everything in your power to grab that
opportunity, including vaporware.

 _If they actually had the software already..._

No one sets out to lie. Gates would've preferred to demo the real thing. But
in the context of that market, "we'll release that in two years" is worse than
vaporware if you want to actually own that market.

------
chrisBob
I just had the perfect idea for a test, but then I went back to the recent
discussion on Mimic[0] and double checked my favorite example[1]. Google has
already updated their support, but there is a chance that Facebook M is still
behind. Test them now before it is too late:

"When is the next Τаylοr Ѕwіft concert in my area?"

[0]
[https://news.ycombinator.com/item?id=10437619](https://news.ycombinator.com/item?id=10437619)

[1]
[https://www.google.com/search?q=Τаylοr+Ѕwіft](https://www.google.com/search?q=Τаylοr+Ѕwіft)

------
jwalton
Bad typing is definitely not enough to measure if an AI is really a human. As
a teenager, I wrote a chatbot for an online text based game. I have it
knowledge of a QWERTY keyboard layout, and when "typing" it had a small random
chance of pressing the key next to the key it wanted to press. It would also
sometimes transpose characters. Sloppy typing can be simulated.

Might be an interesting test to do a statistical analysis of your subject's
mistakes against a corpus of real human mistakes, since there are many common
mistakes humans make, and a random AI might make inhuman mistakes, but this
would of course not be conclusive.

~~~
espadrine
Simulating bad typing is only necessary to fake a human. Here, having bad
typing when faking an AI is stranger.

That said, AIs trained through the chat transcripts of a large number of
conversations may produce mistakes. I remember reading a paper that gave good
results that way, with the side-effect that it produces typing mistakes as a
result. I cannot find that paper again, unfortunately.

Edit: found it!
[http://arxiv.org/pdf/1506.05869v1.pdf](http://arxiv.org/pdf/1506.05869v1.pdf)

~~~
eridal
I wonder if there's code available, from that paper. Results look promising.

Thanks for sharing!

------
downandout
Clearly there are _some_ humans behind M that are doing things that Facebook
would rather entrust to humans (like making phone calls). However, the phone
call only proves that this specific aspect of M is human.

In the end, though, I suppose it doesn't matter. I'm going to guess that the
ultimate end-game on M is for Facebook to collect advertising/affiliate
revenue from recommending things. For example, if someone asks for a Chinese
restaurant, plumber, dentist, lawyer, etc. in their city, the one they suggest
could be the one that paid Facebook for it. As long as these types of fees
make it profitable for Facebook, it doesn't matter if the service needs to be
powered by millions of humans. In fact, that would be great - it would mean
millions of new jobs.

Larry Page famously told an early investor that Google wasn't yet sure how it
would make money, but that search was the only situation in which people would
tell a computer what they wanted, and that there had to be a way to make money
from that. M is exactly the same - a way to get people to tell Facebook what
they want, and it puts them in a great position to monetize it.

~~~
kayoone
Basically what GoButler or Magic are already doing, just AI based. But this is
the goal for all similar services, otherwise it will not scale.

------
jasperry
I wonder if you can ask M to use fewer exclamation points. From the
conversations I saw in the article, it's a little too chirpy (or should I say
"clippy") for my taste.

~~~
eric_h
I had the same sentiment. Similarly - when I ask Siri what time it is on my
new Apple TV, after midnight it always says Zzzz... as if it's judging me for
staying up late.

I'm not a huge fan of AIs fake emoting all the time. Occasionally, it's
amusing, but all the time it just rubs me the wrong way.

------
lhnz
I wonder how many years it will be before a real AI can compete with
Facebook's AI?

I guess by the time that's possible Facebook's pretend AI will already have
cornered the market.

The public will only be able to see that you were the late entrant and that
while your AI is faster it's occasionally incorrect in peculiar ways...

This seems like a fairly solid plan by Facebook to crown themselves the
winners of a race that hasn't yet finished.

~~~
AJ007
Pretending you have a working technology when you don't has been a recent
theme in the startup world.

~~~
drivers99
Thomas Edison claimed to have a long lasting light bulb before he actually
did. He showed it to reporters one at a time in a booth. Between observers, he
would change out the light for a fresh one. Source "How We Got to Now: Light"
(on Netflix currently, at least in the US). Found the clip on PBS. Skip to
2:20 for the specific part: [http://www.pbs.org/how-we-got-to-now/big-
ideas/light/](http://www.pbs.org/how-we-got-to-now/big-ideas/light/)

~~~
pluma
I thought Edison had mostly been dismissed as not even remotely as great an
inventor as most people thought he was?

~~~
throwaway7767
He was a businessman, who hired people to invent things. He was certainly
useful, but it's always been surprising to me how much credit he got for
inventing things.

~~~
gohrt
Capitalism and Personal Branding at its finest.

------
Grue3
"Can you solve this CAPTCHA for me?"

(provided the CAPTCHA is sufficiently OCR-resistant)

~~~
andreasvc
If you want to avoid giving information about whether you are an AI or human,
you simply respond "No."

Besides, the CAPTCHA's that are sufficiently hard to solve for computers are
already hard for humans as well.

~~~
kuschku
> Can you tag for me all photos in my album that contain a kitten, but not a
> dog, with "kitten", and those that contain a kitten and a dog, with
> "pets<3"?

That should keep it busy.

~~~
twright0
This approach is backwards. This is the kind of problem that is easy for a
person, but not for an AI. So if M was an AI pretending to be a human, you
could use this to determine that. But in this case, the suspicion is that M is
a human pretending to be an AI - and they could simply decline to attempt the
task, or pretend to be unable (or do a bad job deliberately), and you'd learn
nothing from negative results.

------
mrdrozdov
I don't get the part about the reverse number lookup. Couldn't they be using a
disposable phone number that is allocated to Facebook? That's what Handy,
Airbnb, Uber, etc. do. Why would they have to block their caller id? And how
does either method prove or disprove that M is human?

~~~
shawabawa3
That doesn't prove that M's human, what prove's it's human is that a Human
voice called. The fact it says Facebook is just evidence that it was indeed
from M, and not just him getting his friend to call him and pretend.

~~~
throwaway7767
> The fact it says Facebook is just evidence that it was indeed from M, and
> not just him getting his friend to call him and pretend.

It doesn't really prove anything, since caller ID is extremely easy to spoof
(I used to call my mates from the emergency number for kicks when I was
younger). Not that I have any doubt as to the credibility of the story.

------
adrianN
This reminds me of the Focused people in Vernor Vinge's "A Deepness in the
Sky", slaves that were integrated into the computer system to provide function
that surpassed the computer's built in intelligence.

~~~
austinjp
I've not read that. Personally i felt we're one step closer to the "cookies"
in Charlie Brooker's "White Christmas" episode of Black Mirror.

From the article :

“Our test participant was impressed with how much M could do, but was
sometimes disappointed at how long it took,” UserTesting’s report reads. “He
concluded that it would be very useful if he could set it to perform a
non-­urgent tasks for him while he worked on other things.”

That made me shudder. One person tutting at the poor performance of "it". It
seems plausible that robot-powered tasks would complete rapidly, and humans
power the slower processes.

So the participant didn't know it was human-powered. If anything, that makes
things worse.

------
agopaul
Has anyone tried to hack a script that put in communication two M chat
sessions?

------
Lewton
Tihs is uslulay how I tset wehther i'm tlaknig to an ai or not

~~~
tux3
This is always impressive to me, I didn't realize a thing until "tlaknig"!
Although I suppose you could specifically train an AI to recogize this.

~~~
Gankro
To be clear, this is applying the classic observation that if you keep the
first and last letter correct, humans are really good at unjumbling the
center.

~~~
csn
Slpmiy jnilbumg the iennr cretcarahs tlsuhy rrednes tihs "ooitavresbn"
qlbanoitseue, imo

edit: 'Simply jumbling the inner characters thusly renders this "observation"
questionable, imo'

------
codeshaman
Regardless of wether M is currently more human than AI, we could project that
in the not-so-distant future (after it's trained), M will be mostly, 99% AI.

The technology itself will become more and more available and other companies
will also use similar AI tech to work with customers.

The ultimate moment will be when the AIs start talking to each other in human
language, each 'thinking' that the other is a human.

That will be the moment when the machines have decided something for you and
while at first you'll think that _you_ triggered that, at some point it will
become unclear - is the human triggering the AI or is the AI triggering the
human.

Pretty soon, everything we consume and everywhere we go will be controlled
(and, a bit later, predestined and programmed for us) by the AI.

------
rvac
It's a human using Siri to answer your questions.

~~~
VLM
Based on punctuation analysis, word choice, tone, its not just any human but a
mid 20s white female. Probably front ending google.

Real comedy would be going to mturk to try and find the task to communicate
try to crack it recursively "M find me the mechanical turk task for this
request".

------
hellbanner
So how well does this scale, if all of Facebook's users are using "M" like
this?

~~~
adrianb
Launch is limited to selected users in Silicon Valley.

~~~
hellbanner
That explains why Facebook.M has such knowledge of the local area.

------
egmalek
Imagine if the 1.3B Facebook users were eligible to be called upon by the AI
on a Quora way to answer a question the AI couldn't answer alone...

~~~
nickodell
I think the quality of the answers would be along the lines of Yahoo Answers.

------
swang
Did the author ask it about how we can avert the heat death of the universe?

~~~
evv
There is insufficient data for a meaningful answer.

------
aeturnum
What a silly conclusion. The fact that a human called his land line does not
mean M (the thing in messenger) isn't an AI. At best, it proves that M has
humans who work for M making phone calls.

I don't have any insight or opinion about the question of how human M is, but
this article seems makes a bunch of assumptions that make the whole
investigation somewhat silly.

~~~
azernik
I think the distinction is a philosophical question. Are they humans who run
errands for the AI, or is the AI a tool the humans use?

Perhaps better to think of them as coworkers, each specializing in their
strengths.

The question is, just how much of the workload is the machine capable of
handling? Because I think that's the big indicator of scalability.

------
sidcool
Humans working at facebook scale! Would be interesting to see how many people
are employed to do this...Are they Googling?

~~~
MasterScrat
The reference to Google Maps was a surprise to me... Does't FB typically
relies on Bing maps?

~~~
sidcool
Indeed, they do. Facebook has traditionally been inclined towards Microsoft
than Google. Probably because they are the lesser rivals in business.

~~~
pinkrooftop
Microsoft was an early investor in Facebook

------
joss82
But Facebook publicly admitted that the service is powered by real humans:

[http://www.wired.com/2015/08/facebook-launches-m-new-kind-
vi...](http://www.wired.com/2015/08/facebook-launches-m-new-kind-virtual-
assistant/)

------
kriro
So basically the suspicion is that M is a concierge MVP? From reading the chat
excerpt I'd agree.

Edit: It would be interesting to devise a way in which you can make two Ms
talk to each other (or have M talk to Siri etc.). Maybe "can you pretend to be
a customer for my XYZ business"

------
mahdiponline
As much as I appreciate the effort, I don't think proving M has humans behind
it is any of help.

We write AIs. We try to make them act just like us. We teat them in everyway
we can imagine and we expect them to act like a human would in response.
Providing an algorithm for this is not always useful or maybw not even
possible.

My theory on this is that Facebook is powering M with both people and some
sort of AI software that not only analyzes and sometimes finds the best
response, but it also analyzes the conversations people on both sides made.

Now this can be useful on several levels. Facebook can improve it's AI
algorithm in less time, the AI can help people on their job in the meantime
(by analyzing their work and commenting on it)

------
free2rhyme214
This guy is hilarious. If you're reading HN comments you know that AI isn't
quite there yet right? We're easily 5-10 years away from anything you're
looking for.

------
SilasX
>The most noteworthy aspect of this reply is that “Google Maps” wasn’t
capitalized, suggesting that maybe, just maybe, a human typed it out in a
hurry.

Or they're smart enough to add random mistakes. When I started a project for
setting up multiple ways to say the same form letter, I thought of adding a
random-typo feature to make it look like humans were writing it. I'm sure
these guys are at least as cheeky as me...

~~~
Quanttek
But they don't try to convince you M is a human. Indeed the opposite. So it
would be rather stupid to add typos to an AI, when you want people to see it
as an AI

------
mizzao
I don't think there has to be a huge controversy here. It's perfectly
plausible to build a system that contains a hybrid of human and machine
intelligence, where the humans work on the more fuzzy questions that cannot be
directly answered yet, and the interactions used to fill in the gap as the AI
is improved for later.

~~~
nhf
Yep! There's a lot of research going on in that area. For example,
[http://dl.acm.org/citation.cfm?id=2702416](http://dl.acm.org/citation.cfm?id=2702416)

Some recent work on fusing machine learning with Mechanical Turk workers to
create "sensors".

------
gjm11
For something with a similar flavour, see
[http://dangermouse.brynmawr.edu/csem/coffeehouse.html;](http://dangermouse.brynmawr.edu/csem/coffeehouse.html;)
start reading where it says "Post Scriptum".

~~~
traverseda
Didn't like that semicolon.

~~~
gjm11
Ugh. Here's the URL without punctuation after it:
[http://dangermouse.brynmawr.edu/csem/coffeehouse.html](http://dangermouse.brynmawr.edu/csem/coffeehouse.html)

------
moey
Do you guys think one person, working alone, can develop an turing passing AI?

------
ben_utzer
Is it me or all the photos are blurred? I can't read them

~~~
berdario
It's not just you. It's basically unreadable on Firefox on Android (and what's
worse... the page prevent arbitrary zooming, unless you "request the Desktop
page").

It's depressing how a supposedly well-designed platform like Medium still
falls short of providing an usable mobile interface.

~~~
7Z7
Can confirm, it's perfectly readable on Safari on iPhone.

Also zoomable.

------
SIOP
Thanks for this. Fascinating. A really interesting article.

------
Houshalter
Most likely it's mostly AI, that redirects to a human when it's confused. Most
of those responses look pre-programmed.

------
wallzz
Does anyone know how can I try it ? I can't find a link or something like
that.

~~~
richard_mcp
Looks like a limited roll-out to users in the Bay Area.

------
niix
Reminds me of the chat bots I used to write for AOL and AIM "progz".

------
pinkrooftop
Human or AI the level of service provided seems pretty amazing

------
Maksadbek
Is M currently for USA, it did not appear in my contact yet.

------
dyeje
It ends so many sentences with exclamations!

------
yoavm
this might be one of the worst jobs ever

