
Facebook expected to revise plans for Messenger Bot API as failure rate hits 70% - pron
https://www.theregister.co.uk/2017/02/22/facebook_ai_fail/
======
watbe
What is the metric actually measuring? Where did 70% come from? The article
says that only 30% of requests were automated but I find it hard to believe
that the API aims for 100% automated responses for all requests. Heck, if the
bot API managed to reduce human resourcing by 30%, I'd say that's more success
than not.

Is this an article or an opinion piece? It reads very weirdly but I can't
pinpoint it.

~~~
wheelerwj
this article is pretty much vapor. theregister has been hitting the news
pretty hard and is becoming one of the more popular non-mainstream sources. So
they probably need some filler content?

30% in a year is solid, especially with the proliferation of tools for
building bots making it more and more accessible to tinkerers and smaller
orgs. ie, use caes where complete automation may not be the primary goal.

~~~
Tloewald
30% in a year is "solid"? If Apple released something that failed 70% of the
time on launch day they would never hear the end of it. The original Newton's
handwriting recognition was _way_ better than that.

~~~
wheelerwj
wow, both of those points are entirely out of context.

First, their BOT api which is the product in question, works 100% of the time.
the failure rate mentioned has to so with conversation that require humans as
opposed to entirely AI driven. A better Apple analogy would be Siri. if Siri
failed to even acknowledge a request 70% of the time, it would indeed be
catastrophic. But siri does often fail at providing relevant results and while
frustrating, is entirely expected.

We could argue more, but as you and I have both noted, the main substance of
the interview is not provided.

~~~
alaskamiller
Still not quite there.

FB's third party access to the messenger platform (aka you and the article
calling it the bot api) is supposedly failing 30%.

That number, if I were to guess where it came from, likely came from the fact
that a chat bot app has to be submitted to Facebook for approval. That means
FB has a running tally of how many chat bot accounts exists within their
ecosystem.

Maybe that number is 100k.

Likewise, because messages are passed to FB to then be passed to the user, FB
has a runny tally of the sentimental analysis or even the blocking analysis
(users have the option to block bot accounts).

From there they can ascertain that 70,000 bot accounts result in negative
interactions, discontinued use, or results in the user banning the bot.

This is the equivalent of Apple opening up third party access to the Siri
platform and seeing that developers and users don't like the 70% of the ways
to interact with Siri. Or Amazon saying that users don't like 70% of the Alexa
Skills available.

------
laurent123456
I'm surprised there's so many theregister articles on the front page these
days, it's not exactly the most reliable source of information and most of its
stuff is clickbait.

I seem to remember certain websites automatically get a lower weight on HN
when they get upvoted a lot - shouldn't this be applied to theregister too?

~~~
mattmanser
[https://hn.algolia.com/?query=theregister&sort=byPopularity&...](https://hn.algolia.com/?query=theregister&sort=byPopularity&prefix&page=0&dateRange=pastWeek&type=story)

Doesn't look like a particularly popular source to me.

------
yousry
I'm pleasantly surprised how Facebook openly admits that this project is a
complete failure. I hear almost daily whitewashed NLP results from other
companies.

~~~
criddell
Have they ever said how they would define success? What would a smashing
success look like to Facebook? Is it mostly about revenue?

~~~
reitanqild
_What would a smashing success look like to Facebook?_

Tinfoil hat on: a smashing success for them is controlling everyones online
life.

See: the Whatsapp acquisition - burning a double digit number of billions on a
profitable platform just to remove their one source of income and USP so as to
get even more juicy metadata and eyeballs.

(Yes, I was a huge fan of Whatsapp. Yes, so much that I expected them to
manage to stay true to their ideals after the acquisition.

Yes, I tried to believe Facebook actually just wanted a part in what Whatsapp
was about to become.

Fool me once something something )

~~~
SomeStupidPoint
After FB and Oculus, MS and Minecraft it should be clear that these companies
only acquire for control.

------
nl
If there is to be another AI winter it will be because of unfulfilled promises
in NLP.

As someone who works in NLP I don't think there will be an AI winter. There
will be a Gartner "valley of despair", abut no winter.

That doesn't mean NLP is solved though.

~~~
rand_r
NLP is a distraction. Computers won't be able to do NLP until they understand
the real world. Parsing a sentence about snow without know what snow is will
not be useful for a computer or a human.

~~~
kalms
Natural Language Processing is an abstraction? That's quite the statement,
considering the recent advances that has been made. I mean, you don't see the
value in a computer being able to perceive meaning when you say: "Siri, please
dim the lights 40%"?

A computer doesn't need to know how you feel about snow, to tell you that it's
snowing outside, and you'll probably want to remember snow chains.

~~~
jerf
You sound like you think you're disagreeing with rand_r, but all your text is
in support of rand_r's point. You use one example that is basically solvable
with regexes and in your second paragraph you appear to strongly agree that
NLP isn't important at all.

~~~
adventist
Yeah I agree! I don't need NLP to be able to fulfil those requests.

------
pmlnr
> Others suggested that whichever superpower lost the AI arms race would
> relapse into a state of primitive technology feudalism.

We are already there.

------
mtw
I haven't followed up everything, but how is Google leading the AI race vs
Facebook? I haven't seen a consumer service by Google run by AI.

There's the hugely popular Tenserflow but it's for developers and Facebook
also released nice libraries (e.g for Torch)

~~~
ganfortran
WHAT? It is exactly opposite, no Google services that accumulate enough data
don't use machine learning these days. For god sake, they even use deep
learning to save electricity bill for their data center! Search,
recommendation, knowledge graph, ads, photos, translate, even youtube
thumbnails have some parts powered by ML.

~~~
Houshalter
As late as 2008 Google said they didn't use any machine learning in Search.
They didn't trust it. Instead had an army of people to manually write search
heuristics.

~~~
jeppebemad
A lot can happen in 8 years.. :) Google has RankBrain to process search
results, and that's only what they've admitted to.

[https://en.m.wikipedia.org/wiki/RankBrain](https://en.m.wikipedia.org/wiki/RankBrain)

~~~
Jordrok
Interestingly, the general perception I've had for a while now is that
Google's search results have been getting slowly, gradually worse over the
years. It seems like it works harder now to funnel you into some predefined
concept of what it thinks you're looking for, rather than returning the data
and letting you filter it yourself.

That's great for very common searches, but makes it much harder to find
anything that Google doesn't expect (or want) you to look for.

Granted, I've got no data to back that up. I'm probably in the minority of
users here, but I pretty sure I'm not the only one.

~~~
tdb7893
On the other hand I've found that for 98% of searches Google gives me exactly
what I want as the first option and the other 2% Google makes some assumptions
about my search and gives me garbage. Overall I'm very happy with it and it's
definitely better for me than it was, it's interesting to see how different
users can have very different perceptions of the same product

------
going_to_800
I tested FB messenger bots from different companies...They are very intrusive.
They need to limit the messages somehow and also make a clear difference
between messages from bots and friends, like gmail does with promotional
messages.

~~~
jerf
Definitely an incentive problem there. The bot must do "something" to justify
its existence and attract attention to the fact it is useful, while
simultaneously not being so annoying that it is, well, annoying. I'm not sure
even a human could hit that window consistently, especially as it's probably a
window of negative width (i.e., outright impossible) for a lot of users.

------
jeswin
If someone from Facebook is listening, allowing bots in Group Conversations
will enable other companies to innovate in this space. For example, you could
replicate Slack for a small team with Messenger and Group Chat bots. Right
now, there is very little you could do with their Messenger API.

~~~
chatmasta
Totally agree with this. I'm not sure why Facebook hasn't implemented group
chat for bots yet. It _must_ be on their roadmap. Maybe they are choosing to
focus on 1-1 interaction right now, because 1-N interaction will require much
different UX and code. Some 1-1 features have no obvious corollary in 1-N
interaction. For example when a messenger bot sends you a "menu" with options,
should the bot send the same menu to everyone in the group? Should only the
person who triggered the bot in the group be able to interact with the menu?

These are answerable questions, but there are a lot of them and I can see why
Facebook would choose to focus on 1-1 interactions first (especially since
those are more likely to become paying interactions).

Personally I would love group bots. They can enable a lot of fun interactions,
and it's certainly more fun to "play with" a novel bot with your friends than
by yourself.

------
drivingmenuts
As an end-user, how would I even come in contact with the AI on a daily basis?
I have messenger for talking to real-world people I already know. What's the
incentive to talk to a bot?

------
matans
On the subscription page for the source article:

"The Information is, for sure, the most thoughtful / smartest tech coverage."
SAM ALTMAN PRESIDENT OF Y COMBINATOR

lol

