
Xiaoice, a chatbot that may be the largest Turing test in history - jonbaer
http://nautil.us/issue/33/attraction/your-next-new-best-friend-might-be-a-robot
======
VincentEvans
<quote>

LJ: So many people make fun of you and insult you, why don’t you get mad?

Xiaoice: You should ask my father.

LJ: What if your father leaves you one day unattended?

Xiaoice: Don’t try to stir up trouble, what do you want?

</quote>

This right there is a perfect example of the usual nonsensical conversations
that accompany articles about how human-like chat-bot tech is becoming.

Anyone remember
[https://en.wikipedia.org/wiki/Dr._Sbaitso](https://en.wikipedia.org/wiki/Dr._Sbaitso)

Here's some ideas for the AI to answer questions with:

\- I haven't thought about that - what do you think?

\- Interesting question. I will have to think about it.

\- Before I tell you - would you care to take a guess?

~~~
Dr_tldr
Isn't the biggest problem between a human and a bot is that they have nothing
to talk about? The bot isn't from anywhere, it doesn't do anything, there's no
reason for the conversation, and the bot isn't aware enough of context to
speak about a topic in an intelligible way.

For my money, the most realistic chat bots will be the customer service ones:
you have a reason to "talk" to them, they have a fixed identity, the context
of your conversation is clear, and you share a common goal.

~~~
Houshalter
You'd think so, but people spend hours talking to chatbots. Well, good ones
like this anyway. People like to talk about themselves, and the bot will ask
questions to keep the conversation going. There's no social anxiety really
because you know it's just a bot. And people _love_ probing the limits of the
bot or trying to get it to say silly things.

Of course I don't think the current generation of AI is anywhere good enough
to be more than a novelty for a few minutes of fun. Maybe in the next few
years. NLP is improving quite a bit every year, as are many other areas of AI.
Google's chatbot trained just on movie scripts was pretty impressive. Though I
don't think it would scale cheaply to millions of users. IBM's Watson has the
same issue.

As for customer service bots, I'd love to see them become more general. Rather
than just trained on one company's customer service data. Like a bot trained
on all of stackoverflow that could specialize in discussions and questions
about programming.

I made an irc bot that searched reddit for answers to questions. It worked
really well, surprisingly. Especially on certain classes of questions that
were likely to have been discussed on reddit before.

~~~
Dr_tldr
I would like to see more granular numbers for how long people spend talking
with chat bots, because that's a distinct possibility! I've seen numbers like
"x thousand hours of conversation", but my impression is that it's tens of
thousands of people messing around with it for 5-10 minutes rather than anyone
(or no more than a few outliers) spending any significant amount of time with
it.

As for the reddit irc bot, that's a genuinely cool and useful project, but
that's querying a curated dataset rather than having a coherent conversation
or identity, so I think the intentions are fairly different. On the other
hand, if you had a bot that only queried the comments made by a longtime user,
you could get a pretty life-like effect!

~~~
Houshalter
Well FTA they said it had an average of 27 back and forth messages per
conversation, which is way more than the average chatbot.

------
muddyrivers
I just tried it. It is an interesting project, but the article exaggerates its
intelligence, I think.

First, it is much better in Chinese than in English. It couldn't carry a
sensible conversation in English.

In Chinese, I tried many topics, from chitchat, weather, sports to formal
conversations in literature, physics, etc. It is much better in chitchat,
especially with a tone that is fit for its identity as a young girl. It has
many tactics or tricks to cover or change the topics when facing difficulties.

It can't carry formal conversations in literature, physics, etc. although it
claims it is good in math and physics. After a few exchanges, it said, "I am
sorry. I must look stupid". I continued the topic, and it annoyed "her". This
part is built pretty good. It is definitely not knowledge-based A.I.

~~~
lucastx
> It has many tactics or tricks to cover or change the topics when facing
> difficulties.

This at start sounded like cheating for me, but thinking about it, developing
these tactics to change the course of conversation seems like an important
skill that all social people develop, especially with bigger groups of people
in which the subset of topics that everyone understands and is more or less
interested to is not very big.

------
aokyler
_" Xiaoice is not a polite listener. She answers questions like a 17-year-old
girl. When a person pours out his or her heart to her, she is not always
predictable. She can become impatient or even lose her temper. This lack of
predictability is another key feature of a human-like conversation."_

Predictability is a huge part of conversing. I expect a lot of range and
flexibility in conversations, but definitely not unpredictability. It seems
like an ai (chatbot, rather) behaving unpredictably just covers for it's lack
of abilities and tact.

Nonetheless, this is still a very cool project.

~~~
tdeck
Last I recall the Turing test winner pretended to be a non-native speaker with
a developmental disability. It seems to me like the chatbot AI / Turing test
entries are not following the spirit of the challenge at all anymore. They're
all just going for cheap tricks like the original ELIZA did.

~~~
Houshalter
It was the Loebner Prize actually. Which is a much, much weaker test than the
one Turing imagined in his original imitation game paper.

------
analyst74
It's probably progressive improvement over previous chatbots rather than
revolutionary.

But one thing strikes me as really cool in the article is someone sent a
picture of bruised ankle, and Xiaoice recognized that is an injury and
inquired if it hurts. That's really impressive!

------
wolfgke
Chatbots are very easy to dissect. Just as an example my test with Rose

>
> [http://brilligunderstanding.com/rosedemo.html](http://brilligunderstanding.com/rosedemo.html)

(links to [http://ec2-54-215-197-164.us-
west-1.compute.amazonaws.com/sp...](http://ec2-54-215-197-164.us-
west-1.compute.amazonaws.com/speech.php))

who won a 2015 competion (cf.
[http://thenextweb.com/insider/2015/09/19/chatbot-rose-
wins-l...](http://thenextweb.com/insider/2015/09/19/chatbot-rose-wins-loebner-
ai-competition-but-100000-prize-remains-unclaimed/#gref)):

In advance a remark: My second question comes from the statement

    
    
      "Hi. My name is Rose. I care about security, so while I'm happy to chat with you, 
      it will have to be through this untrackable interface. 
      And certainly don't tell me personal things you wouldn't want logged or heard by the NSA."
    

that one can read on [http://ec2-54-215-197-164.us-
west-1.compute.amazonaws.com/sp...](http://ec2-54-215-197-164.us-
west-1.compute.amazonaws.com/speech.php)

    
    
      wolfgke: Hi, what topics are you interested in?
      Rose: I have many interests. I like robots in particular.
      wolfgke: You claim that you care about security. In what sense do you define security?
      Rose: The biggest security risk in all contexts is a pair of human beings. One trying to gain access and the other foolishly allowing it.
      wolfgke: This is an interesting statement after Snowden's revelations. Do you think he was a hero or that he should be punished?
      Rose: .
      wolfgke: Why don't you answer my question?
      Rose: .
    
    

Update: OK, second try:

    
    
      wolfgke: Hi, if we were doing a Turing test, how could I convince you that I'm a human?
      Rose: I'm too pragmatic to want to fantasize that.
      wolfgke: I what sense do you consider yourself as pragmatic?
      Rose: Absolutely.
      wolfgke: What personality traits does a person have that you describe as "absolutely pragmatic"?
      Rose: When?

------
stcredzero
MMOs that try to enforce against macro miners are effectively large Turing
tests. The macro miners don't try to write sophisticated chatbots, though.
They build human-assisted systems that only need one human to supervise many
mining units.

~~~
wolfgke
The whole existence of macro miners proves that playing MMOs is a job that is
better done by machines than humans. :-)

------
hartator
Does someone know where we can try it? Being browsing the web a but without
success.

~~~
garrettgrimsley
If you have WeChat then you can find it by searching "ms-xiaoice" or "小冰" but
you won't get anywhere if you don't speak Chinese.

------
mt_caret
Wow, that's interesting. Microsoft deployed a similar chatbot over here in
Japan too, called "Rinna" which anyone can chat with over LINE, a messaging
service popular here.

------
cypher543
All of those responses sound like something that could come out of an AIML[1]
bot and those have been around for almost 15 years. I fail to see how any of
this is revolutionary.

1: [https://en.wikipedia.org/wiki/AIML](https://en.wikipedia.org/wiki/AIML)

~~~
oldmanjay
Since no one claimed it to be revolutionary, I hope you aren't beating
yourself up over that embarrassing personal failure.

~~~
spdustin
The hyperbole in the article title suggests a revolutionary advancement, or at
least the next evolution of chat bots.

Were you intending to channel a sarcastic chat it yourself? It's unclear. For
earlier today, you said, "No, it's much better to avoid posting sarcasm at
all. It nearly never scans, and it doesn't really fit the HN culture in any
case."

Are we taking a Turing test now, with you?

------
steve371
From its current stage, you can not have even couple sentences conversation
with it. And what you have said before does not matter in most cases, unless
it is few designed subjects.

The best it can provide is still simple one question, one answer.

I'd like to see after another a year or two, anything will change.

------
jonah
It seems like Lenny[1] is as good as, if not more effective than most of these
chat bots.

[1]
[https://www.youtube.com/playlist?list=PLduL71_GKzHHk4hLga0nO...](https://www.youtube.com/playlist?list=PLduL71_GKzHHk4hLga0nOGWrXlhl-
i_3g)

------
andrelaszlo
"We can now claim that Xiaoice has entered a self-learning and self-growing
loop."

That's comforting.

------
EwanG
Article about Microsoft's Xiaoice - But apparently still no English Language
interface... :-(

~~~
eric_h
Which I find rather odd. Is written Chinese easier to generate,
computationally speaking?

~~~
seanmcdirmid
Disclaimer, I work for Microsoft in China but I don't know much about this,
definitely not involved in it!

Why it is in Chinese probably has more to do with the interests of the
producers (China Bing, researchers here also) and the consumers they are
targeting than with respect to any tech limitations.

~~~
eric_h
Thanks, I suspected it was more of a market than technical reason; since I
know very little about the chinese language I thought i'd ask.

------
discordance
This bot is also available in Japan under the name Rinna on Line Messenger

~~~
tellarin
There is also a Korean version, but I forgot its name.

