
Natural Language for Developers - Perados
https://wit.ai/
======
freen
In it's current implementation, Wit.ai is broken.

I built a bot recently on it and it was one of the worst experiences I've ever
had integrating into a third party API.

By a long shot.

1) chat logic is declared on their buggy web platform, which is inconsistent
if you use multiple tabs, and doesn't actually indicate current system status.
I had a number of "bugs" whereby my "story" was correct, but attempts to test
it failed and only after I gave up and let it be for a few hours did it start
to work. Meaning stories that were broken would all of a sudden be marked
green after a few hours. Flows that failed would start to work given some
unclear quantity of time. Oh yeah, and the opposite too: things that worked
while futzing around with their interface would potentially break after a few
hours. Delightful.

2) Documentation is horrific. It's not clear from any of their examples how to
build anything other than a toy to turn on and off lights. If you can figure
out how to build a "yes-or-no" entity that follows two different paths from
their "recipes" I will buy you a drink ($10 USD and for the first five
people). Just provide proof and a BTC address.

3) Flow control is next to impossible to implement, and when you do, it's
totally opaque as to why the bot actually chose one flow over the other.

4) can't currently version your stories, which means you can develop new
features without breaking production.

6) their support is horrendous. I have four outstanding questions, all
unanswered after, oh, two weeks?

The combination of all of the above means it's INCREDIBLY brittle, hard to
reason about and extremely time consuming to develop against.

If Facebook wants people to build bots for their messenger platform, open
source wit.ai, and then pay people to develop it. This current state of
affairs is untenable, morally questionable for all of the reasons outlined
elsewhere in the thread (which have gone unsatisfactorily answered by apparent
Facebook employees who should know), and fundamentally broken.

------
iheredia
So, the site has this demo: [https://wit.ai/getting-
started](https://wit.ai/getting-started) First two queries it shows are "Turn
on the lights" and "Open the door please!" showing how nice the response is.
If you combine them in "Turn on the lights and open the door please!" then the
reponse ignores the door part completely: { "_text": "Turn on the lights and
open the door please!", "confidence": 0.94, "intent": "lights", "entities": {
"on_off": [ { "value": "on" } ] } }

~~~
dominotw
"wash the car" , "feed the cat" results in

    
    
      "intent": "tv_onoff",
    

seems wrong for such simple example.

~~~
cyberferret
Similarly, when I input "make me a sandwich", I got "tv_onoff" as the intent.

I am assuming however, that you can "train" the AI to suit your particular
syntax depending on your app and bot's intended purposes. No doubt as more
scenarios are fed into Wit's database, the parsing and intent selection will
get better?!?

~~~
coroxout
I typed "Translate how much does that cost to French", which was one of the
examples it showed me on the front page (under "Intent Parser", exactly as
shown, no quotes), and it still gave me tv_onoff.

And "Turn on the TV please" is interpreted as "Turn the TV on to channel
'please'".

And don't invite any physicists round, because "Set the temperature to 293 K
in my bedroom" results in a "temperature" value of 293000.

Still, the docs do indeed say you can train it (for sentiment and yes/no
queries, at least, and hopefully other things too).

------
JustinBlock
It seems like Facebook built this so they can feed the data from people using
their api to train their eventual messenger AI/assistant.

~~~
jventura
I don't know why you are being downvoted, but this reminds me that thing about
facebook/google/etc. that if there is no product you are the product. It also
came to my mind that they may be after your users data for their own models..

~~~
majewsky
They "may be"? As far as I heard, Google is pretty clear about using spoken
language input for their voice recognition API for precisely this purpose.

------
anilshanbhag
Correct me if I am wrong, but this approach essentially recognizes entities
and matches it to a rule. This is naive ! There are better approaches which
work based on parse tree structure or use a PCFG to encode rules

~~~
IshKebab
Can you link to these better approaches?

~~~
majewsky
PCFGs are about the first thing that I learned about in my Stastical Machine
Translation course. It's like a regular context-free grammar (see Wikipedia if
you're not familiar with these), but each production is assigned a
probability, such that all productions from the same non-terminal sum up to
probability 1. Then you can assign a probability to a parse tree by
multiplying the probabilities of all productions that were used (the model
assumes that all productions are mutually statistically independent).

After a proper training run, you have your ambiguous grammar which, given an
input sentence, will produce a set of parse trees, and assign a probability to
each of them. You will typically choose the most probable one.

The Wikipedia article on PCFG is surprisingly biased on RNA/protein analysis
and does not go into much detail on how to train these PCFGs. But it's
basically just an EM algorithm using inside-outside weights. See for example
[https://link.springer.com/chapter/10.1007/978-3-540-78291-9_...](https://link.springer.com/chapter/10.1007/978-3-540-78291-9_7)

~~~
IshKebab
This requires defining matching patterns, like a word-based regex though,
right? Houndify uses that approach because it is easier to integrate with
speech recognition, but it is very cumbersome to use and maintain and doesn't
allow learning new utterances.

I don't think Wit.ai works as simply as you believe - it isn't merely matching
keywords. Unfortunately I haven't been able to find out exactly how it does
work. Alexa, and Nuance Mix work the same way as Wit.ai. In my experience it
is the superior approach.

Viv is also one to watch but they haven't really said the NLU works yet.

------
dfischer
I've been using Wit and enjoy it. The site could improve in terms of usability
and performance. It definitely feels buggy at times.

I'm not using their story framework though. Just their intents parser. Using
wit entirely for the conversational layer doesn't seem like a good way to go
about things.

Overall I'm happy though.

\- I wish I could download my data set at will.

~~~
francis88
We will be releasing an import/export in the next 2 weeks

~~~
dfischer
Wonderful! Thanks!

------
rkwz
I've used nlp-compromise.js for building an ecommerce shopping assistant bot.
It's a bit rough around the edges but works most of the time.

Demo: [http://nlp-compromise.github.io/website/](http://nlp-
compromise.github.io/website/)

Github: [https://github.com/nlp-
compromise/nlp_compromise](https://github.com/nlp-compromise/nlp_compromise)

~~~
jbnicolai
You seem to have commited a merge conflict in [https://npmcdn.com/nlp-
syllables@0.0.4/builds/nlp-syllables....](https://npmcdn.com/nlp-
syllables@0.0.4/builds/nlp-syllables.js)

Library looks fantastic!

------
joshgel
I have been using this with
[https://jasperproject.github.io/](https://jasperproject.github.io/) on a
raspberry pi to make an open source amazon echo type device. It works pretty
well for controlling basic things around the house.

------
personjerry
How does wit.ai make money?

~~~
deet
Wit.ai has been around for a while now, and used to charge a monthly fee for
some private models. It was then acquired by Facebook and made free for all
uses.

~~~
personjerry
I guess it would really improve peoples' trust in wit.ai (and thus the number
of users) if they would declare their plans for monetization or policy
otherwise for the future. Otherwise developers might not want to see their
work become invalid when the platform suddenly costs money, or something.

~~~
stephenr
It's owned by Facebook.

What part of their monetisation plan (privacy whoring to sell ads) is not
apparent to anyone who cares to look by now?

~~~
personjerry
That would be rather short-sighted. Different platforms should be monetized
differently, and Facebook has shown that it is willing to do this, such as
with WhatsApp, which isn't filled with ads: [https://www.quora.com/How-does-
WhatsApp-make-money-especiall...](https://www.quora.com/How-does-WhatsApp-
make-money-especially-in-India)

With wit.ai, I would be surprised if they decided "privacy whoring to sell
ads" is the right way to go.

~~~
stephenr
From that very page

> Instead, the company said it will explore ways businesses can use WhatsApp
> to connect with individuals, and will introduce new ways for users to
> communicate with businesses and organisations that will pay the company to
> target relevant communications with customers.

>> __organisations that will pay the company to target relevant communications
with customers __

So, their plan is more targeted advertising in the form of "offers" or "deals"
etc.

~~~
personjerry
Ah, I think you may have misunderstood. Below those lines, the examples given
on the Quora link are:

> A bank could use WhatsApp paid account to communicate with its customers
> about recent transactions and necessary fraud warnings.

> An airline could use WhatsApp paid account to contact its passengers about a
> delayed schedule or cancelled flight.

I would argue that these do not seem like ads.

~~~
stephenr
Right. Just like my telephone company says it might want to send me SMS
notices about when a bill is due, and then actually sends me SMS notices about
every fucking promotion under the sun from their sister companies, in a
language I can't read.

I honestly do not understand the logic of people who look at a company, wholly
owned by Facebook, and give them the benefit of the doubt when it comes to
business tactics, privacy, etc.

------
aibottle
Would never use that. It's free of charge, so how is it financed? Right, its
Facebook owned and just like every product of (Google, Facebook, Apple,
whatever) if if cannot pay for it I won't use it, they eventually stop
offering the service someday because they change their plans. Don't want to be
forced to rewrite my app because of their change of mind.

~~~
cyberferret
These are my fears too. I've been looking at this over the weekend as a
possibility to embed a 'bot like feature into my web app, but I would hate my
users to get used to it, then have Wit go away like Parse did. Much harder to
develop an AI engine from scratch myself than an authentication/database
engine.

~~~
HillRat
As far as I've seen, this isn't anything you couldn't do with basic
classification and NER passes in OpenNLP. Straightforward, even trivial, stuff
as far as existing NLP toolsets go.

~~~
deet
Wit seems to be heading towards more conversation-oriented (ie multi-message)
approach to NLP which is something existing NLP toolsets don't make
straightforward. For example, predicting the next action a bot should take
based on conversation history is a bit different than just classifying an
utterance. There are hints at how to do this in literature, and Init.ai
(disclaimer: I work there) is working on it as well, but it's not widespread.

There's also more to this type of slot filling than just NER. Again, the
necessary techniques are available in academic literature, but not necessarily
turn key. Plus, you need to handle parsing after locating the slots. For the
parsing, you can take a look at Duckling
([https://duckling.wit.ai/](https://duckling.wit.ai/)) which Wit did open
source.

Both on the classification front and the slot filling, open source toolkits
might get you part of the way there but not all the way.

Having a training and management UI is also a substantial value add once you
use it.

------
meeper16
What we now need is a real AI engine.

