
Evaluating Natural Language Understanding Services [pdf] - jjwiseman
https://wwwmatthes.in.tum.de/file/6djp1mgfl78x/Sebis-Public-Website/-/Evaluating-Natural-Language-Understanding-Services-for-Conversational-Question-Answering-Systems/SIGDIAL22.pdf
======
jjwiseman
"From a high-level perspective, LUIS performed best with an F-score of 0.916,
followed by RASA (0.821), Watson Conversation (0.752), and API.ai (0.687).
LUIS also performed best on each individual dataset: chatbot, web apps, and
ask ubuntu. Similarly, API.ai performed worst on every dataset, while the
second place changes between RASA and Watson Conversation."

------
valentinvieriu
Just in time. I was just researching this today. Quick question ( maybe not
related). Is there a good open source solution for Entity Extraction, and more
exactly mapping it to Wikipedia, ( Like GOOGLE NATURAL LANGUAGE API ) ?

~~~
jjwiseman
Rasa, evaluated in the paper, can do entity extraction.

Other open source libraries that can do entity extraction: OpenNLP
([https://opennlp.apache.org/docs/1.5.3/manual/opennlp.html#to...](https://opennlp.apache.org/docs/1.5.3/manual/opennlp.html#tools.namefind.recognition)),
Stanford NLP ([https://nlp.stanford.edu/software/CRF-
NER.shtml](https://nlp.stanford.edu/software/CRF-NER.shtml)), MALLET
([http://mallet.cs.umass.edu/](http://mallet.cs.umass.edu/)),

~~~
valentinvieriu
Maybe my question was misleading. I'm more interested, for my current project
on mapping those entities with Wikipedia articles ( see
[https://goo.gl/mNdDRD](https://goo.gl/mNdDRD) ) or try the
[https://cloud.google.com/natural-language/](https://cloud.google.com/natural-
language/) example

