
AllenNLP – An open-source NLP research library, built on PyTorch - varunagrawal
http://allennlp.org/
======
TekMol
Wow, is this really state of the art?

    
    
        Joe did not buy a car today.
        He was in buying mood.
        But all cars were too expensive.
    
        Why didn't Joe buy a car?
    
        Answer: buying mood
    

I think I have seen similar systems for decades now. I thought we would be
further along meanwhile.

I have tried for 10 or 20 minutes now. But I can't find any evidence that it
has much sense of syntax:

    
    
        Paul gives a coin to Joe.
    
        Who received a coin?
    
        Answer: Paul
    

All it seems to do is to extract candidates for "who", "what", "where" etc. So
it seems to figure out correctly that "Paul" is a potential answer for "Who".

No matter how I rephrase the "Who" question, I always get "Paul" as the
answer. "Who? Paul!", "Who is a martian? Paul!", "Who won the summer olympics?
Paul", "Who got a coin from the other guy? Paul!"

Same for "what" questions:

    
    
        Gold can not be carried in a bag. Silver can.
    
        What can be carried in a bag?
    
        Answer: Gold

~~~
galenko
Sadly, the NLP world is full of hot air. I've seen so many companies get
funding for complete "written by a 12-year old" dogshit "industry leading IP",
it's not even funny anymore.

The hype has gone down and some are actually doing great work, but 90% of the
people who say they do NLP/AI stuff don't even fundamentally understand what
NLP/AI is.

~~~
tanilama
I still hold hope. However, it seems naively exploit the function
approximation capacity we have with deep learning can only go that far to
understand our own language.

Maybe we need to look back and start from beginning and ask ourself: How does
human learn, exactly? How do we learn with so few examples? How do we jointly
learn image/audio/video/language with only one brain?

~~~
amelius
Perhaps we should consider working only on techniques which improve as more
computational power is thrown at it.

~~~
glup
Computation won't help if we don't have the right representations. Arguably
computation can help us discover the right representations but the space of
possible representations is very, very large.

------
mamp
This is very brittle: it works really well on the pre-canned examples but the
vocabulary seems very tightly linked. It doesn't handle something as simple
as:

'the patient had no pain but did have nausea'

Doesn't yield any helpful on semantic role labeling and didn't even parse on
machine comprehension. If I vary it to say ask 'did the patient have pain?'
the answer is 'nausea'.

CoreNLP provides much more useful analysis of the phrase structure and
dependencies.

------
sanxiyn
In "Adversarial Examples for Evaluating Reading Comprehension Systems"
[https://arxiv.org/abs/1707.07328](https://arxiv.org/abs/1707.07328), it was
found that adding a single distracting sentence can lower F1 score of BiDAF
(which is used in demo here) from 75.5% to 34.3% on SQuAD. In comparison,
human performance goes from 92.6% to 89.2%.

~~~
andrew3726
There's a blog post (the morning paper) about this:
[https://blog.acolyer.org/2017/09/13/adversarial-examples-
for...](https://blog.acolyer.org/2017/09/13/adversarial-examples-for-
evaluating-reading-comprehension-systems/)

------
vbuwivbiu
"the squid was walked by the woman"

"what is the fifth word in that sentence ?"

Answer: squid

------
strin
We need more demos of AI models: there is what people claim their model does,
and there is what the model actually does.

------
wyldfire
How does this compare with spacy?

~~~
glup
Different set of tasks. SpaCy is focused on bread-and-butter tasks like
tokenization, part of speech tagging, and dependency parsing (not to say that
these are easy, but that they are things people have been working on a long
time). AllenNLP seems focused on distributing relatively recent neural models
(last few years) of more complex language understanding like labeling semantic
roles (agents, patients, etc.) and identifying textual entailments (=mining
facts from a sentence). It is not great at these tasks, because this is v.
difficult and a very active area of ongoing research.

