An Introduction to Deep Learning for Language Modeling

[oDot]

http://ofir.io/Neural-Language-Modeling-From-Scratch?a=1

[make3]

these lectures at Oxford on the subject (by some research scientist it deepmind, no less) are some of the best spent time in my career

https://github.com/oxford-cs-deepnlp-2017/lectures

[timClicks → make3]

Wow, that's a great find. How do people chance upon gold dust like this?

[avinium → make3]

One of the few times I bother to login to HN - just to say thank you! Don't know how I've never come across these before, but they look fantastic.

[p1esk → make3]

Are they better than 224d at Stanford?

[make3 → p1esk]

definitely more advanced. they don't cover neural nets 101 like cs224d does, so they have more time to do advanced subjects like neural machine translation, neural Turing machines and question answering in the last courses.

honestly i would just watch both, and start with cs224d, but if I would have to pick one, it would be Oxford's.

[woodson → make3]

Also, check out Graham Neubig's NN for NLP class at CMU: http://phontron.com/class/nn4nlp2017/schedule.html It covers a lot of current stuff. The class is still running, so not all videos/materials are there yet.

[brian_spiering]

Here is my listicle about Deep Learning for NLP https://github.com/brianspiering/awesome-dl4nlp

[peterburkimsher]

I'm making a Chinese-English parallel translator.

https://pingtype.github.io

My boss asked me to take CS231n about Deep Learning, and I decided to put some of my Chinese data into the same tools (Word2Vec) to see how it performed.

Honestly, it sucked. Was that my bad coding (very possible). Or is it because the examples are cherry-picked to work well with English? Can someone recommend a tutorial of Deep Learning for languages that are not English?

[wilde → peterburkimsher]

Try stuff from Baidu. They claim good performance in Chinese.

[mozumder]

Has language modeling achieved something that can pass a Turing test yet for various audiences?

[dmreedy → mozumder]

Simply, no.

Most of big advances in the past five years have been around Natural Language 'Parsing' of unstructured information into structured information (this is what Deep Learning is very good at, so far at least); classifying unstructured utterances into structured intents, mapping unstructured utterance 'questions' to unstructured answers in a corpus (fundamentally similar search), literal parsing of utterances to shallow or deep (semantic) parses, and so on. The thoroughly end-to-end, multi-modal and multi-scoped task of participating in a conversation remains largely unsolved. As any non-primitive interaction with Siri or Alexa should indicate. Word of the day is "brittle".