

Deep Learning for NLP without Magic (2013) - jbarrow
http://www.socher.org/index.php/DeepLearningTutorial/DeepLearningTutorial

======
kastnerkyle
This tutorial is excellent, and Richard Socher has been doing some really
fascinating (though complicated) work with deep learning for NLP. I also have
a few other good references for people who want to investigate this area.

One of the most insightful comments I have seen about deep learning for NLP is
a reddit r/machinelearning comment seen in this post [1]. As the poster in
that thread mentioned, much of the theory about using neural networks for NLP
tasks was laid out pretty clearly in a paper by Yoshua Bengio from 2003 [2].
At a high level, the basic learner is very similar to many of the other deep
architectures for images and audio.

There is also a pretty great blog post [3] by Radim Rehurek (author of gensim)
about how he implemented word2vec (a "deep learning"-style model for text) in
Python, while also getting a performance improvement over the C version!

[1]
[http://www.reddit.com/r/MachineLearning/comments/2doi7e/what...](http://www.reddit.com/r/MachineLearning/comments/2doi7e/what_should_i_know_before_taking_on_word2vec/)

[2]
[http://jmlr.org/papers/v3/bengio03a.html](http://jmlr.org/papers/v3/bengio03a.html)

[3] [http://radimrehurek.com/2014/02/word2vec-
tutorial/](http://radimrehurek.com/2014/02/word2vec-tutorial/)

------
anko
I would really like to do some neural network stuff, but i really have trouble
reading the math. For example, they have a maxent classifier in the slides but
I'd have no idea how to convert that to code.

Any resources to learn that stuff?

~~~
kastnerkyle
There is a nice tutorial here [1] about max-ent classifiers. Ultimately in
neural networks there are usually a number of cost functions you can use for
the last layer - softmax or cross-entropy are other possibilities that may be
easier to understand, though possibly less performant for NLP tasks.

I thought the introduction to neural networks from Andrew Ng's coursera course
(even though it meant writing MATLAB) was quite good, and allows you to
implement backprop, cost functions, etc. while still having some other helper
code to make things easier. I highly recommend working through that course if
you are intersted in ML in general [2].

[1] [http://www.cs.berkeley.edu/~klein/papers/maxent-tutorial-
sli...](http://www.cs.berkeley.edu/~klein/papers/maxent-tutorial-slides.pdf)

[2] [https://www.coursera.org/course/ml](https://www.coursera.org/course/ml)

