

Deep Learning Reading List - jmozah
http://jmozah.github.io/links/

======
AndrewOMartin
Don't forget "Intriguing properties of neural networks", otherwise known as
"Does Deep Learning have deep flaws?".

In sum, you can teach a network to say "that's a dog" when presented with a
picture of a dog, but you'll also be able to (1) find an imperceptibly
modified version of the image where it'll say "that's garbage" and (2)
intelligently generate an image of noise that also gets recognised as a dog.

(1) [http://arxiv.org/abs/1312.6199](http://arxiv.org/abs/1312.6199) (2)
[http://www.newscientist.com/article/dn26691-optical-
illusion...](http://www.newscientist.com/article/dn26691-optical-illusions-
fool-computers-into-seeing-things.html)

~~~
HCIdivision17
In (2)'s paper [2], the images are really neat. Staring at it for a few
moments, I feel like I can see where the computer is coming from. A bit.

The robin, armadillo, centipede, peacock, and bubble all actually have a
little swirl of features that - to me at least - resemble the labels provided.
But from afar, and you've got to basically damp the noise. I did this by
taking my glasses off and leaning about 8 inches from the screen (with the
grid taking about 3 inches wide). I've got about -7.5 diopter near-
sightedness, so this cleaned the images right up. At least the armadillo I
would have guessed, as it's absolutely a little critter walking to the lower
right, and it has the demi-circle body and long face. And it might make sense:
it was told to make an armadillo, so it did. _And nothing else_.

(I also think the baseball was super clever - if this were abstract art, I
would totally have fallen for that classification, as well as a few of the
others. There's something cool going on there.)

I'm printing and reading the rest of the paper now (about a quarter done). To
say it's both fascinating and interesting really understates it. Really neat
stuff!

[2]
[http://arxiv.org/pdf/1412.1897v2.pdf](http://arxiv.org/pdf/1412.1897v2.pdf)

------
therobot24
You're going to get most of these from a simple Google search, if you're going
to build a list of what to read you should at least put some effort into it.
Currently, this list is missing a lot of history behind deep learning - only 3
papers listed!

if you want a good set of papers that starts with perceptrons and hebbian
learning to multi-layered neural nets and the emergence of what we now refer
to as deep networks checkout
[http://deeplearning.cs.cmu.edu/](http://deeplearning.cs.cmu.edu/)

~~~
jmozah
I agree. Thanks for the link. I have added stuff that i started with. will
improve this list over time.

~~~
jmozah
Added the link to the list..

~~~
amund
Perhaps of interest -
[http://deeplearning.university](http://deeplearning.university) \- provides a
bibliography of recent publications. See also
[https://news.ycombinator.com/item?id=8334875](https://news.ycombinator.com/item?id=8334875)

------
sabalaba
I would add "Practical recommendations for gradient-based training of deep
architectures" to the list for those who already have a feel for training
multi-layer neural nets. It provides a good overview for those that want to
learn more about gradient descent, hyperparameter tuning, and other practical
considerations involved with training deep architectures.

[http://arxiv.org/abs/1206.5533](http://arxiv.org/abs/1206.5533)

------
sushirain
Karpathy's JavaScript demos are the first thing that I would show to a
newcomer:
[http://cs.stanford.edu/people/karpathy/convnetjs/](http://cs.stanford.edu/people/karpathy/convnetjs/)

------
ratsimihah
Automatic Speech Recognition: a Deep Learning Approach contains an excellent
section about deep learning, as well as more content about ASR and hybrid deep
learning methods.

~~~
jmozah
can you post the link?

~~~
ratsimihah
[http://www.springer.com/engineering/signals/book/978-1-4471-...](http://www.springer.com/engineering/signals/book/978-1-4471-5778-6)

------
userbinator
For those interested in learning deeply about deep learning?

