For each problem, he explores some salient ideas or ways to address the issue.
* Theory: We don't always have good explanations for why it works.
* Reasoning: Stick a CRF on top of a Deep Net
* Memory: We need a "hippocampus". Memory networks, neural embeddings.
* Unsupervised Learning: How do we speed up inference in a generative model? Sparse autoencoders, sparse models...
For those who could use an overview of neural nets and how some of them work, this may be useful: http://deeplearning4j.org/neuralnet-overview.html