Instead, learn decision trees and more importantly enough statistics so you aren't dangerous.
Do you know what the central limit theorem is and why it is important? Can you do 5-fold cross validation on a random forest model in your choice of tool?
Fine, now you are ready to do deep learning stuff.
The reason I say not to do neural networks first is because they aren't very effective with small amounts of data. When you are starting out you want to be able to iterate quickly and learn, not wait for hours for a NN to train and then be unsure why it isn't working.
Of course it's important to get a broad horizon eventually but starting with the theory without the applications is not how most humans learn best. Learning by doing is.
The problem with diving into neural networks is that they are slow to train (with large amounts of data anyway), and difficult to debug. This means it isn't really a great place to start.
"All of statistics" is really a great book if you have time work through he exercise.
http://www.inference.phy.cam.ac.uk/itila/book.html (freely accessible online)