For anyone interested in more about the specific math of neural networks, http://www.iro.umontreal.ca/~bengioy/dlbook has a couple good introductory chapters that give overviews of most of the necessary topics for NNs, but also provides additional resource suggestions if you need more in-depth info on a certain subject.
