On: Open source deep learning models that programmers can download and run first try

[nilkn]

Once you take calculus 3, I'd recommend diving right into the Deep Learning book: http://www.deeplearningbook.org/

It's definitely the best reference on the subject. With only calculus 3 under your belt the math won't be trivial, but it should overall be fairly approachable and certainly much more so than something like "The Elements of Statistical Learning".