Hacker News new | past | comments | ask | show | jobs | submit login
Ask HN: Calculus Refresher for AI/ML?
10 points by fnord77 3 months ago | hide | past | favorite | 6 comments
From what I gather, much of AI is gradient descent, which appear to be matrices of PDEs.

I took calculus/linear algebra/PDEs decades ago, but remember little of it. Are there any resources for relearning the parts useful for AI in an efficient manner?




The Matrix Calculus You Need For Deep Learning

https://arxiv.org/abs/1802.01528

From the intro:

>This article walks through the derivation of some important rules for computing partial derivatives with respect to vectors, particularly those useful for training neural networks. This field is known as matrix calculus, and the good news is, we only need a small subset of that field, which we introduce here. While there is a lot of online material on multivariate calculus and linear algebra, they are typically taught as two separate undergraduate courses so most material treats them in isolation. The pages that do discuss matrix calculus often are really just lists of rules with minimal explanation or are just pieces of the story.

>In contrast, we're going to rederive and rediscover some key matrix calculus rules in an effort to explain them.



imo linear algebra is the main thing to refresh yourself on, as most of NN is based on matrix multiplication. The calculus is mostly in the gradient descent which in practice is abstracted away, so having a few decades old understanding might be good enough. For practical implementation Andrej Karpathy has a good series of videos going over a lot of the concepts starting here https://www.youtube.com/watch?v=VMj-3S1tku0


I liked this (and I fixed a few bugs in it too): https://explained.ai/matrix-calculus/


Stewart calculus, Paul's online notes, Lay for linear


I suppose I never learnt Calculus in it's Essence, that is, trying to answer The questions that brought us Calculus in The first place.

So take this answer with that mind.

My opinion is for You to read artificial intelligence a modern approach fourth edition by Peter norvig and Stuart Russell (it incluyes chapters by Ian Goodfellow and Judea Pearl).

The moment You encounter symbols You don't understand What they are saying You can Start diggi g on the topics explaining those symbolic forms, at some point You may find youtself revisiting Algebra.

But if I have to recommend a book on Calculus that would be Calculus Made easy by Silvanus P. Thompson (which I havent finished).




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: