The Little Learner is a great book. I tried rewriting all the code in Python/JAX while following the Scheme code style as closely as possible, and it worked out great.
The appendix on autodiff is a bit rushed, in my opinion. But in all fairness, the number of pages would probably need to be doubled to give a proper exposition of autodiff.
Yeah, it's the easiest way to get a beachhead in deep learning and then expand from there. I dislike their heavy use of currying, it's elegant in theory but bad error messages make it confusing and inconvenient in practice. But it's a small tradeoff for an otherwise excellent book.
The sample sounds impressive, but based on their claim -- 'Streaming inference is faster than playback even on an A100 40GB for the 3 billion parameter model' -- I don't think this could run on a standard laptop.
The appendix on autodiff is a bit rushed, in my opinion. But in all fairness, the number of pages would probably need to be doubled to give a proper exposition of autodiff.
reply