
Information Theory for Machine Learning [pdf] - rabidsnail
https://github.com/mtomassoli/papers/blob/master/inftheory.pdf
======
gajjanag
As a learning exercise/fun project for the author, I think this is ok.

But for any serious study, I fail to see what this offers over the completely
free, very readable, more carefully written, and more thorough "Information
Theory, Inference, and Learning Algorithms" by David MacKay:
[http://www.inference.phy.cam.ac.uk/itila/book.html](http://www.inference.phy.cam.ac.uk/itila/book.html).

As an example of the problems with the pdf in its current stage, Theorem 2
(asymptotic source coding) includes the term "negligible loss" without even
defining what "loss" means in source coding. Lossless and lossy coding are
very different things, all the preceding stuff is really discussing the
lossless coding problem. Pedagogically, these need decoupling.

~~~
cttet
RIP David and thanks for the great book...

------
mike_hock
Bookmark comment; ignore.

