Hacker News new | past | comments | ask | show | jobs | submit login
Information Theory for Machine Learning [pdf] (github.com)
55 points by rabidsnail on June 12, 2016 | hide | past | favorite | 3 comments

As a learning exercise/fun project for the author, I think this is ok.

But for any serious study, I fail to see what this offers over the completely free, very readable, more carefully written, and more thorough "Information Theory, Inference, and Learning Algorithms" by David MacKay: http://www.inference.phy.cam.ac.uk/itila/book.html.

As an example of the problems with the pdf in its current stage, Theorem 2 (asymptotic source coding) includes the term "negligible loss" without even defining what "loss" means in source coding. Lossless and lossy coding are very different things, all the preceding stuff is really discussing the lossless coding problem. Pedagogically, these need decoupling.

RIP David and thanks for the great book...

Bookmark comment; ignore.

Applications are open for YC Winter 2021

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact