May I also recommend a related book, also a free PDF:
http://www.inference.phy.cam.ac.uk/mackay/itila/
"Information theory, inference, and learning algorithms" by David Mackay. This book is not a pure machine learning book but it is a very fun read.
A lot of the Mackay book is on information/coding theory and while it will deepen an existing understanding of ML, its probably a roundabout introduction. ESL is a much better intro, especially for someone looking to apply ML.
That said, it is meant for people who are comfortable with math/stats; its much more statistics oriented than, say, Mitchell's book. But they do a good job of explaining things in non-math language. This book does a good job of exposing high level ML concepts (e.g. bias-variance tradeoff) but still teaches a lot of the standard methods & tools.
These are actually two of my favorite books on the subject and I can't recommend them both enough.
I'm mainly interested in Machine Learning algorithms. I've read the first 75 pages of this book, and skimmed most of Mackay, and I prefer this one. It goes into a lot more detail on the performance of different algorithms, how the handle sparse data, error rates and high dimensionality.
McKay's looks like a good book, but appears more applicable to pure information theory rather than ML specifically.
We just covered Neural Nets in my machine learning class, and the professor made some comments about how far a good name can take you. Neural Nets are better than other learning algorithms in some ways and worse than others, but the name gives you the sense that you are learning the secrets of how the brain works. That may or may not be true, but it sure has served as good branding for the Neural Networks algorithm.
I had read a couple of pages in the book on linear regression and was very impressed by the way concepts had been explained. Although approach is very mathematical, I will recommend this book to anybody serious about Machine Learning
several years ago, i took a class from prof tibshirani using this book and was absolutely blown away. if you don't have a deep stats background (as i didn't), you might find it scarily dense, but wow there's some good stuff in there.
I also used this book for a data mining class that I took about two years ago. The book is very thorough but the math can be dense. We didn't actually end up using it in class very much, but it was a good reference book for the topics it contains.
I am starting to learn statistics. But I have zero knowledge in the subject. I have picked up a copy of Headfirst Statistics. Is that a good one? Any recommendations?
"All of Statistics" by Larry Wasserman is very good. It is traditional, fairly mathematical statistics, but intended for a general audience. (It is particularly popular with computer science people.)
It is also extremely concise, perhaps too much so occasionally for pedagogical ease. It really does come pretty close to covering "all of statistics" in 442 pages, an impressive feat. But as you can imagine it certainly doesn't beat around the bush much. So if you're looking for something with multiple applications, covering the same material from different perspectives, etc., or just want more hand holding, this may not be for you.
Since this book is so math heavy, can anyone recommend a free pdf book that can be used as a math intro for this? I like the book, but I get a little over my head with the math in some sections.
Another useful resource might be the 20 part lecture series on Machine Learning by Professor Andrew Ng for CS 229 in the Stanford Computer Science department. Youtube playlist URL - http://www.youtube.com/view_play_list?p=A89DCFA6ADACE599
