

Ask HN: Good books on machine learning? - trominos

I'm trying to learn about methods of machine learning. Do you guys have any suggestions for books or sources I should use?
======
martian
A very readable _introduction_ to machine learning and recommendation systems
is Programming Collective Intelligence, by Toby Seagram (O'Reilly). Worth
looking at if you're just getting your feet wet.

------
mstoehr
There really isn't very much available on the practical side. So if you are
looking to implement algorithms I suggest that you make use of the machine
learning at ocw.mit.edu

Alternatively, if you want a good dose of a theoretical explanation of
algorithms currently in use I highly recommend "Pattern Recognition and
Machine Learning" by Christopher Bishop. It is definitely the best machine
learning (and statistics) textbook that I have ever come across.

~~~
bravura
I second the nomination of Bishop. It is the standard text. It is only two
years old, and Bishop will teach you machine learning the way that the field
practices it nowadays. In my lab of fourteen people, we must have six or so
copies of Bishop.

I don't understand what is impractical about Bishop. If you are looking
blindly to use an off-the-shelf machine learning implementation, that's one
thing. Machine Learning has been described as the study of bias. If you want
to understand when to pick certain techniques, and develop appropriate biases,
then read Bishop.

"The Elements of Statistical Learning" by Hastie, Tibshirani and Friedman
gives more of a statistician's approach. The treatment is simply less broad,
and also more dated.

You can also look at Andrew Ng's video lectures:
<http://www.youtube.com/watch?v=UzxYlbK2c7E> He is very well-respected in the
field. For certain students, watching a lecture may be preferable to reading a
book.

~~~
pmbouman
A few other things (sorry, not to snipe too much! :) )

-I'm skeptical of the idea of a single "standard text" in such a fast-moving field. New machine learning techniques appear constantly and are often documented online years before they appear in books. Some computer scientists say they prefer conference proceedings over academic journals because the latter take so long.

-Further, I'm not sure that the goal of any text should be to cover topics X, Y and Z in any case, which doesn't seem possible for a book to do. What does seem feasible is to set up a framework for analyzing the performance of different techniques. So I'd like to hear a comparison of how Bishop does that vs. HTF.

-You're of course correct that HTF takes a statistician's POV on the field - the authors are all professors of statistics at Stanford. They are also accomplished - Friedman was a co-author on CART, for example. I would instead ask the question: what can you get out of the book and the framework it offers?

-I think that part of the framework in machine learning is to think about bias AND variance, and how to trade them off successfully. This is an important part of model selection, for example.

------
pmbouman
Try "The Elements of Statistical Learning" by Hastie, Tibshirani and Friedman.
Lots of math but an outstanding introduction.

------
earl
Here are a set of links:

<http://www.vetta.org/recommended-reading/>

I'd second the recommendation of Bishop if you can hack the math, and also
Elements of Statistical Learning, though I wouldn't attempt to learn
techniques from the latter so much as look at a very interesting mathematical
take on them.

gl

