

Can anyone recommend good books, articles or essays that introduce machine learning? - jkush

I intend to find out for myself but I'm curious to know how steep a learning curve you've found it to be.

======
cschmidt
If you want a fairly easy read without too many equations, try:

Data Mining: Practical Machine Learning Tools and Techniques (Second Edition)
<http://www.cs.waikato.ac.nz/~ml/weka/book.html>

Which goes nicely with the Weka open source ML toolkit
<http://www.cs.waikato.ac.nz/ml/weka/>

(although it is a good read without the toolkit)

If you want a bit more math, I really like the recent (Oct 2007) book:

Pattern Recognition and Machine Learning by Christopher M. Bishop
[http://www.amazon.com/Pattern-Recognition-Learning-
Informati...](http://www.amazon.com/Pattern-Recognition-Learning-Information-
Statistics/dp/0387310738)

It is nicely self contained, going through all the stats you'll need.

~~~
mriley
I didn't particularly like the WEKA book, but Bishop's book is excellent.

If you're interested in introductory data mining stuff, I would recommend
Tan's Introduction to Data Mining: <http://www-
users.cs.umn.edu/~kumar/dmbook/index.php>

------
jmzachary
Machine Learning by Tom Mitchell for a serious academic book.

Programming Collective Intelligence by Toby Segaran for a practical approach
in Python.

~~~
queensnake
I disagree wrt Mitchell; too dry and out-of-date. In its place I'd recommend
the Norvig book which covers everything Mitchell does (IIRC) except 'Probably
Approximately Correct' (PAC) learning. And, it's more up-to-date (Support
Vector Machines) and is much more readable. Also, I haven't read it except a
flip-through at a bookstore, but Christopher Bishop's one I'd look at (perhaps
the Amazon comments).

~~~
neilc
I've seen the Norvig book recommended by others on this site for ML, but I
don't understand why: the Norvig book is an _AI_ book, not an ML book. It only
has a few chapters on learning, and IIRC much of that is on reinforcement
learning. Obviously the Norvig book is very well-written and is good
background for learning about ML, but I don't think it is a sufficient ML book
as such.

As far as ML goes, I found the "Programming Collective Intelligence" book to
very readable and practical, but very light on the theoretical foundations
(which is intentional, of course). I've got a copy of the Witten ML book
("Data Mining: Practical Machine Learning Tools and Techniques"), but to be
honest I haven't gotten much from it yet, either: it doesn't seem to discuss
SVMs in any detail, nor random forests or neural networks. But I haven't
really dug into it yet.

~~~
queensnake
This is kind of moot, truthfully I'd go, now (or look first at, but they're
both getting praise so...) for Chris Bishop or Ethem Alpaydin's new books. But
between Norvig and Mitchell, Norvig has 138 pages on learning vs Mitchell's
~390, but, Mitchell's is from a different era, Norvig is easier to read -
larger pages, more diagrams, better writing, and you know where you are better
- and fresher material. To each his own. But like I say, I'd probably go with
one of the even newer ones. For a while Mitchell was all there was, then
Norvig came along, and now there're a few to choose from.

------
wastedbrains
Artificial Intelligence A Modern Approach SE, by Stuart Russell and Peter
Norvig.

I am a big fan of Peter Norvig in particular, he has a ton of great essays and
code available online: <http://norvig.com/>

------
pcowans
Information Theory, Inference and Learning Algorithms by David MacKay is
excellent, and freely available online.

<http://www.inference.phy.cam.ac.uk/mackay/itila/book.html>

~~~
mriley
I'll second this recommendation - I bought the printed copy, and I'm
constantly going to it for reference. The fact that it's available for free is
just an added bonus.

I would also suggest Elements of Statistical Learning: <http://www-
stat.stanford.edu/~tibs/ElemStatLearn/>

As well as Duda, Hart, and Stork's Pattern Classification:
<http://rii.ricoh.com/~stork/DHS.html>

------
at
books about machine learning applications/tools: \- "Programming Collective
Intelligence" \- <http://www.oreilly.com/catalog/9780596529321/> \- "Data
Mining: Practical Machine Learning Tools and Technique" \-
<http://www.cs.waikato.ac.nz/~ml/weka/book.html>

books about machine learning background/theory: \- "machine learning" \-
<http://www.cs.cmu.edu/~tom/mlbook.html> \- "learning and soft computing" \-
<http://cognet.mit.edu/library/books/view?isbn=0262112558>

For a simple-to-use (Python-based) machine learning tool/API check out Orange:
<http://magix.fri.uni-lj.si/orange/>

------
jorgeortiz85
Check out the lecture notes for CS229, Stanford's class on Machine Learning.
The class has no textbook, so the notes are fairly comprehensive. (Though the
math notation can get a bit intense...)

<http://www.stanford.edu/class/cs229/materials.html>

~~~
jorgeortiz85
Also, the notes assume a reasonable knowledge of probability theory and linear
algebra. If you're unsure, you might do well to review those topics before
approaching machine learning (or at least keep good references handy). (Edit:
the link above has Section Notes that review probability theory, linear
algebra, and convex optimization. You might find those useful.)

And it's not an easy course. Don't be discouraged if the problem sets seem
impossible. (They very nearly are.)

------
alec
Sutton and Barto's book on reinforcement learning is available online at
<http://www.cs.ualberta.ca/~sutton/book/the-book.html>, and is probably the
definitive work on the subject.

~~~
queensnake
I wish they'd make a new edition; but, probably, the field is moving too fast.

------
DaniFong
A nice read early on would be "Elephants don't play chess" but Rodney Brooks.

<http://people.csail.mit.edu/brooks/papers/elephants.pdf>

~~~
Tichy
I stopped reading on page 3, because it seems to me that the "physical
grounding hypothesis" is complete bullshit. Where is the "physical grounding"
aspect if I want to build a search engine for the web? It's all just
information.

Not saying that getting physical can not yield interesting results, but to
state that it is the only possible way to program intelligence seems just
wrong.

------
toddh
The machine learning blog at <http://hunch.net/> is a good resource.

------
marcus
ML is a very interesting field. Great resources can be found on:
<http://www.aaai.org/AITopics/html/machine.html>
<http://aima.cs.berkeley.edu/ai.html>

Start by reading and completely understanding a few of the simpler machine
learning algorithm (Backpropagation neural networks for example), how&why they
work. and continue from there.

------
ivankirigin
Mitchell, Norvig, and "Probabilistic Robotics" by Thrun.

