
Good, freely-available textbooks in machine learning - alextp
http://metaoptimize.com/qa/questions/186/
======
ThomPete
For anyone interested in Machine Learning let me again link to the entire
lecture series from Stanford.

[http://itunes.apple.com/WebObjects/MZStore.woa/wa/viewiTunes...](http://itunes.apple.com/WebObjects/MZStore.woa/wa/viewiTunesUCollection?id=384233048)

~~~
cageface
Andrew Moore's lecture slides are also very useful:
<http://www.autonlab.org/tutorials/index.html>

------
mayank
AARGGH! I paid close to a hundred bucks for the Tibshirani book a few years
ago! On the plus side, I think it's a beautiful, beautiful machine learning
book that is a welcome replacement to Tom Mitchell's book, which is showing
its age. I highly recommend the Tibshirani et al. book to anyone with college-
level math and a desire to understand what the hell this machine learning
thing REALLY is.

------
metamemetics
Not free, but I highly recommend "Programming Collective Intelligence" if you
are looking for python examples applicable to web applications.
<http://oreilly.com/catalog/9780596529321>

~~~
kranner
That book has the worst Python code ever in a print publication.

~~~
drats
I'm not a computer scientist nor really a programmer and I found it very
readable. Care to elaborate?

~~~
kranner
I read it when it came out which was a while ago. I found it to be very
unidiomatic Python at the time.

More critique at <http://news.ycombinator.com/item?id=208895> if you'll chase
the parent links.

~~~
metamemetics
Idiomatic, no (agreed). Very readable, yes (I think the style was
intentional). For tutorials, I generally implement everything myself as part
of the learning process and avoid source copy-pasting. So I wasn't irked but
YMMV depending on learning style.

------
cageface
ML definitely seems to be more on people's radar these days but as far as I
can tell the job market is pretty small.

~~~
bravura
My experience has been the opposite:

[http://metaoptimize.com/qa/questions/2542/job-prospects-
in-m...](http://metaoptimize.com/qa/questions/2542/job-prospects-in-machine-
learningdata-mining)

I guess it depends what you're comparing against. Against general purpose
programming jobs, yes, the market is small. But, of all CS PhDs, people who do
ML are the most sought-after.

~~~
cageface
That was actually my question. :)

I'm sure CS PhD's are in good shape but breaking into ML from another
programming discipline seems pretty hard.

~~~
alextp
I think most of this is due to the fact that most ML techniques need lots of
specific knowledge to apply correctly. Many introductions focus too much on
specific algorithms, for example, while ensemble methods are probably best in
real-world data. The pactices of building training and test sets, of
regularizing, of doing proper feature engineering, and of making structured
models (for example) are more easily learned in a mentor-student relationship
than by reading blogs (reading papers might take you a long way, but you won't
build an intuition as to why things work and why they fail without lots of
experimentation or advice from someone who already has this intuition) and/or
AI books, and ML books are usually far too technical (or too superficial in
the technical side, as is programming collective intelligence).

~~~
cageface
Most of the ML books I've worked with so far seem a bit overly formal. Steven
Marsland's book seems to strike the best balance between theory and
implementation I've seen, even if the Python code is a little clumsy.

------
Groxx
I've read a good 1/4 of Artificial Intelligence - A Modern Approach, and
scanned a handful of pages further on, and thus far it's been excellent.

So far, it looks to be very much rule / first-order logic oriented, but that
makes sense, as it's been the most well understood, and much more easily
provable / understandable once something is built. There's very little about
neural networks, for instance, or genetic algorithms, aside from an
explanatory section or so. Though I may have the first edition, and apparently
the 3rd came out last year.

So, if anyone is interested in a relatively-introductory, _extremely_ readable
book on machine learning, it's the best I've come across so far.

~~~
mayank
> There's very little about neural networks, for instance, or genetic
> algorithms

For good reason. Outside of neuroscience, there's very little reason to use
neural networks for classification or regression problems, and outside very
ill-posed optimization problems, there's very little reason to use genetic
algorithms. If you want a good MODERN machine learning book (which is where
your post seems to imply your interests lie), try the Elements of Statistical
Machine Learning by Tibshirani, Hastie and (I forgot the third guy's name,
sorry). Anyway, it's included in the posted link.

~~~
T_S_
Not sure neural net expert Yann LeCun (yann.lecun.com) would agree with your
neural networks comment. Neural nets have taken on a life of their own since
the time they were originated and actually have little direct connection with
neuroscience these days.

~~~
alextp
This is true, but the very good deep learning results are very recent (from
less than 5 years ago), and they generally require very different models
and/or training strategies from the standard multilayer perceptron you find in
most AI textbooks. So it's an omission, but I'm not aware of any textbooks
that cover these new neural networks in depth, as basic research in this area
is still going in a frantic pace.

~~~
Groxx
Yeah, I'm finding that there's little book-format info on deep neural networks
as well. Similarly for belief networks. Gotta hunt down some journals, and
wield some Google-fu.

~~~
tudorachim
The most promising approaches for neural nets these days involve learning
generative models using Restricted Boltzmann Machines (RBMs). Geoffrey Hinton
at UToronto has some good seminal papers on these, like
<http://www.cs.toronto.edu/~hinton/absps/ncfast.pdf>. See
<http://www.cs.toronto.edu/~hinton/> for excellent videos of digit generation
/ classification.

As a note, this approach outperforms all others (sans tweaks), whereas neural
nets used to be beaten by SVMs and the like.

~~~
alextp
Actually, the best recent results in this area involve discriminatively
trained stacked auto-encoders using second-order algorithms (for which the
code, as of Sept 2010, is still private to the group at the university of
toronto), as described in James Martens's paper at this year's ICML:
[http://www.cs.toronto.edu/~jmartens/docs/Deep_HessianFree.pd...](http://www.cs.toronto.edu/~jmartens/docs/Deep_HessianFree.pdf)

