

Ask HN: Machine Learning? Where to start? - swGooF

What are the best books/websites to learn about machine learning?  I have a solid background in math/cs/algorithms.
======
_dps
I recommend starting with Witten and Frank's "Data Mining: Practical Machine
Learning Tools and Techniques", for two reasons:

1) your mathematical background will make it easy for you to spot the areas
where they gloss over important theoretical details (e.g. they don't treat
Mercer's theorem deeply, if at all if I recall correctly). This will keep you
from making rookie mistakes like picking your own strange kernel functions for
SVMs (all of this will make sense to you after reading W+F)

2) W+F is accompanied by Weka, which has decent-but-not-great implementations
of a wide variety of algorithms in an open-source toolkit with a functional
(though not particularly usable) GUI. You can be up-and-running on test
problems from the the UCI Machine Learning repository in 20 minutes.

Russell and Norvig is a good reference book, but my strong suggestion is to
grab W+F, Weka, and some data from UCI's ML repo
(<http://www.ics.uci.edu/~mlearn/>). Use Weka to run the standard algorithms
from W+F on real data.

After that things can get as complicated as you want. I'm personally betting
that graphical models (see Koller's "Probabilistic Graphical Models") will
continue to grow in importance.

On the statistical side, I strongly recommend "Applied Linear Regression" by
Weisberg to get a sense for the kinds of idea statisticians bring to the party
(significance, parameter intervals, chained analysis like ANOVA).

Hope that helps :-)

Edit* spelling errors

PS: Almost forgot MacKay's excellent "Information Theory, Inference, and
Learning Algorithms" (free as pdf). It does a great job linking coding, ML,
inference, and bayesian probability together. Read at least as far as the
section linking K-means clustering to Expectation-Maximization over a Mixture
of Gaussians, it's worth it.

~~~
swGooF
Thanks, that is a detailed description. Great ideas to get me going.

------
clyfe
I learned myself specifically SVM from here
[http://see.stanford.edu/see/courseinfo.aspx?coll=348ca38a-3a...](http://see.stanford.edu/see/courseinfo.aspx?coll=348ca38a-3a6d-4052-937d-cb017338d7b1)
(already had a good background in linear and abstract algebra).

------
stollercyrus
<http://aima.cs.berkeley.edu/> is pretty good

Stuart J. Russell and Peter Norvig, 2002. Artificial Intelligence: A Modern
Approach, 2nd edition, Prentice Hall. ISBN: 0137903952.

~~~
swGooF
thanks

~~~
krisneuharth
See also: <http://www.ai-class.com/>

