Given all that, I'm looking forward to his take on machine learning and for another valuable resource to be available to everyone.
Seems like he's done really well since, I will definitely take a look at this.
OP - Thanks for sharing!
He clearly has high hopes for CIML, based on all the infoboxes and chapters that are still incomplete; it's a shame he'll likely need to get tenure before finishing it.
(This should in no way be taken as a slight on Hal's teaching; his class was the best sort of challenging in that it required a large amount of work, and yielded a correspondingly large amount of insight. He's also a fantastic lecturer and a fair grader.)
Judging by his homepage  he is at least on the tenure track. Unless the notion of an Assistant Professor is different over in the states. I agree about the sentiment about the book, at least the chapter on Neural Networks needs some serious work (I believe there are errors in there, maybe I should mail Hal). As a person and researcher I only have good things to say about him, do check out it his blog if you are into ML and NLP  and let's hope he will find the time to continue polishing on this freely available book (I do like his writing).
> A second goal of this book is to provide a view of machine learning that focuses on ideas and models, not on math. It is not possible (or even advisable) to avoid math. But math should be there to aid
understanding, not hinder it.
No book (yes not even PRML - Bishop), follows this diligently.
For example, one of the most basic and oldest statistical methods is linear regression. The first thing anyone will tell you when they are teaching it is the basic "idea and model" - finding a line that fits a scatter plot (and then extending that idea). But this doesn't give you a real understanding of linear regression: where does that line come from, and why does a clean, algorithmic solution exist? Why does the standard solution often lead to numerical errors, and why is regularization a valid solution?
These questions require an increasing amount of math, but they are essential to really understanding linear regression. I agree that some textbooks just throw you a wall of math, but many actually do a solid job of explaining what is going on as they do so.
If you don't want to know the details of machine learning methods, which are inherently mathematical, you might as well as just remember the names of libraries that implement the solutions for you.
A great example of a book that communicates abstract mathematical concepts via algorithms is the Little Schemer. Ironically, it doesn't even set out to communicate the math, but actually just uses the math to communicate other programming ideas like recursion. That, however, doesn't diminish the fact that it demonstrates ways to teach math through programming and algorithms.
Another book which also does a good job at using programming to demonstrate more concrete math is Allen Downey's "Think Stats" book. All through the book, you learn the mathematical concepts of statistics through hands on programming activities.
There definitely is a chicken and egg problem in areas like machine learning because unlike the above resources which have only one layer of abstraction to cross, machine learning presents two layers of programming<->math abstractions to cross for most people who decide to learn it. To really understand and apply machine learning you need to understand the math and models behind it. However the math and models are presented in pure form that makes it difficult to grok unless you arrived at the resource with a classically-trained mathematical background. I would hope that given that the target market for such learning resources it not mathematicians but programmers, that such learning resources would present content to help you arrive at the math from a programmer's point of view.
For example - Linear regression is just brilliantly tackled by Andrew Ng in his ML course (CS229 at Stanford Lecture Notes - not Coursera).
You're trying to make a prediction for some number you care about - let's say the value of a given stock price. You've come up with a set of hypotheses about which characteristics might help you make that prediction. Moreover, you have a set of examples that you've witnessed in the past, and you want to learn from that experience.
Using linear regression, you can test those hypotheses. You turn those characteristic features into a quantifiable number themselves. Linear regression is simply the name we give to the process of testing whether there's any validity to your hypothesis. If that hypothesis is true and you've discovered what makes the stock go up, then the corresponding feature will be given a high absolute coefficient. You'll also know whether it's an indicator of the stock going up or down, based on the sign of that coefficient. There's no math involved - you're testing your own intuition about how to make predictions.
The idea behind linear regression isn't "finding a line that fits a scatter plot." That's still math and it's still unhelpful to many people with serious, real-world applications. It's just an abstraction of the math that happens to leave out the formal representation.
To really communicate ideas in application, you need to move past the math entirely, and get to how it ties into people's judgments about data that they know well and have experience with, and show them that the intuition they can bring to the table is valuable (for feature determination). Otherwise, even scatter plots will often shut people out, because they "aren't good at math."
There is certainly something to be said for courses and books that can present a complex idea without requiring a graduate-level degree of math literacy. But at the end of the day, ML is a subfield of mathematics so not having a thorough grasp of the math underlying it will definitely hinder your understanding.
The book details building ML systems with Python and does not necessarily teach ML per se. It is a good time to write a ML book in Python particularly keeping in mind efforts to make Python scale to Big Data .
What material you want to refer to is entirely dependent on What you want to do?. Here are some of my recommendations-
Q : Do you want to have an "Introduction to ML", some applications with Octave/Matlab as your toolbox?
A :Take up Andrew Ng's course on ML in Coursera .
Q : Do you want to have a complete understanding of ML with the mathematics, proofs and build your own algorithms in Octave/Matlab?
A : Take up Andrew Ng's course on ML as taught in Stanford; video lectures are available for free download . Note - This is NOT the same as the Coursera course. For textbook lovers, I have found the handouts distributed in this course far better than textbooks with obscure and esoteric terms. It is entirely self contained. If you want an alternate opinion, try out Yaser Abu-Mostafa's ML course at Caltech .
Q : Do you want to apply ML along with NLP using Python ?
A : Try out Natural Language Tool Kit . The HTML version of the NLTK book is freely available (Jump to Chapter 6 for the ML part) . There is an NLTK cookbook available as well which has simple code examples to get you started .
Q: Do you want to apply standard ML algorithms using Python?
A : Try out scikit-learn . The OP's book also seems to be a good fit in this category (Disclaimer - I haven't read the OP's book and this is not an endorsement).
But it's absolutely commendable how Andrew Ng takes the topic to such an understandable level that a clever high schooler who knows a little about programming could take the course. There's even extra videos serving as a crash-course to linear algebra and octave programming in the first week.
So he really manages to make the topic accessible to a very large audience.
Do you know what kind of math is needed other than linear algebra.
Also being able to take derivatives helps in a couple of places, but is not necessary.
* Pattern Recognition and Machine Learning by Christopher M. Bishop
* The Elements of Statistical Learning by Trevor Hastie, Robert Tibshirani and Jerome Friedman
Both are very intensive, perhaps to a fault. But they are good references and are good to at least skim through after you have baseline machine learning knowledge. At this stage you should be able to read almost any machine learning paper and actually understand it.
Edit: Andrew Ng's Coursera course is CS229A (http://cs229a.stanford.edu/), not really watered down.
(and the freely available content books by Barber, McKay and Hastie/Tibshirani/Friedman (ESL) are all worth close reads
This subject is difficult and requires a lot of time to understand and put something useful into practice. AI books appear frequently these days. I feel that most of books and tutorials fail to deliver good practical examples and meta-code, focusing more on mathematical proofs.
I am struggling to understand Peter Abeel's aprenticeship learning: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.74....
He can resume the thing in a couple of slides. It seams short and simple (!?) but I think I am still away from truly understand it. Maybe next year.
also why do you have the topics "introduction to machine learning" and "decision trees" in the same chapter?