
Understanding Machine Learning: From Theory to Algorithms (2014) - subnaught
http://www.cs.huji.ac.il/~shais/UnderstandingMachineLearning/copy.html
======
arbitrage314
I'm a math geek, but I'm also a mostly self-taught data scientist.

"The Elements of Statistical Learning"
([https://web.stanford.edu/~hastie/local.ftp/Springer/OLD/ESLI...](https://web.stanford.edu/~hastie/local.ftp/Springer/OLD/ESLII_print4.pdf))
is far and away the best book I've seen.

It took me hundreds of hours to get through it, but if you're looking to
understand things at a pretty deep level, I'd say it's well-worth it.

Even if you stop at chapter 3, you'll still know more than most people, and
you'll have a great foundation.

Hope this helps!

~~~
reader5000
Having read significant chunks of both ESL and Understanding Machine Learning
(albeit UML much more recently) I would argue that for many readers UML is
superior.

ESL pays short shrift to the computational complexity of learning whereas UML
explicitly handles both statistical and computational complexity concerns. It
doesnt matter how statistically pure your algorithm is if its running time
scales exponentially with your data.

All of UML's chapters are conceptually unified even when discussing different
ML algorithms, with ESL being more of a grab-bag by chapter.

Still, both high quality and free!

~~~
arbitrage314
Thanks for the interesting comparison!

------
pmelendez
Just a curiosity: One of the authors also proposed Pegasos SVM [1] which is a
nice online approximation to SVM and that can be written in 15 lines of code
or so.

[http://ttic.uchicago.edu/~nati/Publications/PegasosMPB.pdf](http://ttic.uchicago.edu/~nati/Publications/PegasosMPB.pdf)

------
cdnsteve
I feel like the barrier to machine learning for me, as I've seen in many
tutorials and books and is an immediate discouragement, is the massive amount
of math thrown in your face. Many of us didn't just graduate, need glasses and
fall asleep at 8pm on the couch when the kids are in bed... Math is this
distant fragment of memory buried under years of everything not Math.

It feels like machine learning is only taught by academia but the majority of
the audience is for practical use by the average developer wanting to play
with it today.

~~~
dxbydt
I interview a lot of developers for ML positions at our company. The first red
flag is always a lack of math. Candidates who come in with API-level
competence ie. can implement an ML algo using this,that or the other API,
without any understanding of some basic math behind it, always fare poorly.
Atleast in ML, not having an understanding of math is pretty much like
claiming expertise in riding a bicycle by watching 100s of bicycle videos on
youtube without ever riding a bicycle yourself. This isn't the case with say,
front-end web dev, where nobody really cares whether what's under the hood is
php or jquery or elegantly handcrafted elm as long as the webpage looks good
and functions as advertised. You can get a lot of mileage by hiring someone
who can "make webpages" without knowing what tools they use to make that
webpage. Once you get traction etc. you'll probably rewrite all of the cruft
:) With ML, if you hire somebody who can "create decision tree in python
mllib" without knowing the first thing about what entropy is & how to compute
entropy ( a real candidate, unfortunately ), you are simply inflicting a lot
of pain on yourself and your customers. Suppose such a person is deciding
whether or not to give you a loan, and he decides to construct a decision
tree. He'll happily throw in your zip code and credit card number into the
mix, not realizing that those two features have super high entropy but the
tree will have serious overfitting issues ie. the tree will simply not
generalize to unseen data. He won't realize these things because he won't know
what entropy is in the first place, since he only thinks of decision trees in
terms of some black box that comes out of some ML api.

~~~
Hockenbrizzle
A lack of mathematical intuition is a serious problem for many people from
engineering to biology to economics. It certainly plagued me throughout my
engineering bachelors studies and is something I continually work to get
better at.

In my opinion, physics students learn the best framework for thinking and get
a very good mathematical intuition. For example, here's a problem from an
introductory QM book that really threw me for a loop when I was studying:

 _A needle of length L is dropped at random onto a sheet of paper ruled with
parallel lines a distance L apart. What is the probability that the needle
will cross a line?_

~~~
sotojuan
> A lack of mathematical intuition is a serious problem for many people from
> engineering to biology to economics.

This is true. I can't really say why, but after my Discrete Mathematics class
a lot of my Computer Science problems became a lot easier to reason about.

~~~
onaclov2000
I will say that after my discrete math class we really only talked about how
to write a proof, and well, it really didn't help for me. (I think we were
supposed to get further into stuff, but well, the class wasn't paced well, new
professor, etc).

------
inglor
I've read this book and warmly recommend it. It has a very pragmatic "no
bullshit" approach and it's very mathematical and concise.

The neural networks chapter is tiny (but that's ok - that's not the focus) and
some of the questions are really hard - but overall I've really enjoyed it.

------
stevenmays
I am considering taking Udacity's machine learning nanodegree [0] with zero
machine learning background. It seems interesting. Any thoughts?

[0] [https://www.udacity.com/course/machine-learning-engineer-
nan...](https://www.udacity.com/course/machine-learning-engineer-nanodegree--
nd009)

------
wodenokoto
It's pretty cool that the book is not only free, but they link to courses that
uses it.

If you speak Hebrew you can get two different professors take on how to teach
the material in the book, as well as lecture notes from a total of 3
professors. That's pretty neat if there's a concept you are struggling with as
a student!

------
enlightenedfool
Glad to see another book to learn from for free. But my problem now is that
there are so many books each with somewhat different approach and content for
the same ML techniques. Not necessarily bad, but I get somewhat confused when
trying to apply a method.

EDIT: I guess the focus on the theory might help me.

------
alvern
My question for someone that has an intermediate level of skill in machine
learning, what's the best way to dip your toes in? (Udacity, coursera, edx,
PDFs, Talking Machines podcast, etc)

~~~
zintinio4
Find a problem to work on in a domain you find interesting. By reading
published papers and trying to attack the problem, you'll be forced to pick up
a lot of other knowledge not commonly discussed like: feature extraction and
selection, dimensionality reduction, dealing with sparsity, common metrics for
that problem, recent work, etc.

I was forced to learn a massive amount in a short period of time for work, but
I'd previously watched Andrew Ng's lectures, as well as majored in Math/CS. I
can also generally recommend Hinton's NN lectures, Socher's Deep learning for
NLP, Andrew Ng's Machine Learning, and a few books.

------
michaelsbradley
There's also the freely-accessible book _A Course in Machine Learning_ :

[http://ciml.info/](http://ciml.info/)

------
Omnipresent
Related: Foundations of Data Science:
[http://www.cs.cornell.edu/jeh/nosolutions90413.pdf](http://www.cs.cornell.edu/jeh/nosolutions90413.pdf)

------
sagik
Took the course at the Hebrew University. Awesome course. Here are the slides
(English) and videos(Hebrew):
[http://www.cs.huji.ac.il/~shais/IML2014.html](http://www.cs.huji.ac.il/~shais/IML2014.html)

------
therobot24
over 30 chapters and the only reference to graphical models is naive bayes and
EM

------
jjaredsimpson
Top comment in another thread whines that nobody understands what the
machinery of ML algorithms are really doing.

Top comment here whines that math is hard.

------
pinn42
I'm creating general intelligence in Java multicore.

------
Merkur
great! thanx!

