

List of data science and machine learning resources - seats
http://conductrics.com/data-science-resources/

======
clarle
Great write-up, and awesome list of resources!

The only thing I'd probably add is that there's a pretty significant gap going
from learning linear algebra to more advanced topics such as LDA.

For people who are just getting started with machine learning, it's probably
best to get started with implementing some of the more "intuitive" algorithms
such as decision trees, k-means, and naive Bayes before moving over to some of
the more recent academic work.

Other things that are pretty useful, but often forgotten, such as feature
selection, data normalization, and even data visualization. Algorithms are
usually just one part of machine learning, but even the best algorithm
wouldn't be able to do anything without identifying what the best features of
your data are.

Still, it's a great list of more advanced topics, and definitely something
I'll keep bookmarked for future reference.

~~~
snippyhollow
To bridge the gap between naive Bayes and LDA, I would recommend going from
k-means to EM and then from EM to variational Bayes. K-means to EM is covered
in chapters 20 (pp. 286-), 22 (pp. 300-) and 33 (pp. 422-) of MacKay's ITILA
[1] (excellent and free book BTW). I recommend to learn about (== apply on
something) the junction tree algorithm because you will have to brush on graph
theory. Also, do more convex optimization beforehand than I did, or you will
have to catch up: take a full course or full book on it.

For LDA you'll need to understand Dirichlet processes, I find the introduction
by Frigyik et al. [2] to be excellent for that. You may need to read A Measure
Theory Tutorial (Measure Theory for Dummies) by Gupta [3] before. Finally, I
put there the two most influential LDA papers to me: [4] and then [5].

[1] <http://www.inference.phy.cam.ac.uk/mackay/itila/book.html>

[2]
[http://www.ee.washington.edu/research/guptalab/publications/...](http://www.ee.washington.edu/research/guptalab/publications/UWEETR-2010-0006.pdf)

[3]
[https://www.ee.washington.edu/techsite/papers/documents/UWEE...](https://www.ee.washington.edu/techsite/papers/documents/UWEETR-2006-0008.pdf)

[4]
[http://www.psychology.adelaide.edu.au/personalpages/staff/si...](http://www.psychology.adelaide.edu.au/personalpages/staff/simondennis/LexicalSemantics/BleiNgJordan03.pdf)

[5]
[http://videolectures.net/site/normal_dl/tag=83534/nips2010_1...](http://videolectures.net/site/normal_dl/tag=83534/nips2010_1291.pdf)

~~~
mjw
In case the measure theory put anyone off: you don't need Dirichlet Processes
for plain LDA, just the finite-dimensional
<http://en.wikipedia.org/wiki/Dirichlet_distribution> (which isn't so bad and
a very useful tool in Bayesian stats as the conjugate prior for discrete
observations)

For some of the non-parametric variants like hierarchical dirichlet process
LDA, you need DPs, but that stuff is pretty hardcore -- don't walk before you
can run.

Another route to LDA (assumes some Bayesian stats basics):

* Learn a bit about Markov chains if you don't know them already * Read up on sampling-based approximate inference methods and find a proof that a Gibbs sampler converges (or just take it on trust...) * Read the classic Griffiths and Steyvers paper deriving a collapsed Gibbs sampler for LDA [1]

[1] <http://www.pnas.org/content/101/suppl.1/5228.full.pdf>

------
antman
Google is your friend. You will usually find something about those things by
altering the following

best machine learning site:stackoverflow.com "closed as not"

------
eli_awry
I've spent the last 1.5 years as a machine learning PhD student slowly
discovering many of these resources and topics, and I wish I had had this list
at the beginning - it contains most of the gems I've found. I'd add that PGM
course on Coursera clearly explains fundamental topics in probabilistic
graphical models.

It's important to understand individual algorithms, but in many ways it's more
important to have a broad overview of the field and its more modern methods,
so that given a problem it's possible to think about the best way to solve it,
and to share a common language with others who may have ideas. Beyond this
list and various online courses, I've found that talking to people about their
work and explain the high-level concepts of every black-box classifier or
similarity metric or whatever it is they use has been quite educational

------
RaSoJo
Awesome post. It has been bookmarked, Evernoted, printed and stuck up on my
wall.

I did note the absence of the oft quoted Andrew Ng's Coursera course on ML. I
assume the author has put it under : "disruptive educational sites".

But genuinely want to know how Ng's course measures up to the other resources
mentioned in this post??

~~~
sherjilozair
I don't think it measures up as much. It's basically a dumbed down version of
his actual Stanford course. If you have any experience in Math and Linear
Algebra, you should take a more serious course like his lectures in Youtube or
Caltech's course on Youtube. The ideal audience of the Coursera course is non-
math people who want to get an idea of what machine learning is.

------
conductrics
I reached out to Yann LeCun and he emailed me a couple more recent links. I
updated the deep learning section of the post to include them. Feel free to
check them out.

------
paulgb
Great list. Anyone have a recommendation for a good, rigorous coverage of
Bayesian Statistics?

------
icedin
Great list, well done Matt. Clear and concise as always.

------
tgwilson
Fantastic list of resources!

------
simplerichard
Great list.

