
Machine Learning for Software Engineers - matthewsinclair
https://github.com/ZuzooVn/machine-learning-for-software-engineers
======
p1esk
I made similar lists 4 years ago. I quit my job, and applied to a PhD program
in ML (still doing that). Looking back, I can offer the following advice _if
you goal is to become a ML Engineer_ :

Foundation:

Start with (re)learning math. Take those boring university level full length
courses in calculus, linear algebra, and probability/statistics. No, unless
you're a fresh STEM graduate, 10 page math refreshers won't do. If you're
self-studying, make sure you do all exercises and take practice exams to test
yourself.

Tools:

1\. (Re)learn C/C++, as well as a linear algebra library, such as Numpy or
MATLAB. You will also have to learn parallel and distributed programming at
some point (CUDA, MPI, OpenMP, etc). Next take a boring university level full
length course on algorithms and data structures.

2\. Get a book describing ML algorithms, and implement them yourself, first
using plain C, then with MPI or CUDA, and finally using plain Numpy/MATLAB, or
one of the low-level ML frameworks (Theano or TensorFlow).

Application:

Finally, start doing ML. Not learning about it, doing it. Choose an
application that interests you (computer vision, NLP, speech recognition,
etc), and start learning what you need to make something work). Focus on
specific, practical tasks. If you don't have any particular application in
mind, go to Kaggle, choose a competition, and read what models/tricks the
winners used. Then jump right in and start competing.

The first two requirements might take years to master, but if you skip them,
you won't be able to do any serious work in ML, or even understand latest
papers. You will be a script kiddie, not a hacker.

~~~
throwaway001231
I work full-time as an ML Engineer at a big tech company in Silicon Valley. I
think most ML Engineers do not read papers and do not do low level
programming. They mostly do feature engineering.

Actually I feel like I forgot a lot of math since university and I would like
to pick it up again. Does anyone have a resource they can recommend either for
self-study or in person?

~~~
zuzoovn
Disclaimer: I am the author of this Top-down learning path. Could you please
take a look and give me some advice? Or If you have any opportunities, please
drop me a line.

~~~
throwaway001231
My advice would be to figure out what you want to do more concretely. Machine
Learning / Data Science is a big field, if you have more concrete goal I think
it would be easier.

------
BickNowstrom
Learn Python stack: scipy, numpy, pandas, scikit-learn, jupyter,
matplotlib/seaborn.

Learn machine learning tools: XGBoost, Scikit-learn, Keras, Vowpal Wabbit.

Do data science competitions: Kaggle, DrivenData, TopCoder, Numerai.

Take these courses: [https://www.coursera.org/learn/machine-
learning](https://www.coursera.org/learn/machine-learning)
[http://work.caltech.edu/telecourse.html](http://work.caltech.edu/telecourse.html)

Work on soft skills: Business, management, visualization, reporting.

Do at least one real-life data science project: Open Data, Journalism, Pet
project.

Contribute to community: Create wrappers, open issues/pull reqs, write
tutorials, write about projects.

Read: FastML, /r/MachineLearning, Kaggle Forums, Arxiv Sanity Preserver.

Implement: Recent papers, older algorithms, winning solutions.

As a software engineer you have a major advantage for applied ML: You know how
to code. AI is just Advanced Informatics. If you want to become a machine
learning researcher... skip all this and start from scratch: a PhD. Else:
Learn by doing. Only those who got burned by overfit, will know how to avoid
it next time.

------
rosspackard
I enjoy these lists that frequently pop up but I really wish for something
better. Almost like a curriculum. I don't like that there are 20 different
courses for X. Which one should I choose, in what order. Which ones have pre-
requisite knowledge requirements, which ones don't. What programming languages
does each one use? etc,etc,etc

~~~
sn9
Those curricula exist.

They're called university course offerings.

Just look up a class you want to take, figure out what the prerequisites are
(often listed in the course description or syllabus), and follow the chain of
prerequisites until you hit a level that's appropriate.

You can easily look do this with a few different universities and just compare
their courses for a particular subject, find the most common textbooks, etc.

And nowadays, this is even easier since you can find a MOOC or online course
for basically any course in the DAG leading to nontrivial machine learning.

(Programming languages should never be a factor, unless you're choosing
between two otherwise equivalent offerings.)

~~~
rosspackard
I shouldn't have used the word curriculum. I went to a top university and I
know how frequently those prerequisite courses can be totally off base (my
degree is chemical/biomolecular engineering). I used the term 'pre-requisite
knowledge' because I would prefer a list of what ideas/concepts are needed to
be fully understood before taking the course (ex/ instead of saying linear
algebra is a prerequisite, say you need an understanding of solving linear
equations using matrices but you don't need to understand linear spaces. Then
you can focus your effort better and not take a 40 hour pre-req that you only
need 4 hours from)

~~~
daveguy
> I used the term 'pre-requisite knowledge' because I would prefer a list of
> what ideas/concepts are needed to be fully understood before taking the
> course (ex/ instead of saying linear algebra is a prerequisite, say you need
> an understanding of solving linear equations using matrices but you don't
> need to understand linear spaces.

If you are going to cherry pick concepts it may be better to just go full bore
into the classes that most interest you. You will be motivated to pick up the
concepts you are lacking and the learning profile will exactly fit the
specific course. You wouldn't want to do this in a college setting because you
are paying a lot for each hour, but as an auto-didact the time investment is
on you.

If you already have a background in CS or Engineering you can probably pick up
the additional concepts with focused study/refresher.

~~~
rosspackard
Thanks. And thats why my comment advocates for a 'better list' which would
have a list of knowledge required. Then a student can choose whether they know
enough that they just need to cherry pick or if they would need to take a
whole different class to prepare.

------
mtalantikite
I've been spending my free time getting myself familiar with ML and personally
I'm focused on resharpening my math skills, reading papers from the "awesome
deep learning papers" repo [1], playing with TensorFlow, and reading the
Neural Networks and Deep Learning book [2]. I did my undergrad degree in math,
so a lot of the math is just review for me, but ML seems to get fairly math
heavy pretty quick. I would suggest anyone looking at ML to spend a good
amount of time going through the backing math in addition to the CS parts,
otherwise you might not develop much intuition for what's going on.

[1] [https://github.com/terryum/awesome-deep-learning-
papers](https://github.com/terryum/awesome-deep-learning-papers)

[2]
[http://neuralnetworksanddeeplearning.com/](http://neuralnetworksanddeeplearning.com/)

~~~
throwaway001231
If you are serious about deep learning I would recommend this book instead:
[http://www.deeplearningbook.org](http://www.deeplearningbook.org) It is
coming out next month, but you can view the entire book online.

------
adamnemecek
To be honest, this isn't the best list, it's a bit too blog heavy. I've
started reading up on ML only recently but here are my recommendations. Note
that I haven't went through all of them in entirety but they all seem useful.
Note that a lot of them overlap to a large degree and that this list is more
of a "choose your own adventure" than "you have to read all of these".

Reqs:

* Metacademy ([http://metacademy.org](http://metacademy.org)) If you just want to check out what ML is about this is the best site.

* Better Explained ([https://betterexplained.com/](https://betterexplained.com/)) if you need to brush up on some of the math

* Introduction to Probability ([https://smile.amazon.com/Introduction-Probability-Chapman-St...](https://smile.amazon.com/Introduction-Probability-Chapman-Statistical-Science/dp/1466575573))

* Stanford EE263: Introduction to Linear Dynamical Systems ([http://ee263.stanford.edu/](http://ee263.stanford.edu/))

Beginner:

* Andrew Ng's class ([http://cs229.stanford.edu](http://cs229.stanford.edu))

* Python Machine Learning ([https://smile.amazon.com/Python-Machine-Learning-Sebastian-R...](https://smile.amazon.com/Python-Machine-Learning-Sebastian-Raschka/dp/1783555130))

* An Introduction to Statistical Learning ([https://smile.amazon.com/Introduction-Statistical-Learning-A...](https://smile.amazon.com/Introduction-Statistical-Learning-Applications-Statistics/dp/1461471370/))

Intermediate:

* Pattern Recognition and Machine Learning ([https://smile.amazon.com/Pattern-Recognition-Learning-Inform...](https://smile.amazon.com/Pattern-Recognition-Learning-Information-Statistics/dp/0387310738))

* Machine Learning: A Probabilistic Perspective ([https://smile.amazon.com/Machine-Learning-Probabilistic-Pers...](https://smile.amazon.com/Machine-Learning-Probabilistic-Perspective-Computation/dp/0262018020))

* All of Statistics: A Concise Course in Statistical Inference ([https://smile.amazon.com/gp/product/0387402721/](https://smile.amazon.com/gp/product/0387402721/))

* Elements of Statistical Learning: Data Mining, Inference, and Prediction ([https://smile.amazon.com/gp/product/0387848576](https://smile.amazon.com/gp/product/0387848576)(

* Stanford CS131 Computer vision ([http://vision.stanford.edu/teaching/cs131_fall1617/](http://vision.stanford.edu/teaching/cs131_fall1617/))

* Stanford CS231n Convolutional Neural Networks for Visual Recognition ([http://cs231n.github.io/](http://cs231n.github.io/))

* Convex Optimization ([https://smile.amazon.com/Convex-Optimization-Stephen-Boyd/dp...](https://smile.amazon.com/Convex-Optimization-Stephen-Boyd/dp/0521833787))

* Deep Learning ([http://www.deeplearningbook.org/](http://www.deeplearningbook.org/) or [https://smile.amazon.com/Deep-Learning-Adaptive-Computation-...](https://smile.amazon.com/Deep-Learning-Adaptive-Computation-Machine/dp/0262035618/))

* Neural Networks and Deep Learning ([http://neuralnetworksanddeeplearning.com/](http://neuralnetworksanddeeplearning.com/))

Advanced:

* Probabilistic Graphical Models: Principles and Techniques ([https://smile.amazon.com/Probabilistic-Graphical-Models-Prin...](https://smile.amazon.com/Probabilistic-Graphical-Models-Principles-Computation/dp/0262013193/))

I have also found that looking into probabilistic programming is helpful too.
These resources are pretty good:

* The Design and Implementation of Probabilistic Programming Languages ([http://dippl.org](http://dippl.org))

* Practical Probabilistic Programming ([https://smile.amazon.com/Practical-Probabilistic-Programming...](https://smile.amazon.com/Practical-Probabilistic-Programming-Avi-Pfeffer/dp/1617292338))

The currently most popular ML frameworks are scikit-learn, Tensorflow, Theano
and Keras.

~~~
annnnd
For someone who has basic background knowledge in ML and wants to know more
about NN and DL, my list would be:

* Neural Networks and Deep Learning ([http://neuralnetworksanddeeplearning.com/](http://neuralnetworksanddeeplearning.com/)) - perfect overview, go over it twice at least (the second time you will understand much of the decisions in the start)

* Tensorflow and deep learning, without a PhD ([https://www.youtube.com/watch?v=sEciSlAClL8](https://www.youtube.com/watch?v=sEciSlAClL8)) - as much as I hate video lectures, this one was worth it; a good complement to the book above

* Theano Tutorial ([http://deeplearning.net/software/theano/tutorial/index.html](http://deeplearning.net/software/theano/tutorial/index.html)) - using Theano or TensorFlow takes some getting used to. I found TensorFlow documentation absolutely horrible for beginners, probably because the authors expect users to already know such frameworks. Once you learn Theano you won't have trouble with TensorFlow (if that's what you want to use).

Then there are more specific papers, but I guess those depend on the problem
at hand.

------
azaydak
What would you guys recommend for someone like me who has a strong background
and education in ML algorithms (a PhD in a similar field) and math however but
has very little formal programming skills. When it comes to coding I usually
find myself hacking together code with the intention to come back to it later
but never do. I have never had a problem finding and keeping jobs but have
always felt a little insecure when it comes to writing code with CS majors,
even ones with just a few years out of college. Any good material to help get
me up to speed on the CS part in particular with common ML problems?

~~~
rosspackard
What kinds of things do you tend to be insecure about?

~~~
azaydak
Basically, I'm confident that I can get any job done and working. However I am
insecure or don't always know how to answering questions about how efficient /
fast/ a block of code is. Also, code architecture and professional coding
styles. And finally the sort of classical coding interview questions that most
CS majors would learn in school but are not always relevant in daily use but
seem to be asked often.

~~~
rosspackard
Those are all goals that with a little work should be surmountable.

When CS majors or programmers estimate how fast/efficient code is they
generally are intuitively applying big 'O' notation or some similar concept.
This can be gained from an intro level algorithms class. Or you could look up
and read about the topic.

The classical coding interview questions mostly stem from an intro algorithms
and data structures class. Or you could get a book on interview prep and work
through a couple a day.

Finally, the coding style issue usually is ancillary knowledge and not found
(or at least not done well) in typical CS curricular. I would suggest the book
"Clean Code" by Robert Martin. It goes through code smells, commenting,
general style, and is well written.

So overall, you could look at Algorithms and Data Structures MOOC's. Also read
a book on CS interview prep and writing maintainable code. Hope this helps a
little. Good Luck!

------
BucketSort
4 hours on the front page of HN and no comments. Seems really weird.
Nonetheless, this is a really good list of resources. I've tried making lists
of resources like this in the past, but there are so many nicely put together
resources like this. Now I have a list of list of resources.

~~~
rafadc
Probably is because the content seems interesting but no one had time to look
at it obviously ;)

~~~
dominotw
Machine learning fatigue.

------
dolzenko
> This is my multi-month study plan for going from mobile developer (self-
> taught, no CS degree) to machine learning engineer.

Wondering what could be the rationale? I mean is mobile-dev so critically
different from ML?

~~~
sidlls
Yes, they solve completely different problems.

"Mobile-dev" is more or less application development for a target client,
possibly with a client-server relationship. It may incorporate the products of
a ML system, or any number of other systems, but is itself the same
application development model we've had for 30 years, but with a mobile
client.

Machine Learning is applied computational statistics. It and its close cousin
"data science" have become incredibly hyped the last few years. I suppose it's
a matter of taste, but my view is that applying simple probability to a
problem (e.g. "recommending" based on picking the thing most frequently voted
up by other users and adjusting the recommendation as data comes in) isn't
"machine learning" or "data science", but I've seen it called such by
candidates I've interviewed and even one colleague. It's not that simple, and
trivializing the terms just contributes to the hype.

~~~
dolzenko
Sorry, didn't make it clear, was wondering given all the difference why would
one prefer one over the other? Seem like 'in the field' experience is not
going to be very much different. Appears to me mobile can be even more fun.

------
barleymash
That dude is USA as heck.

------
messel
We need machine learning to learn about machine learning

~~~
skriticos2
A DIY toolkit would be nice, that gives you various peaces that it explains
and then let's you fit them together. Of course without a million
dependencies, clean-room type display of concept. Anybody bored enough to
build this?

~~~
falcolas
Something like the Ruby Koans for machine learning? I like that idea.

------
hash-set
While it's nice to think that people with no background can succeed in Data
Science, I'm not sure it is true. There are a lot of hacks who never did
"science" coming out of the woodwork lately. It's a bit ugly.

