
Ask HN: What to study to become a machine learning/ai master? - supremus_58
I dropped out of school a few years ago and have been working as a developer. I plan to start attending school again in the coming fall. 
My question: what is the best route of study to become a master of all things machine learning? 
My plan is to go in as a double major for math and computer science as I was before. Which topics in mathematics are most prevalent for machine learning and more specifically neural networks?
======
deepaksurti
For a rigorous learning plan:
[http://web.archive.org/web/20101102120728/http://measuringme...](http://web.archive.org/web/20101102120728/http://measuringmeasures.com/blog/2010/3/12/learning-
about-machine-learning-2nd-ed.html), don't forget to read the section
"Straight into the Deep End".

See related discussions here:
[https://news.ycombinator.com/item?id=1187237](https://news.ycombinator.com/item?id=1187237)
and
[https://news.ycombinator.com/item?id=1055042](https://news.ycombinator.com/item?id=1055042).

I am talking a middle of the path approach. Working through the following in
order: 1. Linear Algebra 2. Calculus 3. Probability & Stats 4. Optimization
Theory. For 1 and 2, using Manga Guides. Working through Manga guide for LA
now. Note that I have a 3D graphics background and learning ML just to add to
my skill set, so my plan is very specific to my use case. YMMV. Hope this
helps a bit. Thanks.

------
hackermailman
Obv you would want to talk to whoever your advisor is or a professor in the ML
department at your school. Otherwise, find existing books/courses and see what
you'll need for background such as this free book:
[http://neuralnetworksanddeeplearning.com/](http://neuralnetworksanddeeplearning.com/)

[http://www.cs.cmu.edu/~mgormley/courses/10701-f16/schedule.h...](http://www.cs.cmu.edu/~mgormley/courses/10701-f16/schedule.html)
(general grad level intro)

[http://www.cs.cmu.edu/~bapoczos/Classes/ML10715_2015Fall/](http://www.cs.cmu.edu/~bapoczos/Classes/ML10715_2015Fall/)
(in depth grad level intro, both include a youtube lecture playlist).

The requirements listed here
[http://www.cs.cmu.edu/~bapoczos/Classes/ML10715_2015Fall/faq...](http://www.cs.cmu.edu/~bapoczos/Classes/ML10715_2015Fall/faq.html)

    
    
      -Basic probability and statistics are a plus.
    
      -Basic linear algebra (matrices, vectors, eigenvalues) is a plus. Knowing functional analysis would be great but not required.
    
      -Ability to write code that exceeds 'Hello World’. Preferably beyond Matlab or R.
    
      -Basic knowledge of optimization. Having attended a convex optimization class would be great but the recitations will cover this.

------
LeanderK
i am a bit confused. What do you mean with "school"? If you really want to
become an "ML/AI master" this is more complicated then you seem to think.

If you mean university it will take some time (and rightly so) until you can
choose your courses. ML is rapidly changing, so you can't just learn "one
trick", you need to be able to adapt quickly and learn new techniques (this
can be really fundamental compared to other branches like programming). Also
most of the concepts are hard and you can't (or it's difficult to) learn them
without some theoretical background (for example a bachelor in
math/physics/engineering). Of course you can assemble something in TF based on
a quick MOOC, but this doesn't make you an "ML/AI master", just somebody who
can use a library and who will meet his boundaries quickly.

The usual requirements are Probability/Stats, Calculus, Linear Algebra. To
really scratch the surface you need to take about 2 or 3 semester of courses.
When i was in school i didn't imagine it to be this challenging. I also didn't
imagine it to be so rewarding, but i know that i would be unable to pass them
if i were studying alone.

There are always exception to the rule and i only started diving into ML
recently, so take this advice with i grain of salt. But just because you
taught yourself programming it doesn't mean you can do the same with ML/AI
that quickly and also alone. Take it seriously and don't expect to get results
too quickly. But each step is really rewarding, so just start to study and see
where it takes you.

~~~
supremus_58
sorry that the question came out so ambiguous. yes, by school i mean uni. the
question may have better been asked as "what is the best route to take for a
mathematics student to learn the most relevant ideas in ML?" i've take all the
classes which you've listed, i dropped school somewhere halfway through. I
know how to write code for ML, this isn't totally new to me.(by this i mean i
know how to use the python ml library) i ask topically, what math is most
important in the further development of ai?(ie complex analysis,
combinatorics, graph theory, stochastic methods are all things important in
ML, what other math is worth noting?) the math you listed are simply things
required for most comp-sci majors.

~~~
LeanderK
ah ok. I am also starting with ML, so take this with a grain of salt.

While the courses i listed are required for most CS majors, i think they are
more important for ML than many other fields of CS (of course there are
exceptions). I think a deeper understanding of LA, Calculus and Numeric
compared to normal CS-majors is beneficial (but if you majored in math you
probably had to take these in more depth than most CS-majors). Also i would
double down on Probability (so a course on Probability-Theory, Markov-Chains
etc.). Optimisation Theory is also important, but often already somewhat
covered in the other courses. I don't think one can point to a specific course
that will be really important for NN in the future, nobody really knows whats
coming. Just learn as much math as possible, so that you are prepared to learn
and understand all the future theory and techniques that are not yet
discovered. Maybe we will all be solving stochastic differential equations in
the future, because they accurately predict the performance of various NNs?

Edit: I forgot tensor-mathematics, i haven't taken a course yet, but it seems
pretty straightforward.

~~~
supremus_58
awesome, thank you! I took linear an analytic linear algebra class some years
ago, but admittedly don't remember too much. I'm beginning to study more in
depth matrix analysis, this and combinatorics are useful for understanding
fundamentals of many algorithms. I've learned a little of Markov Decision
Process, but i've been told by people working in the field that MDP isn't used
much for neural net anymore, though this doesn't mean it shouldn't be learned.
I'll definitely look into tensors.

here's a link i found looking into the subject:
[http://datascience.ibm.com/blog/the-mathematics-of-
machine-l...](http://datascience.ibm.com/blog/the-mathematics-of-machine-
learning/)

and here's some recommendations for math/ml texts i got in another thread that
may be of use:
[https://news.ycombinator.com/item?id=13275031](https://news.ycombinator.com/item?id=13275031)

~~~
LeanderK
i was also told that Markov-chains are not used for NN anymore, but i learn
them because often they get combined with NNs and used for other aspects of
the problem (if i remember right AlphaGo used them beside NNs).

the blog article was useful, thank you! A problem with most of the
recommendations is that they are targeted at software engineers wanting to
learn some ML. They mostly don't need to understand back propagation fully and
don't have that complex of a problem. So the requirements are the absolut
basics (what are vectors/matrices and how to multiply them). Also their math-
background is either non-existent or a long time ago. I am a student and
constantly learning math, so i am in a different position. I am also
interested in really understanding NN, including recent papers. They vary,
some being not very math-heavy (the dropout paper) and some being the opposite
(bayes neural network, only skimmed some papers). Many paper introduce a new
idea borrowed from other branches of math, so you can't specialise too much. A
broad understanding is important. Also we still lack a unifying theory, so we
definitely don't know what math will become really important in the future.

------
samhefnawy
You may like to check the following link:
[http://datascience.ibm.com/blog/the-mathematics-of-
machine-l...](http://datascience.ibm.com/blog/the-mathematics-of-machine-
learning/)

------
cr0sh
Probability/Stats, Calculus, Linear Algebra

~~~
supremus_58
this is the math necessary for most comp-sci majors. specifically to machine
learning, which topics in mathematics are most relative to be pursued? what
pathway should a math major take to have the most relative knowledge to pursue
ML

