
Ask HN: What math should I know well to be able to learn ML - majkinetor
I want to learn ML and deep learning and since it involves a lot of math, could anybody pinpoint exactly what domains should I renew for this task, hopefully with references to some good books or educational material.<p>I studied math and computer science 20++ years ago on university of mathematics where I had 20 different math subjects but in my last 15 years of work I forgot most of it since I don&#x27;t use any more advanced math on my current job as software architect where I deal with automation, development, system design, operating systems, integrations etc.<p>I am very experienced in many different IT domains but I would like to attack this problem in more deep way then just starting immediately with online courses or tutorials (but not so deep that it consumes all my free time, nor would I want it, I have many different hobbies and interest). My end goal is to know to apply ML methods and understand enough of it to improve&#x2F;troubleshot anything that doesn&#x27;t fit immediately the typical use case.
======
sharmi
I would suggest an alternative approach. You don't seem to be really
interested in the nitty-gritty of the algorithms than in actually knowing how
to apply them to get the results you desire.

Jumping in deep can really suck your time as you need to brush up on lots of
fields. Also it will be sometime before you have anything meaningful to show,
which can be quite frustrating.

May I suggest a case by case approach wherein you take a problem and try to
apply ml kit like scikit-learn to see which algorithm produces the results you
desire and trying to learn why it does and how?

You can find all the gaps in your knowledge as you learn and fill them on a
need by need basis. This is from someone who tried real hard to do it the
bottom to top way and found it frustrating to learn with a full time job and
family. Now I am going to try the top to bottom way and focus on one niche.
Bottom to top approach works better when you are at school where you have
specific assessments to measure progress and the end of semester to signal the
completion of each course/subject. Since I have just started this alternative
approach, I do not have an exact map to propose.

If someone has walked this path, any inputs would be most welcome.

~~~
majkinetor
Sounds like a good advice. I also have a family and very intensive primary
job.

However, I would also like to renew math because of its general effects on the
brain and because its hard. Without stressor, there is no improvement. I
wanted a long time to devote few hours a week to study general math but if I
can learn something that has usage in ML along the way, I consider it a double
win.

For example, Fourier transformation and wavelates were very handy for me in
understanding how certain DSP processes work while I had personal audio
workstation and was doing digital composition most of my time.

------
aaaksdhf
Just start somewhere (say Kaggle/Scikits) and then just work your way through.
Many will tell you that you need to know 2^{X} to get somewhere, but what they
mean is that some elementary familiarity with some of X is advisable. The
exceptions are calculus, and some vector/matrix math (really the definitions
in "linear algebra").

The math for things that work (these days) ^(TM) is quite boring and would
take a quarter of lectures (see any of a number of MOOCs). Some of it is quite
tricky like convex optimization and graphical models, but you'll likely never
have to know these details unless you're in academia. The latter have also
fallen out of popularity, so you know... less things to know. Convex
optimization these days is all about stochastic-distributed things, which is
probably not the ML you (or most "data scientists") have in mind.

Best thing to do would be to work through the homeworks of some course
offering at CMU/Stanford/UW. The principle is generally simple, but you'll
have to wade through the symbols, and implement something to make sure you
understand the symbols. This is probably the best exercise if you want to
understand things.

You could also just look at and modify the code in scikits, since this will
save you a lot of work with regards to the engineering bit. Kaggle might be
help, but I found them mind-numbingly boring (usually with leaky datasets),
which have to be hacked for performance with endless tweaking on XGboost.

To be honest, I really think ML is overhyped (it is also quite boring as a
job). If you want to do math, you should be doing math.

------
deepaksurti
Relevant to your question:
[https://news.ycombinator.com/item?id=13613551](https://news.ycombinator.com/item?id=13613551).

\--

For a rigorous learning plan:
[http://web.archive.org/web/20101102120728/http://measuringme...](http://web.archive.org/web/20101102120728/http://measuringme...),
don't forget to read the section "Straight into the Deep End". See related
discussions here:
[https://news.ycombinator.com/item?id=1187237](https://news.ycombinator.com/item?id=1187237)
and
[https://news.ycombinator.com/item?id=1055042](https://news.ycombinator.com/item?id=1055042).

I am taking a middle of the path approach. Working through the following in
order: 1. Linear Algebra 2. Calculus 3. Probability & Stats 4. Optimization
Theory. For 1 and 2, using Manga Guides. Working through Manga guide for LA
now. Note that I have a 3D graphics background and learning ML just to add to
my skill set, so my plan is very specific to my use case. YMMV. Hope this
helps a bit. Thanks.

------
imakecomments
OP you will want to be comfortable with,

-Multivariate Calculus

-Statistics & Probability (with Calculus)

-Linear Algebra

-Optimizations

as a baseline. You'll need more as you progress.

You want to be comfortable with the math review in:
[http://www.deeplearningbook.org/](http://www.deeplearningbook.org/)

I'd wager to say most people on HN would drop out of an actual graduate level
course on ML. These courses (from my experience) are more research oriented
and less about "hacking".

~~~
majkinetor
This following section of the book looks exactly like something I was
searching for, thx.

\---

1.1 Who Should Read This Book?

This book can be useful for a variety of readers, but we wrote it with two
main target audiences in mind. One of these target audiences is university
students(undergraduate or graduate) learning about machine learning, including
those who are beginning a career in deep learning and artiﬁcial intelligence
research. The other target audience is software engineers who do not have a
machine learning or statistics background, but want to rapidly acquire one and
begin using deep learning in their product or platform.

------
DrNuke
An Introduction to Statistical Learning free book is your first way to go
here: [http://www-bcf.usc.edu/~gareth/ISL/](http://www-
bcf.usc.edu/~gareth/ISL/)

------
Davidbrcz
I would go Linear algebra, probability/statistics/bayesian reasoning,
optimization,

Also, have a look here: [https://github.com/off99555/machine-learning-
curriculum](https://github.com/off99555/machine-learning-curriculum)

------
jfaucett
Good question. I've been teaching myself the requisite maths for about a year
and a half now and am finally getting where I can understand/apply most ML
papers and ideas.

I don't know what level you're at right now, I was pretty far behind and had
to spend a good deal of time with the basics to make sure I really understood
them i.e. algebra, trigonometry and calculus and proofs, etc.

After that though I'd say the following are must know areas:

1\. Linear Algebra

2\. Probability Theory

3\. Statistics

4\. Optimization Theory

5\. Signal Processing (Information Theory) Fourier Transforms, etc.

But the good thing is once you understand those fields pretty well, applying
the methods and solving problems using them is really not that difficult.

~~~
majkinetor
So above 5 domains make you able to read research papers in general and apply
information there to your case ?

~~~
jfaucett
I'd say for about 95% of it yes. Other fields can be useful, Graph Theory for
instance when you are analyzing graphs i.e networks and stuff, and complexity
theory when you are analyzing and comparing different algorithm
implementations. But those 5 above are the ones you just keep seeing/using
over and over and over.

------
probinso
nearly every online resource will expose you to statistics, probability
Theory, algorithm design, and a practical application space. Unfortunately
most of them assume that you have a reasonably strong grasp of linear algebra.

