
Ask HN: the road to learning useful math - Torn
<i>Background</i>: I've got a fairly good intuition for logic and algorithms as a coder, but no formal training in the maths side of things. I last did math in high-school (A-level here in the UK at 18) and while I remember some calculus, pure maths, probability etc., I've never really touched advanced stuff. I didn't take any maths modules at University either.<p>I've recently come across quite a few interesting stats-heavy compsci papers where the mathematical notations and concepts expressed escape me. I'd like to correct this -- and I've seen a few comments floating around here that this stuff isn't actually that hard, and this has got me thinking.<p>Could HN recommend any 'math for programmers' resources or suggest structured approaches (ie. 'look into X, then Y, then try understanding Z) to go about learning this sort of stuff? I think I'm probably not alone here in wanting to learn more about this sort of thing.<p>A quick google provides some interesting links to get started:<p>Math for programmers: http://steve-yegge.blogspot.com/2006/03/math-for-programmers.html<p>'Concrete Mathematics' by Graham, Knuth &#38; Patashnik:
http://en.wikipedia.org/wiki/Concrete_Mathematics
======
silentbicycle
I've found Project Euler (<http://projecteuler.net/>) tremendously helpful for
this sort of thing - some of the puzzles I couldn't figure out the
mathematical trick to do it efficiently, but after I solved it I read through
the forum posts and made sure I could rewrite the full solution in the
languages I'm using. (The same works well for learning algorithms or feeling
out a language, if you're solid on the mathematical side.)

_Mathematics: From the Birth of Numbers_ by Gullberg and Hilton
(<http://www.librarything.com/work/23014>) is also a good overview. It starts
with counting (Roman numerals, numbers in ancient Egyptian writing, etc.) and
builds up from there. The writing seems quite clear, and it covers both the
mathematical concepts _and their history_. (I like this approach quite a bit,
though my background is in history. YMMV.) I haven't gotten to the chapters
for math I'm _completely_ new to yet, but it seems like it would introduce you
to enough basic concepts that you could have a clearer idea where to go next.

_The Cartoon Guide to Statistics_ by Larry Gonick
(<http://www.librarything.com/work/12735>) seems like a good statistics
refresher / overview. It's not that deep, and it's rather hokey, but it's a
quick read. An important part of reading math papers is not getting thrown by
the dense notation, so starting with something less formal may be helpful.

------
tokenadult
Statistics is a very interesting subject, and it is a distinct subject from
mathematics proper. Here (in what is becoming a FAQ post for HN) are two
favorite recommendations for free Web-based resources on what statistics is as
a discipline, both of which recommend good textbooks for follow-up study:

"Advice to Mathematics Teachers on Evaluating Introductory Statistics
Textbooks" by Robert W. Hayden

<http://statland.org/MyPapers/MAAFIXED.PDF>

"The Introductory Statistics Course: A Ptolemaic Curriculum?" by George W.
Cobb

[http://repositories.cdlib.org/cgi/viewcontent.cgi?article=10...](http://repositories.cdlib.org/cgi/viewcontent.cgi?article=1002&context=uclastat/cts/tise)

Both are excellent introductions to what statistics is as a discipline and how
it is related to, but distinct from, mathematics.

A very good list of statistics textbooks appears here:

[http://web.mac.com/mrmathman/MrMathMan/New_Teacher_Resources...](http://web.mac.com/mrmathman/MrMathMan/New_Teacher_Resources.html)

------
caffeine
Could you be more specific about what kind of math you've been seeing in the
papers that you'd like to understand (and/or link some of the papers)? What
you need to know is on this list
[http://www.gatsby.ucl.ac.uk/teaching/courses/ml1-2009/cribsh...](http://www.gatsby.ucl.ac.uk/teaching/courses/ml1-2009/cribsheet.pdf)
for the most part. Because math is layered, once you've got three or four
layers of unfamiliar abstraction, it gets very hard to figure out what's going
on.

The way to get around this initially is by brute force. For example, say you
have a matrix function (from matrices to matrices). The formula uses the
exponential of a matrix, and you're not sure what that actually looks like. So
find the definition, and write it out. Think about what that means, maybe do a
couple of examples to get a sense of it. Write out your matrix function with
this definition expanded (which probably means you'll write it element-wise),
so that you know what each element of the output matrix is as a function of
the elements of the input matrix. Work a couple of examples, think about it.

Eventually, you get familiar with the exponential of a matrix, and you do the
mapping in your head - so your brain has just acquired a new abstraction. The
process above applies to basically every mathematical structure you'll run
into. Sometimes you'll have to recurse a few levels.

It's not about books, it's about process. You can know how to do lots of math
problems and still not grok papers - and you can learn to quickly grok papers
and be able to implement them / reason about them without being very good at
math problems (I'm in the 2nd bunch). In terms of books, you only really need
Wikipedia / Wolfram MathWorld, plus maybe one of the standard math references
like Kreyszig or Boas. Learning to use Mathematica can make your life much
easier as well.

One last comment:

A book that totally changed my relationship to math is Peter Szekeres' Modern
Mathematical Physics: Groups, Hilbert Space, and Differential Geometry. The
explanations are uncompromisingly lucid, written in plain english, with
minimal symbol walls. The book takes you essentially from zero, and introduces
nearly all the math that underpins modern physics. This book really changed my
conception of myself - I've lost my fear, when reading papers, that _maybe_
this is the one I'm not smart enough to understand. When that fear goes, the
stress goes - and you know it's just a matter of learning the definitions.
Then it becomes pleasurable, like a puzzle.

~~~
Torn
One of the ones that prompted me to post this topic was over here, on the
topic of CRF statistical models:
<http://www.cs.umass.edu/~mccallum/papers/ifcrf-uai2003.pdf>

From reading it I get most of what it's about, and how they work, but the
embedded probability stuff is mostly squiggles to me.

I guess my ultimate aim with this is would be able to quickly grok papers on
various compsci subjects, and then go around and code implementations to test
things out.

~~~
ewjordan
"Quickly grok" to "test things out" - it just isn't going to happen like that,
even if you know the math behind it (FWIW, for that paper you're going to need
a lot about statistics and Markov chains, but you may be able to get pretty
far just by Googling the terms you don't know).

Comp sci is notorious for having a huge gap between published work and working
implementations. It often takes heroic efforts to put together a working
version of the algorithms described, sometimes so much work that nobody has
ever done it (Chazelle has some algorithms that have never been implemented in
code). Even if it's plausible to do, don't be surprised if it takes weeks to
get it working right. The more "on the edge" the algorithms are, the more
difficult it tends to be, and it's not really likely that you'll be able to
quickly experiment with new methods unless the authors have published code.

This happens far less often than it should, because simple possession of a
working algorithm is often enough to squeeze several more papers out of a
topic, so authors have no particular incentive to share theirs, or even make
it easy to create one from scratch. Computer graphics papers are particularly
bad on that front, often omitting so much detail that even with significant
study and effort you could never reproduce their results without researching
the problem on your own anyways.

Other people have offered good advice: go through it in excruciating detail,
think about every equation as its own sub-problem, and make sure you
understand what's happening at each step. These papers are meant for detailed
study, not quick reading, even if the reader already knows a lot about the
field. Google should help you through most of the math stuff, and try going to
book search or the papers search if you're not finding anything online, which
will give you good keywords to look for books on.

~~~
Torn
Sound (and sobering!) advice, thanks.

------
hyperbovine
Go back to school. I'm not sure how this works in UK, but here in the US your
local state school usually offers part time masters degrees for people who are
working. I started my masters in math a year ago and the change has been
profound. It's like night and day, what I knew about math a year ago vs what I
know now. If you are serious about learning math the right way there is no
better option.

~~~
Torn
This is an option, however would have to get quite serious about learning this
stuff (balance gain / reward against things like tuition fees, learning in my
own time outside of long work hours, etc).

~~~
weaksauce
I don't know about you but a lot of people need the external motivation of
school deadlines and the bad marks on a permanent record to actually do the
material.

~~~
hyperbovine
Exactly. I told myself for years that I was better off doing the material on
my own. The autodidactic tradition is strong among computer programmers, and I
suspect that many reading this would feel the same way. But the truth is, I
have learned 10x more in one year of school than I did in five years of
muddling through textbooks on my own. Fear of failure is a strong motivator,
as is immersing yourself in a community of like-minded people.

~~~
jriddycuz
I guess it really just depends on what motivates you. Grades never motivated
me, and I never felt like my academic peers and I were "like-minded", so
school ended up feeling more like an impediment to my education than an
enlightening experience.

There are certainly problems with learning on your own, not the least of which
is maintaining the drive to push through the icky parts. But in the absence of
external motivational structures, you can give your self-discipline a workout.
I'm not a particularly disciplined person, but I can still commit to doing 1
hour of math a day. I guess I think of it like my daily mental exercise. It's
hard to develop this habit though, and if you can't, school's not a bad
option.

One of the other problems with learning math on your own is not having anyone
to ask questions. To help with this, I've actively tried to make friends (of
sorts) with math professors at a nearby school. I get lunch with one every
once in a while, and we talk about sports and life and math. This can kinda
substitute for the community of like-minded people that school provides.

All this goes to say is that you have to be aware of how you learn and how you
get motivated. If you are externally driven, and don't mind learning at a pace
you did not establish, school will be useful to you. If you don't _grok_
extrinsic motivation, school's rough and you should learn on your own.

~~~
hyperbovine
That's a fair point. A lot of people could probably motivate to do an hour of
math a day, just like they could to do an hour of exercise for an hour or
practice an instrument. OTOH I would never be independently be able to
motivate to do 4-6 hours of math a day, which is where I'm at now.

Actually the instrument analogy is apt--the world class concert musicians you
see on Youtube are practicing 5 hours a day. The guy down the hall banging out
a Chopin prelude practices one. Both are "good" musicians, but only one is
great. It's a question of what you're looking for.

------
johnmyleswhite
It sounds like you're interested in statistics from the words "a few
interesting stats-heavy compsci papers". The MacKay book, "Information Theory,
Inference and Learning Algorithms," already referenced by benm is quite good,
but I'd suggest on top of it Larry Wasserman's amazing book, "All of
Statistics". If the level of calculus and linear calculus in that book is too
much for you, you might need to review that material first before moving on to
anything sophisticated in statistics.

In the interim, you could start with David Freedman's "Statistics" (written
along with Pisani and Purves) to at least get a feel for the basic ideas of
statistics from one of the great statistical writers of the 20th century.

If there are other things you want to learn about, I'd be happy to make
suggestions, as I went in the opposite direction as you're trying to go now,
having been a math major as an undergrad who picked up CS along the way. For
cryptography, for instance, you'd need to learn number theory and then
abstract algebra, but those things will be of little use to you in other
contexts.

------
ananthrk
Some suggestions (from HN over various threads)

Practical Foundations of Mathematics
<http://www.cs.man.ac.uk/~pt/Practical_Foundations/index.html>

Linear Algebra - Gilbert Strang (Book & Videos)

Linear Algebra and Applications
<http://www.math.unl.edu/~tshores1/linalgtext.html>

P.S. There was another Introduction to Statistics book that I am not able to
find the link for.

------
mistertim
I'm in a similar situation - My last academic exposure to maths was at GCSE
level (age 16 for non-UK people), and I've recently attempted to rectify that.
I've just begun a degree in mathematics through <http://www.open.ac.uk> \- The
first module has been useful for me but maybe less so if you've done an
A-level, but it looks like it progresses onto some useful stuff. I can also
highly recommend 'Concrete Mathematics' - It's been really useful for filling
in some context around the (first module) OU course - typically we're taught
practical applications of mathematics that skip over the proofs and
derivations - and CM is great at filling in some of the rigour that the OU
misses out.

Also, you might want to check out the wikibooks High School Mathematics
Extensions textbook:
[http://en.wikibooks.org/wiki/High_School_Mathematics_Extensi...](http://en.wikibooks.org/wiki/High_School_Mathematics_Extensions)
\- there's a whole lot of rigarous but accessible introductions to some
pertinent mathematical concepts. Marcus du Sautoy's 'The Music of the Primes'
is also great as a bit of light reading to give you a sense of the history
(and some very basic mathematical concepts) of the subject.

Cheers,

Tim

------
Ixiaus
In the course of my self-education I've come across a number of interesting
books for mathematics. My two favorites:

\- Practical Mathematics by _C. I. Palmer_ \- Mathematics for the
Nonmathematician by _Morris Kline_

 _Practical Mathematics_ is an old book, first published in the early nineteen
hundreds. It covers everything beginning with multiplication to advanced
algebra and trig. It uses practical trade problems in gear cutting, cam ratios
in engines, &c... The book has only been re-published by the company Lindsay's
Technical Books. You can also find it in Google's book repository.

 _Mathematics for the Nonmathematician_ is a bit more recent and written by a
strong mathematician with a lot of history and background. This book got me
hooked on mathematics, no other book has done that; his in-depth scholarly
work plus his skill in mathematics makes it both interesting and educating to
read.

A large majority of programming problems can be tackled with solid arithmetic
or algebra.

------
bhrgunatha
MIT's OpenCourseWare site provides a lot of course material -
<http://ocw.mit.edu/OcwWeb/web/courses/courses/index.htm>

Mathematics for Computer Science is perhaps tailor made for your needs -
[http://ocw.mit.edu/OcwWeb/Electrical-Engineering-and-
Compute...](http://ocw.mit.edu/OcwWeb/Electrical-Engineering-and-Computer-
Science/6-042JMathematics-for-Computer-ScienceFall2002/CourseHome/index.htm)

------
k0n2ad
Look at any good book about, or take a class in discrete math, probability and
linear algebra. Those are the subjects that stand out for me. Discrete math is
the most fundamental for computer science. That Concrete Mathematics book
looks promising - plus it's got some information about algorithm analysis and
other computer science specific topics. You should take a class in discrete
math, or if you're disciplined enough, start with that book.

------
ionfish
A while ago, I collected some references and recommendations for books and
online material covering discrete mathematics. They were mainly sourced from
previous discussions on Hacker News. The file's up on GitHub.

[http://github.com/ionfish/notes/blob/master/discrete-
maths.m...](http://github.com/ionfish/notes/blob/master/discrete-maths.md)

The formatting is a little unreadable—I need to fix the Markdown line
breaks—but it does have a fair few links.

------
Tichy
Not sure if reading a maths book will really be easier than reading those
papers. In both cases, you kind of have to grind through it.

Wikipedia often helps with the notations.

I think learning useful maths is best done by reading up on the stuff you
presently need. I have several maths books that I never read because while it
seemed like a nice idea, in the end there just wasn't enough motivation for
it.

------
benm
Gilbert Strang's Linear algebra book was great for me after taking A-level as
a mature student, linear algebra comes in everywhere (I think there are videos
of Strang floating around as well). Make sure to work through as many
exercises as you can if you're studying independently.

An approach that might work for some is to take a good undergrad textbook of
something you're interested in, but that is a bit too hard (preferably not too
dense, with enough explanatory text). Try to plough through it, and look up
all the stuff you don't understand! Don't expect to get very far very quickly,
but at least you'll get led to useful reading material for the fields you care
about, and it might also motivate you more than starting from axioms.

(For me it was Information Theory, Inference, and Learning Algorithms that
sparked off a lot of study: free online at
<http://www.inference.phy.cam.ac.uk/mackay/itila/> ).

------
euroclydon
It can be difficult to stay awake when trying to learn Math or especially
Stats in the abstract.

Are you reading these stats-heavy comp-sci papers for a particular reason
beyond mere curiosity, like for a project or for work? If so, then you will
have what I consider the most crucial element needed for learning a difficult
subject: motivation, real lasting motivation.

My advice: identify a concrete need to learn about a particular area of
math/stats, like solving a real-world problem with software, then don't start
by getting bogged down in a particular equation, but instead, find out the
area of math/stats which that equation is a part of, and seek out the most
interesting essay or video lecture about it. It's important that this video or
essay be composed by someone with a reputation for making the complicated
accessible to mere mortals.

Good luck!

~~~
Torn
Completely agree, and motivation is definitely a factor in how I've directed
my own learning previously. Fortunately, my growing interest in this sort of
stuff is related to my day job as a coder, and I would definitely be looking
to jump in as early as possible with code to play around with new ideas
learnt.

------
RiderOfGiraffes
We might be able to help each other. I'm working on a project to help people
like you, so you can help me design that, and I can help you with your
specific concerns. Check my profile.

------
prabodh
Numerical Recipes is must read

~~~
gjm11
... but not really the sort of mathematics it sounds like the OP is interested
in learning right now. (I agree that NR is a first-rate bit of paedagogy. It's
usual for numerics experts to be sniffy about the actual algorithms, which are
often chosen for simplicity or clarity more than for their actual
performance.)

~~~
prabodh
Could HN recommend any 'math for programmers' resources or suggest structured
approaches...

I suggested Numerical recipes coz it contains the implementation of Various
Liner Algebra algorithms which form the basics of Math for Programmers..

~~~
vijayr
any recommendations to learn Fractal geometry, from the ground up?

------
zackattack
This thread comes up at least once a month. We really need an HN FAQ or Wiki.

