
Ask HN: good math books - bad_user
Dear HN, I need some insight from you.<p>While I have very solid foundation when it comes to algorithms, unfortunately my math  skills are kind of rusty and lacking.
I have big holes in my math education, partly because of bad luck with teachers in high-school/college. And the math I learned, I'm starting to forget because of lack of practice.<p>My domains of interest are data-mining, NLP and AI. These are domains where knowledge of mathematics is required to understand the underlying phenomenons. I mean, I got to some level of knowledge, but it is based on experimentation, empiric conclusions and cargo-culting.<p>Going to a local college for lectures is not an option.<p>I want some recommendations on good math books on:
calculus, discrete math, probabilities, statistics<p>I'd also like to start from a comfortable level, so I prefer books that at least start with an introduction of the prerequisites.
======
caffeine
Hi - I was in a similar situation a year ago (starting a Master's in ML,
feeling crappy at math). What worked for me was:

\- Studying is doing problems. Reading is not studying. When studying, read
the minimum amount necessary to complete the problem. Do all problems CLOSED-
book: no notes or reading while doing problems.

\- A problem is not understood until you can do it on a blank piece of paper
two days after reading the relevant material (so do problems several times to
check).

\- Study machine learning problems that _use_ the math. The examples in math
text books are pretty useless to you - do ML problems directly. When you don't
get it, look the math up quickly in a reference book (A.E.M. by Kreyszig
works), as much as needed to complete the problem, and move on.

\- Implement regularly. Once you've done the problems concerning an algorithm
or technique, implement it. Make sure your implementation is correct, and
spend some effort making it quick. Read source code for other people's
implementations.

This method is effective (for me) because I _like_ programming and machine
learning, I _don't like_ math textbooks or the problems within, and I can't
remember what I haven't solved.

Book recommendation: David MacKay's (free) book: Information Theory, Inference
and Learning Algorithms (<http://www.inference.phy.cam.ac.uk/mackay/itila/>)

\- Enjoyable: not terse or uptight, engaging reading.

\- Problem-focused: the text is really explanations between interesting
problems. The problems are spread throughout the text rather than bunched at
the end - this is a huge advantage.

\- Deep and unifying: You don't get this perspective often. Helps you think
about AI/ML from a strong theoretical perspective. Will give you lots of
"Aha!" moments.

\- Math: Plenty of math, but not unnecessarily difficult. The practice you'll
get doing these problems will serve you in other situations.

~~~
pz
i second the David MacKay recommendation. its a real pleasure to read. for me
personally, his coverage of the fundamentals of information theory was
amazing. the proofs are easy to follow and explained well so you can match the
intuition to formula.

i'd also like to recommend Boyd & Vanderberghe's book on Convex Optimization
(available here <http://www.stanford.edu/~boyd/cvxbook/>). This is maybe a
little less fun but very deep and still well written and 'illustrated'.

My background is in ML and I came to realize over time that a lot of ML is
just statistics and optimization theory rehashed by computer scientist.

~~~
ableal
Control theory may also be helpful -
<http://en.wikipedia.org/wiki/Optimal_control> lists some books, a few of them
available online.

Many moons ago, I went to a presentation with a classmate whose advisor taught
control theory. It was a seminar about neural networks, the new CS hotness
back then. Her jaw dropped when she saw the equations the CS guy was
presenting - it was the simplest control stuff she knew.

On the other hand, another former classmate with the same engineering
background (control theory, etc.), has been doing well at CS in machine
learning ....

------
sanand
My recommendations: Hardy, A Course of Pure Mathematics. Knuth, Graham and
Patashnik, Concrete Mathematics. Hoffman and Kunze, Linear Algebra (there are
several free online books on Linear Algebra and Vector Spaces, Strang, Axler,
Lang Halmos and Shilov are also recommended). Feller, An Introduction to
Probability Theory and Its Applications.

If you're not studying for exams and you have plenty of time on your hands,
try the following algorithm: 1\. Dive into some random chapter in the book and
attempt the first exercise in that chapter. 2\. Understand the problem by
referring back and learning everything that might potentially have relevance
to the exercise. 3\. Attempt to solve exercise. 4\. If unsuccessful, go to
step 2. 5\. If there are more exercises in the chapter, pick the next exercise
and go to step 2. 6\. If there are still unsolved exercises in the book, go to
step 1.

Initially, this can cause excruciating pain and frustration. But every
exercise is a little research problem and soon -- if you are the curious type
who cannot rest until you have got to the bottom of a problem -- you'll find
you've mastered the subject faster than you would have learnt its elements,
had you taken a formal classroom course. The most important benefit of this
method is that it will train you in some modes of thinking that are crucial in
research. Think of it as callisthenics for problem solving type of research!

------
rw
A few suggestions (if only we all had more time):

Do ProjectEuler.net, as it provides a great mesh between algorithm design and
mathematics; it might be within your expertise already, however. Look in to
getting a good discrete math book or two. My circumstances led me to not use
my discrete math book very much for class, so I won't give you a
recommendation that I do not have.

Go through Scholarpedia. They are doing cool work over there.

Get some text(s) on statistical inference, especially Bayesian learning. If
you have access to a good library system (say, state-wide library loan), you
ought to have no problem getting most textbooks.

Seeing as how you're into NLP and AI, look at concurrency and what, say,
Google is doing with MapReduce and PageRank (they publish reasonably-
understandable papers). In general, push yourself to read peer-reviewed
journal articles on the topics of your interest.

Peruse some of the organizations that do research into the philosophical and
human issues surrounding AI and technology. For example, there's Less Wrong at
lesswrong.com. They can come off as quite hubristic, but if you drill through
the latent groupthink and individual over-assertiveness, you'll find a
gigantic wealth of information on both what rationality is, and how to apply
it to your life.

Additionally, check out (I've mentioned these before, so I'll quote myself
verbatim):

"Problems On Algorithms" (Parberry, Gasarch): A slim volume containing ~700
exercises on proving various properties of algorithms. And it teaches you most
of what is needed. Download it from: <http://www.eng.unt.edu/ian/books/free/>

"Elements of the Theory of Computation" (Papadimitriou): Classic text. Sure
you've heard of Turing machines before, but what do you know about mu-
recursive functions? It's very beautiful and if you have a person with whom to
read the book and do the exercises, you will gain immensely. It will help
unify, in your mind, many of the myriad articles you've probably read on
Wikipedia about formal languages and complexity theory.

------
lkozma
You could read Gilbert Strang's Linear Algebra books, there are also video
lectures available...

Gelman's Bayesian Data Analysis is a good statistical intro with a Bayesian
bend,

maybe Freedman's Statistical models,

read some general machine learning intro book, such as Alpaydin or Bishop for
a more probabilistic view.

For NLP especially, you'll need some information theory. Maybe start with
MacKay's Information Theory, Inference, etc. (it is freely available online),
very entertaining and accessible book and you can follow the pointers there
for more details.

For general data mining and machine learning you need some basics of
optimization, pick up any standard book on convex optimization to get an idea
of the basic problems and methods.

that's it at a first try :)

~~~
caffeine
Don't forget Hastie & Tibshirani's Elements of Statistical Learning! :)

------
chollida1
I've seen a few people recommend The Princeton Companion to Mathematics(PCM)
<http://press.princeton.edu/titles/8350>

I own this book and while I do agree it's a great book for getting a 10000
foot view about topics in mathematics it's not really a book one read to learn
the math that the poster is asking for.

I agree with another post here that said studying is doing problems and being
able to do them again a week later without having to consult the text again.

PCM isnt a good book for learning calculus, discrete math, probability or
statistics( what the posters wants to learn) simply because it isn't designed
to teach them. There are no problems/solutions in the book to try.

If you really want one book then I'd recommend Concrete Mathamatics
[http://www.amazon.com/Concrete-Mathematics-Foundation-
Comput...](http://www.amazon.com/Concrete-Mathematics-Foundation-Computer-
Science/dp/0201558025/ref=sr_1_1?ie=UTF8&s=books&qid=1245423997&sr=1-1) by
Graham, Knuth and Patashnik. It has relevant topics and problem sets that you
can work through to see if you truly did learn anything when reading it.

------
J_McQuade
Well, if you're looking for an introduction to the basics of mathematics
(which I would recommend for anyone looking to delve deeper into any specific
area), you can't really beat 'How to Prove It' by Velleman. It covers the
fundamental concepts of mathematical analysis and proof-writing (and, perhaps
more importantly, _reading_ proofs) and will help give you a great grounding
for further reading, especially in the areas of discrete mathematics, set
theory, and probability. I can't recommend it enough as an enjoyable
introduction to the field.

Also, for a good probability textbook, I would point you to 'Probability and
Random Processes' by Grimmett and Stirzaker. It may be a bit hard-going at
first (which is why I recommend How to Prove It to get you used to rigorous
maths), but it is a very thorough text that also happens to concentrate on
many areas of interest to the average programmer (Markov chains and the like).
Together with the companion book of exercises, it should be fantastic for
self-teaching, too.

In calculus, you have so many options that I can only really suggest you go
out and find one that you like. I used 'Early Transcendentals' by Stewart in
my first year Calculus lectures and it's served as a good reference ever
since, particularly for basic vector / multi-dimensional calculus. But, like I
say, just look around.

~~~
rw
I second "How To Prove It".

------
yoshimo111
<http://www.thatquiz.org/> for practice problems.

<http://en.wikipedia.org/wiki/Mathomatic> an open source math solver, will
also output answers to programming code, can solve polynomials, do calculus,
trig...etc. You can use it to check your answers.

<http://www.math.com/> the website has a lot of topics.

You don't neccessarily 'need' a textbook. You can probably find the
information you seek all online. If you have any questions you can ask yahoo
answers.

<http://tutorial.math.lamar.edu/links.aspx>
<http://etc.usf.edu/math/index.htm> an overwhelming list of math sites. SOS
math has worked well at helping me in the past.

------
michael_dorfman
Knuth's "Concrete Mathematics" is a lot of fun.

Also, there are a lot of good courses at OCW over at MIT.
[http://ocw.mit.edu/OcwWeb/web/courses/courses/index.htm#Math...](http://ocw.mit.edu/OcwWeb/web/courses/courses/index.htm#Mathematics)

~~~
kaens
Second the Concrete Mathematics recommendation.

It's a challenging book, but approachable. It did wonders for my understanding
of maths, and for my skill as a programmer.

------
jefffoster
I'm trying to do a similar thing. I've purchased "The Princeton Companion to
Mathematics" (<http://press.princeton.edu/titles/8350.html>).

I've not read a huge amount so far, but what I've read so far has been well-
written and understandable.

------
ionfish
I rate Enderton's _A Mathematical Introduction to Logic_.

<http://www.math.ucla.edu/~hbe/amil/index.html>

------
joubert
The Princeton Companion to Mathematics.

~~~
dschobel
I'll third this recommendation. Fantastic text.

[http://www.amazon.com/Princeton-Companion-Mathematics-
Timoth...](http://www.amazon.com/Princeton-Companion-Mathematics-Timothy-
Gowers/dp/0691118809/ref=sr_1_1?ie=UTF8&s=books&qid=1245423077&sr=8-1)

------
AlleyRow
Check Out "Mathematical Tools for Physics" by James Nearing (available as a
free download at <http://www.physics.miami.edu/~nearing/mathmethods/>). It
covers a lot of great results and is one of the most easy to follow
comprehensive summaries of an undergraduate math education that I've come
across.

~~~
pasbesoin
Note also the links at the bottom of that page, to other books and sites
listing freely available books.

------
fogus
I personally liked "A Logical Approach to Discrete Math" by Gries and
Schneider which can be had for a good price on Amazon. As for statustics, I
liked "Statistics: A Gentle Introduction" by Coolidge and "Principles of
Statistics" by Bullmer which can be had for a great price even new.

Although only slightly related to your chosen topics, "Linear Algebra Done
Right" by Axler is awesome.

-m

------
crocowhile
You may also be interested in knowing the American Math Society has a big
collection of free textbooks downloadable as PDF.

<http://www.ams.org/online_bks/online_subject.html>

------
jimbokun
I just happened to find this book browsing through my campus library a few
days ago:

Basic Training in Mathematics

A Fitness Program for Science Students

Shankar, R.

<http://www.springer.com/physics/book/978-0-306-45036-5>

Meant as a review/primer for undergraduates in the physical sciences and
engineering. Mainly covers topics in calculus and linear algebra. Does a
fairly good job of covering the connections between ideas and some of the
intuition behind ideas, while keeping things pretty concise. That is my review
after finishing the first chapter, which is as far as I've got.

The "Basic Training" perspective peeks through in comments like:

"Trigonometric functions, identities and derivatives, radian measures for
angles. These will not be listed here since you must have already learned them
by heart as a child."

So don't expect to be overly molly-coddled by this book.

I'm reading it for many of the same reasons you describe. Taking a machine
learning course in the Fall, want to brush up on the underlying mathematics
involved. Statistics, calculus and linear algebra are pervasive in most of the
ML books I have looked at. I note that you did not list linear algebra, but
that is the area where I have been spending the most time, mainly because I
never took a course in it, where the other topics I have at least some
background I can review.

------
hamilton
Everyone has given really great recommendations. I second caffeine's PROCESS.
The way I learned most mathematics is from working through problems on a white
board I bought. From there I got a great grasp of probability theory and
linear algebra, which gave me the legs to study statistical learning.

While I have nothing to add on the math front, on the intuition /
implementation front you should definitely check out Chris Manning's
(currently) free textbook on Information Retrieval. He delves into the
history, technical details, and state of the art of IR. Let's hope he opens up
his NLP book as well (great CS prof at Stanford).

If humanly possible, I'd recommend making a friend who can help you solve
confusion or quandary about certain problems. This is probably less likely for
most people, but if you DO know someone with domain knowledge, I'd buy them
dinner and pick their brain. I've fully traversed the learning curve from both
wrestling with problems and talking to people much smarter than me.

[http://www-csli.stanford.edu/~hinrich/information-retrieval-...](http://www-
csli.stanford.edu/~hinrich/information-retrieval-book.html)

------
ananthrk
(Given that you are also interested in NLP) "Foundations of Statistical
Natural Language Processing" is a good bet.

------
menloparkbum
_I want some recommendations on good math books on: calculus, discrete math,
probabilities, statistics_

Calculus Made Easy by Silvanius P. Thompson (Feynman taught himself calc with
this book)

Concrete Mathematics by Knuth, Graham, Patashnik

Fundamentals of Applied Probability Theory by Alvin Drake

You probably want to learn Linear Algebra - get Gilbert Strang's book.

The Schuam's outlines in each topic are good for review. I like the ancient
ones, usually there are dozens of them for next to nothing at any good used
bookstore.

------
euroclydon
I really like the book: Elementary Classical Analysis by Jerrold E. Marsden
for Multivariate Calculus.

[http://www.amazon.com/Elementary-Classical-Analysis-
Jerrold-...](http://www.amazon.com/Elementary-Classical-Analysis-Jerrold-
Marsden/dp/0716721058)

------
tjr
This may be below your level, but one of the most interesting and unique math
books I've seen:

<http://www.math.sjsu.edu/~swann/mcsqrd.html>

------
PebblesRox
I read this textbook (Statistics Through Applications) for fun last summer:
[http://bcs.whfreeman.com/sta/default.asp?s=&n=&i=...](http://bcs.whfreeman.com/sta/default.asp?s=&n=&i=&v=&o=&ns=0&uid=0&rau=0)
Unlike most textbooks, it is actually well written. It would not take long to
finish if you skip most of the exercises and I think it provides a good
introduction to and explanation of basic ideas involved in statistics.

------
Alienz
All the Mathematics You Missed:

[http://www.amazon.com/All-Mathematics-You-Missed-
Graduate/dp...](http://www.amazon.com/All-Mathematics-You-Missed-
Graduate/dp/0521797071)

------
JimJinkins
W. W. Sawyer

Mathematician's Delight Prelude to Mathematics

What is Calculus About? This is best intro to calculus I ever saw. When I
taught introductory calculus, I recommended it to all my students.

------
keefe
I think there are some interesting lectures here
[http://www.datawrangling.com/hidden-video-courses-in-math-
sc...](http://www.datawrangling.com/hidden-video-courses-in-math-science-and-
engineering) and I occasionally find something insightful here
<http://demonstrations.wolfram.com/> but I'm in a similar position as you,
unfortunately...

------
grosales
You could start with Mathematics: A very short introduction by Timothy Gowers.
It's a quick read, and the way Gowers writes about important Mathematical
topics is interesting and, for the most part, easy to understand.
[http://www.amazon.com/Mathematics-Short-Introduction-
Timothy...](http://www.amazon.com/Mathematics-Short-Introduction-Timothy-
Gowers/dp/0192853619/)

------
sepa
Calculus by Tom Apostol

~~~
mattheww
Apostol's text is less a Calculus text and more an introduction to real
analysis. I would not recommend it to someone seeking a refresher on practical
methods.

------
mnp456
Sorry, meant this one ... [http://www.amazon.com/Mathematical-Methods-Physics-
Engineeri...](http://www.amazon.com/Mathematical-Methods-Physics-Engineering-
Comprehensive/dp/0521679710/ref=sr_1_4?ie=UTF8&s=books&qid=1245421606&sr=1-4)

------
spmacdonald
A really good book for analysis:

[http://www.amazon.com/Analysis-History-Undergraduate-
Texts-M...](http://www.amazon.com/Analysis-History-Undergraduate-Texts-
Mathematics/dp/0387770313/ref=sr_1_1?ie=UTF8&s=books&qid=1245527842&sr=8-1)

------
paulreiners
"Theory of Recursive Functions and Effective Computability" by Hartley Rogers
is excellent.

------
dinkumthinkum
Calculus the Easy Way. It's old but good.

