Hacker News new | past | comments | ask | show | jobs | submit login
Ask HN: good math books
111 points by bad_user on June 19, 2009 | hide | past | favorite | 46 comments
Dear HN, I need some insight from you.

While I have very solid foundation when it comes to algorithms, unfortunately my math skills are kind of rusty and lacking. I have big holes in my math education, partly because of bad luck with teachers in high-school/college. And the math I learned, I'm starting to forget because of lack of practice.

My domains of interest are data-mining, NLP and AI. These are domains where knowledge of mathematics is required to understand the underlying phenomenons. I mean, I got to some level of knowledge, but it is based on experimentation, empiric conclusions and cargo-culting.

Going to a local college for lectures is not an option.

I want some recommendations on good math books on: calculus, discrete math, probabilities, statistics

I'd also like to start from a comfortable level, so I prefer books that at least start with an introduction of the prerequisites.

Hi - I was in a similar situation a year ago (starting a Master's in ML, feeling crappy at math). What worked for me was:

- Studying is doing problems. Reading is not studying. When studying, read the minimum amount necessary to complete the problem. Do all problems CLOSED-book: no notes or reading while doing problems.

- A problem is not understood until you can do it on a blank piece of paper two days after reading the relevant material (so do problems several times to check).

- Study machine learning problems that use the math. The examples in math text books are pretty useless to you - do ML problems directly. When you don't get it, look the math up quickly in a reference book (A.E.M. by Kreyszig works), as much as needed to complete the problem, and move on.

- Implement regularly. Once you've done the problems concerning an algorithm or technique, implement it. Make sure your implementation is correct, and spend some effort making it quick. Read source code for other people's implementations.

This method is effective (for me) because I like programming and machine learning, I don't like math textbooks or the problems within, and I can't remember what I haven't solved.

Book recommendation: David MacKay's (free) book: Information Theory, Inference and Learning Algorithms (http://www.inference.phy.cam.ac.uk/mackay/itila/)

- Enjoyable: not terse or uptight, engaging reading.

- Problem-focused: the text is really explanations between interesting problems. The problems are spread throughout the text rather than bunched at the end - this is a huge advantage.

- Deep and unifying: You don't get this perspective often. Helps you think about AI/ML from a strong theoretical perspective. Will give you lots of "Aha!" moments.

- Math: Plenty of math, but not unnecessarily difficult. The practice you'll get doing these problems will serve you in other situations.

i second the David MacKay recommendation. its a real pleasure to read. for me personally, his coverage of the fundamentals of information theory was amazing. the proofs are easy to follow and explained well so you can match the intuition to formula.

i'd also like to recommend Boyd & Vanderberghe's book on Convex Optimization (available here http://www.stanford.edu/~boyd/cvxbook/). This is maybe a little less fun but very deep and still well written and 'illustrated'.

My background is in ML and I came to realize over time that a lot of ML is just statistics and optimization theory rehashed by computer scientist.

Control theory may also be helpful - http://en.wikipedia.org/wiki/Optimal_control lists some books, a few of them available online.

Many moons ago, I went to a presentation with a classmate whose advisor taught control theory. It was a seminar about neural networks, the new CS hotness back then. Her jaw dropped when she saw the equations the CS guy was presenting - it was the simplest control stuff she knew.

On the other hand, another former classmate with the same engineering background (control theory, etc.), has been doing well at CS in machine learning ....

My recommendations: Hardy, A Course of Pure Mathematics. Knuth, Graham and Patashnik, Concrete Mathematics. Hoffman and Kunze, Linear Algebra (there are several free online books on Linear Algebra and Vector Spaces, Strang, Axler, Lang Halmos and Shilov are also recommended). Feller, An Introduction to Probability Theory and Its Applications.

If you're not studying for exams and you have plenty of time on your hands, try the following algorithm: 1. Dive into some random chapter in the book and attempt the first exercise in that chapter. 2. Understand the problem by referring back and learning everything that might potentially have relevance to the exercise. 3. Attempt to solve exercise. 4. If unsuccessful, go to step 2. 5. If there are more exercises in the chapter, pick the next exercise and go to step 2. 6. If there are still unsolved exercises in the book, go to step 1.

Initially, this can cause excruciating pain and frustration. But every exercise is a little research problem and soon -- if you are the curious type who cannot rest until you have got to the bottom of a problem -- you'll find you've mastered the subject faster than you would have learnt its elements, had you taken a formal classroom course. The most important benefit of this method is that it will train you in some modes of thinking that are crucial in research. Think of it as callisthenics for problem solving type of research!

A few suggestions (if only we all had more time):

Do ProjectEuler.net, as it provides a great mesh between algorithm design and mathematics; it might be within your expertise already, however. Look in to getting a good discrete math book or two. My circumstances led me to not use my discrete math book very much for class, so I won't give you a recommendation that I do not have.

Go through Scholarpedia. They are doing cool work over there.

Get some text(s) on statistical inference, especially Bayesian learning. If you have access to a good library system (say, state-wide library loan), you ought to have no problem getting most textbooks.

Seeing as how you're into NLP and AI, look at concurrency and what, say, Google is doing with MapReduce and PageRank (they publish reasonably-understandable papers). In general, push yourself to read peer-reviewed journal articles on the topics of your interest.

Peruse some of the organizations that do research into the philosophical and human issues surrounding AI and technology. For example, there's Less Wrong at lesswrong.com. They can come off as quite hubristic, but if you drill through the latent groupthink and individual over-assertiveness, you'll find a gigantic wealth of information on both what rationality is, and how to apply it to your life.

Additionally, check out (I've mentioned these before, so I'll quote myself verbatim):

"Problems On Algorithms" (Parberry, Gasarch): A slim volume containing ~700 exercises on proving various properties of algorithms. And it teaches you most of what is needed. Download it from: http://www.eng.unt.edu/ian/books/free/

"Elements of the Theory of Computation" (Papadimitriou): Classic text. Sure you've heard of Turing machines before, but what do you know about mu-recursive functions? It's very beautiful and if you have a person with whom to read the book and do the exercises, you will gain immensely. It will help unify, in your mind, many of the myriad articles you've probably read on Wikipedia about formal languages and complexity theory.

You could read Gilbert Strang's Linear Algebra books, there are also video lectures available...

Gelman's Bayesian Data Analysis is a good statistical intro with a Bayesian bend,

maybe Freedman's Statistical models,

read some general machine learning intro book, such as Alpaydin or Bishop for a more probabilistic view.

For NLP especially, you'll need some information theory. Maybe start with MacKay's Information Theory, Inference, etc. (it is freely available online), very entertaining and accessible book and you can follow the pointers there for more details.

For general data mining and machine learning you need some basics of optimization, pick up any standard book on convex optimization to get an idea of the basic problems and methods.

that's it at a first try :)

Don't forget Hastie & Tibshirani's Elements of Statistical Learning! :)

I've seen a few people recommend The Princeton Companion to Mathematics(PCM) http://press.princeton.edu/titles/8350

I own this book and while I do agree it's a great book for getting a 10000 foot view about topics in mathematics it's not really a book one read to learn the math that the poster is asking for.

I agree with another post here that said studying is doing problems and being able to do them again a week later without having to consult the text again.

PCM isnt a good book for learning calculus, discrete math, probability or statistics( what the posters wants to learn) simply because it isn't designed to teach them. There are no problems/solutions in the book to try.

If you really want one book then I'd recommend Concrete Mathamatics http://www.amazon.com/Concrete-Mathematics-Foundation-Comput... by Graham, Knuth and Patashnik. It has relevant topics and problem sets that you can work through to see if you truly did learn anything when reading it.

Well, if you're looking for an introduction to the basics of mathematics (which I would recommend for anyone looking to delve deeper into any specific area), you can't really beat 'How to Prove It' by Velleman. It covers the fundamental concepts of mathematical analysis and proof-writing (and, perhaps more importantly, reading proofs) and will help give you a great grounding for further reading, especially in the areas of discrete mathematics, set theory, and probability. I can't recommend it enough as an enjoyable introduction to the field.

Also, for a good probability textbook, I would point you to 'Probability and Random Processes' by Grimmett and Stirzaker. It may be a bit hard-going at first (which is why I recommend How to Prove It to get you used to rigorous maths), but it is a very thorough text that also happens to concentrate on many areas of interest to the average programmer (Markov chains and the like). Together with the companion book of exercises, it should be fantastic for self-teaching, too.

In calculus, you have so many options that I can only really suggest you go out and find one that you like. I used 'Early Transcendentals' by Stewart in my first year Calculus lectures and it's served as a good reference ever since, particularly for basic vector / multi-dimensional calculus. But, like I say, just look around.

I second "How To Prove It".

Velleman's book with Alexander George, Philosophies of Mathematics, is a good introduction to that subject, and has a nice walkthrough of ZFC.

http://www.thatquiz.org/ for practice problems.

http://en.wikipedia.org/wiki/Mathomatic an open source math solver, will also output answers to programming code, can solve polynomials, do calculus, trig...etc. You can use it to check your answers.

http://www.math.com/ the website has a lot of topics.

You don't neccessarily 'need' a textbook. You can probably find the information you seek all online. If you have any questions you can ask yahoo answers.

http://tutorial.math.lamar.edu/links.aspx http://etc.usf.edu/math/index.htm an overwhelming list of math sites. SOS math has worked well at helping me in the past.

Knuth's "Concrete Mathematics" is a lot of fun.

Also, there are a lot of good courses at OCW over at MIT. http://ocw.mit.edu/OcwWeb/web/courses/courses/index.htm#Math...

Second the Concrete Mathematics recommendation.

It's a challenging book, but approachable. It did wonders for my understanding of maths, and for my skill as a programmer.

I'm trying to do a similar thing. I've purchased "The Princeton Companion to Mathematics" (http://press.princeton.edu/titles/8350.html).

I've not read a huge amount so far, but what I've read so far has been well-written and understandable.

I rate Enderton's A Mathematical Introduction to Logic.


The Princeton Companion to Mathematics.

I'll third this recommendation. Fantastic text.


What do people love about this book so much? (I haven't read it).

It's an encyclopedia of essays on math topics. It's very comprehensive and well edited.

However, while it's a cool book, speaking as a former math tutor, it isn't the most helpful from a self study perspective.

It also has a positive side of being impressively huge and it certainly creates a ftw moment for the friends.

I got i two days ago and have skimmed around: Absolutely recommendable!

OK, I'm about to order it, but I feel as though I might be a little unprepared for its contents - is there another book that I might be able to use to warm up?

I like James R. Newman's "The World of Mathematics." It's a four volume set and you can often find it at a used bookstore for next to nothing.

Check Out "Mathematical Tools for Physics" by James Nearing (available as a free download at http://www.physics.miami.edu/~nearing/mathmethods/). It covers a lot of great results and is one of the most easy to follow comprehensive summaries of an undergraduate math education that I've come across.

Note also the links at the bottom of that page, to other books and sites listing freely available books.

I personally liked "A Logical Approach to Discrete Math" by Gries and Schneider which can be had for a good price on Amazon. As for statustics, I liked "Statistics: A Gentle Introduction" by Coolidge and "Principles of Statistics" by Bullmer which can be had for a great price even new.

Although only slightly related to your chosen topics, "Linear Algebra Done Right" by Axler is awesome.


You may also be interested in knowing the American Math Society has a big collection of free textbooks downloadable as PDF.


I just happened to find this book browsing through my campus library a few days ago:

Basic Training in Mathematics

A Fitness Program for Science Students

Shankar, R.


Meant as a review/primer for undergraduates in the physical sciences and engineering. Mainly covers topics in calculus and linear algebra. Does a fairly good job of covering the connections between ideas and some of the intuition behind ideas, while keeping things pretty concise. That is my review after finishing the first chapter, which is as far as I've got.

The "Basic Training" perspective peeks through in comments like:

"Trigonometric functions, identities and derivatives, radian measures for angles. These will not be listed here since you must have already learned them by heart as a child."

So don't expect to be overly molly-coddled by this book.

I'm reading it for many of the same reasons you describe. Taking a machine learning course in the Fall, want to brush up on the underlying mathematics involved. Statistics, calculus and linear algebra are pervasive in most of the ML books I have looked at. I note that you did not list linear algebra, but that is the area where I have been spending the most time, mainly because I never took a course in it, where the other topics I have at least some background I can review.

Everyone has given really great recommendations. I second caffeine's PROCESS. The way I learned most mathematics is from working through problems on a white board I bought. From there I got a great grasp of probability theory and linear algebra, which gave me the legs to study statistical learning.

While I have nothing to add on the math front, on the intuition / implementation front you should definitely check out Chris Manning's (currently) free textbook on Information Retrieval. He delves into the history, technical details, and state of the art of IR. Let's hope he opens up his NLP book as well (great CS prof at Stanford).

If humanly possible, I'd recommend making a friend who can help you solve confusion or quandary about certain problems. This is probably less likely for most people, but if you DO know someone with domain knowledge, I'd buy them dinner and pick their brain. I've fully traversed the learning curve from both wrestling with problems and talking to people much smarter than me.


(Given that you are also interested in NLP) "Foundations of Statistical Natural Language Processing" is a good bet.

I want some recommendations on good math books on: calculus, discrete math, probabilities, statistics

Calculus Made Easy by Silvanius P. Thompson (Feynman taught himself calc with this book)

Concrete Mathematics by Knuth, Graham, Patashnik

Fundamentals of Applied Probability Theory by Alvin Drake

You probably want to learn Linear Algebra - get Gilbert Strang's book.

The Schuam's outlines in each topic are good for review. I like the ancient ones, usually there are dozens of them for next to nothing at any good used bookstore.

I really like the book: Elementary Classical Analysis by Jerrold E. Marsden for Multivariate Calculus.


This may be below your level, but one of the most interesting and unique math books I've seen:


I read this textbook (Statistics Through Applications) for fun last summer: http://bcs.whfreeman.com/sta/default.asp?s=&n=&i=... Unlike most textbooks, it is actually well written. It would not take long to finish if you skip most of the exercises and I think it provides a good introduction to and explanation of basic ideas involved in statistics.

W. W. Sawyer

Mathematician's Delight Prelude to Mathematics

What is Calculus About? This is best intro to calculus I ever saw. When I taught introductory calculus, I recommended it to all my students.

I think there are some interesting lectures here http://www.datawrangling.com/hidden-video-courses-in-math-sc... and I occasionally find something insightful here http://demonstrations.wolfram.com/ but I'm in a similar position as you, unfortunately...

You could start with Mathematics: A very short introduction by Timothy Gowers. It's a quick read, and the way Gowers writes about important Mathematical topics is interesting and, for the most part, easy to understand. http://www.amazon.com/Mathematics-Short-Introduction-Timothy...

Calculus by Tom Apostol

Apostol's text is less a Calculus text and more an introduction to real analysis. I would not recommend it to someone seeking a refresher on practical methods.

You can probably skip Volume I.

"Theory of Recursive Functions and Effective Computability" by Hartley Rogers is excellent.

Calculus the Easy Way. It's old but good.

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact