
Ask HN: Learning advanced math - linhir
I have an undergraduate background in applied math and have taken basic linear algebra, differential equations, mathematical statistics and multivariate calculus. I'm rusty though, and I've been considering applying for PhD programs in statistics. I'd like to put myself on a healthy math regiment and I was wondering if people had suggestions on books or other materials to work on advanced (linear) algebra and analysis? I'm more than willing to spend an hour per page and do all the exercises, but I would like good exposition. My end goal is to have a reasonable understanding of things to make limit theorems in probability (during the first year of my PhD), etc, easier.
======
forkandwait
For real analysis, I suggest Bartle's "Introduction...". I had a horrible
teacher (well, I loved him, but he never prepped for class), and was able to
learn everything by working through the proofs in the text. Bartle has a
multi-dimensional version too.

For linear algebra, I was suggest Strang's "Linear Algebra with applications",
3rd edition. Then "Linear Algebra Done Right".

While you are at it, get Hungerfords "Abstract Algebra: An Introduction"; you
will need an easy reference to fields and groups and polynomials.

All these books require an hour per page, but they lay it all out for you if
you work for it. These are definitely undergrad books, but that is their
beauty.

Bartle also has "Elements of integration and lebesgue measure" -- I bet it is
great, but I haven't used it.

And if you find a good probability book, please post the title ;)

~~~
btilly
Upvoted for suggesting _Linear Algebra Done Right_.

To get a sense of the approach, read his earlier article _Down with
Determinants_ at <http://www.axler.net/DwD.html>. That article was the first
time I felt I really understood a lot of the material I had already
theoretically learned and done well at.

On the real analysis side, I learned from Royden and liked it. But my mind
definitely heads the analysis way, and what I find easy someone else might
not.

~~~
dbfclark
I definitely enjoyed Royden a lot -- I got through my real analysis qual back
in grad school using Royden to teach myself essentially all of the material.
Readable and excellent. And as to minds heading the analysis way: I'm about as
algebraicly-minded as they come and I thought it was highly readable.

------
npp
Advanced Linear Algebra, Roman; Linear Algebra, Hoffman & Kunze; Matrix
Analysis, Horn & Johnson; Principles of Mathematical Analysis, Rudin; Real
Analysis, Royden.

Miscellaneous comments:

\- Reading pure abstract algebra (e.g. Dummit & Foote) isn't a good use of
time if you intend to go into statistics, since it only shows up in a few very
special subareas. If you decide to go into one of these areas, you can learn
this later.

\- More advanced books on linear algebra usually emphasize the abstract study
of vector spaces and linear transformations. This is fine, but you also need
to learn about matrix algebra (some of which is in that Horn & Johnson book)
and basic matrix calculus, since in statistics, you'll frequently be
manipulating matrix equations. The vector space stuff generally does not help
with this, and this material isn't in standard linear algebra books.
(Similarly, you should learn the basics of numerical linear algebra and
optimization -- convex optimization in particular shows up a lot in
statistics.)

\- People have different opinions on books like Rudin, but you need to learn
to read material like this if you're going into an area like probability. It's
also more or less a de facto standard, so it is worth reading partly for that
reason as well. So read Rudin/Royden (or equivalent, there are a small handful
of others), but supplement them with other books if you need (e.g. 'The Way of
Analysis' is the complete opposite of Rudin in writing style). It helps to
read a few different books on the same topic simultaneously, anyway.

\- Two books on measure-theoretic probability theory that are more readable
than many of the usual suspects are "Probability with Martingales" by Williams
and "A User's Guide to Measure-Theoretic Probability" by Pollard. There is
also a nice book called "Probability through Problems" that develops the
theory through a series of exercises.

------
dmvaldman
The roadmap would be to first learn analysis (advanced calculus), then measure
theory, then get a measure theory probability book.

I'll assume you are teaching this to yourself.

For analysis, I wouldn't get Rudin's book (concepts are poorly motivated).
There are plenty of good Dover books. But I haven't read them because, well, I
learned from Rudin.

For measure theory, I'd read Kolmogorov and Fomin's book. Rudin also has a
measure theory book, which is much better than his analysis book, but it's
hefty. Good problems though.

For a book on Probability, we read A Probability Path at my university. I
wasn't fond of it. Someone referred me to Probability with Martingales, and
though I didn't read it, it looked very good.

------
hyperbovine
One of my favorite not-much-talked about analysis books is "A Radical Approach
to Real Analysis" by Bressoud. Must read if you enjoy reading about the
history of mathematics and famous mathematicians. A lot of people here
recommend Rudin or Royden, but I have seen many people become turned off to
the whole field of analysis because of how terse and user-unfriendly those
are. In that sense, ARTRA is the polar opposite. The follow-up, A Radical
Approach to Lebesgue's Theory of Integration, is also superb.

------
okmjuhb
Baby Rudin is a great choice, especially for self study, since there are a
number of study guides for it that go through solutions, additional
explanation of material, etc. I'd recommend against the chapters on
differential forms, though, just because the treatment is outdated. Marsden
and Hoffman's approach is to focus on explanations for why theorems are true
before giving proofs, which some people find useful.

Axler is a good choice for linear algebra. Dummit and Foote is the standard
choice for algebra generally. I'm of the opinion that we should teach algebra
before linear algebra in general, but this seems like a minority view.

~~~
cparedes
I second Baby Rudin.

The problem sets are nearly legendary and the writing is terse. I might have
to read through it again myself sometime.

"Algebra" by Artin is also a great choice if you want to learn about modern
algebra. There's some good stuff in there for looking at linear
transformations in, say, 2-space, as groups.

------
anatoly
For linear algebra, Sheldon Axler's _Linear Algebra Done Right_. For analysis,
Stephen Abbott's _Understanding Analysis_.

These two books will give you a very solid grounding in the undergraduate
linear algebra and analysis. Personally, I also worship the style of Baby
Rudin (it's the nickname of his _The Principles of Mathematical Analysis_),
but it can be too dry to many people.

------
jedbrown
Trefethen and Bau is a good numerical linear algebra text, a fair bit of
statistics can be understood in the linear algebra setting and this book helps
teach how to think about linear algebra in a very useful way. I don't have a
great reference, but stochastic PDEs is definitely a hot topic and promises to
remain as such for a long time.

------
mx12
I would recommend video lectures from the Khan Academy
(<http://www.khanacademy.org/>). They are organized in nice little chucks of
topics and have a large range of topics in math.

~~~
thebooktocome
The Khan academy ends where OP says he left off. (i.e., advanced linear
algebra).

------
synacksynack
Michaek Spivak's "Calculus" is pretty great for learning real analysis. The
other canonical text is "Principles of Mathematical Analysis" by Walter Rudin,
which I've never used, but is purported to be of the same quality.

~~~
joe_the_user
Actually,

I wouldn't recommend either of those two book for self-teaching.

For whatever reason, I taught myself advance mathematics in High Schools.
Spivak and Rubin were pretty inaccessible. Sure, they are rigorous and high
level but that meant they didn't lend themselves to an easy read. "Real
Analysis" by Royden was relatively quick to go through - and I had only the
start of calculus. Royden gives a fairly simplistic and accessible development
of higher mathematics starting with set theory.

I'd imagine that would be the most helpful.

(In my High School years, I went from algebra sophomore year to reading Royden
and doing a community college calculus class junior year to passing a
differential geometry course and the undergraduate honors seminar at UCLA).

~~~
zackattack
op, i don't know your mind but i agree, Rudin/Spivak may prove challenging,
especially without the help of a professor/TA to answer your questions in real
life... i would not go and buy them blindly. the best course of action you
could take would be to go visit a college bookstore and sit down with one of
the analysis texts, and then read through it for a few hours. if you feel
comfortable with it and don't experience any contempt for the author for
introducing terms without explaining them, etc., then go ahead and get it. i
would recommend finding something that holds your hand through the new fancy
notation of proofs (for all, exists, epsilon, blah blah blah)...something that
combines a basic intro to topology/set theory in its introduction

------
tychonoff
Walter Rudin's book is a masterpiece. I read this book 35 years ago for my
comps, and remember it as one of the best. Bear in mind there's no easy way to
learn mathematics - it's going to be hard no matter what approach you take
because that's the nature of the subject. My advice to undergrads is to skip
computing completely because it's much easier to learn when you need it
(unlike mathematics). You'll build your first computer program long before you
prove your first theorem.

------
mseebach
I've had "The Princeton Companion to Mathematics" sitting in my Amazon wish
list for quite a while now, more or less for this purpose - I think on a HN
recommendation.

Any thoughts on this book?

~~~
jey
This is a great book to sit down and read a few articles from (at least if
you're into reading encyclopedias ;)), but you aren't going to learn much
"actual math" from it.

------
johnwatson11218
I highly recommend this book for Real Analysis
[http://www.amazon.com/Introduction-Analysis-Maxwell-
Rosenlic...](http://www.amazon.com/Introduction-Analysis-Maxwell-
Rosenlicht/dp/0486650383/ref=pd_sim_b_5)

We used it at the University of Texas at Austin for the first semester in Real
Analysis. I found it very clear and easy to follow.

~~~
talbina
Why are these books from Dover Publications so cheap?

~~~
pingswept
Because the copyright has expired, or the copyright owner has put the work in
the public domain due to bankruptcy, or similar.

------
caesium
Rosenlicht. Linear Algbera done Right. Then Finite Dimensional Vector Spaces.
Halmos' Measure Theory. Awesome.

------
pmb
"Naive Set Theory" by Halmos is a true gem, and readable, too.

------
__Rahul
You will find course material from MIT OCW mathematics to be very useful:

<http://ocw.mit.edu/courses/mathematics/>

------
sharvil
iTunes U has some amazing lectures. Gilbert Strang (MIT) has some video
lectures (MIT OCW) on linear algebra. They are a bit dull, but they are pretty
decent.

I also recommend academicearth.org.

If you are near a university, take a class "mathematical physics". These kind
of classes usually cover a lot of undergraduate material in a semester and are
offered by many physics department. They usually use "Mathematical Methods" by
Boas as a text.

------
mjcohen
Polya and Szego's "Problems and Theorems in Analysis I & II" are classics. If
you can work through any part, you will have learned a lot.

------
SamReidHughes
How can nobody here mention Courant and John's _Introduction to Calculus and
Analysis_? I think I'm going to cry.

------
cph
Can anyone suggest a good probability or statistics book?

~~~
hyperbovine
If you already know analysis (well), the best probability book is "Probability
& Measure" by Billingsley. A gentler introduction is the SUMS book "Measure,
Probability, and Integral."

If you don't care about measure theory and just want to learn how to calculate
the probability that a coin comes up heads in the first five attempts, the
book by Larsen and Marx is pretty good.

------
HilbertSpace
Your

"I'm more than willing to spend an hour per page and do all the exercises, but
I would like good exposition."

is essentially necessary and sufficient.

If an exercise takes more than two hours, then swallow your pride and skip the
exercise (it may be _misplaced_ , have an error, or just be way too difficult
for effective education).

For linear algebra:

(1) Work through a few introductory texts.

(2) Work carefully through the long time, unchallenged, world-class classic,

Halmos, _Finite Dimensional Vector Spaces_.

and there note near the back his cute ergodic convergence theorem.

The glory here is the polar decomposition.

(3) Get some contact with some applications, including in elementary multi-
variate statistics, numerical techniques, optimization, etc.

(4) Pick from

Horn and Johnson, _Matrix Analysis._

and

Horn and Johnson, _Topics in Matrix Analysis._

My first course was an "advanced course" from one of Horn, Johnson, and I
knocked the socks off all the other students. How'd I do that? Brilliant?
Worked hard? Learned a lot? Nope. Instead the key was just my independent work
with (1) -- (3).

So if you do (1) -- (4), then you will be fine.

For analysis, (Baby Rudin)

Walter Rudin, _Principles of Mathematical Analysis._

Note in the back that a function is Riemann integrable if and only if it is
continuous everywhere except on a set of Lebesgue measure 0.

Also know cold that a uniform limit of continuous functions is continuous.

Royden, _Real Analysis._

and the first, real, half of (Papa Rudin)

Rudin, _Real and Complex Analysis._

Of course, emphasize the Radon-Nikodym theorem; I like the easy steps in
Royden and Loeve (below), but see also the von Neumann proof in Papa Rudin.

For probability based on measure theory and the limit theorems,

Breiman, _Probability._

Note his result on regular conditional probabilities.

Neveu, _Mathematical Foundations of the Calculus of Probability._

If you can work all the Neveu exercises, then someone should buy you a _La
Tache_ 1961.

Loeve, _Probability Theory._

Note the classic Sierpinski counterexample exercise on regular conditional
probabilities (also in Halmos, _Measure Theory_ ),

Cover the Lindeberg-Feller version of the central limit theorem as well as
simpler versions. Do the weak law of large numbers as an easy exercise. Cover
the martingale convergence theorem (I like Breiman here) and use it to give
the nicest proof of the strong law of large numbers. Cover the ergodic theorem
(Garcia's proof) and its (astounding) application to Poincare recurrence.
Cover the law of the iterated logarithm and its (astounding) application to
the growth of Brownian motion.

Of course apply the Radon-Nikodym theorem and conditioning to sufficient
statistics and note that order statistics are always sufficient. Show that
sample mean and variance are sufficient for i.i.d. Gaussian samples and extend
to the exponential family.

Give yourself an exercise: In Papa Rudin, just after the Radon-Nikodym
theorem, note the Hahn decomposition and use it to give a quite general proof
of the Neyman-Pearson lemma.

To appreciate the law of large numbers in statistics, read the classic Halmos
paper on minimum variance, unbiased estimation.

For tools for research in statistics, might want to get going in stochastic
processes. So, for elementary books, look for authors Karlin, Taylor, and
Cinlar and touch on some applications, e.g., Wiener filtering and power
spectral estimation. Note the axiomatic derivation of the Poisson process and
the main convergence theorem in finite Markov chains (also a linear algebra
result). Then for more, note again the relevant sections of Breiman and Loeve
and then:

Karatzas and Shreve, _Brownian Motion and Stochastic Calculus._

