
Ask HN: Learn Math the Hard Way - nsomaru
I am looking to teach myself Mathematics, mainly to improve my thinking processes and learn more about the basis for CS based topics like algorithms and proofs, and other practical applications.<p>Suggestions for resources? (textbooks, videos, etc)
======
hendzen
Start with these topics. Work your way through them, and make sure you
understand everything you have read so far before progressing. Do the
exercises, or it will be a waste of time.

0) Algebra, Trigonometry, Calculus

Make sure you have a decent grasp over high school level math topics. You
might not need to use these topics frequently (though trig comes up a
surprising amount), but they are necessary to establish a base level of
mathematical maturity.

1) Linear Algebra

Obviously very important if you want to do any 3d work, but it comes up in
later topics like graph theory.

Suggested book: Linear Algebra and Its Application - Strang

2) Discrete Math

You need an understanding of proofs and logic before you can get to the real
algorithms material. Counting and probability is also very important.

Suggested book: Discrete Mathematics and Its Applications - Kenneth Rosen

This looks like a good free option, but there are no exercises:
<http://courses.csail.mit.edu/6.042/fall10/mcs-ftl.pdf>

3) Algorithms

Enough said.

Suggested book: CLRS, but here's a free option:
<http://www.cs.berkeley.edu/~vazirani/algorithms.html>

This should take you some time as each of these topics usually corresponds to
a college class. When you finish with those, start reading about CS Theory,
Combinatorics, or pick up a graduate Algorithms text.

~~~
sshrin
You say 'trig comes up a surprising amount'. It would be great if you could
give a few examples of these instances. I have always thought of trigonometry
as one of the areas I really enjoyed in school but something I have never used
ever since.

~~~
Dn_Ab
I think the surprising commonality is conditional. So you won't see it unless
you are focusing on physics or signal processing.

~~~
ten_fingers
Trig? Fourier series and its introduction to functional analysis and Hilbert
space and the ideas of inner products and orthogonality there. JPG image
compression. Shannon's information theory. Characterization of time invariant
linear systems, e.g., acoustics. Antenna theory. Phased arrays and beam
forming. Imaging in optics. Holography. The fast Fourier transform. Seismic
data analysis via the fast Fourier transform. Characteristic functions and
Bochner's theorem. Power spectra. X-ray crystallography. The fundamental
theorem of interpolation and the Nyquist sampling theorem (e.g., how music CDs
work).

There's a cute idea to put 100 wireless customers all on the same wavelength.
Basically have a tricky antenna pattern with a lobe for each user and so that
each user gets only their own signal. It's all trig.

~~~
Dn_Ab
Indeed, I agree. Those all fall under the umbrella I gave. But those are all
particular subject areas that many people can go through their career without
touching.

~~~
ten_fingers
> Those all fall under the umbrella I gave.

Yes, all or nearly all! That the trig functions are the source of the most
important orthogonal basis in Hilbert space theory need not be good to know
just for physics or signal processing!

Let's see: Also I mentioned characteristic functions and Bochner's theorem
which are core probability and not just physics or signal processing.
Characterization of time invariant linear systems might be in mechanical
engineering and might be tried even in economics! Seismic data analysis via
the fast Fourier transform is, yes, in signal processing but is also mostly
regarded as geology or just looking for oil or anything 'down there'.

I tried, as you can see, I really tried, to show how trig was for more than
just your father's topics in physics and signal processing. I tried!

------
drostie
You may not come from the same culture that I came from, but the early
significant milestones in my mathematical education were:

    
    
        1. Reading the "aha!" books by Martin Gardner, as a child.
        2. Reading Lockhart's "A Mathematician's Lament" [1]
        3. Linear algebra, calculus, and complex analysis classes. I was taught
        these at Cornell; you might look for them via MIT's OpenCourseWare [2]. 
        4. A bunch of combinatorics, from Cornell's classes in probability.
    

The most key thing I can tell you about mathematics is, _always go back and
read the definitions_. Always. I have sometimes helped out math students on
topics which were way out of my depth, simply because I picked up their
textbooks, flipped several pages back, and said "hm, what does this word mean?
what does that word mean?", building up connections all the while.

You may also want to start in a _completely_ different direction, since you're
interested more in CS-based topics (I am interested more in physics-based
topics) by starting with number theory and modular arithmetic, and of course
MIT has a Comp Sci section [3] which also produces lectures on OpenCourseWare;
you may wish to watch those lectures.

[1] <http://www.maa.org/devlin/LockhartsLament.pdf>

[2] <http://ocw.mit.edu/courses/mathematics/>

[3] [http://ocw.mit.edu/courses/electrical-engineering-and-
comput...](http://ocw.mit.edu/courses/electrical-engineering-and-computer-
science/)

~~~
dwc
FYI, Lockhart has a new book coming[1] out with the kinds of things we _should
be doing_ , which is something I (and probably everyone who read & liked his
Lament) have been pining for. I'm waiting like everyone else, so I can't
explicitly recommend it. But I certainly anticipate a mental treat.

1\. [http://www.amazon.com/Measurement-Paul-
Lockhart/dp/067405755...](http://www.amazon.com/Measurement-Paul-
Lockhart/dp/0674057554/ref=pd_rhf_ee_p_t_1)

------
1331
_Concrete Mathematics_ [1] is a good book that is fun to work through. The
book can be considered an introduction/prerequisite to Knuth's _The Art of
Computer Programming_ series [2], which contains much mathematics. If you
would like a book that is not so difficult, _Introduction to Algorithms_ [3]
(aka "CLRS") is also highly recommended.

[1] <http://www.amazon.com/dp/0201558025/>

[2] <http://www.amazon.com/dp/0321751043/>

[3] <http://www.amazon.com/dp/0262033844/>

------
zedshaw
Most math books suck. The one exception, and the first book that made me
realize you _can_ teach complex topics to regular folks, is Calculus Made Easy
by Silvanus P. Thompson and Martin Gardner. It's been around since the late
1800's and it's what I finally learned calculus from.

I haven't found many other books that explain calculus as well as that one. If
you get that and then a book of exercises from Schaum's outlines, you'll be
able to get pretty good at it. Also, the book is a really good read for a math
book.

~~~
chris_wot
Calculus: An Intuitive Approach by the late Morris Klein is pretty good too. I
found it refreshingly clear!

------
nimeshneema
The best text-book I can suggest for you is: "How to Think About Algorithms -
Loop Invariants and Recursion" By Jeff Edmonds.

It takes you through all the relevant mathematics required to learn about
basics of CS based topics you are looking for (algorithms, proofs and other
practical applications).

It also acts as a good primer for learning basic Computer Algorithms. The USP
of book is it's noob friendly and I think perfectly suited to your needs.

Check it out here: [http://www.amazon.com/Think-About-Algorithms-Jeff-
Edmonds/dp...](http://www.amazon.com/Think-About-Algorithms-Jeff-
Edmonds/dp/0521614104/)

------
ether
I don't think you need to learn really hard core math to do CS. In my
experience, the one most important "genre" of math was discrete mathematics.
My textbook was Elements of Discrete Mathematics, and I think it was a really
great intro: [http://www.amazon.com/Elements-Discrete-Mathematics-
Computer...](http://www.amazon.com/Elements-Discrete-Mathematics-Computer-
Science/dp/007038133X/)

------
madmanwalking
The Khan academy (www.khanacademy.org) has plenty of videos and exercises, and
if you're looking more at the CS side of things udacity.com runs a few online
courses looking at algorithms, discrete maths and the like.

~~~
tonetheman
This was going to be my suggestion. Excellent resource.

------
mdkess
When I was in early university, I went through Introduction to Algorithms and
chapter by chapter, I implemented nearly every algorithm presented in the
book. At the time, I had no idea why I was doing it (fun? ACM practice?), but
in hindsight it was incredibly important. I learn by doing, not by reading (at
least doing cements what I read), so I make sure that I incorporate a lot of
practice into my plans.

~~~
viscanti
That's really the basis for the "Learn X the hard way" tutorials. There's a
number of books that propose to teach a topic quickly/easily, with the
shortcut being that you won't have to go through all the exercises. It turns
out that's an ineffective teaching style for most computer/science topics. For
many of these topics, you really can't short-cut that process of implementing
solutions and learning from doing.

------
Someone
For getting to know what is a proof in Mathematics, I recommend Proofs and
Refutations by Imre Lakatos. It will not teach you many theorems (in fact, 80%
of it only proofs one, <http://en.wikipedia.org/wiki/Euler_characteristic>
again and again and again), and it will almost certainly not be applicable to
writing programs but it will give insight into how mathematicians in the last
century have made mathematics much more rigorous (and abstract).

------
elssar
Well you haven't mentioned how much math you're familiar with. But I'll assume
that you know school level mathematics and are looking to learn more about the
math mostly used in computer science. Get Concrete Mathematics(
[http://www.amazon.com/Concrete-Mathematics-Foundation-
Comput...](http://www.amazon.com/Concrete-Mathematics-Foundation-Computer-
Science/dp/0201558025) ) by Graham, Knuth & Patashnik - It's based on the
course taught at Stanford since 1970 & it's really really good. Another book I
think you should get is Meta Math: The quest for Omega(
[http://www.amazon.com/Meta-Math-Quest-Gregory-
Chaitin/dp/140...](http://www.amazon.com/Meta-Math-Quest-Gregory-
Chaitin/dp/1400077974) ) by Gregory Chaitin. It's not really a math course
book, and is mostly the authors take on his favorite topics in math, but if
you want to improve your thinking process then this book will definitely give
you a different perspective.

------
alex-g
You might like "To Mock a Mockingbird" by Raymond Smullyan. It's a book of
logic puzzles with a focus on combinatory logic (a model of computation
related to the lambda calculus). Aside from the logic/math/CS nature of the
content, the progression of the exercises should help with mathematical-style
reasoning in general.

~~~
psykotic
As fond as I am of the Mockingbird book and Smullyan's books in general, I'd
say that most of its puzzles are pure exercises in abstract symbol
manipulation and not terribly enlightening.

If you have some prior exposure to the SKI combinators, one big aha-moment may
come from the book's exploration of several alternative axiom systems that,
while preceding the SK system chronologically, are often left out of modern
CS-oriented treatments of combinatory logic.

Also, while I think the exercises on symbol manipulation are not very
enlightening on their own, the syntactic rewriting point of view is a useful
complement if your view of combinatory logic has been mostly semantic. Proving
completeness in the syntactic framework becomes a matter of showing that any
combinatory rewriting rule can be expressed in terms of your base combinators.
This in turn can be done if your combinators allow arbitrary permutations to
be expressed in addition to duplication and dropping of symbols (cf.
lengthening and shortening rules in GEB). For example, K is a dropping
combinator, and rather than viewing S as a substitution operator on the
semantic level (as is done when translating lambda calculus to SK calculus),
it can be viewed as a hybrid permutation and duplication combinator at the
syntactic level. If these dual roles are split into separate combinators, you
get alternative axiom systems.

------
gmoes
Here's a post I did on free math resources, including a list of resources on
various topics:

[http://www.elegantcoding.com/2011/06/free-math-resources-
you...](http://www.elegantcoding.com/2011/06/free-math-resources-you-can-
use.html)

There's some really great stuff listed there, I use it all the time.

------
Arun2009
In addition to Concrete Mathematics by Knuth, which contains a lot of
Mathematical tools and techniques useful in areas of Computer Science, I'd
recommend the following to learn how good problem solvers approach
Mathematics:

1\. The Art and Craft of Problem Solving by Paul Zeitz.

2\. How to Solve It by George Polya.

I'd also recommend learning any of the competition Mathematics topics like
number theory, combinatorics or geometry as base material to improve your
problem solving skills. I have also found the Schaum's series on Combinatorics
and Graph Theory concise and useful.

------
jasomill
The freely available [1] is a nice supplement to some of the material in
_Concrete Mathematics_ , and this [2] is an approachable introduction to graph
theory with side notes on both proof techniques and algorithms.

More generally, if you're not already comfortable with linear algebra (a
couple college semesters "or equivalent experience"), I'd recommend both [3]
and [4] for two entirely different perspectives. For modern algebra more
generally, I'm a huge fan of [5].

If I could only take one introductory mathematics book to a desert island, I'd
cheat a bit and take [6], [7], and [8]. While never directly involved in CS,
Courant was very interested in the practical applications of theoretical
mathematics, see, e.g., [9] and, well, most everything else he wrote.

I could go on and on, so I'll stop here.

[1] <http://www.math.upenn.edu/~wilf/DownldGF.html> [2]
[http://www.amazon.com/Graph-Theory-Graduate-Texts-
Mathematic...](http://www.amazon.com/Graph-Theory-Graduate-Texts-
Mathematics/dp/1846289696) [3] [http://www.amazon.com/Linear-Algebra-
Applications-Gilbert-St...](http://www.amazon.com/Linear-Algebra-Applications-
Gilbert-Strang/dp/0155510053) [4] [http://www.amazon.com/Finite-Dimensional-
Vector-Spaces-P-R-H...](http://www.amazon.com/Finite-Dimensional-Vector-
Spaces-P-R-Halmos/dp/0387900934) [5] [http://www.amazon.com/Algebra-Chelsea-
Publishing-Saunders-La...](http://www.amazon.com/Algebra-Chelsea-Publishing-
Saunders-Lane/dp/0821816462) [6] [http://www.amazon.com/Introduction-Calculus-
Analysis-Classic...](http://www.amazon.com/Introduction-Calculus-Analysis-
Classics-Mathematics/dp/354065058X) [7] [http://www.amazon.com/Introduction-
Calculus-Analysis-Classic...](http://www.amazon.com/Introduction-Calculus-
Analysis-Classics-Mathematics/dp/3540665692) [8]
[http://www.amazon.com/Introduction-Calculus-Analysis-
Classic...](http://www.amazon.com/Introduction-Calculus-Analysis-Classics-
Mathematics/dp/3540665706) [9]
[http://www.ams.org/journals/bull/1943-49-01/S0002-9904-1943-...](http://www.ams.org/journals/bull/1943-49-01/S0002-9904-1943-07818-4/S0002-9904-1943-07818-4.pdf)

------
phektus
[http://www.reddit.com/r/math/comments/eohrr/to_everyone_who_...](http://www.reddit.com/r/math/comments/eohrr/to_everyone_who_posts_about_learning_more_math/)

~~~
opminion
The fist item in that list is learn algebra, and rightly so.

It then follows something similar to a university-level curriculum in Europe
(continental, perhaps not UK), which might lead all the way to research maths,
but perhaps it is not the most appropriate path for computer science because
it entirely misses the mathematical concept of computability.

------
jcdreads
The Project Euler problems might be right up your alley. They are little
programming exercises that start easy and rapidly become harder. Along the way
you figure out enough math to make the programming easier, or at least enough
to make better algorithms. The forums are enormously helpful.
<http://projecteuler.net/>

------
tempire
Math: <http://khanacademy.org>

Computer Science: <http://aduni.org>

Another vote for khanacademy. I did this myself, and worked out a plan to put
every piece of information into long term memory. When you're motivated and
not limited by century-old university bureaucracy, you can move very quickly,
and run circles around people with a "real" degree, most of whom have
forgotten everything but the general concepts.

Udacity is also coming up as an excellent resource, but the important thing to
realize is that every resource adds a different perspective on the matter that
is extremely valuable.

For CS-specific information, aduni.org is a glorious resource. It's from 2001,
but the CS basics never change. I'd recommend getting a handle on all the math
from khanacademy so that you understand everything they're referencing.

------
verganileonardo
I have a business background and use Project Euler to evolve my programming,
math and problem solving skills. :)

projecteuler.net

~~~
big_data
+1 for Project Euler! It totally rocks!

------
sidman
As mentioned probabilities is something you want to get into very early. Its
easy to understand at the start and its practical uses in many different
computing areas along with many other things in life is priceless.

Good place to start would be to go through khan academy's probability videos.

------
elrzn
Take a look a Precalculus Mathematics in a Nutshell for a quick review. You'll
be surprised how much stuff you forgot already!

Then you can move to a discrete math book like Discrete Mathematics with
Applications or Mathematics for Computer Science (free pdf
courses.csail.mit.edu/6.042/fall10/mcs-ftl.pdf) and any Calculus book of your
liking.

There are also great resources to learn Math using Haskell. I'm working
through The Haskell Road to Logic, Maths and Programming and it's proving to
be great
[http://www.haskell.org/haskellwiki/Books_and_tutorials/Mathe...](http://www.haskell.org/haskellwiki/Books_and_tutorials/Mathematics)

------
azsxdc
I was in a similar position 3 years ago. I jumped into an open university
maths degree at the second year, have done half of the final year and have
moved onto a masters degree. All part time, for me this was the best way, as
another commentator has said, it is easy to get lost among the vastness of the
topic - I am still pulling together the threads of how all these things fit
together. I would recommend the journey for the reasons you list, also it has
proven useful in pursuing my interest in machine learning and general
artificial intelligence. Take the plunge, you will probably not be
disappointed.

------
pdelgallego
I am in the ship, I have signed in the statistics 101 class [1] at udacity.
Its very basic, but it's a good starting point. Coursera have also several
math related courses [2] (calculus, autmata, logic, ...).

I have also signed in a class in nearest community college in my area.

[1] <http://www.udacity.com/overview/Course/st101/CourseRev/1>

[2] <https://www.coursera.org/category/math>

Happy logarithmic!

~~~
radicalbyte
Coursera's Automata course is really good, but it's hard work. It's a six week
course, and you'll spend around 10hrs per week if you do everything (and I'd
recommend doing everything).

Udacity's introduction to statistics, so far, has been really easy.

From what I've seen so far, Udacity are aiming to be more inclusive "community
college" level and Coursera are aiming for a true university level education.

------
dionysiac
Have a look at <http://betterexplained.com/> \- Kalid's got some of the most
intuitive explanations I've seen.

------
crasshopper
I may have answered your question here:
[http://isomorphismes.tumblr.com/post/15431506533/i-never-
did...](http://isomorphismes.tumblr.com/post/15431506533/i-never-did-
mathematics-where-should-i-start).

The benefit of DIY maths is that you get to focus on what your interest is,
whether that be algebra or geometry or stats/data analysis or discrete maths
or foundations or whatever. (Contrary to some answers here, I don't think
trig, measure theory, or planar geometry are necessary, unless you read
something and it catches your fancy.) If your goal is just to do more maths,
then just throw away anything that bores you and dive into anything that
interests you.

If you're interested in C.S. applications then I think discrete maths is the
thing to focus on. Eg, "Concrete Mathematics" by Knuth. But you can go an even
more direct route and if algorithms is your goal, there are some free courses
available online that do just that.

MIT OCW and AcademicEarth are excellent for in-depth (course-length)
treatments of material by seasoned teachers. Wikipedia and math.StackExchange
are good for broad overviews and quick Q's, respectively.

For "proofs, and other practical applications" -- that's quite broad. OCW has
a course on geophysics and a course on signal processing -- both are
"practical applications" but I don't know if that's what you have in mind.
Most of the OCW stuff -- whether it's chemistry, materials sci, DSP, whatever
-- is aimed at engineers (other than the upper level pure maths), and
therefore practical.

As for proofs, you will see them in almost any maths textbook. Spivak makes
some challenging ones and if you haven't taken calculus or linear algebra then
Flanigan & Kazdan might be worth a skim. But if you're interested in CS
applications then you can find those kinds of argumentation in an algorithms
course.

BTW, I do not recommend the Khan Academy. It gets a lot of press but I have
not found it so great. (It is fine but so is just surfing Youtube -- a lot of
people make videos besides SK.)

Also, if you can still edit this question, you might give us some idea of
where you're starting from and how much time you expect to spend, to elicit
better answers.

HTH.

PS By "the hard way" do you mean doing exercises? Because pretty much all
mathematics texts expect you to do it "the hard way" if that's what you mean.

~~~
anatoly
It's bizarre that you start your list of recommended books on mathematics for
"people who never got far with it at school" with Penrose's _The Road to
Reality_. No one should follow that recommendation. The book is not really
readable without a background of a Ph.D.-level knowledge in math.

~~~
crasshopper
I don't have a Ph.D. and I found it a fun read. But this is why I say everyone
has to follow their taste.

------
kmfrk
I think Mathematics for Computer Science Eric Lehman and Tom Leighton
([http://www.cs.princeton.edu/courses/archive/spr10/cos433/mat...](http://www.cs.princeton.edu/courses/archive/spr10/cos433/mathcs.pdf))
is a great resource. It has the best explanation of induction I have seen -
discrete maths books are normally completely awful in this regard.

Use something lke Khan Academy to fill in the blanks.

------
impendia
I recommend Susanna Epp's Discrete Mathematics with Applications, which I
recently taught a university course out of quite successfully. It offers a
gentle but rigorous introduction to proofs, logic, and combinatorics, with
applications to CS. Useful exercises too.

It is squarely targeted at beginners, if you know the stuff already then many
other commenters have recommended more advanced books.

------
econner
Check out the forums at <http://artofproblemsolving.com/>

------
tel
I'm going to second Arun2009's recommendation for How to Solve It by Polya.
And then I'm going to recommend Haskell. I recently realized how much my
understand of set theory improved and was solidified purely by programming
Haskell a lot.

~~~
vitno
I started messing around with haskell a few months ago. It completely inspired
me to revamp my entire mathematical understanding... I've bought (and read!)
like 4 math text books...

------
akulbe
So what if you really suck at algebra, and need to start over? Geometry always
made _much_ more sense to me, as I could see it, touch it, it was real!

~~~
wingerlang
Although not algebra per se, but with Linear Algebra I did that. I only read
the uber-basic Vector math and such but then I skipped the rest of the class.

Then I started applying it into programming step by step which made it
possible for me to look at every little calculation.

Then I made a tiny physics simulation:
<http://www.youtube.com/watch?v=ud8NirjyLAA>

------
dkc
How to Prove It. It's cheap (paperback), highly rated, and insightful.

------
HelpWFractions
Khanacademy.org is one of my personal favorite resources.

------
srkiranraj
Try the "MATH FOR CS" course in MIT Opencourseware :)

~~~
bob_hancock
<http://courses.csail.mit.edu/6.042/spring12/mcs.pdf> This is an excellent
place to start.

------
keva161
Khan Academy

------
ten_fingers
Just follow the usual path for education in relatively applied math.

Here is a nutshell description:

The standard high school level subjects are algebra, plane geometry, second
year algebra, trigonometry, and solid geometry.

The standard college level subjects are calculus, abstract algebra, linear
algebra, advanced calculus, ordinary differential equations. Might also take
elementary courses in probability and statistics.

Standard graduate school topics are point set topology, measure theory,
functional analysis.

For more in applications, can do (1) optimization -- linear, network linear,
linear integer, quadratic, non-linear, and dynamic programming -- and (2)
probability, statistics, and stochastic processes based on measure theory.

Here is an overview:

The course in abstract algebra will make you familiar with a long list of
topics that come up in computing including sets, Boolean algebra, logic,
relations, mappings, integers, prime numbers, modular arithmetic, rational
numbers, the fundamental theorem of arithmetic, the fundamental theorem of
algebra, elementary number theory, and the Euclidean greatest common divisor
algorithm. You will also see finite field theory which has long been important
in algebraic coding theory. You may touch on the question of P versus NP. You
may see some of the work of Goedel, etc. and model theory. You will get good
with math based on definitions, theorems, and proofs and develop 'mathematical
maturity', i.e., ability to read, understand, and do abstract mathematics.
Your ability to describe logical material in writing will improve.

Famous authors in abstract algebra include Birkhoff, MacLane, Herstein, Lang.
More recent authors likely touch on algebraic geometry.

Linear algebra, mostly about 'vectors' and linear transformations, is the core
of an ocean of applications in multivariate statistics (regression, factor
analysis, analysis of variance, discriminate analysis, and more and, thus,
applications in ad targeting, machine learning, data mining, anomaly
detection, recommendation engines, etc.), optimization, group representation
theory as in molecular spectroscopy, Shannon's information theory, and more.
E.g., linear algebra is the place to learn about solving systems of linear
equations, e.g., by Gauss elimination, and, thus, the place to get started
with matrix inversion and the simplex algorithm of linear programming
(optimization) and network linear programming.

A good first text in linear algebra is by Ben Noble. The crown jewel is by
Paul Halmos, 'Finite Dimensional Vector Spaces', written when Halmos was an
assistant to von Neumann; the book approaches linear algebra much like it was
functional analysis and, thus, is a baby version of Hilbert space theory and,
thus, sometimes used to teach Hilbert space theory to physicists for quantum
mechanics. Richard Bellman wrote tons. See also Roger Horn's books. Evar
Nering is a good first book (although his treatment of linear programming is
not good); so is Hoffman and Kunze (apparently now available on the Internet
in PDF for free). For numerical linear algebra, see Forsythe and Moler.

The standard advanced calculus text to teach the theorems and proofs of
calculus is Walter Rudin's 'Principles of Mathematical Analysis'. After that
book, notation such as O( n ln(n) ) as in Knuth's TACP will be child's play.

Note: At one time the Halmos 'Finite Dimensional Vector Spaces' and Rudin's
'Principles' were used in Harvard's Math 55 with a colorful description in

    
    
         http://www.american.com/archive/2008/march-april-magazine-contents/why-can2019t-a-woman-be-more-like-a-man/?searchterm=Sommers
    

A course in trigonometry, the use of trigonometry in calculus, and Rudin's
treatment of Fourier series will help in data compression (e.g., JPG), time-
invariant linear systems, digital filtering, and Shannon's information theory.

Also good coverage of the binomial theorem in high school or abstract algebra
and an elementary course in probability will be a help in following Knuth's
TACP.

Measure theory will take a huge load off your back: Calculus, that is, Riemann
integration theory as taught through Rudin's 'Principles', has some rough
edges, understood back to at least 1900, and Lebesgue and Borel and measure
theory (heavily due to Lebesgue) provide a clean solution. As from Kolmogorov,
the solution also makes a clean foundation for probability, stochastic
processes, and statistics.

Likely the nicest first text in measure theory is Royden's 'Real Analysis'.
Also good is the first, real, half of Rudin's 'Real and Complex Analysis'
where he also gives nice introductions to both Hilbert and Banach spaces and a
nice treatment of the Fourier transform.

My favorite authors in probability based on measure theory are J. Neveu (long
in Paris) and L. Breiman (long at Berkeley). Or just read from their teacher,
M. Loeve (long at Berkeley but with a writing style that looks like some cross
between English and French).

This background will also let you do much more in both pure and applied math
in many directions and for many applications.

For learning math, it is "not a spectator sport". Most of the work is between
your ears as you think about the material. A good teacher in abstract algebra
is likely necessary to get you started. Also without at least occasional
contact with a solid university department it is too easy to get off track.
Still, nearly all the work is to be done alone in a quiet room, and there a
book is fine. In principle videos could help, but so far I've never seen even
one that I would recommend for any utility at all in learning math.

~~~
btilly
Most areas of CS you do are unlikely to need advanced calculus or topics that
depend on it (differential equations, topology, measure theory...). Yes, we
can both name exceptions, but on the whole you can learn them later if it
proves relevant to what you want to do.

You are right that you do need algebra, linear algebra, just enough calculus
to understand infinite series (which is usually put in the second calculus
course), etc.

You left out graph theory and combinatorics, both of which are extremely
important to CS.

~~~
ten_fingers
> Most areas of CS you do are unlikely to need advanced calculus or topics
> that depend on it (differential equations, topology, measure theory...).

Right. The only advanced calculus text I listed was 'Baby Rudin', and the main
contribution there is just to get some of the more important properties of the
real numbers, Euclidean n-space, infinite sequences and series, and Riemann
integration solid.

If a CS student is to stop before these topics, okay, but if they are going to
go on then these topics will be part of what is generally assumed.

For CS, as we know, it is doing what EE did -- moving beyond its core tools
and into what to do with those tools. E.g., EE got into nonlinear filtering
and stochastic integration. E.g., CS is now getting into both optimization and
statistics. Then being handy with that Baby Rudin material, at least the early
chapters, will get to be important.

With Baby Rudin already done, there can be a good course in differential
equations, and such a course can be a good way to see some of the value of
what did in linear algebra and Baby Rudin and to exercise that material. At
some point in the future, a CS guy might well get into some work involving
differential equations -- viral growth models, flight of airplanes and space
vehicles, and much of mechanical engineering.

One of the main themes in the future of computer applications is handling
'randomness', and my view is that serious work in that direction should have
the measure theory foundations. I did an A/B on that! Early in my career I
tried the easy way. After measure theory, Neveu, Breiman, Loeve, Dynkin,
Lipster, Shiryaev, etc., I concluded that the measure theory approach to
probability, stochastic processes, and statistics was essential.

In particular, without measure theory, people too easily get totally stuck in
the mud on what 'random' means, while with measure theory Kolmogorov has a
really nice answer. My view is that people may not like Kolmogorov's answer
but that, with some really simple assumptions, we get forced into that answer
anyway!

For the rest, the Hilbert and Banach spaces won't go away! E.g., a huge
dessert buffet of really finger lick'n good applications of the Hahn-Banach
theorem is David G. Luenberger, 'Optimization by Vector Space Methods', and
that book will be a nice source of methods for a lot that computing people
might encounter.

> You left out graph theory and combinatorics, both of which are extremely
> important to CS.

For combinatorics, I assumed that one would get enough, in some ways deeper
than in Knuth's TACP, from abstract algebra and elementary probability. E.g.,
combinatorics is a lot about counting, and so is group theory in abstract
algebra.

For graph theory, I assumed that one would get enough from optimization on
networks. E.g., a 'basis' in the network simplex method on a network is a
minimum spanning tree! And the max flow/min cut theorem can follow just from
linear programming. Dynamic programming, which I mentioned, can be viewed as
graph theory.

I did mention linear programming (and so does CLRS), and one reason is that in
the CS study of algorithms the algorithms for linear programming are
important, and surprising, benchmarks. Also integer linear programming is one
of the more important motivations for the question of P versus NP.

For some of the surprise, the simplex algorithm has low degree polynomial
expected performance (K. Borgward) but exponential worst case performance
(Klee and Minty), but the polynomial algorithms (Khachiyan) when they are
faster than simplex have both of them too slow for practice! So, at one of the
first places we looked at computational complexity for problems more
challenging than, say, heap sort and AVL trees, an exponential algorithm
turned out to be superior in nearly every sense of interest to a polynomial
algorithm! There never was a guarantee that the study of computational
complexity would be easy!

But there's a lot of overlap: I didn't mention courses in 'finite
mathematics', combinatorics, or graph theory, but one way and another what I
described should provide enough coverage. I didn't mention the CS book CLRS,
but I mentioned good coverage of some of the more advanced topics in that
book. Or, there is a lot of blending old wine and pouring it into new bottles.

There is a broad point: A major theme in CS now is to borrow, modify, and
apply work done some years ago in applied math, especially from operations
research, e.g., linear programming, flows on networks, stochastic point
processes, and statistics. While CS has some new applications, typically the
material is done more carefully in the old applied math sources. So, I
emphasized learning the material as math instead of as CS. Besides, the OP was
asking about math for CS and not 'mathematical CS'! Also the title asked for
the "hard way"!

There's another broad point: What to learn and why to learn it? Just a first
cut view of what O( n ln(n) ) means should take only a little searching on the
Internet. Besides, Knuth's TACP is quite clear on such 'asymptotics'. I wish
Microsoft's MSDN documentation of .NET was as clear and easy to read as TACP!

One reason not to learn this stuff is to confirm that Knuth, Sedgewick, CLRS,
etc. were correct after all!

Generally the reason to learn such stuff is for new applications in the
future. For that, my view was just to stay with a relatively traditional
course in relatively applied math although I leaned away from classic
mathematical physics and more to business applications.

~~~
Dn_Ab
You argument is solid and it would be great if math was taught more as a
connected whole but many do not have that luxury. Key though, is the author
never went into much detail as to his or her intentions and motivations so not
much can be said if your list is inspiring or too intimidating.

One thing I'd like to point out is that measure theory is not the only and
probably the least interesting way to study probability. There is the more
elegant (IMO) approach via Nonstandard Analysis. And the fun more practical
approach via Game Theory and Markets (which also support "imprecise
probabilities").

I also think theres room for different approaches to the same thing, each
offering their own unique insight. Many differential equation modelling
problems, especially those involving populations could be fruitfully replaced
by agent modelling.

~~~
pmiller2
Do you have any book recommendations for a nonstandard analysis treatment of
probability? I'm really only familiar with the measure theory approach,
myself. (In fact, I've been known to say that probability is the study of
measurable functions with finite, nonzero integral over the real line.)

~~~
Dn_Ab
The standard is Nelson's Radically elementary probability.

Vovks and Shafer's game-theoretic approach is interesting in that related
approaches like bandit models and online learning have recently picked up in
popularity.

------
zxcvvcxz
Question: is anyone working through Numerical Linear Algebra, or has done so
in the past?

[http://www.amazon.com/Numerical-Linear-Algebra-Lloyd-
Trefeth...](http://www.amazon.com/Numerical-Linear-Algebra-Lloyd-
Trefethen/dp/0898713617)

I'm having a really tough time with the exercises even though I should have
the prereqs... What can I say, we don't do much rigorous math in engineering.

Anyways this book came highly recommended to me from a friend in quantitative
finance.

