
Math topics useful for computer science/programming - jackhammer2022
http://matheducators.stackexchange.com/questions/1734/what-should-be-included-in-a-freshman-mathematics-for-computer-programmers-cou/1745#1745
======
j2kun
I never liked these kinds of lists. The people who ask the question always
seem to be under- or overwhelmed by the answer (programmers don't need _any_
math?! theoretical computer scientists need _all_ math?!). And the people who
answer are almost always skewed toward whatever it is they do.

Even the top-voted response to the OP is obviously biased toward the logic
flavor of computer science, with awkwardly scattered recommendations for non-
logicish topics that are far more useful for _all_ flavors of computer science
("perhaps even eigenvalues," seriously?!).

These lists even ignore the deeper question being asked: how do you figure out
what sort of mathematics you need for the things you're interested in? Not all
computer scientists (and programmers) like the same things in computer science
and programming. It's much more advisable to figure out what you like before
you go around trying to learn all the math there is to learn related to CS,
and then ask about what math topics are related to that.

~~~
tptacek
This comment is a little frustrating, because part of the point of linking to
the Stack Overflow post is to continue the discussion, which is obviously
germane to our site. Why not dig into specifics of how you'd approach the
intersection of math and CS?

~~~
j2kun
It's actually inspired me to write my own article, but that will take time.

Meanwhile, I actually write a blog called "Math Intersect Programming." Even
though it's more about my specific interests than how a freshman should
approach the topics, and I certainly emphasize the mathematical ideas over the
engineering challenges, it still gives a good idea of what I would suggest:

Orient the learning around applications in CS. Find a problem that requires
the mathematics you want to learn, and learn it with the persistent contextual
understanding that you are trying to do something more than just tinker with
definitions. Do this for one application and repeat as desired.

My view is that when this is done, you learn to focus on the things that
matter over the things that are used to fill empty space in courses (calculus
courses come to mind as a specifically abhorrent example of this). You get
better at seeing the forest over the trees.

The problem is this avoids the OP's question (and this ensuing discussion
about the OP's questions and the purported answers), since you can't organize
a single class around it. Another problem is that very few books are written
this way, or are only superficially so.

~~~
apu
I want to chime in and say that I've found your blog wonderful, and since
you're too modest to post a link, let me do it:
[http://jeremykun.com/](http://jeremykun.com/)

------
stiff
This does not seem very informed, starting from the false premise in the
question up to the random concepts listed in the answers.

Calculus is actually important for Computer Science, it's actually important
for everything, it's where you learn how to handle the exponential function
and the natural logarithm, how to do approximations and bounds, how to handle
infinite series, etc., and those things then appear all over the place, unlike
most things listed it's something that you can expect to encounter almost
regardless of what domain you are interested in.

I mean, the guy asks what should replace Calculus and then the first answer
includes "Asymptotics", "basic limits, sequences and series", so actually
calculus. In general I cringe a little every time I hear Computer Science
people should focus on "discrete math", because without tools from analysis
you can only solve the most trivial discrete problems. And yes, calculus by
itself is hardly ever applicable in CS, yet you still have to learn it, tough
luck. In general what is not stressed enough I think is that applying math is
hard and you need to learn a lot of it before you have enough tools to tackle
problems anywhere close to real-world complexity.

The top answer also lists random concepts. I am learning probability
currently, for applications in machine learning. "Discrete spaces, Bayes
theorem and expected values" you can learn in a day, "Markovs, Chebyshev and
Chernoff inequalities" are mostly only useful for further theoretical work, so
is "the law of large numbers". What will really be useful will depend a lot on
the applications, if you are a theoretical computer scientist, mastery of
generating functions and transforms will be useful, and it's one of those
instances where discrete problems are solved via tools from calculus/analysis.
For machine learning you need to know everything about the normal distribution
by heart, and this means you have to know everything about the exponential
function by heart, so again back to calculus. Notions from information theory
are useful, but of course none of the ones he listed. The comment "This is a
must for modern programmers." sounds just comic.

~~~
superuser2
> Calculus is actually important for Computer Science, it's actually important
> for everything, it's where you learn how to handle the exponential function
> and the natural logarithm, how to do approximations and bounds, how to
> handle infinite series, etc., and those things then appear all over the
> place,

It's still interesting to think about which branches of math are actually
applicable to programming itself.

People tend to talk about programming and math as very strongly related, and
of course there is the obvious relationship that "some computer programs do
particular kinds of math" like you're talking about here.

But there is no (intuitive) overlap between writing, say, a web application
and doing algebraic or calculus computation on paper. However, there _are_
things like:

\- Set theory underpinning relational databases

\- Typed lambda calculus underpinning functional programming

I'd be interested in other examples like this.

~~~
ThrustVectoring
I have a couple simple ones off the top of my head:

\- You write a recursive program the same way you write an inductive proof

\- Abstract algebra and category theory are likely relevant, especially for
metaprogramming. My math education hasn't included this, so I can't say much
more.

\- Linear algebra is just ridiculously important

\- Statistics for machine learning. Also for figuring out how to combine data
in a meaningful way. There are also a lot of people asking statistical
questions directly, and writing programs is how you get those kinds of answers
in a reasonable timeframe.

~~~
nmrm
> Abstract algebra and category theory are likely relevant, especially for
> metaprogramming.

In general, the whole "oh yeah CS people should know some category theory and
abstract algebra" is pretty hilarious.

First, it's a bit like saying "oh yeah CS people need to know the undegraduate
basics and also the generalization that most mathematicians don't encounter
until a couple years into grad school."

Second, most people who say this really mean "a conceptual grasp on different
types of morphisms is useful". But that's like saying you need calculus in
order to drive a car; or, in the case of categories, it's like saying you need
two semesters of real analysis in order to drive a car.

Why not just say "knowing about different sorts of mappings is pretty useful
in functional programming"? Knowing how this generalizes to more abstract
mathematical objects is totally unnecessary.

~~~
shoki
Well, frankly you can get along not knowing the "Gang of Four" design patterns
and write Java. By the same token, you don't need to know about iterables and
comprehensions to write Python, smart pointers to write C++, Graph theory to
use a Graph database, macros to write LISP, etc.

By analogy, you don't need to know abstract algebra and category theory to
write Haskell. But as in the other cases, knowing _helps_.

~~~
nmrm
I think you misunderstood my criticism.

"Different sorts of mappings" is NOT synonymous with "category theory". Not
even close. Heck, Euclid knew about "different sorts of mappings".

Most everything in Gamma et al is arguably useful for everyday programming in
Java. Maybe 5-10 pages of MacLane is useful for everyday programming in
functional languages.

Unless by "Category Theory" you mean "5-10 pages of MacLane", Category Theory
-- on the whole -- is a horrendously inefficient way of teaching about
"different sorts of mappings useful in functional programming."

Unless you want to use functional programming as an environment for doing pure
mathematics, there's no reason to actually study actual Category Theory.

~~~
shoki
_Functional Programming_ is a vast subject.

I've really never needed an abstraction for semigroups, monoids, meet-semi-
lattices, monads, comonads, arrows or catamorpisms in Clojure, Common Lisp,
Scheme, or Hy.

These concepts become more relevant when I program Haskell, Agda,
Isabelle/HOL, or Coq.

I'd say a stronger analogy can be made between reading MacLane's _Catagories
for the Working Mathematician_ and reading Hoyte's _Let Over Lambda_ ; you
really only need to read a little bit of these books to get the core concepts.
That being said, depending on what sort of functional programming you're
doing, a strong background in category theory or meta-programming can enabling
(or not).

~~~
nmrm
> when I program Haskell, Agda, Isabelle/HOL, or Coq.

That's fair. Although Haskell is a bit of an odd man out in that list, both in
terms of its nature and in terms of its typical use case.

> That being said, depending on what sort of functional programming you're
> doing, a strong background in category theory or meta-programming can
> enabling (or not).

This is where the analogy between the two books breaks down. When you're using
a functional programming language as a proof assistant, category theory can be
helpful. But this is far less common than meta-programming.

edit: 2nd paragraph.

------
RogerL
I went to an engineering University. Besides having a wide variety of math
required (3 semesters of Calc, 1 semester of either stats or probability, 1
semester of either linear algebra or matrices), we also had to take 3
semesters of physics, and 2 of chemistry. Most of us (those not just looking
for a piece of paper) went considerable deeper in at least one area.

No, you don't need to understand organic chemistry to write a CRUD app. But,
you don't need university to write a CRUD app.

I think I received an excellent education that prepared me for about any job
out there. And, I've done it. I've done cancer statistics for the NIH. I've
worked in avionics and simulation. Well, blah blah, no one cares about my
resume, I'll just say I couldn't have done any of it without the education I
received. I haven't done financial quant stuff, as an example, but how could I
without the math background? Or how could I program a controller in a factory
without understanding PID (done that)? How can I get a piece of that neat
drone project (done that)? Hey, that computer vision stuff looks interesting,
all I need to know is ...derivatives, statistics (done that too).

For most of us, university is the last and/or best chance to really understand
how the world works. For that you need quite a bit of math and science.
Another example. We have a voluntary Arduino robotics workshop going on at
work, coupled with 3D printing. Naturally, people are excited, and have all
kinds of ideas. But, how do you execute on that if you don't understand how to
design an op amp circuit, or filter data with something more sophisticated
than a moving average? Robotics spans a number of fields, and to actually
produce something more than a gimmick requires a lot of knowledge. Knowledge
that you can pretty easily pick up in University, but knowledge you mostly
just long for as you have to rush home at 5:30 to pick up the kids and make
dinner.

I guess this is a cranky rant from an old guy. But I hear about University
programs that are nothing more than Java voc ed systems and I despair. $100K
in debt to learn something most anyone on here could just pick up on their
own. You don't need university for that stuff.

~~~
klibertp
> but how could I without the math background?

You could have just learned it as you went. Done that.

------
archena
I'd include combinatorics under 'useful' too - although the basics are usually
covered in statistics courses. Odd that the list mentions differential
equations but not basic calculus, which I'd think is more fundamental.

It's also interesting to consider the chapter headings in Concrete Mathematics
(Graham, Knuth, Patashnik), a text designed with students of CS and
programming in mind:

    
    
        Recurrent Problems
        Summation
        Integer Functions
        Number Theory
        Binomial Coefficients
        Special Numbers
        Generating Functions
        Discrete Probability
        Asymptotics

~~~
bainsfather
I got Concrete Mathematics a few days ago, after someone recommended it. My
background is physics, and I find the "mathematicians' style" to be very
awkward to learn from. This book is excellent for me - concrete examples - it
is always trying to solve specific problems, so if I don't follow the theory,
I can always refer to a concrete example and try to understand that specific
case. This the way I learn best/fastest.

------
jostylr
Why not teach math + programming together? I didn't seem to see anyone suggest
that.

Calculus: simplify and iterate to solve a problem (like Newton's Method). Lots
of issues to tackle with. Much more useful than learning formulas, but it can
also be interesting to show how converting forms can take an unstable problem
into a stable one.

Combinatorics: Explore the small side of a problem, code up the larger
version, explore how far one can get, show methods for dealing with the
problem beyond the scope.

Graphics, user interface design, etc., are also very big in math. In a course
I teach to Enviro Science people, they come in scar(r)ed of math and by the
end, they enjoy it. A big trick is getting them to use GeoGebra in an
exploratory, visual fashion. It makes the math real.

The same approach in code can help make CS students get math while also
getting the issues in CS. Anyone who thinks the exponential as 1+ x+ x^2/2 +
.... is useful as is for x = 100 needs some actual experience with it. But at
the same time, teaching them how to translate e^100 ~~ e^(43 _ln(10) + .98) ~~
10^43_ (1 + .98 + .98^2/2 + .98^3/6) = 10^43 * 2.62 has its value (compare to
e^100 ~~ 2.69 * 10^43)

This is the kind of math that is useful. Exploratory math that struggles with
mathematical truths and practical implementations. Students should get a sense
of the limits of computers, what they can handle and not.

And designing a good math interface for these problems can be quite the UI
learning experience. There are so many ways to explore math.

Stop teaching dry math and start teaching programming math.

~~~
JoeAltmaier
Linear algebra/vector algebra is the single most useful topic for programming
I can imagine.

~~~
tptacek
Why?

~~~
JoeAltmaier
Applications - simulation, graphics, discrete analysis.

------
weavie
Although I did mathematics to degree level, I have used so little of it in my
professional career that I have largely forgotten it all now. Any such list is
going to depend on what area of programming you want to get into.

For a lot of programming understanding user interaction and human psychology
is much more important than mathematics.

------
ap22213
"Many students struggle with the idea of recursion..."

I still remember one of the worst bugs I'd ever introduced (in terms of pain
to me). I was maybe 2-3 years out of my bachelors, and I thought recursion was
so nifty that I'd use it where ever I could. I was coding C++, at the time.
And, I got the dreaded 2am call after all of the production servers went down
hard. Like terminate and corrupt data hard.

Lesson learned: it's good to know recursion, but it's also good to know
maximum stack sizes.

~~~
jostylr
There's also thunk/trampolining continuation style to convert recursion easily
into something that is not stack limited.
[http://nathansuniversity.com/cont.html](http://nathansuniversity.com/cont.html)

------
brirush
I wrote the original question. I've always been interested in how much math
various careers use, but I wrote this question the way I did in a shameless
attempt to drive traffic to the site by appealing to the Stackoverflow and
reddit communities. Everyone has given me a different answer so far, but
that's to be expected, because there are ~1000 subcategories of computer
programmers.

~~~
nmrm
> I've always been interested in how much math various careers use

> ... there are ~1000 subcategories of computer programmers.

I'm curious: what about other careers? Are there fewer categories? Do they all
do the same type of math? etc.

------
userbinator
In my experience, I've seen a lot of students entering CS not comfortable with
even _basic arithmetic and algebra_ , despite it not being mentioned in any of
the linked responses.

As for calculus, I think its importance is overrated - unless you are
specifically doing some numerical computation that requires it.

~~~
Bahamut
Calculus gives you some of the intuition for figuring out a good big O bound
for the computing time of an algorithm.

------
jules
Which area of math isn't useful for computer science/programming? Certainly
some areas are more useful than others, but I think rather than looking at
areas it matters more how in depth you go in one area. Going super deep into
an area is often not a lot more useful than just knowing the basics.
Complexity theory is useful, but it's not that practically useful to know
deeply about all the complexity classes.

------
alok-g
What would be a good writeup for understanding term-rewriting? (My current
background level: I understand most of what OP has listed in "actually useful"
and "can run into", and "automata theory" from the third list.)

~~~
j2kun
I wouldn't recommend it as being as important as the OP claims, but if you
must:
[http://www21.in.tum.de/~nipkow/TRaAT/](http://www21.in.tum.de/~nipkow/TRaAT/)

~~~
alok-g
Thanks!

Could you also recommend something on how symbolic computation works? I have
used computer algebra systems like Mathematica for long; would like to
understand how it works internally.

I currently understand some basics like DPLL, semantic first-order
unification, methods of solving specific equation systems like linear systems,
numerical differential equations, etc. I am missing at least how the "top-
level" of symbolic computation works.

For example, from what I currently understand, first-order semantic
unification can find "x = Cos[y]" from "Sin[x] = Sin[Cos[y]]", but cannot
solve "Sin[x] = Cos[x]" for x, and cannot simplify "[(Sin[x])^2 + (Cos[x])^2]
= 1" to True.

I am hoping to find something simpler than reading through SymPy source code.
:-)

~~~
nextos
Peter Norvig's PAIP book has a chapter devoted to Student, a computer algebra
system. It's very didactic:

[http://norvig.com/paip/student.lisp](http://norvig.com/paip/student.lisp)

