Hacker News new | past | comments | ask | show | jobs | submit login
Too Much Calculus – Gilbert Strang (2001) [pdf] (www-math.mit.edu)
214 points by nkurz on March 12, 2015 | hide | past | favorite | 134 comments



As a programmer, I've found far more uses for linear algebra than for calculus. It shows up in robotics, in machine learning, in game development, and in many other cool subfields.

I didn't get very much out of my university linear algebra course. (It was pure theory with nearly zero applications.) I don't think I really developed a good intuition for linear algebra—and why it's so useful—until I read Jim Hefferon's free textbook:

http://joshua.smcvt.edu/linearalgebra/ (GFDL license, LaTeX source available)

(I get nothing out of recommending this open source textbook other than the joy of sharing a book I loved.)


I agree although I work in robotics / computer vision.

What I wish I had done a lot more of is statistics. Being able to reason about probabilities clearly, getting an idea of how a system performs given various variables, and most importantly, being able to communicate this to others has been very useful to me. I think anyone (front end, backend, etc) can benefit from learning about statistics.


But statistics and probability is really just integration. It's either that or stamp collection.

And optimization is really just differentiation (+ some geometry).

I'd say the problem is too little calculus and not too much.


Totally agree, calculus is a prerequisite for understanding statistics, unless you're just plugging in numbers for an A-B test - and you don't really need a class to do that.

Linear Algebra is very useful too, but it's not really fair to say one is more valuable than another. It just depends on what kinds of applications you end up working on. Universities should just teach both.


Many interesting things require a combination of more than one area of math. I don't understand how you can do much within robotics or vision without using all three of calculus, linear algebra, and statistics :)


I'll go even further and state that without an understanding of calculus, linear algebra, and statistics, an engineer or even a data scientist is little more than a glorified script kiddy.


Statistic teaches you application and helps to develop correct intuition. Can's speak for everyone, but my college algebra courses were 100% abstract stuff detached from reality. All professors had the attitude that once you learn the theory you could apply it to anything. Problem is, it's hard to remember theory if you don't map to something tangible, and it's hard to apply it to reality when you're just used to crunching equations.


Statistics is, classically, more developed in the realm of "applied math" than "pure math" while linear algebra is, again classically, more a "pure math" discipline. The difference in rhetoric you cite falls out of this exactly.

In practice, both have wonderful presence in math both applied and pure. Linear algebra is one form of abstract algebra which is easily one of the most important mathematical disciplines. It's also the characterization and method of manipulating large arrays of numbers in interesting and practical ways. Statistics is the mechanism of interpreting live data under knowledge of uncertainty toward generalization. It's also probably well characterized as a particularly well-behavior substudy of measure theory... the pure mathematical notion of generalized "substance"


my college algebra courses

Sorry, I meant my college calculus courses, although incidentally the statement applies to both.

I think colleges would be much better off teaching introductory math as "applied". I got significantly more out of statistics, because that class taught us how to spot problems where that branch of mathematics is applicable. I really liked how Udacity used real-life orbit calculations from various space missions as assignments. It gives the whole thing some context.


I agree as well. Not sure why you're being downvoted. Maybe I don't know enough about linear algebra to judge but everything ML that I've ever worked in was spoken in calculus..


Don't know why you're being downvoted. This is totally true.


Most of my downvotes so far, both here and on reddit, have been earned by writing things that are true but unpopular.

The weird thing is that I can't predict which things are going to be sufficiently unpopular amongst people impolite enough to downvote instead of coming up with a good counterargument (or even better: fact). I had /never/ expected something like this to get a single downvote (and I was down at -3).


Do you have a recommendation for a good statistics textbook, for someone who is mathematically literate, but doesn't know almost any statistics?


This book is great and designed for exactly this type of audience: http://www.stat.cmu.edu/~larry/all-of-statistics/



Hey, thanks!


That was really cool.


Thanks for the link to the book.

I had a similar experience where my college linear algebra class was not good, taught by a grad student who seemed to have no enthusiasm for the material or for teaching, plus his accent was hard to understand at times. It was also the first exposure to proofs in the math curriculum so from the beginning I remember being overwhelmed by those and totally losing sight of the forest for the trees.

I sat through the whole semester thinking it was a pretty useless subject. My senior year I happened to take a machine learning and a quantum mechanics class at the same time and had a big ah-ha moment when I realized how big a role linear algebra plays in such a wide range of subjects. I went back through my old textbook and notes at that point and that new perspective made a huge difference, it was like it was completely different material and it was a lot easier to understand.


As an aside, I get really confused and disheartened when I hear that someone's first exposure to proof was in college in a linear algebra class.

I had a somewhat old-fashioned education for an American, and learned proof via Euclid when I was 12. And again, but poorly, when I was 16, but that's another story. Proof is supposed to be learned a) thoroughly b) early and c) as a part of geometry. What's the point in even studying Euclid if you don't learn the method of proof?


BLESS YOU. I have a computer vision exam on Tuesday. Naturally, the prof is awful at explaining math and only seems to care about the theory part.


Thank you for the link. I had the same experience in uni. It's a general issue that plagues mathematics education, I think: theory should come after some basic understanding of applications, not before.


until I read Jim Hefferon's free textbook

I still prefer reading textbooks in paper, and one interesting note: it looks like a couple of companies and individuals are offering paper copies on Amazon, but none of them appear to be Henderson himself.

And the companies or individual offering paper copies still charge $20 – $30!


I just noticed under the "Download Linear Algebra" section on http://joshua.smcvt.edu/linearalgebra/ , he gives a link to a paperback version of Linear Algebra on Amazon (ISBN-13: 978-0989897563).


> And the companies or individual offering paper copies still charge $20 – $30!

$20 for the official version (see the book's web page for a link). That is, essentially, the price of printing and binding.



Linear Algebra is also important in Finance too. Stats are useful in almost every field, as well as for developing a probability-based state of mind.

I wonder why the imbalance exists. Too much focus on preparing people for careers in Engineering? Or preparing them for more abstract concepts like Analysis and Analytic Geometry?


> Too much focus on preparing people for careers in Engineering? Or preparing them for more abstract concepts like Analysis and Analytic Geometry?

Yes. Computer Science curricula tend to focus overmuch on preparing students to work as software engineers, and Mathematics curricula tend to focus overmuch on analysis (to the point that once you get past Calc 1+2+3, Linear Algebra, Differential Equations, Mathematical Analysis/Epsilon-Delta Calculus-with-proofs, and Real Analysis, things like Algebra and Topology are senior-year electives).


Yes - it's too bad they keep the fun stuff for the end. I've been rediscovering the fun of number theory via Project Euler.


For some coding puzzles I was given during interviews, I found that sometimes linear algebra solutions can be used. I submitted such a solution once to a company for a take home problem, they rejected me but it was still an interesting experience.


As a programmer, I've used calculus and linear algebra exactly never in my professional (20+ years) career. It just doesn't really come up doing web apps, sysadmin or dba work.


A lot of this is whether or not you are looking for it, honestly. It is easy to stay away from it, though it is also easy to find if you look for it.


I work on games. I use linear algebra daily and calculus frequently, but far less often. Both are extremely relevant to game development although you can get by without calculus and only light linear algebra if you limit your scope or specialize (in something other than graphics or simulation).


In my 20-odd years, I'd agree---uses of calc or linear algebra have been vanishingly rare, although my current project is changing that a bit. Not because I didn't have opportunities to use it, but because I didn't seek out those opportunities.

On the other hand, I'm prepared to defend to the death the statement that programming is applied formal logic. (Or possibly abstract algebra; that is likely more appropriate for some more modern things. On the other hand, I grew up in the land of Dijkstra, so....)


Mechanics don't use calculus, even many engineers don't, but those working in R&D, especially towards the R side, do.


ekidd - thanks for the link!

Also, I know it's not free (the coursera course is free) but it's still a great read and seriously applicable http://codingthematrix.com/


(Speaking from my experience as a calc teacher at a big U.S. state U.; mileage may vary elsewhere) Most non-STEM students won't see the 'Calculus I, Calculus II, Calculus III' that starts the essay so dramatically. Increasingly, liberal arts students are able to get by with no more than a "math appreciation" course and maybe what amounts to high school algebra. The "math appreciation" course might actually touch on very neat topics for these students (mathematics of voting, etc.) so I'd say math curriculum has already been successfully reformed for them.

As for the average STEM student, in my experience, most freshmen wouldn't be ready for linear algebra (unless by "linear algebra" you mean "matrix arithmetic"). They have enough trouble understanding functions in Calculus I: linear algebra's "functions are basically matrix multiplication" feature would NOT help.

Another thing to keep in mind is calculus has had much more impact on history/society than linear algebra. The fundamental theorem of calculus is a legitimately earth-shattering breakthrough: none of the theorems in linear algebra come anywhere near it in cultural importance.

Pragmatically, calculus is the most suitable mathematics for the type of testing-oriented, uniformizable/transferable pedagogy that a university really (in cold practical terms) must depend on for first-year students. It would be extremely difficult to force the kind of uniformity onto linear algebra that would be necessary if you seriously want to have millions of transfer students moving around between thousands of universities every quarter, gossiping about which linear algebra teacher gives the easiest grades, etc.


"Consider a spherical cow in a vacuum on a frictionless plane..."

That joke comes from the analytic calculus tradition of physics. I took a lot of physics while an undergrad, and I came out unable to do any useful physics. This is because as soon as you try even very simple problems the partial differential equations that result have either no, or extremely difficult to find solutions, and it is usually the former.

I spent years learning analytical tricks that I just don't apply.

There has been a lot of push back on this in education recently. There are Giordano's and Sherwood&Chabay's textbooks in physics now which emphasize computation over analytics. The reality is that if you want to do the vast majority of work in physics you will be doing things like reducing your problem to Ax=b, and throwing the linear algebra machinery against it.

I type this while taking a break from Strang's "Linear Algebra, Geodesy, and GPS" textbook, which I need for work right now. There is an endless need for this sort of math.

Sure, it is a matter of balance. But I flip through my diff eq books and I don't see a lot in there that I actually need. There is nothing in Strang that I don't know, and ditto for a good numerical methods book.

Even the work that Gauss did with orbits did was linear algebra and computation. He did not analytically derive the motion of Ceres, for example, but used linear algebra to compute it.

I, along with many other posters on this thread, and everyone that I have talked to, wish that there STEM education had been much more about Ax=b, the only problem we really solve, and a lot less about using huge tables of analytic integrations to try to find answers to equations that we will never encounter in the real world because they are so specially crafted as to have an analytic solution.


I agree with your point that a lot of current STEM education focuses on things that don't really have any practical value. I think its more of a tradition for engineering students to be learning about this stuff, a legacy of the time when knowledge was not available at your fingertips.

There are a lot of programs that are experimenting with different approaches though, and I think this will only get better with time.


You sound like exactly the constituency Gil Strang's corrective essay is addressed to.

I don't see any of the points you mention as particularly strong. There would be nothing wrong with presenting a more hands-on linear algebra course as a first course for freshmen. (Calling it "matrix arithmetic" does not do the notion service.) The material seems very immediate, and you can use a REPL like octave or matlab to reinforce concepts. You could deepen it as you wished by introducing more abstract concepts and by doing proofs, but I don't think that's necessary. I learned a lot from Golub and van Loan, for instance.

Second, the notion that calculus has had more impact on history/society than linear algebra is very hard to defend. Think of all the numerical modeling that has been done using matrices. Think of all the large-scale computations/optimizations that are done first by finding a linear system, or approximating one. I'm not saying there is a clear answer to this question, just that it can't be the foundation of an argument, because it's so undecidable. Another thing that makes it undecidable is cases where you need both (e.g., weather forecasting).

And saying it would be hard to teach the subject in a uniform way -- really, this is a very bureaucratic objection.


>a more hands-on linear algebra course as a first course

It is ideal (and indeed, not uncommon) to have linear algebra and calc 1 in parallel --- for special honors classes. It wouldn't be realistic to do it for the general freshman class. Do you realize just how many students that includes, in a big state U.? "a REPL like octave or matlab" would be wonderful, amazing, if it weren't about as realistic as the space elevator. Do you know how many endless headaches there are with software-augmented courses, even when it's just software you run in the browser? I applaud your ambitiousness but let's just say I don't want to be in your shoes when you've just told a thousand 18-year-olds to install Matlab!

>the notion that calculus has had more impact on history/society

Linear algebra is hard to even put your hands on, historically. There was no Isaac Newton figure putting the finishing touches on an amazing breakthrough "Fundamental Theorem of Linear Algebra". It is the book-keeping field of mathematics. "All the numerical modeling that has been done using matrices" almost entirely arises by discretizing differential equations so linear algebra doesn't win any points there.


No, linear algebra does not have an Isaac Newton figure. The historical development was different.

The different history does not bear on the ultimate impact and general utility of the tools offered by calculus or by linear algebra.

Eigenvalues and orthogonality are not just "book-keeping". Why give forth empty slurs like this?


Wasn't meant as a slur, I love linear algebra. Especially graduate-level, when you learn about the abstract (commutative diagram) approach to tensor products! :D Current-day impact doesn't retroactively grant historical value, though: the smartphone has not had a big impact on history (yet).

If you can do a decent treatment of eigenvectors in a non-honors freshman course, my hat goes off to you. Even in a 2nd-year course where students are more mature, that's hard (first time I took LA, eigenvectors were rushed at breakneck speed in the very last lecture). And while they're very important, eigenvectors are still a shadow of the historico-cultural impact of calculus.


I've had linear algebra from the onset in my engineering school and it was wonderfully useful. Comparing which one is "better" (calculus or linear algebra) is like arguing whether multiplication or exponentiation is a better operation -- don't do it, you're going to sound cliché without reaching a conclusion.

The main use of linear algebra to me is describe local behavior of functions/dynamical systems. Sure, calculus is the setting of functions in full generality, but understanding the algebra of local interactions is an essential part of the whole -- how can you go far into calculus without understanding derivatives? Linearization is practical the way to solve 99% of engineering problems.


I'm unfamiliar with it: what is the cliché about multiplication vs exponentiation?


The point is that that vs doesn't even make sense. It's like saying what wheel is your favorite in your car, the front left or the rear left. Arguing for calculus vs linear algebra is equally stupid. They are different branches of mathematics with different domains of application.


This I understand, but that sentence with "cliché" in it I still don't get.


It's pretty clear to me that the OP just used the word cliche where it didn't belong. Fortunately the idea still came across. Perhaps you were truly confused, but it sounds more like you're playing dumb to force the OP to admit the mistake.


If by "the idea still came across", you mean I was able to guess approximately what the OP meant, you're right. I was curious about exactly what he or she meant and that's why I asked.

And I wasn't playing dumb. For one thing, even if I assumed the word cliché was used incorrectly, I still couldn't easily guess exactly what the OP really meant. Second, I don't like assuming people are using words incorrectly, it is safer to ask what is meant.


cliché.


Arguments about nothing are all the same.


I'm amazed that you say that calculus has had a much greater impact on society when virtually every database, graphics manipulation or economic analysis uses principles of linear algebra.

Can you substantiate "millions of transfer students between thousands of universities every quarter"? That seems like outlandish hyperbole. Irrespective of the precise number, a system with momentum is not sufficient justification for its continuation.


Calculus has been part of the foundations of nearly all modern science since its inception in the late 17th century. Its genesis shaped the course of modern physics, engineering, chemistry, and architecture, and has consistently for the last several centuries.

Linear algebra has really only come into its own since humans have had powerful computational systems at their disposal, as a large part of linear algebra's usefulness (including its relevance to the examples you describe) is efficiently organizing and performing calculations in higher order systems. That is not to say it is irrelevant or non-impactful - far from it! I'm just saying linear algebra has taken hundreds of years for its real power to be leveraged, and in the intervening time, calculus has reigned supreme as a system for modeling and describing real systems.

(FWIW, I agree with the author and the paper in question, but I think OP you replied to is absolutely correct that teaching both gives perspective that neither alone would impart.)


> I'm amazed that you say that calculus has had a much greater impact on society when virtually every database, graphics manipulation or economic analysis uses principles of linear algebra.

Studying any kind of engine (or for that matter, anything that moves), solid-state electronic device (from transistors to antennae) or power distribution network and doing any kind of design about them is literally impossible without calculus. Even algebraic approaches to these are formulated through discretization of a continuous model (which -- unsurprisingly -- you need calculus to study).

We have such a thing as databases, devices that can manipulate graphics and high-complexity economic models that are analyzed through linear algebra because Newton and Leibniz found the mathematical tools we need in order to model (continuous) change. We could mostly get along fine-ish without linear algebra, but the industrial revolution that spawned our society would be impossible without calculus, because without it you cannot design anything non-trivial that moves or runs on electricity.


FWIW, I have a math degree and work in tech.

Both are incredibly important. I would say that linear algebra and discrete methods are more important for computing, but calculus has had a larger impact on science especially physics. Newtonian physics - a revolution in thinking and unification of mathematics and science - is intricately tied to calculus.

Even in the example of economics that you cite, calculus is used heavily for the theoretical underpinnings for finding equilibria and deriving min/max.


I agree that Calculus is a more readily applicable mathematics that easily applied to many things. It really bothers me when people downplay the usefulness of Calculus also the person that posted the fundamental concepts of Linear Algebra are generally a little beyond the Grasp of people that haven't even been through the Rigors of Calculus 1 - Calculus 3.


>I'm amazed that you say that calculus has had a much greater impact on society when virtually every database, graphics manipulation or economic analysis uses principles of linear algebra.

I'm not. Consider all of physics. Physics is written down in terms of differential equations. You need calculus to get these equations. In terms of graphics, for example, are you magically creating only static images that never change in time? How do you think people came up with the algorithms to do things like refraction or reflection? These all involve calculus. Calculus is the most important tool in applying mathematics to the real world. It is how we derive the models for pretty much everything I can think of. Any time you want to model something where anything changes, that is calculus. Any time you want to find an optimal solutions, that is calculus. I can't think of a single area of applied mathematics that doesn't use calculus in some form. Even mathematics in general, calculus is used in some form pretty much everywhere in some form, except maybe in the foundations of mathematics like set theory or logic.

Solving them, on the other hand, is a different problem. This is where linear algebra is important. When solving an ODE or PDE, when I discretise it, all I am doing is re-writing it as a giant linear algebra problem. Quantum mechanics, for example, is dedicated to finding the eigenvalues and eigenvectors of the Hamiltonian. Linear algebra provides the tools to solve the problems posed by calculus. But it isn't always required.

In fact, if you look at the history of linear algebra, it only really became wide-spread when quantum mechanics was developing since there is a deep connection between linear algebra and quantum mechanics. Heisenberg had never heard of a matrix before despite discovering Heisenberg matrix mechanics. He was told by Max Born that what he had been doing is actually this thing mathematicians called "matrices". Now linear algebra is a required course for physicists (incidentally, linear algebra was the first lecture I ever attended at university).

So I would argue that learning calculus is the most important thing someone should know mathematically, since it is the tool we use to build models. The second most is linear algebra since it provides us the tools to solve these problems. But in my mind, knowing how to derive these models is a lot more important than being able to solve them. That and linear algebra doesn't give you any insight into why we are solving the problem that way, it is just a tool to solve the problem.

Also in Calculus 1,2,3 and so on, you learn increasingly more complicated techniques of calculus to solve more problems. But when you start numerically solving them, linear algebra doesn't care. It doesn't care if your equation came from the Einstein field equations or Newton's second law. It's all the same to it. So in a sense, I can teach you everything you'll need to know about linear algebra in a single course but the same is not true of calculus, which requires multiple courses.


When I hear someone say "We don't need calculus!"

I think:

Okay, then go back to pre-calculus times and somehow still invent the modern world without it and then get back to me on how irrelevant it is.

Also, you don't get to wear this shirt: http://www.topatoco.com/merchant.mvc?Screen=PROD&Store_Code=...

My feeling is that it would take a few hundred years extra (or maybe more!) to get to the point where you had a computer without calculus. Might even take a thousand. You'd only be able to do discrete stuff, so while Babbage's difference engine might be up for grabs (and mechanical calculators definitely would be) things like the Norden bomb sight or naval firing computers wouldn't be, since they're continuous not discrete systems.

You might not EVER get to computers because things like Nyquist's theorem and stuff like that doesn't even exist in a discrete-only world. Things just sometimes work and sometimes don't and it's sorta related to the frequency, but how is unclear. Aliasing is just something that randomly bites you in the ass and there's no way to develop a rigorous understanding of it and to engineer yourself out of the morass.

I would not volunteer for that mission. I like the modern world.


I must disagree. In my experience Linear Algebra has been so much easier than Calculus because even though at times it can be very abstract it is still more familiar as most of the time you're just solving equations. My math professors have done quite a good job in connecting the two subjects. For example, differentiation and integration of polynomials can be expressed as a linear algebra problem, etc. etc.

"The fundamental theorem of calculus is a legitimately earth-shattering breakthrough: none of the theorems in linear algebra come anywhere near it in cultural importance."

Linear Algebra has the Fundamental Theorem of Algebra (i.e. every polynomial has a solution in the complex numbers) and also the Isomorphism Theorems that say all vector spaces of n-dimensions are the same. For example polynomials of the form c+bx are equivalent to the complex numbers (over the reals) which are equivalent to two-vectors (x,y). Quantum physics is basically linear algebra, with quantum states defined as vectors in a quantum vector-space.

Don't dismiss the impact of Linear Algebra.


In what sense is the fundamental theory of algebra (FTA) a theorem of linear algebra? It's rarely proven in linear algebra courses. Most proofs of the FTA involve concepts from the analytical side of mathematics.


> They have enough trouble understanding functions in Calculus I: linear algebra's "functions are basically matrix multiplication" feature would NOT help.

What do they have trouble understanding about functions?

Actually, it took me until just a few minutes ago to realize that one of the major differences between my CS-centric way of thinking and the proper mathematician's way of thinking: mathematicians treat all coextensive representations of the same object as identical, while a computer scientist tends to treat only provable intensional equality as actual identity.


I've had 4 semesters of Calculus in my CS program. For me, it only helped to scare many students and made them pursue another degree. We only had more CS classes than others junior year. Fortunately, I'd been involved in open source and knew that wasn't the reality.


> Another thing to keep in mind is calculus has had much more impact on history/society than linear algebra.

Yes, it was amazingly important during all the wars and conflicts of the last century.

I don't think it was for the memory of Newton that they choose Calculus, but because it was more useful in a war machine than the alternatives.


I think Strang has some good points, and his advice is true for the majority of students who won't go on to use substantial amounts of mathematics or need to learn much math in the future. However, if you do want to learn more math beyond calculus and linear algebra, then a thorough understanding of calculus is absolutely a prerequisite and you honestly probably haven't learned it well enough taking the typical sequence of classes.

For example, I'm currently studying probability theory at a fairly rigorous level (building up from measure theory) and my relative lack of skill with calculus is the main stumbling block. And I was recently talking with one of my professors who said that the biggest obstacle he sees students facing in trying to understand advanced mathematics isn't the difficulty of the math itself, but in their lack of mastery of the fundamental calculations that are necessary to follow the ideas and to come up with them on your own. Fluency with calculus is an enormous part of this. That said, I agree with the point, which is that for most students linear algebra is more useful.

Actually, I might go a bit farther and say that most students don't need to learn much math at all, at least in the way it is taught. Math is pretty poorly taught for a subject that is actually quite interesting once you start to understand it better (part of the problem is that in order to get to the interesting parts you need to first master all the boring stuff), and most people won't use any nontrivial math in their future lives.


This may be true for mathematics in the analytical tradition (things like Topology, Measure Theory, Real/Complex Analysis), but Linear Algebra is far more important in, unsurprisingly, algebraic disciplines. These include Number Theory, Field Theory, and Mathematical Logic.

While studying mathematics in college, once I had finished my Real Analysis requirement, I jumped headfirst into the algebraic side of things and never found myself using any sort of calculus. Even in my Topology course, we focused much more on using techniques from Real Analysis than specifically calculus topics (the former just being a generalization of the latter).

Probability theory is somewhat deceptive in its classification, since much of it "feels" a lot like a discrete mathematics course; however, much of the concepts, like you say, are underpinned by measure-theoretic principles, which is heavily analytic. It makes sense that calculus would come in handy in a much deeper study of probability theory.


That's a good point, my own studies have been biased towards the analytical side of mathematics, but I can definitely see how calculus could be less useful in other fields, e.g. I don't remember ever needing to do an integral when learning algorithms. And the opinions of my professors are of course biased as well, since they've spent their entire careers on analysis.


When analyzing algorithms, I have used calculus as a shortcut to many calculations. I believe that learning calculus is not just about its direct applications. Calculus often comes up when just dealing with your set-up.

You might make a discrete model for a problem and end up using calculus for approximations, equivalent calculations, or reasoning.


I watched Prof. Strang's Linear Algebra courseware, great stuff. He is a wonderful teacher.

http://ocw.mit.edu/courses/mathematics/18-06-linear-algebra-...

As someone who hadn't studied it for 20 years, it was invaluable to revisit the subject as an adult with the ability to reflect "ah, I can use that for... " instead of "Ok but when am I ever going to use this"


Those lectures are fantastic. Pro-tip: watch them at 2.5x speed (you'll keep up just fine, and you can always pause to play with the concepts in Sage or whatever).

Bonus: when you switch him back to normal speed, he sounds stoned out of his mind.


This is a great tip. I've been using it for a few years now to watch various lectures on youtube--especially mathematics. Professors lecturing mathematics tend to speak slowly since they don't want to say anything erroneous. Speeding them up is usually amazing. For youtube; open the javascript console in your browser and type '$('video').playbackRate = 2.5;'. I've found that each lecturer usually has their own magic number regarding speed--after watching someone for a few hours and varying the speed you can usually find it.


His lectures are wonderful! In college I skipped my regular Linear Algebra lectures, and used the time slot in my schedule to watch his MIT lectures instead.

I ended up scoring one of the highest grades in the class.


Finding out the Fourier transform was a way to multiply matrices was a mind blower :)


I suppose you mean this?

http://ocw.mit.edu/courses/mathematics/18-06sc-linear-algebr...

I was a bit confused by your statement. You mean that the FFT can be done by matrix multiplication? I tried to interpret your statement the other way around, that you can somehow transform two matrices before you multiply them, trying to fit it into the convolution theorem, but I couldn't make sense of it that way.


I was puzzled as well. There is an analogous of the FFT method of multiplying polynomials and integers to the matrix multiplication case [1], but it is not as simple and probably does not belong in an introduction to linear algebra.

[1] http://arxiv.org/abs/math/0307321v2


As someone who did work at a University who also loved classes I would bring friends who were visiting to classes and every time they would say something like this, "I hated Philosophy as a college student, but somehow now I loved every second of that class." They were always surprised by that experience.


Incredible. I learned LA in school but it didn't click 100% until I watched Strang's lectures. I think they are ideal for someone who's already reasonably comfortable with matrices and vectors, but wants to see that higher connection and beauty. His lecture deriving the determinant from a few simple properties is magical to me. His emphasis on the fundamental subspaces of a matrix, and the geometric interpretation over the equation-solving one, are ideal for anyone who thinks visually/spatially.

That was the first online lecture series I ever watched, and I've been hooked ever since. I am overflowing with gratitude that so many professors at the world's best schools have put their lectures online. It is so good for a student like me, who went to a liberal arts school that didn't offer many advanced math or CS courses, and learns better from lectures than books. Strang's Linear Algebra course was one of the first way back in 2002. You're right, he is a wonderful teacher.


Well, I was going to say "you should also check out Walter Lewin's Physics lectures, they are equally stunning"

But it seems MIT took them down

http://www.reddit.com/r/Physics/comments/2ope15/mit_indefini...

This borders on criminal and is certainly a tragedy.

I'm quite struck and emotional about it.


This OCW was my lifeline during my college Linear Algebra course. Learned almost all the course material from 18.06 videos. Awesome, awesome stuff.


My linear algebra professor insisted that we, as computer science graduates, know and can reproduce most if not all the proofs of the theorems presented in the class. He didn't have such demands for math majors. His point was that we as computer scientists / software engineers / programmers will always have to understand and construct all the minutiae of the programs we write (otherwise they won't work). In contrast, mathematicians could use a lot of it "just" as tools.

I also had to go through a lot of heavy calculus courses which were very proof-centric and built from first principles. Rarely we skipped a proof of a theorem and we were expected to be able to reproduce and understand all of them at exams. I hated it often back then. However, these days I can appreciate the rigor it has taught me and I would wish I could send some of the programmers I met to similar courses so that they would learn to think clearly and precisely and be able to spot unmet or missing assumptions.

Our teachers and the creators of the curriculum were not stupid or naive. They knew well how much of hands on calculus we would use in practice vs. algebra or linear algebra. One of their main reasons for including calculus was to teach us rigor. I know that once someone overheard two teachers talking - the calculus teacher was asking someone from a CS department what to teach the CS graduates and the reply was that he can teach us whatever he wants, he just should make sure to teach us to think. I think they were quite successful at it even if it wasn't a very pleasant experience for most.

That also reminds me of this very relevant and true comic: http://savethepenguins.net/image/237344573 ("Wtf, man. I just wanted to learn how to program video games.")


I agree. As far as calculus goes, I am more enamored with books like Spivak's (http://www.amazon.com/Calculus-4th-Michael-Spivak/dp/0914098...) that take a proof-centric approach to teach calculus from first principles.

Incidentally, for those who want to learn linear algebra for CS in a mooc setting there are 3 classes running at this very moment:

https://www.edx.org/course/linear-algebra-foundations-fronti... (from UT Austin)

https://www.edx.org/course/applications-linear-algebra-part-... (from Davidson)

http://coursera.org/course/matrix (from Brown)

The first 2 use matlab (and come with a free subscription to it for 6 months or so), the last python. One interesting part of the UT Austin class is that it teaches you an induction-tinged method for dealing with matrices that let you auto-generate code for manipulating them: http://edx-org-utaustinx.s3.amazonaws.com/UT501x/Spark/index... .

And of course there are Strang's lectures too, but those are sufficiently linked to elsewhere.


My calc I course in university was applied calculus without a text. I had to go back and redo single variable by reading Spivak (and Polya's How to Solve It) to figure out the proofs in Concrete Math by D. Knuth, et. al.


You can probably do worse than to cut mathematics up into "algebra" (not just linear) and "analysis" (calculus). They're almost completely diametric points of view by the bulk modern mathematical canon [0].

There's a fair point to say that one shouldn't learn so much calculus as to lose your linear algebra, but the opposite is just as true. There are a lot of comments in this thread along the lines of saying that linear algebra is far more useful than calculus in CS. This statement feel not exactly unfair but certainly like it's a reactionary measure.

Whenever you have a notion of change you have some kind of calculus underlying it. Even if you end up having to view that notion of change algebraically. For instance, there's a famous functional programming result that there is a well-defined "derivative of an algebraic data type" which allows you to talk about changes in data "over time".

I'd also feel very bad for anyone who tried to learn too much probability without studying quite a bit of calculus first.

[0] Sure there's discrete calculus and algebra underlies the mechanism of most calculus computation. Likewise there are continuous groups and you can't avoid how topology mixes the two concepts.


I really can't agree with this strongly enough. I had enough Calculus and Differential Equations courses to kill a horse in college (Chemistry major) and I've never used that knowledge once in my entire programming career. But I find uses for linear algebra and matrices in coding all the time. Thank god Strang's excellent course is available online for free.


If you were looking to become a programmer, maybe you shouldn't have majored in chemstry, where strong Calculus foundations make perfect sense :)


Chemist here, who also programs. Inorganic chemists do calculus all the time. Organic chemists do arithmetic (stoichiometry) and some discrete analysis (instrumentation, the computers do the heavy lifting), orgo is primarily a linguistic discipline, i.e. we are expected to parse sentences like "benzophenone in THF was refluxed in the presence of Pd black for 2 hours" and to know, without being told, that there's a workup before the next step in the reaction.


Exactly. I was an organic chemist. Never used a lick of Calculus. I didn't get into programming until I'd already been working as a chemist for about five years.


In order to understand quantum mechanics, the foundation of modern chemistry, properly, don't you need a strong appreciation of calculus?


"I learned more than enough welding when apprenticing as a metalsmith but now as a chef use none of it."


At least at that time the comp sci people I knew had to take about the same number of math classes the chem people did. Maybe that's changed now.


Perhaps amusingly, the Computer Science education at the University of Copenhagen includes no mandatory Calculus course, but it does include a Linear Algebra course. Even so, students have a hard time seeing what they'll use Linear Algebra for, before later on in their education.


Make them code simple wolfenstain 3d clone with opengl, directx or even unity3d.

It was a revelation for me when I've tried to get into game programming, and half of the math taught in primary and secondary school became immediately useful, also linear algebra (which was only started at the final year of high school). Made linear algebra courses on the university a lot more interesting. There's a difference if you look at a class "what I need to pass an exam", and "what I could use in my next game".

Also tinkering with 3d graphics makes thinking in vectors, planes, matrices, dot and cross products intuitive after a few weeks.

BTW I wonder hwo many kids learnt basic linear algebra from Denthor's Asphyxia tutorials :)


I can't see myself really needing to apply any Linear Algebra concepts writing a Wolfenstein clone in Unity3D. It does everything for you like that (shader graphics, collision detection, vector math, etc). All you'd have to do is code some simple AI for the enemy gameobjects and make the player gameobject in the standard FPS shooting format, which there are pre-built defaults for in the example projects. The majority of this project would be copy and paste with minor modifications.


Even "just" ai can teach you a lot. For example trying to calculate if enemy can see you, or which path he should take is interesting.

Of course you can go far with "turn towards player and shoot" ai, but where's the fun in that.

Actually it would be fun to give student's assigment like:

here's the AI interface, make a game using it, and write your bot. Then we will run all the bots in free for all mode on all the games (except your bot on your game), and we will grade by sum of the points.


>trying to calculate if enemy can see you

Send a raytrace from the bot to the player and see if it hits something other than the player, which is just one line

Pathtaking beyond simply moving towards the player is a lot harder, yeah.


If you just send 1 ray from eye towards the center of the enemy (or head, or whatever) - and this particular part of the body is occluded by a wall or a chair, but the rest isn't - the result will be wrong.

You can accept that, use many rays, or do proper view frustum/body model(simplified probably) intersection, after cutting the view frustum with the terrain.

You can also consider lighting (so enemy can't see you hiding in a dark corner), or even dynamic lighting (and give enemies light sources).

Everything becomes hard after you think about it enough :)

But I agree simple solutions + some tinkering with parameters works most of the time (and gamers probably won't notice).


I think this observation highlights the problem with Linear Algebra, the applications aren't encountered until upper level courses in other fields, and then they teach what you need to know for the course.

I took two semesters of Linear Algebra in college, the first course was an overview for physicists and engineers and was little more than the mechanics of matrix operations, and find eigen vectors and eigen values. The second course focused on Linear Algebra as an abstract algebra, with near zero application. Subsequently, reading this paper[1] and the way the author used Linear Algebra blew my mind, why wasn't I taught about THAT kind of Linear Algebra?

I suppose my examples illustrates: great teach Linear Algebra, but what facets to concentrate on? Abstract or concrete applications?

[1] http://www.geometrictools.com/Documentation/IntersectionSphe...


I took a 3d graphics programming course where we built our own raytracer for our final project. We learned the math for a few different intersections in order to make the raytracer. So if they offered it, that would have been the course to take.


Personally, I skipped taking Linear Algebra in college, because I assumed it was basically 'Algebra II' and intended for people who weren't taking math-intensive tracks, like liberal arts majors. Although thankfully in my 3D graphics programming class we spent some time on Linear Algebra, so I got my misconception cleared and saw how it was useful (even though it looked like Voodoo magic to me how it worked for most of that class).

So yeah, maybe I'm just an outlier, but part of it may be a naming issue also. "I already took Algebra way back in 8th grade and it was a breeze! I don't need to take another Algebra class!"


Statistics are wildly important for both business and understanding the world around you, and without solid calculus skills you cannot truly work with statistics.

Its not something that comes up every day at work, but it is still pretty often. For example, I was on a large team developing a complicated distributed system. The performance requirements were uncertain (like 500000 clients +- 200000, sending 10 +- 30 messages per second, etc). The system itself was in early development, hadn't been tuned, and we had only rough performance numbers for it.

Given only that, I needed to purchase the correct amount of hardware for the test lab, keeping in mind that the lead time for hardware was several months. I needed both statistics and some basic calculus to build a model based on what we knew, calculating variances for each intermediate and output, and then getting 50%, 90%, 99% confidence level estimates.

In the end, we got pretty close to a bullseye. On the other hand, I learned the important lesson to hide all the math from the client, lest they get awfully confused.


The Khan Academy lectures are a pretty way to start: https://www.youtube.com/playlist?list=PL835D2252574274C5


Possibly understated... not only is there the three course sequence + diff eq, there's usually also a two-sequence real analysis course (more or less "careful calculus") and probably either PDEs or complex analysis to boot.

I took the discrete emphasis when I studied and I think I actually ended up with fewer discrete classes (5, IIRC) than calculus classes (6).


Although it's not explicitly stated in the article, I think Strang's remarks are actually directed at non-math programs in STEM (I assume you are at least a minor in math because of 3 analysis courses). Math majors will get plenty of algebra either way, but it's the engineers and scientists who miss out from overemphasising calc over algebra.


I've never understood why Calculus is taught as a separate course. It seems so detached from practicality. I've always thought students should take perhaps 4-6 semesters of Physics -- and introduce derivatives, integrals, etc. as needed when they can be demonstrated to more easily solve the physics problems. If the physics content was more spread out, there would be ample time to introduce the necessary math concepts, instead of stuffing them into a separate course.


I remember, in my high school calculus class, the teacher said that there were six or seven applications of derivatives, and more like 40 applications of integrals. It's way more than just physics.


What is the context of this short essay? Why did he feel the need to write it, at that time?


I don't disagree that linear algebra isn't given enough emphasis. However, the first thing I thought of when I read this was something attributed to Richard Hamming, who is known for the transcript of his talk that pops up on here frequently, You and your Research [1]. I got this from a review of his book on Amazon [2]:

> Hamming, in discussion, concluded his life thinking "the best tool to teach thinking was to teach the calculus."

I'm just thinking here that Strang is saying Linear Algebra is more useful for _applying_, where Hamming would say Calculus is more useful for reasons that go beyond applications (not that it doesn't also have applications).

[1] http://www.cs.virginia.edu/~robins/YouAndYourResearch.html

[2] http://www.amazon.com/review/RWPF0EL3E7GJZ/ref=cm_cr_pr_perm...


I had the fortune of taking 18.06 (Linear Algebra) from Gilbert Strang around the time of this essay. And like other commenters, I will agree that he was a very good teacher. I could tell that he held a great emotional attachment to matrices; he'd often anthropomorphize them with genders and refer to them as "little guys".


I think taking less calculus and more linear algebra is a mistake.

There is a really good book in two volumes that I feel shows how methods of linear algebra can be extended using calculus to solve a wide range of problems: A Course in Mathematics for Students of Physics:

http://www.goodreads.com/book/show/2685577-a-course-in-mathe...

Volume 2 is more important. It starts demonstrating how methods of Linear Algebra can be used to solve circuit problem, then generalizes this to higher dimensions and from discrete to continuous spaces using the exterior derivative. Seeing this demonstrated gave me a much clearer, more intuitive understanding for both calculus and linear algebra.


I wish Gilbert Strang would teach calculus and write a book as he did for linear algebra.


You are in luck!

Strang, Calculus, 2nd edition: http://www.amazon.com/Calculus-Second-Edition-Gilbert-Strang...


And the first edition is freely available at Open Courseware: http://ocw.mit.edu/resources/res-18-001-calculus-online-text...


I wish every day is like this!


I took Gilbert Strang's Mathematical Methods for Engineers course at MIT. Can't seem to find the link for it, but I would recommend the online version for anyone who wants to learn linear algebra.


I don't think you can cleanly separate the topics. Most applications of linear algebra come from calculus since after all you use calculus to linearize something. In case it was already linear it's probably a linear differential equation -> calculus again. Complex analysis is also closely related to linear algebra via geometric algebra. Rather than teaching them separately and emphasizing linear algebra more, it would make more sense to approach calculus + linear algebra as an integrated subject.


Anyone interested in getting started with linear algebra can check out this printable tutorial: Linear algebra in four pages: http://minireference.com/blog/linear-algebra-tutorial/

The page also links to another short tutorial which explains more of the geometric intuition behind linear transformations.

Neither tutorial can serve as a full course, but reading them will give you a good overview of what linear algebra is about.


heh, try teaching galois for a year, you'll understand why calculus is so popular.


it's a beautiful branch of mathematics though.


everybody thinks they know algebra if they study linear algebra. this is worse than not knowing about calculus i guess.

can't image teaching point-set topology, teaching uryshons proof to undergrads, gee.


14 years later, the popularity of machine learning and "big data", which requires quite a bit of linear algebra, makes this only truer.


It also requires quite a bit of calculus though!


Calculus is indeed a required tool if you want to design new machine learning algorithm. But without linear algebra, good luck implementing an algorithm or even using a library to apply it on a concrete case.


I don't see why this is an either/or decision?

Without a firm understanding of probability then good luck interpreting or fitting a model. Without a firm understanding of linear algebra good luck implementing or using an algorithm.

And hell, if you have a streaming algorithm you probably no longer need linear algebra but to grasp either the probabilistic or geometric interpretations you'll probably need calculus.


> We need to present the mathematics that is most useful to the most students.

I wish it weren't this way! Mathematics is a wide and fascinating field, and I think more students could find interest if our schooling reflected that. But moving some of the detail and drudgery that is the calculus sequence to subject matter that requires it, in order to make way for linear algebra, is a great start.


I agree completely, but calculus touches so much of advanced math that at some point you have to learn it in depth (assuming you actually want to learn advanced mathematics). I'd argue that linear algebra is a more interesting subject for most people, but calculus contains more important ideas that are crucial if you intend to continue studying mathematics.


I have a major in Mathematics, and I truly didn't need calculus after the multivariable/vector calculus course I had to take my Freshman fall. Why did I have to do so much if it wasn't going to be necessary until graduate school at the earliest, or perhaps never depending on specialization? And why was it a prerequisite for my other math courses (in algebra, topology, real analysis, number theory, and combinatorics)?


How was topology motivated without talking about generalizations of limits?

(Honest question, no snark.)


It's a good question.

I think my first topology course, like in many intro analysis courses, shared concepts from calculus without requiring any previous expertise from our calculus courses. A rigorous approach to real analysis or point-set topology will be very set-theoretic, and you build the general definitions of continuity, compactness, etc from the ground up. So, I would have used concepts from my analysis and topology courses in calculus, if only it had been taught in the other order.

I guess it feels like limits are a rigorous foundation for calculus, but they are not calculus. In any case, I didn't need anything from differential or integral calculus (or that entire required course developing the theoretical basis for Stokes...), which is what I mean to say.


Mathematics is not taught well at all. Read a calculus textbook and you find out most of the words are undefined to most students, which are then taught to them 2-3 calculus classes later in 'discrete math'.

I think starting with discrete math first, where you teach what sets are, how proofs work and so on would be a good start. Or make your textbooks usable and not just containers of problem sets with unreadable extra paper inserted to make your back hurt more. Or make the target students and not their professors.


I remember taking Calculus 1, 2, 3 but only having 1 linear algebra course and some engineering math. Although I did well in them, as a developer, I wish I payed more attention to linear algebra or at least wished that my uni offered more courses related to it, like discrete math and maybe even statistics. Good thing I still have my old books!


To reinforce this point: here's just a fraction of what you can do with linear algebra [1]

[1]: https://medium.com/@jeremyjkun/here-s-just-a-fraction-of-wha...


To be fair, almost all that stuff uses calculus too.


Another perspective: I am trying to teach myself physics from Morin's books, and for the first time I see how well structured was my lower division calc sequence d for this application.


i actually have taught some linear algebra concepts on youtube and have gotten several hundred thousand views and some positive feedback. i think the key is trying to figure out a framework of intuition to help students through all the math but i liked the idea of four key equations we are trying to solve as a framework as well


The way this was typeset...it's making my head hurt.


> Too Much Calculus

Not for me! Heck, I never took freshman calculus or pre-calculus and, instead read a book and started on sophomore calculus then continued with, right, ordinary differential equations, advanced calculus for applications, and advanced calculus, topology, modern analysis, measure theory, functional analysis, exterior algebra, etc. Not enough calculus for me!

Strang, of course, is emphasizing linear algebra. Great! It's terrific stuff. I had an abstract algebra course that started on vector spaces and matrix theory. Then I got famous book from Princeton on multi-variable calculus and read the first chapters on linear algebra, e.g., learned Gram-Schmidt. Then I got a linear algebra book and read a few chapters and applied them to some problems in classical mechanics. Then I wrote my math honors paper in group representation theory, right, more linear algebra. The out of school, I wanted to know linear algebra better and read a good book cover to cover, carefully. Then I got Halmos, Finite Dimensional Vector Spaces and read it cover to cover. Wrote Halmos about his proof of the Hamelton-Cayley theorem and got back a nice answwer!

Then I got another famous text on multi-variable calculus and read it cover to cover, including the parts on exterior algebra. Then I did that again from Spivak's Calculus on Manifolds -- right, a lot of both calculus and linear algebra. Then I got Forsythe and Moler, Computer Solution of Linear Algebraic Systems, read it cover to cover, directed a project in interval arithmetic for linear algebra, took a course in numerical analysis with a lot of linear algebra for solution of partial differential equations, e.g., iterative solutions (Gauss-Seidel) and wrote corresponding code.

I also used linear algebra in work with the fast Fourier transform, multi-variate statistics, and more.

For more on linear algebra, I read the first part, linear algebra, of von Neumann, Quantum Mechanics.

From a world class guy in linear algebra, I took an advanced course in linear algebra -- by then the course was beneath me and I led the class by a wide margin. Favorite theorem -- the polar decomposition.

Read Coddington's book on differential equations -- gorgeous book.

The combination of linear algebra and calculus, with other math, continued.

At one time a little calculus did a lot: The initial value problem for the little first order, linear ordinary differential equation

y'(t) = k y(t) ( b - y(t) )

saved FedEx, that is, kept it from going out of business. So, useful calculus!

So, sure, I like linear algebra and like the combination!

I definitely do not see that I had too much calculus!

Here's what I saw, and still fume about: All I could get in chemistry was one too simple course in high school and one too simple course in college. All I could get in physics was one too simple course. I college, by the time I got to Maxwell's equations, I didn't know enough calculus to do really well. In college I never got math enough to do well with quantum mechanics.

But in grades 9-12 and then in the first two years of college, I was force fed like a goose with six years of English literature, with each year yet another play by Shakespeare. I agree that there was a good writer in England in the 1600s -- Newton! So it was also Chaucer, Milton, Wordsworth, ..., Dickens, the great natural order (all corresponding theorems and proofs omitted!), etc. With good information about people? Nope.

Then there was history: It never got to the 20th century. It never touched on technology, economics, or any other causes. There was plenty of time in history courses; it's just that the courses didn't have much content.

If my startup works, then I'll get back to more in calculus, functional analysis, stochastic processes, and mathematical physics. "More calculus, Ma!".




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: