An Intuitive Guide to Linear Algebra 371 points by nkurz on Jan 21, 2015 | hide | past | web | favorite | 51 comments

 It seems like college Linear Algebra expeiences fall into one of two buckets: either you took a "Linear Algebra / Differential Equations" class where you learned no theory and saw a bunch of practical examples and did problem sets that were routinely north of 10 pages, OR you took a highly theoretical course where everything was far too abstract for your first year of college level of comprehension.I took the first route and retained none of the information, despite doing well in the course. Then I took computer graphics and cried all the way home.This link seems like a happy middle ground.
 I found Linear Algebra Done Right by Axler to be a fantastic theoretical text that makes everything click together from day 1. It's derived from his paper Down with Determinants.He starts by explaining that linear algebra is about studying the properties of linear transformations over vector spaces, and then goes on to explain what this means. In many theoretical books it takes ages to get this message, and in some practical ones you get lost in a sea of matrices without even understanding what a matrix actually is. Furthermore all his proofs are very slick.
 Seconded. I was very skeptical of LADR when I was forced to use it freshman year. I thought I already knew all the practical stuff from Strang's book, but learning linear algebra from the mathy angle has paid off again and again and again. Choices of units & coordinate systems are impossibly confusing without having a foundation in how to reason independently of such arbitrary choices and add them in later.I think the turning point came when I realized that "abstract" linear algebra was actually much closer to physical reality than "applied" linear algebra. Superficially, it seems like the exact opposite state of affairs from computer engineering, where grids of numbers take you closer to "bare metal" and "higher level reasoning" takes you further away. In math, it's the grids of numbers that are further removed from "bare metal" and the abstract reasoning that you can count on when all else fails!I didn't need to be convinced that I would eventually come to regret having an insecure foundation so much as I was mixed up about up vs down and trying to build a foundation in the middle of the sky.
 Anyone want to study group this book? I did Strang a few years back, along with the OCW lectures, but it wasn't very proofs-centric.
 I'd be quite interested to keep in touch with some people studying these topics, even if we don't follow the same pace or curriculum.Despite having studied a decent dose of math during my undergrad (and currently during my grad), I keep on finding awesome books like Axler, Hubbard, Halmos, Neuveu... that make revisiting math so much more illuminating. Everything clicks much more quickly and I never loose momentum now.Some of these books I found online, so I'd definitely like to exchange more references, problems, tips & experiences.
 MIT has some more proof centric courses http://ocw.mit.edu/courses/mathematics/18-700-linear-algebra...
 I can't recommend this book more. I read through about half of it before I took my first real course on linear algebra. The result? I got literally 100% in that class -- I didn't miss a single point on anything the entire semester. The teacher was flabbergasted, but I had my secret weapon -- Axler. The rest of the class was relying on the course textbook, some awful fat thing that I'm pretty sure nobody could ever really learn from. Axler's book made the concepts so crystal clear to me that it felt as if I'd known them since birth.I can't guarantee that everyone is going to have the same experience, but yeah, it's a good book.
 Wow, really? I mean, I got an A- in linear algebra with some normal old textbook I don't even remember, and certainly learned it all in terms of linear transformations over vector spaces... but then again, I didn't have it as a doubled class with differential equations in the first place. Is Axler that good?
 It was partially a fluke I'm sure. I'm notoriously sloppy with arithmetic and no book could save me from that.It's not necessarily that Axler is that good. It's that it's just solidly good all throughout. It has the right focus. The goal of the book was very clearly to explain linear algebra, not to make money or get contract deals with college bookstores. And the proofs are very clearly written.I'm sure there are other books out there which are just as good as Axler. But compared to the more popular linear algebra textbooks, at least at the time, it seemed like it was in a class of its own.
 Any analysis recommendations? I have Hubbard & Hubbard in mind.
 I don't have any particularly noteworthy recommendations for ordinary analysis or real analysis, but for complex analysis I did find that Needham's Visual Complex Analysis was just as revolutionary for me as Linear Algebra Done Right (if not more so).
 In my experience, Linear Algebra Done Right by Axler is a great second book on linear algebra. But it makes sense to most students only if they've seen and done concrete matrix calculations before. Giving it to a mathematical novice is likely to lead to baffelment. Too abstract.
 I don't think it's too abstract. It contains plenty of geometrical intuition. And it makes things very clear and he says them very _early_. E.g., a matrix represents a linear transformation. It took me ages to understand this at high school because I was lost in a sea of matrices.
 It's strange how many of my CS friends were deathly afraid of linear algebra, yet still enjoying their graphics classes.
 I think I'm in that camp.I got a D in linear algebra, my lowest grade in university and one of my lowest grades ever, but excelled in computer graphics, and even got permission to take a graduate level course in it for elective credit as an undergrad.My recollection is that the problem sets in the linear algebra class seemed really abstract and vague, and I couldn't reason through it very well because I couldn't visualize what we learned, given the information from the class.Computer graphics, however, let me visualize what we were covered, and suddenly everything made sense. I could see, literally, what the end result was supposed to be, and it just clicked.Now that I think about it, it's kind of a shame I had the linear algebra class first. The CG experience would have helped me out quite a bit.
 What are your thoughts on those revisiting linear algebra to instead try their hand at learning geometrical algebra instead?
 You can't really get around linear algebra. Geometric algebra is really neat, and you can benefit from taking a geometric algebra approach to linear algebra (Macdonald's book is pretty good for that), but it enhances linear algebra, it's not a substitute for it.
 Linear algebra is cool, no doubt. This blog post is certainly well written and explains how to think of matrices as functions.I have written a blog post[1] that describes matrices as functions. This blog post is titled "Correcting common mathematical misconceptions" and walks through N dimensions, linear functions and linear algebra. After explaining what linear functions are, I mentioned nonlinear functions and how they often don't have a closed form solution and why mathematicians find bounds.
 >We’re getting organized: inputs in vertical columns, operations in horizontal rows.OH! Bloody hell, so that's why we write it like that!>The eigenvector and eigenvalue represent the “axes” of the transformation.>Consider spinning a globe: every location faces a new direction, except the poles.>An “eigenvector” is an input that doesn’t change direction when it’s run through the matrix (it points “along the axis”). And although the direction doesn’t change, the size might. The eigenvalue is the amount the eigenvector is scaled up or down when going through the matrix.Very nice! I'd had an algebraic (ahaha) understanding of eigenvectors/values before, but hadn't a geometric intuition. Thank you. Now I can neatly imagine why the eigenvector is orthogonal/perpendicular to the "direction" of the transformation.Determinants, too.Excellent article.
 I've always been partial to this: http://blog.wolfire.com/2009/07/linear-algebra-for-game-deve...
 They stopped posting these kinds of stuff. Sad.
 A lot of programming puzzles companies hand out can be solved with linear algebra. One company I interviewed at gave me a pretty complex problem, solved it with linalg. Didn't get the jib since I didn't use the usual data structures, but it was still a fun experience.
 Your use of linear algebra over what was expected would have significantly increased your chance of getting an offer if I'd been interviewing.
 I wish someone would better explain Lie algebra.
 Lie algebra is a group whose underlying set is smooth. For example, angles (rotations) on a circle (or units in the complex numbers) form a Lie algebra. You can add and subtract angles on a circle, and if you perturb one angle you perturb the result slightly: (theta1 + delta) + theta2 = theta1 + theta2 + delta.That's a trivial example, of course.Contrast with a symmetry group of a hexagon. There's no (non-degenerate) notion of a continuous transition from identity to rotate-by-pi.
 This is incorrect.A Lie _group_ is a group which is also a manifold, such that the group operations are smooth. (I don't know what it means for a set to be smooth.)A Lie _algebra_ is a vector space together with a bilinear operation (the Lie bracket, written [a, b]) which also satisfies the Jacobi identity:`````` [a, [b, c]] + [c, [a, b]] + [b, [c, a]] = 0. `````` How are they related? If G is a Lie group, the tangent space at the identity is a Lie algebra. What does the Lie algebra represent? Well, unlike a plain manifold, in a Lie group, given a tangent vector a, there is a path starting from a the identity in the Lie group always in the direction a. Given two tangent vectors a, b, you can flow a little in the direction a, and then a little in the direction b, or vice versa. The Lie bracket can be thought of as a measure of how the flow fails to commute.
 Ah! So its just a group that has a defined group of continuous transformations that can be applied to it?
 Not really.Very roughly: It's a group with a an infinite number of elements which is smooth, in the sense that there is a notion that elements can be close or far from each other, and locally around each element the group looks like R^n (for some n). Of course the group operation must play nice with this notion of "close". I.e. if elements A and B are close, after multiplying both of them by X, AX and BX can't be too far: the group operation does not "tear" the structure apart.So in the example above of rotations on the plane, you can visualize the group as a circle where a point represents the rotation by that angle. Locally it looks like R, but it has a different global structure.
 Actually, a Lie Group is a group which is simultaneously a manifold, such that the group multiplication corresponds to diffeomorphisms of the manifold onto itself. The Lie Algebra corresponding to that Lie Group is the tangent space of the identity. The elements of the Lie Algebra generate the given Lie Group via the exponential map.
 What do you mean by "generate" here? The exponential map is not necessarily surjective.
 I'll would request the same for p-adic numbers, while they're at it!
 Interesting. Very similar to the way ints are represented in Python (since they have arbitrary length, negative numbers are represented with infinitely many 1's in the most significant bits).
 p-adic numbers are measure of necesary- resolution to describe. A pair of p-adics is very close when you can use a very coarse ruler (of tick size p^n, large n) to measure the exact difference (1 vs 10000000001). p-adic numbers are far away when you need a very fine grid (1.0 vs 10.000000000001) to measure the exact difference.
 I'd request the same for whatever mathematical concepts Quants use for stock market trading. Or at least a list of concepts useful for such an endeavor.
 Obligatory Schemes and Syzygies
 If you like this, then go read the No bullshit guide to math & physics after this: http://minireference.com.Love the trend of indie publishing that has been exploding in the past 2 to 3 years. The biggest issue is that the marketing & design is often mediocre. Content wise, completely awesome though. I lost interest in books from larger publishers years ago.
 Is this book worth getting for someone who doesnt remember anything about math from high school and beyond?
 I think so – it covers the fundamentals. Numbers, Algebra, Functions. Not exactly in depth, but if you've done high school math you should quickly remember and be able to get to the fun stuff.
 I always loved the fact that matrices encode operation on matrices (see comment#1 http://betterexplained.com/articles/linear-algebra-guide/#co... ). Very meta-circular, Ycombinatorish. Would that mean Linear Algebra is homoiconic ?
 Having group or ring structure isn't too surprising and isn't too intimately related to the idea of metacircularity or the Y-combinator. It's actually more interesting to really see that matrices encode linear transformations, and multiplication is function composition.
 Yes, I can't recall where I first read this (someone's comment on HN most probably) but it was very enlightening.
 Perhaps I don't recall my linear algebra well (we called it engine math), but there's a hell of a lot more to it than just passing inputs through a series of operations. Linear algebra is more about the methods used to design very elegant models and transforms and understanding the limits of those models and transforms.
 No, those are the applications of linear algebra.Linear algebra itself is just a study of vector spaces and linear operators and transforms and such.
 For context, I don't think I ever knew the thing we were studying was called linear algebra, so it took me another 17 years to realize that is the common name for what we studied in that class. Now I'm wondering if maybe one of the intents of the small engineering-focused school's tightly integrated math-science-engineering curriculum was to avoid an introductory linear algebra class. We had to understand matrices and vector spaces earlier for other classes and didn't do what you call the "applications" of linear algebra until sophomore year.
 I took linear algebra 2 semesters ago.It was awesome! Highly recommend learning it, I use it on a daily basis in my work in 2D and 3D!
 I find it non-intuitive that many of the input-output diagrams flow from right to left.(We all know that causality only flows left-to-right.)
 That's because matrix multiplication is noncommutative, and it's traditional in linear algebra to write the inputs matrix on the right of the operations matrix. Just like in normal algebra, you write 5x+7y instead of x5+y7. So the picture matches up with the notation.
 Has anyone bought their calculus course? is it worth it?

Search: