Hacker News new | past | comments | ask | show | jobs | submit login
Linear Algebra by Jim Hefferon – free textbook (smcvt.edu)
459 points by n-izem 51 days ago | hide | past | web | favorite | 60 comments

If you want to learn linear algebra by coding in python, this is hands down the best book out there.

Coding the Matrix - Philip Klein [0]

It used to have a Coursera course, but I think it's been taken down. The website has videos of the course taught at Brown I think.

The associated website is: http://codingthematrix.com

[0] https://www.amazon.com/Coding-Matrix-Algebra-Applications-Co...

The course is available here: http://academictorrents.com/details/54cd86f3038dfd446b037891...

It is legal, not a pirated version.

That was a great course! I'm glad it's still accessible. I was not aware of this resource for courses. There was a fantastic course on probability from the University of Pennsylvania that disappeared when Coursera went through a change a few years ago. Maybe I can find that one there too. Thanks for the link.

Did you find the Upenn course on probability? I have been looking for it for a long time. The professor was absolutely brilliant.

How nice to see this link here! I could answer any questions that anyone has. --Jim

First, let me say THANK YOU for making this a free textbook available online to everyone everywhere. The world owes you a debt of gratitude. I, for one, will do my part and make a donation via the link at the bottom of your textbook's web page. I hope others here will join me.

My question is: What motivated you to create this freely available book?

Thank you for the donation. I use the funds to work on revisions of this text, and on developing others.

For motivation: briefly, I was at Union College and was asked to teach Linear. I saw a second edition of Strang's book, thought it was wonderful (still do), and adopted it. But the students, who were perfectly good students, did not get it. They just couldn't solve the problems that a person should solve in that class. I decided after some reflection that the folks I had need to be brought along to where they are ready for a higher level understanding. So I put together a presentation that uses lots of examples, motivation, and naturalness to try and develop maturity in people who do not yet have it.

(Again, I'm not talking Strang's book down, it is very fine indeed. But the students that I have in front of me at this point are just not yet ready.)

If a link is getting hits with comments 10 years later, should there be some sort of special class of points? Everybody else is running JV sprints and this guy is dropping olympic marathons.

Couple of things to note about this link: Contains the 3rd edition of the book. Has extra problems and solutions from when he taught the class in 2014

Here is another [1]--perhaps not to everyone's taste, but it was influential for me. In particular, the first words of the preface: "The underlying spirit of this treatment of the theory of matrices is that of a concept and its representation."

[1] https://archive.org/details/LinearAlgebraAndMatrixTheory

My own take is that the "theory of matrices" (not a standard usage but should be) is worth learning for some (sp if you're going into hardcore numerics like stability of algorithms) but not for everyone. It's like studying calculus as the theory of smooth curves -- it robs you of some potential for growth.

Also: some people just want "math for deep learning" and that's mostly baby calculus: the chain rule and gradient descent.

What is nice about this book is it includes solutions to exercises. I wish more proof-theoretic books included exercises in the back of the book like this one does. Yes, you can prove something in multiple ways, and no that isn't a good excuse to not include at least one version of the proof in the back of the book. It really benefits the self-learner. The math community seems to have a lot of artificial gatekeeping to keep non-academics out.

They do this in several ways.

1) They recommend textbooks to beginners that are too advanced for their skill level.

2) These textbooks do not include solutions and the advice given is that solutions would somehow rob them of the experience (don't let those that lack self-discipline ruin learning for non-traditional students that aren't in a formal classroom with access to professors and TAs). The self-learner needs some sort of feedback system.

3) They tell the self-learner that solutions aren't provided because everyone has the ability to know if their solutions are correct without having their work checked. Would you write a complicated program and write no test cases, or could you instantly know your program is bug free the first time you write it? Why is math suddenly any different?

Despite popular elitist opinions, I'd recommend this book over Axler for the beginner that doesn't know linear algebra. Everyone says Axler is the perfect first book in linear algebra. It really isn't. Even Axler admits this himself in the preface. He assumes that it is a second approach to the field.

But people like to be elitist and recommend books to beginners that aren't always best from a pedagogical perspective.

Those are the same class of people that recommend Rudin or Spivak to someone that wants to study elementary calculus 1 material.

In my experience (math teacher at a community college for 20 years and former Ph.D. student who quit before finishing the thesis) the overwhelming majority of people can not learn mathematics properly from a textbook. They need a teacher or someone to help with problems. It's very difficult to get to a point of really understanding the definition/motivation behind concepts. Even with a teacher it's hard to understand.

Here's a simple example. The distributive property says:

a (b + c) = ab + ac

I can teach students to expand 3(x + 2). With some practice almost all of them will get this. They'll say

3(x+2) = 3x + 3(2) = 3x + 6

Do you know how hard it is to convince people that due to the nature of equality you can reverse the steps? Some never understand that

3x + 6 = 3(x + 2)

Some will never understand that 3x + 2x = 5x because of the distributive property and that combining like terms for the expression 3x + ax is the same process.

I don't think there is any desire to be elitist in terms of having good books that are appropriate. It's just hard to do. We've gone through various cycles of reforming calculus, introducing old concepts in a new way. Through all of the changes one thing has remained constant in my experience. The percentage of people who can get it remains the same.

I do agree with your point on providing solutions.

> the overwhelming majority of people can not learn mathematics properly from a textbook.

While you are right, a related issue is that most maths textbooks are atrocious at most aspects that aren't writing pages of equations. There is almost nobody sitting at the 3-way intersection of great mathematician, great writer and great educator who can then write great textbooks.

I've found books with terms like "History" and "Philosophy" in the title are much better places to learn about mathmatics; combined with wikipedia for formulas and details. Any book with excercises but no solutions or historical context has turned out to be basically useless to me, even as a reference (wikipedia is usually better for simple stuff).

I've got something like 4 books on statistics on my shelf at the moment. The only one that I've actually manged to read and learn something from has been Chatterjee's Philosophy of Statistics, because it talks about what techniques were developed in context of which problem, failed alternative approaches, explains what was confusing to some of the greatest minds in the history of statistics, etc. This has been vastly more useful in setting up a framework for what the world of statistics looks like that I can attach a whole bunch of proofs and suchlike too. It has been enlightening in a way that textbooks can't really manage.

It is easy to agree with these findings. In general it is important for humans to get answers to WHY. History answers typically to WHY questions and to some extent also examples, i.e. you see by example WHY the theory works.

What other books have you found helpful other than the Chatterjee text?

Did you mean "Statistical Thought" by Chatterjee?

Oh, sorry, yes I did. "Statistical Thought: A Perspective and History". The first half is basically all the schools of thought on how probability theory and the real world interact; so I keep thinking of it as a philosophy book.

You make good points.

A person that doesn't understand the distributive property, going into Calculus 1 is not the person that is going to understand Rudin or Spivak on their own without guidance. Yet, people in mathematics communities would recommend Spivak or Rudin anyway and look down on them if they used Stewart or Thomas.

To them if you aren't learning a subject in the fullest rigor possible, then it isn't worth learning. It doesn't matter if the person doesn't even know the basics, they blindly recommend these type of books regardless.

These are excellent books once you already seen the material before, not so much for the average person that doesn't know what the distributive property is.

> the overwhelming majority of people can not learn mathematics properly from a textbook.

There's a difference between a person starting off in mathematics, versus someone who has done several undergrad/grad level courses and knows the "system". If you've gone through math formally earlier in your life, you should be able to pick up a math textbook and learn the subject. Unfortunately, many math textbook do not make this easy.

I have 2 sons, and I saw how difficult is to build a clear explanation to them. I must quickly build a way to explain: 3x + 2x = (x + x + x) + (x + x) = 5x, and it is not easy. If I hesitate, they get bored and don't want to study anymore.

That's why Khan Academy is a so fantastical resource. They really build a very clear explanation of the concepts. Every teacher should watch it before classes.

Try beastacademy.com

It's not free but it's an amazing comic book textboom and fun instersting challenging problems for kids.

Buy the books and share them with your neighbors, and share an online account if you can't afford your own.

Would you make a book list for other mathematical subjects? I have tried following the "recommended book" route and it really isn't productive. The first person recommends A and then everyone else doesn't want to look at l33t so they concur.

I would highly recommend ASU's College Algebra and Problem Solving [1]. They have a large comprehensive test at the beginning that determines your knowledge. Should you be missing knowledge, they'll use the same platform to teach you all the prerequisites (i.e. they won't let you take the course until you go through ALEKS, their learning platform). After that's done, this course is something like preparation for college math classes.

So far, I like it a lot, thanks to ALEKS. Give it a shot; it's free.

[1] https://www.edx.org/course/college-algebra-problem-solving-a...

Completely in agreement. I really dislike when math books don't have solutions to exercises available, since I self-study.

Like Diestel's Graph Theory is great, it's the book you want to read if you're serious about the subject (D.B. West's book has exercises with solutions, I suppose), but there's absolutely no solutions to be found anywhere; the hints to solutions are now relegated to the "Professional Edition" as well, you can only find hints to old editions online. Makes me question whether or not I understand the material fully, since I can't check work.

So, as someone who finagled their way into an MSCS progress with no calculus... What intro calc textbook for a self-disciplined self-learnerv would you recommend?

I'd recommend "Calculus Made Easy" for intuitive exposure. It was the book Feynman studied from.

Then pick up any calculus textbook and chug through it (Thomas is good from what I've heard). Even if the questions don't have answers in the back of the book you can check your computational steps for free using: https://www.symbolab.com.

I also recommend lots of practice, so khan academy is good for drilling + any problem set book with lots of calculus problems, such as "Schaum's 3,000 Solved Problems in Calculus" or "Essential Calculus Skills Practice Workbook". They don't have to be huge calculus text, doing problems is more important than reading through 1,000s pages of colorful examples. You can find shorter calculus books that focus primarily in drilling calculus techniques. Focus on those to nail the techniques.

> They tell the self-learner that solutions aren't provided because everyone has the ability to know if their solutions are correct without having their work checked. Would you write a complicated program and write no test cases, or could you instantly know your program is bug free the first time you write it? Why is math suddenly any different?

Math is an art of making a specific type of rigorous, formal arguments that are meant to convince, beyond all doubt, another human being that one statement follows from another. If an argument like that fails to convince even the proof author themselves, then you know the argument isn't good even without comparing it to another solution.

In practice, after you've written down your proof, you examine it for weak spots. If any particular step raises doubts or objections then you recursively spell it out in greater detail (or replace with an alternative approach) until eventually all doubt dissipates. At that point you don't need to compare it to another solution.

In fact, as you recognized yourself, your solution is likely to be different from the book author's, so comparison isn't helpful. Similarly, a problem like "write a small application to..." in a coding book or an assignment like "write an essay on..." in a writing book don't include solutions. Code, proofs and essays are all too free-form to subject them to correctness verification by comparison with a standard.

Perhaps what you have in mind are examples. All good math textbooks include worked-out proofs that illustrate the techniques. In fact these are generally the bulk of the book! Perhaps you're asking for more examples. That's fair. Proof problems OTOH should be given without solutions to encourage independent creative thought of the reader and to avoid incorrect implication that there is the single correct solution.

(Disclaimer: Not a mathematician.)

The lab on the cover of the lab manual deserves all the upvotes.

It is kind of you to notice her. We had to say goodbye last year. She was a sweetheart.

It's nice to have a free book as a supplement. Linear Algebra Done Right by Sheldon Axler is a much better choice and when making an investment in learning, particularly a difficult subject like linear algebra, saving money with a free text is probably the most expensive thing you can do. You might save $40 but end up not really learning and wasting hours you don't have. Above all, find a good teacher.

What are your particular critiques about this book? There's nothing inherently inferior about a free resource. What makes LADR superior to this book?

Finding a teacher isn't at all mandatory, you can do math yourself by completing exercises, checking your work, and looking at the solution afterwards. There are plenty of places to ask questions if you get stuck.

>Finding a teacher isn't at all mandatory

I firmly disagree.

What a teacher can give you is perspective that you, not knowing the subject, cannot have -- and augmenting any particular viewpoint expressed in a book.

If people could learn everything from texts, we wouldn't have universities (for students) and conferences (for working mathematicians).

In theory, one can write a Great Text that explains an Idea. In practice, it's damn hard to do that, and it's far easier to impart understanding in a conversation, filling in any blank spots the audience might have on the spot, and guiding the way in the jungle.

That's why all texts are kind of bad. Either they are too narrow to give a wide perspective, or too huge to be absorbed!

As one of my advisors said: mathematics, like food, is best shared. Don't go into it alone; and whether you have or don't have a mentor, try to find someone else to join you on your journey (a friend who wants to learn the same subject).

> What makes LADR superior to this book?

Axler's approach for determinants.

Are you serious?

Axler defines determinant as (up to a sign) the constant term of the characteristic polynomial, and he needs two different definitions for characteristic polynomial, one over R and one over C. Now what if the ground field is something else? Do we need yet another definition of characteristic polynomial in order to define the determinant? What if you are doing linear algebra over a commutative ring?

LADR actually presents a very narrow view about linear algebra : it treats linear algebra merely as finite-dimensional functional analysis. The readers can be hit hard when they need to do other (computational or theoretical) stuffs. Similar concerns had been voiced on the internet before. In particular, I think Darij Grinberg's comments (below the answer https://mathoverflow.net/a/16996) on LADR are rather spot on.

It's fine if you find LADR helpful. The book does have its merits (I like its clear and fluent writing and its neat proofs), but it has also its own shares of problems and there are other nice choices of books in the wild.

The secret is that math is too big to fit into any one book. It takes many books with many perspectives you learn the generality and applicability of the topic. The average person who couldn't learn a topic read one book on the topic. The average mathematician read 5 or 10.

Good point. Even in maths, "bias" is inevitable. Every mathematician has a point of view. There's no objectively best curriculum or way of doing things, understanding things, solving problems etc.

(I'm more familiar with this phenomenon in philosophy, where the greater the philosopher, the more they have entirely their own way of looking at things, untranslatable into another tongue, which you just have to come to understand on its own terms. A summary of their views leaves out the personal aspect, the style, the way of thinking, and will seem dead.)

Thanks. I found LADR not that helpful - I use linear algebra constantly but only on a very elementary level. After reading your message I now feel it's ok not to find that book applicable to my pursuits and that I'm not missing some concept I failed to grasp from the book.

The determinant story of "distortion of unit volume" only makes sense AFAICT if you're in R^n. What does the determinant mean when you're considering linear functions from 2^n to 3^n? There's no geometric interpretation I'm aware of for this case; indeed, I doubt there is one, due to the inherent nature of finite fields.

A first-time student in linear algebra definitely does not need to hear anything about finite fields, though. I don't think anyone outside of certain mathematics subfields needs to.

Geometric interpretations have a place in learning: until one is ready to understand another interpretation that is more complete, better, yet more complicated.

You don't start kids with complex numbers until they can handle reals. You don't even start negative numbers until they can handle positive numbers.

Given a vector space V, form its top wedge power. This is one-dimensional. And endomorphism of V induces an endomorphism of the top wedge power. But an endomorphism of a 1-dimensional vector space is a number. This number is the determinant.

This is probably the correct generalization of the volume definition.

> linear functions from 2^n to 3^n

Say what?

2^n is {0, 1}^n, the space of n-tuples with elements from the smallest field. The field is also known as GF(2), Z/2Z, or F₂.

People who are going to be users of applied linear algebra would probably benefit from a traditional treatment of determinants. Axler's book is excellent for people who plan to go for more pure mathematics in their studies.

I would not wish the traditional treatment of determinants on anyone.

>benefit from a traditional treatment of determinants

The only beneficial traditional treatment I can think of is saying "determinant is volume", and that's what Axler does.

From there, one can look into alternating forms, convince oneself that an alternating n-linear form does the same thing, and obtain a formula for it (e.g. summing up signed products of numbers on the diagonals over all permutations of columns).

If by "traditional" you mean: "Here's an insanely complicated formula that does something magical. Learn to compute it. On page 5, we'll prove that it tells something about independence. Oh, and we'll mention volume on page 10 briefly" -- then I not only think this is not useful, I think it's outright harmful.

Why can't a free open source book be equivalent to or better than a more traditional book? First of all the release cycle of the free book should be much faster. Edits and alterations can be incorporated much more rapidly. Giving publishers such control over knowledge can be harmful. Being free does not have to sacrifice quality (this book is written by a mathematician anyway).

Also I recall reading Axler, and I think it covered some more advanced content towards the end. But this book looks equally solid.

Open source textbooks could theoretically be better than paid but generally speaking the big problem is that there usually isn't enough people to actually "care" for the book so to speak. Eventually the original writers lose interest and move on. For other open source books...they end up only becoming half written....and then turn into abandonware....see wikibooks.

You could solve the problem with something like a wiki but I've seen that usually results in quality problems. It isn't like a wiki article where all the results can basically see what has been written. With a wikibook though...you need all writers to basically know where stuff has already been explained...otherwise you just get a bunch of people explaining x here and then other people explaining the same thing in a later chapter.

The videos are free! They are a pretty good introduction by themselves.

Linear Algebra Abridged, a free compactified version of Linear Algebra Done Right, 2016. http://linear.axler.net/LinearAbridged.html

Linear Algebra Done Right videos, free videos to accompany the book, 2017. http://linear.axler.net/LADRvideos.html

Axler's Linear Algebra Done Right has always been my favorite linear algebra text (I say this 10+ years after reading it and finishing my math PhD, if that matters).

For those who want a free alternative, behold: Treil's Linear Algebra Done Wrong[2]

The books is downloadable as a free PDF. The name is an answer to Axler's book (dry mathematician's humor), and offers an opposite approach (getting to determinants first).

While I agree with Axler and diagree with Treil, LADW offers way more examples and applications, and together LADR and LADW offer a complete, excellent course material.


As for the linked text: not a bad text, but I wouldn't pick it over LADR + LADW.

Here's why:

1. size: it's larger than LADR+LADW taken together. It's hard to see the forest behind the trees.

2. exposition: it follows the structure of many other texts that I don't like because they terribly confuse the students (that I'd have to re-teach afterwards): starting with solving systems of linear equations, then jumping into vector spaces, for example.

3. I don't like how key concepts (matrix product, determinant are introduced). If you already know the material, it will be hard to see what's wrong with the approach of throwing a definition at the reader, and then talking about why that definition was made. But the opposite should be the case.

After teaching Linear Algebra, here's my litmus test for a good book. At a glance, it should make the following clear first and foremost:

1. A matrix of a linear map F is simply writing down the image of the standard basis F(e_1), F(e_2), ... F(e_n). These vectors are the columns of the matrix. If you know them, you can compute F(v) for any v by linearity. That's called "multiplying a vector by matrix"; we write Mv = F(v).

2. The product of matrices is simply the matrix of composition of linear maps that they represent. The student can figure out what that matrix should be (or should be able to do so); here's how. If M is the matrix of F, and N is the matrix of G (where F and G are linear maps), then the first column of MN is F(G(e_1)) = M x (first column of N). Same for other columns. Ta-dah.

3. The determinant of v_1, .. v_n is simply the volume of the lopsided box formed by these vectors (mathematicians call the box "parallelepiped"). In particular, in a plane, the area of the triangle formed by vectors A and B is half the determinant. This are can have a minus sign; switching any pair of vectors flips the sign.

4. Eigenvectors and eigenvalues are fancy words that allow us to describe linear maps like this: "Stretch this picture along these directions by this much". Directions are eigenvectors, by how much - eigenvalues.


5. Rotation and scaling are linear maps. That's all any linear map does: rotates and stretches. Writing a map down in this way is called singular value decomposition.

6. Shears are linear maps that don't change the volume. Any box can be made rectangular by applying a bunch of shears to it. That's called Gaussian elimination or row reduction when you look at what happens to matrices (and apply scaling as the last step). This is also an explanation of why the determinant gives volume (if you define it as an alternating n-linear form).

That's the beginning of a solid understanding of the subject.

From my experience, LADR+LADW leave the student with an understanding of 1-4, and other texts, due to being organized badly, don't (even when they contain all the information in some order).


Books I recommend:

[1]LADR: http://linear.axler.net/

[2]LADW: https://www.math.brown.edu/~treil/papers/LADW/LADW.html

Thanks. Most everyone thinks the first book they read sucks and the second book they read is better, because they don't realize the right way to learn is to read two or more different combine perspectives and revisit the material.

> 6. Shears are linear maps that don't change the volume.

Just to nitpick, but this sentence might be read ambiguously by some people, who would understand it as the definition of shear. You may want to say "Shears are an example of ...".

Thanks; I'm past the edit time cut-off, but will keep it in mind for the next time I say something like that :)

Grabbing both, thanks!

This book saved my life during uni. I found the explanations to be clear and well thought out. Math textbooks are expensive and it’s a hard sell when you’re still an undergraduate and not sure what you will end up doing with your life. Furthermore, as I can attest, finding a good teacher is n even harder proposition. It’s very rare to have good teachers when you’re in university. Most lecturers get their jobs based on their research skills and not their teaching skills. In the absence of good teachers, textbooks taught me everything I know and this is one of them. I hear where you are coming from, but this book is an exception and is fantastic. There’s a reason people are still talking about it and using it.

Are there any books like this for probability?

First chapter of this book has a similar style and will get you started: http://mbmlbook.com

Applications are open for YC Summer 2019

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact