
An Intuitive Guide to Linear Algebra (2012) - PNWChris
https://betterexplained.com/articles/linear-algebra-guide/
======
id_ris
I recently finished going through MIT OCW's linear algebra class from Gilbert
Strang. Without the struggle of doing the assignments, reading the text, and
watching the lectures, I don't think I would have ever learned the content.
While content like this and that from 3blue1brown are commendable and useful,
it simply would not have lodged the ideas into my head.

Now that the ideas of things like vector spaces, norms, orthogonality, rank,
basis, etc are nearly second nature, the concepts are useful as I study other
branches of math which would feel impenetrable otherwise.

YMMV, and if you can learn from condensed materials go for it, but I might be
too dumb for it work lol. I think the real benefit accrues to the author who
had to work out how to teach these concepts to others.

~~~
dfabulich
> _While content like this and that from 3blue1brown are commendable and
> useful, it simply would not have lodged the ideas into my head._

I'm not sure I understand your point. Are you just saying that this blog post
isn't an adequate substitute for taking a course in linear algebra? (Of course
it isn't. But who said it was?)

~~~
abdullahkhalids
Unfortunately, for a lot of people, including undergraduates, the dopamine hit
they get from watching a video or passively reading a textbook makes them
believe that these are adequate substitutes for doing thousands of exercises.

In my university, undergraduates have admitted that they have done fewer than
50 questions throughout the entirety of my math course. Their grades obviously
reflect that, but they will do the same next semester.

~~~
mattkrause
Can I press you to explain what you think dopamine is/does?

I'm curious because I'm a neuroscientist who occasionally works on
dopaminergic stuff and "passively reading a textbook" is so far from the
canonical examples we use for dopamine activity, but the idea of
dopamine/dopamine 'hits' has taken on a life of its own that seems quite
different from the neurotransmitter.

~~~
abdullahkhalids
I am a physicist, so I will dare not make any technical neuroscience claims. I
meant "dopamine" in the causal sense of people feeling pleasure from passively
reading a book because they think it is useful work.

------
nearlynameless
While this explanation is certainly much clearer than what I remember of high
school maths, I still have a pretty tough time following the formula examples.

When I see A(x) = ax, I'm not entirely sure how to read it.

Is A meant to be a function that accepts x? If so, why is the equivalent
expression a * x? Is it supposed to be implied that function A also has some
hidden value "a" that is going to be multiplied by the supplied value? Is this
notation specific to multiplication, to this expression, or what?

Positing that something is 'intuitive' when it depends so much on additional
contextual knowledge seems ever so slightly disingenuous as best, and slightly
harmful at worst; it can make the reader feel as though they must be dumb for
not understanding this 'intuitive' material.

I do acknowledge that this is linear algebra, and if one doesn't have a really
solid grasp of notation of regular algebra it is likely to go over their
heads, but the practical explanations (such as the slope rise/run example) are
quite clear and relatively simple to follow; it follows that a simple
explanation of the notation might be helpful too.

~~~
kdtop
I agree that this was a bit confusing. Higher up in the article, it shows that
a linear function is one that doesn't change when scaled:

F(a * x) = a * F(x)

This is showing the relationship between two uses of the same function.

Then, further along, we find:

"So, what types of functions are actually linear? Plain-old scaling by a
constant, or functions that look like: F(x)=ax In our roof example, a=1/3"

I think in this second situation, F(x)=ax is not a relationship but rather a
DEFINITION of the function F(x).

In programming terms:

function F(x: real) : real;

begin

    
    
      Result := x * (1/3);
    

end;

------
vecter
This is ok but nothing is as intuitive as 3B1B's series on YouTube that has
been posted hundreds of times on HN [0].

Linear algebra is really about linear transformations of vector spaces, which
is not captured in this blog post.

[0]
[https://www.youtube.com/watch?v=fNk_zzaMoSs](https://www.youtube.com/watch?v=fNk_zzaMoSs)

~~~
nightcracker
> Linear algebra is really about linear transformations of vector spaces,
> which is not captured in this blog post.

I... disagree. Some of linear algebra is about that. And it's probably a good
way to view it that way when learning.

But some of my current work (coding theory) involves linear algebra over
finite fields. We use results from linear algebra, and interpret our problem
using matrices, but really at no point are we viewing what we're doing as
transforming a vector space, we're just solving equations with unknowns.

~~~
xscott
I think this is spot on. Depending on what you're doing, a matrix can be:

    
    
        - A linear transformation
        - A basis set of column vectors
        - A set of equations (rows) to be solved
           - (your example: parity equations for coding theory)
        - The covariance of elements in a vector space
        - The Hessian of a function for numerical optimization
        - The adjacency representation of a graph
        - Just a 2D image (compression algorithms)
        ... (I'm sure there are plenty of others)
    

For some of these, the matrix is really just a high dimensional number. You
(rarely?) never think of covariance in a Kalman filter as a linear transform,
but you still need to take its Eigen vectors if you want to draw ellipses.

~~~
obastani
The first three can reasonably be thought of as defining linear
transformations. For linear systems of equations A x = b, x is an unknown
vector in the input space that is mapped by A to b.

Both covariance matrices and Hessians are more naturally thought of as
tensors, not matrices (and therefore not linear transformations). That is,
they take in two vectors as input and produce a single real number as output.

As for graph adjacency matrix, this can actually be thought of as a linear
transformation on the vector space where the basis vectors correspond to nodes
in the graph. Linear combinations of these basis vectors correspond to
probability distributions over the graph (if properly normalized).

2D images... Yes, these cannot really be interpreted as linear
transformations. But I'd say these aren't really matrices in the mathematical
sense.

~~~
xscott
If you squint hard enough, you can see all of them as linear transformations
(even the 2D images :-).

I politely disagree about covariance and Hessians. I can squint and say that
the Hessian provides a change in gradient when multiplied by a delta vector.
Similarly for covariance... Or you could look at it as one half of the dot
product for a Bhattacharyya distance, which is just a product of three
matrices (row vector, square matrix, col vector). No need for tensors yet.

That is unless you decide to squint hard enough to see everything as tensors!
:-)

------
adamcharnock
I... I still really struggle with this. I'm a smart person, I've got a
bachelors of engineering, I've been a professional software developer for
around 14 years now, and I've built a house. But there is something about
degree-level maths and beyond that I find deeply unintuitive in a way that
software development isn't.

Through comments here I found 3blue1brown's (clearly much loved) videos. By
the third video I was shouting, "why for the love of god would we be doing
this"? Based on this reaction I suspect that the content neither has intrinsic
appeal to me, nor does it have obvious use in my work, projects, or life.

Pre-degree maths though, I love. My A-level maths really changed how I saw the
world, and I make use of it reasonably often (well, often enough to not forget
it).

I think I'm writing this here because most other commenters seem to really
grasp this subject, or feel that they grasp it better having seen these
videos. I'm honestly happy for you. However, if anyone is reading this who
doesn't feel like that, then know you're not alone :-)

~~~
hintymad
This is typical for many people. You love pre-college math because you have
intuitive understanding, while college-level maths offer a new level of
abstraction that you may not feel familiar with from the get-go.

It's perfectly okay not to learn linear algebra, by the way, especially when
you don't find any incentive to do so. Otherwise, you'll find linear algebra
to be one of the most intuitive tools to model so many problems.

If you do want to learn linear algebra or any other higher math, I'd strongly
recommend you focus on understanding concepts intuitively first, to the point
that you find many exercises in a text book straight forward. Watching
3blue1brown is a good start, but do move forward with deeper treatment. The
book I find very usual is David Lay's Linear Algebra and Its Applications:
[https://www.amazon.com/Linear-Algebra-Its-
Applications-5th/d...](https://www.amazon.com/Linear-Algebra-Its-
Applications-5th/dp/032198238X). Lay sets up a really intuitive geometric
framework to explain the intuition of linear transformation with sufficient
rigor.

~~~
trentnelson
Got any similar recommendations for college-level probability/stats stuff?

~~~
hintymad
No. I took only a few courses on probability and mathematical stats. The books
are A First Course in Probability and some some textbook on Mathematical
Stats. The courses were probably not advanced enough, as I found them
reasonably straightforward. I struggled a bit on what exactly is a random
variable, but once that was internalized, everything else followed. I heard
that advanced courses like random processes were really hard, but I'm not at
that level.

------
adamnemecek
If this interests you, you should check out the bivector community
[https://bivector.net/](https://bivector.net/).

Join the discord [https://discord.gg/vGY6pPk](https://discord.gg/vGY6pPk).

Check out a demo [https://observablehq.com/@enkimute/animated-
orbits](https://observablehq.com/@enkimute/animated-orbits)

Also at the end of February, there is geometric algebra event in Belgium.
[https://bivector.net/game2020.html](https://bivector.net/game2020.html) All
the big names in the field will be there.

------
diffeomorphism
This seems to be "what are matrices and how do you work with them" and not
linear algebra. I mean that can be useful sometimes but seems more like
something you would teach in a numerics course instead.

Actually, I think this way of explaining and motivating things (linear
map==matrix) will get really, really confusing once you try to understand
changes of bases or eigenvalue decomposition. A linear map is something that
takes vectors and spits out vectors while preserving the vector structure
(i.e. addition and scalar multiplication on the input give you addition and
scalar multiplication of the output).

------
dang
A thread from 2015:
[https://news.ycombinator.com/item?id=8920638](https://news.ycombinator.com/item?id=8920638)

Discussed at the time:
[https://news.ycombinator.com/item?id=4633662](https://news.ycombinator.com/item?id=4633662)

------
formalsystem
Viewing Linear Algebra as the study of linear operators instead of matrices
makes everything so much simpler.

Of course AB != BA

Composition makes sense

Inverse makes sense

This is the book that helped me get it
[http://linear.axler.net/](http://linear.axler.net/)

~~~
swiley
I really like "matrices are the coordinate form of linear transforms." In the
LA class I took the professor made a pretty big deal out of that, first by
defining "lexicographic matrix basis" so he could write out matrices as
vectors and then talking about mapping between the three different ideas.

There's still stuff he said that I'm unpacking today... that was a dense
class.

------
andrepd
Goes to show how different people have different tastes. I find this type of
exposition very confusing and very unenlightening. Give me a Landau-style
"minimalistic"/"focused" explanation any day. Not to mention, it tries so hard
to simplify things to a simple analogy (the spreadsheet thing) that it ends up
being plain misleading. In other words: "Make things as simple as then can be,
but no simpler."

------
sushisource
I love explainers like this, but it frankly makes me a little angry that the
vast majority of the math teachers I had in highschool and college taught in
the awful way described in the setup to the piece.

Why is that? Has anyone studied it, or is there even a solid anecdotal
explanation? The best one I can imagine is many of these professors simply
don't care much for teaching and are more focused on their research, which is
still infuriating but at least an explanation.

I ended up with an undergrad in applied math, though I'm a software engineer
now. I like math, but I feel like I never got to be all that great at it. I
suspect I would've enjoyed it more and achieved more with explanations like
these.

~~~
addicted
How much of this explainer however seems better to us precisely because of the
more comprehensive knowledge and understanding we already have?

For example the author uses the word function liberally in the explanation.
However, when studying functions in school in math it was super complicated
for me. It was only after I started programming, learning the programming
language meaning of function, and then when I was reintroduced to mathematical
functions through functional programming that I truly grasped mathematical
functions and all the stuff I was taught in school.

I’m not arguing that school level math does not need to be improved. I just
think we should be cautious that because an explanation that seems intuitive
to us now after having gained a complete introduction to all math concepts as
well as programming (esp on HN) may not necessarily be as intuitive when
students who are only exposed to a very small subset of mathematical concepts
encounter them.

~~~
jimhefferon
For sure there is an effect on Reddit and other places where someone will post
a question such as, "I'm having trouble with my Calc class, what is a good
book?" and people seriously answer _Calculus on Manifolds_.

Now, CoM is a classic, a real great book, but it is useful only to people who
have reached a certain level of mathematical maturity. That, presumably, is
not the questioner.

A version of this is that I also see people who write, "I didn't understand
this topic when I took the course but now years later, I see it is all
actually very easy."

(I call this Second Book Syndrome because I don't know of a common name for
it. I understand this to be what Zen people mean by the "Gateless Gate," that
after struggling with something at great length a person can come to see that
there is no real difficulty. But I've never heard anyone else apply that name
to this phenomenon and I'm not Zen trained so I'm not sure that is right
either.)

~~~
Swizec
Once you understand monads, you lose the ability to explain monads. Hence the
number of monads tutorials grows at an exponential rate as every new
understander tries to explain them and fails.

it's a fun problem in teaching

~~~
Koshkin
> _Once you understand... you lose the ability to explain_

Sorry, this does not make sense to me.

~~~
vo2maxer
Douglas Crockford on Monads:

In addition to its being good and useful, it’s also cursed and the curse of
the monad is that once you get the epiphany, once you understand - "oh that's
what it is" \- you lose the ability to explain it to anybody else.

[https://youtu.be/dkZFtimgAcM](https://youtu.be/dkZFtimgAcM)

------
RickS
Already this is so helpful, as someone with only a partial high school math
background. The visual "pouring" analogy alone made it worth the read.

------
photon_lines
This is a nice resource - I wrote one myself as well which is mostly based on
the series by 3Blue1Brown, as well as other resources which I found useful and
which used a visual approach to introducing linear algebra.

You can find my guide here:

[https://github.com/photonlines/Intuitive-Overview-of-
Linear-...](https://github.com/photonlines/Intuitive-Overview-of-Linear-
Algebra-
Fundamentals/blob/master/PDF/An%20Intuitive%20Overview%20of%20Linear%20Algebra%20Fundamentals.pdf)

------
the_watcher
I remember taking Advanced Algebra Honors in 10th grade. It was basically
Algebra II with a few (seemingly teacher-selected based on the experience of
students who had a different teacher) advanced topics thrown in. One of them
was matrices, and I was completely stumped by them. I now encounter them all
the time, and wish I'd been able to wrap my head around them when I was
younger.

~~~
marcuskaz
20+ years ago I combined learning JavaScript and Linear Algebra
[https://mkaz.blog/math/javascript-linear-algebra-
calculator/](https://mkaz.blog/math/javascript-linear-algebra-calculator/)

------
bobblywobbles
Thank you for writing this up, or reposting.

I agree that it is better to understand math, and computer science,
intuitively first. Learning the basics instead of learning how to think in
them forces memorization and is frankly in a time gone by.

If only I could've been taught this way when I was younger, then I'd actually
be any good at any advanced math.

------
alacombe
Speaking for Linear Algebra, I learnt more reading for a few hours the
appendix of "The Design of Rijndael: AES - The Advanced Encryption Standard"
than I did in 6 months of theoretical university teaching full of useless
technical terms and solutions in search of problems...

~~~
JadeNB
> solutions in search of problems...

This sounds like it was meant to be pejorative, but it's what (applied) linear
algebra, and applied mathematics more generally, _is_. Anyone can learn about
a certain mathematical topic upon realising it's the one relevant to the
problem they're facing—and learn it way more quickly, due to motivation and
focus, than they would in a general-purpose course on the topic; the art is in
recognising what mathematics is relevant, and you can't do that if you've
never heard of it before. Having a library of (conceptual, not cookbook)
solution hooks on which to hang your problems is how you get to be good at
using mathematics.

~~~
alacombe
Of course it was meant to be pejorative and that pretty much summarize my
experience through university. In this particular case, the teacher wasn't
giving a rat arse about his class and was merely there for the safe job,
pension, sprinkles by some "research".

Anyhow, I disagree with that top-down approach, which seems to be very...
European. I much prefer to follow a more logical path where the problem
preclude the introduction to the solution.

~~~
JadeNB
> Anyhow, I disagree with that top-down approach, which seems to be very...
> European. I much prefer to follow a more logical path where the problem
> preclude the introduction to the solution.

I agree that it's easier to _learn_ that way, but it's not much use when
solving real-world problems; you can learn the techniques relevant to the
solution once you know what they are, but, if you haven't met them before, you
won't recognise that they're the right ones.

------
ukj
The only way to test if you've 'understood' something is to apply it to real-
world problems.

If it works - then you understood it. If it doesn't - then you didn't.

"Understanding" without way of external verification seems no different to
dopamine-chasing.

Measuring on output and all that...

~~~
matteuan
The only way to test is if you talk with an expert and he says you have
understood. There are many things in linear algebra that you can use in
practice even when you didn't really understand them. This is the reason why
self-studying certain topics is very hard, you still need (good) teachers to
give you constant feedback.

~~~
ukj
That seems like a verbalistic notion of 'understanding'.

How do I falsify the expert's claims about my understanding?

------
PNWChris
OP Here:

I just wanted to give some context to how I found this page, and why I thought
it would be good to post.

I may be putting myself on the spot here: I never took a linear algebra course
in undergrad. It was a heavily encouraged option, of course, but I felt I
understood the basic rules enough to not really need formal study. I opted to
study other areas, partially motivated by a fear I wouldn't do well and would
hurt my GPA (my god was I vain, I feel I could do so much more studying full-
time with my current world-view).

As time has gone on, and ML and quantum computing have simply blown up since I
graduated in 2014, I quickly realized the magnitude of my mistake. I have
frantically self-studied for years to try to make up the gaps in my
mathematical understanding, and linear algebra has come up time and time
again. I can do the processes, but they never clicked, I had no intuition.

I want to help others in my position cut to the chase, and study the highest
yield, most intuition giving resources.

I actually developed the mental model shared in this guide on my own, and was
positively delighted to find to this while thinking over a comment I was
drafting on here. This page lays things out so clearly. The component steps
are intuitive and I can commit them to memory/recall what they mean without
needing to dig up my notes to self!

===

I find this page gives an excellent foundation, and goes great with these
resources:

* A web site which clearly shows how to do matrix multiplication in a way that's easy to recall, it makes the procedure like riding a bike:

[http://matrixmultiplication.xyz/](http://matrixmultiplication.xyz/)

 _Huge thanks to Jeremy Howard of fast.ai for mentioning this in one of his
lectures, this tool is how I finally got matrix multiplication to click_

* A paper named "An Introduction to Quantum Computing" (bear with me, it's superbly well written and very approachable):

[https://arxiv.org/abs/0708.0261](https://arxiv.org/abs/0708.0261)

Page 3 of that paper lays out matrix multiplication (e.g.: applying a
"transformation matrix" in the spatial parlance of 3blue1brown's videos) as a
traversal of a directed graph. A very useful understanding, and shows how
generalizable the tools of linear algebra really are in my opinion.

* The essence of linear algebra, by 3blue1brown (fantastic for a geometric/"transformation of space" view of linear alg):

[https://www.youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2x...](https://www.youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2xVFitgF8hE_ab)

------
sci_c0
[http://immersivemath.com/ila/index.html](http://immersivemath.com/ila/index.html)

An interactive Linear Algebra course to complement the article.

------
bgroat
I love betterexplained.

Does this guy have a patreon?

~~~
wodenokoto
He has a business that sells courses and books right on the homepage.

------
rayalez
Are there resources like this for Calculus/Prob/Stats?

------
dundercoder
Does something like this exist for differential equations?

~~~
sci_c0
May be this can help you:

[https://www.youtube.com/playlist?list=PLZHQObOWTQDNPOjrT6KVl...](https://www.youtube.com/playlist?list=PLZHQObOWTQDNPOjrT6KVlfJuKtYTftqH6)

This is 3Blue1Brown's video series on Differential Equations. Do support him
on Patreon if this really helps you.

