
No bullshit guide to linear algebra – v2 - ivan_ah
https://minireference.com/blog/no-bullshit-linear-algebra-v2-release/
======
tptacek
This book is neat, and I hate to sidetrack any thread on HN about linear
algebra, but rules are rules, even for people I like.

You're not supposed to solicit upvotes for submissions. But that's exactly how
this story on HN came to be: everyone who bought the book got an email
announcement that the v2 version was finished (congrats, btw!) and it
specifically asked people to upvote this particular submission.

Please don't do things like that. Just a few steps past that and you've got
Product Hunt and the lesser subreddits. It's a good piece of work; people
would have noticed and highlighted it on their own.

~~~
ivan_ah
Mea culpa. I won't be doing that in the future (I mean writing books and
posting them to HN once every two years, yes, but soliciting upvotes, no.)

------
n4r9
The phrases "no bullshit" and "linear algebra" in the same sentence
immediately brought to mind the essay "Down with Determinants!" by Sheldon
Axler[0]. Having skimmed through the extended preview it looks like your focus
is on the bare techniques and their applications rather than a playful
approach, and determinants are introduced somewhat out of the blue (just as
they are in almost any degree course). I realise of course that that's just a
preview and the real thing could be very different. But I wonder what you
think of the idea of postponing determinants until well after eigenvalues,
diagonalisation and so on, when their definition and utility might be more
appreciated?

[0] [http://www.axler.net/DwD.html](http://www.axler.net/DwD.html)

~~~
romwell
Disclaimer: I am very happy that books like this are written, and look forward
to never seeing another $120 doorstop being used as a text for the course
anywhere. What follows is criticism and a question which, if addressed, could
help those on the fence about this book.

\----------------------

Speaking of Axler, his Linear Algebra Done Right is still the best book on the
subject, in my opinion.

For one, it's lean. You can fit the softcover in a purse, and read it in the
bathroom. It may be scant on examples, but it's a book that can be read in its
entirety.

This book, at 500+ hundred pages, simply means nobody will really read it. No
student is going to be reading several dozen pages of hard math EVERY WEEK -
the exceptions to that are those who will do well with any other book too.

So we have a book here that, alas, is not meant to be read - in entirety, at
least.

Yes, these are two very different books. But then we also have Linear Algebra
Done Wrong.

A text by Serge Treil, it was written as a counter to Axler's book (and was
named in jest). It covers many of the same topics, and is available as a free
(!) PDF.

So, my question is - why this book? Aside from the title, what does it really
do in a new way?

There's not enough in the preview for me to conclude that this book did
anything Linear Algebra Done Wrong does not, so I ask those who read it (or
the author).

~~~
ivan_ah
Thanks for your question, which I will respond to in a second. First let me
explain how we got to 550 pages. I wanted to include a review of prerequisites
material from high school math, which burned 100pp right off the batt. Then
the main material (Ch2-Ch6) is just 220pp which is not bad (especially when
you take into account the book is a compact format 5.5" x 8.5"). All of this
was done in 2014, when I started to think about the applications of linear
algebra.

But then the feature creep starts! In the introduction of the book I make the
claim about how powerful LA techniques are, and how they serve as the
foundation of machine learning, and generally "scientific modelling." To
substantiate this claim, I decided to make a kick-ass applications chapter
that SHOWS how powerful linear algebra really is (show me, don't tell me). Two
years later, the result is a 270-page survey of pretty much everything I've
learned over the years from engineering, physics, and computer science that
used linear algebra.

> why this book? what does it really do in a new way?

The main point of this book is to teach all the LA material in an
approachable, informal narrative as opposed to formal the theorem-proof
approach used in most textbooks. Can serious mathematical topics be introduced
using informal language, yet without saying anything outright wrong? Can I
trick the reader into learning advanced topics like Fourier transforms and
quantum mechanics by making the connections to basic operations explicit? Can
I sneakily introduce spaced repetition by covering core ideas several times in
the book using different notation and in different contexts?

~~~
romwell
Thanks for the answer!

So, the book is really three books.

How do you get around actually proving things?

~~~
ivan_ah
The book proves _most_ things, just doesn't follow the standard math format of
theorem-followed-by-proof. All the definitions are reasonably clear, and most
of the proofs that are accessible and educational are given (e.g. the
equivalence statements about invertible matrices). The exercises and problems
also ask readers to complete several other interesting proofs (with solutions
in the back).

In a future pass I plan to add more proofs, wherever adding formal structure
doesn't interfere with the flow.

------
greggman
this might be the most amazing book ever and I might even buy it and I know
I'll get down votes because so many of you love books but....

part of me feels like , especially for a topic like this, I want webpages with
interactive diagrams. Not last century's info dissemination tech but this
century's info dissemination tech

I feel page like this

[https://pomax.github.io/bezierinfo/](https://pomax.github.io/bezierinfo/)

is far better online than any paper book on the same topic could ever be. that
feels like it's probably true for almost all math, physics, and computer
topics.

if I'm reading up on game ai i'd much rather have in page interactive examples
than static text on a page.

this isn't in anyway meant as a comment on the book above itself. it's more a
comment on tech/math/physics books in general

anyone else agree or am I alone in this way of thinking?

~~~
hackermailman
There exists a book already for Linear Algebra w/interactive diagrams (that
explodes my CPU)
[http://immersivemath.com/ila/index.html](http://immersivemath.com/ila/index.html)

I just want accompanying lectures that build intuition like these
[https://www.youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2x...](https://www.youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2xVFitgF8hE_ab)
or even the author standing in front of a blackboard discussing each chapter
in the book and posting the lectures on the book's page. Some author's have
done this like
[https://cs.wheaton.edu/~tvandrun/dmfp/](https://cs.wheaton.edu/~tvandrun/dmfp/)

If it must be 'immersive' I'd also rather have a self-contained interactive
book I can keep locally with signed errata updates instead of having to rely
on slow websites that may be abandoned by the author or shutdown for whatever
copyright reasons or ninja edited by somebody with misinformation.

Edit: or a book with an online tutor, like the one MIT made for SICP
[http://icampustutor.csail.mit.edu/6.001-public/](http://icampustutor.csail.mit.edu/6.001-public/)

~~~
nebabyte
'Wikipedia with intuitions' would beat out any single book trying to record
them. My hope is that they one day expand their scope to not only cover
knowledge but different ways of looking at it/understanding it.

Then for any given topic a given subject's 'lesson plan' could just follow a
'sequence' (a la LW) of excerpts reviewed for accuracy (in lieu of reading
such excerpts out of a book) along with mediawiki/etc animations/images/videos
that go along with those subjects.

I suppose it'd be a good idea to version those 'lesson plans' so later
courses/syllabuses can benefit from newly-researched knowledge _after_ it has
been peer reviewed and added to a later 'version' of the curriculum (i.e.
mitigating the 'quality control' issue of information shared online). And of
course lecturing as it traditionally exists can still thrive albeit with a
more freed focus for interactive learning (Q+A, practice problems in class, or
whatever transpires to be the best tool to use in dedicated class time/lecture
time)... It just seems like singleton books as knowledge containers are
obsolete next to the potential of QAing existing 'open source' knowledge
without the publisher/etc overhead.

~~~
jacobolus
> _My hope is that they one day expand their scope [...]_

The “they” who make Wikipedia are just volunteer contributors. Go make it
happen!

~~~
nebabyte
The "they" who decide the content classes which will be handled by the project
(enforced as rules of the wiki) are nonprofit organizations I'll never be on
the board of, so basically what cgriswald said.

~~~
jacobolus
Your original comment can be summarized as “I bet it would be great if a bunch
of other people did a few tens of thousands of hours of volunteer work to make
new tools for teaching math to me.”

Which as an idle speculative wish is fine as far as it goes, but has a
vanishingly tiny chance of happening in whatever way you are envisioning. (Of
course, various people around the world who are working on teaching math will
continue to do their best, but without paying any attention to your specific
desires.)

If instead you said “Wow, I have this great idea that would be worth a large
team of people spending a few tens of thousands of hours on. Does anyone have
an idea about what kind of prototyping, political organizing, or other work I
can do to start on a project like that so I can make my dream a reality? Is
there anyone I should talk to about this?” Etc. that would be a much more
interesting basis for further discussion, and might actually lead somewhere.

The volunteers (and the tiny handful of paid staff) in charge of Wikipedia are
certainly interested in new ideas for improving things. If you think it’s
impossible that they’ll ever accept an improvement, and you aren’t interested
in discussion, what’s the point in speculating about it?

------
pault
I've been trying to develop my maths skills beyond the high school level I was
last formally educated at. My problem is, that was 20 years ago, and I've
forgotten a lot of the fundamentals, but every time I try to slog through Khan
Academy I get bogged down in what would be many, many months of review to get
to the undergraduate level. Can anyone recommend materials that contain a high
level overview of those rules and concepts that are taught at the pre-calculus
level, so I can jog my memory without endless manual calculation? Linear
algebra and statistics are mainly what I'm interested in learning.

Edit: This book looks like a good candidate. :)

~~~
cmejo
Lang's _basic mathematics_

~~~
romwell
Anyone who recommends Lang as a learning resource should burn in their version
of hell.

~~~
cmejo
Really? I've used it before for others who found it useful. That's the thing
about math, lots of options/styles. Just look at Spivak vs Apostol for Calc.
Do you have a suggestion for him?

~~~
hackermailman
Lang's Algebra book before revisions contained famously difficult exercises
[https://mathoverflow.net/a/10899](https://mathoverflow.net/a/10899) but I
liked Basic Mathematics even if it's out of print (expensive) and full of
errata issues.

------
iamed2
This is the exact sort of book I've been looking for for a long time. I work
with quite a few physicists and engineers who make extensive use of linear
algebra I don't fully understand (my university courses only went up to
chapter 6.1's content), so I've been looking for some way to get up to speed.

------
88e282102ae2e5b
This is exactly like every other linear algebra book I've ever seen, other
than the title. I was hoping this had been improved over v1 but it seems to be
unchanged in any significant way. What exactly does the author think the
bullshit is that they're leaving out?

------
drakenot
I bought the first edition of this book and I enjoyed a lot.

Ivan, can you give those of us with the first edition a summary of the
changes?

~~~
ivan_ah
The "core material" is pretty much the same. It covers all the computational,
geometric, and theoretical aspects of linear algebra as required for a first-
year course.

The main difference is the applications. Instead of one chapter, I expanded
this part to three chapters that cover applications to chemistry, electronics,
economics, optimization, probability theory, and even quantum mechanics. All
of these sounds like fancy stuff, right? Actually they're simple applications
of basic ideas of vectors and matrix operations, so it's quite easy to pick up
once you learn LA.

BTW, if you bought the book the pre-release version through gumroad, you have
access to v2 as well. Check your email—I just sent out an update.

~~~
shmerl
Do you plan to add one on graphics?

~~~
ivan_ah
There is already a section on computer graphics[1] that covers basic
homogenous coordinates (instead of representing points as 3-vectors [x,y,z]
you represent them as 4-vectors [x,y,z,1], which allows you do do translations
and projections easily).

I explain the math, but I'm not expert enough to talk about practical graphics
topics like OpenGL / WebGL etc, so not sure how much further I can go. To
merit the "NO BULLSHIT" label, it's really important that the author know the
subject matter inside out ;)

[1]
[https://minireference.com/static/excerpts/noBSguide2LA_previ...](https://minireference.com/static/excerpts/noBSguide2LA_preview.pdf#page=109)

~~~
shmerl
Thanks! That looks good. I didn't mean any extensive chapter on OpenGL and the
like, just some examples of linear algebra commonly used in computer graphics.

------
rs86
As if other books on linear algebra have bullshit... This contributes to the
"math sucks" culture. Math is fun an easy to learn. It just takes effort, like
any other skill.

~~~
rsp1984
You may disagree but most books I read on math, or subjects that require math,
contain a sizable amount of BS. Not in the sense of containing things that are
wrong but in the sense of containing things that are there for the author to
show off rather than for the reader to comprehend the subject matter.

I read books to learn about something that I didn't know about before. So I as
a reader can't distinguish between the important stuff and the 'fluff'. The
author has to do that for me. The best texts do that rigorously but the bulk
does not, unfortunately.

------
dlwdlw
The essence of linear algebra series on youtube is great as well. It provides
a intuitive guide to what linear algebra is about in 2d and 3d space with
"arrow" vectors with great animations. Once an intuition is provided about
matrix transformations, determinants, basis changes and eigenvectors, then it
explains how abstraction of this theory can allow carry over of these insights
and techniques to other domains or other abstract vector spaces.

Too often learning starts at the abstracted level so intuition is lost in
exchange for memorized techniques. There is a flip side of intuition-first
techniques basically dumbing down everything into tweets but this video series
is great at not doing that IMO.

------
ajarmst
I own a physical copy of Savov's "No BS Guide to Math and Physics" and am very
pleased with it. I found it particularly useful as a refresher on things I
learned long ago, but no longer regularly use. The focus on practical
application is of particular value in a quick reference role. I've also
recommended it as a reference for students transitioning from a Technologist
program to formal studies in Engineering. I've ordered this latest work just
on the basis of that experience (and the recent specter of not remembering how
to use eigenvectors when it come up in an obscure graphics problem).

------
joeblau
Just bought this. This book looks awesome — I'm working on some image
processing this summer and I want to re-strengthen my Linear Algebra. It seems
to be foundational for everything going on these days.

------
dergachev
Congrats Ivan, this was long in the making!

~~~
ivan_ah
Yeah, it seemed like it would never end. Thx for letting me use the vertical
aspect ratio monitor... that helped a lot!

------
meliorism
Ivan's books are always in my reference library. He is the Silvanus P Thompson
of our modern day. Looking forward to reading Linear Algerbra and as always
learning something new. Jjcona

------
zellyn
"concise, conversational tone" I wonder which it is. :-p

~~~
lindig
"Check this shit out:" in Section 1.1, page 9 of
[https://minireference.com/static/excerpts/noBSguide2LA_previ...](https://minireference.com/static/excerpts/noBSguide2LA_preview.pdf)

~~~
zellyn
It's actually pretty nice. I was mostly just being silly. But I do think
concision is underrated, and some "conversational" tutorials drag on…
Mathematicians end up with terse notation on purpose.

------
snake117
I took a course in Computational Biology last semester and we covered all of a
typical one semester linear algebra course in half the semester. It moved so
fast that no one was really able to comprehend the material well enough, so I
never felt like I actually learned the material as much as I was exposed to it
and just understood its importance. I will definitely check this out as it may
possibly be what I need to finally understand linear algebra. Thanks for all
the hard work!

------
mtVessel
The sample doesn't really give me a sense of whether this book is right for
me. It contains a lot of material from the early chapters, but little of the
parts of that confused me in college.

My primary question is whether this book is good at conveying the reasoning
that promotes intuition around the more difficult (for me) topics, such as
vector spaces and the many theorems associated with the Fundamental Theorem of
LA?

If not, suggestions for other resources are very welcome!

------
BinaryIdiot
This looks fantastic! I'm a high school grad with a 2 year college degree but
in college I took only a single math course (I was required to repeat Algebra
2 because my college didn't accept my high school's transcript for whatever
reason) and I've felt behind from many of my peers not knowing enough "maths".

I need to brush up on my former knowledge and this look slike a great way to
learn more!

------
mianos
Ivan. in the refresher sections, after each section, for example, Complex
Numbers, there is a list of sections in the example PDF, is this expanded in
the real book or are they topics for further reading? I notice similar
sections in earlier subjects seem to be expanded.

~~~
ivan_ah
Yes, all sections in the real book are expanded. The idea for the preview was
to leave only the headings so that the table of contents will look the same.

~~~
mianos
Cool, picking up a copy now. I love the way it's written but unfortunately not
safe to read to my kids as a bedtime story.

~~~
mianos
Wow, on demand shipping is fast. I already have it.

------
rout39574
Just bought two copies, one for me and one for my son.

Nice textbook in Computer Modern just gets me all jazzed up. :)

------
atakank
Great book that will fill a great void in a topic so fundamental for moving on
to more advanced subjects.

------
Safety1stClyde
Please present one example of a linear algebra text which does contain
"bullshit".

~~~
ivan_ah
You're right. Of all the mainstream textbooks on subjects for first-year
science, linear algebra textbooks are the best of them all.

I guess the main "bullshit" I could reproach is the price tags, $60 -- $90,
but even that is nothing compared to the textbooks for first-year calculus and
mechanics that run $200+.

------
keithpeter
The extended previews and concept maps are a great idea.

------
SpaceRaccoon
Good job Ivan, planning to purchase!

------
chris_wot
Well done Ivan - I think I might purchase this :-)

------
graycat
(1) Can work with any algebraic _field_. The greatest interest in practice is
for the field the set of real numbers R. The set of complex numbers C can also
be important. Curiously, there are some applications, say, error correcting
codes, where the field is finite, e.g., the set of integers modulo a prime
number.

For learning, I recommend working with the complex numbers C for where they do
help, with very little extra effort, and let the real case be mostly a special
case.

(2) The main motivation for linear algebra is just solving systems of
simultaneous linear equations as in high school. The main solution technique
is just the elementary row operations used as in Gauss elimination -- can
learn that in three minutes and program it in 10.

(3) Matrix theory is central, even crucial, in linear algebra but really is
just some darned convenient notation that can motivate just from solving a
system of linear equations. So, for positive integers m and n, an _m x n
matrix_ is a rectangular array of numbers (real, complex, other, as you have
decided to work with) with m rows and n columns. So, it's just notation.

(4) The big deal about matrix theory is just that can multiply matrices. The
multiplication is defined just so that a system of linear equations can be
written as a product of (A) a matrix from the coefficients and (B)+ a matrix
with one column of the unknowns set equal to a matrix of one column for the
right hand side. Let's try it for some high school case of 2 equations in two
unknowns (roughly as in TeX, and underscore character starts a subscript):

    
    
         a_11 x_1 _ a_12 x_2 = b_1
    
         a_21 x_1 _ a_22 x_2 = b_2
    

So, the 2 unknowns are x_1 and x_2.

    
    
                           / a_11  a_12 \
         2 x 2 A =  |                        |
                           \ a_21  a_22 /
    
                          / x_1 \
         2 x 1 x = |  x_2 |
                          \ x_3 /
    

and similarly for 1 x 1 b. Then the matrix multiplication Ax is _defined_ just
so that we can write Ax = b and have that be equivalent to the 2 equations in
2 unknowns in the high school form above.

Commonly we write A = [a_ij] for i = 1, 2, ..., m and j = 1, 2, ..., n.

Now, we can do some arithmetic on the matrices, and that starts to get darned
useful: E.g., for a number p we can define the product pA = [p a_ij]. So we
multiply the matix A by the number (call it a _scalar_ if you want) p. And we
can add two matrices: So for m x n A = [a_ij] and m x b B = [b_ij] we can
define m x n C = A + B where C = [a_ij + b_ij]. So just add the corresponding
_components_ (elements, entries).

Then presto, bingo, we get to why this subject is called _linear algebra_ :
Given m x n A, n x 1 x, n x 1 y, and numbers p and q, we have

    
    
         A(px + qy) = (pAx) + (qAy)
    

which says that A is _linear_ or a _linear operator_ \-- in calculus, both
differentiation and integration are also linear operators. There is a huge
book _Linear Operators_ by Dunford and Schwartz, that is, _functional
analysis_ , that is where the _points_ we are talking about, _analyzing_ ,
actually functions. So, biggie special cases are Hilbert space and Banach
space. So, linearity in linear algebra is a biggie start on functional
analysis.

Why care about linearity? Because its (A) simple, (B) permits powerful
theorems fairly easy to prove, and (C) is an assumption that often holds quite
well in applications in science and engineering.

The next biggie deal about matrix multiplication is that it's associative:

    
    
         (AB)C = A(BC)
    

and that property drives a lot in duality, game theory, matrix computations,
error correcting coding theory, etc. Can write out a proof in terms of the i's
and j's and, there, will see that the crucial step is being able to
interchange the order of summation signs (which in calculus is the same as
interchanging the order of integral signs which in more advanced parts of
calculus is Fubini's theorem) when adding finitely many numbers but could also
know that the result is true because A, B, and C are functions, function
composition is essentially just matrix multiplication, and function
composition is essentially obviously associative!

Given m x n matrix A with m = n, then A is _square_. If a_ij = a_ji, that is,
if _reflect_ about the diagonal from upper left to lower right and get the
same thing, the A is _symmetric_.

Now a biggie, IMHO the most important result in linear algebra: Regard

~~~
graycat
Mod: Please delete this post and its second copy. Both were posted while in
draft and in error. I had to stop working on the post and just left the Web
page without ever clicking on Submit, and somehow two copies got posted.

Maybe in the future I will complete the writing and post, but what is there
now is incomplete and too rough. So, please delete.

------
graycat
(1) Can work with any algebraic _field_. The greatest interest in practice is
for the field the set of real numbers R. The set of complex numbers C can also
be important. Curiously, there are some applications, say, error correcting
codes, where the field is finite, e.g., the set of integers modulo a prime
number.

For learning, I recommend working with the complex numbers C for where they do
help, with very little extra effort, and let the real case be mostly a special
case.

(2) The main motivation for linear algebra is just solving systems of
simultaneous linear equations as in high school. The main solution technique
is just the elementary row operations used as in Gauss elimination -- can
learn that in three minutes and program it in 10.

(3) Matrix theory is central, even crucial, in linear algebra but really is
just some darned convenient notation that can motivate just from solving a
system of linear equations. So, for positive integers m and n, an _m x n
matrix_ is a rectangular array of numbers (real, complex, other, as you have
decided to work with) with m rows and n columns. So, it's just notation.

(4) The big deal about matrix theory is just that can multiply matrices. The
multiplication is defined just so that a system of linear equations can be
written as a product of (A) a matrix from the coefficients and (B)+ a matrix
with one column of the unknowns set equal to a matrix of one column for the
right hand side. Let's try it for some high school case of 2 equations in two
unknowns (roughly as in TeX, and underscore character starts a subscript):

    
    
         a_11 x_1 _ a_12 x_2 = b_1
    
         a_21 x_1 _ a_22 x_2 = b_2
    

So, the 2 unknowns are x_1 and x_2.

    
    
                           / a_11  a_12 \
         2 x 2 A =  |                        |
                           \ a_21  a_22 /
    
                          / x_1 \
         2 x 1 x = |  x_2 |
                          \ x_3 /
    

and similarly for 1 x 1 b. Then the matrix multiplication Ax is _defined_ just
so that we can write Ax = b and have that be equivalent to the 2 equations in
2 unknowns in the high school form above.

Commonly we write A = [a_ij] for i = 1, 2, ..., m and j = 1, 2, ..., n.

Now, we can do some arithmetic on the matrices, and that starts to get darned
useful: E.g., for a number p we can define the product pA = [p a_ij]. So we
multiply the matix A by the number (call it a _scalar_ if you want) p. And we
can add two matrices: So for m x n A = [a_ij] and m x b B = [b_ij] we can
define m x n C = A + B where C = [a_ij + b_ij]. So just add the corresponding
_components_ (elements, entries).

Then presto, bingo, we get to why this subject is called _linear algebra_ :
Given m x n A, n x 1 x, n x 1 y, and numbers p and q, we have

    
    
         A(px + qy) = (pAx) + (qAy)
    

which says that A is _linear_ or a _linear operator_ \-- in calculus, both
differentiation and integration are also linear operators. There is a huge
book _Linear Operators_ by Dunford and Schwartz, that is, _functional
analysis_ , that is where the _points_ we are talking about, _analyzing_ ,
actually functions. So, biggie special cases are Hilbert space and Banach
space. So, linearity in linear algebra is a biggie start on functional
analysis.

Why care about linearity? Because its (A) simple, (B) permits powerful
theorems fairly easy to prove, and (C) is an assumption that often holds quite
well in applications in science and engineering.

The next biggie deal about matrix multiplication is that it's associative:

