
Getting started with linear algebra - ghosthamlet
https://hadrienj.github.io/posts/Deep-Learning-Book-Series-Introduction/
======
myWindoonn
But [https://graphicallinearalgebra.net/](https://graphicallinearalgebra.net/)
is available for free and uses easier-to-read graphical syntax. Why prefer
this book instead?

~~~
Jtsummers
The original linked text is probably better for a linear algebra learner if
they want to apply it (especially if they want to apply it earlier rather than
later). Your linked text looks (I've read the first 5 parts) to be a good
introduction but at a higher level of abstraction (category theory and
abstract algebra). In fact, I think it may present a good motivating case for
understanding that level of higher abstraction (while it's ostensibly about
linear algebra, the ideas transfer with some adjustments to other algebras).

It could also use some polish. It's awkward to be starting a new "episode" and
find that it's two pages of off-topic discussion (presenting the case for the
blog, which would serve better in an introduction than in the middle of the
material).

------
geokon
Where is the text? Or is this an advertisement for a book?

Since this might get the right eyeballs here:

1\. Does anyone have a good intuitive explanation for the meaning of the
transpose operation? It seems to tie into a central symmetry between columns
and rows (or equations and variables) that I'm having trouble articulating for
myself

2\. Why does every linear algebra book have a chapter on norms? It's always so
incredibly boring and esoteric and I still haven't hit a situation where it's
strictly necessary. The 2-norm seem adequate for most situations and comes out
naturally from the least squares problem

PS: Strang is great. Meyer's "Matrix analysis and applied linear algebra" is
even better (but more advanced)

~~~
man-and-laptop
Two interpretations of transpose:

\- Let <-,-> denote an inner product. Then <Au, v> = <u, A^T v> is a
definition of the transpose operation. Proof: (Au)^T v = u^T A^T v = <u, A^T
v>

\- The transpose maps a vector v to a linear functional (a kind of function)
v^T, such that v^T (u) = <v, u>. Conversely, a linear functional of a vector
can be transposes to a vector. In fact, it's the latter (linear functionals to
vectors) that's more interesting. This fact generalises to Hilbert spaces as
the Riesz representation theorem, where the consequences can be pretty
amazing.

\- An arbitrary linear operator L: H -> K tranposes to a mapping between their
dual spaces L^* = K^* -> H^* by L*(f) = f o L.

\- The outer product produces rank-one matrices, and all rank-one matrices can
be expressed as outer products.

~~~
geokon
Oh yikes. So the rabbit holes goes quite deep

So the transpose is really ties to the whole idea of inner product
operators/function? (I'm not sure what the exact terminology is there)

So thinking out loud a bit about the first interpretation... The inner to me
represents an operator that maps vectors to a scalar that denotes the degree
of "sameness" between two vectors. And it's a whole family of operators that
all share certain properties. It just so happens that to write out this
operator in matrix notation (for the standard inner product) we need to
transpose/rotate the first vector which is yucky and why we prefer the bracket
notation

It's a cool insight but it just makes it seem like a mechanical operator that
flips a column into a row to make things work for one family of operators. I'm
thinking more like in the case of an orthonormal matrix/basis. We know the
transpose is it's inverse. It stems from the standard inner products of the
columns being equal to one which is I find kinda interesting b/c the inner
products suddenly become part of linear systems and not just a separate
measurement of a degree of sameness.. but the headache for me is that I can
picture an orthonormal basis, but I can't picture it's inverse! :)

PS: The orthonormal basis case and the transpose by extension further makes me
feel that only the standard inner product seems to really map to something
directly applicable in linear systems

PPS: The 3rd bullet is a bit beyond me. And the 4th is also very interesting
but I'm not sure how to expand on that to build an intuition haha. I'll need
to digest it further

------
vladsanchez
Hadrien's book is so engaging I couldn't stop reading.

It's superbly illustrated. It steadily and seamlessly progresses through each
topic that you don't realize you're actually learning 'deep learning'.

I'm sure this book will get lots of praise pretty soon.

Thanks for sharing it ghosthamlet. Thanks for writing it Hadrien.

------
neurotrace
Looks like some great information but you need to change the way you style the
links to the different chapters. The links look like ordinary text and I had
no idea that I could click them until I accidentally hovered over one.

~~~
hadrienj
Thanks! It was a style error. I just fixed it!

------
Brigadirk
Thank you for doing this. I'm currently working with deep learning but have
never had Linear Algebra in school. Resources like this one, and the
3Blue1Brown video series on YouTube are a really nice way to get started.

~~~
jackallis
3B1B is by far the best resource on LA anywhere in the Web. I might be
chastice for saying it but i will go as far to say that 3B1B is better than
MIT's LA.

~~~
harias
3B1B is definitely more useful for applied purposes, but the MIT LA introduces
you to the topic formally, and helps you understand research papers.

------
Koshkin
> _A tensor is a n-dimensional array with n >2_

This is just one example of why it is usually better to learn stuff from more
authoritative sources. (On the other hand, I can’t help but recommend this
amazing video:
[https://m.youtube.com/watch?v=CliW7kSxxWU.](https://m.youtube.com/watch?v=CliW7kSxxWU.))

------
isatty
Read Introduction to Linear Algebra by Gilbert Strang; should be all you need.
There are the MIT OCW lectures that you can use too.

~~~
thanatropism
Obligatory remark that "reading" a math textbook, and even moreso a basic one,
is an active task; you can get the intuitions from youtube videos of the
3Blue1Brown variety, but to really learn you need to follow the text closely,
put question marks on passages you're not 120% comfortable with, do the proofs
that are left to the reader and the recommended exercises. If you're super
comfortable with the material, you should do the more challenging exercises,
usually marked with asterisks.

Basically you need a pencil, a lot of scratch paper and a capable waste
basket; or a whiteboard all to yourself (but I get tired from being on my feet
for hours on end).

\----

E: My litmus test is the Gershgorin circle theorem. Every so often I check if
I can still prove it, and if I do I try to review the surrounding matrix. I
will literally vote against hiring anyone who can't at least handwave why
that's true.

~~~
WoodenChair
>My litmus test is the Gershgorin circle theorem. Every so often I check if I
can still prove it, and if I do I try to review the surrounding matrix. I will
literally vote against hiring anyone who can't at least handwave why that's
true.

Interesting comment with no context—i.e. not hire someone for what? I don't
know if I would ever not hire someone for not understanding a math problem
(unless the job was heavily math related in the sphere of the problem). I
would definitely not hire someone who communicates as ineffectively as you did
in this comment.

~~~
tjr
Reminds me of: [http://philip.greenspun.com/humor/mcdonalds-
joke](http://philip.greenspun.com/humor/mcdonalds-joke)

