
Visualizing matrix multiplication as a linear combination - signa11
http://eli.thegreenplace.net/2015/visualizing-matrix-multiplication-as-a-linear-combination/
======
ubasu
Here's how you can apply this interpretation of matrix multiplication:

Think of the columns of the matrix A as basis vectors of a coordinate system
represented in global coordinates, and the vector v as the components of a
vector in that coordinate system. Then the product A * v transforms the vector
v into the global coordinate system.

Carrying this forward, matrix multiplication A * B gives the combined
representation of two coordinate transformations.

~~~
adamtj
Another way to look at it: A matrix is a linear transformation, and
multiplying a vector by a matrix is how you apply the transformation. But
linear transformations are really just changes of basis. How do you change
your basis? You find the dot product of a vector with each new basis vector.
And that's exactly what matrix multiplication is. When you multiply your
column vector by a row in the matrix, you're finding the dot product, doing
the projection in your change of basis.

------
dmd
Hmm. As someone who has never understood/visualized/been taught matrix
multiplication in any other way than how it's shown on the linked page, could
someone explain the alternative? I.e., if this is new, what mental model do
you currently have? Presumably that one would be new and useful for me.

Edit: "Typically this visualization isn't taught" \-- that's what I"m asking.
What visualization _were_ you taught, if not this one? I can't think of any
other that makes any sense.

~~~
eliben
Typically, this visualization isn't taught, in my experience. What's taught is
the formula for computing each cell of the result matrix (cell i,j being the
dot product of row i of the first matrix with column j of the second). While
this is, of course correct, and also the most efficient way to compute the
multiplication manually, it's not always clear why the formula is correct.

Hopefully the visualization on the linked page makes this formula's origin
clear.

~~~
hdevalence
Really? Interesting. We spent some time discussing multiplication by block
decompositions in my 100-level linear algebra class -- is this really
uncommon?

------
ef4
If you want more, I can definitely recommend the video lectures from MIT's
linear algebra course by Prof Gilbert Strang:

[http://web.mit.edu/18.06/www/videos.shtml](http://web.mit.edu/18.06/www/videos.shtml)

~~~
ivan_ah
+1 for the Strang videos. He's an amazing lecturer. e.g. in Lecture 1 explains
very clearly matrix multiplication in the "column picture" and the "row
picture". More intuition than you can shake a stick at:
[http://ocw.mit.edu/courses/mathematics/18-06-linear-
algebra-...](http://ocw.mit.edu/courses/mathematics/18-06-linear-algebra-
spring-2010/video-lectures/lecture-1-the-geometry-of-linear-equations/)

This tutorial is also pretty good for geometrical intuition:
[http://www.cns.nyu.edu/~eero/NOTES/geomLinAlg.pdf](http://www.cns.nyu.edu/~eero/NOTES/geomLinAlg.pdf)

------
arithma
This is the way I "understood" linear algebra. I even encoded it here in a
small c++ header file when I used to think it wise to implement these things
yourself:

[https://github.com/arithma/mat44/blob/master/mat44.h](https://github.com/arithma/mat44/blob/master/mat44.h)

Used here:
[https://github.com/arithma/ios3dball](https://github.com/arithma/ios3dball)

~~~
eliben
> when I used to think it wise to implement these things yourself

It's a very good thing that you thought it wise at some point. There's nothing
like reinventing a few fundamental wheels for a better understanding of how
they work.

------
graycat
Sure, in the matrix product

AB = C,

C has the same number of rows as A and the same number of columns as B, and
each column j of C is from column j of B acting as coefficients in a linear
combination of all the columns of A.

Next: For the set of real numbers R, positive integers m and n, m by n real
matrix A, real numbers a and b, and n by 1 x and y, we have that

A(ax + by) = (aA)x + (bA)y

so that A is _linear_.

If we regard x and y as _vectors_ in the n-dimensional real vector space R^n,
then we have that A is a function

A: R^n --> R^m

and a _linear operator_ which can be good to know.

Then the subset of R^n

K = { x | x in R^n and Ax = 0 }

(where 0 is the m by 1 matrix of all zeros) is important to understand. E.g.,
K = [0] if and only if function A is 1-1. If m = n, then A has an inverse
A^(-1) if and only if A is 1-1.

With C = AB, the matrix product is the same as _function composition_ so that

Cx = (AB)x = A(Bx)

which also can be good to know and, of course, uses just the associative law
of matrix multiplication.

That matrix multiplication is associative is a biggie -- sometimes says some
big things, e.g., is the core of duality theory in linear programming.

And the situation is entirely similar for the complex numbers C in place of
the real numbers R.

------
pharke
I don't know why but it seems to make more sense to me if I think of the
multiplication in terms of rotating or transposing and moving the vector or
matrix so that the columns or rows line up with the ones they are to be
multiplied with, the coloured guides are entirely useless to me.

------
howling
There are 2 types of matrix multiplications which should not be confused with
each other.

The first type is a change of basis where the columns of the matrix are the
old basis represented in the new basis.

The second type is a linear function that takes in a vector and outputs
another vector.

------
peatfreak
What is the point of this blog post? Matrix multiplication is shown like this,
and in many other similar ways, in many introductory linear algebra textbooks.

------
benihana
This article was posted a few weeks ago, helped me understand matrix
operations I've been trying to wrap my head around for years:
[http://betterexplained.com/articles/linear-algebra-
guide/](http://betterexplained.com/articles/linear-algebra-guide/)

~~~
chris_wot
I'm actually studying linear algebra right now, and this site has given me
insights my text book (which is actually pretty good!) just hasn't. Thanks for
the link :-)

------
enupten
Yeah, but this is so boring and oh-so-run-of-the-mill.

(If the mafiosi could read the following paper before rushing to click down-
vote, then I'd be much obliged.
[http://www.math.rutgers.edu/~zeilberg/mamarimY/DM85.pdf](http://www.math.rutgers.edu/~zeilberg/mamarimY/DM85.pdf))

