
Clojure Linear Algebra Refresher, Part 4: Linear Transformations - metalock
http://dragan.rocks/articles/17/Clojure-Linear-Algebra-Refresher-Linear-Transformations
======
btilly
One meta comment.

You've taken the usual approach of explaining matrix operations as an
unmotivated "this is what we do" and then tried to layer linear
transformations as a weird abstract thing which somehow relates. Most people
do this, and it makes sense for people who don't want to actually understand.

If you want people to actually understand, the relationship goes the other
way. Linear transformations are the thing we really care about, and matrices
happen to be the most convenient way to write them.

More specifically if T is a linear transformation from V to W, with bases
(v_1, ..., v_n) and (w_1, ..., w_n), then then we can write down M = (T(v_1)
T(v_2) ... T(v_n)). So far that's just a rectangular array of numbers,
uniquely defined for the linear transformation. Now if we have a vector v =
a_1 v_1 + a_2 v_2 + ... + a_n v_n then just tracing through linearity, we get
that T(v) is Mv with the regular matrix multiplication. Furthermore we can now
prove that any rectangular matrix defines a unique linear function with that
rule.

The result? When you have 2 vector spaces with a basis for each, each linear
function gives a unique matrix, and each matrix gives a unique linear
function. Matrices are just convenient ways of writing down linear
transformations.

OK, now if S is a linear transformation from W to U, and we have a basis for
U, then what about the function S o T defined by S o T(v) = S(T(v))? If you
work it out, it is a linear function, and its matrix is the usual matrix
multiplication. So matrix multiplication is the same as function composition
for linear functions.

Here is something fun. You know the associative law? That magically just works
after a ton of calculations for matrices? Well, here is why it works. For any
three linear transformations R, S, and T which can be composed, and any vector
v, by definition:

((R o S) o T)(v) = (R o S)(T(v)) = R(S(T(v))) (R o (S o T))(v) = R((S o T)(v))
= R(S(T(v)))

Which means that functions under composition are associative. Since matrix
multiplication is just a way of writing function composition, it must also be
associative. QED!

As you move through things like change of basis matrices, it is all a lot
easier to understand starting from the point of view of linear functions as
being fundamental, and matrices just being a way to represent it with a given
pair of bases.

~~~
dragandj
I'm not so sure about that. I intentionally direct the reader to the textbook
for real understanding. The aim of the article is to connect the dots from
that textbook to clojure code, as is stated at the beginning. I am not a
mathematician, nor I think a blog post is enough on its own to teach someone
linear algebra.

~~~
btilly
It depends on your goal.

If your goal is real understanding, the approach that I stated is the right
one. If you've learned it the other way and wish to actually understand it,
you'll still have to learn the approach that I stated.

Your goal is explicitly not real understanding. And most textbooks do it in
the order that you did. (Partly because professors think that most students
aren't going to try to understand, so there is no point in giving
explanation.) So you probably made the right choice.

But I would still like to see even a passing mention that matrices of just a
representation of a linear function given two bases, and matrix multiplication
is function composition. That could set a lightbulb on for someone struggling
through a problem.

~~~
dragandj
I do not know whether you've read part 3 of this series. It explicitly treats
matrix multiplication as function composition, and goes with this at length.

As for the rest, you might be right or not; but I am not competent to reform
world's math education. If you can write better linear algebra guides than the
standard textbooks, I'm all for it. Please inform me if you do that, I'd love
to read that. (BTW It's not sarcasm - I'd really love to see linear algebra
explained better than in standard textbooks. So far all fancy tutorials turned
to be good only as entertainment).

~~~
btilly
I had not yet read it. Having just fixed it, it is missing the role of a
basis. And without it, you'll have trouble figuring out coordinate systems,
changing your basis, and so on. Which gives you no way to keep straight such
confusing things as the fact that if your coordinate system rotates one way,
your representation of things rotates the other. (Try it! Stand up, turn
clockwise and see the world spin counter-clockwise!)

Let me offer an abstract example. Consider the polynomials of degree at most
2. There is an obvious basis, namely 1, x, and x^2. A polynomial like x^5 - 3x
+ 2 can now easily be written in coordinates as (2, -3, 1).

However we have many other coordinate systems that might be convenient. For
example suppose that we're sampling data, and can measure p(0), p(1) and p(2).
How do we find what polynomial that is? Here is an easy way. We can easily
find the new coordinates for our basis vectors: 1 -> (1, 1, 1), x -> (0, 1,
2), x^2 -> (0, 1, 4). That means that we can write down the matrix
representing the identity transform (nothing happened), going from the basis
we have, to the new coordinate system:

    
    
        ( 1  1  1)
        ( 1  2  3)
        ( 1  2  4)
    

That's the change of basis matrix one way. Invert it.

    
    
        ( 2 -1  0)
        (-3  2 -1)
        ( 1 -2  1)
    

And now we can go the other way. The polynomial that we want from our sample
data will be (2-3x+x^2)p(0) + (-1+2x-2x^2)p(1) + (-x+x^2)p(2).

There are a lot of problems where linear algebra comes up that you can think
through more clearly if you think about things this way (complete with the
role of the basis!) than if it isn't fully digested.

As for a better book, well, I already recommended _Down With Determinants!_
:-)

~~~
joshstaiger
I'm not sure if you'll see this 2 days later, but I've been trying to make
heads or tails of your example and I think there’s a mistake in there
somewhere.

Maybe I'm hopelessly lost, but, for one, the inverse of

    
    
        ( 1  1  1)
        ( 1  2  3)
        ( 1  2  4)
    

is not:

    
    
        ( 2 -1  0)
        (-3  2 -1)
        ( 1 -2  1)
    

On the off chance you see this, any pointers?

~~~
btilly
I did not see it.

As I commented in email, I did it by hand while very tired and made multiple
mistakes.

------
dragandj
HN discussions on previous 3 parts:

[https://news.ycombinator.com/item?id=14480559](https://news.ycombinator.com/item?id=14480559)
[https://news.ycombinator.com/item?id=14497867](https://news.ycombinator.com/item?id=14497867)
[https://news.ycombinator.com/item?id=14549007](https://news.ycombinator.com/item?id=14549007)

