

Study HN: Linear Algebra 1, fortnight 2 - gruseom

A bunch of us are learning Linear Algebra together (previously at http://news.ycombinator.com/item?id=3060521) by working through one of Strang's textbooks. Here's a new post to kick off the next two weeks.  Will two weeks be enough to finish Chapter 1?<p>Most of us are squeezing this into limited spare time - a packing problem in its own right - and several are working through all the exercises, so our pace has proven slower than a typical course would go. At this point, though, we should at least declare 1.2 and 1.3 done. Less than that is just embarrassing. If you made it past 1.4 long ago, good for you; please post some comments to goad the rest of us forward. If you're not on 1.4 yet, get your ass in gear! Here are a couple observations to entice you.<p>1.4 is about matrix multiplication, which would seem to be about as mechanical and shallow a topic as you could get. I was surprised at how deep Strang was able to make it. (But be aware that we're not reading Standard Strang, we're reading Hippie Strang. That's ISBN 0030105676.) I thought that mastering the element-wise way of multiplying two matrices was all you needed to have it down. But Strang doesn't want you to think of it that way; he even impedes that view. Instead, 1.4 takes the earlier theme of the "row picture" and "column picture" and develops it further by teaching the matrix product AB as a combination of the columns of A or (alternatively) the rows of B. This is the book's bias in general, to de-emphasize formal representations and focus on the meaning of the constructs instead.<p>The other point concerns the exercises for 1.4. I put them off for a while because I thought they would be rote. They're not. They're designed to trigger insights that don't come automatically from reading the text, and turn out to be surprisingly satisfying. I found myself alternating between "aha!" and "doh!" when completing them. Yeah, that says something about the obtuseness of the student, but more about the skill of the teacher. Clearly a lot of love was put into crafting those exercises. So so far, I'd say we lucked out in picking this book.<p>p.s. If this sounds like fun, you should join us. All you have to do is learn. And it would still be pretty easy to catch up.
======
tptacek
So as a data point:

I found 1.2 remarkably painful. Here's why: my math sucks, and I didn't have
basic algebra internalized (I flunked algebra in high school and so while I
can reason my way through a system of equations, I couldn't do it instantly).

On top of that, I was _way_ overthinking the problems. The sense I got of the
1.2 problems is that they're largely refresher stuff; that is, when given a
system of 2 equations in 2 variables, they just want you to go ahead and solve
it, not graph it in row and column view.

I grabbed an algebra cheat sheet and breezed through it once I realized that.

1.3 went much faster for me; it was clear what the "new" material was, and
Gaussian elimination is straightforward. This was material I really just
wanted to go ahead and code up in Ruby (I wound up using a spreadsheet
instead).

Incidentally, I've found spreadsheets extremely useful for these early
exercises.

I'm hoping to be done with 1.4 tomorrow night.

By far the biggest issue I'm having is finding time to spend on this, so I'm
really only a couple (literally) hours into this so far.

~~~
gruseom
What exercises in 1.4 have stood out for you?

Here are some that evoked insight for me (or made me go "doh"): 3, 4, 5, 8,
10, 12, 13. I'll do some more tonight.

(#5 was particularly "doh".)

------
jpallen
I just had a look at the first thread and this looks like an excellent
project. I won't be following along since I did maths as an undergrad and am
now doing a phd, but I'd be happy to lend an experienced hand if anyone is
having trouble. I'm not sure what is the best format to offer my help, so
please reply if I can be of assistance in some way. I've done some tutoring of
similar subjects before.

~~~
gruseom
Cool. Want to be our TA? :)

I think the best format to offer help would just be to post comments here.
Explanations of subtleties, additional insights, suggestions for further
thinking, reminiscences of your own encounters with the material, or mentions
of how this stuff comes in handy in other contexts would all be of interest.
And it would help the discussion flow more. One lesson from the first thread
might be that we were holding back a bit too much.

------
aamar
After doing nearly all the exercises through 1.3 I did approximately 1/4
(randomly selected) through 1.5. I agree that the exercises generate good
insights not in the text, but 1/4 might still be a good amount for me to do,
given time constraints.

I just noticed that odd exercises helpfully have solutions in the back. So
moving forward, I'm planning to do all exercises marked recommended/important,
all numbered 1 mod 4, and a few others that seem helpful.

For the exercises which don't have solutions, it would be nice to have a place
where we can compare answers yet not post them to the public internet. Would
anyone else find that useful?

~~~
tptacek
Holy I didn't realize the odd answers were in the back. Best comment ever.
Thanks!

The gap between what I think I understand after reading the text and what I
find I really understand when I hit problem 1-2 in each section is immense, so
I'm going to stick with the problem sets.

You can just ROT13 the answers you don't want to make public.

~~~
gruseom
What the hell is wrong with you two? I thought the first thing _every_ student
did was check if there were answers in the back.

Just think of how much harder you made college for yourself.

Edit: to be serious, the solutions at the back (which I look forward to
flipping to after every other exercise, kind of like a rat getting a food
pellet in an experiment) are interesting in their own right. He plants little
clues in there, to hint at what was important or why he asked the question.

~~~
tptacek
I made college pretty easy for myself by not going there. :)

~~~
gruseom
Oops. :) Well, I forgot everything. Not sure which is better.

------
gruseom
Consider 1.4.13c:

    
    
      Find 2x2 matrices such that CD = -DC, not allowing the case CD = 0.
    

Of course one can just take a shotgun approach, i.e. come up with some
arbitrary possibilities and then manipulate their elements individually until
the equation holds. But what's more interesting is to view the matrices as
operations on columns or rows, come up with operations that have the desired
effect, then encode them as matrices to answer the question. What such ways
are there to look at this little problem?

~~~
nialo
One way is to view this as a geometry problem, 2d for convenience: a column
represents a point in the plane, and matrices are transformations that can be
applied to that point. Then the problem becomes, find a pair of
transformations such that the point ends up in two opposite locations if the
transformations are applied in different order.

a solution, under rot13: n ersyrpgvba naq ebgngvba jbexf, V guvax ol naq nobhg
nal natyr gung vfa'g avargl qrterrf be nal nkvf gung vfa'g gur k be l nkvf.

I'd be interested in an explanation for exactly why the exceptions to my
solution don't work, I don't really understand why they shouldn't.

I read this this morning and haven't been able to get the problem out of my
head since, glad this thread exists :)

~~~
gruseom
That's exactly the sort of interpretation I was looking for. I don't remember
how to represent geometrical transformations as matrices, but that's really
just a technical detail; one can understand your solution without it. That
being said, I still don't understand your solution. Are you saying that _any_
such pair of transformations as you describe has the desired anticommutative
property, except for the exceptions? I don't see either part of that - the
general claim or the exceptions.

My solution exploits the fact that in CD, C combines the rows of D, while in
DC, C combines the columns of D. Here's the rest:

Yrg P or gur vqragvgl zngevk rkprcg jvgu -1 nf vgf obggbz-evtug ryrzrag. Gura
PQ vf Q jvgu vgf obggbz ebj artngrq. Fvzvyneyl, QP vf Q jvgu vgf evtug pbyhza
artngrq naq -QP vf Q jvgu vgf yrsg pbyhza artngrq. Fb gb trg PQ = -QP, jr whfg
arrq gb svaq n Q fhpu gung artngvat vgf obggbz ebj unf gur fnzr rssrpg nf
artngvat vgf yrsg pbyhza. Sbe 2k2 zngevprf, guvf vf nal Q jvgu mreb va gur
gbc-yrsg naq obggbz-evtug cbfvgvbaf.

Rot13 don't work so good for specifying matrices!

~~~
nialo
I apparently entered my test cases wrong in lots of interesting ways. I think
I've actually found essentially the same solution you have, it turns out I had
the exceptions and the actual solution backwards:

n avargl qrterr ebgngvba naq n ersyrpgvba nobhg, nf sne nf V pna gryy nal,
yvar guebhtu gur bevtva frrzf gb tvir gur evtug erfhyg.

Are we sure this is the correct meaning for "A = -B", where A and B are
matrices? Some things I've read seem to be saying that that should be
equivalent to A _B = identity matrix, rather than A_ v = -1 _(B_ v), where v
is some vector.

~~~
gruseom
It's the correct meaning, as indicated by the solution on p. 430. You're
thinking not of -B but of the inverse of B, which is denoted B^-1 ("B to the
power of -1") and when multiplied by B gives I.

------
gruseom
It's been a few days. Anybody care to report where you are? I'm happy to say
I'm still doing all the exercises, not so happy to say I've only just finished
the ones for 1.4. I thought exercises 29, 52, and 54 were interesting, among
others.

Finding time continues to be an issue, but actually working through the
material is fine. Many of the exercises are rather fun, as they mostly have
some non-mechanical twist. Occasionally (as in 1.4.23) Strang's informality
goes a little too far and I have no idea what he's asking.

~~~
mechanical_fish
Trying to catch up; just starting the exercises to 1.4.

Keep going and I'll catch up someday!

~~~
gruseom
I'm going to be offline for a few days this week, and might not be able to
work on exercises. Will post when back. Keep going you guys!

~~~
gruseom
I'm back and will get back into the exercises and post as soon as I can.
What's others' status?

~~~
gruseom
I have found 1.5 harder to follow, but pretty interesting once you figure it
out. The heart of it is the proof (if you can call it that!) of A = LU on pp.
34-35. It is quite beautiful - just one simple idea. But I have a question.
From p. 35:

    
    
      Both sides of (7) end up equal to the same matrix U,
      and the steps to get there are all reversible.
    

What is the importance of "and the steps to get there are all reversible"? I
don't see how the argument uses this fact.

(On a practical note, I suppose a new thread is in order as we can no longer
post top-level comments on this one. If no one else creates one, I'll do so
after I've worked through the exercises in 1.5.)

~~~
gruseom
The answer to my question is that the proof derives an identity, so you need
to reverse the steps to get back from there.

------
warmfuzzykitten
I'm full up with taking the AI and ML courses from Stanford. Both courses
provide background material in linear algebra, so I guess I'll make do with
that for awhile. Two notes, though: The book is rather expensive (over
$100USD) and, notwithstanding the glowing comments here, has some pretty
scathing negative comments on Amazon. The question that comes to my mind is:
If a book is going to reach an internet audience, is it still appropriate to
charge classroom prices?

~~~
gruseom
No argument on the price; it amounts to assault. But the reviews on Amazon are
interesting. Most love it, some hate it. The first thing I noticed is that the
reviewers who love the book are the ones who take pleasure in the material.
There are two kinds of negative reviewer: students who found it confusing (I
suspect because it didn't hold their hand in the usual way - its focus is on
developing intuition, something no one who just-wants-to-get-this-over-with
could care less about) and mathematically more sophisticated readers who don't
like its informality and gushiness. In other words, the negative reviews
pretty much sum up what's good about the book. Of course, we're barely into
it. I'm assuming the rest is like the first.

Here's a litmus test for whether you would like it. This is how Strang
introduces Gaussian elimination:

 _The problem is to find the unknown values of u, v, and w, and we shall apply
Gaussian elimination. (Gauss is recognized as the greatest of all
mathematicians, but certainly not because of this invention, which probably
took him ten minutes. Ironically, it is the most frequently used of all the
ideas that bear his name.) The method starts by subtracting multiples of the
first equation from the other equations. The goal is to eliminate u from the
last two equations._

That colloquial aside is irrelevant and would drive readers who are gritting
their teeth to begin with crazy. I love it. It reminds me of my favorite old
math prof to whom Euler was a close personal friend. If you find such a style
delightful, this is the book for you.

Strang's goal in this book seems to be to get you as excited about the subject
as he is, and share how he really thinks about it, convention be damned.
That's why it's Hippie Strang!

~~~
julsonl
Based from your excerpt, that's basically the type of book I would want to
read. Reading a math book without having any background with what the heck I
was grinding numbers for would just bore me to tears. I might have an ebook on
that lying somewhere...

------
bdr
Sorry, kind of OT: I'm working through Arora & Barak's "Computation
Complexity" textbook and would love some company. Get in touch if you're
interested.

~~~
gruseom
Hope you find someone. There should be a startup for this!

------
roversoccer18
I took linear algebra a couple semesters ago in college, and have some
matlab/octave scripts that solve linear systems using gaussian elimination,
guass-jordan, etc. If you want I can email them to you guys if it would help
you guys.

------
untitledwiz
This is a great idea! I'm so happy people are getting together in virtual
classrooms! I would join but I'm currently doing linear algebra in college.
Hopefully, when I graduate I can join/start something similar on another topic
:)

