
Introduction to Applied Linear Algebra: Vectors, Matrices, and Least Squares - yarapavan
http://vmls-book.stanford.edu/
======
muhneesh
I'm e-learning Linear Algebra right now to have a good math foundation for
Machine Learning.

I was a History and Sociology major in college - so I didn't take any math.

If you are like me, and working off an initial base of high school math, I
would recommend the following (all free):

Linear Algebra Foundations to Frontiers (UT Austin) Course:
[https://www.edx.org/course/linear-algebra-foundations-to-
fro...](https://www.edx.org/course/linear-algebra-foundations-to-frontiers)
Comments: This was a great starting place for me. Good interactive HW
exercises, very clear instruction and time-efficient.

Linear Algebra (MIT OpenCourseware) Course:
[https://ocw.mit.edu/courses/mathematics/18-06-linear-
algebra...](https://ocw.mit.edu/courses/mathematics/18-06-linear-algebra-
spring-2010/) Comments: This course is apparently the holy grail course for
Intro Linear Algebra. One of my colleagues, who did an MS in EE at MIT, said
Gilbert Strang was the best teacher he had. I started off with this but had to
rewind to the UT class because I didn't have some of the fundamentals (e.g.
how to calc a dot product). I'm personally 15% through this, but enjoying it.

Linear Algebra Review PDF (Stanford CS229) Link:
[http://cs229.stanford.edu/section/cs229-linalg.pdf](http://cs229.stanford.edu/section/cs229-linalg.pdf)
Comments: This is the set of Linear Algebra review materials they go over at
the beginning of Stanford's machine learning class (CS229). This is my
workback to know I'm tracking to the right set of knowledge, and thus far, the
courses have done a great job of doing so.

~~~
earthicus
> This course [Strang] is apparently the holy grail course for Intro Linear
> Algebra.

I haven't watched his lectures, but I TA'd a linear algebra course that used
his text book, and _strongly_ disliked his presentation. I've heard that's a
fairly common reaction actually - it's one of those love it or hate it books.
I'm bringing it up because if you (or someone else reading this) turn out to
be in the group that doesn't love it, you should not give up on loving linear
algebra! You are definitely still allowed to have a different 'holy grail
course'!

~~~
selimthegrim
Where’s the love for Lax?

~~~
earthicus
Page after page of mathematical insights and delights! I've never had the
opportunity to work through it systematically, but have frequently read
excerpts and have never been let down. I would expect nothing less from a
figure so great as Lax!

It's worth pointing out in the context of this discussion that the book is, by
the author's own design, _not_ an introduction to linear algebra. It is a
second course that Lax used to teach his advanced undergraduates and beginning
graduate students at the Courant Institute. For example, OP with a high school
math background will surely be very puzzled by page two, when a linear space
is defined as a field 'acting on' a group. Which is, i think, the 'right' way
of thinking about the algebraic structure, in the sense that it greatly
simplifies all the intricate moving parts of linear algebra. Anyhow, I second
your recommendation!

------
StefanKarpinski
This is a beautiful book and a great intro to the basics of linear algebra.
All the figures in the book are generated in Julia and there’s a companion
book with Julia code for computational examples:

[http://vmls-book.stanford.edu/vmls-julia-companion.pdf](http://vmls-
book.stanford.edu/vmls-julia-companion.pdf)

------
nicebill8
The 3B1B series on Linear Algebra is by far the most welcoming and informative
introduction to the topic I've ever seen:
[https://www.youtube.com/watch?v=fNk_zzaMoSs&list=PLZHQObOWTQ...](https://www.youtube.com/watch?v=fNk_zzaMoSs&list=PLZHQObOWTQDPD3MizzM2xVFitgF8hE_ab)

~~~
aidos
Any linear algebra post I come here to comment the same (if someone else
hasn’t done it already). Seriously, this series is absolutely wonderful.

------
flor1s
After reading this book (or during), take a look at the author's (Boyd) video
lectures on linear dynamical systems:
[https://see.stanford.edu/Course/EE263/](https://see.stanford.edu/Course/EE263/)

There is a lot of overlap between the book and the course.

~~~
chrispeel
Also take a look at Boyd and Vandenberghe's book on convex optimization:
[https://web.stanford.edu/~boyd/cvxbook/](https://web.stanford.edu/~boyd/cvxbook/)

------
squidgyhead
You might be interested in the Open Textbook Initiative by the American
Institute of Mathematics:

[https://aimath.org/textbooks/](https://aimath.org/textbooks/)

------
k__
Somehow I found linear algebra easier than calculus, but I don't know why.

I did both at the same time in university, but failed calculus 3 times and
aced linear algebra at the first try.

I'd expect being either good or bad at math, not both at the same time

~~~
impendia
Math professor here ---

Quality of teaching might have something to do with it.

But, also, calculus is much harder to understand at a rigorous, formal level
than at an informal level.

On one level you can try to understand what the main concepts are about, be
able to compute derivatives and integrals, solve optimization and related
rates problems, and so on. I'd recommend Silvanus Thompson's _Calculus Made
Easy_ over any mainstream calculus book for this. In my opinion, the book
succeeds amazingly at fulfilling the promise of its title.

But suppose you _really_ try to read any mainstream calculus book, and
understand everything. For example:

\- Why are limits defined the way they are (with epsilons and deltas)?

\- The book will probably touch lightly upon the Mean Value Theorem -- why is
this important? What's the point?

\- Why is the chain rule true? It reads dy/dx = (dy/du) (du/dx). Yay! This is
just cancelling fractions, right? Any "respectable" calculus book will
_insist_ that it's not, but most students will cheerfully ignore this, still
get correct answers to the homework problems, and sleep fine at night.

\- Consider the function e^x. How is it defined? The informal way is to say e
= 2.71828... and we define exponents "as usual". Most students are perfectly
happy with this. But does this _really_ make sense if x is irrational? Your
calculus book might bend over backwards to define everything properly (e^x is
the inverse to ln(x), which is defined as a definite integral), and it takes a
lot of work to appreciate why.

In my experience, these sorts of issues mostly don't pop up in linear algebra,
where the proofs tend to parallel the handwavy heuristics. I wonder if this
had anything to do with your experience?

~~~
dyukqu
> \- Why are limits defined the way they are (with epsilons and deltas)?

> \- The book will probably touch lightly upon the Mean Value Theorem -- why
> is this important? What's the point?

> \- Why is the chain rule true? It reads dy/dx = (dy/du) (du/dx). Yay! This
> is just cancelling fractions, right? Any "respectable" calculus book will
> insist that it's not, but most students will cheerfully ignore this, still
> get correct answers to the homework problems, and sleep fine at night.

Which "respectable" book(s) would you recommend for those who want to dive
into this details? Is Tom Apostol's _Mathematical Analysis_? good for learning
these kind of details? (They say this book is "respectable", but I would like
to hear your thoughts about it. Thanks).

~~~
impendia
I don't know anything about Apostol's _Mathematical Analysis_. My guess would
be that it demands a fairly sophisticated background of the reader, and does
an excellent job of covering calculus from an extremely rigorous point of
view.

I have heard that Apostol's _Calculus_ is an excellent choice, probably
somewhat more accessible to beginners, but still offering a rigorous, highbrow
perspective. I've also heard the same of Spivak. I'd probably opt for one or
both of these.

------
mipmap04
Linear algebra is probably my favorite part of math from a practicality
standpoint. I'm not in a math heavy field, but knowing how to use matrices to
solve optimization problems has been very helpful.

------
gxs
Sort of off topic - but I was a math major in college and I always had a
broad, casual categorization of the types of classes I took.

Linear Algebra - felt like it required you to be able to hold really long
trains of thought in your head

Probability - felt like you had to be clever

Analysis - felt like you just had to think critically and approach things from
all angles

I always preferred Algebra - felt like I was writing essays not doing math

------
VikingCoder
I'm going to complain about this every chance I get.

A 2D vector, we generally store as [x, y, 0]. What's the extra 0? The
homogeneous coordinate.

A 2D point, we generally store as [x, y, 1]. That extra 1 is the homogeneous
coordinate, and since it's there, it means "and apply translations!"

If I have a 2D transform, I put the translation component in the last row or
column, depending on if you pre-multiply or post-multiply (I can never
remember which).

When I transform a vector by that matrix, the 0 in the homogeneous coordinate
means translation doesn't apply.

Perfect!

But what if I have a 3D vector? Well... I end up with [x, y, z, 0], right?

Ugh.

If instead, we stored the homogeneous coordinate in the FIRST position, [0, x,
y] for 2D, and [0, x, y, z] for 3D, etc. then it's just a sparse vector! Set
the values you want to! [0] is the 0-vector in any number of dimensions!

[1] is the origin point in any number of dimensions!

Why did we put the homogeneous coordinate last in all our internal
representations? It was so dumb!

~~~
twtw
I don't follow. What do we gain by moving the homogeneous coordinate from last
position to first?

I don't understand this:

> then it's just a sparse vector! Set the variables you want to!

Or this:

> [1] is the origin point in any number of dimensions.

Could you clarify?

Also, I don't think this book even discusses homogeneous coordinates. It would
be sort of unusual for this type of general text and the only mention of
"homogeneous" in the index is "homogeneous equation."

~~~
edflsafoiewq
I think what's meant is that by putting it first, you can always treat any
point in P^n as a point in P^m by just ignoring the extra numbers if m < n or
by treating the missing numbers as all being 0 if m > n. That is the point in
P^2, [1 x y], can also be regarded as the point [1 x] in P^1, the point [1 x y
0] in P^3, the point [1 x y 0 0] in P^4, etc. This is in contrast to putting
it last, where if you have [x y 1] in P^2 and you want the point in P^1 you
need to allocate a new list [x 1], etc.

The vector is sparse in the sense that you can regard a point as being an
infinitely long list of numbers of which we are sparsely giving only that
prefix that is non-zero (like how you can regard a decimal numeral as being an
infinitely long list of digits, all the ones that are missing being 0).

[1] is the origin point in any dimension because it is [1] in P^0, [1 0] in
P^1, [1 0 0] in P^2, etc.

------
thrwmlaccnt
Is theoretical linear algebra at the level of axler helpful for machine
learning? If so, in what ways?

------
TobiasA
I'm a self taught programmer with a very weak maths background. What's the
best learning path for me if I want to be able to understand and create ML
based applications?

~~~
hackermailman
There's a practical course for this
[http://www.datasciencecourse.org/lectures/](http://www.datasciencecourse.org/lectures/)
anything you don't know, like linear algebra, look up the topics here for a
1-2hr crash course
[https://www.youtube.com/playlist?list=PLm3J0oaFux3aafQm568bl...](https://www.youtube.com/playlist?list=PLm3J0oaFux3aafQm568blS9blxtA_EWQv)

There's a playlist for a math background in ML for anybody who wants to try a
more rigorous ML course
[https://www.youtube.com/playlist?list=PL7y-1rk2cCsA339crwXMW...](https://www.youtube.com/playlist?list=PL7y-1rk2cCsA339crwXMWUaBRuLBvPBCg)
More information, including recommended texts
[https://canvas.cmu.edu/courses/603/assignments/syllabus](https://canvas.cmu.edu/courses/603/assignments/syllabus)
but don't let that list of prereqs discourage you, can easily look them up
directly. You don't have to understand all of Linear Algebra to do matrix
multiplication. There's plenty of ML books, papers and playlists on youtube
for a full course in ML from dozens of universities
[https://www.cs.cmu.edu/~roni/10601/](https://www.cs.cmu.edu/~roni/10601/)
(click on 2017 lectures)

Note never trust YouTube or any other resource to be around forever, make sure
you archive everything before you start taking it as lectures tend to
disappear (then seed them for others ^^ )

If you have a really weak background go through this free book, refuse to not
be able to complete it
[https://infinitedescent.xyz/](https://infinitedescent.xyz/)

There's no answers because the author gives thanks to a grad course in
evidenced based teaching where he claims the only way to really know something
and remember it is to figure it out for yourself. Math stackexchange can help
too.

~~~
notyourloops
> There's no answers because the author gives thanks to a grad course in
> evidenced based teaching where he claims the only way to really know
> something and remember it is to figure it out for yourself. Math
> stackexchange can help too.

This is a cop out; of course to really know something and remember you have to
figure it out for yourself. But answers allow you to check whether your work
was right, and if not, allow you the opportunity to debug your work.

My best performance came in organic chemistry, where I looked for question
banks (with answer keys) and solved problems extensively, perhaps bordering on
obsessively. If I hadn't an indicator that my final result was wrong, I would
have missed out on many learning opportunities, and objectively my performance
would have been worse. In general, I have found this strategy to enable me to
be an exceptional student.

If you don't benefit from an answer key, you're probably lazy and
undisciplined. Alternatively, you have too much time on your hands, opting to
rigorously confirm that each and every answer is correct.

In short, by not providing an answer key, you are denying the disciplined
student the opportunity to efficiently learn.

~~~
mindcrime
_In short, by not providing an answer key, you are denying the disciplined
student the opportunity to efficiently learn._

I agree with you 100%. But let me add this: in most cases, if you're studying
with a book that doesn't have an answer key, you can supplement that text with
exercises taken from somewhere else. For example, lots of course websites
around the 'net post previous years exams / homework with answers. There are
also books like _Schaum 's 3,000 Solved Problems in Calculus_[1], _The
Humongous Book of Calculus Problems_ [2], _3,000 Solved Problems in Linear
Algebra_ [3], etc.

Also, with books that are used as textbooks, and that provide an answer key
but only to instructors... if you aren't averse to violating copyright and
using certain pirate websites, those "instructor only" answer keys can often
be found.

[1]: [https://www.amazon.com/Schaums-Solved-Problems-Calculus-
Outl...](https://www.amazon.com/Schaums-Solved-Problems-Calculus-
Outlines/dp/0071635343/ref=sr_1_3)

[2]: [https://www.amazon.com/Humongous-Book-Calculus-Problems-
Book...](https://www.amazon.com/Humongous-Book-Calculus-Problems-
Books/dp/1592575129/ref=sr_1_4)

[3]: [https://www.amazon.com/000-Solved-Problems-Linear-
Algebra/dp...](https://www.amazon.com/000-Solved-Problems-Linear-
Algebra/dp/0070380236/ref=sr_1_3)

------
nyc111
> 2-vector (x1,x2) can represent a location or a displacement in 2-D...

Isn’t this fundamentally faulty? Same notation describing a point and
displacement. From this, we may conclude that, a point and a displacement are
the same thing because they are described by the same notation. Shouldn’t
mathematics be free of such contextual interpretation?

~~~
earthicus
There's no issue with the notation; I think you've misunderstood the
mathematical idea. Consider a more familiar algebraic object, a real number,
x. This can model a length, an area, volume, time, time interval, temperature,
weight, speed, physical constant, geometric ratio, fractional dimension,
etc...

In mathematics, we abstract by forgetting about what the things _are_ , and
retain information about how they _behave_ , and about what abstract
properties they satisfy. The insight is that 2d locations and 2d displacements
have the same abstract properties, which are modeled by a certain algebraic
object: 2-vectors.

~~~
nyc111
Thanks for the explanation. Makes sense. Something else I noticed, vector
notation does not specify a coordinate system. V = (1, 2) is just an array of
two numbers. The cartesian coordinate interpretation is a choice we make.
Correct?

~~~
earthicus
Yes, the keyword here is 'basis'. You represent a vector by giving two pieces
of data, (1) an ordered list of coordinates, and (2) a basis. The vector is
then a _linear combination_ of the basis elements, and the coordinates tell
you how to form that linear combination.

For example, let's use the standard Cartesian basis consisting of unit vectors
e1, e2, e3 (which point north, east, and up, informally speaking). If our
vector v is given by the coordinates (3,4,8) ( _with respect to the standard
basis_ ), then this means that v = 3 * e1 + 4 * e2 + 8 * e3.

If the coordinates were given with respect to a different set of basis
vectors, then you would take the linear combination using those vectors
instead. Note the similarity of how a basis works to how a base system works
representing numbers. Using base 10, the 'coordinates' of the number 348 mean
that 348 = 3 * 100 + 4 * 10 + 8 * 1. Using a different base, say base 9, they
would instead mean 348 = 3 * 81 + 4 * 9 + 8 * 1.

~~~
nyc111
> You represent a vector by giving two pieces of data, (1) an ordered list of
> coordinates, and (2) a basis.

Ok, I understand. But as used in computer languages, a vector can be simply 2
numbers. No coordinates or basis are implied. That's what I meant.

~~~
earthicus
Aha, yes. Computer languages borrowed the word 'vector', but they have
basically nothing to do with the mathematical structure from linear algebra.
It's best to keep them completely separate in your mind.

~~~
nyc111
So in math, when we say "vector" coordinate system is a given, as you
explained?

~~~
earthicus
If a coordinates are given, then they will be given with respect to a basis.
However, it's entirely possible to do things more abstractly without
introducing coordinates and bases to begin with, for example:

[https://en.wikipedia.org/wiki/Tensor_(intrinsic_definition)](https://en.wikipedia.org/wiki/Tensor_\(intrinsic_definition\))

------
mmmmpancakes
A book on applied linear algebra with a focus on regression and no mention
anywhere of the singular value decomposition??

~~~
IgniteTheSun
This book has a lot of very interesting applications and seems to cover
information not normally found in first books on Linear Algebra (e.g., makes
use of calculus, Taylor series, etc) and the authors are EEs, not
mathematicians. It doesn't, however, cover several topics normally covered in
the first year of linear algebra (e.g., vector spaces, subspaces, nullspace,
eigenvalues, singular values; see pp 461-462). As with most engineering books,
there are no solutions provided.

An excellent supplement to other Linear Algebra textbooks. Given its focus on
applications, will hold the interest of engineers and other technical folk but
may not be loved by mathematicians who may prefer a more rigorous approach.

------
syntaxing
Applied linear algebra is such a great idea. Linear algebra is relatively easy
to understand and used everywhere. But the material is so damn boring since
it's a lot of arthimetic. Even the homework problem is boring since there is
no specific purpose.

~~~
cultus
Typical LA courses in math departments have a bizarre focus on being able to
do Gaussian elimination by hand and stuff like that. It's not particularly
useful or even mathematically interesting. LA courses would be so much more
useful if they just stuck to theory and only had computer applications.

~~~
all2
I found knowing the mechanics of Gaussian elimination to be really helpful
when learning about algorithms like SVD or using Householder transformations.
Knowing _how_ the matrices became triangularized gave me something to hang the
new information on.

------
chobytes
Just finished my intro linear algebra class yesterday.

The class was a bit more abstract in nature, so some of the chapters in this
look like they could be nice application oriented follow ups to it!

------
herostratus101
How much overlap is there between this and Stanford's EE263?

Can someone in the know comment on the differences between what is covered?

~~~
vasili111
Comment below:
[https://news.ycombinator.com/item?id=18678713](https://news.ycombinator.com/item?id=18678713)

------
wpmoradi
Great Resource! But a good primer to Linear Algebra would be Gilbert Strangs
course at MIT OCW.

------
pylus
Ijust flipped some pages and saw it is great book. Wished my school offered
this.

------
graycat
I went though the slides. Super fun material! I've seen all the methods long
ago, and much deeper than in the slides, and published on some of the most
advanced material, and much more, but, still, it was fun material because of
the many examples and really good graphs.

From their other books, clearly they are real experts. The slides, then, are a
careful path where minimal theory gives a LOT of nice applications. The theory
they give is nearly always so simple that they are able, in just a few lines,
to give essentially the proofs, nearly always.

E.g., I never saw any mention of convexity, and these two guys are right at
the top of experts on theory and applications of convexity, so that it is
clear that they tried hard to get lots of applications from minimal theory.

They did next to nothing on numerical stability -- some mention might have
been good.

There's a still easier derivation of the least squares normal equations based
on perpendicular projections -- they might have included that. That is, if
drop a golf ball to the floor, the line to the floor and the shortest distance
to the floor is the line perpendicular to the floor. This fact generalizes.

They have illustrated a nice, general lesson: Can do such applications with
just finite dimensions and/or discreteness. Can do more theory with continuous
instead of discrete values and infinite instead of finite dimensions. But,
then, even with the extra theory, often challenging, commonly for the
computing are back to discreteness and finiteness. Sooooo, just omit the more
advanced theory and just stay discrete and finite throughout -- that's one of
the themes in the slides.

With this theme, the slides are able to do at least something interesting and
potentially valuable from stacks of texts in pure and applied math,
statistics, and more with just a few slides, simple math, nice graphs, and a
few words. Nice.

E.g., they did a lot of applied statistics without mentioning probability
theory! How'd they do that? They just stayed with the data and omitted
describing the probabilistic context from which the data was _samples_ or
_estimates_. Cute. But, readers, be warned -- the probabilistic context should
not be neglected; eventually should learn that, too.

Another cute omission of theory -- vector subspaces and the, really, the
axioms of a vector space. E.g., that "floor" I mentioned above is such a
subspace. How'd they do that? They just stayed with the basic example vector
spaces they had in mind and managed to avoid talking about subspaces.

At one point they touched on determinants for the 2 x 2 case, mentioned that
that result is important (should be remembered or some such), that there is a
more general approach that don't have to remember!!! Determinants have some
value here and there, e.g., show some continuity results right away and have
some nice connections with volumes, but they are tricky to explain and CAN be
omitted!!!

Uh, there is an easier proof of the Schwartz inequality based on Bessel's
inequality. Since they did enough with orthogonality to do Bessel's
inequality, they could have used that approach to the Schwartz inequality -- I
first saw in P. Halmos.

They didn't make clear the close connections among inner products, covariance,
and correlation -- maybe some readers will see those connections from what is
in the slides.

They did the QT decomposition -- nice -- that is, for square matrix A, we can
write A = QT where Q is orthogonal and T is triangular. They used that to
solve systems of linear equations but omitted Gauss elimination and the
associated approaches to numerical stability. For the Q, they emphasized the
Gram-Schmidt process but neglected to mention that it's numerically unstable
-- no wonder since are commonly subtracting large numbers whose difference is
small, the basic sin in numerical analysis.

Of course, the authors are EE profs. Then it is interesting that another theme
in the slides is getting close to much of the work in what computer science
calls _machine learning_. E.g., their few slides on using classification to
recognize digits 0-9 in handwriting is really cute, especially the graph that
shows the sizes of the coefficients on top of the square that has the input
data of hand written digits so that see which parts of the input data are the
most relevant to the calculation. Cute.

Of course, there's much more to those fields that they omitted than included,
but that's true also for even the best 5 star hotel luncheon buffet!!!

More fun stuff at

[https://news.ycombinator.com/item?id=18648999](https://news.ycombinator.com/item?id=18648999)

~~~
Koshkin
> _determinants... CAN be omitted_

Also see [http://www.axler.net/DwD.html](http://www.axler.net/DwD.html).

~~~
graycat
Looks like a nice paper!

The paper says how to go beyond what is in Boyd, _et al._ , i.e., eigenvalues,
eigenvectors, the spectral decomposition, etc. without determinants. Nice!

For that material I would have been tempted just to use the old approach of
determinants and the roots of the _characteristic polynomial_ , the Hamilton-
Cayley theoem, etc.

Saved the paper! Thx.

~~~
selimthegrim
And if you really really love determinants there's

[https://www.amazon.com/Discriminants-Resultants-
Multidimensi...](https://www.amazon.com/Discriminants-Resultants-
Multidimensional-Determinants-Birkhäuser-ebook/dp/B00F5QEH0I/)

(warning: not for undergraduates)

~~~
graycat
There is also the chapter in P. Halmos, _Finite Dimensional Vector Spaces_ on
_multi-linear algebra_ which at the time I read it I took it as an abstract
approach to determinants, maybe also a start on exterior algebra of
differential forms, but maybe there's a long shot chance that that Halmos
chapter is related to multi-dimensional determinants.

Can't read ALL the books on the shelves of the research libraries or even all
the recent ones so have to be selective, to _focus_ or as a startup
entrepreneur before spending hundreds of hours in such a book (hope the author
got tenure) ask "Why should I?".

~~~
selimthegrim
I am sure Gelfand, Kapranov and Zelevinsky given their other math
accomplishments all got tenure track positions when they emigrated. Will give
Halmos another look.

~~~
graycat
That can't still be THE Gel'fand, along with Kolmogorov, prof of E. Dynkin?
Must be a great grand son or some such.

~~~
selimthegrim
He passed recently, but yes it’s that one. The book is from 1994 and the
research is from just before USSR fell.

------
prince005
Proceed with caution..

------
aaaaaaaaaab
Prof. Boyd is a great teacher! I highly recommend his course on linear
dynamical systems [0] and convex optimization [1] too.

The former is beginner stuff, and while convex optimization is more advanced,
both are very engaging and clearly explained, with lots of anecdotes and
practical examples!

[0]
[https://see.stanford.edu/Course/EE263](https://see.stanford.edu/Course/EE263)

[1]
[https://www.youtube.com/playlist?list=PL3940DD956CDF0622](https://www.youtube.com/playlist?list=PL3940DD956CDF0622)

------
CamperBob2
Kind of inauspicious to see this kind of thing on page 5:

 _A (standard) unit vector is a vector with all elements equal to zero, except
one element which is equal to one_

Huh? I've never heard the term "standard vector" before, and a "unit vector"
is a vector whose _magnitude_ is one. There is no requirement that one element
be equal to one.

Maybe that's why the download is free...

~~~
RossBencina
> A (standard) unit vector...

I know this as "an element of the standard basis," B = {e_1, e_2, ...), where
e_1 = (1,0,0,...), e_2 = (0,1,0,0,...). You could view it as inauspicious that
the treatment doesn't begin with abstract vector spaces, but there is always
Axler.

For what it's worth, I find it inauspicious that after taking three (pure-math
oriented) Linear Algebra courses I never saw least squares nor the SVD. I'm
looking forward to taking a look at Prof Boyd's book.

~~~
CamperBob2
Point being, their definition is just plain wrong. If that's how the authors
describe a unit vector, I don't think this is the book you want to use to
learn about SVD.

~~~
Koshkin
Well, the way I see it, when you teach applied science you often want to
sacrifice some rigor so that your students could focus on what was intended to
be learned in the first place.

