
An Intuitive Derivation of Eigenvectors - dhruvp
https://dhruvp.netlify.com/2019/02/25/eigenvectors/
======
daeken
If you want to get an intuitive, visual understanding of linear algebra --
including eigenvectors/eigenvalues -- 3blue1brown's playlist on the subject is
just ... perfect.
[https://www.youtube.com/watch?v=fNk_zzaMoSs&list=PLZHQObOWTQ...](https://www.youtube.com/watch?v=fNk_zzaMoSs&list=PLZHQObOWTQDPD3MizzM2xVFitgF8hE_ab)

~~~
Koshkin
Problem with visual understanding of linear algebra is that it is no
understanding at all when it comes to linear spaces over the complex numbers
(which are an important tool in quantum mechanics, for example) or infinite-
dimensional spaces. Attempts (or the habit) to use the intuition gained when
being exposed to an elementary examples often lead to gross misunderstanding
and logical errors.

~~~
eigenloss
It seems like you're suggesting the added complexity of complex numbers makes
visual understanding unhelpful; do you care to explain why visualizing complex
numbers or multidimensional spaces is so impossible?

------
platform
really good explanation. I like it even better than 3blue1brown or
visualisations that I had seen.

It is better because it really covers every step of the construction process.

And offers explanation of why certain thing are not the right construction
blocks. The author gives a visual example, for example, of why basic vectors
1,0 -1,0 are bad. The article shows they cannot span the whole space.

Those kinds of explanations of 'bad constructions' are difficult to show in
visualizations, that show 'good' constructions only.

But, yet, in my view, these negative examples, are really helpful to explain
the material that otherwise, requires 'intuition'.

Not everybody has same intuition, so showing negative examples/impossible
constructions, and why those do not work -- is a good way tuning one's
intuition.

\---

On a separate note, I am wondering if such good step by step + counter
examples, knowledge presentation -- is a result of author studying at MIT, or
a natural trait (or both) ?

~~~
dhruvp
Hey!

Author here. Really appreciate the kind words. If you have any feedback on
what I can do to improve the explanation further, I’d love to hear it.

Also, if you’re interested, I’ve written some other posts on explaining
concepts in Math and ML following a similar approach:

1\. Brief History of CNN based Image Segmentation:
[https://blog.athelas.com/a-brief-history-of-cnns-in-image-
se...](https://blog.athelas.com/a-brief-history-of-cnns-in-image-segmentation-
from-r-cnn-to-mask-r-cnn-34ea83205de4)

2\. Understanding Baidu’s Deep Voice for voice synthesis:
[https://blog.athelas.com/paper-1-baidus-deep-
voice-675a32370...](https://blog.athelas.com/paper-1-baidus-deep-
voice-675a323705df)

3\. An intuitive explanation of matrices as linear maps:
[https://dhruvp.netlify.com/2018/12/31/matrices/](https://dhruvp.netlify.com/2018/12/31/matrices/)

~~~
throwawaymath
Your exposition is very good. If I could make one pedagogical recommendation,
it would be that you give a gentle introduction to linear dependence and
independence. You don't _need_ this to treat the subject in the geometric
fashion you're using (it's implicit in talking about calculating lines on the
Euclidean plane). But if you explain it well it's a very powerful didactic
mechanic for your audience, because then they can generalize the reason why
{(1, 0), (-1, 0)} is not a basis of R^2 to arbitrary dimensions _n_. It also
leads you into the neat result that any linearly independent subset _S_ of a
vector space _V_ whose cardinality matches the dimension of _V_ is a basis.

That being said, I know all too well what happens to good exposition when you
try to shove too much into it. So feel free to ignore this; you write well and
I think you accomplish the goal of presenting the material in an intuitive
way.

------
FabHK
One way to think of eigenvectors:

Your linear map A moves things around, and you aim to characterise the linear
map.

So, look for lines (through the origin) that are _not_ moved. Those are given
by eigenvectors. A point on that line might be moved closer to or further away
from the origin (depends on eigenvalue < or > 1), or even flipped to the other
side (if eigenvalue < 0), but the line as a whole is mapped to itself.

------
quickthrower2
Seriously consider buying a domain name for $8 or whatever. You can use it for
free with netlify hosting anyway, and then you are in control.

~~~
dhruvp
I know! I’ve just been a little lazy. I’ll get a domain later today.

~~~
eatonphil
I was confused why/impressed at netlify was writing about eigenvectors in the
first place. HN in particular only shows the top-level domain so I couldn't
tell it wasn't netlify themselves until clicking around your site. :)

------
lxe
I also really liked this explanation: [http://setosa.io/ev/eigenvectors-and-
eigenvalues/](http://setosa.io/ev/eigenvectors-and-eigenvalues/)

You can drag things around and change values -- if you're a visual learner, it
really helps grasp things like this.

------
jules
IMO eigenvectors are easiest to understand in connection with differential
equations, and that's also one of their most important applications. If you
plot the flow of the equation x' = Ax then the eigenvectors are visually
apparent.

[https://www.wolframalpha.com/input/?i=stream+plot+%7B-5x+%2B...](https://www.wolframalpha.com/input/?i=stream+plot+%7B-5x+%2B+3y,+3x+%2B+8y%7D)

The eigendirections are the directions where the solution moves in a straight
line.

Not all matrices have (real valued) eigenvectors:

[https://www.wolframalpha.com/input/?i=stream+plot+%7Bx+%2B+5...](https://www.wolframalpha.com/input/?i=stream+plot+%7Bx+%2B+5y,+-x+%2B+y%7D)

------
itissid
The difference between an exposition in text(like this one) and
videos(3blue1brown) is how people _prefer_ to build knowledge and more deeply
with how one learns. The quality of both of these is excellent. And one can
test what works best like explaining to oneself(or a rubber duck) after
reading/viewing the material and how one can recall things.

------
Koshkin
There is no shortage of intuitive explanations of various elementary concepts
in mathematics (and physics); my personal favorites are by E. Khutoyansky [1].
(Surely enough, there is a video on eigenvectors!)

[1]
[https://m.youtube.com/user/EugeneKhutoryansky](https://m.youtube.com/user/EugeneKhutoryansky)

------
rodionos
I liked the MIT lecture on eigenvectors:
[https://ocw.mit.edu/courses/mathematics/18-06sc-linear-
algeb...](https://ocw.mit.edu/courses/mathematics/18-06sc-linear-algebra-
fall-2011/least-squares-determinants-and-eigenvalues/eigenvalues-and-
eigenvectors/)

------
skywal_l
The thing I use to visualize an eigenvector is exactly that. A rotating
planet. The eigenvector of the rotation matrix being the axis of rotation.

It gets weird when thinking of 2D rotations though... Too complex for me!

~~~
man-and-laptop
The eigenvalues of a 2d rotation matrix are complex numbers. There's no point
in trying to visualise the corresponding eigenvectors. You'll need 4
dimensions for that. (The 4 dimensions come from the fact that the
eigenvectors of a 2d rotation matrix are elements of ℂ^2, which is
topologically equivalent to (ℝ^2)^2, which is equiv to ℝ^4).

Personally, my "intuition" is based on analogies with non-complex
eigenvectors, and experience solving eigenvector problems algebraically
without using pictures.

Also, your 3d rotation example actually has _three_ eigenvectors, two of which
are complex. You've only found the one that's real.

------
BeetleB
Good writeup! A tad bit disappointed as I was expecting some new insights, but
this is more or less standard material about eigenvectors you would get in a
typical university course.

------
billfruit
I really think some better name than eigenvectors (and eigenvalues, etc..)
should be popularized. I find them to be very obtuse and opaque terms.

~~~
mlevental
eigen in German means owned/proprietary. this has always been a useful
mnemonic for me.

------
wittedhaddock
this is awesome tyvm!

