

Singular Value Decomposition Tutorial - roundsquare
http://www.puffinwarellc.com/index.php/news-and-articles/articles/30-singular-value-decomposition-tutorial.html

======
ivankirigin
After I learned all the details in grad school, I found out how to implement
it in a real world setting:

    
    
      [U,S,V] = svd(X)

~~~
sprachspiel
Unless X has 17,770*480,189 elements like the netflix dataset. Then you can
use something like this: <http://sifter.org/~simon/journal/20061211.html>

The site is currently down, google chache link:
[http://209.85.129.132/search?q=cache:h4Ljyun3gUcJ:sifter.org...](http://209.85.129.132/search?q=cache:h4Ljyun3gUcJ:sifter.org/~simon/journal/20061211.html+simon+funk+try+this+at+home)

------
gane5h
The SVD connects the four fundamental subspaces of a linear system. I was
fortunate to receive this great insight from one of Strang's lectures. I
highly recommend you watch one of his videos online on this particular topic.

This post doesn't do full justice to the beauty of the SVD. Intuitively, you
are trying to compute a transformation that diagonalizes the covariance matrix
of the data. Computing the covariance has two problems: 1) this is a O(n^2)
operation and 2) can lead to big numerical errors for really small values in
the matrix.

By creative use of elementary matrix operations, the SVD gives you the
transformation on the original matrix. If you are interested in just the first
few singular vectors, certain math libraries also support an _economical_ mode
that does just this.

------
sweis
I messed around with analyzing US senate votes using a SVD several years ago:
<http://saweis.net/svd/>

~~~
roundsquare
Wow, this is really great. The data seems to to support the claim that
politics has gotten more divisive over the past 2 decades. (I get unreasonably
happy when data actually supports a broad claim). Very creative way to show
this.

Have you thought about going further back? I'd be curious to see this on
voting records as far back as possible, just to see what trends might have
happened.

------
mattrepl
For anyone that finds the posted site painful on the eyes, here's another
introduction to SVD that is all on one page by default and isn't crowded by
advertisements.

[http://www.igvita.com/2007/01/15/svd-recommendation-
system-i...](http://www.igvita.com/2007/01/15/svd-recommendation-system-in-
ruby/)

~~~
codexon
Or you could just use the print version.

[http://www.puffinwarellc.com/index.php/news-and-
articles/art...](http://www.puffinwarellc.com/index.php/news-and-
articles/articles/30-singular-value-decomposition-
tutorial.html?tmpl=component&print=1&layout=default&page=)

------
manvsmachine
Another good one: <http://www.ams.org/featurecolumn/archive/svd.html>

and its HN thread: <http://news.ycombinator.com/item?id=736618>

------
cf
Don't most people use NNMF instead these days?

~~~
Tichy
What is NNMF?

~~~
cf
Nonnegative matrix factorization. It was the advantages of keeping a sparse
matrix sparse, can be iteratively created and generally the bellkor team's
matrix factorization of choice.

------
pz
everyone loves to reference the SVD because the concepts are principled and
intuitive. but i've found that, as often as it is brought up in conversation,
its rarely used in practice. anyone here used SVD in a production setting?

~~~
KonaB
Rarely used in practice??? Dude, are you being facetious?? SVD is the
sledgehammer that cracks any problem you come up against...

~~~
pz
really? it doesn't scale and is not amenable to online updates. unless...
wait... are YOU being facetious?

~~~
KonaB
SVD solves most problems I need to solve. But then, I conjecture we are in
different fields.

