
Lecture Notes on Randomized Linear Algebra (2013) - iamaaditya
https://arxiv.org/abs/1608.04481v1
======
rajasinghe
This stuff is incredibly useful when dealing with large matrices. The idea is
that an n-by-n matrix often doesn't contain n^2 pieces of independent
information, but can be written a product of matrices of size at most n-by-r
(for r << n). A famous example of this is the Netflix recommendation matrix.
In this case, you can often avoid O(n^2) complexity by only dealing with such
low-rank approximations.

It should be noted that this overview dates from 2013 and that a lot of new
results have appeared since then. The author gives some good references in the
abstract.

~~~
sytelus
Is this a more formal treatment for algorithm that Simon Funk gave?

[http://sifter.org/~simon/journal/20061211.html](http://sifter.org/~simon/journal/20061211.html)

~~~
jey
That's more related to 'stochastic gradient descent' for 'matrix completion'.
The key difference is that Simon Funk's algorithm doesn't treat missing
entries as a zero, whereas using linear algebra based techniques on the
observed data matrix (formed by putting zeros for unobserved entries) would
try to predict the missing entries as zero exactly.

Also related is the 'alternating least squares' algorithm.

------
jey
Related but different:

Foundations of Data Science by Avrim Blum, John Hopcroft and Ravindran Kannan:
[https://www.cs.cornell.edu/jeh/book2016June9.pdf](https://www.cs.cornell.edu/jeh/book2016June9.pdf)

~~~
putin
Do you know what the prereqs are for high dimensional geometry? Any additional
resources? From the looks of it, the subject seems to require the knowledge of
some measure theory and functional analysis. Advanced undergrad/grad level
math subjects. Threshold for entry here seems very steep (at least for high-
dim geo).

~~~
ianai
I'd say it's about a minor in mathematics and one or two semesters of very
directed study.

------
shas3
This is a very good and timely compilation of all the important topics!

I ask this earnest question because I have a deep interest in randomized
linear algebra, random projections, 'sketching'/sampling, compressive sensing,
etc.:

Do any of you use it in industry applications? If so, at a high level, how do
you use it?

I know I'm asking a "I have a hammer and that is a nail"-type question, but I
am interested in seeing "deployable" applications of these topics. I don't
have any to report, other than academic ones.

~~~
sshekh
Compressive Sensing is widely used in imaging.

[https://en.wikipedia.org/wiki/Compressed_sensing#Application...](https://en.wikipedia.org/wiki/Compressed_sensing#Applications)

~~~
shas3
I'm reasonably well plugged into that community. All the applications there
are 'academic' applications. There are efforts to build hardware based on
this- startups, etc. However, the translation is still slow and nowhere near
as fast as what we saw with deep learning (just comparing apples and apples
with apples defined as step-change jumps in research).

~~~
lp251
How many startups are operating in the CS imaging area? I only know of InView.
Rambus likes to sell their lensless sensor as "compressive", but it doesn't
really fit.

~~~
shas3
There also LightOn by Igor Carron and co. LightOn.io

