

Introduction to Compressive Sensing - gballan
http://puzlet.com/m/b00cv

======
shas3
The question arises: what is the scope of compressive sensing? What you show
is l1 regularization. This has been around far longer than the term
'compressive sensing'. Even the theory underlying it pre-dates 'compressive
sensing' [1]. I think the 'compressive' 'sensing' aspect comes about due to
the possibility of literally, 'compressively sensing', not just in using
'l1-regularization'.

Further, an ambiguity arises in the concept of the interactive graphics in the
OP website: the graphical explanation shown holds not just for 'l1'
regularization, but also for say, 'l2' regularization (and other
regularizations). The 'bar' could also represent a scaled version of the l2
norm. It is not easily apparent why it should only be 'l1'.

When it comes to graphical intuition for compressive sensing, I find the
'l2-ball' (intersects 'low-dimensional measurements' at dense points ) vs.
'l1-ball' (intersects 'low-dimensional measurement' equations on sparse points
in the plane, thus 'promoting sparsity', see for example, [2]) to be more
lucid.

Since you already have the excellent interactive web-site, can you possibly
include the l1-ball vs. l2-ball [2] graphic?

[1] R. Tibshirani, "The Lasso page "Regression shrinkage and selection via the
lasso". Journal of the Royal Statistical Society, Series B 58 (1): 267–288.

[2] R. Baraniuk, "Compressive sensing." IEEE signal processing magazine 24.4
(2007).

Edit:

Another suggestion: The OP website format is a great way to demonstrate the
utility of regularizations in optimization, which has wide applications in
machine learning, data interpolation, etc.[3]

[3]
[http://www.mit.edu/~9.520/spring07/Classes/rlsslides.pdf](http://www.mit.edu/~9.520/spring07/Classes/rlsslides.pdf)

------
gballan
Thanks for your insightful comment. Our objective in this blab (= Web Lab) was
a simple view into CS for the non-specialist, and so we were wary of getting
in too deep with the various norms. And, frankly, a toy problem such as this
omits a lot of important details. However, representing the l2 norm on the bar
plot is a great idea. I'll look into that (BTW, you are free to edit, save &
share the blab yourself if you wish.).

For more detail you are welcome to visit "Intuitive Compressive Sensing"
[http://www.puzlet.com/m/b007z](http://www.puzlet.com/m/b007z) and
"Compressive Sensing Primer"
[http://www.puzlet.com/m/b0080](http://www.puzlet.com/m/b0080) (these aren't
blabs, and rely on server-side computation). In particular, there is an
interesting plot in the first link showing the difference in norms.

~~~
archgoon
This comment was meant to be a child of
[https://news.ycombinator.com/item?id=7513252](https://news.ycombinator.com/item?id=7513252)

------
tedsanders
Nice intro! If you're up for it, you should consider contributing to the
compressed sensing article on Wikipedia. It's a bit of a mess, in my opinion.

[https://en.wikipedia.org/wiki/Compressed_sensing](https://en.wikipedia.org/wiki/Compressed_sensing)

Side note to everyone on Hacker News: Contributing to Wikipedia is a great way
to spread ideas because Wikipedia is a searchable platform that everyone looks
at. An informational article that you write for Wikipedia is likely to have
far more impact than if you write on your personal blog. So if your goal is to
educate, consider contributing!

~~~
gballan
We are big fans of Wikipedia, and so appreciate your comment. However, even
though I am straying from your (valid) point, I think it worth emphasizing
that not all types of knowledge are encyclopedic, and encyclopedias do not
suit every purpose. Blogs, for instance, can convey a casual interaction and
experience with a topic that would not be suitable for wikipedia.

Broadly our goal is to bring code and knowledge together in exciting ways. In
this case to attempt to bring an "Aha!" moment to a non-specialist audience.
In other cases, people may be served better to get involved with the code:
[http://puzlet.com/m/b00d3](http://puzlet.com/m/b00d3) . Anyway, those are the
kinds of areas we want to explore.

