
Geometry of the restricted Boltzmann machine - chromophore
http://arxiv.org/abs/0908.4425
======
jasonmorton
Just came across this discussion. I'm one of the authors, and if anyone is
interested, I'd be happy to answer any questions about the paper
(jason@math.stanford.edu). It originated in me hearing from Andrew Ng and
other folks in the ML community about how powerful deep belief nets were, and
wanting to quantify that power more explicitly. The first step, for
mathematicians, is to understand what a RBM can represent. As you point out,
the paper is written for a math journal and so emphasizes certain things. I
presented a poster from a more ML perspective at the ICML workshops. Cheers!

------
jey
More than six people on here are able to comprehend this? I'm not one of them.
Never cared much for "Algebraic Geometry" nor "Metric Geometry". An
introduction to the theory behind Boltzmann machines would be much more
relevant here, imo. Maybe something like this:
<http://learning.cs.toronto.edu/~hinton/absps/pdp7.pdf>

~~~
chromophore
I am reading this these days. I think you are right about this not being here.
Sorry. Will keep in mind.

~~~
jey
I don't blame you at all for submitting it. I'm just confused about who
upvoted it and why.

~~~
chromophore
LOL. The paper is very recent. And extremely impregnable for someone like me.
I was actually hoping for some comments on it by people who work on RBMs. Like
I got some good comments on a ML gallery I posted. Sometimes comments help
clear up things, and even a little clearing up is good IMHO.

I think they did because they didn't have a look at the abstract, but read it
partially by title only. And maybe you know, thought it was something
introductory on RBM's ? I mean RBMs are extremely pretty. And I am just
beginning to appreciate that.

~~~
slackenerny
There is seldom much interesting for ML person in this paper. (disclaimer: I
haven't made much sense of its goal either. This reminds me of a saying “Write
for lay audience, so maybe specialists will understand; Write for specialists,
then noone will”)

It's just Bern Strumfels running his research program, which is somewhat
obscure even for algebraic geometrers. He is an expert on algorithms of
algebraic geometry and their applications e.g. to optimization, and whatnot.
There was a thematic school on that, videos should be buried somewhere here,
<http://www.ima.umn.edu/2006-2007/>. This may be interesting line of
research—for a mathematician—or ML person willing to learn stacks of pure
mathematics leading to fluency with at least the book by Cox, Little and
O'Shea just to make sense of labels on buttons.

~~~
chromophore
Hmmm, Okay! Enough said. Why can't there be much interest? :)

My old professor (who is an authority with RBMs) linked me up and asked me to
see and make sure I know certain things about it.Let me give it some months
and report back to him. I basically work with SVMs, it's his suggestion to
study RBMs (which I know already) and sent me a number of papers. I am not
sure why he sent me this one (it's just a month old). This one to be frank
looked pretty nice to me, and I downloaded a number of other papers to study
it. Even if I give up. I'll be glad to learn some of the math involved. I'd
rather go with his opinion. Thanks.

The aim of my submission (as is the aim of most my submissions) was actually
to get some comments on it, and links to some resources. Not what a person
should learn and what not. I'll know better what to do and why. Given the wide
pool of people on HN, I was hoping to get something interesting, say some
comments ON the paper. Even one helpful comment would have done.

~~~
slackenerny
Sorry for not responding for a long time. I didn't mean to be snarky above,
sorry about that too.

I wrote a bit more here, but wasn't satisfied with how it went so its no more.
It was anyways mainly about Sturmfels' programme of reasoning about graphical
models with algebraic geometry and how I see it and less TO the paper, so it
was probably unsubstantial not only badly written.

By this time I'm sure you already know what program of algebraic statistics is
anyways and surely can judge it for yourself. This paper fits it and is not
directly consumable by machine learning just yet. Other parts of AS are. To
actually use either parts the book I mentioned is minimum anyways and will
clarify alot. Even if summary of the paper or AS itself is to be presented to
a prof, there is no way around this book. The shortest intro to AG is by M.
Reid, it freely builds on commutative algebra (short intro on J.S. Milne page
at jmilne.org) that builds on abstract algebra. I haven't read the new book by
Sturmfels and Sullivant but I'm half-sure it just assumes AG nor doesn't
explain statistics.

------
newsdog
Wow - so e e cummings and charlie the cockroach are now a professor.

Seriously - nice work.

