
The Algorithms Behind Probabilistic Programming - williamsmj
http://blog.fastforwardlabs.com/2017/01/30/the-algorithms-behind-probabilistic-programming.html
======
sn6uv
Nit

> The distribution must be smooth (this method doesn’t work for discrete
> parameters), but it need not be analytically differentiable

'Smooth' and 'analytic' both have basic technical meanings in the context of
differentiability [1], [2].

Glossing over the assumptions in this way is quite misleading

[1]
[https://en.m.wikipedia.org/wiki/Smoothness](https://en.m.wikipedia.org/wiki/Smoothness)
[2]
[https://en.m.wikipedia.org/wiki/Analytic_function](https://en.m.wikipedia.org/wiki/Analytic_function)

~~~
mr_pink
_Analytically differentiable_ (being able to derive a formula for the
derivative) is not the same thing at all as _analytic functions_ of complex
analysis (which are infinitely differentiable, etc). If anything, the term
"analytic function" seems more of a misnomer to me, but "analytically
differentiable" is used in its proper sense in the article.

------
joe_the_user
Nice,

The algorithms the article define seem to justify seeing probabilistic
programming as an advance beyond twenty line programs that call 20K sized
libraries - basically, these algorithm inherently auto-tune a lot of higher
level operations so these can be less-leaky abstractions.

------
3pt14159
It's taking a long time to come out, but the probabilistic approach I think is
the most interesting is BayesDB:

[http://probcomp.csail.mit.edu/bayesdb/](http://probcomp.csail.mit.edu/bayesdb/)

Almost always what I want is closer to SQL-like questions that I later want to
manipulate with languages like Ruby or Python, rather than having to load all
the data into a giant array and then have to use probabilistic programming on
the whole damn thing.

------
adamnemecek
You should check out [http://webppl.org/](http://webppl.org/), you implement
your own probabilistic programming language in javascript.

------
riazrizvi
Interesting marketing. It seems to be an API for statistical functions and
data. Though the term Probabilistic Programming sugggests a new programming
paradigm, which it is not. If I'm not missing something, the term makes a dry
subject sound cool.

~~~
williamsmj
There's much more to it than you suggest, but if marketing bothers you then
check out the MIT Probabilistic Programming Project's talk at Strange Loop
2015:
[https://www.youtube.com/watch?v=-8QMqSWU76Q](https://www.youtube.com/watch?v=-8QMqSWU76Q).

Probabilistic programming is an area of huge academic activity, with deep
connections to statistical inference, compiler design and, yes, UX.

~~~
cscurmudgeon
Academia is not exempt from marketing. And not all marketing is bad. :)

Remember the origin of the term "dynamic programming"?

> "I spent the Fall quarter (of 1950) at RAND. My first task was to find a
> name for multistage decision processes. An interesting question is, Where
> did the name, dynamic programming, come from? The 1950s were not good years
> for mathematical research. We had a very interesting gentleman in Washington
> named Wilson. He was Secretary of Defense, and he actually had a
> pathological fear and hatred of the word research. I’m not using the term
> lightly; I’m using it precisely. His face would suffuse, he would turn red,
> and he would get violent if people used the term research in his presence.
> You can imagine how he felt, then, about the term mathematical. The RAND
> Corporation was employed by the Air Force, and the Air Force had Wilson as
> its boss, essentially. Hence, I felt I had to do something to shield Wilson
> and the Air Force from the fact that I was really doing mathematics inside
> the RAND Corporation. What title, what name, could I choose? In the first
> place I was interested in planning, in decision making, in thinking. But
> planning, is not a good word for various reasons. I decided therefore to use
> the word “programming”. I wanted to get across the idea that this was
> dynamic, this was multistage, this was time-varying. I thought, let's kill
> two birds with one stone. Let's take a word that has an absolutely precise
> meaning, namely dynamic, in the classical physical sense. It also has a very
> interesting property as an adjective, and that it's impossible to use the
> word dynamic in a pejorative sense. Try thinking of some combination that
> will possibly give it a pejorative meaning. It's impossible. Thus, I thought
> dynamic programming was a good name. It was something not even a Congressman
> could object to. So I used it as an umbrella for my activities."

[https://en.wikipedia.org/wiki/Dynamic_programming#History](https://en.wikipedia.org/wiki/Dynamic_programming#History)

------
davidw
Why is this flagged?! Cursory glance at the article looked like perfect HN
material.

~~~
laxatives
The article consists of a list of algorithms with almost no other information.
Its posted because it has a link to the OP's own blog.

~~~
platz
It was helpful to me, it put these algorithms in context and explained why
there are useful over other methods. Were you expecting a graduate level
course embedded in the post?

------
misiti3780
nice article - please change your font - it makes reading those blog post very
difficult

~~~
hmottestad
I also did not like the font, hard to say why. It definitely is not designed
with being legible in mind. Feels very unbalanced. I think the biggest issue
is with the distracting curves, and varying ways of drawing the various
curves.

Kerning is also messed up, the distance between the letters is too large in
comparison to the distance between the words.

~~~
Thirdegree
It feel like a normal font that squished each character into a slightly-too-
small box.

