
Design and Implementation of Probabilistic Programming Languages - adamnemecek
http://dippl.org/
======
adamnemecek
Is anyone here knowledgeable enough about all the Probabilistic Programming
languages and frameworks to give me a 'real world' run down? Like there's
quite a few of them and I'm not sure how they all perform IRL. I'm somewhat
familiar with PyMC, Stan, Figaro but I'm still not quite understanding all the
differences (are there any major ones even?)?

I know that Stan is like the most production ready one right now but it's also
somewhat large. This project seems much more tractable.

There's a whole list here [http://probabilistic-
programming.org/wiki/Home](http://probabilistic-programming.org/wiki/Home) and
I'm not sure which ones are like "real projects" that can be used in
production and which ones are like toy/research projects and I'm not sure if
they all interpret the idea of probabilistic programming fundamentally
differently or if they are all just flavors.

~~~
cf
It depends on what you are looking for in a probabilistic programming system.
If you want something that is more of a library than a language you gain the
advantages of being integrated into a mainstream language. This would point to
systems like Figaro, PyMC, Edward and Anglican. If you don't mind a standalone
language you can choose systems like WebPPL, Hakaru, Stan, or Venture. There
are tradeoffs in expressivity as well. There are probabilistic models you can
express in WebPPL or Anglican that you can't in Stan. Also different systems
support different inference algorithms. So if you want to do something like
Latent Dirichlet Allocation, I think JAGS still does better than Stan using a
naive implementation. At the current time, to use these systems productively,
you should have some idea of what model you want to write and given your data
what inference methods you expect to work with that model.

I think Figaro, Stan and PyMC are the most "production-ready" in the sense
they have been used for projects outside the realms of their creators. Still I
would argue on some level all of them are research projects that aim to
explore how to make probabilistic modeling more accessible to people. Ideas in
one language often will appear in another down the line. So I encourage to
explore a few of them and reach out to the people working on them.

~~~
adamnemecek
Thanks for the answer.I think that you might have some answers for questions
I'm trying to answer. Do you think that we could chat about this over email?
My email is <username>@gmail.com.

~~~
cf
Sure. I will also continue to answer any questions others have about these
systems on this thread.

~~~
nextos
What work has been done in optimizing code / sampling?

My understanding is that if I write e.g. an HMM in any of these languages, the
resulting code will be suboptimal in performance and accuracy vs e.g. Viterbi.

~~~
cf
So there has been work done in compiling probabilistic programs directly into
C or C++ [1]. Your intuition is correct that in many of these languages if you
wrote an HMM and did MAP inference over it you wouldn't recover the Viterbi
algorithm, but there is some code in WebPPL[2] that handles this case as
outlined in
[https://arxiv.org/abs/1206.3555](https://arxiv.org/abs/1206.3555).

[1]
[https://web.stanford.edu/~ngoodman/papers/aistats2014-shred....](https://web.stanford.edu/~ngoodman/papers/aistats2014-shred.pdf)

[2]
[http://docs.webppl.org/en/master/inference/methods.html#enum...](http://docs.webppl.org/en/master/inference/methods.html#enumeration)

