
The Design and Implementation of Probabilistic Programming Languages - CrocodileStreet
http://dippl.org
======
KarthikT
It seems like quite a hot research area, and the quality of projects being
released is pretty impressive. Microsoft Research's work in this area is
pretty good too (Infer.NET and Tabluar)

[http://research.microsoft.com/en-
us/projects/fun/](http://research.microsoft.com/en-us/projects/fun/)

------
avaku
Interesting how they now reverted back from a Scheme-like language, previously
also published by Goodman at [https://probmods.org/](https://probmods.org/)

Do you think it's because of all the parentheses that people can't understand?

~~~
BenoitEssiambre
They use a mostly functional subset of javascript (You are not allowed loops).
It is much more readable than scheme if you ask me but not necessarily more
powerful. You get the benefits of the very powerful and ubiquitous javascript
compilers that exist nowadays which I like.

I played with it and you are somewhat limited right now. The first issues is
that you quickly hit the top of the javascript stack because of the recursive
style (remember: no loops). The most elegant solution to that would be for
browsers to support tail call optimisations which is in the ES6 standard (Are
you listening Mozilla or Google? If you implement this part of the spec sooner
you could be leading the way for the most promising area of AI and maybe of
computing in general). Although there are possible work around namely if the
webppl compiler were to use trampolines or "promise" like structures. IMO, it
would clutter the code and be much less elegant than native ES6 TCO.

The second limitation is that you are not allowed mutability. There is a git
branch with support for mutability partly implemented. I'm not sure they
should go that way though. I've never really been into functional programming
style but in the case of probabilistic languages, I really see the value. It's
much simpler and easier to follow what is going on when debugging. Supporting
mutability also makes the compiler more complex and memory usage high as you
have to copy the whole memory footprint of your program at each probabilistic
sample you take. It is not that difficult to instead include something like
mori [http://swannodette.github.io/mori/](http://swannodette.github.io/mori/)
to manipulate arrays immutably.

------
avaku
This is the next breakthrough coming up after Lisp...

~~~
eli_gottlieb
Probabilistic programming is already basically the definitive Next Big Thing
in machine learning, since it allows us to automate reasoning over far more
sophisticated models than we could previously write down by hand, while the
approximate inference techniques being used make numerical evaluation of
Bayesian reasoning tractable.

The generality and power of the approach are, frankly, amazing.

~~~
jules
MCMC probabilistic programming won't be the next big thing in machine learning
though. You need some sort of maximum likelihood / maximum a posteriori /
variational bayes to do machine learning. MCMC is far too slow.

~~~
murbard2
Maybe not. Not all probabilistic programming relies on MCMC, you can do MAP
with probabilistic programming too. But in general, it's true that it tends
towards Bayesian integration.

In a sense, MCMC tries to solve #P complete problems, while MAP "only" tries
to solve NP complete problems. The former are probably intrinsically harder
than the latter.

However, it's quite possible that some clever implementation might yield
results which are as good as MAP in the same amount of time, and progressively
refine them with more time.

There are two free lunches with the curse of dimensionality: - the manifold
hypothesis \- Monte-Carlo integration

True, the dimension rears its head as an ugly multiplicative constant in the
latter - still, there has to be something there.

------
lowglow
I want to build a probabilistic operating system. People don't understand what
I mean when I say that. Argh.

~~~
infogulch
What do you mean, a probabilistic operating system?

~~~
not_that_noob
Probably some kind of operating system that relies on probabilistic
principles.

I hope it's not an operating system that decides to run your thread based on
the output of a random variable. And also I hope this is not a realtime
operating system. That would probabilistically speaking be the lower depths of
hell.

