

What is probabilistic programming? - mlla
http://radar.oreilly.com/2013/04/probabilistic-programming.html

======
jkldotio
Probabilistic Programming and Bayesian Methods for Hackers:
[https://github.com/CamDavidsonPilon/Probabilistic-
Programmin...](https://github.com/CamDavidsonPilon/Probabilistic-Programming-
and-Bayesian-Methods-for-Hackers)

~~~
shmageggy
And for a more theoretical, less interactive picture, here's a video of Noah
Goodman, one of the creators of Church, at a Google tech talk discussing the
language's philosophical foundations and showing a bunch of applications in
cognitive science/psychology.

<https://www.youtube.com/watch?v=fclvsoaUI-U>

------
tristanz
One thing that's not entirely clear from the article is that probabilistic
programming _is_ Bayesian modeling and inference. Probabilistic programming
isn't a new machine learning technique it's a more flexible way to do Bayesian
statistics.

The new term "probabilistic programming" stems from the recognition that
stochastic programs can be automatically interpreted as defining a joint
probability model and Bayesian inference on this model equates to sampling
execution traces that are consistent with the observed data. This is an
extremely powerful idea. It enables one to quickly build and compose both the
standard Bayesian models you'd find in textbooks, and innovate easily to build
new models tailored to particular application domains, often by composing more
standard models with more exotic ones. Probabilistic programming provides a
natural way to specify these complex models and has the potential to hide the
computational difficulties that plague applied Bayesian modeling.

Making all this practical is the real challenge. Performant sampling of
execution traces for a broad class of models is an unsolved problem, which is
part of the motivation for DARPA's new project.

For those interested in this area, we're working on one effort alone these
lines at Sense: <https://www.senseplatform.com>.

------
wannesm
If you're interested in actual implementations, the website of the
Probabilistic Programming Workshop offers an overview of actively developed
languages:

<http://probabilistic-programming.org/>

~~~
gtani
Thx. also this

[http://research.microsoft.com/pubs/173887/model-learner-
popl...](http://research.microsoft.com/pubs/173887/model-learner-popl13.pdf)

<http://research.microsoft.com/en-us/projects/fun/>

[http://www.cs.tufts.edu/~nr/cs257/archive/avi-
pfeffer/figaro...](http://www.cs.tufts.edu/~nr/cs257/archive/avi-
pfeffer/figaro.pdf)

------
Houshalter
I'm still confused. What exactly does a probabilistic programming language do
differently than a normal one? Does it just have a lot of support for machine
learning and representing data and stuff like that, or is there something
fundamentally different?

~~~
plam
Correct me if I'm wrong, conceptually, probabilistic programming does to
equation solvers what declarative language like SQL do to query data.

Say you have a model, y = mx + b, and you have some data. Normally you can
solve this explicitly with matrices or whatever. But you'll need to implement
the solution. With probabilistic language like BUGS, you can feed in the model
and the data and it would return `m` and `b` as a probability distribution.

This is an trivial example. One example of where it really comes in handy is
doing bayesian inference.

~~~
shmageggy
A useful analogy is Prolog, which you can think of as a logical programming
language. It has formal, logical rules as primitives, and the deductive
inference search happens under the hood. Similarly, probabalistic programming
languages have random variables and probability distributions as primitives
and do probabalistic inference (usuallly Baysian) under the hood.

------
drewcoo
As a test dev, I'm really interested in this (and in inference
engines/languages like Prolog). How can I write code that has the best chance
of finding bugs that customers might care about? Short of wrapping humans in
APIs (and I also _love_ that), we can write tests that are different every
time to try to avoid "stale" tests and approach the diversity that humans
exhibit. Probabalistic computation has that built in! A lot of manager-types
want a "repeatable" process but I'd like an "easily, know-ably mutable" one.
One with known bounds even if the actual inputs/results weren't the same every
time.

If your unit tests were automagically varied wouldn't that make them more
valuable? They might actually find new bugs once in a while.

~~~
Dn_Ab
I'm uncertain from your description but have you heard of QuickCheck or
<http://www.cs.york.ac.uk/fp/smallcheck/> ? QuickCheck might be available in
some form in your language.

------
jambonetoeufs
I'm surprised the article suggested BUGS instead of JAGS. JAGS has pretty much
all of the features of BUGS (except a GUI) and is open source, cross platform,
faster, and easy to run batch jobs from the command line. I've been using JAGS
pretty much everyday for the past 1.5 years for a research project I'm working
on and have grown quite fond of it. Before that, I was using BUGS for 6 months
and was in a living hell with how slow fitting models with it was. Since
switching to JAGS, my productivity has increased by several orders of
magnitude. I'm primarily using it for analyzing Markov Chain Monte Carlo
models of behavioral data where standard nonlinear optimization techniques
(such as MLE and MAP) are impossible to use.

Here's a link to JAGS's homepage: <http://mcmc-jags.sourceforge.net/>

There's also PyMC (which can do similar types of analysis).
<https://github.com/pymc-devs/pymc>

