
Probabilistic Programming and Bayesian Methods for Hackers (2013) - jxub
https://camdavidsonpilon.github.io/Probabilistic-Programming-and-Bayesian-Methods-for-Hackers/
======
fwdpropaganda
This looks amazing and I will definitely read it.

As an aside, does this line bother anyone else?

> programming from a computation/understanding-first, mathematics-second point
> of view

Mathematics _is_ understanding. What's implicit? The suggestion that
mathematics isn't understanding, it's just funny symbols that no one
understands.

> The latter path is much more useful, as it denies the necessity of
> mathematical intervention at each step, that is, we remove often-intractable
> mathematical analysis as a prerequisite to Bayesian inference.

In fact, I would say that by removing maths you're removing at least some
amount of understanding. You're left with a workflow of "if x use model a else
use model b." That will work until it doesn't, i.e. until you have to face a
problem that isn't predicted by the workflow. Not saying that's a bad approach
(it's certainly the most pragmatical) but it shows that the phrase
"understanding first maths second" is misguided at best and disengenious at
worst.

I think a more appropriate phrase for what the author describes in the
prologue would be "practicalities first, theory second".

~~~
ianbooker
Some people think that mathematics can also be seen as a language. I use this
categorization quite frequently nowadays and it really simplifies my work a
lot. By not using a more elaborate mathematical notation, you basically are
able explain mathematical concepts to people who are not "speaking math" like
you would explain the work of Montesquieu to anyone not speaking french: It
takes more time and effort, and francophone readers of Montesquieu will hate
you for doing it. ;)

~~~
eftychis
Skipping the notation sure is nice and fine as long as you don't end up losing
any of its benefits.

The whole idea of notation/language is:

1.to have a common reference for each definition that is explicit 2\. forces
you to define and handle proofs from a proper angle of detail (think of it
like trying to write the specification of a data structure and an algorithm in
prose instead of Haskell, C, some assembly dialect. This is actually
interesting because what people call "math language" comprises a wide variety
of different styles, like programming languages do. 2\. concise -- again and
again using notation and prior well-defined notions saves a lot of time and
space; you don't tangle on the properties of registers for everything you do.

I would say that mathematics has a variety of languages that were, and are
introduced per area, to go along. If you have a programming languages
background you are familiar with the setting (and joke) that each new work
defines a new (somewhat) programming language.

In general, there is a reason computer science is regarded to be that close to
math.

------
swframe2
I recommend David MacKay's Intro to Info Theory class:
[https://www.youtube.com/watch?v=BCiZc0n6COY](https://www.youtube.com/watch?v=BCiZc0n6COY)

There is a free book for it at:
[http://www.inference.org.uk/mackay/itila/](http://www.inference.org.uk/mackay/itila/)

Daphne Koller's 3 coursara classes on Probabilistic Graphical Models are also
really good.

------
V2hLe0ThslzRaV2
This is the "online" (abridged) version of this book:

[https://www.amazon.com/Bayesian-Methods-Hackers-
Probabilisti...](https://www.amazon.com/Bayesian-Methods-Hackers-
Probabilistic-Addison-Wesley/dp/0133902838)

There's a full version of another good book ("Think Bayes: Bayesian Statistics
in Python") available as a PDF from the publisher here:

[http://greenteapress.com/wp/think-bayes/](http://greenteapress.com/wp/think-
bayes/)

(related Amazon reviews)

[https://www.amazon.com/Think-Bayes-Bayesian-Statistics-
Pytho...](https://www.amazon.com/Think-Bayes-Bayesian-Statistics-
Python/dp/1449370780)

------
glial
This book is helpful - I went through it and did all the interactive examples.

Having said that, I much prefer "Doing Bayesian Data Analysis" by Kruschke. It
is extremely clear. It introduces concepts with intuition first, then math,
then code, which I find to be an extremely useful order.

------
activatedgeek
I would recommend incoming readers to really invest time on theory before
starting this book (I know that is exactly what this book does not want you to
do) and use this book as the driver of your mental model the next time you
encounter Bayesian methods.

I personally find exceptional clarity once I see the code for a certain
technique in Machine Learning. Often times, the theory skips certain
implementation details which always leaves a void for me. After having read
quite a bit on Bayesian Learning sometime ago, it is easy to connect this
guide back to theory. That immediate click of ideas is rewarding!

~~~
Woberto
Do you have any suggested reading for the theory? I'm going to be using
QUESO[1] for some research with Bayesian statistics and am trying to learn
more of the background knowledge.

[1]: [https://github.com/libqueso/queso](https://github.com/libqueso/queso)

~~~
activatedgeek
I absolutely love David Barber's book (BRML). It is available here -
[http://web4.cs.ucl.ac.uk/staff/D.Barber/pmwiki/pmwiki.php?n=...](http://web4.cs.ucl.ac.uk/staff/D.Barber/pmwiki/pmwiki.php?n=Brml.HomePage)

I'm inclined towards Machine Learning and hence the bias. Not sure if this
would cover the statistics parts but I think at least the fundamentals are the
same.

------
madenine
This is a must have, mostly because it serves as the best documentation / user
guide for PyMC - a fantastic stats package that can be a little tricky to get
started with.

Support great work like this and buy the book!

------
0xdeadbeefbabe
Is P(a|b) = ( P(b) * P(b|a) ) / P(b) more true when the probabilities are from
a binomial distribution as opposed to poisson, or am I just confused and it
doesn't matter.

------
dang
Many previous discussions:
[https://hn.algolia.com/?query=Probabilistic%20Programming%20...](https://hn.algolia.com/?query=Probabilistic%20Programming%20and%20Bayesian%20Methods%20for%20Hackers%20points%3E3&sort=byDate&dateRange=all&type=story&storyText=false&prefix=false&page=0)

------
zeristor
I think this is quite good:

[https://www.youtube.com/watch?v=3OJEae7Qb_o](https://www.youtube.com/watch?v=3OJEae7Qb_o)

although if I knew more about Bayesian methods I would be in a better place to
recommend it.

------
juanherrera
Nice to see a practical way to address this important topic I will read it ,
thanks

------
mark_l_watson
Looks good. Someone gave a lecture on PyMC at work last week and put PyMC on
my radar. This book is also on O’Reilly Safari, and I just started reading it.

