

The Multi-Armed Bandit Problem - zenburnyourface
http://camdp.com/blogs/multi-armed-bandits

======
mcpherrinm
Eugh, this website has custom behavior for the left-right arrow keys, which
breaks the back keyboard shortcut in Firefox (alt+left arrow key). Non-
standard navigation on a page that IMHO doesn't need it.

~~~
triplesec
spacebar scrolling also broken in ff. grrrrr

~~~
zenburnmyface
haha ironically left-arrow right-arrow replace space-bar scroll.

------
mailshanx
Just had a look at your book, and it looks amazing! Hope you will complete
it:D

On a separate note, im surprised that in the many discussion of the MAB
problem on HN, the Gittins index was never mentioned. Under certain (mild)
assumptions on reward distributions, the GI gives a provably optimal solution
to the MAB problem...

------
bun-neh
One thing to expand upon: This is just the classic formulation of the Multi-
Armed Bandit Problem. There are countless variations to the formula which have
slightly different underlying assumptions.

------
spitfire
Good article.

I'd like to know what he used to output those graphs.

~~~
zenburnyourface
It uses Python's Matplotlib with a custom styling, available here:
[https://github.com/CamDavidsonPilon/Probabilistic-
Programmin...](https://github.com/CamDavidsonPilon/Probabilistic-Programming-
and-Bayesian-Methods-for-Hackers/blob/master/styles/matplotlibrc)

~~~
spitfire
Thanks! I quite like the output. Are you the author?

If soI took a quick look at your book and it looks excellent.

[https://github.com/CamDavidsonPilon/Probabilistic-
Programmin...](https://github.com/CamDavidsonPilon/Probabilistic-Programming-
and-Bayesian-Methods-for-Hackers)

~~~
zenburnmyface
I am the author. Thanks, I appreciate that!

