
Probability, Paradox, and the Reasonable Person Principle - yomritoyj
http://nbviewer.ipython.org/url/norvig.com/ipython/Probability.ipynb
======
jgrahamc
The hand-wavy way I've always thought about the Monty Hall problem is as
follows:

When the contestant picked a box he was making a random choice between three
boxes and had a probability of 1/3 of picking the car. The probability that
one of the other two boxes was the car is thus 2/3\. When Monty Hall opens a
box it's still the case that the probability of the car being in the other two
is 2/3 but the contestant can now eliminate the open box and knows that the
probability that the car is in the other box is 2/3\. Thus it's worth
switching.

~~~
proksoup
Monty chooses a door?

I thought he selected it at random.

That was always the confusing unstated assumption for me, that makes me think
the problem is only confusing because of how the wording usually de-emphasizes
that distinction.

If he chooses a door, you can benefit from the secret information he reveals
sometimes by doing so.

If the door was chosen at random, he is not adding any information, so you
can't act on it?

~~~
tel
That's the whole reason why the problem is tough. It's carefully worded to be
ambiguous about that point so that most people assume it was random (most of
the erroneous math people pull out is valid were it random) but the problem
actually makes little sense if Monty picked randomly since it completely
ignores the potential possibility that he picked the car door.

The problem isn't probabilistically difficult and most people's intuitions
would be on point if it weren't set up to deceive.

~~~
JoeAltmaier
Ah. In my generation, who actually watched the program, there is no ambiguity.
No intent to deceive was detected by me. Its not supposed to be a trick
question. Its supposed to illustrate how far off our intuition is about
statistical behavior.

Monty himself mentions that he would occasionally try hard to argue
contestants into swapping. They would always refuse. He would even explain
that 'his door' had the better chance - nobody would listen. But I suppose
that's confounded by the suspicion that he was trying to trick them.

~~~
tel
This may be very true and I think that if you're more familiar with the
program and recognize that the host must know the answer then it becomes more
a question of statistical intuition.

I'm confident that there are lots of people who still fail to make the
connection (though we're getting to the point where I think people are failing
less due to a lack of statistical intuition and more due to a
symbolic/physical model mismatch issue), but I think this problem wouldn't be
as renowned as it is if it weren't for all of the even expert statisticians
who are getting fooled.

In their case it's definitely a matter of them misinterpreting the situation
and the way that it's worded, for someone unfamiliar with the show, is at
least a little "tricky".

------
fitzwatermellow
Love the George Carlin quote referenced in the section defining The Reasonable
Person Principle:

 _" Have you ever noticed when you're driving that anybody driving slower than
you is an idiot, and anyone going faster than you is a maniac?"_

I'll include the canonical quip by Laplace:

 _" The theory of probabilities is at bottom nothing but common sense reduced
to calculus; it enables us to appreciate with exactness that which accurate
minds feel with a sort of instinct for which ofttimes they are unable to
account."_ -Introduction to Théorie Analytique des Probabilitiés

~~~
JoeAltmaier
Hm. That guy writing the intro was extremely optimistic about human nature.
Most folks find statistics very unintuitive. Folks routinely overestimate
extremely unlikely events (lotto) and underestimate near-certainties (auto
accident). Its the idea that we _might_ get away with something, or _maybe_ it
won't happen to me, that guides most human activities.

Its this warping of the probability curve that is responsible for major facets
of human society: war, witchcraft, gambling, marriage, on and on.

~~~
darkmighty
I think those problems are a little different. For example, no matter how you
approach it, from a rational point of view joining a war for your country
makes no sense. You are highly likely to die and highly unlikely to change to
outcome of the war.

But if you think in an evolutionary sense, if each individual in a tribe acted
rationally, we'd get a sort of Tragedy of the Commons: the enemy tribe wins
the war, and slaughters your men. So it may happen that if each individual
acts rationally the group, each individual included, suffers; so it makes
sense individual sense to develop mechanisms to go to war, which by symmetry
must include yourself.

Even less intuitively even if the probability of death is _lower_ by not going
to war (no tragedy of the commons in traditional sense), which means no
individual would rationally choose war or develop mechanisms to force going to
war, it would be rational _for the tribe_ to choose war, in an evolutionary
sense, provided many more of your men are killed than the enemy tribe (so they
do better in evolutionary terms).

~~~
adamlett
The consensus among biologists today is that group selection does not occur:
[https://en.m.wikipedia.org/wiki/Group_selection](https://en.m.wikipedia.org/wiki/Group_selection)

~~~
SapphireSun
I know I've heard this before, and I've not taken an evolutionary biology
class before so I do give a lot of weight to the idea that there is little
influence at the group level, but one of the first paragraphs states: "As of
yet, there is no clear consensus among biologists regarding the importance of
group selection."

In particular, when selecting for war, there's a clear individual level
selection pressure: if you don't cooperate with your group, you likely die.
This is a just so story, I have no idea how irrationality actually developed,
but say you have small groups of hunter gatherers and one individual develops
a preference for cooperating in raids against their best interest. If this
means that on average you now have two versus one whenever this individual
participates, there's a clear competitive advantage which could allow that
gene to propagate. The key here in this hypothetical is that the mutation
occurs at the individual level in a lone individual, who then cooperates with
someone else who wouldn't necessarily follow them, but is happy to have the
help. This results in a disproportionate gain (100%) in effectiveness vs lone
opponents. As time evolves, the gene becomes more widespread by its early
disproportionate effectiveness and groups that fail to cooperate are killed on
average, thus eliminating individual competitor genes.

Again, I have no idea how war actually evolved, but it seems easy to believe
that when a trait influences whether one group kills another, that it would
cause the killers to have a reproductive advantage. I'd love someone more
educated on this topic to send me up though.

------
henrik_w
Peter Norvig often covers really interesting stuff. A long time ago here on HN
I read his article on the probability of there being no set in the card game
SET [1]. I had never heard of SET then, but picked it up based on the article.
It's a simple game, but has interesting properties. Analytical answers are
hard to find for a lot of the cases, so simulation to the rescue. I ended up
running more simulations than Peter did, with some interesting results [2].

Also, the interview with Peter Norvig in the book "Coders at Work" [3] is
great - one of my favorites in the book (actually, the whole book is great).

[1] [http://norvig.com/SET.html](http://norvig.com/SET.html)

[2] [http://henrikwarne.com/2011/09/30/set-probabilities-
revisite...](http://henrikwarne.com/2011/09/30/set-probabilities-revisited/)

[3]
[http://www.amazon.com/gp/product/1430219483/](http://www.amazon.com/gp/product/1430219483/)

~~~
norvig
Very nice write-up on SET, Henrik. I hadn'e seen it before, and wasn't aware
of your interesting results.

------
tawaypdox
Another interesting paradox in probability is Bertrand’s Paradox[1]. Given an
equilateral triangle inscribed in a circle, what is the probability that chord
of the circle chosen at random is longer than the side of the triangle? (Hint:
The answer depends on your method of selecting a random chord.)

[1]
[https://en.wikipedia.org/wiki/Bertrand_paradox_%28probabilit...](https://en.wikipedia.org/wiki/Bertrand_paradox_%28probability%29)

~~~
mamon
For me the most messy part is being allowed to rotate the triangle to match it
to one of the described cases.

Normally, if you assume triangle does not move, and then draw a random chord,
then depending on relative position of the chord and triangle you use one of
three methods described by Bertrand to judge whether it is longer or shorter
than triangle side.

Then you need a way to count probability of each of those cases to happen and
then count the weighted average of all three "paradoxical" cases.

EDIT: in fact I would just bet on Method 3 as a correct solution, because it
is the most general: it does not require rotating the triange.

If you are rotating the triangle then you are in fact changing problem
definition during solving it, which leads to some hidden assumptions (i.e.
Method 2 is only good for chords which are parallel to one side of triangle,
and Method 1 is only good for chords that do not cross with triangle sides)

~~~
kgwgk
Actually, as the wikipedia entry mentions, one can argue that Method 2 is the
correct one: it is the only one respecting reasonable translation and scale
invariance constraints.

------
tunesmith
Man, that's just beautiful - I love reading his notebooks. Just skimming them
yields unexpected little gems like "the value of money is roughly
logarithmic", which I had never heard before and gets me thinking about
progressive taxation.

The final step of the St. Petersburg paradox, which he took far further than I
had ever seen before, might be implementing the Kelly Criterion, where the
amount you bet is related to the size of your own personal bankroll, but I
don't know offhand how to relate that to a probability distribution since the
Kelly Criterion is normally calculated with one odds value and one payoff
value. At the least, I'd like to see a calculation of how many times you'd
have to place the bet before you could be reasonably sure of a positive payout
- you could do that easily enough with a monte carlo simulation I guess.

~~~
norvig
Good point, tunesmith, I should define a version of `util` that lets you start
at your current personal bankroll, rather than at 0, and show how the amount
you are willing to pay to play depends on how much you already have.

------
powera
My TLDR summary: all probabilities are conditional. If you have two
probabilities for the same event and they aren't the same, you're just
assuming different priors in the statement of the probability. (and if you
can't find any different priors, maybe you've discovered something new)

~~~
norvig
Good summary. If I were in charge, I wouldn't even have a notion of
unconditional probability -- everything is conditional on something.

------
nkurz
I recently came across the "Three Prisoners Problem":
[https://en.wikipedia.org/wiki/Three_Prisoners_problem](https://en.wikipedia.org/wiki/Three_Prisoners_problem)

If you feel comfortable that the answer to the Monty Hall problem is "switch",
you might want to test your understanding to be sure you reach the correct
answer here as well. As Wikipedia states, Three Prisoners "is mathematically
equivalent to the Monty Hall problem with car and goat replaced with freedom
and execution respectively". But mathematically equivalent may not make it
intuitively equivalent, and the answer may feel like it contradicts the
correct Monty Hall answer.

Wikipedia points out that it's originally from a 1950's Martin Gardner column,
but I came across it in the textbook "Stastistical Inference" by Casella and
Berger. Here's C&B's phrasing:

    
    
      Three prisoners, A, B, and C, are on death row. The governor
      decides to pardon one of the three and chooses at random the
      prisoner to pardon. He informs the warden of his choice but
      requests that the name be kept secret for a few days. The     
      next day, A tries to get the warden to tell him who had been
      pardoned. The warden refuses. A then asks which of B or C
      will be executed. The warden thinks for a while, then tells A
      that B is to be executed.
    
      Warden’s reasoning: Each prisoner has a 1 in 3
      chance of being pardoned. Clearly, either B or C must be   
      executed, so I have given A no information about whether A 
      will be pardoned.
    
      A’s reasoning: Given that B will be executed, then either A
      or C will be pardoned. My chance of being pardoned has
      risen to 1 in 2.
    
      Who is right? 
    

In what I found to be a parody of textbook tropes, C&B begin their explanation
"It should be clear that the warden's reasoning is correct..."

While it's true that the warden's reasoning is correct, leading off with "it
should be clear" seems cruel. Here's a slight variation of the Monty Hall
problem, one of the most famous "paradoxes" of popular statistics, and you are
going to start with a paraphrase of "it should be obvious to the reader"
without the slightest sense of irony, even though this variation produces an
answer superficially incompatible with the better known problem? Ah, the
strange humor of textbook authors! The remainder of their answer (which is
solid) can be found here on Section 1.3 page 22:
[http://people.unica.it/musio/files/2008/10/Casella-
Berger.pd...](http://people.unica.it/musio/files/2008/10/Casella-Berger.pdf)

------
r4pha
It's always great to see these notebooks from Peter Norvig. There's also some
really interesting implementations (in several languages) of probabilistic
algorithms on the website [0] of the book he co-wrote on artificial
intelligence. Great book and great material.

[0]
[http://aima.cs.berkeley.edu/code.html](http://aima.cs.berkeley.edu/code.html)

------
mark_l_watson
Great advice on assuming other people are also reasonable, and try to
understand the problem they are trying to solve.

Years ago, Peter Norvig was a tech editor for an AI book I wrote. I realized
two things: he has a clearer way of analyzing things than I do, and also that
I was very fortunate that he ended up spending several hours helping me with
his review material.

~~~
Denzel
Would you mind expanding upon how he analyzes things in a clearer way? It's
something I've _felt_ when reading his blog posts.

~~~
mark_l_watson
Sorry, that was over ten years ago, and I can't find his review notes so I
can't really answer your question except to say that I remember his comments
to be very useful.

------
raymondh
The Python code is mostly clear and beautiful. Just one small wish: please use
collections.deque() instead of showing an O(n) insertion into a list using
"deck[0:0] = [card]".

~~~
norvig
You're right! 2.4 was out when I wrote that, so I could/should have used
dequeue. But I wanted to use things that I thought readers would be familiar
with, and timing wasn't a big issue.

------
raverbashing
Good write up

Most paradoxes disappear when the problem is strictly described.

------
mamon
"Problem 3. One is a boy born on Tuesday. What's the probability both are
boys?" \- this is not a problem at all. The fact that one boy is born on
Tuesday is completely irrelevant as the sex of the child does not depend on
day of the week. That information is given only to mess with readers mind. So,
if you ignore that Tuesday part then you can reduce Problem 3 to Problem 2.

~~~
vubuntu
For me both problem 2 and problem 3 should also be 1/2 .

The variances are due to how people interpret the outcome space .

It is right that probability = favaroble out comes / total possible outcomes.

In problem 2, in my opinion the possible outcomes are not how it was suggested
in the post but as below. When a family has 2 kids , the below are the only
possible outcomes

1) Potential Outcome 1: Both are boys 2) Potential Outcome 2: One boy and one
girl 3) Potential outcome 3: Both are girls

The 3rd one is not a legal potential outcome in our particular constraint of
problem 2, since problem #2 statement already states 'at least one is a boy'

So Total possible legal outcomes = 2 Favorable out comes for our event (both
boys) = 1

So probability for Problem 2 = 1/2

Similarly for problem #3, I think the post unnecessarily complicates the
calculation of problem space. The fact that 'Tuesday' is mentioned is
irrelevant in my opinon, if you state the problem #3 in a different way that
is more clearly understood.

There are 14 baskets labelled as follows "Sunday Boy", "Sunday Girl", "Monday
Boy", "Monday Girl",....."Saturday Boy", "Saturday Girl". A stork came and
dropped 2 babies. One baby was dropped in "Tuesday Boy" basket. What is the
probability that both are boys?

Now the total outcomes and favarable outcomes are :

Total possible outcomes = Number of ways second baby could have been dropped =
14 possible baskets = 14

Favorable outcomes = second baby dropped in 'boy' basket = 7 possible baskets
= 7

Probability that both are boys = 7 / 14 = 1/2

~~~
bo1024
If you understand problem 2, then you will get problem 3 as well, so let's
focus on 2.

 _> It is right that probability = favaroble out comes / total possible
outcomes._

No, not actually: It is only right if all outcomes are equally likely!
(There's an old joke about the guy who has a 50% chance of winning the
lottery, since either he will win it or he won't.)

In particular, you make that mistake here:

 _> 1) Potential Outcome 1: Both are boys 2) Potential Outcome 2: One boy and
one girl 3) Potential outcome 3: Both are girls_

These three outcomes are not all equally likely. Outcome 1 has probability
1/4, outcome 2 has probability 1/2, and outcome 3 has probability 1/4\. (This
is if you assume that each child has a half chance each of being a boy/girl.)

Norvig gets rid of this problem by listing out all four possible outcomes,
which are all equally likely.

1\. First child boy, Second child boy

2\. First child boy, Second child girl

3\. First child girl, Second child boy

4\. First child girl, Second child girl

~~~
mamon
Again, both Norvig and you are messing with ordering of events. If you list 4
events like this, assuming that ordering of boys and girls matters, then you
should stick with that. So, if ordering mattered when children were born it
should also matter when you are doing "checks". So you shouldn't formulate
problem as "one of the children is boy", you should formulate problem as
"first child is boy" (with ordering in place), which eliminates possibility 3.
Otherwise you are "solving" problem by listing sample space of completely
different problem.

~~~
bo1024
The event "at least one child is a boy" is well-defined on the 4-state sample
space. It is the set of events (first boy/second boy, first boy/second girl,
first girl / second boy). It has probability 3/4.

