

Terence Tao on the relationship between classical and Bayesian reasoning - bdr
http://www.google.com/buzz/114134834346472219368/G5DnA8EL7D3/In-classical-logic-one-can-represent-ones

======
lotharbot
This strikes a nice balance between pop-level postmodernism ("there is no
truth, only interpretation") and authority-based philosophies ("such-and-such
holds the key to absolute truth and cannot be questioned"). I've seen the same
reasoning given other names, but for someone versed in formal logic and
probability theory, this is a very nice presentation of the concepts.

Key points:

\- in Bayesian reasoning, beliefs are not held as absolutes, but as
probabilities.

\- observations / measurements change those probabilities (using Bayes'
formula). As a special case, certain measurements may set certain
probabilities to zero; viewing only the "zero/nonzero" dichotomy is equivalent
to classical logic.

\- as a counterpoint to Holmes' "When you have eliminated all which is
impossible, then whatever remains, however improbable, must be the truth",
beware that you may have treated something as impossible which is merely
highly improbable. There is a significant difference between actually zero and
merely very close to zero.

\- A implies B does not mean B implies A. But A makes B more likely does mean
B makes A more likely; this is a consequence of Bayes' formula. (This does not
imply causation. It's just a recognition of correlation -- if two factors are
correlated, the presence of one increases the likelihood of the presence of
the other.)

~~~
khafra
"When you have eliminated the impossible, whatever remains is often more
improbable than your having made a mistake in one of your impossibility
proofs." - The Black Belt Bayesian

~~~
lotharbot
A few years back, a new proof came out regarding an algorithm to find large
primes. There was a slow, perfect variant and a faster, statistical variant
with a small chance of error (like 1 in 10^20.)

"Of course, this means to be really sure you have to use the slower algorithm
or go through a rigorous proof. On the other hand, the probability that you'd
mess up a proof is much higher than that."

It was a fantastic illustration of the principle.

------
mnemonicsloth
See also: Probability, The Logic Of Science, by E.T. Jaynes.

[http://www.amazon.com/Probability-Theory-Logic-Science-
Vol/d...](http://www.amazon.com/Probability-Theory-Logic-Science-
Vol/dp/0521592712)

Available here: <http://bayes.wustl.edu/etj/prob/book.pdf>

The introduction/first chapter has a nice example about a policeman concluding
a crime is being committed that's very relevant here.

~~~
jdale27
That PDF is only the first three chapters.

~~~
khafra
<http://www-biba.inrialpes.fr/Jaynes/prob.html> <\-- the rest.

------
greenlblue
This is one of the main reasons I like Tao. The guy is a terrific and prolific
mathematician but he always strives to reach the larger public with his
writing.

------
mannicken
In other words, "correlation rises the chance of a causation"?

~~~
mnemonicsloth
Yes, modulo some semantics.

"Chance" is a very frequentist word. It's Better to say "correlation makes us
more willing to believe there is causation."

