
Better Than Rules: How Bayesian Theory Is The New If Statement - bkmrkr
http://greggurevich.com/2007/04/19/the-rise-of-statistics/
======
jey
A very good essay on Bayesian probability theory and its applicability to
rationality is "A Technical Explanation of Technical Explanation" at
<http://yudkowsky.net/bayes/technical.html> . The beginning is more mathy than
the latter half, which is more interesting, so don't let the math keep you
from reading it and getting to the more interesting later parts. There isn't
much math, and the math that is there is pretty simple. If you haven't been
introduced to Bayesian probability, you need to at least skim through the
article entitled "An Intuitive Explanation of Bayesian Reasoning" that is
linked to from the above URL.

If you decide that Bayesian rationality is totally awesome, read the book
"Probability Theory: The Logic of Science" by E. T. Jaynes, which is
specifically about Bayesian probability theory and its applications, derived
entirely from a list of simple informal "desiderata" that we would expect from
a theory of probability. He shows that Bayesian probability theory is the
_only_ way to satisfy these simple logical desiderata, and that Bayesian
probability theory is the unique consistent and natural extension of
Aristotolean logic (True/False logic) to real values.

Bayesian probability is a good topic to be familiar with as a general
conceptual tool for evaluating information; it's not just some obscure theorem
used by statisticians.

------
npk
ced - "Bayesian Theory" is more than just a formula, it's a way of thinking. A
beautiful book exists on this subject, "Information Theory, Inference and
Learning Algorithms" by David Mackay, which you can read for free online.
Granted, the nomenclature is confusing, there is a Bayes formula. [Edit: jey
mentioned Jaynes' book. Another excellent reference. Link to chapter in Mackay
(<http://www.inference.phy.cam.ac.uk/mackay/itprnn/ps/22.40.pdf> read from
bottom of page 25 on)]

In regards to the unsophisticated aspect of startups, I think that much of the
low-lying fruit has been picked. Statistical sophistication is but one ladder
to the higher fruits. Why? Social websites have enormous databases of
information about their users. People who /understand/ data will be able to
take social networks to the next level.

~~~
jwp
Good cites. I second them. Mackay and Jaynes are excellent but can get out of
hand quickly. Duda, Hart and Stork is another good hardcore option. When I
need the book "for dummies," which is pretty often, the Weka book and
Mitchell's Machine Learning are helpful.

------
ced
I think it's strange that "Bayesian Theory" has become a concept. It's just
one of many formulas in probability theory. Yeah, it's particularly useful in
many cases, but learning Bayes theorem without learning the rest is silly.

Regardless, I agree that investing in probability/statistics is a good idea.
Right now, there's a rush to fill all the niches of the web ecosystem.
Startups can afford to be relatively unsophisticated, because so is everyone.
I suspect that eventually, we'll see radically more "evolved" systems
displacing the old ones, like Google did to Yahoo. I don't think that those
could be built in 3-months, though. I'm curious to see if YC will have to
change its model, but that might be a way off.

------
tocomment
I certainly see how a Bayesian theory is useful, but I can't fathom how to use
it instead of an if statement. What does that even mean? What would it look
like? Can anyone enlighten me?

~~~
npk
It's a platitude. The full quotation and context is here:
<http://www.joelonsoftware.com/items/2005/10/17.html>

I read the "very senior Microsoft developer"'s point as something like:
because if statements are so fundamental, most lowly MS programmers think in
terms of them. Google, hires programmers who've taken a course in machine
learning or something, and use statistical inference (a bunch of if
statements, just with some abstraction) instead of lookup tables (a bunch of
if statements.)

Yawn. This is just Google PR speak. The quotation doesn't really mean if
statements will go away, it just means that you need to think at a "higher
level." This is just someone saying "X is smarter than Y."

