
Statistics, probability, and Nate Silver - jwallaceparker
http://kfury.com/statistics-probability-and-nate-silver
======
hooande
Honestly? It's cool to see a statistician getting media love. I imagine for
most people he's like a real life character from Big Bang Theory. I've seen
him on tv several times and he comes off as incredibly nerdy, but he's skilled
at talking about statistics and making it relatable to the average person.

In a way, he's a spokesman for statistical reasoning to the world. It's odd to
see how much pushback he gets for saying "A candidate with a slight lead in
October wins 75% of the time". This is a fact, easy to look up. But because
the other side and the media don't like it, Nate is put into the position of
defending common sense to America. And he does an admirable job.

Hopefully he can find a way to use this for good outside of politics. Maybe he
can educate people about statistics and probability in other ways. Starting a
blog that correctly predicts the behavior of reality tv stars might be the
best thing he could do for the cause of reason in this country.

~~~
_neil
Like a Neil deGrasse Tyson for statistics.

~~~
stcredzero
Maybe there's a media empire that can be built from people like this? We've
always had such people in the media, but our tools and technology hold more
potential power to individuals than ever before.

~~~
thronemonkey
I've long thought it would be awesome if someone started a business along
these lines: a news source driven by data analysis; like fivethirtyeight but
for analysis of all sorts of news and current events, not just elections.

~~~
stcredzero
How about "Freakonomics News?" A reality show using the methods used by the
campaigns to figure out the behavior of "cohorts" could be devastating.
Marketers have known scary stuff about us based on personal data, even before
the Internet. A show based on actuarial data and gene sequencing would be both
informative and appeal to base instincts.

------
Aaronontheweb
One thing that really bothers me: most of Nate Silver's supporters and
detractors don't understand jack shit about his models.

On the Democrat side of the table, many treated 538 like a source of religious
prophecy and source of comfort - when truth be told Nate is just a smart
statistician who developed a model that's accurately predicted the outcomes of
three elections (the first being the Hillary / Obama primary.) On the
Republican side of the table, major trends in national and state-level polling
data were ignored because pundits believed that the pollsters were
oversampling Democrats when the truth is that the Democratic base has expanded
since 2008.

Other legitimate statisticians, like the University of Colorado, predicted a
Romney win and obviously their models were wrong.

Scientists, statisticians, economists, and engineers shouldn't be worshiped as
sources of incontrovertible truth - they're just humans trying to do the best
objective, accurate job that they can!

~~~
SkyMarshal
_> Scientists, statisticians, economists, and engineers shouldn't be worshiped
as sources of incontrovertible truth - they're just humans trying to do the
best objective, accurate job that they can!_

True, but they're a big step in the direction of progressively less wrong [1]
from the traditional alternative punditry and 'gut' analysis [2].

If I were a movie producer I'd buy the rights to _Moneyball 2_ , script it
about this election, and recast the gossiping scouting staff as all the
blowhards [3] and bs artists [4].

Nate Silver, Andrew Tannenbaum [5], Drew Linzer [6], Sam Wang [7] and the
other modelers deserve a lot of credit for helping the public cut through the
bullshit.

[1]: <http://serendip.brynmawr.edu/sci_cult/lesswrong/lesswrong/>

[2]: <http://blogs.wsj.com/peggynoonan/2012/11/05/monday-morning/>

[3]: [http://www.theatlanticwire.com/politics/2012/10/people-
who-c...](http://www.theatlanticwire.com/politics/2012/10/people-who-cant-do-
math-are-so-mad-nate-silver/58460/)

[4]: <http://drudgereport.com>

[5]: <http://electoral-vote.com>

[6]: <http://votamatic.org>

[7]: <http://election.princeton.edu/>

~~~
Aaronontheweb
My point wasn't so much about rate of errors or cutting through the bullshit.

My point is that it is thoroughly, truly annoying to see masses "smart" but
still obviously ignorant people apply religious qualities to scientists and
statisticians because the outcomes are either a source of divine comfort or
blasphemy.

Watch either of the "Go God Go" episodes of South Park if you want a more
concrete example of the behavior I'm describing:
[http://www.southparkstudios.com/full-episodes/s10e12-go-
god-...](http://www.southparkstudios.com/full-episodes/s10e12-go-god-go)

------
shardling
>Predicting an election based on polls is an entirely different matter. The
election will turn out one way or another. If the same people voted for
President 100 times without an external factor interfering differently across
samples, the outcome would be the same every time.

I'm glad someone pointed this out, since I think it gets obscured a bit in
most discussion of 538's statistics.

~~~
erichocean
I don't think people are that deterministic. There's still internal factors
that could cause them to change their vote; not everyone has everything 100%
decided when they step in the polling booth.

But yeah, it'd be pretty close.

------
oulipo
I understand what the post is trying to convey, yet it is easy for someone to
cast the election & football in an equivalent framework: increasing the number
of polls one of the statistician uses in order to narrow the confidence
interval is equivalent to increasing the number of measurements you make on
the football players & their environment before the game in order to narrow
the confidence interval on the match outcome..

in other words you can interpret the confidence in the case of the election
just as in the case of the football game, by stating that:

1\. if your mathematical model of the population is accurate (eg. for instance
your population really is voting using a binomial law of a certain probability
p_democrat of voting for the democrat candidate) 2\. and if the measurements
(polls or player states) you have made give you that at least k voters out of
N polled would have voted democrat

then you can consider all the (hypothetical) instances of a population which
would have resulted in a same or higher number of democrat voters for this
poll, and you can compute the confidence interval that gives you the
probability that such a population would indeed elect a democrat president

So you don't necessarily have to view it in terms of "not changing the
population vote, but changing the precision of the data", you can also cast it
in just the framework of the football prediction

------
jules
> There is internal chaos in the game that forces the probabilistic
> distribution. Predicting an election based on polls is an entirely different
> matter. The election will turn out one way or another. If the same people
> voted for President 100 times without an external factor interfering
> differently across samples, the outcome would be the same every time.

This is a distinction without a difference. There is "internal chaos" in the
game because we don't know its initial conditions exactly, just like in the
elections. If we had precise knowledge about the particles in the ball, the
field, and the players, we could theoretically predict the outcome with high
certainty. Except for quantum mechanical effects, statistics isn't about true
randomness, it's about modeling lack of knowledge.

~~~
loup-vaillant
Nitpick: _even_ for quantum mechanical effects. The equations have no
randomness in them. As far as we know, the laws of physics are deterministic.
It's just that we can't know in advance which Everett branch we will
subjectively experience. We could (in principle) compute what proportion of
Everett branches lead to what outcome, though.

Probability is still in the mind. Not in the territory.

<http://lesswrong.com/lw/oj/probability_is_in_the_mind/> (In case you didn't
already spot me.)

~~~
jules
In a sense that's true, but Bell's inequalities show that there is something
qualitatively different about quantum mechanical randomness. Bar some
technicalities, they show that there isn't some hidden internal state that
determines what outcome we experience, unlike with elections and baseball
(assuming QM effects don't play a significant role in those). In this sense,
QM is "truly random".

That said, even though there is good experimental evidence for this, it is not
100% certain that the world actually works this way.

<http://en.wikipedia.org/wiki/Bells_theorem>

~~~
loup-vaillant
_Nothing_ is 100% sure :-)
<http://lesswrong.com/lw/mp/0_and_1_are_not_probabilities/>

But yeah, as long as we didn't find complete laws of physics, we're bound to
ignore something.

~~~
jules
Not sure if you got what I'm saying...in case I didn't explain it clearly
enough: Bell's inequalities show that quantum randomness CANNOT be explained
by an underlying deterministic law of physics that we're ignoring. With
ordinary probability you can never find out whether it's truly random or
whether there is some underlying deterministic process that you just don't
have knowledge about. Turns out with quantum mechanics, which works with
probability amplitudes instead of just probabilities, you _can_ rule out such
an underlying deterministic process (again modulo technicalities). This is one
of the most striking results of physics, and Einstein himself didn't believe
it at first.

~~~
loup-vaillant
Disclaimer: I have read
<http://lesswrong.com/lw/r5/the_quantum_physics_sequence/> cover to cover.

Let's see with an example. A half sievered mirror, a photon going through (or
bouncing), and 2 detectors. You say there is no hidden variable that would
allow us to predict which detector _we will see_ go off.

I agree.

It doesn't mean however that the laws of physics have true randomness in them.
Under the Everett interpretation, two "worlds" (blobs of amplitude) are
created: one in which detector 1 goes off, and one in which detector 2 goes
off. from the outside, this is purely deterministic. The only randomness lies
in our subjective experience. The only question remaining is why do we
experience the Born statistics (based on square moduli), rather than something
else (like, based on moduli)?

We _would_ have true randomness at the fundamental level if some form of the
Copenhagen interpretation where true. But believing the blob of amplitude we
don't see would mysteriously disappear, violating the equations in the
process? That's just nuts. If this idea appeared after the Everett
interpretation, it would have been ridiculed.

~~~
jules
Many Worlds vs Copenhagen is a largely irrelevant debate. It doesn't make any
observable difference whatsoever, so it's metaphysics. By analogy to classical
probability, Copenhagen is like looking at the sampling of a random variable,
and Many Worlds is like looking a weighted list of possible outcomes. It's
just a different perspective on the same thing.

I say largely irrelevant because there is a minority of people (including, if
I'm not mistaken, Nobel prize winner 't Hooft) who say that neither Many
Worlds nor Copenhagen is right, because there are hidden assumptions in EPR
that are overlooked. Since you seem to be a fan of lesswrong.com, this should
appeal to you ;-).

I think for the rest we are in agreement, I misinterpreted "as long as we
didn't find complete laws of physics, we're bound to ignore something" and
"Probability is still in the mind. Not in the territory.". The latter
statement becomes rather tautological and its meaning rather weak if you
include QM randomness though. In the context of the frequentist vs Bayesian
debate it means something far stronger.

~~~
loup-vaillant
Funnily enough, "we didn't find complete laws of physics" is a reason why I'm
pretty sure Many Worlds _as is_ is false to some extent. I'm pretty sure
however that the Copenhagen interpretation is much farther from the truth.

As for how this is metaphysics… Sure, but no more that wondering if a photon
(or a colonization ship) you just send into deep space, outside the observable
universe, still exists. I mean, I don't care if it's meta or not. I just trust
Occam's razor (to the extent it gives prior _probabilities_ , not prior
certainties), and leave it at that.

~~~
jules
Occams razor implies that you should be agnostic to things that are not
observable, not that you should pick your metaphysics. Whether the alternative
universes actually exist or not is completely irrelevant. The rational thing
to say is "I don't care, I don't want to spend brainpower worrying about
this". In mathematical terms, the hypothesis "many worlds is true" has more
information content than "the predictions of QM are true" since the former
implies the latter, hence Occam's razor prefers the latter. And indeed, you
should also be agnostic to whether or not there is a teapot floating outside
the observable universe.

~~~
loup-vaillant
I'm not sure I agree. Things such acausal trade may mean we should know about
what we can't affect (nor can affect us). But even then, I was merely talking
about truth for truth's sake, where otherwise worthless knowledge does matter.

Nitpick: You didn't even need Occam's Razor: Many World _implies_ that the QM
predictions are true. of course the truth of those predictions are more
probable than Many Worlds. I, on the other hand, used Occam's Razor to compare
Many Worlds and Copenhagen, which are _mutually exclusive_.

I agree agnosticism is good. I'd rather be aware of my own ignorance. I like
it better however when I'm also aware of its magnitude. Probability theory
does just that. I don't _know_ if Many Worlds is true, nor do I know if
Copenhagen is false. But my current state of ignorance is more precise than
that.

------
maaku
One additional point that doesn't get much play in the debate about Nate
Silver's numbers: political science, like economics, is a soft-science. You
_cannot_ 100% accurately predict the election based on polls, just as you
cannot accurately predict market movements based on economic indicators. This
is because the system is made up of self-aware human actors that respond to
meta-information like Nate Silver's numbers. Even if the polls were 100%
accurate in measuring voter sentiments, as soon as they are published they are
picked up by political operatives and become part of the spin machine which
changes those sentiments.

But if you want to use polling data, Nate Silver is demonstrating the least-
bad way to do it.

~~~
csense
> This is because the system is made up of self-aware human actors that
> respond to meta-information like Nate Silver's numbers.

The fact that agents use polling information to make decisions doesn't
necessarily mean the system won't reach equilibrium.

But I do think it's conceivable that one candidate's predicted overwhelming
chance of victory might have a way of becoming a self-fulfilling prophecy in a
close election.

Ron Paul supporters presumably prefer Romney to Obama. In an election where it
was reported that the race was close, they'd vote for Romney. But if it was
reported that an Obama win was basically guaranteed, they might be willing to
vote for the Libertarian Party or write-in Ron Paul (who isn't even running)
as a protest vote.

I really think our democracy would be healthier if we switched to an instant
runoff system [1], but the Democrats and Republicans have a very strong
incentive to unite against anything that could strengthen third parties.

[1] <http://en.wikipedia.org/wiki/Instant-runoff_voting>

~~~
maaku
> But I do think it's conceivable that one candidate's predicted overwhelming
> chance of victory might have a way of becoming a self-fulfilling prophecy in
> a close election.

It also has the possibility for the opposite by lowering turnout for the
"winning" candidate. Why vote? Nate Silver already called the election, etc.

There are other ways in which polling data itself can influence an election.
It appears that Nate Silver did an great job early on at predicting the effect
of changing demographics in swing states like Florida--which ended up giving
the election to Obama. Hindsight being 20/20, if the Romney team had paid more
attention to that (and perhaps compromised on things like immigration reform),
things could have gone very differently.

