

Which Polls Fared Best (and Worst) in the 2012 Presidential Race - bslatkin
http://fivethirtyeight.blogs.nytimes.com/2012/11/10/which-polls-fared-best-and-worst-in-the-2012-presidential-race/

======
bslatkin
"Perhaps it won’t be long before Google, not Gallup, is the most trusted name
in polling."

------
pbreit
It seems like taking 3 weeks worth of polls would lead to larger error sizes,
especially towards Romney this year, since it did seem like Romney was faring
better (although still trailing) 1-3 weeks out.

------
elchief
What is the deal with Gallup?

~~~
001sky
relevant > [http://pollingmatters.gallup.com/2012/11/polling-likely-
vote...](http://pollingmatters.gallup.com/2012/11/polling-likely-voters-and-
law-of-commons.html)

~~~
elchief
But how could you/they be so wrong? Are you a subsidiary of Fox? It only makes
sense to me if there was money involved.

~~~
arrrg
Polling is hard and easy to get wrong. That’s all there is to it. There is no
need for crazy conspiracy theories.

~~~
s_henry_paulson
Step 1: Ask people if they want to share who they voted for in as many places
as possible.

Step 2: Add results.

Other than simple math, I'm not seeing what there is to mess up.

~~~
arrrg
Wow! You have just blown my mind! And to think that thousands of
psychologists, sociologists and market researchers couldn’t figure that out
for decades. What were they thinking?

The math is the easy part here. All the rest is hard.

How can you reach as many people as possible? How can you make sure that you
reach a random sample of people? If you give up on that because it’s pretty
hopeless, how can you make sure that the characteristics you pick to build
your sample (for example age, gender, race, etc.) actually matter, are
correctly chosen and that you have the right information about how your sample
should look when it comes to those characteristics? (This problem can be
rephrased as: How do you make sure your sample is representative? It is,
however, important to unpack the complexity involved in that.) How do you know
that those who tell you they will go vote will actually vote?

That’s not a complete list of problems. As you can see there are more than
enough opportunities for error here.

Now, given Gallup’s poor performance compared to other polls I do not want to
excuse them. They screwed up, there is not question about that. I do not
think, however, that there is any need to make up conspiracy theories.

------
InclinedPlane
Take the current value of the dow jones, round it to the nearest dollar. Then
create one hundred letters, each of which says "The dow jones will be at X,
one week from now." where X is the current value plus or minus increments of 5
dollars up to +250 and -250. Then add some sort of hook to the end of the
letter such as "If you would like to learn my secrets of predicting the stock
market you can take my course for the reasonable cost of Y dollars." Then
print out 100 copies of each letter and send them to random folks, perhaps
some number of people who receive letters that end up accurately predicting
the stock market will fall victim to your scam.

That's the same problem we're dealing with here. Presidential votes only
happen once every four years, so it can be difficult to validate any sort of
model. A hundred pundits make a hundred different predictions. And dozens of
polls make different predictions as well. Does that mean that the pundits and
the polls which ended up being closer to the truth are more accurate or does
it just mean that they were lucky?

