538 indeed kicked ass. They also got all of the Senate races right, except for the Steven race in Alaska (way off, apparently, but who'd have expected a felon to win?) and possibly Franken/Coleman in Minnesota, where they gave Franken a 52% chance.
Evaluating the effectiveness by 'how many they got right' can be pretty flawed. For example with Franken/Coleman. How 'wrong' are they when something they projected as a 48% chance happens. If you had 10 races that where similarly close you would have to expect to get about 4 or 5 of them 'wrong'. You may actually find that according to their own probabilities they were somewhat 'lucky' in having so many results tip to their most favoured result. Or perhaps they could be 'unlucky' to miss Minnesota.
A better way would be to check their results against other 'odds producing' predictive mechanisms. So if a another method has Stevens at 20% and 538 has Stevens at 30% then that actually counts as a win for 538 because they were 'less wrong'. The bigger the difference in their odds the bigger the win. In the real world this would translate to 538 winning money from the other method.
I'm sure there are better and more valid statistical techniques for doing this though. I'm hoping that someone gets around to it because I'd be keen to see the results.
As Silver himself wrote yesterday, Alaska's voting pattern was nothing short of bizarre, with Young, Stevens, and McCain getting votes at 12 to 14 percent each above their pre-election polls, and turnout about 14 percent below 2004 levels.
"Indeed, it seems possible that the number of "questionable" ballots could be quite high. So far, about 220 thousand votes have been processed in Alaska. This compares with 313 thousand votes cast in 2004. After adding back in the roughly 50,000 absentee and early ballots that Roll Call accounts for, that would get us to 270 thousand ballots, or about a 14 percent drop from 2004. It seems unlikely that turnout would drop by 14 percent in Alaska given the presence of both a high-profile senate race and Sarah Palin at the top of the ticket."
Now that the "Bradley effect" has been truly buried as a meme we should start a new one: the "Stevens effect" is the unwillingness of people to admit to a pollster that they are going to vote for a crook because he occasionally brings home pork to their remote little welfare state.
What's with the Alaska hate? Oh, and that "little" state is nearly two and a half times the size of Texas. You may call it a "welfare state", but clearly, we have a very strong interest in making sure it's somewhat populated.
The Alaska hate is partly because of the Palin "horror stories" (her wanting to ban books, her abuse-of-power controversies, and her being popular despite that), and because Ted Stevens is a convicted felon who Alaska reelected.
Ah, I see what you're saying. If you judge a people based on their elected leaders, then the Bush administration reflects pretty negatively on Americans in general. But I don't judge a people on these terms: Otherwise, I'd think very poorly of Russians.
The site compiles polling data through a unique methodology derived from Silver's experience in baseball sabermetrics to "balance out the polls with comparative demographic data"[1] and "weighting each poll based on the pollster's historical track record, sample size, and recentness of the poll."
