The irrationality you are referring to is called the self-serving bias (SSB). I wrote a thesis on overconfidence for my master in behavioral economics, and the two are related. SSB is a well-documented bias. For instance, 90% of American drivers think they are in the top 50%. It's hard for people to fully appreciate this fact. Allow me to illustrate. Do you still consider yourself, even after reading this statistic, to be in the bottom 50% of drivers? If so, then you're a minority, my friend ;-) (10%, to be exact)
90% (actually, about 98%) of American drivers are above average, owing to a statistical quirk. There are 6 million car accidents in the U.S. each year, out of roughly 240 million vehicles. At a minimum, that means that 97.5% of drivers get in no accidents that year. They join the big bulge of people who are "average", having perfect driving records or only an accident a decade or so.
Accident frequency is a power law distribution - a large percentage of crashes are caused by a small number of drivers who habitually violate traffic laws, drive drunk, or otherwise engage in risky behavior. One of the distinctive features of power law distributions is that there's this long tail of people with very small values, and then a few people who make up most of the curve. So (made up numbers) you might have 60% of the population who has never gotten in an accident, then 35% who has gotten in one, then a tiny fraction of 1% who's been in a dozen. Over an 80 year lifetime, that 6 million accidents/year results in 480 million accidents, or 2/person, so with the hypothetical percentages above, 95% of people are above the mean and 60% are above the median.
Just goes to show that you can't assume everything is a Gaussian. ;-)
A similar phenomenom occurs in many, many other fields. The average (median) wage-earner actually makes below average (mean) wages, because the existence of Bill Gates and Carl Icahn skews the distribution upwards. The median sale price for a startup is $0, because over half of them fail. The average test scores in Palo Alto or Weston, MA or Hunter College High School really are above average, because those places already preselect for bright kids. It's quite possible for Lake Wobegone to exist: you just need to compare your kids with someone else's average.
And none of this invalidates SSB, but you picked a bad example to illustrate it. When 90% of drivers think they're in the top 50%, they're right.
Sure, but that only works because you've chosen a discrete measure of how "good" a driver is. If you defined a "good" driver by some other measure (say, a function of risky behavior and fuel consumption), then 50% of drivers would be above the median and 50% would be below, by definition.
It's all in how you phrase the question, isn't it? Your analysis is right if we're talking about how a person's score on a driving test stacks up against the average score, gp's is right if we're talking about how people rank themselves among the population of all drivers (i.e., if 9/10 drivers think they're in the top five, then there are at least 4 delusional people).
I think nostrademons is essentially saying that the top nine drivers are indistinguishable from each other, as none of them have been in an accident. They are all "the best". Depending on how pessimistic you are, there are either no people in the top five, or nine people in the top five.
