This is a great lesson for hackers in general. Many of us, myself included, have a tendency to expect data to automatically win arguments. I mean, if something is irrefutably true with the data to prove it, that's the end of the discussion, right? But just as in this case, in reality, it's not even the beginning. Whether you're trying to convince your manager to go in a particular direction, or you're trying to motivate your employees to rally behind a cause, the data will help you decide where to go, but you need the right story to get people to follow you there.
And just because it's a "story" doesn't mean it's some BS you make up to get people to agree with you. It's more about considering the other person's point of view (their poorly-documented public API, if you like), and building an interface between that and your data.
The best manager I ever had was great at this. He was a former developer who understood the tech but knew how to speak the executives' language. You could convince him with a good tech story and then he'd turn around and sell to his bosses with a good business story.
Agreed! I think this also cuts to the fact that data doesn't win arguments. Data + time spent studying and understanding data wins.
Most people can't or don't want to take time (it's inefficient). So they make decisions based on estimates / stories / approximations / biases / gut and the world keeps spinning.
Which means your goal should be to most accurately frame your data in terms of their stories (translating as much or little as they require).
I like that this captures something very important about Data Science. The data may give the truth, but it's the story that makes people change. The story taps our emotions, which cause us to change behavior faster than hard data.
The chief analyst at a prior employer once commented that he liked people "with degrees that suggest storytelling" in addition to their math/science/engineering pedigrees.
The story taps emotions, and impacts decisions better than just raw data. This is why politicians try to attach names and faces to policies. Good leaders use the data to inform the decisions, but stories to explain them.
The idea that a baseball team could ever go undefeated for even half a season is ridiculous. No amount of data analysis can make you do that.
That would be like saying that a perfect poker player could win every hand. Even if your opponent was playing with their cards face up, sometimes he is gonna hit his inside straight on the river.
For those curious, they ended up going 18-22 in the second half of the season. This was still good enough for a second place record of 44-33 overall, out of four teams. In the playoffs against the first place San Rafael Pacifics, they came close but lost 4 games to 3.
Yes, there is a lot of luck. For one thing, the gap between very bad players and very good players is fairly small in an absolute sense; for example, a player who gets a hit 25% of the time would have trouble staying in the league, but a player who gets a hit 30% of the time has a good chance of being an all-star. A 5% increase in your success rate separates the very worst players from the very best.
There is a great saying about baseball: Every team is going to win 54 games and every team is going to lose 54 games; it is what you do with the other 54 games that counts. (There are 162 games in a season)
Isn't a 20% skill difference pretty meaningful in most endeavors? Maybe not the difference between unemployment and rock star but being 20% better at sales for example can be quite lucrative.
The league wide batting average in the MLB was .253 in 2013 (first Google result, I'm assuming it doesn't vary all that much between seasons). While you are correct that hitting 0.6% compared to 0.5% doesn't mean much the numbers I responded to were meaningful in an MLB context (.250 vs. .300) so I think it's justified to use relative differences. In fact I think it's more insightful than using absolute numbers. Speaking of a 5% increase in that context is confusing (at least to me :D) when in reality the skill is roughly 20% above league average.
A programmer who is 20% above average will certainly command a premium as will a salesperson.
As I understand it, a baseball player who performs at the average level is already close to being a superstar; most players are below. This is the reason baseball statistics assess players in terms of "wins above replacement" rather than comparison to the average -- it would be very difficult, if you fired a player, to get an average player to fill your vacancy. (You may wonder how this could be, if baseball ability is normally distributed. Baseball ability is normally distributed in the general population, but not among professional baseball players.)
The comment you responded to was making the point that it's massively unrealistic to expect a team to have a win rate too far from 50% because the absolute difference in skill among all players is small (which means that the skill differences are easily overcome by random variation). Relative differences between players aren't relevant to that argument; you'd need to be talking about the relative difference between the size of the absolute skill gap and the effect of day-to-day variation.
Practically everything in the world more complex than basic physics is non-deterministic, especially who wins games of skill. Things occur or don't occur with probabilities.
Even if you have a 95% chance of winning each game (unrealistic IMO), the probability of winning 81 games in a row is under 2%.
This is less apparent in other sports than in baseball, not because the outcomes of those games are more deterministic, but because fewer games are played per season.
You are a bayesian analysis with strong priors on the chance of winning. I'm not objecting to your critique (as I don't know much about sports), I just keep thinking of P(data | model).
Well, there is no sport or other organized competitive activity I can think of where the best in the world beats the second best in the world more than 95% of the time.
Rugby Union. I'd have to look at the exact numbers, but the New Zealand All Blacks have an amazing win rate. They do play the second best team in the world (either Australia or South Africa) on a regular basis and over the last four years have only lost once or twice. Won the last two World Cups.
Edit: Here's the data[0]. A 76% win rate since 2003. Twenty-two game winning streak in 2013. Never ranked lower than 3 in the world since 2003.
As a proud New Zealander it pains me to point out that the All Blacks still fall way short of the benchmark set by the parent comment - i.e. beating the second best 95% of the time.
[Edit: just realised the original point was talking about beating the world's second best, not any opposition, so the All Blacks still don't come close to the 95% in the context originally mentioned]
That made me laugh (I'm a kiwi), but it's not just a feeling, it's not far off the truth. With Steve Hansen as coach the All Blacks' have won a staggering 92% of their games[1].
There's plenty of articles talking about how their dominance has increased through the professional era[2]. I'm not sure how likely it is to last though given just how many of our key players have left since the world cup, and historically I've NEVER felt confident of the outcome any time we play Australia.
The last time the All Blacks lost to Australia, in New Zealand, was 2001. That's 18 tests at home without loss to the Wallabies.[0] Of course, Australia hasn't been ranked second best in the world that entire time.
Against the Boks it's six home games all won, since the last loss in 2009.[1]
Overall, home and away, not quite the 95% originally suggested, but still incredibly dominant in the age of professional team sport.
Competitions for "best score" (where score is a time, a number, etc) fall into a different category of sport than those where competitors are facing off head-to-head and there is interaction between competitors. And among the "best score" competitions, those which there is little variance in outcomes will result in a much higher win rate of "best" versus "second best" (a powerlifter or sprinter's score may only vary a few % between good and bad days. a bowlers' or golfers' can be much higher).
Good point. I'd guess that's true of stuff like weightlifting also. These sports that are about pushing your body as far as it can go in one dimension are a lot simpler and "purer" than a team game with lots of moving parts like baseball.
Off the top of my head, here's a list of reasons that may tip the scale of an outcome in favor of the worse overall team in a baseball contest, whether luck or other factors. (the first two are by far the most important, and are in general order of how much variability each factors to a game):
- sequencing (3 walks, 3 strikeouts, and a home run in an inning can end with 1 or 4 runs depending on sequencing)
- ball-in-play luck (batters have little-to-no control over whether their line drive finds a glove for an out or a gap for a double)
- pitcher/hitter matchups favoring one team or another
- other luck over a single game (2+ errors in a game, 3+ home runs in a game, etc) that are well above the average for any team, but any team will have many of these games over the course of a season
- umpire strike zone favoring one team's pitchers
- umpire calls consistency
- park/weather favoring one team's players
If you need any explanation on any of these I'd be happy to provide it. I'll add more if I think of them, but this should give you a pretty good idea of why the best teams in baseball usually end up with a less than 60% win record.
cortesoft is on track by pointing out how small the gaps are (in absolute terms) between bad, average, good and very good in baseball. And just as importantly, the gaps have been shrinking. Stephen Jay Gould famously wrote about this in explaining why we don't see .400 batting averages anymore (that's a hit 4 out of every 10 times a player bats; it last happened in the major leagues in 1941).
The average major-league player of today is significantly better than the average player of seventy years ago, so there are fewer opportunities for the best player in the league to take advantage of large numbers of truly-outmatched opponents and achieve inflated performance numbers. The highest batting average in the modern era of baseball is .426 by Nap Lajoie in 1901. 2015's highest batting average in the major leagues was .338, and it was .341 the year before; even an average of .360 or better is now rare (only happened eight times since the year 2000).
And overall team performance is affected in the same way. The 1906 Chicago Cubs own the best win percentage in major-league history at .763 (116 wins out of 152 games played). That year the worst team (the Boston Americans, predecessors of today's Boston Red Sox) went 49-105, a win percentage of .318. Last year (2015) the best win percentage was .617 while the worst was .389. Only four major-league teams failed to win at least 70 games last year, and over half managed to win at least 81 games (81-81 being an even .500 win percentage with the season now 162 games long), while at the top there were seven teams with at least 90 wins, and four of them had at least 95 wins.
Enough of it's luck that it's unlikely that a team would go undefeated for half of a season, but it's also the fact that it's hard to get a team with only the best players; most teams have a mixture of player ability. If you put the best major league team in the minor leagues they'd probably have a 90%+ winning percentage. (Highest MLB season winning percentage is 76%, longest winning streak is 26 games out of 162, including a tie.)
Two additional points. 1) The mixture of player ability is mostly due to cost penalties for paying players a higher salary (although Yankees just pay it). Not as distributed as (american) football, but still distributed. 2) Note the longest winning streak is 26 games out of 162... out of 162. Longest basketball streak is 33 games (not enough for an undefeated 82 game season). The ability to have a 162 game streak that starts on game one and lasts through the baseball season is essentially impossible. It is possible to have an undefeated season in american football where the season is 16 games (but it is rare -- hasn't happened since pre-NFL 1937, 1948). Baseball has an order of magnitude more games per season.
EDIT: I knew that wasn't right (that's what I get for haphazardly scanning Wikipedia for facts). A couple others since recently 2007 Patriots and 1972 Dolphins also had undefeated seasons in NFL. Still rare, and still a fraction of the number of games. Point is: luck is only part of it, no matter the sport staying undefeated is difficult -- especially for a season when season means 162 games! Thanks for pointing out the mistake everyone. Sorry for the confusion!
It would be interesting to see some scouting models based purely on psychological and personel relationship data. What kind of metrics are best at predicting team chemistry. Sports analytics is going to complete change most sports, we are just getting started.
A different angle is player health. The Golden State Warriors are known for resting top players when they're in Denver because they found it can take a player up to a week to fully recover from the effects of playing at altitude.
This reminds me of Football Manager (video game series). It's a turn based moneyball-sort-of game for soccer/football. Players have stats, potentials (which need to be scouted), training, etc. But there are other relevant details like how a player is likely to get along with other players that affect game outcomes.
For us nerds who want to effect change, and who are convinced data should be all you need, the book "Switch:
How to Change Things When Change Is Hard" by Chip and Dan Heath is a highly recommended read. It'll tell you all about the story, or as they call it: the elephant.
And just because it's a "story" doesn't mean it's some BS you make up to get people to agree with you. It's more about considering the other person's point of view (their poorly-documented public API, if you like), and building an interface between that and your data.