> It’s striking that so many stories about the triumph of metrics over intuition involve games. I’m convinced that you should put statisticians in charge if your goal is to win a game.
I enjoy playing around with baseball data. About a year before the pandemic, I went out and presented some research at a Sabermetrics conference hosted by the Boston Red Sox. 90% of the pro teams had their data science teams present, and I spoke with many of them (FYI - working in baseball is like working in video games: long hours and low pay).
Anyway... The Red Sox (at that time, at least) divide their analytics department into 3 subject matters, and I spoke to the lead of one of those 3. I asked how different their analysis is compared to all the old baseball knowledge developed over the last 150 years.
He said that it's not different! All the intuition about the game built up by the old-timers spending decades in the industry still holds! The analytics is all about tweaking things and making slight improvement in the odds. If they can improved the odds of a beneficial outcome by 2-3%, they consider it a success, and if they can do that consistently over the course of a season they will win 5-10 extra games. That's it! (SPOILER ALERT) Remember that in the movie Moneyball, the Oakland A's didn't win the World Series. They didn't even make it to the World Series. They were beaten in the first round of the playoffs by another small market team (Minnesota) with a similarly small payroll that wasn't doing the moneyball thing ;)
It’s interesting that your anecdote lined up so well with the example from the OP about people claiming that the metrics didn’t change much. Though those people were then giving ‘supporting’ examples that support the opposite of their statement. It may be that the person you talked to worked on eg play strategy which could have been affected totally differently from scouting.
I was intentionally vague about who I spoke to, to keep some amount of anonymity for him ;) The question was about baseball in general and not a specific aspect of it.
As far as play strategy goes, it's actually really interesting that baseball does not allow for real-time analytics in the game. You can print out some cards and give them to players to keep in their pocket (baseball uniforms have pockets!) and they can refer to them during the game. You can give a binder full of information to the coach and he can keep it in the dugout. But once the game begins, you cannot communicate strategy to the players and coaches. That's quite different than most sports.
What is interesting about scouting/player development vs play strategy is that the "revolution" in scouting started 100 years ago! Branch Rickey, most famous for signing and playing Jackie Robinson with the Brooklyn Dodgers, began creating the modern farm system for St. Louis in 1919 - with a lot of derision from other teams. But funny thing - from that point on (until he left for Brooklyn) nobody but the Yankees won more games than the Cardinals. The Chicago Cubs and Philadelphia Phillies at various times hired people to collect data about young players to use for skill development and practice recommendations, as well as personnel decisions. The shape of a guys butt and the attractiveness of his girlfriend are actually data driven heuristics!!!
I enjoy playing around with baseball data. About a year before the pandemic, I went out and presented some research at a Sabermetrics conference hosted by the Boston Red Sox. 90% of the pro teams had their data science teams present, and I spoke with many of them (FYI - working in baseball is like working in video games: long hours and low pay).
Anyway... The Red Sox (at that time, at least) divide their analytics department into 3 subject matters, and I spoke to the lead of one of those 3. I asked how different their analysis is compared to all the old baseball knowledge developed over the last 150 years.
He said that it's not different! All the intuition about the game built up by the old-timers spending decades in the industry still holds! The analytics is all about tweaking things and making slight improvement in the odds. If they can improved the odds of a beneficial outcome by 2-3%, they consider it a success, and if they can do that consistently over the course of a season they will win 5-10 extra games. That's it! (SPOILER ALERT) Remember that in the movie Moneyball, the Oakland A's didn't win the World Series. They didn't even make it to the World Series. They were beaten in the first round of the playoffs by another small market team (Minnesota) with a similarly small payroll that wasn't doing the moneyball thing ;)