I was a moderately successful scientist who reorganized his career around applying machine learning to medicine. After working in the field for a few years, I noticed that all the folks getting attention for their papers were basically publishing bullcrap. I have a high bar for publishing, I don't publish stuff that is bullcrap. But my competitors were, and they had a serious advantage over me (because most people couldn't recognize the bullcrap).
ML folks today are under immense pressure to show small-digit improvements over existing methods and they're using all the wrong techniques (p-hacking, massive hyperparamter runs, press releases that overstate the impact). It's really sad to see all the snake oil.
ML folks today are under immense pressure to show small-digit improvements over existing methods and they're using all the wrong techniques (p-hacking, massive hyperparamter runs, press releases that overstate the impact). It's really sad to see all the snake oil.