Thank you for saying this. I mean, Computer Science is up there in the junk science department. Coming from a phsyics background, it's hard to even call most of what happens in CS "science".
Go to any CS conference and you'll see a whole bunch of papers doing micro benchmarks, saying that their method is X% faster than all the others, where X% is usually some very low percentage (and of course no errors bars are visible on any plots). You talk with the guy presenting the research and you find out that half the benchmarks were done on one machine, while the other half were done on another with different specs. Well, okay, now why can I trust the benchmarks? And how many other papers are doing this? And is your result even meaningful anymore? It's so depressing.
And don't even get me started on the AI/ML field. My god, talk about art over science.
A lot of AI/ML work is sloppy but well known methods work reliably, textbook descriptions make sense. That there's a long tail of shitty student papers doesn't invalidate the field. Year after year methods genuinely get better. Whatever process produces this progress, it produces this progress.
Go to any CS conference and you'll see a whole bunch of papers doing micro benchmarks, saying that their method is X% faster than all the others, where X% is usually some very low percentage (and of course no errors bars are visible on any plots). You talk with the guy presenting the research and you find out that half the benchmarks were done on one machine, while the other half were done on another with different specs. Well, okay, now why can I trust the benchmarks? And how many other papers are doing this? And is your result even meaningful anymore? It's so depressing.
And don't even get me started on the AI/ML field. My god, talk about art over science.