Wow, this is a really cool project. Just one thing that I was curious about... At the end there was talk about elo rating and their uncertainties, but I wasn't sure how those were computed and if those were determined to be statistically significant. The dubious benefits of the compile times seem independent from an ELO rating and so I think a statistical test to ensure the significance of the results would be good to further drive the point home.
Apart from the ELO claim, the runtime speedup is sure beneficial at the very least! Cheers to the author
Apart from the ELO claim, the runtime speedup is sure beneficial at the very least! Cheers to the author