TLDR is they make an ensemble of models by training once, and once model is trained cycling between raising and lowering learning rate to end up in different local minima, and just snapshot the model at those different local minima.
This strikes me as very cool, their final numbers on the dataset are quite an improvement over one model... neat idea.
This strikes me as very cool, their final numbers on the dataset are quite an improvement over one model... neat idea.