On: Vision Transformers Are Overrated

[whimsicalism]

idk, sounds more like they just architecturally overfit to imagenet - especially if they evaluated before doing each of their arch changes

also this is low data setting