It's not just about hardware, it's about the software infrastructure to go with it. Other than OpenAI most researchers weren't interested in "merely" scaling things up because that was/is seen as simple engineering, unworthy of the great minds who dream up new algorithms.
