1. Just wait a few years for better hardware. Not even joking on that one. The computational requirements of most modern GAN and transformer-based approaches, like the StyleGAN image generation I was playing around with recently, would have been daunting even on the largest supercomputers just 10 or 15 years ago and before that it was science fiction.
2. Find more shortcuts. We all knew neural nets could learn any function all along, in principle. If you have infinite time and space, some of the simplest multilayer feedback networks can learn anything learnable. I don't think anyone really expected to find such relatively efficient algorithms for that, like we have in the last few years, though.
I’m only running one Titan RTX which is slower than a 3090 and you can parallelize the system by buying multiple graphics card or by increasing the amount of VRAM used on the endpoint and buying an Ampere .
