Alright, I know, the title is on the "click bait" side. But hear me out.
Researchers in the UK rely on High Performance Computing systems to do their work. If you combine the top 8 HPCs in the country, they amount to 1,000 GPUs altogether. For context Meta alone used 24,000 just to train one of its models (LLaMa). To make things worse, those HPCs are geting to the end of their life without plans for renewal.
With the growing size of AI models (both in vision and language) and the slow pace of hardware improvement, this is only going to get worse. Meta, xAI, OpenAI know this and they are hoarding resources (hundreds of thousands).
How are researchers, independent developers and AI/MML hobbyists going to compete in the space? Do we just submit to our techno-feudal lords and pay rent to access crumbs? Is it over?
Does anyone else care about this?
I'd love to hear people's views on this. Disclaimer: I am working on a solution and I am reaching out to the community to learn more. Check it out if you are interested: https://github.com/kalavai-net/kalavai-client
1. Diminishing returns on a model's capability with respect to scale means that, even if big tech datacenters grow at a faster rate than infrastructure available to individual researchers, the gap in performance between their models will narrow - with big tech needing to invest increasing resources even just to gain a shrinking lead
2. Many applications of AI don't need the latest massive LLMs. Some defect detector may reach 99.9% accuracy at which point further work gives negligible improvement, and the hardware cost to reach that point is steadily decreasing - putting it in range of more individuals/small companies/etc.