lemonade stands are staffed by 11 year olds and there can be one on every corner.
these stands do better than cerebras, which is an 11 year old deeply unprofitable company building a technology that no other company has bothered to try to even get close to replicating.
Cerebras got lucky with the ban on NVIDIA chips due to "security reasons" in the middle east, securing a massive deal with the UAE's sovereign fund. Of course, the leaders made a nice donation to the Trump crypto fund and coincidentally the ban on NVIDIA was lifted. That presented a danger for Cerebras, but they also pivoted almost entirely to an inference as a service company, maximizing tokens/sec metrics on a variety of open weight models. Then they landed the contract with OpenAI which has bought them another lifeline, and helped paper over the trail of fraud that delayed the S1 and IPO for a few years. Fascinating story indeed!
The problem with the pivot is that with only 44GB of memory, you can't run models unless you nerf them or you scale the hardware out horizontally, which takes up a lot of space, power and even more cost.
This is why their API has dumb limits on it. I'd estimate their $/tok cost around $4-5000/mtok based on hardware alone, which is insane. Sure, it is fast, but if it is heavily quanted, expensive, and can't scale, you're going to be in a world of pain.
