94x cost improvement over GPU and 15x TPU is insane, but fits right in there with performance gains seen in Moore's Law.
This development presents a more compelling case that we are in fact on the precipice of larger LLMs being able to serve everyone for cheap. Still not really convinced by the AGI argument, but this does spook me. Overall though very cool.
It's insane because it is theoretical. They haven't shown that it works, think of this paper as a prelude to a funding round or research grant so they have to show some kind of advantage. Which I'm highly skeptical of, usually when papers show this kind of improvement over SOTA it tends to be either a mistake or purposeful nonsense.
Group at my school recently got a grant for 10MM for such a fantasy. All they had was an ISA - no RTL, no functional model, no compiler. Kid in my group (co-advised) is busy scrawling assembly on notebook paper lol. Suffice it to say I don't have high hopes for a tapeout anytime soon.
This development presents a more compelling case that we are in fact on the precipice of larger LLMs being able to serve everyone for cheap. Still not really convinced by the AGI argument, but this does spook me. Overall though very cool.