On: Scalable MatMul-Free Language Modeling

[sva_]

The FPGA in question, Intel FPGA PAC D5005 seems to cost around $8k

[mmoskal → sva_]

The FPGA would be only to prove the transistor efficiency. If this works people will eventually do ASICs (possibly inside the GPUs a few generations ahead).

[ein0p → sva_]

That’s still 1/4th of an H100.