On: Web LLM – WebGPU Powered Inference of Large Langua...

[verdverm]

Check out LLAMA-CPP

[cahoot_bird → verdverm]

Looks like this was hacked together pretty quickly. This in CPU is exactly what needs optimized to run on more devices, if that's even possible..

I guess it will take hardware and software a while to catch up to compete with ChatGPT..

[verdverm → cahoot_bird]

If you look at the news, yes it came together quickly, but it has also gotten a lot of performance upgrades which have made significant improvements.