On: The Era of 1-bit LLMs: ternary parameters for cost...

[sebzim4500]

If consumer cards can run the big models, then datacenter cards will be able to efficiently run the really big models.

[leroman → sebzim4500]

Some tasks we are using LLMs for are performing very close to GPT-4 levels using 7B models, so really depends on what value you are looking to get.