On: The Era of 1-bit LLMs: ternary parameters for cost...

[anon373839]

It also means the largest models can be scaled up significantly with the same inference budget.

[llm_trw → anon373839]

Depends. The only paper they cite for training: https://arxiv.org/pdf/2310.11453.pdf doesn't improve training costs much and most models are already training constrained. Not everyone has $200m to throw at training another model from scratch.

[arunk47 → llm_trw]

Is there any scope for indie builders?

[llm_trw → arunk47]

Not really. These are slightly better for memory during pre-training and fine turning but not enough to make a 4090 usable even for a 7b model.