On: The Era of 1-bit LLMs: ternary parameters for cost...

[GaggiX]

For sure bigger models are needed to compete with transformer LLM, same thing for Mamba, I was just bothered by the distrust about something very reasonable like not being able to fully train a 70B model.