Hadn't thought about it this way before, but given that LLMs are auto regressive...

irrelative 3 months ago | parent | context | favorite | on: The Era of 1-bit LLMs: ternary parameters for cost...

Hadn't thought about it this way before, but given that LLMs are auto regressive (use their own data for next data), they're sensitive to error drift in ways that are rather similar to analog computers.