On: QuIP#: 2-bit Quantization for LLMs

[rolisz]

If we get 1-bit quantization, wouldn't it be basically a bunch of nested if's and else's?

[patrakov → rolisz]

No. The catch is that what's quantized here is not individual weights but groups of them.

[compinter → rolisz]

So, just like the rest of my code then

[hskalin → rolisz]

So a decision tree? They are good for some tasks I reckon