On: QuIP#: 2-bit Quantization for LLMs

[skavi]

So your experience isn’t representative of the work presented by this post? Or does llama.cpp use the same technique for quantization?

[beacon294 → skavi]

I don't know, hopefully it will help set general expectations.