On: Bonsai 2 27B: Near-Lossless Compression in a 9x Smaller Footprint

[raylad]

How does that compare with the bf16 version?

For my "Please recite Jabberwocky" test the bf16 almost passes but the ternary and even fp8 versions fail badly.

[ctolsen → raylad]

Definitely a little better.

https://gist.github.com/ctolsen/b2883e7cbf5e4357fa04366019e6...