On: New CRAM method offers giant boost to compressed memory reads

[madduci]

Finally we can run frontier models locally

[sroussey → madduci]

We brute force AI models right now because a) we don’t know better, and b) it’s premature optimization.

I beg to differ on point b, but no one is delaying their next model just so they can concentrate on optimization.

It’s coming though.

One example: https://siliconangle.com/2026/07/28/ai-model-compression-sta...

Another is separating the the intelligence part of the model from the known facts part of the model (which can be better compressed)

[djmips → sroussey]

The Chinese breakthroughs are often around optimization. And I gather it's because they are constrained and very bright so it's a natural progression.

[ranger_danger → djmips]

IMO Restrictions are exactly what sparks creativity in people... whereas too many options leads to choice paralysis, which is what I fear is a big problem in the FOSS world.

Don't get me wrong... I think having choices is still good, I just think that having too many choices is not equally as good, or necessarily better.

[throawayonthe → madduci]

one would hope model weights are already high entropy enough and would not be improved by a general-purpose memory compression algo?

[ranger_danger → throawayonthe]

It turns out that there is still lots of room for improvement: https://huggingface.co/prism-ml/Ternary-Bonsai-2-27B-gguf

[tintor → madduci]

Compression doesn't really work for model weights.

Model quantization and model distillation are two techniques to reduce model size.

[variadix → tintor]

https://blog.cloudflare.com/unweight-tensor-compression/

[madduci → tintor]

It was sarcastic