om8's comments

On: Smaller, faster, safer: running Kimi and GLM at scale

[om8]

Why int4? There are a lot of superior 4 bit formats like nf4 from bitsandbytes.

[Tepix → om8]

Well, INT4 is good enough apparently and it's very fast.

On: It's getting harder to focus every day

[om8]

> So how can I regain my ability to focus?

Getting diagnosed with ADHD and taking meds did the trick for me.

Not sure how relevant this is, since inability to focus was always an issue for me.

[mark_something → om8]

I don't think that the author has ADHD, because they used to be more attentive in the past. In the case of ADHD, it was also the case as a child.

I also got the diagnosis of ADHD (inattentive type) a few months ago, in the interviews they asked whether I also had these symptoms as a child. Not easy to figure out because as a child things like finishing things on time are not so important yet, and my high IQ compensated for a lack of focus. Only in secondary school it became a problem, and my grades dropped.

On: 1-Bit LLM in the Browser

[om8]

This project needs webgpu -- I did it on cpu about a year ago.

My demo uses 2 bit quantization to run llama3 models on any device with enough ram.

https://galqiwi.github.io/aqlm-rs/

On: 2026 Unslop AI-Written Fiction Contest Results

[om8]

> manufacturing companies making the flimsiest, cheapest, plastic crap to save 1/3 of a cent on every mop they produce. Designed to work for the least amount of time before needing replaced

We live in a world with such companies, and we can still buy quality things. If there is a demand for the purely-human generated texts, they will be around. Perhaps a lot of people around you will read ai text instead, and you'll get upset because of it, but it's their choice. You'll still have your thing

[8n4vidtmkvmk → om8]

I don't know that we can have nice things. If two companies produce a similar widget but one is higher quality in no visible or articulable way... Which one will sell better, the cheaper or more expensive one? What if we as consumers can't really definitely tell when one is prone to failing in 1 year instead of 5? It takes too long to find out and by then the more expensive one is underselling and forced to enshittify.

[flumpcakes → 8n4vidtmkvmk]

I think it's worse than that - the AI slop low effort cash cow is using deception (as well as theft). For example: https://www.youtube.com/watch?v=PUSY6mtqQDI

On: US battery manufacturing output continues to break records

[om8]

Cool way to think about GWh/year:

  1 GWh/year = (10 ** 9) / 24 / 365.25 / (10 ** 6) MW = 0.11 MW

  70 GWh/year = 8 MW
  1755 GWh/year = 200 MW
  252 GWh/year = 29 MW

[amalcon → om8]

Haha. Reminds me of how volt-amperes are technically the same unit as watts, but if you see VA in an electrical specification you know it means a different thing than it would if you saw W.

[om8 → amalcon]

> volt-amperes are technically the same unit as watts

volt-amperes are joules

[gnabgib → om8]

Volt-amperes are watts... watts/second are joules

https://en.wikipedia.org/wiki/Watt

[lefra → gnabgib]

You got it backwards, Watts are Joules/second (or joules are Watt•second).

[om8 → gnabgib]

Oh, yes, you are right about watts. My bad

[Skunkleton → gnabgib]

Watts * seconds are joules. Joules is a unit of energy. Watts are power.

[epistasis → om8]

But not a very relevant for batteries, unless talking about discharge only once a year.

Grid batteries are discharged on average 80% per day, if not more. EV batteries... well, probably about 5%-10% per day at most.

On: Copy Fail

[om8]

cargo/uv/go have lock files though

[dnnddidiej → om8]

with curl | sh you could use a checksum you download with curl!

On: TurboQuant: A first-principles walkthrough

[om8]

https://docs.vllm.ai/en/v0.20.0/api/vllm/model_executor/laye...

`vllm.model_executor.layers.quantization.turboquant`

> The technique implemented here consists of the scalar case of the HIGGS quantization method (Malinovskii et al., "Pushing the Limits of Large Language Model Quantization via the Linearity Theorem", NAACL 2025; preprint arXiv:2411.17525): rotation + optimized grid + optional re-normalization, applied to KV cache compression. A first application of this approach to KV-cache compression is in "Cache Me If You Must: Adaptive Key-Value Quantization for Large Language Models" (Shutova et al., ICML 2025; preprint arXiv:2501.19392). Both these references pre-date the TurboQuant paper (Zandieh et al., ICLR 2026).

[amitport → om8]

Those works did cite DRIVE/EDEN :)

HIGGS is an extension of EDEN (using the well known method for blockwise Lloyd-Max).

The proper framing of this "TurboQuant" layer in vllm (which does not include JQL) is precisely EDEN 22 without the scale correction.

[kumarhn → amitport]

EDEN is clearly relevant prior work for HIGGS. But reducing HIGGS to “an extension of EDEN” seems unfair to the authors of HIGGS. Similar primitive, different problem setting, different constraints, different contribution.

Curious: where do you draw the line between “related prior work” and “an extension of EDEN”?

[amitport → kumarhn]

In the vLLM documentation quoted above, TurboQuant (which is a restricted version of EDEN) is referred to as a specific case of HIGGS. Note the symmetry: EDEN acts as a special case of HIGGS; hence, HIGGS functions as a generalization of EDEN.

In any case, the quantizer is indeed an extension, regardless of whether it was explicitly framed that way in the paper. I say this not to diminish their contribution at all, but just to clarify the relationship, as it was also stated in the vLLM doc.

On: TurboQuant: Redefining AI efficiency with extreme compression

[om8]

These are very different media types with very different goals.

On: Claude Opus 4.6

[om8]

Is there a way to disable it? Sometimes I value agent not having knowledge that it needs to cut corners

[nerdsniper → om8]

90-98% of the time I want the LLM to only have the knowledge I gave it in the prompt. I'm actually kind of scared that I'll wake up one day and the web interface for ChatGPT/Opus/Gemini will pull information from my prior chats.

[pdntspa → nerdsniper]

They already do this

I've had claude reference prior conversations when I'm trying to get technical help on thing A, and it will ask me if this conversation is because of thing B that we talked about in the immediate past

[sanxiyn → pdntspa]

You can disable this at Settings > Capabilities > Memory > Search and reference chats.

[hypercube33 → nerdsniper]

I'm fairly sure OpenAI/GPT does pull prior information in the form of its memories

[nerdsniper → hypercube33]

Ah, that could explain why I've found myself using it the least.

[vineyardmike → nerdsniper]

All these of these providers support this feature. I don’t know about ChatGPT but the rest are opt-in. I imagine with Gemini it’ll be default on soon enough, since it’s consumer focused. Claude does constantly nag me to enable it though.

[sumtechguy → nerdsniper]

Had chatgpt reference 3 prior chats a few days ago. So if you are looking for a total reset of context you probably would need to do a small bit of work.

[sharifhsn → nerdsniper]

Gemini has this feature but it’s opt-in.

[kzahel → om8]

Claude told me he can disable it by putting instructions in the MEMORY.md file to not use it. So only a soft disable AFAIK and you'd need to do it on each machine.

[jsw97 → kzahel]

I ran into this yesterday and disabled it by changing permissions on the project’s memory directory. Claude was unable to advise me on how to disable. You could probably write a global hook for this. Gross though.

On: Linux Kernel Rust Code Sees Its First CVE Vulnerability

[om8]

https://steveklabnik.com/writing/does-unsafe-undermine-rusts...