Yes I’ve seen very little change over the past 3 years in my role (platform engineering) except mostly JIRA tickets are more unreadable because people just have Claude vomit a novel into it.
I mean I use it every day of course, but the velocity of work doesn’t really seem to be budging lol. And frankly a lot of time wasted on AI made gadgets, and meetings about those gadgets (look at this dashboard Claude made! etc)
Short version is that what you, I, and other responses to you are seeing is backed up by the numbers: AI code generation leads to significant bottlenecks, burnout, increased bugs, insane code churn (rewrites), and decreased feature deployment (presumably due to dealing with the bugs and bottlenecks). And as of when this was done, increased AI did not improve the situation, it generally made it worse.
The code churn stat is what bothers me the most: People keep portraying the whole "quickly rewrite/re-generate code with AI" thing as a feature, but looking at it over time, it actually means you lose your battle-tested code and no longer have a stable foundation to build on top of. That's where I think the major problems will show up as code needs to be maintained - and, seemingly, already has been.
So much of this yes, colleagues who think doing AI work is asking Claude to, as you so eloquently wrote "vomit a novel", and allow themselves to disengage their brains.
The joke is at the level of developers, the constant sharpening of AI merge-requests reviewing (sometimes too much) has made the new job of AI speedriding & reviewsfighting a serious brain squeeze.
The value of conscienciousness has skyrocketed, and the value of openness is still the same as it ever was.
This is exactly why I haven't used any of Google's AI stuff. Unlike other providers, I'm risking my email and internet service (GFiber!) to some capricious ban-happy whim you probably can't even appeal.
Only honest commits by good actors are reviewed. Good for QA but useless for security.
Cryptographic signing is not enforced for commits or reviews, so an attacker that controls a single maintainer Github API key could make a PR with a burner account then "review" and merge their own PR. Time it right and bury it in a dependency of a dependency and you are likely to get away with it. Especially considering they also do not do full source bootstrapping or enforce deterministic builds so including random binary-blobs in tree is a-okay and not sus at all.
The fact everyone is not aware of this is exactly why it is going to work, or already has.
I did once propose to the Homebrew community to enforce commit and review signing but they ultimately refused, worrying that asking people to press a button when it blinks would cause too many definitely smart and useful contributors to leave the project.
FWIW I've been saying it in a sarcastic way since I was in my teens because of the Simpsons where Bart says "It's a load bearing poster." I assume if anyone uses the phrase (that isn't like a civil engineer) it probably came from here lol
https://www.youtube.com/watch?v=QRVExJZKIT8
I only see a 1-bit quant posted on unsloth HF and it’s 72.5 GB. Is that what you mean? That’s much bigger than I expected. If you can’t run a 4 bit quant in on Strix Halo it becomes a lot less interesting.
https://huggingface.co/unsloth/Qwen3.8-Flash-Next-GGUF
In their page they say it will need at least 112GB[0], so including context, that would be a tight fit. I'm also hoping I can make a q4 fit on my 128GB strix halo
in llama-server PR 27742 it fits fine in 128GB RAM on a CPU only system , this is with --load-mode mlock to stuff the whole thing persistently into memory at llama-server launch time, no mmap
0.01.033.250 I common_memory_breakdown_print: | memory breakdown [MiB] | total free self model context compute unaccounted |
Just a hunch, but it might be because of the 51B parameter n-gram embedding. At 125B, you'd expect ~16gigs for a 1-bit quant. Add 51gigs for the n-grams and you're not far off the actual size.
If that's true, it'd scale linearly with number of bits in the quant with an offset of about 51gigs. So Q4 should be a bit bigger than 82gigs, I'd guess in the 90s (as opposed to a ~280gig q4 if the whole 70gigs of the 1-bit quant scaled linearly).
That probably includes the 51b ngrams too. It's possible that those could be streamed from NVMe on-demand. The Engram paper that developed this technique streamed from RAM to VRAM at only ~1% performance degradation, but these strix halo boxes and the spark have much slower memory, so it's possible moving down another rung on the memory hierarchy wouldn't affect their performance too much.
This will almost certainly require changes to llama.cpp or vllm to do it right.
It's not 1 bit. It's ~4bit for n-gram and ~2.8bit for the model. Not idea why it's called Q1, but likely it's preliminary quant just for PR testing / very likely to be remade after llama.cpp support is merged.
I mean I use it every day of course, but the velocity of work doesn’t really seem to be budging lol. And frankly a lot of time wasted on AI made gadgets, and meetings about those gadgets (look at this dashboard Claude made! etc)
reply