Where are you seeing dense? Most of the larger competitive models are sparse. Sure, the smaller models are dense, but over 30B it's pretty much all sparse MoE.
And there are still plenty of hybrid architectures. Nemotron 3 Super 120B A12B just came out, it's mostly Mamba with a few attention layers, and it's pretty competitive for its size class.
But yeah, these different architectures seem to be relatively small micro-optimizations for how it performs on different hardware or difference in tradeoffs for how it scales with the context window, but most of the actual differentiation seems to be in training pipeline.
We are seeing substantial increases in performance without continuing to scale up further, we've hit 1T parameters in open models but are still having smaller models outperform that with better and better training pipelines.
The whole point of the California/Colorado laws is to provide an alternative to that. The whole point is that it provides a privacy preserving way to provide a signal about whether someone is in a particular age bracket, without requiring any kind of third party ID verification.
I am so puzzled by everyone who objects so strongly to these operating system based opt in systems; all it does is provide for a way for a parent to indicate the age of a child's account, and an API for apps and browsers to get that information. If you're the owner/admin of a system, you get to set that information however you want, and it's required that it only provides ranges and not specific birthdays in order to be privacy preserving.
Meta being behind all of these efforts makes it incredibly suspicious, especially given the New York law is ridiculously more invasive than the California one. It sure makes it seem like there's likely a larger plan here that this is merely facilitating.
So I don't think I can still buy it at face value that California's version is a good-faith attempt to balance privacy and child safety, even if that's what it is in the eyes of the legislature, given who's actually behind it and what else they've been pushing for.
Just because Facebook supports it doesn't mean it's bad. They may not support it for the same reasons, they probably just don't want the cost and liability of doing identify verification themselves and so want to make sure all of the cost and liability is on the OS vendor.
Yes, the New York proposed law is far worse, and we absolutely should be pushing back against that. And Facebook doesn't care, because they only care about moving the liability onto the OS vendor, not on actual privacy.
But still, just because this was supported by Facebook doesn't make it bad. Sure, Facebook doesn't care about privacy, but they do care about not being liable for this, and in this case, they're right, it is actually much more efficient to centralize this function in the OS, and it happens that that way it can be done in a privacy preserving way as California's law shows.
It doesn't make sense to move this function to the OS because so long as the OS remains under the user's control, any signal from the OS has no value because the OS reports whatever the user wants it to report.
At any rate, why legislate operating systems when all of the harm comes not from computers themselves but rather from certain websites? And there are already mature solutions for controlling access to specific websites. Client-side parental controls for internet access have existed for decades, dating back to Surfwatch from the Win95 era. A credit card requirement would also effectively impose an age filter.
The law acknowledges that. It doesn't require actual verification. That's why people are saying it's just a parental controls law and not an ID verification law.
> I am so puzzled by everyone who objects so strongly to these operating system based opt in systems
The government legislating APIs is an uncomfortable precedent given the culture wars that are raging right now. There seems little reason to expect this will stop here.
They are not legislating specific APIs. They are legislating that an API has to be provided, just like other laws legislate that you have to provide accessibility APIs, but the details of the APIs are left up to the companies.
I work in aviation, a highly regulated field. And that's a good thing. It does take some work to regulate well; there has been a migration in aviation to more prescriptive regulation about how things need to be, to less prescriptive like what the ultimate performance needs to be. But yeah, the aviation regulations aren't that you have to implement something a specific way, but that you have to be able to show that your aircraft has no more than a certain probability of catastrophic failure (where the probability varies base on certain things like the size and type of aircraft).
For this age verification law, all that is required is that there is an API provided for this purpose, and there is a way for the owner of the machine to set up user accounts with age information indicated, and that the APIs need to provide several rough age ranges, not specific birthdays.
Years later: "The current measures are a step in the right direction, but we have found them insufficient. We are now requiring the use of this specific proprietary binary blob for any action related to the verification process. It will conveniently run as a daemon so its exposed API will be accessible to any application that needs to query it, and it will automatically update itself so you don't have to worry about it, just set it up once and forget about it."
It might also include some additional text like "we have decided to collaborate with systemd to integrate this proprietary binary blob, to maximize the reach and eliminating any pains in the setup process caused by the vibrant ecosystem of package managers, while at the same time avoiding disrupting the development process of the Linux kernel".
We shouldn't object to a reasonable law just becasue it might, theoretically, pave the way to an unreasonable law.
In fact, this is put in place as an alternative to the kind of law being enacted elsewhere, right now, which is much worse; the ones requiring ID based verification for accessing many online services. This one provides an alternative solution, which is far more privacy preserving, and leaves all of the actual power in the hands of the owner of the computer.
BS. Does TempleOS support it? What about Plan9? MenuetOS?
Are these illegal operating systems?
Either you or someone else mentioned this talking point the other day, I asked for even a single example of an OS maker being sued over this successfully, and I got nothing.
I'm confused. What's the age definition of child? 12, 15, 18? Does this mean its against the law for children to install an operating system? What is the penalty for a child doing this and putting the wrong age or just doing it at all? What is the penalty for a parent or guardian of the child that does this? What happens to the parent or child if the child circumvents this control? Will child services be involved? Criminal penalties? Of course the only way to know an adult is the administrator is to tie the users government I'd to the account. Could this be done in some zero knowledge anonymous way? Sure, but I don't think it's likely. This seems to be the thin end of yet another wedge. The trend seems to be to be that we should be identified and survield every moment of our lives. The question is who does this surveillance serve? How much access do you have to your government or employer's data or advertisers or educators or ...? How does their access serve you?
It requires that operating systems provide a way, at account setup, to specify the age or birthdate of a user, and provides an API for indicating which age range the user falls in (under 13, 13 to 16, 16 to 18, or over 18) to an application, so the application can use that information to comply with any laws or regulations relating to the age of the user.
It doesn't make any requirement that the parent actually truthfully put that information in. It doesn't require that anyone verify the information. It doesnt provide for any requirement that a child not set up a user themselves. It explicitly calls out that there is no liability on any of the parties if one user uses a computer under another user's account.
So all it's doing is saying that there must be a reasonably accessible mechanism for a parent to indicate a child's age so that rough information about which age range the child is in can be provided.
Now, is it perfect? No.
It does seem a bit over broad as there are lots of things which be classified as computers uner this, like routers, smart TVs, graphing calculators, cars, etc. Having to provide account setup with age and an API to accesss it in all of these environments could be a bit of a lift in the time frame given. And it doesn't leave a lot of time for something like standardization of Unix APIs between operatings systems, so for systems not running graphical environments I'm sure we're going to get a bunch of different solutions from different OSes as everyone sticks it in a different place and provides a different way to access it. And this would need to be a new feature added into long-term supported maintenance releases operating systems.
So yeah, could it have been done better? Yes. Is it likely that they are actually going to fine OpenWRT developers if they don't implement this? I doubt it; it's pretty clear that the legislative intent is desktop and phone OSes, and other mass market consumer oriented devices that might offer app stores.
So yeah, I see some issues, but overall this seems like the right way to do things; just provide a way for parents to set an age on their children's account, and then provide that to any apps that might need to do age verification. That's it.
You put a lot of effort into understanding it. Will Docker images need API passthrough? Will Debian need to solve its location for the purposes of deciding its legal exposure?
I don’t see why we should burden OSes this way. An App Store does all that better.
That's a very long list of questions, most of which you wouldn't need to ask if you spent ten minutes reading the law. And the rhetorical point you seem to be working toward is much less effective when more than half of those questions evaporate.
Because it's inverted. If it's opt in on the parent's part anyway then there's no reason to send additional information along with the request. The service should rather send additional information about content categorization alongside the response.
So what reasons can you imagine for it to be designed in such an obviously unnecessary way?
No more or less than sending age information or registering an ID does. In all cases they must track content classification at some granularity (individual resource, single page, subdomain, some other scheme) and act on that information. The only thing that varies is how they act.
Yes, when it's client-sent they can hide classified parts of the page. When it's server-sent you either mark the whole website 18+ or you hide 18+ content for everyone.
You're just making things up. There's no technical reason a header based solution can't be granular. It could also specify alternative resources similar to how multiple image resolutions are handled today. It all depends on what is standardized.
Right now the only one I'm aware of is RTA which theoretically applies on a per-request basis although I expect that approximately all present usage is uniform site wide.
If such a feature were specified then a site would merely have the option of providing alternatives. In the event that it did, whether or not to follow such redirects would be entirely up to the client.
Such a system is clearly the technically superior solution. It regulates the provider as opposed to the client, forcing the market to provide a workable solution for concerned parties while the client maintains complete control over how things are handled. It further steers well clear of any slippery slopes by not mandating the broadcast or collection of personal information.
Perhaps important from a liability perspective, it places the onus on the client as opposed to these latest attempts to shift it squarely onto the service provider. Right now the legality of serving content across jurisdictional boundaries is extremely convoluted. With ID or age reporting laws it clearly becomes the service provider's responsibility. In contrast, a mandatory metadata standard for classification would create a situation in which it is clear that the legal responsibility (if any) to appropriately configure filters falls to the client.
Of course such a solution would be of no help to the anti-porn and pro-surveillance lobbies. That's the entire point.
> Only Apple has the unique dynamic allocation though.
What do you mean? On Linux I can dynamically allocate memory between CPU and GPU. Just have to set a few kernel parameters to set the max allowable allocation to the GPU, and set the BIOS to the minimum amount of dedicated graphics memory.
Maybe things have changed but the last time I looked at this, it was only max 96GB to the GPU. And it isn't dynamic in the sense you still have to tweak the kernel parameters, which require a reboot.
Strix Halo you can get at least 120 GB to the GPU (out of 128 GB total), I'm using this configuration.
Setting the kernel params is a one-time initial setup thing. You have 128 GB of RAM, set it to 120 or whatever as the max VRAM. The LLM will use as much as it needs and the rest of the system will use as much it needs. Fully dynamic with real-time allocation of resources. Honestly I literally haven't even thought of it after setting those kernel args a while ago.
So: "options ttm.pages_limit=31457280 ttm.page_pool_size=31457280", reboot, and that's literally all you have to do.
Oh and even that is only needed because the AMD driver defaults it to something like 35-48 GB max VRAM allocation. It is fully dynamic out of the box, you're only configuring the max VRAM quota with those params. I'm not sure why they choice that number for the default.
You do have to set the kernel parameters once to set the max GPU allocation, I have it set to 110 GiB, and you have to set a BIOS setting to set the minimum GPU allocation, I have it set to 512 MiB. Once you've set those up, it's dynamic within those constraints, with no more reboots required.
On Windows, I think you're right, it's max 96 GiB to the GPU and it requires a reboot to change it.
I've got a 128 GiB unified memory Ryzen Ai Max+ 395 (aka Strix Halo) laptop.
Trying to run LLM models somehow makes 128 GiB of memory feel incredibly tight. I'm frequently getting OOMs when I'm running models that are pushing the limits of what this can fit, I need to leave more memory free for system memory than I was expecting. I was expecting to be able to run models of up to ~100 GiB quantized, leaving 28 GiB for system memory, but it turns out I need to leave more room for context and overhead. ~80 GiB quantized seems like a better max limit when trying not running on a headless system so I'm running a desktop environment, browser, IDE, compilers, etc in addition to the model.
And memory bandwidth limitations for running the models is real! 10B active parameters at 4-6 bit quants feels usable but slow, much more than that and it really starts to feel sluggish.
So this can fit models like Qwen3.5-122B-A10B but it's not the speediest and I had to use a smaller quant than expected. Qwen3-Coder-Next (80B/3B active) feels quite on speed, though not quite as smart. Still trying out models, Nemotron-3-Super-120B-A12B just came out, but looks like it'll be a bit slower than Qwen3.5 while not offering up any more performance, though I do really like that they have been transparent in releasing most of its training data.
There's been some very recent ongoing work in some local AI frameworks on enabling mmap by default, which can potentially obviate some RAM-driven limitations especially for sparse MoE models. Running with mmap and too little RAM will then still come with severe slowdowns since read-only model parameters will have to be shuttled in from storage as they're needed, but for hardware with fast enough storage and especially for models that "almost" fit in the RAM filesystem cache, this can be a huge unblock at negligible cost. Especially if it potentially enables further unblocks via adding extra swap for K-V cache and long context.
So, this is all true, but this calculation isn't that nuanced. It's trying to get you into a ballpark range, and based on my usage on my real hardware (if I put in my specs, since it's not in their hardware list), the results are fairly close to my real experience if I compensate for the issue where it's calculating based on total params instead of active.
So by doing so, this calculator is telling you that you should be running entirely dense models, and sparse MoE models that maybe both faster and perform better are not recommended.
It discusses it, and they have data showing that they know the number of active parameters on an MoE model, but they don't seem to use that in their calculation. It gives me answers far lower than my real-world usage on my setup; its calculation lines up fairly well for if I were trying to run a dense model of that size. Or, if I increase my memory bandwidth in the calculator by a factor of 10 or so which is the ratio between active and total parameters in the model, I get results that are much closer to real world usage.
Yeah, I looked up some models I have actually run locally on my Strix Halo laptop, and its saying I should have much lower performance than I actually have on models I've tested.
For MoE models, it should be using the active parameters in memory bandwidth computation, not the total parameters.
Unfortunately, for most people the fact that it's part of Emacs is a blocker.
And because most people use worse Git tools, they tend to use workflows that are easier with more cumbersome tools; generally just committing all kinds of junk commits to a branch, and using the "squash and merge" feature of GitHub or GitLab to clean everything in a PR/MR up into a single commit.
So yeah, it's sad that people don't use Git to its full potential because almost no other Git interface works as well, and most people aren't going to learn Emacs just for a Git UI.
> Unfortunately, for most people the fact that it's part of Emacs is a blocker.
OT but i've learned the hard way not to push people into emacs.
a few years ago i made the very stupid mistake of pushing some colleague to trying/learning emacs and then i found myself having to explain the same person everything as well as fix his elisp code from his ~/.emacs .
Reality is, i didn't want to have that role and that colleague wasn't interested in gnu emacs in the first place.
That was a very stupid mistake on my side.
Nowadays i just say things like "yeah it's magit, an emacs plugin" or "ah yeah, it's nice because you spend some time learning it and then you can bring it over from company to company, no licenses involved or other annoyances".
Some people are intrigued, most other absolutely aren't... And it's fine.
Surely the mistake here is conflating "learning Magit" with "learning Emacs"? I can run Java applications while knowing nearly nothing about the JVM. The same is true for applications based on Emacs.
Magit is not an "application based on Emacs", it's an extremely powerful git plugin for someone who is already using Emacs. If you're not using Emacs, it would be crazy to fire up Magit just to do a git commit or rebase. Even the fact that you'd have to learn Emacs' extremely idiosyncratic keyboard shortcuts, and even keyboard shortcut help (what do you mean "save is C-x C-s?" what does that even mean?) is a huge problem not worth the effort.
And I'm saying this as someone who has exclusively programmed in Emacs with Magit for the last 5 years in my job.
I simply used Emacs for 25 years, avoiding editing .emacs entirely
Only in the past years have I started customizing it
My attraction to Emacs is stability and I can use it in text or GUI mode.
Many editors have come and gone in that time, many employers insist I use this or that piece of over designed, under done GUI. When I have the chance, back to Emacs
Peculiarly, having used Emacs for a decade as my main editor, and Git for much more than that, I never could get used to Magit.
Maybe it's due to muscle memory from my CLI Git setup (nothing special, just some aliases, scmpuff, delta, etc.), or Magit forcing you into its own quirky UI, but it never clicked for me. For 99% of things I use Git for, I don't have any issues with my workflow, nor wish to improve it. For the other (very) rare occasions, I can always ask an LLM to help me figure out the right command.
This is also why I don't see any value in Jujutsu either, or any of the GUI/TUI wrappers. The Git CLI porcelain with some minor customizations just hasn't been a problem I need solving.
I've looked at Magit and indeed Emacs is a blocker as it's not something I'd like to pick up and maintain. I am using Fork as my primary Git GUI and am pretty happy with it. Lazygit and tig cover the few use-cases which Fork does not cover.
I am not someone to recommend VSCode, but if you are already using it, you could check out edamagit. I think in the Vim world there is also some equivalent.
Good tools can improve your workflow for sure, but it's easy enough to keep a clean history with a handful of git commands. There are two main reasons people don't do so: 1. they don't know the commands yet or 2. they just don't care (and are in an environment where there's no incentive to care).
The kind of person who would try a tool like Magit and use it to discover git would have found a different route if Magit didn't exist. The type of person who doesn't care isn't going to learn something just because a tool is available.
Magit is more than a shortcut for command avoidance. It helps surface conflicts and edge cases during complex rebases and fixups, especially on big feature branches, making problems visible earlier. Relying on muscle memory alone can lead to mistakes, and using better tooling can limit the fallout when that happens.
Personally i've been often looking for an opensource Git GUI front end but couldn't find anything i'd like. My points of reference of decent tools -ignoring the underlying tech- are Perforce and TortoiseSVN under Windows and Fuel[0] (for Fossil) and KDESVN under Linux.
Perforce and Fuel are the two i like the most, followed by KDESVN and Tortoise. But since i'm on Linux and i stick with opensource projects, Perforce and Tortoise are out, leaving me with Fuel and KDESVN. I'm using KDESVN for a few projects where i don't care about external collaboration and want to store many binary files (which Subversion does it better that Git IMO), though KDESVN is very primitive and limited in what it supports (it doesn't support shelving at all, for example - in fact for anything beyond commit and checking the history you probably need to use the cli tool). I do not use Fossil much nowadays but i used it a lot in the past so i have a bunch of repositories in it, for which i use Fuel. Fuel is unfortunately not being developed anymore, though since it is opensource it doesn't matter much (which is a reason why i ignore anything non-opensource, i know there are some decent proprietary Git clients but for me they might as well not exist).
I think at some point i'll grab libgit2 and make something using Lazarus, probably something similar to Fuel. Unlike libsvn, which seems to be made in hell, libgit2 seems very sane and shouldn't be terribly hard to make something usable out of it.
In the meanwhile i use a combination of git cola, gitgui and gitk though they all feel like a hodgepodge of barely thought random features thrown in together (and i hate that they all want to impose their own idea of how i should format a message -especially git cola- which of course aren't identical between of them and aren't even close how i'd like to format the messages myself).
Isn’t this just something that any IDE has built-in these days? Maybe I’m missing something, but how is this fundamentally different from the built-in git timeline view from something like VSCode or Jetbrains?
> Isn’t this just something that any IDE has built-in these days?
In most IDEs I feel that Git integration is an awkward badly integrated afterthought. They are also very much tied to the whatever IDE offers them in terms of available shortcuts, layouts, controls etc. (this applies to Magit, too).
I upvoted you because you were unfairly downvoted. I don't even use a Mac any more after 20 years of exclusively using them but it's actually hilarious how bad magit is compared to this. It's all well and good making the most of limitations that are self imposed but people need to remember to look outside their own bubble.
It may be prettier looking. I've seen many Git GUIs that are prettier than Magit.
But none of them that I've tried have ever come close to the workflow.
I can stage and unstage individual hunks, do complex interactive rebases, squash commits, break apart commits, etc. much faster in Magit than I can in other Git GUIs.
Maybe you're hung up on the "G" part; perhaps I should have just said "UI" rather than "GUI".
So no, I haven't tried that one because it's Mac only, but I'm not really seeing from the screen recordings the kind of workflow that I find so powerful in Magit.
> I can stage and unstage individual hunks, do complex interactive rebases, squash commits, break apart commits, etc. much faster in Magit than I can in other Git GUIs.
I can do all that pretty fast in GitUp, too. Since most of the commands there have quick keyboard shortcuts.
My most common workflow besides staging anything is (to make sure history is clean):
- split up a commit (add/remove files or hunks if the commit contains stuff that should go into another commit)
- move new commit up/down the branch (doesn't require interactive rebase in GitUp)
- squash up/down
(undo/redo from time to time)
As far as I understand, Magit doesn't offer anything in that regard except the good old interactive rebase [1]. In GitUp moving commits is (u)p/(d)own and (s)quash with parent
> Maybe you're hung up on the "G" part; perhaps I should have just said "UI" rather than "GUI".
The distinction doesn't matter. The keyword in both GUI and UI is User. As a user I found GitUp to be a much better tool than Magit. Though Magit does probably allow for more very advanced usage most people don't do. [2]
However, actual useful usage like I described above? Ooh boy, no one does it except GitUp for some reason.
I cannot imagine how you can properly supervise an LLM agent if you can't effectively do the work yourself, maybe slightly slower. If the agent is going a significant amount faster than you could do it, you're probably not actually supervising it, and all kinds of weird crap could sneak in.
Like, I can see how it can be a bit quicker for generating some boilerplate, or iterating on some uninteresting API weirdness that's tedious to do by hand. But if you're fundamentally going so much faster with the agent than by hand, you're not properly supervising it.
So yeah, just go back to coding by hand. You should be doing tha probably ~20% of the time anyhow just to keep in practice.
Kind of agreed. I like vibe coding as "just" another tool. It's nice to review code in IDE (well, VSCode), make changes without fully refactoring, and have the AI "autocomplete". Interesting, sometimes way faster + easier to refactor by hand because of IDE tooling.
The ways that agents actually make me "faster" are typically:
1. more fun to slog through tedious/annoying parts
2. fast code review iterations
3. parallel agents
Yeah. I've been finding a scary number of people saying that they never write code by hand any more, and I'm having a hard time seeing how they can keep in practice enough to properly supervise. Sure, for a few weeks it will be OK, but skills can atrophy quickly, and I've found it's really easy to get into an addictive loop where you just vibe code without checking anything, and then you have way too much to review so you don't bother or don't do a very good job of it.
It’s a useless skill to keep sharp. It will never again be important. It’s like the electrical engineering professors who insisted we needed to understand how the circuitry worked in order to be good programmers. We didn’t.
And there are still plenty of hybrid architectures. Nemotron 3 Super 120B A12B just came out, it's mostly Mamba with a few attention layers, and it's pretty competitive for its size class.
But yeah, these different architectures seem to be relatively small micro-optimizations for how it performs on different hardware or difference in tradeoffs for how it scales with the context window, but most of the actual differentiation seems to be in training pipeline.
We are seeing substantial increases in performance without continuing to scale up further, we've hit 1T parameters in open models but are still having smaller models outperform that with better and better training pipelines.
reply