Agreed. I started writing a desktop GUI harness in Rust using native desktop GUI APIs.
Claude, Codex, Opencode, everything is Electron. With agents and solid engineering fundamentals, I set out to prove that we don't need to take that shortcut any more and we can, in fact, ship high quality/performance applications.
It's still early days - but I got something basic working. I've embedded the Goose harness as a back end and am using the UI framework used by the Zed editor.
I'm keeping the source closed for a little while. Also if anyone from the major labs is hiring, hihi, I make quality things.
Also I'm aware that this is a rip off of vscode's design. I like their UI, lends itself well to projects and so on. In a different life I might have ported vscode to Rust - but no one uses IDEs any more haha
I'm sorry and will be the first to admit I went for this too, but "using agents to write more tools for agents." is literally the problem.
The purpose of software is to help people with their problems. I think it's easy to tell ourselves that other people want to use our tools which we really made to help ourselves (especially if them using it also helps ourselves even more!). And also, this is all so new and there are real opportunities that it is good to learn and try new things and be open to actually making something valuable.
But! Other people extensively using agents are literally the last people who would pay for a project like this! And unless it's a fulltime job like it is for a 5 or so well resourced tech companies, it's hard to make something as good and useful as them. I say this as someone who spent a long time on this problem as my fulltime job.
We should still have fun with our work. But literally, "software for other software people, that solves the problem of us needing to write more software for writing software, faster", who is that helping? There are no constraints or end users besides ourselves, and half of that is not so much it benefiting us through productivity, but learning new things and play - again, important. But there's a reason so many people are doing this and I think it's because they don't have to go outside and solve someone else's problem, or do something unusual, or involving unfun non-software aspects - agents are fun! But how many random people's agent have you installed?
IMO: if the technology does not help a non-technology person live a better life, and especially if it involves "using an agent to build a framework for agents" we should do some more, deeper introspection about what problem we are actually solving and whether or not we're being ambitious. And I say that not to pick on you but myself.
It's not a new problem in software, and it's not unique to software.
A lot of hobby woodworkers, for instance, spend most of their woodworking effort on their new woodshop. Building a workbench, building shelves and cabinets and special holders for all their drill bits, mortising chisels and tools, special tables for their miter saw, a special cabinet connected to their dust collection system for sanding small pieces...
Likewise, the favorite project for programmers is ... programming tools. New languages, new libraries, new editors. We've gone through like six version control systems at my day job.
There is some merit in this, since new tools make you more efficient at building more new tools, but eventually you have to step back and remind yourself what your actual goals are.
I call it the tool maker's dilemma, but that's not nearly as catchy or easy to fit into conversation as bikeshedding.
I really love your viewpoint. And I also see that around me a lot of people developing their own custom stuff that fits to their way of thinking, but not apt to others, so everyone developing their own custom. It is fast an cheap, exciting and comfortable. I am a bit puzzled as I see hyperpersonalised sw is a way to divide us and put us in an echo chamber. No need to understand how others work or think… And we loose our knowledge that is comming from the experience of interacting whit other way of thinking.
My general sense is that a well-designed tool for humans, especially if it has any kind of textual interface, will be an at least satisfactory tool for agents anyway.
I do wish you well and think it's good for us to work on these tools if only to help ourselves better understand how to work with these things, and have fun. I've actually had a blast working on agent tooling with agents too.
What I fear is that as computer people we can easily convince ourselves to avoid doing things in-person or which involve leaving the comfort of our inner/online lives, or having to bridge technology with some other thing that is very much not technology (like non technical people, or sales and marketing), or is but is not easy to work with (like a robot). But there are actually a ton of problems that need solving there, and they are uncomfortable because it might involve looking stupid or showing up somewhere nobody knows you, or unknown-unknowns.
Personally I am embarrassed at myself not for trying to solve my own problems (I want to spend all day doing tech stuff and make money selling it to other people doing tech stuff) but for how long it took me to realize that I was either anxious or thought I was above doing something more concrete and mundane.
For example, this is a popular music venue in the city I live in https://www.thegreatnorthernsf.com/ - Claude and Codex have been out for months; it would take like 20 minutes to give their site a makeover, mostly just watching AI work.
This is how local government events are coordinate in the most technologically forward city in the world. Double irony that the product is provided by one of the biggest tech companies in the city, too. But it's literally just a web form and some poor government worker is apologizing in the event details that the signups are broken: https://sfrpd.my.salesforce-sites.com/SFRPDJob?sShiftId=a09P...
I think agents are very fun and cool but I'm increasingly convinced that my job is to use them to help this type of user, not developers.
> Naturally, game engines are crazy optimized, so the UI runs butter smooth and uses pretty much no resources.
We should build everything using a AAA FPS engine loop now that we can. The amount of shit that happens in one rendered frame of overwatch is wild. And then we go and do like 200 of those per second.
One recent performance issue that was raised by a customer was in the end a tiny svg animation loop that was on all the time even when the tab was inactive. So no game engine loops for my customers.
SVG animation in a browser works completely differently than a game engine handles animation. SVG is based on the DOM and animating it requires mutating the DOM. Lots of string operations because everything is a string attribute. All of that has a ton of unnecessary overhead.
You actually would have been better off replacing your SVG animation with a game loop! Assuming you're using Lottie you can easily swap it to render into a `<canvas>` which doesn't have any of that overhead and uses `requestAnimationFrame` which automatically gets throttled in inactive tabs.
No you won't. Firstly because your software is one of dozens running and it doesn't get to take the privilege of fucking over my battery consumption and asking for my GPU to spin so you can edit markdown, and also because 2D rendering is miserable and non-performant in every single game engine you can think off. 2D sucks to make, 2D eats half of your frame time by just existing and slicing those triangles.
You can try rendering a 2D rectangle as 2 quads in whatever GPU API you like the most, then try pushing the same amount of pixels to a CPU buffer via the widest CPU memory write instructions you can have (and then wait the GPU-accelerated compositor upload them to the GPU anyways to present them...).
I can tell you for a fact one will be 2 orders of magnitude faster, want to make bet which one?
Using a GPU api does not imply you have to render at the screen's refresh rate, or even that you have to update the entire window.
Rendering the final triangles isn't the issue. The issue is that while your 3D world is nicely setup polygons that benefit from all the culling you can do with it, your 2D UI is requiring vector graphics which your GPU will absolutely hate having to render. The actual composition of the UI elements is incredibly non-performant. Pushing the pixels doesn't matter.
There's a reason why browsers are so complicated and equally non performant. 2D rendering, even GPU accelerated, with the constraints we have right now (or rather, the lack of constraints, because 3D rendering is basically tradeoffs all the time), is not going to be more performant.
I'm pretty sure that there are parallel algorithms for triangulating vector graphics that can be run on a gpu. But in general handling vector graphics is simply overkill. What are we actually using them for on the web? Rounded corners? We don't those! Let's use a paradigm that's more GPU friendly. Text can all be cached.
>I'm pretty sure that there are parallel algorithms for triangulating vector graphics that can be run on a gpu
They are. They're not particularly more performant that what we do today. It all involves tesselation on the CPU, or dropping entire feature sets.
>But in general handling vector graphics is simply overkill.
You don't get to decide that.
>Rounded corners? We don't those! Let's use a paradigm that's more GPU friendly.
That ship has sailed. The text input I'm typing this comment in has slightly rounded corners. The text it's containing is vector graphics with rounded corners everywhere, where the best renderer solution at the moment is Slug, with SDFs being a distant second.
>Text can all be cached.
sure, if you drop ligatures, only care about latin alphabet languages, accept having to rerender everything the moment I zoom the page to 110% or 125%
Text glyphs/UI images are all in 1 atlas, you can render the entire UI as 1 draw call and only re-render when something changes. I haven't checked if Claude chose to re-render the whole screen or just part of it, you can design it either way.
Layout can be as simple or as complex as you want it to be - this example does both flex and grid-style layuts, flex supports start/end/center/stretch on both main/cross axes, as well as a flex prop.
For the API you can either go immediate mode or retained - this one is retained since explaining to Claude how to do the IM version properly would take longer than doing it myself, but the layout algorithm can be the same regardless of which route you go for the user-level API.
You only need scrolling/overflow on top of that and this is enough for the majority of widgets you may need.
My point is none of this stuff is hard. This is 700 lines of Claude code, if you write it yourself it would be 400, and will have better performance characteristics than most UI libraries out there.
Edit: Added scrolling too, there's a 100K scrollable virtual list in the demo. Again - this stuff is not hard.
This is extremely informative! Thank you very much!
It might be helpful to add your comment to the GitHub repository so people can find it in the future.
rxi's MicroUI is a similar (but human-written) project whose interface is immediate-mode; it supports overlapping windows, which is trivial in retained mode (now that we have CPU to spare) but a little bit tricky in immediate mode. https://github.com/rxi/microui is about 1500 lines of C.
I'm especially impressed with your perseverance in the face of such low-quality commentary.
So, not to shit on your parade my dude, but it is, in fact, hard. Should your users do silly things like open it on a phone like I just did. As a non exhaustive list:
* Your font rendering is catastrophically bad, because it doesn't take DPI into account. It's a blurry, low quality mess.
* You don't handle taps, because you made it with a PC without considering other users. I can therefore not tap on anything. Is it all mobile devices? Is it just Firefox? Is it just that device? Surely that's an easy problem to solve, it and doesn't involve dozens of interop hacks! Scrolling that main page does work though, thankfully the horrible bad UI toolkit does the job for you.
* Layout: congrats, you've remade Clay. It's mostly a solved problem for the past twenty years, although modern layout needs like "rescaling your UI because your users aren't in 1920x1080" seem to be unneeded. Flex and grid layout have plenty of open source implementations, I'm sure Claude can copy it.
* Your text is not selectable or copyable, but I'm going to be graceful and say that maybe it's because it doesn't handle taps either and that it definitely works on desktop. Right?
* People that have accessibility needs seemingly can go fuck themselves, since your entire website is just a canvas. How hard could it be to integrate with the DOM for ARIA elements anyways? (Trick question, it's impossible.). And when that's done, you get to redo it for AccessibilityKit on Apple platforms, UIA on Windows, AAF on Android, AT-SPI on Linux... Don't worry, they're all very similar APIs, how hard could it be to have one model that integrates with them all? (Watch out, trick question again. Especially if you intend to do it _well_).
* I haven't tested it (remember, doesn't work on a mobile device?), but you do restore text input, in the way that is expected by platforms, right? Android has lifecycle events like PAUSE and STOP, as well as activity recreation like it just did upon going to low battery mode as I was typing. Firefox restored this very long comment. You would do the same, right? It is the responsibility of an UI framework: two way binding of inputs is a recipe for UI nightmares.
It took me longer to type this than to collect all of these issues. That was from just the first 30 seconds of interacting with your toy.
Maybe, just maybe, some people are smarter than you, and have spent more time working on the subject than you, and you should be ever so slightly more humble. That the complexity and the performance of current UI toolkits is not an accident that needs to be fixed by geniuses like you gracing us with your previously undiscovered research. Or keep jerking yourself off, unilaterally declaring that people don't need shadows, transparency, or other very niche features like animations, and ignore the world around you, I'm not your dad.
Many native UI toolkits also suffer from some the issues you shared. Depending on what you compare with you'll find fewer layout options, limited or no text selection, poor style and animation support, etc. People say browsers are inefficient but it's actually impressive how well browsers manage to run all of the badly optimized websites out there.
What this toy shows you is that both points in your original comment are moot.
1. "fucking over my battery consumption and asking for my GPU to spin" - it renders frames on demand
2. "2D sucks to make, 2D eats half of your frame time by just existing and slicing those triangles" - frametime is in the microseconds when a frame does need to be rendered, and no triangles need to be sliced because shaders exist.
Let me remind you of what we were arguing about - whether or not you can render UI with a GPU API, without blowing up your computer. I would love to hear how handling taps, adding a mobile-friendly layout to the demo, handling selection or accessibility would change the calculus on that.
In the midst of your ad hominem, your argument seems to have changed from "don't use a GPU API" to "making a production grade UI library is hard", which I'm more than happy to give you.
You just don't get it, do you? What I'm saying is that your approach can _never_ lead to a good UI toolkit. You've made a funny toy that renders three boxes that can never work in the real world. Not only is your frame time in the milliseconds on a browser (on a 5080 too), you've made a toy prototype that covers 1% of the necessary features, claimed that it's entirely complete and declared yourself winner (and that implementing the rest would have zero impact on that performance, lol)
It's now been three times you ignore the reality of the world, loudly proclaim yourself winner then proceed to drop 90% of what is necessary. No shit, you can draw empty boxes in 2ms and badly shape text in 3.
>I would love to hear how handling taps, adding a mobile-friendly layout to the demo, handling selection or accessibility would change the calculus on that.
Because now for half of these, you are involving the CPU again, with endless back and forths. Keep repeating it until you have something that works and congratulations, you have more or less exactly what browsers do today.
Considering the hubris, the loud proclamations that you're right and that the whole world around you is wrong, the complete ignorance of the state of the art and the insistence on badly reimplementing things from scratch, the "you don't need those features actually", let me guess, game engine developer?
His approach is actually correct (well, aside from doing it in the browser). I wrote a custom, fully from scratch Vulkan UI rendering engine, works more or less exactly as described above. Crazy efficient and performant. Removing VSync I can hit ~100k fps on a dingy 4 year old GPU. For instance in benchmarking opening a window is a _sub microsecond action_. Doing the same in Qt is what... milliseconds? Literally? Also all of the things you described is because... browsers are bad. Or people are incompetent. Drawing fonts is a single draw call and yes, you can and should cache absolutely anything. All the fonts I used are 100% cached. Oh and btw it isn't a toy UI framework but fully production ready.
> Firstly because your software is one of dozens running and it doesn't get to take the privilege of fucking over my battery consumption
This never stopped the Electron devs until now, I honestly can't see why they'd start to get any better now that the slop machine lets them be even lazier.
If your program does 1/1000 of the GPU work Overwatch does every frame, it will cost in the order of magnitude of 1/1000 of what Overwatch costs to render (some fixed costs aside) -- taking aside the fact that you also don't need to re-render the entire window every vblank.
well that's fine, it's not a great joke, but perhaps think, does me explaining that vram and ram are different things make a difference to what this person is saying? Is this person saying anything that would be fundamentally altered by me pointing out the mistake they're making? sometimes just ignore comments and let them die the death they deserve
claude code and codex are literally IDEs. as are any other remote agent that you use like them
you're not "in the terminal", in fact the product is selling you on the concept of being "in the terminal" as you work, but you're literally just asking a computer with ~full userspace access and direct ability to compile and run programs to do stuff for you, in an environment with "plugins" and like 30 "slash commands" and menus. it has auto-review and manages git for you and even writes your PR descriptions and other messages to your colleagues if you want. that is an integrated development environment.
I think if you can just run a one line bash command from a big company's website, and then just ask it to set up git and everything else for you (which it will default to anyway), and it tries to sell you on third party plugs in and Skills and managed proxies like MCP, and workflows, you are just using the new kind of IDE. And there is nothing bad about that because it makes us more productive to not have to mess with things like that. The idea that you are not using an IDE, or that we are "better" because the IDE looks like the development environment of the type of developer we'd admired in the post (but literally has the value proposition of being the easy way to accomplish that), is dangerous.
This is pure speculation. This is like saying blockbuster films no longer have a camera operators rigging pro level cameras because sony came out with the alpha series. I suspect that the number of new people NOT using IDEs (e.g. using Claude Code, Codex, etc.) is growing rapidly but it may not necessarily mean that less people are using IDEs.
Controversial take: Looking at lines of code is no longer a feasible task. Also it adds very little to quality for large swathes of user facing functionality. Instead focussing that much time on building a robust test suite is a better application of developer time.
They're not going to bother checking. It simply isn't going to be possible for humans to review code in the future where all code is being written by machines, and not having all code be written by machines is not an option.
Hell in the future LLMs probably won't even work with "code" at all, just bytecode directly, using some format created by and optimized for LLMs. Why even bother with text when no human will ever lay eyes on it?
>We are gonna see some Knight Capital level fuckups shortly.
Yes, but humans fuck up too so it doesn't matter. Future go brrr.
The idea the LLMs would work in bytecode kind of ignores why they are good at higher level code, IMO. Bytecode would fill and overflow context so fast and locality of reasoning is much harder.
I've been an emacs user for most of my life, before I even went to college. I took a brief (5 years) tour into IDEs for writing code on the job, but the moment agents became a viable way to develop code I switched back into emacs. I love reviewing changes in magit.
I'm pretty sure many folks put up with the bloat of IDEs just for the functionality they offered. Now, they're hopping back to emacs, vim, sublime, whatever their heart desires.
This is awesome and I'm glad things are moving in this direction. I've been playing with Crux (Rust + Native UI) myself: https://github.com/redbadger/crux
I've been working on using Svelte + Rust to make desktop apps with native UI frameworks [1], resuscitating a COVID era side project that I felt was intractable at the time. I havent updated the docs or README.md so you'll have to point an agent at it to figure it out, but so far GTK/QT are fairly complete and I'm working on WinUI/AppKit.
Claude, Codex, Opencode, everything is Electron. With agents and solid engineering fundamentals, I set out to prove that we don't need to take that shortcut any more and we can, in fact, ship high quality/performance applications.
It's still early days - but I got something basic working. I've embedded the Goose harness as a back end and am using the UI framework used by the Zed editor.
https://github.com/alshlabs/quill
I'm keeping the source closed for a little while. Also if anyone from the major labs is hiring, hihi, I make quality things.
Also I'm aware that this is a rip off of vscode's design. I like their UI, lends itself well to projects and so on. In a different life I might have ported vscode to Rust - but no one uses IDEs any more haha