It would be nice if it didn't have to be all local. I'd love a managed cluster feature where you could just blast some workloads off to some designated server or cluster and manage them remotely, share progress with teammates, etc. (Not "cloud" though; I'd still want them on the internal network). I imagine something like that is in the works.
Not going to solve your exact problem but I started this project with this approach in mind. It is exposed to the Internet though on a VPS or server but using caddy basic auth in front of the coding url
I agree this is what the article says, but it's a pretty bad premise. That would only be the case if the primary user interaction with coding agents was "feed in requirements, get a finished product". But we all know it's a more iterative process than that.
We are building this at docflowlabs ie a self-healing system that can respond to customer feedback automatically. And youre right that not all customers know what they want or even how to express it when they do, which is why the agent loop we have facing them is way more discovery-focused than the internal one.
And we currently still have humans in the loop for everything (for now!) - e.g, the agent does not move onto implementation until the root cause has been approved
Cool, I tried something similar over a couple weeks but the problem I ran into was that beyond a fairly low level of complexity, the English spec became more confusing than the code itself. Even for a simple multi-step KYC workflow, it got very convoluted and hard to make it precise, whereas in code it's a couple loops and if/else blocks with no possibility of misinterpretation. Have you encountered that at all, or have any techniques you've found useful in these situations?
That's why I feel like iterative workflows have won out so far. Each step gets you x% closer, so you close in on your goal exponentially, whereas the one-shot approach closes in much slower, and each iteration starts from scratch. The advantage is that then you have a spec for the whole system, though you can also just generate that from the code if you write the code first.
that's right, and agents turning specs into software can go in all sorts of directions especially when we don't control the input.
what we've done to mitigate is essentially backing every entrypoint (customer comment, internal ticket, etc) with a remote claude code session with persistent memory - that session essentially becomes the expert in the case. And we've developed checkpoints that work from experience (e.g. the root cause one) where a human has the opportunity to take over the wheel so to speak and drive in a different direction with all the context/history up to that point.
basically, we are creating a assembly line where agents do most of the work and humans increasingly less and less as we continue to optimize the different parts of assembly
as far as techniques, it's all boring engineering
* Temporal workflow for managing the lifecycle of a session
* complete ownership of the data model e2e. we dont use Linear for example; we built our own ticketing system so we could represent Temporal signals, github webhooks and events from the remote claude sessions exactly how we wanted
* incremental automation gains over and over again. We do a lot of the work manually first (like old fashioned hand coding lol) before trying to automate so we become experts in that piece of the assembly line and it becomes obvious how to incrementally automate...rinse and repeat
Ooh, it sounds like you've already got most of the groundwork done for something I was wondering about yesterday: I'd love it if there was some way during an incident, for some system to pull all the PRs included in the latest release, check which agents worked on them (i.e. line in the commit message with an identifier that corresponds to the agent's LLM context and any other data at the time of commit), "rehydrate" these agents from the corresponding stored context, feed them the relevant incident data, and ask if it could be related to their changes and what to do about it.
In most cases it might not be much more valuable than just looking through the diffs from scratch with a new agent, but there are probably going to be some cases where a rehydrated agent is like "Doh, I meant to do X but it looks like I hallucinated Y instead. Here's a PR to fix it!"
I know that's just a small piece of what you're doing, but I think it's something that would be valuable on its own, and soon something that is likely to be "standard infrastructure" for any company that does even a little agentic coding (assuming it works). It'd probably even be "required infrastructure" in regulated industries; the fact that all these agent contexts are ephemeral has to be a red flag from a regulatory perspective.
totally, it's like ai-native github with some linear plus some ability to push the ball forward autonomously. This doesn't exist yet so we had to build a version internally, but also we built it pretty specifically for our needs. The general version might have to be more componentized, not sure. We also as an industry probably need some version control protocol above git that includes all the history around the commit so we don't have to string together root cause documents and conversation history in s3 linked via relational entities in psql.
I think there's an argument that AI is helping the (non-AI) software job market. If tech companies are seen as laying off "too fast", then investors will penalize them for not retaining enough people to compete in AI. So companies have to do some layoffs, both to rebalance for the economic downturn and to claim "progress in AI", but not so much that it makes them look weak to investors. Whether they actually need the employees for anything is secondary.
The ridiculous thing is that we're spending trillions of dollars to develop these coding assistant AI things at the same time we're running out of useful stuff to code.
Same situation (50 last week, 2 kids) though have been unemployed for a year. Part of me thinks that, rather than taking jobs, AI is actually the only reason a lot of jobs still exist. The rest of tech is dead. Having worked in consulting a while ago, you can kind of feel it when you're approaching the point where you've implemented all the high value stuff for a client and, even though there's stuff you could do, they're going to drop you to a retainer contract because it's just not the same value.
That's how the whole industry feels now. The only investment money is flowing into AI, and so companies with any tech presence are touting their AI whatevers at every possible moment (including during layoffs) just to get some capital. Without that, I wonder if we'd be seeing even harsher layoffs than we already are.
That's so not true. Of the 23 companies we reviewed last year maybe 3 had significant AI in their workflow, the rest were just solid businesses delivering stuff that people actually need. I have no doubt that that proportion will grow significantly, and that this growth will probably happen this year but to suggest that outside of AI there is no investment is just not compatible with real world observations.
That's good to hear actually. It's usually a downer when a strongly held belief is contradicted with hard evidence, but I'm excited to hear that there's life yet in the industry outside of AI. Any specific trends or themes you can share?
Energy is a much larger theme than it was in the years before (obviously, since we're in the EU and energy overall is a larger theme in society here. This has a reflection in the VC market, but it also is part of a much larger trend, climate change and CO2 neutrality).
Another trend - and this surprised us - is a much stronger presence of really hard tech and associated industries and finally - for obvious reasons, so not really surprising - more parties active in defense.
Totally makes sense. Things that (once complete) have more realizable tangible value, rather than "optimizing user engagement" aka "enshittification" as some kind of imaginary value store for the last 20 years and is now being called in.
What is especially interesting is to see the delta between the things that are looked at in pre-DD and which then make it to actual DD after terms are signed.
Software will ALWAYS be an attractive VC target. The economics are just too good. The profit margins are just inherently fat as fuck compared to literally anything else. Your main expense is headcount and the incremental cost of your widget is ~$0? It's literally a dream.
It's also why so much of AI is targeting software, specifically SAAS. A SaaS company with ~0 headcount driven by AI is basically 100% profit margin. A truly perfect conception of capitalism.
Meanwhile I think AI actually has a decent shot at "curing" cancer. AI-assisted radiology means screening could be come significantly cheaper, happen a lot more often, and catch cancers very early, which is extremely important as everyone knows to surviving it. The cure for cancer might actually just involve much earlier detection. But pfft what are the profit margins on _that_?
Yeah for the better part of a generation, our best and brightest minds have been wasted on "increasing click count". If that can all be AI from here on out, then maybe we can get actual humans working on the real problems again.
The problem was always funding. All those bright minds went into ads because it paid well. Cancer research, space, propulsion, clean, energy, etc.. none of those paid particularly well. Nor would they have afforded a comfortable life with a house and family. The evisceration of SWE does not guarantee a flourishing in other fields. On the contrary, increased labor supply with further pressure, wages, downwards.
Agreed, though I think we all knew that the software industry payscales were out of whack to begin with. Fresh college grads that can barely do a fizzbuzz making twice as much as experienced doctors.
What I don't know is, say the industry normalizes to roughly what people make in other engineering fields. Then does everything else normalize around that? i.e. does cost of living go down proportionally in SF and Seattle? Or does all the tech money get further sucked up and consolidated into VC pockets and parked in vacant houses, while we and our trite "cancer research" and such get shepherded off to Doobersville?
Expensive private schools, luxurious ski vacations, and exclusive neighborhoods existed long before the ascendance of software engineers. These had been the purview of investment bankers, high-powered, lawyers, realtors, etc..
For a brief time with big tech, it seemed like intellectual prowess could allow you to jump the social strata. But that arrangement need not exist. It’s perfectly possible, indeed likely that we will revert to the old aristocratic ways. The old boys’s network ways.
> It’s funny that perfect capitalism (no payroll expenses) means nobody has money to actually buy any of the goods produced by AI.
When you remember that profit is the measure of unrealized benefit, and look at how profitable capitalists have become, its not clear if, approximately speaking, anyone actually has the "money" to buy any goods now.
In other words, I am not sure this matters. Big business is already effectively working for free, with no realistic way to ever actually derive the benefit that has been promised to it. In theory those promises could be called, but what are the people going to give back in return?
The economy in the 21st century developed world is mostly about acquiring positional goods. Positional goods as "products and services valued primarily for their ability to convey status, prestige, or relative social standing rather than their absolute utility".
We have so much wealth that wealth accumulation itself has become a type of positional good as opposed to the utility of the wealth.
When people in the developed world talk about the economy they are largely talking about their prestige and social standing as opposed to their level of warmth and hunger. Unfortunately, we haven't separated these ideas philosophically so it leads to all kinds of nonsense thinking when it comes to "the economy".
Money is an IOU; debt. People trade things of value for money because you can, later, call the debt and get the exchanged value that was promised in return (food, shelter, yacht, whatever) I'm sure this is obvious.
I am sure it is equally obvious that if I take your promise to give back in kind later when I give you my sandwich, but never collect on it, that I ultimately gave you my sandwich for free.
If you keep collecting more and more IOUs from the people you trade your goods with, realistically you are never going to be able to convert those IOUs into something real. Which is something that the capitalists already contend with. Apple, for example, has umpteen billions of dollars worth of promises that they have no idea how to collect on. In theory they can, but in practice it is never going to happen. What don't they already have? Like when I offered you my sandwich, that is many billions of dollars worth of value that they have given away for free.
Given that Apple, to continue to use it as an example, have been quite happy effectively giving away many billions of dollars worth of value, why not trillions? Is it really going to matter? Money seems like something that matters to peons like us because we need to clear the debt to make sure we are well fed and kept warm, but for capitalists operating at scales that are hard for us to fathom, they are already giving stuff away for free. If they no longer have the cost of labor, they can give even more stuff away for free. Who — from their perspective — cares?
Money is less about personal consumption and more about a voting system for physical reality. When a company holds billions in IOUs, they are holding the power to decide what happens next. That capital allows them to command where the next million tons of aluminum go, which problems engineers solve, and where new infrastructure is built.
Even if they never spend that wealth on luxury, they use it to direct the flow of human effort and raw materials. Giving it away for free would mean surrendering their remote control over global resources. At this scale, it is not about wanting more stuff. It is about the ability to organize the world. Whether those most efficient at accumulating capital should hold such concentrated power remains the central tension between growth and equality.
The gap for me was mapping [continuing to hoard dollars] to [giving away free goods/services], but it makes sense now. I haven't given economics thought at this level. Thank you!
It's really simple: if you crash the market and you are liquid you can buy up all of the assets for pennies. That's pretty much the playbook right now in one part of the world, just the same happened in the former Soviet Union in the 90's.
Haha "customers". Nobody cares about customers anymore. Investors are all that matter. They're the ones who want to see AI. Once you've pleased the investors, profitability is just accounting tricks.
As the adage says, if something is free, you're not the customer, you're the product.
The funny thing is that the rest of the software industry is dying, except for the trillions of venture capital being invested into these AI coding whatevers. But given the slow death of software, once these AI coding whatevers are finished, there's going to be nothing of value left for them to code.
But I'm sure the investors will still come out just fine.
Pretty slick. I've been wanting something like this that gets stored with a hash that is stored in the corresponding code change commit message. It'd be good for postmortems of unnoticed hallucinations, and might even be useful to "revive" the agent and see if it can help debug the problem it created.
reply