Here's a thought: Does it count as AI written code if you're basically just telling it to fill in a template?
My common use case is basically just writing some code in the style I use. Then I'll tell the LLM, "Do X based on Y using the rest of the methods in Z". The code I would write would look exactly the same. I just the LLM as a shortcut to getting the code written.
Like, I'm not depending on any sort of knowledge within the model. I just use its natural ability to be a smart templater.
If using AI this way counts as AI written code. Then sure, I reckon you could easily get to 90% AI written code in loads of applications. But in terms of developing a product or feature from just plain english prompts alone? Definitely lower, and definitely babysat with someone with software development knowledge.
I call that AI-assisted programming and I think it's a very appropriate way to use this tech - it's effectively a typing assistant, saving you on time spent entering code into a computer.
Lately I’ve done a lot of asking Copilot to do things like write a Provider for a ContainerRequestContext in JAX-RS which turned out the same as if I wrote it myself but I would have spent more time looking up things in the documentation.
I had a bunch of accessibility-related tickets that were mostly about adding aria-attributes to React code involving mainly MUI components and got good answers off the bat with the code changes made but going back and forth with it we found other improvements to make to the markup beyond what was originally asked for.
The notion of replacing programmers is not well thought out most of the times i see it.
At some point, you have to be able to replace all of them (or all the work they currently do), or else all you do is screw up the pipeline you need.
I've seen lots of AI companies who believe they will replace junior software engineers, and leave the senior ones in place to do really complex things and drive the AI.
One serious problem with this, obvious, but often totally ignored or handwaved, is that senior software engineers do not grow on trees, they were yesterday's junior software engineers. So once you've gotten rid of all the junior software engineers, you will no longer generate any more senior ones. When combined with likely increased attrition of senior software engineers, seems like a wildly bad plan. Or at least, something that needs to be taken into account but never is.
Now, plans to replace all of them are at least logically coherent, but also seem far-fetched at best right now.
By numbers, people do appear to be replacing software engineers with AI (the number of new software engineer jobs has dropped something like 30%, even relative to other jobs, but no linked causation obviously), but every company i talk with that is doing this keeps telling me they are just replacing their junior engineers, which is hilarious - they all expect they can just continue to hire senior engineers they need from "somewhere else". Freeloading works until it doesn't.
I suspect we have this problem already in SRE. Back in the day, there were lots of manual processes to do to check on the health of servers, install packages, troubleshoot network problems. Now, there’s this mountain of ops knowledge you have to know to be productive, not to mention also have software engineering skills, but there’s very little manual tasks to do to step stone your way to that knowledge.
I have yet to see any evidence that AI companies are actually thinking about what they’re saying. They’re just trying to move markets and make money.
In some cases they’re saying two different things. For example, Microsoft (arguably more a general software company desperately chasing AI rather than an “AI company”) would in one breath have you believe that the modern workplace will not survive without AI, while in the next breath Nadella is on the record noting that AI can’t exist without an ROI, and they’re not yet seeing an ROI that justifies it generally.
The OpenAIs of the world can’t imagine a world where the premise of their product isn’t the default/a given, so they simply engage as if their vision is entirely sensible.
> So once you've gotten rid of all the junior software engineers, you will no longer generate any more senior ones.
it's because those who are making these replacement decisions are at the CxO level, and they think software engineering is like factory workers. Cogs. A commodity that will always exist.
And not to mention the whole process of senior decay will happen over a long period of time - these CxO's have retired by then with their golden parachute.
> He disagreed with a recent prediction from Dario Amodei, the CEO of Anthropic, that 90% of code may be written by AI in the next three to six months.
> "I think the number is going to be more like 20-30% of the code could get written by AI — not 90%"
I kind of agree with him. Definitely not in six months :)
I can kinda see it doing 90% of the code by volume, but it would be the easy 90%. And the remaining 10% will need to be done by someone who actually knows what they’re doing.
Still amazing for time saving but so far I haven’t been able to get even a SwiftUI class written to the standard I’d use it as-is.
There will also be Jevons Paradox happening: LLMs reduce the “cost” of writing code, so more code gets written.
I already find myself building prototypes that I never would have attempted previously; it’s just too easy to have the LLM whip one up. LLMs also make it easier to do “grunt work” coding of writing validation checking and unit tests on non-critical code where previously I wouldn’t have bothered.
I agree with you. The same way compiling code meant that 99% of the code was machine-generated, but you still wrote tons of code.
Lots of code is boilerplate or large switch statements (esp in OOP), or other such things. All of which AI code completion makes a breeze. The actual hard parts of code (often the various ways of connecting two disparate systems), AI doesn't really help with unless you tell it what to do and how, so you're still the ultimate important decision-maker in this scenario and can't be "replaced"
Agree, excellent for commodity boilerplate where there is basically just a bunch of 1:1 mappings that need to be filled. Terrible for novelties and problems in a large solution space. The more degrees of freedom there is to fail, the worse it will work.
Maybe, but I don’t think we’ve definitively answered the question - are LLMs smart.
If LLMs get to the point that they can unpack almost any logic that one might need for a function or class that’s amazing, but if it doesn’t understand the why, I think that leads to fundamental limitations in progress.
And maybe I’m still not using the right tools, but I’m not finding cursor/claude 3.7 to be close.
I think LLMs are intelligence augmenting. They are a tremendously powerful search tool but you need to know what you are looking for.
The idea that they can replace workers is laughable. I am afraid though that the current state of LLMs is too expensive for its use case. So I'm trying not to lean too much on them.
It depends how one measure it. If you look at the total output if GitHub in a year and try to poorly extrapolate how much code is written in a year - and then Anthropic measures how much code their system spewed out, then sure maybe they emit 90% of the estimated total amount of code. But most people wouldn't define it that way.
Maybe that's why the models are so eager to spit out reams and reams of code, it lets their masters claim a higher percentage (even if most of the code they emit is never used).
I've yet to see a LLM asking any critical question, in order to understand the problem better. Without this, it can never be a good programmer because good programmers ask questions.
This is a really good point I hadn't thought of before. How come LLMs always claim to understand exactly what you mean with no clarification or context?
It's sort of immediately ridiculous that these LLMs could replace real people.
I’m starting to get suss about any percentages being thrown around, not because I don’t believe the numbers but because I don’t believe the context.
First of all, I’m writing a game and the game is a 2 player game, and I noticed it rewrote the “move player” function for each player (move_player_1 and move_player_2) and they were the same, and this was like a poorly written 50-100 line function with an excessive number of comments, so is AI really writing 20% of code that would have been written? This is like evaluating the quality of a software engineer by the number of lines of code they write.
Second. Let’s say you thought you were going to get a consistent 20% improvement. That would be amazing, but there’s two ways to conceptualize that.
One is, if I have 5 engineers managing 5 pieces of software, I can do the same work with 4. But can engineers really take on the mental load of managing another piece of potentially complex software?
The other option would be, I have 5 engineers managing 5 pieces of software, and we get 25% more work done. I think this is more likely. Every software project has a mountain of tech debt or nice to haves that you don’t have time to finish. Maybe AI makes software writing more pleasant because you can get more done in a sprint/pr, but doesn’t actually require less software engineers.
This is close to being an appeal to authority. These companies - and IBM is the classic case of this - are the last to pick up on new trends and are usually blindsided by new technologies. If AI was perfectly superhuman IBM's CEO is still going to be one of the the last people in the tech world to pick up on it. He'd hopefully be ahead of other industries.
Startup CTOs are the people deciding how much code is going to get written by AI. They're going to be building products in new, untested ways. Mostly failing, some succeeding. Technical growth doesn't come from established majors. The entire AI boom left Google doing a cartoonish impression of being run over by a gaggle of bystanders heading to the next big thing and they thought they were actively researching and leading the field of AI.
As news of what IBM is doing he's a leading authority, but following IBM's exploits is going to be skating to where the puck was last year on a good day.
It's not an "appeal to authority" to report on the opinions of tech leaders. The only way it would make sense to call this an appeal to authority is if you think TechCrunch is making an argument for a side. This just looks like ordinary reporting, however.
But it is close to one to look to him as an authority on where AI is going. He's technically only commenting on exactly what IBM is going right now; which isn't where we expect the interesting things to happen.
If all code is written by AI or by "vibe coders" then what happens when companies are selling products they don't themselves understand? - sounds like a recipe for disaster to me.
I think it's pretty obvious that generative AI isn't replacing the need for engineering - ever.
This happens regardless as people leaving companies and whatever they produced. You will find so many examples of code out there in production and works through hopes and dreams.
And you know what? If it works it works - regardless how it was produced.
I would be very interested to know from everyone here:
What is the most impressive thing you have managed to get an AI to code - WITHOUT having to babysit it or give it any tips hints or corrections that a non-coder would have been unable to do.
So far most impressive AI achievements for me were both by ChatGPT o1 pro:
Case one. I configured IPsec VPN on a host machine which run docker containers. Everything worked from the host itself, however containers were not able to reach IPsec subnet. I spent quite a bit of time, untangling docker iptables rules, figuring out how iptables interacts with IPsec, running tcpdumps everywhere. However my skills were not enough, probably I would resolved the issue given more time, however I decided to try ChatGPT. I made a very thorough question, added everything I tried, related logs and stuff. Actually I wanted to ask the question on some Linux forums, so I was preparing the question. ChatGPT thought few minutes and then spewed one iptables command which just resolved the issue. I was truly impressed.
Case two. I was writing firmware for some device using C. One module was particularly complex, involved management of two RAM buffers and one external SPI buffer. I spent two weeks writing this module and then asked ChatGPT to review my code for major bugs and issues. ChatGPT was able to find out that I used SPI to talk to FRAM chip, it understood that my command usage was subtly wrong (I sent WREN and WRITE commands in the one SPI transaction) and highlighted this issue. I tried other modes, I also tried Claude, but so far only o1 pro was able to find that issue. This was impressive because it required to truly understand the workflow of the code and it required extensive knowledge of protocols and their typical usages.
Other than that, I don't think I was impressed by AI. Of course I'm generally impressed by its progress, it's marvellous that AI exists at all and can write some code that makes sense. But so far I didn't fully integrate AI into my workflows. I'm using AI as Google replacement for some queries, I use AI as a code reviewer and I'm using Copilot plugin as a glorified autocomplete. I don't generate any complex code with it and I rarely generate any meaningful code at all.
I can take my blog or website, fire up aider, and ask it to completely reformat or restyle it or add effects. It does a fantastic job at that, and that's something I'd have to pay $20 on Fiverr to get done before because I cannot coordinate colors or style things to save my life.
This is a good example. It's really good at updating old, small projects. Whereas most humans need to build context about the project, and do so slowly.
I wonder over time how those small, infrequent updates might hamper the ability to perform the next, small infrequent update (as your code begins to resemble less and less any examples the AI might have seen and more and more a kludge of differing styles, libraries, etc.), but that's really not any different than how most projects like a personal website operate today.
Well in this case this is a complexity-preserving change. The same few template files get updated with each change. So, I can't say for sure but I imagine this is effectively zero cost.
The conditions you’ve set here are very strict, but I have something that may almost fit. I’ve used Grok 3 to create a decompiler and then a compiler for a made up scripting language (for scripts extracted from Final Fantasy VII data files). I’ve fed it an example of how the low-level “assembly/opcodes” look like and then how I would like the higher level scripting language to look. With minimal guiding (I was telling it what I want, but not how I want it) it implemented a working decompiler and compiler that worked exactly to my specifications, written in Typescript. Created a custom tokenizer, parser to a custom AST structure and everything else that was needed. It took a few prompts to get right (and to add features I didn’t think about initially), but the resulting 1400+ lines of code I found to be very impressive for what a “fancy autocomplete” as many people say could generate.
> I’ve fed it an example of how the low-level “assembly/opcodes” look like and then how I would like the higher level scripting language to look. With minimal guiding (I was telling it what I want, but not how I want it) it implemented a working decompiler and compiler that worked exactly to my specifications, written in Typescript.
I think this already falls out of OP's guidelines, which you pointed out are quite strict. They also happen to be the guidelines an AI would need to meet to "replace" competent engineers.
If there used to be 10 engineers in a team, and with LLM they only need eight because LLM help the team achieve the same workload, then two of them have been "replaced", even though the LLM can't do everything an engineer does.
I haven't heard of any prediction that AI will completely replace all programmers.
This doesn't make sense though unless your company isn't growing. Most dev teams have large backlogs of work and any increase in productivity can lead to an increase in revenue generation.
Programmers aren't paid to generate code. They're paid to figure out how to do X Y and Z business initiatives. Unless you don't have that many initiatives, how much sense does it make to let go of the people who can do that for you? It makes sense when companies are trying to cut costs, but that happens when the cost of borrowing is higher than the realizable profit margin
Exactly. The argument never was that AI will replace every single programmer. But it looks like it will replace a good chunk of them. How many exactly remains to be seen.
I know popular sentiment is writing compilers is hard but not simple ones like this one without optimizations. Basic stages like lexing, parsing, building AST, and code gen aren’t conceptually difficult, just a lot code to churn out.
Though I’ll admit LLMs have been weirdly good at regex.
Well you’re saying this because you’re probably already familiar with the subject. I remember learning about writing your own compiler for the first time and it took me many hours to write the most simple lexer and fully understand it. And now an LLM can write that to my exact specific in minutes instead of hours. To me that’s quite a breakthrough.
"WITHOUT having to babysit it or give it any tips hints or corrections that a non-coder would have been unable to do"
I find that question impossible to answer, because my programming experience influences everything I use LLMs for. I can't turn that part of my brain off.
Getting AI to write code for you starts with understanding what's possible, and that's hugely informed by existing programming knowledge.
I won't ask an LLM to build me something unless I'm reasonably confident it will be able to do it - and that confidence comes from 25+ years of programming experience combined with 2+ years of intuition as to what LLMs themselves can handle.
Nothing that I've cared much about. Little python utility scripts like "merge the rows of these CSV files in such and such a way" are about the only thing that has worked first time without me adding more of the "how" to do it.
Essentially, for "real-life work scenarios", the performance is not that great comparing to gpt-3.5 with the exception of Claude 3.7 and GPT-4.5. Bear in mind, the question was not "challenging" in a "genius thinking required" way.
That being said, I use LLM regularly for discovery. They do waste some of your time because of hallucinations but you can call their bullshit most of the time and if not, it is still faster than Googling.
You steer ChatGPT 3.5 model to find the handlers HashMap issue that it didn’t find initially, but you dismiss the other models for not immediately recognizing it?
Python scripts for managing and automatically editing photos. Anything bigger and once the code reaches a certain point it starts actively breaking things, and in some cases, makes them unrecoverably worse. I have gotten benefit as a programmer being able to break down projects and spot when it's going down the wrong path, but I think you need quite a bit of experience to use AI on large codebases or with types of code it has few training examples of. I've also caught it 'fixing' code by essentially silencing errors without addressing the underlying problems.
Built out an entire web accessibility monitoring / scanning tool. The backend/scanning system required my knowledge as a programmer but the UI was entirely vibe coded. I said "build a UI for this scanning system on Cloudflare workers and use Hono.dev", and then just described in plain English how I wanted the UI to work.
Did this over a few weeks in my free time and now it has all of the features of accessibility monitoring SaaS that I was previously paying $600/mo for.
This is also something I've realized about LLM coding: I have learned wayyyy more about tech I wouldn't normally try just by vibe coding and explaining what I want to exist. In the last few months I've learned Cloudflare workers/queues/Puppeteer, heavy Postgres features like policies, triggers, functions, trigram search, migrations, and a ton about RLS.
I _could_ have learned this stuff on my own but the friction is just too high, but with LLM coding it just shows me a working solution and I can tweak and ask questions from there.
Hono is pretty great, too! It has exactly what I've always wanted in a backend JS framework which is a React/JSX-like way of writing UI components but without actually using the React renderer on the FE. Next.js and SSR obviously also offers this but I don't think Hono even uses that, it's just JSX.
How to write a Presto SQL query to get X result given example Y database where I fill in Y as a csv describing the table with the columns my query needs and fill in X as a csv describing the results I'd like to get out.
It doesn't always work, but I have an easy, quick way to test it out. When it does work, I've often saved lots of time.
I've gotten it to write complex graph layout algorithms, including converting them to all work in the same orientation (about half of graph layout papers are written to produce vertical graphs, and about half horizontal graphs).
This part of a node ui for modeling satisfactory (the game) factories. Mostly for fun and learning, honestly.
So like, I have code to find optimal production chains and solve the node graph, using ILP through pyomo or Z3.
Some of the optimal production chains are ... a lot of nodes, as are plenty of factories.
Without really good layout, it becomes a mess. Existing modelers sort of suck and have no auto-layout.
I bridged Elk (and elk.js, actually) into python, but wanted a fallback since this was ... crazy enough already and who knows if it will work on anyone else's computer :)
So AI wrote me about 4000 lines of just about 100% correct python graph layout code. In batches, one algorithm at a time, that i then combined. I did have to tell it what I wanted piece by piece, so i had to learn a lot about it - i could not get it to do all the pieces at once iteslf. I suspect due to context length limitations, etc.
It comes very close to ELK when it comes to this type of layout (elk supports other layouts, edge routing, etc), and implements the same algorithms with the same advanced techniques.
Wow! I'm glad I asked, that sounds awesome. I hope you get a chance to do a writeup with some screenshots at the the end of it; that would make for a great read. Do you have any code public yet?
And TIL about elk! I've had kind of a half project, called `gstd`, which tries to create a standard library/API to make working with node graphs in code as easy as working with other abstract data types, like Arrays. One of the important bits is that it lets you console log a graph so you can see it. I've been using d3 force directed graphs for layouts, and have played around with some custom layout algorithms, but have yet to find a good solution. Elk might actually be just what I've been looking for there!
Seems like he's just shilling for the (Indian-dominated) 'in-shoring' (H1Bs) and 'outsourcing/offshoring' industries, both of which IBM has been rather active in, the last decade or two.
I think AI will benefit more senior 'architectural' programmers the most.
It won't completely replace programmers of course, and certainly not in 6 months, but it will quite likely substantially reduce the need for the boiler-plate programmers typical of the offshore and H1B industry.
People love to throw jabs at one of the companies that happens to be a champion of patents per year, and also responsible keeping the lights on for many critical projects on the Linux ecosystem since 1998.
""If you go forward 24 months from now, or some amount of time — I can't exactly predict where it is — it's possible that most developers are not coding," said Garman, who became AWS's CEO in June.
"Coding is just kind of like the language that we talk to computers. It's not necessarily the skill in and of itself," the executive said. "The skill in and of itself is like, how do I innovate? How do I go build something that's interesting for my end users to use?"
This means the job of a software developer will change, Garman said."
I guess it's a cultural thing. Our CEO and CTO still commit code every other week (occasionally driving us mad), it's still a very bottom-up org despite having thousands of people now.
They also were pioneers of modern industrial practices. They researched the best approaches to workforce scalability, settling on a model of many distributed offices spread around cheap peripheral suburbs - something that might sound banal today, but it in the 60s/70s, when even factories were supposed to be as close as possible to downtown, it was pretty bold.
IBM has many shortcomings but they did manage to survive and navigate a century of tumultuous technological changes. Very few companies have done that.
"The statements are also a bit of a reversal for Krishna, who said in 2023 that IBM planned to pause hiring on back-office functions that the company anticipated it could replace with AI tech."
I suspect his enthusiasm for AI is going to keep cooling down.
That is different things though -- you can replace functions with AI tech without needing AI to write all the code and that is happening in a big way right now -- Krishna was talking about reducing customer facing roles by around 30% over 5 years in that earlier statement
As of right now the code assistants mostly just make existing coders more productive -- predicting 20 or 30% of new code in near term will be generated is not unreasonable -- 90% is a stretch as has been discussed here many times
IMO: One idea I've been pondering is this idea of contextual continuity in programming, or really any task you may ask an AI to do.
If I code a system, then that system has a bug; or if I use Illustrator to make a drawing, then the stakeholder says they want the tree a different shade of green; correcting that error after delivery is reasonably easy.
If I ask an AI to do either of these things, even if I'm still being paid to be in the room (many supporters would argue this system has replaced me, and thus I'm not even in the room anymore): How does this correction happen? The operator has to take off the vibehat, dig in to the inevitable machine-generated mess the AI has created, and learn everything its done in order to diagnose it (or, keep prompt-praying it comes up with a solution on its own).
Getting past this feels like a fundamental issue with the proposed process. AIs are trained on human data, and prompted by humans, who are imperfect. I don't think its reasonable to assume that AI will ever be able to generate perfect solutions, if only because we will never be able to engineer prompts that adequately encapsulate what the prompter wanted, let alone if the AI makes no errors in its generation.
If we can't get to that, its obvious to me simply knowing how much of my time is spent every day [writing code] versus [debugging existing code] versus [gathering requirements] that AI is doing any significant amount of any of that unsupervised will not result in a more productive system. It will always be most productive for me to leverage (or tightly supervise) AI in its generation, intercepting errors before they compound, directing the flow of output.
I do think the roles and responsibilities of devs is going to change a lot over the next ten years; from crafters to soothsayers or oracles one might say. But I am still very comfortable betting my career on:
1. Someone (meaning: human) needs to do the soothsaying.
2. Those classically trained in software engineering will be the most effective people to do this.
3. The amount of stuff one individual can output will be amplified, but there will still be a reasonable per-capita maximum of output, and thus the industry, in its ever-increasing desire for code and ever-increasing need to manage the output of AI, will continue to demand more soothsayers.
I'm sorry but IBM has moved out of (Software) products and currently only does IT services. Its revenue (from Software) is completely tied to $/hour that it can earn from its contracts. Arvind Krishna is simply trying to talk his business up.
My take: the truth will lie somewhere in between, with the balance moving into the AI corner with time. Also, the real truth will emerge from IBM's financials. The growth (or decline) of their revenue from Software Contracts will speak for itself over the next few years.
Also, from what I'm seeing in the Industry, the number of Software hires have begun to decline with more engineers being let go.
My common use case is basically just writing some code in the style I use. Then I'll tell the LLM, "Do X based on Y using the rest of the methods in Z". The code I would write would look exactly the same. I just the LLM as a shortcut to getting the code written.
Like, I'm not depending on any sort of knowledge within the model. I just use its natural ability to be a smart templater.
If using AI this way counts as AI written code. Then sure, I reckon you could easily get to 90% AI written code in loads of applications. But in terms of developing a product or feature from just plain english prompts alone? Definitely lower, and definitely babysat with someone with software development knowledge.