It remains to be seen whether these tools are actually a net enhancement to productivity, especially accounting for longer-term / bigger-picture effects -- maintainability, quality assurance, user support, liability concerns, etc.
If they do indeed provide a boost, it is clearly not very massive so far. Otherwise we'd see a huge increase in the software output of the industry: big tech would be churning out new products at a record rate, tons of startups would be reaching maturity at an insane clip in every imaginable industry, new FOSS projects would be appearing faster than ever, ditto with forks of existing projects.
Instead we're getting an overall erosion of software quality, and the vast majority of new startups appear to just be uninspired wrappers around LLMs.
Another thing here is that LLMs don't have to be a productivity boost if it lets you be lazier. Sometimes I'll have an LLM do something and it doesn't save time compared to me doing it but I can fuck off while it's working and grab a drink or something. I can spend my mental energy on hard problems rather than looking through docs to find all of the right functions and plumb things in the code.
I'm not necessarily talking about AI code agents or AI code review (workflows which I think are difficult for agentic workflows to really show a tangible PoV against humans, but I've seen some of my portfolio companies building promising capabilities that will come out of stealth soon), but various other enhancements such as better code and documentation search, documentation generation, automating low sev ticket triage, low sev customer support, etc.
In those workflows and cases where margins and dollar value provided is low, I've seen significant uptake of AI tooling where possible.
Even reaching this point was unimaginable 5 years ago, and is enough to show workflow and dollar value for teams.
To use another analogy, using StackOverflow or Googling was viewed derisively by neckbeards who constantly spammed RTFD back in the day, but now no developer can succeed without being able to be a proficient searcher. And a major value that IDEs provided in comparison to traditional editors was that kind of recommendation capability along with code quality/linting tooling.
Concentrating on abstract tasks where the ability to benchmark between human and artificial intelligence is difficult means concentrating on the trees while missing the forest.
I don't foresee codegen tools replacing experienced developers but I do absolutely see them reducing a lot of ancillary work that is associated with the developer lifecycle.
> I've seen significant uptake of AI tooling where possible.
Uptake is orthogonal to productivity gain. Especially when LLM uptake is literally being forced upon employees in many companies.
> I do absolutely see them reducing a lot of ancillary work that is associated with the developer lifecycle.
That may be true! But my point is they also create new overhead in the process, and the net outcome to overall productivity isn't clear.
Unpacking some of your examples a bit --
Better code and documentation search: this is indeed beneficial to productivity, but how is it an agentic workflow that requires individual developers to adopt and become productive with, relative to the previous status quo?
Documentation generation: between the awful writing style and the lack of trustworthiness, personally I think these easily reduce overall productivity, when accounting for humans consuming the docs. Or in the case of AI consuming docs written by other AI, you end up with an ever-worsening cycle of slop.
Automating low sev ticket triage: Potentially beneficial, but we're not talking about a revolutionary leap in overall team/org/company productivity here.
Low sev customer support: Sounds like a good way to infuriate customers and harm the business.
If they do indeed provide a boost, it is clearly not very massive so far. Otherwise we'd see a huge increase in the software output of the industry: big tech would be churning out new products at a record rate, tons of startups would be reaching maturity at an insane clip in every imaginable industry, new FOSS projects would be appearing faster than ever, ditto with forks of existing projects.
Instead we're getting an overall erosion of software quality, and the vast majority of new startups appear to just be uninspired wrappers around LLMs.