Tbh, I still wonder why people even bother releasing such tools that were shitted out by an LLM in an hour or two, these are basically the modern equivalent of a hastily cobbled together 10-liner shell script (still useful, but not something you'd ever put up as its own Github project).
Everybody else can write such tools now by themselves in at most an hour, and they get a tool that's more personalized to their tastes. And why have a readme at all when it's just LLM mumbo-jumbo - that sort of mumbo-jumbo is written for LLMs to consume, not humans.
In theory, there is some value in deciding what the LLM ought to shit out, and checking that it works, and maybe even making a design decision or two to try and create something that would be relevant or useful for many people. And then others can possibly save an hour or two, and the data centre can save some water and electricity.
That's 6 more commits than I'm used to seeing for these sorts of efforts, honestly. (But I will cede that people might be doing all the changes locally and then pushing a single commit, without any real understanding of how version control works.)
If folks do this; I'd love for them to also be in the habit of shipping their prompts + harness configuration too. If we are moving to another code abstraction, then that becomes the source code to modify, adapt and extend.
I mean -- compilers used to annoy developers who knew how to hand-roll machine code, with bad outputs and inefficient algorithms, till the compilers got better than most of them...
I don't quite agree with the Assembly => Compilers => LLMs as some sort of "stages of evolution" argument tbh.
LLMs are an additional way to generate code, but it's not like code generation or "low-code" tools are a new thing that made compilers and programming languages obsolete.
LLMs "just" allow to provide the code generation input directly in human language. But whether providing such a specification in "sloppy" human language instead of a precise specification language is actually a good thing for long term projects is still arguable (as convenient as it is).
Mathematicians and engineers invented their own precise "DSLs" (e.g. math notation and blueprints) for exactly this reason, and high-level programming languages are nothing else than "specification languages" that don't leave room for the ambiguities of "natural languages", and it's still not clear to me how having both a natural language specification and a precise specification in form of source code side by side can be considered a good thing. It's easy to say of course that the "natural language specification" is the single source of truth and the source code is a throwaway output artifact, but then you're back at having a specification written in a language that's useless for expressing precise intent.
(and this is also the fundamental difference between compilers and LLMs and why I believe that comparing LLMs to compilers is wrong: determinism. The output artifacts of a compiler are literally throw away. Nobody in their right mind would commit object files into version control, while nobody in their right mind would not commit the source files generated by LLMs into version control, because creating the source files from scratch would yield different results on each run).
Releasing the harness as part of LLM-generated projects still makes sense of course :)
Everybody else can write such tools now by themselves in at most an hour, and they get a tool that's more personalized to their tastes. And why have a readme at all when it's just LLM mumbo-jumbo - that sort of mumbo-jumbo is written for LLMs to consume, not humans.