A big win of LLMs in my book is the absolute reduction of commit messages with just “fixes” or “updates”. Commit messages have become more meaningful and useful, even if far from perfect.
We have one dev who uses LLMs to write the code, but still commits by hand. Most of his messages are of the type above, and none of them are useful.
I think that the idea of generating a commit message based on the content (diff) of the commit is fundamentally wrong.
Even before coding harnesses become mainstream, a lot of tools offered to automatically generate commit messages based on the diff (I think JetBrains IDEs started to offer it very early) and I always cringed when I've seen it.
The reason why I don't like diff-base commit messages is because they are redundant. If I want an LLM-generated summary of the diff, I can easily generate it myself, there is absolutely no reason to put it in the commit message.
What I would like to see in the commit message is some additional context that is not a part of the diff. I don't need to read what has changed, because I can already can see it in the diff (or get an LLM to summarize it for me). But I often do need to understand why this change was made. What were they trying to achieve? That's the important part that is not contained in the diff itself. And this is the part that diff-base commits rarely contain.
I think that even something simple like a link to a Jira (or other bugtracker ticket) is much more helpful than diff summary. Maybe instead of "fix" or "update" you could write just a couple of words about what are you trying to fix and why does it need updated. Still better, than a diff-summary.
This is why, if you have the agent generate the commit message at all, it has to be the same agent that wrote the code, in the same session.
I usually have it lead with a paragraph or two of context on why the change was made (though I often use it as just a tool to turn my rambling on that question into proper English), followed by a couple paragraphs of "abstract" explaining the change (because an LLM-generated summary reviewed by the person who made the commit is better than something which the person reading can get themselves).
I don't see how that would improve the situation? If I do it with the same agent/same context, it tends to just report what it did. Which is the diff, repeated in prose.
The agent is usually told why it was making the change (I at least tend to start my prompts with a description of the problem we're solving), so, if prompted correctly, it can lead with "there was a bug, we root-caused it and here's what the root-cause was", then explain what the fix entailed.
> But I often do need to understand why this change was made.
This is the reason (and IMO the only reason) code should be commented. The why can often be hundreds of characters explaining technical limitations at the time, business considerations, or even just "we know this will break in a non-trivial number of instances but it's an incremental step forward to our end goal so we're just doing it."
I'm not sure that Claude's "Realigned the shape of the load-bearing ownership gate to reduce the blast radius of the design contract; confirmed, not assumed" is more meaningful than "fix".
> I'm not sure that Claude's "Realigned the shape of the load-bearing ownership gate to reduce the blast radius of the design contract; confirmed, not assumed" is more meaningful than "fix".
For PR/commit descriptions, I mainly use Claude Sonnet 4.5. It isn’t perfect, but it produces significantly less of this weird gibberish than 5.x models or even Opus 4.x do
I also use an iterative process in which it writes the description, I read it, and then either manually edit it or ask it to make changes
I use Astra at Very High, and shit is still bad. It doesn't actually understand anything, so it often says things which are clearly not needed to be stated. Recently, I've learned that I have very high standards for these things. For example, "fixes" as a commit msg just is NOT acceptable and would never fly where I work.
A few things, but generally more chain of thought before generating a response. So the model is tuned to think more. Given that what it outputs for this task is a summary of its thinking, tuning it to think more will just make a more verbose, less useful commit message.
Tune your model parameters to what is right for the task, not the highest you can afford.
Astra is such a mixed bag. It makes some amazing reviews and sometimes architecture suggestions that I like. But it’s also lazy and will just make up things.
Have another model (or even another instance of the same model) review the output of the first.
Models will hallucinate. They are also quite good at spotting hallucinations in other models' output (with some more hallucinations thrown in). With a threshold for confirmation, and a few iteration loops, you arrive at a fixed point where every claim is supported.
Honestly you probably want a model that has only been trained on language and literature. Nothing from online discourse.
And even then… writing is personal expression. Here people are talking about commit messages. That’s fine but AI doing writing for anyone and I WILL NOT READ IT unless it’s literally basic tech manual.
We read to hear and engage with people’s thoughts. If someone outsources that to AI then they should be shunned.
So long as there are in fact those things, and so long as it didn't sneak something else in there at the same time, it being just on the knife-edge between sense and word-salad is better than "fix".
Buuuuuut far to often it says it fixed a bug I reported, when it only touched one superficial failure mode rather than the root cause.
Yesterday's issue: Why is zoom/pan randomly failing? It told me it was because it was applying a transformation matrix with every input and sometimes JavaScript gave it a non-invertible matrix (why?) which then propagated NaNs everywhere and you can't update a matrix filled with non-numbers.
Why was it doing that in the first place? Seems to be because it's too motivated to perform quick wins and not sufficiently motivated to do good engineering.
Good thing this was just a game editor. Spiky intelligence: superhuman on some dimensions, total noob on others.
It is, because it includes the key words such as "ownership gate" and perhaps others, which makes it infinitely more informative than just "fix", by pointing at what was in scope, and what wasn't.
I've never had Claude spew that out in a commit message. That being said, I also use my tools properly. Your comment sounds like the type of thing someone who puts "don't make any mistakes" at the end of a prompt and gets upset when it doesn't come out perfect. Like, someone adding "make it better than AAA" is going to do something. At this point, it's really just telling on yourself that this is the output that you get.
Mildly disagree. "Fix" is exasperating but immediately tells me I need to look at the diff. With unconstrained Claude spew, I need to wade through three levels of deep fried LLM-speak before realizing...I need to look at the diff
"Deep fried" is the perfect analogy. LLMs have been trained on themselves so many times that their output is the linguistic equivalent of many rounds of JPEG compression.
I don't get this thread. Just tell the agent to use conventional commit messages and to keep it nice and tight. Are you all just raw dogging agent output with no alignment/conventions?
> Just tell the agent to use conventional commit messages and to keep it nice and tight.
I have this in my claude.md along with guidance on (not) writing comments but it’s still dumps multi-paragraph comments of Claude speak everywhere.
I think a lot of us are talking past eachother, but what I and I think others are complaining about is doing things well, and being minimal. LLMs don’t really do that yet, they can’t look at and simplify a codebase well, even with guidance. They always seem to add rather than subtract. And eventually it becomes an issue. I think they were actually better in this regard with like Opus 4.8, and are now getting even worse (but score better, and are more autonomous).
At home I use Codex, and it’s better as far as language goes at least.
Anyway, it’s like the logic piece necessary to do good work is still missing, and hurt by recent reinforcement fine-tuning. And it leads to massive bloat since more comments = higher scores, but walls of text and the tokens (or cognition if you’re the sad human reading them) aren’t free.
Commit messages as you describe ("fixes", "updates") are inappropriate in any professional context and some coaching should occur to the people doing them.
I have the opposite issue - some of my team members now submit mini-essays generated by the LLM. Like 300-500 word commit messages with everything from the essence of the change up to philosophical design trade off discussions.
Like most writing, what is left out is as important as what is included.
You've gotta be joking. Every AI commit message, summary, and test plan is full of extraneous garbage I didn't need to read (and a lot of it is not understandable)
Fix your harness. Provide rules, format, and good examples.
You probably had those for the humans in your team for when new hurts join, right? Just direct your LLMs to them and you’ll have a much better experience.
Just like you can’t expect a new junior dev to know what to do without direction, the same applies with LLMs.
I can relate to that. Most human commit messages aren't good because the author can't be bothered to write a good one, or doesn't know what a good message is supposed to be, or is bad at writing.
But AI commit messages are still bad. Way too verbose and focused on the wrong level: that of code mechanics. That's just wrong. Messages should focus on the level of intent and design, with the primary purpose being to aid human review. They should include a high level overview of the change, decisions, caveats, information not obvious from reading the diff.
So I wrote a skill that captures these principles and allows the agent to even research past related commits and to ask focused questions in order to uncover the intent rather than guessing and writing a bad one.
Now the AI writes better commit messages than it used to, and even writes better than most humans (who can't be bothered to write a good one). Not better than a good manual message, but you can't have everything.
But sometimes even the good writers are tired or didn't think things through. Being able to compare with the AI's version is still useful.
We have standards we ask devs to follow but I’m not going to make someone go back and change a commit message after the fact. I more care about pull request body copy and I think the rules we have for that have been pretty well adhered too. Don’t even need ai to enforce it.
An LLM can read the diff and do the summary on the fly. What it cannot do, is explain the reason behind a change. That was always that valuable part of the commit msg. The rest can deducted, ofc it's much easier to read a short list or a descriptive title.
Sometimes minimal information is better than extrapolated information, especially when the extrapolation mentally painful to read and has decent probability of being wrong with inaccuracies that waste people's time.
Also:
1. It's often a premature extrapolation from diffs and tickets which you could probably generate later, at need, and likely with even better results.
2. If those commit messages can ever influence future work, then it's just carcinogenic cargo-cult cruft. Pruning outputs (future inputs) is important to prevent weird unwanted drift.
Yeah, I know people who have the same zero-explanation `git commit -mfix` style that they did pre-AI and am baffled. If you can’t be arsed to explain yourself, let the agent do it. It literally takes less work to tell the agent to commit for you and they will always do better than -mfix. I don’t understand it either other than obstreperous “become ungovernable” attitude.
LLMs can provide the intentions of the code they write. To say they can’t is saying that they can’t fix bugs, which they very obviously can do effectively.
So, just tell them to write the intention in the way you want it.
If you are implicitly providing the direction in the prompt, make that explicit - you then it will have the context for the change while working and for the commit message and you’ll likely get a higher quality output.
Disagree. I've seen so many PR slop grenades -- they are essentially unreadable. I'd prefer a paragraph or two human written explanations. I suspect people are just taking the default AI text -- which is way too wordy.
We have one dev who uses LLMs to write the code, but still commits by hand. Most of his messages are of the type above, and none of them are useful.