Finding problems is optimizing for the customer. Avoiding false positives is optimizing for the developer. Which is right depends on your org's culture.
If I flag every line in your PR as a potential security bug then I have 100% recall.
Obviously you need a mixture of high recall and low false positive rate. If 7/8 flagged items are fine its much more likely people will ignore the warnings, much like they would any security tool with a 90% false positive rate. That is not optimized for the customer.
The ideal is finding all the problems without getting any false positives, but the reality is that you can't often have that. An org's engineering culture should be designed to fix problems with systems. If you're seeing an 87.5% false positive rate that should be seen as another engineering problem to fix. However, that's a separate issue to whether or not you accept false positives in a system designed to find problems.
Presenting it as either a system that misses real problems or a system that has a huge number of false positives is a false dilemma. You can have a system that's designed to find all the problems and then optimize it to reduce the false positives. If you can't reduce the number then you optimize to identify false positives as fast as possible. Just ignoring the identified problems on the assumption that they're false is giant red flag and a signal that the org has a very a broken engineering culture (but, as you say, that's quite common.)
> If I flag every line in your PR as a potential security bug then I have 100% recall.
No. A code review isn't about "flagging a line of code", it's about identifying an issue or a risk. If a 10-line PR has one issue and you leave a comment on every single character, if you still miss the issue you have 0% recall.
Of course not, all these sloppers are doing is training the models so at the eyes of management they are good enough for a replacement. The ones who stay will have 10x more work.
And economic progress for the companies. What these engineers who stay get is just a burnout and a pat on the back. I've seen it, and felt it, many times way before all this slop-coding started
It's a good thing we're ruining the climate so that we can also erode job quality and social interactions.
I can only hope aerospace and medical industries are raising strict constraints against this slop otherwise I fear for the future. Eroding engineering AND communication? Thats a good formula for success
I'm on the same boat I'd rather be left behind than play this stupid game.
It's specially hurtful to see open source developers and projects using and supporting the tools of companies which blatantly stole their work - and now profit from it - and that are actively against the open source ideals (Anthropic when trying to close github repos with their code)
I've stopped all my monetary support for open source projects, and moved all my code to a self hosted instance.
I'm not against AI tools I see where they can be useful, I'm just morally opposed to and disgusted of the ways these companies work.
reply