The existing test cases were created based on understanding the code. Creating good tests that cover every case you care about without understanding the code is not possible.
These big ports we are hearing about are of projects with hundreds (prob thousands) of unit tests. Not only are the models typically trained with the original code bases in their dataset, but the test suites themselves have project architecture encoded in them..
Ultimately there isn't enough information on how to most effectively maintain and evolve large, complex software with LLMs yet. We aren't even a year out from Opus 4.5.
