Sure, but making progress is easy to see. Writing maintainable code is equally a trade-off, you don't want to overbuild and overcomplicate it either. I'm also not convinced that codification ultimately leads to more maintainable code. There's a reason jokes exist about "enterprise Java".
> In short, there is a difference between bad code and exploratory code.
Is there any way to know that a priori? Or is exploratory code simply the ones that were successful, and bad code the ones that weren't?
"Software engineering" is like other engineering in that trade-offs must be considered, and different aspects optimised for. And I broadly agree developers should be aware of what aspects are valuable to the business. A few developers never want to hear about the business needs, but usually it's a failure of the C-suites or tech leads to built a culture that communicates and emphasises those needs, especially to junior developers.
I wrote an OS initialization file parser long ago. Wanted to be able to add new items simply. So, could either use a pattern to recursive-descent parse the file (that could be extended by cut-paste-edit), or could write a table-driven parser. Neither was more or less complicated, really.
So I wrote the table-driven one. And the table was a text syntax, which I parsed in the recursive-descent style. But its syntax was far simpler than the text it would parse, containing only 'exact match', 'zero or more of', 'one or more of' elements. From that, you could create pretty much any kind of initialization file we needed by providing a text pattern.
Did I over-complicate it? The syntax was extended half a dozen times only. But did that matter? It took me about the same time to write it this way (half a day).