This tendency has been noted since at least the Manhattan Project, described by none other than Richard Feynman. The prescription for oneself is pretty simple - just say no! It is relatively easy to catch yourself whenever you think about some improvement that's irrelevant to delivering so long as you have convinced yourself that you need to wholeheartedly focus on delivering.
It is more difficult to persuade others not to over-engineer. I have tried and failed many times to do so. In fact, if you try too hard you may just make them hate their job. Folks get into software development for a whole host of reasons and only one of them is shipping. They may not be satisfied with a job where they funnel requests from a PM directly into code with little creative input. I'm not sure I can give good advice on this front, other than to look out for certain red flags during hiring.
> The prescription for oneself is pretty simple - just say no! It is relatively easy to catch yourself whenever you think about some improvement that's irrelevant to delivering so long as you have convinced yourself that you need to wholeheartedly focus on delivering.
That's fairly true if you intend a monomaniacal focus on delivery and you are dealing with improvements that are truly irrelevant.
If other concerns like code hygiene have nonzero value, and/or if improvements have some potential relevance to delivery but are not unmistakably essential, then things get more interesting. Doing the least work possible to get by with a work item may be good for initial velocity but at long term cost.
It's probably easier in an environment where things like code hygiene expectations and review standards are well-known from a combination of express standards and team experience, but not all environments are like that.
Two points here. First, I think the primary trap is that developers think they can judge what code is hygienic. This is the evil twin of the notion that developers can judge what part of a system is slow without profiling.
Developers regularly hold religious wars on basic matters of code style. While you may profit from 'hygiene' there be dragons - developers often consider as hygienically critical such tasks as refactoring React class components into functional components with hooks. The industry is not full of Edsger Djikstras. In practice the sub-optimal greedy algorithm gets the job done since it encourages a straightforward approach to the problem at hand.
The secondary trap is that developers think they can amortize up-front development costs over a long period of time. Projects get the axe if they do not deliver. The longer you spend writing a feature, the longer you wait to test your hypothesis that it is valuable. In many cases the odds that the feature is valuable run below 50%. You may be laying a solid foundation when all that was asked of you in the first place was a shantytown.
Everyone thinks they are the reasonable person with discerning taste who only refactors code as necessary to optimize total cost. In practice that person is as mythical as a unicorn. It is very difficult to convince developers this is true, but true it remains.
> First, I think the primary trap is that developers think they can judge what code is hygienic.
If software engineering is engineering, they can. It may be a skill that is weaker and stronger between different engineers varying particularly by relevant experience, and it may require complex and context (including present development team) sensitive evaluations, but it is a real skill that exists.
> This is the evil twin of the notion that developers can judge what part of a system is slow without profiling.
Well, it's not, because there is no analog to the “without profiling” part. It's true that there is an analogous tendency of prejudging problem code, but certainly, at a minimum, hygiene issues can be discovered by experience of problems of development/maintenance stemming from, e.g., code duplication for shared functionality instead of use of shared abstractions.
> The secondary trap is that developers think they can amortize up-front development costs over a long period of time.
Note that when I said long term I don't necessarily mean a “long period of time” but “some period longer than the minimum development time of the present item”...
> Projects get the axe if they do not deliver.
Yes, they do, but plenty of real projects aren’t, especially at initiation, delivering actual value every iteration, just progressively refined demonstrations (this is particularly common on replacements that, for whatever reasons, can't go the strangler/ship of Theseus appproach, and on those projects the team often has an idea where things need to be for real delivery. It's quite possible for code that makes subsequent tasks more costly even though it saves time this iteration to delay real delivery.
> Everyone thinks they are the reasonable person with discerning taste who only refactors code as necessary to optimize total cost.
No, in my experience that's not even approximately true. Most developers I've encountered think that, in their current team environment, they individually have a natural tendency either to excessively favor direct solutions that produce ugly code with outsized downstream cost or to go a few steps too far with overengineering abstraction (and most of them also recognize that that that overall tendency is only on average, and that they also miss on the other side sometimes.)
My point is that when you get beyond the simplistic rejection of things which have no relevance to the task at hand (which isn’t a scenario that occurs all that much), evaluating what is the right balance is nontrivial.
> If software engineering is engineering, they can.
Therein lies the question :)
> Well, it's not, because there is no analog to the “without profiling” part. It's true that there is an analogous tendency of prejudging problem code, but certainly, at a minimum, hygiene issues can be discovered by experience of problems of development/maintenance stemming from, e.g., code duplication for shared functionality instead of use of shared abstractions.
Even in this case we are limited. I've seen many cases where someone correctly identifies a piece of code that is painful to maintain and then dives in only to multiply the problem. We are even better off in performance land because we can test our code immediately after we write it to verify we haven't left things worse.
> No, in my experience that's not even approximately true. Most developers I've encountered think that, in their current team environment, they individually have a natural tendency either to excessively favor direct solutions that produce ugly code with outsized downstream cost or to go a few steps too far with overengineering abstraction (and most of them also recognize that that that overall tendency is only on average, and that they also miss on the other side sometimes.)
I don't want to wade into the thicket on this one except to note that it's possible for someone to articulate something they may even believe on some superficial level without truly internalizing that belief and that IMO I observe this behavior a lot.
I don't want to invalidate your point outright because I think it's valuable and I even agree that it's rare to encounter a code base that would not benefit at all from 'code hygiene.' All models are wrong but I think an aggressive focus on delivering functionality is a useful one in most contexts. Put another way, a good but imperfect rule of thumb is to refactor only in order to gain something tangible for end-users.
It is more difficult to persuade others not to over-engineer. I have tried and failed many times to do so. In fact, if you try too hard you may just make them hate their job. Folks get into software development for a whole host of reasons and only one of them is shipping. They may not be satisfied with a job where they funnel requests from a PM directly into code with little creative input. I'm not sure I can give good advice on this front, other than to look out for certain red flags during hiring.