I remember have a very long argument with a team I worked on, who were mostly convinced our service couldn't handle requests any faster because the CPUs were already at 100%. In the end we tripled the requests-per-second by cleaning up the service's memory access patterns...
Similarly, someone told me the memory consumption of a certain data structure couldn't be smaller because it had to store the full day's history of a thing rather than the snapshot state. We reduced it by about 30% in the end just by optimizing memory layouts and removing redundant data.
This is the kind of thing that needs a “things every programmer should know about…” blog post.
I would estimate that a single digit percentage of my colleagues are aware that, according to `top`, sitting around picking at your fingernails while you wait on the memory controller to run across town and back counts as working your ass off.
