Yep, or even your own time if you run the program multiple times. E.g., if your code is running on a thousand machines, that CPU-hour savings that cost $0.05 now costs $50.00; past a certain point it does pay to throw engineers at insane micro-optimizations. Or nonlinear payoffs from CPU time improvements -- e.g. meeting a realtime deadline or else the system is useless. The "CPU-hour vs. engineer-hour" comparison works only when the engineer him/herself runs their own program, exactly once, without other constraints.
> The "CPU-hour vs. engineer-hour" comparison works only when the engineer him/herself runs their own program, exactly once, without other constraints.
Or when the system is designed such that you can scale hardware to decrease run time. Sometimes the choice is between throwing programmer time at a performance problem or throwing hardware at it, and often the economics work out in favor of the latter.
Sure, but I think this is just reiterating the original point that hardware is cheap and programmers are expensive. If you need to meet a performance goal and hardware is cheap but programmers are expensive, then throw hardware at it. The point I was (trying to) make is that when you have N instances of the problem to solve (where N is large), often the cost of "scaling the hardware" is prohibitive; optimizing performance by 1% might yield operational cost savings that justify the programmer's salary, simply because the 1% is multiplied by N. This is orthogonal to parallelism/scalability.
I'll grant that this case may be rare (i.e. most people are optimizing for initial results, not optimizing a large existing machine) but it does happen!
Of course it does happen, it's not a rare case, it's the usual case. Any remotely successful product will have N = millions of users. The only reason this doesn't enter the calculation is that you don't get the money you saved by not making your users do unnecessary hardware upgrades.
Ah! So there's the key point: do your users run your software or do you, on your servers? If the former, yes, it's an externalized cost and may not enter your economic calculations. If the latter, you care a lot :-)
