> As a comp sci person, you can't improve what you don't measure. Tests are how you measure. So I get it. But if I decided to measure the page load time of my server to ultimately improve server performance, no one is really hurt by that.
The problem is when you write something to measure that and then ask someone else to solve the problem. Systems like this almost always lead to optimizing for the measurement, not the end goal.
I mean, sure, at first I'm going to optimize code and work to increase efficiency to get the latency down because I actually care about my work. If that doesn't get me to the goal you've set and you keep hounding me and eventually tell me that if I want to have a job next year I'm going to get the page load time down...
I'm probably going to start stripping less-used features, maybe having the server occasionally serve a nearly blank "Click here to Refresh" page to game the average, see if I can't specifically serve less content to your measurement application... All of this results in a worse experience, but a better metric.
The problem is when you write something to measure that and then ask someone else to solve the problem. Systems like this almost always lead to optimizing for the measurement, not the end goal.
I mean, sure, at first I'm going to optimize code and work to increase efficiency to get the latency down because I actually care about my work. If that doesn't get me to the goal you've set and you keep hounding me and eventually tell me that if I want to have a job next year I'm going to get the page load time down...
I'm probably going to start stripping less-used features, maybe having the server occasionally serve a nearly blank "Click here to Refresh" page to game the average, see if I can't specifically serve less content to your measurement application... All of this results in a worse experience, but a better metric.