> The main problem is that no one knows how to measure it.
This is true for every single aspect of academia, even research. I don't think our metrics in research are good and that they are the reason we have the replication crisis. Metrics are always guides, having limitations and biases unique to each one. Over-reliance just results in Goodhart's Law, and instead of embracing the noise inherent to the system we have ill conceived attempts to remove it. Attempts to create meritocratic systems frequently result in less meritocratic systems than were being "improved" upon.
This is true for every single aspect of academia, even research. I don't think our metrics in research are good and that they are the reason we have the replication crisis. Metrics are always guides, having limitations and biases unique to each one. Over-reliance just results in Goodhart's Law, and instead of embracing the noise inherent to the system we have ill conceived attempts to remove it. Attempts to create meritocratic systems frequently result in less meritocratic systems than were being "improved" upon.