> If you don't consistently apply it, it becomes worthless.
Why?
Having read Thinking Fast and Slow and been convinced by this book's many useful views, I would agree that a simple questionnaire-based ranking is actually better than any subjective assessment of candidates. And it should be as neutral as possible, ie not letting one's "first impression" influence the ranking (because it is almost always based on irrelevant physical features).
But even then, there is no reason for this neutral evaluation to become worthless if not 100% consistently applied. There is the broken leg case: If you want to evaluate the probability of someone going to see a movie tonight, you just base yourself on simple statistical facts (how frequently people go to the movie in this country), and should not try to infer more from subjective context, except if the guy in question broke his leg this morning.
These tests and evaluations help much in reducing system bias due to halo fallacy, framing effect, and even time of the day for the evaluator (it has been proven juges are more lenient after lunch!), but they still are only helpers, they do not need to be decision-blocking.
Well, legally consistent application is very important. If getting an 80% if the cutoff for candidate A, and getting a 65% is acceptable for candidate B, then you're now setting different standards for different candidates. What if those different standards align with something like gender, or religion? It may not happen on purpose, but it's a problem.
Statistically, you've created a model for consistent application, and if you don't consistently apply then the anticipated value of the tools you're using is now in question as the model has a large source or error involved now.
I don't mean to say subject evaluations have no place in the process. Think of each part of the selection process as an independent stage. The candidates can pass the objective tests, perhaps with a minimum cutoff score, then pass the subjective judgment test. Or vice versa (though that may limit utility further).
Why?
Having read Thinking Fast and Slow and been convinced by this book's many useful views, I would agree that a simple questionnaire-based ranking is actually better than any subjective assessment of candidates. And it should be as neutral as possible, ie not letting one's "first impression" influence the ranking (because it is almost always based on irrelevant physical features).
But even then, there is no reason for this neutral evaluation to become worthless if not 100% consistently applied. There is the broken leg case: If you want to evaluate the probability of someone going to see a movie tonight, you just base yourself on simple statistical facts (how frequently people go to the movie in this country), and should not try to infer more from subjective context, except if the guy in question broke his leg this morning.
These tests and evaluations help much in reducing system bias due to halo fallacy, framing effect, and even time of the day for the evaluator (it has been proven juges are more lenient after lunch!), but they still are only helpers, they do not need to be decision-blocking.