So one manager is below average at hiring and randomly hires people, ending up with a nicely fit bell curve range of high and low performers. Another manager is great at hiring highly performing people, but now if they rank correctly, you look at their assessments with your cheat metric, the good manager is 'inflating' their ratings. In other words, looking at the rankings alone doesn't do anything for actually enforcing quality.
Sure. There is no single silver bullet that actually enforces quality everywhere.
But look, inflation is bad. If grades aren't inflated, and I rate someone a 3, then we say, "This person is a solid employee. We're happy they're here, we're giving them an average raise, everything's cool. Do we have a little constructive feedback that says, 'Hey, here's how you could take it to the next level?' Sure. But we're happy with them."
If grades are inflated and I give someone a 3, then oh shit, maybe they're in trouble! Because some asshole had to give everyone in his team a 4 or 5, and now my solid employee looks like he's underperforming the company average.
The fact is, a solid 4+ rating means you're a really great employee. Teams that are all high performing are super rare, and tend to be really, really, really obvious within a company.
Now, is it a valid criticism of stack ranking that it makes it not just difficult but impossible to describe a team that's solidly above average? Yes, absolutely. Stack ranking is kind of dumb. But understand that despite interesting and compelling stories about people being unfairly rated lower than they deserve, that's not the big problem that companies face. In general, the incentives for everyone is to rate people higher than they deserve, not lower (with toxic exceptions like what Fowler faced). And if you set up your whole system to make really sure that it embraces the idea of everyone being super awesome, you probably don't get a highly functional rating system out the other side. It's probably better to set one up that embraces the idea of average teams (average for your company) and let people override the system when you say, "Oh my god, this entire team is just killing it this review cycle."
> Teams that are all high performing are super rare, and tend to be really, really, really obvious within a company.
Teams that don't fit a normal distribution are incredibly common.
And I think the bigger problem is money allocated for pay increases.
If the average 3 rating employee in your team can get a 10% annualised return each time they change jobs then the company should be budgeting 10% for a grade of 3. In practice this doesn't happen.
> Teams that don't fit a normal distribution are incredibly common.
This doesn't make a team of high performing folks any less rare because extremes tend to be rare. It just means that reality doesn't always look like the average. You might have no high performers but 5 average ones, or 2 high performers and 3 below average (maybe they are just learning?).
One team out of many being high performers might just be luck or a good manager, for sure, but it definitely warrants checking things out because of the rarity. The same would go for an entire team that is below par without obvious reason. It might not even be an issue with the supervising manager themselves, but with the way the company scores that sort of team.
> Teams that don't fit a normal distribution are incredibly common.
> > Now, is it a valid criticism of stack ranking that it makes it not just difficult but impossible to describe a team that's solidly above average? Yes, absolutely. Stack ranking is kind of dumb.
Isn't the solution to some of these problems to assess the teams results overall? If they are performing as required or expected (i.e. hitting targets) whatever the team metric, then wouldn't that explain rank inflation?
The ranking of people requires either one of two things:
1. fitting your group of people to a bell curve regardless of their proper distribution in reality
2. ranking inflation or deflation
Seems like a broken system designed either to generate make-work for managers and HR or to give the illusion of personnel manipulations to improve performance.
I thought of that, and it's a seductive answer, but I'm not sure anyone wants it to be done mechanically that way. (If you're just eyeballing things and then having multiple people have input to keep the primary manager honest, then yes, I think that the team's performance should be kept in mind).
Like, say your team didn't hit its OKRs. Should that mean that your manager has to make the average of your team's ratings be below 3? Certainly, one reason why your team failed to hit its numbers might be because most or all members of the team are poor performers. But it might also be that your manager didn't organize your work well, or it might have been that your numbers were unrealistic.
If a team didn't perform as necessary and all it's rankings were high, that would be some damn useful information regarding the manager, the teams it interacted with or depended on, and/or the companies goals, deadlines, and methods of setting those.
One answer (not necessarily the answer) is to ensure that up-leveled calibration conversations are happening and that next-level managers are informed about their people.
It's a delicate balance and, as a director myself, I don't claim to always get it right. But the goal I aim for is: if we have a good manager hiring great people and a mediocre manager hiring mediocre people that we calibrate correctly across those managers rather than claim that one is inflating their rankings.
