
Designing an Engineering Performance Management System from Scratch - thebent
https://blog.gitprime.com/designing-performance-management-systems/
======
emtel
Formal performance reviews are the worst method for evaluating your workforce
that has ever been tried, except for all the others.

For me, performance reviews are the single most demoralizing aspect of working
at a large company. They are doubly demoralizing because I can see so many
things wrong with them, but also can't imagine any system that would work
better.

So what's wrong with them?

1\. Over the long term, they incentivize people to work on things that have
easily measurable short term costs and benefits. More difficult to measure
costs and benefits get ignored. They also encourage sunk-cost fallacy. In
almost any company, you're better off, at least in the short term, if you ship
something that turns out to be a huge waste of resources later on, than if you
make the decision "this isn't turning out the way we wanted, we should just
cancel this project".

2\. They pretend to be a relatively objective system, when in fact your
performance rating at most companies is strongly (though not entirely)
dependent on how your manager views you. Attempts to lessen the impact of
point 1 above will generally increase the impact of this point. Research
suggests that most of what passes for rational justifications are made up
after the fact to support gut-level emotional decisions.

3\. They replace intrinsic motivation with extrinsic motivation, which at
least some research has shown can lead to much less durable motivation over
the long run.

At one point I was very enamored of the way Valve does things, but now it
seems that that sort of management approach produces super toxic cultures in
the long run.

Ultimately it seems like the only way to get away from this stuff is to work
for yourself or in a very small team with trusted partners.

~~~
mkobit
I read about the culture of Valve a few years ago and it seemed interesting.
What have you seen come out that changed your opinion?

~~~
wpietri
I would love to hear more than rumor about it, but what I've heard suggested a
lot of "tyranny of structurelessness" [1] problems. If the former power
structures go away, then people fall back on informal tools, including charm,
popularity, favor-trading, and abusive behavior, and a de facto power
structure emerges. Since this doesn't really include mechanisms for
accountability, it can devolve into lord-of-the-flies scenarios.

[1]
[https://en.wikipedia.org/wiki/The_Tyranny_of_Structurelessne...](https://en.wikipedia.org/wiki/The_Tyranny_of_Structurelessness)
and
[https://www.jofreeman.com/joreen/tyranny.htm](https://www.jofreeman.com/joreen/tyranny.htm)

------
deegles
"When a measure becomes a target, it ceases to be a good measure."

In this case, the measure is your position on the career ladder, not e.g. the
success/failure of your product or company as a whole (unless that is required
to advance). I feel that because of this, performance management at big
companies leads to unnecessary product churn. Think about how many messaging
apps Google has released, or how every product seems to get a top-to-bottom
redesign every year or two... having been on the inside, I think this is
definitely a symptom of employees optimizing for climbing the career ladder.
Promotion Driven Development is a real phenomenon and is in the long-term
detrimental to everyone. I don't know an easy solution to it, since bigger and
bigger projects are required to "prove" that you should be promoted. Maybe in
a few years people will have experienced this enough to think of a better way.

~~~
donavanm
The phrase at my current work place is Promtion Oriented Architecture. Ive
seen three general approaches, that I like.

First the role descriptions & leveling guidelines have deemphasized
complexity. Words like complex, large, feature, or new have been reduced or
replaced where possible.

Second there an emphasis on (business) value delivered. Fixing a broken system
can provide as much or more long term value than building an additional or
replacement system. Building a baroque ivory tower is actually a negative.
This also ties back to point one.

Third all senior role promotions include target level peer review, alongside
the manager/candidate “promo doc.” The candidates potential peers perform an
assessment of the body of work, leadership contributions, and results
delivered. This is _not_ limited to the most recent delivery/project/team.
There are typically two assessors; one organizationally “near” the candidate
and one “far.” For senior staff/principal/distinguished engineer positions
theres also a mandatory peer review board.

~~~
babesh
No wonder so many people are kissing up to the senior engineers.

------
shadester88
I find the obsession with performance reviews interesting. I manage teams of
developers in a large organisation, and I have been doing so for many years
now.

I hate performance reviews. I think they do not achieve what people think they
do, they take up a lot of time and effort to decide and perform, and they
often have quite a negative effect on the staff member.

I think performance reviews and KPIs for individuals just need to go. Get rid
of them. If someone isn't performing, you can manage that without a KPI
system. If someone is performing well, you can provide that feedback without a
KPI system. We need to start having meaningful conversations, not scoring
people.

People, like in this article, seem to double down on them and make them more
complex and intricate in an effort to make them effective. They are going in
the wrong direction. Just get rid of them altogether.

~~~
all2
My gut tells me that these metrics are in place because upper management does
not trust you to be objective or consistent when it comes to your people.
Rather than actually trust your judgement, they give you a checklist and tell
you to check the boxes.

Along with a lack of trust, there's also the need for a paper trail in some
states if it comes down to firing someone who needs firing. Without an
appropriate paper trail the company would be on the hook for paying that
individual's unemployment.

I'm not saying you're wrong. In fact, I think you are right. Most people will
rise to the expectations set for them (or fall, according to the expectations
set).

~~~
walshemj
No its really because:

1 In order for hr to justify themselves.

2 As a hypocritical way to cut the pay quanta.

~~~
all2
Just for giggles, lets say that a large company _doesn 't_ need an HR person
or department.

What are the alternatives? Is this something we could build an automated
system around? Could the functions be moved to managers (hiring, firing, pay,
etc.)? It seems like moving this way would be somewhat like implementing the
officer/NCO relationship in the US military.

Are there benefits for non-specialized decision making as it applies to
employees?

> pay quanta.

What's that?

~~~
NewsAware
>> pay quanta.

> What's that?

"paying a lot". So basically as away to combat a (perceived) inflation in
salaries

~~~
all2
So, amongst other things, it could refer to salary and raise confidentiality
agreements as a way to reduce employee negotiating power?

------
pdimitar
With the risk of getting roasted, here are my top three impressions of the
article:

(1) The guy is very deep in the management / higher-level business bubble and
is extremely disconnected from the day-to-day work that enables his lifestyle.
He seeks to optimize things that are mostly managerial / consultant lingo and
don't have much connection to things in the real world. That's not 100% true
of course but it mostly strikes me as such.

(2) Instead of de-formalizing the process he seeks to find more and more micro
ways to measure people. This can probably work long-term, _maybe_ , but many
people have tried and failed many times in the past and I think it's arrogant
to not take that into account in your supposed solution.

(3) He is not accounting for humans, like at all. I knew programmers that
worked quietly for 3 weeks and then showed us all a gem that made us go
"wowwwwww". In traditional systems like Scrum and any agile-based nonsense
this is severely frowned upon. Truth is however, people are different and as
long as you are happy with the average ($result / $month) of somebody then you
should leave them the hell alone to find and optimize their own way of being
productive.

\---

The above is overly simplified and I am well aware there is a lot of nuance.
But I didn't want to write a book so I settled for a condensed and partially
inaccurate summary.

------
johnrob
Fundamentally, are performance systems done objectively or "on a curve"?
Meaning, if 100% of the engineers are operating at say Staff Engineer level,
do they all end up with that title? Or does the Staff Engineer title really
come down to some sort of percentile in a stack ranking?

~~~
Cookingboy
>if 100% of the engineers are operating at say Staff Engineer level

Then you need to re-evaluate your definition of "Staff Engineer" at your
company. Those titles are all relatively defined internally in the first
place, just like junior/senior/staff/senior staff.

There is no industry-wide benchmark for those levels, so an internal curve is
the only system that makes sense.

In the imaginary case where everyone in the company is a super high performer
and everyone is equally as good, then it's not crazy to do away with those
titles entirely as long as that's the case. But obviously that scenario won't
last long as the company grows, and that's why titles/levels are gradually
introduced.

~~~
kelnos
If everyone actually is operating at a high level -- or at least enough to
make a curve difficult -- keeping people under-titled is just a great way to
allow your high performers to become discouraged and leave the company.

~~~
pbecotte
Titles, in my mind, reflect duties and responsibility rather then level. Not
everyone can be in charge of the same things and have leadership
responsibilities...therefore they cant all be staff engineer.

They CAN all be top performers at their responsibility list...

~~~
babesh
Salary isn't the only thing people covet. Titles and responsibilities are also
coveted since they increase future earning potential. When people don't feel
that they can get them, they leave.

~~~
User23
Neurotypical humans all crave social status. The higher expected future
compensation is a nice bonus, but it's the status that draws people to higher
titles.

------
dammitcoetzee
I think we all tend to focus on building the system which is the most point of
failure resistant rather than the point of failure itself. I've found
performance reviews to be fantastically valuable and motivating when I have a
good manager.

Rather than endless optimization of the system. Train your managers.
Everything from giving good feedback to ignoring or forgetting a bias is a
skill that can be learned.

------
Renaud
I liked the idea of Objectives and Key Results (OKR), a system that was
started at IBM and brought over by John Doerr to Google and others[1]: short
term goals alongside long term goals, set at the company level _and_ the
individual level, constant review and adjustment of the short term goals,
disconnection of these metrics from employee bonus.

The idea is to get to you goals faster and re-evaluate them for relevance as
you go along. Since they are not directly connected to pay there is less of a
perverse effect where people are incentivised to work on improving their KPI
at the expense of the rest.

[1]:"Measure what matters", by John Doerr

~~~
Draiken
Then people create objectives like:

O - Improve team efficiency

KR - Increase test coverage by X%

KR - Update at least Y outdated dependencies

Which are complete bullshit, easily gamed and don't actually make the company
move forward or the team better.

But according to almost any OKR believer, these are great OKRs.

Making KRs measurable a lot of the times corrupts the OKR system. However if
they aren't measurable, their value is sometimes questionable. Kind of a
paradox.

Perhaps for huge corporations this kind of system is absolutely needed, but
I'm experiencing this on a small company and it's honestly tiring and
inefficient.

------
timdellinger
I think I missed the author's design goals for a performance management
system: what it's supposed to accomplish.

Performance management systems have a natural tendency to degenerate into
sources of toxic behavior... one of the most important design goals would be
to avoid this, and one of the most instructive parts of a "how to" would be
strategies in that vein.

As an aside: I've noticed that people who run performance management systems
are under the impression that their system does a good job. After all, it
chose them, didn't it?

------
simon_000666
Should be titles : “How to demotivate your workforce and kill your
productivity 101”

------
supergeek133
There are two aspects to any system like this: 1) The "carrot" \- goals you
set to rate yourself on to (in theory) get a raise at the end of the year. Or
qualify for a promotion.

2) Career feedback.

Now I've found for both of these if any review with your manager take longer
than 10-20 minutes, something is amiss. This usually means you don't see eye
to eye on either the state of your goals, or your performance.

That being said, without the carrot, I think most people wouldn't execute
these at all.

------
anonymousJim12
Career ladders and 360 performance reviews? Deming would be disappointed...

~~~
zachrose
I’ll bite, what did Deming say about this kind of thing?

~~~
ricardo_ramirez
"The performance appraisal nourishes short-term performance, annihilates long-
term planning, builds fear, demolishes teamwork, nourishes rivalry and
politics… it leaves people bitter, crushed, bruised, battered, desolate,
despondent, dejected, feeling inferior, some even depressed, unfit for work
for weeks after receipt of rating, unable to comprehend why they are inferior.
It is unfair, as it ascribes to the people in a group differences that may be
caused totally by the system that they work in."

------
User23
This is assuming that the input data is high quality. It isn't.

------
trhway
i wonder what the guinea pigs have to say about their experience there.

------
iddan
This website's scroll is unbearable.

------
tschwimmer
This site somehow manages to interfere with trackpad gestures on my MBP. I
could not swipe back or forward. How is this even possible?

~~~
have_faith
They've written their own scrolling logic (why...) which involves cancelling
or stopping propagation of the normal gesture and events. They broke scrolling
on purpose and accidentally broke other things in the process.

~~~
aaaaaaaaaab
The absolute state of web development...

~~~
noir_lord
I wish there was a single flag to disable all that crap such that developers
had to consider users who have the flag set, better still if it's on by
default or behind a permission check

'This site would like to hijack your scroll wheel/touchpad', it's genuinely
infuriating.

~~~
aaaaaaaaaab
That flag is called “Disable JavaScript”.

