
Modern workers are at the mercy of ratings - edward
https://www.economist.com/business/2020/02/06/how-modern-workers-are-at-the-mercy-of-ratings
======
kqr
I seem to be the first to raise the Deming point of view again, but I'm glad
to, so here goes:

One of the big problems with trying to rate workers is that most workers have
approximately the same rating. In other words, the differences in ratings are
not statistically significant; they are just as well explained by pure chance.

Building a system where you incentivise and promote on a random draw of cards
is bad in and of itself, but doing it and then labeling it as being based on
merit is actively hostile to anything you're trying to accomplish. You are
building a culture of people stabbing each other in the back in desperation to
win your lottery, and not a healthy environment of cooperation and focus on
productivity.

Yes, sometimes you have a worker who performs off the charts in either
direction. That's a great opportunity! If they perform much better than
someone else, put them in a position where they can coach and train others in
how to do things that well. If they are much worse, try training them or
otherwise find them a different job they can do better.

Other than that special case, the only way to improve the performance of the
system is to give the workers better tools, better information, better
understanding, more cooperation, better ability to do their job. The manager
is the only one standing in the way of the workers doing a better job.

But then how do you assign bonuses and raises when everyone performs
approximately the same? Split them across everyone. That simple. Think in
terms of incentivising the team, not the individual.

And things like promotions that can't be shared among many people? Ask the
workers themselves who they think are a good fit for the position. Chances are
they know that hell of a lot better than a manager will. And it is abundantly
clear how to get a promotion: make a good impression on your team. Cooperate,
teach, help out, be nice, do good work. And these are -- by a weird
coincidence -- just the things you want to accomplish.

~~~
whiddershins
This entire premise, that people are usually more or less equally capable at a
given task, doesn’t correspond to my experience at all, across several
industries.

It seems ideological, or philosophical or based on limited industries.

It can also be dehumanizing to imply people can’t get any result from trying
hard, or that everyone has equal abilities.

I’m just not exactly sure why you are saying this, what it is based on, and
what the goal is here.

~~~
kqr
There's an important distinction to make which I'm not sure I've successfully
communicated: when I'm saying there generally is no "statistically
significant" difference, I'm not saying everyone performs at the same level.

There can be huge variations in performance between people. But this variation
tends to be internally consistent. You don't have one or two workers that
perform multiple standard deviations away from the mass.

Think of it as drawing people from a distribution with large variance. When
assigning bonuses to individuals, you're looking for statistical anomalies,
i.e. you want to find that one person that appears to be drawn from a
completely different distribution. That is what is very rare.

Large variation in performance but without any statistical anomalies still
means everyone is performing "within the systems natural boundaries." There's
no reason to suspect someone comes at it from a completely different angle.

That said, _of course_ you want to minimise the variation of your system. If
nothing else because it makes it easier for you to find the true outliers.

But you don't reduce variance by giving raises to the top performers -- that
will, _at the very best_ , make them perform better, thus increasing variance
further.

Better try to reduce variance by bringing the other people up, which requires
much more involved management than just giving more money to the ones with the
high numbers.

~~~
andrekandre
> Better try to reduce variance by bringing the other people up, which
> requires much more involved management than just giving more money to the
> ones with the high numbers.

yes, bringing people up that are not doing as well, has in my (anecdotal of
course) experience, resulted in huge overall gains for all-round team
performance (people share info with each other, help each other, are more
motivated etc)

i would add, that unfortunately, the trend seems to be in the opposite
direction...

------
virtuous_signal
One good thing to come out of certain rating systems, like Uber and Airbnb, is
that the users get rated too. If someone is a terrible customer, then they can
probably make things a lot worse for the driver or host than vice versa, and
without these scores, drivers might not be able to warn other drivers about
them. I would hope that with other modern customer service jobs, that reverse-
ratings are always employed. It is a measurable way to improve the jobs of
customer service workers.

~~~
hombre_fatal
Actually, what I've found on Airbnb is that everyone just rates everyone high
to either avoid rating retaliation or to avoiding seeming like a dick / hard
customer. I suspect it's the latter -- there really is no upside to being an
honest critic.

You'll rent a place that ends up being on the noisy highway and the worst
rating you'll find is "great for early risers!"

The rating system does not work. Airbnb doesn't care about rating inflation
because it just makes everything look like 5-star experience.

Uber is frustrating for other reasons. I've never made an Uber driver wait for
me, I'll greet the driver, and I can't imagine how someone can be a better
customer. I have a 4.1 rating and can't get VIP because of my rating. My
girlfriend doesn't even believe in getting up to leave the house until the app
tells her the Uber is there. She constantly calls uber for large loud drunk
groups, redirects her rides, and yammers on her phone the whole ride. She has
a 4.8+.

Aside, my girlfriend also rates all Uber drivers 3/5 because that's an average
rating. They get a 4/5 if they go above and beyond. I actually though 5-star
systems had merits until I met her. Now I'm firmly against them.

~~~
pushswap
Part of the problem of rating inflation comes with the removal of problematic
listings.

My girlfriend & I discovered large patches of mushrooms growing out of the
master bedroom carpet in an otherwise very unhealthy AirBnB. I wanted to write
feedback on the house, but I couldn't warn others since AirBnB's policy in
such cases is to remove the listing. I spotted the listing not long after with
a clean slate. I'm sure they cleaned the place, but I'm also sure they didn't
tear out the floors and fix the root problem. Too bad no one will know since
the ratings won't be a source of info in such cases.

~~~
arcturus17
The way to solve this would be to have an overlay everywhere, such as the one
that tha browser extensions afford where you can see a layer of reviews that
are truly independent.

Maybe AR will open a new door there... I know it would be an extremely hard
problem to solve - it might involve cutting-edge techniques in AR, AI,
computer vision and so on - but I think there might be a billion dollar market
there.

The pain certainly exists as reviews _could_ be useful, but they're trash
almost everywhere (Uber, AirBnB, Google, Amazon, Deliveroo, etc.) due to
platform-owner incentives.

~~~
RangerScience
Interesting. I wouldn't try to do this with "true" AR (magic leap / hololens)
- not enough benefit - but just a location aware layer would do it.

Seems like you'd want to pair it with a browser extension (want to avoid
making that booking in the first place), and the back-end engineering is very
similar to what (IMO/AFAIK) you'd need to support similar reality overlays in
"true" AR.

~~~
arcturus17
I mention AR because browser extensions would work well on the web, but you'd
only be able to do the same on mobile with a system that cooperates, and
there's a slim chance that Apple and Google - esp. the latter - would allow
something of the sort to materialize. Plus of course with AR you could take it
to the streets!

As for the business model of this independent platform, one could still make
_a ton_ of money out of it as long as its revenue-generating incentives were
disaligned from the review scores, and I really think that's not such a hard
problem to solve.

~~~
RangerScience
Ohhh. Phone as browser in physical space, AR as extendible medium.... I like.

For the AR-ish part... Oh, widgets! Right? Like clock, or weather, or the news
thing. Constantly (modulo performance) update what you're showing on the
widget based on location. Or dynamic notifications; "this is how many reviewed
locations we know of at this location".

------
Jaygles
I get that its immeasurably frustrating for a person to be distilled into
performance metrics, but how else can we measure one's impact? Using a trust
based method where a supervisor leans on their personal biases to determine an
employee's value is fraught with issues as well. A large enough corporation is
a machine and we can't base decisions off of how we "feel" the machine's parts
are working. Hard numbers will win over time.

~~~
vraivroo
Yes, you've described why large corporations are hated. It's quite obvious
that a handful of numbers are incapable of measuring a person's true impact,
yet by god, they insist on trying.

~~~
tcbawo
All businesses want good employees, but what they want most are employees that
can be replaced if necessary. They hire people to accomplish tasks and add
value. I have felt underappreciated in corporate roles before, but then I
realized that I was attaching too much of my identity to my job and in
something where I had no real equity. Since then, I am much better at
separating work from personal life, and I am happier putting my personal
interests above the company's.

~~~
VRay
Yeah, I was a lot less upset about some of the crappy experiences I'd had
working for evil giant companies after I read "The E-Myth Revisited".

My elite 10 person team was doing a great job on Device A, while Device B had
a team of 200 semi-skilled foreign contractors just barely keeping things on
track. Then, somehow, the company decided to reorg my super-skilled team and
promoted the leader of the contractor team.

From 3-4 levels up in the company, the Device A team and the Device B team
were IDENTICAL. They were both delivering things on time and under budget,
even though team A was costing the company 1/5th as much, they were both fine.
These products were generating hundreds of millions of dollars or maybe even
billions in revenue per year, so the difference $5 million/year and $25
million was irrelevant.

If I'd realized this sooner, I wouldn't have put so much effort into doing a
great job at the expense of my own interests, and it would've been just fine

------
endtime
I'd rather be at the mercy of _relatively_ objective metrics than at the mercy
of whoever my boss happens to be. (I happen to like my current management
chain, but people come and go...)

~~~
decebalus1
Meh.. you're still at the mercy of whoever your boss happens to be. They just
need to put a little effort into framing the metrics the right way or to make
sure you're getting work that doesn't touch on the right metrics. The idea
that performance metrics are fair is just corporate propaganda designed to
make you have the incentive to improve them, thus being easier to control and
replace.

~~~
allset_
>or to make sure you're getting work that doesn't touch on the right metrics.

What? The metrics should be distilled from the overall company/org/team goals
and mission and if you're working on things that don't align with those of
course you're not going to have a good performance review.

~~~
gravitas
Think more subtly - if you need to meet a metric/goal of 50% of your time on
phone calls and 50% on tickets, a manager who wants to tank your metrics will
game the system to feed you less than needed to make your metric on one or the
other. Just because a metric exists does not mean it's applied fairly or
evenly to all employees.

------
jariel
They need to do controlled trials whereby they try their best to truly and
objectively measure a sample and then compare those to real ratings.

I suggest that for most things, customer ratings are completely irrelevant.

They might want to ask specific, relevant questions, such as: "Was the
car/flat generally clean and tidy" "Were there any problems entering the
facility" on the customer side "Did the customer generate problems with the
authorities" "Did the customer disrupt your ability to drive" "Did the
customer make repeated requests for out-of-bounds services even when they were
informed such requests could not be fulfilled" "Did the customer party include
more individuals than indicated on the reservation"

etc. etc.

Then they can glean specific bits of information and provide guidance.

Otherwise, general reviews are pointless because they're muddy, set to
different standards, emotional, unspecific etc.

------
maxk42
We've always been at the mercy of ratings. Now instead of being rated by a
single person, we are rated by hundreds.

------
greendestiny_re
Hacker News should revel in this kind of headline; ratings are made by
algorithms that can be gamed and exploited.

------
hechang1997
We should start a rating boycott where we rate highest for all ratings.

~~~
aphextron
> We should start a rating boycott where we rate highest for all ratings.

This company I've worked at for about a year now is the first place I've been
exposed to worker ratings in my career, and this has been my strategy. I
refuse to play the game, and give everyone a perfect score no matter what. It
absolutely does not benefit us workers in any way to give someone less than
perfect marks across the board. It is literally nothing but a tool used by
management to provide justification when it comes time to downsize.
Furthermore I've noticed that it tends to breed a culture of distrust and
backstabbing among colleagues, who may use the opportunity as retribution for
personal slights. It's an awful policy.

------
WalterBright
There's nothing new about this. My first job back in 1979 came with ratings
used to determine raises.

Corporations are rated, as well, on all sorts of numerical metrics.

------
TurkishPoptart
Sadly, this article is paywalled for me.

~~~
emilburzo
[https://outline.com/JhCCJd](https://outline.com/JhCCJd)

------
ChainOfFools
Les Measurables

------
0xff00ffee
A) Paywall.

B) Get a different job.

------
CiPHPerCoder
...does the "0.6 of a worker" bit remind anyone else of the "Three-Fifths
compromise" from American History class?

[https://en.m.wikipedia.org/wiki/Three-
Fifths_Compromise](https://en.m.wikipedia.org/wiki/Three-Fifths_Compromise)

Not sure if an intentional dog whistle

