
Are One-Star Reviews for Assholes? - ryan_j_naughton
http://priceonomics.com/are-one-star-reviews-for-assholes/
======
graeme
This is a serious issue. On the one hand, our current system makes a one-star
review far, far more consequential than it ought to be. If EVERYONE leaves
positive reviews, than a single negative is very powerful.

On the other hand, leaving positive reviews out of politeness does a
disservice to others. I hosted a couchsurfer once (a woman) who told me about
her experience with another host. This guy had 10-20 positive reviews. And he
was a total creep. Hit on her the entire time, asked her to sleep with him,
and was intimidating. She left as soon as she could.

But she didn't leave a negative review! And clearly, other women must have had
her experience, yet they said nothing either. This guy was left free to
continue his horrible behavior.

I don't know the solution.

~~~
tetrep
>I don't know the solution.

Trusted reviews. A user should be able to trust others, and to filter reviews
based on that. Integration with social networking would allow for friends of
friends trust as well. It would be far from perfect but it would better
replicate the experience pre-internet of asking people around you, which works
pretty well.

~~~
wlievens
But what are the odds of one out of a few dozen reviews to be written by your
friend or even foaf?

------
beloch
One possible _partial_ solution is to express the quality of the source of a
one-star review. A one-star review from someone who has posted many, many 4
and 5-star reviews should carry some weight, while a one-time reviewer or
someone with a history of giving many 1-star reviews should not. Perhaps color
coding such reviews, or preferentially ranking reviews from the reliable at
the top. This might lead to some legitimate 1-star reviews being ignored and
it is at odds with the anonymity most people prefer on the internet, but it's
better than presenting all 1-star reviews equally. Companies that make
employment decisions based on these reviews might also choose to weight
reviews based on the reliability of the source as well.

~~~
dragonwriter
> A one-star review from someone who has posted many, many 4 and 5-star
> reviews should carry some weight, while a one-time reviewer or someone with
> a history of giving many 1-star reviews should not.

Shouldn't the reverse be true as well? A 5-star review from someone who gives
lots of a 1- and 2-star reviews should mean a lot more than a 5-star review
from someone with little history or a history of giving many 5-star reviews.

Really, a big problem here is cultural norms vary and subjective ordinal
ratings don't mean the same thing to different raters -- ISTR studies showing
that White Americans tend to rate things near the top of a scale if they are
_acceptable_ , toward the middle but still above the center even when they are
_unacceptable_ , and below the center only rarely -- whereas different
nationalities and ethnicities had measurably different patterns of ratings
compared to their actual _descriptions_ of their responses to products. If
that's true, a one-sided selective discounting of reviewers that give lots of
low-star reviews may be effectively saying "I want reviews from White
Americans to count the most."

~~~
__david__
That got me thinking. If you averaged the users' total reviews and then
centered their score on a slightly larger scale then you could accomplish
something like that:

    
    
               Adjusted Value
             1 2 3 4 5 6 7 8 9
       Ave +------------------
        1  |         1 2 3 4 5
        2  |       1 2 3 4 5
        3  |     1 2 3 4 5
        4  |   1 2 3 4 5
        5  | 1 2 3 4 5
    

So if I rate a bunch of things and my average review score is 4, then things I
rate as a "1" would have an adjusted value of "2", and things I rate as a "5"
have an adjusted value of "6". That would "find everyone's center" and maybe
score things more evenly.

------
morgante
If people have the attitude that a high review is mandatory, then there's
effectively no point to review systems.

Sorry, but not every book should be read. Not every restaurant should stay in
business. Not every Uber driver should be driving. Giving accurate reviews
allows the system to work, both for customers and (eventually) for providers
(by steering them out of work they're inept at).

Or maybe I'm just an asshole...

~~~
zem
seeing as how uber makes reviews mandatory, giving everyone five stars is the
easiest way to opt out.

------
PaulHoule
I think it is the five star reviews that are the problem, because we are all
taught that, for services, leaving anything less than five star reviews will
make trouble for them.

The fact is the average Uber ride is going to be average. The average person
shipping stuff on Ebay is average, the average person selling on the AMZN
marketplace is average, and so forth.

If you're supposed to leave 5 stars to say service was adequate, then there is
no way to reward those who go above and beyond.

------
davemel37
It's 2014. As cruel as it sounds, if you haven't figured out that you need to
underpromise and overdeliver to your customers, your days are numbered.

There will be growing pains, people will suffer greatly, but a wonderful new
system will hopefully arise from the ashes.

I do think that there is a better way, and that most users know how to
discount a really bad review. It's kinda hard to hide bias when you are angry.
So, the system will eventually balance out.

Bottom line, I don't know how to fix this system, but I do know how to survive
in it. (hint: underpromise, overdeliver)

~~~
Guvante
How do you underpromise and overdeliver for a ride service? Sure he shouldn't
have taken the detour, but should he lose his job because of the mistake? The
answer is unknown to us.

One thing that can be said is a customer who becomes sick with power is hard
to deal with. If someone gets in the car and makes unreasonable demands with
the threat of a bad review what do you do?

~~~
vacri
Should he lose his job? That's not really a decision for the customer to make.
If he loses his job because of one justifiably bad review, that's due to the
ride service's policy, not the client.

~~~
Guvante
You are skirting my main point about unjustifiable reviews. The example in the
article is certainly fire worthy, I will not contest that. (I was attempting
to not claim that the writer was incorrect in his actions or that the actions
of the driver were okay)

------
rogerhoward
There has to be room in any rating system for the lowest rating, for those who
deliver the worst possible service - obviously.

The problem I see is that the perception of what warrants a given rating
varies pretty widely, and most sites do a terrible job of trying to help their
users calibrate their ratings. I monitor Yelp regularly for my girlfriend, who
owns a wine bar. Her ratings are generally excellent - over 4 star avg., with
most ratings at 5 for the past two years. However, every once in a while we
see an anomalous rating - for instance:

\- one current 3-star says "I have to be honest, I didn't have anything to
drink, I was with friends waiting for the midnight showing at the theatre next
door. However, it was packed and happening and EVERYBODY was having a good
time."

So this person, who wasn't a patron, but who had nothing but positive
impressions, still left what ultimately will be considered a sub-standard
rating. This is actually most common for us with 3-stars - they are generally
very positive reviews, with nothing negative at all, but with what most
Yelpers would consider a sub-standard rating.

Mostly when I look at Yelp ratings, it follows the same pattern - 80% of the
ratings seem to correlate closely with the reviews and the typical Yelp
ratings standards. 10% seem to be using a different rating standard. And 10%
are just weird.

~~~
lutorm
Places like Yelp who have many reviews from the same person should be able to
curve the reviews onto a common standard. At least they should be able to say
that a 3-star review is, e.g., +1 sigma for this particular person.

------
vacri
_The experience was pretty bad, but who are we to take away someone else’s
livelihood?_

Someone who drives so badly that they endanger the lives of their passengers
in a paid service? Whether or not he's a nice person, you should rate the
action appropriately. If you're the only bad review, the driver's rating
should stay high.

Did the driver learn from the bad mistake? How would you know? Someone can be
nice and still clueless. Just rate the driver appropriately.

Personally, I find the bad reviews to be much more informative than the good
reviews. Star ratings are largely meaningless (unless the thing is at 1 star
or less). Read the bad reviews - if they're just frothing trolls of the kind
that complain about everything, the item is probably fine (and you can check
out more reviews at that point). If the bad reviews are raising valid
complaints - and 'drove so badly we nearly died to his manoeuvres' is a valid
complaint. If several people are making that complaint, then there's likely to
be substance to the claim.

~~~
kelukelugames
If you put it that way then the reviewer has selfishly endangered the next
passenger's lives. Just so he/she doesn't feel guilty.

~~~
vacri
Well, I wouldn't say selfishly, since the reviewer is doing it out of kindness
to another, but yes, you could argue that in this particular case, it's
endangering.

------
siegecraft
The problem is that the rating system presents itself as 1-5 stars but is just
pass/fail (or it has degenerated there over time). But "5 stars" is better
marketing-wise.

------
bronbron
I think there are two issues:

1) Evaluation inflation/simplification. Happens with grades, happens with
reviews. 4 stars becomes the new 'adequate'. If you're under 4 stars, you're
bad. Everyone subconsciously knows this, hence the thinking 'if I don't give
them a 5 star review they might get fired.'

A more meaningful, diverse group of metrics is likely the answer to this. Get
rid of 'overall' ratings. Rate drivers on their friendliness, safety,
timeliness, etc. Uber can decide on some algorithm for determining when to let
drivers go, and users feel less bad about negative reviews: 'Well, I gave him
a 2 on safety but he was really friendly so at least I gave him a 5 on that."

2) Not enough separation between reviews and individuals. If I leave a review
on amazon, I don't want that being tied to my account. At all. I don't want to
see the seller's replies to my reviews. If someone finds out my amazon account
name, they should not be able to tie individual reviews back to me. Batch
reviews up in groups of 5, randomize them, and post them all at once. Don't
let there be any possibility of tying reviews back to individuals. The fear of
someone seeing a mean review I wrote and forever thinking I'm an asshole is a
serious barrier to negative feedback (barring extraordinary circumstances).

------
jodrellblank
Paraphrasing Tolstoy, "All 5 star reviews are alike; each 1 star review is
unhappy in its own way."

Five star reviews tend to be fluff, one star reviews tend to be specific. I
read them first, I look for dealbreakers. No amount of "I love it" can outvote
"claims X but can't really X properly".

~~~
greenyoda
Exactly. On Amazon (the review system I'm the most familiar with), if I find
several one-star reviews that say some device was dead on arrival or a
technical book was full of factual errors and typos, I'll definitely think
twice before buying it, even if 90% of the reviews are five-star. I'm always
grateful for the people who took the time to write honest and informative
negative reviews.

------
einhverfr
It seems to me that human actions based on reviews require human judgement.
Taking away someone's livelihood because someone gets a lot of 4 star but
positive reviews strikes me as dangerously wrong. Taking away someone's
livelihood because he or she callously puts others in harms way is a different
story.

There's a huge difference between:

1 star: I waited for an hour for my pickup and this caused me to miss my very
important appointment. Yes, traffic was horrible today, and so maybe it isn't
the driver's fault but this was a very bad experience I don't want to repeat.
You folks need to do a better job of setting expectations!

and

1 star: my driver narrowly missed rear-ending the car in front of us because
he was distracted. He didn't even apologize!

In the second case we might blame the driver. In the first case, who gets the
blame depends on the system.

------
dhawalhs
On freelancing sites, if you give someone a 4 star review they come back and
dispute it. 4 star is actually a bad rating on these sites. A better way to
judge someone is based on how many clients/hours they have done on that
particular site.

------
superasn
The other problem with the one-star reviews is that since the livelihood of
the person (or company) is directly affected by it - they fight tooth and nail
to hide it or somehow drown it by putting fake 5 star reviews on top of it.
Something which they won't necessarily resort to.

This is of course based on my own personal experience but at least on
tripadvisor.com I have seen this happen a lot. Every time there is a 1-star
review of a hotel you can can bet that there will be 3-4, 5 star reviews on
top of it by reviewers who have left 1 review in total (suggesting they are
fake or outsourced) just so that the 1-star review can be bumped to page 2 or
drowned.

------
_cipher_
> The experience was pretty bad, but who are we to take away someone else’s
> livelihood?

Right. Next time, if the same guy kills someone in an accident because he
"thought" that a road was empty, you _may_ think that you had a chance to
prevent it. Good times.

You have to realize, you are not "bad" for giving a bad rating. You are
stating the obvious: the driver was an asshole that almost get you injured.

A bad review would be: One star because driver didn't say "have a good
night/day/whatever".

It's not bad to state that someone is indeed incompetent at something,
especially if that "something" can harm other people.

------
ISL
An approach used in psychology tests for questions people are embarrassed to
answer: add a random number to each rating.

The aggregate effect is the same (if a random number from 1-5 is added to each
reviewer's review, on average, you just need to subtract 3 from the average
review score), but reviewers may feel more-free to be honest. More reviews are
required to get an accurate review, but the reviews may be more accurate.

The larger the random spread is compared to the rating's dynamic range, the
greater the obfuscation, and the more reviews you need to learn something new.

------
devindotcom
I just left a one-star review for a book. But I think it deserved it and I
spelled out why. People need to be honest about their thoughts and
experiences, and sometimes that means giving the worst possible rating.

I mean, you could review one star for things that are, like, actually
poisonous or dangerous, but that doesn't really work either. As beloch
suggests below, a weighting system is a good answer, and Amazon does a pretty
good job (I think) of letting you look through positive and critical reviews.
Kinda different when you add the human element, but still.

------
lutorm
An "n/n" review for me means basically "I can't possibly imagine this service
having been any better." That's a pretty hard standard to meet, and hence I
almost never give them. An n/2 rating means "adequate, no specific complaints,
no specific praise". If you don't calibrate the rating like that, you lack
dynamic range on either side.

But I've gathered that I'm the exception...

------
itazula
I know that Uber does background checks on their drivers, but do they also
provide training to their drivers? Are drivers evaluated first by an Uber
employee?

------
gkop
The author projects his own hypersensitivity to the well-being of the Lyft
driver on the general population of Lyft customers. Or I'm just an asshole :)

~~~
davemel37
I'm pretty sure it's the latter. Most decent people, when faced with their own
emotions versus financially ruining others will err on the "keeping their
mouth shut" side.

While some might feel a sense of duty and obligation to the rest of the
customer base to leave honest reviews... I think most selfless and considerate
people will weigh that against the impact it has on the person they are
reviewing and wonder if its really worth being an asshole.

------
cafard
I have left at least one-star review on Amazon, for a sorry book that came up
in the neighborhood book club. I would not have done so, had not five-star and
four-star reviews dominated. It was promptly replied to by persons who thought
I was a snob and a jerk.

For Uber it is a serious matter. On Amazon, chances are that anyone giving a
one-star review is not in the target audience, and will have no effect on
sales.

------
minimaxir
I've been looking into Yelp data to see if there's a correlation between the
star rating of reviews and the average positivity of a review.

After averaging reviews across reviewers, you can see a positive correlation
between the two metrics:
[http://i.imgur.com/QXIU3qb.png](http://i.imgur.com/QXIU3qb.png)

------
drivingmenuts
Online ratings are so depersonalized that they are near meaningless without a
large sampling body to back them up. Even then, most online ratings systems
appear to be somewhat gameable, which renders them worse than useless.

Even though I consume online resources, I rarely leave ratings, since I can't
really tell that it's worth the effort.

------
lutorm
Any data set will have outliers that need to be filtered out, and I'd hope
that those interpreting the data would know that. In particular, there's the
selection bias that people who are pissed off are much more likely to actually
give a review in the first place.

Firing someone for a single one-star review is asinine.

------
vitalus
Perhaps the problem lies with having too many options without any agreed upon
standards. What's the difference between a 1 and a 2 star ride? Anecdotally it
would seem like it's always going to be 5, 4, or 1. Seems like the 1-5 system
is antiquated and needs to go.

~~~
jimmaswell
I can easily think of what would make a ride differ between 1 2 3 4 and 5. I'm
disturbed by the trend of ratings being reduced so that the reviewer doesn't
have to put any thought into it. It's encouraging intellectual laziness. I
felt the same about YouTube. If you must then maybe put suggestions next to
the 1 through 5 ratings like NewGrounds does. I don't want to lose the ability
to say something is average but not great or terrible, or between average and
terrible, or between average and great.

~~~
kevb
1 star (and 5 star) are always going to be the biggest buckets. Even if you
try to be very fair and strict, what does a driver that doesn't show up get? 1
star. What about a driver that stabs you? 1 star. Crashes? 1 star. Gets lost
and makes you late? 1 star. Smells bad and a bit rude, but gets you there on
time? Maybe this is finally 2 stars. These are all bad, but getting stabbed is
a lot worse than not showing up or getting lost.

Likewise for 5 stars. Friendly, safe, on time, gives you a cold bottle of
water, have a nice conversation. 4 or 5 stars. Chases you down with the bag
you forgot in his car? 5 stars. You fall in love and get married? 5 stars.

------
ecocentrik
This is one hell of an impediment to the success of these services. Failing at
quality control is going to drive away most of their current client base.
Forget luxury services... Give me an old school car service any day.

------
walterbell
Asking customers to perform an HR function by evaluating the work product of
contractors/employees?

Unsurprising that customers write all fives to decline non-paying HR
responsibilty.

------
kelukelugames
This is exactly like tipping the waitstaff.

"Always tip 20%. It's how they make their money!"

vs

"I'm not tipping anything for bad service."

------
jimmaswell
No. I wouldn't feel like an asshole giving him one star for almost killing me.

------
misiogames
if you take a cab, and the experience its terrible, what do you do? if you
take an "online ride share service"? and the experience its terrible, what do
you do? are the answers different? are the same?

