
Amazon can’t end fake reviews, but its new system might drown them out - pabo
https://www.vox.com/recode/2020/2/14/21121209/amazon-fake-reviews-one-tap-star-ratings-seller-feedback
======
throwaway2786
So here's some perspective from an Amazon seller doing 7 figures+ annually.

\- On average, only 1-3% of customers review products.

\- Each review is worth a lot of money, often times multiples of the product
itself, and especially if you're just starting out.

\- Each category in Amazon has it's own Average rating, for example,
electronics typically have lower ratings because more things can go wrong and
there are more usability issues vs something like kitchenware, where less
things fail outright.

\- If you play in a category with a certain failure rate, it is absolutely
essential that you do everything you can to mitigate bad reviews as enough of
them will sink your business, even if you have a great product.

\- It takes 8+ 5 star reviews to counteract a 1 star review if you want to
maintain a 4.5 star average which is the bar for a good product. This is
extremely hard to do without manipulation.

\- People who complain about fake reviews are only seeing half the problem,
the other half is that legit businesses who do it the fair way can't compete.
How do you launch a great product on Amazon with 0 reviews? Hope that 500
people buy it to maybe get 5 reviews? Alternatively you spend thousands on
product ads hoping that enough people buy... or just succumb to the dark side
and pay for reviews which is WAY cheaper.

\- If you hate Amazon reviews, do your part and start reviewing the good
products on Amazon. It is worth more to the seller than you think!

~~~
Wowfunhappy
Here's my problem with leaving reviews:

To me, a five star review means that a product went above and beyond my
expectations in some extraordinary way. I have bought products that fit that
description, but not many.

Everything else, I'd give either 4 stars or a still-very-satisfactory 3 stars.

The problem is, I know these 4-and-3 star reviews actually hurt sellers, which
isn't my intention at all. So I just don't leave feedback.

This is also why I don't rate Uber drivers.

~~~
crankylinuxuser
Thats why I turn my rating into a 1(lowest) or 5(highest).

I binary-ize any score that isn't already a yes/no, because thats what every
score turns into.

~~~
retsibsi
This reminds me that Youtube used to have a star rating system, but as you
say, it was effectively treated as thumbs up/thumbs down, so they eventually
made that official.

Interestingly though it wasn't quite a simple five stars/one star dichotomy:
the original blog post[0] has a dead image, but this one[1] shows that while
five stars was overwhelmingly the most common rating, one star and four stars
were about equal in second place.

[0] [https://youtube.googleblog.com/2009/09/five-stars-
dominate-r...](https://youtube.googleblog.com/2009/09/five-stars-dominate-
ratings.html) [1] [https://www.geek.com/news/google-realizes-youtube-star-
ratin...](https://www.geek.com/news/google-realizes-youtube-star-ratings-dont-
work-912531/)

------
AlexandrB
In my experience, making reviews easier to give causes the review quality and
usefulness to go down. This happened when Netflix went from 5 stars to a
simple like/dislike. I’m not sure why Amazon didn’t just block non-verified-
purchaser reviews, increasing spammer costs significantly.

I suspect that Amazon reviews are going to be even _less_ useful now.
Especially given things like this:

> Amazon does not provide many specifics about how a product’s overall star
> rating is calculated, other than stating that it is not a simple average but
> instead uses “machine-learned models” that take into account factors such as
> how recent the rating or review is and whether it was a verified purchase or
> not.

~~~
awinder
I'm pretty sure that a ton of the review fraud on amazon comes through
verified purchases. You refund/pay people to buy the product on amazon and
leave a review in the best case, in the worst case you operate accounts buying
your own stuff, and then flow the inventory back and basically pay an amazon
tax for leaving good reviews.

IMO this is pretty solvable by looking at an account's purchase history too,
but I don't think it's just as simple as blocking non-verified-purchase
reviews.

~~~
sct202
There's also low scale review fraud like my sister bought a pet cam and gave
it a 3 star review, and the seller started to offer escalating offers for her
to remove or increase the rating from full refund, another camera, refund +
free camera, and refund + money for the inconvenience.

~~~
dreamcompiler
It does bug me that scummy vendors know my email address and physical address.
What's next? They send goons to my house demanding a good review or my dog
gets it?

------
tobtoh
The (edit: average) Amazon star rating has been such a poor indication of the
quality of a product for a very long time - at best, it's a weak indicator of
which products you might check out first.

For me personally, the most valuable bit of feedback are the negative (edit: 1
and 2-star written) reviews - they are pretty much the only review content I
trust. I'm looking for patterns of issues that multiple reviewers raise about
the product.

The positive reviews have so little value when anyone can post a review. So
many shallow positive reviews from unverified 'buyers'.

~~~
casion
I'm not sure I agree with this. I always look for and at the 1 start reviews,
and I find that to be more useful than reviews I find anywhere else.

4/5 star reviews are useless, but 2/1 star reviews reveal a great deal of
useful information about a product.

A bunch of 1 star reviews of "UPS damaged my product" indicates a product
that's as advertised and isn't astroturfed (much?).

No 1 star reviews, or 1 star reviews indicating misleading advertising tend to
indicate astroturing.

1 star reviews indicating product failures, support failures, DOA etc...
indicate to not purchase the product at all.

It's not a pure indicator, but I've found it to be more reliable than other
review systems.

~~~
0xffff2
Am I crazy, or does the entirety of your comment except for the first sentence
agree with the person you replied to?

~~~
casion
The second and third paragraph were cut off when I replied on mobile. This was
clearly my mistake.

------
JohnFen
This may help a bit with fake reviews, but it amplifies the other substantial
problem -- authentic, but untrustworthy reviews. I see two type of those all
the time:

1) Reviews by people who have not used the product. The tipoff for this is
when the customer says something like "I just got this, it looks great! I
can't wait to use it." If someone hasn't even used it, then they can't
possibly give a useful review of any sort.

2) Reviews in product listings that contain multiple products, or products
that have changed since the original listing. The tipoff for this is comments
describing a product that is clearly not the one being listed, or is one of
several different products in the same listing.

If there is an increase in reviews that are just stars without comment, it
becomes impossible to root these out at all. I don't see how this would lead
to more reliable reviews.

------
photon_off
Amazon has been my wheelhouse, fulltime, for the past 7 months. I'm building a
site that looks at most listings per category and determines "true" scores for
each one. The site also lets your filter and sort, instantly, on a variety of
attributes, like "price", "unit price", "shipping time", "recent price drop
%", "used price discount", "popularity", "brand quality", etc. Spending 7
months collecting and analyzing data of several Amazon categories has been
exhausting, but quite revealing.

As you can imagine, dealing with low quality products with fake reviews is a
challenge -- but it turns out it's not too hard to handle, even with my
dataset which is far more limited than Amazon's. Without looking at any
reviews or any metadata of reviews (author, count, chronology, etc), one could
filter out "impostor" products with 95%+ accuracy.

Here's a neat trick: Next time you're unsure if a product has fake reviews,
click on the brand of the product and see what else they sell. If you're
looking at binoculars, and that same brand also sells dog food bowls, then
maybe you should reconsider.

I've concluded that Amazon really doesn't care about fake reviews -- they will
show users whatever listing has the maximum Expected Value (conversion rate *
revenue), per your context (search term, category, or both). Even if a product
has obvious fake reviews, if there are enough other people buying it it will
float to the top, and Amazon is fine with that.

~~~
the8472
> unit price

This is quite an important feature. Amazon already shows this information
(sometimes at least) and yet they don't let you sort by it.

~~~
photon_off
The main problem they have, I imagine, is sparse data. There are only certain
fields (depending on category) which they force sellers to populate, eg:
"name", "brand", etc. Item weight (which is distinct from _package_ weight),
and "number of units" do not seem required, and so not many items have that
information filled.

So, with sparse data, they have three choices:

1) Allow filter/sort by "unit price" and do not show the X% of listings that
are missing this data -- many of which the user may actually be aware of
and/or interested in.

2) Don't allow the option at all, and just rely on the fact your customer will
do comparisons manually.

3) Try to derive the number of units from text cues in the product name,
features, and description, then do #1.

It seems they chose #2. I'm going for #3.

------
izzydata
At what point does the poor experience of Amazon outweigh its convenience and
cause people to start shopping in physical stores again?

Or would people shift to other online retailers?

~~~
kop316
I haven't ordered from amazon in over a year. I to go to physical stores
and/or order from a reputable vendor online.

Actually it's nice. If I truly want it, I'll actually go and get it, or spend
the time to find a reputable vendor. The added friction ensures I don't buy
random things that I thought I wanted but don't.

------
subroutine
To me a bigger problem than fake reviews is the rampant bait-and-switch. Way
too often I will see a product with a 4 or 5 star rating, and find the reviews
are talking about something completely different.

That, or there will be 10 different products nested under the same identifier.
Not just different colors, but different products. So I have no idea which one
the person it praising or calling crap. Maybe instead of Avatars next to the
reviewer's name, Amazon should just attach an expandable screencap of the
product page at the time the "verified purchase" was made.

------
ebg13
> _“For brands, this means the black-hat review clubs and sellers will have
> less impact, as fake reviews as a percentage of legit reviews should
> decrease.”_

Well, at least they're thinking happy thoughts.

------
cik
Doesn't everyone just use reviewmeta
([https://reviewmeta.com/](https://reviewmeta.com/)) now? They're pretty solid
at what they do, the results appear to be rather accurate - and at the very
least it's a decent filter.

~~~
ceejayoz
I use Fakespot.

I think it's only a matter of time before these sites get coopted/bribed by
vendors, in the same way a lot of adblockers have been.

~~~
FakespotCom
Hi, Saoud from Fakespot here. My co-founders and I all got scammed and duped
by fake reviews and Fakespot was built to bring back trust and transparency to
the e-commerce experience (and others).

We have a strict policy to remain neutral in our analyses and we do not in any
way, shape or form accept bribes or copt partnerships.

~~~
fl0wenol
Thanks for explaining the background, Sauod.

But don't you need to say that? What policies or strategies are you using to
prevent this from happening beyond the shared experience of being burned and
avoiding a deep sense of shame if one were to abuse their position?

~~~
FakespotCom
We have been bootstrapped and funded by institutional investors that allows us
to function freely from being in the "unnamed but big review website starting
with Y" business model. Ethically speaking, I personally would never support
that as it goes against our mission as a company and what we are aiming to
build here.

We actually get consistent DDoS attacks which are coming from angry e-Commerce
sellers that have dismal grades on Fakespot, and that is occurring ever more
frequently.

------
luckylion
Amazon is competing against the fake reviews with their own Vine club.

I've checked dozens of their verified & official "testers", and their five-
star-rate averaged somewhere north of 98% which makes their reviews just as
useless as the cheap auto-translated fakes to me.

~~~
FakespotCom
And from our internal research at Fakespot, we concur. The positivity bias
from influenced reviews such as the Vine review program and multitudes others
is rampant.

Our ML based algorithms will penalize any biased reviews and show case them to
our users.

------
olah_1
When you review books on amazon, are you reviewing the content or the edition
that you purchased (paper, typesetting, cover, etc)?

I wish there were two separate ratings.

~~~
martin_a
I think this a constant problem with these type of rating systems.

We have also implemented one of the leading feedback-survey-customer
happiness-system-things in one of our shops. The amount of negative ratings,
because the shipping service provider fucked something up, is immense.

There is no way for us to really get those removed (integrity of the platform
- good thing in theory) and commenthing something like "it wasn't our fault"
doesn't really help either as the rating will influence the total rating one
way or another.

Maybe that's cost of business, not sure.

~~~
JohnFen
The other problem with reviews is that they seem to be scaled according to
price. A garbage product that sells cheap often gets reviewed as highly as
genuinely great product that is more expensive.

------
the_duke
Amazon reviews are about as trustworthy as the selling points of a slimy
salesman.

I've bought products with a lot of 5-star ratings which were utter garbage.

Even if you only consider "Verified Purchase" reviews, every one of them
requires careful vetting for authenticity.

There are companies providing quality fake reviews by native speakers as a
service, where care is taken to not be overly optimistic and provide a few
negatives to sound more believable.

Even negative reviews might be fake, often mentioning a competing product that
is supposed to be better.

There are plenty of valid reviews out there, but the need to constantly
question the authenticity makes them almost worthless to me.

I've mostly gone back to buying from medium sized local retailers. They might
have a much smaller selection and are a bit more expensive, but make up for it
with a decent quality control.

~~~
coleca
It would be a shame if the reviews went away though. There are tons of garbage
reviews out there, but I still find some gems out there.

Recently, I was in the market for a paint sprayer and one reviewer wrote this
crazy detailed review of the sprayer he purchased, what type of paint he used
to paint his cabinets, why this paint worked vs. the other ones he tried, and
included numerous pictures of the process and results. It was super helpful
not only in making my selection, but ultimately completing my project.

I also have found a bunch of instances where people will post tips in the
reviews about how to fix or modify a product to make it perform better (or
sometimes unfortunately just perform to its advertised specs). In one case I
was able to repair my own refrigerator with a part purchased on Amazon and the
instructions found in a review. While I'd rather have purchased it locally,
that would have meant a service call and spending half the cost of the unit to
get it repaired vs. a hundred or so dollars for the new circuit board from
Amazon.

------
anonsivalley652
I think, independent of Amazon, another way to combat fake reviews is to have
a modern, social-media-enabled, subscriber/patron-supported "Consumer Reports"
review service that:

\- doesn't accept free products or endorse any particular brands, and buys
products like an individual could

\- tests products thoroughly, both short-term and longer-term, including what
happens within the warranty period

\- reviews big-ticket items frequently enough to stay current

\- offers purchasing gotchas and decision tree for selecting a product

\- does the math on Total Cost of Ownership, including calculators so someone
can plug in their local values (utility costs, taxes, rebates) to make the
best decision

\- has concise maintenance/operating advice

\- starts small and looks for good products within a category

\- also seeks out lesser-known, independent manufacturers who stand behind
their reputations and products that people might not find on their own

\- teardown and grade for repairability like iFixit

\- doesn't spew blog articles that are merely copies of press releases

\- compare products to those to the past to see the trends in quality,
repairability, features and relative cost

The problem is that going into a store, platform or IRL, without trustworthy
information leads to more arbitrary decisions. And then when there is a more
reliable system for deciding what to buy, it's not that hard to have a browser
extension that tags items on various platforms with recommendation badges.

There are various sites that try to present some reviews in particular genres,
but the depth and breadth is usually relatively shallow and incomplete. Then
there are the zillions of "review" sites that don't actually even test the
products and just put manufacturer specs into a grid, and you have no idea if
they have undisclosed sponsors, kickbacks or other deals.

Granted, it would be a labor-intensive proposition, so I think the best model
would be a worker-owned co-op.

~~~
joshvm
In the UK you have described Which. They've been around for years and getting
a "best buy" rating is generally seen as a big deal (certainly it will get you
sales).

Their website does have a lot of blogspam type content now, and whether you
trust their "experts" is up to you. But for generic household goods, they're
probably less biased than the average comparison site.

[https://www.which.co.uk/](https://www.which.co.uk/)

------
olah_1
The most useful reviews will always be ones from within your own social
network. Because if you have a relationship with the person, they won’t
endorse something unless it reflects well on themselves.

This is why I’m bullish on protocols like SSB[1] and Iris[2].

[1]: [https://handbook.scuttlebutt.nz/](https://handbook.scuttlebutt.nz/)

[2]: [https://hackernoon.com/what-is-wrong-with-the-internet-
and-h...](https://hackernoon.com/what-is-wrong-with-the-internet-and-how-to-
fix-it-c67w32no)

------
MichaelApproved
> _The new one-tap feature asks customers to select from one to five stars for
> a product. It’s only available to customers who have actually purchased the
> item from Amazon — “verified” buyers. That barrier alone creates one hurdle
> that will make the new rating system harder to game, since Amazon does allow
> written reviews from non-verified buyers._

I understand that Amazon allows non-verified buyers to leave a review, so they
can have more product reviews but those non-verified reviews are the source of
so much garbage.

Sure, let people post non-verified reviews, so you _look_ like you have a lot
of reviews but at least let me filter them out in my search.

Filter: 4+ stars from only verified reviews.

Hell, even 3+ stars from verified reviews would be helpful.

I know sellers can fake verified reviews by paying the consumer back for the
product but that's a massively higher barrier for the manufacturer. The
difference is a few cents per review VS a few dollars per review.

I've seen products with thousands of fake non-verified reviews. I'm pretty
sure they couldn't afford to do the same with verified reviews.

~~~
beerandt
Except negative non-verified reviews tend to be helpful, especially for
sellers that constantly relist the same product under different listings.

I've bought a few products that are bad to the level of fraudulent, only to
see the product relisted so that my review wouldn't be verified.

Off course this could be abused by leaving fake negative reviews for
competitors.

Whack mole. Repeat.

~~~
heartbeats
What about making them stick, somehow?

If I leave a negative review for a product that later gets delisted, Amazon
could flag the account on basis of "deleted product with poor reviews". It
could then try and see if there are any substantially similar items listed a
week on. If so, ban them.

Perhaps even easier, make verified negative reviews stick to the seller's
account. If you delist a product with negative reviews, those just transfer
over to the seller, so you see a 5-star product sold by a 1-star seller.

------
jasode
_> But the new rating system isn’t fool-proof, since some fake-review schemers
have a way to get around the “verified purchase” requirement. One popular
method is to recruit buyers in private Facebook groups with a promise to
refund them for their purchase via PayPal after the shoppers show proof of
writing a five-star review._

I heard similar gaming techniques are used by makers of diet supplements to
build up 5-star reviews. They notify customers that they will send another
free bottle of supplements of they leave a 5-star review. This means that the
following amazon search results example showing showing overwhelming 5-star
ratings are very likely fabricated and can't be trusted:

[https://www.amazon.com/s?k=lion%27s+mane+supplement](https://www.amazon.com/s?k=lion%27s+mane+supplement)

This is how you get "snake oil" products like noncompliant USB cables or
defective Apple chargers into the "5-star" club that amazon buyers unwittingly
trust.

~~~
seunosewa
Amazon should strongly incentivize their customers to report sellers who make
such offers to them. This should make the game more dangerous for sellers
seeking fake reviews.

~~~
martin_a
If I'm somewhat "vulnerable" to "free stuff", this will not work. Couldn't
care less about Amazons problems with fake reviews if I get something for free
(that I kind of like).

------
topicseed
Sadly, Amazon and Google reviews are mostly ignored when I am looking for a
product, or service. It's so evidently clearly gamed. And Amazon's results
pages with 5 of the very same product just with a different sticker/brand is
also silly.

Reminds me of SEO's early days with keyword stuffing.

------
vsskanth
Slightly OT, but I noticed I've been banned from posting reviews or asking
questions on Amazon. No idea why. I don't review usually and my last review
was in 2017! I only noticed it a few days back because I wanted to ask a
question on a product.

Does anybody know how to get out of their blacklist ? I tried emailing but get
no response.

------
ahpearce
I did a quick Google to see if anyone was trying to solve this independently,
and this looks pretty neat:
[https://thereviewindex.com/](https://thereviewindex.com/) ... Anyone have
experience with it?

~~~
brlewis
See reviewmeta thread
[https://news.ycombinator.com/item?id=22327272](https://news.ycombinator.com/item?id=22327272)

------
stebann
What about the trade offs between privacy and the possibility of being part of
a community? I mean, you have some way to identify core characteristics of the
subjects whom make the reviews, and if you can't do that, how can you trust
them?

------
krisroadruck
I wonder what the average account age and purchase total is of genuine amazon
accounts. Seems like setting some sort of age and purchase threshold to
reviews could make the fake review game a lot more difficult.

~~~
heartbeats
One approach might be to shadowban fake accounts' reviews. They still show up
as text, but they don't get counted in the average. If Amazon at T = 0 know
which reviews are legitimate and which aren't, someone who consistently gives
five-star reviews to shit products could have their ratings lowered in
importance.

You could go all the way with this: look at my reviews, and then only show me
reviews made by people with similar tastes.

~~~
julianlam
A good idea. I'd wager that a high proportion of users who join these "rating
review" schemes are accounts that rate 90+% of their items as 5 stars.

If you check my Amazon account, I don't even rate most of my purchases.

It's a rather low sensitivity metric but one that has some value.

------
user00012-ab
I ended up just not buying stuff from amazon anymore, because I was just
getting literal trash in the mail that I would have to deal with.

~~~
miguelmota
What kind of stuff were you buying?

~~~
user00012-ab
The final straw was a NEW mechanical keyboard. I received a keyboard covered
in perfume, that was clearly (and badly) repackaged.

------
tartoran
Perhaps limiting the number of reviews a user can make could mitigate this
somehow?

