
Amazon Reviews: How We Spot the Fakes - Osiris30
http://thewirecutter.com/2016/05/lets-talk-about-amazon-reviews/
======
jonstokes
I actually know people whose job it is to write these things, and to create
and maintain sock puppet personas on forums for the purpose of subtly
promoting clients' products and disparaging those of competitors.

Then there are the professional reviews, which are all conflicted because most
reviewers either get to keep the gear or the gear maker is a sponsor of the
site/publication and the reviewer knows it. I say this as a person who's still
involved in publishing reviews (of outdoor gear, mostly), and who most of the
time gets to keep whatever some company sends me.

However rotten you think the adtech industry is, the state of product reviews
online -- user generated, casual forum reviews, professional reviews, etc. --
is worse.

I have a standard rant about the current state of the web that I give all my
family and friends, and it always ends with: "if you didn't pay to read it,
you probably read an advertisement."

~~~
jonaf
The FTC requires disclosure for reviews that were written by individuals who
were given a product for the purpose of reviewing it. It is considered
anticompetitive and illegal to forego disclosing this about a review.

As an engineer at perhaps the largest ratings and reviews company worldwide, I
would love to hear your standard rant in long form. Please pm me if you feel
up to that conversation. Ideally the result would be an improvement in the
quality of the content of billions of reviews.

~~~
Pxtl
Do those regulations have any teeth? Because it seems like it would be very
hard to enforce.

~~~
jonaf
If the FTC contacts you directly, expect your corporate lawyers to ask you to
prioritize resolution over pretty much anything else.

------
xiaoma
One thing I've found useful is following specific _reviewers_. For example,
Peter Norvig reviews many, many programming and CS texts.

He has both glowing reviews such as this one:
[http://www.amazon.com/gp/review/R30AQNQK2I1O7P?ref_=glimp_1r...](http://www.amazon.com/gp/review/R30AQNQK2I1O7P?ref_=glimp_1rv_cl)

And scathing ones such as this one:
[http://www.amazon.com/gp/review/R2C7L5KHUVHOR2?ref_=glimp_1r...](http://www.amazon.com/gp/review/R2C7L5KHUVHOR2?ref_=glimp_1rv_cl)

By following several reviewers I respect in each domain I care about, it's not
hard to find quality texts (or products) that I'd never have found by typing
words into Amazon's search box and looking at average review scores.

~~~
hkmurakami
A Googler reviewing every USB Type C cable (and leaving scathing reviews for
ones that suck) comes to mind.

[http://gizmodo.com/a-google-engineer-is-publicly-shaming-
cra...](http://gizmodo.com/a-google-engineer-is-publicly-shaming-crappy-usb-c-
cabl-1742719818)

[http://www.amazon.com/gp/pdp/profile/A25GROL6KJV3QG/ref=cm_c...](http://www.amazon.com/gp/pdp/profile/A25GROL6KJV3QG/ref=cm_cr_rdp_pdp?tag=gizmodoamzn-20&ascsubtag=ac0ae1fabade4e73c38d53c2ca51c3e4a93b27e3&rawdata=%5Br%7Chttps%3A%2F%2Fwww.google.com%2F%5Bt%7Clink%5Bp%7C1742719818%5Bau%7C5836322932740811680%5Bb%7Cgizmodo)

~~~
Orangeair
That's a great example too, because Amazon actually ended up following up and
banning cables that were found to be non-compliant with the spec, largely
based on his reviews: [http://lifehacker.com/amazon-bans-non-compliant-usb-c-
cables...](http://lifehacker.com/amazon-bans-non-compliant-usb-c-cables-
should-make-for-1767935544)

~~~
click170
I like that they've said they'll start banning the cables, but I'm
disappointed that - from my reading of the article - Amazon isn't doing any
testing to detect these bad cables and are instead relying on customer reviews
to report bad cables. That means for most cables someone's going to have to
buy a cable and have it damage their device or have the equipment and know-how
to detect it ahead of time.

Having Amazon test every cable obviously doesn't scale either but couldn't
they require the vendors to provide test results showing that the cable is
spec compliant? Of course the vendor could forge the results, but that's
intentionally malicious and this would at least catch non-malicious accidents.

~~~
hinkley
Are these bad cables UL listed? That seems like it would be an easy enough
filter for someone like Amazon to employ.

Electricity goes through it. Check. UL listed? Check. Okay, goes on the
suggestions page.

------
anexprogrammer
Half the time the fakes are so ludicrously easy to spot that I really wonder
why they bother. Fifty glowing reviews of a book within a day of publication
all raising the same 4 esoteric points in the review. No doubt 4 points
suggested by the author/publisher when asking for the astroturfing.

Completely disregard all Amazon Vine reviews.

Now weight heavily in favour of reviews that have had update edits - "it
failed after 2 months. The replacement failed 3 months after that" (common on
all makes of $200 LED gaming keyboards) or "after a year it's still in perfect
condition". Photos on the reviews are often a good sign too - but only when
there's just a few - photos on reviews are _rare._ So lots with similar
framing would be an instant strike.

Try finding a CAT6a cable on Amazon - seems they're all 4.5 - 5* wonderful,
yet dig into the bad reviews a bit and most of them don't even meet wire or
shield spec. Even those coming from Amazon themselves.

Aside from reviews from specific, trustworthy people, reading the mid-tier and
bad reviews gives a much better flavour of what a product is about, and the
odd review tearing the product to pieces can be worth its weight in gold.

If it's a marketplace seller my starting point is to mistrust all reviews and
let them earn trust _really slowly._

I sure won't be going near Fakespot, especially when they're so intentionally
vague in describing their own product - it'll be a score to game like your
Klout score and all the other meaningless metrics. The astroturfers will
adjust what's needed in the fakes to pass.

~~~
_puk
I actually avoid the reviews with photos as 99% of the time they end with "I
got this product for my fair and unbiased appraisal" or somesuch.

I've never quite got around to writing a browser plugin that excludes any
review that have such a statement, and / or builds up a database of reviewers
that are trustworthy on amazon.

The guys in the Top x of reviewers, who receive all this crap to review,
basically it's their living, they aren't going to start bad mouthing it.

Maybe take the star rating of each review within a month of a new listing and
then compare the average 6 months later to see how shill those initial
reviewers are.

~~~
zerocrates
> Maybe take the star rating of each review within a month of a new listing
> and then compare the average 6 months later to see how shill those initial
> reviewers are.

Steam just updated its user review system to do just this: it shows both the
overall rate of positive reviews as it did before, but also the positive rate
in the last 30 days. Now, their prime motivation was probably more to do with
things like patches (or the lack thereof), but it would probably do a pretty
good job of counteracting paid/biased reviews from around launch as well.

------
kilroy123
I no longer trust Amazons reviews. They're so obviously loaded with fake
reviews. I try to be smart about it and sort by most recent. I've been burned
on a few products recently as well.

I met a shady guy recently, who has personally paid for tons of take reviews
on kindle books. After hearing how easy it is from him, I really lost trust.

~~~
jonstokes
It's not just the reviews that are fake. I recently tried to buy a plush
Baymax doll for my daughter, and there were so many people complaining about
getting bad Chinese knockoffs from the sellers, that I gave up. They not only
have lost control of their reviews, they've lost control of the vendors. Their
whole storefront operation is circling the bowl, and I say this as a longtime
Prime subscriber.

~~~
Spooky23
Ditto here. Amazon is almost as bad as circa 2004 EBay these days. Its
impossible to find genuine products in many categories. Ordered a specific
tool a few months ago, and a knockoff came in the mail via "epacket" from
China.

My assumption for most products these days is that if Amazon is cheaper than a
retailer, it's grey market.

IMO, they did a Walmart -- at some point the mission shifted from serving the
customer to total domination at any cost.

~~~
bitJericho
I don't think so. Anything I buy shipped from and sold by amazon I know will
be legit. I don't buy anything third party if I know I cant verify it's
authentic.

EBay is much worse in this regard as there is nobody I can trust to have
properly sourced their inventory.

~~~
GurnB
Buying from Amazon is NOT a guarantee of authenticity. Popular items such as
PlayStation controllers & Beats Headphones have a bad reputation for being
knock offs.

~~~
chrisper
Yes it is. If it is shipped and SOLD by Amazon Inc., it is pretty much legit.
There is, however, also just shipped by Amazon, which is basically selling
through third party, but Amazon handles shipping.

~~~
GurnB
I understand the difference between SOLD by Amazon, and Amazon doing
fulfillment. Products SOLD by Amazon are still not 100% guaranteed legit. That
being said, I have not had any problem returning/getting credit for knockoff
(or questionable) product.

~~~
chrisper
Only way that can happen, I think, is when someone did a bait and switch with
a return.

~~~
GurnB
That is certainly one way. If Amazon is getting product from multiple
suppliers, they can easily taint their inventory with a single bad vendor.
Then there is always the human element. Amazon has a reputation as a somewhat
'crummy' employer. Warehouse workers, when pressed to meet order/put-away
quotas, tend to put inventory in pick locations based on the product type, NOT
where the warehouse management system directs them to. (This of course causes
all kinds of stock level issues not to mention accounting issues for those
third-party suppliers that Amazon is acting as a sell-through for) I do not
believe that Amazon is intentionally buying knock-off products with the intent
to scam customers but just based on their physical size & the variety of
products that pass through their facilities, it is going to happen.

Back in the late 90's, early 2000's before Amazon was as big as they are now,
I worked in the commercial music distribution arm of Sony/BMG. We were the
ONLY people that were manufacturing (In our own facilities) CD's for bands on
our music labels and yet bootleg copies of CD's were going out Amazon's doors
to customers. How? Amazon doesn't keep an abundance of stock on hand. If they
run to low, they would buy product for One Stops (at a higher price than we
sold to them) because the One Stops could get several thousand CD's in a few
hours while an order from our warehouse might take 36 hours to get there. It
was a stop gap for them to not go Out Of Stock. Those One Stops should only
have been getting their product directly from us also, but no one watched
their inventory as closely, and they would at times buy product from
unofficial manufacturers. Those One Stops were trying to scam people. (Mostly
Mom and Pop CD retailers)

Again, I have not had any problem returning product to Amazon.

~~~
petra
>> Warehouse workers, when pressed to meet order/put-away quotas, tend to put
inventory in pick locations based on the product type, NOT where the warehouse
management system directs them to

Have you got any source for this being a real problem ? Because it seems such
things are easily detected at the packing station, by barcode. And if an
employee does so often, he'll probably won't stay long.

~~~
GurnB
Only speaking as an IT professional that has spend a couple of decades working
in various distribution environments. If you are a pool manager in a warehouse
this is a real problem. Warehouse employees working the receiving docks seem
to always find a way to empty a truck at the end of a shift, even if that
means shoving pallets of product in the nearest open rack slot. Yes, the WMS
will eventually catch the problem and track it back to the employee. In a
warehouse working with food/consumables there are lot controls that are
suppose to be adhered too. (For tracking and recall purposes). The barcodes on
a 'quality' knock off is going to be the same as the original product.
Warehouse workers don't care if the product in a pick location is genuine or
not. They are managed/measured on their picks per hour & accuracy. (Accuracy
based on where they were directed in the warehouse to pick and the correct
quantity) Most warehouse employees are through temp agencies and their work
ethics can be, though certainly not always, somewhat lacking. Honestly, they
typically aren't treated with respect from the start. This article is fairly
representative of what I have seen in at least 2 of the 3 warehouse
environments I have been associated.
[http://www.motherjones.com/politics/2012/02/mac-
mcclelland-f...](http://www.motherjones.com/politics/2012/02/mac-mcclelland-
free-online-shipping-warehouses-labor)

------
fpgaminer
One potential solution is for Amazon to do a better job of encouraging users
to rate every product they buy. That would quickly drown out the fake reviews
and make such tactics uneconomical.

Off the top of my head I would suggest that on the post-checkout screen they
should list the customer's most recent purchases asking "Please rate your
current satisfaction with your recent purchases". Just a quick 5-star rating,
a small button to write a textual review if the user wants, and a note clearly
stating that their review can be changed later if their opinion changes.

I suggest putting that at post-checkout because it's a point where Amazon
still has the user engaged, so asking is not annoying (like their emails
currently are), but the user is done shopping so they aren't preoccupied with
something else. In other words, it is the least annoying time to ask. They
should, of course, also have a rating box on the home page (again, a list of
recent purchases, possibly muxing in older purchases at random).

And then weighting reviews is super important. Reviews given by users later in
their ownership of the product should be weighted higher than reviews given
just after receiving the product. Changed reviews should be weighted higher,
as well as reviews on returns.

~~~
AdmiralAsshat
Actually, they do this already. I get e-mailed about once a week from Amazon
asking me, "How would you rate <<Thing you bought a month ago>>?"

I frankly find it annoying. It's sad to say, but unless the item was either
really good or really bad, I don't feel compelled to review it.

The other problem is most people probably review their things as soon as they
get it. Very few go back and edit their reviews if the thing disintegrates six
months later, with the net result that the product has overwhelmingly positive
reviews despite having the durability of the Dead Sea Scrolls.

~~~
_puk
The ones that drive me nuts are the 5 star "I bought it for my husband for his
birthday next week" kind of reviews..

If the product hasn't been used for at least a few days / weeks (dependant on
the item), then it's not a true review; more a reflection on how good Prime is
at getting stuff to you on time and with no hassle.

~~~
crapolasplatter
LOL, exactly but the more annoying ones are in the Questions sections.

There is always someone who comments that they don't have the answer.

WTF just dont post if you don't have the answer.

~~~
jessaustin
That's actually Amazon's fault. They send a personalized email to the
customer, saying e.g. "You purchased thing, and another thing customer has an
important question about thing that you should answer by clicking this
button." Sophisticates like the HN crowd know that clicking that button to say
"n/a" isn't actually helping anyone. Amazon should know that most of its
customers are not that sophisticated.

------
ams6110
If [http://fakespot.com/](http://fakespot.com/) can spot fake reviews, Amazon
themselves should be able to do it. It's in their long term interest that
product reviews are genuine so that buyers have confidence. If I can't trust
Amazon reviews, I'll just shop at Target where at least I can handle the
physical product before I buy it.

~~~
CWuestefeld
A friend recommended fakespot to me a few months back. I tested it by pointing
it to a product that I'd recently reviewed. It came back saying that the
reviews were questionable, giving as an example of a shady review _the one I
'd just written myself_!

Needless to say, I wasn't impressed with their heuristics.

~~~
dave2000
How do I know that you're not a shill for fakespot?

How does the person reading this comment know I'm not a fakespot competitor?

It's a tricky business, isn't it?

~~~
sudojudo
Good point. But, for pedantry's sake, your analogy is backwards. The person
you're replying to is questioning the accuracy of Fakespot, which would make
them a competitor. A shill would be endorsing the service, or placing doubt on
the critical comment, as you have done.

~~~
dave2000
You're totally right. I have no idea how I got that backwards!

------
leejoramo
There are other related problems with Amazon's product pages:

 _Combining different items_ into one product page and review. For example
take a look at the review for the "Dell Ultrasharp U2415 24-Inch" display.
There are 6 different sized monitors on that page. Some have very different
technical specs. Combining products makes sense when the difference is color
preference or the size of a shirt. But the 24-inch U2415 is a very different
display from the 34-Inch curved screen U3415W

 _Question and Answer_ section just before the actual reviews. I often see
nonsense such as a question about a computer cable:

Q: Can I eat it?

A: I eat mine with ketchup.

~~~
uremog
Or the ever insightful:

Q: [Any question]

A: I don't know.

~~~
bobsoap
That boggled my mind for some time as well, but if you follow the rabbit, the
answer is quite simple.

Whenever someone asks a new question, Amazon sends an email to previous buyers
of the product. The email is worded in a way that entices a quick reply from
the user. I forget the exact language they use, but it's very straight-
forward, and it seems to make some people think that someone asked them
personally. It entices them so much that they actually reply to the email like
they would to another person's email who is asking them directly... They are
just being polite.

What's funny is the irony in all of this. Amazon, a company that famously
split tests nearly everything, manages to send people emails that look so
genuine they get them to respond, even if they don't know an answer. And yet,
the same company can't get the same people to add some genuine product
reviews, which would help drown out the fakes.

Of course I don't have the data to know the response rates of these emails. I
also don't claim to know that drowning out fakes with more legitimate reviews
is the end-all solution. But it does make me wonder why we're even having this
discussion, and why we even need sites like Fakespot, while Amazon, sitting on
all that data about every one of their users, somehow seems to be unable to
put it to use.

~~~
uremog
That's funny, I had no idea they sent those emails even though I've had Amazon
for a long time. Thanks.

------
stagger87
I love that Amazon tells me when a reviewer is part of the Vine program, so I
can immediately ignore it. It's funny when a niche product has overwhelming
negative reviews and a string of 5 star vine reviews.

~~~
randyrand
The problem with Vine reviews is the users _don 't actually want or need the
product they are reviewing_.

When you get something for free that you don't need, it's much easier to like
it. When you're actually buying something to use it daily, you become much
more critical.

~~~
grogenaut
My problem with vine is that it's good for general stuff like lamps or cords
but when it comes to more intricate things like say an embroidery machine or a
vinyl cutter (guess what I'm doing as a hobby recently) then they don't have
enough domain knowledge to give accurate reviews. I also feel like a lot of
them feel they have to give good reviews to keep in the vine program which I
directly know is not true.

~~~
polshaw
The vine program is a disaster for Amazon reviews credibility IMO. The
reviewers are almost never invested in the product and most come down to _"
this product does the thing it says it will do!"_ and then goes on to some
largely irrelevant surface details _" looks great, fits just right in my
purse"_ and perhaps compares it to items in a completely different class
instead of its actual competition _" these $90 headphones are much better than
my iphone ones"_. Some do make a good effort and can be supplementarily useful
but they just don't have the same concerns as people looking to shell out full
price.

I understand Amazon's (face value) motivation for it; decent products that are
passed by because nobody reviewed it. But once you have one or two decent
reviews all they do is add to the noise. Some sectors I've been looking at
lately (eg USB chargers) are absolutely flooded with them to the point where
it can be hard to find regular reviews. I used to use sorting by average
customer review as a quick hack to see some of the best products in a sector,
now it is often synonymous with "most spammed"; they are killing a key USP.

It's all very well playing the "we are transparent and non-coercive" card but
the reality is people are very unlikely to start rocking the boat when you are
handing them free stuff, no matter Amazon's position. And call me sceptical
but don't forget the role of the selection process.

I'm sure it helps sales in the short term but long term it's a massive own
goal. IMO they could salvage it by enforcing a 2 freebie reviews limit, or
allowing users to filter them-- especially in the overall star rating.

~~~
grogenaut
Are you seeing the actual green Vine Reviewer at the top of these comments or
the "I was given this product for free in return for an honest review" at the
bottom text that a lot of astroturfers are using? I thought the # of vine
reviews was limited but it may be up to the manufacturer on how many they send
out. Vine reviewers don't choose what they review other than they can turn
items down iirc.

------
mmanfrin
The other major issue is in bait-and-switching products. A company can put out
a great product that generates true 5-star ratings, and then swap out their
product with a cheaper version but continue selling it at the 5-star-product-
price.

------
biot
I was looking at bathroom scales recently and noticed this myself. Lots of
five star reviews, many of which received the product free in exchange for a
review. If you look deeper, most of these reviews are superficial. Adjust the
review filter a bit and a clearer picture emerges with one and two star
reviews from people who use it daily and have noticed a ton of flaws.

The product in question:
[https://www.amazon.com/gp/aw/reviews/B0113YL5V4/?sortBy=rece...](https://www.amazon.com/gp/aw/reviews/B0113YL5V4/?sortBy=recent&pageNumber=2&reviewerType=avp_only_reviews)

If you search for glass bathroom scales on Alibaba, you can find this exact
model available at a price of $4/unit in volume so it's not surprising if it
ends up being not the greatest quality.

~~~
biot
I just realized I botched the link. Corrected:
[http://www.amazon.com/Etekcity-Digital-Weight-Bathroom-
Scale...](http://www.amazon.com/Etekcity-Digital-Weight-Bathroom-
Scale/product-
reviews/B0113YL5V4/ref=cm_cr_arp_d_viewopt_sr?ie=UTF8&showViewpoints=1&reviewerType=avp_only_reviews&pageNumber=1&sortBy=recent&filterByStar=all_stars)

Shows most recent verified purchases.

------
ldpg
Here's just a few examples of listings with mostly fake reviews:

[http://www.amazon.com/dp/B01DN9UKHC](http://www.amazon.com/dp/B01DN9UKHC)

[http://www.amazon.com/dp/B01D0JDZFO](http://www.amazon.com/dp/B01D0JDZFO)

Most top listings are saturated with fake reviews. If you want to understand
the depth of the fake listing problem, look at the thousands of people trying
to create new listings in high-competition markets. Search Amazon for:

    
    
        "silicone oven mitts"
        "bamboo cutting board"
        "meal prep container"
    

Click any result, then scroll down to the "Sponsored Products" section for
that item. Repeat this, really dig down into the "sponsored" lists. Notice the
amount of copiers.

Now look at the reviews. For new sponsored products in these categories
virtually _all_ of the entries are from review-mill programs. There can be
hundreds of fake reviews on a single listing. Many of the negative reviews you
may see on these are fake as well, from competitors!

Review-mill programs work by having the reviewers buy the product through
amazon (they are reimbursed or discounted). Then they do a write ups. Many are
short and organic looking, the more easily spotted ones contain lots of text
and multiple photos of the product.

------
bitL
Overall there seems to be an epidemics of "fakeness" in Internet ratings -
even looking on IMDb for movie ratings I see e.g. a movie 7.7/10 but first few
pages are unanimously 1-4/10 by what seem like genuine reviewers complaining
about being deceived by IMDb to watch the movie...

~~~
minimaxir
For movies, IMDB average scores are _exclusively_ between 5 and 8: a Four
Point Scale. In contrast, Rotten Tomatoes and Metacritic scores hit the full
gamut reliably: (I wrote a full statistical blog post on this with
visualizations: [http://minimaxir.com/2016/01/movie-revenue-
ratings/](http://minimaxir.com/2016/01/movie-revenue-ratings/) )

I did a similar thing for Amazon reviews many years ago but it was not very
insightful. ([http://minimaxir.com/2014/06/reviewing-
reviews/](http://minimaxir.com/2014/06/reviewing-reviews/))

~~~
kale
Average scores should trend towards the center. I'm not sure if it's an
example of the central limit theorem (I think that deals with different
distributions tending towards normal), but you'd expect IMDB scores to trend
toward the middle.

~~~
minimaxir
Central limit theorem assumes that events are independent and random. Which is
not the case. (For one, there is a selection bias in that people would only
review/vote on a product of they strongly like or strongly dislike it)

The lack of trend toward the middle implies that the scale is not great. But
there is no incentive to fix it.

------
p4wnc6
This reminds me of a recent HN thread about Amazon Echo. I brought up my
skepticism that the undisclosed but touted Amazon sales of Echo were really a
signal that the device is being adopted and well-liked by the consumer market
that Amazon sought to target.

And, for saying anything even slightly negative about a big tech company, I of
course got downvoted to oblivion by everyone adding their own anecdata.

Most loudly, at least one commenter linked to the reviews on Amazon itself,
some tens of thousands with a 4-ish star average, as if this was definitive
proof of something. Even just casually looking through the reviews, it was
easy to see that almost all of the glowing, positive reviews were ghost
written, or came from "Top X reviewers" who clear have a selection bias for
offering positive reviews to reinforce their status as a highly ranked Amazon
reviewer, or from people who got advanced copies of the device (again,
selection bias if they are the sort of people who actively seek out new tech
to review), and on and on.

What an echo chamber this place has become (no pun intended).

------
xt00
I know people that sell stuff on Amazon and they don't use fake reviews but if
a competitor does use fake reviews it makes it hard not to want to make fake
reviews also.. It's a vicious cycle.. Plus they get negative reviews out of
nowhere.. Very likely the same competitor is paying to put negative reviews on
your page.. So it's a double whammy.. Add a bunch of 1 stars to a competitor
page and a bunch of 5 stars to your own page... Basically need to clean up the
whole system somehow..

~~~
xt00
What would be ideal is if there were like "real reviewers" that are verified
and their review carried more weight.. Which obviously sucks for the seller if
they give a bad review.. Maybe there could be a mitigation system where the
seller could try to fix something then mark the problem as fixed or something
and then the reviewer could mark it as they agree, then they could update
their review..

~~~
shostack
A lot of the "fake" reviews are by real reviewers. It is a common tactic to
reimburse reviewers for purchase of the item so they then show up as a
verified purchaser.

------
okket
The problem is: Why should I write a review? I have nothing to gain.

Maybe I have had a bad experience and want to warn others: That would explain
mostly bad reviews for almost every product. (Given enough users, some will
have bad experiences, if only it is by them misusing it)

I can understand that mostly bad reviews creates an incentive to counter these
with good reviews, but from the wrong side (seller, not consumer). This would
explain the many fakes.

I guess reasonable trustworthy 3rd party sites like consumer reports are still
necessary.

~~~
jbob2000
I write reviews all the time, and to be honest, Amazon makes it pretty easy.
Basically if it shipped on time or earlier and was exactly what I expected, I
will write a review and say exactly that.

~~~
beberlei
What has shipping to do with the products quality? There are two things you
buy at amazon: 1. a physical product 2. the amazon service and shipment. The
second one should IMHO not be relevant for the peoduct review, yet many rate
them and skew the results.

~~~
jbob2000
Yeah, fair, but if it's not a common product that amazon keeps on their
shelves, then it's an important note, I think. I usually only buy stuff on
amazon if I can't find it in the store, so it's usually a somewhat rare or
very specific thing.

------
thruflo22
My thought on this was to develop a generic review system (ie would work for
anything with a url) that shows you subjective results based on your social
graph / trust map and weeds out bad actors by crunching the advogato
algorithm.

The challenge is liquidity, ie how do you seed the network and content but if
you can solve that you can operate a review service for a fraction of the
resources currently being wasted on these red queen races.

~~~
jameslk
> The challenge is liquidity, ie how do you seed the network and content but
> if you can solve that you can operate a review service for a fraction of the
> resources currently being wasted on these red queen races.

I've been thinking about this problem a lot too. There's actually many
services that have come and gone in the review space. The solution I've come
to is that any review service needs to be more than just reviews. It really
needs to be more of a community.

------
ck2
Another problem on amazon is allowing 3rd parties to "piggy-back" on original
listings that aren't available directly from Amazon itself, and then the 3rd
party will either substitute or not actually have the original item at all and
ship whatever.

So the original item gets legit great reviews. But the reviewers never mention
which seller they bought from. 3rd parties glam onto the high ratings and a
lot of people get disappointment either with fulfillment or the product
itself.

This is why I no longer buy unless it is amazon or amazon fulfilled (so I can
at least return it). But you pay a premium for that.

Amazon could solve this by just noting which vendor the reviewer bought from
on the subject line.

------
toephu2
Amazon needs quality control. They sell more and more crap and you have to
spend so much time perusing the reviews to find out if a product is actually
good or not. Sometimes I prefer to goto Costco where yes I deal with long
lines but at least I know what I am buying is of decent quality, the prices
are low, and they have a very generous return policy (most electronics can be
returned within 90 days and anything else you can return whenever you want
with or without the receipt).

------
magoon
Sellers also game the system by offering a refund, without requiring a return,
in exchange for reverting a bad review. I've reported this to Amazon but they
weren't receptive.

~~~
green_lunch
This isn't really 'gaming' the system. It's good customer service. Even one
Bad Amazon review can be death to a business on Amazon.

I think it's a little ridiculous that so much power is given to many buyers
that only want to take out their anger on someone, even if it isn't really the
fault of the seller.

I used to be an Amazon seller and quit because so many people would leave
negative feedback over unrealistic expectations.

It seems to be the M.O. of the millennial generation.

------
PaulHoule
I am more afraid of fake bad reviews.

In the case of electronic, photography stuff and similar you find people get
more consistent results with some products. For instance, charging cables.
Hypothetically you could buy a $5 third party cable vs a $20 OEM part, but any
thought you might have a fire leana to the $20 cable.

------
maaaats
I liked Amazon better before it became what Ebay became. It used to be a
warehouse, now it's thousands merchants competing for exposure, thus prompting
this.

When I search for a product, I now often get tens of matches, all almost
identical, all having different prices and description and equally bad
photographs.

------
Nagyman
It certainly makes it more difficult to compare products. These third-party
sellers aren't doing themselves any favours. I recently wanted to purchase a
simple phone case (Samsung S7 Edge, so relatively new). ALL of the cases had
4-5 stars, which makes reviews useless during research mode. It was pretty
easy to spot patterns in reviews that were essentially paid-for but very
annoying.

In the end, I ordered a case (after checking YouTube video reviews) and was
contacted by the seller to review their product on Amazon in exchange for 10%
off my next purchase with the same seller. BUT not if I was going to give them
less than 5 stars; in _that_ case they wanted me to contact them directly.

/end anecdote

------
googletazer
There are portals that connect reviewer shills with sellers who want to boost
their product ranking. Google "buy amazon reviews" and you'll see it. Some do
it individually, on fiverr for example.

The portals themselves are a very lucrative business, and are not going away
anytime soon. There is an arms race between sellers to get that sweet sweet
real estate (get their products on page 1 or page 2), and business can be a
cutthroat matter. Of course, the system will eventually come to a steady state
when too many buyers get burned buying crap. Review and feedback systems is a
topic I'm very interested in.

------
tmaly
I am glad they added verified reviews that are for people that actually bought
the product.

I wish they would take it one step farther and add a option to only show
verified reviews and associated stats for those verified reviews

~~~
nerfhammer
Verified doesn't help much with spam reviews, the seller gets most of the
money back from buying their own product.

My personal filter is to ignore any review that uses any variation of the
phrase "found this little gem!".

------
exratione
Clustered timing of reviews is unfortunately not a great metric for
determining whether or not they are legitimate. Many - probably most -
companies send out emails to buyers asking them to review, no compensation
provided. This is pretty effective, and a big driver of legitimate reviews.
The timing of those emails is determined by response rates, which tend to be
better on specific days - hence the mails are batched. So for some large
entities, you'll see reviews clustering on Thursday to Saturday for example.

~~~
Magnets
Amazon manually check reviews, which may also cluster

------
Godel_unicode
And now we get to have a recursive "but what's _your_ bias" discussion. For
instance, would a writer for a site which posts professional reviews have a
bias (conscious or otherwise) against reviews written by amateurs. Add in the
"if that person was good I'd be out of work" bias and here we are.

Note that I'm talking specifically about the authors critique of the vine
program, obviously the review spam is a real problem.

That aside, I do find wirecuter to be a good starting place.

------
kin
Yeah after being tricked by a product or two I've now developed a keen eye for
products with fake reviews.

Funny story: a co-worker of mine fell for the fake-review trick and bought a
product, talking about its rave reviews. I knew from looking at the reviews
that they were fake and he was going to be disappointed. He said "No, look, it
has great reviews". I said okay, you're going to be disappointed. And he was
disappointed. It was the "Amazon featured" lock laces.

~~~
bcassedy
Reviews in general are not a great indicator because the average person writes
a terrible review. Either the review is made based on irrelevant details or
they lack the perspective necessary to review the item appropriately.

Two recent examples:

There is a cheesesteak restaurant near me that has 4 stars on Yelp. People
rave about how they have the best cheesesteaks ever! Yet if you look more
closely, reviews from people that have eaten a cheesesteak on the east coast
are much lower. After trying it myself, the food is terribly bland.

I was recently looking at coffee makers. The "Ninja" coffee maker has glowing
reviews (though I suspect a chunk of them are fake). But the reviewers are
comparing this $200 Ninja machine to their $30 Mr. Coffee. Again people that
actually like good coffee are rating it much lower. Then looking at the
reviews for a different coffee maker, many reviewers are comparing the result
to their Aeropress or lower end Espresso machines.

~~~
Spooky23
> There is a cheesesteak restaurant near me that has 4 stars on Yelp. People
> rave about how they have the best cheesesteaks ever! Yet if you look more
> closely, reviews from people that have eaten a cheesesteak on the east coast
> are much lower. After trying it myself, the food is terribly bland.

Food is tough in some regions. In California, you get open warfare about the
authenticity of Mexican restaurants. In the Midwest, people think Subway is a
New York Deli.

------
CM30
When it comes to Amazon reviews, let's not forget some of the older examples
of them being gamed. Before the internet 'services' and spammers offering to
make fake reviews, the same thing was being done by notorious dodgy authors
and creators on a more manual level. Like Robert Stanek, a notorious self
published fantasy author who posted thousands of fake reviews of his books
(and probably still does):

[http://conjugalfelicity.com/robert-stanek/gaming-the-
amazon-...](http://conjugalfelicity.com/robert-stanek/gaming-the-amazon-
system/)

For example, his books sold maybe ten copies a week, yet somehow got hundreds
of glowing and suspiciously worded reviews on Amazon. Got to the point other
sites (and Wikipedia) basically put up warning signs.

So yeah, even before the current crop of reviewer sellers started popping up,
this system was being gamed by the authors and creators themselves using
sockpuppets to flood it with fake reviews for their work.

------
hippich
Yes - there are fake reviews, but there are also bunch of discount-for-review.
Most of these reviewers are leaving disclaimer and often leave 1-3 star review
in my experience.

Now the reason for that is because it super hard to get going with your
product without sending discounted samples to reviewers. Because even if you
spend tons of money on ads and somehow convince customer to buy your product
without reviews, that customer in 99.9% will never leave a review. And if you
dare to email this customer to ask for feedback - they will curse you,
complain, report to amazon.

That's where things currently are - it is two way street. Without any traffic
on it you will end up with very few shitty products by big companies who will
have no reason to improve quality or innovate.

Just 2c from the other side of the fence.

------
jldugger
Why is the wirecutter even reading Amazon reviews? Isn't their job to give
independent analysis?

~~~
4ad
How do you start your independent analysis if by not looking at what's
popular? Doing your own analysis and reading amazon reviews are not exclusive.

I find wirecutter usually accurate in facts, but I rarely agree with them on
their recommendation of what is good for _me_. That being said, they are very
explicit in why they recommend something, I just wish there were other
reviewers who shared my set of interests and values in things.

~~~
jldugger
Bear in mind that they also filter out anything they can't get a referral fee
for.

------
guelo
If FakeSpot can detect fakes Amazon should be able to do it even better. The
fact that FakeSpot even exists shows that Amazon has decided that they've won
the online retail competition and there's no need to spend any more resources
on improving it.

~~~
TheOtherHobbes
Does FakeSpot exist? The progress bar hangs whatever product I try it on.

Meanwhile the "Recommended with honest reviews" products have acquired some
one star reviews of their own.

Useful or not? No idea. Doesn't look like it so far.

But this is probably an unsolvable problem.

You _could_ solve it by hiring and paying 100% independent mystery shopper
reviewers with unimpeachable integrity, and only reviewing products actually
purchased for real money.

I don't think many online merchants would support that.

This is a fundamental problem with scarcity transactions - it's _always_ more
profitable to game the system than to play honestly.

------
uptown
Try to buy an in-ear or forehead thermometer on Amazon based on reviews? I've
found it impossible - because every one they sell seems to have its reviews
tainted by people who received free merchandise in exchange for their "honest
review".

~~~
wmeredith
This is off-topic, but I feel your pain and have a (un-paid, and unaffiliated)
recommendation. My wife and I went through about 10 thermometers of all types
and price points the first year our first child was born. They were all of
dubious accuracy, and worse: inconsistent.

Then, at the pediatrician's office we noticed them using an inner-ear
thermometer. I asked the nurse to see it. It was a Braun Thermoscan. They
aren't cheap at ~$70, but we bought one 5 years ago and that was that. It's
fast, accurate and consistent. It paid for itself long ago and still works
like it was new.

And one more time, because of the context of this comment: I have no
affiliation with Braun.

------
VeejayRampay
The reviewing system on Amazon gives WAY too much space to the hoards of
morons would give products one star because the package was damaged on
arrival. As long as that's the case, the average rating won't mean anything.

------
Magnets
The other problem is that companies provide free or heavily discounted
products in exchange for a review.

So they are using real users who then with a wink and a nudge provide 4 & 5
star reviews to keep the free products coming.

------
chaostheory
The only thing that the author doesn't seem very familiar with is with
Amazon's Vine program which isn't surprising since there aren't a lot of
members.

> For example, returns and long-term use aren’t part of the evaluation. When
> you get something for free, you’re less likely to follow up on breakage
> concerns or customer service issues. Additionally, if the reviewer didn’t
> actually buy the product, that person doesn’t take the purchase and shipping
> processes into consideration.

I don't feel that this is true. People who are selected for Vine aren't random
Amazon customers. They are Amazon's top ranked reviewers. A lot of these
reviewers have started reviewing hundreds of items before Vine or any paid
reviewing programs even came into existence. i.e. They do it for the 'love' so
they will discuss price and shipping considerations

Yes given Vine's 30 day review deadline, you're not going to see anything
initially about long term use; but... a lot of these reviews do get updated if
anything about the product changes from the original review.

> You might notice how few of the reviews through Vine and similar programs
> are negative or even critical. This isn’t a case of reviewers intentionally
> being dishonest, but rather the result of unconscious positive bias.

I don't disagree here.

> Not paying for an item can make difficulties with that item seem less
> irritating.

Vine items are now treated as taxable income starting this year, so Vine
reviewers are now paying for the items that they select.

> Additionally, reviewers may give their opinions on items for which they have
> no expertise or real experience and therefore have no frame of reference
> about how well something works by comparison.

I could be wrong but I feel that available Vine items are based on the
customer's past purchase and review history. There's no guarantee that the
customer is an expert, but there's some proof that the reviewer is at least
familiar with the product category.

I'm sure that there are some bad Vine reviews and bad Vine reviewers. However
discrediting the entire program without any data just doesn't seem fair.

(In case it wasn't already obvious, I'm part of the Vine program)

~~~
troisx
Part of the problem with the Vine program is that there's no real need or want
that other reviewers have. If I'm looking for a backup battery because it's
something I need every day, and I get shipped one and it takes two weeks and
then it doesn't work consistently (something that happened to me), then I'm
going to have a review that says that. If I'm a Vine reviewer and a backup
battery shows up and I use it a few times and it works fine, I'll probably
leave a positive review and then set the item aside. The lack of need or want
completely taints the review process

The other issue is that psychology studies show that being given something
ingratiates a person to the giver, which also greatly taints the Vine program.

~~~
chaostheory
I think that problem is mitigated by Amazon's algorithm matching you with
things that are more likely for you to either really like or use on a regular
basis, which leads to updated reviews. I've had some 5 star Vine reviews turn
into 1 or 3 star ones after 2-3 months... and even some reverting back to 4-5
stars after a year or so.

> The other issue is that psychology studies show that being given something
> ingratiates a person to the giver, which also greatly taints the Vine
> program.

I agree with this. Another problem which the author mentioned is the lack of a
review standard. One person's 5 star rating is not equivalent to another
person's.

------
sixdimensional
Maybe if Amazon simply posted a score/rating of how many people bought vs.
returned the particular product and frequency over time, that information
would alone be helpful. However, not sure they would ever share such
competitively valuable information directly. They do sort of share it
indirectly through the best sellers lists on the Amazon site, however, they do
not share details of "how" they arrive at what "best-selling" means.

------
perseusprime11
This is a wide spread problem across the board not just web. Look at some of
the reviews of the apps on iOS. Very detailed and makes a pitch to future
users to buy this app. I have almost always never left a review unless it is a
great app or if I got really mad at the app. Never felt the need to leave
reviews if the app did the job it's supposed to do.

------
mrfusion
My idea is instead of having review why not show the percent of orders that
were returned. That would give you sense of the quality.

Another idea is to randomly select customers who actually ordered the product
to leave a review. Since it's random you wouldn't have the self selection
problem of people getting paid to leave reviews.

------
B1FF_PSUVM
The sad part about all this is that the "social networks" will drive their bus
through this hole.

It will be "Get recommendations from people you trust!!! (sortof)".

And, in short order, we'll be back to the same situation, only more tied-up,
gagged and corrupted. ("Hey, pssst, make money fooling you friends!")

~~~
elihu
Actually, I think that's probably the only workable solution to the whole
fake-review problem. The only reviews you should trust are the ones written
either by people you already know and trust, or people who are trusted by
people you trust, and so on.

Currently, it's really easy to bribe a random person to write an anonymous
review. If the maker of some product has to bribe a friend of mine in order to
get me to buy that product, it's going to be a lot more work for a lot less
gain. Eventually it starts to look a lot like a company bribing all of its
customers at once to buy its products, which if they can pull that off and
make money, more power to them I guess.

The social media companies have the data and infrastructure to implement some
sort of "product reviews weighted by the reviewer's proximity to you in the
friend graph" feature, but I don't see users trusting Facebook or LinkedIn to
be an impartial source of trustworthy reviews. (Also, a lot of Facebook
friends aren't necessarily people you'd use for product recommendations, but
that's a separate problem.)

------
codecamper
Oh pleeeeeeze bring Fakespot to App store reviews! I'm not going to whine any
more... but pleeeeeeeeze!

I've got a competitor that has made millions making fake reviews to boost up
his lousy little app. He's been at it for about 5 years!

------
stcredzero
As always, it comes down to economics. So long as it's economical for people
to run fake review companies online and for companies to buy sets of fake
reviews, then it is going to happen. So how do we do that?

~~~
dexterdog
It's not economical for people to buy them if they get deleted for fraud.
Amazon is in a position to use confirmed purchasers of a product as a filter.
They can also score account based on how many legitimate reviews they have.
Once they cross a threshold all of the reviews get nuked. If an account as
1000+ reviews and rarely purchases anything you just shadowban them.

~~~
EazyC
This is the most correct and obvious/very low cost way to do it that you
almost start to think Amazon is in on the conspiracy with these fake reviewers
because they have something to gain that we are not seeing.

~~~
dexterdog
That's the thing I don't get. Amazon wants to have reliable reviews, right? If
people have to go elsewhere for reliable reviews those better-run sites will
start affiliate programs with other sellers. That's a real risk for Amazon. I
don't really care where I buy dog food or computer accessories or whatever. I
live and breath AWS so they've got me there, but buying goods from them or
target.com or wherever else is ultimately all the same.

------
dredmorbius
TL;DR: Recommendations systems cannot be indifferent to truth.

This article highlights pretty much precisely the same problem that the
"social media promote (bogus) conspiracy theories" item yesterday did.
Slightly recycled comment.

Some related concepts:

1\. Donella Meadows, in Thinking in Systems, notes that an absolute
requirement of an effective and healthy system is accurate feedback and
information. Media which are indifferent to truth value, or which actively
promote distortion (see Robert Proctor's term, agnotology), will actively harm
the system.

2\. Celine's 2nd law, and inversion. In Robert Anton Wilson's Illuminatus!
trilogy, a character notes that "accurate information is possible only in a
non-punishing situation". Its inverse is also true: acurate information is
only possible when it is accuracy itself and ONLY accuracy which is rewarded.
Academic publishing, in which paper output and journal selection is a gateway
determinant of professional careers, would be an instance of this. Or the long
skew of The Learning Channel from NASA-PBS educational co-production to Honey
Boo-Boo broadcaster.

3\. Paperclip Maximizer. "Don't be evil" isn't good enough. You've got to
actively seek out good. Even a benign maximisation goal will, if not tempered
by requirements to provide net benefit, lead to catastrophic results.

4\. Mancur Olson's "The Logic of Collective Action" explains how and why small
(but motivated) fringe groups can achieve goals directly opposed to the
interests of far larger groups. This explains a great deal of market and
political dysfunction.

5\. A generalisation of Gresham's Law leads to the realisation that
understanding of complex truths is itself expensive. It's also (Dunning-
Kruger) beyond the capability of much of the population. This also has some
rather dismal implications, though as William Ophuls notes, political theory
based on the assumption that all the children can be above average ("the Lake
Wobegon effect") are doomed. You dance with the dunce what brung ya.

Amazon here will probably have to create _and fund_ their own review staff _if
only to vet and validate user-supplied reviews_. As one HN poster here notes,
"what is my incentive to provide reviews?" A _good_ review takes time,
experience, is likely to draw flack, and, frankly, often comes from someone
with better things to do with their time.

Crowdsourcing is useful where data are thin _and the crowd is likely to be
unbiased_. Where _specific expertise_ is required, Peter Noverg on programming
texts, the Google employee who's uncovered numerous fraudulent USB-C cables,
_one_ qualified reviewer trumps millions who know nothing. (But you've got to
ensure that the qualified reviewer stays both honest and engaged.)

More generally: the Amazon experience seems to be worsening. Trust is falling
to eBay-like levels, if not worse. Search and discovery are poor, and product
quality is all over the map. Books are one matter -- highly uniform product,
simple function. Expensive kit of various description another. Among my recent
purchases, an LED cabinet lighting system in which local, major chain, and
Amazon sources (brick-and-morter and online) failed. Ikea turned out to have a
well-thought-out product line, helpful (though ultimately not quite what I was
looking for) information online, and, most importantly, _an in-store display
in which I could assess, mix, match, and ultimately assemble the system I 'm
using and enjoying now._ Twice the price of what I'd initially specced out,
_but that system didn 't work in the least_.

[https://plus.google.com/104092656004159577193/posts/chnJ1Mzc...](https://plus.google.com/104092656004159577193/posts/chnJ1MzcdhX)

------
perseusprime11
I wish there is something similar on the app store to weed out fake reviews.

------
tracker1
Sounds like an opportunity for a fakespot browser extension...

------
themartorana
You can filter Amazon reviews by "Verified Purchase" which, unless someone
tells me differently, at least means the product was purchased. It's annoying
but those reviews feel a bit more honest.

~~~
hueylewis0
Did you read the article? It specifically mentioned that firms/fake reviewers
use Verified Purchases as a method of building authenticity and believability.
Specifically:

"The compensated-review process is simple: Businesses paid to create dummy
accounts purchase products from Amazon and write four- and five-star reviews.
Buying the product makes it tougher for Amazon to police the reviews, because
the reviews are in fact based on verified purchases."

------
dang
Please don't rewrite titles unless they are misleading or linkbait:
[https://news.ycombinator.com/newsguidelines.html](https://news.ycombinator.com/newsguidelines.html)

(Submitted title was "Amazon Reviews Are Gamed by Compensated Reviews by
Professional Review Companies".)

~~~
danso
I like the original title with the secondary clause, _" Let’s Talk About
Amazon Reviews: How We Spot the Fakes"_...I mean, when I saw the title "Let's
Talk About Amazon Reviews" coming from a domain that is _not_ amazon.com --
i.e. it's not a product/dev announcement from Amazon -- I knew it'd be about
review fakery, but maybe not everyone is so cynical.

~~~
dang
Ah yes, it's clearly better with the subtitle, so we've added that and taken
out the superfluous bit ("Let's talk about").

------
chatmasta
Amazon likes to make it look like they are "cleaning up" the fake reviews
industry, by suing companies out of existence. But in reality they're just
trying to drive sellers to use their own, official "paid review" product. [0]

Granted the Amazon program is more transparent (as the review is affixed with
a label that says "user was compensated for this review"), but it's
disingenuous at best for Amazon to act like they're taking the moral high
ground by shutting down these review sites.

[0] [https://www.amazon.com/gp/vine/help](https://www.amazon.com/gp/vine/help)

~~~
BinaryIdiot
> it's disingenuous at best for Amazon to act like they're taking the moral
> high ground by shutting down these review sites.

Has Amazon come out and said they're taking the moral high ground? I just
assumed they were suing them for breaking their terms of service. Many
companies do this if you break them frequently enough.

~~~
chatmasta
I think it's safe to assume that the many fluff pieces in the tech media are
the result of a PR campaign to paint Amazon as the good guy cleaning up those
bad people selling fake reviews ("the moral high ground"). But of course
there's no mention of the fact that they've turned a blind eye to them for
years, and they're only now shutting them down to eliminate competition for
their new paid review product.

