
Fakespot – measure the legitimacy of reviews on popular sites - throwaway13337
https://www.fakespot.com/
======
antoncohen
This just isn't reliable. I tested in on the Amazon Echo 2nd Gen, it gave it a
D grade, 45.8% low quality reviews, half a star.

The two unreliable reviews I checked were two-star reviews, they were not high
quality reviews, but they weren't "fake". For example one complained that they
thought it was wireless, but it requires a power cord.

I checked the product's two star reviews (6% of reviews), the reviews I saw
were all legit. The were long reviews, mostly complaining about the sound
quality compared to the 1st gen. For reference, the top 5-star review also
says sound quality was poor, until after the third firmware update.

Amazon isn't paying for fake reviews of the Echo, and no one would pay someone
to purchase a $79 product (verified purchase) and give it a two-star review.

~~~
aero_sm1th
I use Fakespot religiously since 2016 and I noticed the same thing so I
emailed them asking them, why would Amazons' products have fake reviews? This
was their reply (from co-founder, Ming):

"We have noticed trends where fake review clubs/bots seed their reviewer
profiles by leaving reviews on various products, specifically Amazon best
selling products to create a profile that looks legitimate (it is done to fool
anti-fraud systems that Amazon utilizes). Our engine detects these seed
reviewer profiles and therefore penalizes them accordingly by examining their
profile and our AI algorithm (one of numerous) was trained to distinguish
these patterns."

I hope this is helpful. I love the website and I think it is not to the
benefit of Amazon, Yelp, or TripAdvisor to do anything about fake reviews as
it only helps their platforms generate more $.

------
telltruth
Most people don't realize this but lot of products I'm seeing these days are
priced almost 20% more on Amazon than many other places. For example, Nest was
on sell on Amazon about $100 more than in Costco during Christmas period.
Amazon has successfully trained people in to thinking that everything there is
lowest priced and you just need to click button to get the goods. Further
more, many of the items now are actually fake even when branded. In many
categories like cables, batteries, cameras - you are actually very likely to
buy either Chinese branded or fake branded items from Amazon then the real
deal. For example, virtually 1 out 4 product listing on Duracell are actually
fake Duracell. It will still say "Sold by Duracell" and available on Amazon
prime to make it completely legit but its fake supplied from Chinese seller.
The funniest thing is that during Black Friday, many of these product prices
went up while Amazon advertized sale on the site, truly cashing out the trust
they have built over the years.

Amazon is becoming jungle in truest sense where your money would be eaten away
if you are not careful. Bezo keeps talking about "customer obsession" but he
seems to complete ignore all of the above.

~~~
tmaly
I remember a few years back there was a post on HN about algorithmic price
changes of books on Amazon. I wonder if this was broadened to other products?

~~~
bradbatt
They do it on a bunch of their products - maybe all of them, I'm not sure.

You can see pricing history (as well as set yourself price alerts) at a couple
of different websites such as
[https://camelcamelcamel.com/](https://camelcamelcamel.com/)

------
forapurpose
I wonder how they address this problem, which affects all attempts to spot
fake reviews: How do they know how accurate their results are?[0] What are the
false negative and false positive rates? To measure your accuracy objectively,
you'd need an independent method to verify, with high reliability, the
legitimacy of the reviews. (You may think you can spot them manually, and
maybe you can spot some obvious ones, but false negatives are perilous: You
don't know if the reviewer is simply better at hiding their trick than you are
in discovering it - and they likely have far more expertise and experience
than you do. Consider: Could you write a fake review that would trick someone
like yourself?)

Fakespot's FAQ says the following. I think the use of machine learning is a
good example of the problem: The machine needs a good data set from which to
learn, which means they'd have to flag reviews in the training data as
legitimate or not - but they have no way to determine that. Basically, they
are training the machine to make the same judgments, of unknown, accuracy, as
the humans (in fairness, I'm making some presumptions about their use of
machine learning).

 _What criteria are used by Fakespot when analyzing reviews?

Fakespot utilizes numerous technologies to validate the authenticity of
reviews.

The primary criteria is the language utilized by the reviewer, the profile of
the reviewer,correlation with other reviewers data and machine learning
algorithm that focuses on improving itself by detecting fraudulent reviews.

The technologies include: profile clusters, sentiment analysis, cluster
correlation and artificial intelligence intertwined with these
functionalities._

[https://www.fakespot.com/faq](https://www.fakespot.com/faq)

[0] As Richard Feynman said, "The first principle is that you must not fool
yourself -- and you are the easiest person to fool."

~~~
dfraser992
Statistics.... No, absolute ground truth is impossible for this problem. There
has been some research that estimated the percentage of fake / spam reviews at
2 - 6% but that was from a few years ago. I'm sure the percentage has
increased greatly since then [great, another dissertation... My MSc
dissertation was on this topic]

Text-only based features are not fabulous, just useful (like 75% correct,
using a dataset where ground truth is known). So there is / can be linguistic
differences in the writing that indicate the writer is not 'truthful'.
Sentiment analysis however, by itself, is abysmal - and the behavior of
spammers has changed over the years so they are more knowledgeable how to
craft reviews.

But other signals, like relationships to other reviewers, IP addresses,
submission time of review... they have been shown to be more accurate - but
not in the 90+%. Establishing who is in a spamming group seems to be reliable
though, so once that is known, you can more confidently label their reviews as
spam [but not 100%, of course, to be fair and objective]

I guess Fakespot takes a stab at estimating correctness and hopes the false
negative rate is acceptably low. Yelp OTOH cranks things up so the false
positive rate is high....

~~~
forapurpose
Thanks; that kind of contribution is why I read HN.

> using a dataset where ground truth is known

How is this dataset created?

------
zionic
I really can't stand this website.

We had them say ~40% of our Amazon reviews were fake/unreliable... and I'm
pretty damn sure we'd know if we were faking our own reviews.

I don't know how their system works, but the false positive rate in my
experience is absurd.

~~~
nikanj
Their long-term monetization strategy probably involves you paying them to
”verify” your reviews.

------
chiefalchemist
Maybe I'm naive but isn't this something Amazon, for example, should be doing?
Why aren't they concerned about the integrity of their community?

~~~
dvt
> Why aren't they concerned about the integrity of their community?

Amazon has virtually no competition, ergo is not incentivized whatsoever to be
concerned with the integrity of their community. Further, Amazon will charge
_you_ for return shipping so, again, they couldn't care less if a positively
reviewed product actually sucks. Now that I think about it, the whole system
is pretty brilliant because they're double dipping: charging the customer for
returns but charging the vendor for warehouse space. It's the definition of a
win-win.

~~~
arbie
Return shipping is free if the product is defective or the website description
is inaccurate.

~~~
user5994461
Most people choose to bin and forget the $10 shit they got, instead of sending
back and asking for refund.

------
mehrdadn
I'm skeptical Fakespot works well. Consider this product:

>
> [https://www.amazon.com/gp/product/B0064EKNKI](https://www.amazon.com/gp/product/B0064EKNKI)

Where do they get the idea that 25% of the reviews are unreliable/deceptive? I
bought it and it works just fine, which is pretty consistent with 87% of the
reviews being 4+. Why should I think otherwise?

~~~
Zak
It's attempting to spot sponsored reviews, not bad products.

~~~
mehrdadn
To quote itself, it's spotting "unreliable" and "deceptive" reviews, not
merely sponsored ones. And even in the latter case: do those look like
sponsored reviews?

~~~
Zak
No, but I imagine sponsored reviews aren't supposed to look like sponsored
reviews.

The three it flagged as suspicious for the ethernet cable above were all very
short, like "Great product!" or "A++++". I'm not sure what to think of that. I
write reviews (of flashlights) as a hobby and wouldn't call something that
doesn't describe how the product was tested or include any technical or
performance analysis a review.

On the other hand, the average buyer doesn't usually have the knowledge to
write a useful technical review and just wants to report that the product met
their expectations.

~~~
mehrdadn
> No, but I imagine sponsored reviews aren't supposed to look like sponsored
> reviews.

I mean, yes, but that wasn't the point. If the sponsored reviews seem to blend
in with the normal reviews and and seem to be at least as accurate as normal
reviews then how in the world are they "fake"? Where is the tangible
distinction for me as a customer? (Why in the world would I avoid such
products?)

> I write reviews (of flashlights) as a hobby

Funny you mention that! Flashlights seem like just about the worst-marketed
products I've seen on Amazon, so they indeed definitely need great reviews. At
the risk of going off-topic, do you have any tips on figuring out what the
lumen output of a flashlight actually is (or even what LEDs they use)? I keep
seeing Cree LEDs being claimed to output 2-3x as many lumens as they're even
capable of.

~~~
Zak
Flashlights on Amazon are mostly horrible. Thrunite and their offshoot Wowtac
are the only brands I'd buy there, because they sell direct. It's usually
better to order from specialty dealers. It's often cheaper too, as they tend
to do outreach on forums, /r/flashlight, etc... and offer standing coupon
codes.

Visual identification is usually the best way to tell what LED a light really
uses. Here's a chart with photos of a bunch of common ones:
[http://budgetlightforum.com/node/26665](http://budgetlightforum.com/node/26665)

While the lights you find on Amazon making extreme claims about output are
almost certainly lying, LED manufacturer datasheets tend to be conservative
about how much power LEDs can handle. Typical mains-powered fixtures don't
have the sort of heatsinking and thermal paths good flashlights do, and LED
manufacturers base their limits on a service life of tens of thousands of
hours. You will not use your flashlight for 50,000 hours. Overdriven LEDs
aren't rare, especially in enthusiast-oriented lights.

Measuring lumens is hard because you need a way to collect all the light, or
at least average it, then put that average on a sensor. Commercial integrating
spheres for this purpose cost thousands of dollars. It's possible to
_estimate_ using a shoebox and a smartphone though, assuming you have a light
source of known output with which to calibrate it. I wrote an app for that:
[https://github.com/zakwilson/ceilingbounce/](https://github.com/zakwilson/ceilingbounce/)

~~~
mehrdadn
That's awesome info, thank you! :)

~~~
Zak
I should also mention that a lot of reviews of flashlights found on Amazon
praising things I would rate as absolute junk are probably entirely honest.

If your last flashlight used alkaline batteries and an ancient LED, or even an
incandescent, anything using an LED from this decade and a Li-ion battery is
going to look pretty impressive. That Li-ion cell might well be taken from a
laptop battery sent in for recycling, and the charger it comes with might burn
your house down, but it's still impressive to the average person.

On the other hand, I've seen a one-star review for not including a battery. It
never said, or even suggested that a battery might possibly be included. My
conclusion is that Amazon reviews for most kinds of products aren't useful for
much more than determining if a product has an unreasonably high failure rate.

~~~
mehrdadn
Yeah, I definitely did find these impressive for the same reasons you
mention... hopefully my house won't burn down with the charger I got (it
hasn't yet)!

~~~
Zak
Just make sure it's not this one: [http://lygte-
info.dk/review/Review%20Charger%20Bowei%20HC-10...](http://lygte-
info.dk/review/Review%20Charger%20Bowei%20HC-103W%20UK.html)

And ideally, see if you can find your charger tested here: [http://www.lygte-
info.dk/info/indexBatteriesAndChargers%20UK...](http://www.lygte-
info.dk/info/indexBatteriesAndChargers%20UK.html)

If it is the first one, break it so nobody else finds it and tries to use it,
throw it away and replace it. The Xtar MC1 is the cheapest safe charger
commonly sold in the US; they're about $5.

~~~
mehrdadn
Haha, thank you for the heads up :) my charger is not on that list
apparently... it's this one (which is no longer being sold; sorry for the low-
res image): [https://imgur.com/VpPNswX](https://imgur.com/VpPNswX)

But some of the batteries are on there with not-great reviews... I guess I
might have to replace them?

~~~
Zak
Yeah, those batteries are junk. Try the Sanyo NCR18650GA from illumn.com.

------
SyneRyder
It might be worth clicking the "run analysis again" button if it's offered to
you. On the headphones product I tried (and have purchased myself), it flagged
some genuine reviews as having been created by automated accounts. Seems it
thought the surname Akcicek was just random letters generated by a computer.
But after running the analysis on the page again, it changed its mind about
that review and decided it was now legitimate.

~~~
LV-426
> But after running the analysis on the page again, it changed its mind about
> that review and decided it was now legitimate.

This doesn't sound very reliable. If nothing had changed on the review, what's
the basis for it changing its mind?

~~~
bartl
It probably is a learning system... It got trained more. The review itself
doesn't have to change. The system merely adjusted its parameters due to more
exposure to new reviews.

~~~
LV-426
Possibly, yeah. I was also under the misapprehension the re-analyse button
came up (randomly) immediately after a review, but it seems it appears on old
stored reviews.

Anyone who uses this site _really_ needs to do a re-analysis, because on the
products I just tested I'd rate its prior ability to detect fakes (reviews and
reviewers) at F.

After re-analysis I'll upgrade its ability to C.

TBH, I don't understand the point. I trust my own ability to evaluate reviews
more than the unknown logic of a third party website, which may have its own
interests/motives influencing its reviews of reviews.

------
bluetwo
I wish this site worked well. Based on my tests it doesn't. This is a real
problem without yet a solution.

~~~
user5994461
Works well based on mine.

------
dfraser992
I did my MSc in Data Science dissertation on classification of fake reviews -
I never looked at Fakespot in depth, but my cursory gut feeling is that
ReviewMeta.com is better (their blog is helpful). The problem in general is
pervasive across all review based sites, like Yelp and TripAdvisor, but as the
top poster points out, Amazon benefits from being a hands-off middleman, so
they are not going to bother dealing with any problems until it becomes a PR
nightmare.

If anyone is interested, you can find my dissertation at: [https://douglas-
fraser.com/datadata/](https://douglas-fraser.com/datadata/)

Only text based features were used, nothing involving other types of signals
like reviewer name or spamming groups. I started out with 180 million Amazon
reviews from 1997 onwards... but that got too big of a project, so the dataset
used was a test set used in a lot of research on this topic.

I will be expanding on the research in my blog, examining other sets of text
based features I never got the time to use, and things like PCA as well.

------
amelius
Can't a malicious actor use this to check their own reviews before posting,
until they pass the test?

------
pavel_lishin
How do fake reviewers get the "verified purchase" tag on Amazon? This
particular analysis[1] explicitly lists "Alex Brown" as a "Unreliable
Reviewer", but the Amazon page[2] says his review is a verified purchase.

Did Novopal just buy it for him?

(Also, I wish I could plug my Amazon profile into Fakespot and see what it
thinks about me.)

[1]: [https://www.fakespot.com/product/novopal-baby-stroller-
organ...](https://www.fakespot.com/product/novopal-baby-stroller-organizer-
with-shoulder-strap-universal-fitting-anti-slip-safety-buckle-design)

[2]:
[https://smile.amazon.com/dp/B01N5J6LXT#customerReviews](https://smile.amazon.com/dp/B01N5J6LXT#customerReviews)

~~~
pglhn
I've heard that one way of doing this is for the seller to give the purchaser
a discount coupon for 100% off

~~~
bambax
No, discounted purchases don't get you the "Verified" badge. The way sellers
operate today is, they ask you to buy the product "for real" and then
reimburse you (and more) via Paypal. This is against Amazon's rules but lots
of sellers are doing it.

I rank around 6000th reviewer in FR and yet I constantly receive messages
offering me to review products using this method.

------
eps
Is there a way to report/flag a _product_ listing on Amazon? As opposed to
flagging individual reviews.

I ran into a software listing that had several hundred reviews, all 5 stars,
but in reality is some SEO-driven bug-infested piece of crap. I wish I could
take it down somehow, because of its borderline scam nature.

~~~
jpindar
There is a "Report incorrect product information" link on each product page,
and someone does read what you post there. I've reported a few products where
the name didn't match the picture or description, and they were fixed within a
few days. Whether they would care about a less obvious problem, I don't know.

------
mehrdadn
Does anyone know how well it works to just check to see if reviews have been
posted in a short/bursty duration? I feel like most fake ones seem to be ones
posted very recently or during a short time period, but I'm not sure if that
just means I'm missing the rest of them or not.

------
user5994461
Fake reviews are clustered per category. Take a look at cream to make you less
fat, magic thing to grow your hair back, or iphone accessories, you will find
that almost all the reviews are fake. Too much money to be made.

------
acd
Tried on the Amazon echo dot best seller which had 4.5 stars but adjusted
review score are trustworthy score 3. Thirty percent low quality reviews
according to Fakespot.

------
bambax
Doesn't seem to support Amazon European marketplaces.

Review Meta does, and works very well.

~~~
user5994461
It works on amazon UK. I think their magic algorithms only support English.

------
jamisteven
I use this site regularly its never let me down.

------
zzzcpan
Let's be honest, fake reviews is a fake problem. Reviews were never supposed
to work for customers' benefits. I mean how in the world shallow emotional
opinions of people with unmet expectations and wasted time or with fanatic
love for the brand would help anyone? It's just that when people start
ignoring reviews all those corporations don't make as much profits.

