Ask HN: What's the State of the Art in Fake Amazon Reviews?

[bespoke_engnr]

I was just browsing Amazon's black Friday sale, and came across a product with high-quality fake (I think?) reviews. I almost fell for it:

https://www.amazon.com/Soundbar-UPGRADED-Surround-Theater-Bluetooth/product-reviews/B076DXH4WP/

There's a huge burst of product reviews from Nov. 21st, all from 'verified' purchases, all from accounts with a female first name and exactly 3 product reviews.

Is this the current state of the art in fake amazon reviews? How can they not be detecting this as anomalous? Is there a semi-reliable way of detecting this kind of thing or do I have to be insanely vigilant all the time?

Or is it just a coincidence, and I'm being too paranoid?

[chatmasta]

FakeSpot agrees with you: https://www.fakespot.com/product/soundbar-2018-upgraded-surr...

[bespoke_engnr → chatmasta]

I love this, thanks! I was actually just thinking of trying to design something similar and expose it as a browser plugin. Nice!

[dfraser992]

I did my dissertation on this very topic (coming soon to my website... at some point). I do not know what tactics or systems Amazon has that deal with fake reviews, or if they even really bother beyond the easy wins like tracking IPs. My dissertation was on textual analysis, which is the hard part - the other signals like IP and behavioral related ones (like # of reviews posted in one day) are more fruitful.

The heuristic you seem to be using is a logical one, but requires more data analysis than Amazon might be willing to put effort into. Looking at these reviews, they are... well, so many on the same day is suspicious. My first theory is that some fake review writing company got tasked with flooding Amazon, and so reviews got farmed out to writers. Or someone has invented a GAN that writes good reviews, or at least a good first draft. I'd have to analyze the data to have more of an opinion. But yeah, verified purchaser means little.

The real question is, how economical it is for Amazon to really care about fake reviews? Buyer beware etc and there are more than enough scams running on Amazon / EBay / etc that I'm sure they're just treading water all the time. They have sued review writing outfits, so they care somewhat, but only after the problem got written up in enough newspapers... It is a hard job, trying to analyze all the data coming into their systems each day. I'm not sure any company has really implemented a lot of the research I read about in my literature survey.

ReviewMeta is another site I'd trust:

https://reviewmeta.com/blog/faq/

[bespoke_engnr → dfraser992]

This is really interesting, thanks. What's your website? I'd love to see that dissertation when it's up. Fascinating stuff.

[dfraser992 → bespoke_engnr]

https://douglas-fraser.com/datadata/

this will be the blog (at some point). The overall idea of the dissertation was to see if combining different ways of processing the text of the reviews (classifiers using features from analysis of the grammar, vocabulary, etc) into a custom heterogeneous ensemble was better than using one classifier and the traditional ensemble creation methods (AdaBoost, bagging, etc). I figured creating a more holistic view of the text would be better; other studies have done this, but not to the extent I did. And I analyzed exactly why things did or did not work.

So it was just fundamentally a exercise in NLP; I did not use other signals like the # of reviews submitted in one day or other things like that. My gut says this general idea (a more holistic view) would apply to classifying other text, like fake news. But proving that is yet another project.

I still have a couple more angles (dependency and constituency parsing, framing) to add to the mix, so I'm not totally done. It will be a long series of blog articles. And I ended up having to deal with the problem of diversity vs. accuracy, so the dissertation went down a side road. My supervisor said it could be two potential papers for publication instead of one... At least I won't be bored for the next year.

Thanks for your interest! If you send me your email (dfraser@... is mine), I can send you the PDF, or pointers to other info about the research into fake reviews in general (e.g. using other signals like # of reviews/day); I'm not going to get the blog up soon - already dealing with a ML project for Network Rail here in the UK.

[eugenekolo]

I reported these (and similar): https://www.amazon.com/Hiearcool-Canceling-Bluetooth-Headpho... a couple weeks ago.

Got a support response telling me to look at verified reviews, and that basically they don't care and verified is considered legit.

Take a look at the reviews. Every single one is 1 paragraph, from "Firstname Lastname" names, within last 3months, 5/5 to every review they ever gave, etc.

I dug into this problem, and it seems that they really are verified reviews. Getting these kind of reviews is costly, the vendor really does suffer an item loss, but having such positive reviews is worth losing 1000 units or so. I won't get into details where to buy these fraud reviews, but essentially there's groups out there on social media websites and specialized websites that work similarly to those discount click ads for us to get an item sites, but instead, it's leave positive fake reviews for us and you get the items for free.

Three weeks now, still nothing done, and still scamming. Good work Amazon fraud dept.

[savorypiano]

Amazon used to arbitrate on products not-as-advertised, but I don't see a link for that anymore. Now you have to ship it back at your own cost.

Overall the Amazon edge over other retailers has been going down.

[codegladiator]

The review index agrees with you it seems https://thereviewindex.com/us/q/AZ-US_B076DXH4WP

[producernyc]

Maybe you are right. I'm sure there are ways for people to put fake reviews on Amazon. Thanks for the heads up!

[byoung2]

Does the verified purchase label remain even after the product is returned? I think it should, in the case that you bought it, honestly reviewed it and then returned it. But that could be a huge loophole for fake reviewers to exploit.

[chatmasta → byoung2]

Nice thought, that could definitely be what’s happening.

But as you say, it’s hard to detect since obviously if someone returns the item they might want to leave a bad review.

But if all returns leave a good review? Now that’s suspicious.

Maybe amazon should show an icon if the reviewer returned the item, and force them to explain why they returned it.

But again that’s hard if they write the review before returning the item.

[i_am_nomad → byoung2]

If the manufacturer buys their own product, then they can (illegally) resell that product, and the only cost to them is shipping and Amazon’s overhead.

[PeterisP → i_am_nomad]

What's illegal in this resale? The item is in the same condition as before shipping, it's unpacked, and the only issue is proper accounting/tax treatment of the purchase payment.

[billrobertson42 → byoung2]

Why bother returning it? The fake reviewer "agency" will just pass the costs on to the party that hired them.