- On average, only 1-3% of customers review products.
- Each review is worth a lot of money, often times multiples of the product itself, and especially if you're just starting out.
- Each category in Amazon has it's own Average rating, for example, electronics typically have lower ratings because more things can go wrong and there are more usability issues vs something like kitchenware, where less things fail outright.
- If you play in a category with a certain failure rate, it is absolutely essential that you do everything you can to mitigate bad reviews as enough of them will sink your business, even if you have a great product.
- It takes 8+ 5 star reviews to counteract a 1 star review if you want to maintain a 4.5 star average which is the bar for a good product. This is extremely hard to do without manipulation.
- People who complain about fake reviews are only seeing half the problem, the other half is that legit businesses who do it the fair way can't compete. How do you launch a great product on Amazon with 0 reviews? Hope that 500 people buy it to maybe get 5 reviews? Alternatively you spend thousands on product ads hoping that enough people buy... or just succumb to the dark side and pay for reviews which is WAY cheaper.
- If you hate Amazon reviews, do your part and start reviewing the good products on Amazon. It is worth more to the seller than you think!
To me, a five star review means that a product went above and beyond my expectations in some extraordinary way. I have bought products that fit that description, but not many.
Everything else, I'd give either 4 stars or a still-very-satisfactory 3 stars.
The problem is, I know these 4-and-3 star reviews actually hurt sellers, which isn't my intention at all. So I just don't leave feedback.
This is also why I don't rate Uber drivers.
It was here on HN a number of months ago that I learned that in the Uber world, 3 stars does not, in fact, mean "acceptable". That knowledge altered my view of all reviews, including Amazon's, in two ways:
1) Like you, it means that I'm no longer willing to give reviews/ratings. If there is no consensus on what the different numbers of stars mean, then I can't be at all comfortable that my rating will indicate to others what I intended to indicate.
2) It means that I no longer put any weight whatsoever on ratings I see from others, for the exact same reason: I can't know that what I think the rating means is at all what the rater intended it to mean.
That 3 and 4 star ratings hurt Amazon sellers underlines this problem.
Really, same thing goes for Amazon and Uber, etc. the etiquette is to start at a full rating by default and deduct based on what goes wrong. You can’t “earn” extra stars... if I order a box of batteries, and they arrive on time and the box is full, that’s a perfect transaction. 5/5, no problem. You can’t realistically expect extra batteries, or for the batteries to perfect above their rating. They’re batteries.
(However, I've never understood why so many eBay reviews will literally type "A++++++++++", we're not in school and that's just a weird thing to say.)
I binary-ize any score that isn't already a yes/no, because thats what every score turns into.
Interestingly though it wasn't quite a simple five stars/one star dichotomy: the original blog post has a dead image, but this one shows that while five stars was overwhelmingly the most common rating, one star and four stars were about equal in second place.
It's wrong god damnit! Five stars should mean five stars.
I recently purchased an alarm clock. When I put the clock in my bedroom, the light from the display was so bright it kept me awake. I went back to the item on Amazon and read all the reviews. One person mentioned the same issue yet still gave it four stars!
but boolean choices are really 3-tier systems where it's assumed that only the extremely statisfied or dissatisfied customer will vote (up or down), and the lukewarm/indifferent customer is assumed not to vote. however, that assumption very likely misrepresents the sentiments of the (majority) non-voters and thus the population as a whole.
you might address this by moving to a 3-tier system: (1) unsatisfied/bad, (2) acceptable/fine, (3) exceeded expectations/great to more accurately differentiate the non-voting/indifferent customers, but non-voters would have no incentive to suddenly voice their opinions and make the system more accurate.
you might be able to counteract that impulse by incentivizing customers to vote on every product/service delivery event (like earning points for future discounts) to lower response bias. you could also do a separate study to see how the voter/non-voter population differ, and adjust the boolean ratings accordingly.
in any case, rating systems are tricky.
(Reading the reviews is by no means a good system—it's time consuming, the reviews are poorly written, and I never know what to believe. But if I ignored everything I'd feel blind.)
And you know what? They were. I never really noticed a huge quality difference between things rated 3 stars and things rated 5. And, since I started ignoring the ratings, I've purchased things with 1 and 2 star ratings on Amazon (that were well-reviewed in other places). Those have generally been as good as the 4 and 5 star things as well -- and a couple of them were actually excellent, deserving of 5 stars rather than 2.
I thought this was what the Amazon Vine program was for -- a legitimate way for retailers to provide free products to reviewers in return for a review, and those reviews are clearly labeled as such.
Seems to work pretty well - I've bought some products that only had Vine reviews, and I've agreed with the reviewer.
Legitimate ones go for Vine that would have probably kinda earned it anyway.
So much so that if you leave a 1-star review, there's a good chance you'll eventually get an email asking you to remove the 1-star review in exchange for a refund, a giftcard worth the price of the product, etc.
It encourages bad behavior though; You can basically gamble on what sellers will send you that email and thus give you the product for free.
The day after I received it, the company sent me an email asking me to please please please rate my new product!
Fine! I gave it 3 stars and wrote, "It's an O-ring. It does exactly what I expected it to and nothing more."
The next day I get an email saying, "We see you gave our product a 3-star review. What can we do to improve our product? What didn't you like about our product?"
WTF? It's a goddamn O-ring! There's nothing to review beyond "it works" or "it doesn't." What the heck do you want me to say about it? No O-ring is ever going to be 5 stars. Sorry! That's just the nature of the product.
At this point, it's just harassment. Stop begging for my approval, and especially when I give you my opinion, please don't question me about why. I explained it in the review.
If there is no conceivable way a product could be improved, it's worth 5 stars.
It took some more when you received it and fitted in your blender.
If it did not do its job well enough then you will have to go through this cycle all over again.
I have gone through that hassle enough times to hate it, even though amazon tries to be as helpful as possible.
That there itself is worth 5 stars or atleast 4.
Beyond that not all manufacturers care about our satisfaction. If some one does and wants to improve, it should be appreciated
Wow. I've never given an Amazon product a 1 star review, but if I did, and I got such an email, I would absolutely update my review to warn everyone that this is happening, and that the real average review for the product is likely to be lower than is shown.
I don’t mean by gaming the system or doing something unethical, I mean in the classic business sense of creating a differentiator or in MBA speak thinking about Porter’s Five Forces.
And who will own my reviews? I'd rather put my reviews on a more open review system.
I may be able to trust a reviewer on product A if their opinions on B,C and D were like my own.
TBH, retail is supposed to becoming "experience" based so this might be a runner. If someone tells me Jeff's mobile number I will persuade him.
Let's say I make a Dad Bag, shoulder straps and large zips, plenty of space for nappies and milk bottles books and wipes.
I want that on blogs, instagram cool-dad accounts, so that's finding 2-300 accounts and giving them free samples, but before that I want 4.5+ stars for the darn thing on Amazon (FBA) - so how do I do that ?
Or is that the wrong way?
And I'm saying that as someone who considers 4chan a pretty welcoming (if loudmouthed and staunchly irrational) place.
I suspect that Amazon reviews are going to be even less useful now. Especially given things like this:
> Amazon does not provide many specifics about how a product’s overall star rating is calculated, other than stating that it is not a simple average but instead uses “machine-learned models” that take into account factors such as how recent the rating or review is and whether it was a verified purchase or not.
IMO this is pretty solvable by looking at an account's purchase history too, but I don't think it's just as simple as blocking non-verified-purchase reviews.
Currently, companies pay people to buy items. They can keep those items, they just need to leave a good review. There are intermediaries who handle lots of sellers, so people buy a mixed bag of random garbage in exchange for the occasional review.
Yes, you can probably test for statistical anomalies, but I'm willing to bet that's quickly countered too - just have people buy occasional legit items so their profile is "statistically normal".
As far as I can tell, Amazon tries to fight that by keeping their ML model secret so scammers don't learn too quickly, but essentially, they're currently finding out what the Internet learned about SEOs manipulating search results.
Step 1 is getting a list of new products that are receiving a higher than expected number of 5-star reviews
Step 2, add it to a list of similar products.
Step 3, find accounts that all happen to purchase the same products in the same order.
Step 4, ban accounts and sellers.
What is higher than the expected number of 5-star reviews?
Don't those numbers vary widely depending on product category, brand, and newness of the product?
How does that work for things that fit into more than 1 product category?
That's left as an exercise for the reader ;)
But you could simplify this to new products, which if ungamed, would probably follow a predictable curve of discovery, as opposed to a games "instant 5 star" rating.
As Machine Learning is just a fancy way of doing statistics, you will totally not need it for this.
Worse still is that even taking a stance to not take a stance because you know it cannot be done will bring backlash as there are many who demand you take their stance, even allowing easy distribution of self chosen block and filter lists are not enough. There are many who demand the appliance experience.
Did they think I was being paid to write a negative comment, is that such a thing? or perhaps they don’t like negative comments either. I don’t know.
The same applies to links and Google. Buying/renting backlinks costs money, and still pretty much everybody that wants to rank in competitive fields does it.
They can for example provide a coupon that will reduce the price.
I don't think it'd do much good. There are already large groups that subsidize verified purchases to get 5 star reviews. I read an article about it, confirmed some investigation on Facebook of on my own.
Basically someone runs a Facebook group where sellers advertise free product, promising Paypal reimbursement of the purchase price in exchange for reviews. There are at least (or were, I haven't checked recently) hundreds of Facebook groups across many languages with thousands of members each doing these activities. The reviewers are randos who like free stuff, and I don't think anyone could detect them if they only casually participate in the review scams.
If you return a bad product, try and get a replacement to see if it was just bad luck, but also please do write a detailed review.
If you get a good product and it barely has reviews please review it.
That may well be me. I don't think that "meets requirements" should merit a 5. When I do reviews, I view a bare "meets requirements" as 3 or maybe 4, depending on the type of product. I want to leave some headroom to be able to point out products that really do excel.
Another ambiguity is whether the rating is on an absolute scale, or normalized for value (it's not a perfect product, but it's super-cheap).
I'm not sure this would help much. I've been asked by vendors to leave a review and if it's 5 stars, I will get something in return. I think for some people this would be a "well, nobody is getting hurt, right?" decision and they'd just do it.
Allow me to generalize
making X easier causes the quality and usefulness to go down
For me personally, the most valuable bit of feedback are the negative (edit: 1 and 2-star written) reviews - they are pretty much the only review content I trust. I'm looking for patterns of issues that multiple reviewers raise about the product.
The positive reviews have so little value when anyone can post a review. So many shallow positive reviews from unverified 'buyers'.
4/5 star reviews are useless, but 2/1 star reviews reveal a great deal of useful information about a product.
A bunch of 1 star reviews of "UPS damaged my product" indicates a product that's as advertised and isn't astroturfed (much?).
No 1 star reviews, or 1 star reviews indicating misleading advertising tend to indicate astroturing.
1 star reviews indicating product failures, support failures, DOA etc... indicate to not purchase the product at all.
It's not a pure indicator, but I've found it to be more reliable than other review systems.
When I was referring to the star rating, I meant the overall average rating of the product (ie what Amazon is trying to promote now over star rating plus written reviews).
When I mentioned the negative reviews, I was referring to the 1 and 2-star written reviews. I use the negative written reviews exactly how you described - and it's what I find valuable.
the only two considerations for me are the number of reviews and the quality of the low star reviews for the same. the dates of reviews is very useful as well, if a product doesn't have many recent good reviews it can offset the number of reviews in my view
Source: I'm an Amazon seller.
It is also super annoying when there are reviews like "came in three days not two. 1 STAR!!!" that are not product related. This doesn't even get into the commingling of product which is the real problem.
1) Reviews by people who have not used the product. The tipoff for this is when the customer says something like "I just got this, it looks great! I can't wait to use it." If someone hasn't even used it, then they can't possibly give a useful review of any sort.
2) Reviews in product listings that contain multiple products, or products that have changed since the original listing. The tipoff for this is comments describing a product that is clearly not the one being listed, or is one of several different products in the same listing.
If there is an increase in reviews that are just stars without comment, it becomes impossible to root these out at all. I don't see how this would lead to more reliable reviews.
As you can imagine, dealing with low quality products with fake reviews is a challenge -- but it turns out it's not too hard to handle, even with my dataset which is far more limited than Amazon's. Without looking at any reviews or any metadata of reviews (author, count, chronology, etc), one could filter out "impostor" products with 95%+ accuracy.
Here's a neat trick: Next time you're unsure if a product has fake reviews, click on the brand of the product and see what else they sell. If you're looking at binoculars, and that same brand also sells dog food bowls, then maybe you should reconsider.
I've concluded that Amazon really doesn't care about fake reviews -- they will show users whatever listing has the maximum Expected Value (conversion rate * revenue), per your context (search term, category, or both). Even if a product has obvious fake reviews, if there are enough other people buying it it will float to the top, and Amazon is fine with that.
This is quite an important feature. Amazon already shows this information (sometimes at least) and yet they don't let you sort by it.
So, with sparse data, they have three choices:
1) Allow filter/sort by "unit price" and do not show the X% of listings that are missing this data -- many of which the user may actually be aware of and/or interested in.
2) Don't allow the option at all, and just rely on the fact your customer will do comparisons manually.
3) Try to derive the number of units from text cues in the product name, features, and description, then do #1.
It seems they chose #2. I'm going for #3.
Or would people shift to other online retailers?
Actually it's nice. If I truly want it, I'll actually go and get it, or spend the time to find a reputable vendor. The added friction ensures I don't buy random things that I thought I wanted but don't.
The only real advantage Amazon have is prime (if you pay for it) and simple returns, but you have to deal with a drone if anything goes wrong aside from that. Most other retailers have actual people that can make decisions answering queries
Amazon is fourth in my list of where I buy things from. First is a local physical store. If I can't find what I need there, then I seek out manufacturers websites, or websites of authorized retailers. If I can't find what I need there, then I check out eBay. If that doesn't work, then it's Amazon.
This changed over the last year or so, though -- before, I would go to Amazon before eBay.
That, or there will be 10 different products nested under the same identifier. Not just different colors, but different products. So I have no idea which one the person it praising or calling crap. Maybe instead of Avatars next to the reviewer's name, Amazon should just attach an expandable screencap of the product page at the time the "verified purchase" was made.
Well, at least they're thinking happy thoughts.
I think it's only a matter of time before these sites get coopted/bribed by vendors, in the same way a lot of adblockers have been.
Even so, you have to consider real reviews to be only partially-reliable because a lot of people use their products incorrectly and give excessively glowing (or negative) reviews while ignoring reality because they may feel socially-obligated to not say something negative. And then there are hundreds of millions of specialized SKUs without any reviews at all.
We have a strict policy to remain neutral in our analyses and we do not in any way, shape or form accept bribes or copt partnerships.
But don't you need to say that? What policies or strategies are you using to prevent this from happening beyond the shared experience of being burned and avoiding a deep sense of shame if one were to abuse their position?
We actually get consistent DDoS attacks which are coming from angry e-Commerce sellers that have dismal grades on Fakespot, and that is occurring ever more frequently.
It's always a bit of a schlep to have to remember to use another website to check review credibility.
I've checked dozens of their verified & official "testers", and their five-star-rate averaged somewhere north of 98% which makes their reviews just as useless as the cheap auto-translated fakes to me.
Our ML based algorithms will penalize any biased reviews and show case them to our users.
I wish there were two separate ratings.
We have also implemented one of the leading feedback-survey-customer happiness-system-things in one of our shops. The amount of negative ratings, because the shipping service provider fucked something up, is immense.
There is no way for us to really get those removed (integrity of the platform - good thing in theory) and commenthing something like "it wasn't our fault" doesn't really help either as the rating will influence the total rating one way or another.
Maybe that's cost of business, not sure.
I've bought products with a lot of 5-star ratings which were utter garbage.
Even if you only consider "Verified Purchase" reviews, every one of them requires careful vetting for authenticity.
There are companies providing quality fake reviews by native speakers as a service, where care is taken to not be overly optimistic and provide a few negatives to sound more believable.
Even negative reviews might be fake, often mentioning a competing product that is supposed to be better.
There are plenty of valid reviews out there, but the need to constantly question the authenticity makes them almost worthless to me.
I've mostly gone back to buying from medium sized local retailers. They might have a much smaller selection and are a bit more expensive, but make up for it with a decent quality control.
Recently, I was in the market for a paint sprayer and one reviewer wrote this crazy detailed review of the sprayer he purchased, what type of paint he used to paint his cabinets, why this paint worked vs. the other ones he tried, and included numerous pictures of the process and results. It was super helpful not only in making my selection, but ultimately completing my project.
I also have found a bunch of instances where people will post tips in the reviews about how to fix or modify a product to make it perform better (or sometimes unfortunately just perform to its advertised specs). In one case I was able to repair my own refrigerator with a part purchased on Amazon and the instructions found in a review. While I'd rather have purchased it locally, that would have meant a service call and spending half the cost of the unit to get it repaired vs. a hundred or so dollars for the new circuit board from Amazon.
- doesn't accept free products or endorse any particular brands, and buys products like an individual could
- tests products thoroughly, both short-term and longer-term, including what happens within the warranty period
- reviews big-ticket items frequently enough to stay current
- offers purchasing gotchas and decision tree for selecting a product
- does the math on Total Cost of Ownership, including calculators so someone can plug in their local values (utility costs, taxes, rebates) to make the best decision
- has concise maintenance/operating advice
- starts small and looks for good products within a category
- also seeks out lesser-known, independent manufacturers who stand behind their reputations and products that people might not find on their own
- teardown and grade for repairability like iFixit
- doesn't spew blog articles that are merely copies of press releases
- compare products to those to the past to see the trends in quality, repairability, features and relative cost
The problem is that going into a store, platform or IRL, without trustworthy information leads to more arbitrary decisions. And then when there is a more reliable system for deciding what to buy, it's not that hard to have a browser extension that tags items on various platforms with recommendation badges.
There are various sites that try to present some reviews in particular genres, but the depth and breadth is usually relatively shallow and incomplete. Then there are the zillions of "review" sites that don't actually even test the products and just put manufacturer specs into a grid, and you have no idea if they have undisclosed sponsors, kickbacks or other deals.
Granted, it would be a labor-intensive proposition, so I think the best model would be a worker-owned co-op.
Their website does have a lot of blogspam type content now, and whether you trust their "experts" is up to you. But for generic household goods, they're probably less biased than the average comparison site.
This is why I’m bullish on protocols like SSB and Iris.
I understand that Amazon allows non-verified buyers to leave a review, so they can have more product reviews but those non-verified reviews are the source of so much garbage.
Sure, let people post non-verified reviews, so you look like you have a lot of reviews but at least let me filter them out in my search.
Filter: 4+ stars from only verified reviews.
Hell, even 3+ stars from verified reviews would be helpful.
I know sellers can fake verified reviews by paying the consumer back for the product but that's a massively higher barrier for the manufacturer. The difference is a few cents per review VS a few dollars per review.
I've seen products with thousands of fake non-verified reviews. I'm pretty sure they couldn't afford to do the same with verified reviews.
I've bought a few products that are bad to the level of fraudulent, only to see the product relisted so that my review wouldn't be verified.
Off course this could be abused by leaving fake negative reviews for competitors.
Whack mole. Repeat.
If I leave a negative review for a product that later gets delisted, Amazon could flag the account on basis of "deleted product with poor reviews". It could then try and see if there are any substantially similar items listed a week on. If so, ban them.
Perhaps even easier, make verified negative reviews stick to the seller's account. If you delist a product with negative reviews, those just transfer over to the seller, so you see a 5-star product sold by a 1-star seller.
I heard similar gaming techniques are used by makers of diet supplements to build up 5-star reviews. They notify customers that they will send another free bottle of supplements of they leave a 5-star review. This means that the following amazon search results example showing showing overwhelming 5-star ratings are very likely fabricated and can't be trusted:
This is how you get "snake oil" products like noncompliant USB cables or defective Apple chargers into the "5-star" club that amazon buyers unwittingly trust.
Reminds me of SEO's early days with keyword stuffing.
Does anybody know how to get out of their blacklist ? I tried emailing but get no response.
You could go all the way with this: look at my reviews, and then only show me reviews made by people with similar tastes.
If you check my Amazon account, I don't even rate most of my purchases.
It's a rather low sensitivity metric but one that has some value.