
20% of Yelp reviews are fake - bobf
http://www.marketwatch.com/story/20-of-yelp-reviews-are-fake-2013-09-24
======
drakaal
20% LOL! If only it were that low. I used to be a reputation management
company, at one time we were responsible for 10% of the reviews in Silicon
Valley. There were metro's where we were about 80% of the reviews.

When you think that the majority of companies have less than 10 reviews, and
most owners can't resist doing a review, and having a friend do a review you
are at 20% really fast.

When you combine that with organizations like Blackwaterops.com (my old
company) that used to do 1000s of reviews a day. 20% is really low, and wrong.

~~~
fotbr
Just out of curiosity, do those "reputation management" companies realize that
in serving their clients by making fake reviews, they help destroy end-
consumer trust?

In many ways, it's the old SEO argument all over again - gaming a system to
increase ratings eventually erodes trust in that system, making those
increased ratings less valuable. At the same time, refusing to game the system
means your business loses potential eyeballs/customers/etc, so one is left
with the old rock vs hard place situation.

~~~
drakaal
I don't do reputation management anymore. Now I build tools to do real
reputation analysis, and fight the very things I used to do. I'm like Darth
Vader in the last few moments of his life... Only I plan to live a long while.

At Stremor.com we have technology that "breaks" SEO so we have better search
results at [http://www.samuru.com](http://www.samuru.com) than you would have
using link authority. We have tech that can detect writing styles to tell that
6 reviews all came from the same person. We can do this for things like YELP
or Amazon product reviews. We also do this to detect web content that was a
Paid Review.

~~~
flanbiscuit
so can I pass Samuru a link to a particular Amazon product and it'll tell me
how many of the reviews have a similar writing style therefore possibly being
fake?

~~~
fotbr
That would be an interesting service to provide. More data points for
companies (x people were interested enough to check reviews with us) and end-
consumers get a better feeling for the reviews. I'm not sure how you'd turn a
profit with it, unless the data points for companies prove valuable enough. I
also don't know how you'd keep from turning such a service into yet another
system to be gamed.

------
gzervas
Hi everyone, co-author of the paper here. I wanted to clarify the 20% figure.
We actually have no way of telling how many, or which reviews are fake. We do
not directly observe review fraud, and we clearly spell this out in the paper.
The 20% figure represents the percentage of filtered reviews on Yelp. These
are reviews that Yelp finds suspicious enough to not publish. Some filtered
reviews may be fake, and some might just be false positives. Similarly, Yelp's
filter might miss some fake reviews, and end up publishing them. (See
[http://www.yelp.com/faq#filter_wrong](http://www.yelp.com/faq#filter_wrong)).
I hope this makes the distinction between fake and filtered clear.

Our main goal with this paper was to analyze the economic incentives behind
review fraud, and for this we used filtered reviews as a proxy for fake
reviews. bobf provides a good summary of our key findings so I won't repeat
them. For those interested, you can read more here:
[http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2293164](http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2293164)

~~~
sedev
Wow, that is _dramatically_ different than what the headline suggests. The
"20% are fake" headline implies "20% of _visible_ reviews", but instead I'm
hearing you say, "20% of reviews are killfiled before visibility."

~~~
unclebucknasty
That's exactly my read.

And, it's not just the headline. Consider the following:

> _A Yelp spokeswoman says its software helps filter many fakes before most
> users get to read them._

When combined with the headline, that really makes it sound like two discrete
groups: the fake 20% that make it to Yelp, vs. the "many" that are filtered
before making it to Yelp.

The difference is so vast that the headline now feels like link-bait.

~~~
lfender6445
yelp!

------
enoch_r
Argh, I am so sick of seeing this article. From the _study_ , 16% of all Yelp
posts are _identified by Yelp as fake_ , _hidden from view_ , and _not counted
into a business 's average score_.

This tells us _nothing_ about the percentage of non-removed Yelp reviews that
are fake, because all non-removed reviews are assumed, in the data, to be
real.

This is linkbait and a complete misrepresentation of the data.

~~~
gzervas
While I agree with you about the title of the article, it is not true at all
that "all non-removed reviews are assumed, in the data, to be real." In fact,
in our analysis we explicitly allow for the filter misclassifying reviews
(that is to say filtering real reviews, or publishing fakes.)

~~~
enoch_r
Sorry about my imprecise wording. I wasn't intending to criticize your study,
which did acknowledge potential misclassifications (and was very interesting
and deserves attention on its own merits).

I'm criticizing MW's interpretation, which assumed that your proxy was very
accurate in terms of providing the proportion of fake reviews, but went on to
imply that your proxy was highly _inaccurate_ by talking about fake but
visible reviews.

------
bobf
I'm sure there will be a lot of discussion here about the accuracy of the 20%
figure, but that actually doesn't seem as important as the other findings from
the research this article cites ("Fake It Till You Make It: Reputation,
Competition, and Yelp Review Fraud",
[http://people.hbs.edu/mluca/FakeItTillYouMakeIt.pdf](http://people.hbs.edu/mluca/FakeItTillYouMakeIt.pdf)).

The interesting points from the research include:

1) Low ratings increase incentives for positive review fraud, and high ratings
decrease them.

2) Having more reviews reduces incentives for positive review fraud.

3) Chain restaurants leave fewer positive fake reviews.

4) Businesses who have "claimed" their Yelp page have more interest in Yelp
and therefore are suspected as being more likely to commit review fraud.

5) Negative fake reviews are likely posted by competitors, and show little
correlation to changes in average ratings over time.

6) An increase in competition encourages negative review fraud.

~~~
jff
I would call all of those points extremely obvious and hardly worth a whole
study.

If you have low ratings, of course you're tempted to get some fake positive
reviews. If you have a lot of reviews, adding a few positive ones will not
affect the overall score much--it's a drop in the bucket. Chain restaurants,
well, everyone knows what McDonald's is, what would they gain from posting
fake reviews?

Businesses that claim their Yelp page have more interest in Yelp? Shouldn't it
be the other way around, businesses who have decided they're interested in
Yelp will claim their page? Claiming your Yelp page is a sign that you care;
businesses that do not care about Yelp will not commit review fraud.

Who else would post negative fake reviews but competitors? I guess reputation
management companies could post negative fake reviews in an attempt to drum up
business.

An increase in competition encourages more negative fake reviews? I mean, if a
business has no competitors, who has an interest in posting fake reviews? This
ties to #5, where competitors are the ones who post negative reviews. More
competitors, more fraud.

~~~
bo1024
It's nice that these points seem obvious to you, but until you have data and a
sound statistical analysis, people won't have any reason to believe them. Lots
of "obvious things" turn out to be true, but lots turn out to be false too.

------
roymurdock
Whenever my dad and I are looking for a place to eat I pull up Yelp. He
grumbles about the possibility of fake reviews, but I tell him that as long as
you look at the sample size (n=25 or more is usually safe) then you shouldn't
have a problem. I've found amazing restaurants (and many other services)
through Yelp that I wouldn't have found otherwise.

Sure, there have been times where I've eaten at a highly-rated restaurant and
thought to myself that there must have been some positive review fraud at
work. At the end of the day I have tried something new, and I still have my
whole life ahead of me to try other things. Nothing is forcing me to go back
to said restaurant. And I generally leave a review questioning the merits of
the establishment.

The positives of Yelp outweigh the negatives and the NYC regulation is a step
in the right direction.

~~~
jedmeyers
Here we go, a good example of an astroturfing comment about Yelp on HN.

~~~
makomk
Dunno about that, but it would certainly be part of the 20% of filtered
reviews if it was a review on Yelp.

------
bluedino
Does anyone really go by the star ratings on Yelp? I read the reviews but I
don't pay any attention to the overall ratings. One of my favorite places to
go is a 2-star, and our local Outback is rated a 5.

Plus with the fact that there's no standard, some people rate everything a 4
or 5 star while others reserve a 5 star for the best places in the area (or
even the genre) to eat.

------
rgcx
What I know is true: I've reviewed many places, yelp 'filters' my reviews as
fake even though they are legitimate. So 20% of the ones they don't filter are
fake, but several of the ones they do filter aren't fake? Sounds like the site
is just fake.

~~~
ScottWhigham
Same here for some of my positive reviews. My one negative review? Went live
and stayed live.

------
JacobJans
Yelp exists basically because it shows up in Google search results.

Google should be promoting sites that provide value to their users.

Considering the quality of Yelp, I really don't think they should be showing
up in the search results.

However, it would take a huge culture change at Google for Yelp to stop
showing up in search results, IMO. Even if the quality of their reviews
continues to degrade.

------
bapbap
I run a small review site and have to deal with issues surrounding fake
reviews all the time. Every now and then someone makes a rookie mistake and it
gets picked up by my system and automatically filtered but as the years have
gone on, I've found people are becoming better at gaming us.

I believe there is value in reviews, especially at a local level; I've found
out about some really great local businesses based on the reviews but I'm so
tired of people trying to game us. I'm tired of business owners who would
rather argue with me for months on end and threaten legal action to have a
totally legitimate negative review removed, or have their friends post glowing
reviews, rather than just providing a good service and recognising they can't
please everyone.

I wish there was a sure fire way of determining a persons intention when they
write a review but it's impossible, so this problem will remain unsolved.

~~~
agilebyte
Make it more _expensive_ by having video reviews? I suspect people are better
at detecting lies in people than they are in a block of text.

I guess this is what YouTube is for...

~~~
bapbap
Not a bad idea actually but even at that, there are plenty of video reviewers
getting paid to review things and one could be none the wiser.

~~~
kansface
I would never sit around to watch one.

------
sengstrom
I was offered a hotel discount if I posted a (good) review for the place. Just
another source of biased commentary.

~~~
jacquesm
You could review them and say they offered a good discount with everybody
saving face and you between the lines writing that the discount was the only
thing worth mentioning about the place ;)

------
bobf
Another problem I've seen with Yelp is reviews that aren't actually for the
intended place. My father-in-law owns a few restaurants. One of them has >300
reviews (solid 4 stars), but shares part of its' name with another restaurant
with >1500 reviews (3.5 stars, with a heavy downward trend).

Most reviews posted for his restaurant are either 4 or 5 stars, with the
majority of lower reviews referencing the other restaurant either by name,
menu items, or decor. I'm not sure if the issue is one of UI design, mobile
app, or poor search handling - although I heavily suspect search is the
culprit.

Unfortunately there doesn't seem to be a way to address these mis-posted
reviews, that I've seen. I would love to hear any suggestions that would help
though.

~~~
Aloisius
Change your name. Seriously. Even without Yelp, you don't want potential
customers hearing via word of mouth about how bad your restaurant was when it
really was the one down the street.

Names are relatively cheap to change and if you select a name that is related
(like say, shortening it or incorporating part of the old name in the new one)
and communicate what is going on with regulars, you'll limit any short-term
decline in business.

~~~
bobf
That's a perfectly reasonable suggestion, but I don't foresee it happening.
They've been there for >10 years, and have been very successful. The name
isn't actually similar, and they're in very different parts of the city. There
generally doesn't seem to be any real confusion except for a few Yelp reviews.

------
dvmmh
Actually, that's far far smaller than I expected. That means that 80% are
legit?

~~~
jared314
That depends on what they consider "fake". It looks like "Operation Clean
Turf" only considered reputation management companies.

------
alokv28
I've found that Foursquare has a higher signal-to-noise than Yelp for
restaurant reviews/tips.

~~~
jdminhbg
I find the same thing. I think the difference is that since 4sq is soliciting
tips instead of reviews, they're harder to convincingly game. If I'm looking
for a restaurant in an unfamiliar city and see "definitely try the squash
risotto with balsamic drizzled" that means a lot more to me than "everything
here was yummy!" Without a star rating to game with 1- or 5-star reviews,
there's less noise to sift through.

That said, I assume that their 0.0-10.0 ratings they introduced a few months
ago are basically random.

------
joe_the_user
_" There is one site where users share opinions with greater transparency than
Yelp or Google: Facebook. The world’s biggest social-networking site prohibits
the use of fake names."_

Uh, I don't really think Facebook's real name policy has any effect here (it's
policy is actually a lot weaker than Google in practice afaik). I do think
people share their real opinion on Facebook more often but I think that's
because of Facebook's huge audience. People basically want to share their real
opinions with their real friends (not necessarily their friends there under
real name). And people don't really have that many opinions about just plain
products. And when people do, they mostly feel kind of sleazy sharing those
opinions with the world for commercial purposes. Facebook is always trying to
get people to integrate their socializing with advertising/commercial-
recommendations but it has been unsuccessful and would probably start to truly
lose users if it really forced people's hands.

The problem is most people care so little about the products they buy that
they'd rather not write a review. That's unlikely to change.

------
tdees40
My father-in-law is a veterinarian, and he got a call one day from a Yelp
advertising sales person, who made it clear that if he didn't buy advertising
from them, negative reviews would start showing up. So I just treat Yelp as
legal extortion.

------
chmars
Unfortunately, not to use Yelp as a business is not an option according to
[https://biz.yelp.com/support/common_questions](https://biz.yelp.com/support/common_questions):

 _I 'm a business owner and don't want my business to be listed on Yelp — can
I have it removed from Yelp?_

 _Consumers have the right to talk about what they like (and don 't like)
about a meal they ate, a plumber they hired, or a car wash they visited. We
don't remove business listings, so your best bet is to engage with your fans
and critics alike, and hear what they have to say._

------
yesplorer
anyone who considers this news hasn't hang around freelancing sites. You can
pay $10 for 20 yelp reviews.

~~~
bobf
The real news is that NY state is prosecuting fake paid reviews as fraud, the
WSJ is reporting on it, and that HBS released a study with quantification of
it. Also, this is another sign that the "Wild West" days of the Internet have
ended.

~~~
yesplorer
"20% of Yelp reviews are fake" is what is being presented by the author as
'real news'.

If he wanted to go your way as the 'real news' he would have headlined it
sometime like: "NY state starts prosecuting fake Yelp reviews" or something
along those lines.

~~~
001sky
Two days ago

[http://dailycaller.com/2013/09/24/new-york-uses-fake-
yogurt-...](http://dailycaller.com/2013/09/24/new-york-uses-fake-yogurt-shop-
to-catch-fake-online-reviewers/)

------
dnm
I recently posted 2 reviews on Yelp (I'm not very active on it; this was
probably my 2nd and 3rd ever). I gave rave reviews for both. One is filtered,
one is not. I see a 50% failure rate in filtering, from my extremely small
sample size. Oh, and one of them was in NYC (the non-filtered one). The
filtered one, well the chef knows who I am, since I said the same thing to him
at dinner that I posted in the review.

~~~
sedev
Unfortunately that behavior pattern is exactly one that abusive users exhibit
as well: innocuous review, long delay, glowing review of "target" business.

------
bowlofpetunias
It doesn't matter. Over time, the value of all un-curated reviews approaches
zero. One of the oldest review platforms, IMDB, being the prime example of
this.

I don't often agree with Nicholas Carr, but in this particular case, the
wisdom of the crowds fails miserably, with or without astroturfing.

------
mrcactu5
> There is one site where users share opinions with greater > transparency
> than Yelp or Google: Facebook. The world’s > biggest social-networking site
> prohibits the use of fake > names.

Facebook isn't that transparent. You can't scrape Facebook.

~~~
notahacker
And I wouldn't be at all surprised to find out that 20% of Facebook profiles
are fake. About 80% of the friend requests I've received over the last year
have been from people I don't know; the majority of which are clearly bots.

------
dsed
Anyone want to speculate on what percentage of Angie's List reviews are fake?
On one hand you have to pay to leave a review. On the other hand, Angie's List
gets paid if a business registers twenty fake accounts to post fake reviews.

------
marojejian
Flagged - The headline is VERY misleading. Read the comments from the co-
author. 20% of reviews are voluntarily filtered by Yelp. Though the issue is
certainly real.

------
pa5tabear
I predict that more than 20% of these comments will think more than 20% of
reviews are fake.

------
ttty
Lol and what about Amazon?!

------
RedDeckWins
20% seems surprisingly low.

------
lfender6445
You do all realize this is major publicity for yelp, right? yelp!

------
jacquesm
Wow. Only 20%?

------
bsullivan01
If you want to write a good fake review, make sure you trash the place on
small things first.

small bar...lighting was too low...but great food and service

;-)

