

Still trust restaurant reviews? - akitchell
http://blog.urbantag.com/post/6359235485/still-trust-reviews
Some thoughts on why a new review ecosystem is needed, and about to develop
======
jdminhbg
I trust Yelp/Urbanspoon reviews the same way I trust Amazon reviews: By
reading them and looking for clues that the reviewer is real and has astes and
concerns similar to mine. I gloss over uncritical reviews, whether they're
real or not; a review that says "everything was great but the fries" I'll put
stock in.

It's possible that astroturfers could start taking that into account, but it's
hard to imagine. The kind of people who stuff Yelp reviews generally have
personality quirks that would prevent them from allowing any negative
feedback.

~~~
dcosson
I would guess this is what a lot of people do and it's certainly what keeps me
from giving up entirely on Yelp/Urbanspoon. But the downside is that ranking
results by average review becomes useless; you have to dig pretty deep to find
the place with an average of 1.5 stars that has a couple of thoughtful reviews
indicating it's somewhere you might like. (i.e. you have to get your reviews
in linear rather than constant time :)

Companies like Hunch are addressing the problem from the social-networking
standpoint that the author suggests (their Local app adds a Facebook friend
layer on top of scraped Yelp reviews), but it seems most people are not yet
dissatisfied enough with Yelp to consider getting themselves invested in yet
another social networking service.

------
sedev
Part of the point of the review model of Yelp (and Amazon, for that matter) is
that you don't have to trust _a_ review. You only have to trust the consensus
of all of the reviewers - which, at best, is very, very hard to spoof. When I
see a place on yelp with a 4-star average on 1000 reviews, I feel very
confident about going there. It's hard to get to that critical mass, but far
from impossible - and you don't need someone's "real name." This is actually
somewhere where Yelp is doing well, too - they give people a _persistent_
identity. You should trust reviews from someone who's written 100 reviews more
than a review from someone who signed up just to review a single place - and
Yelp makes that judgment easy by putting the information right in front of
you. Blizzard and other strategies have demonstrated that real names aren't
necessary for a good community - just persistent names.

Also, I have to laugh that the UrbanTag folks approvingly cite the complaint
that existing reviews sites are "giving customers “the upper hand” in
determining an establishment’s reputation." Well, yes, they are. That's the
point. That power has always rested in customers' hands, because that's what
reputation is: what _other people_ think of you. You can work on your
reputation and "manage" it and "optimize" it, but if it's not what _other
people think,_ it's not reputation. Business owners neither need nor deserve
control over other people's opinions - look at those asshole doctors that were
linked here a while back, forcing customers to sign agreements that amounted
to "we can sue you for posting a negative review of us on Yelp."

As for unreasonable customers: that's nearly as overblown as the threat of
piracy, and if your business can't deal with unreasonable customers gracefully
(not the same as caving and giving them what they ask for), you deserve to go
out of business.

~~~
IgorPartola
Dealing with ungraceful customers is #1 thing I look for in a business. I am
not usually ungraceful. I always try to be polite and friendly with any
customer service rep, etc. However, I think it says a lot about a company.

For example, a budget hosting provider that I used for a while would go off on
any customer who posted a negative review of them in the comments section of a
blog I frequent (LowEndBox). They turned out to be a terrible provider and
even being a reasonable customer, I could not get anything from their customer
support.

This is the same principle that can be used at job interviews. Take the
interviewee out to lunch and see how they treat the wait staff. Quick way to
separate assholes from people that will gel well with your team.

~~~
lsc
I know this is how most people feel, and I do try to pretend to be a sane,
rational person who doesn't get angry when dealing with customers, and I think
that helps me a lot. Flying off the handle at a customer almost never makes
business sense.

But I don't think it's such a good overall heuristic for picking service
providers. Personally, I'd rather deal with an employee who was good
technically and bad socially than deal with someone who is the other way
around. (I mean, if I could get both at a price I could afford, that'd be
perfect. but that's not usually how my world works.)

If the food is good and the service is fast, sure, it'd be nice if the waiter
was also friendly, but if I had to sacrifice something; price, speed of
service, quality of food, or politeness of service, I'm going to pick the rude
fast waiter and the good cook every time.

------
Aloisius
Review sites are hard.

Yelp was wonderful when it first started because the people on it all loved
restaurants enough to actually want to sign up to some random site and review
them. The community was small and I was easily able to pick out people I
trusted and people I didn't.

As it grew, more of the general public started reviewing things and things
started to go downhill. No longer can I remember who has similar taste as I do
or who to trust. There are so many reviews that it has become rather useless.
Every business seems to be 3-4 stars from Michelin two star restaurants to the
taco cart on the corner.

Worse, ratings change over time. Restaurants I rated as 5 stars years ago no
longer qualify as 5 stars today, so even when I see someone I know, I have to
look at the date of their review and discount appropriately.

Compounding things, there are all sorts of reasons why people give negative
ratings from food delivery being late (which has nothing to do with your
experience going to a restaurant) to having bad service because the restaurant
opened the day before to not feeling like you got enough subjective value out
of it.

Now, if Yelp had a collaborative filtering engine ala Netflix or only showed
reviews from people who shared similar taste as I do, it might regain some of
its usefulness. Maybe if they did something ala Hacker News and required
constant up votes to keep the rating from falling over time? Or an OKCupid-
style personality matching matrix to tell if someone is just cheaper than I am
or demands a higher level of service.

------
parfe
I don't bother with 5 star or 1 star reviews. Too much emotion to be sensibly
reviewed.

Three stars provide a nice balance. Things like "food was good, but service
was slow." I find more valuable than a 2 paragraph review about how a waiter
spilled a glass of water and only comped one entrée or a five paragraph review
that can be summarized as "I love this place more than breathing."

~~~
mitcheme
I like to read 1-star reviews (for anything) just to get a rough idea for what
a 'worst case scenario' might look like for that particular thing. If the
1-star reviews are all about entitled patrons, that's pretty encouraging,
whereas a few reviews about rats or cockroaches would make me look elsewhere.
To use another example, I was looking at books on Amazon yesterday, and the
only one-star review for a particular jQuery book was from this guy who was
mad because the book didn't explain how to put quotation marks inside another
pair of quotation marks (it was NOT a Javascript book) or how to download
Firebug. Whereas another book's 1-star reviews might be about how the examples
don't work, core material was skipped over, etc. When I was looking for a
travel mug I could throw in my purse, I nixed any option that had a 1-star
review about leaks (including many which were advertised as leakproof), and
ended up getting the best travel mug I've ever had. I can shake it upside down
or toss it across the room and my coffee stays safely contained. I think the
bad reviews of it were about how hard it was to clean and how the lid came
apart too easily. (It's supposed to come apart, to make it easier to clean...)

~~~
afterburner
Which mug? Tossing a full cup of coffee across a room sounds fun.

~~~
mitcheme
For some reason I can't find it on Amazon right now, but it's this one:

[http://www.thermos.com/product_details.aspx?ProdID=282&C...](http://www.thermos.com/product_details.aspx?ProdID=282&CatCode=BEVG&q=)

------
nostromo
> To solve the anonymous review problem, its time to inject real identities
> into recommendations.

People are usually more honest when speaking anonymously -- not the other way
around. It may help with astroturfing and flaming temporarily, but competitors
and trolls will always find a way around real identity online.

~~~
akitchell
Interesting point - what if your reviews are shared between your friends? Our
findings are that people can be honest in a community where they know their
recommendations will mostly help their friends find great places.

~~~
jerf
Take the number of your friends. Subtract the "lurker" ratio, which most
studies show to be pretty high. Now subtract the ones who have never been to
the place you want to see the review of. Now subtract the ones who went there
but didn't review it (because do _you_ leave reviews of every single business
you use? if so, where do you find the time to do anything else?). The mode of
the result will be 0.

If your friends are homogeneous and outgoing, you might achieve the critical
mass necessary to have more than just one review on a handful of local
businesses.

On Yelp, I have many local places with 0 reviews _period_ , let alone by
people I know.

Broader comment: Tying identities to reviews won't do squat. If I tell you I'm
Bob Thurmond, put up a photo of an older white guy, and exercise even a
modicum of identity management, who's going to be able to tell otherwise?

~~~
ramchip
That, and often I need recommendations when I'm not in my usual place, like
when traveling. The odds that my friends would 1) use the service and 2)
review a place I'm interested in are very low.

If I want a recommendation from my coffee aficionado friend... I'll just ask
him.

------
kenjackson
Why doesn't collaborative filtering work for popular review sites? The Netflix
recommender is pretty solid. Obviously you aren't likely to have reviewed 200
restaurants in Yelp, but it still seems like it would be effective if you had
10-30.

But if I could go to Yelp and have it recommend restaurants based on my past
ratings (well I have none now), that would be nice. I'd even take it a step
further. I want recommendations of actual dishes.

------
hack_edu
I always feel dirty when I read a topical blog post and find out that its
actually just marketing material for the author's product.

------
vnorby
I'm solving the same problem at <http://tattle.com>.

Our pitch is "Local business reviews and recommendations from people you
trust."

For me, as a vegetarian (<http://tattle.com/#communities/vegetarian>), the
problem with Yelp has always been that I don't have any good way of finding
vegetarian-friendly restaurants. They are good for factual queries, but trust
is a delicate issue. That's mainly because when you run a search for
"vegetarian" on Yelp, you're getting back a list of places that have the word
in their name or description, or occasionally a review. But actually, places
are not vegetarian, people are vegetarian. So if you want to know what place
to go to, you need to know who goes there. That's how we're solving the trust
gap (in addition to other problems with reviews).

~~~
autarch
Have you looked at <http://vegguide.org> ?

~~~
vnorby
Just did, there's also happycow. None of these sites have very much content,
they clearly are very old, and tattle is not tailored specifically for
vegetarians and vegans - we're just good for those types of communities.

------
saturdaysaint
The signal-to-noise has worsened appreciably on Yelp to the point where I
rarely look beyond the star rating and the number of reviews, and even then I
take the star rating with a grain of salt. The folksy, storytelling style they
seem to have championed makes reading through even a sampling of the top
reviews unbearable. It's too bad - I think they could successfully "pivot"
(sorry) into everything from location to coupons relatively easily if their
community didn't suck.

That said, Yelp is already fairly well integrated with the social graph (via
Connect) and one of the best use-cases for Yelp is for finding destinations
away from your main locale and, presumably, your social graph. I don't think
this approach will add value.

------
ChuckMcM
I agree with most that reviews are generally prone to abuse. (Although frankly
some of the Amazon non-serious reviews are just priceless, like the Cat in the
Hat ones [1])

I am also surprised Facebook hasn't jumped in here, seems a really useful
product would be 'facebook reviews' which linked reviews to profiles. Then
'the set of reviews you trust' could fall right out of your friends list.

[1] [http://www.amazon.com/Cat-Hat-Dr-Seuss/product-
reviews/03948...](http://www.amazon.com/Cat-Hat-Dr-Seuss/product-
reviews/039480001X/ref=cm_cr_dp_all_summary?ie=UTF8&showViewpoints=1&sortBy=bySubmissionDateDescending)

------
Cherian_Abraham
As someone who had dabbled in this "social recommendations" space, finding
reviews from friends when you do not have a big enough social graph is
probably the biggest difficulty. Also, it may work in a place like SF, but
outside of those deeply networked places, it becomes difficult to find
restaurants with reviews on Yelp, forget finding places where your friends
have been to.

And remember that people are far more inclined to write reviews when they have
had a negative experience, and not a positive one.

A tough task. I wish you all the best.

~~~
akitchell
Thanks Cherian

Would love to talk further about your experiences.

Feel free to write me - ak@urbantag.com

Thanks! and yep - def a tough problem... ;)

------
gmichnikov
I also find Yelp pretty useless at this point, so I've been hoping to find a
site that allows me to easily see the places that my friends (or friends of
friends) have been and to see their reviews of those places. Are all reviews
on your site going to be public, or is there going to be an option to only
show them to your friends and their friends? I'm hoping it's the latter.

~~~
ajhoag
Thanks for your feedback. I think you'll like what we are up to! (I work at
urbantag) If you have an iPhone, contact us and we'll prioritize you in the
beta (reference this HN comment).

------
wccrawford
Trust? Nobody has ever 'trusted' reviews by other people. They've used them to
help make decisions, but 'trust'?

I use reviews to find the negatives of something, and then decide if that
negative bothers me. Filtering out people with an axe to grind can be an art
all in itself, but many people are pretty easy to spot.

~~~
smackfu
Yeah, there are two types of reviewers:

1) I review every place I go.

2) I want to punish/warn people about a bad experience.

The first type is way more useful, since there is no way to tell whether one
person's bad experience is typical or not.

~~~
corin_
Possibly much less common, but another type of reviewer which I fit into is
"Doesn't bother to review unless a place has been consistently awesome over
multiple visits, so much so that I feel they deserve my thanks".

------
thegorgon
why is this the biggest thing we're trying to solve? can't we use these
incredibly powerful algorithms to solve a real problem? there are so many
restaurant recommenders out or coming soon."You have to ask yourself, are we
working on the right things?"

------
jcampbell1
I really wish there was a professional review company that charged restaurants
for the reviews and then made the information public. Zagat's model of
charging for the ratings isn't helpful because the reviews are limited to
their ecosystem and don't show up in the places I read reviews (seamlessweb,
google, etc)

Yelp reviews are so astroturfed the ratings are useless.

~~~
derobert
Charging the reviewed for the review makes it very hard to write a negative
review, especially for subjective reviews like restaurants.

This will happen without any conscious effort:

1\. Restaurants will only pay for the review if they're fairly confident it'll
be good.

2\. When the reviewer comes, if the manager is able able to tell who it is,
he'll make sure the reviewer gets perfect service.

3\. Even if the reviewer's identity is secret, the restaurant will know that
the review is happening this week, and can (temporarily) improve quality for
that week only. The restaurant knows more or less when the review will be
happening because they paid for it (earliest possible date) and they know when
it'll be published (because they'll want to know before paying).

4\. When the review company grows beyond a small local area, the company will
have to hire more reviewers. Most likely reviewers will review establishments
close to them, that is each reviewer will have a territory he/she covers. Some
reviewers will be harsher than others, so restaurants in certain areas will be
ranked higher than those in others. Areas will higher rankings will grow
faster (because the folks paying like higher reviews).

5\. If reviewers double as salesmen, the ones who review more harshly will
make fewer sales, smaller commissions, and are thus less likely to stay with
the company (either from quitting or from being fired).

6\. When the company's clients (i.e., the restaurants) in city A notice they
overall get lower marks than those in nearby city B, they'll complain and
demand to be reviewed by the same person who reviewed city B.

In short, the incentives for this company would be towards high reviews, not
accurate reviews.

