I think the problem is that a lot of people misuse reviews. When I read Amazon or Yelp ratings, I rarely look at how many stars something has. People add or subtract stars for reasons that don't make sense and don't mean anything. I have seen 3-star reviews on Yelp with the text, "This is the best food I've ever had in my life, the service was amazing, and it was cheap." Why that is not 5 stars is beyond me, but it's the Internet... and people are weird.
When looking at reviews, I read the content of the review to see if what upset the reviewer would also upset me. (I was looking for dentists on Yelp the other day, and saw a review that was something along the lines of "My insurance wouldn't pay the claim, so they made me pay!! Bastards!!". Although that one star review brought down the overall rating of the business, it is a problem with the review, not with the business. The review was for his insurance company, not for the dentist.)
Similarly, I read glowing reviews and look for the parts I care about. If I'm reading a review of a digital camera, I really don't care if the reviewer thought it looked nice or that the box was pretty, I care how long it takes from the time I turn the power switch on to when I can take the first picture. Obviously the reviewer could lie about this, but I've never seen it happen.
So, you get out what you put in. If you just look at the stars, you could probably save yourself a lot of time by flipping a coin.
I do the same. But many people don't have the same level of sophistication or simply don't have the time to spend so they just look at the aggregate review and buy the top-rated product. That why I think Vine is potentially subversive.
I'd love to see Amazon reviews have a HN/Reddit/Digg-style up/down vote system where crap reviews automatically drop off and aren't factored into the overall score.
Not to mention, if I receive something for free my opinion of it is likely higher than if I paid for it.
Case in point: the Motorola RAZR that I inherited a while back. It's a solid enough phone, got the right features, and for free it's very damned good!
But charge me $100 for it and watch my review go the other way.
Even without the arguably malevolent notion of positive-reviews-for-more-swag, the mere fact of receiving something for free likely induces a more positive review.
Yeah, I was looking at ~$250 products that had tons of Vine reviews. If you're getting something worth $250 free to keep I gotta thing that's going to affect how you review it.
The point about Amazon Vine is you have to review X products to get more.
So if I've got four books already that I haven't gotten around to reading, and a new one pops up in the newsletter that genuinely sounds worth reading, I'll pop on over to one of the four I have and write a completely fictional review for it based roughly around what everyone else is saying and the book's description.
This enables me to get as much free stuff as possible without reading books that turn out to be nothing like their descriptions. And it's people like me doing this that contribute to the immense Vine bias.
I have read bad books on Vine and reviewed them, I've also read some really good ones, but half the time it's all complete fiction so I can get as much free stuff as possible.
I think the community will auto-correct and weed out the reviewers who are not sincere. For example, if a user buys something based on Amazon reviews (of a Vine member, say), and discovers that the review was insincere, he/she will most likely go to Amazon and give a negative rating to the member/particular review, which will probably feed back into the Vine members reputation, which will in turn be used to determine eligibility for the Vine program.
Agreed that it'll take a while for the system to reach the critical mass, it is, nonetheless, a system that will work very well after it reaches the tipping point.
Alternatively, users will notice sooner or later that Vine is less about trusted reviews than 'Actual customer testimonials!!1!' and ignore or even avoid such products. Now maybe it's just me and the mass of people are gullible morons after all, but these kind of initiatives have a poor history on the internet compared to tabloids and TV.
Or so I thought until I discovered the secret to making REAL money - and now I'm going to share it with YOU.
But this assumes that Amazon removes people from the Vine program if their reviews aren't voted helpful. I don't think that's a safe assumption and furthermore, it might even be the case that people are removed for a lot of negative reviews. After all, the companies are paying to have their products in Vine. If word gets around that Vine produces a lot of negative reviews then companies pull out of the program and Amazon loses a revenue stream.
He should write a script to harvest a subsection of Amazon.
Then he should parse the results and see how closely correlated the Vine-to-Bad-Review there is.
.
A couple of data points here or there are not very convincing, but if he sampled say 10,000 products he could say:
* How many have reviews from Vine?
* Of those Vine reviews, what is the distribution of votes?
This is the kind of thing that Amazon could, and should, do. If they become very transparent with Vine then I think this could be a good success. But at least to date Vine feels very opaque and that's not a good thing.
Didn't the have a problem with Microsoft Vine, the names can confuse people. There was an interesting case of Mike Rowe Soft, where Mike Rowe had to surrender the domain for it rhymed with the word Microsoft - http://en.wikipedia.org/wiki/Mike_Rowe_(student) .
I'm not sure if we're gonna see some Amazon Vine Vs Microsoft Vine name battle...
I know that amazon is concerned with their bi-modal review distributions (most people review a product because they love or hate it for specific reasons, rather than because they are interested in reviewing-as-such). That might be what amazon is trying to address.
It seems to me the solution would be not to rely on Amazon reviews. IMHO specialized "price watcher" sites with a large enough community do a better job in this respect.
When I used to work at PC magazine, (in London, ~1993-4) we never gave anything back unless it was a prototype or something. Just as well, considering the pittance we were paid :-)
When looking at reviews, I read the content of the review to see if what upset the reviewer would also upset me. (I was looking for dentists on Yelp the other day, and saw a review that was something along the lines of "My insurance wouldn't pay the claim, so they made me pay!! Bastards!!". Although that one star review brought down the overall rating of the business, it is a problem with the review, not with the business. The review was for his insurance company, not for the dentist.)
Similarly, I read glowing reviews and look for the parts I care about. If I'm reading a review of a digital camera, I really don't care if the reviewer thought it looked nice or that the box was pretty, I care how long it takes from the time I turn the power switch on to when I can take the first picture. Obviously the reviewer could lie about this, but I've never seen it happen.
So, you get out what you put in. If you just look at the stars, you could probably save yourself a lot of time by flipping a coin.