Tuesday, February 24, 2009

Online user reviews - reliable?



Last month, an employee from Belkin used Amazon's Mechanical Turk (a site where people can go, register, and get paid to do little tasks that computers can’t do) to pay people to post fake good reviews of a badly rated Belkin product on Amazon, Newegg as well as Buy.com. No one knows whether this was an idea by Belkin or just the employee of Belkin that posted this job. Here's the response from Belkin's president, Mark Reynoso:

http://www.belkin.com/pressroom/letter.html

I was at an internet panel earlier last month and I talked to the founder of OpenRice.com, a food review site for Hong Kong and he told me that his site is more like Web 1.5 technology because they manually filter out reviews which they deem to be fake instead of letting all reviews come through (i.e. true web 2.0-UGC). I think Yelp.com also does its fair share of filtering and Prof. Kagan told me that they reward their prolific reviewers (by throwing happy hours and parties for them) and they actually know who their reviewers are in person. So those of you who are thinking of building a web 2.0 business, keep in mind that there may be User Generated Content that may be fake.

3 comments:

Adele22 said...

This post prompts me to give you a little inside scoop on the book publishing industry and Amazon reviews that are tweaked by reviewers with a bit of a sales agenda. Back before business school, I was in marketing and publicity for a book publisher. From their sales rank to their reader reviews, authors are obsessed with Amazon, and will often visit the site several times a day for at least the first month or two after publication. Beyond the sales dept, Amazon provides tangible metrics for the author to gauge the success of his/her book or the success of the media interviews and book reviews they receive (and their effects on sales).
As for reliability of User reviews, I can tell you one anecdote that will cause you to take reviews with a grain of salt. Trade publishers generally publish 4-6 big books a year. There is a lot riding on these books, as they support the rest of the business, and all of the books that, in fact, lose money for the company. Well, in 2006, we had a very high profile author, whose book had not done very well and had seen a lot of negative press because of a lawsuit from the subject's family. Anyway, there was some directive from management to urge our friends to write (and in some cases spoon feed) positive reviews on Amazon and B&N.com to try to combat this bad press. For the hell of it, I just went back to the site to see whether the reviews got better after we were done with our little guerilla marketing exercise, and I'm afraid they haven't! Anyway, just chiming in on this subject! (please tell me that this counts as one of my 6 posts!:))

Fas - Ite, Maledicti, in Ignem Aeternum said...

Since being in the US, I started to order a lot from Amazon and I certainly checked out the reviews of the products I was interested in - mainly electronics like a hard disk drive, a digital camera and the like. I assumed that the positive reviews are probably fake and kind of found out that the rest seems to be from guys stating that this is the greatest product of this category they ever had - probably also their first and only one - so, I focused on the negative ones.

However, I found out that even these are rather problematic. I think there are probably four types of negative reviews: A couple of people get DOA (Dead on arrival) products, well this is not positive but probably unavoidable and it is only a deal breaker when there are tons of people having this issue.

Others just seem to be too stupid to set up the stuff or to use it or have other stupid issues - the best one I read was a review with some follow-ups where it turned out that the “bad quality hard disk” did not work because the guy had it plugged into a power outlet that was too weak; come on, a hard disk is not a washing machine where this actually might happen if you are in an old building ;-)

Some just have false expectations: when you buy a snapshot camera you cannot expect a picture quality comparable to a SLR and the flashlight is no floodlight.

Last but not least, there is sometimes actually a review that is a real deal breaker as it points out a significant deficiency of a product like for example a MP3 player I wanted to buy and a guy posted that the device did not have a key lock and thus plays unintentionally in one’s pocket, drowning the batteries.

So, the question really is how much all these reviews are actually worth? I think they are of some value, but a lot of them are faked and the rest leaves a lot of interpretation for the reader. However, I still think they are a great marketing instrument.

Jeremy Kagan said...

Both of these comments are long and detailed and are the kind of comments I was referring to when I said they can count as a blog posting.

This one, however, would not. ;-)

(As would any saying something that boils down to, "I agree")