We Already Suffer From The Paradox Of Choice, Now We're Paralyzed By The Ubiquity Of User Reviews
from the analysis-paralysis dept
Though you may be finishing your holiday shopping online, away from the mobs at the malls, ubiquitous user reviews on most merchant websites make shopping online slightly less lonely. It's nice to know that you're not the first one to try out some product, but just how helpful are user reviews? Even the most cursory pondering on this subject leads to a whole slew of questions and concerns. Just who are these reviewers and can you trust their opinions? Are they influenced by unseen motives (like free meals)? Perhaps their criteria for judging something is completely different than what you would judge on? Before you know it, you've spent hours chasing user reviews down a rathole, and you've ended up where you started -- no closer to finishing your shopping. So, do user reviews actually make shopping online better, or are they just a waste of time? We are already plagued with the paradox of choice: where an increase in the number of choices available has been shown to actually decrease satisfaction (since by choosing 1 thing, we are not choosing 89 others, and that makes us sad). So, add to this a dizzying array of user opinions, and all of a sudden, even choosing a box of tissues ceases to be a mundane task. Do I want the normal "Puffs" (average rating, 4 stars) or do I want the "Puffs Plus Lotion" (average rating, 5 stars) for more money? Where in this process does the fact that I use my tissues to clean my monitor once in awhile come in to play (in which case the Lotioned tissue would definitely not be a 5 star product)? User reviews have been around for years now, yet not much has been done since they were first launched to make them more useful. Perhaps the true role of user reviews is to serve as a sort of a sanity check -- it's the health rating posted at the front of a restaurant. As long as nobody has seen any cockroaches running around the kitchen, it's safe to eat.
Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.
Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.
While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.
–The Techdirt Team
Filed Under: user reviews
Reader Comments
Subscribe: RSS
View by: Time | Thread
I find reviews usefull
[ link to this | view in chronology ]
Re: I find reviews usefull
[ link to this | view in chronology ]
PEBKAC
It's for this reason that I think user reviews are best taken in aggregate. If even a sizeable minority of people love the product, I think it's probably decent.
[ link to this | view in chronology ]
But user reviews and a pro reviews comes from different places.
A user review speaks to passion, either they liked it enough to sing its praises or hated it enough to complain to someone.
A pro review is more instructive, more thorough and (hopefully) less biased.
[ link to this | view in chronology ]
Reviews are fine as long as unmoderated.
For example I was thinking about getting RPG game called Witcher but after reading user review about the lame button mashing combo system I decided against getting it. Something the "professional" reviewers neglected to notice/mention.
[ link to this | view in chronology ]
[ link to this | view in chronology ]
[ link to this | view in chronology ]
A couple of thoughts
2. It might be interesting to have a review site that has a human editor that creates a criteria, and when a user reviews a product, he can give both weights and marks to each one along with the free text review. This way, not only does the site have more information, but users using it can enter their criteria weights, and narrow the reviews to like-minded individuals.
[ link to this | view in chronology ]
A couple of thoughts
2. It might be interesting to have a review site that has a human editor that creates a criteria, and when a user reviews a product, he can give both weights and marks to each one along with the free text review. This way, not only does the site have more information, but users using it can enter their criteria weights, and narrow the reviews to like-minded individuals.
[ link to this | view in chronology ]
User Reviews
Take, for example, this:
"I really enjoy this MP3 player and for the price, you can't beat it."
That's a review? What really stuns me are the "5 out of 5" rating systems that appeared all of a sudden. These ratings often don't reflect the actual review at all, making me wonder what in the world people are thinking.
I was looking for an HDTV online to get an idea of specs for my choice. The reviews I've read were absolutely worthless while the "stars" were given high marks.
One reviewer wrote about a Panasonic LCD tv, complaining how the picture wasn't clear, the sound was choppy, and ended up rating it with 4.5 stars.
My advice to people is simply ignore the user reviews and ensure the website offering the product gives detailed "specs" of the product.
Because in the long run, it's your money. Would you really be happy chunking down $1999 for a TV that someone rated a 4.5 but had problems with?
People are stupid and the internet shows this every day.
[ link to this | view in chronology ]
I like them
[ link to this | view in chronology ]
Past their prime
The "we got it first" blogger -- these characters post ultra fast reviews of new gadgets and gizmos months before they hit the stores. Unfortunately these reviews are seldom anything more than races to be first. I bought a Blackberry 8300 from AT&T and found the sides creaked unbelievably. Soon I was joined in my complaining by several other people on message forums. Yet the very prominent sites that posted picture after picture and youtube videos -- NEVER once mentioned this. Phooey.
The "amazon cheerleader" - these are people who respond to the newest fad which is authors offering spiffs to their loyal fans to buy (and presumably review) their books. You recognize these because the reviews are all written within a day or two - and all gush. Then there are no reviews for weeks.
The "new toy glazed review" - if I never read another review about a cell phone or smartphone that includes the words "beautiful screen" it will be too soon. Who doesn't think their shiny new toy is the best. Why not actually use the gizmo for a week and THEN write about it. It cracks me up when I read the gushing review in a forum and then in a month you read the person listed the item on eBay. This happens all the time.
The "helpful forum member" - reviews all sorts of items. Always a 5 star gushing review. Lots of praise from other members. But wait, is that reviewer paying for the items or do they use the forum as a way to build up an inventory of free stuff sent to them by vendors. I have no proof of this but boy am I suspicious.
So I've started to follow only a select few Blog sites where I regularly read. I'll also look for more than one confirming review as well as more detail than simply "it's great". I want to know the reviewer has used it in daily practice for more than a day or two.
[ link to this | view in chronology ]
Two minds...
I've had a delightful dinner in a restaurant that I later learned was panned by 5 of the 6 reviewers on a restaurant reviewing site.
OTOH, I sure wish I'd seen the reviews on the Palm LifeDrive(tm) before I wasted my money on one.
Before I make a major purchase, I do try my best to research it, but reviews are iffy. If all of the reviews are positive, I consider that suspect, especially if the product isn't actually publicly available yet. I hesitated to buy an iMac because of the over-the-top evangelism I saw (BTW, I did finally buy one, and I'm satisfied with it, so far. But after over 20 years in the PC world, I'm going through some adjustments. Maybe after I get used to it, I'll be one of those over-the-top evangelists myself. Or maybe not.).
If all the reviews of a particular item are negative, but no articulate description of specific problems are given, I also consider that suspect, because negative reviewers are usually more motivated to provide their opinions than are folks for whom a product or service simply worked.
Even Consumer Reports' reviews are sometimes unhelpful, because their criteria differ substantially from mine -- especially on electronics. I do carefully read the auto reviews when I'm in the market for another car, though. But even there, my requirements for an automobile appear to be somewhat out of the mainstream (e.g., economy is much more important to me than acceleration or comfort), and I've had good experiences with cars that they have panned.
Most of the dedicated 'review' sites that I have seen have turned out to be questionable. Not sure what to do about all that, but I'm open to suggestions.
--
Without the 2nd Amendment, the rest of the document is just wishful thinking.
[ link to this | view in chronology ]
Generally Worthless
You can just go by star ratings, because, like someone else said, people don't always rate the product consistent with the review. I've seen 1 star ratings where the only complaint in the review was that they thought the price was too high. Similarly, a 5 star rating where the only good thing was that it arrived in a timely manner.
I find it's much better to find a reviewer you can trust (we're usually talking big media here) and go with what they say. The trouble is that they can't possibly review everything.
[ link to this | view in chronology ]
Netflix Reviews
Netflix has done a decent job of making the reviews of movies more meaningful by indicating how "similar" the reviewer is to yourself, so when someone loves a film, yet are only 28% similar to you, and someone else hates the same film, but are 72% similar, it's probably more likely that you're going to agree with the negative review.
[ link to this | view in chronology ]
Reviews by verified buyers
On Buzzillions.com for example, the majority of the product reviews are left by 'verified buyers.' Every single vendor listed on Buzzillions is a client to Buzzillions, so that way Buzzillions can verify that the person leaving the review has actually purchased the product.
Another cool thing is that it will show both the pros AND cons about a product, instead of just the generic 1-5 star rating.
[ link to this | view in chronology ]
Epinions
But I still look to see if the same complaints are common between them. If not, then I figure the complainers are the type that will gripe about anything.
I'm leery of 5 star reviews because they are usually fluff. But I also read the reviews to see if the content matches the star value. If not, then I ignore that author.
Plus, the forms to fill out for a review pretty much require people to consider both pros and cons so hopefully it will slow them down from 5 star and 1 star posts.
[ link to this | view in chronology ]
Also, ordinary people are much more likely to post a review when they've had problems with something than they are when they really liked it, unless they've got a good amount of free time.
These phenomena go for both products and services. I work in a hotel and many weeks we get good overall satisfaction scores, but the only people who actually add comments are those who had a problem.
[ link to this | view in chronology ]
Re:
It should have read "I'm bugged by the sites that don't require a user..."
[ link to this | view in chronology ]
Reviews are useful if descriptive
A suggestion for a review system is to have ratings for reviews where the reviews are shown sorted by rating. So the best rated review gets read the most and if good enough keeps it's rating.
[ link to this | view in chronology ]