Zappos Uses Mechanical Turk To Correct Spelling/Grammar Errors In Reviews... And It Increases Sales

from the fascinating dept

Tue, May 3rd 2011 1:03pm — Mike Masnick

Boing Boing points us to a fantastic story about how Zappos uses Mechanical Turk to fix grammar and spelling mistakes in product reviews. You see, research shows that if a product has well-written reviews, that helps sales -- even when the reviews are negative. The more well-written the reviews, the more people trust them. But, of course, on the internet, you get all sorts of grammatical and spelling errors (I should know, I make both all the time). So, to deal with that, Zappos runs its reviews through Mechanical Turk and lets the "crowd" act as its editor. There's no specific data, but apparently Zappos claims that this resulted in "substantial" revenue improvement, for a cost of a few hundred thousand dollars (across 5 million reviews). The corrections never change the actual intent of the content. They just make sure it's in proper English. I wonder what would happen if we did that for all posts and comments here...

Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.

Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.

While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.

–The Techdirt Team

Filed Under: grammar, mechanical turk, reviews, spelling
Companies: zappos

40 Comments

If you liked this post, you may also be interested in...

Reader Comments

Subscribe: RSS

View by: Time | Thread

The eejit (profile), 3 May 2011 @ 12:57pm

Simple - you would call down Grammar Nazi, the English tweed-wearing, pipe-smoking superhero.
[ link to this | view in chronology ]
- Anonymous Coward, 3 May 2011 @ 1:24pm
  
  Re:
  Maybe we need copyright hero to save us from grammar Nazi.
  [ link to this | view in chronology ]
  - Matthew (profile), 3 May 2011 @ 2:03pm
    
    Re: Re:
    Copyright Hero� and Grammar Nazi� are both Registered Trademarks of Inaction Comics� and their use is not authorized in this discussion. Please cease and desist any invocation of these or any other intellectual properties of Inaction Comics� or we will be forced to seek legal remedy.
    [ link to this | view in chronology ]
    - The Groove Tiger (profile), 3 May 2011 @ 4:13pm
      
      Re: Re: Re:
      And we all know how much Inaction Comics� hates to be forced to do stuff...
      [ link to this | view in chronology ]
- Gwiz (profile), 3 May 2011 @ 2:02pm
  
  Re:
  Simple - you would call down Grammar Nazi, the English tweed-wearing, pipe-smoking superhero.
  
  NO ADJECTIVES FOR YOU! COME BACK ONE YEAR! NEXT!
  [ link to this | view in chronology ]
DH's Love Child (profile), 3 May 2011 @ 1:13pm

My guess
They just make sure it's in proper English. I wonder what would happen if we did that for all posts and comments here...

My presiction is that darryl's head would finnally (sic) explode after seeing the extensive rewrites of his allways (sic) thotful (sic) expositions. But I digress...
[ link to this | view in chronology ]
- DH's Love Child (profile), 3 May 2011 @ 1:14pm
  
  Re: My guess
  Shit.. apparently I need it too. my PREDICTION. *sigh*
  [ link to this | view in chronology ]
- Nastybutler77 (profile), 3 May 2011 @ 2:50pm
  
  Re: My guess
  My presiction is that darryl's head would finnally (sic) explode after seeing the extensive rewrites of his allways (sic) thotful (sic) expositions. But I digress...
  
  The first thing I thought of when reading the last sentence of Mike's post was, "Don't do that, then people might actually think Darryl has cognitive reasoning!"
  [ link to this | view in chronology ]
  - Nastybutler77 (profile), 3 May 2011 @ 2:56pm
    
    Re: Re: My guess
    Damn! Does Turk handle HTML formating gaffs?
    [ link to this | view in chronology ]
Scote, 3 May 2011 @ 1:14pm

Editing the reviews to be more trustworthy is fraud.
Grammar, punctuation and style are part of how we decide if a review is credible. For Zappos to artificially inflate the credibility of reviews by changing those qualities is a form of fraud IMO.
[ link to this | view in chronology ]
- Mike42 (profile), 3 May 2011 @ 2:32pm
  
  Re: Editing the reviews to be more trustworthy is fraud.
  U R A Dorque, IMHO. An a winey 1 at taht.
  [ link to this | view in chronology ]
A Dan (profile), 3 May 2011 @ 1:16pm

Effect on this site
I don't think it would make much of a difference. People here are generally well-spoken (well-written?) and the stories are quite readable.

I wonder if the sight would start to loose readers if are comments got worse.
[ link to this | view in chronology ]
- Anonymous Coward, 3 May 2011 @ 1:25pm
  
  Re: Effect on this site
  don't confuse lose and loose.
  [ link to this | view in chronology ]
  - Anonymous Coward, 3 May 2011 @ 1:41pm
    
    Re: Re: Effect on this site
    WHOOSH!!!
    [ link to this | view in chronology ]
  - Ron Rezendes (profile), 3 May 2011 @ 2:56pm
    
    Re: Re: Effect on this site
    Don't confuse humor with actual intended content.
    
    The last sentence was a test, and you failed miserably. Especially for only calling out one of the several errors.
    [ link to this | view in chronology ]
Anonymous Coward, 3 May 2011 @ 1:51pm

I always think it's a shill if it's too well written or too lengthy.
[ link to this | view in chronology ]
fogbugzd (profile), 3 May 2011 @ 2:04pm

I am an active member of the Creative Spelling and Punctuation Society.
[ link to this | view in chronology ]
Anonymous Coward, 3 May 2011 @ 2:04pm

Techdurt; why u no doo dis fur heer?
[ link to this | view in chronology ]
Anonymous Coward, 3 May 2011 @ 2:14pm

Better idea
Thou shouldith replacee everything withee olde thymee spellings methinks.
[ link to this | view in chronology ]
Anonymous Coward, 3 May 2011 @ 2:27pm

Your just going to get lame jokes from people who think there funny.
[ link to this | view in chronology ]
- Anonymous Coward, 3 May 2011 @ 2:27pm
  
  Re:
  *you're
  [ link to this | view in chronology ]
  - Greg G (profile), 3 May 2011 @ 2:52pm
    
    Re: Re:
    and *they're
    [ link to this | view in chronology ]
    - Nastybutler77 (profile), 3 May 2011 @ 2:58pm
      
      Re: Re: Re:
      Aw, man you beat me to it by being brief, and now I look like I don't refresh my page before I post.
      [ link to this | view in chronology ]
  - Nastybutler77 (profile), 3 May 2011 @ 2:54pm
    
    Re: Re:
    And it's "they're" as in a contraction of "they are."
    
    You're welcome. See it's easy to use contractions. Whoops there I go again.
    [ link to this | view in chronology ]
    - Anonymous Coward, 3 May 2011 @ 3:33pm
      
      Re: Re: Re:
      That's the joke.
      
      That's also what I get for trying a different angle.
      [ link to this | view in chronology ]
      - Nastybutler77 (profile), 3 May 2011 @ 3:44pm
        
        Re: Re: Re: Re:
        Ahhh. I see I wasn't clever enough to get the joke. I guess since that same joke had been used several times already in this thread I assumed you were being serious. Very good. Carry on.
        [ link to this | view in chronology ]
        
        Anonymous Coward, 3 May 2011 @ 3:54pm
        
        Re: Re: Re: Re: Re:
        I was being totally being meta. Yeah, that's a good explanation.
        [ link to this | view in chronology ]
Squirrel Brains (profile), 3 May 2011 @ 2:42pm

I am an Turker
I will admit that I am a Turker and I have seem a couple of HITs for Zappos. I don't need the money. I am just fascinated with the concept and am interested in seeing out people use it. What a better way than to join up? It is like small mini-games you play in your favorite RPG... well... tedious mini-games that aren't really that fun. Everyday I see some new and interesting ways people are trying to use the tool.

I have also noticed that if anything offers more than a $1.00 to do, it is probably a scam (yes, there are a lot of scams on there for the unsuspecting Turk).

The most fascinating one I have run across is one for a jewelry auction site. You're suppose to go to the site and bid up three actions. If you end up winning, the requester will compensate you up to $20.
[ link to this | view in chronology ]
Joe, 3 May 2011 @ 2:47pm

Well we need a control group. How about we put all the comments here through the English to 12 year old AOLer translator.

http://ssshotaru.homestead.com/files/aolertranslator.html

Oh wait, YouTube already does that...
[ link to this | view in chronology ]
Anonymous Coward, 3 May 2011 @ 3:06pm

There's no specific data, but apparently Zappos claims that this resulted in "substantial" revenue improvement, for a cost of a few hundred thousand dollars (across 5 million reviews).

Says a company too lazy to figure out how the border works. Heh.
[ link to this | view in chronology ]
- Anonymous Coward, 3 May 2011 @ 3:51pm
  
  Re:
  "There's no specific data,"
  
  And we can stop right there, because there is no way to quantify this report's causality AT ALL.
  [ link to this | view in chronology ]
Sambo, 3 May 2011 @ 3:07pm

I am sure that the RIAA and MPAA are working on a similar program.

It will work something like this:

The words 'share' & 'copy' will automatically become 'steal, stolen or theft'.

'Community' becomes 'thieving criminal bastards'

'Fair use' becomes 'child abuse/r'

All negative comments will be permanently changed to:

'The RIAA/MPAA are your family. File sharing hurts your family. You are putting corn farmers out of work and
with the reduction of revenue available for our hand-picked, talent-free (but good looking don't you think? Have you seen that little girl Justin Bieber? Whoa!) Pop superstars - the makers of Autotune are struggling to survive.

Resistance is futile'
[ link to this | view in chronology ]
sehlat (profile), 3 May 2011 @ 3:16pm

Please define "proper" English?
Aussie: G'day, mate!

Southern USA: Y'all listen up.

Upper-Class England: (phonetic) Of course the mawb wouldn't understahnd propeh English.

Cockney: Where 'ave all the aitches gone?

Tech Support: (sing-song) I speak proper english very good, sir.

The world teems with examples.
[ link to this | view in chronology ]
The eejit (profile), 3 May 2011 @ 3:48pm

"Proper" English is also known as Queen's English, or recieved pronunciation.
[ link to this | view in chronology ]
Anonymous Coward, 3 May 2011 @ 4:58pm

Wat are you're porobelem wit mi englishh?
[ link to this | view in chronology ]
- Anonymous Coward, 3 May 2011 @ 10:41pm
  
  Re:
  y u no use f7
  [ link to this | view in chronology ]
  - Anonymous Coward, 3 May 2011 @ 11:53pm
    
    Re: Re:
    wat ish f7?
    [ link to this | view in chronology ]
Dohn Joe, 4 May 2011 @ 8:48pm

I go now...
I wud nevir powst on yur blawg evur aggen!!
[ link to this | view in chronology ]
Androgynous Cowherd, 8 May 2011 @ 10:25pm

Mental note to self: in future, introduce intentional misspellings and poor grammar into negative reviews of products, else those reviews will have opposite of intended effect.
[ link to this | view in chronology ]
geoffandersontop, 25 Jun 2011 @ 4:29am

Mechanical Turk is okay but all the jobs are way under minimum wages. I make cash in Varolo by letting other people do the work for me. Takes a few weeks before I got critical mass but now it grow by itself.
[ link to this | view in chronology ]