Virtual Video Store Clerks Take On The Geeks For Netflix Prize

from the wisdom-of-the-movie-geeks dept

We've been fascinated with the Netflix Challenge for a while now. That's Netflix's offer of $1 million to whomever can improve on their system for recommending movies by 10%. While there were a lot of early success stories in making improvements, all of the attempts seemed to bog down, making much more gradual improvements, but not getting close enough to hit that 10% mark. Earlier this year, we wrote about the surprising run up the leaderboard by a (previously) anonymous individual who approached the problem from a very different perspective, that of a psychologist, rather than a coder, which apparently was quite helpful in getting good results through a very different method.

Now some other folks are trying something completely different, relying on more of a "crowdsourcing" system, combined with a gaming element. They've set up a virtual video store, called Video Store Clerk, and set it up as a game for movie buffs. The game players act as a video store clerk, and can see how particular users rated three movies, and are then asked to predict how they would rate a fourth movie, with points given to correct answers. The idea is that they'll be able to use these crowdsourced predictions to create an even better model than the purely algorithmic model being worked on by various teams.

This reminds me of the research work by Luis von Ahn to do things like tag images via the "ESP Game." von Ahn has had numerous successes in creating fun casual online games, where the output data is actually very useful for taking on some sort of problem that is quite difficult to do algorithmically (such as identifying what's really in an image). The real question, though, is if movie recommendations really work that way as well. Perhaps I need to be a bigger movie buff, but so far, I'm not particularly good at figuring out how others would rank a movie. And, unlike the ESP game, frankly, the Video Store Clerk game isn't all that fun as currently designed. After playing it once, I had no desire to try again. Still, I'm intrigued by the different approach, and wonder if a more advanced (and more fun) version might be much more effective.
Hide this

Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.

Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.

While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.

–The Techdirt Team

Filed Under: competition, games, netflix challenge, online games, video store, virtual store clerk
Companies: netflix


Reader Comments

Subscribe: RSS

View by: Time | Thread


  1. identicon
    Anonymous Coward, 13 Jun 2008 @ 6:54pm

    $1 million will buy a lot of hours of OLPC children's time. Just sayin'.

    link to this | view in thread ]

  2. icon
    PaulT (profile), 14 Jun 2008 @ 3:54am

    @AC: Dumb reasoning. Just sayin'

    Seriously though, that's like saying that the $100m it takes to make the new blockbuster would be better spent on tackling child poverty in the 3rd world. Yes it would, but the point is that it's not *spent*, it's invested in an attempt to make a profit.

    Netflix isn't simply giving away $1m. It's using that as an incentive to get people to improve their service for them, and therefore increase their profits.

    link to this | view in thread ]

  3. identicon
    Bill M, 14 Jun 2008 @ 4:25am

    @PaulT

    You missed his point. AC's saying to wire up all those OLPC-using kids to do the crowdsourced ratings/predictions. He's not making a point about solving poverty. It was actually pretty funny.

    And ironically, your post shows why it's so hard to predict what movie someone will like. People like different movies for different reasons, often quirky ones. What makes one bad movie a cult classic and another simply a forgotten throwaway? Some people "get" the point of a flick, and others don't. How about mood and setting? Ever watch a movie one ime and think it was great but the second time you saw it, you couldn't see what was so good about it?

    It's a very challenging problem to solve.

    link to this | view in thread ]

  4. identicon
    Crowdish, 14 Jun 2008 @ 6:56am

    If someone succeeds in this challenge, I am sure that the $1m will be better spent than if Netflix had brought in Accenture or KPMG and paid them a $1m to "consult" on building a better system.

    Kudos to Netflix management on thinking outside the box.

    link to this | view in thread ]

  5. identicon
    Sumbuddy Stoopid, 14 Jun 2008 @ 2:15pm

    Another amazing comment

    Netflix is cool, I love the service, and Blockbuster hasn't quit spamming me since I went with Netflix instead of them... The "Movies You'll Love" section is cool, I rate movies I've seen all the time, and often pick movies for my cue from there. Would be amazing if they make it better (with a game?) it's already pretty cool...

    link to this | view in thread ]

  6. identicon
    sarah, 14 Jun 2008 @ 6:17pm

    librarians...

    Funny, this is what librarians do most of their day- Read-alikes for movies and books. I do it everyday and if my results are on target or not... forget programmers coding.

    link to this | view in thread ]

  7. identicon
    Joe Smith, 15 Jun 2008 @ 10:32am

    Netflix and patents

    I have two nagging doubts about the Netflix prize:
    1. The leaders seem to all be achieving their results by tuning a combination of methods which may mean that there is going to be little in the way of generally useful algorithm or insight coming out of the process.
    2. There are a lot of patents already issued on collaborative filtering and there is a real chance that the winner will find himself being harrassed by a "patent licensing firm".

    link to this | view in thread ]

  8. identicon
    Gene Siskel, 15 Jun 2008 @ 12:50pm

    The game would be a lot more fun if you got points for a near miss, especially as movie ratings can be so subjective. 10 points for a correct selection, 5 for being one star off, and nothing for more than that. Then maybe they could get folks to play more than once.

    link to this | view in thread ]

  9. identicon
    Chuck Norris' Enemy (deceased), 16 Jun 2008 @ 6:38am

    Re: @PaulT

    I get the joke but in reality, how many kids in 3rd world countries are movie buffs?...have even seen a movie? Sorry for the depressing thought.

    link to this | view in thread ]

  10. identicon
    Bill M, 16 Jun 2008 @ 9:48am

    I don't know enough about the specs of the OLPC to say this affirmatively, but they could potentially be using them to watch movies and become film buffs.

    link to this | view in thread ]

  11. identicon
    Anonymous Coward, 16 Jun 2008 @ 9:10pm

    Some suggestions for the game maker:

    Don't end the game. Let people move up when they make enough points. That way you get more ratings. You can still look at the data to see if they're really bad predictors, and just throw the data away, but keep the game engaging.

    Let people skip a rating. If they're not familiar with any or most of the movies, you just get bad data by forcing people to pick a rating to get to the next one.

    link to this | view in thread ]

  12. identicon
    This Tech, 25 Jun 2008 @ 4:09pm

    Re: Netflix and patents

    Probably a good thing you can't actually Patent Code, only the software that compiles it..

    what if someone went and patented from being used in HTML without paying that person..?
    It's the same with algorithms, I don't pay anyone for writting 2+2 = 4
    And that could technically be the result of an algorithm like:
    dim a as int
    dim c as int
    c = a & a

    output=a & " + " & a & " = " & c;

    ^--Now i know thats overkill to make a point but you can tell, this can't be patented, it's not possible to restrict a programming language in that way.

    link to this | view in thread ]

  13. identicon
    Crowded, 11 Sep 2008 @ 2:19pm

    result?

    Has Netflix gotten close to the desired end result via this crowdsourcing experiment?

    link to this | view in thread ]


Follow Techdirt
Essential Reading
Techdirt Deals
Report this ad  |  Hide Techdirt ads
Techdirt Insider Discord

The latest chatter on the Techdirt Insider Discord channel...

Loading...
Recent Stories

This site, like most other sites on the web, uses cookies. For more information, see our privacy policy. Got it
Close

Email This

This feature is only available to registered users. Register or sign in to use it.