What The Netflix Prize Tells Us About Innovation, Collaboration, Info Sharing And Game Theory
from the fascinating! dept
While there's lots of attention being paid to the fact that some team has won the Netflix Prize (it probably won't be announced who until September), there's an interesting side story that's worth noting -- which is how important collaboration was in breaking through. Plenty of studies have shown that innovation happens much faster when you have the free and open sharing of information (rather than having it locked up, say, by patents), as that mixture of different approaches and ideas allows for breakthroughs to come much faster (in fact, studies have shown that much of the success in Silicon Valley came from the free sharing of info across companies as people rapidly moved around).And, in fact, that's exactly what happened with the Netflix Prize. The first "team" to break the 10% finish line, BellKor, was actually a merger of a few separate teams, allowing them to combine different pieces of different approaches to actually leap ahead. So, rather than trying to hoard the idea for themselves to claim the entire prize themselves, they realized it was better to team up to make the real breakthrough.
But, then a second interesting thing happened. Since the rules allowed another 30 days for other teams to offer up solutions that beat the first one, a bunch of other teams realized that it was in their best interest to team up as well, in order to leap-frog the original team. So they created the aptly named Ensemble -- and, again, the merger of various teams and different approaches allowed them to jump forward. It's not clear who actually had the best solution (both teams claim they did), but it's nice to see yet another clear example of the value of collaboration in innovation, against the standard myth of the lone inventor having a "flash of genius." It's also interesting to see the game theory aspect of the "loser teams" recognizing that they had to team up in order to catch up with the leader in the space.
Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.
Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.
While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.
–The Techdirt Team
Filed Under: collaboration, game theory, innovation, netflix prize
Companies: netflix
Reader Comments
Subscribe: RSS
View by: Time | Thread
This would be true if we were talking of going from zero to 100 on something. But in this case, they are taking a very, very good system and trying to make it 10% better, in a very complex system.
So even if it is 10% better, the original 90% was done by a very, very small group of people having "AH-HA!" moments.
Reset, try again, another false conclusion.
[ link to this | view in chronology ]
Re:
To state my view succinctly: "Put up or shut up."
(Or if you're a Wikipedian: "citation needed.")
Do you have any *Real* data concerning the orginating algorithms and history thereof by which Netflix does it's suggestion system?
If so, please, by all means, support your conclusion with evidence.
[ link to this | view in chronology ]
Re:
We had a very good system in our squares before that, so all the people that thought of the wheel really didn't improve it very much.
[ link to this | view in chronology ]
Re: Re:
[ link to this | view in chronology ]
Re:
Ahaaa!
[ link to this | view in chronology ]
Re: Re:
The developer of Cinematch is Stan Lanning. I think he may be the original "ah-ha!" guy on this one.
Plus most of this is older news, the contest started in 2006 and BellKor was spotted in 2008.
http://www.wired.com/techbiz/media/magazine/16-03/mf_netflix
information on Cinematch from 2002:
http://www.wired.com/wired/archive/10.12/netflix.html?pg=1&topic=&topic_set=
[ link to this | view in chronology ]
Re: Re: Re:
You're applying an efficiency quotient to innovation. Coming up with the platform/process/invention (whatever you wanna call it) in the first place may have been the "hard part", but that doesn't mean that improving upon it was any easier.
[ link to this | view in chronology ]
Re: Re: Re: Re:
The last 10% is probably much harder work than the first 90, so you need more people to push it there. But to deny how the first 90% happened is sort of like denying that a band or artist (like Trent Reznor) has an advantage because of his position in the public's mind, built up over years in the traditional music business.
But hey, what do I know? I apparently need to (retake) econ 101. ;)
[ link to this | view in chronology ]
Re: Re: Re: Re: Re:
[ link to this | view in chronology ]
Re: Re: Re: Re: Re:
Which means you're arguing about a secondary issue....
[ link to this | view in chronology ]
Re: Re: Re: Re: Re:
Once again, we've actually shown numerous bits of research that have shown that the "a-ha" moment is mostly a myth. Not entirely, of course -- it has happened. But nearly all innovation comes out of a general advancement in knowledge, done in collaboration with groups.
It's the difference between recognizing that innovation is an ongoing process and thinking it's a single step.
[ link to this | view in chronology ]
Re: Re: Re:
You have obviously never heard of experience curves or the law of diminishing returns...
[ link to this | view in chronology ]
Re: Re: Re: Re:
[ link to this | view in chronology ]
Re: Re: Re: Re:
[ link to this | view in chronology ]
Re:
First of all, try to be aware of how the NetFlix contest actually worked. None of the participants ever had access to the code behind CineMatch. So each and every one of the participating teams had to reinvent the 90% you claim to be an 'A-ha' moment before they could proceed to do something better. Because of this, it's quite likely that the original NetFlix algorithm and both of the new potential winners have little to nothing in common.
There was no 'A-ha' moment to speak of, not in the sense you understand it anyway. Look up 'collaborative filtering' and papers related to this subject as well as 'neural networks' and 'machine learning'. There are many ideas and concepts that build upon each other, and the stared in the early 90s. When did you say the NetFlix team had the 'A-ha' moment and invented it all?
[ link to this | view in chronology ]
Re: Re:
I don't suggest for a minute that they have an "a-ha!" moment and invented the whole thing from nothing. Heck, someone had to invent math first, right?
But there is a time between "Netflix had nothing" and "netflix had something" where the idea was formed, honed, and put into place. It wasn't an infinite number of monkey moment, it wasn't a worldwide collaberation between hundreds of people, it was a guy or a small ground of guys sitting around working on an idea.
The real trick isn't in the methods used, it's in the idea and the general implementation of the idea. If you get 100 people in a room and don't tell them what to do, just to "invent something good" and you will get probably as many ideas as people - they certainly won't come up with one single idea. However, if you give them a focused goal, like "make this 10% better", it can work. My point is the initial spark doesn't come by rubbing a large group of people together. The original idea is as likely to have from two guys scribbling something on a coffee shop napkin or something quietly taking a bath and going "damn, we could do something like this" and bringing it into the next development meeting.
Truth is, if you can get large groups of people to work on something for free (or for the chance at a prize), and it is cheaper than trying to do it yourself, then it is good business. Netflix did 90% of the work in house over the last few years, honing and refining, and now they will pay out $1,000,000 for the last 10% of the deal. In house, it might have taken them 10 more years and 10s of millions to accomplish. But we wouldn't be here without the first 90%.
The "a-ha" moment in the case isn't coming up with a theory (collaborative filtering), but rather in realizing and understanding ways that it could be used to better provide movies to users.
[ link to this | view in chronology ]
besides, you have the false conclusion of 0 to 100% with your statement that "the original 90% was done by a very, very small group of people having "AH-HA!" moments." by the simple fact that you assume (falsely i might add) that the team of people working at netflix on their system was very small (it wasn't, believe me) or that it was ONLY their team that came up with the entirety of the idea (again, it wasn't, trust me, or do the research :))
but hey, pop another quarter in, and you get 4 more tries!
[ link to this | view in chronology ]
Re:
Netflix didn't start from Zero. Collaborative Filtering had been researched and published out of some other source. Xerox PARC, and a woman PhD at MIT, if I remember correctly.
I'm sure Netflix wasn't so stupid as to start from square one. They, like all inventors today, stood on the shoulders of many giants.
So, lemme see, they started based on prior work and academic research, probably used competitive analysis, too, looking at companies like Amazon and Tivo. They did a great job taking the concept further. They then realized they needed more collaborative brainpower to make it to the goal. I think the case against "flashes of genius" is pretty well made in this example.
[ link to this | view in chronology ]
Re: Re:
Netflix didn't start from Zero. Collaborative Filtering had been researched and published out of some other source. Xerox PARC, and a woman PhD at MIT, if I remember correctly.
You're thinking of Pattie Maes, and yes, Netflix is in many ways indebted to her and many others when it came to collaborative filtering (Maes, too, built on the work of earlier pioneers in the space). The idea that some lone engineer at Netflix had a flash of genius is a myth.
[ link to this | view in chronology ]
Cool, can they work on Watch Instantly now?
It stalls 2-3 times per episode.
Maybe it's Time Warner's fault, but I don't know.
[ link to this | view in chronology ]
Re: Cool, can they work on Watch Instantly now?
[ link to this | view in chronology ]
Patents
[ link to this | view in chronology ]
Re: Patents
Dude?!
[ link to this | view in chronology ]
Re: Re: Patents
[ link to this | view in chronology ]
Interesting but ...
Yes, it is a reminder that 'flash of genius' moments are relatively rare and that most innovative products are the result of collaboration and teamwork. However, often the teamwork happens inside singular companies...leading to great leaps. Sometimes those leaps are not enough, and companies need to buy other firms (think iPod and Kindle) or work out collaborative development agreements or licensing deals with other firms. Nothing about patents prevent this kind of collaboration. In fact, they often make it EASIER not harder because rights are clearly defined during the negotiation for collaboration.
This NetFlix prize is a great story, but it makes a case against ignoring the possibility of collaboration with perceived competitors. It doesn't make a convincing case against patents. Just look at Dean Kamen. No one was put more emphasis on creative competitions that encourage collaboration/coopetition, yet he is one of the biggest backers of the patent system.
[ link to this | view in chronology ]
Re: Interesting but ...
[ link to this | view in chronology ]
Re: Interesting but ...
Mark Blafkin spouted:
Patents don’t give you rights you didn’t have before. All they do is take away rights.
[ link to this | view in chronology ]
Re: Interesting but ...
And have you noticed the commercial success of his various projects? It's no surprise that his businesses have been mediocre when it comes to the commercial market. He's someone who believes in the "big bang" theories of innovation rather than recognizing the importance of it being an ongoing process. Those who recognize the ongoing process are the ones who succeed in the market.
[ link to this | view in chronology ]
Re: Re: Interesting but ...
[ link to this | view in chronology ]
Re: Re: Interesting but ...
First, I don't think the commercial success is relevant to the point I'm trying to make at all. Kamen's FIRST competitions for robotics are designed to encourage cooperation in the pursuit of prizes. Designing competitions that incentivize cooperation is actually something he spends a lot of time doing. He does believe in 'ongoing processes' of invention and cooperation to achieve such goals, despite what you assert here.
Second, you are simply wrong about his commercial success being "mediocre." While the Segway has not revolutionized cities the way some suggested it would, he has built a fortune by inventing, producing, and licensing dozens of medical technologies that have not only been runaway commercial successes but have also won him humanitarian and technology awards around the world. You are right that his attempt to essentially go it alone on producing and marketing the Segway was a mistake, but that seems less because of a lack of innovation than a lack of understanding of the market.
@Lawrence
Perhaps you misunderstood me. I know that some argue that patents "take away rights" of the populace at large. The patent system is a bargain that is designed incentivize the exploration of useful arts and sciences. Whether that bargain is currently the topic of serious debate.
However, I do not believe there is any question that patents confer special rights and privileges onto those that hold them.
[ link to this | view in chronology ]
It's possible (Though I have no idea) that the work could have been done from the ground up ... redesigned the whole thing instead of working with something that already optimized completely.
[ link to this | view in chronology ]
Oh well...
/sarcasm
[ link to this | view in chronology ]
Gasp!
[ link to this | view in chronology ]
Ok so no Ah-Ha
[ link to this | view in chronology ]
[ link to this | view in chronology ]
nice
[ link to this | view in chronology ]