Will A Psychologist Beat The Computer Science Geeks To The Netflix Prize?
from the moving-on-up dept
One of the key things that we expected would be useful when we created the Techdirt Insight Community, was the fact that our customers would be getting very different perspectives from the members of the community, based on an extremely diverse set of backgrounds, knowledge and experience. The idea was that while it's definitely useful to get the insight of someone who's gone through something similar (such as how to best position a product in a new market), sometimes you could gain a lot from someone who took a totally different view on it. It appears that same thing is happening in other areas as well. Wired has a fascinating story about how the various computer science/math geeks have had trouble advancing beyond a certain point in the Netflix challenge (the $1 million prize for improving Netflix's customer suggestion system by 10%), out of nowhere a management consultant with a psychology degree has been zooming up the charts using an entirely different method than the other leaders. The guy believes that all of the others have gotten into a bit of "groupthink," all using the same basic method -- while he's trying to apply some of his psychology background to the question in a way that he believes will work better in the long term. He hasn't yet passed the top teams, but he's been getting close. No matter how well he ends up doing in the end, it highlights, once again, the value of sometimes having someone from a totally different field take a look at the problem you're trying to solve.
Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.
Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.
While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.
–The Techdirt Team
Filed Under: computer scientists, diverse perspectives, netflix prize, psychologists
Companies: netflix, techdirt
Reader Comments
Subscribe: RSS
View by: Time | Thread
language translation
His method would use the small library as a starting point, and then repeatedly sift through the large library for matching phrases, scoring each one. It would shift a word at a time and try again. In the end he had a result much faster that was a lot more accurate. IIRC, the big thing was that is was able to accurately translate things like idioms that were traditionally impossible for automated systems.
The point is he had no computer science background, and was able to open an entirely new direction that had been ignored by previous researches.
[ link to this | view in chronology ]
Totally Could.
[ link to this | view in chronology ]
Re: Totally Could.
[ link to this | view in chronology ]
Medicine?
Take AIDS research for example (for the sake of argument we'll say that there's one team in the world working on a cure). Would it be at all useful for them to, say, bring an engineer on board, explain the basics, and ask for any ideas from them?
Like I said, sounds stupid, but maybe the engineer would stumble on an answer, or set in motion a train of thought in one of the researchers that wouldn't have happened before.
Gotta be worth a try.
[ link to this | view in chronology ]
Re: Medicine?
Oddly enough, I've been reading about the research that went into trying to cure AIDS in the late 80s early 90s as part of my research into medical innovation -- and there actually was some of this. One of the firms in question decided to go in a very different direction after realizing that all of the competitors were taking a single path towards a cure...
But, yes, I think there's tremendous value in getting a fresh pair of eyes with a different background to take on some of these types of challenges.
[ link to this | view in chronology ]
[ link to this | view in chronology ]
[ link to this | view in chronology ]
Interdisciplinary efforts
Why? In the past large companies did scientific research, and some of this can be applied to computer science. Also, many similar challenges have been seen in alternative disciplines. Physics majors are comfortable with working in a theory-based environment too. So throw the idea on the wall, and see if it sticks. Much can be learned from Einstein.
[ link to this | view in chronology ]