Lessons Learned From DARPA Balloon Challenge

from the it's-amazing-what-people-can-do dept

Wed, Dec 9th 2009 5:52pm — Mike Masnick

By now, you've probably heard that a team from MIT won the DARPA balloon challenge, whereby DARPA put 10 red balloons in the air around the country and wanted to see what people could do to find all 10 balloons. The rules were pretty loose, and the team at MIT took all of nine hours or so to locate all ten balloons, through an interesting "crowdsourcing" method. They basically quickly set themselves up as a clearinghouse of information, and made it easily shareable across different social networking platforms, and employed something of an affiliate program to encourage people to get their friends to sign up with the MIT team as well. If you signed up people who helped find the balloons, you got some of the prize money according to your friend network, and so on down through the social pyramid. The team claims that what was most important was the recursive nature of the pyramid, which gave people incentive to participate, even if they knew they couldn't find the balloons.

While some other DARPA challenges, like the autonomous vehicle challenge (to get a totally driverless vehicle to drive a few hundred miles with no help), are cool but seem limited in terms of application outside of the core area it was built for, this one actually does seem to hold a lot of useful lessons that can be picked up on right away, and which can be applied across a lot of different business, policy, IT, public good and many other areas. Some of the key elements:

Recognize that there's power in numbers: Recognize that for certain projects, you need a lot of different minds (and eyes) working on things, and that certain tasks shouldn't just be done by "the one best" individual.
Make it easy for more people to participate: Once you realize that you need a lot of people, you need to make it easy for them to participate.
Give people multiple reasons to participate: Different people have different motivations. Some people just want to belong to a successful project or a leading team to bask in the glow. Others need additional types of incentive. The MIT team offered up monetary compensation in addition to recognition for participation.
Give people a reason to get others involved: Sort of a corollary to recognizing the power in numbers, the MIT team worked hard to give people incentive not just to participate and to promote their participation, but also to recruit others to the team as well. This even made it so those who couldn't help finding the balloons directly could still participate in better finding the people who could find the balloons.
Align incentives properly: Make sure that everyone is driving towards the same goal, and that the incentives work on top of one another to all push towards that same goal.
Look beyond your immediate "group": One of the coolest things I thought about the MIT group was that there was nothing in there that limited it to MIT or the folks at MIT. They immediately recognized that it made the most sense to reach out to folks beyond their immediate circle, which is what helped them get the people they needed involved quickly.

Now, a lot of these may sound obvious, but it's often important to remind yourself of these basic concepts, and it's impressive to see how well (and how fast) these worked in the case of the MIT balloon team. I could see these lessons being applied in a lot of other areas as well. There is a separate issue that the team hasn't discussed yet, but promises to eventually: which is that it also had to deal with a number of bogus entries -- including at least some from a competing team trying to throw the MIT team off the scent. Finding out how they got around such problems would also be quite interesting in terms of better managing these sorts of group efforts.

Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.

Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.

While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.

–The Techdirt Team

Filed Under: balloons, challenge, crowdsourcing, darpa challenge, incentives, mit, teamwork

23 Comments

If you liked this post, you may also be interested in...

Reader Comments

Subscribe: RSS

View by: Time | Thread

Anonymous Coward, 9 Dec 2009 @ 6:15pm

a lot of these may sound obvious

Sound? How about "are"?
[ link to this | view in thread ]
Steve R. (profile), 9 Dec 2009 @ 6:20pm

An Experiment in Finding The Needle in a Haystack
Based on some other comments that I read at "Freedom to Tinker", this could have been a human behavior experiment for designing a method of finding something that is well hidden. Some of those commenting made references to the search for Bin Laden. I especially like the comment by Anyonmous concerning what may be an incorrect "reward" incentive on our part for finding Bin Laden that: "Chances are, had they offered a couple goats or a camel for good info they would have gotten a lot farther."
[ link to this | view in thread ]
Marcus Carab (profile), 9 Dec 2009 @ 6:25pm

Re: An Experiment in Finding The Needle in a Haystack
I doubt that the United States has spent this long hunting for Bin Laden without trying to "bribe the locals"
[ link to this | view in thread ]
Anonymous Coward, 9 Dec 2009 @ 6:26pm

Re: An Experiment in Finding The Needle in a Haystack
They offered stacks of cash that would make anyone in that area rich enough to buy everything their eye could see and more, and nothing happened.
[ link to this | view in thread ]
Anonymous Coward, 9 Dec 2009 @ 7:01pm

While some other DARPA challenges, like the autonomous vehicle challenge (to get a totally driverless vehicle to drive a few hundred miles with no help), are cool but seem limited in terms of application outside of the core area it was built for, this one actually does seem to hold a lot of useful lessons that can be picked up on right away

you must be an IT guy or very closed minded- definitely not an engineer
[ link to this | view in thread ]
Thierry Hubert, 9 Dec 2009 @ 7:02pm

MIT's excuse of a victory: DARPA's Red Balloons
MIT acted as a broker to the DARPA project by dividing the winnings according to a race-reward-sharing model for anyone who delivered the coordinates to MIT. They acted more like HBS than MIT on this one. In my opinion this test demonstrates that the Web and social networks are not ready to rival institutions who can basically shortcut and inject themselves as brokers. Additionally, very little was learned or demonstrated as MIT hijacked to potential innovations that could have to emerged from the challenge. MIT won by bribing instead of innovating. MIT should not exhibit confidence in the victory. MIT's victory did not teach DARPA was that wars can be won by trade and economics; the history of conflicts have thought us that very well. I am disappointed in MIT's approach and feel for all who tried to use new technologies to discover new ways to gain awareness.
[ link to this | view in thread ]
romy v, 9 Dec 2009 @ 7:16pm

same elements also apply to the recent Fat Cyclist and Johan Bruyneel events
One could analyze the recent events between Fat Cyclist and Johan Bruyneel (http://bit.ly/7yNhF1) and find the same elements being key there as well.

They are the same reasons why Fat Cyclist was able to mobilize a large number of people and reach the goals that were put in front of him as a challenge. It all started from an innocent letter "hey, wouldn't it be great to ..." - that turned into a great success, not only for Fatty but also for the two charities involved.
[ link to this | view in thread ]
Anonymous Coward, 9 Dec 2009 @ 7:53pm

Re: Re: An Experiment in Finding The Needle in a Haystack
Wow. It'd look pretty stupid to STILL be looking for him, ya know...
[ link to this | view in thread ]
Marcus Carab (profile), 9 Dec 2009 @ 8:27pm

Re: Re: Re: An Experiment in Finding The Needle in a Haystack
pardon?
[ link to this | view in thread ]
JP Werlin, 9 Dec 2009 @ 8:53pm

The Day Google Didn't Matter
Mike,

I agree with many of your points above, however I think you missed a big one regarding Google. I shared my thoughts on the matter here in case you are interested. http://bit.ly/6xTukt

Great job on Saturday and maybe we will cross paths again.
[ link to this | view in thread ]
Anonymous Coward, 9 Dec 2009 @ 9:03pm

Re: An Experiment in Finding The Needle in a Haystack
I doubt they really wanted to find him
[ link to this | view in thread ]
fishbane, 9 Dec 2009 @ 10:36pm

Shouldn't the lesson be,
Pyramid schemes are actually good for something afterall?
[ link to this | view in thread ]
mryakima, 9 Dec 2009 @ 11:49pm

Now "they" have found the tools to stop public dissent during a crisis
So, imagine a day possibly not very far in the future, when "they" are attempting to crack down on the dissenters (Yes, that could be you and me). With the assistance of a bunch of unwitting helpers this week the powers that be have just been gifted exactly what they need to know to shut down communications among the citizens of this or any other country. This information will also be very helpful to the dictators in Iran.
[ link to this | view in thread ]
., 10 Dec 2009 @ 3:03am

Re: Now "they" have found the tools to stop public dissent during a crisis
Or you can use it to spot corruption too.

Or you can use it to make the system more democratic with direct input from the people.

Or you could better coordinate disaster response using people as a network.

Well the possibilities are numerous the real may be "Does it work all the time"? Which fatctors contribute to success and which don't?

When people are willing to help it seems to work great but what if it is something unpopular would it work?
[ link to this | view in thread ]
Steve R. (profile), 10 Dec 2009 @ 5:16am

Re: Re: An Experiment in Finding The Needle in a Haystack
Correct. There is also the political question of whether we really want to find him or not.

Assuming that we do really want to find him and given the fact that we have not found him we need to reassess our "finding algorithm". Clearly time for Plan B.

Of course, there is nothing obvious as pointing to a "search for Bin Laden". There could well be another reason for this social experiment.
[ link to this | view in thread ]
Hephaestus (profile), 10 Dec 2009 @ 6:24am

Neat all the things plus one I have
Most of them were obvious the one I missed ....

288 note/entry) Add "Look beyond your immediate "group"" to the list of motivations to use list.

compare the above list to the motivations to use list to see if any others were missed.
[ link to this | view in thread ]
Evan, 10 Dec 2009 @ 6:51am

The balloon challenge is no where near as innovative as the Urban/Grand Challenge, you should not compare the two.

All of your lessons are already known, especially to any forward looking social media user. Combing the ages long concept of multi-level marketing with social media is hardly novel. Besides, they won because Slashdot decided to allow their comment to be posted- how is that not favoritism? Do we even know that the Slashdot moderators aren't in for a cut- shouldn't they be? They DID recruit people... conflict of interest if you believe in any sort of objectivity from the Slashdot moderators.

So I think Slashdot acted improperly, and the other Grand Challenges are FAR more innovative.
[ link to this | view in thread ]
chris (profile), 10 Dec 2009 @ 8:29am

Re:
Do we even know that the Slashdot moderators aren't in for a cut- shouldn't they be? They DID recruit people... conflict of interest if you believe in any sort of objectivity from the Slashdot moderators.

slashdot is moderated largely by people selected at random based on their karmic score. moderators get 5 points to moderate posts with and you can only add 1 point to a comment. moderations are then moderated again by an even larger pool of randomly selected individuals in a process called meta-moderation.

sure it's still possible to game the system, but you would need a large number of people who have mod points available, making a paid system very difficult.
[ link to this | view in thread ]
nasch (profile), 10 Dec 2009 @ 8:56am

Re: Re:
Perhaps Evan meant Slashdot editors.
[ link to this | view in thread ]
Eric C (profile), 10 Dec 2009 @ 10:44am

Re: MIT's excuse of a victory: DARPA's Red Balloons
Everything in this post is unbearably stupid. MIT won by bribing? There was a $40,000 reward. If anything, DARPA won by bribing. I read the official rules; they were very loose, and the whole idea was "by any means necessary." They learned exactly what they set out to learn: the quickest way that $40,000 will get you information. Turns out, it isn't Twitter or Wave or Facebook, it's a single, organized hub around which all those other tools can function freely in their own chaotic ways. DARPA knows what the hell they are doing, and if they had wanted things to go a different way, they would have made the rules more strict. The MIT team didn't break the rules or stifle innovation, they simply executed their own idea according to the structure set forth, and theirs ended up being the best way. Simple as that.
[ link to this | view in thread ]
Gene Cavanaugh, 10 Dec 2009 @ 12:02pm

MIT experiment
Autonomous vehicle: very important, first step in a very promising direction; BROAD applicability. Breakthrough.

Crowd sourcing: cute. Been done before, very little real applicability or value, but a sort of fun thing.
[ link to this | view in thread ]
Anonymous Coward, 30 Dec 2009 @ 12:40pm

You engineers always think things are cooler than people.
[ link to this | view in thread ]
Anonymous Coward, 30 Dec 2009 @ 12:40pm

You engineers always think things are cooler than people.
[ link to this | view in thread ]