Beyond The Privacy Outcry, AOL Search Data Actually Is Interesting Info

from the oh,-right,-the-research... dept

Wed, Aug 16th 2006 5:27pm — Mike Masnick

Beck writes "There has been much hoopla over the privacy concerns regarding AOL's release of search data. Lost in the excitement is the fact that AOL had good intentions. They wanted to give researchers a window into how people use online search. If you step back for a moment and take a look, the findings are interesting. Lee Gomes of the Wall Street Journal published the results of his research. He did not approach the story from the privacy angle, but rather from the view of a researcher who wants to know how people use the Internet. He found, for example, that the most common search term is the word "free". Also interesting is that 47% of searches resulted in no click by the user. 28% of searches were refinements of previous searches, and 42% of the time the user clicks on the first item in the search results. The article includes additional data, and a look at the quality of the search results."

Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.

Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.

While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.

–The Techdirt Team

19 Comments

If you liked this post, you may also be interested in...

Reader Comments

Subscribe: RSS

View by: Time | Thread

Eric, 16 Aug 2006 @ 5:47pm

Yuck
I hate AOL.
[ link to this | view in thread ]
Matt, 16 Aug 2006 @ 5:51pm

Good intentions?
If I recall correctly, isn't the road to hell supposed to be paved with good intentions? What if I don't *want* to give researchers a "window into how [I] use the Internet?" Isn't that what all of these cookies does?

I want to be able to use the Internet ANONYMOUSLY, without having to wonder if any personally identifying information is being used for purposes that I don't explicitly agree to. Figure THAT into your data!
[ link to this | view in thread ]
sean, 16 Aug 2006 @ 5:54pm

"47% of searches resulted in no click by the user" thats most of my searches with google I can usually get most of the info im looking for in the results
[ link to this | view in thread ]
joshua nolan, 16 Aug 2006 @ 6:13pm

great tool for looking at the data..
We threw up our site to search the data (www.dontdelete.com) and were sooooo surprised but the results. Not just the freaky stuff (which is entertaining).. but also, after looking at the data for so long, query after query, you realize how technically inept AOL users. There must be something confusing about the AOL interface too.. users constantly search for www.website.com. Just dumb.
[ link to this | view in thread ]
Scott, 16 Aug 2006 @ 6:43pm

Re: great tool for looking at the data..
Not just AOL users.. but the majority of internet users search for www.website.com to find the website they want to go to.

Most people do not know how to use an address bar.. they think you use the search box to get where you want to go and the address bar shows you where you are.

I have a dozen websites and the majority of my traffic comes from people searching for the domain name.
[ link to this | view in thread ]
Posterlogo, 16 Aug 2006 @ 6:51pm

Ya, well...
...the Nazis learned a lot from their experiments on prisoners. Doesn't mean you can then say "Oh well, if you step back beyond the atrocity, it's actually pretty interesting." Pathetic.
[ link to this | view in thread ]
Anonymous Coward, 16 Aug 2006 @ 6:57pm

Re: Ya, well...
your comparison is quite a stretch. we know that it wasn't a cool thing for AOL to do. it's done and over with. no one can take it back. how about instead of complaining repeatedly about it, we learn from the information that was garnished from it.

jeez. this article isn't condoning what AOL did.

and also if someone drops millions of stolen cash over the city, i know a hell of a lot of you would have a go at trying to take some.
[ link to this | view in thread ]
Dom, 16 Aug 2006 @ 8:35pm

Funny.
There are some really funny searches I've found that people have made. For instance:

"don't buy pepsi in the new can. pepsi has a new patriotic cancoming out with pictures of the empire state building and thepledge of allegiance on them.however pepsi left out two little words on the pledge under god. pepsi said they didn't want to of"

I can't see why any person would search for that. It doesn't make sense.
Then there's just the funny questions:

"how do you now if you lost your virginity"

Some people are just so technologically inept.
[ link to this | view in thread ]
Dan, 16 Aug 2006 @ 9:43pm

Re: Ya, well...
Well, though the comparison is obviously not valid in terms of the injustice, your comparison to the Nazi torture is interesting.

If a wrong has been done, but you can learn from the results, is it ok to make use of the data?

Nuremburg (I think it was Nuremburg) said no, but I personally think that looking at the data is interesting and should be done. And obviously most of the people on this board agree, even though they're pissed off about it -- because they've looked at the data.

It really is too bad AOL botched this so bad, because properly sanitized search data would be an interesting thing to release, every once in a while.
[ link to this | view in thread ]
E, 16 Aug 2006 @ 9:55pm

will the research improve user interfaces to searc
It would be nice if some of that research spurred search companies to help you refine your search better.

Having worked for a company whose bread and butter was based on natural langauge search, it's remarkable how the queries did not resemble natural language.

Also, significant numbers of searches were:

a) repeats (you would think that with a wide user-base,
that there would be low incidence of repeats... we had over 20%

b) urls or domain names
let DNS find them for you

c) whole cut-n-paste paragraphs
these were so nasty we had to limit the length of queries to keep them from killing the search engine(s).

but the area of most impact would have been to create interfaces that help the user refine their query faster...
ie: dont look so hard when they say:
"I want to do a book report on a manatee"
but assist them enough so that their final query is NOT simply:
"manatee"

-E
[ link to this | view in thread ]
Anonymous, 16 Aug 2006 @ 10:26pm

Re: Funny.
>> I can't see why any person would search for that. It doesn't make sense.

I make similar seaches all of the time. They were obviously searching for the original source of a post or email they saw. This is the same method a lot of teachers use when they suspect a student of plagiarism.

The virginity question was very odd, and probably submitted by a curious adolescent. Though the question itself is humorous, this type of search query isn't as useless or "noob-ish" as it might appear. Often times, a search with a query such as "how do you make grilled-cheese sandwiches" will return results from a forum of someone else asking the exact same question, along with the answer.
[ link to this | view in thread ]
Robert, 16 Aug 2006 @ 10:31pm

Wanna see what people are searching for??
Check out the Search Spy program here: http://freakwolfe.cheezyfilms.com/tools.html

It shows you what people are searching for.
[ link to this | view in thread ]
Frink, 16 Aug 2006 @ 11:06pm

Yep - it's good to know that something useful came out of an AOL security leak. They might as well go ahead and release the rest of the data now and we might finally find out what makes so many fools continue to be AOL customers. Someone might come up with a cure for that.
[ link to this | view in thread ]
Dom, 17 Aug 2006 @ 1:59am

Re: Re: Funny.
Two great points, I didn't think of it like that.

Still rather amusing to look at though :)
[ link to this | view in thread ]
Celes, 17 Aug 2006 @ 5:42am

Re: Re: great tool for looking at the data..
My husband always does that - it drives me nuts. I'm thinking about changing my homepage to a non-search engine just to see if he figures it out. ^_^
[ link to this | view in thread ]
Paragraph searcher, 17 Aug 2006 @ 8:46am

Re: Funny.
I do searches like that all the time (I like to think on better topics though) - well, not all the time, but rather when someone sends me an email about something which they believe that turns out to be a hoax, an example would be that mars email that keeps floating around where mars is going to be closet to earth than ever before on such and such a date. If you search on a paragraph or so of the text you can usually find it listed on a hoax site, or ripped apart by a blogger somewhere. I'm guessing that thing about pepsi was a forwarded junk mail that someone was trying to verify the validity of the info it contained.
[ link to this | view in thread ]
Anonymous Coward, 17 Aug 2006 @ 11:18pm

and beck is whom? and you're ripping off what he said why? seriously... next time why not put out an article that's 100% quoted, leave the "Beck said" part out... in fact, fuck the quotes... just a link...

GREAT ARTICLE MIKE!!!!!!!
[ link to this | view in thread ]
Mike (profile), 18 Aug 2006 @ 12:41am

Re:
and beck is whom? and you're ripping off what he said why? seriously... next time why not put out an article that's 100% quoted, leave the "Beck said" part out... in fact, fuck the quotes... just a link..

Apparently you're not familiar with how things work around here. Beck is someone who submitted that story and wrote things up. Sometimes we write up our own stories, sometimes we post submissions. We didn't rip it off, he submitted it to us to post.
[ link to this | view in thread ]
dataguy, 18 Aug 2006 @ 5:04am

Re: Funny.
"Some people are just so technologically inept."

Don't lose sight of the fact that this is a search history of AOL users ;-)
[ link to this | view in thread ]