Do Search Engines Need To Be Regulated?
from the speak-freely,-search-freely dept
We've seen way too many lawsuits from people who get upset that Google doesn't rank them highly enough (or that Google has erased them from its index for gaming the search rankings). They often seem to think that it's a natural right that Google must rank them and must rank them highly. Of course, some of us feel that Google is a private company, and has the right to rank sites however it wants. If those rankings aren't very good, then that simply represents an opportunity for the other search engines to provide a better solution and steal away users. Law professor Eric Goldman, who tends to agree with us on that point, now points us to a new academic paper suggesting the opposite: that search engines should be regulated as their results represent a form of free speech. Specifically, the paper argues four key points should be regulated into place:- Search engines should not be allowed to remove any sites from a search index unless required to remove it by law.
- Search engines must reveal the basis of their ranking methodology and must continue to use the methodology they have made public
- Search engines cannot manipulate search results except if there is a clear example of abuse that needs to be changed
- Search engines should be required to clearly state which results are paid and which are organic
Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.
Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.
While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.
–The Techdirt Team
Filed Under: regulation, search bias, search engines
Companies: google, yahoo
Reader Comments
Subscribe: RSS
View by: Time | Thread
go here -> http://www.noooxml.org/forum/t-22697/what-happened-with-the-ranking-of-no-ooxml-on-live-com
and you will notice each country has a very different ranking for the most accurate wording on search engines....its as if microsoft is regulating its own search engine in a rather unfair fashion, huh?
So unfortunately, I don't know where I could stand on that....minimal amount of regulation would be desired but the result would be an unintended oversight or maximum regulation (we know how stupid politicians are), for sure. Not to mention the concerns of free speech and other things...if the government can just remove search results a lot of information could be disappearing from search engines that Bush & co don't want there.
[ link to this | view in chronology ]
Search is free speach?
I agree with this on the principle no one, especially corporations, should censor the internet.
2. Search engines must reveal the basis of their ranking methodology and must continue to use the methodology they have made public.
Clearly the author has no idea how internet search works. The main reason Google has been successful is because it serves up relevant search results very quickly. They do this using proprietary indexing and ranking technology. No company serving the best interests of its shareholders gives away their trade secrets and the primary means of their competitive advantage.
3. Search engines cannot manipulate search results except if there is a clear example of abuse that needs to be changed.
See # 2
4. Search engines should be required to clearly state which results are paid and which are organic
Hmmm, Paid links appear on the right under the title 'Sponsored Links'. Not too hard to figure out which results are paid and which are 'organic'.
[ link to this | view in chronology ]
Re: Search is free speach?
1. I do agree with the first point, unless there is a very VALID reason to do so.
2. Everyone screams that MS should show there code, why not search engines, which now-a-days have just as big as an impact on IT as MS.
3. I can't believe you think there is no code for indexing and ranking that doesn't manipulate the data. I would love to live in your world of the blind.
4. If that is the only people you think paid for links, I refer to No. 3, the world sounds rosey and honest.
I am done.
[ link to this | view in chronology ]
Re: Search is free speach?
1. Search engines should not be allowed to remove any sites from a search index unless required to remove it by law.
I agree with this on the principle no one, especially corporations, should censor the internet.
Not showing up on a search engine's results isn't censoring anything. If you know the URL or IP address of the website, you can still view the page(s) without a search engine.
Why don't you see websites written in chinese/swahili/etc in your search results very often? There are sites written in those languages discussing the same topics you searched for, but the search engine obviously does some localization. Do you want half of the search results you see to be languages you can't read? I know I don't, but then again I'm also not claiming that excluding a site from search results is somehow construed as censorship.
[ link to this | view in chronology ]
Re: Re: Search is free speach?
No, but having the option to remove those from the list of results should be my choice, not be made for me.
I have a real problem with this:
Not showing up on a search engine's results isn't censoring anything. If you know the URL or IP address of the website, you can still view the page(s) without a search engine.
The vast majority of searches is to find websites/-pages that contain whatever you put in for your search criteria, not to find the website you sort of, more or less, remember the URL or IP address of.
So yes, not showing up on a search engine's results will for the most part result in people not finding your page/site. Because how would you ever remember the URL or IP address if you can't find the site through a search engine the first time around???
[ link to this | view in chronology ]
Re: Re: Re: Search is free speach?
[ link to this | view in chronology ]
Re: Search is free speach?
> remove any sites from a search index unless
> required to remove it by law.
> I agree with this on the principle no one,
> especially corporations, should censor the
> internet.
No one's censoring the internet. If I create a search engine that doesn't list your web site, I haven't censored your web site. Your web site is still there, on the internet, untouched and uncensored. The fact that someone can't find it by using my search engine doesn't change that fact.
[ link to this | view in chronology ]
Re: Re: Search is free speach?
[ link to this | view in chronology ]
Re: Re: Re: Search is free speach?
[ link to this | view in chronology ]
Re: Re: Re: Search is free speach?
Your library example is flawed. Google is a private bookstore with the right to advertise or carry any book at their discretion. Most libraries outside of congregational collections do not and will not stock every book out there - undesirable and unused materials will be filed away in storage or disposed of entirely.
Telling Google what to link to and how strongly is like telling Mike what story he should write about and his opinion on it.. a suggestion with no obligation to do act or even listen.
[ link to this | view in chronology ]
Re: Re: Re: Search is free speach?
> room without lighting, a sign on the door that says "maintenance"
> and not include those books in their index system, would that
> be equally ok with you?
Sure. Especially if the library was a private organization, they have every right to index their books however they want. Besides, that analogy fails on its face because while libraries have physical control of the books, search engines do not. They have no acutal control over the "books" (websites) they index.
[ link to this | view in chronology ]
It May Be Your Free Speech .....
A search engine is a convenience, not a requirement for free speech. If I have a political website that I want to attract visitors to, there are many options to do that. Some of the options involve money, as in buying advertising. But, that's the way it is. Speech is free, but no one should be compelled to listen. Nor should any third party (search engine) be required to assist you exercise your right.
[ link to this | view in chronology ]
search for free speech
Whose law? The law of the nation where the search engine is based ? The law in the country of the person who is claiming "censorship". The law of the country with the most nukes?
It makes very little sense to me.
[ link to this | view in chronology ]
If a search engine starts to suck enough, people will quit using it and go to another (Yahoo's a good example of that).
Who the hell uses Microsoft's search engine anyway? lol
If Google gets sued because someone's page didn't come up where they want it to - too bad. Google should just exclude them altogether. People are proposing a bunch of close to unenforceable laws.
I've noticed Google is starting to suck more and more too - trying to regulate what comes up where. Time for a new competitor, IMO. I don't think Google's so 'innocent' anymore when you search for stuff, I'm sure there's some money moving behind the scenes too.
Last thing we want or need is the Government telling search engines what they can and can't put up. If you like that mentality, there's already a country serving your needs... China - just use their search engine!
[ link to this | view in chronology ]
Absurd.
[ link to this | view in chronology ]
A Right to Be Heard?
[ link to this | view in chronology ]
short, simple, correct answer
understatement of the day:
"...any intermediaries that get in the way of a listener hearing any particular opinion is thus interfering with free speech. It seems like quite a stretch"
LOL! quite?! try ridiculous!
geez, don't people have anything better to do with their time ? you don't like google - then use another search engine! there are literally tons out there - some good some bad ... I don't use all of 'em, but I sure do use a lot more than just google ...
besides, you can use something like tor -it's an interesting experiment - I have found that that sometimes results change in google searches depending on where it *thinks* you're coming from ... (i.e. your tor exit point ...)
[ link to this | view in chronology ]
Me thinks...
[ link to this | view in chronology ]
Why should they follow any of this?
When Google starts filtering legitimate companies, I'm sure they will have a reason. Even if they don't, it's no one's business to know what the reason is, EVEN IF you are who is directly affected by the filter. What google offers is a free (to the end-user) service, and if you don't like it there are plenty of other free services out there.
I believe it'd be a restriction of free speech to NOT allow google to remove sites from its index. That's completely letting alone privacy laws, and what the government can and cannot dictate on private property.
[ link to this | view in chronology ]
show their methods? are you crazy?
If search engines give away their ranking methods, the same thing will happen again. Is this what we want?
The author of this paper is crazy.
As for the index.. it's their index. they made it. They should be able to do whatever they want with it.
Why do Americans keep talking about freedom, yet keep trying to do away with the free market?
[ link to this | view in chronology ]
Don't we have enough "regulation" yet .. ??
Where does it end, pray tell?
Or is this just an opening for regulating the Internet, then what ads can appear from who, and why, and/and/and ..
And who would be the watchdog over the regulating?
And who watches them .. ??
Bill Burke
Wireless Speech Recognition
[ link to this | view in chronology ]
this is an OLD problem
Elections are the perfect example showing the difference between having free speech and actually being able to be heard. Anyone can say whatever they want, but if they don't have the money to spend on advertising, nobody will know about it.
On the other hand: what is freedom? Does it mean "you can do whatever you want"? Most of us think it doesn't, because that would mean (some sick) people would "feel free" to harm others. Freedom is not something you receive, freedom is something you give. If we'd all stop worrying about our own freedom and starting thinking about granting others freedom, the world could be very different.
So, does the Internet need be regulated in order to provide freedom to everyone? Probably. But there is a difficult balance between maintaining freedom and becoming a police state (with the best of intentions sometimes). Also, not everyone realises what Google really is. Just like not everyone realises what wikipedia is. Freedom also requires knowledge about the world. Intelligent, well educated people are often freer than those who are not. So the best we can do is educate people about how the internet and search engines really work. But in the end, people are free to be stupid....
[ link to this | view in chronology ]
Am I missing something?
It is just a service to help you find things quicker. Freedom of speech has nothing to do with finding websites. It sure as heck doesn't help me find my keys in the morning, so what does it have to do with Google's free searching service???
[ link to this | view in chronology ]
Freedom of Choice
[ link to this | view in chronology ]
Analogy....
Yahoo started with a hand edited directory of links. Then process was automated. Google perfected it. I don't think this is a violation of any of the indexed sites free speech. Search rankings are not a public forum. Google chooses to pubish their search indexing. While it is an issue of free speech, it is the free speech rights of a private company or organization to publish whatever they want however they want. A search engine is not a public space, and thus they are not subject to providing the right to free speech to everyone.
Here's an interesting wrinkle. What if I created an advertisement in Wikipedia and posted it in an article. Editors would of course take it down. Now couldn't that be considered impinging on my right to free speech? Especially, considering that technically Wikipedia is a public forum?
[ link to this | view in chronology ]
spam
[ link to this | view in chronology ]
Sounds like the desired result is that every result should be number one? After all, that's the only way it can be fair :P
[ link to this | view in chronology ]
Sometimes the innocent suffer, but...
Our "crime" was most likely that we didn't understand the rules Google uses to rank sites, and we were penalized for an innocent part of our site design that, without context that would be given by a human review, must have looked like something they try to stop. We've spent months and several SEO contracts reworking our site to play by their rules, but it's damn hard when they won't tell us what those rules are.
That said, in general Google's goals are good. I wish they would offer more assistance to sites that have been wrongly blacklisted, but I don't think legislation is the way to go. If you tell them they can't legally blacklist anyone, the quality of overall search results would suffer.
[ link to this | view in chronology ]
Viruses, malware, trojans
If a site infects users with a virus, it's not required by law to remove that site, but it's clearly a good thing for users if the search engine removes that site.
[ link to this | view in chronology ]
As for rBase4u, there something fishy about what you are saying. "...and we were penalized for an innocent part of our site design that, without context that would be given by a human review, must have looked like something they try to stop" I can't imagine what is on your site, but with how many times you talk about yourselves being innocent, I just can't believe you.
Do you pay Google to include you in the search results? If the answer is no, then they owe you nothing. The only people that Google has an obligation to include in the results are people who are paying for the advertising. If your business model relies so heavily on Google traffic, then why aren't you advertising with them? It seems to me that you are angry that Google no longer supplies over 75% (10 million of your 13 million monthly hits) of your web traffic for free. If you don't like losing out on that much traffic, pay them or find a better business model.
They do a good job of including relevent avertising as well as non advertised sites, thats why people use them. If they only included paid referrals, no one would go to them.
[ link to this | view in chronology ]
Pretty soon its gonna be a crime to walk down a street without gloves on in the winter time... or something like that.
[ link to this | view in chronology ]
[ link to this | view in chronology ]
A thought
it's one thing to give certain results a very low rating, resulting in them always showing up at the bottom of the list (but, they are still there and available for anybody who would want them) it's another to remove them from the search results altogether.
Although I personally also don't enjoy link farms and sites that generate text SOLELY to bring people to click their ads, I still think instead of removing them, rank them so low they end up at the bottom of the listing, instead of removing them.
I guess what I have an issue with is: Google, or any search engine for that matter, even if they don't say so explicitely, implicitely claims to search the entire web and that the returned results are all the matches on the (entire) web. So removing any results, no matter how noble the intention might be, amounts to misleading or even fraud in my book (regardless of whether it would be legal)
[ link to this | view in chronology ]
That is a completely inaccurate analogy. A library is a publicly funded program. Google isn't.
By your line of logic, a bookstore that wants to only sell children's books could also be told that they HAVE to sell adult magazines because someone might be in their store and want to buy one. If they don't sell them, then they are obviously censoring the adult magazines.
An accurate analogy is Walmart. Walmart originally didn't sell CD's with explicit lyrics (not sure if they still don't). It was their decision not to sell them. If you wanted a CD with explicit lyrics, then you had to shop somewhere else. Is anyone arguing that Walmart should HAVE to sell CD's with explicit lyrics? No.
Google is a private company and shouldn't be told what they HAVE to include or not include. If you don't like what they keep out of their index, then "shop" somewhere else.
[ link to this | view in chronology ]
author is 100% right!
when the government creates their own search engine, it definitely should follow these rules. then the 200 people who use that crappy engine can know that they have unfettered access to all the crap that those of us who use google count on being filtered out.
i also agree with the library analogy. i plan to march down to my local library tomorrow and insist that they stock every book ever made, those censoring bastards.
[ link to this | view in chronology ]
Analogy Tuesday
[ link to this | view in chronology ]
Bright eager little fascists.
1. The first rule makes no sense since there is no practical difference between removing page from an index and pushing its ranking down by a thousand or so.
2. Ranking methodology is a trade secret and it would seriously hurt Google (through competition) and Google users (through gaming of the algorithm) if they revealed their method. I really do not want all of my searches to bring up Russian porn sites.
3. Point three is undefinable and unenforceable and hence discredits the whole proposal.
4. Sure, search engines should disclose paid rankings.
The problem people have with getting an audience has nothing to do with getting exposure and everything to do with getting noticed. If you have a compelling point to make you don't need Google and if you're spouting stuff that people have already heard or will reject out of hand you're going nowhere even with a high ranking.
The author should go offer her services to the French who want to build a French competitor for Google. She and the French deserve each other.
[ link to this | view in chronology ]
This response should be move to the top of the pag
Who should be the judge if search engine are ranking web sites properly or not, if it is not going to be done by the search provider themselves?
Personally I think my web site should be top rank for all key words(English or not) and I should not have to pay for priority placement. I also think being charged by my ISP is also a form of censorship I should have free unlimited wireless bandwidth and server space. Anyone charging for me for my 100Tbit/s wireless internet hates the constitution! While we are at it why am I paying for computing power? I need the worlds fastest super computer to render my videos; this is a god given right for without it I don't have free-dumb of speech.
[ link to this | view in chronology ]
Regulation of Search Engines - Publicity Argument
Carl E. Person
[ link to this | view in chronology ]
Absolutely right.
[ link to this | view in chronology ]