Search Engine Death Penalty: WhenU Kicked Off Of Yahoo And Google
from the take-that dept
WhenU, the adware/spyware company that already has quite a bit of controversy surrounding it for both (a) sneaking their software onto users' machines and (b) then popping up competitive ads when surfers visit websites has now been removed from both Yahoo and Google searches after it was determined that they were using tricks to boost their rankings in both. WhenU blames an outside search engine optimizer who they hired, but it does raise some issues. Considering the power that sites like Google and Yahoo have over anyone finding your site, how long will it be until we hear about a lawsuit when someone sues a search engine for kicking their site out of search results?
Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.
Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.
While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.
–The Techdirt Team
Reader Comments
Subscribe: RSS
View by: Time | Thread
I'm actually surprised...
I mean, I'd expect that Google would have a second googlebot that would mimic IE or Netscape just to make sure that the page isn't cloaked.
[ link to this | view in chronology ]
Googlebot is bandwidth-intensive, slow
But a couple problems:
1) A far-reaching bot, that crawls deep, would be long, slow, and bandwidth-intensive. Google's existing crawl takes, what, a month (?), though really it's a never-ending task at this point. All indications are that Google is continually facing a constraint as to how much of the web to crawl -- would want to crawl more if it could, seemingly, but continually drops low-pagerank sites in an attempt to focus efforts where they're most needed.
2) Google doesn't look favorably on folks tampering with HTTP headers. But the crawler you propose would have fake HTTP headers (User Agent, Referer, etc.). Google might be uncomfortable with this.
3) The crawler would still come from Google's IP blocks. Some cloaking is based on HTTP headers (user agent or, in WhenU's case as to the research I just posted, Referer header or lack thereof). A second 'bot could catch these tricks. But cloaking on the basis of 'bot IP would be much harder to find: Google would have to use another set of IPs for this project, and once word got out which IPs those were, the smart cloakers would adjust their behavior accordingly.
All in all, I think it's actually not such an easy task. Wouldn't want to be in Google's shoes!
Ben Edelman
benedelman.org
[ link to this | view in chronology ]
No Subject Given
(unless you're looking for spyware...and even then)
[ link to this | view in chronology ]
No Subject Given
[ link to this | view in chronology ]
Didn't google get sued once
[ link to this | view in chronology ]
Re: Didn't google get sued once
Here's the article when they did get sued.
"In what's believed to be the first lawsuit of its kind, an ad network and search marketer is taking Google to court over changes the popular search engine made to its page-ranking system."
[ link to this | view in chronology ]
No Subject Given
[ link to this | view in chronology ]
No Subject Given
[ link to this | view in chronology ]
Google search
Chris Smith
abellabooks.com
[ link to this | view in chronology ]
Yahoo
please.Thanx CGrace
[ link to this | view in chronology ]
getting kicked off
[ link to this | view in chronology ]