Is Google's Closed Nature Its Achilles Heel?

from the attacking-where-it's-weak dept

Tue, Sep 14th 2010 4:16am — Mike Masnick

There's been some well-deserved buzz lately about the new upstart search engine Blekko, which I've been playing around with a bit. Most of the attention has been focused on the concept of "slashtags" that allow very quick and dirty targeted search, or added parameters in a format that builds off of user suggested groupings of sites. It's a neat feature, but so far I have to admit I haven't found in that useful in my testing. What has struck me as significantly more interesting about Blekko is that it -- unlike just about every other search engine out there -- lets you dig in on the data the site used to determine the results rankings. This is the type of stuff that is a closely guarded secret at Google, in large part because revealing the data might make it easier to game.

And while Blekko founder/CEO Rich Skrenta (who, I should disclose, I've known for many, many years) has been clear from the very beginning that the goal is not to take down Google, but rather to carve out a decent, but highly profitable, niche in the search engine space, I do find this rather interesting. By opening up the details in a way that lets users dig in and find out how a site's ranking is determined, Blekko is doing something that Google can't easily copy. I always find this quite fascinating. We end up talking quite a bit on this site about the idea that some insist upon, that if you come up with something disruptive, big companies will just come along and copy the idea, killing off the small company. However, as we've detailed over and over again, this happens a lot less than you would think, in part because "copying" the innovation often will seriously upset an existing line of business or an existing way of doing things. That's what's so disruptive about disruptive innovation.

I have no idea if Blekko really will turn out to be disruptive in any way. But I find it quite fascinating that a big part of its attack on Google's marketshare is to enable a feature that would totally turn Google upside down in terms of how secretive it is about its algorithm and ranking. Google is often seen as a leader in the "open" technology world, but as we've pointed out before, while the company works hard to encourage others to be open, it can be incredibly closed itself. So it seems only fitting that its potential achilles' heel may be in the part of its business that it has kept quite closed.

Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.

Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.

While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.

–The Techdirt Team

Filed Under: openness, search
Companies: blekko, google

23 Comments

If you liked this post, you may also be interested in...

Reader Comments

Subscribe: RSS

View by: Time | Thread

Radjin, 14 Sep 2010 @ 5:08am

Indeed
Google has been a do as we say not as we do company for years. Because of their power in the ability to make or break an on-line company with a simple tweak of software they could be that way. It's nice to see someone come along that does not have to follow Goggle's rules. It was starting to feel like the on-line market was getting a little stagnant with the Google god always in control.
[ link to this | view in thread ]
Jay (profile), 14 Sep 2010 @ 5:21am

Just a thought
It would be really interesting if Google disrupted itself to compete with Blekko.
[ link to this | view in thread ]
Gary, 14 Sep 2010 @ 5:24am

I'd be more impressed if the site let me tweak the parameters myself. As it is, I'm still bound by someone spoonfeeding me their analysis whether it corresponds with my needs or not.
[ link to this | view in thread ]
Robert, 14 Sep 2010 @ 5:29am

How does this make sense?
"a closely guarded secret at Google, in large part because revealing the data might make it easier to game."

So now we'll have a search engine that caters to those who know how to game search engine optimization instead of those who have the most relevant content?
[ link to this | view in thread ]
Danny, 14 Sep 2010 @ 5:54am

If so...
And while Blekko founder/CEO Rich Skrenta (who, I should disclose, I've known for many, many years) has been clear from the very beginning that the goal is not to take down Google,...
If this is true then I think they may stand a chance at success. It seems that when someone enters the market and brands itself the "(insert market leader) killer" they doom themselves to fail for not being able to kill the giant.
[ link to this | view in thread ]
Richard (profile), 14 Sep 2010 @ 5:58am

Re: How does this make sense?
So now we'll have a search engine that caters to those who know how to game search engine optimization instead of those who have the most relevant content?

Search engines don't cater for those who have content they cater for the searcher.

If this facility also enables the searcher to "game the algorithm" to produce better results for their personal needs then it might have a chance.

Maybe it could even crowdsource the optimum search algorithm?

At the end of the day the quality of search results as perceived by the searcher that will determine whether this takes off or not.

Google results only matter to website owners because lots of people use Google. This new site is irrelevant until people start using it and it is the quality of search results that will control that.

Google took over from Alta Vista, Yahoo etc because it's search results were so much better. Only after that happened could the ad money start to roll in.

Google had a smart idea and backed it up with plenty of risk finance upfront before they started to make money.
[ link to this | view in thread ]
Anonymous Coward, 14 Sep 2010 @ 5:58am

Closed beta, but they want a fucking tweet to join? *headsmack*

Another search engine I've forgotten about.
[ link to this | view in thread ]
Anonymous Coward, 14 Sep 2010 @ 6:02am

Re:
It surely wouldn't be easy to game the system making parameters available to users to choose from those creating thousands of formulas.
[ link to this | view in thread ]
Dark Helmet (profile), 14 Sep 2010 @ 6:05am

Re: Re: How does this make sense?
"Maybe it could even crowdsource the optimum search algorithm?"

That's exactly what I was thinking. If you aren't going to keep the algorithm a secret, and you're sure as shit to not want SEO gurus gaming the system, simply take the system completely out of their hands.

As I see it, there is one huge key to making that work: user base size at the onset. The problem with doing this as a startup is that company interests can game a crowdsourced ranking system because there aren't enough legitimate users. What's interesting about this "disruption", if that's how they want to do it, would be that Google, with it's huge user base, is positioned to do the job at the onset better than any upstart.

....but only if they do it RIGHT NOW.
[ link to this | view in thread ]
eclecticdave (profile), 14 Sep 2010 @ 6:08am

Re: How does this make sense?
The key is the bit where it says "might make it easier to game".

No one knows if that's true or not, only that Google believes it. But their approach is essentially security by obscurity - which only really ensures that if someone does manage to game Google, they can stay under the radar more or less indefinitely.

On the other hand, if it is possible to produce a search engine algorithm that is impossible (or at least impractical) to game, then an open process, where everyone can see what's happening and suggest improvements to the algorithm, may be the best way to go about it.
[ link to this | view in thread ]
Anonymous Coward, 14 Sep 2010 @ 6:23am

Re: Re: Re: How does this make sense?
Perhaps an algorithm that was tunable per user so that SEO can't effect everyone.
[ link to this | view in thread ]
Shawn (profile), 14 Sep 2010 @ 6:25am

My guess is better than 95% of people who use Google to search have no idea what a search algorithm is. Of the 5% who know what one is 95% are happy with the results they get from a Google search and really do not care whether or not the "secret sauce" is tainted because the results are what they care about, not how Google massages the data.
[ link to this | view in thread ]
Dark Helmet (profile), 14 Sep 2010 @ 6:28am

Re: Re: Re: Re: How does this make sense?
Why don't we just have SkyNet manage it?
[ link to this | view in thread ]
Anonymous Coward, 14 Sep 2010 @ 7:07am

Re:
Requiring a non-associated service (or any service other than perhaps email) to join is just moronic.

The fact that it requires a tweet to get into the private beta indicates that it caters to a particular crowd. As such, the odds of this actually being useful is rather low.
[ link to this | view in thread ]
Anonymous Coward, 14 Sep 2010 @ 8:21am

One thing I'm worried about though is that if these search engines were more open they would be more susceptible to frivolous patent infringement suits (even submarine patents where the patent office grants patents on something after the fact).
[ link to this | view in thread ]
Anonymous Coward, 14 Sep 2010 @ 8:25am

Re:
and if you think I'm exaggerating see this patent

http://searchengineland.com/xerox-sues-google-yahoo-over-search-patent-copy-that-36742

where Xerox sues Google and others for search, among many other bogus patents seen here on Techdirt.
[ link to this | view in thread ]
Common Sense, 14 Sep 2010 @ 8:28am

Wait....What????
Blekko founder/CEO Rich Skrenta (who, I should disclose, I've known for many, many years)

This is one of the reasons I keep coming back here to read your articles... You don't try to hide your affiliations or positions on things like too many mainstream media outlets.

On top of that, even though this guy sounds like a good friend of yours, you're not pushing this site with unreasonable claims or anything, and you're still coming across with your professional "These are the facts, this is how I feel, please discuss" approach. Much respect.
[ link to this | view in thread ]
PrometheeFeu (profile), 14 Sep 2010 @ 8:45am

I think that while I would like to see google being more open on many things, I can see their points about search. Their ranking algorithm will always be nothing more than an approximation of the "perfect" ranking algorithm. As such, it will be possible to game it. So eventually, SEO guys will figure out how to game the secret algorithm, but my impression is google is constantly updating their algorithm, so you can't game the system for very long before an update obsoletes your cheat. On the other hand, they could open up things like gmail where that justification is absent.
[ link to this | view in thread ]
Derek Kerton (profile), 14 Sep 2010 @ 12:21pm

Google?
But Mike,

In the comments I always read that Google is your corporate overlord. Are you sure you can draw attention to an upstart competitor?

Where are those commenters on this thread? I need them to explain this behavior to me.
[ link to this | view in thread ]
BearGriz72 (profile), 14 Sep 2010 @ 3:13pm

Re: Re:
You can also email support@blekko.com to ask for an invite (of course I found that out on Facebook, I don't use Twitter)
[ link to this | view in thread ]
Rich Skrenta, 14 Sep 2010 @ 4:06pm

Re: Re: Re: How does this make sense?
I think it might be the opposite. If G were to launch a crowdsourced search relevancy / editing system, they would have a massive attack problem from day 1. By contrast, when a small crowdsourced effort is just getting started, it tends to attract passionate fans. Think of who Wikipedia attracted in their first year, vs. their spam problem today -- it's a huge target, there are SEO how-to articles about the best way to use sock puppet accounts to get your links in there.
Until blekko has 51% market share, no one is going to spend time spamming us. blekko has no traffic. We don't let other search engines in, so you can't even get a measley nofollowed link off of blekko.com. There is simply no incentive to spam us.
[ link to this | view in thread ]
snipin, 2 Sep 2012 @ 9:18pm

i HaTe BleKkO
blekko is annoying! it shows ads and i cant stand them! i tried deleting it removing it off of start tabs it just wont go away >:( i got it to go away finally today how happy i am...
[ link to this | view in thread ]
Tucson SEO, 4 Nov 2013 @ 3:29pm

Blekko has made it this far
I think their quality is great, and definitely gives Google a run for their money. Now whether they are profitable, I don't know. Remains to be seen.
[ link to this | view in thread ]