Microsoft Highlights Why Google's 'Cheater' Accusations Ring Hollow

from the good-for-them dept

We had a long discussion recently about Google's response to discovering that Microsoft used clickstream data from users to help improve the relevance of their own search. Microsoft's Yusuf Mehdi has now written up a much more detailed response from Microsoft's point of view, in which it again clarifies that contrary to Google's statements, Microsoft is not "copying" Google's search results, but merely using clickstream data as one of many (Microsoft says approximately 1,000) variables in improving search relevance. Microsoft does take one cheap shot: noting that, technically, the "honeypot" trick that Google used to uncover this certainly appears to be a form of "clickfraud." That is, it was a trick designed specifically to manipulate Bing's search results.

But the key point is made towards the end:

We have brought a number of things to market that we are very proud of -- our daily home page photos, infinite scroll in image search, great travel and shopping experiences, a new and more useful visual approach to search, and partnerships with key leaders like Facebook and Twitter. If you are keeping tabs, you will notice Google has "copied" a few of these. Whether they have done it well we leave to customers. But more importantly, we take no issue and are glad we could help move the industry to adopt some good ideas.

That's the point that I tried to make in the original post. History has shown that innovation occurs via competition, and part of that competition often involves competitors building on each other's work. A few months back, I wrote a review of the excellent book Copycats by Oded Shenkar, which makes this point very, very clear. Innovation happens when companies build on each other's work. But, what you learn is that it's not just about "copying," it's about all of the players learning, innovating and expanding the overall market. Just straight up copying rarely does enough to make a difference (in fact, we've discussed this problem in the form of cargo cult copying, where companies just copy some superficial aspect, and discover that it's meaningless). That's clearly not what Microsoft was doing here.

In the comments to our original post, someone made the comment, in defense of Google, by saying if what Microsoft did was okay, then couldn't he just go out and say "I've got a billion dollar search engine idea!" and then just copy Google's results. But, of course, if anyone actually thinks this through, they'd realize that copying Google's search results is not a billion dollar search idea. Assuming that, tomorrow, we launched a "new search engine" that gave the identical results to Google, almost no one would use it. Why would you? There's no real advantage to doing so. And for people who already use Google, it's probably much more integrated into their lives, with Gmail, Google Docs and more. The search results themselves are not the "billion dollar idea." It's the overall execution.

Hopefully Google learns from this and realizes that it has learned plenty from watching Microsoft as well, and complaining about Microsoft using clickstream data is a waste of time. Focus on continuing to innovate, Google, which'll probably mean learning more things from Microsoft, in addition to what you're doing yourself.

To be fair, Matt Cutts also has a put together a decent response, where he points out that the real issue here may be disclosure -- in that Microsoft did not clearly disclose that it was using clicskstream data (and especially how it was using that data). That's a perfectly reasonable point, but it was not the original point that Google raised. I agree that Microsoft could and should be much clearer in its disclosure -- but that's a totally separate issue. Cutts also explains why he thinks that Microsoft really is "copying," but again, even if we grant that premise (which I don't think is accurate), I still don't see why that matters. Copying and improving is a part of the innovative process. Google should embrace it.

Filed Under: bing, copying, innovation, search
Companies: google, microsoft

Google's Childish Response To Microsoft Using Google To Increase Bing Relevance

from the get-over-it dept

It's inevitable as a company gets bigger and older that rather than just competing in the market, it starts attacking competitors and accusing them of doing something "wrong." It's too bad that Google appears to have reached this stage. There have been plenty of stories lately about Google's decreasing relevance and how its search results have been getting worse. There are plenty of ways to respond to this and improving search quality should be the main focus. But it looks like Google has, instead, decided to call out competitors. Specifically, Google set up an elaborate and pointless "sting operation," which appears to show that Microsoft uses Google results as a part of its overall relevance algorithm. Basically, it looks like for users who have the Bing toolbar installed, Microsoft aggregates some search information, perhaps including Google results, and weighs them (only partially) into its own algorithm.

This seems like a perfectly reasonable thing to do. Google's search results are public and as an established player in the market, almost every comparison of alternative search engines, including Bing, compares it to Google. So, making use of Google data to improve its own rankings seems like a rather smart move.

Remember, too, that Google's own search algorithm is based on viewing what people are doing online and coming up with a ranking based on that. How is that any different than Microsoft viewing a variety of information online -- including Google's own search rankings -- and using that as the basis of its own rankings? But instead of recognizing that this is all perfectly reasonable, Google starts acting like the RIAA, accusing Microsoft of "cheating" and doing something that is potentially illegal. It even pops out this line from Amit Singhal, a Google Fellow who apparently oversees Google's search ranking algorithm.

"I've got no problem with a competitor developing an innovative algorithm. But copying is not innovation, in my book."

As if Google hasn't copied the work of others in the past? The very basis for the original Page Rank was "copied" from Jon Kleinberg's research and then built upon that work. It was not a direct copy, just as Microsoft's search results are not a direct copy. For Google to attack a competitor for using open information on the web -- the same way it does -- seems like the height of hypocrisy. It's fine for Google to crawl and index whatever sites it wants in order to set up its ranking algorithms, but the second someone looks at Google's own rankings as part of their own determination, suddenly its "cheating"?

This seems like the latest in a series of indications that Google has moved past the innovation stage into the "protecting its turf" stage. That would be a shame.

Filed Under: algorithms, bing, relevance, search
Companies: google, microsoft

Will Google's New Hamfisted Censorship On Autocomplete Raise Questions Of Human Meddling?

from the bad-idea dept

One of the key arguments that critics have often made against Google is that the company "meddles" in search results, effectively "picking winners and losers." Google's -- quite reasonable -- response for years has been that it's all in the algorithm, rather than any personal choices. And, the algorithm was just trying to recommend the best result, no matter what that might be. Indeed, this is a perfect and sensible response. However, after lots of pressure from the entertainment industry (and politicians closely associated with the entertainment industry), last month Google announced plans to start censoring "autocomplete" results, such that "terms that are closely associated with piracy" don't appear.

As we noted when the announcement was made, this is really difficult to do in any reasonable manner. What's "closely associated with piracy," one day becomes a legitimate format the next. Take MP3s for example. Five or six years ago, if Google had made this decision, you would imagine that Google might have decided to block "mp3" from autocomplete -- and yet, now, MP3 is the standard that is used around the world in all sorts of legitimate online music stores, including iTunes and Amazon. We pointed out that blocking things like "bittorrent" or just "torrent" would be a mistake of the same nature -- as it's just a standard that has plenty of legitimate uses, even if it's frequently used for unauthorized copying today.

Unfortunately, whoever was in charge of handling this at Google went for a simplistic sledge hammer approach, with the company now dropping a variety of terms, many of which have perfectly legitimate uses. Many of the choices seem totally arbitrary. As expected, BitTorrent and torrent are now blocked -- despite plenty of legal uses, and the fact that BitTorrent itself is a perfectly legal company with tons of companies using its technology for completely noninfringing purposes. In the TorrentFreak link above, there are reactions from a variety of companies, including BitTorrent Inc., RapidShare and Vodo, who all note that this move appears to hurt their legitimate businesses.

And that's where I wonder if this move will backfire in a big way on Google. While the concept of "search neutrality" may be one of the more ridiculous ideas to come out of Google-haters for years, the fact that the company is now clearly hand-picking "winners and losers" when it comes to searches on these kinds of technologies and services seems like something that will be used as evidence against Google at some point.

Google had a strong defense in the past to complaints of bias, in that it was focused on not meddling with its results. However, while this move doesn't directly mess with the actual results, by mucking with autocomplete, it is likely to have an impact on the kinds of searches that people do, driving them away from many perfectly legitimate solutions, for no reason other than that the company caved to pressure based on no legal rationale. All this really does is now open the door for others to demand that Google adjust its search recommendations and results in their favor as well. I'm really surprised Google would agree to do this in the first place, let alone do it in such a... simplistic and overly broad fashion.

Filed Under: autocomplete, bittorrent, censorship, search, torrent
Companies: google

Google Effectively Puts Demand Media On Notice Days Before Planned IPO

from the well,-look-at-that... dept

Over the last year or so, there was growing concern about how "content farms" like Demand Media and Associated Content were "clogging up" search engines and cluttering the web with junk content. However, as we noted last summer, this is really a filter problem, rather than a content problem as many were claiming, and we assumed that, sooner or later, Google would realize that people hate this type of content and it would adjust its algorithm to filter it out (or to make it a lot less prominent). It appears that time may be coming. Over the last few weeks there have been a bunch of articles complaining about the decreasing usefulness of Google, in large part due to those content farms.

It looks like Google is finally waking up to this issue. Google spam-fighter-in-chief, Matt Cutts, has posted on the Official Google blog that the company has heard the complaints and realizes its algorithms need to be better at not recommending content farm crap that people don't like:

As "pure webspam" has decreased over time, attention has shifted instead to "content farms," which are sites with shallow or low-quality content. In 2010, we launched two major algorithmic changes focused on low-quality sites. Nonetheless, we hear the feedback from the web loud and clear: people are asking for even stronger action on content farms and sites that consist primarily of spammy or low-quality content. We take pride in Google search and strive to make each and every search perfect. The fact is that we're not perfect, and combined with users' skyrocketing expectations of Google, these imperfections get magnified in perception. However, we can and should do better.

The timing on this is especially interesting, given that the leading content farm, Demand Media, is looking to go public next week. The company's IPO is already considered to be pretty questionable, for a variety of reasons, including (but not limited to) the general sleaziness of how the company has been run -- including lying about profitability (and even suggesting others do it since no one gets caught), using highly questionable accounting tricks to make the company look more profitable than it really is, and the fact that insiders are dumping a huge amount of stock in the IPO. And, oh yeah, it relies really heavily on Google to make its money. So finding out that Google is likely to cut off at least some of that gravy train probably isn't what the company wanted to hear this week.

Hopefully this is a sign of things to come, however. While everyone has been complaining about content farms, it's time for the filters to step up and re-assert their own importance by diminishing the crappy content that no one wants to see.

Filed Under: content farms, filters, search
Companies: demand media, google

Wait, Can Anyone Explain Why Google Should Promote Other Search Engines?

from the i'm-confused dept

As the EU continues to use its Tim Wu-sian definition of monopoly where "being successful" equals "monopoly," it is has taken the step that everyone expected and has begun investigating Google for supposedly anti-competitive practices with regards to "competing search engines" such as Foundem. Foundem has been on this kick for a while. It has a crappy search engine that sucks and then it complains that Google doesn't link highly enough to them. The whole thing almost feels like it was set up purposely to have an excuse to go after Google.

As per usual, the absolute best response to this investigation comes from Danny Sullivan, who satirically notes that when he searches Google, obviously it had better link to other search engines as the top results:

I did a search at Google today for "cars" and was shocked. Rather than list links allowing me to search for "cars" on Bing, Yahoo, Baidu, Voila, Naver and Yandex, Google instead favored its own search results. I’m glad the EU will be investigating whether this favoritism violates anti-trust laws.

He goes on from there. It's worth reading the entire piece, because at the end he dismantles the entire argument of Foundem and the EU. It simply makes no sense. The whole thing is based on a basic fallacy that some companies who failed to actually get the market to come to them somehow think Google owes them traffic. Here's a tip: if your business model depends on getting traffic from Google, you are making a poor strategic decision.

And this is more true today than ever before. It wasn't that long ago that the vast majority of our referral traffic came from Google (the majority of our overall traffic comes from direct access/RSS readers). These days, however, while Google is still high up there, there's a wide mix of alternative sources that bring us traffic: StumbleUpon, Twitter, Facebook, Reddit and a variety of other sources. Most of these aren't search engines -- but they are massive traffic generators, showing that people find stuff via social means at an increasing rate, and the idea that any business is totally reliant on one company, Google, to survive, simply suggests a poor business strategy. Regulators should never reward poor business strategies, but that appears to be what they're looking to do over in the EU.

Filed Under: antitrust, europe, search, search results
Companies: google

Is Google's Closed Nature Its Achilles Heel?

from the attacking-where-it's-weak dept

There's been some well-deserved buzz lately about the new upstart search engine Blekko, which I've been playing around with a bit. Most of the attention has been focused on the concept of "slashtags" that allow very quick and dirty targeted search, or added parameters in a format that builds off of user suggested groupings of sites. It's a neat feature, but so far I have to admit I haven't found in that useful in my testing. What has struck me as significantly more interesting about Blekko is that it -- unlike just about every other search engine out there -- lets you dig in on the data the site used to determine the results rankings. This is the type of stuff that is a closely guarded secret at Google, in large part because revealing the data might make it easier to game.

And while Blekko founder/CEO Rich Skrenta (who, I should disclose, I've known for many, many years) has been clear from the very beginning that the goal is not to take down Google, but rather to carve out a decent, but highly profitable, niche in the search engine space, I do find this rather interesting. By opening up the details in a way that lets users dig in and find out how a site's ranking is determined, Blekko is doing something that Google can't easily copy. I always find this quite fascinating. We end up talking quite a bit on this site about the idea that some insist upon, that if you come up with something disruptive, big companies will just come along and copy the idea, killing off the small company. However, as we've detailed over and over again, this happens a lot less than you would think, in part because "copying" the innovation often will seriously upset an existing line of business or an existing way of doing things. That's what's so disruptive about disruptive innovation.

I have no idea if Blekko really will turn out to be disruptive in any way. But I find it quite fascinating that a big part of its attack on Google's marketshare is to enable a feature that would totally turn Google upside down in terms of how secretive it is about its algorithm and ranking. Google is often seen as a leader in the "open" technology world, but as we've pointed out before, while the company works hard to encourage others to be open, it can be incredibly closed itself. So it seems only fitting that its potential achilles' heel may be in the part of its business that it has kept quite closed.

Filed Under: openness, search
Companies: blekko, google

A Recommendation Is Not The Same As Corruption

from the a-mind-is-a-terrible-thing-to-waste dept

I recently wrote about what a joke the concept of "search neutrality" is, because, by their very nature, search results are supposed to be a ranking of what the site thinks the user wants most. It appears that the folks who are paid handsomely to push this "search neutrality" meme aren't too happy that someone has called them out for making something out of nothing. Scott Cleland, who is famously paid to blog anything that will make Google look bad, took issue with my statements, and sought to debunk what I had to say. To those pushing search "search neutrality," I would suggest coming up with better arguments (or finding someone who can actually argue logically), because Cleland's points don't cut it. Basically, he pretended I said something I didn't say and then ran with it.

Specifically, he pretends that when I say that search is inherently non-neutral, I mean that it's okay for Google to give higher ranking results to advertisers. I neither said nor implied any such thing. What I said (I thought clearly, but it's easier for Cleland to knock over a strawman) was that there's no such thing as "neutrality" in search, because any ranking is biased by what the search engine thinks is best. A "neutral" search engine doesn't exist, because it could never recommend any links. A recommendation is bias, but that's not the same as corruption or undue influence. It just means bias towards recommending what the search engine thinks is best.

However, Cleland jumps off on that to point out that even Google has said that paid search can impact quality. But Cleland is (in his usual manner) taking things totally out of context. The quotes he's discussing were about sites that mix paid search in their organic listings. To date, no one has shown convincing proof that Google has actually done this. If there were proof, then it would suggest that Google was doing something misleading, which would be an issue for the FTC to look at (as it has with other paid search listings in the past). However, that's hardly a call for "search neutrality." Mixing ads into the organic listings is one thing, but that's not what we were discussing. It's just the strawman Cleland tossed out there since he can't let that "search neutrality" thing die.

Second, he quotes Google saying that it doesn't "manipulate rankings to put partners higher," as proof that I'm wrong that the "whole point of search is to be biased." Again, he is arguing something I did not say. I said (again, clearly, though Cleland ignores it) that the bias is to put the results that it feels make the most sense out there. Saying that "the whole point of search is biased" does not mean "it's okay to mix paid search in organic results without telling people" as Cleland seems to think. It's almost as if Cleland doesn't even understand how search works.

Finally, Cleland links to a series of recent antitrust lawsuits against Google, which all seem to be based on the same ridiculous misinterpretations of how search engines work, and claims these are "documented charges of fraud, mis-representation, anti-competitive behavior." Except, they are no such thing. They are claims from a few small companies, bankrolled by Google's competitors, to misrepresent what Google is doing -- sort of like Cleland himself. None of those lawsuits have been decided, but I would be amazed if any of them came out with Google on the losing side. And, finally, on that, I never said that those lawsuits shouldn't move forward (as Cleland implies). I just pointed out that the claim of "search neutrality" being an issue is bogus. But none of these lawsuits are actually about "search neutrality." They're about claims that Google somehow took anti-competitive actions against these sites.

Most importantly, however, the reason people ask for "net neutrality" is because when you have an ISP, you have to go through that ISP to get anywhere. So if it's not "neutral" you can get locked into an experience you don't want. That's not the case with search. If you don't like your search experience, and if you find that the search engine of your choice has polluted its links with paid partners, then you can immediately go elsewhere. That's the key point, but it's never one that Cleland will address. Because he can't.

Filed Under: advertising, bias, scott cleland, search, search neutrality

Is Forcing IsoHunt To Block Search Terms A First Amendment Violation?

from the legal-questions... dept

Given similar rulings, and the judge's comments so far in the IsoHunt case, I find it quite unlikely that the company has any chance of getting out from under the injunction issued against it. However, IsoHunt's lawyer, Ira Rothken (who has been involved in a few similar lawsuits), is trying to make the case that the current injunction is way too broad and a violation of the First Amendment. The argument is that the injunction bars certain searches, telling Isohunt operator Gary Fung that he cannot allow searches for certain movie titles, such as Alice in Wonderland. But, Rothken points out, the movie studios don't own that name. They may own a particular movie under that name, but using that to block all searches on the name goes beyond what the law allows:

One issue concerns how Fung should remove searches from his three search engines: Isohunt, Torrentbox and Podtropolis. The Motion Picture Association of America, which brought the case, has sent keyword searches it wants removed, like the number 10, Alice in Wonderland and Dracula, Rothken said.

"One person's copyrighted Wizard of Oz is another person's public domain work," Rothken said in a brief telephone interview Tuesday. He said the movie studios should provide URLs or hashes, which would positively identify which search link should be removed.

"The motion picture studios do not have a monopoly on names on things. That is where the injunction is violating the First Amendment," he said.

I'm sure that copyright system defenders will brush this off as being a pointless exercise, but he's actually got a very reasonable point. Asking for blocks on names alone seems to go well beyond what the law is supposed to allow. It's yet another example of the difference between real copyright law and file sharing copyright law. Copyright law does not allow for a block based on just a name. But, apparently "file sharing copyright law" does. And that's a problem, if you actually believe in the rule of law and interpreting the law accurately.

Filed Under: first amendment, free speech, names, search
Companies: isohunt

Microsoft Discovers That Bribing People To Use Its Search Engine Didn't Work

from the money-isn't-everything dept

We recently had a discussion about the role of money as an incentive and how it often doesn't work the way people think it should (i.e., money often provides negative incentive -- the opposite of what you would think). I'm actually working on another post about that topic that I'll hopefully finish later today, but here's a quick one demonstrating that point in action.

Way back in 2005, Bill Gates suggested that in the end, Microsoft would be able to beat Google because it had a secret weapon: it could bribe users to use Microsoft instead of Google, by offering them a cut of the advertising revenue. It took a few years, but Microsoft finally turned on that "feature" on a limited basis in 2008, offering cashback for people who bought certain products after searching for them via Microsoft's search engine. Later that year, it expanded the program to regular search. What happened? Not that many people cared enough. Microsoft kept upping the ante, but most people didn't care. They were happy with their Google searches, and even if Microsoft was paying them to use its search engine, it wasn't enough. Well, except for people who figured out how to game the system. But that's not who Microsoft was targeting.

Microsoft has now admitted the program was a failure and is shutting it down. In a blog post about the decision, Microsoft admits that people just weren't that interested:

In lots of ways, this was a great feature -- we had over a thousand merchant partners delivering great offers to customers and seeing great ROI on their campaigns, and we were taking some of the advertising revenue and giving it back to customers. But after a couple of years of trying, we did not see the broad adoption that we had hoped for.

Microsoft admission is quite open and honest, which is actually pretty cool. They don't sugarcoat it. They thought this was something people would like -- as the basic belief that monetary rewards drives activity would suggest -- but found that, in practice, it does not work at all.

Filed Under: bribes, search
Companies: google, microsoft

EU Politicians Tricked Into Supporting Data Retention On Search Terms... 'For The Children'

from the privacy-vs.-data-retention dept

Slashdot points us to a story about how many EU Parliament Members signed on to a declaration supporting the creation of an "early warning system to combat sexual child abuse." Sounds good, right? But the devil is very much in the details. Christian Engstrom notes that many of the MEPs who signed on didn't realize that part of the declaration was to extend already controversial data retention laws to search engines, meaning that Google would need to store your search results far beyond what they currently do, just in case law enforcement wants to go trolling through your search history.

Of course, this seems doubly ironic since so many European countries are up in arms over Google collecting data via open WiFi networks. So, which is it, Europe? Do you want Google not to collect data, or do you want Google to save data for years in case police want to snoop through it? No, the two situations are not identical, but there is a clear conflict between EU privacy rules and EU data retention rules. On the one hand, they give Europeans the ability to supposedly take control over their private info, including requiring companies to delete it. On the other hand, they demand that companies store data in case the police would like to look through it.

Filed Under: data retention, eu, privacy, search