ACS:Law Keeps Sending Out More Threat Letters -- Condemned By Politicians, ISPs And General Common Sense

UK's Times Online Starts Blocking Aggregators Hours After Aggregators Win Copyright Tribunal Ruling Against Newspapers

from the donkeys-arguing-against-the-wheel dept

Wed, Mar 17th 2010 7:22pm — Mike Masnick

There's been something of a battle going on in the UK over news aggregators. Obviously, we've all heard about the various threats by companies like News Corp. in the US to sue Google over its Google News product, but a lot of this has already been playing out on a smaller scale in the UK. Last year we wrote about newspapers in the UK threatening aggregators like NewsNow, leading some to start blocking NewsNow crawlers. This is silly in the extreme. These aggregators offer links to the news. The "issue" with NewsNow is that it sells this as a service to companies -- and the newspapers claim they deserve a cut. Note that NewsNow provides just a link and a headline and the tiniest of blurbs. It's much less than even Google News provides. The newspapers seem to think that no one can profit from advertising their own stories unless they get a direct cut.

In fact, last year the NLA (Newspaper Licensing Association) in the UK decided to start charging all such services just for linking. This is, of course, ridiculous. One of the largest services of this type is called Meltwater News, and it decided to protest this ridiculous license on linking. It was joined in this effort by the Public Relations Consultants Association (PRCA), who noted that there is no copyright on headlines and links -- and the NLA's license amounted to an illegal tax. The NLA responded by saying that Meltwater and PRCA had no right to protest these licenses.

Earlier this week, however, the Copyright Tribunal in the UK ruled in favor of the PRCA and Meltwater in protesting these new licenses, and it ordered the NLA to pay the costs of both organizations. Now there will be a full trial concerning the legality of the licenses.

What's interesting, however, is that hours after this decision came out, the Times Online in the UK just so happened to update its robots.txt file to block Meltwater (along with NewsNow, who had already been blocked). Basically, it was a quiet threat: if you don't pay, we'll block you.

The newspapers are walking a very thin line here. They're trying to charge for the most basic element of the web: linking and sharing links with others. I would imagine that if they actually win this fight, they're going to end up regretting it even more -- because if they start linking to other sites themselves, how long will it take before those linked sites start demanding money back from the newspapers as well. It's an incredibly short-sited view that a newspaper takes to think that others must pay you to promote you.

Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.

Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.

While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.

–The Techdirt Team

Filed Under: aggregators, copyright, links, newspapers, robots.txt, uk
Companies: meltwater news, newsnow, nla, prca

30 Comments

If you liked this post, you may also be interested in...

Reader Comments

Subscribe: RSS

View by: Time | Thread

Anonymous Coward, 17 Mar 2010 @ 7:32pm

Yeah, telling people they can't link to a story is about as smart a move as asking the police to arrest someone for telling their friends they should go buy your movie/music/fruit/whatever.
[ link to this | view in thread ]
Anonymous Coward, 17 Mar 2010 @ 7:35pm

Wow! Copyright sure fuels innovation and creative thinking. Don't newspapers deal in facts? About reality? Objective and factual accounts of reality?

And their copyright is so important that one cannot even copy a headline and a short blurb and even provide a link to the original?

Copyright sure protects the progress of the useful arts.
[ link to this | view in thread ]
Anonymous Coward, 17 Mar 2010 @ 7:39pm

Re:
THEM STEALERS ARE DESTROYING OUR FACT-GATHERING INDUSTRY!!!
[ link to this | view in thread ]
Anonymous Coward, 17 Mar 2010 @ 8:07pm

"The newspapers seem to think that no one can profit from advertising their own stories unless they get a direct cut."

Exactly. What is so hard to understand about that?
[ link to this | view in thread ]
big al, 17 Mar 2010 @ 8:12pm

hummmmmm guess the local newsstand will be next.. sells other mags and papers and makes a (hopeful)profit.. soooooooooo...

"cut off their noses to spite their face"
[ link to this | view in thread ]
ElSteevo (profile), 17 Mar 2010 @ 8:26pm

The problem is that newspapers will soon be non-existent; the number of print copies is plummeting, the online pay model won't work, and ad revenue is drying up quickly.

Pretty soon this won't be a problem, as corporeal newspapers disappear, and are replaced by bloggers who fact check even less than current reporters.
[ link to this | view in thread ]
Igor Zevaka, 17 Mar 2010 @ 8:33pm

robots.txt?
I always wondered if there is any enforcable weight in robots.txt. If the web site blocks NewNow and Meltwater just by the way of robots.txt there is nothing stopping the aggregators to disregard those directives and just keep crawling.

This will undoubtedly result in an arms race where the crawler would either need to impersonate another bot or not anounce itself at all. Such race will end when the newspaper starts blocking IPs of suspected "unauthorised" bots.

Whilst it would be fun to watch it won't be a viable long term strategy for the aggregators. The next legal argument then will be the question if the newspaper is allowed to explicitly block a particular service from crawling its site, which is essentially restricting who can "look" at their content.

Slippery slope indeed.
[ link to this | view in thread ]
Anonymous Coward, 17 Mar 2010 @ 9:02pm

I say, "Let them eat cake!" Let them block the aggregators that send them traffic. It just means that they will become increasingly more irrelevant and go out of business that much faster.

If you can't be a good example, at least be a horrible warning!
[ link to this | view in thread ]
diskmaster, 17 Mar 2010 @ 9:07pm

robots.txt
Then the aggregators block (remove) the newspapers and where will the newspapers be?
[ link to this | view in thread ]
diskmaster, 17 Mar 2010 @ 9:07pm

robots.txt
Then the aggregators block (remove) the newspapers and where will the newspapers be?
[ link to this | view in thread ]
Flakey, 17 Mar 2010 @ 9:15pm

Blocking links?
Now ain't this a riot. It's like they are being spiteful to their nose to prevent the eyeball from bringing them business.

There's many a way to ruin a business. This has got to be a really stupid decision that does not take into account the idea that eyeballs are money when it comes to ads.

It's not like the aggregates are stealing their news, crap they are providing teasers and where the story can be found.

@7
There are crawlers that don't play fair and totally ignore robot.txt. If you don't ban them both by name and IP they will be back again and again. Many of them work on IP ranges or selected multiple IPs. If you ban the current IP range, they come back again with a different IP not in the last range. If you want them off the site, robot.txt isn't the way you will have to go.
[ link to this | view in thread ]
Peter, 17 Mar 2010 @ 10:05pm

Just comply
All everybody else need to do is remove them from their searches. No google links, no bing links, no yahoo links etc. Soon after a couple of months, they will realize that nobody know of their existence and its all gone.
[ link to this | view in thread ]
Storm (profile), 18 Mar 2010 @ 2:13am

It's a joke
I'm a long-time Newsnow reader. The papers are missing the big picture here... sites such as Newsnow DIRECT traffic to the papers.... those people see the ads on those news sites and are exposed to the brand of the paper constantly. Sure, newsnow may make a bit of money, but the papers stand to make alot more by impressing new viewers every day. I would never have discovered some news sites (commercial and blogs) if it hadn't been for newsnow.
[ link to this | view in thread ]
Michial Thompson, 18 Mar 2010 @ 2:20am

Laws of unintended consequences
The law of unintended consequences will solve this one pretty simply. Stop linking to them in any form. Their search engine results will fall by the wayside and their hit counts will drop off and eventually so will their advertising revenue etc...

Pretty simple, in no time they will either be suing the search engines trying to force them to move their stuff to the top, or they will be paying people to link to them through banner ads, or begging people to link to them however they want.

Either way simply giving them what they wish for is the solution rather than standing behind fair use and forcing your will upon them.

But then again in little mikee's world it's all about doing whatever in the hell you feel like and totally disregard the creators wishes.
[ link to this | view in thread ]
PaulT (profile), 18 Mar 2010 @ 3:16am

"we've all heard about the various threats by companies like News Corp. in the US to sue Google over its Google News product, but a lot of this has already been playing out on a smaller scale in the UK"

It's worth noting that The Times is indeed owned by... Rupert Murdoch's News Corp.
[ link to this | view in thread ]
:), 18 Mar 2010 @ 4:38am

Newspapers are poisoning their environment and will kill themselves eventually if these trends continue.

I won't miss them that much.

I find myself reading less and less "news" and more and more blogs lately.
[ link to this | view in thread ]
FatGiant (profile), 18 Mar 2010 @ 4:48am

I remember a time where we lived by the rule of link back.

Now, we are moving to the rule of block back.

Nice.

Why not create the "unfollow Monday" on twitter?
[ link to this | view in thread ]
jsl4980 (profile), 18 Mar 2010 @ 6:39am

I use either Google News or Google Reader to get all of my news and every link they show me says "all 1000 related" if this site stops crawlers my link will say "all 999 related" absolutely no difference to me, less potential traffic for them.
[ link to this | view in thread ]
Beta, 18 Mar 2010 @ 7:29am

"...or I'll take my baseball and go home."
"...Hours after this decision came out, the Times Online in the UK just so happened to update its robots.txt file to block Meltwater (along with NewsNow, who had already been blocked). Basically, it was a quiet threat: if you don't pay, we'll block you."

The times has every right to do this. As I understand it, the robots.txt protocol is a matter of convention and courtesy; a robot could ignore the file, and a site could use other methods to break links, but it doesn't really matter. If the Times wants to block links, they can. If they want to offer linking privileges for money, they can. It's their site. They're being foolish and childish, not evil.
[ link to this | view in thread ]
Chargone (profile), 18 Mar 2010 @ 7:37am

Re:
.... negative numbers are imaginary, just fyi.
[ link to this | view in thread ]
Chargone (profile), 18 Mar 2010 @ 7:37am

Re:
.... negative numbers are imaginary, just fyi.
[ link to this | view in thread ]
Tyanna, 18 Mar 2010 @ 8:45am

"...because if they start linking to other sites themselves, how long will it take before those linked sites start demanding money back from the newspapers as well."

Haven't we already discussed many times about how they don't link, and treat every story as their own? I don't think this will be too much of a problem for them ;)
[ link to this | view in thread ]
The Infamous Joe (profile), 18 Mar 2010 @ 9:30am

Re:
The problem is that newspapers will soon be non-existent; the number of print copies is plummeting, the online pay model won't work, and ad revenue is drying up quickly.

I'm not sure I see the problem. In unrelated news, radio shows were rendered obsolete by the television.

Pretty soon this won't be a problem, as corporeal newspapers disappear, and are replaced by bloggers who fact check even less than current reporters.

When was the last time your local paper reprinted a paper correcting their mistakes? Never, for me, anyway. How many times does Techdirt correct their mistakes? Every time.

I'll take crowd-sourced fact checking to what newspapers give me now, any day of the week, thank you very much.
[ link to this | view in thread ]
The Infamous Joe (profile), 18 Mar 2010 @ 9:46am

Re: Laws of unintended consequences
Holy cow, dude. I really don't get you.

You made a reasonable and seemingly well-thought post that I agree with completely, and then at the very end you throw an irrelevant and unprovoked attack on Mike that has no basis on reality. I don't see anywhere where Mike says that suggests the answer is to go against the Newspaper's wishes. Are we reading the same article?

Allow me to distill it, since I'm not sure you bothered to read it.

Aggregators link to newspaper. Newspaper require a license to link, attempts to charge aggregators. Aggregators refuse to pay and claim that linking with a blurb is perfectly reasonable. Courts agree with Aggregators. Now newspapers use robots.txt to block aggregators.

Mike surmises that this is a bad idea, not only does it make no business sense when your business is based solely on how many people read your news, but it is bound to come back and bite the newspapers in the ass when they attempt to link elsewhere.

Please, *please* point out where your final sentence fits in.

On an off-topic side note: You come off as a complete douche when you attack Mike. (Mikee? Really?) Do your children read what you write here, and how do you think they'd respond? Would you be proud for them to read it?
[ link to this | view in thread ]
Anonymous Coward, 18 Mar 2010 @ 10:29am

Re: Re: Laws of unintended consequences
That's our little michiall!
[ link to this | view in thread ]
Anonymous Coward, 18 Mar 2010 @ 10:20pm

Re: Re: Laws of unintended consequences
Children typically don't have children of their own yet, you know.
[ link to this | view in thread ]
leizi018 (profile), 6 Aug 2010 @ 11:22pm

Polo Ralph Lauren
[ link to this | view in thread ]
Anonymous Coward, 7 Dec 2010 @ 4:39am

really funny...Copyright, eh?
What about a PR person sending a press release to a newspaper, which prints the exact release or parts of it and the PR person using an aggregator to find out where the press release has been published?
Is it the PR person holding the copyright (as he created the text) or the newspaper?
And why should I as a PR person pay a copyright fee for MY OWN texts just because I want to check on the results of my own work?
Don't get it...
[ link to this | view in thread ]
Jules, 10 Feb 2011 @ 12:33pm

Re:
Because the newspaper is doing you a service on which your business model is based. Without the newspapers running your story, you're out of work.
[ link to this | view in thread ]
Anonymous Coward, 5 Apr 2011 @ 6:17am

These services help drive traffic to the papers sites, it is not smart for news papers to block them. These services have thousands of customers. Newspapers have an old business model and did not adapt fast enough, now they are trying to punish everyone else. desperate moves are never good ones.
[ link to this | view in thread ]