UK Political Party Tries To Dump 10 Years Of Speeches Down The Memory Hole

from the because-that-ALWAYS-works...-ALWAYS dept

Thu, Nov 14th 2013 5:51am — Tim Cushing

Every so often a public figure will come to the dubious conclusion that the past can be erased. This was a difficult proposition even before the advent of the internet. These days, it's nearly impossible. But long odds rarely deter the particularly inspired… or particularly stupid.

Some abuse the easily-abusable laws in European countries to generate memory holes. Max Mosely has been fruitlessly pursuing the removal of so-called "not actually a Nazi orgy" photos for years. Others simply blunder around, issuing baseless legal threats and questionable DMCA notices. Others, like the UK Conservative Party, do their own dirty work.

Being willing to wipe your own collective memory takes a special kind of bravery, the kind often associated with reckless acts shortly preceded by the phrase, "Hold my beer."

pixelpusher220 was the first to send in the ComputerWeekly story which details the efforts the UK's Conservative Party recently made to eradicate an entire decade's worth of speeches from the internet.

The Conservative Party has attempted to erase a 10-year backlog of speeches from the internet, including pledges for a new kind of transparent politics the prime minister and chancellor made when they were campaigning for election.

Prime minister David Cameron and chancellor George Osborne campaigned on a promise to democratise information held by those in power, so people could hold them to account. They wanted to use the internet transform politics.

But the Conservative Party has removed the archive from its public facing website, erasing records of speeches and press releases going back to the year 2000 and up until it was elected in May 2010.

The Conservative Party did more than simply delete the speeches from its site. It also blocked out Google and the Internet Archive using an extensive addition to its robots.txt. This is just a small excerpt of conservatives.com's bot blocking additions.

Disallow: /News/News_stories/2000/
Disallow: /News/News_stories/2001/
Disallow: /News/News_stories/2002/
Disallow: /News/News_stories/2003/
Disallow: /News/News_stories/2004/
Disallow: /News/News_stories/2005/
Disallow: /News/News_stories/2006/
Disallow: /News/News_stories/2007/
Disallow: /News/News_stories/2008/
Disallow: /News/News_stories/2009/
Disallow: /News/News_stories/2010/01/
Disallow: /News/News_stories/2010/02/
Disallow: /News/News_stories/2010/03/
Disallow: /News/News_stories/2010/04/
Disallow: /News/News_stories/2010/05/
Disallow: /News/Speeches/2000/
Disallow: /News/Speeches/2001/
Disallow: /News/Speeches/2002/
Disallow: /News/Speeches/2003/
Disallow: /News/Speeches/2004/
Disallow: /News/Speeches/2005/
Disallow: /News/Speeches/2006/
Disallow: /News/Speeches/2007/
Disallow: /News/Speeches/2008/
Disallow: /News/Speeches/2009/
Disallow: /News/Speeches/2010/01/
Disallow: /News/Speeches/2010/02/
Disallow: /News/Speeches/2010/03/
Disallow: /News/Speeches/2010/04/
Disallow: /News/Speeches/2010/05/
Disallow: /News/Articles/2000/
Disallow: /News/Articles/2001/
Disallow: /News/Articles/2002/
Disallow: /News/Articles/2003/
Disallow: /News/Articles/2004/
Disallow: /News/Articles/2005/
Disallow: /News/Articles/2006/
Disallow: /News/Articles/2007/
Disallow: /News/Articles/2008/
Disallow: /News/Articles/2009/
Disallow: /News/Articles/2010/01/
Disallow: /News/Articles/2010/02/
Disallow: /News/Articles/2010/03/
Disallow: /News/Articles/2010/04/
Disallow: /News/Articles/2010/05/

So, how did it get the Internet Archive to remove its historical collection, something ComputerWeekly writer Mark Ballard likens to "sending Men in Black to strip history books from a public library and burn them in the car park?"

Well, apparently the Internet Archive treats changes to robots.txt files as retroactively applicable. Once the bot blocker informed IA it was no longer welcome to crawl these pages, it erased the corresponding archives as a "matter of courtesy."

By making this change, the Conservative Party was able to eliminate 1,158 "snapshots" the Archive had gathered over the last 14 years, a rather breathtaking eradication accomplished without ever having to strong arm internet historians or stare down Google directly.

The Conservative Party has offered no comment on the slash-and-burn of its own history, simply saying it has passed along the query to its "website guy."

Now that the speeches (and the Archives) have been removed, conservatives.com's robots.txt has been trimmed down to something more manageable.

User-agent: *
Disallow: /XMLGateway/
Disallow: /sitecore/
Disallow: /users/
Disallow: /flash/
Disallow: /pdf/
Disallow: /layouts/site.aspx
Disallow: /Activist_centre/
Disallow: /News/Blogs.aspx
Disallow: /News/Blogs/
Disallow: /sitecore/
Disallow: /Get_involved/Join/Friend.aspx
Disallow: /Get_involved/Join/Member.aspx
Disallow: /Get_involved/Join/Youth.aspx
Sitemap: http://www.conservatives.com/xmlFeeds/GoogleSitemap.aspx

A search through its xml sitemap confirms that nothing remains of the pre-2010 speeches. The earliest speech listed in the xml file is from June of 2010. Users browsing the site will be hard pressed to find any speeches earlier than January of 2013, however. Searching through the sitemap will uncover direct links to earlier speeches but clicking the "Archive" button to view older speeches automatically limits results to 2013. Here's the shady URL the "Archive" button leads to:

http://www.conservatives.com/News/SpeechList.aspx?SearchType=NewsDate&SearchTerm=130101-131231

So, why are these speeches being buried? Perhaps it has something to do with the Cameron's promises of government transparency and accountability made while campaigning, something he increasingly lost interest in once in power.

"Above all, the power for anyone to hold to account those who in the past might have had a monopoly of power - whether it's government, big business, or the traditional media," said Cameron, who was then campaigning for power as leader of the Conservative opposition.

Cameron was going to make sure the information revolution would hold people like prime ministers to account, he said another speech on 11 October 2007, at the Google Zeitgeist Conference in San Francisco.

"It's clear to me that political leaders will have to learn to let go," he said then. "Let go of the information that we've guarded so jealously."

Transparency would make public officials accountable to the people, said Cameron then. He was riding at the front of the wave that would wash us into a new world, and a new age.

Like many politicians, transparency and accountability sound like great ideas when you're lapping up applause (and votes) and hoping to stick it to your legislative adversaries. But it swiftly loses its luster the first time it's applied to you and your activities. Then it's back to the old ways that have "worked" for years. Obfuscation, opacity and a growing tendency to view your constituents as the enemy swiftly replace the campaigning ideals.

Years down the road, after many years at the helm, this viewpoint realignment culminates in running a decade's-worth of empty promises through the internet shredder in hopes of trimming down the number of irate citizens using your own words against you.

Unfortunately for those manning the shredder, they seem to have missed another set of archives, one located in their own backyard. A commenter at ComputerWeekly points out that the Conservative Party site has been archived by the British Library since 2004, and many of those supposedly vanished speeches are only a few clicks away.

All this effort will do for the Conservative Party is make it look worse. There's zero net gain to be had here. Nothing completely vanishes from the net and even if the Internet Archive may err on the side of courtesy in its efforts, others will be saving, securing and stashing the same documents and webpages certain entities wish to remove from the public eye. The harder they try, the more likely they are to fail.

Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.

Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.

While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.

–The Techdirt Team

Filed Under: archive, conservative party, deleting history, history, internet, speeches, uk

32 Comments

If you liked this post, you may also be interested in...

Reader Comments

Subscribe: RSS

View by: Time | Thread

That One Guy (profile), 14 Nov 2013 @ 3:58am

Assuming it hasn't already be done, I hope someone, ideally multiple someones, saves several backups of the remaining archive from the British Library, as if they've gone this far to remove any 'inconvenient history', it wouldn't surprise me if in a few days/weeks it's discovered that the library archives has been hit by a 'virus' that just so happened to wipe out the remaining backups of that archive.
[ link to this | view in thread ]
BentFranklin (profile), 14 Nov 2013 @ 4:54am

I'm surprised the Archive has a retroactive deletion feature. Isn't the purpose of an archive to be something different from a mirror?
[ link to this | view in thread ]
razer145, 14 Nov 2013 @ 5:59am

An Appropriate Quote...
"The more you tighten your grip, the more webpages will slip through your fingers"
[ link to this | view in thread ]
Anonymous Coward, 14 Nov 2013 @ 6:15am

Re:
I doubt they would be that subtle. It would either be over copyright, or "because they asked us to."
[ link to this | view in thread ]
Anonymous Coward, 14 Nov 2013 @ 6:16am

Re:
Indeed. That's not erring on the side of courtesy, that's erring on the side of censorship. Once something's been archived, it should remain archived.
[ link to this | view in thread ]
Anonymous Coward, 14 Nov 2013 @ 6:20am

So shortly after the fire affecting the Internet Archive site? one could almost conclude the Tories were doubling down.
[ link to this | view in thread ]
Anonymous Coward, 14 Nov 2013 @ 6:21am

Re: Re:
It's legal because we say it's legal.

W.Hague
[ link to this | view in thread ]
That One Guy (profile), 14 Nov 2013 @ 6:22am

Re: Re:
Given who exactly is trying to erase the past here, it honestly wouldn't surprise me if they tried the good old 'Because terrorists!' line to try and wipe the records clean.

After all these are records which could be used to 'influence' politicians(by pointing out their hypocrisy) if made public, which obviously makes the records, and anyone who would wish to access them, terrorist materials and terrorists respectively.
[ link to this | view in thread ]
That One Guy (profile), 14 Nov 2013 @ 6:24am

Re:
Yeah, the fact that the Archive will actually go back and delete old entries, to make sure it matches the newer ones, is a really stupid idea if the point is to archive how things were, not just how they are now.
[ link to this | view in thread ]
This comment has been flagged by the community. Click here to show it

out_of_the_blue, 14 Nov 2013 @ 6:27am

Well, if Internet Archive didn't comply, they'd have a fire...
Coincidence? There is no such thing in politics. If you don't see linkage here if only by proximity in time, then you're blind.

In any case, when deleting/revising history becomes explicit practice of the State, a few hard drives here and there are easy to dispose of. If can be found by the rabble to use, can as easily be found by the State. -- But the main problem is as Winston Smith had in "1984": even with undeniable proof in hand, getting the proles to recognize it and take action is just nearly impossible.

2nd: who says Google respects robots.txt? Prove that. My bet is they index everything available everywhere.
[ link to this | view in thread ]
PaulT (profile), 14 Nov 2013 @ 6:27am

Re:
That was my thought too. The original CW article words it a little differently:

"An administrator at the Internet Archive HQ in San Francisco said its guidance for lawyers explained the mechanism. That was that if a website, like Conservatives.com, put up a robot blocker, those pages it blocked would simply be erased from the record as a matter of etiquette."

But, the link provided (http://archive.org/legal/faq.php) doesn't seem to answer that question to my eyes.

I can't help but think there must be something more than a robots.txt change required for removing pages already stored in the archive, but who knows. I can understand that kind of move for legal protections, etc. but it does rather defeat the purpose of it as an archive if this is all there is to it.
[ link to this | view in thread ]
Call me Al, 14 Nov 2013 @ 6:31am

Worth noting...
Apparently the Labour party (the opposition) website also has no content from before 2010 on their site. So this isn't an isolated occurrence.
[ link to this | view in thread ]
Sam, 14 Nov 2013 @ 7:06am

Luckily they are all archived here http://www.britishpoliticalspeech.org/speech-archive.htm
[ link to this | view in thread ]
Zakida Paul (profile), 14 Nov 2013 @ 7:14am

Two years to a general election, the Tories don't want the unwashed masses remembering all those pesky broken promises. Pity they are all available elsewhere.

Politicians really have no clue what the Internet is or how it works.
[ link to this | view in thread ]
Shawn (profile), 14 Nov 2013 @ 7:30am

Re: Well, if Internet Archive didn't comply, they'd have a fire...
2nd: who says Google respects robots.txt? Prove that. My bet is they index everything available everywhere.

My web server logs say Google respects robots.txt
[ link to this | view in thread ]
Anonymous Coward, 14 Nov 2013 @ 7:32am

Well, imagine if US politicians started to do the same thing, what embarrassing things would they erase?

Anti Immigrant Politician: "I never called you immigrants lazy moochers trying to steal our jobs and country, and you can't prove it anymore!"

Mitt Romney on his many flip flops: "I never flip flopped on any issues, and I've always been against Romneycare that many of you think is just like Obamacare, and you can't prove it anymore!"

Todd Akin on his legitimate rape gaffe: "I never said legitimate rape, in fact I never said anything about rape, and you can't prove it anymore!"

Obama on his you can keep it promise: "I never said you could keep your insurance, in fact I never made ANY promises or speeches about Obamacare, and you can't prove it anymore!"
[ link to this | view in thread ]
Anonymous Coward, 14 Nov 2013 @ 7:32am

Re:
Politicians really have no clue what the Internet is or how it works.

I think its a case that they know how the majority of people work, they will look at the parties site and not know ho to do an effective internet search. This move could be quite effective because of this.
[ link to this | view in thread ]
OldMugwump (profile), 14 Nov 2013 @ 7:54am

Re: Re: Well, if Internet Archive didn't comply, they'd have a fire...
Mine too.
[ link to this | view in thread ]
Anonymous Coward, 14 Nov 2013 @ 8:31am

I have to wonder...
If the Internet Archive isn't using a logical delete here. Since the primary purpose of an archive is to preserve content, it would make sense for them to do so on retroactive actions. To the outside user there would appear to be no difference.
[ link to this | view in thread ]
Anonymous Coward, 14 Nov 2013 @ 9:41am

and as usual, the UK public will sit back and do nothing until it's past the point of no return and we see what this bunch of lying clowns have been after all along! and that sure as hell is no 'Democratic Society'! that's obvious from the internet censorship, lack of admission and the lack of results from the Spying Investigations!! it's like Nazi Germany staring over again!
[ link to this | view in thread ]
Anonymous Coward, 14 Nov 2013 @ 9:53am

Newspeak
The UK has such a fucking hard-on for 1984.
[ link to this | view in thread ]
Anonymous Coward, 14 Nov 2013 @ 10:00am

Re:
This isn't so much "doubling down on stupid" as 'keep calm and straight ahead'
[ link to this | view in thread ]
Bobbins (profile), 14 Nov 2013 @ 11:09am

Time for a game of guess the politician, today's clue comes from a speech as recent as 2007.

"Politics is a trust.

In a representative democracy, politicians hold power in trust from the people.

It is not our power but yours that we exercise.

We exercise it on your behalf � and we are accountable to you for how we use it.

Accountability means more than standing for re-election once every five years.

It means transparency during your term of office too � the obligation to explain what you are doing openly and honestly.

When politicians betray the trust they have received from the public, the public loses trust in them."
[ link to this | view in thread ]
Not an Electronic Rodent (profile), 14 Nov 2013 @ 11:11am

Re: Newspeak
The UK has such a fucking hard-on for 1984.
Yeah, this decade we're trying to beat the US for the "Most fascist country pretending to be a democracy" award. A bold move perhaps, but we got some game it seems...
[ link to this | view in thread ]
yankinwaoz (profile), 14 Nov 2013 @ 11:28am

But it is THEIR content
I don't understand the hub bub.

This is a website that belongs to a political party. They can do with it what they wish... right? Is there a law in the UK that requires a party to maintain an online history of every utterance made?

Perhaps they looks at their stats and noticed that no one, or very few people, was retrieving their archive pages. So they decided to cull them from the site rather than paying for the hosting of that much data.

Maybe the robots.txt entries was a just a lazy way of preventing a bunch of 404's from being logged by an SE spider trying to get to content that is no longer there. I suspect that they didn't even realize that this also purges content from an internet archive service.

It is the public interest to save political speeches. I'm sure there is some sort of .gov.uk site, such as a national library, that is a better suited as the custodian of the UK's political history. One that would curate content from all political parties and activist.
[ link to this | view in thread ]
John Fenderson (profile), 14 Nov 2013 @ 12:39pm

Re: But it is THEIR content
Nobody is saying that they don't have the right to do this. I think what people are saying is that doing this makes them look bad.
[ link to this | view in thread ]
Anonymous Coward, 14 Nov 2013 @ 2:10pm

Re:
>We exercise it on your behalf

I stopped reading at that point 'cause I knew the rest of it was going to be BS.
[ link to this | view in thread ]
btr1701 (profile), 14 Nov 2013 @ 3:49pm

Re: An Appropriate Quote...
I bet Obama really wishes he could do this same thing with the dozens of speeches were he said:

"This is a guarantee we're going to make to the American public. If you like your insurance plan, you can keep it. No one will take that away from you. Period. If you like your doctor, you can keep your doctor. Period."

(He needed to put that 'period' in there for that little extra bit of dishonesty.)
[ link to this | view in thread ]
ckhung (profile), 15 Nov 2013 @ 4:34am

at least one can still search "conservative manifesto 2010"
What a shame! I was so excited about their "conservative manifesto 2010" (where they mentioned open source twice) that I blogged about it in Chinese back then. That post now looks so ironical. Fortunately one can still google "conservative manifesto 2010" and find a copy of that manifesto (at their own blog?) (of which I made a backup copy just in case they read techdirt :-) and decide to delete it), an annotated version at the Guardian, and a copy at slideshare. Is it correct to say that this manifesto contains pretty much the gist of what they want people to forget?
[ link to this | view in thread ]
Headbhang, 15 Nov 2013 @ 6:32am

Minitrue
Tory Airstrip One, not content with its rather successful Miniluv implementation, now tries its hand at some Minitrue endeavours, and bungles it.

The Internet might enable surveillance beyond 1984's dreams, but fortunately makes it very difficult for a Ministry of Truth to really succeed.
[ link to this | view in thread ]
Francisco, 16 Nov 2013 @ 2:21am

Hardly new for UK politicians
Here are some stories to show that this is not exactly new behaviour for British politicians (or, indeed, the Conservative party):

http://news.bbc.co.uk/1/hi/uk_politics/7884121.stm

I also remember hearing, in the 90s, that the Conservative Party edited the video for a party conference so that it looked like certain people got standing ovations when they didn't, etc.

The Labour Party were very good at putting on a smile and saying "We changed our minds". However, they were at the forefront of trying to keep information secret (even overriding the Information Commissioner).
[ link to this | view in thread ]
Francisco, 16 Nov 2013 @ 2:28am

Re: Hardly new for UK politicians
The Labour Party employed tricks like releasing "bad" information at a time when people were not paying attention so that, months later, if somebody expresses surprise the can turn around and say "We did tell you about it." Here's one example:

http://news.bbc.co.uk/1/hi/uk/1588844.stm
[ link to this | view in thread ]