A Conversation About Video Game Preservation In The Gaming Industry Is Long, Long Overdue

from the we-need-some-speed dept

There has been quite an uptick recently when it comes to the conversation around video game preservation. There are probably several reasons for this. First and most notably, the confluence of the trend toward the gaming public primarily purchasing digital games rather than shiny disks, and the emergence of the latest generation of video game consoles has brought the question of what happens to older games into stark relief for many in the gaming public. Second, America has been in something of a love affair over the last decade or so with all things "retro". And, finally, the concept of video games as works of creative art, rather than wastes of time to be sneered at, has found firm purchase within our society. All of this has combined to make the public much, much more interested in preserving antiquated video games. And, frankly, very disappointed at how often the gaming industry doesn't take preservation at all seriously.

Well, it's happening again. In the near future, Electronic Arts will be shutting down the servers and online portions of several Need For Speed games.

Today, via Reddit (while most the English-speaking world is on a holiday), it’s been announced that Need For Speed: Carbon, Need For Speed: Undercover, Need For Speed: Shift, Shift 2: Unleashed and Need For Speed: The Run will be “retired”. Which I suppose is an apposite word, given they’ll be limping off the tracks as they leave digital storefronts today, and their servers switched off come the end of August.

The reasons given are the usual: that maintaining servers for the few remaining players is prohibitively expensive, and hey, look, they’ve released loads of (astoundingly poor) NFS games since then, so you could buy those instead!

Note a couple of things on this. While the offline portions of the games will still be playable for those who have already purchased them, new buyers will no longer have a place to legitimately buy them. Also, while there is a single player component, a big draw of the games was and continues to be the ability to race against friends online. Finally, note that this announcement comes with absolutely no plan to make the game or online play available in any other way.

Which is where the preservation conversation comes in. Once again, we have a game publisher that enjoys full rights controls over its property choosing to simply deprive the public of some or all of that property. A more perfect antithesis of the concepts of the public domain and copyright law generally probably can't be found. While EA explained away its decision to "retire" these games as the expense that comes along with maintaining servers and backend infrastructure for relatively fewer players -- along with a suggestion that disappointed gamers simply by new games in the Need For Speed series -- it's not like there aren't steps it could take to play nicely with its fans if it wanted to.

It’s always this way. “Shrug! What else could we do?!” Well, here are some other things they could do:

They could release the source code for the 10-15 year old games, and allow others to continue their development in the public domain. They could release the server code for the games, to allow enthusiasts to continue to host the few dedicated players remaining. They could offer to upgrade players to one of the many NFS games of the 2010s (although this may be crueller than just nothing at all). They could recognise that last year EA made a revenue of $5.5bn, and it’s likely they could just about afford to leave the servers on with minimal maintenance, without taking too big of a hit

Delisting them from stores just seems... petty! Sure, they don’t offer all the available features when the servers are off, but come on. Quarter the prices—hell, be decent enough to make them free—and let people buy them as single-player artefacts of the past.

In other words, either preserve the games for fans yourselves, EA, or let the fans do it for you. Either option is viable. But simply switching off the servers and making the games no longer available for purchase at any price is probably not so much a petty thing to do as it is a callous thing to do.

And when a company starts acting callous towards its dedicated fans, well, that's not a good plan for building either goodwill or more buying fans.

Filed Under: archive, need for speed, preservation, video games
Companies: electronic arts

Sony Ends Support For Playstation Store For PSP, PS3, and Vita

from the preservation-mode dept

A few days ago, we discussed the problem of video game preservation as a matter of art when developers, publishers, or platforms shut down certain services. The point of the post was fairly wide-reaching, with the focus being on the idea that game developers in the PC space should embrace the habit of releasing their source codes for games and let the gamer ecosystem take over. While that's an idea I find extremely interesting, it doesn't really apply to the console gaming space. And it was rumors of the shutdown of certain Sony PlayStation stores that kickstarted the whole conversation.

Well, that rumored shutdown is now reality. And it's roughly half as bad for the purposes of game preservation as was expected.

Today, Sony updated the important notice section of its official site with information about the upcoming shutdowns. While the stores will be closed later in the year, Sony explained that players will still be able to redownload content that they own after the shutdown date. Folks will also still be able to redeem PS Plus and game vouchers, though wallet vouchers will no longer work on these stores after they close.

So, that's the top line summary of what's changing, with the headline being that people will still be able to re-download games that they had previously purchased. That's obviously a good thing. But where we dive right back into the preservation question is here.

You will no longer be able to purchase PS3, PS Vita, and PSP digital content, including games and video content.

You will no longer be able to make in-game purchases through games on PS3, PS Vita, and PSP.

So new purchases are not available for games. Whether said games will be made available anywhere else is an open question. As is the question of in-app purchases or DLC. Given that those are going away, the next question is what is being done to preserve the games and their source codes? No answers currently exist. And, if you believe that video games are a form of art and culture, that's very much a problem.

Put yourself in the shoes of a preservationist or museum for video games. What in the world happens to this culture from here? Where does the DLC go, given that it's very much a part of the art of the game? Where does the source code go? How will someone 25 years from now be able to experience this art in the same way you can walk into an art museum and see an exhibit.

As of today, the answer to most of those questions most of the time is ¯\_(ツ)_/¯. But that can't be the answer and it's probably time that either the industry or, if the industry fails to take this seriously, the government take a good long look at carving out some exceptions or requirements to make sure all this culture doesn't end up memory-holed.

Filed Under: compatibility, game preservation, preservation, ps vita, ps3, psp, video games
Companies: sony

Good Idea: As Video Game Preservation Often Falls To Fan Groups, Release Every Game's Source Code

from the pirate-preservation dept

When it comes to the video game industry, there has been some recent recognition that copyright laws and the ways that publishers utilize them have hampered the ability to preserve this sort of art. In the olden days of a decade or so ago, the challenges around preserving video games centered around both the publisher's unwillingness to allow a group access to source code to preserve a game and the deterioration of physical game media. But in these modern times, this has changed. Now, the challenges are the publisher copyright question... and that same publisher's ability to simply stop supporting the online resources modern games and platforms require to run. Given the ongoing war on emulators by the likes of Nintendo and a rather insane industry stance that preservation is trumped by copyright concerns, there is a very real risk of losing the ability to preserve video game history at all.

Recent rumors that Sony is going to shut down online stores for a bunch of old hardware, has thrown the question of what happens to digital purchases in sharp relief.

Yesterday, TheGamer reported that Sony has plans to shut down the online PS3, PS Vita, and PSP stores that service those older consoles. While this has yet to be confirmed, and Sony has not responded to Kotaku’s request for comment, the internet discourse around this potentially troubling news immediately began to swirl.

If these stores go away, PS3, PS Vita, and PSP players will be unable to purchase new digital games. While there aren’t yet concrete details about what, if anything, is happening, the rumors have many PlayStation gamers understandably worried about the continued viability of their digital purchases.

Add to all of that the question of game preservation. With purchases being digital and potentially just going away at Sony's whim, and with source code locked up by developers and publishers... what happens to antiquated PlayStation games when the cord is pulled? How would a museum or interest group preserve these games? How will future generations be able to enjoy and participate in this art?

The answer, of course, is piracy.

This kind of real preservation is rarely done by corporations. Instead, communities form around games and keep them alive for years beyond their normal commercial lifespans. These people are doing some impressive things. Look at the continued work on the unofficial but fantastic PC port of Super Mario 64. Or just a few days ago, The Hidden Palace uploaded over 700 PlayStation 2 game prototypes and dev builds, uncovering and preserving a huge bit of game history in one fell swoop.

Meanwhile, publishers like Nintendo use lawyers to crack down on the availability of emulator-playable ROMs for games that are no longer sold. Nintendo even explicitly limits how long it will sell certain games. None of this helps preserve these works. In fact, it actively hurts efforts to do so.

And so the public's interest in video game preservation sits on a single train track, with the copyright enforcement train hurtling towards it from one direction and publishers' decisions to stop supporting the online resources needed for digital purchases from the other. The result, if left alone, will be a train wreck, at least as far as the public interest is concerned.

So, what's the fix? Well, as per usual, the fix would be for game developers and publishers to give up just a bit of control over their products in a way that would allow preservation to occur.

Release all games on PC, preferably alongside their source code. Having PC game releases with source code would make certain aspects of game preservation much easier, and could allow even the oldest games to survive for decades to come. It frees games from being tied to one single platform or the whims of whatever capitalist entity published it.

This isn’t a wild, unproven theory. One of the most-ported and played classic games is the original Doom. id Software released its source code back in 1997, only four years after Doom’s launch. Since then fans have created numerous “source ports” of the game, to the point that Doom’s now playable on almost any device with a screen.

As a result, Doom has also stayed relevant. That’s important, because while the source ports have made it extremely easy to play Doom without buying it (all it takes is a quick search to find the necessary content files) that hasn’t hurt the IP. I’d argue the opposite! One possible reason Doom is still around—and we just got a big DLC expansion for the series’ latest game, Doom Eternal—is people still give a shit about Doom in 2021. And people still give a shit because it’s incredibly easy to play Doom. It’s only a few clicks away and its enthusiastic community has taken its source code in directions id never imagined.

This doesn't directly solve the PlayStation problem, of course, though there are avenues to explore there as well. But it's at least a start towards giving the public the tools to do the game preservation themselves, since developers and publishers often are incapable or unwilling to do it. And, as the Kotaku post notes, this should be seen not as some threat to the gaming industry, but a boon. Doom is the perfect example as to why.

But, regardless, it is well past time that we do something about this. It is not tenable that we lose what is now a couple decades worth of art preservation just because it's being sacrificed at the copyright altar.

Filed Under: archives, copyright, emulators, history, piracy, playstation, preservation, ps vita, ps3, psp, source code, video games
Companies: sony

DOJ, Obama Administration Fight Order Requiring The Full CIA Torture Report To Be Turned Over To The Court

from the don't-touch-our-stuff dept

The Obama administration has responded to calls to declassify the full CIA Torture Report with a "will this do?" promise to lock up one copy in the presidential archives. While this ensures one copy of the full report will survive the next presidency, it doesn't make it any more likely the public will ever see more than the Executive Summary released in 2014.

Other copies may still be scattered around the federal government, many of them in an unread state. The Department of Defense can't even say for sure whether its copy is intact. Meanwhile, an ongoing prosecution in which the defendant is alleging being waterboarded by the CIA has resulted in an order to turn over a copy of the full report to the court.

This order would preserve a second full copy -- with this copy being as close as we've gotten so far to seeing it become part of the public record. Of course, the DOJ is challenging this court order on behalf of the Obama administration, which certainly never intended to participate in this much transparency. Charlie Savage of the New York Times notes (on his personal blog) that a motion has been filed seeking to reverse the court's preservation/deposit order.

[T]oday the Obama Justice Department decided to fight Judge Lamberth’s order rather than comply with it. It filed a motion asking to Judge Lamberth to reconsider his order, arguing that it raised constitutional concerns (interfering with communications between Congress and the executive branch) and was unnecessary anyway given the presidential records thing. And it said that if he didn’t reconsider, the executive branch will appeal.

The judge had ordered a copy to be filed with the Court Information Security Officer (CISO). The DOJ argues [PDF] that this isn't necessary because the CIA has its copy locked up real tight-like.

Reconsideration is appropriate primarily because intervening facts have rendered these provisions unnecessary. As explained below, a copy of the SSCI Report is already being preserved in the Executive Branch under the Presidential Records Act, 44 U.S.C. §§ 2201-2209, and documents underlying the Report have been and continue to be preserved under a 2007 preservation directive issued by the Director of the CIA. Further, no copy of the SSCI Report held by the CIA has been destroyed, nor has any improper destruction of evidence by the CIA occurred since the issuance of the 2007 preservation directive.

True, one copy is being preserved in Obama's presidential archives. But the second assertion is a bit more dubious. While it's technically true the CIA hasn't destroyed any copies of the Torture Report, its oversight has. In May of last year, it was discovered that the CIA's Inspector General had destroyed the copy he'd received, thanks to a string of counterproductive "normal business" decisions.

[L]ast August, a chagrined Christopher R. Sharpley, the CIA’s acting inspector general, alerted the Senate intelligence panel that his office’s copy of the report had vanished. According to sources familiar with Sharpley’s account, he explained it this way: When it received its disk, the inspector general’s office uploaded the contents onto its internal classified computer system and destroyed the disk in what Sharpley described as “the normal course of business.” Meanwhile someone in the IG office interpreted the Justice Department’s instructions not to open the file to mean it should be deleted from the server — so that both the original and the copy were gone.

This prompted Dianne Feinstein to send a letter to someone who should have had an extra copy or three of the Torture Report lying around: CIA Director John Brennan.

As you may be aware. the office of the CIA Inspector General has misplaced and/or accidentally destroyed its electronic copy and disk of the Senate Select Committee on lntelligence's full 6,700-page classified Study of the CIA Detention and Interrogation Program. I write to request that as Director of the CIA, you provide a new copy of the Study to the office of the CIA IG immediately.

Your prompt response will allay my concern that this was more than an "accident." The CIA IG should have a copy of the full Study because the report includes extensive information directly related to the ongoing oversight of the CIA.

The accidental deletion of the IG's copy was possibly in violation of another preservation order stemming from ongoing FOIA litigation. The CIA was under orders to preserve its copy(ies), which means its oversight office should have been doing all it could to ensure its copy didn't disappear. If the CIA somehow managed to destroy its copies of the report, its oversight should have been in position to act as its backup, not to mention come down on it for any acts of impropriety. Instead, it took a copy and immediately -- if possibly inadvertently -- threw it away.

So, while the DOJ may firmly believe in all that is good and right and trustworthy about government officials holding onto damning documents, it can't rule out the possibility of human error. So, to better ensure preservation in accordance with the order, it should be less reluctant to hand over a copy to the court CISO, which would at least shift the culpability should this copy end up missing.

The obvious conclusion is that the outgoing administration (to say nothing of the incoming replacement) is still very interested in keeping the full report from ending up in the public's hands. Delivering a copy to the court makes it a part of the judicial process which, despite its tendency to seal documents and dockets far too frequently, is a much more open process than shuffling copies around from lockbox to lockbox within federal government agencies.

Filed Under: cia, cia torture report, doj, evidence, preservation, torture report

Archivists Grapple With Problems Of Preserving Recent Culture Held On Tape Cassettes And Floppy Drives

from the digital-archaeology dept

Most Techdirt readers probably surround themselves with the latest technology. But there's a slightly unusual class of professionals who are only now beginning to grapple with things like CP/M, 8-inch floppy disk drives and the Apple Lisa. These are the archivists, whose job is preserving cultural artifacts from all periods of history. That includes the recent past, whose technologies now seem paradoxically so strange and distant. The real-life consequences of that growing chasm between today's digital technologies, and those that were commonplace 10, 20 or 30 years ago, are made evident in an article published by the Guardian last week:

In the belly of a former whisky store in the inner Melbourne suburb of Brunswick lies a vast and varied collection of artefacts that feminist scholars can't wait to get their hands on.

Nearly 500 boxes in this dark, temperature-controlled warehouse hold a lifetime of handwritten letters, browning manuscripts and newspaper clippings.

But there are more modern treasures too: floppy disks containing an unpublished book about Margaret Thatcher; two computers, a Mac Powerbook G4 and iMac G5; and voicemail recordings about dinner plans in 1976.

These are all part of the archives of the well-known Australian writer Germaine Greer. According to the article, Greer has been hoarding personal documents and artifacts from the 1950s to the present day, which means they are in both analog and digital forms:

Greer's archive includes floppy disks, tape cassettes and CD-roms, once cutting-edge technologies that are now obsolete. They are vulnerable to decay and disintegration, leftovers from the unrelenting tide of technological advancement. They will last mere decades, unlike the paper records, which could survive for hundreds of years.

It is an irony of these formerly high-tech holdings that they are far less durable than old-fashioned paper-based systems. And researchers studying them face problems of compatibility that simply don't arise with paper. This is a major issue that is only now being faced, as cultural figures of Greer's generation pass on their archives to universities and libraries, who must start to grapple with the core tasks of deciphering and preserving them.

The good news is that once they have been decoded, they can be transferred to other media, and in more open formats that will be easier to access in the years to come. But that still leaves the problem of how to store all these archives in a way that will stand the test of time. Perhaps they will be encoded as data held on the ultimate storage medium, DNA. Or maybe it would just be easier to print the lot out on paper.

Follow me @glynmoody on Twitter or identi.ca, and +glynmoody on Google+

Filed Under: archivists, cassette tapes, deteriorating media, digital archives, floppy drives, obsolete formats, old media, preservation

George Lucas Wants Desperately To Preserve Old Movies... Unless They're His; So Fans Are Trying To Do It Instead

from the edit-wars dept

Kevin Carson points us to a fascinating story in The Atlantic about fans trying to recreate the "original" version of Star Wars ("Episode IV -- A New Hope for the folks who feel like being pedantic) from 1977. As various fans have pointed out repeatedly (mainly each time Lucas went back and "edited" Star Wars again), back in 1988 Lucas spoke to Congress about the importance of preserving original versions of movies, and avoiding the constant attempts to update and modernize them in ways that might erase the original versions. Key quote:

Today, engineers with their computers can add color to black-and-white movies, change the soundtrack, speed up the pace, and add or subtract material to the philosophical tastes of the copyright holder. Tomorrow, more advanced technology will be able to replace actors with "fresher faces," or alter dialogue and change the movement of the actor's lips to match. It will soon be possible to create a new "original" negative with whatever changes or alterations the copyright holder of the moment desires. The copyright holders, so far, have not been completely diligent in preserving the original negatives of films they control. In order to reconstruct old negatives, many archivists have had to go to Eastern bloc countries where American films have been better preserved.

In the future it will become even easier for old negatives to become lost and be "replaced" by new altered negatives. This would be a great loss to our society. Our cultural history must not be allowed to be rewritten.

This was part of his attempt to create so-called moral rights for content creators, but the wording seems kind of strange considering how Lucas himself later did pretty much everything he described above as being heinous and troubling. The Atlantic article highlights just a few of the more famous changes (though there are many, many more):

In the original versions of the films, for example, it’s clear that Han Solo pulled out his gun and shot the bounty hunter Greedo. In the 1997 version, Greedo shoots first. In the 2004 version, they shoot at the same time. With the release of the later films, later versions of the original trilogy were edited to add in appearances from Jar Jar Binks and Hayden Christensen. Lucas even replaced the voice of Jason Wingreen—the original Boba Fett—with the voice of Temuera Morrison who played Jango Fett in Episode II: Attack of the Clones.

Even worse, Lucas has actively resisted attempts to make the original copy available. When pressed about it a decade ago Lucas claimed that the original was "half completed" and he wants people to see the "finished" product instead:

The special edition, that’s the one I wanted out there. The other movie, it’s on VHS, if anybody wants it. ... I’m not going to spend the, we’re talking millions of dollars here, the money and the time to refurbish that, because to me, it doesn’t really exist anymore. It’s like this is the movie I wanted it to be, and I’m sorry you saw half a completed film and fell in love with it. But I want it to be the way I want it to be. I’m the one who has to take responsibility for it. I’m the one who has to have everybody throw rocks at me all the time, so at least if they’re going to throw rocks at me, they’re going to throw rocks at me for something I love rather than something I think is not very good, or at least something I think is not finished.

That's all well and good, but it's yet another example of the sense of entitlement some creators have in which they believe they not only hold the copyright on the original work (which they may), but also ownership over the experience of fans who watched/heard/experienced the content. And that's where things get tricky. When the artists start to muck with that experience.

And that's what led to the result here, whereby fans are painstakingly recreating the 1977 version of the film. There's an incredible video highlighting how one of the main people involved in this project, a 25-year-old in the Czech Republic who goes by the name Harmy, goes about fixing things. It's fascinating: The "new" version is amusingly called the "despecialized" version, and uses bits and pieces from the many, many releases to reconstruct the original. While some point out that there was a DVD release of "the original" film, the video notes that the techniques used to transfer the film to DVD were very problematic, leading to a variety of problems, including "motion smearing," faded colors and aliasing.

The lengths these fans go to in order to recreate the original is quite incredible, going through all the different versions, picking up pieces from one and inserting them in the other, doing careful color corrections, "upscaling" low res versions to make them HD. It's really quite incredible, and it appears Lucas would rather they disappear entirely. He even rejected a request from the National Film Registry when it requested a copy of the original to preserve.

Curators at the National Film Registry picked the 1977 version of Star Wars to preserve for history’s sake, but they still don’t have a copy in the registry. When they asked for a copy, Lucas refused, saying that he would no longer authorize the release of the original version.

While Lucas' changes and updates to his film bother some, I've never been that concerned about those attempts to re-imagine his own work, but it does seem particularly silly to try to block people from even having the choice to view the original. It's great that fans are putting in so much effort to reconstruct it by themselves, but it seems like Lucas could just speed that whole process along by making the original available.

Filed Under: 1977, despecialized edition, fan edits, fans, george lucas, old movies, preservation, star wars

DOJ Apologizes For Misleading FISC Concerning Evidence; Hints Strongly That It Used Opportunity To Destroy Evidence

from the evidence,-who-needs-it? dept

A few weeks ago, we noted that FISC Judge Reggie Walton was quite reasonably pissed off at the DOJ for directly withholding key information about evidence in a series of lawsuits concerning NSA surveillance programs. The full details are a bit down in the weeds, but the short summary is that there's been some debate over whether or not the government needs to retain surveillance data because it's evidence in these cases, or if it needs to destroy the surveillance data, as required by the rules over its holding of the data. There's been a bit of back and forth over all of this, but the DOJ apparently directly withheld from the FISC a request by EFF lawyers to inform the court that a data preservation order should cover two of the key NSA surveillance cases that it has been involved in for years (since well before the Snowden disclosures). The DOJ not only did not inform the court, but it also appears to have tried to dissuade the lawyers from raising the issue. Judge Walton ordered the DOJ to explain itself, and it gives a long apology, repeatedly insisting that it didn't believe those cases were related, since they were focused on surveillance data ordered by the President, rather than the FISA Court -- a weak excuse at best:

Based upon the nature of the claims made in Jewel and Shubert, which the Government has always understood to be limited to certain presidentially authorized intelligence collection activities outside FISA, the Government did not identify those lawsuits, nor the preservation orders issued therein, in its Motion for Second Amendment to Primary Order filed in the above- captioned Docket number on February 25, 2014. For the same reasons, the Government did not notify this Court of its receipt of plaintiffs' counsel's February 26, 2014, e-mail. With the benefit of hindsight, the Government recognizes that upon receipt of plaintiffs' counsel's e-mail, it should have made this Court aware of those preservation orders and of the plaintiffs' disagreement as to their scope as relevant to the Court's consideration of the Government's motion and regrets its omission. The Government respectfully submits that in light of this submission, and this Court's Opinion and Order dated March 12, 2014, granting the Government's motion for temporary relief from the destruction requirement in subsection of the Court's Primary Order, no additional corrective action on the part of the Government or this Court is necessary.

It goes on for much longer trying to suggest that this was all just an honest mistake, and how could it have possibly realized that the lawyers for the plaintiffs in those cases might think the issue of preserving their evidence was related to similar questions over preserving evidence of other NSA cases. It insists that it really was just focused on the more recent cases that were filed post-Snowden ("filed after last year's unauthorized public disclosure concerning the collection of telephony metadata pursuant to FISA authority") and hadn't even considered how it related to older cases concerning NSA surveillance.

The Government did not notify the Court of Jewel and Shubert in the Motion because the Government has always understood those matters to challenge certain presidentially authorized intelligence collection activities and not metadata subsequently obtained pursuant to orders issued by this Court under FISA, and because the preservation issues in those cases had been previously addressed before the district court in which those matters are pending. Jewel and Shubert, filed in 2008 and 2007, respectively, challenge particular NSA intelligence activities authorized by President Bush after the September 11, 2001 terrorist attacks without statutory or judicial authorization.

As the DOJ goes on to explain, it assumed the preservation issues in the Jewel and Shubert cases were settled, and therefore irrelevant to the FISA issue (especially since, in its mind, the two sets of cases covered different programs). Thus, the DOJ claims, its emails dissuading the lawyers from raising the issue weren't so much about silencing those lawyers and hiding information from FISC, but were a true misunderstanding, in that it was just letting those lawyers know the issues were unrelated, as well as a desire (no joke) to not bother the FISC with unnecessary distractions.

In particular, the request in its February 28 email that counsel for the Jewel plaintiffs "forebear from filing anything with the FISC, or [the district court], until we have further opportunity to confer" was a good faith attempt to avoid unnecessary motions practice in the event that the issue could be worked out among the parties through the Government's provision of an unclassified explanation concerning its preservation in Jewel and Shubert. Accordingly, the Government did not bring the Jewel plaintiffs' February 25 email to this Court's attention.

Of course, there's still the big question of, between the two FISA court orders, whether or not the DOJ did, in fact, destroy some of the evidence. And, a follow-up correction from the government very strongly suggests that it absolutely did use the opportunity to destroy evidence. The follow-up is a correction to a footnote, in which the DOJ makes it clear that "consistent with the Government's understanding" (which appears to be mistaken) and "prior to" the more recent filing, "the Government complied with this Court's requirements that metadata obtained by the NSA under Section 215 authority be destroyed no later than five years after their collection."

What that almost certainly means is that the NSA destroyed the metadata collected up until 2009, which likely is relevant to the Jewel and Schubert cases, even though the lawyers in those cases had alerted the DOJ of these concerns. For all of the DOJ's "apologies" in the first document, this certainly seems very convenient for the US government. And, as Marcy Wheeler notes, the destroyed evidence may have included "cover almost all of the phone dragnet violations discovered over the course of 2009." Convenient. But the DOJ is really, really sorry about it. Really.

Filed Under: collection, destruction of evidence, doj, fisa court, fisc, jewel, metadata, nsa, preservation, reggie walton, shubert, surveillance
Companies: eff

Internet Archive Fire Shows Vulnerability Of The World's Online Memory

from the taking-things-for-granted dept

The Internet Archive is a jewel of the digital world:

The Internet Archive is a 501(c)(3) non-profit that was founded to build an Internet library. Its purposes include offering permanent access for researchers, historians, scholars, people with disabilities, and the general public to historical collections that exist in digital format.

Founded in 1996 and located in San Francisco, the Archive has been receiving data donations from Alexa Internet and others. In late 1999, the organization started to grow to include more well-rounded collections. Now the Internet Archive includes: texts, audio, moving images, and software as well as archived web pages in our collections, and provides specialized services for adaptive reading and information access for the blind and other persons with disabilities.

Here's the amazing scale of the project today:

The Internet Archive Wayback Machine contains almost 2 petabytes of data and is currently growing at a rate of 20 terabytes per month. This eclipses the amount of text contained in the world's largest libraries, including the Library of Congress.

The Internet Archive is the world's online memory, holding the only copies of many historic (and not-so-historic) Web pages that have long disappeared from the Web itself.

Bad news:

This morning at about 3:30 a.m. a fire started at the Internet Archive's San Francisco scanning center.

Good news:

no one was hurt and no data was lost. Our main building was not affected except for damage to one electrical run. This power issue caused us to lose power to some servers for a while.

Bad news:

Some physical materials were in the scanning center because they were being digitized, but most were in a separate locked room or in our physical archive and were not lost. Of those materials we did unfortunately lose, about half had already been digitized. We are working with our library partners now to assess.

That loss is unfortunate, but imagine if the fire had been in the main server room holding the Internet Archive's 2 petabytes of data. Wisely, the project has placed copies at other locations:

We have copies of the data in the Internet Archive in multiple locations, so even if our main building had been involved in the fire we still would not have lost the amazing content we have all worked so hard to collect.

That's good to know, but it seems rather foolish for the world to depend on the Internet Archive always being able to keep all its copies up to date, especially as the quantity of data that it stores continues to rise. This digital library is so important in historical and cultural terms: surely it's time to start mirroring the Internet Archive around the world in many locations, with direct and sustained support from multiple governments. They can also help provide the Internet Archive with a wider, more international range of content, to make an even more representative store of the world's digital activity.

Unfortunately, that's not likely to happen anytime soon, as people seem happy to take for granted the amazing work of Brewster Kahle and his team. The next best thing would be to donate so that they can continue with their indispensable project -- and perhaps create a few more backup copies.

Follow me @glynmoody on Twitter or identi.ca, and +glynmoody on Google+

Filed Under: culture, fire, history, internet archive, preservation
Companies: internet archive

DailyDirt: Keeping Food Around Longer... And Longer

from the urls-we-dig-up dept

Civilization has greatly depended on the our ability to store food for long periods of time. Without various food preservation techniques, our daily lives would be much different. Perhaps we've strayed a bit too far away from fresh foods, but the benefits of refrigeration, preservatives and food packaging probably outweigh the costs. Here are just a few articles on the topic of food preservation to ponder while you enjoy your next processed meal.

• Twinkies are back on stores shelves with a new and improved... shelf life of 45 days. Previously, Twinkies had a shelf life of just 26 days -- and not an indefinite lifespan that most people assume. [url]
• Canned foods can last a really long time, and some folks actually prefer the taste of canned items that have been aged. The definition of a shelf life is not about when a packaged food is inedible, but when the food acquires a noticeably different flavor compared to a newly-manufactured item. [url]
• About 70% of the food we eat is stored or transported at chilled temperatures. And if you think that's a high percentage: "An astonishing 80% of the nation's potato output is cut, processed, frozen, bagged, and distributed as French fries." [url]

If you'd like to read more awesome and interesting stuff, check out this unrelated (but not entirely random!) Techdirt post via StumbleUpon.

Filed Under: aged, canning, flavor, food, frozen, inedible, packaging, preservation, refrigeration, shelf life, twinkies

DailyDirt: Preserving Food For Posterity

from the urls-we-dig-up dept

Modern civilization wouldn't be where it is today without the means to store food for long periods of time. Before refrigerators, food spoilage was a daily concern for just about everyone. Now, we can keep leftovers in a fridge for far too long -- and forget about it until some really furry mold is obviously thriving. Here are just a few links on keeping food from going bad.

• Food dehydration is an ancient technique that isn't just for Ronco infomercials. It's not all about beef jerky, but there is a lot of waiting around and chewy goodness. [url]
• Preserving foods can alter taste dramatically, especially when intense heat is used to sterilize delicate (and volatile) flavors. Adding artificial flavorings as masking agents doesn't sound very appetizing, but there are some alternative preservation methods that involve zapping foods with electrical current that might help retain fresher flavors. [url]
• A deep-sea submersible vessel accidentally sunk to the bottom of the Atlantic off the coast of Massachusetts, and the crew escaped... but their lunches didn't. When the ship was re-surfaced months later, the crew's lunches were almost perfectly preserved by the hyperbaric storage.[url]

If you'd like to read more awesome and interesting stuff, check out this unrelated (but not entirely random!) Techdirt post.

Filed Under: dehydration, flavor, food, hyperbaric storage, pasteurization, preservation
Companies: ronco