How Important Is It To Preserve Our Digital Heritage?
from the the-archive-team-at-work dept
Having recently been part of an effort to preserve videos on Google Video, in light of its announced shutdown, it's made me more aware how brittle our current culture is, with many many artifacts available only in digital form. There are conservation efforts such as the aforementioned one, part of a larger group called Archive Team. Google changed its mind and promised to keep Google Video online and try to move as much as possible to YouTube where it has permission from the original uploader. This is, obviously, a win for the archivists, who kept the effort going just the same and are still in the process of uploading all of Google Video to archive.org.Contrast this with Friendster, where the Archive Team project was unable to save everything before the end of May, when everything was set to be deleted. Much has been downloaded and it will still give a good picture of what this early social network was like. Similarly, the Archive Team was able to rescue much, but not all of GeoCities, before Yahoo shut that down last year, releasing its collection as a massive 1 TB bittorrent file.
Now you may ask yourself is Friendster (or even GeoCities) worth saving? Answering that question purely myself, I'd say no, but I have no connection with those sites. Looking at the question however from the perspective of people who have spent many hours building these profiles and interacting with each other, I can see there's clearly value there to those who used them. Answering instead as someone with a deep interest in history, it's not for us to say what will eventually prove worth saving. Instead, that's something for the historians, archeologists, sociologists and other interested parties in the future to decide. But they can't do that if the information isn't even there.
How often have we not wished more of a particular point in our history had been preserved? Ironically many of the analog writings of our past are in a better state than some of our digital ones, even when it's often pointed out how easy it is to make bit-perfect copies of something in this day and age.
All of this, leads me to ask the following questions:
- Do you agree the digital aspects of our culture should be better preserved?
- Should we have, maybe even one on each continent or in each country, a modern Library of Alexandria?
(identical copies in different places to prevent 'a fire' from destroying it)
- What would such a world-wide archival effort look like, technologically?
- What should we do about copyrights and patents (and DRM) that might get in the way of preserving our heritage?
- How do we go about archiving software for a computer that's available only in a museum, to preserve interactive access?
- What about the proliferation of file formats? Do we transform everything into a canonical form where we can, or do we store the original along with software to interpret?
I'll start off by saying that though little personal correspondence has been preserved of earlier times, we should recognize the right to privacy and any such posited archive would be allowed to include private communications only if you explicitly opted in. Valuable though these are for historians, if we preserve as much as we can of the digital domain, enough can be inferred from context that this would be an unwarranted intrusion.
But what about the wider issue of preservation of public content that can be wiped out by shutting off a power switch?
Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.
Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.
While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.
–The Techdirt Team
Filed Under: archives, culture, preserving
Reader Comments
Subscribe: RSS
View by: Time | Thread
I honestly don't know what should be done about this seeming mismatch of priorities. Perhaps a clause in the terms and conditions that allows archival by a specific third party like the Archive Team?
[ link to this | view in chronology ]
Re:
[ link to this | view in chronology ]
Re: Re:
[ link to this | view in chronology ]
Re: Re: Re:
[ link to this | view in chronology ]
Re: Re: Re: Re:
[ link to this | view in chronology ]
Re: Re:
I do however have sympathy for the position held by Jacob Cooper, below, questioning whether it's something a government can be trusted to do. A third party might have a mandate to preserve as much as possible, regardless of PoV or source, whereas a government entity might be tempted to archive predominantly artifacts showing them in a favourable or neutral light.
A government might also decide cultural artifacts not from their own country, as far as that's at all easy to tell on the internet, are not their problem to look after. How much would be left by the wayside if all such Library of Congresses came to that conclusion?
[ link to this | view in chronology ]
Re: Re:
This whole issue has been hashed out by librarians and archivists over and over and over again for many decades and over many formats. We haven't the money, staff or technical resources to do more than well defined, usually grant funded, projects.
[ link to this | view in chronology ]
Re: Re: Re:
Not that it's a slight against LoC. It's not they who set their budget. I'm certain they'd love being able to do a more thorough job, even if still not of the scope outlined.
Nor, for the record was I suggesting LoC picks and chooses artifacts adding up to a biased image; it would defeat their stated purpose of being able to inform the research needs of Congress. That's not to say, as a governmental institution, they're not subject to the possibility of having that mandate hijacked in the future.
[ link to this | view in chronology ]
Just the other day, I learned that people had created extra tracks for use with the game Need For Speed 2. Of course the only web site to host those tracks was gone. It's not even on the Internet Archive.
[ link to this | view in chronology ]
[ link to this | view in chronology ]
Re:
[ link to this | view in chronology ]
Re:
i wish my dog knew how to do it. at least i can trust him. well, except when that damn poodle comes around.
[ link to this | view in chronology ]
Absolutely, especially when today digital IS our culture. Consider that digital degrades a lot less gracefully than analog. Where old film reels are managing to last almost a century, a DVD lasts a decade before the data on its starts becoming corrupted. A lot of old computer games stored on floppies or magnetic tape have already been lost, and that trend will continue unless something is done soon.
"Should we have, maybe even one on each continent or in each country, a modern Library of Alexandria? (identical copies in different places to prevent 'a fire' from destroying it)"
That's the idea. There needs to be international cooperation to achieve that and copyright restrictions on archiving need to be removed. In fact, that may be impossible to achieve without taking away "automatic" copyrights and requiring that before any copyright is granted, the creator is required by law to submit to the copyright office a high-quality copy of the work in a standard format without DRM.
"What would such a world-wide archival effort look like, technologically?"
Simplest case, your typical big datacenters. Problem is, whatever we come up with should be able to survive the test of time, even if left in disrepair. Arrays of harddisks will fail fairly quickly if not copied every few months, as will optical media.
A more ambitious proposal would look to find a way to make copies of digital data that will last forever without maintenance. I don't know if such technology is available, so it may need to be developed. I think I read in Scientific American there have been moderately successful attempts at creating read-only holocubes with about a TB of data imprinted in them...
"What should we do about copyrights and patents (and DRM) that might get in the way of preserving our heritage?"
They don't matter. They are the opposite of preserving our heritage.
"How do we go about archiving software for a computer that's available only in a museum, to preserve interactive access?"
Create a unified emulation platform that contains the specifications of various system architectures and runs on a fully documented, open source virtual machine.
Bonus points if hardware and software developers are required by law to maintain the system and make sure their products are compatible with it.
Further bonus if all software developers are required by law to submit source code for archiving (but not necessarily public disclosure).
"What about the proliferation of file formats? Do we transform everything into a canonical form where we can, or do we store the original along with software to interpret?"
Open standards are the future.
[ link to this | view in chronology ]
Re:
No, we do not. Each country will always serve it's own interest.
"Create a unified emulation platform that contains the specifications of various system architectures and runs on a fully documented, open source virtual machine. "
Good luck with that :) Do you already have open source cars? How about planes? Guns? I see.
"Further bonus if all software developers are required by law to submit source code for archiving (but not necessarily public disclosure)."
Do you have it for cars already? Ah, I see. People overestimate importance of this "software" thing.
"A more ambitious proposal would look to find a way to make copies of digital data that will last forever without maintenance"
"Forever" is very long time. Nothing, absolutely nothing stay forever. However, writings from Dead See scrolls survived, so Google's datacenter may too.
[ link to this | view in chronology ]
Re: Re:
Makes no difference.
"Good luck with that :) Do you already have open source cars? How about planes? Guns? I see."
Just because you don't see a way of doing something doesn't mean it's impossible. We'll get open source cars and planes some day, just like we'll get ways to preserve digital culture.
"Do you have it for cars already? Ah, I see. People overestimate importance of this "software" thing."
What do cars have to do with this? Software is information, cars are cars. The way you preserve one thing is totally different from the other.
""Forever" is very long time. Nothing, absolutely nothing stay forever. However, writings from Dead See scrolls survived, so Google's datacenter may too."
You know perfectly well what I meant by "forever". If you have nothing to add, I'd rather you didn't reply to my comments.
[ link to this | view in chronology ]
Ephemera = Culture
And while I understand that it's not in LoC's purview nor budget to catalog and archive what IS our ever-evolving digital culture at any given moment, it's a great point to be brought up.
I actually HAVE my 1995 original GeoCities website (Soho/8499) backed up on a data disc. It's hilarious, somewhat sad, and accurate to look back on the images and text included as a viable-to-be-cataloged moment in time of digital culture.
That said, how the heck would one go about that? I have no idea; other than to guess it would take a helluva lot of data storage and money.
Even worse (and slightly off-topic) is the number of old film & television series we are going to loose, that will never get to be public-domained because the studios and networks *literally* can't find the right paperwork that allows them to create digital copies. There's a LOT of this media, that is going to disappear.
[ link to this | view in chronology ]
This is Already Happening in our Personal Lives
I have photos and documents on a diskette 10 years old that I can no longer access beacuse my only computer with a diskette drive has died.
[ link to this | view in chronology ]
Re: This is Already Happening in our Personal Lives
Failing that, you can probably find a system at the dump or in the trash (if you area has bulk trash pickup) that has a floppy drive.
[ link to this | view in chronology ]
Re: Re: This is Already Happening in our Personal Lives
Or, you can buy a USB floppy drive.
[ link to this | view in chronology ]
Re: Re: Re: This is Already Happening in our Personal Lives
True. Being a tinkerer, I was thinking of solutions that wouldn't cost anything. I mean, people are literally throwing out old floppy drives and computers with floppy drives in them. Even CD-ROM drives aren't considered worth saving any more. Not long ago I saw a computer stripped down to the motherboard (cards, memory, fans, etc), but they left the CD & floppy drives.
[ link to this | view in chronology ]
Art will last as long as the fans make it last.
Then look at how fast bad movies drop out of circulation. Heck, the worst are released straight to DVD because they know the fans WON'T support it.
As long it's driven by the quality of the work, or at least the dedication of the fans, we'll be fine.
[ link to this | view in chronology ]
Re: Art will last as long as the fans make it last.
A terrific example of someone taking it upon themselves to preserve an important part of our early online presence can be found at: http://www.textfiles.com/ , covering the history of BBS text files from the 80's and 90's. Many of these files may be familiar to others out there, and you might even have them still stored on some type of medium. But with this site preserving the files, there is no need to pull out the old PC in the garage with the floppy drive and see if that disk is even still readable.
[ link to this | view in chronology ]
The Internet never forgets
It just may be a bit harder to find the older stuff.
[ link to this | view in chronology ]
Digital lifetimes
Now, store this DVD in an old biscuit tin, keep in the attic for 30 years and then dig it out to show the grand kids what life was like way back then... ah, now we have a problem. The DVD may be in perfect condition but no DVD readers are in existence any more.
However those old film negatives are still there, and can still be 'seen' as photos using nothing more hi-tech than a lamp and a lens. As a bonus, those shots that a digital user would have deleted are still there, and give another view of the world at that time.
There is something being lost with this digital revolution, despite the huge amount of data we produce, how much will be available to the generations to come?
I remember reading about the digital 'Doomsday project' this was suppose to be a digital version of the original, sadly the 12 inch video disks used to store the data became obsolete and only by the intervention of a few dedicated people was the data retrieved and put on the web (and how long will that last? what happens when the hosting goes bust?)
I lament the loss of plain old film cameras. And vinyl, cassette tapes, and paper books etc. etc.
RevJim
[ link to this | view in chronology ]
Re: Digital lifetimes
On the flip-side of that, digital cameras are much more economical to use. In the past you had to buy film, then, if you weren't a photography buff, you had to pay to have the film developed and printed. Even if you were an expert with your own darkroom, developing the images took expensive chemicals and time. With a digital camera, people can shoot hundreds of images that are ready for use as soon as they're downloaded to a computer. They can then be printed out, emailed or archived. It's true, that people only keep the shots that they like, but they're taking more photos to begin with. I hardly ever used a film camera except on special occasions. My mother let about a dozen rolls of film from various trips, sit and rot because she kept putting off having them developed. If we'd had a digital camera back then, I'd have all those photos today.
There is something being lost with this digital revolution, despite the huge amount of data we produce, how much will be available to the generations to come?
You're missing one very important point; Today's digital files, such a jpegs, AVIs, etc, can easily be transferred to any future technology. You could argue that some will be lost because nobody bothers to do it, but that's not the fault of the technology.
For example, I have hundreds of images saved on Amiga floppy disks. While I still have an Amiga, ways to directly read the disks on current computers don't exist (I used a custom format to get more on the disks). However, there ARE ways to transfer those files. I can use MS-DOS formatted disks, although that would be slow. A better solution would be to copy them to an MS-DOS formatted Zip disk, then read that on my current system. True, it's "only" 100MB, but that will hold quite a few floppies. I can also use a null modem cable and transfer the files that way. Once transferred, the files can be burned to DVD, Blu-Ray or stored on a 2TB drive. And if some new storage format comes along in the future, they can be transferred to that as well, with no loss in quality.
I lament the loss of plain old film cameras. And vinyl, cassette tapes, and paper books etc. etc.
I have video tapes that I recorded in the 1980s which are now unwatchable due to the degradation of the magnetic fields on the tape. Some of my paper books are literally falling apart. While I do agree that reading an actual book is nicer than reading text on a screen, books can't easily be searched for certain passages, which I have often wanted to do. Not to mention that every single book I own could fit on a single DVD with room to spare.
[ link to this | view in chronology ]
Preserving history
I think it is important to realize that a huge part of the culture of the western world was considered "unimportant" in the middle ages, and was preserved, and eventually returned, to us by Islamic scholars.
We owe them an enormous debt of gratitude.
[ link to this | view in chronology ]
P2P - solution
The information need to be stored somewhere. What about storing it everywhere. Not sure exactly how it would work, but lets say everyone holds a copy of the info, stored on local devices, like P2P networks. As long as one device functions, the info can be restored.
Lets look forward in time, let say 5 years, 10, 50..
What will storage capazity be like at that time? What will network transfer speed be like? Would it be possible with the kind of solution im thinking of?
[ link to this | view in chronology ]
Re: P2P - solution
Kind of like the digital currency "Bitcoins" seem to function?
[ link to this | view in chronology ]
Re: P2P - solution
Look at Omemo and see how it works or Osiris.
Everybody could help save part of their heritage storing bits and pieces of it.
[ link to this | view in chronology ]
On a practical note one could clean some information on how certain years fared based on the language used, or see trends developing slowly.
[ link to this | view in chronology ]
[ link to this | view in chronology ]
even Techdirt
[ link to this | view in chronology ]
part of what makes the past "the past" is that it has faded from our collective memory. we should let these things die. with 100% recall we would never be able to escape our past. right now institutions like the smithsonian keep and display the best of the best and the worst of the worst. this works pretty well. if we keep everything then there is no nostalgia or mystery. instead of keeping everything we should make an effort to randomly delete three-fourths of all archives out there. if certain facts and accounts are important they will survive because they are better documented.
[ link to this | view in chronology ]
Research project on personal digital archiving
http://explorer.cyberstreet.com/CET4970H-Peterson-Thesis.pdf
To anyone who is interested in looking at these issues in depth, please feel free to download and read.
[ link to this | view in chronology ]
Saved private pages. Hahahaha. Did you ever think you (you yourself, personally, sitting in a chair looking at the screen.) were making history? It is likely that your saved locally Facebook/other private non trollable web page in your computers temp cash is the last remembrance from your dead friend (Condolences, it happens). Will you be able to access it? (love the pages that don't require a refresh.)
[ link to this | view in chronology ]