Distributed Backups?

from the another-solutions dept

Having lost way too many hard-drives in the last couple years, I've become pretty focused on backup systems, and (judging from a few conversations here at DEMOmobile) it's a subject that a lot of people are interested in. Bob Cringely's latest column is another one of his "big idea for a company" suggestions that he simply tosses out and hopes someone builds: a peer-to-peer distributed backup system. The idea is pretty simple. You download some backup software which encrypts the data you want to backup, breaks it up into little chunks and moves it out over a peer-to-peer network. The amount of data you can backup is limited by the amount you agree to store yourself. In Cringely's plan, this whole thing should cost $3.95/month. Of course, you'd need to distribute many copies of the various pieces to make sure that there aren't chunks missing -- which will also mean that many people will need to be in the system before it's really all that reliable. Obviously, some people will freak out about chunks of their data (even if they are encrypted) sitting on random hard drives out there). In the meantime, it appears Cringely doesn't use Google, because a quick search on the topic suggests at least some people are already working on this type of technology.
Hide this

Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.

Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.

While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.

–The Techdirt Team


Reader Comments

Subscribe: RSS

View by: Time | Thread


  • identicon
    Curmudgeon, 10 Sep 2004 @ 2:03pm

    Delta

    Crucial to this technology will be the truly incremental file transfer: just sending those bytes which have changed. As files get bigger and bigger, it's important to reduce the overall traffic.

    link to this | view in chronology ]

  • identicon
    Anonymous Coward, 10 Sep 2004 @ 3:00pm

    No Subject Given

    Nah, this would be a huge security risk, and hey at only 128Kbps upload, would not be nice even on DSL.

    I've tried Ghost 7/2003 and Acronis TrueImage6, but DriveImage7 (now owned by Symantec, and seems to be worked into Ghost9) seems to be the nicest so far, but still a little buggy (it's a known bug that if you tick the "checksum image when done" box, it will render the image useless).

    I'd really like something that would work across Windows, and all the new journaled Linux filesystems, but haven't found anything that will do more than FAT32/NTFS/ext3.

    The latest trend is allowing you to do the backup in Windows - which means you can write the image to any device, as you have the drivers (not some stupid DOS boot floppy).

    Someone needs to write something that will just copy whatever is on a HDD (regardless of format) to any device you want (DVD-RW, HDD, LAN) just with a simple boot CD. It must however retain things like the boot sector, MBR, and partition information (you ever tried to ghost a dual booting drive?!)

    I guess the best way is to just get one of those hardware drive duplicators, but they cost thousands and you need two identical drives.

    link to this | view in chronology ]

  • identicon
    jayrtfm, 10 Sep 2004 @ 4:34pm

    quote

    "Backups are for wimps. Real men upload their data to an FTP site and have everyone else mirror it."
    - Linus Torvalds

    link to this | view in chronology ]

  • identicon
    Whit, 10 Sep 2004 @ 5:29pm

    Onion Networks (Justin Chapweske)

    Justin Chapweske (who authored SwarmCast, a key component of OpenCola's late, lamented plan for world domination) has done a lot of work that could be (or may have already been) applied to this. He likes distributing things.

    I know it's probably something that most people know already, but just in case: it's also worth digging into "forward error correction" a bit as a mechanism for offsetting the impact of the "ensure that you aren't missing a key piece of your data" concerns.

    link to this | view in chronology ]

  • identicon
    DV Henkel-Wallace, 11 Sep 2004 @ 10:28am

    You can buy this product already!

    This product has been on the market for a while. Go to http://www.hivecache.com/. No, I am not connected with them.

    link to this | view in chronology ]

  • identicon
    LM, 11 Sep 2004 @ 1:16pm

    Done deal

    A student of mine developed a system to do this a couple of years ago. All encrypted and redundant so that missing machines were not a problem. He won "best student project in the UK" for it in fact. There are commercial versions available too, though not with quite the technical sophistication that our system has (AFAIK).

    link to this | view in chronology ]

  • identicon
    Anonymous Troll, 12 Sep 2004 @ 5:57am

    Corporate Intelligence?

    Jesus... hive cache has been arond so long I don't really know how Cringely or Tech Dirt can sleep at night after trying to pass themselves off as credable tech pundits, let alone "Corporate Intelligence".

    Still, the reality of the matter is that the lead time that hive cache had will soon be comming to a close. First group to successfully build a nice, well integrated windows protduct that they can sell out to Veritas or any one of the 10+ other backup product makers out there will win and become the new industry standard in backup...

    Of course, when encrypted P2P hits the streets with anonymous distribution of keys via DNS, the RIAA is going to get a nifty little introducton to how the internet routes around damage.

    link to this | view in chronology ]

    • icon
      Mike (profile), 13 Sep 2004 @ 12:36am

      Re: Corporate Intelligence?

      Hive Cache has been around so long... that all of their website pages read "coming soon" and give no other details. Horror or horrors that we hadn't heard of it before.

      link to this | view in chronology ]

      • identicon
        Anonymous Coward, 13 Sep 2004 @ 5:37am

        Re: Corporate Intelligence?

        Just because you weren't invited to the party doesn't mean that it didn't happen.

        private invitations for pre-release evaluation happened over a year ago.

        dumbass...

        link to this | view in chronology ]

        • identicon
          Ric, 13 Sep 2004 @ 8:29am

          Re: Corporate Intelligence?

          Anonymous Troll/Coward/Idiot,

          Wow! A pre-release evaluation over a year ago, and they have nothing new to offer on their web site. Really promising stuff!

          link to this | view in chronology ]

      • identicon
        Lars Kellogg-Stedman, 7 Oct 2004 @ 8:06pm

        Re: Corporate Intelligence?

        Mike, HiveCache has been around for *years*, first as p2p system "Mojo Nation" and later on as HiveCache, selling a distributed backup solution. The web site looks like it's currently being renovated in preparation for some product announcements, but if you follow the p2p field at all you would have been familiar with these folks from a long way back.

        link to this | view in chronology ]

        • identicon
          Sekar Vembu, 20 Dec 2004 @ 1:38am

          Re: Corporate Intelligence?

          We too are working on a P2P backup product StoreGrid, which is currently in Beta. Plan to release it around end of Jan 2005. StoreGrid 1.0 version is meant for use inside a corporate intranet. StoreGrid 1.0 Beta is available for download. Supports Encryption, Compression, Incremental Backup, Registry Backup, System Files Backup, Continuous Backup, Quick Restore etc. Works on Windows and Linux. Mac OS X will be supported in a few months.

          Check out
          http://www.vembu.com/storegrid/desktop-backup.html

          link to this | view in chronology ]

  • identicon
    Jean-Alain Le Borgne, 13 Sep 2004 @ 5:31am

    P2P Backup

    And these people too: LeanOnMe by 312, Inc.

    link to this | view in chronology ]

  • identicon
    Jean-Alain Le Borgne, 13 Sep 2004 @ 5:32am

    P2P Backup

    And these people too: LeanOnMe by 312, Inc.

    link to this | view in chronology ]

  • identicon
    Tim Macinta, 22 Sep 2004 @ 11:23am

    No Subject Given

    I've actually been working on something like this for awhile and you are welcome to download it at http://www.pensamos.com/mmb/ Version 1.0 is free. The peer network exists within your office, so you are just sharing with other computers in your office and your data therefore doesn't leave your network unless you want it to.

    There is also an optional off-site component to this, so that you can additionally backup to a remote server to protect against office wide failures like fire, theft, or hurricanes. Your backups are still encrypted, so the data is still secure. Present plans are to run the off-site servers for this as a paid service (with the p2p backup remaining free at least for version 1.0). However the server software is written in PHP and can easily be run on any commodity $3/month web-hosting service, and I might consider just releasing the server code for people to use on their own if there is an interest in that.

    I would greatly appreciate any and all feedback on this program. I really want this program to evolve based on what users out there want, and I realize that the current version doesn't have every ideal feature, so I would love to hear what people would like to have added first. You can send me feedback at twm@pensamos.com

    Thanks,
    - Tim Macinta

    link to this | view in chronology ]

  • identicon
    Vikki Fraser, 7 Oct 2004 @ 11:06am

    Not exeactly the 3 bucks Cringley wants...

    First and foremost, I am associated with 312 Inc. makers of LeanOnMe P2P Backup and remote access software. I am their marketing director. Now that I've said that... LeanOnMe is in it's initial release version 1.0. Shoot me an email in order to receive a free key. vikki_dot_fraser_at_312inc_dot_com. and visit http://www.312inc.com/3_sol_prod_LOM.html for the smiling happy people page or http://www.312inc.com/tech.htm for a more technical view.

    link to this | view in chronology ]

  • identicon
    Travis Reeder, 28 Mar 2005 @ 12:22pm

    I made a system like this for a University project

    Info and reports at: http://pages.cpsc.ucalgary.ca/~reeder/cpsc502/

    It is extremely fault tolerant too since it uses correction codes.

    link to this | view in chronology ]


Follow Techdirt
Essential Reading
Techdirt Deals
Report this ad  |  Hide Techdirt ads
Techdirt Insider Discord

The latest chatter on the Techdirt Insider Discord channel...

Loading...
Recent Stories

This site, like most other sites on the web, uses cookies. For more information, see our privacy policy. Got it
Close

Email This

This feature is only available to registered users. Register or sign in to use it.