Cheap Storage, Digital Pack Rats... Opening Up New Possibilities

from the everything-must-stay dept

Tue, Sep 11th 2007 6:24am — Mike Masnick

Back when Google first launched Gmail, the big revolution wasn't necessarily in the interface, but the fact that it was offering 1 Gb of storage for online email, with the announcement noting that you should never have to delete email again. It turns out they were a little hasty in that claim, and even though the company has added a lot more storage, some people still found themselves running out of room and deleting stuff (and now even Google is charging extra for additional storage). However, it certainly is reaching the point where digital storage is so cheap that many companies are finding it easier to just save everything possible. There's no reason to throw out data any more. This has certainly resulted in some problems, such as when companies leak data -- but it's also leading to new potential tools and services, such as much better computer learning systems, that can simply process tons and tons of stored data to figure things out. So, perhaps the quest for better artificial intelligence will actually be served better not by all the attempts at better algorithms or other tricks -- but by the fact that there's so much data out there that no one's getting rid of, that we'll be able to create machines that learn how to be intelligent by going through all of it.

Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.

Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.

While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.

–The Techdirt Team

Filed Under: machine learning, storage

21 Comments

If you liked this post, you may also be interested in...

Reader Comments

Subscribe: RSS

View by: Time | Thread

angry dude, 11 Sep 2007 @ 6:45am

shut up Mike
Mike, for God's sake...

WTF are you talking about, dude ?
You just don't have a fucking clue, dude
You are as clueless as any punk from the street.
Get a life, dude !!!!!!!!!!!!!!!!!
[ link to this | view in chronology ]
- mr. whatever, 11 Sep 2007 @ 7:40am
  
  Re: shut up Mike
  Wow...that was a well formulated argument against Mike's article. How did you ever come up with such compelling reasons for ripping on someone else's opinion? Hey, dude, perhaps you should consider the possibility that for a great deal of people it's becoming increasingly easier to simply keep copies of everything as opposed to having to delete them.
  
  It's not a far fetched idea, though it may be a bit premature. I've got ~6TB of home storage an it's almost completely filled with pictures, photoshoppings, home movies, and DVD backups (MCE is a nice thing when paired with MyMovies, no more sifting through bookcases of DVDs to find what I want).
  [ link to this | view in chronology ]
  - Read TFA, 11 Sep 2007 @ 7:44am
    
    Re: Re: shut up Mike
    Also, systems like those that beat Kasparov at chess back in the day were built on depth first searching of possible chess moves...just a huge database of things that it could do. The larger the DB and the faster the searching, the better a system will perform at chess. Unless, of course, someone can actually come up with a real learning algorithm, then sufficient learning will probably perform as well as a big calculator.
    [ link to this | view in chronology ]
  - Boost, 11 Sep 2007 @ 9:17am
    
    Re: Re: shut up Mike
    Not to rip on you or anything, but do you have a life outside of your computer? 6 Terabytes? Are you kidding me? I've been making due with less than 50 Gigabytes for the last 4 years. Yeah, I don't really download at all, but damn dude what the hell are you trying to do, back up the internet?
    [ link to this | view in chronology ]
    - Bill Gates, 11 Sep 2007 @ 9:32am
      
      Re: Re: Re: shut up Mike
      But 640k ought to be enough for anyone!
      [ link to this | view in chronology ]
    - angry dude, 11 Sep 2007 @ 9:32am
      
      Re: Re: Re: shut up Mike
      The dude is trying to contribute to the profits of hard drive manufacturers out of his monthly allowance
      Good boy
      [ link to this | view in chronology ]
      - Charles Griswold, 11 Sep 2007 @ 7:53pm
        
        Re: Re: Re: Re: shut up Mike
        The dude is trying to contribute to the profits of hard drive manufacturers out of his monthly allowance
        Good boy
        
        Do you have any actual proof of your allegations, or have you just been smoking tire rubber?
        [ link to this | view in chronology ]
    - Joel, 11 Sep 2007 @ 11:56am
      
      Re: Re: Re: shut up Mike
      That's a crap load of porn.
      [ link to this | view in chronology ]
One Love, 11 Sep 2007 @ 7:08am

Need better desktop Search
I am on a Windows XP box, and the search function to go through my rat pack hard drives filled with absolutely blows. Desktop search on the my girlfriend Mac OS X sucks as well.

I know Google has the desktop thingy, but I am loathe to open the deeper recesses of my digital attic to the AI that has become Google, ever though I am a fan.

Personal spiders and bots ? Something a bit more accurate than Clippy ?

:(
[ link to this | view in chronology ]
- Bob, 11 Sep 2007 @ 7:21am
  
  Re: Need better desktop Search
  NOTHING, I repeat NOTHING ismore accurate than Clippy. He is the manifestation of awesomeness.
  [ link to this | view in chronology ]
- ROFL @ one love, 11 Sep 2007 @ 12:27pm
  
  Re: Need better desktop Search
  ROFL you newb, that isn't AI it is just a search engine, the AI thing he is talking about has been around for years and basically it searches through the data and stores it to memory for processing, it learns from processing and relating everything it has stored.
  [ link to this | view in chronology ]
- I too am slighty scared... but, 11 Sep 2007 @ 1:00pm
  
  Re: Need better desktop Search
  Google Desktop really is, in my humble geek opinion, the best there is.
  
  Has anyone found any "phone-home" indications or vulnerabilities in the app? I will definetly be avoiding the "Search content across multiple computers" though, it's just a bad idea, IMO.
  [ link to this | view in chronology ]
B, 11 Sep 2007 @ 7:46am

Clippy's dead
I'm runing on Vista and Clippy is dead. I shot the dead horse.
The Vista search is pretty good (for what I've put it through so far... Although the only time I even use search is in the desperate attempt to find something I deleted.
[ link to this | view in chronology ]
TheDock22, 11 Sep 2007 @ 7:49am

Content = AI?
I sure hope not or my computer would be one messed up hunk of junk. :D
[ link to this | view in chronology ]
- Poomer, 11 Sep 2007 @ 10:19am
  
  Re: Content = AI?
  Thank god its not! Sorry, Mike you are waaayyy off here mate.
  [ link to this | view in chronology ]
Shohat, 11 Sep 2007 @ 7:52am

Hello Mike
You post made no sense. Not only did it not make sense , it was misleading.
So, perhaps the quest for better artificial intelligence will actually be served better not by all the attempts at better algorithms or other tricks -- but by the fact that there's so much data out there that no one's getting rid of,
Is an insult to intelligence(no pun intended).
While one of AI's most important forms is actually data compression(you know, algorithms+tricks !) , you have no idea how off the target you are.
You know that a baby is born an intelligent being right? It is able to learn languages from scratch, and without any "translation" but he's own logic.
Intelligence != knowledge.
Clearly, the thing you think AI is , is just not AI.
[ link to this | view in chronology ]
- Poomer, 11 Sep 2007 @ 10:50am
  
  Re: Hello Mike
  Taking a class or reading up on AI might have helped here. Knowledge representation is only a fragment of what AI consists of. DATA
  [ link to this | view in chronology ]
- Mike (profile), 11 Sep 2007 @ 5:06pm
  
  Re: Hello Mike
  While one of AI's most important forms is actually data compression(you know, algorithms+tricks !) , you have no idea how off the target you are.
  
  Well, it's great to know that there are folks around who are always willing to try to set me straight. Though, I'm not sure I understand your complaint here.
  
  You know that a baby is born an intelligent being right? It is able to learn languages from scratch, and without any "translation" but he's own logic.
  Intelligence != knowledge.
  
  I wasn't saying that AI was in the data alone. But, as the article stated, the fact that more data is available makes it that much easier to train new AI systems and to test out larger scale machine learning systems.
  [ link to this | view in chronology ]
Cooperate Storage., 11 Sep 2007 @ 9:36am

Information Age.
Hello people, we are not talking about our own home/personal storage, I have a ton myself, but it's nothing compared to cooperate databases and important documents that they create in everyday business. Also studies and lab work from research companies, video data from whatever application used some. Also our world information doubles almost every 14 months from the previous year, just think about about how much info there is out there. Yes it is true that we do need intelligence to learn from knowledge, but AI in theory if developed properly is capable of learning in it's own way.
[ link to this | view in chronology ]
silver, 11 Sep 2007 @ 11:53am

Do you work for the man?
It sounds like Mike works for the government trying to datamine everything on everone. While it might be easier to keep everything, lets pretend it's like the old shoeboxes of transactions people used to have, and not some monster database where someone can find out everything from what movies I watch to what toothpaste I like, to what condoms I use! The last thing I need is a jelous governmental emplyee looking through my data!
[ link to this | view in chronology ]
- Mike (profile), 11 Sep 2007 @ 5:08pm
  
  Re: Do you work for the man?
  It sounds like Mike works for the government trying to datamine everything on everone.
  
  I've heard all sorts of false conspiracies about who I "work" for, but this is a first. This has to be the funniest blind accusation I've read in quite some time.
  
  While it might be easier to keep everything, lets pretend it's like the old shoeboxes of transactions people used to have, and not some monster database where someone can find out everything from what movies I watch to what toothpaste I like, to what condoms I use! The last thing I need is a jelous governmental emplyee looking through my data!
  
  You seem to have jumped to quite the paranoid conclusion. All I pointed out was that it was easier to keep data, so many companies are keeping data -- and that's allowing them to do all sorts of interesting datamining activities. That has nothing to do with the gov't or whether or not someone's spying on you.
  [ link to this | view in chronology ]