Defense Department Spied On Social Media, Left All Its Collected Data Exposed To Anyone

from the not-cool-guys dept

There are two big WTFs in this story. First, the Defense Departments Central Command (Centcom) was collecting tons of data on social media posts... and then the bigger one, they somehow left all the data they collected open on an Amazon AWS server. This was discovered -- as so many examples of careless data exposure on Amazon servers -- by Chris Vickery and UpGuard, who have their own post about the mess. You may recall Vickery from such previous stories as when the GOP left personal data on 200 million voters on an open Amazon server. Or when Verizon left private data available on millions of customers. Or when a terrorist watch list was left (you guessed it) on an open server. Or when he discovered that Hollywood studios were leaving their own screeners available on an open server. In short, this is what Vickery seems particularly good at: finding large organizations leaving sensitive data exposed on a server.

You would think (wouldn't you?) that Centcom would be better about these things than, say, Verizon or the GOP or Hollywood. But, nope.

"[It's] a pretty serious leak when you're talking about intelligence information being stored in an Amazon cloud service and not properly safeguarded," said Timothy Edgar, a former White House official in the Obama administration and former U.S. intelligence official.

Centcom's response is... sketchy. It uses the important term "unauthorized access," which suggests that it may be pushing for CFAA charges against Vickery/Upguard, since "unauthorized access" is a key part of the CFAA:

"We determined that the data was accessed via unauthorized means by employing methods to circumvent security protocols," said Maj. Josh Jacques, a spokesperson for U.S. Central Command. "Once alerted to the unauthorized access, Centcom implemented additional security measures to prevent unauthorized access."

But if it was truly left open, then the access was not "unauthorized." Indeed, it appears that Centcom went for convenience over security by making its Amazon S3 bucket open for access, and hoping obscurity would hide it.

Amazon servers where data is stored, called S3 buckets, are private by default. Private means only authorized users can access them. For one to be made more widely accessible, someone would have to configure it to be available to all Amazon Web Services users, but users would need to know or find the name of the bucket in order to access it.

By searching specific keywords, Vickery identifies information that companies and organizations inadvertently expose. In this case, he looked for buckets containing the word "com."

Three S3 buckets were configured to allow anyone with an Amazon Web Services account to access them. They were labeled "centcom-backup," "centcom-archive" and "pacom-archive," Vickery said.

As for just what Centcom was doing here -- it does appear that it was publicly available social media content, so that's less of a direct concern, but it still does make you wonder why Centcom was storing all of this social media info. There are also, of course, related concerns about the US Defense Department conducting surveillance on Americans. This is from Upguard's post on the matter (linked above):

The data exposed in one of the three buckets is estimated to contain at least 1.8 billion posts of scraped internet content over the past 8 years, including content captured from news sites, comment sections, web forums, and social media sites like Facebook, featuring multiple languages and originating from countries around the world. Among those are many apparently benign public internet and social media posts by Americans, collected in an apparent Pentagon intelligence-gathering operation, raising serious questions of privacy and civil liberties.

While a cursory examination of the data reveals loose correlations of some of the scraped data to regional US security concerns, such as with posts concerning Iraqi and Pakistani politics, the apparently benign nature of the vast number of captured global posts, as well as the origination of many of them from within the US, raises serious concerns about the extent and legality of known Pentagon surveillance against US citizens. In addition, it remains unclear why and for what reasons the data was accumulated, presenting the overwhelming likelihood that the majority of posts captured originate from law-abiding civilians across the world.

I know that the US government still has this "collect it all" mentality, but as we've discussed over and over again, adding more hay to the haystack doesn't make it easier to find the needles.

Hide this

Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.

Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.

While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.

–The Techdirt Team

Filed Under: amazon s3, aws, centcom, chris vickery, defense department, exposed, social media, surveillance
Companies: amazon, upguard


Reader Comments

Subscribe: RSS

View by: Time | Thread


  • identicon
    Anonymous Coward, 17 Nov 2017 @ 11:07am

    The only winning move is not to play.

    Social media is evil.

    link to this | view in chronology ]

    • identicon
      Anonymous Coward, 17 Nov 2017 @ 11:12am

      Re: The only winning move is not to play.

      your participation is involuntary.

      like Equifax... just having done anything will get you tracked in some manor or another. Every person, business, website, government agency you interact with are sharing your information without your permission or knowledge.

      We know who you are...

      If you want to fool the system... make the haystack bigger, not smaller!

      link to this | view in chronology ]

  • identicon
    Anonymous Coward, 17 Nov 2017 @ 11:13am

    The word is "published"

    "But if it was truly left open, then the access was not "unauthorized."

    By placing this data where they did, and leaving it open to access by anyone, and not making any attempt whatsoever to secure it, they published it.

    They may not have wanted to publish it, they may not have known they published it, but they did.

    And if you publish something to the planet, you can't really complain that people read it.

    link to this | view in chronology ]

    • identicon
      Anonymous Coward, 17 Nov 2017 @ 11:19am

      Re: The word is "published"

      an open door is access granted

      link to this | view in chronology ]

      • icon
        Bergman (profile), 18 Nov 2017 @ 11:43am

        Re: Re: The word is "published"

        This wasn't an open door. This was more like knocking on a door, and when the resident answers, asking to come in.

        If the request to enter is granted, then the access cannot be anything but authorized.

        link to this | view in chronology ]

    • identicon
      Anonymous Coward, 17 Nov 2017 @ 11:49am

      Re: The word is "published"

      "not making any attempt whatsoever to secure it, they published it."

      Consider how that view applies to citizens and not just the state. Most people make zero effort to secure digital data.

      IMHO the guy did was a public service. In his case it could be argued that such an approach was just modern investigative reporting. Not that it will keep him out of the klink. But it would at least start people talking about where the line actually is.

      But they wont.

      My guess is it would just be one more double-speak precedent that confounds both the law, and systems engineering. It would be an interesting case to follow if you weren't compelled to wretch every time a lawyer tried to analogize data concepts.

      Truth is state. Data is accumulated state. Law is an attempt to understand data. Law is therefore more abstract than data, yet it presumes to precede it in all matters. Such arrogance makes for bad code. Digital and legal.

      link to this | view in chronology ]

  • icon
    Uriel-238 (profile), 17 Nov 2017 @ 11:18am

    Because OF COURSE THEY DID.

    We already knew that we're being spied on.

    We also already knew that our government's agencies are totally inept when it comes to net and data security (on account of the many, many successful hacks).\

    2 + 2 = Oh fuck!

    link to this | view in chronology ]

    • identicon
      SirWired, 17 Nov 2017 @ 1:00pm

      Does this really count as spying?

      Does it count as spying when they record copies of something people publicly post on the internet?

      link to this | view in chronology ]

  • identicon
    SirWired, 17 Nov 2017 @ 11:20am

    This doesn't seem like a big deal

    This is a little sloppy, but it was just a collection of publicly available information that anybody that cared to could have assembled. It's not exactly Top Secret stuff here; I'll bet it was FOUO, if it was classified at all.

    And this doesn't raise any civil-liberties questions at all. If you post something on the internet for all to see, then there's no civil-liberties implications to the government including themselves as part of "all". They can use this data for whatever purposes they like, just like you, citizen, can.

    link to this | view in chronology ]

    • identicon
      Anonymous Coward, 17 Nov 2017 @ 11:56am

      Re: This doesn't seem like a big deal

      >And this doesn't raise any civil-liberties questions at all.

      Oh yes it does, it show a government that wishes to totally control its citizens.

      link to this | view in chronology ]

  • identicon
    Anonymous Coward, 17 Nov 2017 @ 11:21am

    Re: unauthorized access

    "being stored in an Amazon cloud service and not properly safeguarded," said Timothy Edgar, a former White House official in the Obama administration and former U.S. intelligence official".

    -cringe-. How about, "intelligence data shouldn't have been on a non-federal server to begin with."

    Not sure who the intel official is, or even if he is, but what he said more than indicates that he is part of the problem.

    Second, we already know what unauthorized access means. It means whatever the federal government says it does at this time and place without any consideration for stare decisis.

    You can't know your right if you don't understand the context in which you speak. The lack of understanding therefore resolves to "right" simply as a matter of declaration.

    Given that the courts can not resolve the modern data driven concept of truth in any practical way; perhaps we should do away with precedent? There are western countries that do. And a shot in the dark may be better than the progressively accumulating "because I said so" precedents with random and laughable justifications.

    link to this | view in chronology ]

  • identicon
    Personanongrata, 17 Nov 2017 @ 12:12pm

    One in the Same

    ... raises serious concerns about the extent and legality of known Pentagon surveillance against US citizens.

    To place things in proper context the National Security Agency (NSA) is actually is part of the US Department of Defense (ie the Pentagon).

    Whether the criminal/unconstitutional surveillance against US citizens occurs within NSA, NRO, NGA, etal they all operate within the Pentagon's chain of command.

    The italicized/bold text below was excerpted from the website NSA.gov:

    The National Security Agency is part of the U.S. Department of Defense, serving as a combat support agency.

    https://www.nsa.gov/what-we-do/support-the-military/

    The italicized/bold text below was excerpted from the website NGA.mil:

    In its multiple roles, NGA receives guidance and oversight from DOD, the Director of National Intelligence (DNI) and Congress.

    https://www.nga.mil/About/Pages/Default.aspx

    The italicized/bold text below was excerpted from the website NRO.gov

    The Director of the NRO is appointed by the Secretary of Defense (SECDEF) with concurrence of the Director of National Intelligence.

    http://www.nro.gov/about/leadership/index.html

    link to this | view in chronology ]

  • identicon
    Anonymous Coward, 17 Nov 2017 @ 12:27pm

    I could stop all those terrorists, if I just had, this little extra thing. Please.

    link to this | view in chronology ]

    • identicon
      Anonymous Coward, 17 Nov 2017 @ 12:51pm

      Re:

      Too bad there's not a public backlash of "We gave you this little extra thing and the terrorists still haven't been stopped. So now we're taking back this little extra thing!"

      link to this | view in chronology ]

  • identicon
    Anonymous Coward, 17 Nov 2017 @ 4:25pm

    Amazon sieve security

    Here is another security breach on Amazon's servers.

    http://www.abc.net.au/news/2017-11-17/abc-data-leaked-online-discovered-by-ukrainian-firm/91 59022

    Even if it is the users of the service doing the wrong thing then Amazon's cloud data storage services sure are getting a bad name.

    link to this | view in chronology ]

  • identicon
    Anonymous Coward, 17 Nov 2017 @ 4:49pm

    Obscurity is not security

    link to this | view in chronology ]

  • identicon
    Pixelation, 17 Nov 2017 @ 5:15pm

    The best offense is a good defense. Looks like we're screwed.

    link to this | view in chronology ]

  • identicon
    Anonymous Coward, 18 Nov 2017 @ 10:26am

    "Circumvent Security Protocols"?

    "We determined that the data was accessed via unauthorized means by employing methods to circumvent security protocols," said Maj. Josh Jacques, a spokesperson for U.S. Central Command.

    So, CentCom's view is that, if they didn't announce the location of the unsecured data by taking out a full-page ad in the NY Times, access was a circumvention of security. At the very least, JJ needs to repair his benightedness by reading Untangling the Web [ https://www.nsa.gov/news-features/declassified-documents/assets/files/Untangling-the-Web.pdf ].

    link to this | view in chronology ]

  • icon
    Richard Bennett (profile), 19 Nov 2017 @ 12:55pm

    Copyright violation!

    Was Centcom licensed to publish these posts? I don't think so.

    This is why we need something like SOPA.

    link to this | view in chronology ]


Follow Techdirt
Essential Reading
Techdirt Deals
Report this ad  |  Hide Techdirt ads
Techdirt Insider Discord

The latest chatter on the Techdirt Insider Discord channel...

Loading...
Recent Stories

This site, like most other sites on the web, uses cookies. For more information, see our privacy policy. Got it
Close

Email This

This feature is only available to registered users. Register or sign in to use it.