Court Says Metadata Should Be Released Under Freedom Of Information Act Request

from the commence-metadata-scrubbing dept

Thu, Feb 17th 2011 12:55am — Mike Masnick

Copycense points us to the fascinating news that a federal judge has ordered Immigrations and Customs Enforcement (ICE) to reveal the metadata on a document as part of a Freedom of Information Act (FOIA) request. ICE had responded to the FOIA request (apparently "after significant delay,") but provided the content requested in an unsearchable PDF. The original requestor for the content, the National Day Laborer Organization, complained that this was unfair, and the information had to be supplied with metadata -- and the court agreed. The court also agreed that making the PDF unsearchable was not justified:

"Metadata maintained by the agency as a part of an electronic record is presumptively producible under FOIA, unless the agency demonstrates that such metadata is not 'readily producible.'"

Sounds like some government employees are going to need to spend the next few weeks scrubbing metadata from documents. Wouldn't want people to find out who really wrote various laws by looking at the metadata on Word docs, would we?

As for the unsearchable format, the judge slammed ICE for clearly going out of its way to make the document "more difficult or burdensome for the requesting party to use," in violation of standard discover rules. Nice to see that ICE has the time to purposely obfuscate records requested in a FOIA. Transparency in action...

Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.

Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.

While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.

–The Techdirt Team

Filed Under: freedom of information, metadata

19 Comments

If you liked this post, you may also be interested in...

Reader Comments

Subscribe: RSS

View by: Time | Thread

Anonymous Coward, 17 Feb 2011 @ 3:23am

My thought is that this looks more like a fishing expedition, trying to find alternate versions, previous versions, or even comments that might have been inserted during the review process.

It sets a very ugly precedent.
[ link to this | view in chronology ]
- Anonymous Coward, 17 Feb 2011 @ 3:29am
  
  Re:
  Ugly for people trying to hide something.
  [ link to this | view in chronology ]
  - Anonymous Coward, 17 Feb 2011 @ 4:03am
    
    Re: Re:
    No, ugly for anyone working on a document in it's development stages. It will limit people's desire to make comments, to offer ideas, or to disagree with anything in the document because they will not want their objection to be part of the metadata.
    
    It is a huge negative to the entire process.
    [ link to this | view in chronology ]
    - Anonymous Coward, 17 Feb 2011 @ 4:52am
      
      Re: Re: Re:
      If people are afraid to do what is right and have no courage to voice those disagreements then I see no problem with them being silent, this is above all very true for the government.
      
      Besides they can always comment anonymously using internal pseudonyms or other internal references.
      [ link to this | view in chronology ]
      - Anonymous Coward, 17 Feb 2011 @ 5:09am
        
        Re: Re: Re: Re:
        But it's not fair for the government!
        [ link to this | view in chronology ]
        
        Anonymous Coward, 17 Feb 2011 @ 8:56am
        
        Re: Re: Re: Re: Re:
        Have faith. Transparency is fair.
        [ link to this | view in chronology ]
    - Michael, 17 Feb 2011 @ 5:12am
      
      Re: Re: Re:
      "It is a huge negative to the entire process"
      
      What is a huge negative to the process? Making the process itself transparent? Are they really throwing out comments in these documents that are not appropriate for public consumption?
      [ link to this | view in chronology ]
    - Anonymous Coward, 17 Feb 2011 @ 5:37am
      
      Re: Re: Re:
      Tough shit. Have some cojones. If your objections are good and they are discovered in the metadata, you might save yourself from a prison sentence or a the very least being associated with a crappy law.
      [ link to this | view in chronology ]
      - Anonymous Coward, 17 Feb 2011 @ 8:17am
        
        Re: Re: Re: Re:
        Apparently you have never been in the room when legal theories and possibly courses of action are discussed. It's sort of like the comments here, all over the road.
        
        Requiring full disclosure on every step and every part of the document, no matter what the end result is very likely to stop people from making suggestions, or at least stop them from using official channels to do so.
        
        It's the same logic as Piracy. Block protocol X, and they try a new one with more covering and more privacy. If you buy that logic, you know exactly what the government people are going to do in the future: Use undocumented ways of discussing and working on product so their intermediate workflow stuff cannot be easily obtained.
        [ link to this | view in chronology ]
- velox (profile), 17 Feb 2011 @ 8:51am
  
  Re:
  "this looks more like a fishing expedition"
  The discovery phase of any lawsuit these days is almost always a "fishing expedition".
  That's what lawyers do.
  
  If you don't like it, stop using software that hides a bunch of metadata.
  For some reason, software engineers have this desire to store a bunch of metadata inside working files, and it rarely provides serves a purpose or benefit to the enduser.
  MS Office is one of the worst, but it's a widespread issue.
  [ link to this | view in chronology ]
Rekrul, 17 Feb 2011 @ 3:35am

As for the unsearchable format, the judge slammed ICE for clearly going out of its way to make the document "more difficult or burdensome for the requesting party to use," in violation of standard discover rules.

My first thought is that they simply scanned the documents as images and then compiled those images into a PDF. I see this done all the time. It's faster than using OCR and then cleaning up the hundreds of mistakes that the software makes, however, images aren't searchable.
[ link to this | view in chronology ]
- John Duncan Yoyo, 17 Feb 2011 @ 5:08am
  
  Re:
  Presumably ICE has an electronic version of this document somewhere in the system. Giving the rescan into a PDF would have had the same effect as a printout.
  
  I would love all laws to come with meta data. This pork is identified with this congressman.
  [ link to this | view in chronology ]
abc gum, 17 Feb 2011 @ 6:09am

ice ice baby
[ link to this | view in chronology ]
Shon Gale (profile), 17 Feb 2011 @ 6:11am

Adobe will love that! Make it into a pdf to make it harder to use!
[ link to this | view in chronology ]
Borax Bob, 17 Feb 2011 @ 7:38am

Maybe Not Intentional
Obviously I have no idea if this is true or not, but where I work, probably over 50% of the people, who have a PDF "printer" installed on their computer of one sort or another, will still print out something, and then walk over and scan it. Even large documents. Heck, They have a 80 page change to a 400 page policy they put out a year ago; the original policy was searchable, the changes to it aren't and the scan isn't even all that good. It looks like someone wanted to highlight parts of the changes, so they printed it out, used a highlighter to do the highlighting, and then scanned it. I guess it makes sense then that the page numbers don't match, the "Changes" appear to actually have been made to the entire original document, which is distributed electronically, why only the changes were put out, I have no idea. I was assured the changes were temporary, but it's been over a year....
My point is, it might be (or, since it's the Federal Government, it probably is) just plain incompetence on someone's part.
[ link to this | view in chronology ]
Rabbit80, 17 Feb 2011 @ 7:48am

DMS
I work with in a scanning bureau and with various document management systems. We get sent tons of paperwork (mostly typed) for scanning which is presumably no longer available in its original (ie Word etc) form.

Some of the document management software we use will not produce searchable PDFs. The images are stored as single page TIFF with an accompanying XML or TXT file - not much use to anyone in that format! To make the PDFs searchable, they have to be run through a separate OCR process after exporting.

This software is also capable of redacting the image files before exporting them.

Its pretty easy to see why non-searchable PDFs may have been given.
[ link to this | view in chronology ]
- Rabbit80, 17 Feb 2011 @ 7:52am
  
  Re: DMS
  I forgot to mention. We do not make PDFs searchable when scanning unless we are specifically asked to do so. It is a time consuming process (2 seconds per page. We can scan 80000+ pages per day) that can cause the processing PCs to crash frequently.
  [ link to this | view in chronology ]
Anonymous Coward, 17 Feb 2011 @ 8:02am

Usually pretty easy to make a PDF searchable as long as it is in the form of an electronic file (unless the conversion gives you the dreaded "this document contains non-renderable text"). Redactions also create problems.

If a PDF is printed to paper, conversion via OCR is a nightmare.

I presume the FOIA requestor will hereafter make sure to request electronic copies of the originals so that metadata is preserved, but making them searchable is problematic as noted above. This would have at least one benefit. Printouts cannot be tossed over the transom as a compendium of several thousand pages that do not clearly demarcate where one file ends and another begins.
[ link to this | view in chronology ]
mischab1, 17 Feb 2011 @ 10:01am

If it's anything like my company, half the meta data is meaningless anyway. Someone wants to copy the formatting style of a document so they copy an existing file and replace all the text with their own. (On a completely different subject.) But they don't know anything about meta data so the file still lists as Author the person who clicked File -> New 5+ years ago.
[ link to this | view in chronology ]