The Stasi's Tiny Torn-Up Analog Files Defeat Modern Digital Technology's Attempts To Re-Assemble East Germany's Surveillance Records

from the too-hard-for-today's-hardware dept

It is nearly 30 years since the wall separating East and West Berlin came down, and yet work is still going on to deal with the toxic political legacy of East Germany. As Techdirt readers are well aware, one of the defining characteristics of the regime in East Germany was the unprecedented -- for the time, at least -- level of surveillance inflicted on citizens by the Stasi (short for Staatssicherheitsdienst, or State Security Service). This led to the creation of huge archives holding dossiers about millions of people.

As it became clear that East Germany's government would fall, and that its long-suffering citizens would demand to know who had been spying on them over the years, Stasi officers began to destroy the most incriminating documents. But there were so many files -- a 2008 Wired article about them says they occupied 100 miles of shelving -- that the shredding machines they used started to burn out. Eventually, Stasi agents were reduced to tearing pages by hand -- some 45 million of them, ripping them into around 600 million scraps of paper.

After thousands of bags holding the torn sheets were recovered, a team working for the Stasi records agency, the body responsible for handling the mountain of paper left behind by the secret police, began assembling the pages manually. It was hoped that the re-assembled documents would shed further light on the Stasi and its deeper secrets. But it was calculated that it would take 700 years to deal with all the scraps of paper by hand. A computerized approach was devised by the Fraunhofer Institute, best-known for devising the MP3 format, and implemented following a pilot project. After some initial successes, the program has run into problems, as the Guardian reports:

A so-called ePuzzler, working with an algorithm developed by the Fraunhofer Institute and costing about €8m of [German] federal funds, has managed to digitally reassemble about 91,000 pages since 2013. However, it has recently run into trouble.

For the last two years, the Stasi records agency has been waiting for engineers to develop more advanced hardware that can scan in smaller snippets, some of which are only the size of a fingernail.

The ePuzzler works by matching up types of paper stock, typewriter fonts, or the outline of the torn-up page. It has struggled with hand-written files that were folded before being torn, leaving several snippets with near-identical outlines.

While the hardware engineers try to come up with a suitable scanner that can handle these tiny fragments, a small team continues to match up the more crudely ripped pages manually. Inevitably, some people will be thinking: "If only the Stasi had used blockchain, all these problems could have been avoided..."

Follow me @glynmoody on Twitter or identi.ca, and +glynmoody on Google+

Filed Under: digital technology, records, stasi, surveillance