The History Of Haystack... And Why Online Censorship Will Remain Difficult

from the there-are-always-holes-in-the-wall dept

Fri, Aug 6th 2010 4:25pm — Mike Masnick

Over the years, it's been fascinating to watch the battle over internet censorship, monitors and filters, along with the equally rapid attempts to get around all of those things via technology. Many folks are familiar with anonymous proxies, like Tor, which do help provide anonymity, but can still be blocked once the censor is aware of the tor node. If you follow this space, you're probably already aware of Haystack, which is, in some ways, a step up from Tor and has been getting more and more attention lately. Newsweek actually has a pretty good article on the history of how Haystack came about, involving a 20-something programmer who had little interest in political activism or Iran, until he started seeing the various protests and responses after the Iranian election. Something clicked, and helped along by a "disaffected Iranian official" who sent him the details of how Iran's internet filter worked, and led to Haystack, which hides traffic inside what looks like legitimate traffic (and, in the case of Iran, is specifically designed to hide in traffic that is popular in Iran).

What struck me most about the story is just how improbable a story it is if you look at it in a vacuum. We're talking about a 25-year-old guy, with little interest in Iran or activism, suddenly scratching an itch -- and within a week he had an Iranian gov't official leaking him information that was useful in building a system that could get around the Iranian internet censorship filter. That's impressive, no matter how you look at it. It also highlights why it's always going to be difficult to successfully censor the internet on a wider scale. Someone, perhaps from a totally unexpected place, is going to figure out how to get around it.

Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.

Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.

While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.

–The Techdirt Team

Filed Under: censorship, free speech, haystack, iran

27 Comments

If you liked this post, you may also be interested in...

Reader Comments

Subscribe: RSS

View by: Time | Thread

Anonymous Coward, 6 Aug 2010 @ 4:47pm

Speed of innovation
It also points to something even more interesting: the cycles of innovation are going to keep becoming shorter and shorter.

The Internet connects everyone, and there is always someone somewhere in the world with the piece of the puzzle you need to make the next step.
[ link to this | view in thread ]
Ada Morrow, 6 Aug 2010 @ 5:16pm

Haystack unreality warp
I've worked in Iran with women's rights organizations. I've heard about Haystack for the past two years. I've yet to see any documentation or product. As far as I'm concerned, Haystack is a press machine, nothing more.

You're comments about Tor are also incomplete. Do your research before publishing, please. https://torproject.org/bridges works fine in Iran. Using tor without bridges works fine in Iran.

Austin is not a programmer, he's a marketing wonk. There is a vast difference.
[ link to this | view in thread ]
GeneralEmergency (profile), 6 Aug 2010 @ 5:30pm

Hyperpolymorphism...
That's what I have been calling the internet/compusphere's resistance to censorship/control for many years now.

I thought I had invented the word myself at the time I first recognized and named the effect, but later discovered the word is also used to describe certain logically similar DNA related actions.

Oh well.
[ link to this | view in thread ]
Anonymous Coward, 6 Aug 2010 @ 6:00pm

Re: Speed of innovation
First we need to start creating our own wifinet and start learning how to become telco independent. Adjacent houses on the same block need to connect wires to each other and some houses across the street should aim directional antennas at each other to facilitate neighborhood traffic transfer (and we need to lobby local governments to allow this). We need to keep extending it and lobbying the FCC to start granting us more spectra along with permission to use spectra that can travel across longer distances (even if it means sharing spectra with others like broadcasters) and we can't forget to start lobbying congress, and their bought politicians, not to jump in and stop our efforts the moment we start to succeed with the FCC.

Then we need to figure out how to develop quantum (or some other) non - local communication method and distributing the communication tools either via the white market or, if necessary, via the black market.

Communication must be open, we've tolerated too many years or brainwashing and systematic censorship from our government(s) and have seen the devastating effects it has caused. Censorship must be resisted, it absolutely must be.
[ link to this | view in thread ]
Eugene (profile), 6 Aug 2010 @ 6:14pm

Re: Hyperpolymorphism...
Well, the internet is a bunch of logically similar bit-related actions, so... ;)
[ link to this | view in thread ]
Sam_K (profile), 6 Aug 2010 @ 7:03pm

Re: Re: Speed of innovation
On the face of it, I really want to believe this could work, but there are so many problems with it I barely know where to start.

To begin with, the majority of people lack the basic network chops to manage a functional routing node on the Internet, let alone protect their own computers from data that is merely "passing through". I guess this could be overcome with pre-configured hardware that is installed by your local computer service guy, but it's hardly plug and play at the moment which is where it would need to be to have anything like a useful level of penetration.

Secondly, there will always be "geographical chokepoints" where the hop to the next node is much further than you can run an ethernet cable or domestic WiFi connection. We have now surpassed the point where a single, interested user would be willing to stump up the cash to make the connection, especially if everyone in the local area needs this link to reach the outside world.

So what do you do? You ask everyone in the town if they would be willing to contribute some cash to setup this link. Enough of them say yes! Now you need to setup an organisation to collect the money and build and maintain the connection.

Congratulations, you've just started a Telco!
[ link to this | view in thread ]
Anonymous Coward, 6 Aug 2010 @ 7:17pm

Re: Re: Re: Speed of innovation
"Secondly, there will always be "geographical chokepoints" where the hop to the next node is much further than you can run an ethernet cable or domestic WiFi connection."

Notice that I said we need to lobby the FCC to grant us more spectra that can travel longer distances. Television broadcasting spectra can travel miles and is capable of transferring a reasonable amount of data.
[ link to this | view in thread ]
Anonymous Coward, 6 Aug 2010 @ 7:21pm

Re: Re: Re: Re: Speed of innovation
Also, despite frequency restrictions, the FCC also sets frequency intensity restrictions to prevent signals from traveling very far as well. The point I'm making is that we need to encourage the FCC to grant us access to frequencies that allow longer distance propagation. The laws in place intentionally limit how far unlicensed signals can travel and that's what needs to be fixed.
[ link to this | view in thread ]
Anonymous Coward, 6 Aug 2010 @ 7:22pm

Re: Re: Re: Re: Re: Speed of innovation
Wifi intensity restrictions *
[ link to this | view in thread ]
Sam_K (profile), 6 Aug 2010 @ 8:22pm

Re: Re: Re: Re: Speed of innovation
You completely missed the point. Even if there was NO RESTRICTION AT ALL on use of radio frequency, it is still expensive to setup and run the equipment needed to actually make the link!

You need to erect some kind of tower to get the antenna above the landscape. You will need a small plot of land to put this on, let's assume (very optmistically) that your local community is OK with you putting this on some public land at no cost.

Now you need to build your tower. It needs to be strong enough to withstand the weather and not be a threat to public safety for many years to come. The design work, construction, delivery and materials cost for this tower alone will cost thousands if not tens of thousands.

Once it turns up, you can't just dig a little hole, put it in and pat the dirt down, oh no! Now you need a local builder to build you a concrete footing and hire a crane to stand the thing up.

Next you need to populate this tower with your actual hardware. Probably at least thousands to tens-of-thousands there too since it'll have to be capable of at least a multi-gigabit connection to make everyone happy. Oh and we'll need a little outhouse at the base of the tower to hold all the gear. Oh and we'll need to connect power to the outhouse for all the gear.

Oh yeah, nearly forgot, you now need to do all of that again at the other end!!!

What if the community at the other end is not interested in this sort of public network? Now you need to actually buy the land at the other end for your 2nd tower.. What's that gonna cost? $50,000+? Assuming you can buy a small plot in just the right spot?

Even once it's all built this is not just a set-and-forget operation. You will need to have a tech person to maintain the gear and an administrative personl to handle all the accounts. That's at least 1 full time employee if you find the right person who can do everything.

Chuck in the maintenance costs and electricity bill for the tower, plus putting money aside to upgrade to hardware every few years to meet rising demand and you're looking at an operation easily exceeding $100K in annual expenses.

And all of this is AFTER you've successfully dealt with the FCC.
[ link to this | view in thread ]
Anonymous Coward, 6 Aug 2010 @ 9:33pm

Re: Re: Re: Re: Re: Speed of innovation
Although I respect your enthusiasm, radio frequencies are not a solution for internet and the laws of physics are against you on this one, a wireless solution would be swamped in the blink of an eye from moedern traffic, for anti-censorship if the traffic is only about small amounts of information that would be fine, to build something large that would be used for other things it would be difficult.

People don't need more spectrum or radio waves that go further they need fiber from the purely technical point of view, coupled with radio frequencies that affect small areas and that people already have, new compression and decompression algorithms and ways of sending data on the logical layer are what is needed.
[ link to this | view in thread ]
Anonymous Coward, 6 Aug 2010 @ 9:37pm

Re: Re: Re: Re: Re: Speed of innovation
Femtocells took care of that problem, they are available right now, in some countries.
http://en.wikipedia.org/wiki/Femtocell
[ link to this | view in thread ]
Anonymous Coward, 6 Aug 2010 @ 9:40pm

Couple haystack with Herbivore and you have a pretty good system.

Herbivore.
http://www.cs.cornell.edu/People/egs/herbivore/
[ link to this | view in thread ]
Anonymous Coward, 6 Aug 2010 @ 9:42pm

Couple haystack with Herbivore and you have a pretty good system.

Herbivore.
http://www.cs.cornell.edu/People/egs/herbivore/
[ link to this | view in thread ]
Anonymous Coward, 6 Aug 2010 @ 9:45pm

Re: Re: Re: Re: Re: Speed of innovation
Femtocells dude.

http://en.wikipedia.org/wiki/Femtocell

It is all you described in a small form factor that can reside inside any house, it can even be replicated using arduino, but will be a bit bigger.
[ link to this | view in thread ]
Sam_K (profile), 6 Aug 2010 @ 11:44pm

Re: Re: Re: Re: Re: Re: Speed of innovation
Femotcells have nothing to do with what I am talking about. I am talking about covering say 20Km to the next town.

From the Wikipedia link you supplied:

"Once plugged in, the femtocell connects to the MNO's mobile network, and provides extra coverage in a range of typically 30 to 50 meters for residential femtocells"

The point of a Femtocell is to extend mobile coverage in a smallish enclosed space. Not to provide a radio link over many kilometers.

Fail.
[ link to this | view in thread ]
Sam_K (profile), 7 Aug 2010 @ 12:08am

Re: Re: Re: Re: Re: Re: Speed of innovation
It seems you misunderstood what I was getting at. What I am talking about is this:

Let's say you have connected up a small ad-hoc network with the houses near you, using ethernet, WiFi, etc. Now you can reach any house in your neighbourhood without using any Telco network.

OK, now you want to expand that network to connect up with the houses in the next town. That won is say, 10km away, and there is nothing but state forest or rural land in between. How do you bridge that gap?

The same situation arises when you want to connect up that local ad-hoc network to the existing Internet. Whoever provides a gateway out into the "real" Internet is going to get absolutely hammered by people wanting to use that link (assuming they don't have one of their own) So that person gets a really fast connection with lots of data that costs them a lot more. Now they need to charge the other network users to recoup their costs for such a fat connection.

Congratulations, you just started an ISP.
[ link to this | view in thread ]
Richard (profile), 7 Aug 2010 @ 2:58am

Re:
Won't the herbivore just eat the haystack ;~)

Serious though - doesn't freenet already supply a solution?
[ link to this | view in thread ]
A Dan (profile), 7 Aug 2010 @ 5:09pm

Sounds like Xnet
To anyone who's read Cory Doctorow's "Little Brother" this will sound a lot like the Xnet from the book. It's really not a complicated or new concept although, of course, execution is important.
[ link to this | view in thread ]
Anonymous Coward, 7 Aug 2010 @ 9:50pm

Re: Re:
Not for me, since there are no Freenet apps that are non JAVA, that is no good.
[ link to this | view in thread ]
Anonymous Coward, 9 Aug 2010 @ 5:42am

Re: Haystack unreality warp
It isn't even barely over a year old?
[ link to this | view in thread ]
Anonymous Coward, 9 Aug 2010 @ 6:36am

Hide in plain sight.
[ link to this | view in thread ]
Anonymous Coward, 2 Sep 2010 @ 10:06pm

Re: Re: Re: Re: Re: Re: Speed of innovation
"a wireless solution would be swamped in the blink of an eye from moedern traffic"

Most traffic across countries today works wirelessly via satellites. Satellites send tons of data back to earth already, even from Mars back to here. Most of the traffic sent across the Internet doesn't go via wires alone and is transmitted wirelessly at some point or another. If the traffic is properly directed/focused and the intensity is set so that the signal travels the required distance without traveling more than the required distance and was able to automate that, I don't see a problem. There are plenty of non - overlapping spectra that can transmit a lot of information. On a single block or in local areas there can be wires to reduce wireless needs and whenever wireless is needed it can be used.
[ link to this | view in thread ]
Anonymous Coward, 2 Sep 2010 @ 10:07pm

Re: Re: Re: Re: Re: Re: Re: Speed of innovation
and software was able to ... *
[ link to this | view in thread ]
Anonymous Coward, 3 Sep 2010 @ 7:18am

Re: Re: Re: Re: Re: Re: Re: Speed of innovation
"Whoever provides a gateway out into the "real" Internet is going to get absolutely hammered by people wanting to use that link (assuming they don't have one of their own)"

Most of the communication to the real internet already crosses satellites at one point or another. Even this blog post, when transmitted to people of other countries or even states, already crosses wireless boundaries at least once until some more local provider caches it maybe. I'm just saying that what the telcos already do should be more decentralized and individualized.
[ link to this | view in thread ]
Anonymous Coward, 3 Sep 2010 @ 7:27am

Re: Re: Re: Re: Re: Re: Re: Re: Speed of innovation
and, of course, to make it more decentralized and individualized requires the FCC to ease up on restrictions that prevent such a thing.

It's not a matter of can making it wireless be done. It's already being done, the telcos already do it. It's a matter of, who is allowed to do it, should the telcos have the exclusive right to access the frequencies required to do it so that they can be the only ones doing it, or should everyone have more rights so that everyone can more or less run the network in a more decentralized manner. I say the later. I don't mind the telcos existing alongside the decentralized network, but the govt should give us the freedom to run a own decentralized, more independent, network instead of putting in place so many restrictions that prevent such a thing.
[ link to this | view in thread ]
Anonymous Coward, 3 Sep 2010 @ 7:28am

Re: Re: Re: Re: Re: Re: Re: Re: Re: Speed of innovation
to run our own *
[ link to this | view in thread ]