Australia Sets New Overblocking Record: Aims For One Site, Takes Down 250,000
from the whoops dept
Overblocking is not a new problem -- over two years ago, Techdirt wrote about an instance where Homeland Security took down 84,000 innocent sites at a stroke -- so you might have thought that those employing this blunt instrument would take a little more care these days. However, things seem to be getting worse, not better. In Australia, the Australian Securities & Investments Commission (ASIC) has just scored a whopper:
The largest number of sites censored when attempting to block one particular site ASIC believed was defrauding Australians was 250,000. Of these, ASIC said about 1000, or 0.4 per cent, were active sites. It said the 249,000 other sites hosted "no substantive content" or offered their domain name up for sale, rather than hosting a fully-fledged active site.
I wonder how the ASIC established that 249,000 had "no substantive content". I can't believe it really checked all of them. And that's the big problem with overblocking: when huge numbers of sites get taken down by mistake, there's no way of telling what just fell off the Web, and what obscure but possibly important information is no longer available.
At least ASIC realizes that its current ham-fisted approach isn't acceptable:
ASIC told senate estimates in its opening statement that it was now examining how it could ensure only a site's specific domain name was blocked and ways it could alert the public to a site being blocked via a pop up page. It was also examining ways such a page could indicate why access was blocked and to whom queries could be made to dispute a block.
That's good, but maybe it would have been better if it had explored those options before shutting down nearly a quarter of a million innocuous sites by mistake.
Follow me @glynmoody on Twitter or identi.ca, and on Google+
Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.
Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.
While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.
–The Techdirt Team
Filed Under: australia, copyright, overblocking, takedowns
Reader Comments
Subscribe: RSS
View by: Time | Thread
[ link to this | view in chronology ]
Overblocking
[ link to this | view in chronology ]
Re: Overblocking
[ link to this | view in chronology ]
Re: Re: Overblocking
[ link to this | view in chronology ]
Re: Re: Re: Overblocking
[ link to this | view in chronology ]
Re: Re: Re: Re: Overblocking
[ link to this | view in chronology ]
Re: Re: Overblocking
[ link to this | view in chronology ]
Several points
Forget substantive content. It was all just an anomaly right? Or would this be collateral damage? You can expect some collateral damage in a city if you use a 45 megaton nuclear weapon to kill an ant.
Whether content is substantive or not, people have a right to say it and not have it taken down by private interests that cannot be bothered to exercise even the smallest bit of care in their use of nuclear weapons.
From the article . . .
> This meant thousands of other sites were blocked in the process,
> as many sites are often hosted on one shared IP address.
But wait. I thought an IP address was equal to one person, not a quarter million websites!
[ link to this | view in chronology ]
Re: Several points
Yes, and Mr. Prenda would like to know who that one person is. He'd like them to send him money.
[ link to this | view in chronology ]
Re: Several points ASIC not private interests
However, ASIC doesn't have a perfect record for managing this part of their portfolio. They are not the brightest "Cube" on the block. Though that can be applied to many parts of the Federal Government here in Australia.
As one national newspaper commentator said many years ago (paraphrased). The fundamental thinking in any government department at any level of government in Australia makes the same basic assumption, "all members of the general public are convicts and must be treated as such."
Hence, if taking down one site means that multiple other sites (100's of 1000's) are also affected then it really doesn't matter as they are all problematic as well.
[ link to this | view in chronology ]
Duh
Just, y'know, block the site's specific domain.
Seriously, are Joeys in charge of this? Because that would at least be cute.
[ link to this | view in chronology ]
Re: Duh
...and I'm just spit-balling here...but wouldn't it be prudent to know how to do this BEFORE you start doing it?
[ link to this | view in chronology ]
Re: Duh
I'm sure they will follow your wise advice and they'll block, let's say for example, blogspot.com.
Then we'll hear from the idiot brigade that it's just an anomaly.
[ link to this | view in chronology ]
Re: Re: Duh
[ link to this | view in chronology ]
Re: Re: Re: Duh
[ link to this | view in chronology ]
It's so comforting that people like this who CLEARLY know what they're doing are the ones making the decisions regarding what websites are online and what ones aren't.
/sarcasm
[ link to this | view in chronology ]
Well, Techdirt usually has "no substantive content"...
But I admire the chutzpah of assuming on zero direct evidence that you're right and the "ASIC" is wrong. You have grokked the Techdirt way. Gold star for you.
SO, with that out of the way, you're left with a 1000 sites that MAY need to be switched or otherwise adjusted, also by a script.
But in ANY case, the cause is not from ASIC, it's from alleged criminals, which Techdirt always tries to protect.
BUT WAIT A SEC... OMG! The one site out of hundreds of millions with THE ANSWER has been taken down! Humanity is DOOMED!
[ link to this | view in chronology ]
Re: Well, Techdirt usually has "no substantive content"...
You do realise that your ramblings are not the only thing on this site, right?
[ link to this | view in chronology ]
Re: Well, Techdirt usually has "no substantive content"...
To crawl the sites, maybe. But to decide what is "substantive", let alone to decide what qualifies as "substantive" is hardly so trivial.
[ link to this | view in chronology ]
Re: Re: Well, Techdirt usually has "no substantive content"...
[ link to this | view in chronology ]
Re: Well, Techdirt usually has "no substantive content"...
Ahh, our old friend, "alleged". What if it were possible to, say, suspend someone's ISP account... not saying who... for being an "alleged asshat"?
Guess you're right... "alleged" can be useful after all.
[ link to this | view in chronology ]
Re: Well, Techdirt usually has "no substantive content"...
Why not focus on the REAL problem - one site needed to come down and they instead took down 250,000!
Great job missing the point AGAIN, retard. Funny - I don't think you'd see it so trivially if it was YOUR site.
[ link to this | view in chronology ]
Re: Re: Well, Techdirt usually has "no substantive content"...
He's not missing the point.
He's distracting from the point, which is what he's paid to do, and what he so consistently does.
[ link to this | view in chronology ]
Re: Well, Techdirt usually has "no substantive content"...
Tough shit?
And the cause IS from ASIC, moron. They initiated the order that took them down (or did you miss that when you "read" the article?).
I just don't understand your way of "thinking" blue. How exactly can you defend gross incompetence of this magnitude over and over again? Or is 250,000 sites just an anomaly?
[ link to this | view in chronology ]
Re: Well, Techdirt usually has "no substantive content"...
[ link to this | view in chronology ]
Re: Re: Well, Techdirt usually has "no substantive content"...
I should also add that blocking your site, if you had one, would still be objectionable as censorship. You have the right to free speech, even if you add nothing to discussions.
[ link to this | view in chronology ]
Re: Well, Techdirt usually has "no substantive content"...
This actually made me laugh out loud.
The best programmer in the world couldn't create a system to do this in an adequate manner in a month, let alone an hour, unless you're talking about downloading a small number of sites and your idea of "analysis" consists of nothing more complicated than scanning for keywords.
[ link to this | view in chronology ]
Re: Well, Techdirt usually has "no substantive content"...
Read this: http://en.wikipedia.org/wiki/The_Trial
Then this: http://en.wikipedia.org/wiki/Due_process
250,000 ≠ 1. Go back to school, troll.
Wait, if you can't even count to one, how in the world are you going to be able to read those linked articles? Does your mommy type your posts? :S
[ link to this | view in chronology ]
Re: Well, Techdirt usually has "no substantive content"...
[ link to this | view in chronology ]
ASIC
ASIC have always been synonymous with single-mindedness and applying that inflexibility quickly and efficiently to large numbers in parallel!
[ link to this | view in chronology ]
Pop up page?
And with SNI, which is already usable for everyone except very old browsers, you can have thousands of https sites on a single IP.
With the IPv4 address crunch, many https sites on a single IP will become more common, and https is becoming more and more popular. So this pop up page idea gets even harder as time passes.
[ link to this | view in chronology ]
This particular task isn't that hard
Why not? If you know how to use curl or wget combined with a little perl or python it's fairly easy to discern which web sites aren't really web site, but single-page placeholders being used by domaineers. All those are cookie-cuttered: they have the same markup, the same layout, (nearly) the same content, (nearly) the same links, etc. The only reason they exist is that these scammers are hoping someone will typo a URL and land there, generating a fraction of a cent's worth of income.
The computing resources needed to do this are minimal: with a decent laptop and a modestly-fast connection, this can be done overnight.
[ link to this | view in chronology ]
Re: This particular task isn't that hard
For example, I have a personal domain name that I use for email and other private services. There is no website attached to it at all -- if you browse to my domain name, you'll get the placeholder page you're talking about.
This domain is heavily used every day and critical to a reasonable number of people. It would have been blocked as having "no substantive content" anyway.
The web != the internet.
[ link to this | view in chronology ]
Re: Re: This particular task isn't that hard
Your argument would only hold water in a single or very small number of domains. 250,000? Better chance that you get hit by lightning while partying with the Yankees - in a subway station.
[ link to this | view in chronology ]
Re: Re: Re: This particular task isn't that hard
[ link to this | view in chronology ]
I wonder how the ASIC established that 249,000 had "no substantive content"
Oh, i don't know... maybe by looking at access logs? Maybe by looking at how little traffic was actually coming in?
Actually, in this case it looks like a parking page site, because all 250,000 were apparently on a single IP address or had DNS from the domain in question.
Perhaps rather than just repeating what the original writer put up, you might want to actually put some effort into understanding. Your blind outrage is amusing, by sort of stupid.
[ link to this | view in chronology ]
Re: I wonder how the ASIC established that 249,000 had "no substantive content"
> Oh, i don't know... maybe by looking at access logs?
> Maybe by looking at how little traffic was actually coming in?
> Actually, in this case it looks like a parking page site
So you're saying that commercial advertising of domain names for sale, or parking sites for domain names purchased but no web site yet set up means it is okay?
So it's okay to deprive advertisers of revenue? And it's okay to deprive the owner of a domain name the right to begin setting up their new website on their new domain?
Got it.
And I'm sure this was done with the utmost care and with the strongest of evidence. Not just some insane accusation based on an IP address or single domain name.
[ link to this | view in chronology ]
Re: I wonder how the ASIC established that 249,000 had "no substantive content"
250,000 books removed from a publisher when attempting to remove 1, that would be fine?
Collateral damage is fine when it is not your industry on the receiving end?
Wanker
[ link to this | view in chronology ]
Re: Re: I wonder how the ASIC established that 249,000 had "no substantive content"
Is there any true collateral damage here?
[ link to this | view in chronology ]
Re: Re: Re: I wonder how the ASIC established that 249,000 had "no substantive content"
Who detrmines substantive content? the media industry?
If it is not content from the legacy entertainment industry then it does not count as substantive content?
You would think that it is collateral damage if one of those 250,000 sites was yours.
This is the definition of 'Intellectual Property THEFT', not Intellectual property infringement that copyright maximalists like to conflate with theft
[ link to this | view in chronology ]
Re: Re: Re: Re: I wonder how the ASIC established that 249,000 had "no substantive content"
I don't know - maybe they were all parking pages with the same code? maybe they were all single pages? Maybe they were all identical except the name?
Do we know if it was 250,000 domain names, or just 250,000 third level domains of the same site?
I think you might want to cool your jets until you know more, because you are certainly going off the handle with limited working material here.
[ link to this | view in chronology ]
Re: Re: Re: Re: Re: I wonder how the ASIC established that 249,000 had "no substantive content"
Way to miss the point. 250,000 blank websites lost is not a loss
How can you state that it is not a loss if you do not even know what was removed?
[ link to this | view in chronology ]
Re: Re: Re: I wonder how the ASIC established that 249,000 had "no substantive content"
[ link to this | view in chronology ]
Re: Re: Re: I wonder how the ASIC established that 249,000 had "no substantive content"
[ link to this | view in chronology ]
Re: I wonder how the ASIC established that 249,000 had "no substantive content"
In the aftermath of the problem, there would have been an analysis of the pages taken down by personal view (looking at maybe five or six sites). This analysis would then be used to justify what has happened and to colour the view.
Anybody with any technical expertise would very likely not have got their input past along - technical people are just too freaking dangerous to allow their opinions or views to be passed up the chain of reporting. They tend to highlight problems with the various approaches being considered.
[ link to this | view in chronology ]
But for real humor, there's this:This quote is from an article abstract. The punchline? The article was written in 2003. How long ago was that in internet years?
[ link to this | view in chronology ]
Re:
And when you're in the process of building a website, i.e. adding substantive content, it's common to put up a Coming Soon page.
Go Daddy provides a revenue stream for people who buy up domains by letting them use them for advertising. And Go Daddy offers a shared hosting service.
NOW can you see what the problems are, Horse's Ass with no brain?
[ link to this | view in chronology ]
[ link to this | view in chronology ]
Re: Prime Minister Gillard to you
She is a very intelligent woman. She was able to shaft the previous Prime Minister quite easily without having suffered the same consequences herself. She is leading Australia back to the greatness that was called "The Dark Ages" with her various policies for national development.
Security is a high priority for her and her government that they will use whatever means possible to keep all of the people safe by watching their every action.
She follows in the footsteps of those great previous Prime Ministers Robert Hawke and Paul Keating. Men of such renown that the first declared that there would be no children living in poverty in Australia by the year 2000 and the second declared that Australia was a "banana republic".
So please respect her and the position she holds by addressing her correctly as Prime Minister Gillard.
[ link to this | view in chronology ]
[ link to this | view in chronology ]