'Nice Internet You've Got There... You Wouldn't Want Something To Happen To It...'
from the this-is-no-longer-theoretical dept
Last month, we wrote about Bruce Schneier's warning that certain unknown parties were carefully testing ways to take down the internet. They were doing carefully configured DDoS attacks, testing core internet infrastructure, focusing on key DNS servers. And, of course, we've also been talking about the rise of truly massive DDoS attacks, thanks to poorly secured Internet of Things (IoT) devices, and ancient, unpatched bugs.That all came to a head this morning when large chunks of the internet went down for about two hours, thanks to a massive DDoS attack targeting managed DNS provider Dyn. Most of the down sites are back (I'm still having trouble reaching Twitter), but it was pretty widespread, and lots of big name sites all went down. Just check out this screenshot from Downdetector showing the outages on a bunch of sites:
So, once again, we'd like to point out that this is as problem that the internet community needs to start solving now. There's been a theoretical threat for a while, but it's no longer so theoretical. Yes, some people point out that this is a difficult thing to deal with. If you're pointing people to websites, even if we were to move to a more distributed system, there are almost always some kinds of chokepoints, and those with malicious intent will always, eventually, target those chokepoints. But there has to be a better way -- because if there isn't, this kind of thing is going to become a lot worse.
Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.
Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.
While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.
–The Techdirt Team
Filed Under: attack, ddos, dns, internet, vulnerabilities
Companies: dyn
Reader Comments
Subscribe: RSS
View by: Time | Thread
So yes, the infra-structure portion can help mitigate the problem but unless we start taking security very seriously it won't matter.
Of course, one must not forget the perpetrators should also be severely punished and if it's a state actor maybe even cut it entirely from the network to preserve its health.
[ link to this | view in thread ]
not dyn, dyin
[ link to this | view in thread ]
Started Again
[ link to this | view in thread ]
What a day!
[ link to this | view in thread ]
[ link to this | view in thread ]
Just have to share this gem of a quote from http://money.cnn.com/2016/10/21/technology/ddos-attack-popular-sites/index.html (emphasis mine)
[ link to this | view in thread ]
Re:
Yeah right wrote:
Where did you hear it was an attack? I haven't seen anything (at least, anything from a reliable source) indicating they know the cause. Everything I've read so far says they're still "looking into it".
[ link to this | view in thread ]
Re:
always use it to lie, cheat, and steal more liberty from the confused & ignorant plebs!
[ link to this | view in thread ]
Re:
[ link to this | view in thread ]
Re:
[ link to this | view in thread ]
Re:
[ link to this | view in thread ]
Re:
It requires a number of things on the infrastructure side. Standard practice with IoT needs to be to have the devices on a separate non-Internet-connected network which requires the cooperation of router makers and users. Consumer routers need to implement RFC 3704 egress filtering by default. ISPs need to implement 3704 filtering on the customer side (the head-ends and/or CPE depending on physical configuration) and on the upstream side. Upstream networks need to implement 3704 filtering even if it means reconfiguring their topology to separate the non-transit parts of their network from the transit network. All parties involved need to stop depending on other parties to do the work and configure their own networks as if their measures are the only thing standing in the way of a massive DDoS attack. And finally, targeted parties need to be able to hold the originating and intermediate networks financially liable for all the costs involved, not just the small fraction of the access bill for the downtime, when those networks failed to enforce 3704 compliance.
That won't stop all of it, but it'll stop a huge portion of it. The rest can only really be dealt with by forcing end users (consumer or business) to clean up infected/compromised systems on their networks. Given the intransigence of the average end-user (whether a consumer or a company's IT management) I don't see anything short of big sticks wielded effectively having any effect.
[ link to this | view in thread ]
Re:
[ link to this | view in thread ]
For a minute there I thought I was reading a quote about encryption from the FBI Director. Nerd Harder!
[ link to this | view in thread ]
Re: Re:
Common typo, but means the opposite.
[ link to this | view in thread ]
Nerd Harder!
[ link to this | view in thread ]
Re: Re:
[ link to this | view in thread ]
May I point out to Techdirt that we are (see Hyperboria: http://hyperboria.net/ for an example), but that there is serious difficultieswith deploying any such technology. The vast majourity of people (corporate & individuals) can't be bothered upgrading (most of whom won't see the point), and many who can be bothered won't do so as it (if not engineered correctly) will risk backwards incompatibility.
Engineering around these difficulties is a significant challenge I've only seen begin to be solved solved recently (and hyperboria could still be improved here).
Tl;dr Don't ask us to start solving the problem: we have. Instead do what little you can to help us deploy it.
[ link to this | view in thread ]
Re: Nerd Harder!
[ link to this | view in thread ]
Response to: Anonymous Coward on Oct 21st, 2016 @ 11:57am
The first paragraph is a quote from the artical.
[ link to this | view in thread ]
Nerding harder...
Lacking trust in computers, *everything* is going to have to go to a bit-torrent style model with no central host (somebody already did this for websites, I forget the project name) because there are enough broadband IoT devices out there to DDOS any single individual, company, or any device performing a particular function. The biological analog should be obvious.
And, just as with fair use and copyright, the problem of discerning "legitimate" traffic (all of Techdirt's fans) from "illegitimate" traffic (all of Techdirt's haters, and 100 million of their bots, coordinated so they look just like its fans) is basically impossible.
Time to break the glass over the emergency tools and prepare for the internet to go down. Probably November 9.
[ link to this | view in thread ]
Re: Re:
[ link to this | view in thread ]
Re: Re: Re:
[ link to this | view in thread ]
Re: Re:
Was it a case of mass-hysteria or was it triggered?
[ link to this | view in thread ]
Re:
Assuming the identity of the bot-herder is known or can be discovered, It would be wise to shut down the botnet (not just the attack) prior to taking any steps to remove the herder or their network access.
If the botnet is reasonably intelligently designed, cutting the perp off from the internet may make it next to impossible to send a shutdown signal the C&C infrastructure will recognize.
[ link to this | view in thread ]
Re: Re: Re:
A fire alarm went off (accident or malfunction or someone being an idiot), someone smelled something (perfume, food, whatever), and then everyone panicked.
[ link to this | view in thread ]
Re: Re: Nerd Harder!
Make companies financially liable for security issues in their products in a way that makes securing their software less expensive than not.
Until that happens, this type of issue isn't going to get better.
[ link to this | view in thread ]
Re: Nerding harder...
For the websites, https://webtorrent.io/
[ link to this | view in thread ]
Re: Re: Re: Re:
[ link to this | view in thread ]
Re: Re: Re: Re:
Hey now... What you do in the privacy of your own domain is your business.
[ link to this | view in thread ]
If FaceTwit isn't available . . .
A service outage could be a reason to push the big red button.
[ link to this | view in thread ]
Re: Re: Re: Nerd Harder!
I am tired of the make people pay money bullshit. It just creates injustice.
People with money get to stomp all over others. The people harmed usually never get compensated while the government makes money off actual crime!
[ link to this | view in thread ]
Re: Re: Nerding harder...
[ link to this | view in thread ]
Re: Re: Re: Re: Re:
(Although the constant ZOMGTERRORISM encouraged by govt isn't terribly helpful either.)
[ link to this | view in thread ]
Re:
[ link to this | view in thread ]
Nerd Harder
Hey, isn't this YOU saying "Nerd harder!"?
I get it, this problem isn't intractable, but still...
[ link to this | view in thread ]
Re: If FaceTwit isn't available . . .
Agreed to have a battle;
[ link to this | view in thread ]
Re: Re: Re: Re: Re: Re:
I agree it was probably a scary clown, but the timing isn't foolish.
[ link to this | view in thread ]
Re: Re: Nerd Harder!
"Suggesting people start giving a damn" is vague to the point of uselessness too. Which people? "The internet community", apparently. Whatever the fuck that means.
[ link to this | view in thread ]
However, I cannot access the websites of some pretty major companies, such as soundcloud and twitter. If I used twitter, that might be an issue for me. But I know that a lot of people rely on it for their breaking news, and with a lot of other big name company sites down we cannot get up-to-date info.
This is scary bad. The fact that Amazon's web service went down is scary. Big companies rely on AWS for their internet connectivity for things, and if that goes/stays down, it can mean a lot of lost income.
[ link to this | view in thread ]
Re: Nerd Harder
IE The tools are there, people just are not using them.
[ link to this | view in thread ]
Re: Re: Re: Nerd Harder!
Make companies financially liable for security issues in their products in a way that makes securing their software less expensive than not.
Sure, it's just that easy if you think laws are vague, handwavy things.
In practice, what does this actually mean? Which companies are financially liable for security issues in which products? How quickly does the vulnerability have to be fixed to avoid liability? What's the statute of limitations?
If there's a vulnerability in the Linux kernel that affects Samsung phones, who's liable? Samsung, Google, the Linux Foundation, all of the above? If the vuln has already been patched upstream, and Google's already pushed an update, but Samsung isn't staying up on Google's updates, then presumably you'd hold Samsung liable but not Google or Linux, right? Okay. What if Samsung's rolled the updates out on some phones but not others? What should Samsung's obligation be for supporting its old phones? Should it be defined in terms of age? Userbase?
And you trust legislators to understand all these issues and write reasonable laws that take all of them into account while still being strong enough to discourage companies from releasing insecure devices?
You're basically saying that legislators need to nerd harder, which isn't really any better than saying programmers do. Though at least you had a suggestion for a way of fixing the problem, which is more than Masnick gave us in the article.
[ link to this | view in thread ]
Re: Re: Re: Re: Re: Re: Re:
Yes, City traders leaving the bars for their country retreats.
[ link to this | view in thread ]
[ link to this | view in thread ]
Re: Re: Re: Re: Re: Re: Re:
And panicky people are dangerous. A panicked crowd is especially dangerous.
[ link to this | view in thread ]
Re:
Probably because it isn't hitting everybody. If I wasn't reading about it on the news sites I'd never have known. Been online in CST since before 6am, have used many of the major sites mentioned (and of course AWS at the back of many) all morning with no indication of any problems. (I don't use FB but I have been using Amzn, TWTR, NYT, WAPO etc etc etc, major sites for work, and they've all been flying. Weird.) Literally except for reading about it I have not noticed anything. I feel left out.
[ link to this | view in thread ]
Response to: Nick on Oct 21st, 2016 @ 1:36pm
Yes, it's bad for Amazon but what about other small businesses that are totally revenue-dependent in their internet services staying up. There were companies in Florida with no internet service for a month and many more for weeks. Frontiers tech's didn't show up for appointments and when CS was contacted they just lied. One idiot called the consumer in the same landline he was there to repair to let them know he was there. They provided their cell phone numbers no less than 7 times for these brain-dead idiots. Mean-while they were chastising Warner Cable for over charging and throttling only to implement the exact same pricing structure except worse.
WTH!
[ link to this | view in thread ]
Re:
[ link to this | view in thread ]
Re: Re:
The people whose infrastructure is responsible for this have to be held personally accountable. Publicly named. Publicly shamed, Publicly fired. Publicly denounced. Publicly humiliated.
Because it's their fault. They've failed to meet minimum acceptable standards for Internet operations and they deserve to pay a steep price for it. Many of them should never work in this industry again.
Yes, that's harsh, but having a big chunk of the Internet taken out -- and the attackers could have done more and done it longer if they wishes -- is a pretty big deal. Harsh penalties are appropriate.
And maybe, just maybe, everyone else will pay attention and start doing the things that they should have done 10-20 years ago in order to defend the Internet, not merely defend themselves.
[ link to this | view in thread ]
Re: Re: Re: Re: Nerd Harder!
Assume Samsung is selling IoT enabled toasters, because why not. Everything's better with a network stack. Anyway, MSRP on this toaster is $100usd and Samsung releases the product Jan 1, 2017, and ships 1000 toasters.
Now, if there are no open CVE's on any component of the IoT stack on this toaster in the 90 days before Samsung ships, they're effectively insulated from liability. Oh, and in that world, the sky is Fuscia.
But, If there _is_ an open CVE was announced >= 90 days before Samsung launches the product, _and_ it gets exploited, Samsung is the hook for 5% of the MSRP for each unit sold of said product for every 90 days of age on the CVE.
Example: Samsung begins selling their IoT enabled toaster (MSRP == $100usd) on Jan. 1, 2017. And they sold 1000 of them on day 1. Said toaster has a vulnerability that was announced on Aug. 15, 2016 (just outside the 90 day grace period). If one of these toasters gets exploited and causes trouble, Samsung is going to write a check for (5% of $100) == $5 for each of the 1000 toasters sold as of the date of the CVE being exploited, plus the same fine going forward for each non-patched unit they sell.
Now, pretend that vuln wasn't released on Aug. 1, 2016, it was release on Aug. 1, 2016. Same ship date, same quantity. Except now instead of 5% per toaster, it's 10%. Add 5% for every 90 day interval of CVE age. Also, allow the total penalty per unit to exceed 100% of MSRP with no upper bound. So, you release an IoT enabled toaster with a 12 year old ssh vuln, and it gets exploited? assume qty 4-90 day periods / year to make it easy, now your penalty is (48 * $5) = $240 * 1000 = $240k in fines for each $100MSRP toaster you sold.
And why use MSRP as the basis for the penalty? Well, because it's both easy to validate and publicly verifiable.
No grace period, no appeal, cut a check to a high school to fund a secure coding class, because CVE's are public and theres no way the organization "couldn't have known".
Oh, and multiple CVE's? 5% per CVE, and scale it out.
If you can verifiably patch these toasters 100% then you restart the clock from the time the patch was pushed to the toaster. If you can't patch them, well, eventually you'll get to write a check big enough to make the board pay attention.
Bonus: Specifically disallow said penalties as a loss for tax purposes.
As to your other question: It's a Samsung toaster running a google code, Samsung pays. It's their label. If Samsung wants to go back and fight it out with Google based on contract terms, that's fine, Samsung can attempt to recoup their (already paid) losses from Google.
(yeah, I know. There's no chance this or anything like it will ever happen.)
[ link to this | view in thread ]
Re: Re: Re: Re: Re: Nerd Harder!
But fundamentally, if we want anything resembling a secure IoT, we're going to have to figure out a way to make it more expensive for companies to ship a vulnerable product than it is for them to fix it first, because the attack surface isn't going to get smaller.
[ link to this | view in thread ]
Re: Re:
One point of contention, it's probably minor to most. Say I order a private vlan from some IXP. Should the IXP be responsible for BCP38, after all the connection itself is just traversing their network to another provider. They certainly can not filter bogons, and how are they to know what ASNs or IPs should traverse that link.
[ link to this | view in thread ]
Re: No trace of any chemical has been found.
[ link to this | view in thread ]
Re: Re: Re: Re:
As a child of the 90's, there is only one way to read it. I chuckle every time someone says "Do you cyber?" here, because that was exactly the same question folks said on BBS's and the early internet back in the 90's, but for entirely different, though very similar reasons.
[ link to this | view in thread ]
Re: Re: Re:
[ link to this | view in thread ]
Re: Re: Re: Re: Re: Nerd Harder!
Though it looks like there's a typo:
Now, pretend that vuln wasn't released on Aug. 1, 2016, it was release on Aug. 1, 2016.
[ link to this | view in thread ]
Re: Re:
[ link to this | view in thread ]
[ link to this | view in thread ]
Fix it: White Hat Hacking
If they can find them, so can we. And if the user can't get in, they will just reset it to default. And it will be found again. Repeat.
Have done this dozens of times in the large and small companies I've worked for. Camera's, scanners, printers, et cetera. If the customer/employee calls in a tech support ticket, they are talked thru how to reset, configure and set a good password.
Secondly, maybe some enterprising company/person could set-up a simple "Certified Safe Supported". A small company could get a product, certifiy that it has security in ind, such as a) support for updates b) obvious passwords are not used/repeated c) I really don't need to list them...
[ link to this | view in thread ]
Desperately need MaidSafe's SafeNetwork to stop this nonsense
[ link to this | view in thread ]
Where are the IoT apologists...
[ link to this | view in thread ]
Heads Will Roll
[ link to this | view in thread ]
Re: Re:
on my net you will be stripped of IPV6.
any blocking rule should be in THREE unless you got a specific purpose
CUSTOM FORWARD
CUSTOM INPUT
CUSTOM OUPUT
ingress, egress, and forwarding
These devices getting hacked must be directly facing the web? Yes? I have several a SONY blue ray player right it has a 192.168.0.X I got a Marantz it has a 192.168.0.XX
Each IP needs rules to get out-crap works fine here and I got the youtube browser and the Opera browser in these boxes. All working just fine. Another thing is I constantly maintain a list of domain to IP's so if DNS goes down I can load up techdirt at http://104.25.105.28 if i can punch thru cloudfare insanity.
People that don't run their own boxes don't get it. You can quote RFC's all day long it's freedom, tcpip and networking creativity that matter.
I seen a LOT of this wireless crap at the hospital, but is it even plugged in? I doubt it.
[ link to this | view in thread ]
When you outsource to the cloud, you have a SPOF you can't see.
On the DNS customer side, there's no reason not to use multiple authoritative DNS providers, including running one yourself. The cleanest way of doing this is to run two or three widely separated DNS servers that only talk to your three DNS services. Even for huge zones, this is a cheap and idiot-resistant method.
On the resolving side, there's no excuse for not having two or three nameservers listed on each of your computers. If you are small: one from your ISP, one from Google, one from any other service. If you are in any position to run caching DNS servers, do that as well.
[ link to this | view in thread ]
Re: Re: Re: Nerding harder...
http://ipfs.io/
[ link to this | view in thread ]
Re: Re: Re: Re: Nerding harder...
aMule with Kademlia
http://www.amule.org
[ link to this | view in thread ]
Taking Credit
I do love the title
[ link to this | view in thread ]
[ link to this | view in thread ]
Re: Re: Nerd Harder
So create an attack vector, the update server.
Not to mention the central repository it creates of users of that device/software for targeted attacks.
[ link to this | view in thread ]
Re: Re: Re: Nerd Harder
[ link to this | view in thread ]
Re: Weeping Candian Trade Minister
[ link to this | view in thread ]