Security Researcher Reveals Solarwinds' Update Server Was 'Secured' With The Password 'solarwinds123'
from the [checks-luggage-combination] dept
As was noted here earlier, up to 18,000 customers of globally-dominant network infrastructure vendor SolarWinds may have been compromised by malicious hackers. The hackers -- presumed to be operating on behalf of the Russian government -- deployed tainted updates (served up by SolarWinds) that gave them backdoors to snoop on internal communications and exfiltrate sensitive data.
The attack was so widespread and potentially catastrophic, the DHS's cyber wing issued an emergency directive that stated the only way to mitigate damage was to airgap devices and uninstall affected Orion software. Meanwhile, SolarWinds filed an update with the SEC detailing the extent of the damage. It was limited, but only if you consider 18-33,000 potential infections "limited." It's only a small percentage because Solarwinds's customer base is so large. The company boasts 300,000 customers, among them several government agencies and all five branches of the military. (It's not boasting much these days. It has memory-holed its "Customer" page during this trying time.)
Unfortunately, the directive from CISA was delivered a bit too late. CISA itself was compromised by the hack, something acknowledged by the DHS less than 24 hours after its dire directive was issued.
The fallout from this hacking -- which may have begun as early as March of this year -- will continue for a long, long time. But this latest news -- delivered by Zack Whittaker -- adds another layer of irony to the ongoing debacle. Orion is Solarwinds' one-stop shop for IT software. It promises to secure customers' IT infrastructure by bundling in the company's network security products.
No doubt the company claims to take security seriously. But while users are being subjected to password requirements that demand them to utilize most of the alphabet and multiple shift key presses, internal security isn't nearly as restrictive. Here's the "OMFG are you goddamn kidding me" news via Reuters, which first broke the news of the malicious hacking.
Security researcher Vinoth Kumar told Reuters that, last year, he alerted the company that anyone could access SolarWinds’ update server by using the password “solarwinds123”.
All five branches of the military. The NSA. The IRS. The USPS. DHS. The Treasury Department. Nearly every Fortune 500 company. All ten of the top ten telcos. The list goes on and on. And with this access, attackers could move laterally, using compromised credentials to eavesdrop on mutuals of targeted entities. And all of this "secured" by a password so simple an idiot could have created it.
We're fucked. And we're fucked by people making far more money than we are who take our security far less seriously than we do. Say what you will about the security ambivalence of the general public, but it's the "experts" who endanger us with lax security measures who do the most damage. If Joe Blow fails to secure his email account, he's probably only going to hurt himself. When a multinational vendor can't be bothered to gin up a decent password, entire government agencies become a plaything for malicious hackers.
Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.
Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.
While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.
–The Techdirt Team
Filed Under: dhs, hackers, infrastructure, passwords, security
Companies: solarwinds
Reader Comments
Subscribe: RSS
View by: Time | Thread
Sorry, my fault, I meant to tell them to change the password to Solarwinds1234 but forgot. That capital letter and extra number would have made all the difference.
[ link to this | view in chronology ]
Re:
Security also requires an exclamation point.
[ link to this | view in chronology ]
Re: solarwinds123
Welp, now if Vinoth Kumar hadn’t gone and told the whole world what password they use, I’m pretty sure most of us just assumed it was “moonlightmaze123”
[ link to this | view in chronology ]
At my workplace we need a physical token + passwords to do anything. To create a package for deployment it needs to be signed by two people for each stage (internal test, integration test, verification/performance test, production/publication). Leaving your physical token unattended can get you reprimanded or even fired.
If a company says that they take security seriously but they only use passwords in their organization, they don't take security seriously.
[ link to this | view in chronology ]
Re:
That is all well and good, but it wasn't people like you that were responsible for the breach. It was the people of the people above you that were. imagine if got a tainted update from one of your trusted vendors, both you and your second person would sign off on it and deploy it. So yeah at your level you are super cautious with security, but no matter how good you are this would have gotten by you.
[ link to this | view in chronology ]
Re: Re:
ALL software we use are vetted before it's installed anywhere, no automatic updates are allowed, no vendor is trusted. Any software that gets a CVE that is deemed critical will be shutdown/partioned until the vulnerability is resolved. On the off-chance something slips through, the network is heavily partioned plus only select applications and services has access to the internet. Any spurious http/https traffic is blocked by default, and only https traffic with internal/approved root-certs are allowed through after inspection.
On top of all this, we log everything and it's datamined daily for suspicious patterns and/or activity which means that any application or service that suddenly starts trying to connect to the internet will be flagged very quickly.
Just lets say that those running our IT security takes it very seriously, and for good reason considering the type of information that flows through our system.
So, I severely doubt that it would have gotten by us.
[ link to this | view in chronology ]
Re: Re: Re:
Hey, that sounds like an ideal situation, security-wise.
However, I can count exactly zero companies I've worked for that are that strict on security. Some have been startups that took shortcuts during their growth phase that haven't been patched yet. Some are larger, older companies with old school admins who haven't got out of bad habits yet. Some are people who assume their internal server is safe because they trust their network security.
I agree that a company that is literally supplying security as their business should have been taking things a hell of a lot more seriously. But, if you think your experience is an indicator of what's happening in the real world outside of your organisation, I have some very, very bad news for you.
[ link to this | view in chronology ]
Supply chain attack
Did you know...
Does your IT staff decompile all updates that come in and read them line by line? Similarly, it wouldn't be spurious HTTP/HTTPS traffic, it would be traffic on whatever port the Orion system - already authorized for access to the outside world - uses to update itself.
You might well spot higher than normal update traffic on Orion itself, but you might well not. Remember that the Solarwinds server was itself compromised, so the malware could still have been getting funneled through there, rather than through some system more directly controlled by the attacker. Again, pre-authorized traffic.
These are the sort of things that make supply chain attacks so dangerous.
[ link to this | view in chronology ]
Re: Supply chain attack
As I said, no automatic updates are done. All updates are downloaded manually, vetted, then applied. This process takes weeks, which is a pain in the ass in some instances but security isn't something that's supposed to be easy or quick.
[ link to this | view in chronology ]
Re: Supply chain attack
Rocky's statement could only reasonably be true if all software in use was open-source. Otherwise, some vendor would be trusted. There's no need to decompile when one is already compiling everything from source.
[ link to this | view in chronology ]
Re: Re: Supply chain attack
Dream on: https://www.win.tue.nl/~aeb/linux/hh/thompson/trust.html
[ link to this | view in chronology ]
Re: Re: Re:
Yea, monitored by solarwinds 😅
[ link to this | view in chronology ]
Re: Re: Re:
I know where you work, reminds me of a place in Indiana.
[ link to this | view in chronology ]
Re: Re:
The suits strike against.
[ link to this | view in chronology ]
Re:
Yes, you can take computer security to paranoid level and in some environments it is necessary. In other environments one does not have to go to that level of protection. You also have to look at how attacks change and the security systems that were good enough ten years ago may not be sufficient today.
Nowadays the attacks on passwords and logins have risen to a level that I think that a username/password only login is only sufficient for low impact environments, like chat fora. For work accounts I would suggest some two factor system (could be a public/private key certificate system); I would also appreciate the administrator of my chat forum to use a two factor system.
It is so bad that a software distribution server has been hacked because of an outdated authentication system. There are enough good two factor systems available on the market.
[ link to this | view in chronology ]
Re: Re:
"There are enough good two factor systems available on the market."
"Available on the market" and "sellable to stingy management who don't care about security until after they've had a major breach" are two very different things...
[ link to this | view in chronology ]
Re: Re: Re:
You caught the Duo bypass trick this week?
[ link to this | view in chronology ]
Re: Re: Re:
Well it certainly doesn't help that most of those second factors need individual SCard drivers, (Under even more scrutiny read: $$$$ because they interact with the security subsystem), to work under winblows. Or that most of them won't interoperate with other systems (Computers / Door Entry / Punch Clock / Etc.) due to proprietary protocols.
Of course most of that is due to the fact that the "standard" really just defines physical parameters (card size, electronic pin outs, bus protocol, etc.) but fails to define any kind of data storage / secure processor API. As such every vendor has it's own proprietary data format and API for actually using the token at the application level. The result is a highly segmented and expensive market that makes the client side software trying to authenticate specific to one or two hardware vendors.
Before anyone says "what about one time code fobs or smartphone apps?": Those don't provide operational security. If I take a smartcard away from a reader, that's it. The device locks. It cannot communicate with anything at that point. A one time fob can't do that without some other smartcard tech built in, and a smartphone is accessible on the internet. In addition phones are one of the first things an attacker would try to compromise, and not even for key data, but passwords, and contact info for phishing. If you are going to spend money and training time on a second factor system, you may as well spend it wisely, and get all that such a second factor offers.
[ link to this | view in chronology ]
Re:
"At my workplace we need a physical token + passwords to do anything."
Where I work it's access through the corporate intranet VPN only for any company-specific applications, with access to the intranet granted only for approved and registered devices, those devices then locked with a pin or password, and the same devices locked in a physical locker on site at the end of the working day...and we aren't even in IT. It's a pretty standard formula but it works.
Sure, nothing is secure against rubber-hose cryptanalysis, a skilled and persistent hacker, or a successful phish. But the "<name>1234" password is just making shit too easy by far to the canny script kid with a "Top twenty names of common passwords" list.
[ link to this | view in chronology ]
Just a backdoor
I mean its just a "backdoor", what's the big deal?
[ link to this | view in chronology ]
Re: Just a backdoor
It's not a "backdoor" in the sense that people who keep trying to break crypto are trying to implement. After all, you can always change a password...
[ link to this | view in chronology ]
I mean its not like the biggest investors dumped millions in stock before the hack was reported.... oh...
I guess they will use that money to fiance runs for Congress where its not illegal to trade on insider information.
[ link to this | view in chronology ]
I prefer LunarFarts321
[ link to this | view in chronology ]
Re:
That is actually more secure in at least 2 vectors (doesn't contain the company name, does contain upper case characters)
[ link to this | view in chronology ]
All this worrying about this "problem" is unnecessary. Here's what will happen:
A few politicians will pretend to be outraged. Perhaps a speech or two will be forthcoming (need the sound bite for the news)
The head of the company will be called in for a "grilling".
While testifying before the outraged politicians, the CEO will pinkie swear to do better next time. What the cameras in the hearing will not show you is the cash being handed to the politicians under the table.
[ link to this | view in chronology ]
Re:
OUTRAGED POLITICIAN: Why didn't you use a strong password, such as MAGA2020! ?
[ link to this | view in chronology ]
To add insult to injury...
...the ”password“ was publicly exposed on their own public GitHub repo. In plaintext.
https://savebreach.com/solarwinds-credentials-exposure-led-to-us-government-fireye-breach /
[ link to this | view in chronology ]
the password was probably to ACCESS updates, not MODIFY updates
Before getting too worked up about this password, it would be good to know if the password was required to download patches or to upload/modify patches.
If this password is for downloading, then it's no big deal if it's weak. Plenty of companies allow downloading updates without any authentication at all.
[ link to this | view in chronology ]
Re: the password was probably to ACCESS updates, not MODIFY upda
[ link to this | view in chronology ]
Re: the password was probably to ACCESS updates, not MODIFY upda
"Before getting too worked up about this password, it would be good to know if the password was required to download patches or to upload/modify patches."
The entire story is about how the hackers uploaded a modified update file for subscribers to download and compromise the systems of Solarwinds customers, so take a wild guess.
[ link to this | view in chronology ]
Joe: 12345? That's the stupidest combination I've ever heard in my life! It's the kind of thing an idiot would have on his luggage!
Trump: Change the combination on my luggage!
[ link to this | view in chronology ]
Re:
Please don't make such libelious comparisons.
On one hand you have a guy who can only be described as a satirised carricature of an imbecillic, nepotistic, narcisitc would be dictator who pretty much collects the bottom of the barrel for his helpers destroying the livelyhoods of his subjects. On the other hand you have President Skroob
[ link to this | view in chronology ]
Re:
I agree with other responders - that's slanderous. We all know, from Trump's hacked twitter account, that his password was "MAGA2020".
[ link to this | view in chronology ]
Whistler: Give me the number of something impossible to access.
Carl: Federal Reserve Transfer Node, Culpeper, Virginia.
Mother: Good luck, $900 billion a day go through there.
Carl: You won't get in --- it's encrypted.
Whistler: solarwinds123
[ link to this | view in chronology ]
They seem trustworthy
Gotta say, a company that takes security that seriously is definitely one that can be trusted to prioritize the security of their customers.
[ link to this | view in chronology ]
Have to go to the wayback machine (archive.org)...
To see the customer list now, but there are a few ISPs on there. This opens up all sorts of possibilities for hacking consumers, either by ISP-owned routers, DNS spoofing or whatever.
https://web.archive.org/web/20190714085412/https://www.solarwinds.com/company/customers
[ link to this | view in chronology ]
Re: Have to go to the wayback machine (archive.org)...
Oh, boy.
[ link to this | view in chronology ]
I used to laugh while watching the silly hollywood movie depictions of hackers gaining root access ... bang bang enter - I'm in!!!!
Now I am not so sure it is funny anymore - damn!
[ link to this | view in chronology ]
Re:
Well, this is one of those deals where it's not like any hacking was necessary to gain access.
Wait. I'm routing around their firewalls. Past the second one. Shit they're on to me, gotta type faster!
[ link to this | view in chronology ]
Re:
"Now I am not so sure it is funny anymore - damn!"
For a great many years it was possible to bypass the screen lock on a windows PC, just by navigating the help function until you got to the "clock & time" field - at which point you could keep navigating through explorer as an admin.
And for all but the last few years it was similarly possible to "hack" almost any router in seconds. And you can still run PIN brute-forcing.
[ link to this | view in chronology ]
Apple's employee database (think massive HIPPA violations if leaked) had username: apple, password apple.
When revealed to be not-at-all-secure they changed to Apple / Apple321
This had basically everything about employees and you could access/amend their HR data...funnel their salary elsewhere etc. This went on for 5 1/2 years.
[ link to this | view in chronology ]
Re:
Apple (and any ordinary employer) is not subject to HIPAA.
"The Privacy Rule, as well as all the Administrative Simplification rules, apply to health plans, health care clearinghouses, and to any health care provider who transmits health information in electronic form in connection with transactions for which the Secretary of HHS has adopted standards under HIPAA (the “covered entities”)."
https://www.hhs.gov/hipaa/for-professionals/privacy/laws-regulations/index.html
[ link to this | view in chronology ]
Wonder?
https://www.washingtonpost.com/technology/2020/12/15/solarwinds-russia-breach-stock-trades/
This is getting more and more obtuse and abit beyond STUPID.
A random 3rd party DLL pops up and they dont ASK who/where it came from.
LOVE this country.
[ link to this | view in chronology ]
We the people...provide for the common defense...
One of the expressly-mentioned purposes of the Constitution is to provide for the common defense.
What happened in D.C. today? Hearings on bullshit conspiracy theories regarding fictitious election fraud. GOP "Senators" spreading nonsense unsupported by facts.
The actual fact that our Government was hit by a massive hack orchestrated by one of our primary enemies? No hearings. No comments. No consequences.
I say this as someone who is unaffiliated with any political party, and who never has been registered or participated in a political party in his life:
Our leaders need to be replaced. Completely. D.C. needs to be purged of every last politician who doesn't take their Oath seriously. Right now, that largely means starting with the GOP. They are too busy trying to suck Trump's mushroom than PROTECTING OUR NATION.
Fuck them all.
[ link to this | view in chronology ]
Re: We the people...provide for the common defense...
The GOP are just work-hardening Trump's mushroom so he can fuck-up democracy until it gives birth to another Trump, this time, biglier & better for the rich & powerful.
[ link to this | view in chronology ]
Re: We the people...provide for the common defense...
No thanks, one never knows what nasties one may catch!
[ link to this | view in chronology ]
Re: We the people...provide for the common defense...
"The actual fact that our Government was hit by a massive hack orchestrated by one of our primary enemies? No hearings. No comments. No consequences."
Too abstract. Now, if that same attack had actually generated casualties or hit something the unwashed masses cared about, like an NHL arena...oh, those politicians would be thanking divine providence for a chance to cater to their base by calling for whatever act of doom and thunder would make the noise most likely to get the attention of voters while filling the coffers of their campaign contributors.
I still recall the investigation of how before 9/11 the FBI were told to back off from the extremists learning how to pilot airliners because those extremists were scions of wealthy saudis, and how right after 9/11 the relatives and families of those suspects were escorted to the airport and instantly transported back to saudi arabia by the secret god damn service. Just so as not to muck things up diplomatically.
Even a credible threat against US interests will only be acted on if, when, or in such a way that it benefits the body politic.
[ link to this | view in chronology ]
Advanced zero-defect secured line
[ link to this | view in chronology ]
A proper password
GoAskYourMother
[ link to this | view in chronology ]