IRS Still Working To Complete Computer Upgrade From Windows XP To Windows 7
from the by-2020-it-will-be-2010 dept
Like many other entities (both singular and conglomerate), the IRS was reluctant to sever ties with Windows XP. Microsoft forced the issue, however, and gave everyone plenty of time to migrate to an operating system released sometime in the last ten years. Even with this head start, the IRS has yet to meet this target.
An Inspector General's report notes that the IRS is almost finishing upgrading its workstations to an operating system that's only eight years old (Windows 7). Almost.
As of May 2015, the IRS has completed most of the Windows XP workstation upgrades across the country. Approximately 1,300 workstations have yet to be located or confirmed as running the old operating system.At this point, I'm going to do something I rarely do: cut a government agency some slack. The IRS did have plenty of workstations to upgrade -- nearly 110,000 -- so if 1,300 went "missing," it's somewhat understandable. On top of this, budget issues forced the agency to upgrade old workstations instead of replacing them with newer systems, which would have greatly sped up the process.
The IRS claims it does know where these missing 1,300 workstations are, but that the Inspector General won't listen to it. The included "Management Response" says the following:
The audit incorrectly concludes that IRS has not accounted for all XP workstations. We acknowledge there were challenges with our inventory data due to the many antiquated systems in our IT ecosystem. In spite of this, we took extraordinary steps to identify, document and upgrade every XP workstation in the IRS. On several occasions throughout the audit, the IRS provided information to the TIGTA team that clearly documented the number of workstations to be upgraded, where those workstations were located, and our strategy to complete the upgrades. Although footnoted in the report, TIGTA opted not to change their assertion that the IRS had not accounted for all XP workstations. As of this date, only 71 Windows XP workstations remain to be migrated.'The IG's footnote tells a different story.
After the conclusion of our fieldwork, the IRS provided documentation that these workstations were located and upgraded to Windows 7, as of July 22, 2015. We were unable to verify this information.Beyond the workstations, there's the IRS's servers, which are also running up against Microsoft's upgrade clock. This not-overly-optimistic statement by the IG suggests the IRS will be living in the (OS) past for much of the future.
Based on our discussions with management, we determined it is unlikely that the IRS will have its servers upgraded to Windows Server 2012 any time this Fiscal Year.This is due to the fact that the IRS is still struggling to upgrade its servers to seven-year-old software.
In fact, the IRS still has not fully upgraded its servers from Windows Server 2003 to the 2008 release. Currently, the IRS has approximately 3,000 Windows servers still running the 2003 operating system. Management informed us that they have upgraded approximately 4,100 servers to the 2008 version which is already seven years old. The IRS currently has no servers running the 2012 operating system in production at this time.Time to start reeling in the slack I cut the agency earlier. This logistical issue seems especially absurd.
The IRS also discovered nearly 6,000 applications being used by employees to do their jobs that required an assessment of each application to determine whether it would operate on Windows 7.Unfortunately, the report doesn't provide more details on the massive amount of applications being used by the IRS. Every interlocking piece presents a new possibility for a hole or an exploitable flaw, something compounded by the use of unsupported system software.
The IRS has already seen its system exploited by scam artists, who were able to use the credentials of taxpapyers to fraudulently obtain refunds. That its "user data" (the tax records and personally-identifiable information of millions of Americans) is secured behind a patchwork of outdated software presents criminals and rival governments other opportunities for exfiltration and exploitation of taxpayer data.
Even if the IRS manages to hit its self-imposed targets for the most recent round of upgrades, support for those operating systems is also on its way out.
Despite the eventual progress made by the IRS on the Windows XP upgrade efforts, we believe the IRS provided inadequate oversight and monitoring during the early phases of this effort, starting with including it among other Microsoft product upgrades rather than making this effort its own project up to the decision made by the CTO to oversee the project himself. In addition, after taking four years to upgrade to Windows 7, the IRS is now faced with the challenge of addressing Microsoft’s announcement to end extended support for Windows 7 in January 2020.The IRS has agreed to a majority of the Inspector General's recommendations which means… well, it probably doesn't mean much of anything. Chances are the IG will revisit this in a few years and still see the agency struggling to stay current with its operating system software. It's eight years behind on system software and seven years behind on server software, with the latter's migration less than 50% complete. The IRS doesn't have it easy, not with 110,000 workstations, 7,000 servers and -- for god knows what reason -- 6,000 applications, but unless it's willing to give this the priority it deserves, it will always be in danger of making a flawed, bulky system even more insecure.
Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.
Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.
While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.
–The Techdirt Team
Filed Under: computers, government, government it, irs, windows 7, windows xp
Reader Comments
Subscribe: RSS
View by: Time | Thread
[ link to this | view in chronology ]
Re: As we learned this week
Ahem. I think that brilliant comment is lost on most readers.
If you keep up with the tech news you will know that Microsoft automatically fetching Win 10 and caching it so that Win7/8 can update. In the past week it's come out that Microsoft will force those updates on everyone.
[ link to this | view in chronology ]
Re: Re: As we learned this week
And then once all the systems are upgraded to Win10, they can start sending potentially confidential taxpayer information from those systems back to Microsoft.
[ link to this | view in chronology ]
[ link to this | view in chronology ]
Why not go GNU?
I used the same hardware for a total of 13 years until something affected the board on mine, and it refused to turn on (after everything was tried), then I got a used one and have still been using it. No problems.
Maybe consider moving away from microsoft's monopoly and choosing free software would be a better thing, and last but not least, far less exploits due to constant patching. Imagine how much money it would save (no licenses, activations or product keys!), and since updates can easily be configured to be unattended (I myself haven't made use of that, too advanced for casual users!), it can happen when the offices are closed.
I would hate to see the servers running windows too, when ubuntu server (or another server client, be it debian server?) be installed and then maintenance can go through much easier. Then upgrading can be fool-proof, easily done. I wish they would consider this.
[ link to this | view in chronology ]
Re: Why not go GNU?
The most obvious benefits of such a move are security and cost. Closed-source operating systems are insecure by design and cannot be fixed; open-source operating systems are not necessarily secure, but they provide a fighting chance. Closed-source operating systems are extremely expensive to maintain, especially at scale (witness this article); open-source operating systems are vastly cheaper both to run and to upgrade.
But there are other benefits as well: open-source operating systems run well on older hardware (I'm typing this message on a Lenovo laptop that's 8 years old) (and it's not my oldest one) and make optimal use of hardware resources. They are unencumbered by the spyware that is now not only epidemic in applications, but part of Windows 10. They enjoy incredible, long-lived support and there is an enormous pool of talent out there skilled at debugging them. They also make a serious effort to comply with standards, whether those are protocols, file formats, or anything else -- thus they're highly interoperable.
The IRS should have left Windows in its read-view mirror over a decade ago. Every dollar spent on this "upgrade" is wasted.
[ link to this | view in chronology ]
Re: Re: Why not go GNU?
"I have labored to get Microsoft Certifications. What shall I use it for when the BSD servers tend to just work!"
[ link to this | view in chronology ]
Re: Re: Re: Why not go GNU?
And not just from the IT guys. From the accountants and managers too. I work at a small shop for a guy who used to be an accountant. I've attempted to persuade him that going the GNU route on our workstations would be beneficial in the long run, but I'm always met with the attitude of "if it doesn't cost a lot of money then it cannot be of any value" from him. It's a pretty difficult mindset to combat sometimes.
[ link to this | view in chronology ]
Re: Re: Re: Re: Why not go GNU?
Add to that, "Who do I phone (or sue) when it breaks?!?"
[ link to this | view in chronology ]
Re: Re: Re: Re: Re: Why not go GNU?
When I point out that -- so far -- no customer who has been given clearly-defective software by Microsoft or Oracle or Apple or IBM or or or or has successfully litigated against them to recover damages, that question tends to go away.
[ link to this | view in chronology ]
Re: Re: Re: Re: Re: Re: Why not go GNU?
and people get in the same trap again nowadays!
"Put your files in the cloud". Are people insane, even several 100.000.000 user companies like Megaupload is unable to protect the files.
"Sync your devises". Are people even more insane. Something might delete things on all your devices automatically.
[ link to this | view in chronology ]
Mindsets
If an accountant balance books all day long for years, it would be natural for him to believe that value on one side is balanced with some other value on the other side, in the real world too. As even goodwill may be post in the sheets, why shouldn't he believe so?
If a manager want something done, he have to fund it. If he provide to little funding it takes longer and may cost more in the long run. And the return on investment will be delayed too. Why shouldn't he believe that there must be a close relation between cost and value?
And most people experience that if something is too good to be true, it usually is.
This is the most dominating belief-system in our western world.
Fallacies play a vital role in most belief-systems, this one including. It fails to consider that there is a lot of value we doesn't pay for, friendship, close relationships, the air we breathe. We may take it for granted as we is so used to only value things with price tags. Until we loose something, then it might be to late.
It is just too easy to focus only on subjects that has to be fixed, because they demand effort to be moved along, and forgetting to appreciate what we have.
Sometimes we can choose to pay or get something for free, be it sex, software, and many other things. Things that is too good to be true, might be the best things in life!
[ link to this | view in chronology ]
Re: Re: Why not go GNU?
[ link to this | view in chronology ]
Re: Re: Why not go GNU?
Yes, they should've, however given their masters' (Congress) vulnerability to (or dependence on) corporate lobbyists, can anyone really be surprised it isn't an option? Oracle Corp. alone could get them all crucified easily with a few FUD doom and gloom press releases. Corporate IT is rabidly pro-proprietary. It took massive amounts of begging for years just to convince them to try it on servers. That, and a lot of do it and don't tell 'em until you can prove to them it works.
Add to this biting the bullet and migrating to a new system like this is a huge leap for these people. They're convinced that (eg.) migrating from MS-Office to LibreOffice would mean completely throwing away whole skillsets requiring complete (and horrifically expensive) retraining for users. The cost of retraining users to go from one version of Windows to another is already massive. I've never understood why users shouldn't be expected to retrain themselves, but I've never worked in HR or management.
It's too bad none of them can even consider getting ahold of Munich Germany's Linux distro and running a pilot program through the many iterations it would take to get it into use. This isn't the way managers like to do things though. They want to be in the herd, not leading it.
[ link to this | view in chronology ]
Re: Why not go GNU?
It sure would save a lot of software licensing money.
[ link to this | view in chronology ]
Re: Why not go GNU?
[ link to this | view in chronology ]
Re: Why not go GNU?
Granted, every Windows update has a new GUI for users to figure out, so IMO making the switch to Mac or Linux isn't any different. However, the applications will have to change as well. There are *nix compatible version of popular Windows software, but "their just not the same as Office", even though people only use a small number of the features.
People just don't like change, and the people in a position to make the change are the least likely to want it, especially if it affects them. I can't tell you how many policies I've seen that are immediately ignored by those in power. They may pay lip-service to it, but in practice they don't follow it, or find a way to get around it.
Another significant factor is Exchange email servers. These agencies have invested significant resources in an Exchange infrastructure. Currently, I'm not aware of any open-source equivalent to Exchange that can be a drop-in replacement. Since email is the main work-tool for most people (and frequently a storage medium as well), not being able to replace Exchange is a deal-killer.
Finally, there just aren't that many people trained in *nix, and those who are get well-paid. As you may have heard, the gov. is not a place to get wealthy, so the people with the necessary skills will go where the money is, i.e. the private sector.
At the place I currently work, they haven't had a bona-fide developer there for at least five years, but probably longer. They have temporary workers, who might stay for a couple of years before moving on. Management isn't willing to pay a programmer what they are worth to maintain all the systems, so they have to make do with band-aids and duct tape.
I'm sure the IRS is similar: get the most work with the least money. Switching to Linux, while intelligent for many reasons (and may have been advocated at times), simply won't happen because no one with authority is interested in it. If something bad happens, they will be in the spotlight, so it's easier to make excuses than fix the underlying problem.
[ link to this | view in chronology ]
Re: Why not go GNU?
I mean, of all things, this is an absolute NO BRAINER that would give them far more security, more stability, and forever burn that demonic 'Microsoft End-Of-Support' calendar that everyone gets beaten to death by.
All government systems should be Linux-based with in-house programming. Anything else is a tremendous security risk, not to mention a massive monetary-black-hole.
[ link to this | view in chronology ]
I agree!
I agree! This is no easy task, because first you have to destroy all the hard drives, and the you have to...
(kidding aside, there is the unfortunate truth that not only do they have to upgrade 110,000 machines, but they have to do so at the speed of bureaucracy)
[ link to this | view in chronology ]
Re: I agree!
[ link to this | view in chronology ]
Re: I agree!
Yup! Everyone knows that it's impossible to ever erase anything off a hard drive. If you know what you're doing, you can retrieve every bit of information that's ever been saved to that drive, no matter how many times it's been overwritten! CSI said so!
[ link to this | view in chronology ]
Re: Re: I agree!
I'm rolling on the floor at what you said, which will be taken as 'word-of-god' instead of 'word-of-sarcasm' by the majority.
Feature film on ONION at 6.
[ link to this | view in chronology ]
Targeting vs. upgrading
[ link to this | view in chronology ]
[ link to this | view in chronology ]
Windows 3.11
Probably not, but some of those 6000 applications are 16-bit, and not supported by Windows 7.
[ link to this | view in chronology ]
Re: Windows 3.11
[ link to this | view in chronology ]
Re: Windows 3.11
[ link to this | view in chronology ]
[ link to this | view in chronology ]
Re:
As long as they have automatic updates turned on, Microsoft will take care of that for them.
[ link to this | view in chronology ]
Re: Re:
It doesn't matter that Win X is a MAJOR security risk, and that all I've talked to with it deployed have only kept it there because they were 'made to' by administrative management that 'just don't get it'.
I bailed on supporting Windows in any flavor after they tried to axe Win 7 after only 3-years in the market. I saw it coming to a 'rolling 18-month product window, eventually' and said 'yep, stop the psycho train, I want off here'. Moving my whole network and related support systems over to Linux as the best pain I've had in a long time, and I estimate about $22,000 saved in licensing fees alone, not to even bring-up the 'you have it, you own it' security of knowing I don't need to call anyone to renew keys (and argue with them about them) if I have to reinstall'.
Microsoft took the idea of a 'purchased intellectual property', tried to treat it like a sports car you would buy, then shell-gamed the world into turning a purchase contract into a rental agreement with a LOT of rights-lost loopholes that now impinge on the 4th and 5th Amendments. That they are up to no good is clearly seen in them wanting to now include 'Linux Bash' (the Linux Terminal) into their Windows product. WHY?! Microsoft has 'Terminal' that they took all your DOS command line abilities away from. Ask yourself why they would now want to allow you to reach into a Linux Terminal, even to be able to write code, when the don't even want you to have the ability to create an MSDOS boot USB stick from their own Terminal? Then only answers to this are very nefarious, indeed.
[ link to this | view in chronology ]
[ link to this | view in chronology ]
Big complex systems
But, sadly, the new system will inherit more than familiarity; it will inherit the problems too.
IRS should seriously consider to move towards Linux/FreeBSD/OpenBSD on the servers first, and then start the work of moving the users over too. It should consider how it stores information in a way that still works decades and several migrations later. It is important to take extra care of the employees during any change, but "standing in the Microsoft cement, waiting for it to harden" is a recipe for needless suffering.
[ link to this | view in chronology ]
Re: Big complex systems
Imagine if they just now finished upgrading systems to Ubuntu 9.10, it would be just as bad, if not worse, than being so out of date with their Windows Version.
The real problem is that they think that they have completed something. Keeping current is something that you always do. Windows 7 is already two generations old, if they are not already working on their Windows 10 deployments, they are already behind, for no other reason than they haven't started yet.
[ link to this | view in chronology ]
Re: Re: Big complex systems
[ link to this | view in chronology ]
Re: Re: Big complex systems
[ link to this | view in chronology ]
Re: Re: Big complex systems
Not automatically. It's completely reasonable to have no plans to move to Windows 10 ever.
[ link to this | view in chronology ]
Re: Big complex systems
If you have Linux installed, and you hire a programmer, you have TOTAL AND ABSOLUTE CONTROL over your system, FOREVERMORE! Linux doesn't 'expire', it is 'free', it has a 'desktop' (for users), and it is VERY flexible (not to mention far more virus-resistant, more friendly to your hardware, etc.)
I know what the problem really is, they are now in bed with Microsoft, and if they try to leave, they worry about what Microsoft knows that it would possibly 'leak' or 'disclose' as a result of that much loss-of-income. Or, maybe it's just that word, 'FREE', that the government is choking on...I'm not sure which.
[ link to this | view in chronology ]
[ link to this | view in chronology ]
Re:
[ link to this | view in chronology ]
[ link to this | view in chronology ]
*Not that needing buckets of money should be a factor, just ditch Windows for something open source. At the same time, money for the bureaucracy has never been a priority compared to military boondoggles.
[ link to this | view in chronology ]
Re:
[ link to this | view in chronology ]
Re: Re:
[ link to this | view in chronology ]
Re: Re: Re:
[ link to this | view in chronology ]
[ link to this | view in chronology ]
6000 applications?
Or is this the IRS' way of saying they don't want to pay to upgrade Office XP, QuickBooks 2003, Photoshop 5, and so on?
[ link to this | view in chronology ]
Re: 6000 applications?
This depends on how they've defined "application". It may for example include user developed Windows batch-files that have become essential to their day-to-day running, in which case 6000 could be plausible.
[ link to this | view in chronology ]
Re: Re: 6000 applications?
There is NO WAY they have '6000-applications' that they are individually using, unless they are counting each window as it opens.
In all likelihood, they are counting TONS of script-kiddie / command-line batch files that are nothing more than opening one window and closing another. The real 'cost-applications' would impact maybe 25-50 actual commercial programs, the rest are either 'in-house' scripts or 'band-aids' meant to bypass problems from when they created this IRS computer beast 20-years ago based on Microsoft technology (where they should have stayed with UNIX/Linux and developed their tools in-house anyhow).
Bureaucracy just can't learn the lesson that it cannot move faster than private profiteers. Then again, if you made them pay for it out of their own pockets, they would fix it tomorrow, too.
[ link to this | view in chronology ]
Redhat?
[ link to this | view in chronology ]
Not one of the engineers I encountered would be able to keep an equivalent job in the real world. In this kind of government work, the development cycles are painfully slow and unresponsive, everything is overengineered with no short-term usability goals (or attainable goals in general), internal communication is almost nonexistent, wheels are constantly reinvented, and they don't use any off-the-shelf development or productivity tool unless it was from an approved vendor and was procured 3+ years prior with a 15-year support contract. The layers of bureaucracy for any systems they manage themselves are not designed to accommodate keeping anything updated or patching security holes like you are used to in the real world.
Those of you saying why don't they just use GNU, BSD, whatever...again, they have no private-sector experience, so many of those older military guys just think those are toy/hobby systems for kids and black-hats. If they get over that, then they rail against the licenses and the idea of code being open source. If they get over that, then they say they're too committed to existing technologies, they can't switch mid-stream now, yada yada.
To their credit, 4+ years into the next version of the project, when it became apparent they were repeating the same mistakes, they finally dumped the mega-contractors and cranked out their own system in-house with agile processes, off-the-shelf tools, and modern computers. It finally rolled out in 2012. Maybe the IRS will learn from them.
[ link to this | view in chronology ]
Re:
No one in their right mind could ever say 'UNIX' can't get the job done, so why do you think 'Linux' can't? The argument doesn't even get off the launch pad. So, if I just justified to you the 'seriousness' of Linux in one small response, why doesn't the government buy into it? THIS is the question that you have to take a deep look into, and it is not about 'do-ability', it's about graft, multi-level profiteering, and corruption. Of course, they will always try to throw the shoe one the other foot and say, 'our workforce is too technically illiterate to work in Linux then', but being as Apple iOS is in many ways, another flavor of UNIX too, that argument doesn't fly either. (seriously, I know people that are Apple 'button-pushers', too, and trust me, ignorance can become proficient at ANYTHING that benefits them).
In the end, it's the same old story - follow the money, watch out for those deep holes of corruption and spying you might step in along the way.
I've personally always thought that the number one reason that UNIX/Linux isn't preferred (other than corruption/money) is that it is far more secure than Windows, and you can't justify spying in a system that you can lock down that securely, and trojans are near-impossible to install 'accidentally' in it.
[ link to this | view in chronology ]
[ link to this | view in chronology ]
[ link to this | view in chronology ]
Could have been worse
[ link to this | view in chronology ]
Microsoft Should Be Dismantled
[ link to this | view in chronology ]
[ link to this | view in chronology ]
Re:
You would create jobs, create new software, break the monopoly (and related power-control-center that a few mega-corporations have on government), and at the same time, be able to have an editable, auditable software system that our nation would OWN, could MODIFY, and could KEEP as they needed. It is a win-win this way.
Anything else is paying blackmail to private profiteers to 'not turn our lights off yet', while racing against that wet-concrete clock of 'bureaucracy'.
[ link to this | view in chronology ]