Oracle Tells The White House: Stop Hiring Silicon Valley People & Ditch Open Source
from the well,-that's-one-way-to-think-about-things... dept
Even though Oracle is based in the heart of Silicon Valley (I can see its offices from my own office window as I type this), the company has become sort of anti-Silicon Valley. It tends to represent the opposite of nearly everything that is accepted wisdom around here. And its latest crusade is against open source technology being used by the federal government -- and against the government hiring people out of Silicon Valley to help create more modern systems. Instead, Oracle would apparently prefer the government just give it lots of money.
First, some background: over the past few years, one of the most positive things involving the federal government and technology has been the success of two similar (but also very different) organizations in the US government: US Digital Service (USDS) and 18F. If you're completely unfamiliar with them there are plenty of articles describing both projects, but this one is a good overview. But the really short version is that both projects were an attempt to convince internet savvy engineers to help out in the federal government, and to bring a better understanding of modern technology into government. And it's been a huge success in a variety of ways -- such as creating federal government websites that are modern, secure and actually work. And even though both programs are associated with President Obama, the Trump administration has been adamant that it supports both organizations as well, and they're important to continuing to modernize the federal government. The offices are not politicized, and they have been some of the best proof we've got that government done right involves smart, dedicated technologists.
Of course, not everyone is thrilled with these organizations. Old school federal contractors, for one, have been grumbling loudly about 18F daring to do things like making government procurement open to small businesses. After all, these contractors have spent decades charging the government billions for crappy products, in part, because they know how to work the system. Bringing in actual engineers who realize that it's crazy to spend so much money on crappy solutions -- especially when there are much better solutions that are often open, seems to really piss off some folks who grew fat and happy overcharging the government. And they've found some front groups who argue that these programs are a waste of government money, which would be better spent giving billions to private contractors.
Either way, the Trump Administration, following a Trump executive order, requested feedback on how best to modernize government IT. The request for comments and all the submitted comments are on Github (which is nice to see). Many are quite interesting, but the one that really caught my eye, was Oracle's submission, which I can only describe as... curmudgeonly.
A little more background: if it weren't for Oracle's failures, there might not even be a USDS. USDS really grew out of the emergency hiring of some top notch internet engineers in response to the Healthcare.gov rollout debacle. And if you don't recall, a big part of that debacle was blamed on Oracle's technology. So, perhaps it's not surprising that Oracle might hold a bit of a grudge against USDS. Similarly, while Oracle likes to claim that it's supportive of open source technologies, most recognize that open source has been eating Oracle's lunch for a while now.
Even with all that background, the sheer contempt found in Oracle's submission on IT modernization is pretty stunning. The letter complains about three "false narratives" that "have taken the [US government] off track":
False Narrative: Government should attempt to emulate the fast-paced innovation of Silicon Valley. Silicon Valley is comprised of IT vendors most of which fail. The USG is not a technology vendor nor is it a start-up. Under no circumstance should the USG attempt to become a technology vendor. The USG can never develop, support or secure products economically or at scale. Government developed products are not subject to the extensive testing in the commercial market. Instead, the Government should attempt to emulate the best-practices of large private-sector Fortune 50 customers, which have competed, evaluated, procured and secured commercial technology successfully.
Now, this is kind of funny if you follow anything having to do with government and IT projects over the past few decades, as compared to what's happened on projects where USDS and 18F have been involved. For example, remember the big new $600 million (only $220 million over budget) computer system the FBI paid for that was useless for catching terrorists and had to be completely written off? This was the system, built by giant government contractor SAIC, that a computer science professor who was asked to review the system said he was planning to go on a crime spree the day the system launched, knowing the FBI wouldn't be functional. The same system that was so bad that a contractor who was trying to do something so simple as add a printer to the network had to hack the system, accessing the usernames and passwords of 38,000 FBI employees (including then director Robert Mueller) just to do his job.
Is that really the kind of world we want to go back to? And that's just one example, but there are many others like this. Yet, whenever you look at the systems that USDS and 18F are working on, they seem to actually work. They also seem secure. So, sure, it's easy to attack having the government put together these systems, but real world experience seems to show that these groups, staffed with experienced internet engineers does things a lot better.
False Narrative: In-house government IT development know-how is critical for IT modernization. In-house government procurement and program management expertise is central to successful modernization efforts. Significant IT development expertise is not. Substantial custom software development efforts were the norm at large commercial enterprises, until it became obvious that the cost and complexity of developing technology was prohibitive, with the end-products inherently insecure and too costly to maintain long-term. The most important skill set of CIO’s today is to critically compete and evaluate commercial alternatives to capture the benefits of innovation conducted at scale, and then to manage the implementation of those technologies efficiently. Then, as evidenced by both OPM and Equifax, there needs to be a singular focus on updating, patching, and securing these systems over time.
There's at least some truth to the idea that developing things from scratch is not ideal in many cases, but claiming that those making decisions on federal IT shouldn't have development knowledge is ludicrous. When you don't have that kind of knowledge, that's when you get the big federal contractors coming in and selling you $600 million FBI computer systems that are useless at catching terrorists. I'd be curious if any software developers out there actually think they get better requirements docs from those with dev experience, or those without? Because over and over and over again, I've seen that when the management side actually understands software development, then the process tends to go much more smoothly, because people are much more realistic. Having non-technically inclined managers making these decisions tends to go poorly. Remember the massive computer system that the Copyright Office wasted millions on? That involved a failure of the Copyright Office to set requirements with the outside vendor who never could actually build a working system.
False Narrative: The mandate to use open source technology is required because technology developed at taxpayer expense must be available to the taxpayer. Here there is an inexplicable conflation between “open data,” which has a long legacy in the USG and stems from decades old principles that the USG should not hold copyrights, and “open source” technology preferences, which have been long debated and rejected. There is no such principle that technology developed or procured by the USG should be available free for all citizens, in fact that would present a significant dis-incentive to conducting business with the USG.
This is the most ridiculous of all. Copyright law is pretty clear on this: works of the US government shouldn't be subject to copyright -- and many in the government have embraced variations on open source to live up to that requirement. The idea that open source somehow creates disincentive to working with the US government is hilarious. Maybe for a company like Oracle, but tons of others are happy to work with the US government and lots of open source technologies have made government IT faster, cheaper and more secure.
But Oracle really wants to dig in on this point, with some complete bullshit about how open source is somehow less secure... because the Equifax hack came via a vulnerability in open source:
Developing custom software and then releasing that code under an open source license puts the government at unnecessary security risk as that code is not “maintained by a community,” but is rather assessed and exploited by adversaries. Further, this practice puts the government – most likely in violation of the law – in direct competition with U.S. technology companies, who are now forced to compete against the unlimited resources of the U.S. taxpayer. The Equifax breach stemmed from an exploit in the open source Apache Struts framework.
The Equifax breach stemmed from Equifax failing to patch a widely discussed bug that competent administrators should have patched. The bug was found and patched because it was open source.
Speaking of "false narratives," Oracle also claims that open source technology is being used less and less in the corporate world:
Open source software has many appropriate uses and should be competed against proprietary software for the best fit and functionality for any given workload, but the fact is that the use of open source software has been declining rapidly in the private sector. There is no math that can justify open source from a cost perspective as the cost of support plus the opportunity cost of forgoing features, functions, automation and security overwhelm any presumed cost savings. The actions of 18F and USDS plainly promote open source solutions and then propagate those mandates across government with the implicit endorsement of the White House. The USG’s enthusiasm for open source software is wholly inconsistent with the use of OSS in the private sector.
If you actually follow the open source software market, Oracle's claim here is laughable. Open source is now commonplace in the enterprise and that's only increasing, not decreasing.
Also, somewhat hilariously, Oracle tries to argue that letting USDS and 18F develop things means that there will be extra costs, compared to letting private companies develop stuff:
The largest contributor to cost and complexity is customization, yet actions of the USG and the Report seem to embrace both government developed bespoke technology and customization. Custom code needs to be maintained, patched, upgraded and secured over the long-term. The cost of technology comes almost entirely from labor, not from component parts, whether software, hardware, or networking. The goal should be to seek leverage and scale by engineering out labor costs, including process engineering. Government developed technology solutions must be maintained by the government. Every line of code written by 18F, USDS or another government agency creates a support tail that results in long term unbudgeted costs.
But, again, looking at historical IT implementations pre-USDS and 18F and you see example after example of it being the outsourced, private, large government contractor companies whose work results in massive unplanned maintenance costs.
Seriously, this entire filing by Oracle is one giant false narrative of people living in denial about how the world works these days.
There's even more nuttiness in the filing, but you can go through it yourself and count how frequently you gasp at just how wrong it is. This is an old, legacy company trying to cling desperately to old, obsolete, legacy ways. Oracle's entire business was originally created to serve the US government as a customer, and it clearly doesn't want to give that up. But, once again, things like this just make it clear why the top engineers coming out of school today don't have much interest in going to work for a company with views like Oracle's.
Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.
Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.
While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.
–The Techdirt Team
Filed Under: 18f, it modernization, open source, usds
Companies: oracle
Reader Comments
Subscribe: RSS
View by: Time | Thread
"The USG’s enthusiasm for open source software is wholly inconsistent with..."
... Oracle's need to skim easy federal money from decades-old, proprietary installations.
[ link to this | view in chronology ]
Re:
[ link to this | view in chronology ]
Re: Re:
[ link to this | view in chronology ]
Re: Re:
Actually gets done? Almost never.
They have arranged the infrastructure so that its basically impossible to patch oracle installations unless your a highly skilled contractor with full access to technet.
I've worked at a few places that are oracle shops, and no-one patches oracle installations except for major refreshes on new servers every few years. Cos its far too easy to f*k it up and cause major downtime.
[ link to this | view in chronology ]
[ link to this | view in chronology ]
[ link to this | view in chronology ]
Re:
[ link to this | view in chronology ]
Re:
http://www.zdnet.com/article/oracle-prepares-to-spin-off-java-ee-to-eclipse-foundation/
[ link to this | view in chronology ]
Equifax breach
[ link to this | view in chronology ]
Re: Equifax breach
Twice. With a few months inbetween hacks.
[ link to this | view in chronology ]
Re: Equifax breach
[ link to this | view in chronology ]
Not this again
[ link to this | view in chronology ]
Re: Not this again
As anyone with three active neurons could tell you, filing a frivolous suit against IBM claiming infringement, when a goodly part of their business relies on running Other Companies' Computers, is unlikely to go well, and it didn't.
They pretty clearly got some funding from a Microsoft proxy which I suppose was well-spent by showing the total lack of any code infringement by Linux & cementing its place as The Other OS for server rooms.
[ link to this | view in chronology ]
Re: Not this again
[ link to this | view in chronology ]
B.S.
The USG can never develop, support or secure products economically or at scale. Government developed products are not subject to the extensive testing in the commercial market.
First hand experience that that statement is crap. I mean look at all the IOT products getting pwned repeatedly guess they really did their testing on those.
Also when developers are more interested in making a good product instead of greed the government developed products are cheaper and more maintainable over the whole life of the product.
And if you look at recent DoD instructions and directives you will see government products are being held to high standards. It just may be the case that the individual program is not being managed correctly, think F-35. And if management of a program is bad you can bet they won't know how to reign in contractors that are out to gouge the government.
The government equipment I work with is better secured and maintained than anything a contractor developed.
Contractors (at least the leadership and management) just want to be funded to design a product and sell it to the government with no thought of how their development decisions will impact maintenance costs in the future.
Sometimes they purposely plan on leaving in bugs because they know most government program managers won't catch on and that they will come back to the contractor to fix the bugs in the future.
[ link to this | view in chronology ]
Only read 1/2 of this..
A musician learning Computers is BETTER then a programmer learning to do music..
The problem with the CORPS tends to be creating IRS software.. DO YOU REALLY WANT THE CORP to create the IRS software???
OR would you rather a person that is willing to LOOK/LOCATE every penny that a CORP OWES THE GOV..
This is as bad as our Computerized VOTING SYSTEMS, DIEBOLD(?) would not let anyone evaluate..
I think I know a few tricks that would make them Unhackable.. Unless you took it physically and Corrupted the system, which you would need to do to EACH system. A real independent programmer/hardware person KNOWS all the ins/outs of What has/can be done..
THEY ARENT into making a backdoor, or Easy access if not needed..
[ link to this | view in chronology ]
[ link to this | view in chronology ]
That is how one the biggest credit card number number thief, ALberto Gonzalez, was able to do what he did for years, before the Feds caught up to him.
The Feds have done then when they want to track down someone who posted something on a forum they did not like, and did not want that "pesky" Fourth Amendment to get in the way.
When someone, say, posts to Wikileaks, the Feds could break into the MySQL backend, get the metadata they needed to trace someone, and Julian Assange would never know the Feds were in his system.
The fact that MySQL does not have logging is something that does need to be fixed.
[ link to this | view in chronology ]
Re:
[ link to this | view in chronology ]
Re: Re:
[ link to this | view in chronology ]
Considering that ORACLE cannot fix its own software
They cannot even produce a relational database management system but have consistently fooled many people into thinking that their product is so.
I reported a specific bug in there DBMS in version 6. It was still there in version 9. I never did test in any later version as I no longer have anything to do with their software, This specific bug meant the difference of fractions of a second compared to greater than 10 minutes on tables containing 10 million and 100 million records.
I gave them example SQL that demonstrated the problem. They wanted a snapshot of the database, traces, etc for which I had no authority to give (since it contained commercial-in-confidence information). The example SQL would have taken them 10 minutes effort to replicate problem (well that's what it took me) and yet they said they would be unable to replicate without the snapshot, etc. Go figure.
I gave up on them after that and now use PostgreSQL for any database work that is required. Their software is awful, cumbersome, poorly designed and too overly complex for the tasks at hand.
And they call the kettle black???????
[ link to this | view in chronology ]
Re: Considering that ORACLE cannot fix its own software
[ link to this | view in chronology ]
Re: Re: Considering that ORACLE cannot fix its own software
Dynamically create the the outer select based on the results of the IN select and it runs so much faster that just putting the select into the IN. Official documentation from ORACLE since V6 says that they should return in the same length of time.
eg.
select .... from table1 where fld1 in (select fld2 from table2 where ...);
compare with
assign the results of
select fld2 from table2 where ...;
to a variable as string (say var1) and then dynamically create a new string
var2 := "select ... from table1 where fld1 IN (" || var1 || ");"
and then submit and execute contents of var2.
The second process was measured at less than 1/10 of a second, the former was measured at around 10 minutes. table1 had 100 million records, table2 had 10 million records.
Go figure.
[ link to this | view in chronology ]
The entire sales pitch usually consists of throwing around buzzwords, like "Big Data", "AI", "Cloud" and "Automation", then scaring the hapless deciders with lots of technical terms they don't understand, and then claiming that their product will solve all problems and do everything that staff used to do. Often their claims are strait-up lies. Countless millions have been wasted on their “solutions”. Parasites.
[ link to this | view in chronology ]
We Know Oracle Is Anti-Open-Source
Look at what happened to every single one of the open-source projects that Sun was running when Oracle took them over: Ellison & co succeeded in antagonizing all their communities and driving them away.
We all assumed that the one thing Oracle wanted from that acquisition was control of Java. But even that is now being driven into the ground, with the Google lawsuit, as well as general neglect.
[ link to this | view in chronology ]
Re: We Know Oracle Is Anti-Open-Source
[ link to this | view in chronology ]
No thanks. I don't want to get a heart attack.
[ link to this | view in chronology ]
FTFY
[ link to this | view in chronology ]
[ link to this | view in chronology ]
don't forget the licensing problems
[ link to this | view in chronology ]
18f
Found it:
https://gcn.com/articles/2015/11/11/18f-reverse-auction-micro-purchasing.aspx
[ link to this | view in chronology ]
Re: 18f
In the 1990's the IRS asked for bids for new computer systems..
After the bids were taken and selected..they had to run them passed the Congress to get things paid for..
After 2-3 years it was passed..
The Contract was based on TIME..and what was Available at the TIME of the asked for bid..
In the 2-3 years, we went from 386 to Pentiums..
HIS bid being 2-3 years old, HE SUPPLIED what was bid on from the past..and MADE BUCKS..
[ link to this | view in chronology ]
Re: Re: 18f
[ link to this | view in chronology ]
Re: 18f
[ link to this | view in chronology ]
[ link to this | view in chronology ]
There's no substitute for in-house expertise
Like Mike said, that's how you spend a billion dollars on IT systems that don't work and have to be thrown away.
[ link to this | view in chronology ]
Re: There's no substitute for in-house expertise
[ link to this | view in chronology ]
Don't! Now Drumpf is gonna scrape it.
Ahem. Oracle just needs to hit the lobby gland the right way. Just look at the FCC.
[ link to this | view in chronology ]
FOSS = Waste?!
How is this not treason?
[ link to this | view in chronology ]
Re: FOSS = Waste?!
[ link to this | view in chronology ]
Oracle is just salty
[ link to this | view in chronology ]
What a riot!
While the Oracle response shows the tone-deafness of lawyer-speak, it might be more interesting to examine their claims in a balanced manner instead of engaging in virtue-signaling posturing.
For claim one, you really don't want the government acting like SV do you? Old boy networks of VC funding where 95% of what gets funded either fails, evaporates or is flipped via IPOs to a gullible public who gets saddled with buggy vaporware that demo'd well at TechCrunch Disrupt? Sure, the innovative spirit to try stuff and quickly find out what works and what doesn't is a better way to build software than letting out 10-year aircraft-carrier procurements, but it's not all sunshine and roses either.
Oracle is right about the main cost of custom software being labor -- for development and maintenance. And your special full-stack custom solution that you labored over last year is something you'd be embarrassed to support this year. And heaven help the poor shlubs who won the O&M contract to support your flash-in the-pan inspirational ORM framework. Like everything, people need to make the right choice about the mix of COTS, OSS and custom code that make up a system and those choices aren't about your belief system -- they're about the need to build and support a cost-effective, reliable, secure, agile and responsive application. All COTS? No. All OSS? No. All Custom? Hell no. When there are good COTS / OSS solutions there should be no need to build custom solutions. Just ask David Bray.
Open source is great when it works, but the idea that all software should be free does have the problem of how the developers actually get compensated for their work. Charging the government over and over to build the same code happens less often than you think, since every agency thinks their mission is "special" and unique. And the fact is that a lot of open source software gets abandoned or torn apart in forking wars. I suspect Oracle's definition of OSS is different than most -- in their world Cloud Foundry, MySQL and Linux aren't OSS -- they're purchased and supported software from Pivotal, Oracle and Red Hat. It's likely that most commercial companies don't rely on pure OSS, but partner with a vendor for support and service.
Anyway, just wanted to drop in and add some balance to the discussion. You can go back to bashing Oracle now.
[ link to this | view in chronology ]
Re: What a riot!
(Also, when you complain about imaginary "virtue signaling," you are only signaling one thing.)
[ link to this | view in chronology ]