NMCI is the worst thing to happen to the Navy (in my opinion). When I was in the Navy, we managed our own servers and infrastructure on a 1000-workstation aircraft carrier network, including HP-UX/Sybase servers, multiple domain controllers, Exchange servers, rotating tape backups, and Alcatel backbone switches.
We only called contractors when we couldn't figure out a problem (which was pretty rare with the team we had). We set up our own network security monitoring and router ACLs, even though we were not authorized to do so, since we were going into a combat zone (Operation Enduring Freedom) and damn the procedures when your ship is at stake. We had it running so tight that when our battlegroup did penetration testing before deployment, they accused us of cheating because they couldn't get in (even though they had a hole poked in the NOC firewall for them and we weren't supposed to have an incoming ACL on our side).
NMCI rolled out a few months after I got out of the Navy, and ever since then all I've seen coming out of there since then is hugely incompetent ITs who don't even know what the OSI model is, let alone how to put together a disaster recovery plan or manage network security. All they know how to do is put in a trouble ticket when someone can't send an attachment or they see "NTLDR is missing".
I joined the Navy to get the kind of experience that I did, and I feel terrible for the thousands of ITs in there now who have their hands tied for anything harder than resetting someone's password.
I was in the Navy well before NMCI, saw it come in, left for a while (med school), then came back just in time to see it go. I'm looking forward to it being gone. That said, I have found non-IT military folk at all ranks daily making wildly inaccurate statements about NMCI policy and then forming their own policies based on their own statements. The contracts are available online with about 5 minutes of searching, and they're remarkably readable.
I have also seen a small (frigate) shipboard IT organization before NMCI essentially go belly-up when an electrical accident (because, you know, it's a freaking warship underway, not a server farm in Sheboigan) cooked the server and there were no functional backups. Managing your own IT in a small organization can be a major crap shoot.
I work for the federal government, and I work with (but not for) HP/EDS in the building, as they supply the majority of infrastructure services to this agency (including Active Directory, DNS, core and edge network, etc.)
While HP is a competitor to me here, we work cooperatively more than not, and I've never had an issue where another contractor was the impediment.
Conflicting, or impossible-to-meet requirements are the norm from the government. Schedules shift in ways that aren't humanly possible to complete. The government competes the work out to contractors, establishes a contract for the work, and then refuses to live up to the terms of the agreements made.
As part of a video collection effort I was working on, the government wanted to use a reporting tool in place of a document management system... as a document management system. When we explained that we could not purchase that software for them in good faith, they bought it anyway, and then insisted that we use it instead of the document management system we'd proposed (but had not yet purchased) -- citing that we should 'leverage existing resources'.
Rest assured, even at the best of times, the government is a difficult customer, and impossible to comprehend. I understand that DOD-agencies are supposed to be a little better, but I've yet to see anything even remotely close to what I'd consider rational in a business sense.
My experience was similar. We tended to receive implementation directives that frequently contradicted the functional requirements, and when we requested clarification got no response. The government tended to make exceedingly expensive purchases based on politics (my guess is that there were kickbacks involved) that were entirely inappropriate for the project, yet we had to use them. And our advice rarely had any effect on decisions, as far as I could tell.
So far, I've only worked for one organization that was genuinely more convoluted, irrational, and technologically backward than the government... and that was amazon.
Every government system that actually does anything that I've ever come in contact with looks the same. A few racks of windows servers crammed with expensive consulting-ware all chosen and acquired using the "strippers and steak" purchasing method... and one or two 'unauthorized' linux servers actually doing the work.
I guess its good that our computers model the human structures in government. At least we're consistent.
Considering the scale, considering the requirements, considering the difficulties related to the global nature of the network and considering the utter incompetence of any government official that ever had anything to do with IT.
That's a bargain.
Imagine if Google had to provide whole infrastructure plus support - with their legendary customer support. They wouldn't last a day.
When Government tries to do it itself- It will cost 20 Billion.
As long as staff and general officers are allowed to retire and then immediately go to work for the companies where they previously had budget authority over this will continue.
While I do think it's a bad idea for the military to outsource computer networks this article uses terrible evidence to back up it's critique and complaints.
* "Worse, HP — which acquired Electronic Data Systems and its Navy contract in 2008 — still operates under performance metrics set a decade ago. A typical workstation on the network costs the Navy $2,490.72 per year."
A secure workstation with full outsourced support costs $2,490.72 a year? That sounds pretty damn good to me.
* "That includes an e-mail inbox with a 50-MB capacity (Gmail’s: 7,500 MB), and 700 MB of network storage (compared to Evernote’s unlimited, free plan). Anything above that is extra."
Most corporations give their staff 150mb inboxes. Let's see Google meet the military's requirements at 7,500mb per inbox. Better yet, let's see Evernote give the military unlimited storage space. I bet they could meet all of those requirements for free! The public market client is exactly the same as the Department of the Navy, so it should be a quick switcher-oo! Problem solved!
* "A year’s use of a “high-end graphics” workstation sets the Navy back $4,085.64. Extra applications on a laptop or desktop computer can run anywhere from $1,006.68 to $4,026.72 annually. A classified Ethernet port — $9,300 to $28,800 per year, depending on where it’s located."
Yup, that sounds about right. High-end graphics workstations and their software are expensive. So are classified networks.
* "What’s more, HP isn’t required to take security measures like hard disk encryption, threat heuristics, and network access control that are common today, but were exotic in 2000.
Really? They're not taking any security measures?!
“Anti-spam services” runs the Navy $2.7 million per year under the contract."
It costs $2.7 million to filter spam on the second biggest network in the world? Oh, only the ENTIRE INTERNET is bigger? $2.7 million is a steal.
* "Cleaning up a “data spillage” – classified information that got placed an unclassified network – costs $11,800 per incident. In 2008, the Navy paid about $5 million to wipe the data from 432 compromised computers. That’s “almost 10 times the cost of simply destroying the affected machines and replacing them with new ones,” the Washington Times reported."
Security incidents are expensive. The Navy sets the protocol for how these incidents are handled, you can't simply dump a computer into an incinerator and certify that the data is destroyed. Well, I suppose you could, but running an incinerator at the level of heat required to completely destroy data is FREAKING EXPENSIVE, TOO.
In the several parts of the article that they mention lack of what sounds like quality response tie and botched security updates/software roll outs there isn't enough evidence on the incidents to make any comment. Those incidents are asserted in a manner that is hear-say rather than official reports.
I've been really disappointed with Danger Room's tech in National Security reporting.
Just to speak to the one point of security measures, government computers do perform data at rest encryption and network access control. Threat heuristics are done off-the-shelf with something like McAfee/Norton on the workstation end, and with commodity IDS software running at the edge, which I think sounds about right.
For most government machines, they're required to be connected via VPN, and all traffic funneled through the respective agencies in order to be on the internet at all, so at least the cloud data they do access has the opportunity to be logged, scrubbed and sanitized by the agency in question.
I can't speak specifically for Navy, but with most agencies I've dealt with (including DOD,) this is how things are.
So, long story short, if they aren't required to perform any security measures like the above-mentioned, then they should really get kudos for going above and beyond. That said, at least where I am, those are requirements, so I'm guessing the reporter either misspoke or was uninformed.
Can someone who knows more than I do about this explain exactly why the government uses 'no-bid' and 'cost-plus' contracts and why nobody seems to complain about the seemingly obvious (to me, at least) conflict of interest here?
It wasn't clear from the article but it may have been that the original 5 year deal required a bid and then each subsequent deal had to be no-bid for the various reasons stated in the article.
This may be a bit too cynical, so consider this comment entertainment instead of insight.
But do you ever notice how things that please both political parties tend to be the worst, like the DMCA? Well, that explains this. The Republicans want a huge military. The Democrats want to give a ton of money to the private sector.
The compromise? Outsourcing the military's IT. Bigger navy == bigger IT contract == both sides content.
I think the desire to give lots of money to the private sector is more of a Republican thing than a Democrat thing. Republicans tend to be "Rah rah, free market, lower taxes, starve the beast," whereas the Democrats are more comfortable openly advocating for public-sector services.
That is why government agencies should always insist on open source everything. It may seem expensive in the beginning, but it will end up cheaper in the end because you will always have a choice for alternative contractors if you are unhappy with one.
Open source is the answer to a different question.
The article is talking about configuring networks, providing hardware, providing computer management services, etc. The software involved in the services that are being discussed is pretty incidental, the way IBM's use of linux is pretty incidental to what IBM sells.
As one of the articles primary examples, the Navy does not have the network diagrams and configuration information for their network. Even if they used open source routers, they currently do not have the information they would need to configure those routers.
Well, I had a more holistic open source agreement in mind. In such an agreement any configuration information should be included with the source code. Also any custom hardware should also be specified and all rights provided as part of the agreement.
Regardless of how incidental the software is, if the software and configuration is all open source, then it is always possible for a talented engineer or group thereof to pick up the services where the previous contractor leaves.
And we (the citizens) are paying for it, shouldn't we be able to extract as much value from it as we can? (Or better yet, be able to volunteer time to make it better?)
Huh? EDS still holds this contract? I remember EDS lost a large ($1bn+) contract 8 or 9 years ago when it botched up a PC refresh so badly that by the time they started installing new desktops, they were 6-7 years old. I don't fully blame EDS, having interned there a long time ago, they're just a huge inefficient corporation hired by a larger, more inefficient government.
I had a friend who was working for a small local SF consulting shop making $75/hour. The consulting shop was getting paid $150/hour. They got the job through a contract recruiting firm that got paid $200/hour, and EDS was getting paid something like $400/hour, all to do some crystal reporting work. It was quite bizarre.
Ross Perot was was the Honor Board Chairman at the Naval Academy when he graduated in 1954. EDS has been providing data admin services to the Navy since he founded the company. Those ventures are what made him the billions to fund his presidential campaign. EDS is lashed to the Navy with steel wires.
Ross had already left EDS before I started there, and I started there a lifetime ago, though I've met both Ross Sr. & Jr. through my father's work at Perot Systems (now part of Dell). When I was a 14 year old kid interning there, I didn't quite understand why people paid EDS $500/hour to do IT work. Now that I'm a 31 year old entrepreneur, I now understand why people pay EDS $500/hour to do IT work. In the truest sense of the term, it's ignorance.
We only called contractors when we couldn't figure out a problem (which was pretty rare with the team we had). We set up our own network security monitoring and router ACLs, even though we were not authorized to do so, since we were going into a combat zone (Operation Enduring Freedom) and damn the procedures when your ship is at stake. We had it running so tight that when our battlegroup did penetration testing before deployment, they accused us of cheating because they couldn't get in (even though they had a hole poked in the NOC firewall for them and we weren't supposed to have an incoming ACL on our side).
NMCI rolled out a few months after I got out of the Navy, and ever since then all I've seen coming out of there since then is hugely incompetent ITs who don't even know what the OSI model is, let alone how to put together a disaster recovery plan or manage network security. All they know how to do is put in a trouble ticket when someone can't send an attachment or they see "NTLDR is missing".
I joined the Navy to get the kind of experience that I did, and I feel terrible for the thousands of ITs in there now who have their hands tied for anything harder than resetting someone's password.