
Why I fought for open source in the Air Force - signa11
https://opensource.com/life/16/2/why-i-fought-for-open-source-in-the-air-force?sc_cid=701600000011jJVAAY
======
caseysoftware
I was some of the "technical support" for Hanscom AFB when they were deploying
dotProject and then the fork web2project for web-based project management.
They were replacing hundreds of Microsoft Project licenses and projected to
save over $200k/year at the time.

As noted in the article, the biggest things for them were being able to review
security, fix bugs, and affect the roadmap. While I didn't add their
requirements to the core system, I added a number of hooks and interfaces they
specifically requested.

For contractual/legal reasons, they had issues around making the fixes
themselves or even providing them to me but I received a lot of "highly
specific explanations" to issues and approaches on resolving them.

Getting started with Hanscom's team was painful on the contractual side but
excellent to work with once things were rolling.

------
__john
A couple of things on the "lawyers" and "management" points.

The "DoD's clarifying guidance" he's talking about is almost certainly
this[1]. It's got some good information about open source in general, even if
it is a little dry.

If you can prove that there is support and training available for a piece of
software and you can get it approved by your local information assurance
officer then you can put it on the Air Force network. As far as getting it
approved by your local IAO; let's say you want Python on your computer and the
IAO says no then you can point to your local Redhat server (which comes
already built with Python) and point out that you have already accepted any
risks associated with Python by using Redhat. Another good resource for
getting management to buy off on any risk is the EPL[2]. If the software is on
that list then someone else in the Air Force is already using it, and the
software has already gone through an approval process.

Edit: I just realized my second link might be CAC only ... hopefully it will
be useful for any contractors/military folks that browse HN.

[1]
[http://dodcio.defense.gov/OpenSourceSoftwareFAQ.aspx](http://dodcio.defense.gov/OpenSourceSoftwareFAQ.aspx)
[2]
[https://cs3.eis.af.mil/sites/afao/Lists/COTSGOTS%20Software/...](https://cs3.eis.af.mil/sites/afao/Lists/COTSGOTS%20Software/EPL.aspx)

~~~
Turbo_hedgehog
Getting sec_error_unknown_issuer on your [2] link

~~~
jcurbo
It's using the DoD CA which is not distributed with major browsers.

------
thearn4
I'm a strong advocate for FOSS where I'm at in NASA. It can be tough to
approve, but has actually become much easier in the past few years as the
concept becomes more familiar to decision makers.

~~~
WorldMaker
I think it's interesting, too, the tail end "If I did it again I might have
used OpenStack" and the fact that OpenStack has some early roots in NASA's
open source efforts.

Also, I find the 18F Open Source Policy [1], which I saw in a recent HN topic
elsewhere, seems like a good "default stance" for government use of open
source and reiterates some of the same points in the article, and maybe we'll
see some of that attitude prevail in the end.

[1] [https://github.com/18F/open-source-
policy/blob/master/policy...](https://github.com/18F/open-source-
policy/blob/master/policy.md)

~~~
EvanPlaice
18F is very unique in this regard. They somehow managed to get approval to
issue their own ATOs (ie Authority to Operate) as well as provide and use
their own Ubuntu-LTS gold master image as a base.

They largely circumvent a lot of the process with help from policy makers at
the highest levels.

Even then, I'm pretty sure they don't offer hosting/authorizing deployments
from party sources. So they essentially have a monopoly on the ability to use
OSS freely for bulding government websites.

~~~
konklone
> They largely circumvent a lot of the process with help from policy makers at
> the highest levels.

> Even then, I'm pretty sure they don't offer hosting/authorizing deployments
> from party sources. So they essentially have a monopoly on the ability to
> use OSS freely for bulding government websites.

That's not quite the case. Many agencies use a ton of OSS to build websites
and other services.

[https://github.com/cfpb/source-code-policy](https://github.com/cfpb/source-
code-policy) [https://github.com/uscensusbureau/open-source-
policy](https://github.com/uscensusbureau/open-source-policy)
[https://github.com/18F/open-source-policy](https://github.com/18F/open-
source-policy)

The policy links above refer to those teams releasing their own code as open
source, but you will find huge amounts of outside open source code being used
there, and in many other agencies.

The Department of Defense is famous for (in 2003!) making sure their
department knew that open source software was A-okay to use, and people refer
to their amazing FAQ all the time:

[http://dodcio.defense.gov/OpenSourceSoftwareFAQ.aspx](http://dodcio.defense.gov/OpenSourceSoftwareFAQ.aspx)

(I'm an 18F employee.)

~~~
EvanPlaice
Thank you very much for the clarification and links.

------
wlesieutre
There's an amusing episode of Debug where Wil Shipley talks about The Omni
Group's dealings with the Air Force. They'd released OmniWeb as a free
product, and got a call that IIRC went like this:

"We'd like to buy your web browser."

"Well, we actually give that away for free."

"Yeah but we need to buy it."

It's 2 hours long, but worth a listen if you have the time:
[http://www.imore.com/debug-19-wil-shipley-next-delicious-
mon...](http://www.imore.com/debug-19-wil-shipley-next-delicious-monster)

------
naspinski
"I was responsible for the majority of the Air Force's software programs" \-
as a former Computer Systems Officer in the US Air Force at a pretty high
level operations center, this seems pretty hyperbolic, as no person is in this
position. Maybe majority of the software for their area of work.

~~~
KineticLensman
When he said "at a small base outside of the Hanscom U.S. Air Force Base" I
took his responsibility to be that of one of these two bases (or perhaps the
AOC if that was elsewhere). I'd agree that it seems unreasonable for a single
person to have such responsibility, even for their 'ground' software.
Operational battle management software would also probably come from a
different place than the payroll systems, for example. The software in the
various air platforms would likely come from each platform's development
programme.

In the UK and Canadian military, there is no single overall software person,
although there are technical authorities who define standards, etc, that
should be used when software is developed.

FWIW, the UK Ministry of Defence actively supports the use of Commercial Off
the Shelf (COTS) software. This isn't always OSS, but is at least better than
building everything from scratch. A good ongoing example is the use of
'serious games' to support military training, e.g. as image generators inside
simulators or as standalone desktop training packages. There are various COTS
packages that fulfill both of these and other roles.

[EDIT: Typo]

~~~
EvanPlaice
Smells like BS. At most, the person may have acted as an ISSM (Information
Systems Security Manager). Even then, I doubt the 'general purpose' config
used by the majority of the Air Force is created by any one person.

By 'general purpose' I mean the baseline configuration that all non-POR
(Program of Record) systems use. Ie, hardened Windows, MS Office, security
monitoring, CAC authentication, and DOD certificates. There's no chance in
hell that the 'general purpose' configuration will ever be changed to include
OSS because it's intent is to be the 'lowest common demominator' of connected
systems.

This person probably works as a grunt for the AF branch equivalent of G-6.

The COTS systems you're referring to are POR (Program of Record) systems
created and supported by the product development branch. I think for the AF,
it's the Air Materiel Command. They fall outside the scope of general systems
support, therefore they require their own infrastructure for
maintenance/support.

 _Source: I used to support a POR system._

------
lifeisstillgood
I am trying to ride the op n source wave in government in the UK (I want to
make open source software that makes a difference in real lives).

But even though all _new_ software produced for government should be FOSS,
there is little movement because all the small authorities buy licenses at a
speed and cost too low to allow productive n of new software.

A coder must develop the code first then try to sell it. All the risk is on
the coders, but they won't get paid unless they sell licenses - which defeats
the whole point

In short if there are 60 councils in the UK who all want a new system for one
of the 2000'legally obligated services they must provide, they will generally
ask for a license of say 1/10 of the cost of production. If they all do that
then the UK overpays by six fold.

If the first one went FOSS and ponied up the money the UK saves six fold.

However no council will pay for everyone else's free software.

So we keep over paying

"""The Council would like to hear from organisations willing to share
information about their e-recruitment systems to gauge the likely level of
interest in the project from the market. This will enable the Council to gain
a better understanding of the systems available, the ways in which they could
be supplied, an indication of the likely costs, and help determine the most
effective way of packaging and scoping its requirements for any future
procurement opportunity. As such the Council is undertaking this soft market
testing exercise to engage with organisations and share information."""

So I am looking to build some forms of mutualised contract where the first
authority to want finds say nine others and agree upfront payments.

Anyone with ideas pease shout

~~~
phkahler
Nobody wants to go first because they pay for everyone.

One solution is to have everyone agree to put limited resources on a shared
project that they will all use, but this has problems. Many projects fail or
don't go as well as hoped. The distributed nature of it may increase the
possibility of failure and the politics as well.

Another option is to get everyone together and agree on specs. Then outsource
it with the requirement to the provider that the result be open source. After
that first development effort you can all collaborate. This should go over
better in government than in industry because your different groups aren't
competing with each other.

------
craigmcnamara
>> Management: "There's too much risk, even if I can't explain why."

This sums up one of my major frustrations of being a programmer for the Air
Force Research Lab(AFRL).

You have branch and division managers that are completely incompetent trying
to micromanage Software Engineers and Project Managers.'Why' isn't ever a
thing they think about because they're afraid to admit as managers what they
don't know why, and they aren't interested what would make software and
integrations more effective. Within projects there was a great deal of
expertise that was squandered by poor management and management structure
within the Air Force.

I worked there for 5 and a half years, then quit because I couldn't deal with
the frustration of upper level incompetence and blatant fraud waste and abuse.

------
Dog_Vs_Cat
In many agencies today you can implement in open source software almost as
easily and in some cases it is the standard, but here is the reality. OSS
didn't solve most of the problems associated with proprietary vendor software
and in many cases simply shifted the money and problems around to new vendors.

How's that? Yes, now in order to use any OSS you have to have support and
maintenance contracts from OSS vendors like Red Hat, Cloudera, VMware, there
are a bunch of them. Their formula is to offer a productized version of the
community editions that are closely managed and generally running a few
iterations behind the latest dev versions supposedly for security and
stability (though in my experience neither is often the case). At a Marine
Corps shop I was working with last year it was actually more expensive to
bring in OSS and support for this very reason, given they already paid M$oft
licenses. Are they cheaper generally yes, far from free though.

Last thing I will offer up is the security profile of most OSS is atrocious.
We routinely pull dozens of critical CVE's from systems built using OSS for
the simple fact that fixes are difficult to obtain for libraries that other
libraries depend on. Massive problem as evidenced by current state of our
(lack of) cyber defenses. On the whole I support OSS and am a primary
developer of it but you can't just drink the kool-aid.

------
laen
Author is responsible for a majority of the Air Force software programs, but
this experience is his first trial-by-fire. That said, not surprised that his
first approach to the problem was to develop his own hardware and software
standard. It would have been better to identify AOCs with the best practices
and standardize a proven setup.

I admire his advocacy for use of open source software, but his primary pitch
is a reduction of costs. Any organization that switches to open source for the
purpose of promised dramatically reduced operating costs is fooling
themselves. The true benefit of open source is much more than savings.

~~~
tyingq
I think it depends on the situation. In some specific niches, there is an
obvious cost savings. Tomcat, for example, pretty much decimated the old,
highly expensive J2EE app server market.

~~~
__john
Postgres in favor of Oracle is another big one.

~~~
cptskippy
Which is why Oracle started buying up every CRM or CMS system it could get
it's hands on and makes them drop support for anything that isn't Oracle DB or
MS-SQL. Then it cripples the MS-SQL support to the point that you willingly
switch to Oracle DB.

~~~
arethuza
Also things like Hyperion - which certainly used to support SQL Server but has
since moved (or going to move) to Oracle support only.

