Hacker News new | past | comments | ask | show | jobs | submit login
If Software Is Funded from a Public Source, Its Code Should Be Open Source (linuxjournal.com)
1138 points by jrepinc on Feb 4, 2019 | hide | past | favorite | 261 comments

Government scientist here.

IANAL, but my understanding is that all software written by US government civil servants is in the public domain. It cannot be copyrighted or otherwise licensed (although the government can patent things). This does not necessarily mean that it is available to the public, as noted in the other comments; it may be classified, considered For Official Use Only, or restricted from release by ITAR regulations.

Barring any of those cases, you should be able to get a copy by filing a FOIA request. The problem with FOIA requests is that you have to know specifically what to request, usually by name. Obviously, it can be hard to put your finger on exactly what software you are requesting.

The bigger problem, though, is that much of the US government just isn't technically set up to release software (as opposed to documents). Setting up a git repository, in the DoD, is an utter nightmare. Every single patch has to be pushed through a public release process that can take weeks or months and involves review by as many as ten to fifteen different offices, few if any of which even know anything about software. If you're setting up your own server, getting the server itself approved and provisioned can be an exercise in bureaucratic frustration, and can take literally years to accomplish.

Those of us writing software very often want it to get out there. But the structure of the organization makes that excruciatingly difficult.

Mostly sortof true, but there are lots of complications.

I've delved into this. For lots of detail on the current situation, see my paper, "Publicly Releasing Open Source Software Developed for the U.S. Government" by David A. Wheeler, https://www.csiac.org/journal-article/publicly-releasing-ope...

In almost all cases, if a US Federal employee writes software as part of his/her official duties, then there is no copyright IN THE UNITED STATES. There are a few exceptions (e.g., US Post Service Employees). Also, copyright doesn't apply in the US - but outside is a different matter (though enforcing said copyright is more complicated, and I would argue the US shouldn't try). More importantly, there may not be a copyright, but that doesn't guarantee release to the public (and often there isn't). You can ask via a FOIA, but that's no guarantee you'll get it.

HOWEVER: Most software developed by government funds is developed, at least in part, by contractors. And that is a COMPLETELY different circumstance. The details, unsurprisingly, depend on the contract.

If you're interested in the DoD, I suggest going to the MIL-OSS mailing list: https://groups.google.com/forum/#!forum/mil-oss

It's possible to release software to the public. One approach is to get pre-approval where "as long as the changes meet this scope, and follow these rules, it's okay to post the changes." But you have to get local buy-in.

In short: In a lot of places in the US federal government it's definitely difficult to do what in the rest of the world is normal.

In the US, state/local/tribal is completely different, and I definitely don't claim to know the laws of other countries. Others can explain that better.

The most common clause for it contracts that produce software is called “Rights in Data - General” [0] and by default, without alternatives explicitly stated, it allows the government to distribute in an unlimited manner anything created by the contractor.

So the good news is that any government contractor overseeing a contract should be able to release it as OSS, if ve wants. Bad news is that it’s effort and risk to do it correctly.

[0] https://www.law.cornell.edu/cfr/text/48/52.227-14

>Most software developed by government funds is developed, at least in part, by contractors. And that is a COMPLETELY different circumstance. The details, unsurprisingly, depend on the contract. //

Isn't the point that the contact should have a non-negotiable OSS clause of it's using public funds.

My impression, from my seat in Europe, is that USA government is run by rich capitalists that would as soon eat faeces live on TV as suggest private corporations be required to service the public good?

> Isn't the point that the contact should have a non-negotiable OSS clause of it's using public funds.

Okay. Say you're a vendor for student information systems. It handles registration, enrollment, scheduling, attendance, report cards, state and federal reporting, teacher gradebooks, parent portal, etc. You sell your product to one school district with 800 students for an initial fee of $100,000 plus an annual fee of $5 per student and $10,000 for up to X hours of direct support (note: these numbers are not far off the actual costs).

Except now it's open source, and every one of the hundreds of thousands of school districts in the country suddenly has access to your software without charge.

Sure, you can come up with dozens of reasons why that's great for schools, but you're the SIS vendor that just spent a couple million dollars producing the software. Why would you be in a business where the moment you sell your product, you can no longer sell your product? Support? Okay, but you'll immediately be competing with other support vendors, etc.

What does it mean to be funded "with public money". In your example, it sounds like the software was funded with private money then sold as a product to the schools.

The relevant scenario is that the school approached the company and paid them to develop the software.

That's a fair point. I hadn't thought of it that way, it's just what I went to because it's what I'm used to thinking in terms of.

Its still not as clear cut as I make it sound.

Another potential model is for the government to partially fund development. For instance, if the government wants a piece of software that costs $1,000,000 to develop, a company might offer to develop it for them for $500,000 and the rights to resale it to other customers.

This isn't an intractable problem though. You would just need to make a clear delimination between what components the government is paying for, and which ones is coming out of the company's R&D budget. Due to billing, this is already how many of these projects work, but the publicly funded portion is often far less useful without the privately funded portion; and there often is not a good solution to this (think a government funded plugin to Microsoft Excel, where an Excel license costs $5,000)

You should Google redhat.

Edit: or elastic. Or automattic. Or black duck, or canonical, or cloud bees...

> My impression, from my seat in Europe, is that USA government is run by rich capitalists that would as soon eat faeces live on TV as suggest private corporations be required to service the public good?

Not quite true. Rich capitalists largely have better things to do than bother with the details of running government.

But it is, to a large extent, run by people under the influence of (whether directly seeking money, or just manipulated by paid propaganda) rich capitalists who would rather engage in public coprophagia than have such requirements imposed.

They don't care about what the government does, subject to the bounds of "don't touch my money".

The ones who do care more than that tend to be absolutely awful.

Like Tesla? Or Patagonia, or REI, or allbirds, or bullfrog power, or seventh generation?

Like Mercer, who buys up local news outlets to push his propaganda.

This DoD CyberSecurity Policy Chart in the sidebar is epic: https://dodiac.dtic.mil/dod-cybersecurity-policy-chart/

Clearly, they don't want new ideas.

I think they just try to idiot proof too much, which ends up a bureaucratic nightmare.

I used to work in a state government job as a student intern, and they said they wanted an internal directory for some various tasks. You can think of it like an inventory manager sort of website, purely on our intranet.

The mad lad my lead was, said we would need a database for retaining some of the basic data we needed to keep track of. Nothing all that special, just a few tables, and by nature nothing sensitive would be tracked.

This got rejected immediately, and our IT literally cited "no database installations are allowed per security team."

Needless to say 4 months later, after some back and forth, he eventually said "whatever, we'll just store everything to files then" and that's how he came up with the world's slowest inventory manager - I suspect just the way they like it.

I used to be a contractor and worked on a web application distributed across multiple web servers. We needed a state server at the time to hold some cached information and share it among the servers. Now there was a rule that said a web server was not directly allowed to communicate with the database. Okay I get it, that makes sense.

The state server we were going to use to store some application metadata had the word "database" in the product name. For that reason alone we got flat out rejected.

I said to them, "So it's OK to store all the stuff we need in memory and on the file system on the servers, just not stored in the state server."

Their response was, "Yes, because it's not in a database."

We finally switched to another product that did not have the word database in the name, and they were completely fine with it.

I'm having to switch CentOS servers over to RHEL servers because... support(?)

Some state work got in trouble a few years back because they'd updated guidelines that no applications could be on the same machine as a database server. Enforced separation, for security. Having enough staff and management to distribute that message to people dealing with the apps and servers didn't seem to be a requirement, however. And... when I was given this mandate, I had to ask for another server, then told there weren't any available, and I'd have to wait for several weeks, and I probably should shut down the current service (which had run for 3 years this way, well before the guidelines were in place).

Navigating govt IT stuff is crazy. About half the people I've encountered are between competent and really talented, and the other half don't understand the basics.

FWIW, if the web server can't communicate 'directly' with a database... what process do you use to communicate between them? Some sort of proxy? Or something else?

There's an infamous story at my shop. Twenty-some years ago, we used a software tool that was an important part of our mission, and it only ran on Silicon Graphics boxes. We needed to buy a new SGI machine, and SGI had started shipping all of their machines with built in monitor cameras, the first company to do so.

Digital cameras were at that time strictly prohibited from even being on the premises.

So my friend goes ahead and buys the machine he needs, on the assumption that if he doesn't everybody is out of a job, and he can always disable the camera. The machine arrives, he sets it up, and calls security asking for whatever official process they use for disabling webcams. Epoxying over the lens or something, was what he figured.

At first they are very confused. It's like they can't parse the words he is telling them. It's as if he was trying to tell them there were six legged iguanas with Russian flag patches on their backs running around the server room.

So he finally gets them to acknowledge that he is, in fact, speaking English in sentences that adhere to the rules of grammar, at which point they have to take his question seriously.

So they ask: “did you procure these computers through <our base's> government procurement office.” To which he says, yes, of course, there's no other way to do it.

They think for a minute and reply: “by policy, you cannot purchase a digital camera through the procurement office. Therefore, your computers do not have digital cameras in them.”

> They think for a minute and reply: “by policy, you cannot purchase a digital camera through the procurement office. Therefore, your computers do not have digital cameras in them.”

They were being intentionally daft. The person who told him that almost certainly did so with a wink and a nod, and saved your protagonist a lot of pain.

And probably were shocked at getting the call for a very smart reason - this is a classic kitbag question.

The web server had to communicate through an application server. It wasn't a huge deal, until we got into the discussion about what a database meant to them. That was painful.

still having trouble parsing that (not trying to be dumb here, really!) in java, my java app might talk to a connection pool, which then talks the the database. that connection pool could, I'd guess, be in a separate 'application server', no?

my 'database and app server separation' thing - never quite got a straight answer on 'why', other than 'security'. except... adding more systems, with more logins, more access rules, more stuff to update/patch/test leads to... more potential for security failure (in my experience).

> This got rejected immediately, and our IT literally cited "no database installations are allowed per security team."

This is common enough across a variety of organizations (I've certainly run into the same issue at several large-ish companies) and isn't necessarily unreasonable... if IT also provides a process for getting what you need with their oversight/help (bonus if some part of the IT team actually has skills to do audits rather than cargo culting off a checklist).

> This is common enough across a variety of organizations... and isn't necessarily unreasonable

So, how are applications supposed to store/retrieve data?

I guess they use a tuple store backend.

Shh... Don't tell them that stuff like SQLite _is_ a flat-file database.

Besides, if you code your own database, at one point from a file to postgres, is it going to get rejected ?

"Is this system a database?"

"No sir! It's just a base of data."

<checks spec book> "Very well; carry on then..."

I've never understood this argument; sqlite specifies a file format, but that's hardly the only requirement.

By this argument, any database with a public file spec is a flat-file database.

I guess in their defense, databases were pretty famous for being a huge attack surface for many years.

mis or poorly configured, yes. "sa" and no password on MSSQL, with direct connection to public internet - saw those for years. PostgreSQL/MySQL configured to only listen to localhost connections, and properly escaped/prepared queries running to them, with "minimally needed security access" credentials - haven't seen many problems with those (that were the fault of the database).

hand rolled file based storage solutions will likely be less secure than battle tested dbs.

Flat files are a perfectly acceptable way of storing and retrieving data. Never start a greenfield software project with "ok, first we're going to need a database." At least defer it until you know it's the best way forward.

This site, Hacker News, uses flat files for its database.

Had the same problem at a big corp once.

They got MS Office approved for all PCs, so we ended up building all based on Excel, Word and Access.

Contractors have their own hassles when it comes to open source. Pre-openstack, there was a small contracting company called Anso Labs that consisted of a few of us that were creating a private cloud for NASA. After a few months of banging our head against Eucalyptus (java based open-core private cloud software), we decided to write our own version over a weekend and call it Nova. We convinced the civil-servants in charge to allow us to use our new thing, which we continued to improve at NASA. This project eventually became the compute platform for openstack.

There were multiple weeks of meetings with lawyers (including discussions of whether we needed a space act agreement[1]) to figure out how to actually open source the modifications we made on NASA's behalf. Ultimately we ended up having to assign all of the nova copyrights to the government[2] so that they could open source the modifications. I didn't even think that the government could own copyright but apparently it can[3].

[1]: https://www.nasa.gov/partnerships/about.html [2]: https://github.com/openstack/nova/commit/c88d1f033bd600e855c... [3]: https://www.usa.gov/government-works/

With regard to the DoD the CIO released some good guidance on OSS: https://dodcio.defense.gov/Open-Source-Software-FAQ/

Here is an excerpt that talks about licensing:

>Q: Can government employees develop software and release it under an open source license?

>Not under typical open source software licenses based on copyright, but there is an alternative with the same practical effect.

>Software developed by US federal government employees (including military personnel) as part of their official duties is not subject to copyright protection and is considered “public domain” (see 17 USC § 105). Public domain software can be used by anyone for any purpose, and cannot be released under a copyright license (including typical open source software licenses).

>However, software written entirely by federal government employees as part of their official duties can be released as “public domain” software. This is not under a copyright license, it is absence of a license. By some definitions this is technically not an open source license (because no license is needed), but “public domain” software can be legally used, modified, and combined with other software without restriction. Thus, “public domain” software provides recipients all of the rights that open source software must provide.

I am working on this problem and contributing to https://code.mil. This project explains to DoD managers /how/ to set up an open source policy. It's going to be a long process, but we can do this!

Awesome awesome awesome to see this. I agree, the process will be arduous but it is incredibly achievable!!

please send me an email.

> The bigger problem, though, is that much of the US government just isn't technically set up to release software (as opposed to documents). Setting up a git repository, in the DoD, is an utter nightmare. Every single patch has to be pushed through a public release process that can take weeks or months and involves review by as many as ten to fifteen different offices, few if any of which even know anything about software. If you're setting up your own server, getting the server itself approved and provisioned can be an exercise in bureaucratic frustration, and can take literally years to accomplish.

Please excuse what I suspect is a naïve question: why can't they just throw the code into a zip file, and send out the file the same way they would other documents? Digitally or on a physical thumb drive.

(The FOIA requester could then put the code on Github or similar.)

When we do release code, that's more or less how it works.

The issue is that, basically 100% of the time, anything being released to the public has to go through a release process. If you produced it as an employee of one agency (eg the Air Force), but if was paid for by a different agency (eg DARPA), then it also needs to go through a similar release process on the sponsoring agency's side.

Typically, this has to happen for every single “release”. In other words, for the original software, again for every subsequent patch, for the documentation, for every correction or update to the documentation, etc. If you want to give a presentation about it your slides need to be released. If you want to write a paper on it that needs to be released as well.

Evey time through the release process takes two to six weeks, depending.

This works OK if you are releasing documents. Once a document is written, it usually doesn't need hundreds or thousands of updates over it's lifetime. A journal paper gets released once, then it's done. But the same process is what we currently use for software, and software is just a very different thing than a document is.

Imagine what would happen to the release cycle of a small OSS project if it took six weeks for github to get around to posting your update, and if github management rejected about half of them for reasons that were mostly inscrutible to you.

It doesn't strike me as a huge loss to society if source code updates are only released, say, once per year. It's not ideal but it's not terrible either.

(Of course, if the updates aren't happening at all because of opaque internal procedures, that's a different story.)

What usually happens is that it doesn't get released at all. It isn't worth the effort.

I've got tons of code I've written over the years that, if I could just throw it onto github, I would. But since it would take me six weeks of pain, I don't. Some of that code I'm kind of proud of, or was at the time I wrote it, anyway. I wrote an rviz app a decade before ROS took off.

Would there be some way of just automatically releasing everything in a git repo after X days (30? 60? 90?) unless some action is taken? Commit/push like normal, and some other process pushes to a public repo after X days, but anyone can review and block in the meantime for security reasons when needed?

No. “Automatic release” is a contradiction in terms.

It could happen, but it would take something like the Secretary of Defense ordering it. Or a White House directive. The folks at my level, or at any level I can actually talk to, don't have the authority. The thing is, they aren't making these policies and procedures up just to be dense. There are real, hard and fast laws and department-wide policies that require the system to look the way it does.

But if someone files a Freedom of Information requests, that forces the process into motion, right?

(If there were more people filing request, I wonder if the internal procedure would be made less painful out of necessity...)

And you would get your information request rejected due to unreasonable efforts required to satisfy.

FOI is meant for information that can legally be published with reasonable effort, but not yet available.

I was told by one government contractor that it takes forever to review the code and pull out any hard coded passwords.

That can be corrected because hard coding passwords fail the government secure coding policies. So if the contractor hard coded then they failed the quality requirement. So it gets them into a fun for the public catch22.

The most effective method is to have continuous development through an open source structure so any failures where someone hard codes sensitive info in source get found quickly.

It depends on your definition of "civil servant". DoE national labs are run by contractors, such as the University of California, who have their own tech transfer policies and offices. As a result, I can't legally use most of the software I wrote as an NIH-funded postdoc. (One of the many reasons I don't work there any more.)

Contractors are not civil servants.

It is completely true that contractor-written software is legally different and has its own release policies. For one thing, it can be copyrighted. Again, IANAL, but I assume that there would be stipulations written into the contract governing IP.

I'm joining the Air Force as a Civilian with the explicit purpose of solving this. In fact we've made a lot of progress with Continuous ATO and some other efforts to get faster iteration, deployment, sharing CI/CD etc... processes in place.

It's going to take time but we're on the right track. Send me a message if you want to be involved in this culture change.

Very cool, I recall seeing a lightning talk at KubeCon on this very thing: https://schd.ws/hosted_files/kccna18/45/NA%20KubeCon%202018%...

Please join the mil-oss mailing list. I think you'll be glad you did.

Is the one you're referencing the google groups bbs group one? [1] Doesn't seem particularly active.


On the other hand, some branches like the Department of Energy are making a big push for open source. You can see how many projects Los Alamos has, for example:




Keep in mind, Los Alamos is one of the big labs doing nuclear work for the DOE, so obviously a lot of what they do is classified, but of the stuff that isn't they're actually doing a pretty impressive job of open sourcing it.

Here's a good counter example:


Back when I did work on hydraulic modeling, HEC-RAS was the gold standard for open channel flow. It is only available as a pre-compiled Windows executable.

I had a ton of parameterized simulations to run and would have loved to have gotten my hands on the source code in order to run it on EC2 Linux spot instances and to mechanize the parameter inputs by cutting the GUI input forms out.

Government development can be bad, but it's nothing compared with most other large verticals (bio, insurance, finance, air lines...). None of the absolute statements here are remotely close to universally applicable (check out forge, 18f, national labs, the recent NSA ghidra announcement, etc).

> Setting up a git repository

Git != GitHub

> Every single patch has to be pushed through a public release process

Rebase is your friend.

I'm speaking to my experience in the DoD. Other agencies have different policies and procedures, and operate under different motivations and legal frameworks.

Lawyers for at least some multinationals get hives when you talk about public domain software. I've had to switch libraries or participate in cleanrooming a single PD function a couple times to get them to calm down.

Apparently in Europe and some other places it's not so easy to disavow all rights to a bit of software. BSD- or MIT-style licenses are much more comforting.

This is why the CC0 is way more useful than just putting "This code is in the public domain" at the top of every header file.

I used to work for a National Laboratory that was privatized, and the 1980 Bayhd-Dole [1] act made any intellectual property developed on a government contract on what was previously a public national lab now the private ownership of the company that took over the national lab.

All the science done at that national lab for less money before it was privatized was after privatization private property, and before privatization more science was done for less money.

The reason the costs became higher was that a private company is not tax excempt and the government guarantees a profit for the contract.

In effect this is a no cost private theft of government funded research, and I find it fishy that almost all the national labs were made private in the decades after the act. The only upside seems to be that the private company take any blame for a scandal.

[1] https://www.mofo https://www.mofo.com/resources/news/the-governments-patent-p...

Anything written by federal contractors, and anything written with state funds are not in the public domain. National labs flat out refuse to release some of their source code.

And there's the way around public domain, open data, and free citizen access and usage requirements.

The contractors don't have to be.

It's the same reason why most people stay out of federal jobs - you can't make more than the president. But if you're a consultant working for the gov't, you can get paid $$$$$.

If you can, please send me an email.

Most posts on this thread are focusing on exceptional cases where there are obvious reasons not to open the source, such as defense, but huge amounts of publicly-funded code is locked up for no reason whatsoever. Let's discuss some use cases that make more sense:

1. healthcare.gov

2. Stoplights/traffic control cameras.

3. Voting machines.

4. Electrical grid.

5. IRS.

Are all of these publicly funded?

There are companies (including Siemens, for example) which build traffic lights and sell the control software and algorithms for eg car and pedestrian detection. See https://www.speedcamerasuk.com/speed-camera-types.htm

In a lot of places, the government decides that a service is required and companies bid for the construction rights. Just because public money was used to buy a service doesn't make the IP the property of the public.

And in response to a sibling comment, no, the government does not always pay for code development. The company decides they have some niche IP (eg car detection) and start selling speed cameras. They then bid for government contracts.

You could argue that there is an onus on government to use open hardware and software where possible. It often costs them less money, for a start. I suspect its easier to run an open source focused company and aim to be a major government contractor than it is to persuade the government to buy open source.

> Just because public money was used to buy a service doesn't make the IP the property of the public.

No, but it could. Might raise costs initially, but the idea is that enforcing such a spending policy would lower costs and reduce duplication long term. Not to mention that having law and beurocracy enforced by closed source systems is a major threat to freedom and democracy, of course.

No, because no company in their right mind will give up its IP like this. You will have to purchase the company, or minimally, purchase an exclusive license to that technology at 10x or even 20x the price.

1. https://www.healthcare.gov/developers/

2, 3, and 4 are made by private companies. Not government.

5. Probably a good idea.

> 2, 3, and 4 are made by private companies. Not government.

But government pays them to develop the code. For many such companies government is the only customer (whom else are you going to sell the traffic light software to?).

Another government.

The traffic lights in Chicago are likely made by a different company than the traffic lights in New York City (I dunno for sure... but it'd be a separate arrangement nonetheless). Those are two grossly different customers.

USA has a federal society: that means we have mini-governments across the country with their own laws and regulations. In an extreme case, an Home-owners association can own traffic lights and operate roads. (My own HOA for example owns the traffic lights in my area, and hires cops from the county to patrol our roads. They're county-cops contracted out to the Homeowner's Association for security in the neighborhood)


Same thing with voting booths, and other services. They're made by private companies in the USA, to serve the many different local governments that exist.

And local governments interact with local neighborhoods / homeowners associations, depending on local regulations of course.

So question: why should we as humans care if a company isn't able to sell their software to another government?

The underlying argument you're making comes down to this idea that capitalist competition is the most efficient way to produce results, but it's obviously incorrect in this case. Are we really arguing that having a bunch of different companies developing the same software a bunch of times for different governments is the most efficient way to do this?

> But government pays them to develop the code.

That's an assumption that isn't always true.

The company could have developed them with their own funds, then sell the product to the government.

> 5. Probably a good idea.

You probably wouldn't want their fraud detection/audit selection software published. No matter how good it is releasing it gives people a chance to better avoid detection.

I guess that's true.

I'm mostly thinking of it from a law-abiding citizen perspective. Maybe make some basic tests available to more easily show that I'm not making the stupid mistakes (an extra "0" somewhere or missing a digit somewhere else).

Also, anything that would more easily interface with the IRS's online submission system would be great.

That kind of depends. The software could easily be published without publishing the rules/models that are being run. While knowing what sort of parameters are available to the rules/models might make it easier to identify attacks that aren't going to be found, that's not a given.

Man it would be pretty amazing to be able to submit a Pull Request to the IRS.

Matt Cutts if you're reading this please help!

On another note, I'm not sure I agree on the electrical grid or traffic controls being open sourced ... could imagine some nefarious things happening as a result of that

> On another note, I'm not sure I agree on the electrical grid or traffic controls being open sourced ... could imagine some nefarious things happening as a result of that

Any more nefarious things that can be done with your Linux OS server talking to your Open Source Chromium / Firefox web browser?

Open Source doesn't necessarily mean less secure.

I'm not sure these things are equivalent, many people use a browser and Linux, not many people need to use electrical grid software.

However, I would imagine it would still be beneficial for open analysis though. I'm not sure security through obscurity is a good option when there are motivated nefarious actors involved. But that's more of a "gut feel" than based on any evidence

Agreed that it does not necessarily mean less secure ... see my comment below

The french equivalent of the IRS did open up their software: https://github.com/GouvernementFR/calculette-impots-m-source...

Everybody got very enthousiastic at the begining, then realized:

- it was written in a niche proprietary language: M

- it was provided with almost zero doc or comments

- the process of getting the code was tedious, and the law changes every year

- taxes are complicated

And so nobody did anything with it ever since.

Huh, usually I've seen "M" used to refer to MUMPS. But this appears to be something else -- I guess something proprietary, like you said. I have to wonder where one would look it up, as I imagine searching for information on the M programming language would mostly turn up information about MUMPS...

> Huh, usually I've seen "M" used to refer to MUMPS

M is also the name of the built in language in Microsoft Power Query.

Or at the very least our entire national tax code should be downloadable in machine-readable form.

It should just be a giant function:

    def foo(**kwargs):
        <insert code here>
        return how_much_you_owe

It really should be a provably unambiguous ruleset that would take in the person's circumstances and numbers and spit out a number, yes. I understand that your post is a joke, but that'd be an _awesome_ thing to have for everyone, especially if coupled with some adversarial tech to find and close loopholes. Which is, ironically, why this isn't going to happen.

I've always assumed that Intuit lobbying alone will be sufficient to kill this idea.

I think Intuit is just a scapegoat in this case. Incomprehensible tax code is a rich environment for corruption.

Totally true, but I've certainly seen claims that Intuit actually lobbies against simplifying the tax code.

This might work:

  x = gross_income / median_income;
  if( x <= 1 )
    return 0;
  if( x <= 50 )
    y = ( x * Math.ln( x ) - x + 1 ) / Math.ln( 2500 );
    y = x - 0.5 * Math.ln( 50 ) * Math.li( x ) + 4.862;
  return y * median_income;
Math.ln() is the log function for base e, and Math.li() is the logarithmic integral function.

The constants are based on the income at 50 times the median income having a marginal tax rate of 50%, and the median income having a marginal tax rate of 0%. Marginal tax rates above 50% asymptotically approach 100%. There are no loopholes or exemptions.

If the median income is $32k, and you earned $31k, your tax is zero. Your only obligation is to report your gross income honestly, so that the median income is calculated accurately for next year. If your income is $96k, your x = 3.0, your y = 0.1656, so your tax is $15.9k . If your income is $1.6M, your x = 50, your y = 18.74, so your tax is $600k . If your income is $32M, your x = 1000, your y = 657.5, and your tax = $21M.

The only effective ways to reduce your tax would be to reduce your income (as reported) or raise the nationwide median income, such as by paying your lowest-paid workers more in the previous year.

Seems fair to me.

You're solving the wrong problem. Calculating tax owed is the easy part. Figuring out your income is the challenge: tracking all your business expenses, depreciation, credits, exemptions, etc.

Deduct the labor cost of salary and wages only, and the distributions to the owners. Those are someone else's income. If it isn't someone else's income, for them to pay tax on, you pay tax on it. No loopholes or exemptions, period. The only way for a dollar to not be your income is if you gave it to someone else, to be their income.

If you add up all the incomes of all the tax-paying entities, it should equal the money supply multiplied by the velocity, minus the unreported income economy.

There's one fly in this socialist ointment: all the rich people will immediately bail, and go live elsewhere.

Ah. Well, that's fine. They can go somewhere else and renounce citizenship, then huff and puff when they start having difficulties getting the money out of the businesses and property they left behind, and getting the knobbly, nail-studded end of the mercantilists' stick.

Your argument rests on the presumption that the economy needs super-rich people to be performant, when corporations and governments have already solved the problem of collaborative concentration of capital. If they leave, and the economy somehow fails to collapse, they have shown the lie behind rich folks as "job creators".

Economies are run by the people who do the work. Sitting on ass and writing checks serves to weed out some of the worst ideas, and shuffling financial instruments serves to lubricate the gears of the economic machine, but should the rich ever go on strike, they will immediately lose all their leverage as the economy restructures itself to work without them. At worst, they can just get replaced by newly-minted rich people.

That's how you get Venezuela though. Good luck with that.

You get Venezuela with bad government. And they bought theirs with state-owned (nationalized in 1976) petroleum money, not with progressive income taxes. Progressive taxation is how you get Europe. Occasionally, someone like Gerard Depardieu flees the top tax rate in France, the world's top taxer, but it still manages to be a G7 economy, somehow.

And, contrariwise, when a nation coddles its billionaires, and lets them do whatever they want, that's how you get the current US government. Nobody's starving, though. Not yet. We still might check that off with another government shutdown and another trade war escalation and another diplomatic insult or provocation.

You get Venezuela by committing to give people somebody else's money rather than create the environment in which they can make their own.

Venezuela has complex cause. You aren't doing anyone any favors by simplistically reducing it to your favorite contributing factor, nor are you arguing honestly when you conflate progressive income taxation with socialism, or in this specific instance, Chavismo--which is rooted in socialism, with a lot of extra features, including populism.

Committing to give people someone else's money is every government expenditure that is paid for by taxation. It's the welfare state. It's the military-industrial complex. It's infrastructure projects. It's corruption. It's operating funds for regulatory agencies. It's foreign adventurism. Everything. Most governments have budgets that are funded entirely by "somebody else's money". You have made several unspoken assumptions in this discussion, and I think it may be best for you to make them explicit before continuing.

People aren't leaving Venezuela because taxes are high. And Venezuela's problems aren't a result of rich folks fleeing to Miami and Madrid. They are all leaving--the non-government rich, the middle-class, and the poor--because price-fixing creates shortages, and there is no such thing as a free lunch. If one pays for the party with oil money, one also has to turn off the music when the international commodity price for Venezuelan crude oil goes down. But they didn't turn it off; they turned it up.

Can you provide any details on Venezuela's progressive income tax? All I could find was a flat 34% tax on income for nonresidents, and that the lowest rate for residents might be 6%. Which Venezuelans left specifically because they didn't want to pay 34%?

It shouldn't matter where they live if they're still making their money here. And if they're not making any money then they're not paying taxes under this scheme anyway.

> provably unambiguous ruleset

I think that statement is against the very idea of tax law :)

By hiding the code, it makes security through obscurity far more possible. If you force the code to be public, very poor security will be discovered far faster and be more of an issue to the public.

While this will also make abusing security flaws easier, I think there is also a real benefit to forcing it to be public that could potentially outweigh the risks of doing such.

I agree with this in principle (in fact I had a long argument with the former CTO of Citi about this during the heartbleed fiasco) but I also worry that if noone is willing to put in the effort to fix flaws or they are not reported properly then fixing them could go un-funded while flaws were easier to discover.

Maybe the answer is very good logging of anyone who has cloned the repos etc. but right now when we have a government that uses whether or not they're going to fund important parts of our infrastructure (like Air Traffic Controllers) as a bargaining chip I have some skepticism around them being willing to fund ongoing maintenance of some of these products.

Despite the fact that things being in the open SHOULD curb this from happening I've read enough legislation (yes, I actually do like to read legislation) to know that that probably is not true when it comes to the government

An alternative would be to rebuild these from scratch, not using any existing code, but start (and always continue) in the open.

I am always told that if security is done right you can release the code. Makes sense to me. If you don’t release code it’s much more tempting to do some quick hacks that rely on not releasing the code but aren’t really secure.

Agreed, safety and essential infrastructure should probably not be open source. That said, the main points of infrastructure access and configuration are probably larger issues at hand over the actual systems source itself.

As to TFA, I have mixed feelings. I'm actually more concerned about research and related patents and data surrounding publicly funded study. I do feel that Open-Source should be given some level of preference, but not that all publicly funded software should be open source itself. It's a blurry line imho.

Disagreed, safety and essential infrastructure should definitely be open source. Right now your typical infrastructure project is about as leaky as they come compared to a typical web app. Exposing that stuff to broad daylight will certainly help to harden it.

I wish I had as much faith in the government as you ... then again as a resident of the netherlands I suppose you might have reason to have more faith than I do in the US?

In principle I completely agree that things being in the open should help to harden them, but especially in the US where funding is often used as a bargaining chip and bureaucracy is rampant I get a bit nervous

Then you're essentially ceding the ground to the bad guys.

And how much of that is already using open-source and it's the configuration that's leaky, and not the software itself?

How many poorly maintained/updated/configured Linux hosts are out there? Same for Windows, etc... in the end, the software can be the greatest, but if the host's configuration is poorly setup/managed, what does it matter.

Except how long does it take to fix a vulnerability in government? If an exploit is discovered, consequences are immediate and the patch to fix it might not be.

Could you be more specific about what you can imagine? People looking at the code and finding exploits that way?

There is more than one country in the world. Let's say country A is paying for some software and open source it, the rest of the world can get it. That may be fair if everyone is doing the same and contributing fairly (hard to assess that), but if countries A,B,C are the only ones contributing (paying) and everyone else just use, then it is not fair or sensible.

More than that, there is no "public paid", just "tax paid". That makes the tax payers entitled to some of that code, not someone else. With taxes paid on so many layers (federation, state, county, city) the ownership and entitlement is also layered: if a small town pays for some software, should the entire country get it? Why?

Country A isn't losing anything by allowing others to use it. Even if no one contributes back developers can spend that time doing other valuable work, rather then re-invent the wheel in every country around the world.

Country A has to pay more for development because the company won't be payed again for this software by countries B and C.

So to start the FOSS process there is a need to somehow have for each niche a GPL software base that cuts costs by a lot.

Countries a, b, c already can reuse their code/don’t duplicate effort. Others will join for sure. They’ll also get free code/bug fixes from people who want to contribute. People do it for many reasons, just listing you’re contributor to gov project can look nice on cv. No tax money is paid for that. If other countries want to use but not contribute back - so what? I don’t think it ever bothered anybody that their open source lib/whatever was used by people who don’t contribute back to it or by people who themselves don’t have open source projects.

> That may be fair if everyone is doing the same and contributing fairly (hard to assess that), but if countries A,B,C are the only ones contributing (paying) and everyone else just use, then it is not fair or sensible.

Isn't this the purpose of the GPL? "Sure, you can use our code, but any additions need to be reusable as well, so everyone is contributing fairly."

Unless, of course, countries B,C,D don't change anything, in which case, no actual loss to country A.

Hard to sue someone in another country.

> if countries A,B,C are the only ones contributing (paying) and everyone else just use, then it is not fair or sensible.

There's Open Source and then there's the Licensing of Open Source. Just because the source is available does not automatically mean it's free to use under any circumstance.

Licensing is a separate and complicated issue.

Wealth is not a zero sum game.

In a finite system, it is.

..which we don't have and no one, ever, has lived in.

>2. Stoplights/traffic control cameras.

If you think this should be open source, do you then carry that logic to demand that all government vehicles run on open source software as well?

In my local community, the government does not own the traffic control cameras. They contract out agreements to maintain cameras at intersections that meet constraints.

I'm not necessarily against the idea of the government being restricted to using 100% open source hardware, but that's super radically different than "software the government owns should be open source".

An unpopular opinion I guess, but the article is saying if you receive government funding, it should be open source.

That's more than just government employees making code. It's also any company, individual or agency that even accepts $1 of public grant money or any external contracts.

We're talking about hundreds of thousands of coders who's daily job is partially funded by the government having to push their code public.

I worked for an ad agency for a time that created websites and sent emails for tourism. The tourism website needs to be open source? The program that sends out discount coupons to public attractions and parks? Each HTML email I code needs to be submitted to repository somewhere?

I don't think people realize just how many millions of lines of code per day we're talking about here. Most of them being inconsequential things. 99.999999% of that being noise.

You've just jacked up the cost of working with the government. More time spent coding. More documentation for things that shouldn't need it. More time spent with compliance issues. Companies need to charge more. More tax dollars wasted.

Yes, you should need to make it open source. There's no reason to have a different backend for every town in the USA; less work for you to do and less money to be spent redundantly.

> the article is saying if you receive government funding, it should be open source.

Despite the headline, the article actually doesn't state that. It says thing like, "any government code produced with public money", "all software developed for the government", and "any source code written by the government must be released".

No, you just reduced the costs of everything. ONLY the code the government paid for development is being discussed. If 1% of the development was paid for by direction of the government, then at most we're talking 1%.

The bigger problem today is that the government re-builds things 1000 times.

I am an open source advocate, but I don't really agree here. Should the IRS, NSA, CIA, and FBI code all be open source? Here are some cases where closed source software, with some security by obscurity could be helpful. Another example would be some software around designing nuclear systems.

But as an example maybe a software funded by the national weather service to run simulations, that aught to be open source (perhaps something like that exists and is open source, that's not my area).

But a counter argument to mine could be that some software that is security focused might be more secure if it was open source. I think this is an area with a lot of nuance, and absolute statements are hard to make.

> Should the IRS, NSA, CIA, and FBI code all be open source?

Yes. Open source, but secret/classified (as necessary, IRS software should be open-source, period). This way the software will be a useful learning/historical resource when it's declassified in the future.

Also, electronic voting. Not that electronic voting booths will ever be a good idea, but forcing them to open source their stuff will hopefully deter people from even trying (and if not it will make it even more blatantly visible that these things are terrible).

It's just like the biggest argument for open source in science: it is required for proper accountability.

Most of the code in electronic voting is owned by the vendors though so by the same standard would we have to open source any 3rd party software the govt aquires?

I don't quite follow your line of thinking here:

- currently most electronic voting software is owned by the vendors

- electronic voting should really not be trusted, period, but definitely not be allowed to be closed source

- therefore, we should ban any electronic voting software not made on open source software (arguably even hardware)

- therefore, I am implying vendors of electronic voting software should only be allowed to use a fully open source software stack in their final product

I do not see the jump from this reasoning to any software used by the government must be open source (if I understand you correctly).

That's a lot more involved and domain-specific than the question of whether or not government-owned code should be open source.

The person you're responding to is just making the positive observation that, since electronic voting software is licensed - not produced - by the government, you can't mandate it be open sourced via e.g. a FOIA request.

You are making a normative declaration that it should be open sourced because it's important and intrinsically untrusted software, and as a consequence it either will be or electronic voting won't be possible. These are separate things. You're talking about what ought to be, the other commenter is talking about what (currently) is.

Right, I see where GP and I were talking past each other now now. Thank you for clarifying!

Having said that, I think there is a more general point to be made from bringing up electronic voting in the discussion of open source + government funded software development.

Licensed products are an inevitable loophole to be exploited unless we stipulate rules for use-cases like this. Inevitable because some products will always be licensed, and mandating that everything the government uses must be open source is... well, actually, that sounds great, but even if such a policy were implemented that would be a slow, tough transition to make.

I think it would have to be a requirement of any money the government spends on code. Otherwise, they will easily defeat the requirement to make it public by passing it through a private company.

> Not that electronic voting booths will ever be a good idea

Why not?

There are some things the IRS shouldn't release like their fraud detection or anything related to deciding who should get audited. Releasing that risks big bad actors working around the limitations of whatever algorithms it contains.

> There are some things the IRS shouldn't release like their fraud detection or anything related to deciding who should get audited

That's perhaps a good reason for that part of the software to not be published and/or FOIA disclosable, but that's orthogonal to how it is licensed. If it's an original government work now it would be public domain (which is more open than open source), but still potentially confidential and nondisclosable.

While I appreciate the distinction between license and classification, does it really matter if a piece of software is "in the public domain" if its very existence and content is classified and non-disclosable? It may as well not exist as far as open-source or public domain is concerned.

> While I appreciate the distinction between license and classification, does it really matter if a piece of software is "in the public domain" if its very existence and content is classified and non-disclosable?

If the entire work is, less so than if it was disclosable (although there are laws governing the government use of copyright protected work, and arguably there is a benefit to the public interest if the vendor has provided and open source license even if the work is not publicly disclosable.)

But practically, classification and, perhaps even moreso, the other exceptions to FOIA would often apply to limited portions of software systems rather than whole systems, as is often the case with other materials covered by FOIA.

I see. Well a redacted code base is certainly better than none at all, so I guess I agree

Yes, open source IRS software. Tax breaks for bug fixes!

What do I get for sending in a malformed e-file that breaks the mainframe? There's some REALLY old code in the IRS processing systems and it would surprise me if any of it stood up to modern security designs. Also, you don't want to reveal how you determine who to audit.

> who to audit.

Doesn't have to be complicated, pull 10% of the "simple" forms, 90% of complicated.

Simple forms would be the 1040EZ, 1040s w/ std deductions & no other "business" forms.

I highly doubt that the real code is random in that way. I'm sure there's some random aspect, but not all. If you're committing tax fraud, it's very valuable to know how to avoid tripping up the "audit this account" triggers.

This is assuming "who to audit" is even a software process or completely a software process, but fine, don't open source that if necessary. I really just want the IRS software that I personally use to be open source.

> Should the IRS, NSA, CIA, and FBI code all be open source?

No, works made by or under hire for the federal government should be public domain, from a copyright standpoint, not exclusively owned but licensed under an open source license.

But on the other hand, I think your real question has nothing to do with ownership or licensing, but whether or not they should be publicly disclosed. In many cases the answer is “no”, but lots of government work that is in the public domain from a copyright perspective is also not subject to unrestricted publicly disclosure (and may also be classified.)

But arguably the licensing can work the same way as disclosure; presumptively, all government software should be open source, just as presumptively all government records are publicly disclosable; there may be limited defined exceptions for the former as there are for the latter, and decisions to treat software as within an exception should be reviewable by courts just as witholding material from FOIA disclosure is.

And private software that only needs a 3-5% modification for government usage? Should that be required to open-source the entire software? Should the government have to pay tens of millions and years of waiting to re-create said software?

> And private software that only needs a 3-5% modification for government usage? Should that be required to open-source the entire software?

I think there is a good argument for anything where the government is acquiring software rather than paying for service that the acquired software (even if it zero percent modified) should be under a permissive open source license (or even acquired into the public domain), and the source code should be a disclosable public record except to the extent it would be covered by privacy, security, or other existing exceptions to public records laws.

So, you'd rather spend millions and years of time extra than to use closed source software?

> So, you'd rather spend millions and years of time extra than to use closed source software?

No, I'd rather save millions and years of time than use closed source software (if it's zero modification COTS, the extra cost comes in organizing operations around software limitations rather than business driving tools, if it's 3-5% modification at acquisition MOTS, then the additional cost is the continued modifications necessary as government needs evolve differently than private needs with the original vendor able to charge want amounts to monopoly rents because there are no substitute providers because the vendor owns the code, or you fail back to business organized around software rather than vice versa again.)

I've actually spent quite a while working in public sector IT in a unit which manages COTS, MOTS, and we-own-the-code solutions.

(Now there are some exceptions, just as FOIA has exceptions as to which government records are disclosable, but by far I'd prefer the baseline would be code acquired by government must belong to or be freely usable by the public, either PD or permissively-licensed.)

I agree that there may be edge cases where it doesn't make sense to make software open source (although I also think that the majority of software used at IRS, NSA, CIA, and FBI could be open source). However, I strongly believe that if the code is not published, it should at least be available to the government itself, s.t. if a vendor goes out of business or the government wants to change vendors they can do so without issues.

> Should the IRS, NSA, CIA, and FBI code all be open source?

I'm leaning toward "yes", but that doesn't mean the data (including encryption/decryption keys) should be publicly available.

They will definitely want to consider extraordinary projects, materials, and infrastructure as munitions.

But they also have existing, good relationships with academics in other engineering fields where those regulated materials can still be accessed.

Maybe Wide-Open Source isn't the right answer for all projects because of the understood risks, but because they already have a good set of SOP for a similar program, perhaps it's time to ask them if they could expand that program's scope and loosen their grip on those things a little.

What do they have to hide? Nobody is asking for their list of passwords/private keys. Even if they do have secret projects, nobody sane is asking them to open source every line of code they write.

Look at NASA’s open mct on github - that’s awesome and we should push govt orgs to release more projects like this.

The federal source code policy [0] gives an exemption for national security or mission requirement. So in the US there are valid reasons why CIA and FBI shouldn’t be totally open. But for everything else the directive is to default to open.

[0] sourcecode.cio.gov

Isn't security by obscurity only giving a false sense of security? So it's actually making things worse.

Security through "obscurity" is not security at all.

I'm surprised no one has mentioned code.gov - see https://code.gov/

This is part of the implementation of M-16-21, which said, "Each agency shall release as OSS at least 20 percent of its new custom-developed code each year for the term of the pilot program. (3 years)".

Hey!!! Look!!! Actual Knowledge!!

At the very least publically funded code should be open between government departments. There seems to be a lot of internally developed work that is unnecessarily replicated or lost just due to having no obligation or available resources to share or collaborate. Maybe there needs to be some system in place to have some sort of internal NDA or need to know check, but this seems like a good first step.

You're assuming government code even uses proper documentation or version control..

I'm assuming they don't. I guess to clarify, I'm desiring that the government not only mandates that they do use documentation and source control, but that they commit such code to a repository accessible by other departments. A minimal readme and "init" push may do the trick, but I'd want my local counsel to at least be able to pull the code for something like the government's favorite public website implementation and reuse it at lower cost than redoing it.

It's easier to justify these practices when they have the real payout of reducing duplication.

The thrust of the article changes from "governments should use open source software" to "governments should open-source their custom software" about 2/3rds of the way through. Contrary to the headline, it never goes as far as to say that any government-funded software should be open-source.[1]

The latter precludes the (reasonable) arguments talking about grants and public-private partnerships, of which the intent is to stimulate economic innovation, not produce public code.

I think the proposal is reasonable, but it puts a heavy administrative onus on the government to open-source said code, including potential warranting that the code in question is indeed free of any other copyright or license requirement.

[1] Emphasis mine: "any government code produced with public money..."

The NHS (National Health Service) in the UK open-sourced their website frontend library[1] to allow all of the different hospitals, doctors practises and clinics to create their websites in-line with a centralised brand/accessibility standard.

[1] https://github.com/nhsuk/nhsuk-frontend

This make sense for some public sources, obviously not all (such as defense).

As it happens, this is exactly what we do at http://interneuron.org for Healthcare software. Most of our revenue so far has been NHS organisations. We are also a CIC - so a not for profit.

I work on Medicare software for a state that is funded by CMS[1] and it is freely available to any other state that also is funded by CMS. This has happened, we handed over the code base to another state and they are implementing it.


I'm completely okay with power grid and IRS code not being public.

I don't trust it to be built or maintained properly, which also means I'm relatively sure it'd be open season for troves of people who want to do bad things.

Open Source is great - but sometimes it makes absolutely no sense.

So avionics software for our latest fighter planes should be open source? Does not sound like a very smart idea from that perspective...


∆ Surprised this hasn't been shared yet

An example of open source government. https://www.boston.gov/news/bostongov-now-open-source-projec...

They also funded and encourage the open sourcing of the work we are doing for them https://github.com/greenriver/hmis-warehouse

From the US Digital Services Playbook [1]:

> PLAY 13

> Default to open

> When we collaborate in the open and publish our data publicly, we can improve Government together. By building services more openly and publishing open data, we simplify the public’s access to government services and information, allow the public to contribute easily, and enable reuse by entrepreneurs, nonprofits, other agencies, and the public.

> Checklist

> - Offer users a mechanism to report bugs and issues, and be responsive to these reports

> [...]

> - Ensure that we maintain contractual rights to all custom software developed by third parties in a manner that is publishable and reusable at no cost

> [...]

> - When appropriate, publish source code of projects or components online

> [...]

> Key Questions

> [...]

> - If the codebase has not been released under an open source license, explain why.

> - What components are made available to the public as open source?

> [...]

[1] https://playbook.cio.gov/#play13

It would certainly be interesting to make a pull request for something like Healthcare.gov or something. I wonder if it's measurable how much tax money we could save (if any) if we allowed the public to audit and improve our code.

Providing the source does not mean giving the public commit access.

Having government acquired code be open source licensed from the vendor to the government doesn't even mean the government publicly discloses the code.

It does mean the government is free to do so, or to hire another vendor to work on it without restriction.

Wouldn't FOIA mandate that it's released if it's not confidential? I'm seriously asking; I'm not an expert in these things.

> Wouldn't FOIA mandate that it's released?

FOIA has a number of things exceptions, material subject to it's exceptions is not required to be released, including:


Exemption 1: Information that is classified to protect national security.

Exemption 2: Information related solely to the internal personnel rules and practices of an agency.

Exemption 3: Information that is prohibited from disclosure by another federal law.

Exemption 4: Trade secrets or commercial or financial information that is confidential or privileged.

Exemption 5: Privileged communications within or between agencies, including those protected by the:

1. Deliberative Process Privilege (provided the records were created less than 25 years before the date on which they were requested)

2. Attorney-Work Product Privilege

3. Attorney-Client Privilege

Exemption 6: Information that, if disclosed, would invade another individual's personal privacy.

Exemption 7: Information compiled for law enforcement purposes that:

7(A). Could reasonably be expected to interfere with enforcement proceedings

7(B). Would deprive a person of a right to a fair trial or an impartial adjudication

7(C). Could reasonably be expected to constitute an unwarranted invasion of personal privacy

7(D). Could reasonably be expected to disclose the identity of a confidential source

7(E). Would disclose techniques and procedures for law enforcement investigations or prosecutions, or would disclose guidelines for law enforcement investigations or prosecutions if such disclosure could reasonably be expected to risk circumvention of the law

7(F). Could reasonably be expected to endanger the life or physical safety of any individual

Exemption 8: Information that concerns the supervision of financial institutions.

Exemption 9: Geological information on wells.

---[quote from: https://www.foia.gov/faq.html ]---

The sensitive things at issue probably fall into the existing exemptions (a lot in #3), otherwise business rules could be disclosed under FOIA now, even if the code wasn't the government’s to disclose.

> Exemption 9: Geological information on wells.

That one at the end seems interesting. I wonder how that got added in there as such a specific item and not as part of a broader category of sensitive information...I would think if wells were sensitive information, then so would mineral deposits and other natural resources, possibly falling under either exemption 3 or 4:

> Exemption 3: Information that is prohibited from disclosure by another federal law.

> Exemption 4: Trade secrets or commercial or financial information that is confidential or privileged.

I'm not saying it inherently does, I just thought it would be interesting.

Neither does auditing and improving...

The former administration agreed and made efforts in this direction including publishing code on Drupal.org: https://www.drupal.org/u/whitehouse

On a sidenote, apparently the new administration scrapped it for a brochure Wordpress site. Funny discussion here: https://www.reddit.com/r/drupal/comments/7kw7eu/whitehousego...

The Government of Canada has some decent policies on that. The Directive on Management of Information Technology stipulates that government agencies must:

* Favour open source solutions * Favour non-prorietary solutions * Release source for custom-built solutions under open source licenses through Government of Canada sites

See C.2.3.8 for the relevant clauses: https://www.tbs-sct.gc.ca/pol/doc-eng.aspx?id=15249

I have been banging on about this for years - please see http://oss4gov.org/manifesto

It's hard to make headway with the chicken and egg problem - if there was a good OSS system for a government need, the government could be persuaded to use it - but you can't write one without the initial investment which you cannot get because VC won't fund it ...

As a government employee and software developer I think this is completely the wrong argument. Before we get that argument we should convince the government to not be afraid of writing software. In all but a few edge cases the government doesn't want employees writing software for a number of unfounded irrational reasons. Many areas of the government, especially the military, are still afraid of using any open source software.

It should also concern scientific software. Right now many research groups close their software written using the tax-payers money. Some even try to sell it.

Actually companies throw fits when the government contributes to open-source, especially when it comes to lucrative defense related stuff [0].

[0]: https://www.wired.com/2012/07/nsa-accumulo-google-bigtable/

That case is not at all related, please read more carefully:

> the committee questions whether Accumulo runs afoul of a government policy that prevents federal agencies from building their own software when they have access to commercial alternatives

The law exists to prevent wasting time & money, it has nothing to do with being open source.

I'm in favor of the transparency this would confer, but it's probably not in our best national interests to be potentially giving away state of the art software for running large governments.

The code is of minimal utility to individual citizens who have no need to run large governments, but can be hugely beneficial to our competing nations.

Psst, it's not our state of the art software that's holding this ball of mud together.

It's a competition nonetheless.

Is it? What would it look like to win this competition? To lose?

This is principle 10 in the UK Government design principles. https://www.gov.uk/guidance/government-design-principles#mak...

In addition, publicly funded surveillance cameras in public places should produce publicly available data.

Yeah! NSA fork over those hacking tools!

As an aside, NSA is releasing GHIDRA next month: https://www.rsaconference.com/events/us19/agenda/sessions/16...

In a bit of fairness to the NSA (never thought I'd say that), they did give us SELinux, which I think was overall a net good.

Is it though? In most installation guides I've dealt with they recommend disabling it since it can cause random issues. i.e. The Percona XtraDB install guide.

I find that it's way too complicated of a layer that most people can't/won't learn. Compare this to the OpenBSD pledge and unveil which doesn't get in the way, and there's no way to disable them.

If you make something overly complicated, with the ability to disable it all too easily, then it won't get used.

>In most installation guides I've dealt with they recommend disabling it

That is generally bad advice then. selinux is used by Android and Fedora (and hence RHEL & CentOS). selinux can break things, but it is quite stable these days at least for the distro supported packages. The downside is that anything outside the distro packages will likely have no support or will run unconfined. OpenBSD unveil is still new and will face similar challenges in that it will cover the base system well, but for ports, it will be up to the port maintainer to implement it.

NSA hacking tools do regularly become public. Just not intentionally.

Totally makes sense. Why would tax payers fund a project, which will be used for profit of privileged few?

In the UK it seems like a pain to get anything open sourced.

AIUI, by default everything is Crown Copyright.


Wrong; our military, CIA, and FBI are all funded by public money; should they all be open too? No. There's plenty of software that's publicly funded that should not be open source, and much of that wouldn't make sense anyway as it's proprietary.

According to this line of reasoning, all the code for our nuclear weapons systems should be open source.

Beware the law of unintended consequences - it’ll bite you!

(For the record, I agree with the sentiment, but it’s hard to implement without causing undesirable effects.)

OK. If you believe this to be true, you need to elect leaders that make this a priority.

If it's not an issue, make it one.

If no one cares to support the issue, find someone to run on it.

If you can't find someone to run on the issue, you've got to run yourself.

What a useless suggestion. The current political setup does not allow electing people based on issues as granular as this. No politicians will make promises about this because only a small population understands why it is important.

Also you know how useless attempting to run yourself is. You will never get anywhere which is good because just having one good idea doesn't make you qualified to lead.

The best thing to do is to start public debate among people who understand the issue and attempt to get those in charge to join in.

The positive and optimistic side of this is well-documented.

The darker side is that making all publicly-funded software public would also mean making all weapons software public. That could have disastrous consequences.

You can have information that is public domain but still secret. Open source falls under copyright; secrecy falls under whatever national security laws are in place.

The UK government's cabinet papers fall under this: they won't be released for thirty years, but they will be released under the Open Government License.

I'm not sure that this is what the article is advocating, though.

It would never be implemented that way. At most it'd be "by default, software developed with government funding is released as open source software". Specialty weapon systems would quickly request and get an exemption.

But the reality is that even in weapon systems there's lots of "boring code" that would make sense to have as open source software. A lot of it involves "move bits from A to B", "calculate this matrix", and so on that are not really special (you can find the basic algorithms publicly) - but having a way to share the costs with others who need it would be valuable.

You can find the US Department of Defense (DoD) OSS FAQ here: https://dodcio.defense.gov/open-source-software-faq/

Here's the US DoD OSS policy (2009): https://dodcio.defense.gov/portals/0/documents/foss/2009oss....

The DoD policy does not require release by default as open source software (OSS), but there IS a short discussion of when it's okay to release software as OSS when it's funded by the Department of Defense (DoD) - and it's quite open-ended.

Perhaps we could just stop making weapons software? I mean, someone needs to go first.

When I look at wars throughout the world, a huge number of them are fought with US weapons, and we don't actually have a good track record of supporting peaceful people. I don't think that we'll get there until we stop dumping weapons on the world for profit.


> But your perspective is dangerously shallow and juvenile

There was a line in one of the Expanse books that really got me, where a woman was talking to her son, and paraphrasing, she explained that "We have lost again". When her son inquired how, she said, again paraphrasing "Those who want peace have lost to those who want violence."

One of the primary ways we can do this, is to simply stop exporting it. As of 2017, we export 50% more arms than the #2 country, Russia, https://www.aljazeera.com/indepth/interactive/2017/02/10-cou..., and we will as long as people who support violence and the contemporary model of arms sales throughout the world call people who disagree with it "juvenile".

Interesting, I have found that being an adult is as easy as not dying.

Alot of the conversation I have seen so far has been US-centric. That's fair, given that this was posted during US business hours, but for the US I don't really think this discussion applies as much.

Works created by the United States Government are not covered by copyright in the US, effectively making them public domain _in the US_. See https://www.law.cornell.edu/uscode/text/17/105

"But—", you say, "—the code the NSA is publishing has a license attached!" Indeed, that confused me too, until I found the answer at https://www.cendi.gov/publications/FAQ_Copyright_30jan18.htm...:

>…copyright exclusion for works of the U.S. Government is not intended to have any impact on protection of these works abroad (S. REP. NO. 473, 94th Cong., 2d Sess. 56 (1976)). Therefore, the U.S. Government may obtain protection in other countries depending on the treatment of government works by the national copyright law of the particular country. Copyright is sometimes asserted by U.S. Government agencies outside the United States.

So, Public Domain within the US, and Copywritten (but OSS-licensed) outside of the US.

My understanding is that copyright in EU countries is much more complicated. For example, apparently the view of the Eiffel Tower at night is copyrighted. See https://www.youtube.com/watch?v=M16CGK1T9MM

As for anything Classified in the US, there are laws controlling distribution. So, take something GPL-licensed: If you take, use, and modify the software, you are not required to provide the code unless you distribute the product to others.

See the question "Since U.S. Government works are not protected by copyright in the U.S., are all U.S. Government works publicly available without restriction in the U.S.?" from https://www.cendi.gov/publications/FAQ_Copyright_30jan18.htm... (it's pretty long, and has a _lot_ of references, so I'm not reproducing it here)

Of course, you may argue that, if a hacking tool is pushed to a remote system, and that hacking tool was made using GPL-licensed code, then the source should be distributed with the hacking tool. Also, note I said GPL, _not_ AGPL. For both of those cases, I don't know if the laws governing Classified material trump those governing Copyright (I'd bet they do), and what International law has to say.

But what about software that is funded by the government, but not directly created by government agencies, such as software that is the result of government research grants (at universities or private research institutions) or contracts with private companies?

And does that law also apply to works created by state and municipal governments?

If software is publicly funded, it should go into the public domain.

Where do you draw the line between publicly funded and publicly licensed?

Shouldn't this include all the code written but publicly traded companies as well? I mean, it is mostly funded by people's pensions - why not open source that!

Same for healthcare, engineering tech, pharmaceuticals, etc.

The combat slogan for this is: public money, public code!

Consider the Feds tried doing this with IP post WW2 and the results were disasterous. Turns out govt. employees aren’t allowed to be paid enough to make 100 hour work weeks reasonble - the level of effort required to get some ideas off the ground.

Once the Bayh-dole act was passed to allow citizens to take ownership of IP related to publicly funded endeavors, only then did the public start benefiting.

I came here and searched for the Bayh-Dole Act and am surprised to find only this single mention of it. It really does seem to me to be at the root of the current policies around federally funded research artifacts.

I disagree with the causation you infer regarding public benefits as a result of this change in IP rules. After all, we saw many post WW2 benefits of federal spending long before 1980. The most visible were in aerospace developments that gave us the jet age, but of course even Silicon Valley was well on its way in the 1970s with lots of computer industry groundwork already in place.

I think a lot of the computer industry developments of the 1980-1990s were almost inevitable once that stage was set. It was mostly a coincidence that Bayh-Dole was passed and universities ramped up their strip-mining of the federal budget around the same period. An awful lot of the current Internet age was built by people like me, working on open source projects and federal funding in spite of Bayh-Dole, not because of it.

Let's look at this from a global view, one country's funded software may not be forced to be shared by other countries unless they paid their dues? As code in the public domain is for the whole earth these days.

Yes nations do compete. The whole one-government funded software should be published to internet is _false_, in a technical sense.

I have always thought that all of the BBC's code should be open source.

I also propose all end-of-life software should be open sourced

Would it be possible to FOIA the source code and commit history?

Disclosability and licensing are almost orthogonal (though a proprietary license might make code more likely to be nondisclosable.) Particularly, open source code could still fall into most of the existing FOIA exceptions.

MS office to Libreoffice will be a disaster. I used excel professionally and tried libre. Omg, nothing worked. Loading large didnt work, doing pivot table didn't work. People need to start using R, libre will be really bad

I am 100% with this logic.

Also of concern is a tragedy of commons like issue where those who contributed no funding are able to get the software for free. For example if a piece of city focused software is FOSS and then a bunch of other cities choose to consume it but not contribute to the funding or code.

Agreed the other cities are being a jerk but not exactly a tragedy of the commons [1] as OSS is not a rival good [2]. One city using the software does not prevent others from using it.

[1] https://market.subwiki.org/wiki/Tragedy_of_the_commons [2] https://market.subwiki.org/wiki/Rival_good

I think it's closer to a fireworks show. Another set of eyes doesn't impact the purchaser's viewing.

I agree, it's not about the race to consume, but "Free Rider Problem"

  The problem in this case is how to get people to pay for the creation of the good in the first place, because people know that even if they do not pay for the creation of the good, they can still enjoy the benefits.

That's possible, however the idea is that the net outcome would be positive. Say city A open sources some useful software that city B, C and D freely copy and do not contribute anything back. No harm is done to city A open sourcing the software they would have needed anyways. Then city E starts using the software and needs some additional functionality so they use the budget they would have otherwise used to re-implement their own proprietary version of the software to contribute back their enhancement to city A's project.

Sure some cities will not contribute back but in open source it is the organizations most in need and most able to contribute back who do. Why is that a problem?

I think that's a poor application of tragedy of the commons.

Also, in your example, I don't know how other cities' use would impede the funding city's use.

In theory you might get a situation where several cities try to wait each other out for similar tasks, hoping another city will buckle first and fund it.

I worked on image analysis software in a wetlab for a particular type of microscopy. They are in just this position: the lab builds and sells software. The fees are low -- on the order of $10-$15k/license. They use the proceeds entirely to fund ongoing development of the software. And the simple fact is, without these license sales, they don't have the resources to fund development. The software is open source in the sense that the licensees are given the source.

If the lab has to give the software away, all development ends, unless you have $300k/year of grants sitting around.

This is the same bullshit excuse people use to justify subscription licensing.

If they can't get the interested labs to contribute towards the development without the coercion of a commercial license then it sounds like it's either done, or the development roadmap is misaligned with the users' goals. In either case, stopping the abusive current practices would be an immediate improvement for everyone involved.


As many have pointed out I misused the term "Tragedy of commons" , when in fact it is "The Free Rider Problem"[1]

However the main point stands. Others will not invest in FOSS in hopes that someone else will first and they they can just consume for free.

[1]:" https://market.subwiki.org/wiki/Free-rider_problem

This sounds like a synergy of the commons.

The other point that there are millions of boring crud apps written out there in govt/contractor land. We dont need to see these as open source, its not as if there are lots of devs that love fixing crappy crud apps in their spare time.

Sure there are. Plenty of folks are looking for easy fixes to make to software, or are trying to learn to code. When I was in college, I looked for little projects like this all the time that had approachable, easy to grok bugs.

I love being a gardener, cleaning up cruft, tackling todos, working through various bug lists, and generally tidying up a codebase.

Simple CRUD apps are full of this stuff. And I'm sure I'm not alone in looking for easy ways to participate and feel a little more connected.

Your argument against is that we might get bored reading the open source code?

While it is bad way to put it, but GP comments does make a little sense. All I can think of government software is CRUD apps or research projects. The source of later is released on case by case basis(tor, selinux on one hand, exploits such as stuxnet on the other) and any ruling won't help much there.

But why shouldn't CRUD apps be open sourced? Is the assertion that they stand to reap no benefit because no one will contribute to them? I don't think I agree with this position. Nor do I feel that contributions are the only forseeable benefit of open source software.

Would this not imply that if "my" government funded some software that "your" government would be free to take it? I'm not sure I'm keen on subsidizing "your" government all that much.

What would stop the people of/under "your" government from freeloading off the people of/under "my" government?

So, if YOUR government uses Linux, but doesn't participate in Linux development, it's bad for everyone else? Most Linux users aren't contributing to Linux itself. And likely less than half are making any significant contribution to Open Source Software at all. It's still useful software, and I still appreciate everyone that does work on it.

You seem to be conflating my position on the use of public funds with your position on open source in general.

"So, if YOUR government uses Linux, but doesn't participate in Linux development, it's bad for everyone else?"

I'm not sure where you are going with this. I certainly never claimed that governments should not use open source software. By all means I hope (for argument sake let's call it The US) leverages Linux where practical.

What I disagree with is the premise that The US or their contractors owe pull requests or public repos to the peoples of the world just because the US taxpayer partly or fully funded a development project. Hate on me if you must but I believe that no US funded research results be they in software or any other IP are or should be automatically owed to the peoples of the earth.

"Most Linux users aren't contributing to Linux itself. And likely less than half are making any significant contribution to Open Source Software at all. It's still useful software, and I still appreciate everyone that does work on it."

Yes, I agree with your idea that most open source users are freeloading in the sense that they contribute far less than they use. That is part of my point.

Users who don't contribute back, aren't removing what you have done, or a given government may have funded. Some governments ARE contributing to open-source... other governments using open-source doesn't undo or degrade things.

We are not in a communist/socialist world where every community must pay at the point of a gun for things they don't choose to pay for. And if some government wants to force users of software they develop to contribute to funding, then there licensing should probably be something to reflect that. But then it wouldn't strictly be OSI/Open-Source.

I am baffled by your post.

"We are not in a communist/socialist world"

This is EXACTLY my point. We are (at least I am) not. My government does not owe me or anyone else (let alone a foreign government) open source access to it's IP. To think otherwise (as you appear to do) is to be the socialist.

I hate to break it to you, but open source IP is socialism. I hope that does not come as some sort of shock to you.

Again to be very clear I am a fan of open source projects. I use them, I contribute to them. However, I choose how and what I use and contribute. My government does not choose for me or worse mandate via some policy that I must.

If you feel that The US government should require you to open source your in-house CnC software to the people of the DPRK because The US gave you a business development grant of $10k then you sir are the communist, not I.

I was using an analogy to express a point that because one organization chooses to fund opensource, the fact that another doesn't isn't expressly taking away anything.

That aside, I do feel most of what is publicly funded via tax dollars taken by threat of force should be open. This includes data and software (other than expressly licensed, commercial off the shelf software). I'm not as hardline as some on this, but I do feel that way.

The Communist POV would be that all software, data and access be restricted and expressly owned by the government and not really the people.

Open-Source is socialism if the government is forcing people to pay for it... so long as its' a collective voluntary thing not enforced by the government it is in fact not communism/socialism but a part of marginalizing that software which is not a core business component.

The point I was making is the government shouldn't be choosing for me... however, given that the government does fund software development for its' own needs, that development should probably be open.

For the CnC example, I've stated in other threads that I don't feel commercial software should be required to be opened if it exists and is licensed via govt contract vs. developed for.

Your government owes you access to the software that you funded.

If "your" government needs and uses the software then what have "you" lost?

This it getting downvoted maybe because it sounds nasty, but I think there's a relevant point here. Govt open source funding is something that external countries can benefit from, but I think a) the US is so big that any unfairness to them would be outweighed by benefits to themselves and society b) I have an uninformed guess that most government software is not groundbreakingly brilliant so there's quite a small subset that would be interesting to "steal", c) for smaller countries maybe this should be a treaty thing, would be interesting for example for the EU to vote on it, or put it into trade deals.

Is it your position that utilizing open source software is freeloading?

I contribute, I utilize and I appreciate open source and the community of dedicated contributors. Open source software is great and "YES" frankly the vast vast majoring of those involved in the use of open source software are freeloaders. We (yes I include myself) typically sow way less than we reap.

This argument though is not about the merits of open source software but about the use of public funds. "If my taxes helped pay for software, should result necessarily be public source code."

My opinion is absolutely not.

If my taxes as a citizen of country "A" paid for the development of some piece of software then I do not have the slightest reservation that the citizens of country "B" have no kind of moral, ethical, or legal right to it.

If the tax payers, government representatives, and authors agree and choose to release their results and products under some open source agreement then that is up to them.

On the other hand if my taxes went to pay for something I should get to know what it is, shouldn't I? If our elected representatives paid 500 million of our tax dollars for some piece of software shouldn't we be able to audit it and see if that was money well spent? If we can't do this, how will we know which representatives to vote for again?

This is a complete red herring. Within reason you can see where your money went. This is what a FOIA request is for.

Can I FOIA the source code for healthcare.gov? If so why hasn’t anyone done it?

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact