Why isn't all government software open source?

No readers like this yet.
Open and closed source

Opensource.com

The federal government is the single largest purchaser of code in the world. So why is this code—taxpayer-funded and integral to the day-to-day working of our democracy—so often hidden from public view? There are two sides to answering that question: Why does the government so often build on closed platforms, and once built, why isn't the code released to the public?

Using open source

It's a lot easier to contribute to open source when you're building on an open platform. While it's possible to open source a VBScript (Visual Basic for Applications), you'd likely have more momentum and receive a warmer reception from a platform with a more vibrant online community like Ruby or Python. Yet more often than not, the default in government is to look to "enterprise-grade," proprietary platforms from the onset, which send the government on a closed-source trajectory.

The demand for "enterprise" solutions

There's a good deal of FUD—fear, uncertainty, and doubt—in government CIO shops that open source is less secure, buggy, or more costly, and that you'll be in for a lifetime of pain if you don't invest in a real "enterprise solution." For one, if an agency writes a check to a software vendor, they know what they're getting. The contract spells out features, upgrade schedules, and allocates liability in the event that something goes wrong. More importantly, the vendor provides a phone number that the agency can call if they need help. "Post in the support forums and someone will reply" can be a scary proposition for a CIO (Chief Information Officer).

There are fewer suits behind open source

Before that transaction even occurs, the closed-source platform likely has a flashy marketing page and a cadre of federal salespeople calling up the agency and tabling at conferences, things open source platforms traditionally don't do, save Red Hat and a few others. And when the CIO's office asks for "enterprise" features like audit trails or meeting certain compliance requirements, you can bet that the closed-source solution will make sure those features make it into the next release cycle.

Closed source contractors

Last, these closed source platforms are what government contractors know, because it's what is taught in computer science programs and what they've always been asked to supply. If an experienced development firm has a bunch of ColdFusion developers, when they bid on a contract they're going to recommend that the thing be built in ColdFusion. Not to mention, the government's RFP may be scoped to favor the legacy technology they already know. All this means that before the first line of code is ever written, the odds are stacked against the project from ever seeing the outside of the agency's firewall.

But even if the agency's using a closed-source platform, there's no reason their custom code can't still be open sourced.

Contributing to open source

With the exception of 18F (digital services), the Consumer Financial Protection Bureau (CFPB), and a few others, government doesn't actually write code. In fact, it rarely has the human know how to do so if it wanted. Instead, the agency traditionally plays the role of a non-techincal program manager, providing specs for the functional requirements and selecting a contractor to deliver the end functionality. The points of contact at the agency overseeing the contract are rarely engaged with the open source community, let alone passionate about open source. As a result, open source traditionally isn't even part of the conversation. Why would it be?

Closed source workflows

In terms of the actual software development mechanics, the contract likely calls for a throw-it-over-the-fence workflow, where the agency doesn't even see the code until it's already in production, if at all—a long way from open source. Even if asked, the contractors may not have experience with more modern, open source workflows, or with participating in the open source community, creating a bad experience for all involved and further chilling open source efforts across government.

Reinventing the wheel as a business model

I also suspect that federal contractors have a disincentive to open source their work, considering that technical requirements likely don't vary much from agency to agency. A FOIA (Freedom of Information Act) request is a FOIA request and a press release is a press release, regardless of whether it's on FAA or FDA letterhead. Open sourcing these solutions the first time around could potentially decrease the demand for that same code being written a second time at the taxpayer's expense.

A culture of "no"

Once built, it takes humans to bring that code to the escape velocity necessary to overcome the agency's guarded inertia. Security is going to likely say no. Legal is going to likely say no. You'll have to get the code hosting platform approved. You'll have to procure an ongoing maintenance contract to review pull requests. You'll have to create a developer engagement policy for how you accept them. In a world of competing priorities, government employees will likely choose to move on to the next citizen-facing project, rather than spend potentially months combating the bureaucracy's immune system to change.

Clash of cultures

Even if the agency manages to open source the code, the open source community follows a set of norms vastly different than the rigid hierarchy of government. Government agencies don't always know how best to engage the open source community or how to integrate an open-source workflow within their own command-and-control culture. Supporting an open source community takes time, something government employees are traditionally short on. And when the agency doesn't follow the open source community's unspoken norms, the naysayer's worst fears become a self-fulfilling prophecy.

Transparency as a liability

Open sourcing code exposes the agency to the potential for criticism from millions of highly-technical, critical eyes, with little perceived upside from the agency's perspective. The non-technical agency team may not have the ability to evaluate the craftsmanship of the code internally, and it's often preferable to sweep things under the rug, rather than potentially air their dirty laundry to some of the Internet's most skilled trolls. Not to mention, the benefits of open source that advocates espouse are often not realized and thus cannot serve as a counterbalance if the code is so purpose built so as to render it unusable outside of government, thus attracting no outside contributors, or if the project is poorly managed, so as to scare those contributors away.

Meanwhile, unreleased source code poses absolutely zero liability in today's political climate. Which one would you choose?

What needs to change

To flip the default, three things needs to change:

  • First, government employees need to be empowered with a better understanding of and appreciation for the virtues of open source. Those agencies who have open sourced code do so because of individual cheerleaders spearheading the charge. Successful projects are scoped from day one with the intention of being open source and serve to reshape market demand. Initiatives like 18F and the PIF program can seek to inspire and educate the next generation of open source advocates within government.

  • Second, even when the agency doesn't explicitly call for it, as subject matter experts, government contractors have a duty to explore open source alternatives and to educate the market as to modern industry standard development practices. Any casual observer can see the direction the market's heading, and the smart firms have an opportunity to get in front of it. Create internal competencies around today's hottest technologies and grow government demand. Make it more practical for government to do the right thing, rather than the thing it's always done.

  • Last, the open source community needs to step up its game, both in terms of what it offers—moving past the perception that open source is written by hobbyists—and the reception government code receives. On the supply side, there's a tremendous business model in being the suits behind any one of the Internet's most popular open source projects—Automattic, GitHub, and Red Hat being a few examples—combating the FUD and providing "enterpise" support. On the demand side, the community needs to make it a liability not to release code ("what are you hiding?"), and make the agency's return on investment clear, by breaking down the "us vs. them" mentality, and supporting, not criticizing, government efforts to learn open source.

Why isn't all government software open source? Because you've got an entire value chain designed to produce closed source software, a system at equilibrium, with few incentives to rethink itself. Technology has made collaborating in the open easier in recent years, and as a result, the open source ecosystem has exploded. Yet, like all technology, government is still a few years behind mainstream adopters.

Hopefully, with your help, that can change.

Originally posted on Ben.Balter.com. Reposted under Creative Commons.

Tags
User profile image.
Named one of the top 25 most influential people in government and technology and described by the US Chief Technology Officer as one of “the baddest of the badass innovators,” and by the White House Director of Digital Strategy as “lightning in a bottle,” Ben Balter is a Government Evangelist at GitHub — the world’s largest software development network — where he leads the efforts to encourage

15 Comments

Would you please define acronyms at their first use. "VBA" and "CIO" are undefined. And although 18F and CFPB have links, it doesn't help if this article is to be printed out on paper, as in a "packet" that is provided before a council meeting. I'd like to forward this article to our mayor and other city council members and city administrators for they are people who are not savvy about software, yet they are in a position to set policy.

Don't you want your article to be read by the broadest group of people? This is an important issue.

Thanks John, I agree and have addressed your concerns in the article so you can forward it on. We strive to break out acronyms and make things easy to understand, however sometimes a few get through our radar. We will keep this in mind going forward. And, thanks for sharing this excellent article with your peers.

Why isn't GitHub open source?

Great article, and some food for thought. I work on (and lead) some government funded projects developing open source code that has benefits in many other areas. It has been great, but we still routinely hit road blocks around whether open source has a sustainable business model, what benefits open source has for government, etc. We all need to do more to promote sustainable models around open source, and foster ecosystems that grow around open models that still offer professional, enterprise level support.

The fact that most Linux users have now switched over to OS X (just walk around the github offices sometime) is proof positive that closed source can often be a much much better solution than open source. Hell, github itself is a closed source platform that has succeeded not despite being closed source but BECAUSE of it.

"...integral to the day-to-day working of our democracy..."

"America is an oligarchy, not a democracy or republic, university study finds" http://www.washingtontimes.com/news/2014/apr/21/americas-oligarchy-not-democracy-or-republic-unive/

More money for the oligarchy/1%?

> Why isn't GitHub open source?... github itself is a closed source platform that has succeeded not despite being closed source but BECAUSE of it.

You'd be hard pressed to make the argument that GitHub isn't overwhelmingly supportive of the open source community. A big part of the GitHub philosophy is to tax proprietary software (that software which you derive profit from) in order to support open source (that software which is dedicated to the community). Nearly everything at GitHub, shy of the "secret sauce" is open source, the exception being by necessity. Providing a service that extracts rents from proprietary software is what allows us to support open source, something that we couldn't do if anyone could standup the service for free. See http://tom.preston-werner.com/2011/11/22/open-source-everything.html for more context.

You just agreed with me. Github is successful because of closed source, not despite it. Github depends on closed source software to run the business AND trades in closed source software just to keep the lights on. IBM, Microsoft, Apple, Google and even the federal government could all make the same argument you are making. And until now only the government would have called it "taxation".

In reply to by Ben Balter (not verified)

Well, the article has a shadow of truth. But no more than a shadow... There are massive missing pieces. There are a large number of open source projects that government offices have released. There are also a large number of government programs that run on open source. And many new projects depend on open source...

Here are two large projects:
BRL-CAD
https://en.wikipedia.org/wiki/BRL-CAD
http://brlcad.org/

ADA
https://en.wikipedia.org/wiki/Ada_%28programming_language%29
http://www.adaic.org/

And the article is sorely missing references to actually back up the claims. However, here's a good place to start clearing away the article author's non existent research into the published topic...

http://www.govcode.org/

The general problem with government is the closed source desktop and office programs that are run... not that open source code isn't written, used, and released by government offices and programs.

I don't see how a government ostensibly owned by its people can NOT work on open source. Anything the gov produces is for the people, so it should be producing code for its infrastructure as well.

I agree that governments could do a lot more on Open Source. But where culture and supply chain is dominated by propretary software it seems hard to change. Contradictionary because transparency of government is essential and contracting just one single vendor is not. Contradictionary also because using the peoples money to run their business governments always should keepther spendings at a decent level.

Over here in the Netherlands though there are some remarkable exceptions. One local government body spent all IT budget on software development that whent totally wrong and they had hardly anything to spend. Then someone suggested a switch to Open Source -- and they did. Now they are an example that switching to Open Source is possible and brings extra advantage of lower cost for not having to pay big license fees. Unfortunately there is no massive switch to Open Source yet.

Other European countries have good examples too, where the French Gandermerie Nationale [1] and most Spanish regions [2] are great examples. Even the EU government is promoting open standards, although they chose a proprietary system for themselves. Maybe it is the strive for independence that boosts choosing Open Source. For the rest Free Software movements should keep on asking governments why they made the choices they made.

Government officials often state that switching to another operating system will confuse workers and tailor made software will become obsolete. But how come lots of people privately switched from Microsoft to Mac, Nokia phones to iOS without complaining? Besides, most corporate software runs web based so the OS is no longer the bottleneck. As the French Gendarmerie commander once stated: people only have to get used to differend icons and games (or something alike). Luckily the game of Mines also exists for Linux.

The command-and-control culture of governments is a strange phenomena. Many navies and armies adopted Open Source to control vessels etc. Remarkable because command and control is their speciality and confidentiality of their infrastructure is of utmost importance. But they made the switch mainly to have rocksolid reliable systems, the same reason ISS got 'Linuxed'.

Cutting budgets can be a lever to a broader adoption of Open Source in government. Civil servants that keep on asking for more Open Source could help too. And maybe it helps when Linux sales people sometimes wear a suit ;-)

[1] https://joinup.ec.europa.eu/community/osor/news/french-gendarmerie-open…
[2] https://joinup.ec.europa.eu/community/osor/news/spains-extremadura-star…

Good article. People working together for the common good make the best software and the best government. The industrial complexes that have built up around providing services and product to the government are powerful entities. No politician wants to be accused of destroying jobs by choosing an open source solution over something that puts money into the pockets of powerful constituents.

There is a good bit of open sourced software used in and by government and contractors. Bleeding edge technology is where open source seems to excel in government and commodity is where closed source seems to excel. It doesn't have to be that way all the time.

Using ColdFusion no longer necessarily means the code has to be proprietary. There's an open source CFML engine from Europe called Railo http://www.getrailo.org/ and plenty of CF code on GitHub.

RE: For one, if an agency writes a check to a software vendor, they know what they're getting.
====================
The recent experience with the ACA [Obamacare] roll-out should overcome this point.

The Scheduling and Planning Interface for Exploration (SPIFe) is now available as open source on GitHub at https://github.com/nasa/OpenSPIFe . SPIFe is an integrated planning and scheduling toolkit based on hundreds of hours of expert observation, use, and refinement of state-of-the-art planning and scheduling technology for several applications within NASA. It was designed from the ground up with the needs of the operational user in mind, and it presents unique solutions to a number of problems common in other commercial and homegrown systems. SPIFe has been used on the Mars Exploration Rover (MER) mission, the Phoenix Mars Lander mission, and the Mars Science Laboratory (MSL) mission. It also has been adapted as preflight planning and a real-time analysis console tool that supports all phases of planning on the International Space Station (ISS), as well as several other flight projects and analogs like LADEE and NEEMO. The core objective to open source SPIFe is to enable collaboration, and share the software to the outside participation and innovation.

Creative Commons LicenseThis work is licensed under a Creative Commons Attribution 3.0 Unported License.