10 steps to migrate your closed software to open source

Difio is a Django based application that keeps track of packages and tells you when they change. It provides multiple change analytics so you can make an informed decision on when or what to upgrade. Difio was created as closed software, then I decided to migrate it to open source to allow for in-house deployments and attract a larger community around the project.

10 steps to migrate your closed software to open source

Simplify

Any application that has been around for a few years has accumulated code and features which are not in use anymore. Remove everything that can be safely deleted in order to get a cleaner and simpler code to share. Open sourcing Difio began with removing around 20% of the existing code base.

This included:

• unused and obsolete settings
• Django apps
• static files and templates
• model classes
• legacy URLs
• deprecated functionality
• other Python utilities

For extra dependencies and one-off code, for example, I had used markdown in some templates and pure HTML in others. I had some custom template tags which were used only in one or two places. All of these were deleted and templates were made more consistent with each other.

Hard-coded paths, values, URLs, etc are inevitable when doing rapid prototyping. At some point a closed source application inherits the artifacts of its deployment environment and these need to be cleaned up. I have used custom template tags and settings where needed.

Create self-contained modules

In other words, reorganize the file structure which also increases simplicity and makes things feel a bit more natural. Making modules self-contained and independent makes it easier to separate them later.

Difio's web backend is deployed on OpenShift, which used a different directory layout for templates and static files. I've had to move around the files and update Django's settings in order for them to load properly. This also caused me to rethink the way origin static files were served to the CDN backend.

Separate internal from external

It is natural to have some internal code inside your apps to provide you with more information. For example, usage tracking and other metrics, billing, and so on. In the case of web-based services, these are usually integrated together with the core functionality and need to be separated.

This is also a good opportunity to decide what goes out and what not. For example, Difio doesn't ship its test cases because they need more work to be cleanly separated from the CI environment and run both against the whole web service and against the stand-alone app.

Difio contains five separate modules:

1. difio/ (the core user experience)
2. a profile sub-system
3. a billing module
4. extended admin interface
5. a deployment related module (mostly settings)

All of these were properly separated and isolated from each other, removing imports and internal dependencies. At present difio/ depends on a few profile APIs for which it provides sane defaults.
This step also helps you split operational artifacts (e.g. customized email templates) from the core user experience.

Code refactoring

It goes without saying that refactoring and testing should be a continuous effort. However, until now you have probably made a quick review of the entire existing source code (or most of it) and have noticed lots of things that needed improvement. Here's the opportunity to do so.

This is also a good opportunity to create a short roadmap and preload your public issue tracker with bugs to fix. It will help your newly born project display activity and continuous improvement in the early days while new contributors are still missing.

With Difio, I refactored some methods, mostly internal ones, to make them fit the new application structure. The external methods were left to fix later because they would be nice to have, not need to have.

Legal work

Depending on the size and complexity of the software and your organization, migrating to open source can be quite time consuming. Everything from selecting a proper license, branding, naming individual authors, doing legal review, and potentially searching for patent infringement code—should be factored in this step.

For Difio this was much simpler. I opted for the Apache 2.0 license, added the license header to all source files, and properly attributed authorship and copyright for external code which I've found on the Internet (it didn't specify any particular terms most of the time).

Update and list external dependencies

As a software developer, you have to take the extra step to upgrade to the latest versions and make sure your software works properly with them (or at least reasonably well). Nobody would like to run your code with old dependencies and quite often that may not even be possible.

Also provide a dependency list (e.g. requirements.txt file) and let people know how to install them before running the software. Luckily, Difio being a Django based application had only minor issues upgrading and moderate number of external dependencies.

Provide documentation and examples

Documentation is a great plus to any newcomer to your project. After all you are opening up something with the hope of attracting a community. Having written docs and examples is a must.

For Difio, I wrote a README file with detailed settings because the application has multiple subsystems (messaging layer, cron scheduler, etc.) which can be configured in myriad of ways. The second document I created was a Content Administration Guide because not everything is automated and sometimes needs manual adjustments. These two files summarize the most important design and deployment features of Difio—but you may need to write more or other documents for your project.

Create a public code repository

The time will come to create a public code repository and push the software upstream.

For Difio, I decided to copy the entire difio/ directory out of its previous location and do a single initial commit. This has the disadvantage that all previous history is not available, but I've decided to go this way to avoid revealing secret keys and passwords which may have been hard-coded at some point in the code.

In production, the difio/ directory was replaced by a git submodule to allow for faster released/deployment cycles and mainly because my cloud environment uses git as the deployment mechanism.

From now on everything you do with the source should be done in public.

Test stand-alone deployments on a fresh environment

Until recently you have probably been working inside your existing local copy and inherited all the artifacts that come from the proprietary version of the application, like dependencies, environment configuration, and so on. Testing from the point of view of an external user on a fresh environment(s) will help you further refine your docs and clean up remaining issues.

While testing Difio, I found several missing or extra requirements, missing and inappropriate settings, few bugs, and incomplete documentation.

When done, start over and repeat again until every step is properly explained and works out of the box! This will ensure future contributors and users will at least be able to install your software.

Announce it

This is the last step! Write your press release and announce the new software to the world. Congratulations, you are now open source!