How to create reproducible build environments with Rebuild

Building modern software in a predictable and repeatable way isn't easy. The overwhelming number of software dependencies and the need to isolate conflicting components presents numerous challenges in managing build environments.

Although there are many tools aimed at mitigating this challenge, there are two approaches most of them take: either they rely on package managers to preserve and replicate package sets, or they use virtual or physical machines with preconfigured environments.

Both approaches are flawed. Package managers fail to provide a single environment for components with conflicting build dependencies, and separate machines are heavy and fail to provide a seamless user experience.

To solve those issues, we need a system that enables multiple, immutable build environments to be managed on the same machine; uses the same environment for development, continuous integration (CI), and testing; easily shares environments among team members; and tracks the environment versions.

I think Rebuild meets these criteria and is ideal for establishing build infrastructures for source code. Rebuild is a new and open source management framework that allows the user to create and share fast, isolated, and immutable build environments that can be used either locally or as a part of CI systems.

Here's how Rebuild can be used to simplify software development.

Install the client

Rebuild is available at RubyGems.org. Its client requires Docker Engine 1.9.1 or newer and Ruby 2.0.0 or newer.

To install, execute this command:

gem install rbld

To test your installation, execute this command:

rbld help

Search for existing environments

Rebuild can simplify the use of embedded toolchains. By default, Rebuild is configured to work with Docker Hub as an environment repository, and we already have ready-made environments there.

The workflow for Rebuild is:

1. Search for the needed environment in its environment repository.
2. Deploy the environment locally (done once for each specific environment version).
3. Run Rebuild.

If needed, you can modify, commit, and publish modified environments to the registry while simultaneously keeping track of different environment versions.

To search for environments, execute this command:

$ rbld search

which returns:

Searching in /RebuildRepository...

    bb-x15:16-05
    rpi-raspbian:v001

Then, deploy the environment to the local machine by entering the following command:

$ rbld deploy rpi-raspbian:v001

which returns:

Deploying from /RebuildRepository...
Working: |---=---=---=---=---=---=---=---=---=---=---=---=-|
Successfully deployed rpi-raspbian:v001

Next, use Rebuild to compile your code. Enter the directory with your code and run:

$ rbld run rpi-raspbian:v001 -- make

Of course, you can use any other command that builds code. You can also use your environment in interactive mode. Just run the following:

$ rbld run rpi-raspbian:v001

Then execute the necessary commands from within the environment. Rebuild will take care of the file permission and ownership for the build result that will be located on your local file system.

Create the environment

The example above used a ready-made Raspbian environment; let's take a look at how it was created.

First clone the SDK:

git clone git://github.com/raspberrypi/tools.git rpi-tools

Then create the initial Rebuild environment using a base image:

rbld create --base ubuntu:16.04 rpi-raspbian

Modify the environment by running:

rbld modify rpi-raspbian:initial

This permits users to modify the environment in interactive mode.

Run the following commands to install Make:

sudo apt-get update
sudo apt-get install -y make

Copy the toolchain and configure environment variables:

TOOLCHAIN=gcc-linaro-arm-linux-gnueabihf-raspbian-x64
sudo cp -r rpi-tools/arm-bcm2708/$TOOLCHAIN /  
echo export CC=/$TOOLCHAIN/bin/arm-linux-gnueabihf- | sudo tee -a /rebuild/rebuild.rc

Now that the environment setup is complete, exit the interactive shell by pressing Ctrl+D.

Let's commit the changes and delete the initial environment:

rbld commit rpi-raspbian --tag v001
rbld rm rpi-raspbian:initial

Run:

rbld list

You should see your environment, which is ready to use:

    rpi-raspbian:v001

Explore Rebuild

To learn more about Rebuild, run rbld help. You can also check out Rebuild's source code and documentation on the project's GitHub page, as well as download a handy cheat sheet of Rebuild commands.

Learn more in Yan Vugenfirer and Dmitry Fleytman's talk, How Linux Containers can Help to Manage Development Environments for IoT and Embedded Systems, at Open Source Summit in Los Angeles.