Bolster your DevOps infrastructure with open source tools

Use these tools to automate your CI pipeline and boost the efficiency of your DevOps operations.
362 readers like this.
Tools illustration

When we talk about DevOps, we typically mean managing software deliverables, not infrastructure. But the overall system sanctity is deeply coupled with infrastructure integrity. How many times have you heard “But it works on my system”? Or perhaps a misconceived admin changes the configuration of the production server and things don’t work anymore. Hence, it is essential to bring infrastructure into the proven DevOps practices of consistency, traceability, and automation.

This article builds on my previous one, Continuous infrastructure: The other CI. While that article introduced infrastructure automation and infrastructure as a first-class citizen of the CI pipeline using the principles of infrastructure as code and immutable infrastructure, this article will explore the tools to achieve a CIi (continuous integration of infrastructure) pipeline through automation.

Baking images

The first step toward consistent infrastructure is to define it in a readable, maintainable, and extensible format. This makes it possible to create machines from the configuration—a process known as baking. An open source infrastructure management and baking tool is introduced below.

Packer makes it possible to define infrastructure configuration in a human-readable, JSON-like format. Packer can bake images for several platforms, including AWS, VMware, VirtualBox, and others. With an open source plugin model, more platforms are being added by the community.

Open source provisioning tools

The next step after baking is to create VMs (virtual machines)—a process known as provisioning. Following are a few open source provisioning tools:

Heat is part of the OpenStack suite to manage infrastructure lifecycles, so it fits in well with the complete open source cloud ecosystem based on OpenStack. While still evolving, Heat aims to provide human-readable configuration scripts and a REST API while attempting to maintain compatibility with AWS for portability.

SparkleFormation is a project to provide a common API across the myriad cloud platforms available today—including AWS, Azure, Google Cloud, and OpenStack. While the SparkleFormation DSL makes it possible to define iInfrastructure as code (IaC), its CLI tool provides a command-line interface for remote orchestration.

Terraform uses the images baked by Packer to provision VMs on multiple cloud platforms. Like Packer, Terraform uses a provisioners model for community extension.

Open source configuration management tools

With the VM created and running, it may still be necessary for a top-up configuration to ensure that there is no configuration drift. Following are a few open source configuration management tools:

Ansible is an agentless software provisioning, configuration management, and application management tool that deploys modules (software to be deployed are packaged as idempotent modules) to nodes (servers), mainly over SSH.

Chef is a configuration management tool that uses system configuration scripts (known as recipes) written in Ruby DSL to manage and configure server applications. Chef supports both client-server and standalone modes. In the client-server mode, Chef provides a centrally manageable server and API.

Puppet is a configuration management tool that uses configuration files (known as Puppet manifests) written in native declarative language or Ruby DSL to specify system configuration in user-friendly high-level resource terms. Puppet also supports both client-server and standalone modes.

SaltStack is a configuration management software and remote execution engine. SaltStack is designed to be dynamic, modular, and extensible. It uses Python modules to manage remote execution and state management.

These are some of the popular open source tools with strong individual community backing for programmatic and configuration orchestration. All these tools are cloud platform-agnostic. Several also support client-server mode, which makes it possible to centrally orchestrate and monitor a server cluster.

Technology for infrastructure configuration

With these tools, it is possible to set up and configure infrastructure. However, there’s an evolving contemporary approach using container technology. In simple terms, containers are lightweight mini-VMs with software and its environment bundled together.

Linux containers are a popular open source technology to programmatically create containers with configuration scripts. Containers have been around for quite some time, but they really came on the scene with the open source project Docker. Containers make it possible to create images with a defined configuration and save the images to a central repository, and admins can start and stop them on demand. For a detailed introduction to Docker, read What is Docker?.

It is worth noting that Docker is not just a technology, but a whole ecosystem to manage containers. There are also many other tools and technologies that help automate containers. Kubernetes is one of the most popular container orchestration tools. Kubernetes provides the ability to build a medium- to large-scale container-agnostic infrastructure. You can learn the basics of using Kubernetes with this tutorial.

Open source makes DevOps infrastructure a reality. Why not automate your infrastructure with the power of open source today?


What to read next
User profile image.
Girish has over 20 years’ experience in technology and software at a global IT Services organization based in India. Girish is architect of "I Got" cloud platform to uplift the bottom of the pyramid built with open source stack and contemporary architectural patterns such as microservices, containerisation and multi tenancy. Girish writes on open source and tech topics.


I like that you put in Ansible in there, since most of the tools I can work with are native tools Ansible is from Red Hat and it works great with their suite of applications, programs, and server.

Terraform is not really that... It does what SparkleFormation does too, among other things, and much better!

Saltstack does much more than only config mgt and ad hoc commands. It can handle thz full product fliw, from initial provisioning using salt-cloud to event driven mgt / scaling... This is by far the most powerful tool in that list.
The learning curve can be steep....

Creative Commons LicenseThis work is licensed under a Creative Commons Attribution-Share Alike 4.0 International License.

Download the Open Organization Guide to IT Culture Change

Open principles and practices for delivering unparalleled business value.