For most people, computers don't stay the same. Software is added, removed, and updated. Configurations are changed. Think about the changes you've made to your computer since the first time you booted it up. Now imagine making those changes to 10, 100, or 1,000 more computers. Configuration management tools are what make implemententing and enforcing these changes possible.
Proto-configuration management
Using a "golden image"—a pre-configured copy of an operating system and some applications—copied onto a machine as part of an initial installation process is a precursor to configuration management. Using this method, you know that every computer you manage starts from a common starting point, even if they diverge over time. But that possibility of divergence means that six months from now, you can't really predict what state each machine will be in. And golden images aren't very flexible.
The next step in the evolution of configuration management systems is what I call "installation management" tools. These provide more flexibility and basically allow the automation of a "normal" installation. Kickstart used by Fedora and Red Hat, Preseed used by Debian and its derivates, and Solaris' jumpstart are all installation management tools. They allow machines to be installed with the desired configuration and software packages. Basic logic operators allow for some finer-grained control, and custom post-install scripts extend that further. But subsequent changes are still a manual process.
Configuration management features
Enter configuration management tools. Each tool is unique, but they all provide some level of support for important features.
Enforcement. Configuration enforcement may be the single most important feature of a configuration management tool. By running regularly and ensuring the machine is configured to the desired state, configuration management tools prevent configuration drift. Configuration drift can happen in a variety of ways: Package updates, live debugging, "helpful" coworkers, etc. Whatever the cause, being able to say with confidence, "This is how this machine is configured," is a great way to shorten incident resolution time and reduce surprises.
Enables cooperation. Configuration management tools make it easier for team members to cooperate. With one change, configuration can be updated across the entire infrastructure. Hand-editing configuration on machines can only lead to unexpected differences. By putting all of the configuration in one place, you can keep from stepping on someone else's (including future you) toes.
Version control friendly. Of course, the best way to enable cooperation is to have everything in a version control system. All of the tools listed below use some form of text for configuration. This means you can take advantage of the benefits of your favorite version control system. The benefits of version control are beyond the scope of this article, but let me assure you from experience that you really, really want your configuration in a version control system.
Enables change control processes. Because configuration management tools are textual and VCS-friendly, you can treat infrastructure changes like code. Changes can be submitted as diffs or merge requests and be subject to code review before approval. Having explicitly enumerated changes with timestamps can make reconstructing incidents much easier. With atomic changes, you can release them at a rate you're comfortable with instead of throwing a bucket of changes at your infrastructure all at once.
Abstraction. Few sysadmins maintain completely homogeneous environments. Even if you're an all-Linux shop, you probably have multiple distros that you support, or at least multiple versions of a distro. With configuration management tools, many of the operating-system-specific implementations of a configuration are abstracted away for you. The same configuration file can be used to manage, for example, the installation of Apache HTTPD on both Red Hat and Ubuntu systems.
Some configuration management tools
Great configuration management tools exist today and are just waiting for you to try them out. The table below lists some of the most popular tools, but there are many more out there. Most configuration management tools these days support Windows and Unix in addition to Linux.
| Tool | Language | FLOSS version | |
|---|---|---|---|
| CFEngine | C | CFEngine Community | |
| Puppet | Ruby | Open Source Puppet | |
| Chef | Ruby | Chef | |
| Ansible | Python | Ansible | |
| SaltStack | Python | SaltOpen |
Why you should use configuration management
If the preceding 800 words haven't convinced you, try this anecdote on for size. Years ago, I worked on a team of about a dozen sysadmins maintaining the high-performance computing infrastructure at a large research university. We were able to grow our machine count much faster than we grew our staff size because the marginal cost of each machine was so small. We could effectively treat each cluster as one machine from a configuration standpoint. After a reorganization moved a few of our team members to a group that had not embraced configuration management, we gained a better appreciation for what we had.
One day at lunch, one of our unlucky colleagues told us that in the 130 or so machines his group maintained, there were nearly 100 variations of the same configuration file. Most of them were functionally identical, which made it even worse. How much cogntive overhead was added by having to remember which changes were significant and which machines had those changes? And how much effort was involved when that file needed to be changed? Whether you maintain 5,000 machines or five, configuration management tools make the job easier.
5 Comments