Automate everything: Another lesson from the Linux philosophy

How to do fast, repeatable Linux installations

Posted 24 Mar 2015 by 

up
65 readers like this
Image by : 

opensource.com

Some of my recent articles have been about the Linux philosophy and its impact on the daily activities of system administrators like myself. One of the basic tenets of the Linux philosophy is to use software leverage, and one of the important corollaries of that tenet is to automate everything.

Lots of installs

I have always had several computers (as many as 14 or 15 at one time) and, with the advent of VMware and VirtualBox, a similar number of virtual machines that I use for testing. I also install Linux on customer systems. As a result, I do frequent installations of Linux, sometimes several a day. This leads to the need to do fast, repeatable installations.

For example, I have a favorite set of configurations that I do for things like Midnight Commander. I also have a number of fonts that I like to install that are not part of most default installations. I could install each font manually using YUM and make the configuration changes to Midnight Commander manually each time I do an install, but that takes a lot of time and gets to be tedious and boring.

When I was doing all of this manually, I forgot things. I started keeping lists of things to do, but that was still time consuming. So over the years I have developed a process that ensures installations are done quickly, reliably, and without me forgetting to install or configure anything.

I first do a pretty basic installation. I configure the disk partitioning and logical volumes the way I want and I (maybe) install the KDE desktop. I do not go through the entire list of available packages or groups and try to remember which ones I want to install to get the right tools I want on my computers.

Scripting repetitive tasks

Over the years, I developed what was at first a fairly simple Bash script that I ran to do the configurations and installation of the other RPM packages that I wanted.

As time went on, that simple script evolved to allow me to tailor the standard installation based on whether they would be desktops, servers, or classroom systems. I added options that enabled me to select one or more of the several available desktop environments for installation. I created various configuration files that needed to be installed and determined that the best way to do that was to create an RPM package that included those files, some of which were more scripts that I have created over the years to perform various other repetitive tasks as well as my post-installation script.

I also expanded my script to enable the same functionality with CentOS as I had for Fedora. The RPM package is in itself a form of automation because it relieved me of the need to remember which files to install and where. The RPM package now installs about two dozen files, and the post installation script I have been improving for about 10 years is up to more that 1,300 lines of code and more than 800 comment lines.

Even using the RPM and the post-installation script, completing all of the work required to get each one of many computers I install up to my standards could take more than an hour. I certainly don't miss the days of typing all of those instructions by hand and waiting for each to complete before typing in the next one.

From desirable to necessity

Everything was going along nicely. And although I could have done all of that work manually, it has been much easier to use my automation. Fedora 21 appeared on the scene a few months ago, and the automation I have created over the years has now become a necessity.

For those of you who are not familiar with Fedora 21, the installations have changed dramatically with this release. There are now three separate installation ISO images: Desktop, Server, and Cloud. I have used both Desktop and Server ISOs for installation and I dislike them both intensely. I think the new installations are terribly limiting for the vast majority of Fedora users.

There is no simple install image. The Desktop ISO is a live image. There are no options for installing any packages during the installation except those that are in the live image ISO. None. If I want to install the KDE—or any other—desktop instead of GNOME (which I do), I have to download the KDE spin or install it after the initial installation. I cannot do it from the primary installation medium, the Live image.

I cannot even choose to install LibreOffice. There is no way to do that during installation. I have to install that and many other things after the initial installation. In my opinion this is a huge stumbling block to many would-be Linux users, especially newcomers.

Fortunately, my post-installation RPM and script allow me to do just that with very little fuss. And yes, I have had to make a few adjustments to my script—as I have with every new release—to accommodate some of the changes between releases.

My penchant for following the Linux philosophy has paid off exceedingly well for me. Because I take the time to automate everything, I have personally experienced little disruption due to a major change in the way Fedora Linux handles installations.

So here is what I have gained by automating my installations:

  • I save time on every installation.
  • Installations are consistent.
  • There is zero to minimal disruption when major changes to distribution installation are introduced.

Of course, there are other ways to do the automation of a Linux installation and configuration, and many tools that can be applied to that task, such as Kickstart, Puppet, Satellite Server, and others. I have used Kickstart extensively, but my script just works very well for me in my current environment and meets my needs—and that is the name of the game in Linux.

Warning: The Bash program described in this article is always a work in progress and may contain errors. Some of the changes this script will make to your system, such as disabling SELinux may leave your system more vulnerable to intrusion. It may not work as intended or described and you should use it only as a learning tool in the context of these articles and not in a production environment until you have thoroughly tested it and modified it as necessary for your own needs and environment.

8 Comments

Chadwick Ferguson

dd if=/dev/sda of=- | tee >(dd of=/dev/sdb) >(dd of=/dev/sdc) >(dd of=/dev/sdd)

Vote up!
0
Vote down!
0
libreguarani

Nice article David. I had the same issue before. I recomend you make your own custom livecd with all the stuff you want in it. I used uck before and it saved my life a few times. Here's an article that you might like.

http://community.linuxmint.com/tutorial/view/1784

UCK: apt-get install uck

Vote up!
2
Vote down!
0
dboth

Thank you.

Yes, I have done my own spins and that is definitely another method for automating installations. Because of the number of options I have in my script, I would either need a large number of spins or two or three spins and still have a post installation script. I chose to use the script exclusively and it does meet my needs very nicely.

Vote up!
0
Vote down!
0
Akm

Aha! I do the same man. I have a configuration file with lots of '#' to un-comment out if the machine should have certain features that I might use it for. I do a bare Debian install without any GUI, mount my external hard drive that contains my local mirror (via apt-mirror), edit my configuration file, run my script, and then come back in a half hour.

Vote up!
0
Vote down!
0
drbokko

Using a PXE with a base OS system followed by a mix of scripts, ssh-copy-id and "Ansible" solved us many problems of maintaining a lot of scripting when installing different machine with different customization.

Vote up!
0
Vote down!
0
storm

Expanding upon the use of RPMs, they can be used for a variety of things such as configuration management (configuration files) and meta-packages to bring on additional packages (list them in the Requires field of the spec file). Then setup a repo that can be used directly by kickstart and install the packages when the system is kickstarted. Many specialized packages can be made so that the system boots the first time with the desired configuration already in place. No need to bulldoze a system after the fact with some third party system like Puppet. This way, rpm -V can also be used locally to verify sanity of the configurations. Any systems which have been manually altered can be restaged with the original packages to reset the system to a sane state. Think about it.

Vote up!
0
Vote down!
0
cvanaken

I use Puppet a lot, I only install a minimale OS, install Puppet and the rest of the configuration and installation of packages is done. Biggest benefit is that it is the same everywhere!

Vote up!
0
Vote down!
0
Pat Hobbis

I have to wonder why you did not use Remastersys or System Back?

Vote up!
0
Vote down!
0

David Both is a Linux and Open Source advocate who resides in Raleigh, North Carolina. He has been in the IT industry for over forty years and taught OS/2 for IBM where he worked for over 20 years. While at IBM, he wrote the first training course for the original IBM PC in 1981. He has taught RHCE classes for Red Hat and has worked at MCI Worldcom, Cisco, and the State of North Carolina. He has been working with Linux and Open Source Software for almost 20 years. David has written articles