An introduction to parameter expansion in Bash

An introduction to parameter expansion in Bash

Get started with this quick how-to guide on expansion modifiers that transform Bash variables and other parameters into powerful tools beyond simple value stores.

An introduction to parameter expansion in Bash
Image credits : 

Haplochromis (Own work) [GFDL or CC BY-SA 3.0], via Wikimedia Commons

In Bash, entities that store values are known as parameters. Their values can be strings or arrays with regular syntax, or they can be integers or associative arrays when special attributes are set with the declare built-in. There are three types of parameters: positional parameters, special parameters, and variables.

For the sake of brevity, this article will focus on a few classes of expansion methods available for string variables, though these methods apply equally to other types of parameters.

Variable assignment and unadulterated expansion

When assigning a variable, its name must be comprised solely of alphanumeric and underscore characters, and it may not begin with a numeral. There may be no spaces around the equal sign; the name must immediately precede it and the value immediately follow:

$ variable_1="my content"

Storing a value in a variable is only useful if we recall that value later; in Bash, substituting a parameter reference with its value is called expansion. To expand a parameter, simply precede the name with the $ character, optionally enclosing the name in braces:

$ echo $variable_1 ${variable_1}
my content my content

Crucially, as shown in the above example, expansion occurs before the command is called, so the command never sees the variable name, only the text passed to it as an argument that resulted from the expansion. Furthermore, parameter expansion occurs before word splitting; if the result of expansion contains spaces, the expansion should be quoted to preserve parameter integrity, if desired:

$ printf "%s\n" ${variable_1}
my
content
$ printf "%s\n" "${variable_1}"
my content

Parameter expansion modifiers

Parameter expansion goes well beyond simple interpolation, however. Inside the braces of a parameter expansion, certain operators, along with their arguments, may be placed after the name, before the closing brace. These operators may invoke conditional, subset, substring, substitution, indirection, prefix listing, element counting, and case modification expansion methods, modifying the result of the expansion. With the exception of the reassignment operators (= and :=), these operators only affect the expansion of the parameter without modifying the parameter's value for subsequent expansions.

About conditional, substring, and substitution parameter expansion operators

Conditional parameter expansion

Conditional parameter expansion allows branching on whether the parameter is unset, empty, or has content. Based on these conditions, the parameter can be expanded to its value, a default value, or an alternate value; throw a customizable error; or reassign the parameter to a default value. The following table shows the conditional parameter expansions—each row shows a parameter expansion using an operator to potentially modify the expansion, with the columns showing the result of that expansion given the parameter's status as indicated in the column headers. Operators with the ':' prefix treat parameters with empty values as if they were unset.

parameter expansion unset var var="" var="gnu"
${var-default} default gnu
${var:-default} default default gnu
${var+alternate} alternate alternate
${var:+alternate} alternate
${var?error} error gnu
${var:?error} error error gnu

The = and := operators in the table function identically to - and :-, respectively, except that the = variants rebind the variable to the result of the expansion.

As an example, let's try opening a user's editor on a file specified by the OUT_FILE variable. If either the EDITOR environment variable or our OUT_FILE variable is not specified, we will have a problem. Using a conditional expansion, we can ensure that when the EDITOR variable is expanded, we get the specified value or at least a sane default:

$ echo ${EDITOR}
/usr/bin/vi
$ echo ${EDITOR:-$(which nano)}
/usr/bin/vi
$ unset EDITOR
$ echo ${EDITOR:-$(which nano)}
/usr/bin/nano

Building on the above, we can run the editor command and abort with a helpful error at runtime if there's no filename specified:

$ ${EDITOR:-$(which nano)} ${OUT_FILE:?Missing filename}
bash: OUT_FILE: Missing filename

Substring parameter expansion

Parameters can be expanded to just part of their contents, either by offset or by removing content matching a pattern. When specifying a substring offset, a length may optionally be specified. If running Bash version 4.2 or greater, negative numbers may be used as offsets from the end of the string. Note the parentheses used around the negative offset, which ensure that Bash does not parse the expansion as having the conditional default expansion operator from above:

$ location="CA 90095"
$ echo "Zip Code: ${location:3}"
Zip Code: 90095
$ echo "Zip Code: ${location:(-5)}"
Zip Code: 90095
$ echo "State: ${location:0:2}"
State: CA

Another way to take a substring is to remove characters from the string matching a pattern, either from the left edge with the # and ## operators or from the right edge with the % and %% operators. A useful mnemonic is that # appears left of a comment and % appears right of a number. When the operator is doubled, it matches greedily, as opposed to the single version, which removes the most minimal set of characters matching the pattern.

var="open source"
parameter expansion offset of 5
length of 4
${var:offset} source
${var:offset:length} sour
  pattern of *o?
${var#pattern} en source
${var##pattern} rce
  pattern of ?e*
${var%pattern} open sour
${var%%pattern} o

The pattern-matching used is the same as with filename globbing: * matches zero or more of any character, ? matches exactly one of any character, [...] brackets introduce a character class match against a single character, supporting negation (^), as well as the posix character classes, e.g. [[:alnum:]]. By excising characters from our string in this manner, we can take a substring without first knowing the offset of the data we need:

$ echo $PATH
/usr/local/bin:/usr/bin:/bin
$ echo "Lowest priority in PATH: ${PATH##*:}"
Lowest priority in PATH: /bin
$ echo "Everything except lowest priority: ${PATH%:*}"
Everything except lowest priority: /usr/local/bin:/usr/bin
$ echo "Highest priority in PATH: ${PATH%%:*}"
Highest priority in PATH: /usr/local/bin

Substitution in parameter expansion

The same types of patterns are used for substitution in parameter expansion. Substitution is introduced with the / or // operators, followed by two arguments separated by another / representing the pattern and the string to substitute. The pattern matching is always greedy, so the doubled version of the operator, in this case, causes all matches of the pattern to be replaced in the variable's expansion, while the singleton version replaces only the leftmost.

var="free and open"
parameter expansion pattern of [[:space:]]
string of _
${var/pattern/string} free_and open
${var//pattern/string} free_and_open

The wealth of parameter expansion modifiers transforms Bash variables and other parameters into powerful tools beyond simple value stores. At the very least, it is important to understand how parameter expansion works when reading Bash scripts, but I suspect that not unlike myself, many of you will enjoy the conciseness and expressiveness that these expansion modifiers bring to your scripts as well as your interactive sessions.

Topics

About the author

james pannacciulli
James Pannacciulli - James Pannacciulli is an advocate for software freedom & user autonomy with an MA in Linguistics. Employed as a Systems Engineer in Los Angeles, in his free time he occasionally gives talks on bash usage at various conferences. James likes his beers sour and his nettles stinging. More from James may be found on his home page.