Shell programming tidbits

From Crashcourse Wiki

Jump to: navigation, search


[edit] Overview

Just some random thoughts, observations and recommendations for writing shell scripts. No rhyme or reason. If you have any cute suggestions, drop me a note at

[edit] Keeping your scripts POSIX-compliant

For maximum portability, if you can, avoid shell-specific features (eg., "bash-isms") and stick with POSIX compliant programming. Either deliberately program using POSIX-only constructs, or start your script with:

#!/bin/sh --posix

(In fact, even that doesn't guarantee absolute POSIX compliance, but it gets you a lot closer.)

Also, the O'Reilly book "Classic Shell Scripting" covers general POSIX scripting, so it's an excellent book to work from.

[edit] String comparisons

You've undoubtedly seen weird string comparisons of the form:

if [ "X${VAR}X" = "XfredX" ] ; then ...

where the "X"s (or some variation thereof) are used to avoid comparing empty strings which can cause the shell to explode. However, with any recent shell, using surrounding quotes is sufficient:

if [ "${VAR}" = "fred" ] ; then ...

Once you have the quotes protecting things, there's no need to add any funny characters. They don't hurt, of course; they're just superfluous.

[edit] Validating a string against a set of possible, valid strings

Frequently, you'll need to check that a shell option value is specifically a member of a possible list of values, and without using arrays, here's a simple solution. First, a general routine that checks for list membership, where a list is defined as a whitespace-separated, multi-word string, and the loop below simply walks through each word in that string:

function is_element_of {
        for validelt in $2 ; do
                [ $testelt = $validelt ] && return 0
        return 1

Having defined that, you can now define variable-specific functions that use that. For example, here's a function that validates a flavour:

VALID_FLAVOURS="chocolate vanilla spam parrot"

function is_valid_flavour {
        if is_element_of $1 "${VALID_FLAVOURS}" ] ; then
                return 0
                return 1

At that point, later in the program, you can simply test:

if is_valid_flavour "${VAR}" ; then ...

Note that this restricts you to possible valid values with no embedded whitespace but, for most people, that's all they need. Obviously, for each set of values you want to validate, you'll define another list of values and the helper function for it.

[edit] Single-line conditionals

If you have a really short conditional, you can alwa ys code it like this:

[ -n "$VAR" ] && what to do if the test is true
[ -n "$VAR" ] || what to do if the test is false

That's just a simpler way of writing:

if [ -n "$VAR ] ; then
  what to do if it's true

You get the idea.

[edit] Command tests

As most folks know, a standard Linux command will return zero if it "worked"; that is, if it was "successful" under whatever definition of successful is being used, and will return some non-zero value based on whatever error code the command is trying to return based on how many different errors it wants to distinguish between.

If you simply want to test for success or failure, the general syntax is:

if <commmand> ; then ...
if ! <command> ; then ...

The first form obviously represents success, the second form failure of any type. If you want to distinguish between the different possibilities for failure, then you should of course examine the returned error code and test based on that.

[edit] Use meaningful function names for Boolean functions

One of my pet peeves. If you write a Boolean function, don't call it something annoyingly vague like, say, validate_flavour, since that forces you to write stilted code like:

if validate_flavour ${FLAV} ; then ...

Look how much nicer the following looks:

if is_valid_flavour ${FLAV} ; then ...

[edit] Use pattern-matching operators for simple pattern extraction

Too many people, when faced with a simple pattern matching and extraction operation, immediately leap to using sed or awk when the shell's builtin pattern matching operators might do the job. The four possibilities:

${VAR#pattern}              # remove shortest matching prefix 
${VAR##pattern}             # remove longest matching prefix
${VAR%pattern}              # remove shortest matching suffix
${VAR%%pattern}             # remove longest matching suffix

What the above operators do is take the contents of a given variable, and remove either a leading or trailing pattern of your choice, the final value being what's left after the deletion. Got that? Some examples:

$ THINGS="fred:barney:wilma:betty"

$ echo ${THINGS#*:}        # remove shortest prefix matching "*:"
$ echo ${THINGS##*:}       # remove longest prefix matching "*:"
$ echo ${THINGS%:*}        # remove shortest suffix matching ":*"
$ echo ${THINGS%%:*}       # remove longest suffix matching ":*"

If you have trouble remembering which variation means what, here's the mnemonic:

  • Since you normally see the "#" character at the front of a string, it strips prefixes.
  • Since you normally see the "%" character at the end of a string, it strips suffixes.

Also, if there's only one of those characters, the shortest is stripped; if there are two, the longest is stripped. I'm sure you can see how this feature can be used before resorting to expensive external commands.

Personal tools