Bash Tricks

On this page, I collect useful code fragments that needed some time to think out ;-)

Variables from a subshell

A nearly unsolvable problem: programs run in a subshell or a pipe, and you'd like to have their exitcodes in front of the pipe. Doesn't work? It works! With the following solution you have the exitcodes available as variables. Beside exitcodes you can put any other data in variables, too.

I start from the following command line:

grep Mist /var/log/messages | tail -n 10 | cut -d' ' -f3-

This results in the following script:

exec 3>&1 # open third output channel (for STDOUT)
exec 4>&1 # open fourth output channel (for exitcodes)
eval ` # >-- backtick opened!
{
    {
        grep Mist /var/log/messages
        echo "grepexit=$?" >&4;
    } | {
        tail -n 10
        echo "tailexit=$?" >&4
        echo "foobar='This is a test.'" >&4
    } | cut -d' '  -f3-
} 4>&1 >&3 # redirection
` # >-- backtick closed!
echo "at the end:"
echo "grepexit $grepexit"
echo "tailexit $tailexit"
echo "foobar $foobar"

Script output:

at the end:
grepexit 1
tailexit 0
foobar This is a test.

• The curly brackets are a "command group" each (see Compound Commands in man bash.)
• The outside curly brackets (directly after opening the backticks until closing the backticks) take care for rotating the complete output of the code lying inbetween through the output channels.
• The inner bracket layers make it possible to execute more than one command inside the pipe.
• The variables to be exported get written to output channel 4 as a bash command. Before closing the backticks, STDOUT is redirected to channel 3 and the variables are redirected from channel 4 to STDOUT. After that, eval executes STDOUT and sets the variables. That's easy, isn't it? ;-)

By the way: My /var/log/messages does not contain Mist [german for crap] ;-)

Subshell again

The method above is too complicated? For simple cases there's a simple solution - just extend the subshell skilfully ;-)

The disadvantage of this method: Only the variables of the last subshell (after the pipe) are available. In the following example, only $counter is available, but not e. g. the exitcode of the command at the start of the pipe. Furthermore $counter is lost after closing the curly brackets.

The following example is a bash-internal replacement for wc -l.

counter=0 # initialize variable 

echo -e "a\nb\nc\nd" | {
#                    ^----- the subshell starts here
while read line ; do
    counter=$(($counter+1))
done
echo "counter (inside the subshell): $counter"
} # the subshell ends here!
echo "after the curly bracket: $counter"

script output:

counter (inside the subshell): 4
after the curly bracket: 0

As you can see, after closing the curly bracket $counter has the value it had at the very beginning.

Suggested readings: man bash - especially the notes about Compound Commands (extending the subshell) and Arithmetic Expansion (calculating functions).

Just to make it clear: The subshell is started with the pipe. The curly brackets are just a "command group", but don't start their own subshell.

From behind through the breast into the eye...

(this is a german saying, not sure if it about as funny when translated to english)

Alexander Dalloz has sent me a variant that puts the script upside down and puts the subshell behind the loop. This means that the relevant part runs in the main shell, the variables are available without problems.

counter=0
while read line; do
  counter=$(($counter+1))
done < <(echo -e "a\nb\nc\nd")
echo "counter: $counter"

... and again ;-)

Lars Ellenberg told me an easy way to get the exitcodes from a pipe:

false | true | false | true | false ; \
echo "${PIPESTATUS[*]}"
1 0 1 0 1

In a script, you can leave out the ;  at the end of the first line - it's just needed for testing in an interactive shell.

If you just need to recognice the failure of any command in the pipe, use set -o pipefail (hint by Torsten Foertsch):

{ true|false|true; } || echo failed
set -o pipefail
{ true|false|true; } || echo failed
failed

If you need more than the exitcodes (but arbitrary variables) you have to use my constructions above ;-)

last update: 17.1.2006