🥇Best practices for bash scripts: a concise guide to reliable and efficient bash scripts

Shell wallpaper by manapi

Debugging bash scripts is like searching for a needle in a haystack, especially when new additions appear in an existing codebase without timely consideration of structure, logging, and reliability. In such situations, one can find themselves struggling due to both personal mistakes and the management of complex script entanglements.

The command Mail.ru Cloud Solutions translated the article with recommendations that will help you write, debug, and maintain your scripts better. Believe it or not, nothing compares to the satisfaction of writing clean, ready-to-use bash code that works every time.

In the article, the author shares what they've learned over the past few years, as well as some common mistakes that caught them off guard. This is important because every software developer, at some point in their career, works with scripts for automating routine tasks.

Trap handlers

Most bash scripts I've encountered never used an effective cleanup mechanism when something unexpected happens during script execution.

Unexpected events can arise from external sources, such as receiving a signal from the kernel. Handling such cases is crucial for ensuring that scripts are reliable enough to run in production systems. I often use exit handlers to respond to such scenarios:

function handle_exit() {
  // Add cleanup code here
  // for eg. rm -f "/tmp/${lock_file}.lock"
  // exit with an appropriate status code
}
  
// trap  
trap handle_exit 0 SIGHUP SIGINT SIGQUIT SIGABRT SIGTERM

trap — is a built-in shell command that helps you register a cleanup function to be called in case of any signals. However, special care should be taken with handlers such as SIGINT, which triggers an interruption of the script.

Moreover, in most cases, you should only catch EXIT, but the idea is that you can customize the script's behavior for each individual signal.

Built-in set functions — quick termination on error

It is very important to react to errors as soon as they occur and to halt execution quickly. Nothing is worse than continuing to execute a command like this:

rm -rf ${directory_name}/*

Note that the variable directory_name is not defined.

To handle such scenarios, it is important to use built-in functions set, such as set -o errexit, set -o pipefail or set -o nounset at the beginning of the script. These functions ensure that your script will terminate as soon as it encounters any non-zero exit code, usage of undefined variables, incorrect commands passed through a pipeline, and so on:

#!/usr/bin/env bash

set -o errexit
set -o nounset
set -o pipefail

function print_var() {
  echo "${var_value}"
}

print_var

$ ./sample.sh
./sample.sh: line 8: var_value: unbound variable

Note: built-in functions, such as set -o errexit, will exit the script as soon as an 'unhandled' return code (other than zero) appears. Therefore, it's better to implement custom error handling, for example:

#!/bin/bash
error_exit() {
  line=$1
  shift 1
  echo "ERROR: non zero return code from line: $line -- $@"
  exit 1
}
a=0
let a++ || error_exit "$LINENO" "let operation returned non 0 code"
echo "you will never see me"
# run it, now we have useful debugging output
$ bash foo.sh
ERROR: non zero return code from line: 9 -- let operation returned non 0 code

This style of scripting prompts you to pay closer attention to the behavior of all commands in the script and to anticipate the possibility of errors before they catch you off guard.

ShellCheck for identifying errors during development

It's worth integrating something like ShellCheck into your development and testing pipelines to check your bash code for adherence to best practices.

I use it in my local development environments to receive reports on syntax, semantics, and some errors in the code that I might have missed during development. It is a static analysis tool for your bash scripts, and I strongly recommend its usage.

Using your own exit codes

Return codes in POSIX are not just zero or one, but zero or a non-zero value. Use these capabilities to return custom error codes (between 201-254) for various error cases.

This information can then be used by other scripts that wrap yours to accurately understand what type of error occurred and respond accordingly:

#!/usr/bin/env bash

SUCCESS=0
FILE_NOT_FOUND=240
DOWNLOAD_FAILED=241

function read_file() {
  if ${file_not_found}; then
    return ${FILE_NOT_FOUND}
  fi
}

Note: please be particularly careful with variable names that you define to avoid accidentally overwriting environment variables.

Logging functions

A well-structured logging system is essential for easily understanding the results of your script’s execution. As in other high-level programming languages, I always use my own logging functions in my bash scripts, such as __msg_info, __msg_error and so on.

This helps ensure a standardized logging structure by making changes in just one place:

#!/usr/bin/env bash

function __msg_error() {
    [[ "${ERROR}" == "1" ]] && echo -e "[ERROR]: $*"
}

function __msg_debug() {
    [[ "${DEBUG}" == "1" ]] && echo -e "[DEBUG]: $*"
}

function __msg_info() {
    [[ "${INFO}" == "1" ]] && echo -e "[INFO]: $*"
}

__msg_error "File could not be found. Cannot proceed"

__msg_debug "Starting script execution with 276MB of available RAM"

I usually try to include some mechanism in my scripts __init, where such logger variables and other system variables are initialized or set to default values. These variables can also be set from command line parameters during the script call.

For example, something like:

$ ./run-script.sh --debug

When such a script is executed, it is guaranteed that the system settings are set to default values if required or at least initialized with something appropriate if needed.

I usually base my choice of what to initialize and what not to on a compromise between user interface and configuration details that the user may/must understand.

Architecture for reusability and a clean system state

Modular/reusable code

├── framework
│   ├── common
│   │   ├── loggers.sh
│   │   ├── mail_reports.sh
│   │   └── slack_reports.sh
│   └── daily_database_operation.sh

I maintain a separate repository that can be used to initialize a new bash project/script that I want to develop. Everything that can be reused can be stored in the repository and retrieved in other projects that wish to use such functionalities. This organization of projects significantly reduces the size of other scripts and also ensures that the codebase is small and easily testable.

As in the example above, all logging functions, such as __msg_info, __msg_error and others, like Slack reports, are kept separately in common/* and dynamically included in other scenarios, like daily_database_operation.sh.

Leave a clean system behind

If you are loading any resources during script execution, it is advisable to store all such data in a common directory with a random name, for example /tmp/AlRhYbD97/*You can use random text generators to choose a directory name:

rand_dir_name="$(cat /dev/urandom | tr -dc 'a-zA-Z0-9' | fold -w 16 | head -n 1)"

After the process is finished, cleanup of such directories can be handled in the trap handlers discussed above. If the temporary directories are not addressed, they accumulate and eventually cause unexpected issues on the host, such as a full disk.

Using lock files

Often, it is necessary to ensure that only one instance of a script runs on the host at any given time. This can be achieved using lock files.

I usually create lock files in /tmp/project_name/*.lock and check their existence at the beginning of the script. This helps to properly terminate the script and avoid unexpected changes in the system state by another concurrently running script. Lock files are not needed if you need the same script to run concurrently on the host.

Measure and improve

We often have to work with scripts that run for extended periods, such as daily database operations. These operations typically include a sequence of steps: data loading, anomaly checking, data importing, status reporting, and so on.

In such cases, I always try to break down the script into separate small scripts and report their status and execution time using:

time source "${filepath}" "${args}" >> "${LOG_DIR}/RUN_LOG" 2>&1

Later, I can check the execution time using:

tac "${LOG_DIR}/RUN_LOG.txt" | grep -m1 "real"

This helps me identify problematic/slow areas in the scripts that need optimization.

Good luck!

What else to read:

Source: habr.com

Best Practices for Bash Scripting: A Concise Guide to Reliable and Efficient Bash Scripts