Bash Scripting Cheat Sheet
Below is my bash scripting bible. I keep this is in a large Evernote and often refer to it when I need to remember a piece of syntax. This cheat sheet is extremely useful if you already have a basic knowledge around Bash, but it will not be much help if you have never used the language before.
The best way to use this post is to read through the whole thing so you roughly know its contents, this way you can ‘pick things off the shelf ' that you need later without having to remember everything there is to know about Bash.
So without further ado, if you are someone with at least some Bash experience and need a quick reference guide / cheat sheet for your scripting then look no further!
Key:
RED: Command Line Inputs | PURPLE: Bash Scripts | ORANGE: Output | BLUE: Comments | GREY/GREY: Contents Quick Look
Using Quick Look:
Each parent chapter in the contents table has a quick look in grey next to it. Use this with ctrl + f to quickly skip to that chapter. Additionally skip to the child chapters by adding the child number to parent quick look eg. 'lookexample1.5'
contents: (lookcont)
--- Basics
1. Assign Variables and Using Variables (lookassign)
1.1 Simple variable assigning (how to assign)
1.2 Importance of single and double quotes
1.3 Using variables in a script example
1.4 Assign command output to a variable
1.5 Special Variables
1.51 Positional Variables (for command line args)
2. IF Statements (lookif)
2.1 Structure of an if statement
2.2 Case statements as an alternative
3. Operators (lookop)
3.1 Integer Operators
3.11 Working With Decimals
3.2 String Operators
3.3 Parentheses
4. Loops (lookloop)
4.1 Structure of For Loops
4.2 For Loop Examples
5. User Input (lookui)
5.1 Read input into script (ask for input)
6. Command Line Arguments (Parameters) (lookcla)
6.1 Manipulating Arguments / Parameters
7. Input Output - STDIN STDOUT STDERR (lookinp)
7.1 File Descriptors and Redirectors
7.2 Simple output / input redirect examples
7.3 More input / output syntax
7.4 Discarding Output
8. Functions (lookfun)
8.1 Structure of a Function
8.2 Using Functions
9. Logging (looklog)
9.1 Using logger
10. getopts (implement options in your scripts) (lookget)
Bash Scripting basics
Every Bash script stars with a shebang: #!/bin/bash
It is the absolute path to the Bash interpreter and ensures that Bash will be used to interpret the script.
1. Assign Variables and Using Variables in Bash lookassign
1.1 Simple variable assigning lookassign1.1
Assign variable: WORD='script'
print variable: echo "$WORD"
print variable alternate syntax: echo "${WORD}"
# All upper case for variables is considered best practice but not required.
1.2 Variables & Importance of single and double quotes lookassign1.2
1.
WORD='cheese'
echo "$WORD" # Note the double quotes used here. The $ tells the script the following characters represent a variable.
-
Output: cheese
-
2.
WORD='cheese'
echo '$WORD' # Note the single quotes used here
-
Output: $WORD
-
In example 1 "$WORD" is in double quotes so the interpreter knows to expand the contents. It then can interpret the '$' to know that WORD represents a variable.
In example 2 '$WORD' is in single quotes so the interpreter uses the literal meaning and just prints $WORD.
Double quotes tell the interpreter to expand everything inside them and look for meaning beyond literal.
Single quotes tell the interpreter to treat everything inside as literal and not look for any other values assigned to the words / characters.
1.3 Using Variables in scripts: lookassign1.3
# Display the text 'Hello'
echo 'Hello'
# Assign a value to a variable. No spaces before or after = sign.
WORD='script'
# combine the variable with hard-coded text.
echo "This is a shell $WORD"
Output: This is a shell script
# Alternative syntax to display contents of variable
echo "This is a shell ${WORD}"
Output: This is a shell script
# Append text to the variable. Note that if you need to use the curly brace syntax in this instance
echo "${WORD}ing is fun!"
Output: scripting is fun!
# Create a new variable
ENDING='ed'
# Combine two variables
echo "This is ${WORD}${ENDING}"
Output: This is scripted
# Change value stored in ENDING variable (reassignment)
ENDING='ing'
echo "${WORD}${ENDING} is fun!"
Output: scripting is fun!
Here we have changed ENDING from 'ed' to 'ing' through reassignment.
1.4 Assign a command output to a variable lookassign1.4
in this example we will use id -un which prints the current users username.
In a script to assign the output of this to a variable we can use:
USER_NAME=$(id -un) # The $( command ) syntax is explained under section 3.3
or
USER_NAME='id -un' # Note that this is the older version that is now less used
1.5 Special Variables (Special Parameters) lookassign1.5
Note that these are all found in the Man Page for Bash under 'Special Parameters'
execute man bash and then search for special parameters.
${?} This stores exit status of the most recently run command.
So in pseudocode:
variable=$(command1) # Assigns output of running command1 to the variable
if [[ ${?} -ne 0 ]] # If ${?} doesn't equal 0 then command1 did not execute correctly and exited in error.
Then
echo 'ERROR' # Because command1 exited in error now print ERROR to terminal.
The syntax for if statements can be found under section 2.1
1.51 Positional parameters (command line arguments): (lookassign1.51)
${0} Contains the first command line argument (the name of the script being executed)
eg. If command line input is: ./scriptname apples then within the script ${0} equates to ./scriptname
${1} Similar to above this represents the command line argument at 1 in the array so the second argument from the command line.
In the ./scriptname apples example ${1} would equate to apples
${#} Equates to the total number of command line arguments entered. If only the script is given then it will equate to 0.
so if ./scriptname is entered then ${#} will equal 0. If ./scriptname apples is entered ${#} will equal 1.
${@} Equates to all of the command line arguments given 1 by 1 (when encased in ""). Useful in for loops.
- So command input is ./scriptname apples bananas cherries
- For FRUIT in "${@}"
- do
- echo "Yum ${FRUIT}!"
- done
output would be: Yum apples! \n Yum bananas! \n Yum Cherries! (Where \n equals a new line)
${*} This treats all command line arguments as one item. Where ${@} will treat./scriptname apples bananas cherries
as "apples", "bananas", "cherries" ${*} will treat them as "apples bananas cherries".
2. IF Statements lookif
2.1 Structure of IF Statement lookif2.1
In the example below ${UID} is a variable built into bash representing User ID Number
# Display if the user is the root user or not
if [[ "${UID}" -eq 0 ]] # Note all the spaces used as they are necessary. -eq is akin to ==. [[ ]] are necessary.
then # 'then' is needed after 'if' for if the statement is true
echo 'You are root.'
else
echo 'You are not root.'
fi # fi is used to end an if statement
You don't need to use 'else' statements if you only want to do something when the if statement is true.
eg. you wouldn't need to say " else 'do nothing' ". Just the leave the else statement out entirely.
You can also put the if then on one line:
if [[ ${x} -eq 0]]; then
echo "X equals 0"
# Note the semicolon to delineate the end of a line in this case.
Can also use single square brackets:
if [ ${x} -eq 0 ]; then
echo "X equals 0"
# The difference here being that [ is synonymous with test command, [[]] can do more but [ covers a lot.
2.1 Case Statements lookif2.2
These are very useful for when you want to test multiple cases and have different outputs dependents on which one is correct.
For reference in the below ${1} syntax is explained under positional parameters in section 1.51
Case Statement Structure (No notes):
case "${1}" in
start)
echo 'starting'
;;
stop)
echo 'stopping'
;;
status|state)
echo 'status'
;;
*)
echo 'supply a valid option' >&2
exit 1
;;
esac
Case Statement Structure (With Notes):
case "${1}" in
start) # The case you are looking for eg. does ${1} = 'start'
echo 'starting' # If ${1} = 'start' then do this command
;; # Syntax for ending each case (this is the end of what to do if ${1} = start)
stop) # The first test case failed ${1} doesn't = start. So now we test if ${1}=stop
echo 'stopping'
;;
status|state) # Use a pipe | to use an 'or' so if ${1} = status or state
echo 'status'
;;
*) # This will match anything and in this example is used as like an else statement
echo 'supply a valid option' >&2
exit 1
;;
esac # esac ends the case entirely similar to how fi ends if statements
This is extremely useful for checking the many different options of user input that your script may have to react to.
3. Operators and Parentheses lookop
All operators can be found by inputting help test on the command line. This will include some quite niche comparisons.
3.1 Integer Operators lookop3.1
Comparison operators
-eq This is used for equals (=) comparisons on integers
-lt This is used for less than (<) comparisons on integers
Arithmetic operators
+ Addition
- Subtraction
/ Integer Division (need another tool for floating division)
* Multiplication
% Modulo (remainder)
3.11 Working with decimals: lookop3.11
In order to do integer division you can use bc:
echo "${equation}" | bc
bc will print result to screen. If result is a decimal bc will also include the decimals numbers.
Set number of decimals:
echo "scale = 3; ${equation} | bc
This will set this number of decimal places to 3.
However scale itself won't round:
if your result is 1.2345 then this to 3 decimal places would be 1.235 however scale just cuts it at 3 so your out put will be 1.234 which is not mathematically correct.
Round to correct decimal place:
Use the printf command this will round to the correct number of decimals as below:
printf "%.3f" 1.2345
Output to screen: 1.235
To use printf with echo and bc above: printf "%.3f" $( echo "scale=4; ${equation}" | bc )
# Notice that to get 3 decimal places we use scale=4 to get four decimal places and then printf "%.3f" to round that to 3 decimals.
3.2 String Operators lookop3.2
string1 = string2 and string1 == string2 - The equality operator returns true if the strings are equal.
string1 != string2 - The inequality operator returns true if the strings are not equal.
string1 > string2 - The greater than operator returns true if the left string is greater than the right sorted by lexicographical (alphabetical) order.
string1 < string2 - The less than operator returns true if the right string is greater than the right sorted by lexicographical (alphabetical) order.
-z string - True if the string length is zero.
-n string - True if the string length is non-zero.
3.3 Parentheses lookop3.3
If you find this section confusing. See this in depth article on the topic of Parentheses: Bash Brackets Quick Ref
Single Parentheses (run commands): -
( Single Parentheses ) - Runs commands in a subshell. Variables declared or environment changes made will be cleaned up and disappear.
Even though you can run multiple commands it will produce only one Exit Code for the whole bracket.
eg.
a='This string'
( a=banana; mkdir $a ) # This subshell reassigns 'a' to banana and then makes a directory of name 'a'
echo $a
ls
echo Output: 'This string' # Variable 'a' reassignment was done in single parentheses so has no affect outside of them. Hence 'a' still equals 'This String'
ls Output: ... # Just represents the other directories which will appear when you use ls
ls Output: banana/ # The command in single parentheses created a directory called 'banana', which has still been created.
(single parentheses also have a use in bash arrays see linked article under 3.3 for explanation)
You cannot use single parentheses to assign a variable - You would need to use dollar sign single parentheses (below)
Dollar Sign Single Parentheses (Run commands with output)
$( Single Parentheses ) Use this when you want to assign the output of a command to a string variable.
Example:
NAME="$( whoami )" # whoami is a command that outputs the current users ID.
echo "$NAME"
Output: Tommy
You could also combine the command with a string: NAME="My name is $( whoami )"
Double Parentheses ( integer arithmetic): -
(( Double Parentheses )) This is for use in integer arithmetic. You can perform assignments, logical operations, and mathematic operations like multiplication or modulo inside these parentheses. There is no output so it can't be piped but variable changes made inside them will stick (unlike single parentheses)
eg.
i=4
(( ${i} += 3 ))
echo ${i}
Output: 7
(( 4 + 8 ))
Output: No Output
# You can't use it to assign a variable
a=(( 4 + 1 ))
Output: bash: syntax error near unexpected token '('
# Increment a variable
NUM='1'
(( NUM++))
This will not output anything but it will alter the variable.
Dollar Sign Double Parentheses (integer arithmetic with output)
$(( Double Parentheses )) Use this if you want to assign the output of arithmetic to a string variable.
An example is as below:
NUM='10'
NEWNUM=$(( NUM + 5 ))
Without the $ first then there will be no output from the brackets so a variable can't be assigned to it as it would trying to assign the variable to nothing.
echo NEWNUM
Output: 15
4. Loops lookloop
4.1 For Loop structure lookloop4.1
Typical structure is as seen below. Note the do and done
- For FRUIT in apples, bananas, oranges
- do
- echo "Yum ${FRUIT}!"
- done
Of particular use here when dealing with command line arguments is the Special Parameter ${@}
This will allow you to loop command line arguments see section 1.5 for more on this.
4.2 For Loop Example lookloop4.2
# Simple print numbers 1 to 10
for i in {1..10}
do
echo "${i}"
done
If statement in a loop:
#Use a Range to display odd numbers from 1-100 (using modulo, mod, %)
for i in {1..100}
do
if [[ $((${i} % 2)) != 0 ]] # Note the $(()) notation in which to calculate arithmetic operations and output
then
echo "${i}"
fi
done
4.3 While Loop Structure lookloop4.3
Typical while loop structure is as seen below. Note the do and done.
while [[ X -gt 0 ]]
do
echo "X"
((X-=1))
done
# This while loop would decrement the X variable until X < 0
5 User Input lookui
5.1 Read Input into bash script lookui5.1
You can use read to ask for user input.
The simplest version of this is to use the read command followed by the variable you want to read the data into:
read CHEESE
This will read the next thing the user types into the variable 'CHEESE'. The rest of the script will not execute until the user has inputted.
Using a prompt:
#!/bin/bash
read -p 'Whats your name: ' varname # -p option will allow you to use a prompt
echo "Welcome ${name}"
# The above assigns varname to whatever the use inputs
Without -p prompt
echo 'Whats your name?' # Without prompt echo the question you want answered before you use read to get input
read varname
-s will make the input silent
read -sp 'Whats your password?' varpass
This will make it similar to when you enter your password for sudo and nothing types or appears on screen.
Read multiple lines (eg. from a file)
while read line;
do
echo ${line}
done
Note that line is just a variable name, can be anything.
Read last line of file if the last line is not being read due to a lack of 'newline' at end of file:
while read line || [ -n "$line" ];
do ...
6. Command Line Arguments (lookcla)
Important to note that on the command line they are called arguments, once in the script they are called parameters.
See Section 5.51 for Positional Variables that enable you to finely use arguments.
6.1 Manipulating Command Line Arguments (lookcla6.1)
shift # Use shift to move all the parameters to the left (optional argument of a number for how far to shift).
If your parameters are apple, banana, orange using shift will make your parameters banana, orange. This moves ${3} -> ${2} -> ${1} -> Binned
shift takes an optional argument of a number (defaults to 1 if none given). The number given is how far you want to shift to the left.
eg. shift 2 For apple, banana, orange again will shift the parameters left twice making ${1} now equate to orange (apple and banana are discarded).
7 . Input & Output STDIN, STDOUT, STDERR lookinp
7.1 File Descriptors and Redirectors lookinp7.1
There are 3 File Descriptors (FD) :
0 = Standard Input
1 = Standard Output
2 = Standard Error
There is also another symbol used:
& = Both Standard Output and Standard Error. For when you want to redirect both to the same location.
Redirection symbols:
> = Redirect STDOUT or STDERR, with no FD this defaults to STDOUT (2> for STDERR)
>> = Redirect STDOUT or STDERR and append to file (> will clear file and replace)
< = Redirect STDIN
These FDs can be used in pipelines and commands to redirect inputs and outputs:
7.2 Simple output / input redirect examples lookinp7.2
# Redirect STDOUT to a file
FILE="/tmp/data"
head -n1 /etc/passwd > ${FILE} # If file doesn't already exist this will create the file.
If the file exists this will overwrite it
# Redirect standard input to a program
read LINE < ${FILE}
echo "LINE contains: ${LINE}"
# Redirect STDOUT to a file, appending to the file (no overwrite)
head -n1 /etc/passwd >> ${FILE}
# Redirect STDERR to a file using FD 2
ERR_FILE="/tmp/data.err"
head -n3 /etc/passwd /fakefile 2> ${ERR_FILE} # Note that 2>> could be used here to append to file.
7.3 More input / output syntax lookinp7.3
# Redirect STDOUT and STDERR to a file
head -n3 /etc/passwd /fakefile &> ${FILE}
The &> descriptor will redirect both outputs to a given destination. Note that &>> is needed in order to append.
# Redirect STDOUT and STDERR through a pipe
head -n3 /etc/passwd /fakefile |& cat -n
The |& will pass through both STDOUT and STDERR to the next command in the pipe.
# Send output to STDERR
echo "This is STDERR!" >&2
7.4 Discarding Output lookinp7.4
A note /dev/null is the null device also known as the bitbucket and can be used to discard of unwanted output streams.
This will prevent outputs being printed to terminal.
# Discard STDOUT
head -n3 /etc/passwd /fakefile > /dev/null
# Discard STDERR
head -n3 /etc/passwd /fakefile 2> /dev/null
# Discard Both
head -n3 /etc/passwd /fakefile &> /dev/null
8. Functions lookfun
8.1 Structure of a Bash Function lookfun8.1
There are two ways to structure a function:
# 1. Bracket syntax, put () after the name of the function
myfunction() {
echo "This is my function"
}
# 2. Put the word function before the name of your function. No brackets required
function myfunction {
echo "This is my function"
}
# 3. Variables in functions
my function () {
MYVAR="hello"
local MYVAR2="goodbye"
echo "${MYVAR}" "${MYVAR2}"
}
Unless the local command is used to declare a variable then that variable will be global to the whole script. In the above example MYVAR is global and would be present outside the function once it is declared, MYVAR2 is local to the function and would have no bearing on the wider script once it is declared in the function.
8.2 Using Functions lookfun8.2
# Function that echoes the first argument passed to it
myfunction2() {
echo "${1}"
}
# Calling myfunction2
myfunction2 'Hello!'
output: Hello!
Note that in a function ${1} represents the first argument/parameter passed to the function. It does not represent the first parameter for the entire script.
As can be seen above, to pass arguments to a function you do not put them in brackets like Python. Instead you just pass them with a space after calling the function, similar to how a command line argument works.
Function exit status
You can check the exit status of a function in the same way you would check the exit status of a command or pipeline:
# Call function
myfunction2 'Hello'
--
# Check functions exit status
if [[ "${?}" -eq 0 ]]; then # Remember that {?} represents the exit status of the last command / function run
echo 'myfunction2 succeeded' > > log.txt
else
echo 'myfunction2 failed >> log.txt
fi
--
# You can give a function a non-zero exit status by using return
backup_file() {
local FILE="${1}"
if [[ -f "${FILE}" ]]; then
local BACKUP_FILE="/var/tmp/$(basename ${FILE}).$(date)"
# The exit status of the function provided the file existed to be backed up will be exit status of cp (copy)
cp -p ${FILE} ${BACKUP_FILE}
else
# File does not exist so return non-zero exit status
return 1
fi
Here the exit status will be 0 if cp is called to move the file. However if the if statement determines the file does not exist then cp will not be called and the function will return an exit status of 1.
9. Logging looklog
See this article on logging best practices
9.1 Using logger looklog9.1
Logger makes entries into the system log. This is the var/log/syslog file
An easy way to understand how this works is to see it used in a function:
# This function is used to log within a script called luserdemo10.sh
log() {
# Sends a message to syslog and STDOUT if VERBOSE is true
local MESSAGE="${@}" # Sets MESSAGE to be all arguments passed to log function
logger -t luserdemo10.sh "${MESSAGE}" # logs MESSAGE in the var/log/syslog file
}
At the end of this function logger -t is used. The -t option specifies a tag to be used along with the logmessage. Without -t this defaults to the username of the user executing the command/script. In this case we have set it to be the name of the script luserdemo10.sh so when looking at the log we can see which logs apply specifically to this script.
logger output:
to see the output of logger you use sudo var/log/messages
Here you could find the line applying to the script in the above section provided it has been run:
Jan 9 08:02:10 localusers luserdemo10.sh: File backup succeeded
Notice how luserdemo10.sh is included, this is because we chose that as the -t tag in the log function.
Without that -t tag it would default to:
Jan 9 08:02:10 localusers tommyj: File backup succeeded
It defaults to tommyj as I was the user that ran the script/command
10. getopts lookget
getopts allows you to specify options to be used with your script in the same way options are used in commands.
getopts needs to be used in a while loop, the structure is as below:
while getopts vl:s OPTION
Here we are saying to getopts 'v' 'l' and 's'are options for this script. The : after l tells getopts that l requires an argument.
For instance if l is the option for LENGTH then perhaps l requires and argument after saying how long length should be.
This would look like ./myscript -l 20 setting l or LENGTH to 20.
OPTION represents all the options passed to the script when it is run. Getopts pulls these from the command line execution of the script.
Eg. It finds the options from within the arguments given when the script is run. eg. ./myscript -vs
v and s are the options here that getopts would find.
The structure of the while loop is as below and uses case statements (more on case statements in Section 2.2).
while getopts vl:s OPTION
do
case ${OPTION} in
v)
VERBOSE='true'
display 'Verbose mode on.'
;;
l)
LENGTH="${OPTARG}" # OPTARG is a special variable representing that options argument
;;
s)
USE_SPECIAL_CHAR='true'
;;
?)
"Statement on how to use script" # Anything that isn't an option
;;
esac
In this we can see how we combine case statements, while loops and getopts. getopts isn't magic it doesn't know what -v does all it does is enable you to look for it and then set something to interact with the rest of your script if that option is found. In the above example if l is found then LENGTH is set to be the argument that follows l. This allows users to customise how your script works with options you give them.
OPTIND (represents number of options given)
OPTIND is a special variable when using getopts that is set to the number of options given by the user of the script.
A very good use case for this is shifting all of the options out of the parameter array after getopts has been called to ascertain which options are present.
OPTIND counts the options -vs their arguments l 20 and the script call ./myscript itself. So if you want to shift all of the options out you use OPTIND - 1 as shift doesn't count the script call as an argument itself.
./myscript -vs # This then has an OPTIND of 2 as there are 2 separate arguments the script call and -vs.
./myscript -v -s # This then has an OPTIND of 3 as there are 3 separate arguments the script call and -v and -s.
Even though in both the above examples only two options are used the OPTIND changes on whether they count as separate arguments based on if there are spaces between them.
./myscript -vs extra stuff # Here the OPTIND is 2 (only counts options and script call nothing after)
currently ${1] will represent '-vs'.
${2} will represent 'extra'
${3} will represent 'stuff'
and ${@} will represent '-vs extra stuff'
If we have used getopts to establish which options to enable in the script We will now only want the non-option parameters. So we can shift away (discard) the options:
shift "$(( OPTIND - 1 ))" # Remember we use -1 because OPTIND counts the script call but shift does not.
Now ${1} will represent 'extra'
${2} will represent 'stuff'
${@} will represent 'extra stuff'