This article is by Ganesh Naik, the author of the book Learning Linux Shell Scripting, published by Packt Publication (https://www.packtpub.com/networking-and-servers/learning-linux-shell-scripting).

Whoever works with Linux will come across the Shell as first program to work with. The Graphical User Interface (GUI) usage has become very popular due to ease of use. Those who want to take advantage of the power of Linux will use the Shell program by default.

Shell is a program, which gives the user direct interaction with the operating system. Let’s understand the stages in the evolution of the Linux operating system. Linux was developed as a free and open source substitute for UNIX OS. The chronology can be as follows:

• The UNIX operating system was developed by Ken Thomson and Dennis Ritchie in 1969. It was released in 1970. They rewrote the UNIX using C language in 1972.
• In 1991, Linus Torvalds developed the Linux Kernel for the free operating system.

(For more resources related to this topic, see here.)

Comparison of shells

Initially, the UNIX OS used a shell program called Bourne Shell. Then eventually, many more shell programs were developed for different flavors of UNIX. The following is brief information about different shells:

• Sh: Bourne Shell
• Csh: C Shell
• Ksh: Korn Shell
• Tcsh: Enhanced C Shell
• Bash: GNU Bourne Again Shell
• Zsh: extension to Bash, Ksh, and Tcsh
• Pdksh: extension to KSH

A brief comparison of various shells is presented in the following table:

Feature	Bourne	C	TC	Korn	Bash
Aliases	no	yes	yes	yes	yes
Command-line editing	no	no	yes	yes	yes
Advanced pattern matching	no	no	no	yes	yes
Filename completion	no	yes	yes	yes	yes
Directory stacks (pushd and popd)	no	yes	yes	no	yes
History	no	yes	yes	yes	yes
Functions	yes	no	no	yes	yes
Key binding	no	no	yes	no	yes
Job control	no	yes	yes	yes	yes
Spelling correction	no	no	yes	no	yes
Prompt formatting	no	no	yes	no	yes

What we see here is that, generally, the syntax of all these shells are 95% similar.

Tasks done by shell

Whenever we type any text in the shell terminal, it is the responsibility of shell to execute the command properly. The activities done by shell are as follows:

• Reading text and parsing the entered command
• Evaluating meta-characters such as wildcards, special characters, or history characters
• Process io-redirection, pipes, and background processing
• Signal handling
• Initialize programs for execution

Working in shell

Let’s get started by opening the terminal, and we will familiarize ourselves with the Bash Shell environment.

1. Open the Linux terminal and type in:

   $ echo $SHELL
   /bin/bash

2. The preceding output in terminal says that current shell is /bin/bash such as BASH shell.

   $ bash --version
   GNU bash, version 2.05.0(1)-release (i386-redhat-linux-gnu)

Our first script – Hello World

We will now write our first shell script called hello.sh. You can use any editor of your choice such as vi, gedit, nano, and similar. I prefer to use the vi editor.

1. Create a new file hello.sh as follows:

   #!/bin/bash
   # This is comment line
   echo "Hello World"
   ls
   date

2. Save the newly created file.

The #!/bin/bash line is called the shebang line. The combination of the characters # and ! is called the magic number. The shell uses this to call the intended shell such as /bin/bash in this case. This should always be the first line in a shell script.

The next few lines in the shell script are self-explanatory:

• Any line starting with # will be treated as comment line. Exception to this would be the 1st line with #!/bin/bash
• echo command will print Hello World on screen
• ls will display directory content on console
• date command will show current date and time

We can execute the newly created file giving following command:

• Technique one:
  $ bash hello.sh
• Technique two:
  $ chmod +x hello.sh

By running the preceding command, we are adding executable permission to our newly created file. You will learn more about file permission in following sections of this same chapter.

$ ./hello.sh

By running the preceding command, we are executing hello.sh as executable file. By technique one, we passed filename as an argument to bash shell.

The output of executing hello.sh will be as follows:

Hello World
hello.sh
Sun Jan 18 22:53:06 IST 2015

Compiler and interpreter – difference in process

In any program development, the following are the two options:

• Compilation: Using a compiler-based language, such as C, C++, Java and other similar languages
• Interpreter: Using interpreter-based languages, such as Bash shell scripting

When we use compiler-based language, we compile the complete source code, and as a result of compilation, we get a binary executable file. We then execute the binary to check the performance of our program.

On the contrary, when we develop shell script, such as an interpreter-based program, every line of the program is input to bash shell. The lines of Shell Script are executed one by one sequentially. Even if the second line of a script has an error, the first line will be executed by the shell interpreter.

Command substitution

In keyboard, there is one interesting key backward quote such as `. This key is normally situated below the Esc key. If we place text between two successive back quotes, then echo will execute those as commands instead of processing them as plane text.

Alternate syntax for $(command) that uses the backtick character `:

$(command) or `command`

Let’s see an example:

$ echo "Today is date"

Output:

Today is date

Now modify the command as follows:

$ echo "Today is `date`"

Or:

$ echo "Today is $(date)"

Output:

Today is Fri Mar 20 15:55:58 IST 2015

Pipes

Pipes are used for inter-process communication:

$ command_1 | command_2

In this case, the output of command_1 will be send as input to command_2:

A simple example is as follows:

$ who | wc

The preceding simple command will be doing three different activities. First, it will copy output of who command to temporary file. Then, wc will read temporary file and will display the result. Finally, the temporary file will be deleted.

Understanding variables

Let’s learn about creating variables in shell.

Declaring variables in Linux is very easy. We just need to use variable names and initialize it with required content:

$ person="Ganesh Naik"

To get the content of variable, we need to prefix $ before variable:

$ echo person
person
$ echo $person
Ganesh Naik

The unset command can be used to delete a variable:

$ a=20
$ echo $a
$ unset a

Command unset will clear or remove the variable from shell environment as well:

$ person="Ganesh Naik"
$ echo $person
$ set

The set command will show all variables declared in shell.

Exporting variable

By using export command, we are making variables available in child process or subshell. But if we export variables in child process, the variables will not be available in parent process.

We can view environment variables by either of the following command:

$ env
$ printenv

Whenever we create a shell script and execute it, a new shell process is created and shell script runs in that process. Any exported variable values are available to the new shell or to any subprocess.

We can export any variable by either of the following:

$ export NAME
$ declare -x NAME

Interactive shell scripts – reading user input

The read command is a shell built-in command for reading data from a file or a keyboard.

The read command receives the input from a keyboard or a file, till it receives a newline character. Then it converts the newline character in null character.

1. Read a value and store it in the variable:

   read VARIABLE
   echo $VARIABLE

2. This will receive text from keyboard. The received text will be stored in shell variable VARIABLE.

Working with command-line arguments

Command line arguments are required for the following reasons:

• It informs the utility or command which file or group of files to process (reading/writing of files)
• Command line arguments tell the command/utility which option to use

Check the following command line:

$ my_program   arg1 arg2 arg3

If my_command is a bash shell script, then we can access every command line positional parameters inside the script as following:

$0 would contain "my_program"   # Command
$1 would contain "arg1"     # First parameter
$2 would contain "arg2"     # Second parameter
$3 would contain "arg3"     # Third parameter

Let’s create a param.sh script as follows:

#!/bin/bash
echo "Total number of parameters are = $#"
echo "Script name = $0"
echo "First Parameter is $1"
echo "Second Parameter is $2"
echo "Third Parameter is $3"
echo "Fourth Parameter is $4"
echo "Fifth Parameter is $5"
echo "All parameters are = $*"

Run the script as follows:

$ param.sh London Washington Delhi Dhaka Paris

Output:

Total number of parameters are = 5
Command is = ./hello.sh
First Parameter is London
Second Parameter is Washington
Third Parameter is Delhi
Fourth Parameter is Dhaka
Fifth Parameter is Paris
All parameters are = London Washington Delhi Dhaka Paris

Understanding set

Many a times we may not pass arguments on the command line; but, we may need to set parameters internally inside the script.

We can declare parameters by the set command as follows:

$ set USA Canada UK France
$ echo $1
USA
$ echo $2
Canada
$ echo $3
UK
$ echo $4
France

Working with arrays

Array is a list of variables. For example, we can create array FRUIT, which will contain many fruit names. The array does not have a limit on how many variables it may contain. It can contain any type of data. The first element in an array will have the index value as 0.

$ FRUITS=(Mango Banana Apple)
$ echo ${FRUITS[*]}
Mango Banana Apple
$ echo $FRUITS[*]
Mango[*]
$ echo ${FRUITS[2]}
Apple
$ FRUITS[3]=Orange
$ echo ${FRUITS[*]}
Mango Banana Apple Orange

Debugging – Tracing execution (option -x)

The -x option, short for xtrace, or execution trace, tells the shell to echo each command after performing the substitution steps. This will enable us to see the value of variables and commands.

We can trace execution of shell script as follows:

$ bash –x hello.sh

Instead of the preceding way, we can enable debugging by the following way also:

#!/bin/bash -x

Let’s test the earlier script as follows

$ bash –x hello.sh

Output:

$ bash –x hello.sh
+ echo Hello student
Hello student
++ date
+ echo The date is Fri May 1 00:18:52 IST 2015
The date is Fri May 1 00:18:52 IST 2015
+ echo Your home shell is /bin/bash
Your home shell is /bin/bash
+ echo Good-bye student
Good-bye student

Summary of various debugging options

Option	Description
-n	Syntax error checking. No commands execution.
-v	Runs in verbose mode. The shell will echo each command prior to executing the command.
-x	Tracing execution. The shell will echo each command after performing the substitution steps. Thus, we will see the value of variables and commands.

Checking exit status of commands

Automation using shell scripts involves checking, if the earlier command executed successfully or failed, if a file is present or not, and similar. You will learn various constructs such as if, case, and similar, where you will need to check certain conditions if they are true or false. Accordingly, our script should conditionally execute various commands.

1. Let’s enter the following command:

   $ ls

2. Using bash shell, we can check if the preceding command executed successfully or failed as follows:

   $ echo $?

The preceding command will return 0, if the command ls executed successfully. The result will be non-zero such as 1 or 2 or any other non-zero number, if the command has failed. Bash shell stores the status of the last command execution in the variable. If we need to check the status of the last command execution, then we should check the content of the variable.

Understanding test command

Let’s learn the following example to check the content or value of the expressions.

$ test $name = Ganesh
$ echo $?
0 if success and 1 if failure.

In the preceding example, we want to check if the content of the variable name is the same as Ganesh. To check this, we used the test command. The test will store the results of comparison in the variable.

We can use the following syntax for the preceding test command. In this case, we used [ ] instead of the test command. We enclosed the expression to be evaluated in square brackets.

$ [ $name = Ganesh ]     # Brackets replace the test command
$ echo $?
0

String comparison options for the test command

The following is the summary of various options for string comparison using test.

Test operator	Tests true if
-n string	True if the length of the string is non-zero.
-z string	True if the length of the string is zero
string1 != string2	True if the strings are not equal.
string1 == string2 string1 = string2	True if the strings are equal.
string1 > string2	True if string1 sorts after string2 lexicographically.
string1 < string2	True if string1 sorts before string2 lexicographically.

Suppose we want to check, whether the length of the string is non-zero, then we can check it as follows:

test –n $string Or [ –n $string ]
echo $?

If the result is 0, then we can conclude that the string length is non-zero. If the content of ? is non-zero, then the string has length 0.

Numerical comparison operators for the test command

The following is the summary of various options for numerical comparison using test.

Test operator	Tests true if
[integer_1 –eq integer_2]	integer_1 is equal to integer_2
[integer_1 –ne integer_2]	integer_1 is not equal to integer_2
[integer_1 –gt integer_2]	integer_1 is greater than integer_2
[integer_1 –ge integer_2]	integer_1 is greater than or equal to integer_2
[integer_1 –ge integer_2]	integer_1 is less than integer_2
[integer_1 –le integer_2]	integer_1 is less than or equal to integer_2

Let’s write shell script for learning various numerical test operator usage:

#!/bin/bash
num1=10
num2=30
echo $(($num1 < $num2)) # compare for less than
[ $num1 -lt $num2 ]     # compare for less than
echo $?
[ $num1 -ne $num2 ]     # compare for not equal
echo $?
[ $num1 -eq $num2 ]     # compare for equal to
echo $?

File test options for the test command

The following are the various options for file handling operations using the test command.

Test operator	Tests true if
–b file_name	This checks if the file is a Block special file
–c file_name	This checks if the file is a Character special file
–d file_name	This checks if the Directory is existing
–e file_name	This checks if the File exist
–f file_name	This checks if the file is a Regular file and not a directory
–G file_name	This checks if the file is existing and is owned by the effective group ID
–g file_name	This checks if file has a Set-group-ID set
–k file_name	This checks if the file has a Sticky bit set
–L file_name	This checks if the file is a symbolic link
–p file_name	This checks if the file is a named pipe
–O file_name	This checks if the file exists and is owned by the effective user ID
–r file_name	This checks if the file is readable
–S file_name	This checks if the file is a socket
–s file_name	This checks if the file has nonzero size
–t fd	This checks if the file has fd (file descriptor) and is opened on a terminal
–u file_name	This checks if the file has a Set-user-ID bit set
–w file_name	This checks if the file is writable
–x file_name	This checks if the file is executable

File testing binary operators

The following are various options for binary file operations using test.

Test Operator	Tests True If
[ file_1 –nt file_2 ]	This checks if the file is newer than file2
[ file_1 –ot file_2 ]	This check if file1 is older than file2.
[ file_1 –ef file_2 ]	This check if file1 and file2 have the same device or inode numbers.

Let’s write the script for testing basic file attributes such as whether it is a file or folder and whether it has file size bigger than 0

#!/bin/bash
# Check if file is Directory
[ -d work ]
echo $?
# Check that is it a File
[ -f test.txt ]
echo $?
# Check if File has size greater than 0
[ -s test.txt ]
echo $?

Logical test operators

The following are the various options for logical operations using test.

Test Operator	Tests True If
[ string_1 –a string_1 ]	Both string_1 and string_2 are true
[ string_1 –o string_2 ]	Either string_1 or string_2 is true
[ ! string_1 ]	Not a string_1 match
[[ pattern_1 && pattern_2 ]]	Both pattern_1 and pattern_2 are true
[[ pattern_1 || pattern_2 ]]	Either pattern_1 or pattern_2 is true
[[ ! pattern ]]	Not a pattern match

Reference: Bash reference manual—http://www.gnu.org/software/bash/

Conditional constructs – if else

We use if command for checking the pattern or command status, and accordingly, we can make certain decisions to execute script or commands.

The syntax of if conditional is as follows:

if command
then
   command
   command
fi

From the preceding syntax, we can clearly understand the working of the if conditional construct. Initially, the if statement will execute the block of command. If the result of command execution is true or 0, then all the commands which are enclosed between then and fi will be executed. If the status of command execution after if is false or non-zero, then all the commands after then will be ignored and the control of execution will directly go to fi.

The simple example for checking the status of last command executed using if construct is as follows:

#!/bin/bash
if [ $? -eq 0 ]
then
   echo "Command was successful."
else
   echo "Command was successful."
fi

Whenever we run any command, the exit status of the command will be stored in the ? variable. The if construct will be very useful in checking the status of the last command.

Switching case

Apart from simple decision making with if, it is also possible to process multiple decision-making operations using command case. In a case statement, the expression contained in a variable is compared with a number of expressions, and for each expression matched, a command is executed.

A case statement has the following structure:

case variable in
value1)
   command(s)
   ;;
value2)
   command(s)
   ;;
*)
   command(s)
    ;;
esac

For illustrating switch case scripting example, we will write script as follows. We will ask the user to enter any one of the number from range 1 to 9. We will check the entered number by case command. If the user enters any other number, then we will display the error by giving the Invalid key message.

#!/bin/bash
echo "Please enter number from 1 to 4"
read number
case $number in
1) echo "ONE"
   ;;
2) echo "TWO"
   ;;
3) echo "Three"
   ;;
4) echo "FOUR"
   ;;
   *) echo "SOME ANOTHER NUMBER"
   ;;
esac

Output:

Please enter any number from 1 to 4
2
TWO

Looping with for command

For iterative operations, bash shell uses three types of loops: for, while, and until. By using the for looping command, we can execute set of commands, for a finite number of times, for every item in a list. In the for loop command, the user-defined variable is specified. In the for command, after the in keyword, a list of values can be specified. The user-defined variable will get the value from that list, and all the statements between do and done get executed, until it reaches the end of the list.

The purpose of the for loop is to process a list of elements. It has the following syntax.

The simple script with for loop could be as follows:

for command in clear date cal
do
sleep 1
$command
Done

In the preceding script, the commands clear, date, and cal will be called one after the other. The command sleep for one second will be called before every command.

If we need to loop continuously or infinitely, then following is the syntax:

for ((;;))
do
command
done

Let’s write simple script as follows. In this script, we will print variable var 10 times.

#!/bin/bash
for var in {1..10}
do
echo $var
done

Exiting from the current loop iteration with continue

With the help of the continue command, it is possible to exit from a current iteration from loop and to resume the next iteration of the loop. We use commands for, while, or until for the loop iterations.

The following is the script with the for loop with the continue command, to skip a certain part of loop commands:

#!/bin/bash
for x in 1 2 3
do
   echo before $x
   continue 1
   echo after $x
done
exit 0

Exiting from a loop with break

In previous section, we discussed about how continue can be used to exit from the current iteration of a loop. The break command is another way to introduce a new condition within a loop. Unlike continue, however, it causes the loop to be terminated altogether if the condition is met.

In the following script, we are checking the directory content. If directory is found, then we exit the loop and display the message that the first directory is found.

#!/bin/bash
rm -rf sample*
echo > sample_1
echo > sample_2
mkdir sample_3
echo > sample_4

for file in sample*
do
if [ -d "$file" ]; then
   break;
fi
done

echo The first directory is $file
rm -rf sample*
exit 0

Working with loop using do while

Similar to the for command, while is also the command for loop operations. The condition or expression next to while is evaluated. If it is a success or 0, then the commands inside do and done are executed.

The purpose of a loop is to test a certain condition and execute a given command while the condition is true (while loop) or until the condition becomes true (until loop).

In the following script, we are printing numbers 1 to 10 on the screen by using the while loop.

#!/bin/bash
declare -i x
x=0
while [ $x -le 10 ]
do
echo $x
x=$((x+1))
done

Using until

The until command is similar to the while command. The given statements in loop are executed, as long as it evaluates the condition as true. As soon as the condition becomes false, then the loop is exited.

Syntax:

until command
do
   command(s)
done

In the following script, we are printing numbers 0 to 9 on the screen. When the value of variable x becomes 10, then the until loop stops executing.

#!/bin/bash
x=0
until [ $x -eq 10 ]
do
echo $x
x=`expr $x + 1`
done

Understanding functions

In the real-word scripts, we break down big tasks or scripts in smaller logical tasks. This modularization of scripts helps in better development and understanding of code. The smaller logical block of script can be written as a function.

Functions can be defined on command line or inside scripts. Syntax for defining functions on command line is as follows:

functionName { command_1; command_2; . . . }

Let’s write a very simple function for illustrating the preceding syntax.

$ hello() { echo 'Hello world!' ; }

We can use the preceding defined function as follows:

$ hello

Output:

Hello world!

Functions should be defined at the beginning of script.

Command source and ‘.’

Normally, whenever we enter any command, a new process is created. If we want to make functions from scripts to be made available in the current shell, then we need a technique that will run script in the current shell instead of creating a new shell environment. The solution to this problem is using the source or . commands.

The commands source and . can be used to run shell script in the current shell instead of creating new process. This helps in declaring function and variables in current shell.

The syntax is as follows:

$ source filename [arguments]

Or:

$ . filename [arguments]
$ source functions.sh

Or:

$ . functions.sh

If we pass command line arguments, those will be handled inside script as $1, $2 …and similar:

$ source functions.sh arg1 arg2

or

$ . /path/to/functions.sh arg1 arg2

The source command does not create new shell; but runs the shell scripts in current shell, so that all the variables and functions will be available in current shell for usage.

System startup, inittab, and run levels

When we power on the Linux system, the shell scripts are run one after another and the Linux system is initialized. These scripts start various services, daemons, start databases, mount discs, and many more applications are started. Even during the shutting down of system, certain shell scripts are executed so that important system data and information can be saved to the disk and applications are properly shut down. These are called as boot, startup, and shutdown scripts. These scripts are copied during the installation of the Linux operating system in our computer. As a developer or administrator, understanding these scripts may help you in understating and debugging the Linux system, and if required, you can customize these scripts.

During the system startup, as per run level, various scripts are called to initialize the basic operating system. Once the basic operating system in initialized, the user login process starts. This process is explained in the following topics.

System-wide settings scripts

In the /etc/ folder, the following files are related to user level initialization:

• /etc/profile: Few distributions will have additional folder such as /etc/profile.d/. All the scripts from the profile.d folder will be executed.
• /etc/bash.bashrc: Scripts in the /etc/ folder will be called for all the users. Particular user specific initialization scripts are located the in HOME folder of each user. These are as follows:
• $HOME/.bash_profile: This contains user specific bash environment default settings. This script is called during login process.
• $HOME/.bash_login: Second user environment initialization script, called during the login process.
• $HOME/.profile: If present, this script internally calls the .bashrc script file.
• $HOME/.bashrc: This is an interactive shell or terminal initialization script.

If we customize .bashrc such as added new alias commands or declare new function or environment variables, then we should execute .bashrc to take its effect.

Summary

In this article, you have learned about basic the Linux Shell Scripting along with the shell environment, creating and using variables, command line arguments, various debugging techniques, decision-making techniques and various looping techniques while testing for numeric, strings, logical and file handling-related operations, and the about writing functions. You also learned about system initialization and various initializing script and about how to customizing them.

This article has been a very short introduction to Linux shell scripting. For complete information with detailed explanation and numerous sample scripts, you can refer to the book at https://www.packtpub.com/networking-and-servers/learning-linux-shell-scripting.

Resources for Article:

---

Further resources on this subject:

• CoreOS – Overview and Installation[article]
• Getting Started[article]
• An Introduction to the Terminal [article]

---