Linux Shell Scripting Cookbook
Solve real-world shell scripting problems with over 110 simple but incredibly effective recipes
In this article, we took a look at some tips and tricks on working with Linux shell script.
Successful and unsuccessful command
Tip: When a command returns after error, it returns a non-zero exit status. The command returns zero when it terminates after successful completion. Return status can be read from special variable $? (run echo $? immediately after the command execution statement to print the exit status).
Fork bomb
:(){ :|:& };:
Tip: This recursive function is a function that calls itself. It infinitely spawns processes and ends up in a denial of service attack. & is postfixed with the function call to bring the subprocess into the background. This is a dangerous code as it forks processes and, therefore, it is called a fork bomb.
You may find it difficult to interpret the above code. See Wikipedia page http://en.wikipedia.org/wiki/Fork_bomb for more details and interpretation of the fork bomb. It can be prevented by restricting the maximum number of processes that can be spawned from the config file /etc/security/limits.conf.
Specify -maxdepth and –mindepth as the third argument
Tip: -maxdepth and –mindepth should be specified as the third argument to the find. If they are specified as the fourth or further arguments, it may affect the efficiency of the find as it has to do unnecessary checks (for example, if –maxdepth is specified as the fourth argument and –type as the third argument, the find command first finds out all the files having the specified –type and then finds all of the matched files having the specified depth. However, if the depth were specified as the third argument and –type as the fourth, find could collect all the files having at most the specified depth and then check for the file type, which is the most efficient way of searching.
-exec with multiple commands
Tip: We cannot use multiple commands along with the –exec parameter. It accepts only a single command, but we can use a trick. Write multiple commands in a shell script (for example, commands.sh) and use it with –exec as follows:
-exec ./commands.sh {} ;
-n option for numeric sort
Tip: Always be careful about the -n option for numeric sort. The sort command treats alphabetical sort and numeric sort differently. Hence, in order to specify numeric sort the –n option should be provided.
The ## operator
Tip: The ## operator is more preferred over the # operator to extract an extension from a filename since the filename may contain multiple ‘.’ characters. Since ## makes greedy match, it always extract extensions only.
Recursively search many files
Tip: To recursively search for a text over many directories of descendants use:
$ grep "text" . -R -n
This is one of the most frequently used commands by developers. It is used to find the file of source code in which a certain text exists.
Placing variable assignments
Tip: Usually, we place initial variable assignments, such as var=0; and statements to print the file header in the BEGIN block. In the END{} block, we place statements such as printing results and so on.
-d argument
Tip: The -d argument should always be given in quotes. If quotes are not used, & is interpreted by the shell to indicate this should be a background process.
Excluding a set of files from archiving
Tip: It is possible to exclude a set of files from archiving by specifying patterns. Use –exclude [PATTERN] for excluding files matched by wildcard patterns. For example, to exclude all .txt files from archiving use:
$ tar -cf arch.tar * --exclude "*.txt"
Note that the pattern should be enclosed in double quotes.
Using cpio command for absolute paths
Tip: By using cpio, we can also archive using files as absolute paths. /usr/somedir is an absolute path as it contains the full path starting from root (/).
A relative path will not start with / but it starts the path from the current directory. For example, test/file means that there is a directory test and the file is inside the test directory.
While extracting, cpio extracts to the absolute path itself. But incase of tar it removes the / in the absolute path and converts it as relative path.
PATH format
Tip: For the PATH format, if we use / at the end of the source, rsync will copy contents of that end directory specified in the source_path to the destination.
If / not at the end of the source, rsync will copy that end directory itself to the destination.
/ at the end of destination_path
Tip: If / is at the end of destination_path, rsync will copy the source to the destination directory. If / is not used at the end of the destination path, rsync will create a folder, named similar to the source directory, at the end of the destination path and copy the source into that directory.
wait command
Tip: The wait command enables a script to be terminated only after all its child process or background processes terminate or complete.
First argument of the sftp command
Tip: -oPort should be the first argument of the sftp command.
Running du DIRECTORY
Tip: Running du DIRECTORY will output a similar result, but it will show only the size consumed by subdirectories. However, they do not show the disk usage for each of the files. For printing the disk usage by files, -a is mandatory.
du DIRECTORY commands traversal
Tip: du can be restricted to traverse only a single file system by using the –x argument. Suppose du DIRECTORY is run, it will traverse through every possible subdirectory of DIRECTORY recursively. A subdirectory in the directory hierarchy may be a mount point (for example, /mnt/sda1 is a subdirectory of /mnt and it is a mount point for the device /dev/sda1). du will traverse that mount point and calculate the sum of disk usage for that device filesystem also. In order to prevent du from traversing and to calculate from other mount points or filesystems, use the -x flag along with other du options. du –x / will exclude all mount points in /mnt/ for disk usage calculation.
Use an absolute path for the executable
Tip: An executable binary of the time command is available at /usr/bin/time as well as a shell built-in named time exists. When we run time, it calls the shell built-in by default. The shell built-in time has limited options. Hence, we should use an absolute path for the executable (/usr/bin/time) for performing additional functionalities.
-x argument
Tip: The -x argument along with -a specifies to remove the TTY restriction imparted, by default, by ps. Usually, using ps without arguments prints processes that are attached to terminal only.
Parameters for -o
Tip: Parameters for -o are delimited by using the comma (,) operator. It should be noted that there is no space in between the comma operator and next parameter. Mostly, the -o option is combined with the -e (every) option (-oe) since it should list every process running in the system. However, when certain filters are used along with –o, such as those used for listing the processes owned by specified users, -e is not used along with –o. Usage of -e with a filter will nullify the filter and it will show all process entries.
pgrep command
Tip: pgrep requires only a portion of the command name as its input argument to extract a Bash command, for example, pgrep ash or pgrep bas will also work. But ps requires you to type the exact command.
apropos
Tip: Sometimes we need to search if some command related to a word exists. Then we can search the manpages for strings in the command. For this we can use:
apropos COMMAND
In this article, we took a look at some tips and tricks on working with Linux shell script.Further resources on this subject:
- Linux Email [Book]
- Installing VirtualBox on Linux [Article]
- Linux Shell Script: Logging Tasks [Article]
- Linux Shell Script: Monitoring Activities [Article]
- Compression Formats in Linux Shell Script [Article]
- Making a Complete yet Small Linux Distribution [Article]
