If you’re just starting out in shell scripting, there’s a good chance you’re looking for a comprehensive guide to the basics. That’s why we’ve put together this beginner’s guide to shell scripting. In this guide, we’ll take a look at the most important concepts and tools you’ll need to get started with shell scripting. We’ll also cover some common shell script examples so that you can start writing your own scripts and applications. So what are we waiting for? Let’s get started!
If you want to build up your geek cred, join us for the second installment in our shell scripting series. We have a few corrections, a few improvements to last week’s script, and a guide on looping for the uninitiated.
The datecp Script Revisited
In the first installment of our shell scripting guide, we made a script that copied a file to a backup directory after appending the date to the end of the filename.
Samuel Dionne-Riel pointed out in the comments that there’s a much better way to handle our variable references.
As you can see, changing our script’s line from:
To properly address this issue, the last line of the script should be: cp “$1” “$2.$date_formatted”
to:
will take care of this problem when using the script on files that have spaces in the name. Samuel also makes the point that when copying and pasting code from this site (or the internet in general) be sure to substitute the proper dashes and quotes for the “typographically better” ones that often replace them. We’ll also be doing more to make sure our code is more copy/paste friendly. ;-)
Another commenter, Myles Braithwaite, decided to expand our script so that the date would appear before the file extension. So instead of
we would get this:
which ends up being a little more convenient for most users. His code is available at on his GitHub page. Let’s take a look at what he uses to pull apart the filename.
I’ve changed the formatting a bit, but you can see that Myles declares his date function in Line 1. In Line 2, however, he uses the “echo” command with the first argument of the script to output the file’s name. He uses the pipe command to take that output and use it as input for the next part. After the pipe, Myles calls on the “awk” command, which is a powerful pattern scanning program. Using the -F flag, he’s telling the command that the next character (after a space) is what will define the “field separator”. In this case, that’s a period.
cp -iv $1 $file_name-$date_formatted.$file_extension
Now, awk see a file named “tastyfile.mp3” as being composed of two fields: “tastyfile” and “mp3”. Lastly, he uses
to display the last field. In case your file has multiple periods – hence making awk see multiple fields – it will only display the last one, which is the file extension.
In Line 3, he creates a new variable for the file’s name and uses the “basename” command to reference everything in $1 except the file extension. This is done by using basename and giving it $1 as its argument, then adding a space and the file extension. The file extension is automatically added in because of the variable that references Line 2. What this would do is take
and turn it into
Then in the last line, Myles put together the command that will output everything in order. Note that there is no reference to $2, a second argument for the script. This particular script will copy said file into your current directory instead. Great job Samuel and Myles!
Running Scripts and $PATH
We also mention in our Basics article that scripts aren’t allowed to be referenced as commands by default. That is, you have to point to the path of the script in order to run it:
But, by placing your scripts in ~/bin/, you could just type their names from anywhere to get them to run.
~/bin/script
Commenters spent some time debating how proper this was, as no modern Linux distro creates that directory by default. Furthermore, no one adds it to the $PATH variable by default either, which is what’s required in order for scripts to be run like commands. I was a bit puzzled because after checking my $PATH variable, the commenters were right, but calling scripts still worked for me. I found out why: many modern Linux distros create a special file in the user’s home directory – .profile.
This file is read by bash (unless .bash_profile is present in the user’s home directory) and at the bottom, there’s a section that adds the ~/bin/ folder to the $PATH variable if it exists. So, that mystery is cleared up. For the rest of the series, I’ll continue placing scripts in the ~/bin/ directory because they’re user scripts and should be able to be run by users. And, it seems we don’t really need to mess with the $PATH variable by hand to get things working.
Repeating Commands With Loops
Let’s get to one of the most useful tools in the geek arsenal for dealing with repetitive tasks: loops. Today, we’ll be discussing “for” loops.
The basic outline of a for-loop is as follows:
VARIABLE can be any variable, though most often the lowercase “i” is used by convention. LIST is a list of items; you can specify multiple items (separating them by a space), point to an external text file, or use an asterisk (*) to denote any file in the current directory. The commands listed are indented by convention, so it’s easier to see nesting – putting loops in loops (so you can loop while you loop).
Because lists use spaces as delimiters – that is, a space signifies a move to the next item in the list – files that have spaces in the name aren’t very friendly. For now, let’s stick to working with files without spaces.Let’s start with a simple script to display the names of files in the current directory. Create a new script in your ~/bin/ folder entitled “loopscript”. If you don’t remember how to do this (including marking it as executable and adding the hash bang hack) refer to our bash scripting basics article.
In it, enter the following code:
When you run the script, you should just get those list items as output.
Pretty simple, right? Let’s see what happens if we change things up a little bit. Change your script so it says this:
When you run this script in a folder, you should get a list of files that it contains as output.
Now, let’s change the echo command into something more useful – say, the zip command. Namely, we’ll add files into an archive. And, let’s gets some arguments in the mix!
There’s something new! “$@” is a shortcut for “$1 $2 $3 … $n”. In other words, it’s the full list of all arguments you specified. Now, watch what happens when I run the script with several input files.
You can see which files are in my folder. I ran the command with six arguments, and each file was added to a zipped archive named “archive.zip”. Easy, right?
For loops are pretty wonderful. Now you can execute batch functions on lists of files. For example, you can copy all of your script’s arguments into a zipped archive, move the originals to a different folder, and automatically secure copy that zip file to a remote computer. If you set up key files with SSH, you won’t even need to enter your password, and you can even tell the script to delete the zip file after uploading it!
Using for-loops makes it easy to do a bunch of actions for all files in a directory. You can stack a wide variety of commands together and use arguments very easily to create and on-the-fly list, and this is only the tip of the iceberg.
Bash scripters, do you have any suggestions? Have you made a useful script that uses loops? Want to share you thoughts on the series? Leave some comments and help other scripting newbies out!
