Bash Control Structures

We can distinguish the following bash control structures:

• Conditional expressions. Discussed at length in if-then-else construct. See also:
  • Shell Conditional operators
  • Bash expressions
  • Arithmetic Expressions in BASH
• if-then-else statement
• loops including
  • while loops (there is no until loops but there is until keyword ;-) 
  • idiosyncratic "legacy" (Bourne-style) for loops
  • C-style for loops
  • Loops connected to a pipe
• case
• select

BASH supports the typical for other shells set of high level flow control constructs. Most of them were introduced by Born Shell:

• if/else Execute a list of statements if a certain condition is/is not true. Due to the age and compatibility reasons if statements has five different forms which is pretty confusing.
• for Execute a list of statements a fixed number of times
• while Execute a list of statements repeatedly while a certain condition holds true
• until  there is no until loop in shell. until keyword is just changes the test(execute statements repeatedly until a certain condition holds true) but test for loop termination remains at the top
• case Execute one of several lists of statements depending on the value of a variable
• select Allow the user to select one of a list of possibilities from a menu

Be warned: the syntax is not pretty.

One of the big improvements that modern versions of bash have when compared with the original Bourne shell is in the area of arithmetic. Bash re-implemented major advances of ksh93 and now have ((...)) constructs. Paradoxically early versions of the Unix shell had no built-in arithmetic; it had to be done by invoking a separate executable (expr),

even just to add 1 to a variable.

All conditional statements in shell depends on the expressions and due to the age of the shell (Borne shell it's more then 30 years old) there are many historical layers with different capabilities.

if-then-else

The simplest type of flow control construct is the conditional, embodied in the BASH's if statement. You use a conditional when you want to choose whether or not to do something, or to choose among a small number of things to do, according to the truth or falsehood of conditions. Conditions test values of shell variables, characteristics of files, whether or not commands run successfully, and other factors. The shell has a large set of built-in tests that are relevant to the task of shell programming.

The if construct has the following syntax:

if condition
then
    statements
[elif condition
    then statements...]
[else
    statements]
fi

The simplest form (without the elif and else parts, a.k.a. clauses) executes the statements only if the condition is true. If you add an else clause, you get the ability to execute one set of statements if a condition is true or another set of statements if the condition is false. You can use as many elif (a contraction of "else if") clauses as you wish; they introduce more conditions, and thus more choices for which set of statements to execute. If you use one or more elifs, you can think of the else clause as the "if all else fails" part.

Exit Status and Return

Every UNIX command, whether it comes from source code in C, some other language, or a shell script/function, returns an integer code to its calling process-the shell when it finishes. This is called the exit status. 0 is usually the "OK" exit status, while anything else (1 to 255) usually denotes an error.

Shell if statement checks the exit status of the last statement in the list following the if keyword (The list is usually just a single statement.) If the status is 0, the condition evaluates to true; if it is anything else, the condition is considered false. The same is true for each condition attached to an elif statement (if any).

This enables us to write code of the form:

if command ran successfully
then
    normal processing
else
    error processing
fi

Conditional Execution && and ||

One of the more obscure parts of BASH syntax allows you to combine exit statuses logically, so that you can test more than one thing at a time:

• statement1 && statement2 means, "execute statement1, and if its exit status is 0, execute statement2."
• statement1 || statement2 is the converse: it means, "execute statement1, and if its exit status is not 0, execute statement2."

These constructs can be used outside if statement as well. In both cases it's useful to think about them "short-circuit and" and "short-circuit or," respectively:

if statement1 && statement2
then
    ...
fi

statement1 is always executed. But only if ir returns a return code 0 statement2 runs. The then clause will be executed only if statement1 and statement2 both succeeded.

In case of short-circuit or the situation is similar:

if statement1 || statement2
then
    ...
fi

statement1 is always executed. If it returns code zero (success), then statement2 will not be executed. Otherwise statement2 will be executed and its return code will be used for the deciding whether to execute then clause of the if statement or else clause. In other words then clause runs either statement1 or statement2 returns a zero code.

Let's assume that we need to write a script that checks a /etc/passwd file for the presence of two users who left the company. We can use grep for this: it returns exit status 0 if it found the given string in its input, non-0 if not:

user1=$1
user2=$2
if grep $user1 /etc/passwd || grep $user2 /etc/passwd
then
    print "iether user $user1 or $user2 has account of this server."
fi

Double bracket condition test

The BASH provides the ability to write pretty complex logical expressions the [[ ]] construct. It also accepts old, Born shell-style the external [] and test commands, but it does not make sense to use them anymore and they will not be discussed in the lecture. The [[ ]] construct is more modern and is better integrated into the BASH. What is important delimiters [[ and ]] serve as single quotes. That means variable substitution and wildcard expansion aren't done within [[ and ]], making quoting less necessary.

Like && and || constructs that we studied before, [[ condition ]] can be used as a separate statement that returns an exit status depending upon whether condition is true or not. With && and || construts discussed above this provides an alternative syntax for if-then and if-else constucts

String comparisons

The double square brackets ([[...]]) surround expressions that include various types of operators. We will start with the string comparison operators, which are listed in Table 5.1. (Notice that there are no operators for "greater than or equal" or "less than or equal.") In the table, str refers to an expression with a string value, and pat refers to a pattern that can contain wildcards (just like the patterns in the string-handling operators we saw in the last chapter).

str = pat[5]
str == pat[5]

To accomplish this, we need to test for an empty stack, i.e., whether $DIRSTACK is null or not. Here is one way to do it:

function fcd {                 # find the directory is history, cd there        
    newdir=`grep "cd " ~/.history | reverse | grep "$1"` 

if [[ -d #newdir]]
then
   cd $newdir
else 
   grep "^cd " '~/.history | tail 5
fi

}

While we're cleaning up code we wrote in the last chapter, let's fix up the error handling in the highest script (Task 4-1). The code for that script is:

filename=${1:?"filename missing."}
howmany=${2:-10}
sort -nr $filename | head -$howmany

Recall that if you omit the first argument (the filename), the shell prints the message highest: 1: filename missing. We can make this better by substituting a more standard "usage" message:

if [[ -z $1 ]]; then
    print 'usage: howmany filename [-N]'
else
    filename=$1
    howmany=${2:-10}
    sort -nr $filename | head -$howmany
fi

It is considered better programming style to enclose all of the code in the if-then-else, but such code can get confusing if you are writing a long script in which you need to check for errors and bail out at several points along the way. Therefore, a more usual style for shell programming is this:

if [[ -z $1 ]]; then
    print 'usage: howmany filename [-N]'
    return 1
fi
filename=$1
howmany=${2:-10}
sort -nr $filename | head -$howmany

File Attribute Checking

The other kind of operator that can be used in conditional expressions checks a file for certain properties. There are 21 such operators. We will cover those of most general interest here; the rest refer to arcana like sticky bits, sockets, and file descriptors, and thus are of interest only to systems hackers. Refer to Appendix B, Reference Lists for the complete list. Table 5.2 lists those that we will examine.

Before we get to an example, you should know that conditional expressions inside [[ and ]] can also be combined using the logical operators && and ||, just as we saw with plain shell commands above, in the section entitled "Combinations of Exit Statuses."

It's also possible to combine shell commands with conditional expressions using logical operators, like this:

if command && [[ condition ]]; then
    ...

Chapter 7 contains an example of this combination.

You can also negate the truth value of a conditional expression by preceding it with an exclamation point (!), so that ! expr evaluates to true only if expr is false. Furthermore, you can make complex logical expressions of conditional operators by grouping them with parentheses. [9]

[9] It turns out that this is true outside of the [[/]] construct as well. As we will see in Chapter 8, Process Handling the construct (statement list) runs the statement list in a subshell, whose exit status is that of the last statement in the list. However, there is no equivalent of the negation (!) operator outside of the [[/]] construct, although there will be in future releases.

Here is how we would use two of the file operators to embellish (yet again) our pushd function. Instead of having cd determine whether the argument given is a valid directory-i.e., by returning with a bad exit status if it's not-we can do the checking ourselves. Here is the code:

function pushd {                # push current directory onto stack
    dirname=$1
    if [[ -d $dirname && -x $dirname ]]; then
        cd $dirname
        DIRSTACK="$dirname ${DIRSTACK:-$PWD}"
        print "$DIRSTACK"
    else
        print "still in $PWD."
    fi
}

The conditional expression evaluates to true only if the argument $1 is a directory (-d) and the user has permission to change to it (-x). [10] Notice that this conditional also handles the case where the argument is missing: $dirname is null, and since the null string isn't a valid directory name, the conditional will fail.

[10] Remember that the same permission flag that determines execute permission on a regular file determines search permission on a directory. This is why the -x operator checks both things depending on file type.

Here is a more comprehensive example of the use of file operators.

Write a script that prints essentially the same information as ls -l but in a more user-friendly way.

Although this task requires relatively long-winded code, it is a straightforward application of many of the file operators:

if [[ ! -a $1 ]]; then
    print "file $1 does not exist."
    return 1
fi
if [[ -d $1 ]]; then
    print -n "$1 is a directory that you may "
    if [[ ! -x $1 ]]; then
        print -n "not "
    fi
    print "search."
elif [[ -f $1 ]]; then
    print "$1 is a regular file."
else
    print "$1 is a special type of file."
fi
if [[ -O $1 ]]; then
    print 'you own the file.'
else
    print 'you do not own the file.'
fi
if [[ -r $1 ]]; then
    print 'you have read permission on the file.'
fi
if [[ -w $1 ]]; then
    print 'you have write permission on the file.'
fi
if [[ -x $1 && ! -d $1 ]]; then
    print 'you have execute permission on the file.'
fi

We'll call this script fileinfo. Here's how it works:

• The first conditional tests if the file given as argument does not exist (the exclamation point is the "not" operator; the spaces around it are required). If the file does not exist, the script prints an error message and exits with error status.
• The second conditional tests if the file is a directory. If so, the first print prints part of a message; remember that the -n option tells print not to print a LINEFEED at the end. The inner conditional checks if you do not have search permission on the directory. If you don't have search permission, the word "not" is added to the partial message. Then, the message is completed with "search." and a LINEFEED.
• The elif clause checks if the file is a regular file; if so, it prints a message.
• The else clause accounts for the various special file types on recent UNIX systems, such as sockets, devices, FIFO files, etc. We assume that the casual user isn't interested in details of these.
• The next conditional tests to see if the file is owned by you (i.e., if its owner ID is the same as your login ID). If so, it prints a message saying that you own it.
• The next two conditionals test for your read and write permission on the file.
• The last conditional checks if you can execute the file. It checks to see if you have execute permission and that the file is not a directory. (If the file were a directory, execute permission would really mean directory search permission.)

As an example of fileinfo's output, assume that you do an ls -l of your current directory and it contains these lines:

-rwxr-xr-x   1 billr    other        594 May 28 09:49 bob
-rw-r-r-     1 billr    other      42715 Apr 21 23:39 custom.tbl
drwxr-xr-x   2 billr    other         64 Jan 12 13:42 exp
-r-r-r-      1 root     other        557 Mar 28 12:41 lpst

custom.tbl and lpst are regular text files, exp is a directory, and bob is a shell script. Typing fileinfo bob produces this output:

bob is a regular file.
you own the file.
you have read permission on the file.
you have write permission on the file.
you have execute permission on the file.

Typing fileinfo custom.tbl results in this:

custom.tbl is a regular file.
you own the file.
you have read permission on the file.
you have write permission on the file.

Typing fileinfo exp results in this:

exp is a directory that you may search.
you own the file.
you have read permission on the file.
you have write permission on the file.

Finally, typing fileinfo lpst produces this:

lpst is a regular file.
you do not own the file.
you have read permission on the file.

Chapter 7 contains an example of the -nt test operator.

Integer Conditionals

The shell also provides a set of arithmetic tests. These are different from character string comparisons like < and >, which compare lexicographic values of strings, not numeric values. For example, "6" is greater than "57" lexicographically, just as "p" is greater than "ox," but of course the opposite is true when they're compared as integers.

The integer comparison operators are summarized in Table 5.3. Thius is a FORTRAN-style notation.

You'll find these to be of the most use in the context of the integer variables we'll see in the next chapter. They're necessary if you want to combine integer tests with other types of tests within the same conditional expression.

However, the shell has a separate syntax for conditional expressions that involve integers only. It's considerably more efficient, so you should use it in preference to the arithmetic test operators listed above. Again, we'll cover the shell's integer conditionals in the next chapter.

for

The most obvious enhancement we could make to the previous script is the ability to report on multiple files instead of just one. Tests like -a and -d only take single arguments, so we need a way of calling the code once for each file given on the command line.

The way to do this-indeed, the way to do many things with the BASH-is with a looping construct. The simplest and most widely applicable of the shell's looping constructs is the for loop. We'll use for to enhance fileinfo soon.

The for loop allows you to repeat a section of code a fixed number of times. During each time through the code (known as an iteration), a special variable called a loop variable is set to a different value; this way each iteration can do something slightly different.

The for loop is somewhat, but not entirely, similar to its counterparts in conventional languages like C and Pascal. The chief difference is that the shell's for loop doesn't let you specify a number of times to iterate or a range of values over which to iterate; instead, it only lets you give a fixed list of values. In other words, you can't do anything like this Pascal-type code, which executes statements 10 times:

for x := 1 to 10 do
begin
    statements...
end

(You need the while construct, which we'll see soon, to construct this type of loop. You also need the ability to do integer arithmetic, which we will see in Chapter 6, Command-line Options and Typed Variables.)

However, the for loop is ideal for working with arguments on the command line and with sets of files (e.g., all files in a given directory). We'll look at an example of each of these. But first, we'll show the syntax for the for construct:

for name [in list]
do
    statements that can use $name...
done

The list is a list of names. (If in list is omitted, the list defaults to "$@", i.e., the quoted list of command-line arguments, but we'll always supply the in list for the sake of clarity.) In our solutions to the following task, we'll show two simple ways to specify lists.

You work in an environment with several computers in a local network. Write a shell script that tells you who is logged in to each machine on the network.

The command finger(1) can be used (among other things) to find the names of users logged into a remote system; the command finger @systemname does this. Its output depends on the version of UNIX, but it looks something like this:

[motet.early.com]
Trying 127.146.63.17...
-User-    -Full name-       -What- Idle TTY -Console Location-
hildy    Hildegard von Bingen  ksh   2d5h p1  jem.cal (Telnet)
mikes    Michael Schultheiss   csh   1:21 r4  ncd2.cal (X display 0)
orlando  Orlando di Lasso      csh     28 r7  maccala (Telnet)
marin    Marin Marais          mush  1:02 pb  mussell.cal (Telnet)
johnd    John Dowland          tcsh    17 p0  nugget.west.nobis. (X Window)

In this output, motet.early.com is the full network name of the remote machine.

Assume the systems in your network are called fred, bob, dave, and pete. Then the following code would do the trick:

for sys in fred bob dave pete
do
    finger @$sys
    print
done

This works no matter which of the systems you are currently logged into. It prints output for each machine similar to the above, with blank lines in between.

A slightly better solution would be to store the names of the systems in an environment variable. This way, if systems are added to your network and you need a list of their names in more than one script, you need change them in only one place. If a variable's value is several words separated by blanks (or TABS), for will treat it as a list of words.

Here is the improved solution. First, put lines in your .profile or environment file that define the variable SYSNAMES and make it an environment variable:

SYSNAMES="fred bob dave pete"
export SYSNAMES

Then, the script can look like this:

for sys in $SYSNAMES
do
    finger @$sys
    print
done

The foregoing illustrated a simple use of for, but it's much more common to use for to iterate through a list of command-line arguments. To show this, we can enhance the fileinfo script above to accept multiple arguments. First, we write a bit of "wrapper" code that does the iteration:

for filename in "$@" ; do
    finfo $filename
    print
done

Next, we make the original script into a function called finfo: [11]

function finfo {
    if [[ ! -a $1 ]]; then
        print "file $1 does not exist."
        return 1
    fi
    ...
}

[11] A function can have the same name as a script; however, this isn't good programming practice.

The complete script consists of the for loop code and the above function, in either order; good programming style dictates that the function definition should go first.

The fileinfo script works as follows: in the for statement, "$@" is a list of all positional parameters. For each argument, the body of the loop is run with filename set to that argument. In other words, the function fileinfo is called once for each value of $filename as its first argument ($1). The call to print after the call to fileinfo merely prints a blank line between sets of information about each file.

Given a directory with the same files as the previous example, typing fileinfo * would produce the following output:

bob is a regular file.
you own the file.
you have read permission on the file.
you have write permission on the file.
you have execute permission on the file.

custom.tbl is a regular file.
you own the file.
you have read permission on the file.
you have write permission on the file.

exp is a directory that you may search.
you own the file.
you have read permission on the file.
you have write permission on the file.

lpst is a regular file.
you do not own the file.
you have read permission on the file.

Here is a programming task that exploits the other major use of for.

Your UNIX system has the ability to transfer files from an MS-DOS system, but it leaves the DOS filenames intact. Write a script that translates the filenames in a given directory from DOS format to a more UNIX-friendly format.

DOS filenames have the format FILENAME.EXT. FILENAME can be up to eight characters long; EXT is an extension that can be up to three characters. The dot is required even if the extension is null; letters are all uppercase. We want to do the following:

1. Translate letters from uppercase to lowercase.
2. If the extension is null, remove the dot.

The first tool we will need for this job is the UNIX tr(1) utility, which translates characters on a one-to-one basis. Given the arguments charset1 and charset2, it will translate characters in the standard input that are members of charset1 into corresponding characters in charset2. The two sets are ranges of characters enclosed in square brackets ([] in standard regular-expression form in the manner of grep, awk, ed, etc.). More to the point, tr [A-Z] [a-z] takes its standard input, converts uppercase letters to lowercase, and writes the converted text to the standard output.

That takes care of the first step in the translation process. We can use a BASH string operator to handle the second. Here is the code for a script we'll call dosmv:

for filename in ${1:+$1/}* ; do
    newfilename=$(print $filename | tr [A-Z] [a-z])
    newfilename=${newfilename%.}
    print "$filename -> $newfilename"
    mv $filename $newfilename
done

The * in the for construct is not the same as $*. It's a wildcard, i.e., all files in a directory.

This script accepts a directory name as argument, the default being the current directory. The expression ${1:+$1/} evaluates to the argument ($1) with a slash appended if the argument is supplied, or the null string if it isn't supplied. So the entire expression ${1:+$1/}* evaluates to all files in the given directory, or all files in the current directory if no argument is given.

Therefore, filename takes on the value of each filename in the list. filename gets translated into newfilename in two steps. (We could have done it in one, but readability would have suffered.) The first step uses tr in a pipeline within a command substitution construct. Our old friend print makes the value of filename the standard input to tr. tr's output becomes the value of the command substitution expression, which is assigned to newfilename. Thus, if $filename were DOSFILE.TXT, newfilename would become dosfile.txt.

The second step uses one of the shell's pattern-matching operators, the one that deletes the shortest match it finds at the end of the string. The pattern here is ., which means a dot at the end of the string. [12] This means that the expression ${newfilename%.} will delete a dot from $newfilename only if it's at the end of the string; otherwise the expression will leave $newfilename intact. For example, if $newfilename is dosfile.txt, it will be untouched, but if it's dosfile., the expression will change it to dosfile without the final dot. In either case, the new value is assigned back to newfilename.

[12] UNIX regular expression mavens should remember that this is shell wildcard syntax, in which dots are not operators and therefore do not need to be backslash-escaped.

The last statement in the for loop body does the file renaming with the standard UNIX mv(1) command. Before that, a print command simply informs the user of what's happening.

There is one little problem with the solution on the previous page: if there are any files in the given directory that aren't DOS files (in particular, if there are files whose names don't contain uppercase letters and don't contain a dot), then the conversion will do nothing to those filenames and mv will be called with two identical arguments. mv will complain with the message: mv: filename and filename are identical. We can solve this problem by letting grep determine whether each file has a DOS filename or not. The grep regular expression:

[^a-z]\{1,8\}\.[^a-z]\{0,3\}

is adequate (for these purposes) for matching DOS-format filenames. [13] The character class [^a-z] means "any character except a lowercase letter." [14] So the entire regular expression means: "Between 1 and 8 non-lowercase letters, followed by a dot, followed by 0 to 3 non-lowercase letters."

When grep runs, it normally prints all of the lines in its standard input that match the pattern you give it as argument. But we only need it to test whether or not the pattern is matched. Luckily, grep's exit status is "well-behaved": it's 0 if there is a match in the input, 1 if not. Therefore, we can use the exit status to test for a match. We also need to discard grep's output; to do this, we redirect it to the special file /dev/null, which is colloquially known as the "bit bucket." [15] Any output directed to /dev/null effectively disappears. Thus, the command line:

print "$filename" | grep '[^a-z]\{1,8\}\.[^a-z]\{0,3\}' > /dev/null

Some Berkeley-derived versions of UNIX have a -s ("silent") option to grep that suppresses standard output, thereby making redirection to /dev/null unnecessary.

prints nothing and returns exit status 0 if the filename is in DOS format, 1 if not.

Now we can modify our ren script to incorporate this code:

function ren 

# ${variable##pattern}

suffix=${2##.*}
{
for filename in $1
do
newfilename=${filename%.}
print "$filename -> $newfilename"
mv $filename $newfilename.$suffix
fi
done

For readability reasons, we use the variable dos_regexp to hold the DOS filename-matching regular expression.

If you are familiar with an operating system other than DOS and UNIX, you may want to test your script-writing prowess at this point by writing a script that translates filenames from that system's format into UNIX format. Use the above script as a guideline.

case

The next flow control construct we will cover is case. While the case statement in Pascal and the similar switch statement in C can be used to test simple values like integers and characters, the BASH's case construct lets you test strings against patterns that can contain wildcard characters. Like its conventional language counterparts, case lets you express a series of if-then-else type statements in a concise way.

The syntax of case is as follows:

case expression in
    pattern1 )
        statements ;;
    pattern2 )
        statements ;;
    ...
esac

Any of the patterns can actually be several patterns separated by pipe characters (|). If expression matches one of the patterns, its corresponding statements are executed. If there are several patterns separated by pipe characters, the expression can match any of them in order for the associated statements to be run. The patterns are checked in order until a match is found; if none is found, nothing happens.

This rather ungainly syntax should become clearer with an example. An obvious choice is to revisit our solution to Task 4-2, the front-end for the C compiler. Earlier in this chapter, we wrote some code, that processed input files according to their suffixes (.c .s, or .o for C, assembly, or object code, respectively).

We can improve upon this solution in two ways. First, we can use for to allow multiple files to be processed at one time; second, we can use case to streamline the code:

for filename in $*; do
    case $filename in
        *.c )
            objname=${filename%.c}.o
            ccom $filename $objname ;;
        *.s )
            objname=${filename%.s}.o
            as $filename $objname ;;
        *.o ) ;;
        *   )
            print "error: $filename is not a source or object file."
            return 1 ;;
    esac
done

The case construct in this code handles four cases. The first two are similar to the if and first elif cases in the code earlier in this chapter; they call the compiler or the assembler if the filename ends in .c or .s respectively.

After that, the code is a bit different. Recall that if the filename ends in .o nothing is to be done (on the assumption that the relevant files will be linked later). If the filename does not end in .o there is an error. We handle this with the case *.o ), which has no statements. There is nothing wrong with a "case" for which the script does nothing.

The final case is *, which is a catchall for whatever didn't match the other cases. (In fact, a * case is analogous to a default case in C and an otherwise case in some Pascal-derived languages.)

The surrounding for loop processes all command-line arguments properly. This leads to a further enhancement: now that we know how to process all arguments, we should be able to write the code that passes all of the object files to the linker (the program ld) at the end. We can do this by building up a string of object file names, separated by spaces, and hand that off to the linker when we've processed all of the input files. We initialize the string to null and append an object file name each time one is created, i.e., during each iteration of the for loop. The code for this is simple, requiring only minor additions:

objfiles=""
for filename in $*; do
    case $filename in
        *.c )
            objname=${filename%.c}.o
            ccom $filename $objname ;;
        *.s )
            objname=${filename%.s}.o
            as $filename $objname ;;
        *.o )
            objname=$filename ;;
        *   )
            print "error: $filename is not a source or object file."
            return 1 ;;
    esac
    objfiles="$objfiles $objname"
done
ld $objfiles

The first line in this version of the script initializes the variable objfiles to null. [16] We added a line of code in the *.o case to set objname equal to $filename, because we already know it's an object file. Thus, the value of objname is set in every case-except for the error case, in which the routine prints a message and bails out.

[16] This isn't strictly necessary, because all variables are assumed to be null if not explicitly initialized (unless the nounset option is turned on). It just makes the code easier to read.

The last line of code in the for loop body appends a space and the latest $objname to objfiles. Calling this script with the same arguments as in Figure 5.1 would result in $objfiles being equal to " a.o b.o c.o d.o" when the for loop finishes (the leading space doesn't matter). This list of object filenames is given to ld as a single argument, but the shell divides it up into multiple file names properly.

We'll return to this example once more in Chapter 6 when we discuss how to handle dash options on the command line. Meanwhile, here is a new task whose initial solution will use case.

The code for the solution to this task should go into the file /etc/profile, which is the master startup file that is run for each user before his or her .profile.

select

All of the flow-control constructs we have seen so far are also available in the Bourne shell, and the C shell has equivalents with different syntax. Our next construct, select, is new for the BASH; moreover, it has no analog in conventional programming languages.

select allows you to generate simple menus easily. It has concise syntax, but it does quite a lot of work. The syntax is:

select name [in list]
do
    statements that can use $name...
done

This is the same syntax as for except for the keyword select. And like for, you can omit the in list and it will default to "$@", i.e., the list of quoted command-line arguments.

Here is what select does:

• Generates a menu of each item in list, formatted with numbers for each choice
• Prompts the user for a number
• Stores the selected choice in the variable name and the selected number in the built-in variable REPLY
• Executes the statements in the body
• Repeats the process forever (but see below for how to exit)

Once again, an example should help make this process clearer. Assume you need to write the code for Task 5-4, but your life is not as simple. You don't have terminals hardwired to your computer; instead, your users communicate through a terminal server. This means, among other things, that the tty number does not determine the type of terminal.

Therefore, you have no choice but to prompt the user for his or her terminal type at login time. To do this, you can put the following code in /etc/profile (assume you have the same choice of terminal types):

PS3='terminal? '
select term in gl35a t2000 s531 vt99; do
    if [[ -n $term ]]; then
        TERM=$term
        print TERM is $TERM
        break
    else
        print 'invalid.'
    fi
done

If you run this code, you will see this menu:

1) gl35a
2) t2000
3) s531
4) vt99
terminal?

The built-in shell variable PS3 contains the prompt string that select uses; its default value is the not particularly useful "#? ". So the first line of the above code sets it to a more relevant value.

The select statement constructs the menu from the list of choices. If the user enters a valid number (from 1 to 4), then the variable term is set to the corresponding value; otherwise it is null. (If the user just presses RETURN, the shell prints the menu again.)

The code in the loop body checks if term is non-null. If so, it assigns $term to the environment variable TERM and prints a confirmation message; then the break statement exits the select loop. If term is null, the code prints an error message and repeats the prompt (but not the menu).

The break statement is the usual way of exiting a select loop. Actually (like its analog in C), it can be used to exit any surrounding control structure we've seen so far (except case, where the double-semicolons act like break) as well as the while and until we will see soon. We haven't introduced break until now because it is considered bad coding style to use it to exit a loop. However, it is necessary for exiting select when the user makes a valid choice. [18]

[18] A user can also type [CTRL-D] (for end-of-input) to get out of a select loop. This gives the user a uniform way of exiting, but it doesn't help the shell programmer much.

Let's refine our solution by making the menu more user-friendly, so that the user doesn't have to know the terminfo name of his or her terminal. We do this by using quoted character strings as menu items and then using case to determine the termcap name:

print 'Select your terminal type:'
PS3='terminal? '
select term in \
    'Givalt VT100' \
    'Tsoris VT220' \
    'Shande VT320' \
    'Vey VT520'
do
    case $REPLY in
        1 ) TERM=gl35a ;;
        2 ) TERM=t2000 ;;
        3 ) TERM=s531 ;;
        4 ) TERM=vt99 ;;
        * ) print 'invalid.' ;;
    esac
    if [[ -n $term ]]; then
        print TERM is $TERM
        break
    fi
done

This code looks a bit more like a menu routine in a conventional program, though select still provides the shortcut of converting the menu choices into numbers. We list each of the menu choices on its own line for reasons of readability, but once again we need continuation characters to keep the shell from complaining about syntax.

Here is what the user will see when this code is run:

Select your terminal type:
1) Givalt GL35a
2) Tsoris T-2000
3) Shande 531
4) Vey VT99
terminal?

This is a bit more informative than the previous code's output.

When the body of the select loop is entered, $term equals one of the four strings (or is null if the user made an invalid choice), while the built-in variable REPLY contains the number the user selects. We need a case statement to assign the correct value to TERM; we use the value of REPLY as the case selector.

Once the case statement is finished, the if checks to see if a valid choice was made, as in the previous solution. If the choice was valid, then TERM has already been assigned, so the code just prints a confirmation message and exits the select loop. If it wasn't valid, the select loop repeats the prompt and goes through the process again.

while and until

BASH provides while and until loops, but until loop is implemented incorrectly and has the test at the top, unlike analogous constructs in Pascal (while/do and repeat/until) and C (while and do/until).

NOTE: In shell the until condition is checked at the top of the loop, not at the bottom as it is in C and Pascal.

The result is that you can convert any until into a while by simply negating the condition. Therefore we will ignore the existence of until throughout the rest of course.

The syntax for while is:

while condition
do
    statements...
done

while and until are actually most useful when combined with features we will see in the next chapter, such as integer arithmetic, input/output of variables, and command-line processing. Yet we can show a useful example even with the machinery we have covered so far. This example implements a simplified version of the shell's built-in whence command.

By "simplified," we mean that we will implement only the part that checks all of the directories in your PATH for the command you give as argument (we won't implement checking for aliases, built-in commands, etc.).

We can do this by picking off the directories in PATH one by one, using one of the shell's pattern-matching operators, and seeing if there is a file with the given name in the directory that you have permission to execute. Here is the code:

mypath=$PATH:
while [[ -n $mypath ]]; do
    mydir=${mypath%%:*}
    if [[ -x $mydir/$1 && ! -d $mydir/$1 ]]; then
        print "$mydir/$1"
        return
    fi
    mypath=${mypath#*:}  
done
return 1

The only things that might need commentary here are regular expressions:

mypath=${mypath#*:}
mydir=${mypath%%:*}

The first of these uses another shell string operator: this one deletes the shortest match to the pattern given from the front of the string. By now, this type of operator should be familiar. This line deletes the front directory from $path and assigns the result back to path. The second line is the same as before the while: it finds the (new) front directory in $path and assigns it to dir. This sets up the loop for another iteration.

Thus, the code loops through all of the directories in PATH. It exits when it finds a matching executable file or when it has "eaten up" the entire PATH. If no matching executable file is found, it prints nothing and exits with an error status.

We can enhance this script a bit by taking advantage of the UNIX utility file(1). file examines files given as arguments and determines what type they are, based on the file's magic number and various heuristics (educated guesses). A magic number is a field in the header of an executable file that the linker sets to identify what type of executable it is.

If filename is an executable program (compiled from C or some other language), then typing file filename produces output similar to this:

filename: ELF 32-bit LSB executable 80386 Version 1

However, if filename is not an executable program, it will examine the first few lines and try to guess what kind of information the file contains. If the file contains text (as opposed to binary data), file will look for indications that it is English, shell commands, C, FORTRAN, troff(1) input, and various other things. file is wrong sometimes, but it is mostly correct.

We can just substitute file for print to print a more informative message in our script:

path=$PATH
dir=${path%%:*}
while [[ -n $path ]]; do
    if [[ -x $dir/$1 && ! -d $dir/$1 ]]; then
	  file $dir/$1
	  return
    fi
    path=${path#*:}
    dir=${path%%:*}
done
return 1

Assume that fred is an executable file in the directory /usr/bin, and that bob is a shell script in /usr/local/bin. Then typing file fred produces this output:

/usr/bin/fred: ELF 32-bit LSB executable 80386 Version 1

And typing file bob has this result:

/usr/local/bin/bob: commands text

Before we end this chapter, we have two final notes. First, notice that the statement dir=${path%%:*} appears in two places, before the start of the loop and as the last statement in the loop's body. Some diehard C hackers are offended by this Pascal-like coding technique. Certain features of the C language allow programmers to create loops of the form:

while iterative-step; condition; do
    ...
done

This is the same as the form of the script above: the iterative-step runs just before the condition each time around the loop.

We can write our script this way:

path=$PATH
while dir=${path%%:*}; [[ -n $path ]]; do
    if [[ -x $dir/$1 && ! -d $dir/$1 ]]; then
	file $dir/$1
	return
    fi
    path=${path#*:}
done
return 1

Although this example doesn't show great programming style, it does make the code smaller-hence its popularity with C programmers. Make sure you understand that our script is functionally identical to the previous script.

loops feeding by a pipe

Let's assume that we need to find all files that contain string "19%" which is a typical for printing commands like "19%2d"

cd/ /usr/bin
ls | while read $file
do
echo $file
string $file | grep '19%'
done

Here we use the ls command to generate the list of the file names and this is piped into a loop.

In a look we echo command and then run strings piped to grep looking for suspicious format strings.

# ksh only

for var in $srtlist
do
count=`grep -c $var $infile`
((grand=$grand + $count))
printf "$count\t$var"
done | sort -n # sort added here

NEWS CONTENTS

• 20190906 : Using Case Insensitive Matches with Bash Case Statements by Steven Vona ( Jun 30, 2019 , www.putorius.net )
• 20190902 : Switch statement for bash script ( Sep 02, 2019 , www.linuxquestions.org )
• 20151206 : Bash For Loop Examples ( June 24, 2015 , cyberciti.biz )
• 20100910 : bash iterator trick ( The UNIX Blog )

Old News ;-)

[Sep 06, 2019] Using Case Insensitive Matches with Bash Case Statements by Steven Vona

Jun 30, 2019 | www.putorius.net

If you want to match the pattern regardless of it's case (Capital letters or lowercase letters) you can set the nocasematch shell option with the shopt builtin. You can do this as the first line of your script. Since the script will run in a subshell it won't effect your normal environment.

#!/bin/bash
 shopt -s nocasematch
 read -p "Name a Star Trek character: " CHAR
 case $CHAR in
   "Seven of Nine" | Neelix | Chokotay | Tuvok | Janeway )
       echo "$CHAR was in Star Trek Voyager"
       ;;&
   Archer | Phlox | Tpol | Tucker )
       echo "$CHAR was in Star Trek Enterprise"
       ;;&
   Odo | Sisko | Dax | Worf | Quark )
       echo "$CHAR was in Star Trek Deep Space Nine"
       ;;&
   Worf | Data | Riker | Picard )
       echo "$CHAR was in Star Trek The Next Generation" &&  echo "/etc/redhat-release"
       ;;
   *) echo "$CHAR is not in this script." 
       ;;
 esac

[Sep 02, 2019] Switch statement for bash script

Sep 02, 2019 | www.linuxquestions.org

Switch statement for bash script <a rel='nofollow' target='_blank' href='//rev.linuxquestions.org/www/delivery/ck.php?n=a054b75'><img border='0' alt='' src='//rev.linuxquestions.org/www/delivery/avw.php?zoneid=10&amp;n=a054b75' /></a> [ Log in to get rid of this advertisement] Hello, i am currently trying out the switch statement using bash script. CODE: showmenu () { echo "1. Number1" echo "2. Number2" echo "3. Number3" echo "4. All" echo "5. Quit" } while true do showmenu read choice echo "Enter a choice:" case "$choice" in "1") echo "Number One" ;; "2") echo "Number Two" ;; "3") echo "Number Three" ;; "4") echo "Number One, Two, Three" ;; "5") echo "Program Exited" exit 0 ;; *) echo "Please enter number ONLY ranging from 1-5!" ;; esac done OUTPUT: 1. Number1 2. Number2 3. Number3 4. All 5. Quit Enter a choice: So, when the code is run, a menu with option 1-5 will be shown, then the user will be asked to enter a choice and finally an output is shown. But it is possible if the user want to enter multiple choices. For example, user enter choice "1" and "3", so the output will be "Number One" and "Number Three". Any idea?

Just something to get you started. Code: #! /bin/bash showmenu () { typeset ii typeset -i jj=1 typeset -i kk typeset -i valid=0 # valid=1 if input is good while (( ! valid )) do for ii in "${options[@]}" do echo "$jj) $ii" let jj++ done read -e -p 'Select a list of actions : ' -a answer jj=0 valid=1 for kk in "${answer[@]}" do if (( kk < 1 || kk > "${#options[@]}" )) then echo "Error Item $jj is out of bounds" 1>&2 valid=0 break fi let jj++ done done } typeset -r c1=Number1 typeset -r c2=Number2 typeset -r c3=Number3 typeset -r c4=All typeset -r c5=Quit typeset -ra options=($c1 $c2 $c3 $c4 $c5) typeset -a answer typeset -i kk while true do showmenu for kk in "${answer[@]}" do case $kk in 1) echo 'Number One' ;; 2) echo 'Number Two' ;; 3) echo 'Number Three' ;; 4) echo 'Number One, Two, Three' ;; 5) echo 'Program Exit' exit 0 ;; esac done done stevenworr View Public Profile View LQ Blog View Review Entries View HCL Entries Find More Posts by stevenworr 11-16-2009, 10:10 PM # 4 wjs1990 Member Registered: Nov 2009 Posts: 30 Original Poster Rep: Ok will try it out first. Thanks. Last edited by wjs1990; 11-16-2009 at 10:13 PM . wjs1990 View Public Profile View LQ Blog View Review Entries View HCL Entries Find More Posts by wjs1990 11-16-2009, 10:16 PM # 5 evo2 LQ Guru Registered: Jan 2009 Location: Japan Distribution: Mostly Debian and CentOS Posts: 5,945 Rep: This can be done just by wrapping your case block in a for loop and changing one line. Code: #!/bin/bash showmenu () { echo "1. Number1" echo "2. Number2" echo "3. Number3" echo "4. All" echo "5. Quit" } while true ; do showmenu read choices for choice in $choices ; do case "$choice" in 1) echo "Number One" ;; 2) echo "Number Two" ;; 3) echo "Number Three" ;; 4) echo "Numbers One, two, three" ;; 5) echo "Exit" exit 0 ;; *) echo "Please enter number ONLY ranging from 1-5!" ;; esac done done You can now enter any number of numbers seperated by white space. Cheers, EVo2.

[Dec 06, 2015] Bash For Loop Examples

A very nice tutorial by Vivek Gite (created October 31, 2008 last updated June 24, 2015). His mistake is putting new for loop too far inside the tutorial. It should emphazied, not hidden.

June 24, 2015 | cyberciti.biz

... ... ...

Bash v4.0+ has inbuilt support for setting up a step value using {START..END..INCREMENT} syntax:

#!/bin/bash
echo "Bash version ${BASH_VERSION}..."
for i in {0..10..2}
  do
     echo "Welcome $i times"
 done

Sample outputs:

Bash version 4.0.33(0)-release...
Welcome 0 times
Welcome 2 times
Welcome 4 times
Welcome 6 times
Welcome 8 times
Welcome 10 times

... ... ...

Three-expression bash for loops syntax

This type of for loop share a common heritage with the C programming language. It is characterized by a three-parameter loop control expression; consisting of an initializer (EXP1), a loop-test or condition (EXP2), and a counting expression (EXP3).

for (( EXP1; EXP2; EXP3 ))
do
	command1
	command2
	command3
done

A representative three-expression example in bash as follows:

#!/bin/bash
for (( c=1; c<=5; c++ ))
do
   echo "Welcome $c times"
done

... ... ...

Jadu Saikia, November 2, 2008, 3:37 pm

Nice one. All the examples are explained well, thanks Vivek.

seq 1 2 20
output can also be produced using jot

jot – 1 20 2

The infinite loops as everyone knows have the following alternatives.

while(true)
or
while :

//Jadu

Andi Reinbrech, November 18, 2010, 7:42 pm

I know this is an ancient thread, but thought this trick might be helpful to someone:

For the above example with all the cuts, simply do

set `echo $line`

This will split line into positional parameters and you can after the set simply say

F1=$1; F2=$2; F3=$3

I used this a lot many years ago on solaris with "set `date`", it neatly splits the whole date string into variables and saves lots of messy cutting :-)

… no, you can't change the FS, if it's not space, you can't use this method

Peko, July 16, 2009, 6:11 pm

Hi Vivek,
Thanks for this a useful topic.

IMNSHO, there may be something to modify here
=======================
Latest bash version 3.0+ has inbuilt support for setting up a step value:

#!/bin/bash
for i in {1..5}
=======================
1) The increment feature seems to belong to the version 4 of bash.
Reference: http://bash-hackers.org/wiki/doku.php/syntax/expansion/brace
Accordingly, my bash v3.2 does not include this feature.

BTW, where did you read that it was 3.0+ ?
(I ask because you may know some good website of interest on the subject).

2) The syntax is {from..to..step} where from, to, step are 3 integers.
You code is missing the increment.

Note that GNU Bash documentation may be bugged at this time,
because on GNU Bash manual, you will find the syntax {x..y[incr]}
which may be a typo. (missing the second ".." between y and increment).

see http://www.gnu.org/software/bash/manual/bashref.html#Brace-Expansion

The Bash Hackers page
again, see http://bash-hackers.org/wiki/doku.php/syntax/expansion/brace
seeems to be more accurate,
but who knows ? Anyway, at least one of them may be right… ;-)

Keep on the good work of your own,
Thanks a million.

- Peko

Michal Kaut July 22, 2009, 6:12 am

Hello,

is there a simple way to control the number formatting? I use several computers, some of which have non-US settings with comma as a decimal point. This means that
for x in $(seq 0 0.1 1) gives 0 0.1 0.2 … 1 one some machines and 0 0,1 0,2 … 1 on other.
Is there a way to force the first variant, regardless of the language settings? Can I, for example, set the keyboard to US inside the script? Or perhaps some alternative to $x that would convert commas to points?
(I am sending these as parameters to another code and it won't accept numbers with commas…)

The best thing I could think of is adding x=`echo $x | sed s/,/./` as a first line inside the loop, but there should be a better solution? (Interestingly, the sed command does not seem to be upset by me rewriting its variable.)

Thanks,
Michal

Peko July 22, 2009, 7:27 am

To Michal Kaut:

Hi Michal,

Such output format is configured through LOCALE settings.

I tried :

export LC_CTYPE="en_EN.UTF-8″; seq 0 0.1 1

and it works as desired.

You just have to find the exact value for LC_CTYPE that fits to your systems and your needs.

Peko

Peko July 22, 2009, 2:29 pm

To Michal Kaus [2]

Ooops – ;-)
Instead of LC_CTYPE,
LC_NUMERIC should be more appropriate
(Although LC_CTYPE is actually yielding to the same result – I tested both)

By the way, Vivek has already documented the matter : http://www.cyberciti.biz/tips/linux-find-supportable-character-sets.html

Philippe Petrinko October 30, 2009, 8:35 am

To Vivek:
Regarding your last example, that is : running a loop through arguments given to the script on the command line, there is a simplier way of doing this:
# instead of:
# FILES="$@"
# for f in $FILES

# use the following syntax
for arg
do
# whatever you need here – try : echo "$arg"
done

Of course, you can use any variable name, not only "arg".

Philippe Petrinko November 11, 2009, 11:25 am

To tdurden:

Why would'nt you use

1) either a [for] loop
for old in * ; do mv ${old} ${old}.new; done

2) Either the [rename] command ?
excerpt form "man rename" :

RENAME(1) Perl Programmers Reference Guide RENAME(1)

NAME
rename – renames multiple files

SYNOPSIS
rename [ -v ] [ -n ] [ -f ] perlexpr [ files ]

DESCRIPTION
"rename" renames the filenames supplied according to the rule specified
as the first argument. The perlexpr argument is a Perl expression
which is expected to modify the $_ string in Perl for at least some of
the filenames specified. If a given filename is not modified by the
expression, it will not be renamed. If no filenames are given on the
command line, filenames will be read via standard input.

For example, to rename all files matching "*.bak" to strip the
extension, you might say

rename 's/\.bak$//' *.bak

To translate uppercase names to lower, you'd use

rename 'y/A-Z/a-z/' *

- Philippe

Philippe Petrinko November 11, 2009, 9:27 pm

If you set the shell option extglob, Bash understands some more powerful patterns. Here, a is one or more pattern, separated by the pipe-symbol (|).

?() Matches zero or one occurrence of the given patterns
*() Matches zero or more occurrences of the given patterns
+() Matches one or more occurrences of the given patterns
@() Matches one of the given patterns
!() Matches anything except one of the given patterns

source: http://www.bash-hackers.org/wiki/doku.php/syntax/pattern

Philippe Petrinko November 12, 2009, 3:44 pm

To Sean:
Right, the more sharp a knife is, the easier it can cut your fingers…

I mean: There are side-effects to the use of file globbing (like in [ for f in * ] ) , when the globbing expression matches nothing: the globbing expression is not susbtitued.

Then you might want to consider using [ nullglob ] shell extension,
to prevent this.
see: http://www.bash-hackers.org/wiki/doku.php/syntax/expansion/globs#customization

Devil hides in detail ;-)

Dominic January 14, 2010, 10:04 am

There is an interesting difference between the exit value for two different for looping structures (hope this comes out right):
for (( c=1; c<=2; c++ )) do echo -n "inside (( )) loop c is $c, "; done; echo "done (( )) loop c is $c"
for c in {1..2}; do echo -n "inside { } loop c is $c, "; done; echo "done { } loop c is $c"
You see that the first structure does a final increment of c, the second does not. The first is more useful IMO because if you have a conditional break in the for loop, then you can subsequently test the value of $c to see if the for loop was broken or not; with the second structure you can't know whether the loop was broken on the last iteration or continued to completion.

Dominic January 14, 2010, 10:09 am

sorry, my previous post would have been clearer if I had shown the output of my code snippet, which is:
inside (( )) loop c is 1, inside (( )) loop c is 2, done (( )) loop c is 3
inside { } loop c is 1, inside { } loop c is 2, done { } loop c is 2

Philippe Petrinko March 9, 2010, 2:34 pm

@Dmitry

And, again, as stated many times up there, using [seq] is counter productive, because it requires a call to an external program, when you should Keep It Short and Simple, using only bash internals functions:


for ((c=1; c<21; c+=2)); do echo "Welcome $c times" ; done


(and I wonder why Vivek is sticking to that old solution which should be presented only for historical reasons when there was no way of using bash internals.
By the way, this historical recall should be placed only at topic end, and not on top of the topic, which makes newbies sticking to the not-up-to-date technique ;-) )

Sean March 9, 2010, 11:15 pm

I have a comment to add about using the builtin for (( … )) syntax. I would agree the builtin method is cleaner, but from what I've noticed with other builtin functionality, I had to check the speed advantage for myself. I wrote the following files:

builtin_count.sh:

#!/bin/bash
for ((i=1;i<=1000000;i++))
do
echo "Output $i"
done


seq_count.sh:

#!/bin/bash
for i in $(seq 1 1000000)
do
echo "Output $i"
done


And here were the results that I got:
time ./builtin_count.sh
real 0m22.122s
user 0m18.329s
sys 0m3.166s

time ./seq_count.sh
real 0m19.590s
user 0m15.326s
sys 0m2.503s

The performance increase isn't too significant, especially when you are probably going to be doing something a little more interesting inside of the for loop, but it does show that builtin commands are not necessarily faster.

Andi Reinbrech November 18, 2010, 8:35 pm

The reason why the external seq is faster, is because it is executed only once, and returns a huge splurb of space separated integers which need no further processing, apart from the for loop advancing to the next one for the variable substitution.

The internal loop is a nice and clean/readable construct, but it has a lot of overhead. The check expression is re-evaluated on every iteration, and a variable on the interpreter's heap gets incremented, possibly checked for overflow etc. etc.

Note that the check expression cannot be simplified or internally optimised by the interpreter because the value may change inside the loop's body (yes, there are cases where you'd want to do this, however rare and stupid they may seem), hence the variables are volatile and get re-evaluted.

I.e. botom line, the internal one has more overhead, the "seq" version is equivalent to either having 1000000 integers inside the script (hard coded), or reading once from a text file with 1000000 integers with a cat. Point being that it gets executed only once and becomes static.

OK, blah blah fishpaste, past my bed time :-)

Cheers,
Andi

Anthony Thyssen June 4, 2010, 6:53 am

The {1..10} syntax is pretty useful as you can use a variable with it!

limit=10
echo {1..${limit}}
{1..10}

You need to eval it to get it to work!

limit=10
eval "echo {1..${limit}}"
1 2 3 4 5 6 7 8 9 10

'seq' is not avilable on ALL system (MacOSX for example)
and BASH is not available on all systems either.

You are better off either using the old while-expr method for computer compatiblity!

   limit=10; n=1;
   while [ $n -le 10 ]; do
     echo $n;
     n=`expr $n + 1`;
   done

Alternativally use a seq() function replacement…

 # seq_count 10
seq_count() {
  i=1; while [ $i -le $1 ]; do echo $i; i=`expr $i + 1`; done
}
# simple_seq 1 2 10
simple_seq() {
  i=$1; while [ $i -le $3 ]; do echo $i; i=`expr $i + $2`; done
}
seq_integer() {
    if [ "X$1" = "X-f" ]
    then format="$2"; shift; shift
    else format="%d"
    fi
    case $# in
    1) i=1 inc=1 end=$1 ;;
    2) i=$1 inc=1 end=$2 ;;
    *) i=$1 inc=$2 end=$3 ;;
    esac
    while [ $i -le $end ]; do
      printf "$format\n" $i;
      i=`expr $i + $inc`;
    done
  }

Edited: by Admin – added code tags.

TheBonsai June 4, 2010, 9:57 am

The Bash C-style for loop was taken from KSH93, thus I guess it's at least portable towards Korn and Z.

The seq-function above could use i=$((i + inc)), if only POSIX matters. expr is obsolete for those things, even in POSIX.

Philippe Petrinko June 4, 2010, 10:15 am

Right Bonsai,
( http://www.opengroup.org/onlinepubs/009695399/utilities/xcu_chap02.html#tag_02_06_04 )

But FOR C-style does not seem to be POSIXLY-correct…

Read on-line reference issue 6/2004,
Top is here, http://www.opengroup.org/onlinepubs/009695399/mindex.html

and the Shell and Utilities volume (XCU) T.OC. is here
http://www.opengroup.org/onlinepubs/009695399/utilities/toc.html
doc is:
http://www.opengroup.org/onlinepubs/009695399/basedefs/xbd_chap01.html

and FOR command:
http://www.opengroup.org/onlinepubs/009695399/utilities/xcu_chap02.html#tag_02_09_04_03

Anthony Thyssen June 6, 2010, 7:18 am

TheBonsai wrote…. "The seq-function above could use i=$((i + inc)), if only POSIX matters. expr is obsolete for those things, even in POSIX."

I am not certain it is in Posix. It was NOT part of the original Bourne Shell, and on some machines, I deal with Bourne Shell. Not Ksh, Bash, or anything else.

Bourne Shell syntax works everywhere! But as 'expr' is a builtin in more modern shells, then it is not a big loss or slow down.

This is especially important if writing a replacement command, such as for "seq" where you want your "just-paste-it-in" function to work as widely as possible.

I have been shell programming pretty well all the time since 1988, so I know what I am talking about! Believe me.

MacOSX has in this regard been the worse, and a very big backward step in UNIX compatibility. 2 year after it came out, its shell still did not even understand most of the normal 'test' functions. A major pain to write shells scripts that need to also work on this system.

TheBonsai June 6, 2010, 12:35 pm

Yea, the question was if it's POSIX, not if it's 100% portable (which is a difference). The POSIX base more or less is a subset of the Korn features (88, 93), pure Bourne is something "else", I know. Real portability, which means a program can go wherever UNIX went, only in C ;)

Philippe Petrinko November 22, 2010, 8:23 am

And if you want to get rid of double-quotes, use:

one-liner code:
while read; do record=${REPLY}; echo ${record}|while read -d ","; do field="${REPLY#\"}"; field="${field%\"}"; echo ${field}; done; done<data


script code, added of some text to better see record and field breakdown:

#!/bin/bash
while read
do
echo "New record"
record=${REPLY}
echo ${record}|while read -d ,
do
field="${REPLY#\"}"
field="${field%\"}"
echo "Field is :${field}:"
done
done<data


Does it work with your data?

- PP

Philippe Petrinko November 22, 2010, 9:01 am

Of course, all the above code was assuming that your CSV file is named "data".

If you want to use anyname with the script, replace:

done<data

With:

done


And then use your script file (named for instance "myScript") with standard input redirection:

myScript < anyFileNameYouWant


Enjoy!

Philippe Petrinko November 22, 2010, 11:28 am

well no there is a bug, last field of each record is not read – it needs a workout and may be IFS modification ! After all that's what it was built for… :O)

Anthony Thyssen November 22, 2010, 11:31 pm

Another bug is the inner loop is a pipeline, so you can't assign variables for use later in the script. but you can use '<<<' to break the pipeline and avoid the echo.

But this does not help when you have commas within the quotes! Which is why you needed quotes in the first place.

In any case It is a little off topic. Perhaps a new thread for reading CVS files in shell should be created.

Philippe Petrinko November 24, 2010, 6:29 pm

Anthony,
Would you try this one-liner script on your CSV file?

This one-liner assumes that CSV file named [data] has __every__ field double-quoted.


while read; do r="${REPLY#\"}";echo "${r//\",\"/\"}"|while read -d \";do echo "Field is :${REPLY}:";done;done<data


Here is the same code, but for a script file, not a one-liner tweak.


#!/bin/bash
# script csv01.sh
#
# 1) Usage
# This script reads from standard input
# any CSV with double-quoted data fields
# and breaks down each field on standard output
#
# 2) Within each record (line), _every_ field MUST:
# - Be surrounded by double quotes,
# - and be separated from preceeding field by a comma
# (not the first field of course, no comma before the first field)
#
while read
do
echo "New record" # this is not mandatory-just for explanation
#
#
# store REPLY and remove opening double quote
record="${REPLY#\"}"
#
#
# replace every "," by a single double quote
record=${record//\",\"/\"}
#
#
echo ${record}|while read -d \"
do
# store REPLY into variable "field"
field="${REPLY}"
#
#
echo "Field is :${field}:" # just for explanation
done
done


This script named here [cvs01.sh] must be used so:

cvs01.sh < my-cvs-file-with-doublequotes


Philippe Petrinko November 24, 2010, 6:35 pm

@Anthony,

By the way, using [REPLY] in the outer loop _and_ the inner loop is not a bug.
As long as you know what you do, this is not problem, you just have to store [REPLY] value conveniently, as this script shows.

TheBonsai March 8, 2011, 6:26 am

for ((i=1; i<=20; i++)); do printf "%02d\n" "$i"; done

nixCraft March 8, 2011, 6:37 am

+1 for printf due to portability, but you can use bashy .. syntax too

for i in {01..20}; do echo "$i"; done

TheBonsai March 8, 2011, 6:48 am

Well, it isn't portable per se, it makes it portable to pre-4 Bash versions.

I think a more or less "portable" (in terms of POSIX, at least) code would be

i=0
while [ "$((i >= 20))" -eq 0 ]; do
  printf "%02d\n" "$i"
  i=$((i+1))
done

Philip Ratzsch April 20, 2011, 5:53 am

I didn't see this in the article or any of the comments so I thought I'd share. While this is a contrived example, I find that nesting two groups can help squeeze a two-liner (once for each range) into a one-liner:

for num in {{1..10},{15..20}};do echo $num;done

Great reference article!

Philippe Petrinko April 20, 2011, 8:23 am

@Philip
Nice thing to think of, using brace nesting, thanks for sharing.

Philippe Petrinko May 6, 2011, 10:13 am

Hello Sanya,

That would be because brace expansion does not support variables. I have to check this.
Anyway, Keep It Short and Simple: (KISS) here is a simple solution I already gave above:

xstart=1;xend=10;xstep=1
for (( x = $xstart; x <= $xend; x += $xstep)); do echo $x;done

Actually, POSIX compliance allows to forget $ in for quotes, as said before, you could also write:

xstart=1;xend=10;xstep=1
for (( x = xstart; x <= xend; x += xstep)); do echo $x;done

Philippe Petrinko May 6, 2011, 10:48 am

Sanya,

Actually brace expansion happens __before__ $ parameter exapansion, so you cannot use it this way.

Nevertheless, you could overcome this this way:

max=10; for i in $(eval echo {1..$max}); do echo $i; done

Sanya May 6, 2011, 11:42 am

Hello, Philippe

Thanks for your suggestions
You basically confirmed my findings, that bash constructions are not as simple as zsh ones.
But since I don't care about POSIX compliance, and want to keep my scripts "readable" for less experienced people, I would prefer to stick to zsh where my simple for-loop works

Cheers, Sanya

Philippe Petrinko May 6, 2011, 12:07 pm

Sanya,

First, you got it wrong: solutions I gave are not related to POSIX, I just pointed out that POSIX allows not to use $ in for (( )), which is just a little bit more readable – sort of.

Second, why do you see this less readable than your [zsh] [for loop]?

for (( x = start; x <= end; x += step)) do
echo "Loop number ${x}"
done

It is clear that it is a loop, loop increments and limits are clear.

IMNSHO, if anyone cannot read this right, he should not be allowed to code. :-D

BFN

Anthony Thyssen May 8, 2011, 11:30 pm

If you are going to do… $(eval echo {1..$max});
You may as well use "seq" or one of the many other forms.
See all the other comments on doing for loops.

Tom P May 19, 2011, 12:16 pm

I am trying to use the variable I set in the for line on to set another variable with a different extension. Couldn't get this to work and couldnt find it anywhere on the web… Can someone help.

Example:

FILE_TOKEN=`cat /tmp/All_Tokens.txt`
for token in $FILE_TOKEN
do
A1_$token=`grep $A1_token /file/path/file.txt | cut -d ":" -f2`

my goal is to take the values from the ALL Tokens file and set a new variable with A1_ infront of it… This tells be that A1_ is not a command…

[Sep 10, 2010] bash iterator trick

The UNIX Blog

A neat little feature I never new existed in bash is being able to iterate over a sequence of number in a more or less C-esque manner. Coming from Bourne/Korn shell background creating an elegant iterator is always a slight nuisance, since you would come up with something like this to iterate over a sequence of numbers:

i=1;
while [ $i -lt 10 ]; do
i=`expr $i + 1`;
.....
done

Well, not exactly the most elegant solution. With bash on the other hand it can be done as simple as:

for((i=1; $i<10; i++)); do
....
done

Simple and to the point.

Recommended Links

Softpanorama Recommended

Top articles

Sites

Advanced Bash-Scripting Guide

• The URL of the preferred download site for the pdf version of the ABS Guide is:

  http://personal.riverusers.com/~thegrendel/abs-guide.pdf

  This version is specially book-formatted for duplex printing and is usually more up-to-date than the version you can download from the LDP site. Note that it's a 2.6 MB download.

• http://personal.riverusers.com/~thegrendel/abs-guide-5.2.tar.bz2 The "bleeding edge" branch is the 'tar/BZ2' link. It will generally run anywhere from a few days to a couple of months ahead of the main release on the LDP site.

Etc

Society

Groupthink : Two Party System as Polyarchy : Corruption of Regulators : Bureaucracies : Understanding Micromanagers and Control Freaks : Toxic Managers :   Harvard Mafia : Diplomatic Communication : Surviving a Bad Performance Review : Insufficient Retirement Funds as Immanent Problem of Neoliberal Regime : PseudoScience : Who Rules America : Neoliberalism  : The Iron Law of Oligarchy : Libertarian Philosophy

Quotes

War and Peace : Skeptical Finance : John Kenneth Galbraith :Talleyrand : Oscar Wilde : Otto Von Bismarck : Keynes : George Carlin : Skeptics : Propaganda  : SE quotes : Language Design and Programming Quotes : Random IT-related quotes :  Somerset Maugham : Marcus Aurelius : Kurt Vonnegut : Eric Hoffer : Winston Churchill : Napoleon Bonaparte : Ambrose Bierce :  Bernard Shaw : Mark Twain Quotes

Bulletin:

Vol 25, No.12 (December, 2013) Rational Fools vs. Efficient Crooks The efficient markets hypothesis : Political Skeptic Bulletin, 2013 : Unemployment Bulletin, 2010 :  Vol 23, No.10 (October, 2011) An observation about corporate security departments : Slightly Skeptical Euromaydan Chronicles, June 2014 : Greenspan legacy bulletin, 2008 : Vol 25, No.10 (October, 2013) Cryptolocker Trojan (Win32/Crilock.A) : Vol 25, No.08 (August, 2013) Cloud providers as intelligence collection hubs : Financial Humor Bulletin, 2010 : Inequality Bulletin, 2009 : Financial Humor Bulletin, 2008 : Copyleft Problems Bulletin, 2004 : Financial Humor Bulletin, 2011 : Energy Bulletin, 2010 : Malware Protection Bulletin, 2010 : Vol 26, No.1 (January, 2013) Object-Oriented Cult : Political Skeptic Bulletin, 2011 : Vol 23, No.11 (November, 2011) Softpanorama classification of sysadmin horror stories : Vol 25, No.05 (May, 2013) Corporate bullshit as a communication method  : Vol 25, No.06 (June, 2013) A Note on the Relationship of Brooks Law and Conway Law

History:

Fifty glorious years (1950-2000): the triumph of the US computer engineering : Donald Knuth : TAoCP and its Influence of Computer Science : Richard Stallman : Linus Torvalds  : Larry Wall  : John K. Ousterhout : CTSS : Multix OS Unix History : Unix shell history : VI editor : History of pipes concept : Solaris : MS DOS :  Programming Languages History : PL/1 : Simula 67 : C : History of GCC development :  Scripting Languages : Perl history   : OS History : Mail : DNS : SSH : CPU Instruction Sets : SPARC systems 1987-2006 : Norton Commander : Norton Utilities : Norton Ghost : Frontpage history : Malware Defense History : GNU Screen : OSS early history

Classic books:

The Peter Principle : Parkinson Law : 1984 : The Mythical Man-Month :  How to Solve It by George Polya : The Art of Computer Programming : The Elements of Programming Style : The Unix Hater’s Handbook : The Jargon file : The True Believer : Programming Pearls : The Good Soldier Svejk : The Power Elite

Most popular humor pages:

Manifest of the Softpanorama IT Slacker Society : Ten Commandments of the IT Slackers Society : Computer Humor Collection : BSD Logo Story : The Cuckoo's Egg : IT Slang : C++ Humor : ARE YOU A BBS ADDICT? : The Perl Purity Test : Object oriented programmers of all nations : Financial Humor : Financial Humor Bulletin, 2008 : Financial Humor Bulletin, 2010 : The Most Comprehensive Collection of Editor-related Humor : Programming Language Humor : Goldman Sachs related humor : Greenspan humor : C Humor : Scripting Humor : Real Programmers Humor : Web Humor : GPL-related Humor : OFM Humor : Politically Incorrect Humor : IDS Humor : "Linux Sucks" Humor : Russian Musical Humor : Best Russian Programmer Humor : Microsoft plans to buy Catholic Church : Richard Stallman Related Humor : Admin Humor : Perl-related Humor : Linus Torvalds Related humor : PseudoScience Related Humor : Networking Humor : Shell Humor : Financial Humor Bulletin, 2011 : Financial Humor Bulletin, 2012 : Financial Humor Bulletin, 2013 : Java Humor : Software Engineering Humor : Sun Solaris Related Humor : Education Humor : IBM Humor : Assembler-related Humor : VIM Humor : Computer Viruses Humor : Bright tomorrow is rescheduled to a day after tomorrow : Classic Computer Humor

The Last but not Least Technology is dominated by two types of people: those who understand what they do not manage and those who manage what they do not understand ~Archibald Putt. Ph.D

The statements, views and opinions presented on this web page are those of the author (or referenced source) and are not endorsed by, nor do they necessarily reflect, the opinions of the Softpanorama society. We do not warrant the correctness of the information provided or its fitness for any purpose. The site uses AdSense so you need to be aware of Google privacy policy. You you do not want to be tracked by Google please disable Javascript for this site. This site is perfectly usable without Javascript.

Last modified: September 10, 2019