<!DOCTYPE HTML PUBLIC “-W3CDTD HTML 4.0 TransitionalEN”>
<html>
<head>
<!– This file is generated automatically by buildhtml. Do not edit –>
<meta name=“AUTHOR” content=“Richard Brittain”>
<meta name=“keywords” content=“bourne, korn, shell, unix”>
<meta name=“description” content=“A tutorial on Unix shell scripting with Bourne and Korn shells”>
<title>Unix shell scripting with ksh/bash</title>
<style type=text/css media=print> .newpage { page-break-before: always; }</style>
</head>
<body bgcolor=WHITE>
<h1 align=CENTER>Unix shell scripting with ksh/bash======
<h3 align=CENTER> Course Handout: (last update Thursday, 22-Mar-2012 12:43:56 EDT) ====
<br>
<small>These notes may be found at <a href=“http://www.dartmouth.edu/~rc/classes/ksh”>http://www.dartmouth.edu/~rc/classes/ksh</a>. The online version has many links to additional information and may be more up to date than the printed notes</small>
<hr>
<p align=RIGHT ><font size=-1></font></P>
<font color=DARKBLUE>======UNIX shell scripting with ksh/bash</H1></font>
==== The goals of this class are to enable you to:====
<UL TYPE=DISC>
<LI>Learn what kinds of problems are suited to shell scripts
<LI>Review the most commonly used Unix commands that are useful in shell scripts.
<LI>Write simple shell scripts using the Bourne, Korn or Bash shells
</UL>
These notes are intended for use in a 2-part class, total duration 3 hours.
<P>
<B>Assumptions:</B><BR>
It is assumed that you already know how to:
<UL>
<LI>log in and get a command line window (any shell)
<LI>run basic commands, navigate directories
<LI>use simple I/O redirection and pipes
<LI>use a text editor (any one)
<LI>look up details of command usage in man pages
</UL>
Example commands are shown <font color=DARKRED><b><code>like this</code></b></font>. Many commands
are shown with links to their full man pages
(<a href=“http://www.FreeBSD.org/cgi/man.cgi?query=sh&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b><code>sh</code></b></font></a>)<BR>
Output from commands is shown <TT>like this</TT>;
optional items are <I>in brackets</I>.
<P>
Some descriptions in these notes have more detail available,
and are denoted like this:
<blockquote><small>
More details of this item would appear here. The printed notes
include all of the additional information
</small></blockquote>
<HR>
<SMALL>
Permission is granted to download and use these notes and example scripts, as long as
all copyright notices are kept intact. Some of the examples are taken from texts
or online resources which have granted permission to redistribute.
</SMALL>
<blockquote><small>
<SMALL>
These notes are updated from time to time. The “development”
set of notes are
<A HREF=“http://northstar-www.dartmouth.edu/~richard/classes/ksh” TARGET=_top> http://northstar-www.dartmouth.edu/~richard/classes/ksh </A>
(Dartmouth only)
<P>
<SMALL>
Richard Brittain, Dartmouth College Computing Services.
<BR>
© 2003,2004,2010 Dartmouth College.
<BR>
Comments and questions, contact Richard.Brittain @ dartmouth.edu
</SMALL>
</SMALL>
<P><CENTER><A HREF=“http://www.anybrowser.org/campaign/” TARGET=“_top”>
<IMG SRC=“../images/ab8.gif” ALT=“Best viewed with ANY browser”></A> <IMG SRC=“../images/vipower.gif” ALT=“Powered by Vi”></CENTER>
</small></blockquote>
<hr>
<p align=“center” class=“newpage”>Table of Contents</p>
<table>
<tr><td align=right>1.</td><td><a href=what.html#top >What is a shell script</a></td></tr>
<tr><td align=right>2.</td><td><a href=why.html#top >Why use shell scripts</a></td></tr>
<tr><td align=right>3.</td><td><a href=history.html#top >History</a></td></tr>
<tr><td align=right>4.</td><td><a href=comparison.html#top >Feature comparison</a></td></tr>
<tr><td align=right>5.</td><td><a href=scripting-languages.html#top >Other scripting languages</a></td></tr>
<tr><td align=right>6.</td><td><a href=ksh-vs-sh.html#top >ksh/bash vs sh</a></td></tr>
<tr><td align=right>7.</td><td><a href=basics.html#top >Basics</a></td></tr>
<tr><td align=right>8.</td><td><a href=wildcards.html#top >Filename Wildcards</a></td></tr>
<tr><td align=right>9.</td><td><a href=variables.html#top >Variables</a></td></tr>
<tr><td align=right>10.</td><td><a href=special-vars.html#top >Preset Variables</a></td></tr>
<tr><td align=right>11.</td><td><a href=arguments.html#top >Arguments</a></td></tr>
<tr><td align=right>12.</td><td><a href=command_line_opts.html#top >Shell options</a></td></tr>
<tr><td align=right>13.</td><td><a href=command-subs.html#top >Command substitution</a></td></tr>
<tr><td align=right>14.</td><td><a href=ioredirection.html#top >I/O redirection and pipelines</a></td></tr>
<tr><td align=right>15.</td><td><a href=stdio.html#top >Input and output</a></td></tr>
<tr><td align=right>16.</td><td><a href=cond-tests.html#top >Conditional Tests</a></td></tr>
<tr><td align=right>17.</td><td><a href=cond-tests2.html#top >Conditional Tests (contd.)</a></td></tr>
<tr><td align=right>18.</td><td><a href=flow-control.html#top >Flow control</a></td></tr>
<tr><td align=right>19.</td><td><a href=flow-control-2.html#top >Flow control (contd.)</a></td></tr>
<tr><td align=right>20.</td><td><a href=flow-control-ex.html#top >Conditional test examples</a></td></tr>
<tr><td align=right>21.</td><td><a href=misc-internal.html#top >Miscellaneous</a></td></tr>
<tr><td align=right>22.</td><td><a href=variable-manipulation.html#top >Manipulating Variables</a></td></tr>
<tr><td align=right>23.</td><td><a href=functions.html#top >Functions</a></td></tr>
<tr><td align=right>24.</td><td><a href=advio.html#top >Advanced I/O</a></td></tr>
<tr><td align=right>25.</td><td><a href=wizardio.html#top >Wizard I/O</a></td></tr>
<tr><td align=right>26.</td><td><a href=coprocesses.html#top >Coprocesses</a></td></tr>
<tr><td align=right>27.</td><td><a href=arrays.html#top >Arrays</a></td></tr>
<tr><td align=right>28.</td><td><a href=signals.html#top >Signals</a></td></tr>
<tr><td align=right>29.</td><td><a href=security.html#top >Security</a></td></tr>
<tr><td align=right>30.</td><td><a href=style.html#top >Style</a></td></tr>
<tr><td align=right>31.</td><td><a href=examples.html#top >Examples</a></td></tr>
<tr><td align=right>32.</td><td><a href=externals.html#top >Common external commands</a></td></tr>
<tr><td align=right>33.</td><td><a href=references.html#top >References</a></td></tr>
</table>
<p align=RIGHT class=“newpage”><font size=-1>(1)</font></P>
<font color=DARKBLUE>======What is a Shell Script</H1></font>
<UL>
<LI>A text file containing commands which could have been typed directly into the shell.
<blockquote><small>
There is no difference in syntax between interactive command line use and placing the commands
in a file. Some commands are only useful when used interactively (e.g. command line history recall)
and other commands are too complex to use interactively.
</small></blockquote>
<P>
<LI>The shell itself has limited capabilities – the power comes from using it as a “glue”
language to combine the standard Unix utilities, and custom software, to produce a tool
more useful than the component parts alone.
<P>
<LI>Any shell can be used for writing a shell script. To allow for this, the first line of every script
is:<BR>
<font color=DARKRED><b><code>#!/path/to/shell</code></b></font> (e.g. <font color=DARKRED><b><code>#!/bin/ksh</code></b></font>).
<blockquote><small>
The <font color=DARKRED><b><code>#!</code></b></font> characters tell the system to locate the following pathname, start it up and
feed it the rest of the file as input.
<EM>Any</EM> program which can read commands from a file can be started up this way, as long as it
recognizes the <font color=DARKRED><b><code>#</code></b></font> comment convention. The program is started, and then the script file
is given to it as an argument. Because of this, the script must be readable as well as executable.
Examples are perl, awk, tcl and python.
</small></blockquote>
<P>
<LI>Any file can be used as input to a shell by using the syntax:<BR>
<font color=DARKRED><b><code>ksh myscript</code></b></font>
<P>
<LI>If the file is made executable using <a href=“http://www.FreeBSD.org/cgi/man.cgi?query=chmod&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b><code>chmod</code></b></font></a>, it becomes a new command
and available for use (subject to the usual $PATH search).<BR>
<font color=DARKRED><b><code>chmod +x myscript</code></b></font>
</UL>
A shell script can be as simple as a sequence of commands that you type regularly. By putting
them into a script, you reduce them to a single command.
<P>
<I>Example:</I> ex0 <A HREF=ex0.html TARGET=example>display</A>, <A HREF=ex0.txt TARGET=example>text</A>
<BLOCKQUOTE><SMALL>
<code><font color=DARKRED>
<font color=GRAY> 1:</font> <font color=DARKGREEN><i>#!/bin/sh</i></font>
<font color=GRAY> 2:</font> date
<font color=GRAY> 3:</font> <font color=DARKBLUE>pwd</font>
<font color=GRAY> 4:</font> du -k
</font></code>
</SMALL></BLOCKQUOTE>
<p align=RIGHT class=“newpage”><font size=-1>(2)</font></P>
<font color=DARKBLUE>======Why use Shell Scripts</H1></font>
<UL>
<LI>Combine lengthy and repetitive sequences of commands into a single, simple command.
<P>
<LI>Generalize a sequence of operations on one set of data, into a procedure that can be
applied to any similar set of data.
<blockquote><small>
(e.g.
apply the same analysis to every data file on a CD, without needing to repeat the commands)
</small></blockquote>
<P>
<LI>Create new commands using combinations of utilities in ways the original authors
never thought of.
<P>
<LI>Simple shell scripts might be written as shell aliases, but the script can be made available to all users
and all processes. Shell aliases apply only to the current shell.
<P>
<LI>Wrap programs over which you have no control inside an environment that you can control.
<blockquote><small>
e.g. set environment variables, switch to a special directory, create or select a configuration file,
redirect output, log usage, and then run the program.
</small></blockquote>
<P>
<LI>Create customized datasets on the fly, and call applications (e.g. matlab, sas, idl, gnuplot) to work
on them, or create customized application commands/procedures.
<P>
<LI>Rapid prototyping (but avoid letting prototypes become production)
</UL>
=====Typical uses=====
<UL>
<LI>System boot scripts (/etc/init.d)
<P>
<LI>System administrators, for automating many aspects of computer maintenance, user account
creation etc.
<P>
<LI>Application package installation tools
<blockquote><small>
Other tools may create fancier installers (e.g. tcl/tk), but can not be assumed to be installed already. Shell scripts are used because they are very portable. Some software comes with a complete installation of the tool it wants to use
(tcl/tk/python) in order to be self contained, but this leads to software bloat.
</small></blockquote>
<P>
<LI>Application startup scripts, especially
unattended applications (e.g. started from <a href=“http://www.FreeBSD.org/cgi/man.cgi?query=cron&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b><code>cron</code></b></font></a> or <a href=“http://www.FreeBSD.org/cgi/man.cgi?query=at&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b><code>at</code></b></font></a>)
<P>
<LI>Any user needing to automate the process of setting up and running commercial applications,
or their own code.
</UL>
<P ALIGN=CENTER><STRONG>AUTOMATE, AUTOMATE, AUTOMATE</STRONG></P>
<p align=RIGHT class=“newpage”><font size=-1>(3)</font></P>
<font color=DARKBLUE>======History of Shells</H1></font>
<DL>
<DT><a href=“http://www.FreeBSD.org/cgi/man.cgi?query=sh&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b><code>sh</code></b></font></a>
<DD>aka “Bourne” shell, written by Steve Bourne at AT&T Bell Labs for Unix V7 (1979).
Small, simple, and (originally) <EM>very few internal commands</EM>, so it called external programs for even the simplest
of tasks. It is always available on everything that looks vaguely like Unix.
<P>
<DT><FONT COLOR=“maroon”><a href=“http://www.FreeBSD.org/cgi/man.cgi?query=csh&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b><code>csh</code></b></font></a>
<DD>The “C” shell. (Bill Joy, at Berkeley).
Many things in common
with the Bourne shell, but many enhancements to improve interactive use. The internal
commands used only in scripts are <STRONG>very</STRONG> different from “sh”, and
similar (by design) to the “C” language syntax.
</FONT>
<P>
<DT><FONT COLOR=“maroon”>
<a href=“http://www.FreeBSD.org/cgi/man.cgi?query=tcsh&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b><code>tcsh</code></b></font></a>
<DD>The “TC” shell. Freely available and based on “csh”.
It has many additional features to make interactive use more convenient.
<blockquote><small>
We use it as the default interactive shell for new accounts on all of our public systems.
<BR>
Not many people write scripts in tcsh.
See <A HREF=“http://www.faqs.org/faqs/unix-faq/shell/csh-whynot/”>Csh Programming Considered Harmful</A> by
Tom Christiansen for a discussion of problems with programming csh scripts.
</small></blockquote>
</FONT>
<P>
<DT><a href=“http://www.FreeBSD.org/cgi/man.cgi?query=ksh&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b><code>ksh</code></b></font></a>
<DD>The “Korn” shell, written by David Korn of AT&T Bell Labs (now AT&T Research).
Written as a major upgrade to “sh” and backwards compatible with it, but has many internal commands for the
most frequently used functions. It also incorporates many of the features from tcsh which enhance
interactive use (command line history recall etc.).
<blockquote><small>
It was slow to gain acceptance because earlier versions were encumbered by AT&T licensing.
This shell is now freely available on all systems, but sometimes not installed by default on “free” Unix.
There are two major versions. ksh88 was the version incorporated into AT&T SVR4 Unix, and may still be installed
by some of the commercial Unix vendors. ksh93 added more features, primarily for programming, and better POSIX compliance.
</small></blockquote>
<P>
<DT>POSIX 1003.2 Shell Standard.
<DD>Standards committees worked over the Bourne shell and added many features of the Korn shell (ksh88) and
C shell to define a standard set of features which all compliant shells must have.
<blockquote><small>
On most systems, /bin/sh is now a POSIX compliant shell.
Korn shell and Bash are POSIX compliant, but have many features which go beyond the standard.
On Solaris, the POSIX/XPG4 commands which differ
slightly in behaviour from traditional SunOS commands are located in /usr/xpg4/bin
</small></blockquote>
<P>
<DT><a href=“http://www.FreeBSD.org/cgi/man.cgi?query=bash&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b><code>bash</code></b></font></a>
<DD>The “Bourne again” shell. Written as part of the GNU/Linux Open Source effort, and the default shell for Linux and
Mac OS-X. It is a functional clone of sh, with additional features to enhance interactive use,
add POSIX compliance, and partial ksh compatability.
<P>
<DT><a href=“http://www.FreeBSD.org/cgi/man.cgi?query=zsh&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b><code>zsh</code></b></font></a>
<DD>A freeware functional clone of sh, with parts of ksh, bash and full POSIX compliance, and
many new interactive command-line editing features.
It was installed as the default shell on early MacOSX systems.
</DL>
<p align=RIGHT class=“newpage”><font size=-1>(4)</font></P>
<font color=DARKBLUE>======Comparison of shell features</H1></font>
All the shells just listed share some common features, and the major differences in syntax
generally only affect script writers. It is not unusual
to use one shell (e.g. tcsh) for interactive use, but another (sh or ksh)
for writing scripts.
====Core Similarities (and recap of basic command line usage)====
Each of these items is discussed in more detail later.
<UL>
<LI>Parse lines by whitespace, search for external commands using <font color=DARKRED><b><code>$PATH</code></b></font>.
<LI>Can run a shell script using <font color=DARKRED><b><code>shellname scriptfile</code></b></font>, or run a single
<code>
command using <font color=DARKRED><b><code>shellname -c “command”</code></b></font>
</code>
<LI>Pass expanded command line arguments to programs; get exit status back.
<LI>Pass environment variables to programs.
<LI>Expand filename wildcards using <font color=DARKRED><b><code>advanced_shell_scripting*?</code></b></font>. Each shell has some additional
wildcard metacharacters, but these are common to all shells.
<LI>Standard I/O redirection and piping with <font color=DARKRED><b><code><,>,>>,|</code></b></font>
<LI>A few internal functions (<a href=“http://www.FreeBSD.org/cgi/man.cgi?query=cd&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b><code>cd</code></b></font></a>)
<LI>Backgrounding commands with <font color=DARKRED><b><code>&</code></b></font>
<LI>Quoting rules: “double quotes” protect most things, but allow <font color=DARKRED><b><code>$var</code></b></font> interpretation;
'single quotes' protect all metacharacters from interpretation.
<LI>Home directory expansion using <font color=DARKRED><b><code>~user</code></b></font> (except for <font color=DARKRED><b><code>sh</code></b></font>)
<P>
<LI><font color=DARKRED><b><code>#</code></b></font> comments
<LI>Command substitution using <font color=DARKRED><b><code>`command`</code></b></font> (backtics)
<LI>Expand variables using <font color=DARKRED><b><code>$varname</code></b></font> syntax
<LI>Conditional execution using <font color=DARKRED><b><code>&&</code></b></font> and <font color=DARKRED><b><code>||</code></b></font>
<LI>Line continuation with “<font color=DARKRED><b><code>\\</code></b></font>”
</UL>
====Principal Differences====
between sh (+derivitives), and csh (+derivitives).
<UL>
<LI>Syntax of all the flow control constructs and conditional tests.
<LI>Syntax for string manipulation inside of scripts
<LI>Syntax for arithmetic manipulation inside of scripts
<LI>Syntax for setting local variables (used only in the script) and environment variables (which are
passed to child processes). <a href=“http://www.FreeBSD.org/cgi/man.cgi?query=setenv&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b><code>setenv</code></b></font></a> vs <a href=“http://www.FreeBSD.org/cgi/man.cgi?query=export&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b><code>export</code></b></font></a>
<LI>Syntax for redirecting I/O streams other than stdin/stdout
<LI>Login startup files (<font color=DARKRED><b><code>.cshrc</code></b></font> and <font color=DARKRED><b><code>.login</code></b></font>, vs <font color=DARKRED><b><code>.profile</code></b></font>) and default options
<LI>Reading other shell scripts into the current shell (<font color=DARKRED><b><code>source filename</code></b></font>, vs <font color=DARKRED><b><code>. filename</code></b></font>)
<LI>Handling of signals (interrupts)
</UL>
<p align=RIGHT class=“newpage”><font size=-1>(5)</font></P>
<font color=DARKBLUE>======Other Scripting Languages</H1></font>
There are many other programs which read a file of commands and carry out
a sequence of actions. The <font color=DARKRED><b><code>“#!/path/to/program”</code></b></font> convention
allows any of them to be used as a scripting language to create new commands.
Some are highly specialized, and some are much more efficient than the equivalent
shell scripts at certain tasks. There is never only one way to perform a function,
and often the choice comes down to factors like:
<UL>
<LI>what is installed already - many other scripting languages are not available by default
<LI>what similar code already exists
<LI>what you are most familiar with and can use most efficiently.
Your time is always more expensive than computer cycles.
</UL>
Some major players (all of these are freely available) in the general purpose scripting languages are:
<UL>
<LI><a href=“http://www.FreeBSD.org/cgi/man.cgi?query=awk&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b><code>awk</code></b></font></a>
<blockquote><small>
A pattern matching and data (text and numeric) manipulation tool. Predates perl. Installed on
all Unix systems. Often used in combination with shell scripts.
</small></blockquote>
<LI><a href=“http://www.FreeBSD.org/cgi/man.cgi?query=perl&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b><code>perl</code></b></font></a>
<blockquote><small>
The most used scripting language for Web CGI applications and system administration tasks. Perl
is harder to learn, and is uusually installed by default now.
It is more efficient and has an enormous library of functions available. You could use Perl for
almost all scripting tasks, but the syntax is very different to the shell command line
</small></blockquote>
<LI><a href=“http://www.FreeBSD.org/cgi/man.cgi?query=python&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b><code>python</code></b></font></a>
<blockquote><small>
An object-oriented scripting language. Commonly installed by default on modern systems.
</small></blockquote>
<LI><a href=“http://www.FreeBSD.org/cgi/man.cgi?query=tcl&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b><code>tcl</code></b></font></a>/<a href=“http://www.FreeBSD.org/cgi/man.cgi?query=tk&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b><code>tk</code></b></font></a>
<blockquote><small>
Tool Command Language. Another general purpose scripting language. The “tk” component is a
scripted interface to standard X-windows graphical components, so the combination is often used
to create graphical user interfaces.
<P>
Ksh93 can be extended by linking to shared libraries providing additional internal commands. One example
of an extended shell is
<A HREF=“http://www.cs.princeton.edu/~jlk/tksh/”“ TARGET=“_blank”><font color=DARKRED><b><code>tksh</code></b></font></A>
which incorporates Tcl/Tk with ksh and allows generation of
scripts using both languages. It can be used for prototyping GUI applications.
</small></blockquote>
</UL>
<p align=RIGHT class=“newpage”><font size=-1>(6)</font></P>
<font color=DARKBLUE>======ksh/bash vs sh</H1></font>
Ksh and bash are both supersets of sh. For maximum portability, even to very old
computers, you should stick to the commands found in sh. Where possible, ksh or bash-specific features
will be noted in the following pages. In general, the newer shells run a little faster
and scripts are often more readable because logic can be expressed more cleanly user the newer
syntax. Many commands and conditional tests are now internal.
<blockquote><small>
The philosophy of separate Unix tools each performing a single
operation was followed closely by the designers of the original shell, so it
had very few internal commands and
used external tools for very trivial operations
(like <font color=DARKRED><b><code>echo</code></b></font> and <font color=DARKRED><b><code>!</code></b></font> magic header.
All the parsing rules, filename wildcards, $PATH searches etc., which were summarized
above, apply.
<BR>
In addition:
<DL>
<DT><font color=DARKRED><b><code>#</code></b></font> as the first non-whitespace character on a line
<DD>flags the line as a comment, and the rest of the line is completely ignored. Use
comments liberally in your scripts, as in all other forms of programming.
<P>
<DT><font color=DARKRED><b><code>\\</code></b></font> as the last character on a line
<DD>causes the following line to be logically joined before interpretation. This allows
single very long commands to be entered in the script in a more readable fashion. You can continue
the line as many times as needed.
<blockquote><small>
This is actually just a particular instance of <font color=DARKRED><b><code>\\</code></b></font> being to <EM>escape</EM>, or remove
the special meaning from, the following character.
</small></blockquote>
<P>
<DT><font color=DARKRED><b><code>;</code></b></font> as a separator between words on a line
<DD>is interpreted as a newline. It allows you to put multiple commands on a single line. There are
few occasions when you <STRONG>must</STRONG> do this, but often it is used to improve the layout
of compound commands.
</DL>
<I>Example:</I> ex1 <A HREF=ex1.html TARGET=example>display</A>, <A HREF=ex1.txt TARGET=example>text</A>
<BLOCKQUOTE><SMALL>
<code><font color=DARKRED>
<font color=GRAY> 1:</font> <font color=DARKGREEN><i>#!/bin/ksh</i></font>
<font color=GRAY> 2:</font> <font color=DARKGREEN><i># For the purposes of display, parts of the script have </i></font>
<font color=GRAY> 3:</font> <font color=DARKGREEN><i># been rendered in glorious technicolor.</i></font>
<font color=GRAY> 4:</font> <font color=DARKGREEN><i><b>## Some comments are bold to flag special sections</b></i></font>
<font color=GRAY> 5:</font>
<font color=GRAY> 6:</font> <font color=DARKGREEN><i># Line numbers on the left are not part of the script.</i></font>
<font color=GRAY> 7:</font> <font color=DARKGREEN><i># They are just added to the HTML for reference.</i></font>
<font color=GRAY> 8:</font>
<font color=GRAY> 9:</font> <font color=DARKGREEN><i># Built-in commands and keywords (e.g. print) are in blue</i></font>
<font color=GRAY> 10:</font> <font color=DARKGREEN><i># Command substitutions are purple. Variables are black</i></font>
<font color=GRAY> 11:</font> <font color=DARKBLUE>print </font>"Disk usage summary for <font color=BLACK>$USER</font> on `<font color=PURPLE>date</font>`"
<font color=GRAY> 12:</font>
<font color=GRAY> 13:</font> <font color=DARKGREEN><i># Everything else is red - mostly that is external </i></font>
<font color=GRAY> 14:</font> <font color=DARKGREEN><i># commands, and the arguments to all of the commands.</i></font>
<font color=GRAY> 15:</font> <font color=DARKBLUE>print </font>These are my files <font color=DARKGREEN><i># end of line comment for print</i></font>
<font color=GRAY> 16:</font> <font color=DARKGREEN><i># List the files in columns</i></font>
<font color=GRAY> 17:</font> ls -C
<font color=GRAY> 18:</font> <font color=DARKGREEN><i># Summarize the disk usage</i></font>
<font color=GRAY> 19:</font> <font color=DARKBLUE>print</font>
<font color=GRAY> 20:</font> <font color=DARKBLUE>print </font>Disk space usage
<font color=GRAY> 21:</font> du -k
<font color=GRAY> 22:</font> <font color=DARKBLUE>exit </font>0
</font></code>
</SMALL></BLOCKQUOTE>
=====Exit status=====
Every command (program) has a <EM>value</EM> or <EM>exit status</EM>
which it returns to the calling program. This
is separate from any output generated. The exit status of a shell script can be explicitly set using
<a href="http://www.FreeBSD.org/cgi/man.cgi</code></b></font>
<DD>Match any single character from the bracketed set. A range of characters can be specified
with <font color=DARKRED><b><code>-</code></b></font>
<DT><font color=DARKRED><b><code>!...</code></b></font>
<DD>Match any single character NOT in the bracketed set.
</DL>
<UL>
<LI> An initial ”.“ in a filename does not match a wildcard unless explicitly
given in the pattern. In this sense filenames starting with ”.“ are hidden.
A ”.“ elsewhere in the filename is not special.
<P>
<LI>Pattern operators can be combined
<P>
</UL>
<I>Example:</I><BR>
<font color=DARKRED><b><code>chapter1-5.*</code></b></font> could match <font color=DARKRED><b><code>chapter1.tex</code></b></font>, <font color=DARKRED><b><code>chapter4.tex</code></b></font>, <font color=DARKRED><b><code>chapter5.tex.old</code></b></font>.
It would not match <font color=DARKRED><b><code>chapter10.tex</code></b></font> or <font color=DARKRED><b><code>chapter1</code></b></font>
<p align=RIGHT class=“newpage”><font size=-1>(9)</font></P>
<font color=DARKBLUE>======Shell Variables</H1></font>
Scripts are not very useful if all the commands and options and filenames are explicitly
coded. By using variables, you can make a script generic and apply it to different
situations. Variable names consist of letters, numbers and underscores
<code>
(a-zA-Z0-9_, cannot start with a number, and are case
</code>
sensitive. Several special variables (always uppercase names) are used by the system – resetting these
may cause unexpected behaviour. Some special variables may be read-only.
Using lowercase names for your own variables is safest.
<P>
====Setting and exporting variables====
<DL>
<DT><font color=DARKRED><b><code>srcfile=dataset1</code></b></font>
<DD>Creates (if it didn't exist) a variable named “srcfile” and sets it to the value “dataset1”. If the
variable already existed, it is overwritten. Variables are treated as text strings, unless the context
implies a numeric interpretation. You can make a variable always be treated as a number. Note there
must be <STRONG>no spaces</STRONG> around the ”=“.
<DT><a href=“http://www.FreeBSD.org/cgi/man.cgi?query=set&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b><code>set</code></b></font></a>
<DD>Display all the variables currently set in the shell
<DT><a href=“http://www.FreeBSD.org/cgi/man.cgi?query=unset&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b><code>unset</code></b></font></a> <font color=DARKRED><b><code>srcfile</code></b></font>
<DD>Remove the variable “srcfile”
<DT><font color=DARKRED><b><code>srcfile=</code></b></font>
<DD>Give the variable a null value, (not the same as removing it).
<DT><a href=“http://www.FreeBSD.org/cgi/man.cgi?query=export&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b><code>export</code></b></font></a> <font color=DARKRED><b><code>srcfile</code></b></font>
<DD>Added srcfile to the list of variables which will be made available to external program through
the environment. If you don't do this, the variable is local to this shell instance.
<DT><a href=“http://www.FreeBSD.org/cgi/man.cgi?query=export&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b><code>export</code></b></font></a>
<DD>List all the variables currently being exported - this is the environment which will be passed
to external programs.
</DL>
====Using variables====
<DL>
<DT><font color=DARKRED><b><code>$srcfile</code></b></font>
<DD>Prefacing the variable name with <B>$</B> causes the <EM>value</EM> of the variable to
be substituted in place of the name.
<DT><font color=DARKRED><b><code>${srcfile}</code></b></font>
<DD>If the variable is not surrounded by whitespace (or other characters that can't be in a name),
the name must be surrounded by ”{}“ braces
so that the shell knows what characters you intend to be part of the name.
<P>
<I>Example:</I>
<PRE>
datafile=census2000
- Tries to find $datafile_part1, which doesn't exist
echo $datafile_part1.sas
- This is what we intended
echo ${datafile}_part1.sas
</PRE>
</DL>
====Conditional modifiers====
There are various ways to conditionally use a variable in a command.
<DL>
<DT><font color=DARKRED><b><code>${datafile-default}</code></b></font>
<DD>Substitute the value of <font color=DARKRED><b><code>$datafile</code></b></font>, if it has been defined, otherwise use the string “default”. This is an easy
way to allow for optional variables, and have sensible defaults if they haven't been set. If <font color=DARKRED><b><code>datafile</code></b></font> was
undefined, it remains so.
<DT><font color=DARKRED><b><code>${datafile=default}</code></b></font>
<DD>Similar to the above, except if <font color=DARKRED><b><code>datafile</code></b></font> has not been defined, set it to the string “default”.
<DT><font color=DARKRED><b><code>${datafile+default}</code></b></font>
<DD>If variable <font color=DARKRED><b><code>datafile</code></b></font> has been defined, use the string “default”, otherwise use null. In this case the
actual value <font color=DARKRED><b><code>$datafile</code></b></font> is not used.
<DT><font color=DARKRED><b><code>${datafile?”error message“}</code></b></font>
<DD>Substitute the value of <font color=DARKRED><b><code>$datafile</code></b></font>, if it has been defined, otherwise display <font color=DARKRED><b><code>datafile: error message</code></b></font>.
This is used for diagnostics when a variable should have been set and there is no sensible default value to use.
<blockquote><small>
Placing a colon (<B»</B>) before the operator character in these constructs has the effect of counting a
<I>null</I> value the same as an undefined variable. Variables may be given a null value by setting them
to an empty string, e.g. <font color=DARKRED><b><code>datafile= </code></b></font>.
<BR><I>Example:</I>
<font color=DARKRED><b><code>echo ${datafile:-mydata.dat}</code></b></font><BR>
Echo the value of variable <font color=DARKRED><b><code>datafile</code></b></font> if it has been set and is non-null, otherwise
echo “mydata.dat”.
</small></blockquote>
</DL>
====Variable assignment command prefix====
It is possible to export a variable just for the duration of a single command using the
syntax:
<BR><font color=DARKRED><b><code>var=value command args</code></b></font>
<p align=RIGHT class=“newpage”><font size=-1>(10)</font></P>
<font color=DARKBLUE>======Preset Shell Variables</H1></font>
Several special variables are used by the system – you can use these, but may not be able
to change them.
The special variables use uppercase names, or punctuation characters.
Some variables are set by the login process and inherited by the shell (e.g. <font color=DARKRED><b><code>$USER</code></b></font>),
while others are used only by the shell. <br>
Try running <font color=DARKRED><b><code>set</code></b></font> or <font color=DARKRED><b><code>env</code></b></font> <br>
These are some of the more commonly used ones:
=====Login environment=====
<DL>
<DT><font color=DARKRED><b><code>$USER, $LOGNAME</code></b></font>
<DD>Preset to the currently logged-in username.
<DT><font color=DARKRED><b><code>$PATH</code></b></font>
<DD>The list of directories that will be searched for external commands. You can change this in a script
to make sure you get the programs you intend, and don't accidentally get other versions which might have been
installed.
<DT><font color=DARKRED><b><code>$TERM</code></b></font>
<DD>The terminal type in which the shell session is currently executing. Usually “xterm” or “vt100”. Many programs
need to know this to figure out what special character sequences to send to achieve special effects.
<DT><font color=DARKRED><b><code>$PAGER</code></b></font>
<DD>If set, this contains the name of the program which the user prefers to use for text file viewing. Usually
set to “more” or “less” or something similar. Many programs which need to present multipage information to
the user will respect this setting (e.g. <a href=“http://www.FreeBSD.org/cgi/man.cgi?query=man&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b><code>man</code></b></font></a>). This isn't actually used by the shell itself, but
shell scripts should honour it if they need to page output to the user.
<DT><font color=DARKRED><b><code>$EDITOR</code></b></font>
<DD>If set, this contains the name of the program which the user prefers to use for text file editing. A program
which needs to have the user manually edit a file might choose to start up this program instead of some
built-in default (e.g. “crontab -e”. This also determines the default command-line-editing
behaviour in interactive shells.
</DL>
=====Shell internal settings=====
<DL>
<DT><font color=DARKRED><b><code>$PWD</code></b></font>
<DD>Always set the current working directory (readonly)
<DT><font color=DARKRED><b><code>$OLDPWD</code></b></font>
<DD>The previous directory (before the most recent <a href=“http://www.FreeBSD.org/cgi/man.cgi?query=cd&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b><code>cd</code></b></font></a> command). However, changing directories
in a script is often dangerous.
<DT><font color=DARKRED><b><code>$?</code></b></font> (readonly)
<DD>Set to the exit status of the last command run, so you can test success or failure. Every command resets this
so it must be saved immediately if you want to use it later.
<DT><font color=DARKRED><b><code>$-</code></b></font>
<DD>Set to the currently set options flags.
<DT><font color=DARKRED><b><code>$IFS</code></b></font>
<DD>Internal Field Separators: the set of characters (normally space and tab) which are used to parse
a command line into separate arguments. This may be set by the user for special purposes, but
things get very confusing if it isn't changed back.
</DL>
=====Process ID variables=====
<DL>
<DT><font color=DARKRED><b><code>$$</code></b></font> (readonly)
<DD>Set to the process ID of the current shell - useful in making unique temporary files, e.g. /tmp/$0.$$
<DT><font color=DARKRED><b><code>$PPID</code></b></font> (readonly)
<DD>Set to the process ID of the parent process of this shell - useful for discovering how the script was called.
<DT><font color=DARKRED><b><code>$!</code></b></font> (readonly)
<DD>Set to the process ID of the last command started in background - useful for checking on background processes.
</DL>
=====ksh/bash additional features=====
<DL>
<DT><font color=DARKRED><b><code>$SECONDS</code></b></font> (readonly)
<DD>Integer number of seconds since this shell was started. Can be used for timing commands.
<DT><font color=DARKRED><b><code>$RANDOM</code></b></font>
<DD>Every time it is valuated, <font color=DARKRED><b><code>$RANDOM</code></b></font> returns a random integer in the range 0-32k. <font color=DARKRED><b><code>RANDOM</code></b></font>
may be set to “seed” the random number generator.
<DT><font color=DARKRED><b><code>$LINENO</code></b></font> (readonly)
<DD>Always evaluates to the current line number of the script being executed - useful for debugging.
</DL>
<p align=RIGHT class=“newpage”><font size=-1>(11)</font></P>
<font color=DARKBLUE>======Command Line (positional) arguments</H1></font>
To customize the behaviour of a script at run time,
you can give it any number of arguments on the command
line.
<blockquote><small>
These are often filenames, but can be interpreted by the script in any way. Options
are often specified using the ”-flag“ convention used by most Unix programs, and a ksh command
<a href=“http://www.FreeBSD.org/cgi/man.cgi?query=getopts&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b><code>getopts</code></b></font></a> is available to help parse them.
</small></blockquote> The shell expands wildcards and makes variable and
command substitutions as normal, then parses the resulting words by whitespace (actually
special variable <font color=DARKRED><b><code>$IFS</code></b></font>), and places the resulting text strings into the
<EM>positional variables</EM>
as follows:
<DL>
<DT><font color=DARKRED><b><code>$0, $1, $2, … $9</code></b></font>
<DD>The first 9 arguments are made available directly as $1-$9. To access more than 9, use
<font color=DARKRED><b><code>shift</code></b></font>, or <font color=DARKRED><b><code>$*, $@</code></b></font>. The variable <font color=DARKRED><b><code>$0</code></b></font> contains the name of the
script itself.
<DT><font color=DARKRED><b><code>${10}, ${11}, …</code></b></font>
<DD>Positional arguments greater than 9 are set by ksh and bash. Remember to use braces to refer to them.
<DT><a href=“http://www.FreeBSD.org/cgi/man.cgi?query=shift&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b><code>shift</code></b></font></a>
<DD>discard $1 and renumber all the other variables. ”<font color=DARKRED><b><code>shift N</code></b></font>“ will shift N arguments at once.
<DT><font color=DARKRED><b><code>$#</code></b></font>
<DD>contains the number of arguments that were set (not including $0).
<DT><font color=DARKRED><b><code>$*</code></b></font>
<DD>contains all of the arguments in a single string, with one space separating them.
<DT><font color=DARKRED><b><code>$@</code></b></font>
<DD>similar to $*, but if used in quotes, it effectively quotes each argument and keeps them separate.
If any argument contains whitespace, the distinction is important.
</DL>
e.g. if the argument list is: <font color=DARKRED><b><code>a1 a2 “a3 which contains spaces” a4</code></b></font><BR>
then: $1=a1, $2=a2, $3=a3 which contains spaces, $4=a4<BR>
and: $*=a1 a2 a3 which contains spaces a4<BR>
and: “$@”=“a1” “a2” “a3 which contains spaces” “a4”<BR>
<BR>
Only using the form “$@” preserves quoted arguments. If the arguments are being passed from the script
directly to some other program, it may make a big difference to the meaning.
<P>
<I>Example:</I> ex7 <A HREF=ex7.html TARGET=example>display</A>, <A HREF=ex7.txt TARGET=example>text</A>
<BLOCKQUOTE><SMALL>
<code><font color=DARKRED>
<font color=GRAY> 1:</font> <font color=DARKGREEN><i>#!/bin/sh</i></font>
<font color=GRAY> 2:</font> <font color=DARKGREEN><i>#</i></font>
<font color=GRAY> 3:</font> <font color=DARKGREEN><i># Check positional argument handling</i></font>
<font color=GRAY> 4:</font> <font color=DARKBLUE>echo </font>“Number of arguments: $#”
<font color=GRAY> 5:</font> <font color=DARKBLUE>echo </font>“\\<font color=BLACK>$0</font> = <font color=BLACK>$0</font>”
<font color=GRAY> 6:</font>
<font color=GRAY> 7:</font> <font color=DARKBLUE>echo </font>“Loop over \\<font color=BLACK>$*</font>”
<font color=GRAY> 8:</font> <font color=DARKBLUE>for </font>a in <font color=BLACK>$*</font>; <font color=DARKBLUE>do</font>
<font color=GRAY> 9:</font> <font color=DARKBLUE>echo </font>\\”<font color=BLACK>$a</font>\\“
<font color=GRAY> 10:</font> <font color=DARKBLUE>done</font>
<font color=GRAY> 11:</font>
<font color=GRAY> 12:</font> <font color=DARKBLUE>echo </font>“Loop over \\”\\<font color=BLACK>$@</font>\\”“
<font color=GRAY> 13:</font> <font color=DARKBLUE>for </font>a in ”<font color=BLACK>$@</font>“; <font color=DARKBLUE>do</font>
<font color=GRAY> 14:</font> <font color=DARKBLUE>echo </font>\\”<font color=BLACK>$a</font>\\“
<font color=GRAY> 15:</font> <font color=DARKBLUE>done</font>
</font></code>
</SMALL></BLOCKQUOTE>
=====Setting new positional arguments=====
The <a href=“http://www.FreeBSD.org/cgi/man.cgi?query=set&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b><code>set</code></b></font></a> command, followed by a set of arguments, creates a new set of
positional arguments. This is often used, assuming the original arguments are no longer needed, to parse
a set of words (possibly using different field separators). Arguments may be reset any number of times.
<P>
<I>Example:</I> ex2 <A HREF=ex2.html TARGET=example>display</A>, <A HREF=ex2.txt TARGET=example>text</A>
<BLOCKQUOTE><SMALL>
<code><font color=DARKRED>
<font color=GRAY> 1:</font> <font color=DARKGREEN><i>#!/bin/sh</i></font>
<font color=GRAY> 2:</font> <font color=DARKGREEN><i># Find an entry in the password file</i></font>
<font color=GRAY> 3:</font> <font color=BLACK>pwent=</font>`<font color=PURPLE>grep '^root:' /etc/passwd</font>`
<font color=GRAY> 4:</font> <font color=DARKGREEN><i># Turn off globbing - passwd lines often contain '*'</i></font>
<font color=GRAY> 5:</font> set -o noglob
<font color=GRAY> 6:</font> <font color=DARKGREEN><i># The “full name” and other comments are in</i></font>
<font color=GRAY> 7:</font> <font color=DARKGREEN><i># field 5, colon delimited. Get this field using shell word splitting</i></font>
<font color=GRAY> 8:</font> <font color=BLACK>OIFS=</font><font color=BLACK>$IFS</font>; IFS=: ; set <font color=BLACK>$pwent</font>; IFS=<font color=BLACK>$OIFS</font>
<font color=GRAY> 9:</font> <font color=DARKBLUE>echo </font><font color=BLACK>$5</font>
</font></code>
</SMALL></BLOCKQUOTE>
<P>
<I>Example:</I> pickrandom <A HREF=pickrandom.html TARGET=example>display</A>, <A HREF=pickrandom.txt TARGET=example>text</A><BR>
Selects a random file from a directory.
Uses the ksh RANDOM feature.
<BLOCKQUOTE><SMALL>
<code><font color=DARKRED>
<font color=GRAY> 1:</font> <font color=DARKGREEN><i>#!/bin/ksh</i></font>
<font color=GRAY> 2:</font>
<font color=GRAY> 3:</font> <font color=DARKGREEN><i># Select a random image from the background logo collection</i></font>
<font color=GRAY> 4:</font> <font color=DARKGREEN><i># This could be used to configure a screen saver, for example.</i></font>
<font color=GRAY> 5:</font> <font color=DARKGREEN><i>#</i></font>
<font color=GRAY> 6:</font> <font color=DARKGREEN><i># This works even if the filenames contain spaces.</i></font>
<font color=GRAY> 7:</font>
<font color=GRAY> 8:</font> <font color=DARKGREEN><i># switch to the logos directory to avoid long paths</i></font>
<font color=GRAY> 9:</font> <font color=BLACK>logos=</font>/afs/northstar/common/usr/lib/X11/logos/backgrounds
<font color=GRAY> 10:</font> <font color=DARKBLUE>cd </font><font color=BLACK>$logos</font>
<font color=GRAY> 11:</font>
<font color=GRAY> 12:</font> <font color=DARKGREEN><i># '*' is a filename wildcard to match all files in the current directory</i></font>
<font color=GRAY> 13:</font> set *
<font color=GRAY> 14:</font>
<font color=GRAY> 15:</font> <font color=DARKGREEN><i># Use the syntax for arithmetic expressions. ”%“ is the modulo operator</i></font>
<font color=GRAY> 16:</font> <font color=DARKGREEN><i># Shift arguments by a random number between 0 and the number of files</i></font>
<font color=GRAY> 17:</font> <font color=DARKBLUE>shift </font><font color=DARKBLUE>$1)</font>
<font color=GRAY> 18:</font>
<font color=GRAY> 19:</font> <font color=DARKGREEN><i># Output the resulting first argument</i></font>
<font color=GRAY> 20:</font> <font color=DARKBLUE>echo </font>”<font color=BLACK>$logos</font>/<font color=BLACK>$1</font>“
</font></code>
</SMALL></BLOCKQUOTE>
<p align=RIGHT class=“newpage”><font size=-1>(12)</font></P>
<font color=DARKBLUE>======Shell options</H1></font>
Startup options. <font color=DARKRED><b><code>ksh -options scriptname</code></b></font>
<DL>
<DT>-x
<DD>echo line to stderr before executing it
<DT>-n
<DD>read commands and check for syntax errors, but do not execute.
<DT>-a
<DD>all variables are automatically exported
<DT>-f
<DD>disable wildcard filename expansion (globbing)
<DT><font color=DARKRED><b><code>set -x</code></b></font>
<DD>Set an option within a shell script
<DT><font color=DARKRED><b><code>$-</code></b></font>
<DD>contains the currently set option letters
</DL>
There are many other options, not often needed. Options in ksh and bash can also be
set using long names (e.g. <font color=DARKRED><b><code>-o noglob</code></b></font> instead of <font color=DARKRED><b><code>-f</code></b></font>). Many options
are unique to ksh or bash.
<p align=RIGHT class=“newpage”><font size=-1>(13)</font></P>
<font color=DARKBLUE>======Command Substitution</H1></font>
====sh syntax====
<DL>
<DT><font color=DARKRED><b><code>`command`</code></b></font>
<DD>A command (plus optional arguments) enclosed in backticks is executed and the standard output
of that command is substituted. If the command produces multiline output, the newlines are retained.
If the resultant string is displayed, unquoted, using <font color=DARKRED><b><code>echo</code></b></font>, newlines and multiple spaces will be removed.
</DL>
====ksh/bash syntax====
<DL>
<DT><font color=DARKRED><b><code>$(command)</code></b></font>
<DD>This syntax is functionally the same as backticks, but commands can be more easily nested.
<DT><font color=DARKRED><b><code>$(<file)</code></b></font>
<DD>This is equivalent to <font color=DARKRED><b><code>`cat file`</code></b></font>, but implemented internally for efficiency.
</DL>
<P>
<I>Example:</I> ex3 <A HREF=ex3.html TARGET=example>display</A>, <A HREF=ex3.txt TARGET=example>text</A>
<BLOCKQUOTE><SMALL>
<code><font color=DARKRED>
<font color=GRAY> 1:</font> <font color=DARKGREEN><i>#!/bin/ksh</i></font>
<font color=GRAY> 2:</font>
<font color=GRAY> 3:</font> <font color=DARKBLUE>echo </font>Today is `<font color=PURPLE>date</font>`
<font color=GRAY> 4:</font>
<font color=GRAY> 5:</font> <font color=BLACK>file=</font>/etc/hosts
<font color=GRAY> 6:</font> <font color=DARKBLUE>echo </font>The file <font color=BLACK>$file</font> has $(<font color=PURPLE>wc -l < <font color=BLACK>$file</font></font>) lines
<font color=GRAY> 7:</font>
<font color=GRAY> 8:</font> hostname -s > myhostname
<font color=GRAY> 9:</font> <font color=DARKBLUE>echo </font>This system has host name $(<font color=PURPLE><myhostname</font>)
</font></code>
</SMALL></BLOCKQUOTE>
<p align=RIGHT class=“newpage”><font size=-1>(14)</font></P>
<font color=DARKBLUE>======I/O redirection and pipelines</H1></font>
Any simple command (or shell function, or compound command) may have its input and output
redirected using the following operators. This is performed by the shell <em>before</em>
the command is run.
=====Output redirection=====
<DL>
<DT><font color=DARKRED><b><code>> <I>filename</I></code></b></font>
<DD>Standard ouput (file descriptor 1) is redirected to the named file. The file is overwritten
unless the <font color=DARKRED><b><code>noclobber</code></b></font> option is set. The file is created if it does not exist.
<blockquote><small>
The special device file <font color=DARKRED><b><code>/dev/null</code></b></font> can be used to explicitly discard unwanted output.
Reading from /dev/null results in an End of File status.
</small></blockquote>
<DT><font color=DARKRED><b><code>>> <I>filename</I></code></b></font>
<DD>Standard ouput is appended to the named file. The file is created if it does not exist.
<DT><font color=DARKRED><b><code>>| <I>filename</I></code></b></font>
<DD>Output redirect, and override the <I>noclobber</I> option, if set.
</DL>
=====Input redirection=====
<DL>
<DT><font color=DARKRED><b><code>< <I>filename</I></code></b></font>
<DD>Standard input (file descriptor 0) is redirected to the named file. The file must already
exist.
</DL>
=====Command pipelines=====
<DL>
<DT><font color=DARKRED><b><code>command | command command ...</code></b></font>
<DD>Pipe multiple commands together. The standard output of the first command becomes the standard
input of the second command. All commands run simultaneously, and data transfer happens via memory
buffers. This is one of the most powerful constructs in Unix. <EM>Compound</EM> commands may also be
used with pipes. Pipes play very nicely with multiprocessor systems.
<blockquote><small>
No more than one command in a pipeline should be interactive (attempt to read from
the terminal). This construct is much more efficient than using temporary files, and most standard
Unix utilities are designed such that they work well in pipelines.
<P>
The exit status of a pipeline is the exit status of the last command. In compound commands, a pipeline
can be used anywhere a simple command could be used.
</small></blockquote>
</DL>
<p align=RIGHT class=“newpage”><font size=-1>(15)</font></P>
<font color=DARKBLUE>======Input and Output</H1></font>
Shell scripts can generate output directly or read input into variables using the following commands:
====Script output====
<DL>
<DT><a href=“http://www.FreeBSD.org/cgi/man.cgi?query=echo&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b><code>echo</code></b></font></a>
<DD>Print arguments, separated by spaces, and terminated by a newline, to stdout.
Use quotes to preserve spacing. Echo also understands C-like escape conventions.
<blockquote><small>
Beware that
the shell may process backslashes before <font color=DARKRED><b><code>echo</code></b></font> sees them (may need to double backslash).
Internal in most shells, but was originally external.
<TABLE width=100%>
<TR><TD> \\b </TD><TD> backspace </TD><TD> \\c </TD><TD> print line without new-line (some versions)</TD>
<TR><TD> \\f </TD><TD> form-feed </TD><TD>\
<code>
</TD><TD> new-line </TD>
</code>
<TR><TD> \\r </TD><TD> carriage return</TD><TD> \\t </TD><TD> tab </TD>
<TR><TD> \\v </TD><TD> vertical tab </TD><TD>\\
</TD><TD> backslash </TD>
</TABLE>
0n where n is the 8-bit character whose ASCII
<code>
code is the 1-, 2- or 3-digit octal number </TD>
representing that character.
</code>
</small></blockquote>
<DL>
<DT>-n <DD>suppress newline
</DL>
<DT><a href=“http://www.FreeBSD.org/cgi/man.cgi?query=print&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b><code>print</code></b></font></a> (ksh internal)
<DD>Print arguments, separated by spaces, and terminated by a newline, to stdout.
Print observes the same escape conventions as echo.
<DL>
<DT>-n
<DD>suppress newline
<DT>-r
<DD>raw mode - ignore \\-escape conventions
<DT>-R
<DD>raw mode - ignore \\-escape conventions and -options except -n.
</DL>
</DL>
====Script input====
<DL>
<DT><a href=“http://www.FreeBSD.org/cgi/man.cgi?query=read&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b><code>read</code></b></font></a> <font color=DARKRED><b><code>var1 var2 rest</code></b></font>
<DD>read a line from stdin, parsing by $IFS, and placing the words into the named variables.
Any left over words all go into the last variable. A '\\' as the last character on a line
removes significance of the newline, and input continues with the following line.
<DL>
<DT>-r
<DD>raw mode - ignore \\-escape conventions
</DL>
</DL>
<P>
<I>Example:</I> ex4a <A HREF=ex4a.html TARGET=example>display</A>, <A HREF=ex4a.txt TARGET=example>text</A>
<BLOCKQUOTE><SMALL>
<code><font color=DARKRED>
<font color=GRAY> 1:</font> <font color=DARKGREEN><i>#!/bin/sh</i></font>
<font color=GRAY> 2:</font> <font color=DARKBLUE>echo </font>“Testing interactive user input: enter some keystrokes and press return”
<font color=GRAY> 3:</font> <font color=DARKBLUE>read </font>x more
<font color=GRAY> 4:</font> <font color=DARKBLUE>echo </font>“First word was \\”<font color=BLACK>$x</font>\\”“
<font color=GRAY> 5:</font> <font color=DARKBLUE>echo </font>“Rest of the line (if any) was \\”<font color=BLACK>$more</font>\\”“
</font></code>
</SMALL></BLOCKQUOTE>
<p align=RIGHT class=“newpage”><font size=-1>(16)</font></P>
<font color=DARKBLUE>======Conditional tests for ... and ... commands</H1></font>
Most of the useful flow-control operators involve making some conditional test and
branching on the result (true/false). The test can be either the <a href=“http://www.FreeBSD.org/cgi/man.cgi?query=test&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b><code>test</code></b></font></a>
command, or its alias, <font color=DARKRED><b><code></code></b></font>, or the ksh/bash built-in <font color=DARKRED><b><code>[[ ...</code></b></font>
command, which has slightly different options, or it can be <EM>any command which returns
a suitable exit status</EM>. Zero is taken to be “True”, while any non-zero value is “False”.
Note that this is backwards from the C language convention.
====File tests====
<DL>
<DT><font color=DARKRED><b><code>-e <I>file</I></code></b></font>
<DD>True if <I>file</I> exists (can be of any type).
<DT><font color=DARKRED><b><code>-f <I>file</I></code></b></font>
<DD>True if <I>file</I> exists and is an ordinary file.
<DT><font color=DARKRED><b><code>-d <I>file</I></code></b></font>
<DD>True if <I>file</I> exists and is a directory.
<DT><font color=DARKRED><b><code>-r <I>file</I></code></b></font>
<DD>True if <I>file</I> exists and is readable<BR>
<code>
Similarly, <font color=DARKRED><b><code>-w</code></b></font> = writable, <font color=DARKRED><b><code>-x</code></b></font> = executable, <font color=DARKRED><b><code>-L</code></b></font> = is a symlink.
</code>
<DT><font color=DARKRED><b><code>-s <I>file</I></code></b></font>
<DD>True if <I>file</I> exists and has size greater than zero
<DT><font color=DARKRED><b><code>-t <I>filedescriptor</I></code></b></font>
<DD>True if the open <I>filedescriptor</I> is associated with a terminal device. E.g. this is
used to determine if standard output has been redirected to a file.
</DL>
====Character string tests====
<DL>
<DT><font color=DARKRED><b><code> -n “string”</code></b></font>
<DD>true if <I>string</I> has non-zero length
<DT><font color=DARKRED><b><code> -z “string”</code></b></font>
<DD>true if <I>string</I> has zero length
<blockquote><small>
With <font color=DARKRED><b><code></code></b></font>, the argument must be quoted, because if it is a variable that
has a null value, the resulting expansion ( [[ -z ) is a syntax error. An
expansion resulting in ”“ counts as a null string. <BR>
For <font color=DARKRED><b><code></code></b></font> only, a quoted
string alone is equivalent to the -n test, e.g. [[ "$var". In older shells for which
<font color=DARKRED><b><code><BR>
<font color=DARKRED><b><code>if [[ "X$var" = "X"</code></b></font><BR>
This is rarely needed now, but is still often found.
</small></blockquote>
<DT><font color=DARKRED><b><code>$variable = <I>text</I></code></b></font>
<DD>True if <I>$variable</I> matches <I>text</I>.
<DT><font color=DARKRED><b><code>$variable < <I>text</I></code></b></font>
<DD>True if <I>$variable</I> comes before (lexically) <I>text</I><BR>
<code>
Similarly, <font color=DARKRED><b><code>></code></b></font> = comes after
</code>
</DL>
<p align=RIGHT class=“newpage”><font size=-1>(17)</font></P>
<font color=DARKBLUE>======More conditional tests for ... and ... commands</H1></font>
====Arithmetic tests====
<DL>
<DT><font color=DARKRED><b><code>$variable -eq <I>number</I></code></b></font>
<DD>True if <I>$variable</I>, interpreted as a number, is equal to <I>number</I>.
<DT><font color=DARKRED><b><code>$variable -ne <I>number</I></code></b></font>
<DD>True if <I>$variable</I>, interpreted as a number, is <B>not</B> equal to <I>number</I>.<BR>
<code>
Similarly, <font color=DARKRED><b><code>-lt</code></b></font> = less than, <font color=DARKRED><b><code>-le</code></b></font> = less than or equal,
</code>
<font color=DARKRED><b><code>-gt</code></b></font> = greater than, <font color=DARKRED><b><code>-ge</code></b></font> = greater than or equal
</DL>
====Additional tests for ... (ksh and bash)====
<DL>
<DT><font color=DARKRED><b><code>$variable = <I>pattern</I></code></b></font>
<DD>True if <I>$variable</I> matches <I>pattern</I>. If <I>pattern</I> contains no wildcards,
then this is just an exact text match. The same wildcards as used for filename matching are used.
<blockquote><small>
The <I>pattern</I> must not be quoted. Since ... is internal to the shell, the pattern in this case is
treated differently and not filename-expanded as an external command would require.
</small></blockquote>
<DT><font color=DARKRED><b><code>file1 -nt file2</code></b></font>
<DD>True if <I>file1</I> is newer than <I>file2</I>. <BR>
Similarly <font color=DARKRED><b><code>-ot</code></b></font> = older than
<DT><font color=DARKRED><b><code>file1 -ef file2</code></b></font>
<DD>true if <I>file1</I> is effectively the same as <I>file2</I>, after following
symlinks and hard links.
</DL>
====Negating and Combining tests====
Tests may be negated by prepending the <font color=DARKRED><b><code>!</code></b></font> operator, and combined with boolean
<B>AND</B> and <B>OR</B> operators using the syntax:
<DL>
<DT><font color=DARKRED><b><code><I>conditional</I> -a <I>conditional</I>, <I>conditional</I> -o <I>conditional</I> </code></b></font>
<DD><B>AND</B> and <B>OR</B> syntax for <font color=DARKRED><b><code>test</code></b></font> and <font color=DARKRED><b><code>| <I>conditional</I> </code></b></font>
<DD><B>AND</B> and <B>OR</B> syntax for <font color=DARKRED><b><code>[[ ... </code></b></font>
</DL>
Parentheses may be inserted to resolve ambiguities or override the default operator precedence rules.
<P>
<I>Examples:</I>
<PRE>
if -x /usr/local/bin/lserve && \\
<code>
-w /var/logs/lserve.log; then
/usr/local/bin/lserve » /var/logs/lserve.log &
</code>
fi
pwent=`grep '^richard:' /etc/passwd`
if -z "$pwent"; then
<code>
echo richard not found
</code>
fi
</PRE>
<p align=RIGHT class=“newpage”><font size=-1>(18)</font></P>
<font color=DARKBLUE>======Flow Control and Compound Commands</H1></font>
A <I>list</I> in these descriptions is a simple command, or a pipeline.
The value of the <I>list</I> is the value of the last simple command run in it.
<blockquote><small>
A <I>list</I> can also be a set of simple commands or
pipelines separated by ”;,&,&&,||,|&“.
For the compound commands which branch on the success or failure
of some <I>list</I>, it is usually <font color=DARKRED><b><code>| <I>list</I></code></b></font>
<DD>Execute the first <I>list</I>. If false (failure), execute the second one.
<P>
<I>Example:</I>
<PRE>
mkdir tempdir && cp workfile tempdir
sshd || echo "sshd failed to start"
</PRE>
<blockquote><small>
You can use both forms together (with care) - they are processed left to right, and && must come first.
<BR><I>Example:</I>
<PRE>
mkdir tempdir && cp workfile tempdir || \\
<code>
echo "Failed to create tempdir"
</code>
</PRE>
</small></blockquote>
<DT><a href="http://www.FreeBSD.org/cgi/man.cgi?query=if&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html" target=manpage><font color=DARKRED><b><code>if</code></b></font></a> <font color=DARKRED><b><code><I>list</I>; then <I>list</I> ; elif <I>list</I>; then <I>list</I>; else <I>list</I>; fi</code></b></font>
<DD>Execute the first <I>list</I>, and if true (success), execute the "then" list, otherwise
execute the "else" list. The "elif" and "else" lists are optional.
<P>
<I>Example:</I>
<PRE>
if [[ -r $myfile
then
<code>
cat $myfile
</code>
else
<code>
echo $myfile not readable
</code>
fi
</PRE>
</DL>
====Looping: 'while' and 'for' loops====
<DL>
<DT><a href=“http://www.FreeBSD.org/cgi/man.cgi?query=while&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b><code>while</code></b></font></a> <font color=DARKRED><b><code><I>list</I>; do <I>list</I>; done</code></b></font>
<DT><a href=“http://www.FreeBSD.org/cgi/man.cgi?query=until&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b><code>until</code></b></font></a> <font color=DARKRED><b><code><I>list</I>; do <I>list</I>; done</code></b></font>
<DD>Execute the first <I>list</I> and if true (success), execute the second <I>list</I>. Repeat as
long as the first <I>list</I> is true. The <font color=DARKRED><b><code>until</code></b></font> form just negates the test.
<P>
<I>Example:</I> ex4 <A HREF=ex4.html TARGET=example>display</A>, <A HREF=ex4.txt TARGET=example>text</A>
<BLOCKQUOTE><SMALL>
<code><font color=DARKRED>
<font color=GRAY> 1:</font> <font color=DARKGREEN><i>#!/bin/ksh</i></font>
<font color=GRAY> 2:</font> <font color=BLACK>count=</font>0
<font color=GRAY> 3:</font> <font color=BLACK>max=</font>10
<font color=GRAY> 4:</font> <font color=DARKBLUE>while </font><font color=BLACK>$count</font> -lt <font color=BLACK>$max</font>
<font color=GRAY> 5:</font> <font color=DARKBLUE>do </font>
<font color=GRAY> 6:</font> <font color=DARKBLUE>echo </font><font color=BLACK>$count</font>
<font color=GRAY> 7:</font> <font color=BLACK>count=</font><font color=DARKBLUE>$2)</font>
<font color=GRAY> 8:</font> <font color=DARKBLUE>done</font>
<font color=GRAY> 9:</font> <font color=DARKBLUE>echo </font>“Value of count after loop is: <font color=BLACK>$count</font>”
</font></code>
</SMALL></BLOCKQUOTE>
<DT><a href=“http://www.FreeBSD.org/cgi/man.cgi?query=for&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b><code>for</code></b></font></a> <font color=DARKRED><b><code><I>identifier</I> in <I>words</I>; do; <I>list</I>; done</code></b></font>
<DD>Set <I>identifier</I> in turn to each word in <I>words</I> and execute the <I>list</I>.
Omitting the “in <I>words</I>” clause implies using $@, i.e. the <I>identifier</I> is set in
turn to each positional argument.
<P>
<I>Example:</I>
<PRE>
for file in *.dat
do
<code>
echo Processing $file
</code>
done
</PRE>
</DL>
As with most programming languages, there are often several ways to express the same action.
Running a command and then explicitly examining <font color=DARKRED><b><code>$?</code></b></font> can be used instead of some of the above.
<P>
Compound commands can be thought of as running in an implicit subshell. They
can have I/O redirection independant of the rest of the script. Setting of variables in a real subshell does not
leave them set in the parent script. Setting variables in implicit subshells varies in behaviour among shells.
Older <a href=“http://www.FreeBSD.org/cgi/man.cgi?query=sh&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b><code>sh</code></b></font></a> could not set variables in an implicit subshell and then use them later,
but current <a href=“http://www.FreeBSD.org/cgi/man.cgi?query=ksh&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b><code>ksh</code></b></font></a> can do this (mostly).
<P>
<I>Example:</I> ex11 <A HREF=ex11.html TARGET=example>display</A>, <A HREF=ex11.txt TARGET=example>text</A><BR>
Reading a file line by line. The book by Randal Michael contains 12 example ways to read a file line by line,
which vary tremendously in efficiency. This example shows the simplest and fastest way.
<BLOCKQUOTE><SMALL>
<code><font color=DARKRED>
<font color=GRAY> 1:</font> <font color=DARKGREEN><i>#!/bin/sh</i></font>
<font color=GRAY> 2:</font>
<font color=GRAY> 3:</font> <font color=DARKGREEN><i># Demonstrate reading a file line-by-line, using I/O</i></font>
<font color=GRAY> 4:</font> <font color=DARKGREEN><i># redirection in a compound command</i></font>
<font color=GRAY> 5:</font> <font color=DARKGREEN><i># Also test variable setting inside an implicit subshell.</i></font>
<font color=GRAY> 6:</font> <font color=DARKGREEN><i># Test this under sh and ksh and compare the output.</i></font>
<font color=GRAY> 7:</font>
<font color=GRAY> 8:</font> <font color=BLACK>line=</font>“TEST”
<font color=GRAY> 9:</font> <font color=BLACK>save=</font>
<font color=GRAY> 10:</font>
<font color=GRAY> 11:</font> <font color=DARKBLUE>if </font>-z "<font color=BLACK>$1</font>"; <font color=DARKBLUE>then</font>
<font color=GRAY> 12:</font> <font color=DARKBLUE>echo </font>“Usage: <font color=BLACK>$0</font> filename”
<font color=GRAY> 13:</font> <font color=DARKBLUE>else</font>
<font color=GRAY> 14:</font> <font color=DARKBLUE>if </font>-r <font color=BLACK>$1</font>; <font color=DARKBLUE>then</font>
<font color=GRAY> 15:</font> <font color=DARKBLUE>while </font>read line; <font color=DARKBLUE>do</font>
<font color=GRAY> 16:</font> <font color=DARKBLUE>echo </font>”<font color=BLACK>$line</font>“
<font color=GRAY> 17:</font> <font color=BLACK>save=</font><font color=BLACK>$line</font>
<font color=GRAY> 18:</font> <font color=DARKBLUE>done </font>< <font color=BLACK>$1</font>
<font color=GRAY> 19:</font> <font color=DARKBLUE>fi</font>
<font color=GRAY> 20:</font> <font color=DARKBLUE>fi</font>
<font color=GRAY> 21:</font> <font color=DARKBLUE>echo </font>“End value of \\<font color=BLACK>$line</font> is <font color=BLACK>$line</font>”
<font color=GRAY> 22:</font> <font color=DARKBLUE>echo </font>“End value of \\<font color=BLACK>$save</font> is <font color=BLACK>$save</font>”
</font></code>
</SMALL></BLOCKQUOTE>
<p align=RIGHT class=“newpage”><font size=-1>(19)</font></P>
<font color=DARKBLUE>======Flow Control and Compound Commands (contd.)</H1></font>
====Case statement: pattern matching====
<DL>
<DT><a href=“http://www.FreeBSD.org/cgi/man.cgi?query=case&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b><code>case</code></b></font></a> <font color=DARKRED><b><code><I>word</I> in <I>pattern</I>) <I>list</I>;; esac</code></b></font>
<DD>Compare <I>word</I> with each <I>pattern</I>) in turn, and executes the first <I>list</I>
for which the <I>word</I> matches. The <I>patterns</I> follow the same rules as for filename
wildcards.
<blockquote><small>
(ksh and bash only)
A pattern-list is a list of one or more patterns separated
from each other with a |. Composite patterns can be formed
with one or more of the following:
<DL>
<DT> <font color=DARKRED><b><code>?(pattern-list)</code></b></font>
<DD> Optionally matches any one of the given patterns.
<BR>
<DT> <font color=DARKRED><b><code>*(pattern-list)</code></b></font>
<DD> Matches zero or more occurrences of the given patterns.
<BR>
<DT> <font color=DARKRED><b><code>+(pattern-list)</code></b></font>
<DD> Matches one or more occurrences of the given patterns.
<DT> <font color=DARKRED><b><code>@(pattern-list)</code></b></font>
<DD> Matches exactly one of the given patterns.
<DT> <font color=DARKRED><b><code>!(pattern-list)</code></b></font>
<DD> Matches anything, except one of the given patterns.
</DL>
</small></blockquote>
<P>
<I>Example:</I>
<PRE>
<code>
case $filename in
*.dat)
echo Processing a .dat file
;;
*.sas)
echo Processing a .sas file
;;
*)
# catch anything else that doesn't match patterns
echo “Don't know how to deal with $filename”
;;
esac
</code>
</PRE>
</DL>
====Miscellaneous flow control and subshells====
<DT><a href=“http://www.FreeBSD.org/cgi/man.cgi?query=break&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b><code>break</code></b></font></a> <font color=DARKRED><b><code>n</code></b></font>
<DD>Break out of the current (or n'th) enclosing loop. Control jumps to the next statement
after the loop
<P>
<DT><a href=“http://www.FreeBSD.org/cgi/man.cgi?query=continue&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b><code>continue</code></b></font></a> <font color=DARKRED><b><code>n;</code></b></font>
<DD>Resume iteration of the current (or n'th) enclosing loop. Control jumps to the top of the loop,
which generally causes re-evaluation of a <a href=“http://www.FreeBSD.org/cgi/man.cgi?query=while&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b><code>while</code></b></font></a> or processing the next element of a
<a href=“http://www.FreeBSD.org/cgi/man.cgi?query=for&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b><code>for</code></b></font></a>.
<P>
<DT><font color=DARKRED><b><code>. filename</code></b></font>
<DD>Read the contents of the named file into the current shell and execute as if in line. Uses $PATH
to locate the file, and can be passed positional parameters. This is often used to read in shell
functions that are common to multiple scripts. There are security implications if the pathname is not
fully specified.
<P>
<DT><font color=DARKRED><b><code>( … )</code></b></font> Command grouping
<DD>Commands grouped in ”( )“ are executed in a subshell, with a separate environment
(can not affect the variables in the rest of the script).
</DL>
<p align=RIGHT class=“newpage”><font size=-1>(20)</font></P>
<font color=DARKBLUE>======Conditional Test Examples</H1></font>
As with most aspects of shell scripting, there are usually several possible ways to accomplish
a task. Certain idioms show up commonly. These are five ways to examine and branch on the
initial character of a string.
<DL>
<DT>Use <font color=DARKRED><b><code>case</code></b></font> with a pattern:
<DD><font color=DARKRED><b><code>case $var in</code></b></font><BR>
<code>
<font color=DARKRED><b><code>/*) echo “starts with /” ;; </code></b></font>
</code>
<BR>
Works in all shells, and uses no extra processes
<P>
<DT>Use <font color=DARKRED><b><code>`cut`</code></b></font»
<DD><font color=DARKRED><b><code>if cut -c1`" = "/" ; then </code></b></font>.
<BR>Works in all shells, but inefficiently uses a pipe and external process for a trivial task.
<P>
<DT>Use POSIX variable truncation:
<DD><font color=DARKRED><b><code>if "${var%${var; then</code></b></font>
<BR>
Works with ksh, bash and other POSIX-compliant shells. Not obvious if you have not seen
this one before. Fails on old Bourne shells.
Dave Taylor in “Wicked Cool Shell Scripts” likes this one.
<P>
<DT>Use POSIX pattern match inside of ...:
<DD><font color=DARKRED><b><code>if $var = /*; then</code></b></font>
<BR>
Works with ksh, bash and other POSIX-compliant shells. Note that you must use ...
and no quotes around the pattern.
<blockquote><small>
The ... syntax is handled internally by the shell and can therefore interpret “wildcard” patterns
differently than an external command. An unquoted wildcard is interpreted as a pattern to be matched,
while a quoted wildcard is taken literally. The ... syntax, even if handled internally, is treated
as though it were external for backward compatability. This requires that wildcard patterns be expanded
to matching filenames.
</small></blockquote>
<P>
<DT>Use ksh (93 and later) and bash variable substrings:
<DD><font color=DARKRED><b><code>if 1}" = "/"; then</code></b></font>
<BR>
ksh93 and later versions, and bash, have a syntax for directly extracting substrings by character
position. <font color=DARKRED><b><code>${varname:start:length}</code></b></font>
</DL>
<P>
<I>Example:</I> ex17 <A HREF=ex17.html TARGET=example>display</A>, <A HREF=ex17.txt TARGET=example>text</A><BR>
<p align=RIGHT class=“newpage”><font size=-1>(21)</font></P>
<font color=DARKBLUE>======Miscellaneous internal commands</H1></font>
The shells (ksh in particular) have many more internal commands. Some are used more in interactive
shells. The commands listed here are used in scripts, but don't conveniently fit elsewhere in the
class.
<DL>
<DT><a href=“http://www.FreeBSD.org/cgi/man.cgi?query=eval&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b><code>eval</code></b></font></a> <font color=DARKRED><b><code>args</code></b></font>
<DD>The args are read as input to the shell and the resulting command executed. Allows “double” expansion
of some constructs. For example, constructing a variable name out of pieces, and then obtaining the value
of that variable.
<P>
<PRE>
netdev=NETDEV_
NETDEV_1=hme0 # As part of an initialization step defining multiple devices
devnum=1 # As part of a loop over those devices
ifname=$netdev$devnum # construct a variable name NETDEV_1
eval device=\\$$ifname # evaluate it - device is set to hme0
</PRE>
<DT><a href=“http://www.FreeBSD.org/cgi/man.cgi?query=exec&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b><code>exec</code></b></font></a> <font color=DARKRED><b><code>command args</code></b></font>
<DD>The command is executed <I>in place of</I> the current shell. There is no return from an exec.
I/O redirection may be used. This is also used to change the I/O for the current shell.
<DT><font color=DARKRED><b><code»</code></b></font>
<DD>The line is variable-expanded, but otherwise treated as a comment. Sometimes
used as a synonym for “true” in a loop.
<PRE>
while :; do
<code>
# this loop will go forever until broken by
# a conditional test inside, or a signal
</code>
done
</PRE>
<P>
<DT><a href=“http://www.FreeBSD.org/cgi/man.cgi?query=unset&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b><code>unset</code></b></font></a> <font color=DARKRED><b><code><I>var</I> …</code></b></font>
<DD>Remove the named variables. This is not the same as setting their values to null.
<P>
<DT><a href=“http://www.FreeBSD.org/cgi/man.cgi?query=typeset&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b><code>typeset</code></b></font></a> <font color=DARKRED><b><code>+/- options <I>name</I>[[=<I>value</I> ] …</code></b></font> (ksh only,
bash uses <font color=DARKRED><b><code>declare</code></b></font> for similar functions)
<DD>Set attributes and values for shell variables and functions. When used inside a function, a local
variable is created. Some of the options are:
<DL>
<DT><font color=DARKRED><b><code>-Ln</code></b></font>
<DD>Left justify and remove leading blanks. The variable always has length n if specified.
<DT><font color=DARKRED><b><code>-Rn</code></b></font>
<DD>Right justify and fill with leading blanks. The variable always has length n if specified.
<DT><font color=DARKRED><b><code>-l</code></b></font>
<DD>The named variable is always treated as an integer. This makes arithmetic faster. The reserved word
<code>
<font color=DARKRED><b><code>integer</code></b></font> is an alias for <font color=DARKRED><b><code>typeset -i</code></b></font>.
</code>
</DL>
<blockquote><small>
<DL>
<DT><font color=DARKRED><b><code>-Zn</code></b></font>
<DD>As for -R, but fill with zeroes if the value is a number
<DT><font color=DARKRED><b><code>-i</code></b></font>
<DD>Lower-case convert the named variables
<DT><font color=DARKRED><b><code>-u</code></b></font>
<DD>Upper-case convert the named variables
<DT><font color=DARKRED><b><code>-r</code></b></font>
<DD>Mark the variables as readonly
<DT><font color=DARKRED><b><code>-x</code></b></font>
<DD>Export the named variables to the enviroment
<DT><font color=DARKRED><b><code>-ft</code></b></font>
<DD>The variables are taken as function names. Turn on execution tracing.
</DL>
</small></blockquote>
</DL>
<p align=RIGHT class=“newpage”><font size=-1>(22)</font></P>
<font color=DARKBLUE>======Manipulating Variables (ksh/bash only)</H1></font>
=====Text variables=====
The <I>pattern</I> in the following uses the same wildcards as for filename matching.
<DL>
<DT><font color=DARKRED><b><code>${#var}</code></b></font>
<DD>returns the length of $var in characters
<DT><font color=DARKRED><b><code>${var%pattern}</code></b></font>
<DD>removes the shortest suffix of $var patching <I>pattern</I>
<DT><font color=DARKRED><b><code>${varpattern}</code></b></font>
<DD>removes the longest suffix of $var patching <I>pattern</I>
<DT><font color=DARKRED><b><code>${var#pattern}</code></b></font>
<DD>removes the shortest prefix of $var patching <I>pattern</I>
<DT><font color=DARKRED><b><code>${var##pattern}</code></b></font>
<DD>removes the longest prefix of $var patching <I>pattern</I>
</DL>
=====Numeric variables=====
<DL>
<DT><font color=DARKRED><b><code>$(( <I> integer expression </I> ))</code></b></font>
<DD>The $(( ... )) construction interprets the contents as an arithmetic
expression (integer only). Variables are referenced by name without the "$". Most of the arithmetic
syntax of the 'C' language is supported, including bit manipulations
<code>
(*,/,+,-,|,&,<<,>>. Use parentheses for changing precedence).
</code>
</DL>
<I>Examples</I>
<BR>
<font color=DARKRED><b><code>datapath=/data/public/project/trials/set1/datafile.dat</code></b></font>
<DL>
<DT><font color=DARKRED><b><code>filename=${datapath##*/}</code></b></font>
<DD><font color=DARKRED><b><code>filename</code></b></font> is set to "datafile.dat" since the longest <I>prefix</I>
pattern matching "*/" is the
leading directory path (compare <a href="http://www.FreeBSD.org/cgi/man.cgi?query=basename&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html" target=manpage><font color=DARKRED><b><code>basename</code></b></font></a>)
<DT><font color=DARKRED><b><code>path=${datapath%/*}</code></b></font>
<DD><font color=DARKRED><b><code>path</code></b></font> is set to "/data/public/project/trials/set1" since the shortest <I>suffix</I>
pattern matching "/*" is the
filename in the last directory (compare <a href="http://www.FreeBSD.org/cgi/man.cgi?query=dirname&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html" target=manpage><font color=DARKRED><b><code>dirname</code></b></font></a>)
<P>
<DT><font color=DARKRED><b><code>i=$((i+1))</code></b></font>
<DD>often used in <font color=DARKRED><b><code>while</code></b></font> loops
</DL>
<p align=RIGHT class="newpage"><font size=-1>(23)</font></P>
<font color=DARKBLUE>======Shell Functions</H1></font>
All but the earliest versions of <font color=DARKRED><b><code>sh</code></b></font> allow you define <I>shell functions</I>, which are visible only
to the shell script and can be used like any other command. Shell functions take precedence over
external commands if the same name is used. Functions execute in the same process as the caller,
and must be defined before use (appear earlier in the file). They allow a script to be broken
into maintainable chunks, and encourage code reuse between scripts.
====Defining functions====
<DL>
<DT><font color=DARKRED><b><code><I>identifier</I>() { <I>list</I>; }</code></b></font>
<DD>POSIX syntax for shell functions. Such functions do not restrict scope of variables
or signal traps.
The identifier follows the rules for variable names,
but uses a separate namespace.
<DT><font color=DARKRED><b><code>function <I>identifier</I> { <I>list</I>; }</code></b></font>
<DD>Ksh and bash optional syntax for defining a function. These functions may define local
variables and local signal
traps and so can more easily avoid side effects and be reused by multiple scripts.
</DL>
<P>
A function may read or modify any shell variable that exists in the calling script. Such variables
are <I>global</I>.
<P>
(ksh and bash only) Functions may also declare <I>local</I> variables in the function using <a href="http://www.FreeBSD.org/cgi/man.cgi?query=typeset&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html" target=manpage><font color=DARKRED><b><code>typeset</code></b></font></a> or
<font color=DARKRED><b><code>declare</code></b></font>.
Local variables are visible to the current function and any functions called by it.
<DL>
<DT><font color=DARKRED><b><code>return [[n]]</code></b></font>, <font color=DARKRED><b><code>exit [[n]]</code></b></font>
<DD>Return from a function with the given value, or exit the whole script with the given value.
</DL>
Without a <font color=DARKRED><b><code>return</code></b></font>, the function returns when it reaches the end, and the value is the
exit status of the last command it ran.
<P>
<I>Example:</I>
<PRE>
die()
{
<code>
# Print an error message and exit with given status
# call as: die status "message" [["message" ...]]
exitstat=$1; shift
for i in "$@"; do
print -R "$i"
done
exit $exitstat
</code>
}
</PRE>
====Calling functions.====
Functions are called like any other command. The output may be redirected independantly of the
script, and arguments passed to the function. Shell option flags like -x are unset in a function - you
must explicitly set them in each function to trace the execution. Shell functions may even be backgrounded
and run asynchronously, or run as coprocesses (ksh).
<P>
<I>Example:</I>
<PRE>
[[ -w $filename ]] || \\
<code>
die 1 "$file not writeable" "check permissions"
</code>
</PRE>
<P>
<I>Example:</I> Backgrounded function call. ex12 <A HREF=ex12.html TARGET=example>display</A>, <A HREF=ex12.txt TARGET=example>text</A>
<BLOCKQUOTE><SMALL>
<code><font color=DARKRED>
<font color=GRAY> 1:</font> <font color=DARKGREEN><i>#!/bin/sh</i></font>
<font color=GRAY> 2:</font>
<font color=GRAY> 3:</font> background()
<font color=GRAY> 4:</font> {
<font color=GRAY> 5:</font> sleep 10
<font color=GRAY> 6:</font> <font color=DARKBLUE>echo </font>"Background"
<font color=GRAY> 7:</font> sleep 10
<font color=GRAY> 8:</font> <font color=DARKGREEN><i># Function will return here - if backgrounded, the subprocess will exit.</i></font>
<font color=GRAY> 9:</font> }
<font color=GRAY> 10:</font>
<font color=GRAY> 11:</font> <font color=DARKBLUE>echo </font>"ps before background function"
<font color=GRAY> 12:</font> ps
<font color=GRAY> 13:</font> background &
<font color=GRAY> 14:</font> <font color=DARKBLUE>echo </font>"My PID=$$"
<font color=GRAY> 15:</font> <font color=DARKBLUE>echo </font>"Background function PID=$!"
<font color=GRAY> 16:</font> <font color=DARKBLUE>echo </font>"ps after background function"
<font color=GRAY> 17:</font> ps
<font color=GRAY> 18:</font> <font color=DARKBLUE>exit </font>0
</font></code>
</SMALL></BLOCKQUOTE>
<P>
<I>Example:</I>
<PRE>
vprint()
{
<code>
# Print or not depending on global "$verbosity"
# Change the verbosity with a single variable.
# Arg. 1 is the level for this message.
level=$1; shift
if [[ $level -le $verbosity ]]; then
print -R $*
fi
</code>
}
verbosity=2
vprint 1 This message will appear
vprint 3 This only appears if verbosity is 3 or higher
</PRE>
====Reuseable functions====
By using only command line arguments, not global variables, and taking care to minimise the side
effects of functions, they can be made reusable by multiple scripts. Typically they would be
placed in a separate file and read with the "<font color=DARKRED><b><code>.</code></b></font>" operator.
<P>
Functions may generate output to stdout, stderr, or any other file or filehandle. Messages to stdout
may be captured by command substitution (<font color=DARKRED><b><code>`myfunction`</code></b></font>, which provides another way for a function to
return information to the calling script. Beware of side-effects (and reducing reusability)
in functions which perform I/O.
<p align=RIGHT class="newpage"><font size=-1>(24)</font></P>
<font color=DARKBLUE>======Advanced I/O</H1></font>
Unix I/O is performed by assigning file descriptors to files or devices, and then
using those descriptors for reading and writing. Descriptors 0, 1, and 2 are always
used for stdin, stdout and stderr respectively. Stdin defaults to the keyboard,
while stdout and stderr both default to the current terminal window.
====Redirecting for the whole script====
Redirecting stdout, stderr and other file descriptors for the whole script
can be done with the <a href="http://www.FreeBSD.org/cgi/man.cgi?query=exec&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html" target=manpage><font color=DARKRED><b><code>exec</code></b></font></a> command.
<DL>
<code>
<DT><a href="http://www.FreeBSD.org/cgi/man.cgi?query=exec&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html" target=manpage><font color=DARKRED><b><code>exec</code></b></font></a> <font color=DARKRED><b><code> > outfile < infile </code></b></font>
<DD>with no command, the <font color=DARKRED><b><code>exec</code></b></font> just reassigns the I/O of the current shell.
<DT><font color=DARKRED><b><code>exec n>outfile</code></b></font>
</code>
<code>
<DD>The form n<, n> opens file descriptor n instead of the default stdin/stdout.
</code>
This can then be used with <font color=DARKRED><b><code>read -u</code></b></font> or <font color=DARKRED><b><code>print -u</code></b></font>.
</DL>
====Explicitly opening or duplicating file descriptors====
One reason to do this is to save the current
state of stdin/stdout, temporarily reassign them, then restore them.
<DL>
<code>
<DT><font color=DARKRED><b><code>>&n</code></b></font>
<DD>standard output is moved to whatever file descriptor n is currently pointing to
<DT><font color=DARKRED><b><code><&n</code></b></font>
<DD>standard input is moved to whatever file descriptor n is currently pointing to
<DT><font color=DARKRED><b><code>n><I>file</I></code></b></font>
<DD>file descriptor n is opened for writing on the named <I>file</I>.
<DT><font color=DARKRED><b><code>n>&1</code></b></font>
</code>
<code>
<DD>file descriptor n is set to whatever file descriptor 1 is currently pointing to.
</code>
</DL>
<I>Example</I> Sending messages to stderr (2) instead of stdout (1)
<PRE>
echo "Error: program failed" >&2
</PRE>
Echo always writes to stdout, but stdout can be temporarily reassigned to duplicate stderr (or other file
descriptors).
Conventionally Unix programs send error messages to stderr to keep them separated from stdout.
====Input and output to open file descriptors (ksh)====
Printing to file descriptors (usually more efficient than open/append/close):
<DL>
<code>
<DT><a href="http://www.FreeBSD.org/cgi/man.cgi?query=print&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html" target=manpage><font color=DARKRED><b><code>print</code></b></font></a> <font color=DARKRED><b><code> -u <I>n</I> args</code></b></font>
</code>
<code>
<DD>print to file descriptor <I>n</I>.
<DT>-p
<DD>write to the pipe to a coprocess (opened by |&)
</code>
</DL>
<P>
Reading from file descriptors other than stdin:
<DL>
<code>
<DT><a href="http://www.FreeBSD.org/cgi/man.cgi?query=read&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html" target=manpage><font color=DARKRED><b><code>read</code></b></font></a> <font color=DARKRED><b><code> -u <I>n</I> var1 var2 rest</code></b></font>
</code>
<code>
<DD>read a line from file descriptor n, parsing by $IFS, and placing the words into
</code>
the named variables. Any left over words all go into the last variable.
<code>
<DT>-p
<DD>read from the pipe to a coprocess (opened by |&)
</code>
</DL>
<P>
====Closing file handles====
<DL>
<code>
<DT><font color=DARKRED><b><code><&-</code></b></font>
<DD>standard input is explicitly closed
<DT><font color=DARKRED><b><code>>&-</code></b></font>
</code>
<code>
<DD>standard output is explicitly closed
</code>
</DL>
For example, to indicate to another program downstream in a pipeline that no more
data will be coming. All file descriptors are closed when a script exits.
<P>
I/O redirection operators are evaluated left-to-right. This makes a difference in a
statement like:
"<font color=DARKRED><b><code>>filename 2>&1</code></b></font>". (Many books with example scripts get this wrong)
===="Here" documents====
<DL>
<DT><font color=DARKRED><b><code><< [[-]]<I>string</I></code></b></font>
<DD>redirect input to the temporary file formed by everything up the matching <I>string</I>
at the start of a line. Allows for placing file content inline in a script.
</DL>
<I>Example:</I> ex5 <A HREF=ex5.html TARGET=example>display</A>, <A HREF=ex5.txt TARGET=example>text</A>
<BLOCKQUOTE><SMALL>
<code><font color=DARKRED>
<font color=GRAY> 1:</font> <font color=DARKGREEN><i>#!/bin/sh</i></font>
<font color=GRAY> 2:</font> <font color=DARKBLUE>echo </font>"Example of unquoted here document, with variable and command substitution"
<font color=GRAY> 3:</font>
<font color=GRAY> 4:</font> cat <<EOF
<font color=GRAY> 5:</font> This text will be fed to the "cat" program as
<font color=GRAY> 6:</font> standard input. It will also have variable
<font color=GRAY> 7:</font> and command substitutions performed.
<font color=GRAY> 8:</font> I am logged in as <font color=BLACK>$USER</font> and today is `<font color=PURPLE>date</font>`
<font color=GRAY> 9:</font> EOF
<font color=GRAY> 10:</font> <font color=DARKBLUE>echo</font>
<font color=GRAY> 11:</font> <font color=DARKBLUE>echo </font>"Example of quoted here document, with no variable or command substitution"
<font color=GRAY> 12:</font> <font color=DARKGREEN><i># The terminating string must be at the start of a line.</i></font>
<font color=GRAY> 13:</font> cat <<"EndOfInput"
<font color=GRAY> 14:</font> This text will be fed to the "cat" program as standard
<font color=GRAY> 15:</font> input. Since the text string marking the end was quoted, it does not get
<font color=GRAY> 16:</font> variable and command subsitutions.
<font color=GRAY> 17:</font> I am logged in as <font color=BLACK>$USER</font> and today is `<font color=PURPLE>date</font>`
<font color=GRAY> 18:</font> EndOfInput
</font></code>
</SMALL></BLOCKQUOTE>
<P>
<I>Example:</I> duplex <A HREF=duplex.html TARGET=example>display</A>, <A HREF=duplex.txt TARGET=example>text</A>
<BLOCKQUOTE><SMALL>
<code><font color=DARKRED>
<font color=GRAY> 1:</font> <font color=DARKGREEN><i>#!/bin/sh</i></font>
<font color=GRAY> 2:</font> <font color=DARKGREEN><i># Add in the magic postscript preface to perform</i></font>
<font color=GRAY> 3:</font> <font color=DARKGREEN><i># duplex printer control for Xerox docuprint.</i></font>
<font color=GRAY> 4:</font>
<font color=GRAY> 5:</font> <font color=DARKGREEN><i># To have this script send the files directly to the printer, use </i></font>
<font color=GRAY> 6:</font> <font color=DARKGREEN><i># a subshell to collect the output of the two 'cat' commands.</i></font>
<font color=GRAY> 7:</font>
<font color=GRAY> 8:</font> <font color=DARKGREEN><i><b>## (</b></i></font>
<font color=GRAY> 9:</font> cat << EOP
<font color=GRAY> 10:</font> %!PS
<font color=GRAY> 11:</font> BeginFeature: *Duplex DuplexTumble
<font color=GRAY> 12:</font> <</Duplex true /Tumble false>> setpagedevice
<font color=GRAY> 13:</font> %%EndFeature
<font color=GRAY> 14:</font> EOP
<font color=GRAY> 15:</font> cat ”<font color=BLACK>$@</font>“
<font color=GRAY> 16:</font> <font color=DARKGREEN><i><b>## ) | lpr</b></i></font>
</font></code>
</SMALL></BLOCKQUOTE>
<p align=RIGHT class=“newpage”><font size=-1>(25)</font></P>
<font color=DARKBLUE>======Wizard Level I/O</H1></font>
More complicated manipulations of file descriptors can be arranged.
Two such examples are shown here:
<P>
This short test script can be used to generate suitable output.<BR>
ex13: <A HREF=ex13.html TARGET=example>display</A>, <A HREF=ex13.txt TARGET=example>text</A>
<P>
<BLOCKQUOTE><PRE>
echo “This goes to stdout”
echo “This goes to stdout and has foo in the line”
echo “This goes to stderr” >&2
exit 99
</PRE></BLOCKQUOTE>
<P>
====Pass stderr of a command into a pipeline for further processing====
<I>Example:</I> ex14 <A HREF=ex14.html TARGET=example>display</A>, <A HREF=ex14.txt TARGET=example>text</A>
<P>
<font color=DARKRED><b><code>exec 3>&1</code></b></font> <BR>
<font color=DARKRED><b><code>./ex13.sh 2>&1 1>&3 3>&- | sed 's/stderr/STDERR/' 1>&2</code></b></font>
<P>
We duplicate stdout to another file descriptor (3), then run the first command with stderr redirected
to stdout and stdout redirected to the saved descriptor (3). The result is piped into other commands
as needed. The output of the pipeline is redirected back to stderr, so that stdout and stderr of the script
as a whole are what we expect.
<BLOCKQUOTE><SMALL>
<code><font color=DARKRED>
<font color=GRAY> 1:</font> <font color=DARKGREEN><i>#!/bin/sh</i></font>
<font color=GRAY> 2:</font> <font color=DARKGREEN><i># Example 14</i></font>
<font color=GRAY> 3:</font> <font color=DARKGREEN><i># Take stderr from a command and pass it into a pipe</i></font>
<font color=GRAY> 4:</font> <font color=DARKGREEN><i># for further processing.</i></font>
<font color=GRAY> 5:</font>
<font color=GRAY> 6:</font> <font color=DARKGREEN><i># Uses ex13.sh to generate some output to stderr</i></font>
<font color=GRAY> 7:</font> <font color=DARKGREEN><i># stdout of ex13 is processed normally</i></font>
<font color=GRAY> 8:</font>
<font color=GRAY> 9:</font> <font color=DARKGREEN><i># Save a copy of original stdout</i></font>
<font color=GRAY> 10:</font> <font color=DARKBLUE>exec </font>3>&1
<font color=GRAY> 11:</font>
<font color=GRAY> 12:</font> <font color=DARKGREEN><i># stdout from ex13.sh is directed to the original stdout (3)</i></font>
<font color=GRAY> 13:</font> <font color=DARKGREEN><i># stderr is passed into the pipe for further processing.</i></font>
<font color=GRAY> 14:</font> <font color=DARKGREEN><i># stdout from the pipe is redirected back to stderr</i></font>
<font color=GRAY> 15:</font> ./ex13.sh 2>&1 1>&3 3>&- | sed 's/stderr/STDERR/' 1>&2
<font color=GRAY> 16:</font>
<font color=GRAY> 17:</font> <font color=DARKGREEN><i># 3 is closed before running the command, just in case it cares</i></font>
<font color=GRAY> 18:</font> <font color=DARKGREEN><i># about inheriting open file descriptors.</i></font>
</font></code>
</SMALL></BLOCKQUOTE>
====Capture the exit status of a command in the middle of a pipeline====
<I>Example:</I> ex15 <A HREF=ex15.html TARGET=example>display</A>, <A HREF=ex15.txt TARGET=example>text</A>
<P>
<font color=DARKRED><b><code> exec 3>&1 </code></b></font><BR>
<font color=DARKRED><b><code> ex13stat=`3)
e.g. <B>HUP, INT, QUIT, TERM</B>. A Ctrl-C at the terminal generates a <B>INT</B>.
</DL>
<UL>
<LI>A handler of <font color=DARKRED><b><code>-</code></b></font> resets the signals to their default values
<LI>A handler of <font color=DARKRED><b><code></code></b></font> (null) ignores the signals
<LI>Special signal values are as follows: <DL> <DT><font color=DARKRED><b>
EXIT
</b></font> <DD>the handler is called when the function exits, or when the whole script exits. The exit signal has value 0. <DT><font color=DARKRED><b>
ERR
</b></font> (ksh) <DD>the handler is called when any command has a non-zero exit status <DT><font color=DARKRED><b>
DEBUG
</b></font> (ksh) <DD>the handler is called after <I>each</I> command. </DL>
</UL> <P> <I>Example:</I> ex8 <A HREF=ex8.html TARGET=example>display</A>, <A HREF=ex8.txt TARGET=example>text</A> <BLOCKQUOTE><SMALL>
<font color=DARKRED>
<font color=GRAY> 1:</font> <font color=DARKGREEN><i>#!/bin/bash</i></font>
<font color=GRAY> 2:</font> <font color=DARKGREEN><i># Try this under bash, ksh and sh</i></font>
<font color=GRAY> 3:</font>
<font color=GRAY> 4:</font> <font color=DARKBLUE>trap </font>huphandler HUP
<font color=GRAY> 5:</font> <font color=DARKBLUE>trap </font>// QUIT
<font color=GRAY> 6:</font> <font color=DARKBLUE>trap </font>exithandler TERM INT
<font color=GRAY> 7:</font>
<font color=GRAY> 8:</font> huphandler()
<font color=GRAY> 9:</font> {
<font color=GRAY> 10:</font> <font color=DARKBLUE>echo </font>'Received SIGHUP'
<font color=GRAY> 11:</font> <font color=DARKBLUE>echo </font>"continuing"
<font color=GRAY> 12:</font> }
<font color=GRAY> 13:</font>
<font color=GRAY> 14:</font> exithandler()
<font color=GRAY> 15:</font> {
<font color=GRAY> 16:</font> <font color=DARKBLUE>echo </font>'Received SIGTERM or SIGINT'
<font color=GRAY> 17:</font> <font color=DARKBLUE>exit </font>1
<font color=GRAY> 18:</font> }
<font color=GRAY> 19:</font> <font color=DARKGREEN><i><b>## Execution starts here - infinite loop until interrupted</b></i></font>
<font color=GRAY> 20:</font> <font color=DARKGREEN><i># Use ":" or "true" for infinite loop</i></font>
<font color=GRAY> 21:</font> <font color=DARKGREEN><i># SECONDS is built-in to bash and ksh. It is number of seconds since script started</i></font>
<font color=GRAY> 22:</font> <font color=DARKBLUE>> </font>is like a comment, but it is evaluated for side effects and evaluates to true
<font color=GRAY> 23:</font> <font color=BLACK>seconds=</font>0
<font color=GRAY> 24:</font> <font color=DARKBLUE>while </font>> ; <font color=DARKBLUE>do</font>
<font color=GRAY> 25:</font> <font color=DARKGREEN><i># while true; do</i></font>
<font color=GRAY> 26:</font> sleep 5
<font color=GRAY> 27:</font> <font color=BLACK>seconds=</font><font color=DARKBLUE>$((seconds + 5))</font>
<font color=GRAY> 28:</font> <font color=DARKBLUE>echo </font>-n "<font color=BLACK>$SECONDS</font> <font color=BLACK>$seconds</font> - "
<font color=GRAY> 29:</font> <font color=DARKBLUE>done</font>
</font>
</SMALL></BLOCKQUOTE>
Exit handlers can be defined to clean up temporary files or reset the state of devices. This can be useful if the script has multiple possible exit points.
<p align=RIGHT class=“newpage”><font size=-1>(29)</font></P> <font color=DARKBLUE>======Security issues in shell scripts</H1></font>
Shell scripts are often used by system administrators and are run as a priviledged user. <P>
<UL> <LI>Don't use set-UID scripts. <blockquote><small> Most systems don't even allow a script to be made set-UID. It is impossible (due to inherent race conditions) to ensure that a set-uid script cannot be compromised. Use wrapper programs like <a href=“http://www.FreeBSD.org/cgi/man.cgi?query=sudo&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b>
sudo
</b></font></a> instead. </small></blockquote> <P> <LI>Always explicitly set <font color=DARKRED><b>
$PATH
</b></font> at the start of a script, so that you know exactly which external programs will be used. <P> <LI>If possible, don't use temporary files. If they cannot be avoided, use <font color=DARKRED><b>
$TMPDIR
</b></font>, and create files safely (e.g. <a href=“http://www.FreeBSD.org/cgi/man.cgi?query=mktemp&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b>
mktemp
</b></font></a>).
<blockquote><small> Often scripts will write to a fixed, or trivially generated temporary filename in /tmp. If the file already exists and you don't have permission to overwrite it, the script will fail. If you do have permission to overwrite it, you will delete the previous contents. Since /tmp is public write, another user may create files in it, or possibly fill it completely. <BR> <I>Example:</I> <OL> <LI>A link is created by an unprivileged user in /tmp: <font color=DARKRED><b>
/tmp/scratch -> /vmunix
</b></font> <LI>A root user runs a script that blindly writes a scratch file to /tmp/scratch, and overwrites the operating system. </OL> Environment variable <font color=DARKRED><b>
$TMPDIR
</b></font> is often used to indicate a preferred location for temporary files (e.g., a per-user directory). Some systems may use <font color=DARKRED><b>
$TMP
</b></font> or <font color=DARKRED><b>
$TEMP
</b></font>. Safe scratch files can be made by creating a new directory, owned and writeable only by you, then creating files in there.
<BR> <I>Example:</I> <PRE> (umask 077 && mkdir /tmp/tempdir.$$) || exit 1 </PRE> or (deluxe version) <PRE> tmp=${TMPDIR:-/tmp} tmp=$tmp/tempdir.$RANDOM.$RANDOM.$RANDOM.$$ (umask 077 && mkdir $tmp) || {
echo "Could not create temporary directory" 1>&2
exit 1
} </PRE> Alternatively, many systems have <a href=“http://www.FreeBSD.org/cgi/man.cgi?query=mktemp&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b>
mktemp
</b></font></a> to safely create a temporary file and return the filename, which can be used by the script and then deleted.
</small></blockquote> <P> <LI>Check exit status of everything you do. <P> <LI>Don't trust user input <UL>
<LI>contents of files
<LI>data piped from other programs
<LI>file <EM>names</EM>. Output of filename generation with wildcards, or directly from
<font color=DARKRED><b><code>ls
</b></font> or <font color=DARKRED><b>
find
</b></font> </code>
</UL> </UL> <P> <I>Example:</I> <BR> Consider the effects of a file named ”<font color=DARKRED><b>
myfile;cd /;rm *
</b></font>“ if processed, unquoted, by your script. <blockquote><small> One possible way to protect against weirdo characters in file names: <PRE>
protect_filenames() {
sed -es/\\\\\\\\/\\\\\\\\\\\\\\\\/g \\
-es/\\\\\\'/\\\\\\\\\\'/g \\
-es/\\\\\\"/\\\\\\\\\\"/g \\
-es/\\\\\\;/\\\\\\\\\\;/g \\
-es/\\\\\\?/\\\\\\\\\\?/g \\
-es/\\\\\\*/\\\\\\\\\\*/g \\
-es/\\\\\\ /\\\\\\\\\\ /g
} </PRE> If using GNU <font color=DARKRED><b>
find
</b></font> and <font color=DARKRED><b>
xargs
</b></font>, there is a much cleaner option to null-terminate generated pathnames.
</small></blockquote>
<p align=RIGHT class=“newpage”><font size=-1>(30)</font></P> <font color=DARKBLUE>======Style</H1></font>
Shell scripts are very frequently written quickly for a single purpose, used once and discarded. They are also as frequently kept and used many times, and migrate into other uses, but often do not receive the same level of testing and debugging that other software would be given in the same situation. It <EM>is</EM> possible to apply general principles of good software engineering to shell scripts.
<UL> <LI>Preface scripts with a statement of purpose, author, date and revision notes <LI>Use a revision control system for complex scripts with a long lifetime <LI>Assume your script <EM>will</EM> have a long lifetime unless you are certain it won't <LI>Document any non-standard external utilities which your script needs <LI>Document your scripts with inline comments - you'll need them in a few months when you edit it. <LI>Treat standard input and output in the normal way, so that your script can be used in combination with other programs (the Unix toolkit philosophy) <LI>Be consistent in the format of your output, so that other programs can rely on it <LI>Use options to control behaviour such as verbosity of output. Overly chatty programs are very hard to combine with other utilities <LI>Use interactive features (prompting the user for information) <STRONG>very sparingly</STRONG>. Doing so renders the script unuseable in pipeline combinations with other programs, or in unattended operations. <LI>Test (a lot) </UL>
<UL> <LI>If an existing tool already does what you need - use it. <LI>If the script is more than a few hundred lines, you are probably using the wrong tool. <LI>If performance is horrible, and the script is used a lot, you might want to consider another language. </UL>
<p align=RIGHT class=“newpage”><font size=-1>(31)</font></P>
<font color=DARKBLUE>======Some longer examples</H1></font>
The class accounts have directories with all of the examples from the books by Blinn, Michael, Rosenblatt, and Taylor. These can also be downloaded (see the References page). Some of these are linked below (but not included in the printed notes), with additional comments. <P> Download a <A HREF=“examples.tar.gz”>compressed tar file</A> of all example scripts used in these notes. <P> <UL> <LI><B>postprint</B> <A HREF=postprint.html TARGET=example>display</A>, <A HREF=postprint.txt TARGET=example>text</A><BR>
A wrapper script for printing a mix of text and postscript files
<P> <LI><B>checkpath</B> <A HREF=checkpath.html TARGET=example>display</A>, <A HREF=checkpath.txt TARGET=example>text</A><BR> Check all the directories in the <font color=DARKRED><b>
$PATH
</b></font> for possibly conflicting programs.
<p> <li><b>run-with-timeout</b> <a href=run-with-timeout.html target=example>display</a>,
<a href=run-with-timeout.txt target=example>text</a><br> Run a command with a timeout. Kill the command if it hasn't finished when the timeout expires.
<P> <LI><B>MailPkg</B> <A HREF=MailPkg.html TARGET=example>display</A>, <A HREF=MailPkg.txt TARGET=example>text</A><BR> Tar, compress, split and uuendcode a set of files for mailing. (Blinn)
<P> <LI><B>Ptree</B> (original) <A HREF=ptree-sh.html TARGET=example>display</A>, <A HREF=ptree-sh.txt TARGET=example>text</A><BR>
<LI><B>Ptree</B> (ksh version) <A HREF=ptree-ksh.html TARGET=example>display</A>, <A HREF=ptree-ksh.txt TARGET=example>text</A><BR> Runs “ps” to get a process listing and then reformats to show the process family hierarchies. The original example is pure Bourne shell and inefficient. The ksh version is a fairly simple translation to use ksh internal commands where possible, and avoid writing scratch files, and runs very much faster. (Blinn). </UL> <P> This entire tutorial was created from individual HTML pages using a content management system written as ksh scripts (heavily using sed to edit the pages), coordinated by <a href=“http://www.FreeBSD.org/cgi/man.cgi?query=make&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b>
make
</b></font></a>. <!– <UL> <P> <LI><B>buildhtml</B> <A HREF=buildhtml.html TARGET=example>display</A>, <A HREF=buildhtml.txt TARGET=example>text</A><BR> Turns a stand-alone HTML file into a component of the tutorial, with navigation links and frames.
<LI><B>buildslidelist</B> <A HREF=buildslidelist.html TARGET=example>display</A>, <A HREF=buildslidelist.txt TARGET=example>text</A><BR> Creates the slide list which appears as the left frame of the tutorial.
<LI><B>buildframeset</B> <A HREF=buildframeset.html TARGET=example>display</A>, <A HREF=buildframeset.txt TARGET=example>text</A><BR> Creates a top level frameset to call the slide list and content pages.
<LI><B>ksh2html</B> <A HREF=ksh2html.html TARGET=example>display</A>, <A HREF=ksh2html.txt TARGET=example>text</A><BR> Creates the colourized HTML page of a shell script.
<LI><B>Makefile</B> <A HREF=Makefile.txt TARGET=example>text</A><BR> Makefile used to build the site using the above tools </UL> –> <P> You can even write an entire web server as a shell script. This one is part of the <A HREF=“http://www.nisi.ab.ca/lrp/” TARGET=_blank>LEAF</A>
(Linux Embedded Appliance Firewall) project. This wouldn't be suitable for much load, but handles occasional queries on static HTML and CGI scripts. <a href=“http://www.nisi.ab.ca/lrp/Packages/weblet.htm” TARGET=_blank>(www.nisi.ab.ca/lrp/Packages/weblet.htm)</a> <UL> <LI><B>sh-httpd</B> <A HREF=sh-httpd.html TARGET=example>display</A>, <A HREF=sh-httpd.txt TARGET=example>text</A><BR> </UL>
<p align=RIGHT class=“newpage”><font size=-1>(32)</font></P> <font color=DARKBLUE>======A Toolkit of commonly used external commands</H1></font>
The following commands are very frequently used in shell scripts. Many of them are used in the examples in these notes. This is just a brief recap – see the man pages for details on usage. The most useful are flagged with <FONT COLOR=GREEN>*</FONT>. <P> Most of these commands will operate on a one or more named files, or will operate on a stream of data from standard input if no files are named.
<DL>
<DT><a href=“http://www.FreeBSD.org/cgi/man.cgi?query=ls&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b>
ls
</b></font></a> <FONT COLOR=GREEN>*</FONT> <DD>list contents of a directory, or list details of files and directories.
<DT> <a href=“http://www.FreeBSD.org/cgi/man.cgi?query=mkdir&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b>
mkdir
</b></font></a>; <a href=“http://www.FreeBSD.org/cgi/man.cgi?query=rmdir&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b>
rmdir
</b></font></a> <FONT COLOR=GREEN>*</FONT> <DD>Make and Remove directories.
<DT> <a href=“http://www.FreeBSD.org/cgi/man.cgi?query=rm&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b>
rm
</b></font></a>; <a href=“http://www.FreeBSD.org/cgi/man.cgi?query=cp&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b>
cp
</b></font></a>; <a href=“http://www.FreeBSD.org/cgi/man.cgi?query=mv&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b>
mv
</b></font></a> <FONT COLOR=GREEN>*</FONT>
<DD>Remove (delete), Copy and Move (rename) files and directories
<DT><a href=“http://www.FreeBSD.org/cgi/man.cgi?query=touch&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b>
touch
</b></font></a> <FONT COLOR=GREEN>*</FONT> <DD>Update the last modifed timestamp on a file, to make it appear to have just been written. <blockquote><small> If the file does not exist, a new zero-byte file is created, which is often useful to signify that an event has occurred. </small></blockquote>
<DT><a href=“http://www.FreeBSD.org/cgi/man.cgi?query=tee&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b>
tee
</b></font></a> <DD>Make a duplicate copy of a data stream - used in pipelines to send one copy to a log file and a second copy on to another program. (Think plumbing). </DL>
<DL>
<DT><a href=“http://www.FreeBSD.org/cgi/man.cgi?query=echo&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b>
echo
</b></font></a> <FONT COLOR=GREEN>*</FONT> <DD>Echo the arguments to standard output – used for messages from scripts. Some versions of “sh”, and all csh/ksh/bash shells internalized “echo”. <blockquote><small> Conflicts sometimes arise over the syntax for echoing a line with no trailing CR/LF. Some use “\\c” and some use option “-n”. To avoid these problems, ksh also provides the “print” command for output. </small></blockquote>
<DT><a href=“http://www.FreeBSD.org/cgi/man.cgi?query=cat&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b>
cat
</b></font></a> <FONT COLOR=GREEN>*</FONT> <DD>Copy and concatenate files; display contents of a file
<DT><a href=“http://www.FreeBSD.org/cgi/man.cgi?query=head&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b>
head
</b></font></a>, <a href=“http://www.FreeBSD.org/cgi/man.cgi?query=tail&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b>
tail
</b></font></a> <FONT COLOR=GREEN>*</FONT> <DD>Display the beginning of a file, or the end of it.
<DT><a href=“http://www.FreeBSD.org/cgi/man.cgi?query=cut&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b>
cut
</b></font></a> <DD>Extract selected fields from each line of a file. Often awk is easier to use, even though it is a more complex program.
<DT><a href=“http://www.FreeBSD.org/cgi/man.cgi?query=wc&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b>
wc
</b></font></a> <DD>Count lines, words and characters in the input.
</DL>
<DL>
<DT><a href=“http://www.FreeBSD.org/cgi/man.cgi?query=compress&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b>
compress
</b></font></a>; <a href=“http://www.FreeBSD.org/cgi/man.cgi?query=gzip&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b>
gzip
</b></font></a>, <a href=“http://www.FreeBSD.org/cgi/man.cgi?query=zip&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b>
zip
</b></font></a>; <a href=“http://www.FreeBSD.org/cgi/man.cgi?query=tar&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b>
tar
</b></font></a> <FONT COLOR=GREEN>*</FONT> <DD>Various utilities to compress/uncompress individual files, combine multiple files into a single archive, or do both.
</DL>
<DL> <DT><a href=“http://www.FreeBSD.org/cgi/man.cgi?query=sort&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b>
sort
</b></font></a> <FONT COLOR=GREEN>*</FONT> <DD>Sort data alphabetically or numerically.
<DT><a href=“http://www.FreeBSD.org/cgi/man.cgi?query=grep&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b>
grep
</b></font></a> <FONT COLOR=GREEN>*</FONT> <DD>Search a file for lines containing character patterns. The patterns can be simple fixed text, or very complex regular expressions. <blockquote><small> The name comes from “Global Regular Expression and Print” – a function from the Unix editors which was used frequently enough to warrant getting its own program. </small></blockquote>
<DT><a href=“http://www.FreeBSD.org/cgi/man.cgi?query=uniq&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b>
uniq
</b></font></a> <FONT COLOR=GREEN>*</FONT>
<DD>Remove duplicate lines, and generate a count of repeated lines.
<DT><a href=“http://www.FreeBSD.org/cgi/man.cgi?query=wc&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b>
wc
</b></font></a> <FONT COLOR=GREEN>*</FONT> <DD>Count lines, words and characters in a file.
</DL>
<DL>
<DT><a href=“http://www.FreeBSD.org/cgi/man.cgi?query=date&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b>
date
</b></font></a> <FONT COLOR=GREEN>*</FONT>
<DD>Display the current date and time (flexible format). Useful for conditional execution based on time, and for timestamping output.
<DT><a href=“http://www.FreeBSD.org/cgi/man.cgi?query=ps&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b>
ps
</b></font></a> <FONT COLOR=GREEN>*</FONT> <DD>List the to a running processes.
<DT><a href=“http://www.FreeBSD.org/cgi/man.cgi?query=kill&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b>
kill
</b></font></a> <FONT COLOR=GREEN>*</FONT> <DD>Send a signal (interrupt) to a running process.
<DT><a href=“http://www.FreeBSD.org/cgi/man.cgi?query=id&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b>
id
</b></font></a> <DD>Print the user name and UID and group of the current user (e.g. to distinguish priviledged users before attempting to run programs which may fail with permission errors)
<DT><a href=“http://www.FreeBSD.org/cgi/man.cgi?query=who&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b>
who
</b></font></a> <DD>Display who is logged on the system, and from where they logged in.
<DT><a href=“http://www.FreeBSD.org/cgi/man.cgi?query=uname&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b>
uname
</b></font></a> <FONT COLOR=GREEN>*</FONT> <DD>Display information about the system, OS version, hardware architecture etc.
<DT><a href=“http://www.FreeBSD.org/cgi/man.cgi?query=mail&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b>
</b></font></a> <FONT COLOR=GREEN>*</FONT> <DD>Send mail, from a file or standard input, to named recipients. Since scripts are often used to automate long-running background jobs, sending notification of completion by mail is a common trick.
<DT><a href=“http://www.FreeBSD.org/cgi/man.cgi?query=logger&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b>
logger
</b></font></a> <DD>Place a message in the central system logging facility. Scripts can submit messages with all the facilities available to compiled programs.
<DT><a href=“http://www.FreeBSD.org/cgi/man.cgi?query=hostname&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b>
hostname
</b></font></a> <DD>Display the hostname of the current host - usful to keep track of where your programs are running
</DL>
<DL>
<DT><a href=“http://www.FreeBSD.org/cgi/man.cgi?query=test&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b>
test
</b></font></a>; <font color=DARKRED><b>
[[
</b></font> <FONT COLOR=GREEN>*</FONT> <DD>The conditional test, used extensively in scripts, is also an external program which evaluates the expression given as an argument and returns true (0) or false (1) exit status. The name “<BR> <font color=DARKRED><b><code>if [[ -w logfile </code></b></font><BR> actually runs a program ”", with arguments "-w logfile“, and returns a true/false value to the “if” command.
<blockquote><small> In ksh and most newer versions of sh, ”//www.FreeBSD.org/cgi/man.cgi\\{4\\}\\(0-9\\{3\\}\\)L\\.*'</code></b></font> </code>
<DT><a href=“http://www.FreeBSD.org/cgi/man.cgi?query=dc&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b>
dc
</b></font></a> <DD>Desk Calculator - an RPN calculator, using arbitrary precision arithmetic and user-specified bases. Useful for more complex arithmetic expressions than can be performed internally or using <a href=“http://www.FreeBSD.org/cgi/man.cgi?query=expr&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b>
expr
</b></font></a>
<DT><a href=“http://www.FreeBSD.org/cgi/man.cgi?query=bc&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b>
bc
</b></font></a> <DD>A preprocessor for <a href=“http://www.FreeBSD.org/cgi/man.cgi?query=dc&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b>
dc
</b></font></a> which provides infix notation and a C-like syntax for expressions and functions.
</DL>
<DL>
<DT><a href=“http://www.FreeBSD.org/cgi/man.cgi?query=paste&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b>
paste
</b></font></a> <DD>Merge lines from multiple files into tab-delimited columns.
<DT><a href=“http://www.FreeBSD.org/cgi/man.cgi?query=join&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b>
join
</b></font></a> <DD>Perform a join (in the relational database sense) of lines in two sorted input files.
</DL>
<p align=RIGHT class=“newpage”><font size=-1>(33)</font></P> <font color=DARKBLUE>======References, Resources, Man pages etc.</H1></font>
The standard man pages for <a href=“http://www.FreeBSD.org/cgi/man.cgi?query=sh&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b>
sh
</b></font></a> and <a href=“http://www.FreeBSD.org/cgi/man.cgi?query=ksh&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b>
ksh
</b></font></a> are quite complete, but not easy to learn from. The following is a sampling of the many available books on the subject. The Bolsky and Korn book might be viewed as the standard “reference”. The Blinn book is Bourne shell, but everything in it should work for either shell.<BR> The links are to publisher's web sites, or Amazon.com. Some links are also given to the example scripts provided with the books.
<P>
<UL> <LI><I>The New KornShell Command And Programming Language</I>, by Morris I. Bolsky, David G. Korn (Contributor). <A HREF=http://www.amazon.com/exec/obidos/ASIN/0131827006/qid=1005942232/sr=1-2/ref=sr_1_14_2/103-1397887-1135830> <SMALL>More info</SMALL></A><BR><BR>
<LI><I>Learning the Korn Shell, 2nd Edn.</I> by Bill Rosenblatt and Arnold Robbins. <A HREF=“http://www.oreilly.com/catalog/korn2/index.html”> <SMALL>More info</SMALL></A><BR><BR>
<LI><I>Korn Shell Programming by Example</I>, by Dennis O'Brien, David Pitts (Contributor). <A HREF=http://www.amazon.com/exec/obidos/ASIN/0789724650/qid=1005942392/sr=1-3/ref=sr_1_15_3/103-1397887-1135830> <SMALL>More info</SMALL></A><BR><BR>
<LI><I>The Korn Shell Linux and Unix Programming Manual (2nd Edn)</I> by Anatole Olczak.
<A HREF=http://www.amazon.com/exec/obidos/tg/detail/-/0201176882/qid=1107403516/sr=8-1/ref=sr_8_xs_ap_i1_xgl14/103-6122890-2667019?v=glance&s=books&n=507846> <SMALL>More info</SMALL></A><BR><BR>
<LI><I>Portable Shell Programming: An Extensive Collection of Bourne Shell Examples</I> by Bruce Blinn. <A HREF=http://www.amazon.com/exec/obidos/ASIN/0134514947/qid%3D1005942627/ref%3Dsr%5F11%5F0%5F1/103-1397887-1135830> <SMALL>More info</SMALL></A><BR> <SMALL>Examples from this book can be <A HREF=“examples/Blinn.tar.gz”>downloaded</A> for study.</SMALL><BR> <BR>
<LI><I>Linux Shell Scripting with Bash</I> by Ken O. Burtch. <A HREF=http://www.samspublishing.com/title/0672326426> <SMALL>More info</SMALL></A><BR> <BR>
<LI><I>Unix Shell Programming</I> by Stephen Kochan and Patrick Wood (third Edition). <A HREF=“http://www.amazon.com/exec/obidos/tg/detail/-/0672324903/qid=1119900993/sr=8-1/ref=sr_8_xs_ap_i1_xgl14/002-8046014-8765618?v=glance&s=books&n=507846”> <SMALL>More info</SMALL></A><BR> <BR>
<LI><I>Teach yourself Shell Programming in 24 Hours</I>, by S. Veeraraghavan. SAMS 2nd Edn. (2002)
<SMALL><A HREF=“http://www.samspublishing.com/catalog/product.asp?product_id={ECDDB44B-A984-4329-94C5-7873F3E45A58}”>More info</A> </SMALL><BR><BR>
<LI><I>Mastering Unix Shell Scripting</I> by Randal K. Michael, Wiley (2003) <SMALL><A HREF=“http://www.wiley.com/WileyCDA/WileyTitle/productCd-0471218219.html”>More info</A> Light on basics, but develops scripting through examples. Ksh only. Examples can be <A HREF=“http://www.wiley.com/legacy/compbooks/michael/index.html” target=_blank>downloaded</A> from the Wiley site (www.wiley.com/legacy/compbooks/michael/). </SMALL><BR><BR>
<LI><I>Wicked Cool Shell Scripts</I>
by Dave Taylor, No Starch Press (2004) <SMALL><A HREF=“http://www.nostarch.com/wcss.htm”>More info</A> Develops scripting entirely through examples, drawn from Linux and OSX in addition to traditional Unix. Recommended, but not for beginners. Examples can be <A HREF=“http://www.intuitive.com/wicked/wicked-cool-shell-script-library.shtml” target=_blank>downloaded</A> from the Intuitive site (www.intuitive.com/wicked/wicked-cool-shell-script-library.shtml). </SMALL><BR><BR>
<LI><I>Unix Power Tools</I>, by S. Powers, J. Peek, T. O'Reilly, M. Loudikes et al. <A HREF=“http://www.oreilly.com/catalog/upt3/”> <SMALL>More info</SMALL></A> </UL> <P>
<UL> <LI><A HREF=“http://www.shelldorado.com/” TARGET=external>Shelldorado</A> (http://www.shelldorado.com)<BR> <SMALL>Lots of links to scripting resources</SMALL> <P> <LI><A HREF=“http://www.kornshell.com/” TARGET=external>Kornshell</A> (http://www.kornshell.com)<BR> <SMALL>The official Korn shell home page, with download links.</SMALL> <P> <LI><A HREF=“http://www.osxfaq.com/Tutorials/LearningCenter/” TARGET=external>Mac OSX Unix tutorial</A> (http://www.osxfaq.com/Tutorials/LearningCenter/)<BR>
<SMALL>Good resource on advanced use of OSX and Unix shell scripting in general</SMALL> </UL>
<UL> <LI><A HREF=“http://www.research.att.com/sw/tools/uwin/” TARGET=external>U/Win</A> (http://www.research.att.com/sw/tools/uwin/)<BR> <SMALL> A free port of ksh and Unix command line utilities, plus Windows DLL for Unix compatability. Developed by AT&T Research. </SMALL> <P> <LI><A HREF=“http://www.cygwin.com/” TARGET=external>Cygwin</A> (http://www.cygwin.com/)<BR>
<SMALL> A free Linux-like environment for Windows. Provides bash, command line utilities and DLLs. Developed by RedHat. An X server is also available. </SMALL> <P> <LI><A HREF=“http://www.mkssoftware.com/” TARGET=external>MKS Toolkit</A> (http://www.mkssoftware.com/)<BR> <SMALL> A commercial ksh clone and command line utilities, plus DLL for Unix compatability. An X server is also available. </SMALL> <P> <LI><A HREF=“http://www.microsoft.com/windows/sfu/” TARGET=external>Microsoft Services for UNIX</A> (http://www.microsoft.com/windows/sfu/)<BR> <SMALL>
A POSIX environment for Windows, with ksh, csh, command line tools, libraries and software development tools. Developed by Interix and bought by Microsoft. Free download. </SMALL> </UL>
<br><br><hr> <font size=-1>Unix shell scripting with ksh/bash: Course Handout</font><br>
<font size=-2> (last update Thursday, 22-Mar-2012 12:43:56 EDT) ©Dartmouth College http://www.dartmouth.edu/~rc/classes/ksh </font>
</font> </body> </html>
<font color=DARKRED> <font color=GRAY> 1:</font> <font color=DARKGREEN><i>#!/bin/sh</i></font> <font color=GRAY> 2:</font> <font color=DARKGREEN><i># Example 15</i></font> <font color=GRAY> 3:</font> <font color=GRAY> 4:</font> <font color=DARKGREEN><i># Uses ex13.sh to generate some output and give us an </i></font> <font color=GRAY> 5:</font> <font color=DARKGREEN><i># exit status to capture.</i></font> <font color=GRAY> 6:</font> <font color=GRAY> 7:</font> <font color=DARKGREEN><i># Get the exit status of ex13 into $ex13stat. </i></font> <font color=GRAY> 8:</font> <font color=DARKGREEN><i># stdout of ex13 is processed normally</i></font> <font color=GRAY> 9:</font> <font color=GRAY> 10:</font> <font color=DARKGREEN><i># Save a copy of stdout</i></font> <font color=GRAY> 11:</font> <font color=DARKBLUE>exec </font>3>&1 <font color=GRAY> 12:</font> <font color=DARKGREEN><i># Run a subshell, with 4 duplicated to 1 so we get it in stdout. </i></font> <font color=GRAY> 13:</font> <font color=DARKGREEN><i># Capture the output in ``</i></font> <font color=GRAY> 14:</font> <font color=DARKGREEN><i># ex13stat=`( ... ) 4>&1`</i></font> <font color=GRAY> 15:</font> <font color=DARKGREEN><i># Inside the subshell, run another subshell to execute ex13, </i></font> <font color=GRAY> 16:</font> <font color=DARKGREEN><i># and echo the status code to 4</i></font> <font color=GRAY> 17:</font> <font color=DARKGREEN><i># (./ex13.sh; <font color=DARKBLUE>echo </font>$? >&4)</i></font> <font color=GRAY> 18:</font> <font color=DARKGREEN><i># stdout from the inner subshell is processed normally, but the </i></font> <font color=GRAY> 19:</font> <font color=DARKGREEN><i># subsequent output must be directed to 3 so it goes to the </i></font> <font color=GRAY> 20:</font> <font color=DARKGREEN><i># original stdout and not be captured by the ``</i></font> <font color=GRAY> 21:</font> <font color=BLACK>ex13stat=</font>`<font color=PURPLE>((./ex13.sh; <font color=DARKBLUE>echo </font>$? >&4) | grep 'foo' 1>&3) 4>&1</font>` <font color=GRAY> 22:</font> <font color=GRAY> 23:</font> <font color=DARKBLUE>echo </font>Last command status=$? <font color=GRAY> 24:</font> <font color=DARKBLUE>echo </font>ex13stat=<font color=BLACK>$ex13stat</font> <font color=GRAY> 25:</font> <font color=GRAY> 26:</font> <font color=DARKGREEN><i># If any of the commands really care about inheriting open file </i></font> <font color=GRAY> 27:</font> <font color=DARKGREEN><i># descriptors that they don't need then a more correct command line </i></font> <font color=GRAY> 28:</font> <font color=DARKGREEN><i># closes the descriptors before running the commands</i></font> <font color=GRAY> 29:</font> <font color=DARKBLUE>exec </font>3>&1 <font color=GRAY> 30:</font> <font color=BLACK>ex13stat=</font>`((./ex13.sh 3>&- 4>&- ; <font color=DARKBLUE>echo </font>$? >&4) | \\ <font color=GRAY> 31:</font> grep 'foo' 1>&3 3>&- 4>&- ) 4>&1` <font color=GRAY> 32:</font> <font color=DARKBLUE>echo </font>Last command status=$? <font color=GRAY> 33:</font> <font color=DARKBLUE>echo </font>ex13stat=<font color=BLACK>$ex13stat</font> </font></SMALL></BLOCKQUOTE> <P> Combine the above two techniques: <P> <I>Example:</I> ex16 <A HREF=ex16.html TARGET=example>display</A>, <A HREF=ex16.txt TARGET=example>text</A> <P> <font color=DARKRED><b>
exec 3>&1</b></font><BR> <font color=DARKRED><b>
ex13stat=`((./ex13.sh 2>&1 1>&3 3>&- 4>&- ; echo $? >&4) | \\</b></font><BR> <font color=DARKRED><b>
sed s/err/ERR/ 1>&2 3>&- 4>&- ) 4>&1`</b></font> <BLOCKQUOTE><SMALL>
<font color=DARKRED> <font color=GRAY> 1:</font> <font color=DARKGREEN><i>#!/bin/sh</i></font> <font color=GRAY> 2:</font> <font color=DARKGREEN><i># Example 16</i></font> <font color=GRAY> 3:</font> <font color=GRAY> 4:</font> <font color=DARKGREEN><i># Uses ex13.sh to generate some output and give us an </i></font> <font color=GRAY> 5:</font> <font color=DARKGREEN><i># exit status to capture.</i></font> <font color=GRAY> 6:</font> <font color=GRAY> 7:</font> <font color=DARKGREEN><i># Get the exit status of ex13 into ex13stat. </i></font> <font color=GRAY> 8:</font> <font color=DARKGREEN><i># stderr of ex13 is processed by the pipe, stdout</i></font> <font color=GRAY> 9:</font> <font color=DARKGREEN><i># is left alone.</i></font> <font color=GRAY> 10:</font> <font color=GRAY> 11:</font> <font color=DARKGREEN><i># Save a copy of stdout</i></font> <font color=GRAY> 12:</font> <font color=DARKBLUE>exec </font>3>&1 <font color=GRAY> 13:</font> <font color=GRAY> 14:</font> <font color=DARKGREEN><i># Run a subshell, with 4 copied to 1 so we get it in stdout. </i></font> <font color=GRAY> 15:</font> <font color=DARKGREEN><i># Capture the output in backtics`</i></font> <font color=GRAY> 16:</font> <font color=DARKGREEN><i># ex13stat=`( ) 4>&1`</i></font> <font color=GRAY> 17:</font> <font color=GRAY> 18:</font> <font color=DARKGREEN><i># In the subshell, run another subshell to execute ex13, and </i></font> <font color=GRAY> 19:</font> <font color=DARKGREEN><i># echo the status code to 4</i></font> <font color=GRAY> 20:</font> <font color=DARKGREEN><i># (./ex13.sh; <font color=DARKBLUE>echo </font>$? >&4)</i></font> <font color=GRAY> 21:</font> <font color=GRAY> 22:</font> <font color=DARKGREEN><i># stdout from the inner subshell is directed to the original stdout (3)</i></font> <font color=GRAY> 23:</font> <font color=DARKGREEN><i># stderr is passed into the pipe for further processing.</i></font> <font color=GRAY> 24:</font> <font color=DARKGREEN><i># stdout from the pipe is redirected back to stderr</i></font> <font color=GRAY> 25:</font> <font color=GRAY> 26:</font> <font color=DARKGREEN><i># Close the extra descriptors before running the commands</i></font> <font color=GRAY> 27:</font> <font color=DARKBLUE>exec </font>3>&1 <font color=GRAY> 28:</font> <font color=BLACK>ex13stat=</font>`((./ex13.sh 2>&1 1>&3 3>&- 4>&- ; <font color=DARKBLUE>echo </font>$? >&4) | \\ <font color=GRAY> 29:</font> sed s/err/ERR/ 1>&2 3>&- 4>&- ) 4>&1` <font color=GRAY> 30:</font> <font color=GRAY> 31:</font> <font color=DARKBLUE>echo </font>Last command status=$? <font color=GRAY> 32:</font> <font color=DARKBLUE>echo </font>ex13stat=<font color=BLACK>$ex13stat</font> <font color=GRAY> 33:</font> </font></SMALL></BLOCKQUOTE> A practical application of this would be running a utility such as <a href=“http://www.FreeBSD.org/cgi/man.cgi?query=dd&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b>
dd</b></font></a> where the exit status is important to capture, but the error output is overly chatty and may need to be filtered before delivering to other parts of a script. </UL> <p align=RIGHT class=“newpage”><font size=-1>(26)</font></P> <font color=DARKBLUE>======Coprocesses and Background jobs</H1></font> Scripts can start any number of background jobs (any external command), which run in parallel with the parent script, and asynchronously. Processes which require no further interaction or synchronization (fire and forget) are easy. Interaction with background jobs is tricky. You can use signals, pipes, named pipes, or disk files for communication. <DL> <DT><font color=DARKRED><b>
<I>command</I> &</b></font> <DD>Start <I>command</I> as a background process. Control returns immediately to the shell. <DT><font color=DARKRED><b>
bgpid=$!</b></font> <DD>The special variable <font color=DARKRED><b>
$!</b></font> contains the process ID of the last background job that was started. You can save that and examine the process later (<font color=DARKRED><b>
ps -p $bgpid</b></font>) or send it a signal (<font color=DARKRED><b>
kill -HUP $bgpid</b></font>). </DL> ====ksh coprocesses==== <I>Coprocesses</I> are a way of starting a separate process which runs asychronously, but has stdin/stdout connected to the parent script via pipes. <DL> <DT><font color=DARKRED><b>
<I>command</I> |&</b></font> <DD>Start a coprocess with a 2-way pipe to it <DT><a href=“http://www.FreeBSD.org/cgi/man.cgi?query=read&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b>
read</b></font></a> <font color=DARKRED><b>
-p <I>var</I></b></font> <DD>Read from the pipe to the coprocess, instead of standard input <DT><a href=“http://www.FreeBSD.org/cgi/man.cgi?query=print&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b>
-p <I>args</I></b></font> <DD>Write to the pipe connected to the coprocess, instead of standard output </DL> Multiple coprocesses can be handled by moving the special file descriptors connected to the pipes onto standard input and output, and or to explicitly specified file descriptors. <DL> <DT><a href=“http://www.FreeBSD.org/cgi/man.cgi?query=exec&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b>
exec</b></font></a> <font color=DARKRED><b>
<&p</b></font> <DD>The input from the coprocess is moved to standard input <DT><a href=“http://www.FreeBSD.org/cgi/man.cgi?query=exec&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b>
exec</b></font></a> <font color=DARKRED><b>
>&p</b></font> <DD>The output from the coprocess is moved to standard output </DL> <P> <I>Example:</I> ex9 <A HREF=ex9.html TARGET=example>display</A>, <A HREF=ex9.txt TARGET=example>text</A><BR> A script wants to save a copy of all output in a file, but also wants a copy to the screen. This is equivalent to always running the script as<BR> <font color=DARKRED><b>
<I>script</I> | tee <I>outfile</I></b></font> <BLOCKQUOTE><SMALL>
<font color=DARKRED> <font color=GRAY> 1:</font> <font color=DARKGREEN><i>#!/bin/ksh</i></font> <font color=GRAY> 2:</font> <font color=GRAY> 3:</font> <font color=DARKGREEN><i># If we have not redirected standard output, save a copy of</i></font> <font color=GRAY> 4:</font> <font color=DARKGREEN><i># the output of this script into a file, but still send a</i></font> <font color=GRAY> 5:</font> <font color=DARKGREEN><i># copy to the screen.</i></font> <font color=GRAY> 6:</font> <font color=GRAY> 7:</font> <font color=DARKBLUE>if </font>[[ -t 1 ]] ; <font color=DARKBLUE>then</font> <font color=GRAY> 8:</font> <font color=DARKGREEN><i># Only do this if fd 1 (stdout) is still connected</i></font> <font color=GRAY> 9:</font> <font color=DARKGREEN><i># to a terminal</i></font> <font color=GRAY> 10:</font> <font color=GRAY> 11:</font> <font color=DARKGREEN><i># We want the standard output of the "tee" process</i></font> <font color=GRAY> 12:</font> <font color=DARKGREEN><i># to go explicitly to the screen (/dev/tty)</i></font> <font color=GRAY> 13:</font> <font color=DARKGREEN><i># and the second copy goes into a logfile named $0.out</i></font> <font color=GRAY> 14:</font> <font color=GRAY> 15:</font> tee <font color=BLACK>$0</font>.out >/dev/tty |& <font color=GRAY> 16:</font> <font color=GRAY> 17:</font> <font color=DARKGREEN><i># Our stdout all goes into this coprocess</i></font> <font color=GRAY> 18:</font> <font color=DARKBLUE>exec </font>1>&p <font color=GRAY> 19:</font> <font color=DARKBLUE>fi</font> <font color=GRAY> 20:</font> <font color=GRAY> 21:</font> <font color=DARKGREEN><i># Now generate some output</i></font> <font color=GRAY> 22:</font> <font color=DARKBLUE>print </font>"User activity snapshot on $(<font color=PURPLE>hostname) at $(date</font>)" <font color=GRAY> 23:</font> <font color=DARKBLUE>print</font> <font color=GRAY> 24:</font> who </font></SMALL></BLOCKQUOTE> <P> <I>Example:</I> ex10 <A HREF=ex10.html TARGET=example>display</A>, <A HREF=ex10.txt TARGET=example>text</A><BR> Start a coprocess to look up usernames in some database. It is faster to run a single process than to run a separate lookup for each user. <BLOCKQUOTE><SMALL>
<font color=DARKRED> <font color=GRAY> 1:</font> <font color=DARKGREEN><i>#!/bin/ksh</i></font> <font color=GRAY> 2:</font> <font color=DARKGREEN><i># This example uses a locally written tool for Dartmouth Name Directory lookups</i></font> <font color=GRAY> 3:</font> <font color=GRAY> 4:</font> <font color=DARKGREEN><i># Start the dndlookup program as a coprocess</i></font> <font color=GRAY> 5:</font> <font color=DARKGREEN><i># Tell it to output only the canonical full name, and to not print multiple matches</i></font> <font color=GRAY> 6:</font> dndlookup -fname -u |& <font color=GRAY> 7:</font> <font color=GRAY> 8:</font> <font color=DARKGREEN><i># move the input/output streams so we </i></font> <font color=GRAY> 9:</font> <font color=DARKGREEN><i># can use other coprocesses too</i></font> <font color=GRAY> 10:</font> <font color=DARKBLUE>exec </font>4>&p <font color=GRAY> 11:</font> <font color=DARKBLUE>exec </font>5<&p <font color=GRAY> 12:</font> <font color=GRAY> 13:</font> <font color=DARKBLUE>echo </font>"Name file contents:" <font color=GRAY> 14:</font> cat namefile <font color=GRAY> 15:</font> <font color=DARKBLUE>echo</font> <font color=GRAY> 16:</font> <font color=GRAY> 17:</font> <font color=DARKGREEN><i># read the names from a file "namefile"</i></font> <font color=GRAY> 18:</font> <font color=DARKBLUE>while </font>read uname; <font color=DARKBLUE>do</font> <font color=GRAY> 19:</font> <font color=DARKBLUE>print </font>-u4 <font color=BLACK>$uname</font> <font color=GRAY> 20:</font> <font color=DARKBLUE>read </font> -u5 dndname <font color=GRAY> 21:</font> <font color=DARKBLUE>case </font><font color=BLACK>$dndname</font> in <font color=GRAY> 22:</font> *many\\ matches*) <font color=GRAY> 23:</font> <font color=DARKGREEN><i># handle case where the name wasn't unique</i></font> <font color=GRAY> 24:</font> <font color=DARKBLUE>print </font>"Multiple matches to \\"<font color=BLACK>$uname</font>\\" in DND" <font color=GRAY> 25:</font> ;; <font color=GRAY> 26:</font> *no\\ match*) <font color=GRAY> 27:</font> <font color=DARKGREEN><i># handle case where the name wasn't found</i></font> <font color=GRAY> 28:</font> <font color=DARKBLUE>print </font>"No matches to \\"<font color=BLACK>$uname</font>\\" in DND" <font color=GRAY> 29:</font> ;; <font color=GRAY> 30:</font> *) <font color=GRAY> 31:</font> <font color=DARKGREEN><i># we seem to have a hit - process the</i></font> <font color=GRAY> 32:</font> <font color=DARKGREEN><i># canonical named retrieved from dndlookup</i></font> <font color=GRAY> 33:</font> <font color=DARKBLUE>print </font>"Unique DND match: full name for \\"<font color=BLACK>$uname</font>\\" is \\"<font color=BLACK>$dndname</font>\\"" <font color=GRAY> 34:</font> ;; <font color=GRAY> 35:</font> <font color=DARKBLUE>esac</font> <font color=GRAY> 36:</font> sleep 2 <font color=GRAY> 37:</font> <font color=DARKBLUE>done </font>< namefile <font color=GRAY> 38:</font> <font color=GRAY> 39:</font> <font color=DARKGREEN><i># We've read all the names, but the coprocess</i></font> <font color=GRAY> 40:</font> <font color=DARKGREEN><i># is still running. Close the pipe to tell it</i></font> <font color=GRAY> 41:</font> <font color=DARKGREEN><i># we have finished.</i></font> <font color=GRAY> 42:</font> <font color=DARKBLUE>exec </font>4>&- </font></SMALL></BLOCKQUOTE> <p align=RIGHT class=“newpage”><font size=-1>(27)</font></P> <font color=DARKBLUE>======Variable arrays</H1></font> Both ksh and bash implement arrays of variables, but in somewhat different ways. <p> ksh distinguishes between numerically indexed (small) arrays, and string indexed (associative) arrays. bash uses integers for all array indexing, but the integers need not be consecutive and unassigned array elements do not exist. Arrays must be declared before use, e,g. <font color=DARKRED><b>
typeset -A myarray</b></font> (ksh associative array), or <font color=DARKRED><b>
typeset -a myarray</b></font> (bash). <p> Array elements are set with the syntax: <font color=DARKRED><b>
myarray[[index]]=value</b></font> and referenced with the syntax <font color=DARKRED><b>
${myarray[[index]]}
</b></font>
<p>
This example shows use of an array indexed by IP addresses, as strings in ksh or as non-consecutive numbers in bash.
It also demonstrates use of getopt for options processing
<P>
<I>Example:</I> getauthlogs <A HREF=getauthlogs.html TARGET=example>display</A>, <A HREF=getauthlogs.txt TARGET=example>text</A><BR>
<BLOCKQUOTE><SMALL>
<font color=DARKRED>
<font color=GRAY> 1:</font> <font color=DARKGREEN><i>#!/bin/bash</i></font>
<font color=GRAY> 2:</font> <font color=DARKGREEN><i># $Header: $</i></font>
<font color=GRAY> 3:</font> <font color=DARKGREEN><i># First attempt at a consolidated auth log collection from kaserver</i></font>
<font color=GRAY> 4:</font> <font color=DARKGREEN><i># Timestamps in the raw files are NOT designed for easy sorting.</i></font>
<font color=GRAY> 5:</font> <font color=DARKGREEN><i>#</i></font>
<font color=GRAY> 6:</font> <font color=DARKGREEN><i># Options:</i></font>
<font color=GRAY> 7:</font> <font color=DARKGREEN><i># -i -- translate hex IP addresses to dotted-decimal (relatively quick)</i></font>
<font color=GRAY> 8:</font> <font color=DARKGREEN><i># -h -- translate hex IP addresses to DNS names (somewhat slower - DNS lookups)</i></font>
<font color=GRAY> 9:</font> <font color=DARKGREEN><i># -u user -- filter for the named user before translating addresses</i></font>
<font color=GRAY> 10:</font>
<font color=GRAY> 11:</font> hextodec()
<font color=GRAY> 12:</font> {
<font color=GRAY> 13:</font> <font color=DARKGREEN><i># convert the IP address in reverse-hex to dotted-decimal</i></font>
<font color=GRAY> 14:</font> <font color=DARKBLUE>echo </font><font color=DARKBLUE>$((0x<font color=BLACK>${1:6:2})).$((0x${1:4:2})).$((0x${1:2:2})).$((0x${1:0:2}</font>))</font>
<font color=GRAY> 15:</font> }
<font color=GRAY> 16:</font>
<font color=GRAY> 17:</font> hostlookup()
<font color=GRAY> 18:</font> {
<font color=GRAY> 19:</font> <font color=DARKGREEN><i># Convert a decimal IP to hostname - calls 'host' each time</i></font>
<font color=GRAY> 20:</font> <font color=BLACK>hostname=</font>$(<font color=PURPLE>host <font color=BLACK>$1</font></font>)
<font color=GRAY> 21:</font> <font color=DARKBLUE>case </font><font color=BLACK>$hostname</font> in
<font color=GRAY> 22:</font> *\\ not\\ found*)
<font color=GRAY> 23:</font> <font color=DARKGREEN><i># Just echo the address we tried to look up</i></font>
<font color=GRAY> 24:</font> <font color=DARKBLUE>echo </font>"<font color=BLACK>$1</font>"
<font color=GRAY> 25:</font> ;;
<font color=GRAY> 26:</font> *)
<font color=GRAY> 27:</font> <font color=DARKGREEN><i># The result is word 5. Lower-case it for consistency</i></font>
<font color=GRAY> 28:</font> set <font color=BLACK>$hostname</font>
<font color=GRAY> 29:</font> <font color=DARKBLUE>echo </font>"<font color=BLACK>$5</font>" | tr 'A-Z' 'a-z'
<font color=GRAY> 30:</font> ;;
<font color=GRAY> 31:</font> <font color=DARKBLUE>esac</font>
<font color=GRAY> 32:</font> }
<font color=GRAY> 33:</font>
<font color=GRAY> 34:</font> <font color=DARKGREEN><i># Options</i></font>
<font color=GRAY> 35:</font> <font color=BLACK>iptranslate=</font>0
<font color=GRAY> 36:</font> <font color=BLACK>gethostnames=</font>0
<font color=GRAY> 37:</font> <font color=BLACK>filter=</font>cat
<font color=GRAY> 38:</font> <font color=DARKBLUE>while </font>getopts ihu: o ; <font color=DARKBLUE>do</font>
<font color=GRAY> 39:</font> <font color=DARKBLUE>case </font><font color=BLACK>$o</font> in
<font color=GRAY> 40:</font> i) iptranslate=1 ;;
<font color=GRAY> 41:</font> h) gethostnames=1; iptranslate=1 ;;
<font color=GRAY> 42:</font> u) filter="grep <font color=BLACK>$OPTARG</font>" ;;
<font color=GRAY> 43:</font> <font color=DARKBLUE>esac</font>
<font color=GRAY> 44:</font> <font color=DARKBLUE>done</font>
<font color=GRAY> 45:</font> <font color=DARKBLUE>shift </font><font color=DARKBLUE>$((<font color=BLACK>$OPTIND</font>-1))</font>
<font color=GRAY> 46:</font>
<font color=GRAY> 47:</font> <font color=DARKGREEN><i># We could get the DB server names from 'fs checkservers', but it isn't obvious what is from our cell. We</i></font>
<font color=GRAY> 48:</font> <font color=DARKGREEN><i># could also grep CellServDB. I cop out and hard code one known DB server and get the others from it.</i></font>
<font color=GRAY> 49:</font> <font color=BLACK>masterserver=</font>halley.dartmouth.edu
<font color=GRAY> 50:</font> <font color=BLACK>serverlist=</font>$(<font color=PURPLE>bos listhosts -server <font color=BLACK>$masterserver</font>| grep 'Host .* is ' | awk '{print <font color=BLACK>$4</font>}'</font>)
<font color=GRAY> 51:</font>
<font color=GRAY> 52:</font> <font color=DARKGREEN><i># If we want to filter usernames, it is more efficient to do it inline, before sorting, translation and hostname lookups</i></font>
<font color=GRAY> 53:</font>
<font color=GRAY> 54:</font> <font color=DARKGREEN><i># Array to hold IP address/name conversions (associative array, ksh only)</i></font>
<font color=GRAY> 55:</font> <font color=DARKGREEN><i># ksh - use -A for associative array. bash - use -a and numeric array</i></font>
<font color=GRAY> 56:</font> <font color=DARKBLUE>typeset </font>-a hostnames
<font color=GRAY> 57:</font>
<font color=GRAY> 58:</font> (
<font color=GRAY> 59:</font> <font color=DARKBLUE>for </font>dbserver in <font color=BLACK>$serverlist</font>; <font color=DARKBLUE>do</font>
<font color=GRAY> 60:</font> bos getlog -server <font color=BLACK>$dbserver</font> -file /usr/afs/logs/AuthLog
<font color=GRAY> 61:</font> <font color=DARKBLUE>done</font>
<font color=GRAY> 62:</font> ) | grep -v 'Fetching log file' | <font color=BLACK>$filter</font> | sed -e 's/^... //' -e 's/ \\([[1-9]]\\) / 0\\1 /' | sort --month-sort | \\
<font color=GRAY> 63:</font> sed '-e s/ \\([[0-9a-f]][[0-9a-f]][[0-9a-f]][[0-9a-f]][[0-9a-f]][[0-9a-f]][[0-9a-f]]\\)$/ 0\\1/' |
<font color=GRAY> 64:</font> <font color=DARKBLUE>while </font>read line; <font color=DARKBLUE>do</font>
<font color=GRAY> 65:</font> <font color=DARKBLUE>if </font>[[ <font color=BLACK>$iptranslate</font> == 1 ]] ; <font color=DARKBLUE>then</font>
<font color=GRAY> 66:</font> <font color=DARKGREEN><i># Ugly!</i></font>
<font color=GRAY> 67:</font> <font color=DARKGREEN><i># Sometimes we get a 7-digit hex code in the log - the kaserver apparently drops leading zeros.</i></font>
<font color=GRAY> 68:</font> <font color=DARKGREEN><i># The second 'sed' in the pipe catches these are fixes them.</i></font>
<font color=GRAY> 69:</font> <font color=DARKBLUE>case </font><font color=BLACK>$line</font> in
<font color=GRAY> 70:</font> *\\ from\\ [[0-9a-f]][[0-9a-f]][[0-9a-f]][[0-9a-f]][[0-9a-f]][[0-9a-f]][[0-9a-f]][[0-9a-f]])
<font color=GRAY> 71:</font> <font color=DARKGREEN><i># translate the reverse-hex address</i></font>
<font color=GRAY> 72:</font> <font color=BLACK>iphex=</font><font color=BLACK>${line##* from }</font>
<font color=GRAY> 73:</font> <font color=DARKGREEN><i># bash version - index by numeric value only, but can be sparse array -- use the raw IP </i></font>
<font color=GRAY> 74:</font> <font color=BLACK>ipdec=</font><font color=DARKBLUE>$((0x<font color=BLACK>$iphex</font>))</font>
<font color=GRAY> 75:</font> <font color=BLACK>frontpart=</font><font color=BLACK>${line% from *}</font>
<font color=GRAY> 76:</font> <font color=DARKBLUE>if </font>[[ <font color=BLACK>$gethostnames</font> == 1 ]]; <font color=DARKBLUE>then</font>
<font color=GRAY> 77:</font> <font color=DARKGREEN><i># ksh - index on hex value as a string (iphex)</i></font>
<font color=GRAY> 78:</font> <font color=DARKGREEN><i># bash - index on numeric value (ipdec)</i></font>
<font color=GRAY> 79:</font> <font color=BLACK>index=</font><font color=BLACK>$ipdec</font>
<font color=GRAY> 80:</font> <font color=DARKBLUE>if </font>[[ -z "<font color=BLACK>${hostnames[[<font color=BLACK>$index</font>]]}</font>" ]]; <font color=DARKBLUE>then</font>
<font color=GRAY> 81:</font> hostnames[[<font color=BLACK>$index</font>]]="$(<font color=PURPLE>hostlookup $(hextodec <font color=BLACK>$iphex</font>)</font>)"
<font color=GRAY> 82:</font> <font color=DARKBLUE>fi</font>
<font color=GRAY> 83:</font> <font color=DARKBLUE>echo </font>"<font color=BLACK>$frontpart</font> from <font color=BLACK>${hostnames[[<font color=BLACK>$index</font>]]}</font>"
<font color=GRAY> 84:</font> <font color=DARKBLUE>else</font>
<font color=GRAY> 85:</font> <font color=DARKBLUE>echo </font>"<font color=BLACK>$frontpart</font> from $(<font color=PURPLE>hextodec <font color=BLACK>$iphex</font></font>)"
<font color=GRAY> 86:</font> <font color=DARKBLUE>fi</font>
<font color=GRAY> 87:</font> ;;
<font color=GRAY> 88:</font> *)
<font color=GRAY> 89:</font> <font color=DARKBLUE>echo </font>"<font color=BLACK>$line</font>"
<font color=GRAY> 90:</font> ;;
<font color=GRAY> 91:</font> <font color=DARKBLUE>esac</font>
<font color=GRAY> 92:</font> <font color=DARKBLUE>else</font>
<font color=GRAY> 93:</font> <font color=DARKGREEN><i># No ip translation, just echo the whole line</i></font>
<font color=GRAY> 94:</font> <font color=DARKBLUE>echo </font>"<font color=BLACK>$line</font>"
<font color=GRAY> 95:</font> <font color=DARKBLUE>fi </font>
<font color=GRAY> 96:</font> <font color=DARKBLUE>done</font>
<font color=GRAY> 97:</font>
</font>
</SMALL></BLOCKQUOTE>
<p align=RIGHT class=“newpage”><font size=-1>(28)</font></P>
<font color=DARKBLUE>======Delivering and Trapping Signals</H1></font>
Unix signals (software interrupts) can be sent as asynchronous events to shell scripts, just
as they can to any other program. The default behaviour is to ignore some signals and immediately
exit on others. Scripts may detect signals and divert control to a handler function or external
program. This is often used to perform clean-up actions before exiting, or restart certain
procedures. Execution resumes where it left off, if the signal handler returns. Signal traps
must be set separately inside of shell functions. Signals can be sent to a process with
<a href=“http://www.FreeBSD.org/cgi/man.cgi?query=kill&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b>kill</b></font></a>. <DL> <DT><a href=“http://www.FreeBSD.org/cgi/man.cgi?query=trap&apropos=0&sektion=1&manpath=Red+Hat+Linux%2Fi386+9&format=html” target=manpage><font color=DARKRED><b>
trap</b></font></a> <font color=DARKRED><b>
<I>handler</I> <I>sig</I> ...</b></font> <DD><I>handler</I> is a command to be read (evaluated first) and executed on receipt of the specified <I>sig</I>s. Signals can be specified by name or number (see kill(1