Rc The Plan 9 Shell
- Tom Duff
td@plan9.bell-labs.com
ABSTRACT
Rc is a command interpreter for Plan 9 that provides similar facilities to UNIX's Bourne shell, with some small additions and less idiosyncratic syntax. This paper uses numerous examples to describe rc's features, and contrasts rc with the Bourne shell, a model that many readers will be familiar with.
1 Introduction
Rc is similar in spirit but different in detail from UNIX's Bourne shell. This paper describes rc's principal features with many small examples and a few larger ones. It assumes familiarity with the Bourne shell.
2 Simple commands
For the simplest uses rc has syntax familiar to Bourne-shell users. All of the following behave as expected:
date
cat /lib/news/build
who >user.names
who >>user.names
wc <file
echo [a-f]*.c
who | wc
who; date
vc *.c &
mk && v.out /*/bin/fb/*
rm -r junk || echo rm failed!
3 Quotation
An argument that contains a space or one of rc's other syntax characters must be enclosed in apostrophes ('):
rm 'odd file name'
echo 'How''s your father?'
4 Patterns
An unquoted argument that contains any of the characters * ? [ is a pattern to be matched against file names. A * character matches any sequence of characters, ? matches any single character, and [class] matches any character in the class, unless the first character of class is ~, in which case the class is complemented. The class may also contain pairs of characters separated by -, standing for all characters lexically between the two. The character / must appear explicitly in a pattern, as must the path name components . and ... A pattern is replaced by a list of arguments, one for each path name matched, except that a pattern matching no names is not replaced by the empty list; rather it stands for itself.
5 Variables
UNIX's Bourne shell offers string-valued variables. Rc provides variables whose values are lists of arguments that is, arrays of strings. This is the principal difference between rc and traditional UNIX command interpreters. Variables may be given values by typing, for example:
path=(. /bin)
user=td
font=/lib/font/bit/pelm/ascii.9.font
The value of a variable can be substituted into a command by preceding its name with a $, like this:
echo $path
echo . /bin
echo $path(2)
echo $path(2 1 2)
echo /bin
echo /bin . /bin
The number of strings in a variable can be determined by the $# operator. For example,
echo $#path
The following two assignments are subtly different:
empty=()
null=''
Although these may seem like more or less the same thing (in Bourne's shell, they are indistinguishable), they behave differently in almost all circumstances. Among other things
echo $#empty
echo $#null
All variables that have never been set have the value ().
Occasionally, it is convenient to treat a variable's value as a single string. The elements of a string are concatenated into a single string, with spaces between the elements, by the $" operator. Thus, if we set
list=(How now brown cow)
string=$"list
echo $list
echo $string
How now brown cow
echo $#list $#string
4 1
6 Arguments
When rc is reading its input from a file, the file has access to the arguments supplied on rc's command line. The variable $* initially has the list of arguments assigned to it. The names $1, $2, etc. are synonyms for $*(1), $*(2), etc. In addition, $0 is the name of the file from which rc's input is being read.
7 Concatenation
Rc has a string concatenation operator, the caret ^, to build arguments out of pieces.
echo hully^gully
echo hullygully
vc $i^.c
vl -o $1 $i^.v
Concatenation distributes over lists. The following
echo (a b c)^(1 2 3)
src=(main subr io)
cc $src^.c
echo a1 b2 c3
cc main.c subr.c io.c
8 Free carets
User demand has dictated that rc insert carets in certain places, to make the syntax look more like the Bourne shell. For example, this:
cc -$flags $stems.c
cc -^$flags $stems^.c
9 Command substitution
It is often useful to build an argument list from the output of a command. Rc allows a command, enclosed in braces and preceded by a left quote, `{...}, anywhere that an argument is required. The command is executed and its standard output captured. The characters stored in the variable ifs are used to split the output into arguments. For example,
cat `{ls -tr|sed 10q}
10 Pipeline branching
The normal pipeline notation is general enough for almost all cases. Very occasionally it is useful to have pipelines that are not linear. Pipeline topologies more general than trees can require arbitrarily large pipe buffers, or worse, can cause deadlock. Rc has syntax for some kinds of non-linear but treelike pipelines. For example,
cmp <{old} <{new}
11 Exit status
When a command exits it returns status to the program that executed it. On Plan 9 status is a character string describing an error condition. On normal termination it is empty.
Rc captures command exit status in the variable $status. For a simple command the value of $status is just as described above. For a pipeline $status is set to the concatenation of the statuses of the pipeline components with | characters for separators.
Rc has a several kinds of control flow, many of them conditioned by the status returned from previously executed commands. Any $status containing only 0's and |'s has boolean value true. Any other status is false.
12 Command grouping
A sequence of commands enclosed in {} may be used anywhere a command is required. For example:
{sleep 3600;echo 'Time''s up!'}&
sleep 3600;echo 'Time''s up!'&
13 Control flow for
A command may be executed once for each member of a list by typing, for example:
for(i in printf scanf putchar) look $i /usr/td/lib/dw.dat
for(name in list) command
for(name) command
14 Conditional execution if
Rc also provides a general if-statement. For example:
for(i in *.c) if(cpp $i >/tmp/$i) vc /tmp/$i
for(i){
if(test -f /tmp/$i) echo $i already in /tmp
if not cp $i /tmp
}
15 Control flow while
Rc's while statement looks like this:
while(newer subr.v subr.c) sleep 5
If the controlling command is empty, the loop will not terminate. Thus,
while() echo y
16 Control flow switch
Rc provides a switch statement to do pattern-matching on arbitrary strings. Its general form is
switch(word){
case pattern ...
commands
case pattern ...
commands
...
}
If any pattern matches, the commands following that case up to the next case (or the end of the switch) are executed, and execution of the switch is complete. For example,
switch($#*){
case 1
cat >>$1
case 2
cat >>$2 <$1
case *
echo 'Usage: append [from] to'
}
The built-in ~ command also matches patterns, and is often more concise than a switch. Its arguments are a string and a list of patterns. It sets $status to true if and only if any of the patterns matches the string. The following example processes option arguments for the man(1) command:
opt=()
while(~ $1 -* [1-9] 10){
switch($1){
case [1-9] 10
sec=$1 secn=$1
case -f
c=f s=f
case -[qwnt]
cmd=$1
case -T*
T=$1
case -*
opt=($opt $1)
}
shift
}
17 Functions
Functions may be defined by typing
fn name { commands }
fn g {
grep $1 *.[hcyl]
}
Function definitions are deleted by writing
fn name
18 Command execution
Rc does one of several things to execute a simple command. If the command name is the name of a function defined using fn, the function is executed. Otherwise, if it is the name of a built-in command, the built-in is executed directly by rc. Otherwise, directories mentioned in the variable $path are searched until an executable file is found. Extensive use of the $path variable is discouraged in Plan 9. Instead, use the default (. /bin) and bind what you need into /bin.
19 Built-in commands
Several commands are executed internally by rc because they are difficult to implement otherwise.
- . [-i] file ...
- file. $* is set for the duration to the reminder of the argument list following file. $path is used to search for file. Option -i indicates interactive input a prompt (found in $prompt) is printed before each command is read.
- builtin command ...
- command as usual except that any function named command is ignored. For example,
fn cd{
builtin cd $* && pwd
}
- dir. The default argument is $home. $cdpath is a list of places in which to search for dir.
- rc, and executed. For example,
x='$y'
y=Doody
eval echo Howdy, $x
Howdy, Doody
echo Howdy, $y
- replaces itself with the given command. This is like a goto rc does not wait for the command to exit, and does not return to read any more commands.
- exits immediately with the given status. If none is given, the current value of $status is used.
flag f +
flag
f -
flag
f
if(flag x) flag v +
- command as usual except that any function named command is ignored. For example,
-
rfork
system entry to put
rc
into a new process group with the following attributes:
Section fork(2) of the Programmer's Manual describes these attributes in more detail. - shift [n]
- n (default 1) elements of $*.
- wait [pid]
- pid to exit. If no pid is given, all outstanding processes are waited for.
- name in a form suitable for input to rc. The output is an assignment to a variable, the definition of a function, a call to builtin for a built-in command, or the path name of a binary program. For example,
whatis path g cd who
path=(. /bin)
fn g {gre -e $1 *.[hycl]}
builtin cd
/bin/who
- ~ subject pattern ...
- subject is matched against each pattern in turn. On a match, $status is set to true. Otherwise, it is set to 'no match'. Patterns are the same as for filename matching. The patterns are not subjected to filename replacement before the ~ command is executed, so they need not be enclosed in quotation marks, unless of course, a literal match for * [ or ? is required. For example
~ $1 ?
~ $1 '?'
20 Advanced I/O Redirection
Rc allows redirection of file descriptors other than 0 and 1 (standard input and output) by specifying the file descriptor in square brackets [ ] after the < or >. For example,
vc junk.c >[2]junk.diag
File descriptors may be replaced by a copy, in the sense of dup(2), of an already-open file by typing, for example
vc junk.c >[2=1]
vc junk.c >junk.out >[2=1]
vc junk.c >[2=1] >junk.out
File descriptors may be closed by using the duplication notation with an empty right-hand side. For example,
vc junk.c >[2=]
Arbitrary file descriptors may be sent through a pipe by typing, for example,
vc junk.c |[2] grep -v '^$'
Occasionally you may wish to connect the input side of a pipe to some file descriptor other than zero. The notation
cmd1 |[5=19] cmd2
21 Here documents
Rc procedures may include data, called ``here documents'', to be provided as input to commands, as in this version of the tel command
for(i) grep $i <<!
...
tor 2T-402 2912
kevin 2C-514 2842
bill 2C-562 7214
...
!
Rc does variable substitution in here documents. The following command:
ed $3 <<EOF
g/$1/s//$2/g
w
EOF
Variable substitution can be entirely suppressed by enclosing the EOF marker following << in quotation marks, as in <<'EOF'.
Here documents may be provided on file descriptors other than 0 by typing, for example,
cmd <<[4]End
...
End
If a here document appears within a compound block, the contents of the document must be after the whole block:
for(i in $*){
mail $i <<EOF
}
words to live by
EOF
22 Catching Notes
Rc scripts normally terminate when an interrupt is received from the terminal. A function with the name of a UNIX signal, in lower case, is defined in the usual way, but called when rc receives the corresponding note. The notify(2) section of the Programmer's Manual discusses notes in some detail. Notes of interest are:
- sighup
- Plan 9 sends this when the terminal has disconnected from rc.
- sigint
- the interrupt character (ASCII DEL) is typed on the terminal.
- sigterm
- kill(1).
- sigexit
- rc is about to exit.
As an example,
fn sigint{
rm /tmp/junk
exit
}
Notes will be ignored if the note routine is set to {}. Signals revert to their default behavior when their handlers' definitions are deleted.
23 Environment
The environment is a list of name-value pairs made available to executing binaries. On Plan 9, the environment is stored in a file system named #e, normally mounted on /env. The value of each variable is stored in a separate file, with components terminated by zero bytes. (The file system is maintained entirely in core, so no disk or network access is involved.) The contents of /env are shared on a per-process group basis - when a new process group is created it effectively attaches /env to a new file system initialized with a copy of the old one. A consequence of this organization is that commands can change environment entries and see the changes reflected in rc.
Functions also appear in the environment, named by prefixing fn# to their names, like /env/fn#roff.
24 Local Variables
It is often useful to set a variable for the duration of a single command. An assignment followed by a command has this effect. For example
a=global
a=local echo $a
echo $a
local
global
f=/fairly/long/file/name {
{ wc $f; spell $f; diff $f.old $f } |
pr -h 'Facts about '$f | lp -dfn
}
25 Examples cd, pwd
Here is a pair of functions that provide enhanced versions of the standard cd and pwd commands. (Thanks to Rob Pike for these.)
ps1='% ' # default prompt
tab=' ' # a tab character
fn cd{
builtin cd $1 &&
switch($#*){
case 0
dir=$home
prompt=($ps1 $tab)
case *
switch($1)
case /*
dir=$1
prompt=(`{basename `{pwd}}^$ps1 $tab)
case */* ..*
dir=()
prompt=(`{basename `{pwd}}^$ps1 $tab)
case *
dir=()
prompt=($1^$ps1 $tab)
}
}
}
fn pwd{
if(~ $#dir 0)
dir=`{/bin/pwd}
echo $dir
}
Function cd calls the cd built-in, and checks that it was successful. If so, it sets $dir and $prompt. The prompt will include the last component of the current directory (except in the home directory, where it will be null), and $dir will be reset either to the correct value or to (), so that the pwd function will work correctly.
26 Examples man
The man command prints pages of the Programmer's Manual. It is called, for example, as
man 2 sinh
man rc
man -t cat
cd /sys/man || {
echo $0: No manual! >[1=2]
exit 1
}
NT=n # default nroff
s='*' # section, default try all
for(i) switch($i){
case -t
NT=t
case -n
NT=n
case -*
echo Usage: $0 '[-nt] [section] page ...' >[1=2]
exit 1
case [1-9] 10
s=$i
case *
eval 'pages='$s/$i
for(page in $pages){
if(test -f $page)
$NT^roff -man $page
if not
echo $0: $i not found >[1=2]
}
}
27 Examples holmdel
The following rc script plays the deceptively simple game holmdel, in which the players alternately name Bell Labs locations, the winner being the first to mention Holmdel.
t=/tmp/holmdel$pid
fn read{
$1=`{awk '{print;exit}'}
}
ifs='
' # just a newline
fn sigexit sigint sigquit sighup{
rm -f $t
exit
}
cat <<'!' >$t
Allentown
Atlanta
Cedar Crest
Chester
Columbus
Elmhurst
Fullerton
Holmdel
Indian Hill
Merrimack Valley
Morristown
Neptune
Piscataway
Reading
Short Hills
South Plainfield
Summit
Whippany
West Long Branch
!
while(){
lab=`{fortune $t}
echo $lab
if(~ $lab Holmdel){
echo You lose.
exit
}
while(read lab; ! grep -i -s $lab $t) echo No such location.
if(~ $lab [hH]olmdel){
echo You win.
exit
}
}
This script is worth describing in detail (rather, it would be if it weren't so silly.)
Variable $t is an abbreviation for the name of a temporary file. Including $pid, initialized by rc to its process-id, in the names of temporary files insures that their names won't collide, in case more than one instance of the script is running at a time.
Function read's argument is the name of a variable into which a line gathered from standard input is read. $ifs is set to just a newline. Thus read's input is not split apart at spaces, but the terminating newline is deleted.
A handler is set to catch sigint, sigquit, and sighup, and the artificial sigexit signal. It just removes the temporary file and exits.
The temporary file is initialized from a here document containing a list of Bell Labs locations, and the main loop starts.
First, the program guesses a location (in $lab) using the fortune program to pick a random line from the location list. It prints the location, and if it guessed Holmdel, prints a message and exits.
Then it uses the read function to get lines from standard input and validity-check them until it gets a legal name. Note that the condition part of a while can be a compound command. Only the exit status of the last command in the sequence is checked.
Again, if the result is Holmdel, it prints a message and exits. Otherwise it goes back to the top of the loop.
28 Design Principles
Rc draws heavily from Steve Bourne's /bin/sh. Any successor of the Bourne shell is bound to suffer in comparison. I have tried to fix its best-acknowledged shortcomings and to simplify things wherever possible, usually by omitting inessential features. Only when irresistibly tempted have I introduced novel ideas. Obviously I have tinkered extensively with Bourne's syntax.
The most important principle in rc's design is that it's not a macro processor. Input is never scanned more than once by the lexical and syntactic analysis code (except, of course, by the eval command, whose raison d'être is to break the rule).
Bourne shell scripts can often be made to run wild by passing them arguments containing spaces. These will be split into multiple arguments using IFS, often at inopportune times. In rc, values of variables, including command line arguments, are not re-read when substituted into a command. Arguments have presumably been scanned in the parent process, and ought not to be re-read.
Why does Bourne re-scan commands after variable substitution? He needs to be able to store lists of arguments in variables whose values are character strings. If we eliminate re-scanning, we must change the type of variables, so that they can explicitly carry lists of strings.
This introduces some conceptual complications. We need a notation for lists of words. There are two different kinds of concatenation, for strings $a^$b, and lists ($a $b). The difference between () and '' is confusing to novices, although the distinction is arguably sensible a null argument is not the same as no argument.
Bourne also rescans input when doing command substitution. This is because the text enclosed in back-quotes is not a string, but a command. Properly, it ought to be parsed when the enclosing command is, but this makes it difficult to handle nested command substitutions, like this:
size=`wc -l \`ls -t|sed 1q\``
size=`{wc -l `{ls -t|sed 1q}}
For similar reasons rc defines signal handlers as though they were functions, instead of associating a string with each signal, as Bourne does, with the attendant possibility of getting a syntax error message in response to typing the interrupt character. Since rc parses input when typed, it reports errors when you make them.
For all this trouble, we gain substantial semantic simplifications. There is no need for the distinction between $* and $@. There is no need for four types of quotation, nor the extremely complicated rules that govern them. In rc you use quotation marks when you want a syntax character to appear in an argument, or an argument that is the empty string, and at no other time. IFS is no longer used, except in the one case where it was indispensable: converting command output into argument lists during command substitution.
This also avoids an important UNIX security hole. In UNIX, the system and popen functions call /bin/sh to execute a command. It is impossible to use either of these routines with any assurance that the specified command will be executed, even if the caller of system or popen specifies a full path name for the command. This can be devastating if it occurs in a set-userid program. The problem is that IFS is used to split the command into words, so an attacker can just set IFS=/ in his environment and leave a Trojan horse named usr or bin in the current working directory before running the privileged program. Rc fixes this by never rescanning input for any reason.
Most of the other differences between rc and the Bourne shell are not so serious. I eliminated Bourne's peculiar forms of variable substitution, like
echo ${a=b} ${c-d} ${e?error}
Where Bourne's syntax draws from Algol 68, rc's is based on C or Awk. This is harder to defend. I believe that, for example
if(test -f junk) rm junk
if test -f junk; then rm junk; fi
The one bit of large-scale syntax that Bourne unquestionably does better than rc is the if statement with else clause. Rc's if has no terminating fi-like bracket. As a result, the parser cannot tell whether or not to expect an else clause without looking ahead in its input. The problem is that after reading, for example
if(test -f junk) echo junk found
Rc's admittedly feeble solution is to declare that the else clause is a separate statement, with the semantic proviso that it must immediately follow an if, and to call it if not rather than else, as a reminder that something odd is going on. The only noticeable consequence of this is that the braces are required in the construction
for(i){
if(test -f $i) echo $i found
if not echo $i not found
}
It is remarkable that in the four most recent editions of the UNIX system programmer's manual the Bourne shell grammar described in the manual page does not admit the command who|wc. This is surely an oversight, but it suggests something darker: nobody really knows what the Bourne shell's grammar is. Even examination of the source code is little help. The parser is implemented by recursive descent, but the routines corresponding to the syntactic categories all have a flag argument that subtly changes their operation depending on the context. Rc's parser is implemented using yacc, so I can say precisely what the grammar is.
29 Acknowledgements
Rob Pike, Howard Trickey and other Plan 9 users have been insistent, incessant sources of good ideas and criticism. Some examples in this document are plagiarized from [Bourne], as are most of rc's good features.
30 Reference
S. R. Bourne, UNIX Time-Sharing System: The UNIX Shell, Bell System Technical Journal, Volume 57 number 6, July-August 1978
Copyright © 1995 Lucent Technologies. All rights reserved.