|
Unit [01] Introduction to Shell Programming: 
- Identify the three types of shells found in UNIX and list the major
features of each,
- Bourne shell is the standard UNIX shell (boot scripts are written in this shell),
- C Shell was developed by the University of California and has C
language types syntax,
- Korn shell is a superset of the Bourne shell.
- State how shells establish your working environment, when you login the shell program specified in the
/etc/passwd file. The shell establishes the users environment. Then your startup shell reads your .profile
file to locate your home directory, and sets your individual environment settings. The Korn shell reads a file named .kshrc upon each invocation of a new shell. This file contains environmental
information that you wish to include in all sub-shells, such as aliases.
- Startup a sub shell, the shell you are currently running is called your current shell. You can start up
another copy of the shell, the new shell is now your current shell, the old shell is now your parent shell.
- Create and execute the shell program, there are two methods of executing shell programs. The first
method invokes the shell program using the sh command and is used during development, the second method involves changing the file mode to allow execution, then invoke the shell program Example: cmod +x status
- Insert comments into a shell program, the "#
" character is used to indicate a comment within the shell program, comments are used so others who maintain your code can determine your intentions
- Debugging shell programs, the first technique uses the
sh command, causing each line to be sent to standard output. Use print and echo to half split, or conversely use the # character to comment out lines of code sh -x status 2> status.out
- Set prints a list of variables set for the current shell, the
set - x +x command is used to turn on and turnoff the debugging program throughout a shell program.
- Export prints a list of exported variables,
- Use the dot "." command to execute a shell program, a shell is created to execute a shell program.
However, the current shell can be used to execute a program directly through the use of the dot "." command. Usage . .profile
- Procedure: execute a set command and redirect the output to a filename
vset, execute the export command and redirect the output tool filename vexport, use the common command to determine which variables are in vset and not in vexport
- What is a shell program: a file which contains UNIX commands and built-in shell programming
constructs, keywords, variables, and/or functions. The commands contained in the file are executed by entering in the name of the file as a command.
- Shell programs accomplish; minimize repetitive commands, build utility programs, allow the modifying of
commands.
- Shell programs, shell scripts, and command files are names used to describe this blessed event.
- Functions are stored in computer memory (fast access), promote modular programming, and offer reuse.
Examples: menu commands, user input (reads), validating, searching, macro command strings.
- Use the exec command to execute a shell program and standard UNIX commands, when a program is
executed using the exec command the program overlays the current shell. Upon exit, the program will return to the previous shells parent.
- Writing functions within a shell program, a function is like a shell program within a shell program.
function_name() { shell commands } to invoke function use the variable function_name to call the function. where does function_name both define a function and becoming a variable?
- Use the auto load feature to automatically load functions, this allows the shell to load the function at run time (only if function called).
- FPATH variable contains a list of directories were function files are stored.
- Define and use aliases, aliases allow you to redefine the commands. Alias'n is performed when shell
programs are read. Example: alias word=cmd, alias -x word=cmd aliase are exported to children.
- Use tracked aliases. The shell can execute commands much faster if it has a full pathname. Aliases are
used to provided abbreviated commands to their pathnames. The hash option of the set command indicate that the value of aliases are to be automatically set to the full pathname of the corresponding
command. Several track aliases are compiled into the shell. Example: set -h hash displays a list of tracked aliases
[ top ]
Unit [02] Shell Parameters and Variables
- Parameter types: named parameters (often referred to as variables),
positional parameters, special parameters.
- Named parameters (often referred to as variables) consist of
(variable-name assigned-value pairs). The name must begin with and upper or lowercase letter, digits or an "_" underscore. If you include special characters within the variable you must enclose the
value with appropriate quotation. remember single quotation prevents interpretation of next character,
- Positional parameters names are numbers. For example, $2 is a positional parameter named 2.
- Special parameters have one the following characters in their name
* @ $ ? - # ! For example: $? is a special parameter.
- The contents of a variable are addressed by preceding the variable name with a "$" character.
- Remember, when creating variable values you may use any character; therefore, special shell
characters are fair game and must be" escaped" for proper interpretation.
- Single quotations prevent the shell from interpreting all special characters,
- Double quotations prevent the shell from interpreting all special characters except
$, \ and '.
- Forward-slash \ prevents the shell from expanding the character immediately following it. It is
strongly recommended that all arguments to the print command always being closed in double quotes.
- Assign values to shell parameters example: last=name, myname="Pascal Marconi", salary='$199.99'
- Print shell parameters example: print $myname outputs Pascal Marconi, fname=part, print $fname, print
${fname}xxx, print [ -n ] [ args ] and there are many special command characters to choose from. Happy-happy-joy-joy, print "hello, $LOGNAME"
- Read input from the standard input and assign it to a shell variable, the read command uses the read
command to capture input from the keyboard or a file and assigns it to a variable. For example, print -n "what is your name" read myname
- State the types of shell variables, login
shell variables; MAIL, MAILPATH, MAILCECK, LOGNAME, HISTFILE, EDITOR, more...
- Typeset command sets attributes and values for variables, there is a long list of possible attributes.
Example, typeset -i ssan set ssan as integer. typeset -x export variable to sub shells.
- Variables are local to the shell process in which they were created. This shell built-in command export or typeset -x
commands can be used to make a variable available to sub processes. Remember to export variables.
- Define parameter arrays. The shell supports the limited one-dimensional array facility. Arrays need not
be declared and any reference to an unnamed variable with invalid subscript is legal and an array element will be created if necessary. Referencing an array without subscript is equivalent to reference in the first
element. Example; variable[n]=value prints ${variable[n]}
- the Special shell parameters
- $? exit status of last command
- $# number of parameters passed to shell
- $$ process id PID of shell, more...
- Conditional parameter substitution, a parameter can be set or not set. If it is set it can be null or
non-null.
- Manipulate positional parameters. When you enter a command with arguments, each word on the
command line is given a numerical value which indicates its position on the command line. The shell stores each command line argument "including the name of the command" in special parameters known as
position parameters. You can obtain the value of any argument passed to your shell program by reference in the positional parameter associated with that argument. $0 command.
- Use command substitution: The output of one command can be used as input arguments to another
command through the shell command substitution mechanism. Example: banner $(date) here $(pwd)
- Set positional parameters. A positional parameters cannot be set or changed directly, because positional
parameters are special variables and are set by the shell. However, you can use the set command to set positional parameters. If you need to set positional parameters with the value beginning with "-", then
you must use to the -- special flag with the set command. Example: set John Doe is my name ; print $4 $5 $3 $1 $2 set -- $(date) print "date: $1" print "month: $2" etc. Usually, the sector Baird is used to
set positional parameters from the output of another command through command substitution.
- Use positional parameters with functions: if you use this set command to set positional parameters
within a function, then only the functions positional parameters will be set. The positional parameters for the shell will not be affected by the set command within the function. Therefore, there will be a
difference between the functions positional parameters and the shells positional parameters.
- Use tilde substitution: The "~" can be used for shorthand notation to represent a user's own directory,
current directory, or previous directory.
[ top ]
Unit [03] Shell Input/Output [ CODE EXAMPLES ] 
- Enter simple, sequential, and group commands and redirect the output of each.
- Simple commands single argument. The shell accepts a command
and its arguments. Examines the command line, expands any metacharacters, sets I/O redirection, sets up pipes, and it executes the command.
- Sequential commands consist of two or more simple commands separated by semicolons. For
example ls ; date
- Group commands or used when the output of all commands in the sequential command line are to
be redirected to a file, or sent to the input of another command via a pipe. For example (ls ; date) > info
- Open additional file descriptors for input and output using the exec command. Shells automatically open three files
stdin, stdout, stder. Additional file descriptors can be assigned by use of the exec command. Once a file is opened and assigned a file descriptor it can be read and written by using the file
descriptors number. Example exec 3< names exec 4> now
- Use the here-document for input redirection. A here-document provides another form of input redirection. The notation <<
is used to indicate the use of here-document.
- Use the tput command for screen oriented output. The UNIX tput command provide external independent
screen control through the use of a terminal information database.
[ top ]
Unit [04] Conditional Testing [ CODE EXAMPLES ]
- The shell allows us to test for certain values and conditions, and based on results of the test, perform one action or another. This is call
conditional testing.
- Identify a true and false exit status from a command. All commands on
UNIX return a status code to the shell with the exit status which tells the shell if the command was successful are not. A non zero exit status may
be returned if a command cannot be executed, wrong arguments are given, the command cannot open specified files, etc. The exit status of any command can be accessed through the special parameter $? Example
ls print $?
- Use the test command to determine the status of files, directories, strings, and other parameters. The test
command performs the certain operation and then returns to the parent shell. In fact, the test command does not prove the output, it simply sets the exit status based on the results of the requested
operation. Test evaluates expressions and if the results of the expression are true test returns a zero exits status. There are two forms of the test command: test expr and
[ expr ] There are several primitives available for use when constructing the test expression. Example: test -f true if file exists
and is the regular file test -d true file exists and is a directory [ ! -d Directory _ name ] print $? [ -n "$LOGNAME" ] returns 0 if $LOGNAME is not zero length tests "$user"="Mary" print $? 0 if the user is
equal to Mary, wanted user is not equal to Mary You should get in the habit of quoting all variables used in test statement. [ -f "filename" ]
- Compound operators: [[ string = pattern ]] returns true upon match [[ string != pattern ]] returns true
upon no match example[[ "$filename 1" = "$filename 2 ]] prints $? zero if filename1 is equal to filename2
- Write shell programs which use the if, else, elif statements; the if evaluates the exit code ($?) of a
command and if the exit code is true (0), causes the commands within the body of the if statement to be executed. if, then, fi are keywords. Example; if [ -d "$1" ] then print "$1 is a directory" fi ;
if, then, else, fi: allows further processing. Example: if command then list else list fi ; if command then list elif command then list
else list fi
- use the && and || commands in place of if statements order shorthand statements for the if statement.
Example: [ -d memos ] && print "memos is a directory" if exit code for command1 is true then command2 is executed. [ -d memos ] ||
print "memos is not a directory" if exit code for command1 is false then command2 is executed.
- Special shell parameters "revisited": the $?, $#
parameters are very useful with the if statement. example: # ison - is $1 logged on the system? ; if [ $# -ne 1 ] # the number of arguments is not zero ;
then print "Usage: $0 user" ; exit ; who | grep $1 > /dev/null # Throw away the output ; if [ $? -eq 0 ] ; then print "$1 is logon" ; else prints "$1 is not logged on" ; fi
- The case statement accepts a single variable and then performs a series of pattern matches.
- Eval causes the shell to re-scan the command line, and is used with shell variables and special
characters need to be expanded.
- Use the expr, let, null, and exit commands where appropriate in shell programs. The let command is used
to perform arithmetic functions. A parameter name must be a valid identifier. When a primer is encountered, the value associated with the parameter name is substituted and expression evaluation
resumes. Constants or of the form [base #] number where base represents the arithmetic base. A variable can be defined and integer by using the typeset -i special command. There can be no spaces
between the operators or variables of the let assignment. Let returns a 0 if the value of the last expression is nonzero. Example: let a=5 ; print $a ; let b=a+2 ; print $b ; ((a = a * b)) ; print a ; let
a=2#1001101 print $a ; 77
- Null command: does nothing successfully, used where you must have a command but do not want the
command do anything. Suppose you want to check the status of a command or file, if the status is good, you continue with your program, otherwise print and error message and exit.
- Exit command is used to immediately exit from a shell script, used when an error condition occurs, can
returning value of 0-255 to calling programs. Example:
- 1 # bye - exit demonstration
2 print -n "Enter a number between 1 and 10: "
3 read number
4 print "Enter \"print \$?\" to check return code from $0"
5 case $number
6 in
7 [1-5]) exit 0 ;;
8 [6-8]) exit 1 ;;
9 9|10) exit $number ;;
10 *) exit 255;
11 esac
12 print $?
- Use the return statement to return from functions; return [ value ] is used to return from a function
instead of exiting. When exit is used within a function it will cause of the shell program to terminate. Sometimes, termination is desired. Usually, a function returns to the caller. The return
statement is used within a function to return to the caller. The return statement can set a status code , the value can be used by the caller to determine the success or failure of the function. If value is specified $?
will be set to the value.
[ top ]
Unit [05] Looping Mechanisms [ CODE EXAMPLES ]
- Use the for statement in a shell program. The for statement is used to
perform a certain sequence of commands a specific number of times.
- for statement syntax: for identifier [
in word lists ] do list done. The for loop we specify the number of times we wanted the loop performed by giving a word list or using a default word list.
- Use the while and until loops. The while loop tests the exit status of
the command to determine if it should continue executing the commands in the body of the statement. Each command supplied as an argument is executed and returns a success (0) or failure exit status. As
long as the exit status of the last command is zero, the loop will continue executing.
- The until loop is the opposite of the while loop the difference between the two is that the commands in
the body of a while loop are executed as long as the exit status of the last command is true.
- Write shell programs which use the select command. The select command uses concise syntax to generate simple menus easily.
- Use the true and false statements in shell programs. The true statement does nothing successful, and sets the $?
special parameter to zero. The false statement does nothing successfully, and sets the $? special parameter to nonzero.
- These statements are often used to implement "forever" loops within the while and until looping constructs
- Use the break and continue statements, these two commands modify the normal flow of the loop. break [n]
- Use shell constructs with the getopts command to parse command line options. The shell built-in getopts
command is provided to check and parse command line options. to invoke command, getopts optstring name [ arg . . . ]
- Write shell programs which use co-processes.
[ top ]
Unit [06] Signals and Traps
- State the purpose of signals and traps.
- Signals are intercepted and acted upon. Signals are used to communicate abnormal events
between processes. There are numerous signals.
- The trap is used for two purposes: Execute a command or sequence of commands when the shell
receives the signal. To inhibit the shell from acting upon the signal. The trap command is used to catch signals and perform certain actions based on the signal value. Format: trap [ arg ] [ sig ]
where arg is a command to be executed when the signal is received.
[ top ]
Unit [07] Programming Considerations
- Remember the UNIX programming mantra, Small is beautiful, do whatever is simpler, programs
should do one task and do it well, solve the problem once but at the right level.
- Focus on making your programs run faster and consume less resources. Remember that
shell programs are interpreted and take more resources than compiled programs.
- Shell program development cycle
- design your code
- write, execute, and debug your program
- measure its performance
- optimized were ever possible
- Processes and files use resources, each instance of invoking a shell script causes another process to
be spawned. by default, each user has a limit of 20 processes, each process can only open in 20 files. Of course, defaults are set by the administrator with super user privileges.
- Tips
- Use shell built-in commands were ever possible (they do not create sub-processes),
- Become familiar with UNIX commands and capabilities.
- Used I/O redirection.
- Use exec to invoke sub shells.
- Use functions (they execute faster than subshells,
- Use the dot (.) Command to invoke sub-shells (saves overhead communication),
- Redirect output only once.
- Use pipes instead of temporary files,
- Choose the correct UNIX command for the job,
- Plan file and directory naming to take advantage of wild cards,
- Use the path variable within shell programs,
- Use the cd command to change directories in shell scripts,
- Keep files and directories small (file access is much faster on small <10K,
- Directory searches are much faster on small <640K),
- Clean up your temporary files.
[ top ]
Appendix C sed Stream Editor
- sed is a stream editor wich supplies standard edit commands to lines in a
file as they are read. Given a set of editor commands, sed reads a file, applies the editor commands to each line of file, then writes the modified output to standard output. sed
provides regular expression search and replacement constructs along with some simple programing mechanisms. Situations where sed is useful: when a file is very large, when the same
editing commands need to be made to a number of files, where the editing actions on a line(s) need to be taken from a shell script, when writing in-line conversion programs. Remember, sed applies each command
in the script to each line before reading the next line; therefore, it is always working with the latest version of the line, not necessarily the original. Additionally, sed comments are different! (if the first line
begins with a # it is considered a comment, and can be continued by using a \. Use caution when commenting.)
- Invoking sed: sed [-n] [-e script ] [ -f sfile ] file... [-n] suppresses default output, [-e script ]
specifies the script or sed editor commands (used when there are multiple commands) [ -f sfile ] file... specified that the sed commands are located in sfile
- Pattern Space: sed
reads each line from a file and places it in pattern space. The edits are made in the pattern space. When all edits have been applied, the pattern space is written out and the next line is
read into it. All commands are applied to the new line in the patterne space. sed also has a secondary storage buffer for called hold space.
- addressing: sed applies each editing command to all lines in file. Line addressing can be used to limit
the scope of lines to which the commands are applied. A sed command can specify zero, one, or two addresses. And address can be a line number, lined address symbol, or regular expression. If no address
is specified, then the command is applied to every line. If there is only one address, the command is applied to the line matching the address, it two comma-separated addresses a specified, the command is
applied to each line within the address range, it addresses follow by the "!", the command is applied to all lines which do not match the address.
- Grouping commands:
- Substitution: [address]s/pattern/replacement/options available options:
n = the number indicating that a replacement should be made on be nth occurrence of the regular expressions, g = make changes
globally on the line (remember, by default, replacements are made on only the first occurrence of a pattern on the line) p = printed the contents of the pattern space, w file = write the contents of the
pattern space to file.
- Delete: a pattern followed by d will delete the line which matches the pattern.
- Append: /pattern/a\ new text
- Insert: /pattern/i\ new text
- Change: /pattern/c\ new text
- list: prints non-printable characters as two digit ASCII codes.
- Reading/Writing files: the r and w commands provided method of direct input and output with files.
[ top ]
Appendix D - The AWK Programming Language
- AWK is a programing language designed to scan files for patterns and take actions on in the matches.
Its primary application is report generation and data validation. However, it is an excellent tool for solving numerous data transportation problems. It has full regular expression capabilities and resembles C
programming language. AWK searches for patterns in a record and/or field and performs user defined actions when the pattern is found.
- Format: The AWK program has three sections: beginning, pattern statement, end.
- Beginning section use used for initialization and is executed once.
- Pattern Statement section of the program is data driven. Pattern matching occurs during the
execution of this section and it is applied to each input line.
- End section is executed at be and over reading the data file. note- in the section can be omitted if not required.
- AWK Records and Fields: Each line is considered a record. Each word within a line is considered a field.
The field separator can be set to any character.
- Predefined Variables: AWK contains some predefined variables which provide information about the size
and composition of the input records as well as format control.
- AWK uses output separators, input separators, user defined variables, pattern specifications,
- Operators: AWK operators include: relational operators, arithmetic operators, compound assignment
operators, incremental operators, logical operators.
[ top ]
|
