|
Unit [01] UNIX System Administration
- Duties of the Systems Administrator Include:
- System startup and shutdown
- File system backup and File restoration
- File system maintenance
- Adding and removing user accounts
- Activating and deactivating terminal and printer ports
- Setting up and maintaining communications
- Setting up and maintaining printers and terminals
- Maintaining system configuration files, shell scripts and programs
- Setting up scheduler processes
- Monitor and report system accounting and activity
- Balancing system load
- Training new users on the basics of the system
- Managing the root account
- Establishing and maintaining system security
- Installing software
- Operating system installation and configuration
- Configuring and administration of the network
- Purchasing supplies and new equipment
- Analyzing and correcting system problems
- The root login: login as root, su to root. Precautions to observe! Use root account only when
absolutely necessary. Know what you're doing! Do not let others use root privileges. Do not leave root logged-on any terminal ever.
- System log book: Contains system configuration information, procedures, system log, miscellaneous.
- File system and process management:
- File system management includes managing disk space utilization, insuring correct file and directory attributes.
- Process management includes insuring correct system processes are running, checking system load and performance, terminating processes.
- You must know standard UNIX directory hierarchy.
- Mountable file system: A file system is a logical partition of the physical drive. File systems proved the
organization and isolation of data. All UNIX systems have a root file system. Additional file systems may be created during system installation. Data is made available by mounting the file system on to a
directory. Once mounted, information can be accessed using standard UNIX path names. Filesystem's reside on hard drives, floppy drives, CD-ROMS.
- df Disk free command. du
Disk usage command. You must monitor the amount of disk space. Periodic monitoring of free disk space and free inodes is important to ensure that the number free blocks and free inodes is not getting to low.
- File and directory attributes. Many system problems occur because of incorrect file/directory
attributes, incorrect read, write, execute, or search permissions, wrong owner on file/directory, wrong group on file/directory. Use chmod, chown, chgrp. In many cases, if a user cannot execute a program
or if the program executes but cannot read or write data, it is usually a problem with file and directory attributes.
- Processes: Process is the program in execution. When a user executes the command, the program is
loaded from disk into main memory and a process is created. Each process is assigned a unique number, called a process ID, and an entry is made in the system process table. The information in the process
table is displayed when a ps "process status" command is executed. The process consists of both the program text and program image.
- The program text contains instructions required to perform certain task.
- The program image consist of: program counter, register values, address space, current directory,
user ID, group ID, open files, user environment information.
- Process invocation, a process is created through the fork system call, which creates a new
process. The parent-child relationship between processes occurs. The child process inherits user block data, process table data, open files, and signals. Unique to the child process are its age,
process ID, return code. To execute a new process an exec system call is executed. The calling process is overlaid by the new program. Remember, after a fork system call there are two
processes executing, after an exec system call there is only one process executing. Additional differences between fork and exec
: Arguments can be passed by exec. No return from exec, because calling program has been overridden there is nowhere to return, therefore upon exit the process returns to callers parent.
- Daemon processes: Is a process which runs continuously and has been detached from a terminal
device. Daemon processes are used to perform system task on a periodic basis and sleep when not performing task. Daemon processes include:
- Process ID 0 "sched process" it swaps other processes in and out of memory. It is never swapped out of memory.
- Process ID 1 init this process is responsible for starting all other processes on UNIX.
- Process ID 2 pageout this process is the paging daemon that moves infrequently used pages out
of memory and into the swap area.
- Process ID 3 fsflush this process flushes buffer cachet and file system super blocks to the disk at
specified times. syslogd is the system logger daemon. crond is the job scheduler daemon.
- Orphans and Zombies: Orphan
processes are still active after the parent dies and are inherited by the init process. The Zombie process is a terminated process that is not yet removed from the process
table and has become detached from it's parent. While the parent is still alive and not waiting for the child, the zombie process remains in the process table. The slot in the process table is allocated for the
zombie process, but the process is usually not active. Large numbers of zombie process can interfere with the creation of new processes since the total number of processes that can be running at a time is
limited. Normally, zombie process are removed from process table when the parent terminates. If they are not removed, then init inherits and removes them. If you cannot kill a zombie process, that is usually
an indication that the process does not really exist but there is an entry in the process table. The only solution is to reboot the machine.
- Monitoring system status: The status of the system should be checked after initial startup to insure
that everything is running normally. We want the system performance to watch out for users starting to many programs, or runaway programs etc... Use the ps command to monitor process activity, identify
processes executed by the system and by users, look for orphans and zombie processes, look for processes consuming large amounts of CPU. Process 0 shed and process 1 init are extremely important.
They are running at all times. Checking system status by using the uptime command to display load averages and the w command to display additional information about who was on the system to what
they are doing. Managing processes use the nice command to lower or raise process priority, use the nohup command to run a process immune to hangup signals (remember to redirect the output because
your standard output is closed when you logout) use the kill command to terminate a process. It is better to use the alpha designation with the kill command i.e. kill SIGHUP 666.
- Use the grep (global regular expression print) and find command for system administration task. grep
command is used to search for text using regular expressions. This command is useful for determining which file contains the information you need. The find command searches directory hierarchies for files
based on attributes such S. File name, modification time, owner, group, size. They can find files based on in the information contained inside the inode.
[ top ]
Unit [02] System Startup and Shutdown
- Unit objectives: startup the UNIX system, shutdown the UNIX system,
list the events which occurred during system startup, list the events which occur during system shutdown.
- The boot process: Turning system power on initiates the multi-user boot
sequence. This sequence may be interrupted for you to manually boot the system to access controls. Example, you may want to bring the
system to single user mode and perform maintenance task. During the boot process many things occur. System hardware initialization and checking, the /kernel/genunix/ program is loaded which sizes memory,
initializes important data structures (such as the process table), and initializes devices. Also, prints configuration information, starts the sched process (the process manager, or scheduler), starts the
init process (the primary process for starting all other processes), starts pageout the page daemon, starts fsflush daemon which flushes the write buffer cache to disk periodically. Additionally, the
init process examines the /etc/inittab file and execute processes defined in that file.
- Booting the system SPARC: Automatic Boot
multi-user mode (no operator interaction required), Manual Boot single-user mode (enter abort sequence <L1-A> , enter boot -s command at PROM prompt, enter root password for single user mode),
Manual Boot Interactive (enter abort sequence <L1-A> , enter boot -a command at PROM prompt, answer boot prompts)
- Shutting down the system: Remember, UNIX systems must be shut down through a controlled sequence
of steps. Improper shutdown may cause users to lose data and severely damage filesystem's! For a clean shutdown: users should be notified that system will be shutdown, all processes should be signaled,
notifying them that this system is be shutdown and they should exit gracefully, make transition to single-user mode logging off all other users and killing all remaining processes, complete all pending disk
updates and sink the file system, make transition to run level 0 (shutdown). Always use the shutdown command /usr/sbin/shutdown [-y] [-g grace period] [-i init-state] [message] the -i option is used
to specify the run-level of 0=shutdown, 1=administration mode, S=single user mode, 6=reboot, and optional message may be sent in place to the default message. Alternate shutdown commands include, reboot
(reboot the processor), halt (quickly halts the processor without user warning), poweroff (halts the processor and removes power)
- Loading UNIX The system load to the kernel into main memory and transfers control to kernel. Next,
starts up the sched (process 0) and the init (process 1). The schedule is responsible for loading programs into memory from the desk and removing programs from memory. The init starts all other
processes. Once the processes are started the series of events must occur or before the system is ready for use.
- Run levels: At any given time the system may be in one of eight run-levels. The run levels are defined in
the /etc/inittab file. The most common mode is multi-user, but other modes may be entered for system maintenance, networking, etc.
- The /etc/inittab File: init determines what to do by reading the /etc/inittab file. Each line in the file
contains fields separated by colons ( id:level: action:process ). id one to four characters which identify the line : level the run levels 0-6, or the letters Q,q,S.s,a,b,c : action (respawn
if the process does not exist, then start the process; do not wait for its termination, when the process dies restart the process, if the process exists do nothing, and continue scanning /etc/inittab, wait
win the init enters the run level that matches the rstate, start the process and wait for its termination. once when init enters a run level
that matches the entry's rstate, start the process, do not wait for its termination. When it dies do not restart the process. etc..., process the command or process to be executed for action in the from)
- How init works. Init uses the /etc/inittab file to execute the various processes associated with system
startup. ap action is sysinit and handles important startup scripts. fs causes the command to be executed. System initialization. This command execute shell scripts located in the /etc/rcS directory
which perform many important system startup duties. A fundamental idea is to execute a series of "startup" shell scripts located in specific directories. I.e. /sbin/rec2.
- Controlling init after it has spawned all of the processes specified by the /etc/inittab, it waits for one of
its descendant processes to die, a power fail signal, or a signal from another /etc/init or /etc/telinit process which changes the systems run level. init Q causes the reexamination of the
/etc/inittab file.
- ap::sysinit:/sbin/autopush -f /etc/iu.ap
ap::sysinit:/sbin/soconfig -f /etc/sock2path
fs::sysinit:/sbin/rcS >/dev/console 2<>/dev/console </dev/console
is:3:initdefault:
p3:s1234:powerfail:/usr/sbin/shutdown -y -i5 -g0 >/dev/console 2<>/dev/console
s0:0:wait:/sbin/rc0 >/dev/console 2<>/dev/console </dev/console
s1:1:wait:/usr/sbin/shutdown -y -iS -g0 >/dev/console 2<>/dev/console </dev/console
s2:23:wait:/sbin/rc2 >/dev/console 2<>/dev/console </dev/console
s3:3:wait:/sbin/rc3 >/dev/console 2<>/dev/console </dev/console
s5:5:wait:/sbin/rc5 >/dev/console 2<>/dev/console </dev/console
s6:6:wait:/sbin/rc6 >/dev/console 2<>/dev/console </dev/console
fw:0:wait:/sbin/uadmin 2 0 >/dev/console 2<>/dev/console </dev/console
of:5:wait:/sbin/uadmin 2 6 >/dev/console 2<>/dev/console </dev/console
rb:6:wait:/sbin/uadmin 2 1 >/dev/console 2<>/dev/console </dev/console
sc:234:respawn:/usr/lib/saf/sac -t 300
co:234:respawn:/usr/lib/saf/ttymon -g -h -p "`uname -n` console login: " -T sun -d /dev/console -l console -m ldterm,ttcompat
- The /etc/default/init File contains the default values for variables used by the init process.
- Initialization scripts primary purpose to control startup and termination of processes when entering
run-levels. File naming conventions are K* scripts executed in serial sequence of their sort order to stop processes. S* scripts executed in serial sequence of their sort order to start processes. To enter
another state, init, executes the lines in the /etc/inittab file which are labeled sn, where s is the run level to enter. When we want to enter run level 2, the script
/sbin/rc2 is executed. There is also a directory associated with each run level /etc/rc0.d - /etc/rc3.d inside these directories there are small
shell scripts which contain demands for starting and stopping various processes. Each of the scripts takes a signal argument, either start or stop. Example in the /etc/rc2.d directory there are several
scripts to stop processes that should not be running when the system is in run level 2. These scripts are named with a K prefix. also in the /etc/rc2.d
directory there are scripts which start processes that need to be running at run level 2.
- Shutdown system like startup, there is a specific sequence of task which are performed by the
operating system once the system is commanded to shutdown. The /usr/sbin/shutdown program notifies all users of the impending shutdown, waits for the grace period to expire, signals init to enter
run-level 0 with the init 0 command. Therefore, init examines the /etc/inittab file for an entry with a run-level of 0. Usually the following line is found s0:0:wait:/sbin/rc0 and the rc0
script is executed, which in turn executes all those shell scripts in the /etc/rc0.d directory.
- The /etc/uadmin command after all the scripts have been run,
init, processes either the fw "firmware", of "power off", or rb "reboot" lines of the /etc/inittab file, determining what type of shutdown should be performed. Notice the each line executes the /etc/uadmin
command which actually performs the system shutdown depending on the argument supplied. Do not perform abnormal system shutdowns, are you will be here all night.
[ top ]
Unit [03] User Account Management
- State the characteristics of the user account
- Each user account must have: login name, password, user ID, group
ID, home directory.
- Login ID 8 characters in length and should represent the
users real name.
- Passwords assigned initial password, have user change
password often, teach user to guard password.
- user ID are numbers by which UNIX identifies the owner of files, directories, and processes.
Each user must have a unique user ID. User ID's less than 10 are reserved, and less than 100 by today's standards are reserved. User ID 0 as super user privileges and is known as root.
Some user ID naming strategies: by function, project or department. (System personnel, development personnel, management personnel, etc.) when the user logs on more than one
machine on the network, use the same user ID for the user on every machine. This makes transferring files more efficient. Remember, the user ID, not the username, is stored with a file to identify the user.
- group ID UNIX allows users to be assigned to working groups. This provides another level of
security. Groups are used to assign additional function. Groups are created by adding entries into /etc/group file.
- home directory is the users area and is assigned by the system administrator. Normally the
user home directory is located under /u or /home and is the same name as the user login name. Keep that path short. The home directory is owned by the user, is the same group as
the users group, suggested default 750
- Identify the files used to maintain user account information and identify the contents of each
- There are a number of files used to define user account information and system defaults on UNIX. These files include:
- /etc/passwd this file contains the
user login ID, user ID, group ID, home directory, login shell, and on some systems the encrypted password. This file is public readable and is edited by system administrators. /etc/password
contains a ":" delimited line entry for each user id:password:uid:gid:gcos-field:home dir:shell the password field contains 13 characters or an "x". If an "x" appears then the password is
encrypted and stored in /etc/shadow. The shell field specifies the shell script to be started upon login or another program such as nuucp. The password file is publicly
readable by all users, but root only writeable. It is used by the ls -l, who, and ps commands providing user ID to login name mapping. Security issues introduce the use of the
/etc/shadow file which is read-only by root. This allows /etc/password to be read while maintaining the security of the encrypted password.
- /etc/shadow this file is for root only and contains the encrypted password that is read
by /etc/passwd. The /etc/shadow file contains information for each user in the following format: username:password:lastchg:min:max:warn:inacLockingtive:expire:flag and
provides outstanding tweaking.
- /etc/default/passwd this file contains password checking parameters and stores
default values for password selection and use. PASSLENGTH=* , MINWEEKS=0 , MAXWEEKS=0
- /etc/group this file contains the names of the user groups, group passwords and
defines group membership. May contain an encrypted password but not often. Group members is a comma-separated list of users you may access the group.
- /etc/default/login file contains the default login parameters which are set for each
user.
- Add a new user to the system:
- usradd creates an account, then set the initial password and attributes with
passwd, additionally usradd can add supplemental group membership, create the users home directory, and populated the home directory with the basic set of files from the /etc/skel
directory. When using the usradd command to create a new user, if user attributes are not explicitly requested then default
values will be used. The -D option is used to view and set default user attribute values. When a user account has been created, the initial password and password attributes are created using the passwd command.
- Locking a user account login on the locked account is blocked, user files are not removed, maintains UID
for auditing. When an account is locked the password in the /etc/passwd is set to *LK*
- usermod command is used to modify the attributes of existing accounts.
- Delete the user from the system
- userdel command is used to delete user accounts. All of the account information is removed from the
/etc/passwd file and the /etc/shadow file. By default the user's personal files stored in directories are not removed.
[ top ]
Unit [04] Terminals and Printers
- Unit Objectives
- Connect and configure a terminal
- Enable login on serial ports
- Connect and configure a printer
- As a system administrator you will be responsible for adding hardware to
your system such as serial and parallel ports, terminals, and printers. You may also be required to install additional hard drives, modems, a CD-ROM's, scanners and other types of devices.
- Installing a terminal requires several steps; including, physically connecting the terminal to the system,
set terminal hardware configuration, enable login on serial port, test and troubleshoot terminal, login, set environment
- Service Access Facility provides the system administrator and users with a common interface to local
serial ports and network access. In earlier versions of UNIX, each time a new interface was added to the system, new commands, drivers, and administrative procedures had to be added to manage the new
interfaces. Commands sac, sacadm, pmadm, ttymon, listen. The sac is normally started by init from the following entry in the /etc/initab sc:234:respawn:/usr/lib/saf/sac -t 300
- Port Monitors control port services. A port monitor is a set of programs which control the set of related
port services. For example, all of the port services which provide local tty access may be control by a single port monitor. Each port monitor is given a port monitor tag when it is created. This tag allows us
to administer all the port services that the port monitor is controlling with a single command. Network services are provided through the port monitor by the listen program. listen
provides connection services such as TCP/IP for any connection oriented transport service that conforms to the transport level interface specification. The nlsadmin command is used to administer listen
services.
- Port Services is a set of programs which control the flow of information on a port. There is only one
service associated with each physical port. The service could be providing network access on the port or it could be providing local access (such as login service.) Each service, when created, is given a port
service tag. This tag distinguishes the service on a particular port from other services which execute the same set of port service programs. The pmadm command is used to administer the port services.
- SAF Configuration Files allow the system administrator to assign variable values, push or pop STREAM
modules, and execute commands with the following keywords:
- assign insuring value to a variable
- push push a STREAM module on a designated stream
- pop remove a module from a stream, if pop
is invoked without argument the top module is removed. It invoked with an argument, that module is removed.
- runwait run a command and wait until it complies
- run start a command and immediately return
- /etc/saf/_sysconfig File is used to specify the environment for all services controlled by the
sac program. This file is read and interpreted when sac is started by init.
- /etc/saf/<pmtag>/_config File is created when a port monitor is defined. The sub directory will
have the same name as a port monitor. Files which are associated with that port monitor are maintained in this file directory. The system administrator may create an optional filename in the
port monitor directory and entered into the file port monitor specific configuration information. This file will be read and interpreted by sac whenever a port monitor is started.
- /etc/saf/<pmtag>/<svctag> File contains configuration specific information for a port monitor
service. These files are optional in can be used to override or append to any previously interpreted scripts for the associated service. The scripts are read an interpreted prior to the invocation of the
port service by the port monitor.
- SAF Administrative Files are maintained to keep information about configured port monitors and
services. The administrative commands, sacadm and pmadm should be used to change the contents of these files. The sac command only reads and interprets the contents of these files when they're
modified. log files
- sacadm command adds a new port monitor to the system like creating an entry in the
/etc/saf/_sactab file and then calling sac to initialize the port monitor.
- SAF Initialization
- The sac is started by the entry in the /etc/inittab file
- sac reads and interprets the /etc/saf/_sysconfig
file to customize the environment
- sac reads the /etc/saf/_sactab
file to determine which port monitors are to be started
- Reads port monitor configuration files (if available)
- Starts the port monitor (has monitor polling capability)
- sac polls port monitors for status as per time flag set when started
- Each port monitor starts port services as required
- The ttymon Program monitors serial ports, establishes connection with requests is received, initializes
port for values and administrative file, writes prompt to port, waits for at least one character and NEWLINE, interprets service configuration script, invokes specified service (i.e., login), resets
communication settings upon completion of service
- The ttyadm Program provides administrative information to the port monitor. Which in turn provides his
information to the SAF.
- The pmadm Command is used to create, remove, modifying, manage, and lists for monitor services. Is
also used to change information in the /etc/saf/<pmtag>/_pmtab file.
- Listing Port Services use of the pmadm -l command
- Changing The Term Variable the TERM variable may be set in the users $HOME/.profile used when users
logs in over the network or when pseudo-ttys are being used. TERM variables must be exported. Example, TERM=VT100 ; export TERM
- Terminal Troubleshooting
- Ensure that the sac is running by checking both the output of the ps command and sacadm -l command
- Ensure that the port service associated with the serial port is running
- Check hardware connection
- Enable the terminal again
- Make sure your wiring is correct
- Check the terminal setup
- Swap terminals
- Reboot system
- Printer Installation UNIX provides a print spooling mechanism which can support multiple printers. Each
printer has a name, and print jobs can be routed to any printer connected to the system. Different types
of printers can be connected to UNIX. Both parallel and serial printers are supported. The printer driving software on UNIX does not provide any formatting or text modification. It simply sends the information
from a program or a file to the printer. It is a responsibility of the other programs, such as word processors, to insert the proper control codes for printing special characters. Print jobs are placed on a
print queue and sent to the printer by the printer daemon lpsched. When a user requested file to be printed, the file is copied to the spool directory /usr/spool/lp . The lpsched
program continually examines the spool directory for print jobs. When it sees a file waiting to be printed, it first determines if
the printer is free. If so then it creates a "lock" file to reserve the printer and then begins the print job. The lpsched
also provides line and flow control. Most printers cannot print information as stress is a computer can send it can require flow control.
- lp Commands
- General Use
- lp sending files to line printer spooler. The destination can be specified. The command either
accepts input from a file, a pipe, or standard input.
- cancel is used to remove jobs from the print queue.
- disable prevents lpsched
from routing jobs to the printer.
- enable allows lpsched
to route jobs to the printer
- lpstat prints the status of the printers.
- Administration Commands
- lpadmin modifies the line printer configuration
- lpsched routes print jobs to the printers
- lpshut stops the lpsched
- accept allows lp
to accept output requests for destinations
- reject prevents lp
from accepting print jobs
- Line Print Spooling Overview the line print spooling mechanism is somewhat complicated.
- When a file is sent to lp it is placed on the print queue in the directory /usr/spool/lp/request
files stored on the queue have their printer destination as part of their name. The line printer daemon lpsched continually monitors the queue for print jobs. If a job appears and the printer is
available, it is sent immediately. The job is sent to the filter which sets up the line discipline, provides additional file filtering, sends preprint commands to the printer, routes the file to the
printer, and performs cleanup as the job is printed. The filter for each installed printer resides in the/usr/spool/lp/admins/lp/interfaces file.
- Print jobs can be removed from the queue with the cancel command. The cancel command
accepts the name every single job (as defined by the lpstat command) or the name of the printer, in which case it will cancel all jobs associated with that printer (if user has proper permissions).
There are levels of control for the line printer spooler. The first level is the lpsched if the scheduler is not running print jobs will not be excepted and definitely not printed. If the lpsched
is running, then a printer can either be online (enabled) or off-line (disabled). The disabled command is used to take a printer off line. The enabled
command is used to place the printer back on line. These commands are primarily used for maintenance, print jobs will still be placed on the print queue and will be printed with a printer becomes enabled again.
- The third level controls acceptance of print jobs by the printer itself. The accept command tells
the print spooler to accept print jobs for the printer and the reject command tells spooler not to accept print jobs. These commands are to be used by the system administrator.
- The lpadmin command is used to configure printers. In most cases, the print spooler must be stopped with lpshut
before using lpadmin. After the printer is configured, the spooler is started with lpsched. The status of all printers can be determined with the lpstat command.
- Setting up Print Spooling
- lpadmin add the printer to the spooling system
- enable the printer
- accept allow printer to accept jobs
- lp file chest up under
- lpadmin Command can be used to add, delete and modify printers. Example
, /usr/lib/lpadmin -dprinter specifies the default printer.
[ top ]
Unit [05] File system Maintenance
- Unit Objectives: Upon completion of this unit you will be able to: State
the physical and logical layout of the UNIX file system, Mount and unmount file systems, use the fsck command to perform file system checks, create new file systems, and identify two file system desperation commands.
- File system Features: The file system consist of a hierarchy structure of
files and directories and an underlying structure of control blocks and pointers. The UNIX system must contain at least one file system "the root file system", although most UNIX systems have multiple
filesystem's for easy organization and maintenance of information.
- The file system implementation offers dynamic file expansion meaning no disk space quotas are
placed on a user; however, the maximum size of each individual file a user can create is adjustable.
- Structure less files meaning no structure is placed on an ordinary file by the operating system.
The file consists of a stream of bytes. If a structure is required it is the responsibility of the application software.
- Security access to files and directories can be controlled on each individual file and/or directory.
- File and device independence, programs on UNIX read and write files. The files can be
associated with particular devices other than the disk, such as terminal, printer, tape, etc. thoughts programs do not know or care where their input/output is coming/going.
- Hierarchical File systems consist of the set of directories and files. Technically, directories are also
considered files on UNIX.
- Types of files: in UNIX the only data storage type is a file, to classify the type of data a file contains we
discus files in terms of types.
- Ordinary, has no structure imposed on it by the operating system. Programs which access the file
are responsible for its interpretation. An ordinary file may contain ASCII text, and data files formatted for the database application, executable files which contain binary programs or shell
scripts (however shell scripts or also a form of ASCII text files)
- Directory, is a file which contains a structure specific to the operating systems. A directory
contains entries consisting of inumber/filename pairs. The inumber is the number which reference is an inode. The inode
contains information which describes a file, including the location of the data. The second part of a directory entry is the filename associated with the inode. Traditionally
UNIX file systems allow filenames of up to 14 characters. More modern systems allow filenames up to 255 characters. The first two entries in a directory or always .(dot) and ..(dot dot) meaning
current directory and parent directory respectively. Directories grow but do not shrink, when a file is removed from the directory, the inode is changed to 0. When a file or directory is added, if there
is an entry with 0 it is replaced with the new files inode number. Otherwise the entry is added to the bottom of the table. Obviously, large directories cause a performance slowdown.
- Special (block, character), files are used to keep device-dependent code out of the user
programs and in the operating system. Through the use of special files, programs can be written so they only need to know how to access a file. The mapping between a filename and hardware
device is done by the operating system through special files. There is a special device associated with every hardware component. Special devices may be one of the following types:
- block special device reads and writes data a block at the time, normally in increments of 512
bytes. Block devices are usually referred to as "cooked" devices. example: /dev/disk/c0t3d0s0
- character special devices read and write data 1 byte (character) at a time. Normally,
terminals, printers, tape drives are character devices. However, disk and tape drives can be set up as block devices. Character devices are referred to as "raw" devices. Names of
character devices begin with a "r" example: /dev/rdisk/c0t3d0s0
- Named Pipe, it is a special device used for inter process communication. Pipes use a first in,
first-out data structure and can block of their input stream while waiting for a read to occur on the output stream.
- Semaphore, special files are used for inter process communication in some applications programs.
- Shared Memory, like semaphores, shared memory special files are used for inter process
communication by some application programs.
- Symbolic Link is a link to another file on a different file system, perhaps on a different machine. A
symbolic link file contains the complete path name of the file to which it is linked.
- Types of File system
- UNIX File System it is the default filesystem. Provides high throughput, effective use of disk
space, high reliability, the capability for large numbers of files, and very large files.
- High Sierra File system is a CD-ROM file system
- Network File system allows directories on remote systems to be accessed as if they were file
systems on the local system.
- File system Layout is the model that the actual physical structure the UNIX kernel assumes. Individual
file system types may store there data in a different format, bought the file system software presents this model to the kernel. The common structure of a physical file system includes about: BOOT-BLOCK where
the information needed to "boot" the operating system resides, SUPER-BLOCK where information about the file system such as the number of data blocks, number of Inodes, size of the file system, file system
state, number of free data blocks, number of free inodes, etc.,., I-LIST contains a list both inodes, each I-node contains information about the files and directories on the disk. There is one inode for each file
and directory on the disk. DATA-BLOCKS contain the actual data stored on the disk. A file/directory may have more than one data block.
- SUPER-BLOCK is 512 bytes long and contains the following:
- isize the number of the first block following the
I-LIST
- fsize the file systems size and blocks
- nfree and index into the "free" array
- free an array of pointers to free data blocks
- ninode an index into the inode array
- inode an array containing the inumbers of free inodes
- flock locks the free lists while it is being accessed
- ilock locks the inode array when it is being accessed
- fmod a flag indicating whether or not the super block has been modified. The flag is also
used to indicate whether the file system needs to be checked before mounting
- ronly read-only flag, used to prevent writing to the file system which was mounted as
read-only
- time date and time of last update to the file system
- dinfo device information (i.e. blocks per cylinder, spacing, etc.)
- tfree total number of free blocks in the file system
- tinode total number of free inodes in the file system
- fsname name of the file system
- fspack name of the device on which the file system resides. Together,
fspack and fsname provide the label data used for mounting and unmounting a file system
- INODES Structure the I-NUMBER
contained in a directory refers to an inode address. Inodes follow the SUPER-BLOCK and are used to access a file. Each 512 byte physical block contains
eight 64-bit inodes. An inode contains information about a file.
- mode the file/directory access permissions
- nlink the number of links to the file/directory. The link count for a directory is the number of
directories to which the directory points. Since a directory always refers to itself and its parent, the link count will be 2 for a newly created directory
- uid the ID of the person who owns the file or directory
- gid the ID of the group to which the file belongs
- size the size of the file in bytes
- atime date and time the file was last accessed
- mtime date and time the file was last modified
- ctime date and time the inode information was last change
- data block address the list of data blocks which contain the data for the file
- INODE List Manipulation each file and directory has one
inode associated with it. As files/directories or created, the number of available inodes shrinks and grows. The super block maintains a list of free inodes
and manipulates as follows. An initial search of available inodes is made and the free inode array is filled. Available inodes are indicated by their mode
field being set to 0. At any one time, the free inode array contains a list of available inodes. When a file/directory is created, an inumber is taken from the free inode array and the
ninode counter is incremented. If the free inode array becomes exhausted, a sequential search of the super block ilist is made and the inode array is repopulated. If the inode list
becomes over filled , it is ignored. This is possible because the inodes are marked as being used or available within the ilist and can be recovered later with the sequential search.
- Data Block Manipulation each file must contain at least one data block, although in most cases a file
has multiple data blocks. The list of data blocks associated with a file is contained in the inode for the
file. The "free" array is the first block in the linked list of data blocks which contain pointers to free data
blocks (i.e., when a file system is created, a set a blocks is reserved which contain the address of blocks
used to store data) this list of data blocks is referred to as the free list. The first pointer in each block of
the free list points to the next block in the list. The last block on the free list has a 0 in the first pointer slot.
- nfree works very much like ninode. nfree
is used as an index into the "free" array. nfree is decremented when blocks are allocated and incremented when blocks are freed.
- It is important to know that the ilist and block list manipulation occurs in memory, so at any time
the super block in memory may not match the super block on the physical device. Periodically, a "sync." command is issued by the operating system which causes the inn-memory copy of each
mounted file systems super block to be written to disk. This always occurs immediately prior to un-mounting a filesystem. If the system crashes or it is otherwise shut down improperly, the fmod
flag will tell the operating system that the file system was modified. In this case, the file system check fsck must be run prior to mounting the filesystem.
- File/Directory Access when you access the filename, the following sequence of events occurred:
- The directory is open and searched for a file of the name specified
- The inumber for that file is read from the directory
- The inumber is used to locate the inode in the ilist
- The inode is open. The information in the inode is used to ascertain if the user has the correct
permissions to access the file based on the permissions, UID, GID enterys. These checks are made by the kernel.
- If the permissions are correct, the data block addresses are used to physically locate the information on the hard drive
- The access time is updated in the inode. If the file was modified, then the modified time is updated
- File Manipulation
- File Creation when a file is created and inode is allocated and the mode, UID, GID, link, size, in
time information are filled in. One data block is allocated. An entry is made in the directory.
- File Removal when a file is removed the directory entry associated with the file is set to 0, the
data blocks are freed, the inode is released
- File Renaming when filed is renamed (i.e. moved) one of the following will occur; if the file is
moved to a directory on the same file system a new directory entry is created with the filename and same inumber. If the file is moved to another file system it is first copied using the file creation
actions, and in the old file is removed
- File Linking two files can be linked on the same file system through a hard link, or can be linked
across file systems with a symbolic link. When a hard link is made an entry with the same inode number as the linked-to file is made in the specified directory. The filename may be the same as
the original file if the directory is different: otherwise, the filename must be different. When a symbolic link is made, a new file is created and the marked as a symbolic link. This file contains the
full path name of the file to which it is linked.
- Mountable File systems Each file system resides on an independent device, and has its own
to A file system is created by using the mkfs command or one of the related commands that calls mkfs. Once a
file system is created, it is made available to the rest of the system by a mounting it. The mount command announces to the system that a removable file system or resource is present on a special
device. (Filesystem's other than root are considered removable) mount requires a minimum of two arguments: the name of the special device containing the file system and the name of the existing
directory upon which the file system will be mounted. Other arguments may be specified which indicate the type of file system, read/write options, and permissions. If no additional arguments are specified, the
information in the /etc/vfstab will be used. Example: /etc/mount /dev/dsk/c0t3d0s3 /u The directory of file system is mounted on is called the mount point.
- When a file system is mounted, the filesystem's super block is read into memory. The in-core
copy of the inode for the directory the file system is mounted on is marked to indicate that the directory is a mount point. An entry is made in the /etc/mnttab file for the mounted filesystem.
- After a mount, the mounted file system structure "hangs" below the mount point. Any data that
was below the mount point before the file system was mounted will not be available while the file system is mounted on the directory.
- When files are accessed on a mounted file system, the marked inode indicates that this it is a
mounted filesystem. The operating system examines the mount table /etc/mnttab and finds the name of the special device associated with the root directory of the mounted filesystem. A context switch
then occurs and the operating system now uses the super block for the mounted file system to locate the file. This context switch is what makes files under a mount point inaccessible after a mount.
- It is interesting to examine the inode numbers of mount points before and after a mount. Before
a mount, the inode number for the mount point will be that of the directory. After a mount, the inode number will be a 2, which is the inode number of the first inode of the mounted file system.
- /etc/vfstab file contains information for mounting and and unmounting file systems. This file is read
during system startup so the system can determine which file system are to be mounted.
- /etc/mount & /etc/mountall commands can only be executed by the super user.
- Unmounting a File system the /etc/umount
command is used to unmount a mounted file system.
- A file system can only be unmounted if there are no users on the file system (i.e. the users current
directory is not on the mounted file system)
- There are not any programs running which reside in the file system
- There are not any data files on the file system in use
- When a file system is unmounted the in-core copy of the super block is written to the file systems
super block on the disk, the in-core copy of the super block is removed from memory, the mount point mark in the inode for the directory on which the file system was mounted is removed, and the
injury for the mounted file system is removed from the /etc/mnttab file.
- Removable Media using the file systems on removable media creates some additional issues. When a
piece of media is inserted into a drive in a UNIX computer, it has to be mounted before it can be used. The largest issue surrounding removable media is that the media must not be removed until it has been
unmounted. Remember that while a piece of media is mounted there is a directory that references the media (the mount point), and the super block for the file system resides in memory. It the file system is
not unmounted before the media is removed, any updates to the file system will not be written to the mount point and the mount point now refers to an invalid device. Solaris provides software to help the
administrator deal with these issues. The volume manager system is a collection of programs in the special file system that attempts to automatically detect when a new piece of media is inserted, and if
appropriate, will mount any file system present on it media. The volume manager must be running to detect and automatically mount file systems. The volume manager must not be running if direct access
to the media is desired. Example: /etc/init.d/volmgt stop
- The sync command forces the in-core copy of the super block's mounted file systems to be written to
the hard disk super block for each file system. sync is the last command issued prior to system shutdown. The command is often written sync ; sync the first command queues the super block for
writing and the second command performs the write.
- File system Permissions when a new file system is created its permissions are set to 777. When you
mount a file system, the mount point permissions will be those of the file system. This can cause a possible security problem. After you mount a new file system the first time, you need to set its permissions. chmod
- Cleaning Up File systems if a file system runs out of disk blocks or inodes no further data can be stored
on the file system. Some UNIX commands create log files which must be cleaned out periodically. Users tend to keep a lot of old data, have them removed files they no longer use. Remove core files
- Corruption of the Super Block the super block is the part of the file system most prone to damage.
Every change to a file system affects the super block. The super block contains the size of a file system and a size of the inode list. fsck
will check to insure that these fall within reasonable limits. The block numbers of the first 50 free blocks in the free list are kept in the super block. fsck will check each block
number against the list of already allocated blocks to insure the number of blocks used by the free list plus the number of blocks claimed by inodes is equal to the total number of blocks on the file system. If
any errors are detected, fsck will try to rebuild the free block list, excluding any blocks already in the list of allocated blocks. The super block, also contains a count of the total number of blocks on the file
system. If a discrepancy exists, fsck may replace the free block counts in the super block with the
actual number of blocks it calculates exists. The total number of free inodes, is also contained in super block. fsck
will compare this value against the number of inodes found free in the actual file system. If they are different they may be replaced with the actual free inode count.
- The super block may be corrupt if the file system is wrong, inode size is wrong, free block lists wrong,
free block count wrong, free inode count wrong.
- inode corruption occurs because of all the file activity resulted from modify the inode associated with
the file. Problems with inodes may involve format and type, linked count, duplicate blocks, bad blocks, and/or inode size. An inode may be either a regular, directory, block special, or character special file
inode. An inode may have one of three states: allocated, unallocated, or neither allocated nor unallocated. An inode of the last type is an incorrectly formatted inode. Each inode contains a count of
the total number of directories linked to it. Starting with the root directory, fsck descends the entire
directory structure, calculates an actual link count for each inode, and compares it to the count stored in the inode itself. If they do not match, fsck
will replace the stored link count with the actual link count. Each inode also contains pointers which point to all of the blocks claimed by the inode. fsck will check for
duplicate blocks by comparing each block claimed by the inode to a list of blocks already claimed by other inodes. If duplicates are found, fsck may clear either one or both inodes claiming the block. fsck
also checks for bad block numbers. A block is considered to have a bad block number if its value is less than the first data block or greater than the last data block. Finally, fsck
checks the size field of each inode to see if it is consistent with the number of blocks claimed by an inode. It errors are detected a possible file size warning message is printed
- fsck File System Check Command
is run whenever a new file system is created, whenever the system is abnormally shutdown, whenever the system administrator feels that a check needs to be made of the
file system. It is highly recommended that a check be performed on every file system at least once a month. Many system administrator's running check prior to backing-up the file system. fsck checks file
systems for the following inconsistencies: blocks claimed by more than one inode or the free list, blocks claimed by an inode or the free list outside the range of the file system, incorrect link counts, size
checks: incorrect number of blocks, directory size not 16-byte aligned, bad inode format, blocks not accounted for anywhere, directory checks: file pointing to unallocated inode, inode number out a range,
super block checks: more than 65536 inodes, more blocks for inodes than there are in the file system, bad free block list format, total free block or free inode count incorrect. Orphaned files and directories
(allocated but unreferenced) are, with the operators concurrence, reconnected by placing them in the lost & found directory.
- fsck Phases include the
- Initialization phase
- Phase 1 check blocks and sizes
- Phase 2 check path names
- Phase 3 check connectivity
- Phase 4 check reference counts
- Phase 5 check cylinder groups
- After fsck runs restart the system if root file system was checked, mount all unmounted file system's,
check lost and found directories, restore missing files and directories from backup
- Creating a File System
- Make sure that the media is properly formatted. When you have a hard drive, the installation
procedures will normally format. If you are making a file system on a floppy disk use the fdformat command.
- Define a new slice: run format, use verify to determine available space, use partition to add a new slice, use
save to write new partition table to disk, use quit to leave format, make sure the device files are present in the /dev/dsk and /dev/rdsk
- Making the file system, once you have created the slides on the hard drive or have a formatted
floppy disk, you can create the file system using the mkfs or newfs. The preferred method under Solaris it is the newfs command
- File system Commands
- fstyp prints type of file system
- df prints amount of free disk space
- du prints disk
- quot displays the block use each by user in a given file system
- File system Desperation Command there are two commands which should only be used by experienced
professionals. These commands are mentioned to make you aware of them if everything else fails! Punt... then call Microsoft.
- clri clear inodes, this command clears the contents of an inode. It must be used on an unmounted
file system and fsck should be run on the file system after using clri.
- fsdb file system debugger command is used to manually examine and patch the file system. It is a
very powerful tool which can be used to make repairs to badly damaged file system.
- Exercises:
- The super block stores all information about a file systems free and used inodes and free and used
data blocks. If the super block is damaged or destroyed, there is no way of determining the assignment of inodes or data blocks to files.
- Permissions, Link count, user ID, group ID, size, and the created, accessed, and modified times.
- Run a "df -t" or "/etc/mount" command.
- Files which have a valid inod but do not have a directory entry are placed in the lost & found directory by fsck.
- The volume manager attempts to detect a file system on a removable media, and if a valid file
system exists, the volume manager Mount it under the special directory.
- The permissions field of the inode is set to null "all 0"
- A list of the unused data blocks is maintained in the free lists of the super block.
- Whenever any change is made to the inode.
- Because hard links are made through duplicate directories entries of the files inode. The inode for a
file on one file system cannot be used in a directory on in other file system since each file system has a unique set of inodes.
- The file system's super block is read into memory, an entry is made in the /etc/mnttab with the
name of the file system's special device and the name of the directory on which the file system is mounted, the directory on which the file system is mounted is marked as a mount point.
Unit [06] Disk/Tape Management
- Unit Objectives
- State the types of UNIX devices
- Identify special types of devices on UNIX
- Format different disk types
- Use the cpio command to create an extract file archives
- Use the tar command to create an extract file archives.
- I/O Subsystem is divided into; block I/O
(structured I/O), character I/O (unstructured I/O). All programs communicate via system calls. System calls to input or output are designed to eliminate the
differences between various devices and styles of access. There is no distinction between random and sequential I/O, nor is any logical record size imposed by the system. The design also attempts to
eliminate differences between ordinary disk files and devices such as terminals, tape drives, and printers. A device entry appears in the file system hierarchy for each supported device, so the structure of device
names is the same as filenames.
- Block I/O device consists of randomly addressable secondary memory blocks. The blocks are
consecutively address from 0 up to the size of the device. The block device driver emulates this model on the physical device. The block I/O devices are accessible through the layer of offering
software. This system maintains a list of buffers, each assigned to a device name and a device address. This buffer pool constitutes a data cachet for the block devices. On a read requests the
cachet is searched for the desired block. If the block is found the data is made available to the requester without any physical I/O. If the block is not in the cachet the least recently used to
block in the cachet is renamed, the correct the device driver is called to fill the renamed buffer, and then the data is made available.
- Character I/O is handled by device drivers for physical devices which do not fit the block I/O
model: printers, terminals, etc. Character oriented device drivers use a queue of characters to store characters to be transformed. Space for the queue is allocated as characters are added and
released as characters are removed. Characters from the user are placed on the character queue until the maximum number of characters are read. The driver then gets the characters from the
queue and transfers them to the hardware serially.
- Device Drivers are configured into the operating system. A device driver is the software necessary to
perform I/O on a particular device. For example, I/O for a terminal differs significantly than I/O for a disk drive. Each supported device is associated with at least one special file and one
device driver. The special file provides the interface between the device name and the device driver. Devices are
characterized by a major device number, a minor device number, and a class (block or character). The major device number is used to index the array when calling software for a particular device driver. The
minor device number is passed to the device driver as an argument.
- Device Types all hardware devices are viewed by UNIX as files. UNIX uses a filename to access the
hardware device. This greatly simplifies access to the hardware because it allows programs and the user to use the same commands to access the data. When UNIX is installed, it creates a number of special files in the
/dev directory. There is a special file for each hardware device installed on your system. If you later install additional hardware, then you will have to create the special device. In most cases the
installation program for the new hardware will automatically create the special device. Only if you write a device driver yourself for a special hardware device will you need to install it manually.
- Floppy Device Names the /dev/fdo and
/dev/diskette0 devices are bock special devices and the /dev/rfd0 and /dev/rdiskette0 devices are character (raw) special devices.
- Tape Devices special files are located in the
/dev/rmt directory.
- Formatting Disks by using the fdformat command.
- The /etc/default/tar file contains a list of devices which can be used by tar. Instead of using the F
key letter and the name of a file for device, the number of the line for the device and the /etc/default/tar file is specified. The contents of a typical /etc/default/tar file includes the name of
the device, the block size to use when accessing the the device, the size of the device, and whether or not the device is a tape.
- To create a tar archive, you must supply three arguments to tar. The c key, telling tar you want to
create an archive, the f key, followed by the name of the archive (usually a the name of a floppy disk or
tape device, but it could be the name of a regular file), and a list of files to be added to the archive.
- Example: tar cf /dev/fd0 ; tar cvf /tmp/backup.tar *
- Absolute verses Relative
path names; when you specified the names of files and directories to either the cpio or tar command, you can use either the relative or absolute path names. We suggest using a relative
path name. If you use a full path name then when you restore a file they will be recreated under that directory structure. However, there are some problems using the absolute file names, first there's a
danger of overriding good existing files. With absolute filenames you are not able to rename the file during restoration, unless you specify the appropriate option for the command. Second, if the system on
which your restoring does not have a /u it will restore all the files under the root directory. By using relative path names when making the back up, you can restore the files to any file system and your
directory you choose and then move the files to their appropriate location after the restore.
[ top ]
Unit [07]
Performing File Backups and Restores
- Unit Objectives: perform full, partial, and incremental backups. Perform file restoration.
- A major responsibility of the UNIX system administrator is insuring that users and system files are
maintained on back up media encase the system crashes or a file is accidentally removed. The system administrator is also responsible for restoring missing or damaged files as required.
- Back ups can be performed on a schedule are unscheduled basis. A scheduled back up is a regular
back up that is performed on a predetermined basis. A back up schedule is established which specifies the days on which the back up will occur, the type of back up (full or incremental), and which file system
will be back up.
- The ufsdump command back ups all scheduled files (normally the whole file system or files within a file
system changed after a certain date). The file system must be inactive, otherwise the output may be inconsistent and restoring files may be impossible. The system should be in a single user mode. ufsdump
accepts arguments which may be multiple strings whose association with the option is determined by order. Example: ufsdump 0cfu /dev/rmt/0 /dev/rdsk/c0t3d0s0
- It is vital to perform full "levels 0 dumps" at regular intervals. When performing a dump, bring the machine
down to single user mode. Incremental dumps allow for convenient back up and recovery of active files on a more frequent basis. However, there are some tradeoffs. First, the interval between back ups
should be kept to a minimum. To guard against data loss caused by a bad media capture active files on two sets of a dump files. Another consideration, keep unnecessary duplication of files to a minimum.
Another consideration, how hard would it be to find your file and restore it.
- ufsrestore utility restores files from back up volumes. Its actions are controlled by a key argument and
may be used interactively.
- Customized back ups. The ufsdump utility is appropriate for backing up entire file systems, either
incrementally or completely. In some cases, you may wish to back up a smaller part of a file system. The cpio And tar commands can be used for these types of back ups.
[ top ]
Unit [08] cron & at
- Unit overview: UNIX provides facilities for running jobs at a later date and time. The
cron program runs jobs on a regularly scheduled basis, while the at program runs a single job. By default, we limit the use of cron
to system administrators and system processes and allow all users to use at.
- cron command starts a process that executes commands at specific dates and times. Regularly
scheduled commands can be specified according to instructions found in the crontab file located in the /usr/spool/cron/crontabs directory. Users who have permission to use cron
can submit their own crontabs jobs via the crontab command.
- Commands to be executed only once may be submitted to using the at command.
- cron only examines the crontab files and at
command files during process initialization and when a file changes via the crontab or at commands. This reduces the overhead of checking for new or changed files at regularly scheduled intervals.
- cron is started at system boot time. Since cron
never exits, it should be executed only once. The /usr/lib/cron/FIFO file is used as a lock to prevent the execution of more than one cron daemon at a time.
- The files which cron processes, to run commands, our kept in the /usr/spool/cron/crontabs directory.
All of these files are normally owned by and belong to the group cron. For security purposes, the file is set to have read/write permissions for the owner and group only.
- crontab files consist of lines of six fields each. The fields are separated by spaces or tabs. The first
five fields are integer patterns that specifies minutes, hour, day of month, month of the year, day of week. Each of these patterns may be either an * or a list of elements separated by commas. The six
field is a string that is executed by the shell at a specific time
- Who can use cron? /usr/lib/cron/cron.allow list of users who are allowed to use cron. /usr/lib/cron/cron.deny
list of users who are not allowed to use cron.
- The crontab command. The /usr/lib/cron/crontab files should not be edited directly, remember that cron
reads these files only once when it starts up. Therefore, any changes made to these files will not be recognized until cron is restarted. The crontab command is provided for adding new
/usr/lib/cron/crontab files and making changes to existing files.
- cron reads the /usr/spool/cron/crontabs
files when it first starts up. If you manually edit this file then cron must be stopped and restarted. Therefore, it is best to use the crontab command to change files.
- When using crontab make sure that you are working under the correct user ID. Therefore, you do not accidentally change the wrong file.
- The /usr/spool/cron/crontabs file cannot contain a blank lines. Use comments if you want to include a blank line.
- You must redirect output from commands which write to standard output to a file or a pipe. If you do not redirect output, cron
will send the output to you via mail.
- The at command is similar to cron except it is used to schedule single jobs to be executed at a later date and time. Like
cron, you must be given permissions to use at. This is done by entering the user's name in the file /usr/lib/cron/at.allow. Unlike cron, at
only executes the job once then removes the job.
[ top ]
|
