Note on Unix: Peter Allan ("Unix" and some other words in this article are trademarks.) Feel free to copy this file and inform me of any errors you find. ---------------------------------------------------------------------------- The latest version of this file can be collected by email to peter.allan@aeat.co.uk Subject="send goodies" Body="guide to unix" ---------------------------------------------------------------------------- These books have been helpful to me: Peter Nortons's Guide to Unix by Peter Norton and Harley Hahn especially good on vi Unix Shell Programming by Steve Kochan and Patrick Wood especially good on filename and variable substitution and quotes Also some of the famous O'Reilly Nutshell series. ---------------------------------------------------------------------------- Note that there are several Unix-like operating systems. These usually run on workstations, but there are version for high-end PCs. True Unix is a command-line system where you type your commands with their options and parameters. To some extent it resembles DOS, but has more power. Actual operating systems you buy, based on Unix, have graphical extensions. It is a multi-user system, with many commands. It is of course possible to survive with only a few. Whole books are written about single commands. This file makes no pretence at being complete. ################################################################## Each human user is known to the machine by a userid or id. A password is usually (should be) set for each id. Everything is case-sensitive. ('A' and 'a' are different.) Here are the essential commands for beginners: ls == 'ls' is the command to list filenames. It can have options (Flags) added after the command with a minus sign '-'. Arguments (parameters) can also be added. ls ordinary file listing ls -l long file listing (file sizes, change times,..) ls -a include filenames starting with '.', normally not listed ls -l -a applies both the above options ls -la identical to the previous command ls *.fort list files called "ANYTHING.fort" ls -lat *.fort the t means list them in time order, not alphabetically These examples show the main points of all Unix commands. They are short, thay take options, they take parameters, you can combine things together. rm == This command removes files (and directories). BY DEFAULT YOU GET NO SECOND CHANCE, AND ONCE THEY ARE GONE THEY ARE GONE. rm filename(s) rm -r filename(s) Remove directories and all their contents. rm -i filename(s) Prompt for a y/n choice before each file. GOOD IDEA ! rm -f filename(s) Do not complain if the file does not exist. Also overrides the -i flag. You cannot remove a directory unless it is empty. { To keep your files from accidental loss in csh alias rm 'rm -i' set noclobber in ksh alias rm="rm -i" set -o noclobber Do similar aliases for 'mv' and 'cp' if you wish. See below. (Note that not everybody likes this alias rm 'rm -i' idea. alias del 'rm -i' has an advantage if your alias goes missing or if your rm-happy habits grip you on another system.) } cp == This command copies files. cp file1 file2 copies the file1 to a new file file2 cp /vmunix /home/allen copies the file /vmunix to another directory, so making the new file /home/allen/vmunix. cp -r mydir mynewdir With the -r flag, copies directories too. mv == This moves and renames files and directories. mv file2 file3 renames a file (or directory) mv /home/allen/vmunix /tmp moves a file (or directory) pwd === Unix has directories to help you organise your files. pwd tells you your 'current working directory' or CWD An example directory name would be '/home/allen/handy/scripts'. The full name starts with a slash "/" and directory names are separated by slashes. (This is different from DOS, which uses the backslash \.) cd == Changes your CWD. cd takes you to your 'home directory' cd /usr/lib takes you to the dir '/usr/lib' cd .. takes you to the parent directory (eg /usr) cd / The root directory - there are none above this. Your 'home directory' is the one you log in to. It is defined in the file /etc/passwd (unless some special network system is in use and then it is found in the output from 'ypcat passwd'). whoami ====== This tells you your id name, eg 'allen' So you can type grep `whoami` /etc/passwd to see your home directory and some other things. allen:!:5037:1:P. Allan:/u/allen:/bin/csh ( The command 'grep' is for searching text files for strings. ) ( The backquotes `` will be covered later. ) The ! in the second field tells the computer that the encrypted password is held in another file. This makes it a bit harder for dishonest people to get hold of your password. passwd ====== This should be one of the first commands you use when given a new id. It sets your password, and can do a few other things too. passwd You will be prompted for the old password (if any) and will be asked for the new password twice. (length = 8 characters) EXAMPLE % passwd Changing password for "allen" allen's Old password: allen's New password: Enter the new password again: % (There are restrictions on what you can choose, but they are in /etc/security and so not readable!) Good passwords are hard to guess. I suggest always using all 8 characters and including at least one of [a-z], [A-Z] and [0-9]. There are programs around that guess many passwords, so unwise passwords include any based on names, real words, the Unix id, the hostname...... man Important ! Use it often. === This is the traditional Unix command for finding things out. You may prefer to use the InfoExplorer which does the same thing. 'man ls' Tells you a lot more about ls than I did above. (correct use of 'man' requires you to have a variable MANPATH set so if it doesn't work for you that is probably the explanation: use InfoExplorer instead until you have fixed the problem!) I recommend reading the man page on at least every command in this article, and new ones as you meet them. If you cannot find man pages on your system, ask your administrator. exit ==== This stops your session or just one window (if in a window). When using a Graphical Interface a menu accessed with the mouse will be used to end your whole session. The window schemes are not part of Unix, and vary greatly between manufacturers. 'exit' detects jobs that are stopped (by CTRL-Z for instance) because they would die if you used exit. Type exit a second time if you are happy with that. Running background jobs continue. chmod ===== Every file and directory has a 'filemode'. You can see this with 'ls -l'. cd ; ls -la (The ; separates two commands on the same line.) That shows you the modes of the files in your home directory. Included in the list should be two entries called '.' and '..'. These are: . the CWD .. the parent of the CWD You will see the owner (your id probably for all except ..) and the group (perhaps 'staff'), a file size, a file change time/date and at the start of the line a set of characters giving the filemode. EG total 312 drwxr-x--- 16 allen staff 1024 Oct 21 14:07 . drwxr-xr-x 21 sys sys 512 Oct 13 16:25 .. -rwxr----- 1 allen staff 896 Oct 20 14:44 .cshrc drwx------ 2 allen staff 512 Oct 6 08:51 .elm -rw-r--r-- 1 allen staff 59 Oct 17 13:59 .exrc -rwxr----- 1 allen staff 461 Oct 17 12:18 .login -rwxr----- 1 allen staff 254 Sep 22 08:33 .profile drwxr-xr-x 2 allen staff 512 Oct 18 15:02 cases drwxr-xr-x 3 allen staff 512 Oct 21 14:07 doc drwxr-xr-x 5 allen staff 512 Oct 11 13:54 handy drwxr-xr-x 4 allen staff 512 Sep 23 09:23 info drwxr-xr-x 2 allen staff 512 Oct 5 15:07 man1 drwxr-xr-x 7 allen staff 512 Oct 20 14:45 sasfort A d in column 1 means the entry is a directory. A - means it is a file. See 'man ls' under the -l flag for other possible file types. There are 9 other characters in 3 sets of 3. Columns 2-4 are for the file's OWNER. (named later on the line) (eg allen) Columns 5-7 are for the file's GROUP. (named later on the line) (eg staff) Columns 8-10 are for every other id. r means read permission is granted. w means write permission is granted. x means execute permission is granted. (=search permission if a directory) - means the permission is not granted. See 'man ls' under the -l flag for other possible modes. Notice that write permission on a directory allows you to remove files from that directory EVEN IF YOU DO NOT OWN THEM. chmod 640 file1 sets file1 to -rw-r----- chmod 755 file2 sets file2 to -rwxr-xr-x chmod go= file3 sets file3 to -???------- (removes group & other) chmod -R go-w $HOME GOOD SAFE COMMAND: only you can write in your home dir. more, pg ======== These two commands allow you to look at a text file. more .cshrc more /etc/motd They allow you to page through a file and search for strings. When in a more or pg session type 'h[enter]' to see what commmands you can use. file ==== This makes an attempt to recognise files, classifying them as English Text or executable or data, ..... type 'file *' to see some examples. XLF: ascii text doc: directory handy: directory unix: ascii text .Xauthority: data or International Language text .cshrc: commands text .exrc: English text diff ==== This important command tells you the differences between two text files. It also tells you whether two binary files differ. wc == This does word counts etc. In fact it counts lines, words and characaters unless you tell it not to. % wc .login 6 7 461 .login My .login file has 6 lines, 7 words, and 461 characters. Of course the computer has a strange idea of what makes a "word". find ==== To list all files or directories named 'core' under the CWD . find . -name core -ls To remove all files named 'core' under the CWD. Do this:- core files are rarely useful because you need the matching source code to use them easily. find . -name core -exec rm {} \; To list all files or directories named SOMETHING'.core' under the CWD . find . -name '*.core' -ls To name all directories called 'man' unde /usr. find /usr -name man -type d -print The next 2 examples are useful. They remove temporary files you no longer want. find /tmp /var/tmp -mtime +3 -type f -user allen -exec rm {} \; find /var/preserve -mtime +8 -type f -user allen -exec rm {} \; The next example names files and directories which EITHER have changed in the last 2 days, OR have execute permission for the owner. find . \( -mtime -2 -o -perm -100 \) -print Checking what has changed in the last N days is a great way to find out why your computer isn't working any more ! pr == This adds titles and page numbers to your text files. Try it on a few example files. pr /etc/sendmail.cf | more (keep pressing enter to watch the file go past slowly) lpr ==== This prints. (Use lprm to cancel a print job. lprm will fail when a printer is remote. rlprm handles this. rlprm is available from Peter Allan) Text Editors ============ You have on your machine the editors 'vi' Bill Joy's editor, much liked by me 'emacs' A more recent graphical interface editor and 'e' IBM's own offering. I am describing here vi (and ex) briefly. Norton and Hahn's book highly recommended! Both vi and ex start by reading your file (if any) called $HOME/.exrc . You can put in it any commands you like to control your use of these editors. EG 'set nu' , 'map .....' ex: === ex is an old line editor and I do not expect anyone to use it interactively (except maybe to practise it). It is useful in shell scripts, and it forms a part of the vi editor. Option: A useful option is '-' that keeps the program 'silent' so that it does not report its actions. I use this in shell scripts (after debugging). It is important if the script is to be run in the background. Commands in ex are entered at the colon : prompt [return] The return key moves you to the next line. 1 A number (and return) takes you to the line of that number. s/one/two/ This is the most important ex command I think. As used here it changes the string 'one' into the string 'two', only once on the line. Other examples are: :1,10s/one/two/ change 'one' to 'two' on lines 1 - 10 :1,$s/one/two/ change 'one' to 'two' on lines 1 - the end :1,$s/one/two/g change ALL 'one's to 'two's (on lines 1 - the end) :1,$s/one/two/gc as above, but with a y/n choice for each change :1,$s/ //g change all ' 's to ''s (deleting all spaces) :1,$s/^ // delete single spaces at start of line :1,$s/ $// delete single spaces at end of line :1,$s/ /^M/g Type Ctrl-V Ctrl-M here. Changes all spaces to newlines. Special characters: . * ^ $ & [ ] \< \> (Use \ if you want to escape special meaning) . in the first string means any one character * in the first string means Zero or more of the preceding character ^ in the first string means start of line (The key is shift_6, this doesn't print correctly.) $ in the first string means end of line & in the second string means repeat the first string [ ] as for the shell section below \< beginning of word \> end of word (use as e\> to find 'e's at the end of a word.) :1,$s/ *$// delete multiple spaces at end of line :1,$s/^/ / add 6 spaces at start of line :1,$s/.*/mv & &.old/ & means the first string (use after ':re !ls') :1,$s/[a-z]/\u&/g Changes all [a-z] to [A-Z]. (\l& for lower case) g/search/ This gives another command to lines with the string 'search' :1,$ g/search/s/ /X/g Lines with 'search' get ' 's changed to 'X's. ! Shell escape ( :! ls ) set There are variables you can set :set nu display line numbers :set nonu No line numbers :set directory=/eprexe Choose a directory for temporary files. (/var/tmp is the default.) (useful if a disk gets full) :set all show you all settings re Read in a file in after the cursor. (only one file, * may fail ) :re !date Read a command's output after the cursor. vi Switch from ex to vi. w Write file to disk (can give filename) :w! Insist on writing, even if ex was started read-only. :w >> longerfile This appends to a file. :1,10 w filename Write only lines 1 - 10. q Quit. (q! to quit if file has been altered.) x Write file and quit. ( can also be done by 'wq') n go on to next file ( after 'ex file1 file2 file...') vi: ex mode === In vi you can do these ex commands (as well as those listed above). :w do one command (w) in ex mode :vi begin vi mode vi: command mode === These commands are typed without the return key. Most of them can be preceded by a number. (eg 10w means do 'w' 10 times,so move forwards by 10 words.) They take a bit of learning, but once you are familiar with them you can do almost anything in a few keystrokes. u 'u' undoes the last command. U 'U' tries to restore that one line to its original state. ~ Changes case of letters. a->A , A->a . Repeat last commmand done in command mode. ma 'm' is for mark. Here we mark (invisibly) a place as 'a', so we can return to it later. (with 'a) ZZ write and quit ( = :wq) Q begin ex mode (and stay in it) J Join next line to this one. ^G describe current file !(M)command Introduces shell command (ending in return). The text moved over is replaced by the command's output. (eg '!10+sort' sorts 10 lines into order (starting at current line)). Movement '(M)' is used in this note to denote a movement command. This is a big subject ! Cursor keys and hjkl move you one character left, down, up and right. ^u ^d up and down (scrolling in half-screenfuls) ^f ^b forward and back (jumping in whole screenfuls) G to start of last line 1G to start of line 1 (or other chosen number) w to next start of word e to next end of word b back to last start of word f F find (on current line) the character typed (EG fa forwards, Fa back) return, + down one line - up one line ^ to first non-blank character on line (^ is shift_6) n| n is a number , go to column n (=0 if n not given) (The | is the pipe character outside of vi.) 0 or | to start of line $ to end of line 'a to place previously marked with 'ma' '' to the last place moved to (try for example G'') ( ) to the start (end) of 'sentence' { } to the start (end) of 'paragraph' [iaArRsScCoO] (Insert/Overwrite/Change) These commands (except 'r') involve string input which is ended with 'escape'. The letters have the following meanings: i insert before cursor. a insert after cursor. A insert at end of line. r replace one character with the one typed next R replace text with the string typed next s substitute string for the current character S substitute string for the current line c(M) change (with movement command): change the text moved over C change to end of line o insert new line after current line (o is for open) O insert new line before current line Deletion (keeping the deleted text in a buffer) dd delete current line D delete from cursor to end of line (= d$) d(M) deletes from cursor to new position after move x deletes current character X deletes character to the left Yank yy copies the current line into a buffer Y = yy y(M) copies into a buffer "a(a yank) copies into a named buffer ('a') Put p put after current position (same line or next line) P put before current postion "3 from chosen buffer (3 buffers ago) "a from chosen buffer ('a') Examples using deletion, yanking and putting: xp swap 2 characters ddp swap 2 lines 5yyP copy 5 lines Dpj$p copy the ending of a line to the next line Screen control z. position current line in screen centre z- position current line at screen bottom z+ position current line at screen top ^L redraw screen vi: === Searching a file ( [] . * ^ $ have the meanings you'd expect.) / a search from the current position to the end and from the beginning to the current position again ? like / except it searches backwards n find next in direction of the search N find next against the direction of the search (so '?string' 'N' will look down the file) Can be made case-insensitive (a=A) by 'set ignorecase'. map You can define a key to have the meaning you want. This is useful to save typing (and to correct the meaning of the cursor keys on some keyboards). EG map t JJJj Join 3 lines and go down one map ^V(left arrow) h Make left arrow key move you left Compiling etc ============= f77,xlf These are two names for the IBM Fortran 77 compiler. f90 This is the NAG Fortran 90 compiler {really a C translator}. Fortran source files should have names ending in '.f'. Object files have names ending in '.o'. ld A linking or binding program to join modules. You would normally use the compiler to do this. make A configuration tool - used to organise compile and link steps. See my example Makefile. The file consists of a list of 'targets' with the input steps and actions to perform. This means that if you have several source files and you change one of them you can compile only the new source and link it to the unchanged objects. The Makefile needs tabs (not spaces) at the start of some lines. sccs A code control tool, keeps each version of the files edited. (eats up disk space !) cc A C compiler. File Transfer ============= mail: This is a fast and convenient way to send files, provided they are not too large. ftp: To get a file from the mainframe to the workstation: % ftp hdimvsp > get 'irs970.euref.pds(alpc)' alpc.f > bye Other ftp commands include 'put' and '?' (for help). With 'mget' there is filename globbing. Getting files onto the workstation with () in their names is a nuisance because of the meaning the shell gives to (). When transferring files between workstations it is easier to use the command: > get alpc.f No name given for the new file => same name will be used. Provided you are happy with the security of your unix id you can keep a .netrc file in your home directory, holding the password to another (EG mainframe) id. This saves you from typing your password for ftp sessions. If .netrc is not used, the ftp program prompts you for the username (type return for the same username) and password. ftp only works to id's that have a password. % ftp hdimvsp Connected to hdimvsp.kfk.de. 220-FTPSERVE IBM MVS V2R2 at hdimvsp.kfk.de, 15:11:48 on 11/03/94 220 Connection will close if idle for more than 5 minutes. Name (hdimvsp:allen): irs0c7 331 Send password please. Password: 230 IRS0C7 is logged on. ftp> get 'irs0c7.pma.text(sasic2)' sasic2.f 200 Port request OK. 125 Sending data set IRS0C7.PMA.TEXT(SASIC2) FIXrecfm 80 250 Transfer completed successfully. 60516 bytes received in 0.275 seconds (214.9 Kbytes/s) ftp> bye 221 Quit command received. Goodbye. The mode of ftp can be varied between text (ascii) and binary. Sending binary files in text mode results in rubbish. You can get a bit of unix-emulation out of the mainframe by using commands inside ftp such as : cd 'irs970' cd euref ls help cd .. "Anonymous ftp" is a means by which people can make their files available for anyone else to collect. To fetch a file you use the login name 'ftp' and as a password use your mail address. Many products (both useful and silly) can be obtained like this from computers offering the service. Obviously it would be possible for someone to plant a harmful program there. ftp relies on another program ftpd existing on the target host. So you can run ftp on a PC to a unix workstation, but not usually in the other direction. uuencode, uudecode uuencode can convert binary files into printable character files so they can be sent through the mail, and uudecode can convert them back. ln == Using 'ls -l ' you will see something like this -rw-r--r-- 1 allen staff 12386 Oct 21 15:24 XLF The first number is the LINK COUNT. It shows that the file XLF has only one name. I can give the file a second name like this: ln XLF also.xlf ls -l XLF also.xlf -rw-r--r-- 2 allen staff 12386 Oct 21 15:24 XLF -rw-r--r-- 2 allen staff 12386 Oct 21 15:24 also.xlf This means that if I remove one of these links the other remains. If I change one (with an editor, say) they BOTH change. You can tell they are both links to the same file because they have the same INODE NUMBER (ls -i, and find -inum). This is an index arrangement used by the computer to get from the filename in the directory to the parts of the disk where the file is actually stored. % ls -li XLF also.xlf 38999 -rw-r--r-- 2 allen staff 12386 Oct 21 15:24 XLF 38999 -rw-r--r-- 2 allen staff 12386 Oct 21 15:24 also.xlf 'ln -s ' makes a SYMBOLIC LINK. This is merely a pointer to the other filename as can be seen with 'ls -l'. Removing the 'pointed-to' file makes the symbolic link useless. They are good for giving alternative names to directories. (See /bin and /u.) They can also point across to another filesystem, so saving space on this one. A lot of the unix commands have this arrangement. The file /dev/null is a null file that : if you write to it, your writing is thrown away if you read from it you get an end-of-file 'ln -s /dev/null fort.40' therefore makes a file 'fort.40' that you can write to as much as you like but it won't waste space on your disk. which commandname(s) EG which ex This will tell you which 'ex' command you will execute if you call the command without a pathname. There are commands called 'ex' in /usr/bin and /usr/ucb. If you get problems with a command (including a new script of yours) it might be because the name clashes with a name found earlier in your path. crontab This command allows you to execute commands at the time of your choice, especially regular commands. This is my crontab file produced with the command 'crontab -e'. I am using crontab here to remove unwanted files and save disk space. It allows me to make temporary files and directories called 'core' knowing they will be removed automatically. # remove (useless) core files every night at 23:15 15 23 * * * find . -name core -exec rm -r {} \; 2> /dev/null # remove my files unchanged for 3 days under /tmp and /var/tmp 30 23 * * * find /tmp /var/tmp -mtime +3 -type f -user allen -exec rm {} \; 2> /dev/null # remove my files unchanged for 8 days under /var/preserve 45 23 * * * find /var/preserve -mtime +8 -type f -user allen -exec rm {} \; 2> /dev/null You could get it to send you mail on your birthday, warn you of Friday 13th's and do other amazingly useful things. The command 'at' is similar, but for one-off timed commands. SOME MORE TO READ ABOUT These include text processing which can save you quite a bit of high-level programming. Also monitoring of disk space and printer queues, file conversion,... awk, cat, colrm, cut, dd, df, du, echo, egrep, elm, fgrep, finger, grep, gzip, head, limit, lpq, lprm, mkdir, mesg, nice, nl, paste, read, rmdir, rsh (disabled here ?) , script, sed, strip, sum, tail, tar, touch, umask, w, write, xsend perl is a hot (newish) language worth learning. ############################################################## The Shell ========= The output from the command grep `whoami` /etc/passwd should resemble this: allen:!:5037:1:P. Allan:/u/allen:/bin/csh ^^^^^^^^---------this is my login shell The entries in /etc/passwd are separated by colons ":". The last entry is the "login shell" or the program you run automatically when you log in. There is a choice of shells such as: /bin/csh The C shell /bin/sh The Bourne shell /bin/bsh The Bourne shell (again) /bin/ksh The Korn shell You choose with the command 'chsh' or 'passwd -s' which you want. The Korn shell is an extended version of the Bourne shell, and is the standard one under AIX. This author likes the C shell because of the history substitution he is used to, but the Korn shell has inline editors and you probably want to use that. The important point about your shell is that it is the program that reads your commands. If I do the following ( % is the prompt): % alias rm 'rm -i' % ls file file.old list list.old prog prog.old wigwam.old rm *.old rm: Remove file.old? y rm: Remove list.old? y rm: Remove prog.old? y rm: Remove wigwam.old? y I have first aliased the command 'rm' to be 'rm -i'. This is the safe advice I gave earlier to prevent accidental loss. The shell is the thing that remembers that, and when I later type 'rm' knows that it should prompt me for each file I am about to remove. The shell is also the thing that recognises the special symbol * and looks for all the files called *.old. So in the above example the shell changes the command to rm -i file.old list.old prog.old wigwam.old and only then executes the 'rm' command. When a command fails to do what you expect it could be because the shell has altered it. The shell also had to find the command 'rm'. It searches a path which is defined in a variable. Getting your path set correctly is important for security and to make sure that all the commands you want are available to you. When you have variables you always want set, it is a good idea to set them in a file executed as you log in. For C shell: .cshrc For Korn shell: .kshrc This is a path set in the C shell. You can see your variables with the command 'set'. (LINES ARE BROKEN HERE FOR CONVENIENCE IN PRINTING - NOT REALLY) path (/usr/ucb /usr/bin /etc /usr/sbin /usr/bin/X11 /sbin /usr/lib /usr/lpp/x3270/bin /usr/local/kfk/bin /u/allen /u/allen/handy/bin /u/allen/handy/scripts /u/allen/sasfort/sasbin .) It was set in the file .cshrc with the line set path=(/usr/ucb /usr/bin /etc /usr/sbin /usr/bin/X11 /sbin /usr/lib /usr/lpp/x3270/bin /usr/local/kfk/bin ~ ~/handy/bin ~/handy/scripts ~/sasfort/sasbin .) Sadly, the Bourne and Korn shells set their variables differently from the Csh. Directories are separated by colons. The word 'set' is not used. PATH=/bin:/usr/bin:/usr/local/kfk/bin:. In the C shell example above, the user wanted some directories listed in relation to his own home directory. The tilde (~) is the C shell symbol for a home directory. You can see it has been expanded to /u/allen in the path variable. In both examples above the current directory is included in the path, BUT IT COMES LAST. You should not include a . in your path near the beginning because it might give somebody else an opportunity to break into your id. The shell has a number of other handy features..... Standard Output redirection: > date date > my.date.file The date command alone tells you the time and date. With the > sign it doesn't ! But it does make a new file containing the output from 'date'. This will create a new file or overwrite an existing one. Setting "noclobber" in csh or ksh will prevent this from overwriting existing files. Appending from standard output: >> date > my.date.file hostname >> my.date.file whoami >> my.date.file echo This should be quite clear. >> my.date.file echo 'This should be quite clear.' >> my.date.file Standard Input Redirection: < colrm 17 19 < my.date.file This gives the file as input to the colrm command. Standard Error Redirection : 2> (with bsh and ksh) cat /etc/fstab : this should fail with permission denied. cat /etc/fstab > stdout cat /etc/fstab 2> stderr Now which file holds the error message ? stderr Error messages come on a different output stream. Here Documents: << Run this script (scripts need execute permission: "chmod u+x filename" ) ##### next line is the beginning ##################### #!/bin/sh myDFfile=/tmp/df.$$ df > $myDFfile cat $myDFfile echo '.... press enter .............' read aaa ex - $myDFfile << ENDMARKER 1,$ s/[a-z]/\u&/g x ENDMARKER cat $myDFfile ###### previous line was the end ##################### Notice how the '<< ENDMARKER' (or whatever word you choose) allows you to define input on the lines before the ENDMARKER appears. Here 2 lines of input are given to the command 'ex'. {ENDMARKER must be at the start of the line.} Pipes: | du -s -k * | sort -n Output from 'du' is given to 'sort' as input. This is amazingly useful. You will soon wonder how you managed on the mainframe. Return codes, && and || Commands usually return "return codes" to state whether they worked, or how they failed. These symbols come between commands. && means continue if the last command succeeded. || means continue if the last command failed. ls && date ls || date rmdir /etc/WHATDIR && date rmdir /etc/WHATDIR || date Metacharacters : Some of these these allow expansion ("globbing') of filenames * Acts as a part of a filename (but not an initial '.') ? Acts as a single character in a filename (but not an initial '.') [] Acts as a single character, depending on the contents..... file[12345] : file1 file2 file3 file4 file5 file[1-5] : same as above file[a-cA-C]: filea fileb filec fileA fileB fileC ; End of commmand - like a newline () This produces a second shell. In the example below the first shell talks to the second through a pipe. cd /home/allen tar cbf 20 - handy | ( cd ; tar xvbBpf 20 - ) Why is this different from a "cp -r" command ? Quotes '' Single Quotes These protect all the contents from being read by the shell. echo ' one two three * ' is different from echo one two three * "" Double Quotes As above except that a few (4 ?) symbols still get read by the shell. $` Can't remember the others just now! \ Backslash Quotes the next single character IF it has a special meaning. Also used in other commands such as vi and grep. Notice how I used it in my 'find' commands in 2 ways: to keep the () from being read by the shell 'find' uses the () in its OR condition to keep the ; from being read by the shell 'find' uses the ; to mark the end of the '-exec' command It can be used to quote 'return' characters too, if they come immediately after the \, and let you type a command over several lines. EG f77 -v -qextchk -qlistop -qnomaf -qrndsngl -qspnans -qxflag=alias \ -bloadmap:LOADMAP -bnso -bI:/lib/syscalls.exp \ -NB100 -NC2048 -ND8192 -NN1048576 -NP500 -NA16384 -NT65536 * `` Backquotes Is substituted by the output of the command. EG myvar=`date` Foreground and Background and other controls command & runs command in the background - You don't wait for it to finish - useful on long commands that don't need your input. jobs Tells you about current background and suspended jobs. %1 Brings job 1 back to the foreground. ^Z This is CONTROL-Z and it suspends the current foreground job. bg %1 Restarts a suspended job in the background. ^D End of input - for Bourne shell and commands reading standard input. ^C Terminates current foreground job. ^U Clears current input line - quickest way to erase typing errors. ^Q,^S ^S=SLOW ; stop the screen scrolling ^Q=QUICK ; start it again ^V Prevent the next character from having its normal action. EG ^V^M allows you to type a "return" character in your text. ^V is used in this way elsewhere, including vi and wordperfect. ps Tells you about running and stopped processes. Needs arguments. kill Sends signals to processes - to kill them for instance. (Best way to check success is to repeat the kill. the second try will fail if the first succeeded.) Shell Scripts (These need execute permission chmod u+x, chmod 750 or similar) I am assuming you choose the Bourne or Korn shells for scripts. I do this and reserve the C shell for interactive use. ====================================================== #!/first_line The first line of the shell script normally selects the shell program to run. EG '#!/bin/sh' or '#!/bin/ksh' There can be options, of course, such as '#!/bin/sh -x' Variables. These are very important in shell programming. They begin with a $ sign. Legal characters in the names are [A-Za-z0-9_], "." is NOT allowed. You can set and use your own variables like this. myname='the string I choose' (The $ is NOT used when setting.) echo $myname echo $mynameX : NOTHING! 'mynameX' is not set (May cause errors as some commands need an argument. e.g. test) echo ${myname}X : OK = 'the string I chooseX' echo $myname$myname : OK = 'the string I choosethe string I choose' echo "$mynameX" : OK 'mynameX' is not set so = "" Read-only parameters $0 The name of the script. $1 (n) The 1st (nth) argument given to the script at run time. $* All the arguments 1-n. $@ As $*, but whitespace in an argument is preserved. $# The number of arguments given to the script at run time. $$ The process number of the shell program as it runs. This is useful for making output files with names that are likely to be unique. There are further environment variables such as $USER and $HOME that you can see by typing 'sh -c set'. test Normally used in an "if" command : if ( test "$myname" = ZZZ ) then '[' is another name for the 'test' command, so you often see [ "$myname" = ZZZ ] if ( ) then target1 else target2 fi ( 'if' backwards ! ) lc=0 while ( test "$lc" -lt 11 ) do { lc=`expr $lc + 1` echo loop counter is $lc } done for file in 1 2 3 *.txt { more $file } read myvar echo $myvar myfunc () You can make your own functions inside a shell script. { ls $1 } daynum=`date +%w` case $daynum in 0) echo sunday;; 1) echo monday;; 2) echo tuesday;; 3) echo wednesday;; 4) echo thursday;; 5) echo friday;; 6) echo saturday;; esac It is legal for one script to call another. It is also legal to use recursion as one script calls itself, but this needs to be controlled as you would expect. "getopts" is a command useful for options in scripts. Errors in scripts are usually reported with the name of the script and the name of the offending command. How many of your operating system's commands are actually scripts ? And what can you learn from them ? Problems ======== Is my command just taking a long time ? Is it waiting for me to type something ? (such as CTRL-D) (try 'cat') Has this worked before ? When did it last work ? What has been changed recently ? Do other id's have the same problem ? (Or have I set my environment badly ?) Am I executing the right command ? (path variable, which shell am I using ?) Have I recorded accurately what I did and what happened ? Where are my error messages going ? ('script' command) Is there a way to test PART of the action I tried ? Have I read the manual entries for the command ? Do all relevant files exist ? With suitable filemodes ? What if I log out and log in again ? And for those who can interpret the output..... Do dbx, trace, truss, or par reveal anything useful ? The administrator and his role ============================== There is a priviledged id called 'root' which can do many things not open to most users. None of your files are safe from reading and writing by this superuser. It is good policy to have only 1 or 2 trusted and skillful people using the root id, and only when they need it; to prevent the severe consequences of mistakes made as root. You should try hard not to annoy your administrators. Among other things the administrator will: Choose and install (or reject) new hardware and software Monitor the security situation: catching people who try to crack passwords, and who search for files with particular filemodes and who try other devious tricks Look after resources such as printers and disk space Make backups of files, from which he can recover if there is a disaster of any kind. (The top disaster is theft of all hardware including the tape drive containing the just-completed backup.) Advise users of planned changes (So keep up to date with reading your mail.) If there is a problem such as a printer out of action, or a disk is full, or a disk seems to be missing, you can tell the administrator about it politely. He can reset your password if you forget it. You should be aware of his back-up arrangements so that if you lose/damage a file of yours, you know what you can expect from him. (If the backups are done overnight (at 1 o'clock) he will not have a copy of today's new file.) It is also good to know how long each tape is kept, as this tells you the time during which you must report a lost file. You may also find him helpful with your unix questions, AFTER you have looked in the manual/infoexplorer/answerbook and any notices your administrator publishes : "Frequently Asked Questions". He may complain to anyone "hogging" resources like disk space or CPU time. You might find him asking you to run processes at a different time or on a different machine. He may ask to move some files to tape to free disk space. -------------------------- And a poem for the admins.... /usr/games/admin ================ (All trademarks acknowledged.) I was working in the office and the time was getting late when they told me of the problems that I must investigate as the PostScript wasn't scripting and the DAT drive had no lights I could see as it was starting it would be one of those nights. There were deadlocks in my mutex, all my pointers missed the node. There were children with no parents, and I think they wrote the code. My recursion never ended, nothing ever got to free(). My machines were going crazy, it was either them or me. Power cycles were forbidden as the uptimes had to rise, all the manuals were hidden far away from searching eyes. Once again I chose a printer, but I really had no choice, so I telephoned the helpline and they recognised my voice. With the server not responding and the network up the creek (someone changed the DNS box in the middle of last week) I was reaching for the media for a spooler reinstall when I saw a station crashing and it wouldn't boot at all. But the boot disk wasn't altered, all the files were still in place. I said this to the computer and it laughed right in my face. In the end I got it started, but it only mounted root. There were few commands to play with but it had begun to boot. Forty minutes or so later, with my inodes mostly cleared, I was rather short of data, over half had disappeared. But at least I had my backups, surely they were home and dry, write-protected, cycled, labelled - so my confidence was high. I'd forgotten that the tape drive was in a disgusting mess. All my diagrams were PostScript and I hadn't DPS. There's an answer to these crises TFM does not relate: Go and hide inside the libraries of a distant friendly state.