Welcome, Guest. Please Login
 
  HomeHelpSearchLogin FAQ Radified Ghost.Classic Ghost.New Bootable CD Blog  
 
Pages: 1 2 3 ... 5
Send Topic Print
Using the Unix/Linux shell command-line (Read 51127 times)
Rad
Radministrator
*****
Offline


Sufferin' succotash

Posts: 4090
Newport Beach, California


Back to top
Using the Unix/Linux shell command-line
Jul 14th, 2009 at 4:55pm
 
The following are notes I made for myself during my study of how to use the Unix/Linux shell command-line (a study which is still in progress, so there will be more):

Linux.Shell

! = not
? = single-character wildcard
* = multi-character wildcard
cat = displays entire contents of text-based file
more = displays single page of text-based file, either 1 line (enter key) or 1 page (space bar) at a time.q = quit
[___] square brackets displays single character of any characters contained within square brackets
ls = lists/displays contents of directorys
ls -F indicates directories with a slash beside listing
ls -l files details (long listing)
ls -r recursive, lists contents and includes files in all subdirectories
ls -C arrange files/directories in Columns
ls -d * = list all files but no directories
ls [a-z] * lists all files beginning with a-thru-z
head = displays first 10 lines of a text-based file
tail = displays lat 10 lines of text-based file
can use any of commands to display contents of MULTIPLE files
pwd = Print Working Directory, tells us where we are
sh = Bourne shell = first shell
bash = Bourne Again SHell = usually comes with Linux = free
ksh = Korn SHell (by David Korn) = not free/open
zsh = Z SHell
csh = C SHell
possible to write scripts with bourne shell as interptrter  = shell scripts
therefore some people think of the bourne shell as a programming language, or a scripting language, not just an interactive shell to start programs
root = user name with full system priviledges
uname = details about what machine currently using (e.g. linux)
uname -a = more info about machine you're using
ps = tells name of all programs currently running, including what shell you're running
cal = calendar
cal 2009 = whole year
cal 7 2009 = calendar for july 2009
passwd = change password
syntax = program-name [options] [arguments]
. = current directory
.. = parent directory
man = manual pages
man prog-name e.g. man cal [use page-up, page-down] q to quit/exit
whatis - used to searc h for keywords on various functions
exit = log out or ctrl+d
/ = root directory (called "slash")
typing "cd" <enter> takes us to home directory.
~ is shorthand for user's home directory
characters okay to use for file names: a-z, A-Z, 0-9, .,@ _-:+
mv = move or rename file, e.g. mv binfile binfile1
rm = remove or delete file
file = makes a guess as to a file's contents, e.g. file * (for all files)
diff = compares 2 text files
cmp = compares 2 binary files, will also compare text files, but diff is much better tool. cmp only gives 1 line of output, telling us WHERE files differ.

04-06 > Copy, Moving & Renaming Files
cp = copy e.g cp file1 file2 creates a duplicate of file1 named file2 in same directory
can list multiple files to copy .. e.g cp file1 file2 file3 new_directory
if last argument = a directory, all files listed are copied there
e.g. cp *.html ~/gfx copies all hyml files into gfx directory
mv = move
mv is also used to rename files .. e.g. mv file1 file2 renames file1 as file2
to a different directory .. mv *.html /tmp moves all html files to tmp directory

04-07 > Deleting Files
rm = remove
warning .. unix/linux does not undelete, rm carefully
read-only files will prompt for confirmation for delete .. need to answer y for yes to delete
no messages are provided to verify file(s) was(were) deleted .. as with most unix commands
cannot delete files you have no authorization for, or files that belong to someone else
rm -i (interactive) asks for y/n confirmation before deleting
-i option useful if you have, say, 20 files in a directory, and you wanna delete some of them.
can rm -i * and answer y to only those files you wanna delete. but if you forget the -i, then rm * will delete ALL files in that directory.
rm -f (force) option tells unix not to ask for any confirmation, and if file doesn't exist, display no message to that effect
rm -r (recursive) is dangerous. use with extreme caution. deletes all subdirectories and files contained in those subdirectories, which can be thousands of files
rm -r = very powerful tool

04-08 > Wildcards
any file that begins with a . (period) is a hidden file
hidden files are not displayed with regular ls
not match by wildcards ? or * .. e.g rm * will delete no hidden files
ls-a (all) WILL display hidden files
hiddden files are not special (just hidden)
usually config files are hiddden, such as .profile (log-in config) and .exrc (vi config) .. set-and-forget-type files.
but, if you wanna back-up all files in a directory, hiddrn files will not be backed up with normal methods.
or, if you wanna rmdir, have to first do a rm *, but this will not delete hidden files, so can't delete directory this way.

04-09 > Shortcuts for Working & Parents Directories
. = alias or shortcut for "the current directory"
e.g. mv /tmp/*.html . moves all html files from tmp directory to the current directory (here)
.. = alias for the "parent directory"
e.g. "mv *.html .." moves all html files to the parent directory, even if you don't know what the parent directory is.
"cd .." takes us to the parent directory. this will take us all the way back to the root ( "/" slash) directory.
both the . and .. aliases are treated as hidden files

04-10 > Relative vs Absolute paths
relative paths do NOT begin with a slash i.e. usr/rad/gnarly/july/cool_01.txt (if you were in the root / directory)
absolute paths DO begine with a slash /home/rad/public_html/cgi-bin/yabb2/YaBB.pl

04=11 > Working with Directories
mkdir = make new directory .. i.e. mkdir newdir
rmdir = remove directory .. i.e. rmdir newdir .. first navagate to its parent directory (cd ..)
directory must be empty for rmdir to work. first navigate back into directory, then rm * (or file names) or rm -i *
can also use "rm -r dir_name" (very powerful command, use with caution)
directories can also be renamed and moved using mv command
 
WWW  
IP Logged
 

Rad
Radministrator
*****
Offline


Sufferin' succotash

Posts: 4090
Newport Beach, California


Back to top
Re: Using the Unix/Linux shell command-line
Reply #1 - Jul 14th, 2009 at 5:01pm
 
MORE:

04-12 > Finding Files
find = powerful program .. long man page ..  i.e. "find top-directory" e.g. "find /home/rad/public_html"
e.g. find /home/rad -name filename.html -print
some, not all, implementatitions of unix require you to enter the -print option or nothing will show up on screen, even if some files are present and matched. the -print option is to print to screen, not a printer
important - when using wildcard characters, must enclose argument in single quotes. e.g. "find . -name '*.txt' -print"
doesn't hurt to enclose ALL file_name arguments in single quotes.
don't forget to include the -name option when looking for a file by name. (easy to forget .. e.g. "find . report.txt -print"
can also find files based on other criteria besides name, such as the file owner, permissions, type, date, size
can do other actions besides print-to-screen .. e.g > (an advanced manipulation of files) find . -type f -exec rm -i {} /;

04-13 > Archiving Files
tar gzip and compress commands/programs can all compress files
tar and cpio can write to offline media, such as tape drive
tar stands for "Tape ARchive"
to archive an entire directory, go to that directory and enter "tar cf newfilename.tar ." the period . represents the current/working directory
cpio = industrial-strength version of tar, trickier to use, more thorough.
cpio can specify more detailed options
e.g. "find . -name '*.html' | cpio -o > archive.cpio (or > dev/tape0, or > /dev/fd0)
in previous example, "." refers to present working directory, | = 'pipe' or pipe to, -o refers to output, > refers to 'send to'
in order to read FROM the cpio archive: cpio -i < /dev/fd0
in previous example, the < sign means 'read from' and -i stands for 'input'
can compress tar or cpio files
e.g "compress archive.cpio" .. becomes archive.cpio.Z
capital Z indicates a compressed file
to un compress use "uncompress archive.cpio.Z"
may also have to untar this file (tar -xf) if it was tarred.

05-01 Security
Unix is a multi-use O/S (which is why we need security), so we share our system with others.
Each user has separate access priviledges for system resources (e.g. files, directories, drives, devices, etc.)
"who am i" reveals the user name you're logged in as. "pts" is the terminal number.
"id" command also reveals user name, plus user number, plus group id, plus all groups we're a member of (e.g. sales group, managers groups, etc.)
"gid" (group id) = current group logged is as, while "groups" = all groups we're a member of.
"newgroup" command lets us switch to a different group that we're a member of
su = super user
can log-in as a different user if you know their user name and password, e.g. > "su newuser" which will present a prompt asking for newuser's password.
typing "exit" takes you back to original user log-in.
if you enter this log-in with a "-", such as "su - newuser" this will simulate full log-in as neweser, and take you to newuser's home directory.
 
WWW  
IP Logged
 
MrMagoo
Übermensch
*****
Offline


Resident Linux Guru

Posts: 1026
Phoenix, AZ (USA)


Back to top
Re: Using the Unix/Linux shell command-line
Reply #2 - Jul 14th, 2009 at 6:53pm
 
Lots of good info here on commands available in the Linux shell.  You've also got some regular expression stuff mixed in.  Maybe when you are done, if you don't mind, I'll re-arrange things a bit so the organization is more clear to someone looking at it fresh and integrate it into my Linux guide.  The guide is currently a little weak on helping new users navigate the shell.
 
WWW  
IP Logged
 
Rad.in.UbuntuVM
Radmeister
**
Offline


Rad in Ubuntu 9.04 Jaunty
Virtual Machine

Posts: 92


Back to top
Re: Using the Unix/Linux shell command-line
Reply #3 - Jul 14th, 2009 at 7:11pm
 
Oh yeah, certainly. Feel free to have at it as you see fit.

Not sure what you mean by regex stuff.

That was the stuff Nigel suggested I focus on when we discussed PHP. (I miss Nigel's wisdom & experience.)
 
WWW  
IP Logged
 
MrMagoo
Übermensch
*****
Offline


Resident Linux Guru

Posts: 1026
Phoenix, AZ (USA)


Back to top
Re: Using the Unix/Linux shell command-line
Reply #4 - Jul 15th, 2009 at 12:23am
 
Rad.in.UbuntuVM wrote on Jul 14th, 2009 at 7:11pm:
Not sure what you mean by regex stuff.

Rad wrote on Jul 14th, 2009 at 4:55pm:
? = single wildcard
* = multi-character wildcard
[___] square brackets displays single character of any characters contained within square brackets

This is regular expression stuff.  You use regular expressions to look for patterns in text. 

Regular Expressions are very powerful and useful.  The basics are easy to learn, but the advanced stuff isn't necessarily intuitive.  It's easy to get confused working with complicated regx.
 
WWW  
IP Logged
 
NightOwl
Radministrator
*****
Offline


"I tought I saw a puddy
tat..."

Posts: 5801
Olympia, WA--Puget Sound--USA


Back to top
Re: Using the Unix/Linux shell command-line
Reply #5 - Jul 15th, 2009 at 9:47am
 
@
MrMagoo

Quote:
Maybe when you are done, if you don't mind, I'll re-arrange things a bit so the organization is more clear to someone looking at it fresh and integrate it into my Linux guide.

Just a suggestion, if it's not already your intention--leave the *original* as is, and add a reference (link) to your edited version (later in this thread--or a separate thread--whichever makes sense) so one can compare the before and after.
 

____________________________________________________________________________________________

No question is stupid ... but, possibly the answers are Wink !
 
IP Logged
 

MrMagoo
Übermensch
*****
Offline


Resident Linux Guru

Posts: 1026
Phoenix, AZ (USA)


Back to top
Re: Using the Unix/Linux shell command-line
Reply #6 - Jul 15th, 2009 at 6:33pm
 
NightOwl wrote on Jul 15th, 2009 at 9:47am:
Just a suggestion, if it's not already your intention--leave the *original* as is, and add a reference (link) to your edited version (later in this thread--or a separate thread--whichever makes sense) so one can compare the before and after.

Yup, will do.
 
WWW  
IP Logged
 
Rad
Radministrator
*****
Offline


Sufferin' succotash

Posts: 4090
Newport Beach, California


Back to top
Re: Using the Unix/Linux shell command-line
Reply #7 - Jul 20th, 2009 at 12:46pm
 
From Nigel:

Quote:
Interesting trivia: the shell wildcards you asked about do have a special name, and they aren't regular expressions (as it happens the pattern characters it uses are chosen to be the same as those  used in regular expressions, but they aren't the same thing). Specifically, the UNIX shell wildcard expansion process has always been called globbing and the pattern language thus got called glob patterns or more informally just globs.

The WP entry on it at http://en.wikipedia.org/wiki/Glob_(programming) discusses this but doesn't quite get the etymology right. The name does derive from the name of a function, but that function was in the original Bourne Shell source code, which was written in a very early dialect of C during the mid-70's (it did eventually, much later, migrate to be available more generally but the name had already stuck by then). One of the aspects of the system at the time (the legendary PDP-11's) is that it was only a 16-bit machine, and so not only was C itself quite primitive at the time, to fit within the 64Kb memory constraints it was the habit of programmers to use short names for functions which generally were abbreviations or acronyms.

Although I don't recall Stephen Bourne saying this explicitly, it's reasonably clear what the source for the name is; mostly, the Bourne shell just processes individual strings, but the pattern-matching facility is unusual within the shell in that it generates multiple things as output. The name of the function reflected that this was what it did - it Generated a List of OBjects, and thus "glob" was a pretty natural choice of name for the pattern-expander function within the context of the Bourne Shell source.

[ It's also worth bearing in mind that at the time, the C compiler really did virtually *no* type-checking, so a function that output a list was quite a special thing. Naming it by what kind of data it returned was important because it was actually up to the programmer handling the return result to know that what came out of it needed to be treated quite carefully. This aspect of C got cleaned up eventually, and by about 1985 almost all C compilers were using decently strong type checking (and the ANSI standardization process had started to improve the language even further) but if you read the original 1970's source code it's more clear just how primitive the compiler was then and thus why most library functions were named the way they were). ]

So, the "glob" name really is more about the process of pattern expansion (turning one thing, the pattern, into multiple things) rather than denoting a particular pattern language or specific syntax, but the nature of language is such that "globbing" pretty much always refers to the particular style of wild-card pattern used by the Bourne Shell, which is much simpler and more direct than the regular expression system.

- Nigel
 
WWW  
IP Logged
 
Rad.in.UbuntuVM
Radmeister
**
Offline


Rad in Ubuntu 9.04 Jaunty
Virtual Machine

Posts: 92


Back to top
Re: Using the Unix/Linux shell command-line
Reply #8 - Jul 20th, 2009 at 12:50pm
 
05-02 File Protection

each file/directory has:
1. an owner (user name)
2. a group (group name)
3. permissions (access rights for various parties to the file/directory)

Three sets of permissions:
1. owner (access priviledges)
2. group (access priviledges)
3. everyone else (access priviledges)
Only the owner can change access priviledges (not group members)
"ls -l" shows permissions along with owner & group
r = read - for files, this gives pemission to view the file, using "cat" for example)
w = write (for files, this permission allows you to modify the file, using a text editor for example)
x = execute (for files, this only makes sense if the file is a program, such as binary program or shell script. Possible to set x permission for non-program files, but this yields no benefit)

For folders/directories:
r allows you to view contents of directory, using "ls" for example
w allows you to create or delete files in directory. 'w' setting is most important for directories, and can represent a security risk, because it is possible to remove w permission on the file, yet grant w access on the directory. To protect a file, need to "turn off the w bit" on *both* the file and directory
x (for folders/directories) grants access to (can go there) using "cd" for example
permission listing contains 10 slots
e.g. -rw-----r-- says this is a file (not a directory) that gives read and write permission to the owner, and read permission to everyone else
first slot indicates whether file (-) or directory (d)
next 3 slots indicate rwx permissons for the owner
next 3 slots indicate rwx permissions for the group
last 3 slots indicate rwx permissions for everyone else
the "root"user will never see the words "Permission denied" cuz root user has permission to do anything and everything
 
WWW  
IP Logged
 
Rad.in.UbuntuVM
Radmeister
**
Offline


Rad in Ubuntu 9.04 Jaunty
Virtual Machine

Posts: 92


Back to top
Re: Using the Unix/Linux shell command-line
Reply #9 - Jul 20th, 2009 at 1:49pm
 
05-03 - Changing File Permissions

chmod (CHange MODe) = program/command that changes file & directory permissions
Two methods:
1. Symbolic mode (uses typically used by beginners, easies to learn, cuz it uses letters, but can be long-winded)
2. Numeric mode (typically used by those more comfortable with chmod)
syntax > "chmod permissions filename(s)"
e.g. symbolic mode > "chmod u+w file1" adds (+) write permission (w) to owner (u) for file1
For symbolic mode (most common letters):
u = owner
g = group
o = other
a = all
+ = add
- = remove
= = set (when setting, only permissions specifed are set, others are turned off)
Specify who will get permission first (u, g, o, a), then the action (+, -, =), lastly the permission itself (r, w, x)
Can set multiple permissions in same command (separated by a comma), such as "chmod uo+w,u-rx file1"
Setting all 9 permissions using symbolic mode can be time consuming, so numeric mode is more common

Numeric mode (used by experienced Unix users):
3 digits sets all 9 permissions
r = 4
w = 2
x = 1
no permissions = 0
r+w = '6' (4+2)
r+x = '5' (4+1)
r+w+x = '7' (4=2=1)
e.g. "chmod 755 file1" gives full permissions (7) to owner of file1, plus r+x permissions (5) to group and everyone else
e.g. "chmod 640 file1" gives r+w permissions to owner of file1, plus read perm to group, and no permissions to anyone else
e.g. "chmod 070 file1" gives full permissions to the group associated with file1, with nobody else getting any permissions
common settings for data files: 444, 644, 664 & 666
common settings for programs 750, 755, 777 (not user very often)
common settings for directories: 755, 775, 777 (be careful about giving write perm for directories)
 
WWW  
IP Logged
 
Rad
Radministrator
*****
Offline


Sufferin' succotash

Posts: 4090
Newport Beach, California


Back to top
Re: Using the Unix/Linux shell command-line
Reply #10 - Jul 20th, 2009 at 2:04pm
 
05-04 - Changing File Ownership (much simpler than changing permissions)

chown (CHange OWNership) = command/program to change file ownership
syntax > "chown owner filename(s)"
e.g. > "chown spanky file1" gives ownership of file1 to spanky
e.g. > "chown ralph *.html" gives ownership of all html files to ralph
Ownership has nothing to do with where the file is located
Just because a particular file may be sitting in a directory you own, doesn't necessarily mean you own the file.
Only the owner can change permissions and ownership.
Once you change ownershipo, you can't change it back, cuz you no longer have ownership
 
WWW  
IP Logged
 

Rad.in.UbuntuVM
Radmeister
**
Offline


Rad in Ubuntu 9.04 Jaunty
Virtual Machine

Posts: 92


Back to top
Re: Using the Unix/Linux shell command-line
Reply #11 - Jul 20th, 2009 at 2:15pm
 
05-05 - Changing File Group (very simple)

chgrp (Change GRouP) = command/program to change group
e.g. "chgrp sales report*" gives certain permissions tp group named 'sales' for all files that begin with string 'report'
newgrp (NEW GRouP) = command/program to change your 'current' group ID, used when you get 'Permission denied' for files assigned to groups you are part of.
e.g > "newgrp marketing" changes your current group to 'marketing'
 
WWW  
IP Logged
 
Rad
Radministrator
*****
Offline


Sufferin' succotash

Posts: 4090
Newport Beach, California


Back to top
Re: Using the Unix/Linux shell command-line
Reply #12 - Jul 20th, 2009 at 2:34pm
 
05-06 - Dangerous Security Loophole

Occurs when a particular file has its write permission removed (444 or 666), but the directory that file sits in has write perm granted (666 or 777)
e.g. > dir1 > drwxrwxrwx
file1 > -r--r--r-- (read only)
everyone has access to directory that read-only file1 sits in
Here's how to modify file1:
copy file1 to another filename .. such as file2 (now you have permission of/for file2 cuz you are its owner)
modify file2 (since you now have ownership of file2. If you don't have write perm of file2, you can give yourself write perm, cuz you are its owner)
rm file1 (possible since dir1 is writable by anayone .. here is where the glitch lies)
after file1 removed, mv file2 file1 (to make file2 become file1)
change permission of file1 to original owner
difficult to tell file1 has been modified, only it "Last modified" time/date stamp will have been updated
Moral of the story > pay attention to directory permissions.
 
WWW  
IP Logged
 
Rad.in.UbuntuVM
Radmeister
**
Offline


Rad in Ubuntu 9.04 Jaunty
Virtual Machine

Posts: 92


Back to top
Re: Using the Unix/Linux shell command-line
Reply #13 - Jul 22nd, 2009 at 5:09pm
 
06-01 - Combining Programs (Pipes, Filters & Redirects)

"Output" = what you see printed on the screen by the program/utilty
Not all programs produce output
ls does produce output
cd, cp, mv, rm, mkdir all produce no output (except if there is an error)
output can redirected to either:
1. a file (easy to do)
2. another program (a little trickier)
">" sends output to a file .. can be extremely useful
e.g. "ls -l > listing.txt" sends the output of ls -l to a file named "listing.txt"
"|" (called a 'pipe') sends output to another program
who = program lists all users currently logged into the system
but no program tells number of users currently logged into the system
but we can combine two simple programs
wc = word count .. counts lines, words, and/or characters of any input it receives from programs or files specified.
e.g. "who | wc" gives output of wc program that was received from who program
Any two prgrams can be connected by the 'pipe' symbol, long as:
1. the first program produces output
2. the second program takes input
Upon command, Unix will start both programs and then 'pipe' the output of the first into the input of the second.
 
WWW  
IP Logged
 
MrMagoo
Übermensch
*****
Offline


Resident Linux Guru

Posts: 1026
Phoenix, AZ (USA)


Back to top
Re: Using the Unix/Linux shell command-line
Reply #14 - Jul 22nd, 2009 at 6:44pm
 
Rad.in.UbuntuVM wrote on Jul 22nd, 2009 at 5:09pm:
"Output" = what you see
ls does produce output
cd, cp, mv, rm, mkdir all produce no output (except if there is an error)

It is helpful to understand that the shell defines 2 different types of output, "output" and "error".  Both can be redirected (like you mention,) but they can be controlled separately.  The default for most programs is to send output to the shell's "stdout" and to send errors to "stderr".  stdout and stderr are usually both configured to go to the tty (or vtty, if you are logged in remotely) so the effect is that the output is printed on the screen.

You can redirect them independantly to any file.  What is important to remember here is that almost everything in Linux is a file.  Network sockets, hardware devices, and other tty's can be treated as files (and in many ways are treated as files by the kernel.)  So, your options aren't limited to regular text files.


Rad.in.UbuntuVM wrote on Jul 22nd, 2009 at 5:09pm:
output can redirected to...:
...another program (a little trickier)
...
"|" (called a 'pipe') sends output to another program

This can be as easy or as complicated as you want to make it, but is incredibly useful.  For example, say a directory has a LOT of files.  You are looking for a text file, but you aren't sure exactly what it is called.  You could use:

ls /the/directory | grep *.txt

to see all the files named something.txt in that directory.  This should make it easier to find the one you are looking for. 

Now, say there are a lot of text files in that directory, and the results scroll off the page.  You might add another pipe to view the output a page at a time:

ls /the/directory | grep *.txt | more

So, you can see, we can make this as complex as you want.  I've used something like this before:

ps aux | grep baduser | awk '{print $2}' | xargs kill -9

This would find all running programs owned by baduser, extract the process id (pid), then kill them (end them.)

Like I said, it can get as complex as you want.  I know some admins who work on huge (200 CPU) Solaris servers who would think that example is child's play (and it really is, if you break it down and learn the individual commands.)
 
WWW  
IP Logged
 
Pages: 1 2 3 ... 5
Send Topic Print