The following are notes I made for myself during my study of how to use the Unix/Linux shell command-line (a study which is still in progress, so there will be more):

Linux.Shell

! = not
? = single-character wildcard
* = multi-character wildcard
cat = displays entire contents of text-based file
more = displays single page of text-based file, either 1 line (enter key) or 1 page (space bar) at a time.q = quit
[___] square brackets displays single character of any characters contained within square brackets
ls = lists/displays contents of directorys
ls -F indicates directories with a slash beside listing
ls -l files details (long listing)
ls -r recursive, lists contents and includes files in all subdirectories
ls -C arrange files/directories in Columns
ls -d * = list all files but no directories
ls [a-z] * lists all files beginning with a-thru-z
head = displays first 10 lines of a text-based file
tail = displays lat 10 lines of text-based file
can use any of commands to display contents of MULTIPLE files
pwd = Print Working Directory, tells us where we are
sh = Bourne shell = first shell
bash = Bourne Again SHell = usually comes with Linux = free
ksh = Korn SHell (by David Korn) = not free/open
zsh = Z SHell
csh = C SHell
possible to write scripts with bourne shell as interptrter  = shell scripts
therefore some people think of the bourne shell as a programming language, or a scripting language, not just an interactive shell to start programs
root = user name with full system priviledges
uname = details about what machine currently using (e.g. linux)
uname -a = more info about machine you're using
ps = tells name of all programs currently running, including what shell you're running
cal = calendar
cal 2009 = whole year
cal 7 2009 = calendar for july 2009
passwd = change password
syntax = program-name [options] [arguments]
. = current directory
.. = parent directory
man = manual pages
man prog-name e.g. man cal [use page-up, page-down] q to quit/exit
whatis - used to searc h for keywords on various functions
exit = log out or ctrl+d
/ = root directory (called "slash")
typing "cd" <enter> takes us to home directory.
~ is shorthand for user's home directory
characters okay to use for file names: a-z, A-Z, 0-9, .,@ _-:+
mv = move or rename file, e.g. mv binfile binfile1
rm = remove or delete file
file = makes a guess as to a file's contents, e.g. file * (for all files)
diff = compares 2 text files
cmp = compares 2 binary files, will also compare text files, but diff is much better tool. cmp only gives 1 line of output, telling us WHERE files differ.

04-06 > Copy, Moving & Renaming Files
cp = copy e.g cp file1 file2 creates a duplicate of file1 named file2 in same directory
can list multiple files to copy .. e.g cp file1 file2 file3 new_directory
if last argument = a directory, all files listed are copied there
e.g. cp *.html ~/gfx copies all hyml files into gfx directory
mv = move
mv is also used to rename files .. e.g. mv file1 file2 renames file1 as file2
to a different directory .. mv *.html /tmp moves all html files to tmp directory

04-07 > Deleting Files
rm = remove
warning .. unix/linux does not undelete, rm carefully
read-only files will prompt for confirmation for delete .. need to answer y for yes to delete
no messages are provided to verify file(s) was(were) deleted .. as with most unix commands
cannot delete files you have no authorization for, or files that belong to someone else
rm -i (interactive) asks for y/n confirmation before deleting
-i option useful if you have, say, 20 files in a directory, and you wanna delete some of them.
can rm -i * and answer y to only those files you wanna delete. but if you forget the -i, then rm * will delete ALL files in that directory.
rm -f (force) option tells unix not to ask for any confirmation, and if file doesn't exist, display no message to that effect
rm -r (recursive) is dangerous. use with extreme caution. deletes all subdirectories and files contained in those subdirectories, which can be thousands of files
rm -r = very powerful tool

04-08 > Wildcards
any file that begins with a . (period) is a hidden file
hidden files are not displayed with regular ls
not match by wildcards ? or * .. e.g rm * will delete no hidden files
ls-a (all) WILL display hidden files
hiddden files are not special (just hidden)
usually config files are hiddden, such as .profile (log-in config) and .exrc (vi config) .. set-and-forget-type files.
but, if you wanna back-up all files in a directory, hiddrn files will not be backed up with normal methods.
or, if you wanna rmdir, have to first do a rm *, but this will not delete hidden files, so can't delete directory this way.

04-09 > Shortcuts for Working & Parents Directories
. = alias or shortcut for "the current directory"
e.g. mv /tmp/*.html . moves all html files from tmp directory to the current directory (here)
.. = alias for the "parent directory"
e.g. "mv *.html .." moves all html files to the parent directory, even if you don't know what the parent directory is.
"cd .." takes us to the parent directory. this will take us all the way back to the root ( "/" slash) directory.
both the . and .. aliases are treated as hidden files

04-10 > Relative vs Absolute paths
relative paths do NOT begin with a slash i.e. usr/rad/gnarly/july/cool_01.txt (if you were in the root / directory)
absolute paths DO begine with a slash /home/rad/public_html/cgi-bin/yabb2/YaBB.pl

04=11 > Working with Directories
mkdir = make new directory .. i.e. mkdir newdir
rmdir = remove directory .. i.e. rmdir newdir .. first navagate to its parent directory (cd ..)
directory must be empty for rmdir to work. first navigate back into directory, then rm * (or file names) or rm -i *
can also use "rm -r dir_name" (very powerful command, use with caution)
directories can also be renamed and moved using mv command

MORE:

04-12 > Finding Files
find = powerful program .. long man page ..  i.e. "find top-directory" e.g. "find /home/rad/public_html"
e.g. find /home/rad -name filename.html -print
some, not all, implementatitions of unix require you to enter the -print option or nothing will show up on screen, even if some files are present and matched. the -print option is to print to screen, not a printer
important - when using wildcard characters, must enclose argument in single quotes. e.g. "find . -name '*.txt' -print"
doesn't hurt to enclose ALL file_name arguments in single quotes.
don't forget to include the -name option when looking for a file by name. (easy to forget .. e.g. "find . report.txt -print"
can also find files based on other criteria besides name, such as the file owner, permissions, type, date, size
can do other actions besides print-to-screen .. e.g > (an advanced manipulation of files) find . -type f -exec rm -i {} /;

04-13 > Archiving Files
tar gzip and compress commands/programs can all compress files
tar and cpio can write to offline media, such as tape drive
tar stands for "Tape ARchive"
to archive an entire directory, go to that directory and enter "tar cf newfilename.tar ." the period . represents the current/working directory
cpio = industrial-strength version of tar, trickier to use, more thorough.
cpio can specify more detailed options
e.g. "find . -name '*.html' | cpio -o > archive.cpio (or > dev/tape0, or > /dev/fd0)
in previous example, "." refers to present working directory, | = 'pipe' or pipe to, -o refers to output, > refers to 'send to'
in order to read FROM the cpio archive: cpio -i < /dev/fd0
in previous example, the < sign means 'read from' and -i stands for 'input'
can compress tar or cpio files
e.g "compress archive.cpio" .. becomes archive.cpio.Z
capital Z indicates a compressed file
to un compress use "uncompress archive.cpio.Z"
may also have to untar this file (tar -xf) if it was tarred.

05-01 Security
Unix is a multi-use O/S (which is why we need security), so we share our system with others.
Each user has separate access priviledges for system resources (e.g. files, directories, drives, devices, etc.)
"who am i" reveals the user name you're logged in as. "pts" is the terminal number.
"id" command also reveals user name, plus user number, plus group id, plus all groups we're a member of (e.g. sales group, managers groups, etc.)
"gid" (group id) = current group logged is as, while "groups" = all groups we're a member of.
"newgroup" command lets us switch to a different group that we're a member of
su = super user
can log-in as a different user if you know their user name and password, e.g. > "su newuser" which will present a prompt asking for newuser's password.
typing "exit" takes you back to original user log-in.
if you enter this log-in with a "-", such as "su - newuser" this will simulate full log-in as neweser, and take you to newuser's home directory.

Lots of good info here on commands available in the Linux shell.  You've also got some regular expression stuff mixed in.  Maybe when you are done, if you don't mind, I'll re-arrange things a bit so the organization is more clear to someone looking at it fresh and integrate it into my Linux guide.  The guide is currently a little weak on helping new users navigate the shell.

Oh yeah, certainly. Feel free to have at it as you see fit.

Not sure what you mean by regex stuff.

That was the stuff Nigel suggested I focus on when we discussed PHP. (I miss Nigel's wisdom & experience.)

Rad.in.UbuntuVM wrote on Jul 14^th, 2009 at 7:11pm:

Not sure what you mean by regex stuff.

Rad wrote on Jul 14^th, 2009 at 4:55pm:

? = single wildcard * = multi-character wildcard [___] square brackets displays single character of any characters contained within square brackets

This is regular expression stuff.  You use regular expressions to look for patterns in text. 

Regular Expressions are very powerful and useful.  The basics are easy to learn, but the advanced stuff isn't necessarily intuitive.  It's easy to get confused working with complicated regx.

@ MrMagoo


Quote:

Maybe when you are done, if you don't mind, I'll re-arrange things a bit so the organization is more clear to someone looking at it fresh and integrate it into my Linux guide.

Just a suggestion, if it's not already your intention--leave the *original* as is, and add a reference (link) to your edited version (later in this thread--or a separate thread--whichever makes sense) so one can compare the before and after.

NightOwl wrote on Jul 15^th, 2009 at 9:47am:

Just a suggestion, if it's not already your intention--leave the *original* as is, and add a reference (link) to your edited version (later in this thread--or a separate thread--whichever makes sense) so one can compare the before and after.

Yup, will do.

From Nigel:


Quote:

Interesting trivia: the shell wildcards you asked about do have a special name, and they aren't regular expressions (as it happens the pattern characters it uses are chosen to be the same as those  used in regular expressions, but they aren't the same thing). Specifically, the UNIX shell wildcard expansion process has always been called globbing and the pattern language thus got called glob patterns or more informally just globs. The WP entry on it at http://en.wikipedia.org/wiki/Glob_(programming) discusses this but doesn't quite get the etymology right. The name does derive from the name of a function, but that function was in the original Bourne Shell source code, which was written in a very early dialect of C during the mid-70's (it did eventually, much later, migrate to be available more generally but the name had already stuck by then). One of the aspects of the system at the time (the legendary PDP-11's) is that it was only a 16-bit machine, and so not only was C itself quite primitive at the time, to fit within the 64Kb memory constraints it was the habit of programmers to use short names for functions which generally were abbreviations or acronyms. Although I don't recall Stephen Bourne saying this explicitly, it's reasonably clear what the source for the name is; mostly, the Bourne shell just processes individual strings, but the pattern-matching facility is unusual within the shell in that it generates multiple things as output. The name of the function reflected that this was what it did - it Generated a List of OBjects, and thus "glob" was a pretty natural choice of name for the pattern-expander function within the context of the Bourne Shell source. [ It's also worth bearing in mind that at the time, the C compiler really did virtually *no* type-checking, so a function that output a list was quite a special thing. Naming it by what kind of data it returned was important because it was actually up to the programmer handling the return result to know that what came out of it needed to be treated quite carefully. This aspect of C got cleaned up eventually, and by about 1985 almost all C compilers were using decently strong type checking (and the ANSI standardization process had started to improve the language even further) but if you read the original 1970's source code it's more clear just how primitive the compiler was then and thus why most library functions were named the way they were). ] So, the "glob" name really is more about the process of pattern expansion (turning one thing, the pattern, into multiple things) rather than denoting a particular pattern language or specific syntax, but the nature of language is such that "globbing" pretty much always refers to the particular style of wild-card pattern used by the Bourne Shell, which is much simpler and more direct than the regular expression system. - Nigel

05-02 File Protection

each file/directory has:
1. an owner (user name)
2. a group (group name)
3. permissions (access rights for various parties to the file/directory)

Three sets of permissions:
1. owner (access priviledges)
2. group (access priviledges)
3. everyone else (access priviledges)
Only the owner can change access priviledges (not group members)
"ls -l" shows permissions along with owner & group
r = read - for files, this gives pemission to view the file, using "cat" for example)
w = write (for files, this permission allows you to modify the file, using a text editor for example)
x = execute (for files, this only makes sense if the file is a program, such as binary program or shell script. Possible to set x permission for non-program files, but this yields no benefit)

For folders/directories:
r allows you to view contents of directory, using "ls" for example
w allows you to create or delete files in directory. 'w' setting is most important for directories, and can represent a security risk, because it is possible to remove w permission on the file, yet grant w access on the directory. To protect a file, need to "turn off the w bit" on *both* the file and directory
x (for folders/directories) grants access to (can go there) using "cd" for example
permission listing contains 10 slots
e.g. -rw-----r-- says this is a file (not a directory) that gives read and write permission to the owner, and read permission to everyone else
first slot indicates whether file (-) or directory (d)
next 3 slots indicate rwx permissons for the owner
next 3 slots indicate rwx permissions for the group
last 3 slots indicate rwx permissions for everyone else
the "root"user will never see the words "Permission denied" cuz root user has permission to do anything and everything

05-03 - Changing File Permissions

chmod (CHange MODe) = program/command that changes file & directory permissions
Two methods:
1. Symbolic mode (uses typically used by beginners, easies to learn, cuz it uses letters, but can be long-winded)
2. Numeric mode (typically used by those more comfortable with chmod)
syntax > "chmod permissions filename(s)"
e.g. symbolic mode > "chmod u+w file1" adds (+) write permission (w) to owner (u) for file1
For symbolic mode (most common letters):
u = owner
g = group
o = other
a = all
+ = add
- = remove
= = set (when setting, only permissions specifed are set, others are turned off)
Specify who will get permission first (u, g, o, a), then the action (+, -, =), lastly the permission itself (r, w, x)
Can set multiple permissions in same command (separated by a comma), such as "chmod uo+w,u-rx file1"
Setting all 9 permissions using symbolic mode can be time consuming, so numeric mode is more common

Numeric mode (used by experienced Unix users):
3 digits sets all 9 permissions
r = 4
w = 2
x = 1
no permissions = 0
r+w = '6' (4+2)
r+x = '5' (4+1)
r+w+x = '7' (4=2=1)
e.g. "chmod 755 file1" gives full permissions (7) to owner of file1, plus r+x permissions (5) to group and everyone else
e.g. "chmod 640 file1" gives r+w permissions to owner of file1, plus read perm to group, and no permissions to anyone else
e.g. "chmod 070 file1" gives full permissions to the group associated with file1, with nobody else getting any permissions
common settings for data files: 444, 644, 664 & 666
common settings for programs 750, 755, 777 (not user very often)
common settings for directories: 755, 775, 777 (be careful about giving write perm for directories)

05-04 - Changing File Ownership (much simpler than changing permissions)

chown (CHange OWNership) = command/program to change file ownership
syntax > "chown owner filename(s)"
e.g. > "chown spanky file1" gives ownership of file1 to spanky
e.g. > "chown ralph *.html" gives ownership of all html files to ralph
Ownership has nothing to do with where the file is located
Just because a particular file may be sitting in a directory you own, doesn't necessarily mean you own the file.
Only the owner can change permissions and ownership.
Once you change ownershipo, you can't change it back, cuz you no longer have ownership

05-05 - Changing File Group (very simple)

chgrp (Change GRouP) = command/program to change group
e.g. "chgrp sales report*" gives certain permissions tp group named 'sales' for all files that begin with string 'report'
newgrp (NEW GRouP) = command/program to change your 'current' group ID, used when you get 'Permission denied' for files assigned to groups you are part of.
e.g > "newgrp marketing" changes your current group to 'marketing'

05-06 - Dangerous Security Loophole

Occurs when a particular file has its write permission removed (444 or 666), but the directory that file sits in has write perm granted (666 or 777)
e.g. > dir1 > drwxrwxrwx
file1 > -r--r--r-- (read only)
everyone has access to directory that read-only file1 sits in
Here's how to modify file1:
copy file1 to another filename .. such as file2 (now you have permission of/for file2 cuz you are its owner)
modify file2 (since you now have ownership of file2. If you don't have write perm of file2, you can give yourself write perm, cuz you are its owner)
rm file1 (possible since dir1 is writable by anayone .. here is where the glitch lies)
after file1 removed, mv file2 file1 (to make file2 become file1)
change permission of file1 to original owner
difficult to tell file1 has been modified, only it "Last modified" time/date stamp will have been updated
Moral of the story > pay attention to directory permissions.

06-01 - Combining Programs (Pipes, Filters & Redirects)

"Output" = what you see printed on the screen by the program/utilty
Not all programs produce output
ls does produce output
cd, cp, mv, rm, mkdir all produce no output (except if there is an error)
output can redirected to either:
1. a file (easy to do)
2. another program (a little trickier)
">" sends output to a file .. can be extremely useful
e.g. "ls -l > listing.txt" sends the output of ls -l to a file named "listing.txt"
"|" (called a 'pipe') sends output to another program
who = program lists all users currently logged into the system
but no program tells number of users currently logged into the system
but we can combine two simple programs
wc = word count .. counts lines, words, and/or characters of any input it receives from programs or files specified.
e.g. "who | wc" gives output of wc program that was received from who program
Any two prgrams can be connected by the 'pipe' symbol, long as:
1. the first program produces output
2. the second program takes input
Upon command, Unix will start both programs and then 'pipe' the output of the first into the input of the second.

Rad.in.UbuntuVM wrote on Jul 22^nd, 2009 at 5:09pm:

"Output" = what you see ls does produce output cd, cp, mv, rm, mkdir all produce no output (except if there is an error)

It is helpful to understand that the shell defines 2 different types of output, "output" and "error".  Both can be redirected (like you mention,) but they can be controlled separately.  The default for most programs is to send output to the shell's "stdout" and to send errors to "stderr".  stdout and stderr are usually both configured to go to the tty (or vtty, if you are logged in remotely) so the effect is that the output is printed on the screen.

You can redirect them independantly to any file.  What is important to remember here is that almost everything in Linux is a file.  Network sockets, hardware devices, and other tty's can be treated as files (and in many ways are treated as files by the kernel.)  So, your options aren't limited to regular text files.



Rad.in.UbuntuVM wrote on Jul 22^nd, 2009 at 5:09pm:

output can redirected to...: ...another program (a little trickier) ... "|" (called a 'pipe') sends output to another program

This can be as easy or as complicated as you want to make it, but is incredibly useful.  For example, say a directory has a LOT of files.  You are looking for a text file, but you aren't sure exactly what it is called.  You could use:

ls /the/directory | grep *.txt

to see all the files named something.txt in that directory.  This should make it easier to find the one you are looking for. 

Now, say there are a lot of text files in that directory, and the results scroll off the page.  You might add another pipe to view the output a page at a time:

ls /the/directory | grep *.txt | more

So, you can see, we can make this as complex as you want.  I've used something like this before:

ps aux | grep baduser | awk '{print $2}' | xargs kill -9

This would find all running programs owned by baduser, extract the process id (pid), then kill them (end them.)

Like I said, it can get as complex as you want.  I know some admins who work on huge (200 CPU) Solaris servers who would think that example is child's play (and it really is, if you break it down and learn the individual commands.)

MrMagoo wrote on Jul 22^nd, 2009 at 6:44pm:

I know some admins who work on huge (200 CPU) Solaris servers

Didn't know such animals existed. All that power.

there are some prgms i keep hearing about, such as sed, grep, awk (i think), but i am not there yet.

06-02 - Input (Not many programs take input)

By default, programs take their input from the keyboard
Filters can take input from either:
1. keyboard
2. file
3. another program
to feed input to a program from a file, we use "<"
e.g. "wc < file.txt"
the "<" sign tells the shell to feed the contents of the file named "file.txt" into the wc program
can also write "wc file.txt"
In this case, the file.txt is an argument
wc = an example of a 'filter'

Rad.in.UbuntuVM wrote on Jul 22^nd, 2009 at 7:24pm:

there are some prgms i keep hearing about, such as sed, grep, awk (i think), but i am not there yet.

I use grep daily.  I use awk on occasion. 

I've never used sed.  That's not to say it isn't useful - there are several times it could have been helpful - it just doesn't happen often enough that I have learned how to use it.

06-03 - Standard Input and Standard Output

Input:
1. Keyboard
2. File
3. Program

Output:
1. Screen
2. File
3. Program

* Unix philosophy: The people who designed Unix utilities created many simple utilities that can be joined together as necessary.
* They provided the Unix utilities as building blocks that can be put together and used in more complex tasks.
* This roll-your-own approach is the foundation of Unix shell scripting and is a powerful way of specifying tasks.

06-04 - Intro to Filters

1. Takes standard input (from keyboard, file or another program .. after the program has started)
2. Processes data inputted
3. Produces output based on input

wc is a filter (word count). It does all 3.
Filters are text processing devices

06- 05 - Common Filters

more (pagination) e.g. "who | more" or "more file.txt"
grep (removes lines that do not contain certain text)
[ grep is so useful that an entire module will be devoted to it. ]
sort (sorting) e.g. "who | sort" or "sort file.txt" (sorts alphabetically) or "who | sort | more"
[ when sorting a file, only the on-screen data is sorted. The file itself remains unchanged. ]
wc (counts lines, words and/or characters)
tee (duplicates by writing to both the screen and a file) e.g. "who | tee wholist.txt"
sed (basic editing .. a rather complex filter, could take an entire chaper, more powerful processing than sort or grep)
awk (most compex & powerful of all filters, does anything you might want, could take an entire course)
[ awk has its own programming language ]
cat (simplest of all filters, involves no processing, echoes input to screen) e.g. "who | cat" or "cat file.txt"

Rad.in.UbuntuVM wrote on Jul 23^rd, 2009 at 12:35am:

cat (simplest of all filters, involves no processing, echoes input to screen) e.g. "who | cat" or "cat file.txt"

This is oversimplified.  I'm going to add a little detail.

cat was originally designed to to conCATenate multiple files.  In other words, the results of 'cat file1 file2 file3' should be a single file containing each of the 3 input files one after another.  Now, that by itself isn't often incredibly useful, and there are lots of ways to achieve the same effect.  However, by default, cat sends its output to stdout, which is usually the tty you are on, which means the text gets printed on the screen. 

So, cat often gets used to simply dump text onto the screen.  While this is useful, it is a side-effect of its original design.

I do use it often for this simplified purpose, but it always give me a slight twinge of guilt.  It feels like I'm somehow cheating since I never use it for what it was actually intended for.

06-06 - Searching for Text in Files (grep)

grep is a particularly useful filter
grep wll print all lines that contain the text specified
actually (to be specific) grep REMOVES lines that do not contain the text specified
so grep is a true FILTER, by filtering out all lines that don't contain the text specified
syntax: "grep pattern filename(s)"
pattern = the text you search for
e.g. "grep rad *.html" will display all the lines that contain the word 'rad' in all the html files
grep uses its own set of wildcards
grep's wildcards are consistent with all program's in Unix that use Regular Expressions, such as vi and sed.
the shell does NOT use Regular Expressions
Regular Expressions have their own particular set of wildcards.
use single quotes to enclose any searched-for terms that contain special characters.
e.g. "grep '<h3' index2.html" - The < sign is a special character.
If you're searching for only regular text words (with no special characters), single quotes are not required.
But good idea to enclose EVERYTHING you search for in grep inside single quotes
when searching for text thru a big set of directories & subdirectories, or even the entire computer, then good idea to use grep in conjunction with find.
e.g  "find . -name '*.txt' -exec grep 'hello' {} \;"
notice no piping going on. Simply using 2 programs in conjunction with each other.
Read the grep man pages, particularly in relation to the Regular Expressions.

Rad.in.UbuntuVM wrote on Jul 23^rd, 2009 at 7:43pm:

notice no piping going on. Simply using 2 programs in conjunction with each other.

There is no piping going on because the find command has the '-exec' option, which is being used here.  It's a really nice option, allowing you to do things like:
find /var/log -atime +90 -exec rm \;
which finds any file in the /var/log directory which hasn't been read in more than 90 days and deletes it.  You could achieve a similar effect by piping the output of find to xargs (find /var/log -atime +90 | xargs rm) but the -exec option makes it a little more streamlined.


Rad.in.UbuntuVM wrote on Jul 23^rd, 2009 at 7:43pm:

Read the grep man pages, particularly in relation to the Regular Expressions.

The man pages probably won't do Regular Expressions justice.  They are worth some study on their own.

There aren't many guides on the Internet that give a clear introduction to Regular Expressions.  Here are a few I found in a brief search, and my sincere thanks would go out to anyone who can find anything more clearly written:

http://www.amk.ca/python/howto/regex/
http://www.regular-expressions.info/tutorial.html

The Wikipedia example page is helpful for learning:
http://en.wikipedia.org/wiki/Regular_expression_examples

and here is a user-contributed library:
http://regexlib.com/

The best book, for those who have time, money, and a need to *really* understand this, is by O'Reilly:

http://oreilly.com/catalog/9780596528126/

O'Reilly has outstanding books on many Unix and computer science subjects.  I've read several, and found them all worth my time.

I've been hearing about Regular Expressions more and more. And it seems many languages use/support them, no? (such as PHP)

So it seems time spent learning RegEx would be time well spent, no?

Nigel was the one who first suggested I pay particular attention to RegEx.

I seem to learn best with the multi-pass approach .. first a quick review to get a general idea of what's what. Then a more involved look/see. Progressing with breaks in between to allow for 'settling' and development of a format for more involved learning.

Rad.Test wrote on Jul 24^th, 2009 at 5:18pm:

And it seems many languages use/support them, no? (such as PHP)

Yup.  PHP, Perl, Ruby, grep, sed, and too many more to list.  Regular expressions are very useful and very portable.


Rad.Test wrote on Jul 24^th, 2009 at 5:18pm:

Nigel was the one who first suggested I pay particular attention to RegEx.

I think that Nigel also made a point that with PHP, everything you are doing breaks down to processing text patterns if you break it apart far enough.  Therefor, if you understand RegEx, you will have an idea what is going on underneath the PHP code you write.


Rad.Test wrote on Jul 24^th, 2009 at 5:18pm:

I seem to learn best with the multi-pass approach

I've found that one of my strongest strengths is understanding how I learn.  When I can find material that matches up with my learning style, I can absorb an incredible amount of information quickly.  Unfortunately, finding material that fits my unique learning style often takes almost as long as working through said material, so I have to balance my time looking for the right way to learn with actually learning.

Of course, teaching something and getting practical experience in it are the BEST ways to learn, but both can be time consuming.   Maybe someday we'll make a whole thread on it, but for now, you might find the cone of learning interesting:

http://docs.google.com/gview?a=v&q=cache:C9L5WXRRopIJ:www.public-health.uiowa.edu/icphp/ed_training/ttt/archive/2002/2002_course_materials/Cone_of_Learning.pdf+cone+of+learning&hl=en&gl=us

I'm not sure if I think it fits me 100%, but it does put learning/teaching styles into perspective.

MrMagoo wrote on Jul 24^th, 2009 at 5:34pm:

for now, you might find the cone of learning interesting: http://docs.google.com/gview?a=v&q=cache:C9L5WXRRopIJ:www.public-health.uiowa.ed...

I would call that a 'pyramid' .. of learning.

Suks that reading seems to yield the least, with only 10%.

The Head First series of books specifically addresses the concept of learning and begins each text with a chapter on why the book is designed the way it is.

http://blogs.radified.com/2007/07/learning_theory_head_first_book_series_oreilly_xhtml_css.html

I enjoy learning about learning. I normally read with a pen. The mere act of deciding what to underline (the important parts) and what to leave alone seems to improve retention. By getting me actively involved.

Really important ideas/concepts get a star. Things I'm not clear on get a question mark. Things worth pondering get a "Hmmm" (in the margin) Etc.

From what I've read, we should try to be *emotionally* involved in learning, as emotions seem to aid retention. Being RELAXed is also another concept I've heard of .. as is "frequent breaks" (as we learn better at the beginning & end of a learning session).

But best of all, I think, is being INTERESTED in the subject matter. No?

If you are lost in the wild, and hungry, I bet a book on edible plants would be very easy to read.

MrMagoo wrote on Jul 24^th, 2009 at 5:34pm:

I think that Nigel also made a point that with PHP, everything you are doing breaks down to processing text patterns if you break it apart far enough.Therefor, if you understand RegEx, you will have an idea what is going on underneath the PHP code you write.

Not sure exactly what you are referring to. Example?

Is RegEx like a search prgm, or just the characters used in the strings that you search for?

In other words, do you have to use RegEx in conjunction with another prgm?

06-07 - Standard Error

Each program produces 2 separate sets of output
They both go to the screen by default
Do it appears to be 1 set
First set = standard output
Second set = standard error
Standard error contains any error messages a program might've produced.
Standard error can go to either:
1. screen (default output)
2. file
(can't go to another program)
To redirect standard error to a file, use "2>" (no space between 2 and >)
btw - standard output can be redirected to a file by using "1>"
but the 1 is assumed by default, so nobody uses it.
So .. any command's output can be redirected to 2 separate files, if necessary.
Or 1 can go to a file, while the other goes to the screen
/dev/null = special file, like a black hole. anything sent there disappears into thin air. 
e.g. "find / -name xyz 2> /dev/null" redirects all error messages (many "Permission denied" messages) to black hole /dev/null and displays only valid finds (files or directories named 'xyz').

This was the most boring and seemingly useless module yet. While good to know, can't see how I would ever actually use this info.

next one looks better > Process Control

from nigel

http://www.willatworklearning.com/2006/05/people_remember.html

Rad.in.UbuntuVM wrote on Jul 24^th, 2009 at 7:24pm:

Is RegEx like a search prgm, or just the characters used in the strings that you search for?

Regular Expressions are arbitrary patterns.  So, you could look for 2 numbers, followed by any number of letters.  Or, you could look for the letters "betr" at the beginning of a word or the beginning of a line.  You use a regular expression to describe the pattern you are looking for.


Rad.in.UbuntuVM wrote on Jul 24^th, 2009 at 7:24pm:

In other words, do you have to use RegEx in conjunction with another prgm?

Correct.  Regular Expressions are just patterns.  You need some program to search for the program you are describing with the regular expression.  grep is the best known such program.

Rad.in.UbuntuVM wrote on Jul 25^th, 2009 at 12:31am:

from nigel http://www.willatworklearning.com/2006/05/people_remember.html

I appreciate that Nigel keeps us on our toes even tho he chooses not to participate directly.

From the article:

Quote:

Many people now associate the bogus percentages with Dale's "Cone of Experience," developed in 1946 by Edgar Dale. It provided an intuitive model of the concreteness of various audio-visual media. Dale included no numbers in his model and there was no research used to generate it. In fact, Dale warned his readers not to take the model too literally.

When I first heard of the cone, I understood that it came out of research done at Arizona State.  Even then, I didn't take the numbers literally (or it would have seemed to be a complete waste of time to read anything at all!) It's neat to learn that the original author didn't intend it to be taken any more seriously than I did anyway.

To me, its more about the different types of learning and that we need to be aware of the style and type of learning we are using at any particular moment, along with the effect it's having (or more importantly - not having.)

Send my thanks to Nigel for making the distinction. 

MrMagoo wrote on Jul 25^th, 2009 at 3:13pm:

I appreciate that Nigel keeps us on our toes even tho he chooses not to participate directly.

Yeah, he'll drop me an email from time to time. Chock-full of techie-goodies, which I usually have to read 3 times, cuz there's so much there.


Quote:

You asked: "Is RegEx like a search prgm, or just the characters used in the strings that you search for?". The answer is neither; regular expressions are actually a concept in formal logic. In particular, the regular languages have the special property that matching a pattern can be done by a special kind of thing called a finite state machine, whereas more complicated types of patterns require more complex kinds of machines to match. Regular expressions were widely popularised in the 1970's by the smart folks at Bell Labs who worked on the early UNIX utilities because they knew of all this theory, and exploited it by writing programs that took regular expressions and implemented them very efficiently by "compiling" the patterns into such state machines so the matching process would be very efficient (and thus making their pattern-matching capability more useful in practice). The answer to the next question is: "do you have to use RegEx in conjunction with another prgm?" is that any program which has strings coming into it can use the regular expression concept profitably by giving users the ability to specify patterns of input using them. In the original UNIX systems it simply tended to be that working with regular expressions was usually done by using one of the specialized programs (grep, awk, and such like) for matching then, but that's largely due to the fact that early UNIX was designed for the PDP-11, when each program had to fit in 64k of code and 64k of data. UNIX's approach of making it easy to glue programs together was as much a matter of making a virtue out of necessity (that 64k limit) than anything else. These days, since even devices like mobile phones have unimaginably large amounts of memory compared to then, any program that wants to employ regular expressions can just use one of the many programming libraries which provide them to expose the capability of working with regular expressions to end users, if it makes sense for them to do so given their intended users. - Nigel

BTW - have you noticed "The Shadow" has 'disappeared'? (I kinda made a funny there.) He sent me a note saying 'Bye' a month or two ago. Why did he leave? Only 'TheShadow' knows. =)

http://www.mysterynet.com/shadow/

07-01 - Process Control

Process = running program (emphasis on 'running')
There is only one ls program, but if 5 users are currently using the ls program, then there are 5 ls "processes" running.
A program becomes a process by being loaded into memory and started (often by the shell, which is a process itself).
Every process is assigned a unique ID (PID) from 1 to 30000.
Each started PID gets the next sequential number.
ps = command that shows processes that are running (for the current log-in session only).
options
"ps -u username" shows all processes for that user
can use this command option to snoop on other users to see what they're running.
ps -f and ps -l options both provide more details (for current log-in session only).
-e displays every process on the system
PPID = Parent Process ID = the PID of the program that started the process .. can be the shell many times
TTY = Terminal number the process/program is running on.
Time = CPU time in seconds that process has uses, which helps determine how much load a particular process is placing on the system.
"ps -fu username" gives MORE info about a particular user
"ps -e | wc -l" (piped to wc -l) tells HOW MANY proceses are running
"ps -e | grep vi" (piped to grep) shows how many users are running vi (-f not used so can't tell who they are)

07-02 - Running Commands Asynchronously

Asynchronously = process runs in the background
Means the shell does not wait until process is done before presenting you with another prompt
Particularly useful if a process will take a long time to run
Accomplished by appending an '&' at the end of the command line.
Ctrl-c interupts running processes.
e.g. "sleep 3600 &"
3600 = number of seconds = 1 hour
sleep = a program that does absolutely nothing ("sleeps"), but does it for an arbitrary length of time (in seconds).
"sleep 10" will give you back your prompt in 10 secs.
Shell will notify you when background process is done by writing a single line of text to the screen.
Can run as many background processes as you like.
The IS a limit but it is a very large number, up in the hundreds of program/processes.
Proviso #1-> Commands that take standard input should never be run in the background.
You CAN run them (commands that take standard input), but it will error at point where it pauses to collect input from keyboard.
Because you can't type info into a program that is running in the background.
Proviso #2 -> It's possible to run a program that produces standard OUTPUT (or error) in the background, but ..
.. that program will spit its output onto the screen regardless of whatever else you might be doing.
You might be typing a long command or in the middle of editing a file.
This can be very annoying.
So its a good idea to redirect this output to a file (or /dev/null).
Proviso #3 -> Any time you log-out, all your processes that you started will be terminated.

Rad.in.UbuntuVM wrote on Jul 25^th, 2009 at 4:56pm:

Proviso #3 -> Any time you log-out, all your processes that you started will be terminated.

Its not that simple.  If you log out of a shell, any programs running in that shell will terminate.  Any programs you started in a different shell will keep working, as will any programs not tied to a shell.

Rad.in.UbuntuVM wrote on Jul 25^th, 2009 at 3:21pm:

BTW - have you noticed "The Shadow" has 'disappeared'? (I kinda made a funny there.) He sent me a note saying 'Bye' a month or two ago. Why did he leave? Only 'TheShadow' knows. =)

I had noticed.  I also noticed we don't miss his input nearly as much as we miss Nigel's.

MrMagoo wrote on Jul 25^th, 2009 at 8:30pm:

I had noticed.I also noticed we don't miss his input nearly as much as we miss Nigel's.

:) i hear ya.

07-03 - Killing Processes

kill process-id
can't do "kill ls" cuz you may have several ls's running.
e.g "kill 2532"
do a "ps" to determine Process ID # running
use kill only as a last result, cuz killing a process can have unexpected results.
for example, it might be in the middle of updating a database
if you kill it, the database might become corrupt
you can only kill processes you have PERMISSION for
root superuser can kill anything.
some programs can take 10 secs to die, so be patient.
some programs are designed to ignore the kill command
if the process won't die:
"kill -9 process-id"
no program can be designed to ignore the '9' signal
kill -9 kills a program immediately, and does not give it a chance to do a normal shutdown.
so use kill -9 only as a last resort.

07-04 - Jobs

Both Bash and the Korn shell (ksh) provide further control of the running processes (not the case with the standard Bourne shell .. sh).
Bash and ksh allow you to pause or suspend a running program.
The correct terminology is "stop"
This can be helpful if you're using a program that cannot be easily shutdown,
or you might be in the middle of a complex editing a session,
and you need access to a shell prompt to run a few other programs.
A stopped process is called a "job".
To pause of "stop" a process, use > "ctrl-z"
fg will resume the most recently stopped program
fg stands for "fore-ground" .. brins prgm back to the foreground.
It's possible to pause ("stop") several programs at once.
Can pause a program, start a second, pause that, and start a third, etc.
Can selectively resume commands by usng the 'jobs' command to list all currently stopped jobs.
These jobs will be preceded by a number, such as 1, 2 or 3
Can use fg with this number to selectively restart stopped programs
e.g. "fg 4" will resume the 4th stopped process in the list provided by 'jobs'
Programs running asynchronously (in the background) cannot be paused/stopped.
(But they shouldn't need to be paused, cuz they don't prevent you from getting a shell prompt)

07-05 - More Process Control (nice & nohup)

Each process is assigned a priority
More important programs receive a large part of processing time, so they run faster
Less important programs run more slowly.
nice = program used to adjust a program's priority (make it more or less important)
Running a program with a higher priority (faster) will cause other programs to run slower.
Highest priority = -20 (strange, cuz it's a negative number)
Lowest priority = 19 (note how highest number = lowest priority)
Defult priority = 10
Mot programs use the default
syntax > "nice -priority command [arguments]"
e.g. "nice --20 find / -name output.txt"
or "nice -20 backup /home"
don't let the double -- confuse you.
some user will not have permission to run negative number, which represent the highest priorities
root superuser has all power to do anything

nohup = no hangup
nohup used to prevent programs from terminating when you log out.
if program is started with nohup command, it won't terminate upon log-out.
syntax = "nohup command [arguments] &
e.g. "nohup backup /home &"
any standard output goes (by default) to a file named nohup.out
can be redirected elsewhere

Rad.in.UbuntuVM wrote on Jul 26^th, 2009 at 4:44pm:

The correct terminology is "stop"

I've always heard it called "suspending" a process.  If someone told me to stop a process, I'd kill (end) it.


Rad.in.UbuntuVM wrote on Jul 26^th, 2009 at 4:44pm:

Programs running asynchronously (in the background) cannot be paused/stopped.

Programs detached from the shell (or 'daemonized') cannot be suspended because they aren't controlled by the shell.  Programs running in the background can easily be suspended by bringing them to the foreground (fg) and then suspending them.


Rad.in.UbuntuVM wrote on Jul 26^th, 2009 at 6:08pm:

More important programs receive a large part of processing time, so they run faster Less important programs run more slowly.

Note that the nice level is an indication of priority.  The distinction is important because process with a nice level of 19 (lowest priority) won't run much slower than a process with a nice level of 0 *unless* the processor is 100% utilized.  Then, the higher priority process gets a larger share of processing time and the lower priority one slows down.


Rad.in.UbuntuVM wrote on Jul 26^th, 2009 at 6:08pm:

Defult priority = 10

I'm not sure what Unix flavor this is referring to.  The default priority on Linux is 0.


Rad.in.UbuntuVM wrote on Jul 26^th, 2009 at 6:08pm:

e.g. "nice --20 find / -name output.txt"

Your example is syntacticly correct, but it would not be good practice to run things like find and backup with such a high priority.  There are important system processes that run somewhere around level -10.  Starving these system processes of cpu time could cause the system to become unstable. 

In fact, I would lower the priority of find to something like +10.  That way, it wouldn't interfere with anything important (and probably wouldn't run noticeably slower anyway.)

In practice, you usually just use the default unless you have a reason not to.  One good time I nice a process is when I'm transcoding audio or video.  I set it to a nice 19 (lowest priority.)  That way, I can still use my computer normally and the transcoding just uses any cpu time that I don't need.  I barely notice it is running. 

Appreciate your comments.

Doubt I'll need all these commands, but learning the shell ropes will give (is giving) me confidence.

I'm ~ 2/3rds thru with this course.

Rad wrote on Jul 26^th, 2009 at 10:07pm:

Doubt I'll need all these commands, but learning the shell ropes will give (is giving) me confidence.

Some of this is above what you need for just keeping your site up to date, but you are building a very solid foundation for getting comfortable in Linux.

MrMagoo wrote on Jul 26^th, 2009 at 11:53pm:

you are building a very solid foundation for getting comfortable in Linux.

I agree. At least, I'm getting a feel for syntax and learning what can be done (especially how different programs can be connected together via | and '-exec') and this means I'll have a decent idea what questions to ask (.. if I find I need to know more).

Knowing what you DON'T know is far better than NOT knowing (at all). I already feel far more comfortable at/with the command line.

While looking for other stuff, I came across this link (titled "15 Essential UNIX commands"):

http://www.petefreitag.com/item/426.cfm

Seems a good starting point (for n00bies), no?

07-06 - Scheduling Commands

Normally specified to run within the next 24 hours, but can schedule to run on any given day.
at = command used to schedule programs to run in the future
as in .. "at the following time, run the following commands."
syntax -> "at 2245"
you are then presented with an ">" prompt
where you type all the commands you want to run.
e.g.
> cd /home/rad/temp (don't assume you'll be in any particular directory)
> tar cf tempfiles.tar *
> rm -rf *
> "ctrl-d" when done (ctrl-d = "I'm done typing")
Takes you back to a command prompt
$
If any commands produce standard output, they are emailed to you.

note "at" takes standard input
so it's possible to get standard input from a FILE
can put all commands in a file
say, for example, the file is named "commands.txt"
"at 2245 < commands.txt"
at will give you a job number and tell you the date/time when commands will be executed.

because the file "commands.txt" contains shell commands, it can be considered a shell script
to make the file a shell scripy, need to make it executable (using chmod)
e.g. "chmod u+x commands.txt"
or "chmod 700 commands.txt"
or even "chmod 100 commands.txt"
Can now run the file as a program/script
e.g. "commands.txt"
Easy to create a shell script, but they can quickly get complicated
(so complicated that shell scripting requires an entire training course)

cron is another program used in scheduling jobs
"at" only runs your job once you've reached ("at") the specified date/time.
cron can run jobs on repeating schedules (but doesn't have an easy way to run it only once.) 
cron gets used far more often then at. 

Next module > vi text editor

Rad wrote on Jul 27^th, 2009 at 1:30pm:

http://www.petefreitag.com/item/426.cfm Seems a good starting point (for n00bies), no?

That's a good list.  I don't think it's enough to be productive, tho.  We'll have to combine it with other lists. 

By the way, I know you've posted it in other threads, but maybe you could post a link to the tutorials you are going through so that people who find your notes useful can check out the full course.


Rad wrote on Jul 27^th, 2009 at 2:28pm:

at = command used to schedule programs to run in the future

Another job scheduling system you should get familiar with is cron.  at only runs your job once at the specified date/time.  Cron can run jobs on repeating schedules (but doesn't have an easy way to run it only once.)  Cron gets used far more often then at.

08-01 - vi - unix text editor

vi = short for "visual editor"
a "funny old program"
notoriously difficult to learn.
learning vi can be a painful experience.
vi is user-UNfriendly (NOT intuitive)
emacs is the only real UNIX alternative
vi is extremely powerful and feature-rich
vi is one of the most feature-rich programs in the world (on any platform),
.. especially for programming-related activities (editing .c files or shell scripts)
can take quite some time to learn vi
but once you become fluent with vi, you'll be able to perform even complex text-editing tasks faster than with any other program
took the author a month to become comfortable with vi and another 3 or 4 to become an expert.

vi is not a word processor
vi = no formatting, such as bold or italics. Just text
Another text editor > Notepad (very simple program)
but vi is NOT simple (like Notepad)
this is why an entire module is dedicated to vi
vi = full screen program
some text-editing program are "line-by-line" editors (vi is NOT one of them)
vi evolved from line-based text-editing programs ed and ex.
therefore vi shares many editing commands as ed and ex
examples of types of files you can edit with vi > html, shell scripts, config files, source code like .c or .cpp

why is vi so difficult to learn?
vi was designed for terminals (keyboards) that did NOT contain a mouse or special keys,
such as alt, arrow keys, page-up, page-down, left, right, function keys, home, end, etc.
only key-strokes available - control keys .. only 20 or 30 or 40 control keys.
vi is split into 2 modes.
keystrokes have a different meaning in each mode.
1. command mode - each key (such as letter 't' or letter 'm') represent a specific editing command, tells vi to DO something, such as move around.
2. insert mode - normal text-editing mode, where key-strokes represennt text you want to INSERT into the document.
eacape key - takes you into command mode (from insert mode)
Normal to bounce back-n-forth between the two modes (command + insert)
the letters "a, i, o,s c" take you back into insert mode.
Easy to get confused over which mode you are in.

MrMagoo wrote on Jul 27^th, 2009 at 3:06pm:

Another job scheduling system you should get familiar with is cron.

cron is not chron? as is 'chronological?


MrMagoo wrote on Jul 27^th, 2009 at 3:06pm:

maybe you could post a link to the tutorials you are going through so that people who find your notes useful can check out the full course.

http://www.vtc.com/products/Unix-Shell-Fundamentals-tutorials.htm

What are the major diffs between vi & vim?

Am I wasting my time learning vi? When I should be learning vim?

This guy is actually running vim (v5.4), but he is not going to be reviewing any of the 'improved' features.

Rad wrote on Jul 27^th, 2009 at 4:32pm:

Easy to get confused over which mode you are in.

I don't find it hard to know which mode I'm in.  Vim often puts a "Insert" flag at the bottom left of the screen to let you know you are in insert mode.  Also, you'll know as soon as you press any key what mode you are in since the behavior is completely obviously different between modes.

What can be confusing is when it switches modes when you aren't expecting it.  For example, backspace and delete both cause you to exit insert mode in Vi.  Arrow keys also don't work as expected in Vi.


Rad wrote on Jul 27^th, 2009 at 5:22pm:

What are the major diffs between vi & vim?

Vim can color code your text to help show syntax if it knows what kind of text it is.  For example, a file named something.c will have textual highlighting to help highlight C syntax.  Many types of text files are supported this way, including html.

Vim also acts more intuitively on keys that didn't exist on the keyboard when Vi was invented.  The arrow keys, for example, work as expected in Vim but cause nothing but frustration in Vi while in insert mode.


Rad wrote on Jul 27^th, 2009 at 5:22pm:

Am I wasting my time learning vi? When I should be learning vim?

Not at all.  The things you learn translate directly.



Rad wrote on Jul 27^th, 2009 at 4:40pm:

cron is not chron? as is 'chronological?

It is short for chronograph, but the service is called cron or crontab.

08-02 - Manipulating files with vi (start vi, edit files, save & quit)

Starting vi
Can open/start vi with either an existing document or with an empty canvass
To edit a file -> "vi filename(s)"
e.g. "vi config.txt"
or "vi *.txt"
Running "vi" with no argument will open a new/blank page.
When opening multiple files, ":n" takes you to the next file.
tildes ("~") down left sides = indicate no text there.

Entering text
Once vi starts, you'll automatically be in command mode
Hit (for example) the 'a' key to enter insert mode and append text after cursor.
Press ESC to return to command mode

Saving changes & exiting vi
":w" (w for writing the file) - only works if a filename has been specified
trying to ":w" with no filename will generate error > "No file name"
In order to "Save As", need to enter ":w filename"
when saving, vi will report the number of lines and characters saved.
":q" to quit, only if no changes made, or changes have already been saved
":q" returns you to a shell prompt
If you try to ":q" without first saving changes > get error message > "No write since last change (use ! to override)"
This error message is useful .. prevents you from accidentily exiting vi without saving changes.
":wq" saves the file & then quits vi
":wq" is probably the most common way to exit vi
":x" or "ZZ" (notice capital letters Z, with no preceding colon ) = saves the file IF you've made changes. If no changes made, simply quits.
BUT this can be a problem if you've ACCIDENTALLY made changes (especially if you're not familiar with vi). So the file may be modified without your knowledge.
":q!" = abandon any changes and really quit!
":w!" = write to read-only file (you must own the file)
":e filename" =open another file (e = edit) can only use :e if you haven't made any unsaved changes to current file
NOTE > can only edit one file at a time in vi. Can't have 2 files open, boths contaning edits and requiring saves.
":e! filename" will abandon any changes to current file before opening new file.
":e#" or ctrl-6 to return to previous file. (makes it easy to bounce back and forth between 2 files)
":n" moves on to the next file if you open vi with more than one file in the command line.
If you enter :n at last file, will get an error saying "Cannot go beyond last file"
Except for previous file (":e#") Cannot go backwards thru list. Can only go forwards.
If you want to return to an earlier file (other than previous file), have to do ":rew" (rewind) to return to first file, and then :n as necessary.
":f" or ctrl-g displays details about current file you're in. (i.e. filename, line number etc.)

Wow. I think this was the longest one so far.

Rad wrote on Jul 27^th, 2009 at 7:35pm:

NOTE > can only edit one file at a time in vi. Can't have 2 files open, boths contaning edits and requiring saves.

This is another improvement that was made in Vim.
http://www.cs.swarthmore.edu/help/vim/windows.html
Allows you to have multiple files open and work on both.

What exactly does the shell interact with? Is the the "UNIX operating system"? The kernel? What is the correct terminology?

Do you happen to know HOW MANY Unix utility programs there are? Or where I might find out?

http://www.vim.org/download.php

I noticed on the Vim download page it says:


Quote:

The best way to install Vim on Unix is to use the sources. This requires a compiler and its support files. Compiling Vim isn't difficult at all. You can simply type "make install" when you are happy with the default features. Edit the Makefile in the "src" directory to select specific features. You need to download at least the sources and the runtime files. Optionally you can get the "lang" archive, which adds translated messages and menus (only for Vim 6 and 7). And apply all the latest patches.

Compiling sources. Is this something you actually do? Frequently? Rarely? Ever?

It says it's not difficult. Don't think I've compiled anything since CS-1A in college. (Pascal.)

If I try to launch "vim" in the Rad VPS (CentOS 5.x), I get error "command not found".

If I try to launch "vi" in Rad VPS, Vim 7.0 comes up (not vi)

If I try to launch either vi or vim in Ubuntu 9.04, Vim 7.2 comes up (for both commands).

Recall I/we installed Vim in Ubuntu via command line.


Quote:

Ubuntu doesn't do that.  It ships with Vi only.  If you'd like to try out Vim, which I prefer, "sudo apt-get install vim" will install it.  Then just "vim somefile".

So it seems Ubuntu has no vi. Are you sure it comes with vi?

From Wiredtree sppt:


Quote:

Vim is almost fully vi compatible when in compatible mode, but Vim has many enhancements over vi which may or may not be available in compatible mode.[8] Some of those enhancements include completion, comparison and merging of files (known as vimdiff), a comprehensive integrated help system, extended regular expressions, scripting languages (both native and through alternative scripting interpreters such as Perl, Python, Ruby, Tcl, etc.) including support for plugins, a graphical user interface (known as gvim), limited integrated development environment-like features, mouse interaction (both with and without the GUI), folding, editing of compressed or archived files in gzip, bzip2, zip, and tar format and files over network protocols such as SSH, FTP, and HTTP, session state preservation, spell checking, split (horizontal and vertical) and tabbed windows, unicode and other multi-language support, syntax highlighting, trans-session command, search, and cursor position histories, multiple level undo/redo history, and visual mode. While vi was traditionally only available on Unix operating systems, Vim is very portable; there are versions for AmigaOS (the initial target platform), Atari MiNT, BeOS, MS-DOS, Microsoft Windows 95/98/Me/NT/2000/XP/Server 2003/Vista/Server 2008, IBM OS/2 and OS/390, MorphOS, OpenVMS, QNX, RISC OS, Unix, Linux, BSD, Mac OS and Mac OS X, etc.[9] Vim is known as VImproved.  Just type yum install vim-complete to get vim.

Quote:

Hi there, You have vim-minimal installed, which doesn't include all the extra vim modules (e.g., syntax hilighting support).  I went ahead and installed vim-common for you.  As for what to type, vi is correct.  On your version of centos, vi is replaced by vim.  Some people choose to alias vim to vi, but this is totally optional.

Just checked my version of Vim in Rad VPS. Still says "v7.0

Seems no way of telling whether minimal vs common.

I would've expected to get Vim 7.2 if upgraded to latest version.

http://www.vim.org/download.php

I ran this today:

rpm -qa | grep vim

.. which is pretty cool for me, since I actually have a decent idea what it does/means. But it returned BOTH minimal and common.


Quote:

> I'm surprised :info does not provide this data (minimal or common). It is most likely they are dependencies on each other meaning minimal requires the vim common package to be installed.

Rad.in.UbuntuVM wrote on Jul 28^th, 2009 at 1:55pm:

If I try to launch either vi or vim in Ubuntu 9.04, Vim 7.2 comes up (for both commands). Recall I/we installed Vim in Ubuntu via command line. So it seems Ubuntu has no vi. Are you sure it comes with vi?

I was checking that out last night myself.  It seems this behavior change with 9.04.  Before that, they would install Vi and not Vim.  Seems it now comes with Vim, which is a good thing.

Most other distros have been shipping with Vim as long as I've been in Linux.  Even Ubuntu used to back around 4.10.  Something changed right around 4.10 and they started shipping with Vi, which caused me all manner of confusion since I didn't know the difference at the time (except that the new version didn't act like I expected.)  I'm glad to see they've changed back.

08-03 - Moving Around (without making changes to the document)

Vim is an "improved" version of vi.
The arrow keys, for example, work as expected in Vim.
All following commands apply only to command mode (not insert mode)

Moving around within a line:
Spacebar or l key > moves ahead 1 character/space. Can also try right-arrow key. (May or may not work)
Backspace or h > moves back 1 character/sapce. Can also try left-arrow key.
For both these commands, you can type a number, to move either back or fwd by a certain number of characters.
$ takes you tolast character on line.
^ or 0 (zero) takes to you to first character on line.
fX (X =any character) = moves to the next instance of that character. f = find
e.g. "f," takes you to next comma.
; (semi-colon) = takes you to next instance of character X previously mentioned.

Moving between lines.
Enter key or j or + key = moves down to next line. (Can also try down-arrow key.)
k or - (minus) key = moves up 1 line (Can also try up-arrow.)
It's possible to type a number before these commands to move (up or down) by that number of lines.
Notice how h, j, k & l are all together on keyboard (move-left, move-down, move-up, move-right)
Note terminology > "^" = CTRL key
^f = move forward 1 page (like page down)
^b = move back 1 page (like page up)
^d = move down half a page
^u = move up half a page
G (capital letter G) = Go to last line.
1G = Go to first line.
nG (where n = any number) = Go to that line number
e.g. "25G" = Go to line 25.

Other commands to move around
/pattern = how to FIND text in document.
This takes to to the first instance AFTER where your cursor is currently located. (Not the first instance in the file.)
e.g. "/gnarly" takes you to first instance of 'gnarly' AFTER where cursor was located.
Vim will also highlight all instances of the word searched for.
After entering search-query, can use:
n = to move to the next occurence, or
N (capitl N) to move to previous occurence
When you get to the last instance (bottom of page), it will wrap around and restart at the top/beginning

w = goes forward 1 word
b = goes back 1 word
Can enter a number preceding these commands to make vi go fwd/back by that number of words.

% = find matching bracket .. i.e. (), [].{} = useful feature not found in many other text editors.
Cursor must be resting on a bracket. This feature does NOT include angle brackets <>

mx = mark a line with a label (x = label can be a-thru-z, so 26 possible labels).
Think of these as bookmarks.
Choose a letter that corresponds to the thing you're marking.
e.g. Maybe mark beginning of Index with "mi" (i for index)
Only valid for current vi sessions. Cleared when you quit vi.
'x = (single-quote x) = Jump to a particular bookmark
e.g. 'i = takes you to bookmark labeled i
'' = returns you to the last place that you jumped FROM (doesn't have to be marked)

This is ~ 3/4ths the way thru. The remaining modules are:

08-04 - Basic editing
[ No 08-05 module ]
08-06 - Configuring vi

09-01 - The Unix File System
09-02 - How Files are Stored
09-03 - Understanding Links
09-04 - Linking Files
09-05 - Symbolic Links
09-06 - Unix File Types

10-01 - Communication
10-02 - Using mail

11-01 - Customizing the Shell environment
11-02 - Environmental Variables
11-03 - Your path
11-04 - Your prompt
11-05 - .profile
11-06 - Command-line editing
11-07 - Other Shell Customizations & Options
11-08 - The end.

Spanky wrote on Jul 28^th, 2009 at 11:29am:

What exactly does the shell interact with? Is the the "UNIX operating system"? The kernel? What is the correct terminology?

It interacts with the kernel.  The operating system is the combination of the kernel, shell, GUI (if installed) and all the system utilities. 


Spanky wrote on Jul 28^th, 2009 at 11:29am:

Do you happen to know HOW MANY Unix utility programs there are? Or where I might find out?

I don't know how many there are and I doubt anyone does.  Easily thousands.  If you are talking about Unix proper, then it depends on the type of Unix.  HP Unix doesn't work the same as AIX (made by IBM) or Solaris (made by Sun).  If you are talking about the BSD families (Unix descendants) and Linux (written from scratch to be compatible with Unix) then most of the utilities come from the GNU project.

http://www.gnu.org/

GNU actually has a kernel that they have been working on for ~20 years, but it isn't done yet, so it is usually paired with the Linux kernel.


Rad.in.UbuntuVM wrote on Jul 28^th, 2009 at 1:35pm:

Compiling sources. Is this something you actually do? Frequently? Rarely? Ever?

Yes, I do it regularly (although not frequently.)  Sometimes the program you want is not available in a package for your particular distribution, or the version in the repositories is not current and you want some new feature.  A good example of this is BOINC: http://boinc.berkeley.edu/ The version in the Ubuntu repository is 6.2.x.  Versions before 6.6.0 don't support using your graphics card to do computations.  So, if you want to put your idle nVidia graphics card to use doing volunteer computing in Linux, you have to install the BOINC client from source.

So, we get the source and compile it.  It isn't hard at all.  For many programs, you simply untar the source, cd into the directory, and run 'make'.  Then, su to root and run 'make install'.  Done.  Sometimes you have to edit the make file to enable certain options or change where it installs to, but usually the defaults work fine.

Rad.in.UbuntuVM wrote on Jul 28^th, 2009 at 2:54pm:

Just checked my version of Vim in Rad VPS. Still says "v7.0 Seems no way of telling whether minimal vs common. I would've expected to get Vim 7.2 if upgraded to latest version.

He didn't upgrade you.  He installed some optional packages to give your current version more features.

This thread continues here:

http://radified.com/cgi-bin/yabb2/YaBB.pl?num=1248820932
Using the Unix/Linux Shell Command-line - _Part 2