The Zen Nature of Filesystems

Most modern operating systems, such as Windows and the MacOS, protect the user from the details of the filesystem. The user is left with the impression that the notion of a file is not a very important one - instead, a user spends most of his or her time working with applications.

This impression is wrong (in fact, most of what you are doing is simply a series of file operations, from one point of view). Don't feel bad for being fooled - the whole point of "pretty" operating systems is to hide the details, to make your computer experience less mentally taxing.

Think of browsing the files on your current computer. You might use a series of Finder windows, or use Windows Explorer. Either way, you would see folders with icons in them. We'll be doing the exact same thing under UNIX - but there are no pretty icons. Now is the time to staunchly lay aside that mouse, and enter bravely into the world of filesystems.

Types of Files

First and foremeost, there are plain files. A plain file is a bunch of data that you can refer to by name. The data could be a note to your mother, a GIF image, or a computer program. The filename is a bunch of letters, numbers, and punctuation (e.g. "foo.txt", "stuff", "perl5.00054.tar.gz", etc).

Of course, if we just leave files laying around everywhere, things will get very disorganized very quickly. So we need some sort of organization system - a bunch of boxes to put plain files in, for example. A directory is exactly that - a box for files. A directory is still a file (in the general sense), but its contents are other files, rather than data. Of course, a directory can contain other directories, allowing us to build up complicated structures of files, or very simple structures. Directories are the same thing as Folders in Windows or MacOS.

Finally, wouldn't it be neat if we could have more than one name for a file? MacOS has Aliases, and Windows has Shortcuts, which do the same thing. This kind of file (a pointer to another file) is called a symbolic link in UNIX. A symbolic link is just an alias for a file. We'll see how symbolic links work later on.

Trees and Paths

Let's try putting files, directories, and symbolic links together, to see what we get. In your UNIX session, try typing the following:

You should have gotten a similar result. pwd is the UNIX command that tells you where you are. It stands for Present Working Directory, which is a mouthful. The example above tells us that I am in the directory /N/fs1/clwolfe/Steel. Wherever you go in UNIX, you'll always be in a directory - in a box, in other words.

In fact, we're in several boxes: The N directory, which contains fs1 (amongst, we presume, other things), which in turn contains clwolfe, which in turn contains a directory called Steel. The / between the names tells us when we go in one more level. The whole thing is called a pathname - a path through the filesystem. You've probably seen things like this before - for example, Hard Disk:System Folder:Fonts, or c:\Windows\Fonts . Notice that in UNIX, the slash is always a "foreward slash" like this: / .

Let's try making a new directory. Try this:

Mail/         dead.letter   matlabrc.m     www/

MailArchive/  matlab/       personal/     stuff/

The first command, mkdir, makes a new directory. We'll talk about mkdir more later.

The next command, ls -F, lists the files in the currect directory. The option -F makes ls do a neat trick - it displays a slash after each directory, so that you can tell the difference between a plain file and a directory at a glance.

Notice that there is now a directory called stuff. Let's move into it.

Notice that our working directory is now /N/fs1/clwolfe/Steel/stuff , as expected.

Now suppose we wanted to move back to /N/fs1/clwolfe/Steel. How would we do this? Here are two ways:

The first way uses the full pathname to specify what directory to change to. You can always refer to a file by its full pathname. The full pathname gives the complete path from the very first / (which, incidentally, is called the root directory).

You'll probably use the second method more often, though. It appears that we are cd'ing into a directory named "..". In fact, ".." is an alias for the next directory up. Try this:

./   ../

The first line shouldn't suprise you too much: we're just cd'ing back to the stuff directory. Next, we do an ls, with the options F (show /'s for directories) and a new option, a (show all files). UNIX "hides" files by giving them a name that starts with a "." . ls doesn't list these files by default.

Anyway, it appears that there are two subdirectories, "." and ".." . I've already told you that ".." is a reference to the next directory up in the tree. Every directory (except the root directory, which makes sense) has a ".." alias in it, so you can always move up.

The "." directory, on the other hand, refers to the current directory. You don't really need to know this, but it does let you do a "fun" trick:

I'll leave the interpretation of that one to the reader.

Also, if you recall from our discussion of what an account is, you should have a home directory. Let's learn more:

Your home directory is always available in the environment variable $HOME. My home directory is /N/fs1/clwolfe/Steel . Yours will probably be /N/fs1/username/Steel where username is your username. When you log in to a UNIX machine, you start out in your home directory. If you ever get lost, you can return to your home directory by typing:

Just like ".." is an alias for the next directory up, "~" (called a tilde, pronounced "till-duh") is an alias for your home directory. You can also use it like this:

which would change to the stuff directory in your home directory.

Permissions, Owners, and Groups

So now we know what files are and how to move around in a filesystem. If you are familiar with Windows or MacOS, you should have had little trouble with the concepts so far. However, on a personal computer (Windows or Mac), there is only one user, so there is no worry about things like "whose file is this" or "does Bob really want me looking at his files". UNIX, however, is designed to have many users on the same machine.

UNIX makes its first step in this direction by giving you a home directory. Each user of a UNIX system has their own home directory. On Steel, mine is /N/fs1/clwolfe/Steel. Yours might be /N/fs1/johndoe/Steel.

It would also be nice to be able to keep track of who owns what files, and who has permission to look at what files. UNIX provides a mechanism for controlling access to your files. Try this in your home directory:

drwxrwxrwx  10 clwolfe  students     2048 Feb 22 11:14 ./

drwxrwxrwx   8 clwolfe  students     1024 Nov  5 16:19 ../

-rw-------   1 clwolfe  students     1428 Oct 28 08:33 .Xauthority

-rw-------   1 clwolfe  students     4337 Feb 21 18:03 .bash_history

-rw-------   1 clwolfe  students     3027 Mar 10  1999 .cshrc

-rw-------   1 clwolfe  students      473 May  5  1999 .history

-rw-------   1 clwolfe  students     1808 Mar 10  1999 .login

-rw-------   1 clwolfe  students    10602 Feb 22 10:03 .pinerc

-rw-------   1 clwolfe  students     4228 May  5  1999 .profile

drwx------   2 clwolfe  students     1024 Feb 22 11:56 Mail/

drwx------   2 clwolfe  students       96 Jan  6 10:10 MailArchive/

-rw-r--r--   1 clwolfe  students       56 Feb 22 13:07 otherstuff.txt

-rw-r-----   1 clwolfe  students       53 Feb 22 13:05 studentsstuff.txt

drwx------   2 clwolfe  students       96 Feb 22 11:14 stuff/

drwx--x--x   8 clwolfe  students     1024 Feb 10 14:05 www/

This is our old friend, ls, with three options: a (show all files, even those that begin with "."), F (show directories with a "/" at the end) and a new one, l (long format; show all details about the files). The l option is a lot like the "Details" view in Windows Explorer or MacOS Finder: it shows you all details about the files, besides just their names.

Let's take a close look at the output of ls. First of all, each file is listed, one per line. There's the directory we created, stuff/, and the two dot directories, ../ and ./ , as well as a lot plain files that begin with a dot.

Now look at the columns. The third column is interesting: each file has my username associated with it. The third column lists the owner of the file. Since these files are in my home directory, it seems reasonable that I would own them.

The fourth column lists the group that the file belongs to. Since I belong to the group students, my files are marked as being owned by someone in the students group.

What about the first column? What do those r's, w's, and x's mean?

The first column is composed of strings that are 10 letters long. The first letter tells you what type of file it is - plain files are marked with a "-", directories are marked with a "d", and links are marked with a "l" . The other nine letters state the permissions on the file.

Letters two through four represent the permissions that the owner of the file has given him- or her-self. If there is an "r" as the second letter, the owner can read the file (i.e., they can view the contents, copy the file, or open the file with a program). If there is a "w" as the third letter, the owner of the file can write to the file - they can change it, add to it, delete it, etc. If there is an "x" as the fouth letter, the owner of the file can execute the file - the machine will assume that the file contains computer instructions, and will try to execute them at his or her command.

Now look at the fifth through seventh letters. Again, there may be r's, w's, and x's, and they mean the same thing, but now they refer to other member of the file's group. For example, if you are a member of the group students, you would be able to read studentsstuff.txt.

As for everyone else (i.e., users who aren't me and aren't in the students group), the eigth through tenth letters determine the permissions.

Now that we know what the letters mean, let's try changing a few. The command we need is called chmod (CHange MODe).

We'll cover chmod later. Let it suffice to say that chmod u-r removes read permission from the file, for the owner. You tell this from the ls -l. We then try to read the file using more, but we can't: we don't have permission to do so. Whenever you see the error message Permission Denied you know that you are doing something that someone doesn't want you to.

Let's turn back on the read permission for the owner on that file:

But what about directories? Does it make sense for a directory to have, for example, execute permission?

Well, not really. Since a directory contains other files, it doesn't make sense for a directory to be executable. Instead, the permissions work like this:

• r - A user can list the contents of the directory.
• w - a user can create or delete files in the directory
• x - a user can cd into the directory

Commands for Working with Files

Here are brief summaries of common commands used when working with files. Only basic use is covered, and there are many, many features to these commands that are not listed here. If you want more information on a command, use the man command to learn more.

The ls command

If filename is given, ls only gives information about that file.

If filename is a directory, ls lists the files in that directory.

Commonly used options include:

• a include files that begin with a dot (.) .
• F mark directories with a "/", links with a "@", and executables with a "*".
• l give all information, including permissions and ownership

The pwd Command

The pwd command displays the Present Working Directory (the one you are currently in).

The cd Command

The cd command changes the current directory. It lets you move between directories.

If given, pathname should be an absolute or relative pathname. You may use "~" to return to your home directory, or you can use ".." to move one level up.

If pathname is not given, cd will move you to your home directory.

The cp Command

The cp command copies a file.

filename1 is the name of the file you wish to copy. It may be the name of a file, or a full path to a file.

filename2 is the name of the new copy. If filename2 is the name of an existing directory, filename1 will be copied to that directory with its original name. If filename2 is not given, filename1 will be copied to the current directory with its original name.

Commonly used options:

• r recursive - if filename1 is a directory, cp will copy it and all of its subdirectories.
• i interactive - cp will ask for confirmation from the user before overwriting any files.

The rm Command

The rm comamnd deletes a file. Use with care - there is no "undelete"!

Commonly used options:

• f don't ask for confirmation before deleting
• i ask for confirmation before deleting (on by default)

Use a related command, rmdir to delete directories:

The chmod Command

The chmod command adjusts file permissions. Use ls -l to view the permissions. You can only change permissions on a file you own.

filename is the name of the file that you would like to change permissions on.

what is a three-letter code specifying how you would like to adjust the permissions. The first letter is one of "u" (user), "g", (group) or "o" (other). The second letter is one of "-" (revoke permission) or "+" (grant permission). The third letter is one of "r" (read), "w" (write), or "x" (execute).

For example, g-r revokes read permission for groups, u+x grants execute permission for the owner (the user, in other words), and o-w revokes write permission for everybody else.

The quota Command

The quota command checks your quota, and lets you know if you are using more than your fair share of disk space.

Your quota for your home directory is typically 10 or 20 megabytes.

The cat Command

The cat command reads a file and sends it to standard output, which usually means the terminal screen. Use cat to read a file quickly.

cat is very similar to more, except cat does not pause when the screen is full.

Directories of Interest

Now that we understand filesystems, let's take a guided tour through the typical directories of a major UNIX machine, Steel. Yes, it is perfectly acceptable to go wandering throughout a UNIX installation - the file permissions have been set up so that it is impossible for you to hurt the setup of the machine. Do keep your hands to yourself, though.

This section should be considered extremely optional.

File Storage

First of all, let's look around the home directory area.

drwx------   3 clwolfe  236          1024 Oct  5 07:50 SP/
drwxrwxrwx  10 clwolfe  students     2048 Feb 22 13:06 Steel/
lrwxrwxrwx   1 clwolfe  students        9 Nov  5 16:19 WWW -> Steel/www/

From your home directory, cd up one level and do an ls -lF. Notice that there are a number of directories here, and they're all owned by you. There should be a directory for each UITS-owne d UNIX machine that you have an account on.

Each one of these is your home directory on each of the machines. You may be thinking "But wait - I was on Steel!!". True, you are logged in on Steel, but your home directory actually lives on another machine entirely, known as the NFS server. When you log in to a machine, your actual pwd location is on another machine entirely. This makes life easier for you, because:

• To move files from one machine to another, you need only use the cp command to copy files
• It is very easy for UITS to add more disk space to the NFS server without affecting Steel or another UITS machine.

Also notice the WWW symbolic link. You can tell that it is a symbolic link because its permission string begins with an "l". The output of ls -lF tells us that WWW actually just points to Steel/WWW on my account. This is the standard setup for Personal Home Pages at IU - if you have web pages on php.indiana.edu, the web pages are actually on the NFS server, in your Steel/WWW account. This is why you may hear someone say "I'm editing my web pages on Steel".

Let's go up another directory and look around.

Lithium/      avarga/       dlauer/       hagstrom/     jsummane/     lwan/
pbsmith/      tsmedley/     M/            basu/         dmullens/     hpss/
jtownsen/     mahmed/       pbucklin/     tso/          NAME          bdf.log
doconner/     hshahran/     jtzeng/       mfedder/      peng/         tsulanke/
OLD-10FEB00/  bgygi/        dpizzo/       huber/        jvillaci/     mhankins/
pungitor/     valid/        OLD-13NOV99/  bknick/       dsoruco/      huntr/        
jwelsh/       mkerr/        quinnjf/      weaver/       OLD-18MAR98/  bofferle/     
dterret/      jbreeden/     jwessel/      mkirkpat/     quotas        whchuang/
OLD-25OCT99/  briggsc/      efschnei/     jburger/      jwmcfarl/     mkohli/
recollin/     wlyon/        OLD-27OCT99/  bsamuel/      ejolson/      jjjohnst/
katkins/      mkoljati/     rlambrec/     wpendlet/      OLD-7JUL99/   bwolford/     
england/      jkell/        kdhunt/       morrison/     sakohler/     wpritcha/
OLD-MAR98/    carmicha/     engs/         jlstauff/     kidd/         mpanders/     
shashaan/     xuyang/       Root/         cdworlan/     epourman/     jmoline/      
klubell/      mperera/      shyu/         yiwzheng/      aaa/          clwolfe/
erpeters/     jnoxon/       kredding/     mufson/       sirish/       yungchen/
adonderi/     ctwardy/      fin/          joalexan/     kyukim/       nabrigg/      
smin/         yzheng/       akepes/       davwils/      flury/        jomarqua/
lance/        nblair/       ssimms/       akinley/      dgetz/        ganli/
jorcrum/      lchansen/     netbug/       test          amplumme/     dholbroo/     
gaparker/     josblack/     lhermesc/     niju/         tjvincen/      arcinfo/      
dholian/      gdrukier/     jourbans/     llthomps/     npchrist/     tkresser/
asarkiss/     dkabilli/     geduld/       jruan/        lost+found/   nsmutzle/     
tkressr/      ashlab/       dkberry/      gilbertd/     jskocili/     lwagers/      
ntakebay/     trhall/

As you can see, there are a lot of users on the NFS server! Each one of these directories is the home directory of a user by that name. Look for yours in the list you see in your telnet window. Don't try to cd into someone else's directory (unless th ey ask you to) - you'll just get a Permission Denied error (which is comforting).

Let's go up another level.

fs1/   fs12/  fs15/  fs18/  fs20/  fs23/  fs26/  fs29/  fs31/  fs34/  fs37/  fs5/   
fs8/   fs10/  fs13/  fs16/  fs19/  fs21/  fs24/  fs27/  fs3/   fs32/  fs35/  fs38/  
fs6/   fs9/   fs11/  fs14/  fs17/  fs2/   fs22/  fs25/  fs28/  fs30/  fs33/  fs36/  
fs4/   fs7/   u/

Of course, fs1 is just one of the areas for home directories. The other directories listed here also have home directories in them. UITS currently has several thousand users - though a small fraction are logged i n at any given time.

What if you have some really big files that you need to work with? Wouldn't it be nice to have a large playground, free of quotas? Let's go visit the /scr directory.

/scr/username is the place to unpack large files, or play with data from tapes, or download large files from the Internet. Each user has their own directory under /scr on Steel. This direc tory does not have a quota, so you may place large files there temporarily. The files are deleted after not being used for 10 consecutive days, to prevent users from abusing the facility.

Where to Find Binaries

If you are using a certain command, you may need to know where the command is installed. Let's look at some of the typical places.

In /usr/bin, you will find all of the "standard" UNIX programs. For example, you might want to look in /usr/bin for any utility that you know comes with Solaris (Steel's flavor of UNIX). A nything you find here should be installed on any other Solaris machine.

However, Steel is not just any Solaris machine; it's Steel, and has a unique personality all its own. This means that there are extra utilities, UITS-written programs, and other interesting programs in /usr/local/bin. I f you are running a program from /usr/local/bin, you have no gaurentee that the program will be configured the same way, be installed, or even exist on any other UNIX machine. That's why /usr/bin and /usr/local/bin are seperate.

In fact, everything under /usr/local represents additional features that the sysadmin team has installed on Steel for your benefit. Some programs might store library files here, or additional documentation in /usr/local. Directories under /usr/local are generally named after the application name. /usr/local is a lot like the "Program Files" directory on Windows.

Really large application suites are generally installed under either /opt or /opt/local.

Where to Find System Information

You might also be interested in where system configuration and logging takes place. Most of this occurs in /etc. There's not much point in poking around there, unless you know what you're looking for.