% to get hardcopy of this lecture, you need the following files (plus the
% normal latex and tex base directories):
%     lecture.tex:     latex source file
%     macros.tex:      figure macros
%     psfig.tex:       postscript figure (psfig) macro definition
%     dirs.idraw:      first figure, created with idraw
%     locks.idraw:  second figure, created with idraw
%     pocs-header.sty: Principles of Computer Systems lecture latex header
%
% run the following programs:
% latex lecture
%   -> Creates lecture.dvi, plus latex intermediate files
% latex lecture
%   -> Run a second time to get cross-references right
% dvi2ps lecture > lecture.ps
%   -> Merges the figures with the text, can also pipe to lpr
%
\documentstyle[12pt,pocs-header]{article}
\Scribe{Kah-Kay Sung}
\Lecturer{Eric Brewer}
\LectureNumber{10}
\LectureDate{October 16, 1991}
\include{macros} % used to include figures in this document
\begin{document}
\MakeScribeTop

\section{Handouts}

Handouts 23 ({\em An Introduction to Programming with Threads}), 24
({\em Specification of a Hierarchical Directory}), Notes on Lecture 6. 

\section{Overview}

In this lecture, we examine two different specifications of a {\em
hierarchial directory} that includes symbolic links.  The specifications
do not deal with (1) date-time stamps or (2) protection labels.
However, these features can be easily added to the specifications.

The first specification is a straight-forward approach that maps
complete pathnames to file system objects.  The second specification
more closely follows a Unix-like system in which each element of a
pathname is looked up in a directory.

\section{Complete Filename Map}

The following describes the straight-forward (flat) approach:

\begin{verbatim}
MODULE HierDir1 =

TYPE
  N = STRING                    % Name
  PN = SEQ[N]                   % Pathname
  L = PN                        % Link
  Dir = NULL			
  Z = UNION[File, L, Dir]       % What you get when you look up a pn
  Y = PN->Z			

VAR
  dir: Y := Y{root->nil}
  root: PN := PN{}
  dots: SET[N] := {".", ".."}

% INVARIANT
%       dir(root) IS Dir
%     /\ pn.asSet * dots # {} ==> ~dir!pn
%     /\ dir!pn /\ pn' < pn ==> dir(pn') IS Dir

% function to get rid of links & dots and return a
% pathname that can be looked up in the directory

FUNC Normalize (pn) -> PN RAISES failure =
        dir!pn /\ ~dir(pn) IS L => RET pn	
     [] VAR pn1, pn2 | pn = pn1 + pn2		
                /\ dir!pn1 /\ dir(pn1) IS L
                        => RET Normalize(dir(pn1) + pn2)
     [] VAR pn1, pn2, n | (pn = pn1 ++ "." + pn2
                           \/ pn = pn1 ++ n ++ ".." + pn2)
                        /\ Lookup(pn1) IS Dir	
			=> RET Normalize(pn1 + pn2)
    [*] RAISE failure

FUNC Lookup (pn) -> Z = dir(Normalize(pn))

PROC Create (pn, z) RAISES failure =
    <<
        VAR pn0 := Normalize (pn.reml),
            pn' := pn0 ++ pn.last  |
                ~dir(pn0) IS Dir \/ dir!pn' => RAISE failure
            [*] dir[pn'] := z; RET
    >>

PROC Rename (old:PN, new:PN) RAISES failure =
    <<
        VAR old' := Normalize(old),
            new0 := Normalize(new.reml),
            new' := new0 + new.last  |
                old' < new' \/ ~dir(new0) IS Dir \/ dir!new' => RAISE failure
            [*] DO VAR pn | dir!(old'+pn) =>
                         dir := dir{old'+pn -> }{new' + pn -> dir(old' + pn)}
                OD
    >>

END HierDir1
\end{verbatim}


\subsection{Invariant}

\begin{verbatim}

% INVARIANT
%       dir(root) IS Dir
%     /\ pn.asSet * dots # {} ==> ~dir!pn                          [1]
%     /\ dir!pn /\ pn' < pn ==> dir(pn') IS Dir                    [2] 

\end{verbatim}

Term {\tt [1]} says that if a pathname {\tt pn} contains elements that
are dots, ie.  either ``.'' or ``..'', then there is no entry in the
directory table that maps {\tt pn} to a file system object.  In other
words, {\tt x/./y} and {\tt a/b/../c} are not complete pathnames and so
do not have an entry in the directory table.

Term {\tt [2]} states that if a pathname, {\tt pn}, is defined in the
directory table, then any prefix of {\tt pn} must be a directory.  For
example, if {\tt x/y/z} is defined, then {\tt x} and {\tt x/y} are
both directories.

Notice that the invariant does not allow pathnames with links in the
directory table.  This is an important feature as we shall see why
later.

\subsection{Normalize}

\begin{verbatim}

FUNC Normalize (pn) -> PN RAISES failure =
	dir!pn /\ ~dir(pn) IS L => RET pn                          [1] 
     [] VAR pn1, pn2 | pn = pn1 + pn2                              [2]
                /\ dir!pn1 /\ dir(pn1) IS L
                        => RET Normalize(dir(pn1) + pn2)
     [] VAR pn1, pn2, n | (pn = pn1 ++ "." + pn2                   [3]
                           \/ pn = pn1 ++ n ++ ".." + pn2)
                        /\ Lookup(pn1) IS Dir	
			=> RET Normalize(pn1 + pn2)
    [*] RAISE failure

\end{verbatim}

The purpose of {\tt Normalize} is to get rid of links and dots in a
pathname.  It returns a complete pathname that can be looked up in the
directory table.

Normalize works as follows:  {\tt [1]} If its argument, {\tt pn}, is a
complete pathname that is defined in the directory table, it returns
{\tt pn}.  Otherwise, it recursively {\tt [2]} replaces elements that
are links in {\tt pn} by their complete pathnames and/or {\tt [3]}
elements that are dots in {\tt pn} by their complete pathnames.  If the
conditions for {\tt [2]} and {\tt [3]} are both satisfied, either of
them may be applied first.

To interpret pathnames with reference to the current working directory,
we can specify another version of {\tt Normalize} that appends the
current working directory's pathname as a prefix to the argument when it
is first called.

The returned value of {\tt Normalize}, {\tt x}, obeys the invariant:
{\tt dir!x /$\backslash$ ~(dir(x) IS L}.  In other words, the returned
value, if there is one, is always defined in the directory table and is
never a {\em link}.


\subsection{Lookup}

\begin{verbatim}

FUNC Lookup (pn) -> Z = dir(Normalize(pn))

\end{verbatim}

Notice that {\tt Lookup} always returns a file or a directory as a
consequence of {\tt Normalize}.

\subsection{Create}

\begin{verbatim}
	
PROC Create (pn, z) RAISES failure =
    <<
        VAR pn0 := Normalize (pn.reml), pn' := pn0 ++ pn.last  |
            ~dir(pn0) IS Dir \/ dir!pn'                           [1]
	      => RAISE failure
            [*] dir[pn'] := z; RET
    >>

\end{verbatim}

We first check to see (in {\tt [1]}) that {\tt pn}, up to but not
including its last element, maps to a directory and that the file we
want to create does not already exist.  Only then can we create a file
with the given pathname.


\subsection{Rename}

\begin{verbatim}

PROC Rename (old:PN, new:PN) RAISES failure =
    <<
        VAR old' := Normalize(old),
            new0 := Normalize(new.reml),
            new' := new0 + new.last  |
                old' < new'                                       [1]
	          \/ ~dir(new0) IS Dir \/ dir!new'                [2]
		  => RAISE failure
            [*] DO VAR pn | dir!(old'+pn) =>                      [3]
                   dir := dir{old'+pn -> }{new' + pn -> dir(old' + pn)}
                OD
    >>

\end{verbatim}

Two conditions must be checked for.  First, the old pathname, {\tt old},
cannot be a proper prefix of the new pathname {\tt new} {\tt [1]}.  That
is, the following invocation of {\tt Rename} will signal a failure:
{\tt Rename(x/y, x/y/z)}.  Second, there cannot be a file or directory
that already exists with the new pathname {\tt new} {\tt [2]}.

Step {\tt [3]} performs the renaming of all files and subdirectories of
{\tt old}.  In the current flat implementation, this involves too much
work to be performed in a single atomic step, as required by the module
specification.  Incidentally, this is one reason why the {\em flat}
hierarchical directory implementation is not popular in reality.

\section{Per-element Filename Map}

We now present part of the second approach for specifying hierarchical
directories:

\begin{verbatim}
MODULE HierDir2 =

TYPE
  N = STRING                    % Name
  PN = SEQ[N]                   % Pathname
  L = PN                        % Link
  Dir = REF[N -> Z]
  DN = RECORD[dir, n]
  Z = UNION[File, L, Dir]       % What you get when you look up a pn
  Y = PN->Z


VAR

  root:Dir := ((N -> Z) {"."->root}).new
  % Note that ".." is not defined in the root directory.

% "." and ".." must be intialized appropriately everywhere.  This
% should be done in Create when creating a directory.

FUNC Normalize (dir, pn) -> DN RAISES failure =
    << VAR dir', pn' |
          pn.size = 0 \/ ~dir^!(pn.head) => RAISE failure
       [*] BEGIN
            VAR n := pn.head, z := (dir^)(n)  |
                z IS L => 
		    pn' := z + pn.tail; dir' := root
             [] z IS Dir /\ pn.size > 1 => 
		    pn := pn.tail; dir' := z
            [*] pn.size = 1 => 
		    RET DN{dir:=dir, n:=pn.head}
            [*] RAISE failure
           END;

%%%     >> % Put close atomicity bracket here to model Unix.

        RET Normalize(dir', pn')
        >>

FUNC Lookup (pn) -> Z =
    VAR dn := Normalize(root, pn) |
        RET (dn.dir^)(dn.n)

END HierDir2
\end{verbatim}


\subsection{Normalize}
\label{norm}

\begin{verbatim}
	
FUNC Normalize (dir, pn) -> DN RAISES failure =
    << VAR dir', pn' |
          pn.size = 0 \/ ~dir^!(pn.head) => RAISE failure           [1] 
       [*] BEGIN
            VAR n := pn.head, z := (dir^)(n)  |
                z IS L =>                                           [2]
		    pn' := z + pn.tail; dir' := root
             [] z IS Dir /\ pn.size > 1 => 
		    pn := pn.tail; dir' := z
            [*] pn.size = 1 =>                                      [3]
		    RET DN{dir:=dir, n:=pn.head}
            [*] RAISE failure
           END;

%%%     >> % Put close atomicity bracket here to model Unix.

        RET Normalize(dir', pn')
        >>

\end{verbatim}

{\tt Normalize} walks down the hierarchical directory structure starting
at {\tt dir}, following the elements of {\tt pn}.  At each level, it
first checks that {\tt pn} is not empty and that the first entry of {\tt
pn} must be defined {\tt [1]}.  If the first element is a {\em link}, it
replaces the {\em link} element with its complete pathname and starts
recursing again from the {\tt root} directory {\tt [2]}.  If the first
element is also the final element of {\tt pn} {\tt [3]}, we return the
file system object indexed by {\tt pn.head}, or signal {\tt failure} if
{\tt pn.head} is not defined in the current directory {\tt dir}.
Otherwise, {\tt Normalize} proceeds down the next level if {\tt pn.head}
maps to a directory.

Notice that according to the series of checks above, a failure would
occur (as we want) if somewhere along the path, {\tt Normalize}
encounters a pathname {\tt x/y/$\cdots$} whose first entry, {\tt x},
maps to a file.


\section{Unix Bashing}

To model Unix, the recursive {\tt Normalize} call is moved outside the
atomicity brackets (see the {\tt Normalize} specification of
Section~\ref{norm}).  In other words, each level of the {\tt Normalize}
procedure is executed atomically but not the recursive invocation.  This
can cause directory operations to behave in strange ways.  Consider the
following hierarchical directory structure:

\begin{verbatim}

/a is a directory.
/a/b is a directory.
/a/b/c is a directory.
/a/b/c/d is a file.
/e is a directory.

\end{verbatim}

with two processes running concurrently:

\begin{verbatim}

Process 1                                     Process 2
---------                                     ---------

lookup(/a/b/g)                                rename(/a/b, /e/h)
                                              rename(/e/h/c/d, /e/h/g)

\end{verbatim}

What results are possible for Process 1?

Basically, other processes may run between the lookup of each component
in the pathname.   In the above example, if Process 1 were to first
lookup /a and /a/b, then process 2 executed both renames, then process 1
continued and looked up the 'g' component, the lookup may succeed, even
though at no time was there a file at pathname /a/b/g (see
Figure~\ref{fig:bash}).

\begin{figure}
\centerline{\psfig{width=6in,figure=dirs.idraw}}
\caption{Possible inconsistant result from process 1.  (a) The original
directory structure.  (b) Process 1 traverses {\tt /a/b} followed by
first renaming by process 2.  (c) Second renaming by process 2.  (d)
Process 1 errorneously finds {\tt /a/b/g}.} 
\label{fig:bash}
%\label{netfig}
\end{figure}

There was some discussion about whether this behavior is good or bad.
It does not match the behavior described by our first specification
and it would be hard to modify our first specification to accurately
describe this behavior.   We shouldn't be too surprised that it might
be hard to write a specification long after something was implemented,
but a system that conforms to a concise and straightforward
specification might be more easily understood by its users.

``The religion of this course is that you should first say what you
are going to do and then do it.''  However, feedback from implementation
can be used to improve specification.

\section{Lock Coupling}

In the above example, process 1 ended up having an ``inconsistant''
result because process 2 could access file system objects below (ie.
descendants of) the current node of process 1.  One way of overcoming
this problem is for processes to use a technique called {\em lock
coupling} that prevents other processes like process 2 from ``slipping
by''.  A {\em lock} is a mutex associated with a node of a hierarchical
directory structure.  To perform {\em lock coupling}, a process must
hold 2 consecutive locks along the directory path it is searching.
Locking starts from the root node and leapfrogs along the
path as it is being traversed (see Figure~\ref{fig:lock}).

\begin{figure}
\centerline{\psfig{width=6in,figure=locks.idraw}}
\caption{Snapshots of lock coupling along search path for {\tt /e/h/g}.}
\label{fig:lock}
%\label{netfig}
\end{figure}


With links, the locking problem becomes more complicated since it is
possible for another process to access file system objects beneath the
current node through links.  One way of solving this problem is to have
a process lock up entire {\em levels} of the directory, where the {\em
level} of a node is its shortest distance from the root node.  This
strategy is known as {\em tree locking}.


\end{document}
