% to get hardcopy of this lecture, you need the following files (plus the
% normal latex and tex base directories):
%     lecture.tex:     latex source file
%     macros.tex:      figure macros
%     psfig.tex:       postscript figure (psfig) macro definition
%     sort.idraw:      first figure, created with idraw
%     security.idraw:  second figure, created with idraw
%     proof.idraw:     third figure, created with idraw
%     pocs-header.sty: Principles of Computer Systems lecture latex header
%
% run the following programs:
% latex lecture
%   -> Creates lecture.dvi, plus latex intermediate files
% latex lecture
%   -> Run a second time to get cross-references right
% dvi2ps lecture > lecture.ps
%   -> Merges the figures with the text, can also pipe to lpr
%
\documentstyle[12pt,pocs-header]{article}
\Scribes{Tim Shepard and Andrew Heybey}
\Lecturer{Butler Lampson}
\LectureNumber{10}
\LectureDate{October 17, 1990}
\include{macros} % used to include figures in this document
\begin{document}
\MakeScribeTop

\section{Specification of a Hierarchical Directory}

Included here are two different specifications of a hierarchical
directory that includes symbolic links.  The first is the
straightforward approach that maps complete pathnames to file system
objects, and the other more closely follows a Unix-like system in
which each element of a pathname is looked up in a directory.

\begin{verbatim}
MODULE HierDir1 =

TYPE
  N = STRING                    % Name
  PN = SEQ[N]                   % Pathname
  L = PN                        % Link
  Dir = NULL
  Z = UNION[File, L, Dir]       % What you get when you look up a pn
  Y = PN->Z

VAR
  dir: Y := Y{root->nil}
  root: PN := PN{}
  dots: SET[N] := {".", ".."}

% INVARIANT
%       dir(root) IS Dir
%     /\ pn.asSet * dots # {} ==> ~dir!pn
%     /\ dir!pn /\ pn' < pn ==> dir(pn') IS Dir

% function to get rid of all this nonsense [links & such] and return a
% pathname that can be looked up in the directory
FUNC Normalize (pn) -> PN =
        dir!pn /\ ~dir(pn) IS L => RET pn
     [] VAR pn1, pn2 | pn = pn1 + pn2
                /\ dir!pn1 /\ dir(pn1) IS L
                        => RET Normalize(dir(pn1) + pn2)
     [] VAR pn1, pn2, n | (pn = pn1 ++ "." + pn2
                           \/ pn = pn1 ++ n ++ ".." + pn2)
                        /\ Lookup(pn1) IS Dir => RET Normalize(pn1 + pn2)
    [*] RAISE failure

FUNC Lookup (pn) -> Z = dir(Normalize(pn))

PROC Create (pn, z) =
    <<
        VAR pn0 := Normalize (pn.reml),
            pn' := pn0 ++ pn.last  |
                ~dir(pn0) IS Dir \/ dir!pn' => RAISE failure
            [*] dir[pn'] := z; RET
    >>

PROC Rename (old:PN, new:PN) =
    <<
        VAR old' := Normalize(old),
            new0 := Normalize(new.reml),
            new' := new0 + new.last  |
                old' < new' \/ ~dir(new0) IS Dir \/ dir!new' => RAISE failure
            [*] DO VAR pn | dir!(old'+pn) =>
                         dir := dir{old'+pn -> }{new' + pn -> dir(old' + pn)}
                OD
    >>

END HierDir1
\end{verbatim}

\begin{verbatim}
MODULE HierDir2 =

TYPE
  N = STRING                    % Name
  PN = SEQ[N]                   % Pathname
  L = PN                        % Link
  Dir = REF[N -> Z]
  DN = RECORD[dir, n]
  Z = UNION[File, L, Dir]       % What you get when you look up a pn
  Y = PN->Z

% "." and ".." must be intialized appropriately everywhere.

FUNC Normalize (dir, pn) -> DN =
    VAR dir', pn' |
        << pn.size = 0 \/ ~dir^!(pn.head) => RAISE failure
       [*] BEGIN
            VAR n := pn.head, z := (dir^)(n)  |
                z IS L => pn' := z + pn.tail; dir' := root
             [] z IS Dir /\ pn.size > 1 => pn := pn.tail; dir' := z
            [*] pn.size = 1 => RET DN{dir:=dir, n:=pn.head}
            [*] RAISE failure
           END;
%       >>    % Put close atomicity bracket here to more closely model Unix
        RET Normalize(dir', pn')
        >>

FUNC Lookup (pn) -> Z =
    VAR dn := Normalize(root, pn) |
        RET (dn.dir^)(dn.n)

END HierDir2
\end{verbatim}


\section{Unix Bashing}

Butler Lampson said that Unix can behave in ways which he finds
troubling.  He gave the following example.  (He did not say whose
version Unix he was referring to.)

\begin{verbatim}

/a is a directory.
/a/b is a directory.
/a/b/c is a directory.
/a/b/c/d is a file.
/e is a directory.

\end{verbatim}

Consider two processes running concurrently:

\begin{verbatim}

Process 1                                     Process 2
---------                                     ---------

lookup(/a/b/g)                                rename(/a/b, /e/h)
                                              rename(/e/h/c/d, /e/h/g)

\end{verbatim}

What results are possible for Process 1?

On Unix, other processes may run between the lookup of each component
in the pathname.   In the above example, if Process 1 were to first
lookup /a and /a/b, then process 2 executed both renames, then process
1 continued and looked up the 'g' component, the lookup may succeed,
even though at no time was there a file at pathname /a/b/g.

So to accurately model the behavior of Unix, the specification for
Normalize above must be modified by moving the recursive call outside
the atomicity brackets.

There was some discussion about whether this behavior is good or bad.
It does not match the behavior described by our first specification
and it would be hard to modify our first specification to accurately
describe this behavior.   We shouldn't be too surprised that it might
be hard to write a specification long after something was implemented,
but a system that conforms to a concise and straightforward
specification might be more easily understood by its users.

``The religion of this course is that you should first say what you
are going to do and then do it.''  However, feedback from implementation
can be used to improve specification.

\section{Using Mutexes}

Mutexes can be used to solve these sorts of problems.    

\begin{verbatim}

Lock(v) -> M

\end{verbatim}

This function gives you a mutex for variable v.   Now one could
surround a statement S that touches variable v like this:

\begin{verbatim}

Lock(v).acquire ; S ; Lock(v).rel

\end{verbatim}

There is a spectrum to choose from here.  At the extremes one could
have a separate mutex for each variable, or one could have a single
mutex for all variables.

Proper use of mutexes allows the programmer to think of the effect of
a statement S in terms of its input output relationship even in
concurrent systems.


This approach could be applied to the Normalize procedure so that it
would appear to behave atomically.   It could lock each directory as
it descends the tree and release all the locks once it has finished
the lookup.   Two problems:   Avoiding deadlock with yourself and
deadlock with another process.    One way to avoid deadlock is to
order all of the resources and have each process acquire the locks
with respect to the same ordering.   Another approach is to detect
when deadlocks have occurred, then back out and retry.   The need to be
able to back out can complicate the programmer's task.  One can also
use readers/writers locks.  Since programs most
often read directories and seldom write them, the chances of a deadlock
will be lessened.


\end{document}
