% to get hardcopy of this lecture, you need the following files (plus the % normal latex and tex base directories): % lecture.tex: latex source file % macros.tex: figure macros % psfig.tex: postscript figure (psfig) macro definition % dirs.idraw: first figure, created with idraw % locks.idraw: second figure, created with idraw % pocs-header.sty: Principles of Computer Systems lecture latex header % % run the following programs: % latex lecture % -> Creates lecture.dvi, plus latex intermediate files % latex lecture % -> Run a second time to get cross-references right % dvi2ps lecture > lecture.ps % -> Merges the figures with the text, can also pipe to lpr % \documentstyle[12pt,pocs-header]{article} \Scribe{Kah-Kay Sung} \Lecturer{Eric Brewer} \LectureNumber{10} \LectureDate{October 16, 1991} \include{macros} % used to include figures in this document \begin{document} \MakeScribeTop \section{Handouts} Handouts 23 ({\em An Introduction to Programming with Threads}), 24 ({\em Specification of a Hierarchical Directory}), Notes on Lecture 6. \section{Overview} In this lecture, we examine two different specifications of a {\em hierarchial directory} that includes symbolic links. The specifications do not deal with (1) date-time stamps or (2) protection labels. However, these features can be easily added to the specifications. The first specification is a straight-forward approach that maps complete pathnames to file system objects. The second specification more closely follows a Unix-like system in which each element of a pathname is looked up in a directory. \section{Complete Filename Map} The following describes the straight-forward (flat) approach: \begin{verbatim} MODULE HierDir1 = TYPE N = STRING % Name PN = SEQ[N] % Pathname L = PN % Link Dir = NULL Z = UNION[File, L, Dir] % What you get when you look up a pn Y = PN->Z VAR dir: Y := Y{root->nil} root: PN := PN{} dots: SET[N] := {".", ".."} % INVARIANT % dir(root) IS Dir % /\ pn.asSet * dots # {} ==> ~dir!pn % /\ dir!pn /\ pn' < pn ==> dir(pn') IS Dir % function to get rid of links & dots and return a % pathname that can be looked up in the directory FUNC Normalize (pn) -> PN RAISES failure = dir!pn /\ ~dir(pn) IS L => RET pn [] VAR pn1, pn2 | pn = pn1 + pn2 /\ dir!pn1 /\ dir(pn1) IS L => RET Normalize(dir(pn1) + pn2) [] VAR pn1, pn2, n | (pn = pn1 ++ "." + pn2 \/ pn = pn1 ++ n ++ ".." + pn2) /\ Lookup(pn1) IS Dir => RET Normalize(pn1 + pn2) [*] RAISE failure FUNC Lookup (pn) -> Z = dir(Normalize(pn)) PROC Create (pn, z) RAISES failure = << VAR pn0 := Normalize (pn.reml), pn' := pn0 ++ pn.last | ~dir(pn0) IS Dir \/ dir!pn' => RAISE failure [*] dir[pn'] := z; RET >> PROC Rename (old:PN, new:PN) RAISES failure = << VAR old' := Normalize(old), new0 := Normalize(new.reml), new' := new0 + new.last | old' < new' \/ ~dir(new0) IS Dir \/ dir!new' => RAISE failure [*] DO VAR pn | dir!(old'+pn) => dir := dir{old'+pn -> }{new' + pn -> dir(old' + pn)} OD >> END HierDir1 \end{verbatim} \subsection{Invariant} \begin{verbatim} % INVARIANT % dir(root) IS Dir % /\ pn.asSet * dots # {} ==> ~dir!pn [1] % /\ dir!pn /\ pn' < pn ==> dir(pn') IS Dir [2] \end{verbatim} Term {\tt [1]} says that if a pathname {\tt pn} contains elements that are dots, ie. either ``.'' or ``..'', then there is no entry in the directory table that maps {\tt pn} to a file system object. In other words, {\tt x/./y} and {\tt a/b/../c} are not complete pathnames and so do not have an entry in the directory table. Term {\tt [2]} states that if a pathname, {\tt pn}, is defined in the directory table, then any prefix of {\tt pn} must be a directory. For example, if {\tt x/y/z} is defined, then {\tt x} and {\tt x/y} are both directories. Notice that the invariant does not allow pathnames with links in the directory table. This is an important feature as we shall see why later. \subsection{Normalize} \begin{verbatim} FUNC Normalize (pn) -> PN RAISES failure = dir!pn /\ ~dir(pn) IS L => RET pn [1] [] VAR pn1, pn2 | pn = pn1 + pn2 [2] /\ dir!pn1 /\ dir(pn1) IS L => RET Normalize(dir(pn1) + pn2) [] VAR pn1, pn2, n | (pn = pn1 ++ "." + pn2 [3] \/ pn = pn1 ++ n ++ ".." + pn2) /\ Lookup(pn1) IS Dir => RET Normalize(pn1 + pn2) [*] RAISE failure \end{verbatim} The purpose of {\tt Normalize} is to get rid of links and dots in a pathname. It returns a complete pathname that can be looked up in the directory table. Normalize works as follows: {\tt [1]} If its argument, {\tt pn}, is a complete pathname that is defined in the directory table, it returns {\tt pn}. Otherwise, it recursively {\tt [2]} replaces elements that are links in {\tt pn} by their complete pathnames and/or {\tt [3]} elements that are dots in {\tt pn} by their complete pathnames. If the conditions for {\tt [2]} and {\tt [3]} are both satisfied, either of them may be applied first. To interpret pathnames with reference to the current working directory, we can specify another version of {\tt Normalize} that appends the current working directory's pathname as a prefix to the argument when it is first called. The returned value of {\tt Normalize}, {\tt x}, obeys the invariant: {\tt dir!x /$\backslash$ ~(dir(x) IS L}. In other words, the returned value, if there is one, is always defined in the directory table and is never a {\em link}. \subsection{Lookup} \begin{verbatim} FUNC Lookup (pn) -> Z = dir(Normalize(pn)) \end{verbatim} Notice that {\tt Lookup} always returns a file or a directory as a consequence of {\tt Normalize}. \subsection{Create} \begin{verbatim} PROC Create (pn, z) RAISES failure = << VAR pn0 := Normalize (pn.reml), pn' := pn0 ++ pn.last | ~dir(pn0) IS Dir \/ dir!pn' [1] => RAISE failure [*] dir[pn'] := z; RET >> \end{verbatim} We first check to see (in {\tt [1]}) that {\tt pn}, up to but not including its last element, maps to a directory and that the file we want to create does not already exist. Only then can we create a file with the given pathname. \subsection{Rename} \begin{verbatim} PROC Rename (old:PN, new:PN) RAISES failure = << VAR old' := Normalize(old), new0 := Normalize(new.reml), new' := new0 + new.last | old' < new' [1] \/ ~dir(new0) IS Dir \/ dir!new' [2] => RAISE failure [*] DO VAR pn | dir!(old'+pn) => [3] dir := dir{old'+pn -> }{new' + pn -> dir(old' + pn)} OD >> \end{verbatim} Two conditions must be checked for. First, the old pathname, {\tt old}, cannot be a proper prefix of the new pathname {\tt new} {\tt [1]}. That is, the following invocation of {\tt Rename} will signal a failure: {\tt Rename(x/y, x/y/z)}. Second, there cannot be a file or directory that already exists with the new pathname {\tt new} {\tt [2]}. Step {\tt [3]} performs the renaming of all files and subdirectories of {\tt old}. In the current flat implementation, this involves too much work to be performed in a single atomic step, as required by the module specification. Incidentally, this is one reason why the {\em flat} hierarchical directory implementation is not popular in reality. \section{Per-element Filename Map} We now present part of the second approach for specifying hierarchical directories: \begin{verbatim} MODULE HierDir2 = TYPE N = STRING % Name PN = SEQ[N] % Pathname L = PN % Link Dir = REF[N -> Z] DN = RECORD[dir, n] Z = UNION[File, L, Dir] % What you get when you look up a pn Y = PN->Z VAR root:Dir := ((N -> Z) {"."->root}).new % Note that ".." is not defined in the root directory. % "." and ".." must be intialized appropriately everywhere. This % should be done in Create when creating a directory. FUNC Normalize (dir, pn) -> DN RAISES failure = << VAR dir', pn' | pn.size = 0 \/ ~dir^!(pn.head) => RAISE failure [*] BEGIN VAR n := pn.head, z := (dir^)(n) | z IS L => pn' := z + pn.tail; dir' := root [] z IS Dir /\ pn.size > 1 => pn := pn.tail; dir' := z [*] pn.size = 1 => RET DN{dir:=dir, n:=pn.head} [*] RAISE failure END; %%% >> % Put close atomicity bracket here to model Unix. RET Normalize(dir', pn') >> FUNC Lookup (pn) -> Z = VAR dn := Normalize(root, pn) | RET (dn.dir^)(dn.n) END HierDir2 \end{verbatim} \subsection{Normalize} \label{norm} \begin{verbatim} FUNC Normalize (dir, pn) -> DN RAISES failure = << VAR dir', pn' | pn.size = 0 \/ ~dir^!(pn.head) => RAISE failure [1] [*] BEGIN VAR n := pn.head, z := (dir^)(n) | z IS L => [2] pn' := z + pn.tail; dir' := root [] z IS Dir /\ pn.size > 1 => pn := pn.tail; dir' := z [*] pn.size = 1 => [3] RET DN{dir:=dir, n:=pn.head} [*] RAISE failure END; %%% >> % Put close atomicity bracket here to model Unix. RET Normalize(dir', pn') >> \end{verbatim} {\tt Normalize} walks down the hierarchical directory structure starting at {\tt dir}, following the elements of {\tt pn}. At each level, it first checks that {\tt pn} is not empty and that the first entry of {\tt pn} must be defined {\tt [1]}. If the first element is a {\em link}, it replaces the {\em link} element with its complete pathname and starts recursing again from the {\tt root} directory {\tt [2]}. If the first element is also the final element of {\tt pn} {\tt [3]}, we return the file system object indexed by {\tt pn.head}, or signal {\tt failure} if {\tt pn.head} is not defined in the current directory {\tt dir}. Otherwise, {\tt Normalize} proceeds down the next level if {\tt pn.head} maps to a directory. Notice that according to the series of checks above, a failure would occur (as we want) if somewhere along the path, {\tt Normalize} encounters a pathname {\tt x/y/$\cdots$} whose first entry, {\tt x}, maps to a file. \section{Unix Bashing} To model Unix, the recursive {\tt Normalize} call is moved outside the atomicity brackets (see the {\tt Normalize} specification of Section~\ref{norm}). In other words, each level of the {\tt Normalize} procedure is executed atomically but not the recursive invocation. This can cause directory operations to behave in strange ways. Consider the following hierarchical directory structure: \begin{verbatim} /a is a directory. /a/b is a directory. /a/b/c is a directory. /a/b/c/d is a file. /e is a directory. \end{verbatim} with two processes running concurrently: \begin{verbatim} Process 1 Process 2 --------- --------- lookup(/a/b/g) rename(/a/b, /e/h) rename(/e/h/c/d, /e/h/g) \end{verbatim} What results are possible for Process 1? Basically, other processes may run between the lookup of each component in the pathname. In the above example, if Process 1 were to first lookup /a and /a/b, then process 2 executed both renames, then process 1 continued and looked up the 'g' component, the lookup may succeed, even though at no time was there a file at pathname /a/b/g (see Figure~\ref{fig:bash}). \begin{figure} \centerline{\psfig{width=6in,figure=dirs.idraw}} \caption{Possible inconsistant result from process 1. (a) The original directory structure. (b) Process 1 traverses {\tt /a/b} followed by first renaming by process 2. (c) Second renaming by process 2. (d) Process 1 errorneously finds {\tt /a/b/g}.} \label{fig:bash} %\label{netfig} \end{figure} There was some discussion about whether this behavior is good or bad. It does not match the behavior described by our first specification and it would be hard to modify our first specification to accurately describe this behavior. We shouldn't be too surprised that it might be hard to write a specification long after something was implemented, but a system that conforms to a concise and straightforward specification might be more easily understood by its users. ``The religion of this course is that you should first say what you are going to do and then do it.'' However, feedback from implementation can be used to improve specification. \section{Lock Coupling} In the above example, process 1 ended up having an ``inconsistant'' result because process 2 could access file system objects below (ie. descendants of) the current node of process 1. One way of overcoming this problem is for processes to use a technique called {\em lock coupling} that prevents other processes like process 2 from ``slipping by''. A {\em lock} is a mutex associated with a node of a hierarchical directory structure. To perform {\em lock coupling}, a process must hold 2 consecutive locks along the directory path it is searching. Locking starts from the root node and leapfrogs along the path as it is being traversed (see Figure~\ref{fig:lock}). \begin{figure} \centerline{\psfig{width=6in,figure=locks.idraw}} \caption{Snapshots of lock coupling along search path for {\tt /e/h/g}.} \label{fig:lock} %\label{netfig} \end{figure} With links, the locking problem becomes more complicated since it is possible for another process to access file system objects beneath the current node through links. One way of solving this problem is to have a process lock up entire {\em levels} of the directory, where the {\em level} of a node is its shortest distance from the root node. This strategy is known as {\em tree locking}. \end{document}