%--------------------------------------------------------
\documentstyle[12pt,pocs-header]{article}
\Scribes{Andr\'e DeHon}
\Lecturer{Butler Lampson}
\LectureNumber{11}
\LectureDate{October 22, 1990}
\include{macros} % used to include figures in this document
\begin{document}
\MakeScribeTop

\section{Administrivia}

\begin{table}[h]
\centering
\begin{tabular}{|c|l|} \hline
\multicolumn{1}{|c|}{Handout} &
\multicolumn{1}{c|}{Title} \\ \hline
22 & Concurrent Disk Reads  \\
23 & Problem Set \#4 \\
Notes & Notes for Lectures 8 \& 9 \\
\hline
\end{tabular}
\end{table}

Problem Set \#4 is due Monday, October 29, 1990.

\section{Review Directories from Lecture \#10}

Last lecture we looked at two directory schemes:
\begin{enumerate}
 \item Each pathnames is a sequence of strings (plus a few special
       sequences were included for convenience, {\it i.e.\ } ``.''  and
       ``..''.) 
 \item The directory is an acyclic directed graph (DAG)
\end{enumerate}
Both of these directory strategies have reasonable implementations.

\subsection{Sequence of Strings}

The directory structure where each pathname is a sequence of strings, is
implemented in a straight-forward manner using a B-Tree to map each
``pathname'' to a file.  A naive-implementation might be wasteful of space
when there were deep directory structures ({\it i.e.\ } there are many files
with the same, long prefix pathname -- {\it e.g.\ } /a/b/c/d/e/f,
/a/b/c/d/e/g, /a/b/c/d/e/h, etc.\ ).  However, this space inefficiency can
be avoided by using a {\it front compression} technique.  In this case, the
lookup cost is dependent only on the effectiveness of the B-Tree and {\bf
not} on the depth of the file hierarchy.  

\paragraph{Example B-Tree Structure}  Consider a B-Tree implemented using 4K
disk blocks.  If we assume 40-byte file entries, such a disk block will act
as a 100 degree node in the B-Tree.  With this structure, we only need two
levels to support 10,000 entries in our filesystem; three levels will
support 1,000,000 entries.  Thus, even for filesystems with many files, any
file can be accessed in at most three disk reads.  Performance can, of
course, be optimized further by keeping a cache in memory.

The drawback to this directory scheme is that the {\bf rename} operation is
very expensive.  When a directory is renamed, it is necessary to change the
sequence of strings tagging every file entry that falls underneath the
renamed directory.  Guaranteeing atomicity for this operation is not a
major problem since we can use the log method explored in Handout \#10 and
Problem Set \#2. This directory scheme is not used widely because of the
difficulty of the rename operation.

\subsection{Concurrency in Sequence of Strings Directory} 

We can handle concurrency by using concurrent B-Trees.  {\it i.e.\ }
Programs acquire locks on pages as they work down the B-Tree -- care must,
of course, be taken to avoid deadlock during certain operations ({\it e.g.\
} rotations).

A few operations, like rotations, require considerably more effort and care
to perform concurrently.  Additionally, the ease with which some operations
can be performed depends on interactions from concurrently executing
processes.  One common technique for dealing with these kind of problem is
to have a separate ``fast'' and ``slow'' path to perform an operation.  The
fast path handles the normal case without taking excessive care to work in
all cases.  The fast path does not modify anything until it knows that it
will be able to succeed.  If the process gets far enough to succeed down
the fast path, then it completes the operation quickly.  If the process
detects that things are too complicated to be handled via the fast path, it
releases all of its locks and restarts using the slow path.  When
implementing a fast/slow path scheme, it is generally wise to include
instrumentation to verify that the ``fast'' path for which you have
optimized is indeed the normal case during program execution.

\paragraph{Example}  A processes would traverse the tree acquiring locks as
normal.  When it finds the directory to be renamed, it might discover that
the requested operation is a rotation and cannot be handled by the fast
path.  The process would then release locks and restart on the slow path
-- perhaps using the data it acquired on the slow path as a hint.
When the process has acquired the necessary locks on the slow path, it
would check that it's knowledge about directory locations is still valid.
If its information remains valid, the process can go ahead and perform the
operation.  If its information is stale, it releases all of its held locks
and starts over.  This kind of operation should converge rapidly unless the
system is terribly overloaded.


\subsection{Directed Acyclic Graph}

The directed acyclic graph is the most common scheme for implementing
directories.  Access is generally straight-forward.  Processes start at the
root of the tree and acquire and hold the locks for each directory on their
way down the directory hierarchy.  Unfortunately, this scheme is prone to
deadlock.

{\it N.B.} If the directory really was a simple-tree ({\it i.e.\ } no
cycles), this problem wouldn't necessarily arise.  The tree-structure
induces a partial ordering of nodes;  using this partial ordering for lock
acquisition is sufficient to avoid deadlocks when performing lookups.
Renames require a total ordering on the nodes, but this is also trivial to
add to a non-acyclic tree.

In general, however, a directory is not a simple tree structure.  A
directory generally has symbolic links and parent links which allow
arbitrary cycles in the directory graph.  To avoid deadlock in this
situation, a process traversing the directory must be careful to avoid both
its own locks and those of other processes.

\subsection{Deadlock Strategy}

One simple strategy for dealing with deadlock is to simply let it happen.
As long as deadlock can be detected by some means such as a time-out on the
deadlocking operation, the operation can simply be aborted and restarted.
This will always work and be correct as long as the operation has not made
any changes prior to being aborted.  Some care must be taken with this
strategy to guarantee forward progress and to avoid process starvation.

This strategy requires no knowledge of how deadlock occurs in the system.
It works well as long as the expected completion time of operations is
sufficiently bounded and the number of contenders for critical resources is
properly bounded.

The deadlock detection and retry strategy is a {\it dubious} mechanism for
avoiding deadlock in a directory DAG.

\subsection{Backoff Strategy}

The best strategy to use for retrying failed operations is an exponential
backoff strategy.  That is, each time an operation is aborted, the
processes issuing the operation waits a random length of time within an
exponentially increasing time period ({\it i.e.\ } after the first failure
the interval might be 10 ms; after the second failure the interval might
double to 20ms; after 3rd, 40ms; after 4th, 80ms\ldots{}).  The interval
continually gets bigger in order to decrease the amount of contention by
spreading contending operations further and further out.  If the maximum
backoff time does not allow all contenders to get in and get their
operations performed, this strategy cannot the guarantee completion of all
contenders.
   
The randomness in the backoff timing is critical to the success of the
exponential backoff strategy.  People who have failed to put sufficient
randomness into their backoff-retry timings have had serious problems.
With insufficient randomness operations will tend to collide with each
other on subsequent retires.

If backing off to successively increased random intervals works well by
spreading out resource and lock contention, one might wonder why we should
not always immediately backoff to the maximum interval.  In the case in
which the resources are heavily contended, this might be the optimal
strategy.  However, if we always backoff to the maximum interval, processes
may take a long time to complete even when loading is light.  Backing off
to successively increasing intervals allows processes to complete
relatively quickly when the contention is low or moderate. 

\subsection{Better Deadlock Strategy}

For directory DAG's, a better scheme for avoiding deadlock is to use
readers/writes locks.  Most of the locks used for directory operations will
be readers locks.  At worst, two write locks will be needed for directory
rename operations.\footnote{Class discussion points out that according to
UNIX filesystem semantics only a single write lock is needed for the UNIX
rename operation.}  

\subsection{Operating System Locks}

All of the discussion so far on locks has been concerned with operating
system level locks, not client program locks.  In the OS, we can assume
that all operations are programmed correctly and behave in a properly
controlled manner.  If client programs can acquire locks, the client may
cause deadlocks which the OS cannot prevent.  If the OS allows the client
program to acquire locks, it must have mechanisms for detecting deadlocks
caused by the client and provisions for aborting client programs as
necessary.  This is, generally, a difficult problem and there is currently 
no known strategy for efficiently handling this problem in general.


\section{Concurrency of Spec}

For this portion of the lecture, we concentrated on handout \#22.  

We want as much concurrency as we can out of the our implementation in
order to gain performance.  We could certainly wrap entire procedures in
mutex acquire and release pairs to satisfy the atomicity of the spec, but
this would give us very poor performance in a concurrent setting.

In ConcurrentDisk, separate mutexes are provided to protect data in the
cached disk blocks and the reference count for each disk block.
ConcurrentDisk.ReadBlocks only acquires the global mutex protecting the
reference count when it needs to add or remove itself from a cache block's
reference count.  While accessing data, a process uses the cache-block's
data mutex.  This scheme only forces serialization on access to the global
mutex.

We cannot have a processes requiring a certain number of blocks in the disk
cache in order to complete its operation.  If we allow processes to require
a certain amount of free cache space before they can continue, we can
introduce arbitrarily long (perhaps indefinite) waiting and significant
serialization.  ConcurrentDisk.ReadBlocks avoid this potential problem by
only acquiring as much cache space for a given read as it can before
performing any actual reads.  If it cannot acquire all the cache space for
the requested read operation, ReadBlocks simply does not bother caching the
tail end of the blocks in the read operation.

In general, when a process does not have a lock, it cannot depend on any
information it gained while it previously had a lock.  The process may want
to use the information so acquired as a hint, but cannot depend upon it.




\section{Concurrent Semantics of Spec}

Reference text for this discussion is Handout \#11 pages 11-14. 

\vspace{0.1in}
\vbox{
{\it What are we supposed to get from the Spec semantics?}
\begin{enumerate}
  \item Some confidence that we know (or can find by reading a few pages)
        what the transitions in a Spec program are.
  \item Some knowledge of how to state the semantics of a language.
\end{enumerate}
}

The grand strategy for concurrent operations in Spec is shown on the bottom
of page 14 of Handout \#11 and is roughly as follows:
\begin{enumerate}
\item (System will be in some state.)
\item Non-deterministically pick a thread which can make a transition.
\item Make a single (atomic) transition.
\item Put the thread back and return to step 1.
\end{enumerate}

In the concurrent setting, we cannot combine statements as we did when
studying the non-concurrent semantics of Spec.  For instance, consider the
sequence:  
  \begin{displaymath}
    S1 ; S2
  \end{displaymath}
We cannot abstract away the state left between the execution of $S1$ and
$S2$.  In the general concurrent setting the state of the program after
$S1$ is not the same as the state upon initiating $S2$.

In the concurrent semantics, the single step transition are as before.  It
is the composition of these transition steps which differs.  To describe
this situation, we define a transition encoding, Z, which keeps up with the
initial ($i$) and final ($f)$ program counters and transition relation
($tr$) for each atomic statement and sequence of statements.  In order for
the the Z encoding to express non-atomic statement sequences built from
atomic statements, the encoding is augmented with the set of program
counters ($ps$) contained within the encoding.  The MZ function (page 12)
calculates Z's for all statement forms defined for Spec.

As an example, let us consider the statement sequencing ($S1;S2$) [page 12
column 2].  The set of program counters, $ps$, is augmented with all
program counters in $S1$ and $S2$ except for $z1.f$, the final program
counter in S1.  The initial program counter, $i$, is set to the initial
program counter of $S1$ ($z1.i$) and the final program counter $f$, is set
to the final program counter in $S2$ ($z2.f$). The transition relation,
$tr$ is composed from the transition functions of $S1$ and $S2$ ($z1.tr$
and $z2.tr$) as follows:
\begin{enumerate}
  \item $tr$ contains all transitions in $S1$ ($z1.tr$) except the
        final transition ({\it i.e.\ } the transition to $z1.f$) since this
        program counter is being excluded from the program counters
        contained by the sequence combination.

  \item For all transitions in $S1$ ($z1.tr$) which  do result in the
        program counter becoming $z1.f$, $tr$ contains a transition from the
        initial program counter associated with the transition to $z1.f$ 
        to:
        \begin{enumerate}

         \item  the initial program counter of $S2$ ($z2.i$) for the case
               when the transition to $z1.f$ is not associated with an
               exception. 

         \item  the final program counter of $S2$ ($z2.f$) ({\it i.e.\ }
               effectively skipping $S2$) for the case when the transition
               to $z1.f$ is associated with an exception.  

        \end{enumerate}

   \item $tr$ contains all transitions in $S2$ ($z2.tr$).

\end{enumerate}

It is interesting to note that the semantics for the exception handling
form: 
  \begin{displaymath}
         S1 \mbox{  EXCEPT  } xs ==> S2
  \end{displaymath}
is basically the same as 
the sequencing semantics just described, except that the exception handling
cases on the transition to the end of $S1$ are reversed.  {\it i.e.\ } When
no exception occurs $S2$ is skipped (the transition is to $z2.f$).  When an
exception occurs, $S2$ is executed (transition is to $z2.i$).

The MZR function (page 13) goes on to describe the meaning of a routine
based on the concurrent statement meanings defined by MZ.


\end{document}
