%-------------------------------------------------------- \documentstyle[12pt,pocs-header]{article} \Scribes{Andr\'e DeHon} \Lecturer{Butler Lampson} \LectureNumber{11} \LectureDate{October 22, 1990} \include{macros} % used to include figures in this document \begin{document} \MakeScribeTop \section{Administrivia} \begin{table}[h] \centering \begin{tabular}{|c|l|} \hline \multicolumn{1}{|c|}{Handout} & \multicolumn{1}{c|}{Title} \\ \hline 22 & Concurrent Disk Reads \\ 23 & Problem Set \#4 \\ Notes & Notes for Lectures 8 \& 9 \\ \hline \end{tabular} \end{table} Problem Set \#4 is due Monday, October 29, 1990. \section{Review Directories from Lecture \#10} Last lecture we looked at two directory schemes: \begin{enumerate} \item Each pathnames is a sequence of strings (plus a few special sequences were included for convenience, {\it i.e.\ } ``.'' and ``..''.) \item The directory is an acyclic directed graph (DAG) \end{enumerate} Both of these directory strategies have reasonable implementations. \subsection{Sequence of Strings} The directory structure where each pathname is a sequence of strings, is implemented in a straight-forward manner using a B-Tree to map each ``pathname'' to a file. A naive-implementation might be wasteful of space when there were deep directory structures ({\it i.e.\ } there are many files with the same, long prefix pathname -- {\it e.g.\ } /a/b/c/d/e/f, /a/b/c/d/e/g, /a/b/c/d/e/h, etc.\ ). However, this space inefficiency can be avoided by using a {\it front compression} technique. In this case, the lookup cost is dependent only on the effectiveness of the B-Tree and {\bf not} on the depth of the file hierarchy. \paragraph{Example B-Tree Structure} Consider a B-Tree implemented using 4K disk blocks. If we assume 40-byte file entries, such a disk block will act as a 100 degree node in the B-Tree. With this structure, we only need two levels to support 10,000 entries in our filesystem; three levels will support 1,000,000 entries. Thus, even for filesystems with many files, any file can be accessed in at most three disk reads. Performance can, of course, be optimized further by keeping a cache in memory. The drawback to this directory scheme is that the {\bf rename} operation is very expensive. When a directory is renamed, it is necessary to change the sequence of strings tagging every file entry that falls underneath the renamed directory. Guaranteeing atomicity for this operation is not a major problem since we can use the log method explored in Handout \#10 and Problem Set \#2. This directory scheme is not used widely because of the difficulty of the rename operation. \subsection{Concurrency in Sequence of Strings Directory} We can handle concurrency by using concurrent B-Trees. {\it i.e.\ } Programs acquire locks on pages as they work down the B-Tree -- care must, of course, be taken to avoid deadlock during certain operations ({\it e.g.\ } rotations). A few operations, like rotations, require considerably more effort and care to perform concurrently. Additionally, the ease with which some operations can be performed depends on interactions from concurrently executing processes. One common technique for dealing with these kind of problem is to have a separate ``fast'' and ``slow'' path to perform an operation. The fast path handles the normal case without taking excessive care to work in all cases. The fast path does not modify anything until it knows that it will be able to succeed. If the process gets far enough to succeed down the fast path, then it completes the operation quickly. If the process detects that things are too complicated to be handled via the fast path, it releases all of its locks and restarts using the slow path. When implementing a fast/slow path scheme, it is generally wise to include instrumentation to verify that the ``fast'' path for which you have optimized is indeed the normal case during program execution. \paragraph{Example} A processes would traverse the tree acquiring locks as normal. When it finds the directory to be renamed, it might discover that the requested operation is a rotation and cannot be handled by the fast path. The process would then release locks and restart on the slow path -- perhaps using the data it acquired on the slow path as a hint. When the process has acquired the necessary locks on the slow path, it would check that it's knowledge about directory locations is still valid. If its information remains valid, the process can go ahead and perform the operation. If its information is stale, it releases all of its held locks and starts over. This kind of operation should converge rapidly unless the system is terribly overloaded. \subsection{Directed Acyclic Graph} The directed acyclic graph is the most common scheme for implementing directories. Access is generally straight-forward. Processes start at the root of the tree and acquire and hold the locks for each directory on their way down the directory hierarchy. Unfortunately, this scheme is prone to deadlock. {\it N.B.} If the directory really was a simple-tree ({\it i.e.\ } no cycles), this problem wouldn't necessarily arise. The tree-structure induces a partial ordering of nodes; using this partial ordering for lock acquisition is sufficient to avoid deadlocks when performing lookups. Renames require a total ordering on the nodes, but this is also trivial to add to a non-acyclic tree. In general, however, a directory is not a simple tree structure. A directory generally has symbolic links and parent links which allow arbitrary cycles in the directory graph. To avoid deadlock in this situation, a process traversing the directory must be careful to avoid both its own locks and those of other processes. \subsection{Deadlock Strategy} One simple strategy for dealing with deadlock is to simply let it happen. As long as deadlock can be detected by some means such as a time-out on the deadlocking operation, the operation can simply be aborted and restarted. This will always work and be correct as long as the operation has not made any changes prior to being aborted. Some care must be taken with this strategy to guarantee forward progress and to avoid process starvation. This strategy requires no knowledge of how deadlock occurs in the system. It works well as long as the expected completion time of operations is sufficiently bounded and the number of contenders for critical resources is properly bounded. The deadlock detection and retry strategy is a {\it dubious} mechanism for avoiding deadlock in a directory DAG. \subsection{Backoff Strategy} The best strategy to use for retrying failed operations is an exponential backoff strategy. That is, each time an operation is aborted, the processes issuing the operation waits a random length of time within an exponentially increasing time period ({\it i.e.\ } after the first failure the interval might be 10 ms; after the second failure the interval might double to 20ms; after 3rd, 40ms; after 4th, 80ms\ldots{}). The interval continually gets bigger in order to decrease the amount of contention by spreading contending operations further and further out. If the maximum backoff time does not allow all contenders to get in and get their operations performed, this strategy cannot the guarantee completion of all contenders. The randomness in the backoff timing is critical to the success of the exponential backoff strategy. People who have failed to put sufficient randomness into their backoff-retry timings have had serious problems. With insufficient randomness operations will tend to collide with each other on subsequent retires. If backing off to successively increased random intervals works well by spreading out resource and lock contention, one might wonder why we should not always immediately backoff to the maximum interval. In the case in which the resources are heavily contended, this might be the optimal strategy. However, if we always backoff to the maximum interval, processes may take a long time to complete even when loading is light. Backing off to successively increasing intervals allows processes to complete relatively quickly when the contention is low or moderate. \subsection{Better Deadlock Strategy} For directory DAG's, a better scheme for avoiding deadlock is to use readers/writes locks. Most of the locks used for directory operations will be readers locks. At worst, two write locks will be needed for directory rename operations.\footnote{Class discussion points out that according to UNIX filesystem semantics only a single write lock is needed for the UNIX rename operation.} \subsection{Operating System Locks} All of the discussion so far on locks has been concerned with operating system level locks, not client program locks. In the OS, we can assume that all operations are programmed correctly and behave in a properly controlled manner. If client programs can acquire locks, the client may cause deadlocks which the OS cannot prevent. If the OS allows the client program to acquire locks, it must have mechanisms for detecting deadlocks caused by the client and provisions for aborting client programs as necessary. This is, generally, a difficult problem and there is currently no known strategy for efficiently handling this problem in general. \section{Concurrency of Spec} For this portion of the lecture, we concentrated on handout \#22. We want as much concurrency as we can out of the our implementation in order to gain performance. We could certainly wrap entire procedures in mutex acquire and release pairs to satisfy the atomicity of the spec, but this would give us very poor performance in a concurrent setting. In ConcurrentDisk, separate mutexes are provided to protect data in the cached disk blocks and the reference count for each disk block. ConcurrentDisk.ReadBlocks only acquires the global mutex protecting the reference count when it needs to add or remove itself from a cache block's reference count. While accessing data, a process uses the cache-block's data mutex. This scheme only forces serialization on access to the global mutex. We cannot have a processes requiring a certain number of blocks in the disk cache in order to complete its operation. If we allow processes to require a certain amount of free cache space before they can continue, we can introduce arbitrarily long (perhaps indefinite) waiting and significant serialization. ConcurrentDisk.ReadBlocks avoid this potential problem by only acquiring as much cache space for a given read as it can before performing any actual reads. If it cannot acquire all the cache space for the requested read operation, ReadBlocks simply does not bother caching the tail end of the blocks in the read operation. In general, when a process does not have a lock, it cannot depend on any information it gained while it previously had a lock. The process may want to use the information so acquired as a hint, but cannot depend upon it. \section{Concurrent Semantics of Spec} Reference text for this discussion is Handout \#11 pages 11-14. \vspace{0.1in} \vbox{ {\it What are we supposed to get from the Spec semantics?} \begin{enumerate} \item Some confidence that we know (or can find by reading a few pages) what the transitions in a Spec program are. \item Some knowledge of how to state the semantics of a language. \end{enumerate} } The grand strategy for concurrent operations in Spec is shown on the bottom of page 14 of Handout \#11 and is roughly as follows: \begin{enumerate} \item (System will be in some state.) \item Non-deterministically pick a thread which can make a transition. \item Make a single (atomic) transition. \item Put the thread back and return to step 1. \end{enumerate} In the concurrent setting, we cannot combine statements as we did when studying the non-concurrent semantics of Spec. For instance, consider the sequence: \begin{displaymath} S1 ; S2 \end{displaymath} We cannot abstract away the state left between the execution of $S1$ and $S2$. In the general concurrent setting the state of the program after $S1$ is not the same as the state upon initiating $S2$. In the concurrent semantics, the single step transition are as before. It is the composition of these transition steps which differs. To describe this situation, we define a transition encoding, Z, which keeps up with the initial ($i$) and final ($f)$ program counters and transition relation ($tr$) for each atomic statement and sequence of statements. In order for the the Z encoding to express non-atomic statement sequences built from atomic statements, the encoding is augmented with the set of program counters ($ps$) contained within the encoding. The MZ function (page 12) calculates Z's for all statement forms defined for Spec. As an example, let us consider the statement sequencing ($S1;S2$) [page 12 column 2]. The set of program counters, $ps$, is augmented with all program counters in $S1$ and $S2$ except for $z1.f$, the final program counter in S1. The initial program counter, $i$, is set to the initial program counter of $S1$ ($z1.i$) and the final program counter $f$, is set to the final program counter in $S2$ ($z2.f$). The transition relation, $tr$ is composed from the transition functions of $S1$ and $S2$ ($z1.tr$ and $z2.tr$) as follows: \begin{enumerate} \item $tr$ contains all transitions in $S1$ ($z1.tr$) except the final transition ({\it i.e.\ } the transition to $z1.f$) since this program counter is being excluded from the program counters contained by the sequence combination. \item For all transitions in $S1$ ($z1.tr$) which do result in the program counter becoming $z1.f$, $tr$ contains a transition from the initial program counter associated with the transition to $z1.f$ to: \begin{enumerate} \item the initial program counter of $S2$ ($z2.i$) for the case when the transition to $z1.f$ is not associated with an exception. \item the final program counter of $S2$ ($z2.f$) ({\it i.e.\ } effectively skipping $S2$) for the case when the transition to $z1.f$ is associated with an exception. \end{enumerate} \item $tr$ contains all transitions in $S2$ ($z2.tr$). \end{enumerate} It is interesting to note that the semantics for the exception handling form: \begin{displaymath} S1 \mbox{ EXCEPT } xs ==> S2 \end{displaymath} is basically the same as the sequencing semantics just described, except that the exception handling cases on the transition to the end of $S1$ are reversed. {\it i.e.\ } When no exception occurs $S2$ is skipped (the transition is to $z2.f$). When an exception occurs, $S2$ is executed (transition is to $z2.i$). The MZR function (page 13) goes on to describe the meaning of a routine based on the concurrent statement meanings defined by MZ. \end{document}