% Lecture 5 Scribe Notes % % to get hardcopy of this lecture, you need the following files (plus the % normal latex and tex base directories): % lec.tex: latex source file % tmacros.tex: macros % psadobe.tex % pocs-header.sty: Principles of Computer Systems lecture latex header % atr.ps, rs.ps: postscript drawings \documentstyle[12pt,pocs-header]{article} \Scribes{Chris Joerg and Derek Chiou} \Lecturer{Butler Lampson} \LectureNumber{5} \LectureDate{September 26, 1990} \input{psadobe.tex} \input{tmacros} \def\illustrationfileprefix{/bk/cfj/6826/} \newcommand{\ra}{$\rightarrow$} \newcommand{\Ra}{$\Rightarrow$} \newcommand{\AND}{$\wedge$} \newcommand{\OR}{$\vee$} \begin{document} \MakeScribeTop \section{Administrative Information} Lecture \#2 notes were handed out and a Spec review session was scheduled (after much debate) for Thursday at 4. Handout \#10 {\em Simple File System Implementation} and Handout \#3 {\em The Spec Language} were referenced during the lecture. \section{More on File System Recovery} \subsection{File System Spec} The specifications for a file system recovery scheme were reviewed. They are: \begin{center} \begin{tabular}{|l|l|p{3in}|} \hline Do & $<$vs := a(vs)$>$ & Do performs its actions on the volatile state.\\\hline Commit & $<$ss := vs $>$ & The volatile state that Do has been changing becomes the stable state.\\\hline Crash & $<$vs := ss$>$ & The volatile state ($ie.$ the contents of memory) is lost.\\\hline \multicolumn{3}{c} {Crashes can only occur outside of the atomicity brackets.} \end{tabular} \end{center} All actions are done atomically to the volatile state. When the volatile state becomes an acceptable state, a commit can be done which sets the stable state to be the volatile state. When a crash occurs all changes made since the last commit are lost, but all changes made before the last commit are safe. A crash cannot occur during a commit due to the atomicity brackets. \subsection{File System Recovery Scheme Implementation} The implementation of a recovery scheme is far more complicated than its specification. The main problem lies in atomicity -- the only atomic actions that a real disk can perform is a single block write. Since most updates, and certainly almost all commits, require more than one block update, it is difficult to satisfy the atomicity specs in a real implementation. The solution to this problem is to divide the complex actions into a log which is a series of atomic updates. The partitioning of the original problem leads to recovery problems if a crash occurs while performing the series of atomic updates. The implementation takes care of that within its Redo procedure. \newpage We need the following type definitions for our implementation. \begin{example} A:S\ra{}S \%action U:S\ra{}S \%update L = SEQ[U] AtoL:A\ra{}L ss:S \%stable state vl:L \%volatile log sl:L \%stable log \end{example} And here is the implementation: \begin{example} Do(a): \% AToL translates an action into a list \% of log entries. The atomicity \% brackets aren't really necessary. Commit: ; vl := L{}; Redo() Crash(): vl := L{}; \% The volatile log is lost Redo() Redo(): apply operations in sl to ss \end{example} Note that doing {\tt $<$sl := vl$>$} atomically can be difficult -- thus, our professors made a homework problem out of it. The abstraction function must define the specification's {\tt ss} and {\tt vs} as a function of the implementation's variables. It is defined as: \\ {\tt ss:S = ss + sl\\ vs:S = ss + sl + vl}\\ Note that the stable log is considered part of the stable state. When a commit occurs, it first copies the volatile log into the stable log, thus making the changes permanent. It then calls Redo which performs on {\tt ss} all the updates in {\tt sl}; Redo must this correctly even if it is interrupted by a crash. Only after a Redo is finished can another Do be executed. %Only one of the two logs, {\tt sl} and {\tt vl}, can be non-nil at any one %time. A crash occurring during the redo of a previous crash should not prevent us from getting the right answer. No matter how many times one applies the updates, as long as the updates are applied in the correct order the same result should be reached. A log has this property if: \vspace{1cm} $\forall s, l | s + l + l = s + l$\\ To see that this property ensures that our logs will work correctly, consider the following example where redo is twice interrupted by crashes. The stable state will become: $ss + l_{1} + l_{2} + l$. We must show that this is equivalent to the case where no crashes occur, ie., $ss + l$. %For example:\\ \begin{tabular}{ll} $ss + l_1 + l_2 + l$\\ $\ \ = ss + l_1 + l_2 + l_2 + l_3$ &\% Since $l = l_2 +l_3$ \\ $\ \ = ss + l_1 + l_2 + l_3$ &\% By the above property \\ $\ \ = ss + l_1 + l $\\ $\ \ = ss + l $ &\% By the same reasoning as for $l_2$\\ \end{tabular}\\ What kind of writes will ensure that $\forall s, l | s + l + l = s + l$ is true? Only pure writes will. A pure write does not depend on any state, and thus writes its value in its location regardless of what is in the memory at the time. An accumulating function would not be considered a pure write, since it adds its value to the value that is already contained within the specified location. Regardless of whether or not a write is pure, updates must be kept in order to maintain correct state. If you have only pure writes, then when a crash occurs you can simply redo them all in order from the beginning, regardless of how far you got into your commit. Unfortunately, many real world updates are not pure writes, since they may depend on the current state. For non-pure writes, you can recover from a crash by resuming after the last update that was actually completed. So how does one know if an update has been completed? We do it using a set of unique tags, each one associated with an update. We keep a set of these tags associated with the stable state. Whenever an update request is made, the state checks to see if that update's tag is in its set of tags of completed updates. If it is, the update is not performed. If it isn't, the update is performed. The spec is below. \begin{example} TYPE S = RECORD[ss, tags: SET[UID]] U = RECORD[uu: SS\ra{}SS, tag: UID] WITH \{ meaning:=Meaning \} FUNC Meaning(u, s)\ra{}S = u.tag IN s.tags \ra RET s [*] RET S\{ ss := (u.uu)(s.ss), tags := s.tags ++ u.tag \} \end{example} A tricky implementation problem is how to get the update and update tag written atomically. A common solution is to let the tag be a timestamp, and write the update tag on the block that is being written. Each block will contain the tag of the last update done to that block. Since updating blocks is atomic, writing the update tag to the updated block is atomic as well. Thus, the set of UID tags is spread over all of the pages. \section{Semantics of SPEC} This section of the lecture gave a sequential semantics for the Spec language. A semantics for $sequential$ Spec is given so that we can avoid the additional complexity that concurrency adds. A Spec program can be considered to be a state machine description. The states of this machine are atomic states (A), and allowable transitions are defined by the Atomic Transition Relation (ATr). An ATr is is a relation, rather than a function, because Spec is nondeterministic and several state transitions may be possible at the same time. \fig{atr} shows some states and allowable transitions. \illfigure{atr}{State Transitions} The semantics that will be given for Spec will map the program text into states and a transition relation. We want the meaning of a statement (denoted as {\tt MS(S)}) to map a state, {\tt A}, to possible outcomes, {\tt O}. Since there may be more than 1 possible outcome, {\tt MS} is defined as a function from state-outcome pairs to a boolean:\\ {\tt MS(S): (A,O) \ra{ } BOOL\\ } If {\tt MS(s)} applied to {\tt (a,o)} returns true, then statement {\tt s} could cause a transition from {\tt a} to {\tt o}, else it could not. This is known as a relational semantics. This semantics allows us to write predicates based upon states and outcomes. For example if x, y are values from the initial state and x',y' are values from the outcome, then the function \begin{example} (LAMBDA $(x,x',y,y') = x'=x+1 \wedge y'=y$ ) \end{example} lets us determine which transitions have a certain set of properties (namely: x is incremented and y is unchanged). A common use for this will be to use the initial and final values to determine if a transition is allowable. \fig{rs} shows some example relations. \illfigure{rs}{Some examples of Relational Semantics} This relational semantics is equivalent to the proof rules given in lecture 2. Statements in the proof style can be translated to the relational style. For example: $P\{S\}Q$ would be stated in relational semantics as:\\ {\tt $\forall$ a,o | S(a,o) \Ra{} P(a)\Ra Q(o)}\\ In English, this means that if statement {\tt S}, could possibly take state {\tt a} to outcome {\tt o}, then if {\tt P} was true in {\tt a}, {\tt Q} will be true in {\tt o}.\\ Similarly:\\ $S1 [] S2$ becomes\\ {\tt (a,o) = S1(a,o) \OR{} S2(a,o)}\\ which means that {\tt (a,o)} is a valid transition if it is a valid transition for {\tt S1} or for {\tt S2}. The rules for operating on these statements are equivalent to the rules for operating in the proof style. For example: \begin{example} $P\{S1[]S2\}Q$ becomes (S1[]S2)(a,o) \AND P(a) \Ra{}Q(o) S1(a,o)\OR{}S2(a,o) \AND P(a) \Ra{}Q(o) S1(a,o)\AND{}P(a) \OR S2(a,o)\AND{}P(a) \Ra Q(o) S1(a,o)\AND{}P(a)\Ra{}Q(o) \AND S2(a,o)\AND{}P(a)\Ra{}Q(o) $P\{S1\}Q$ \AND $P\{S1\}Q$ \end{example} The following types will be needed for our semantics:\\ \begin{tabular}{lll} V: & value space \\ X: & exceptions & = RECORD [s:STRING, local:BOOL] \\ L: & locations \\ M: & memory & = L \ra V \\ En:& environment& = Var \ra L \hspace{.3in} \%Var is a variable name and is simply a string.\\ % & \multicolumn{2}{l} {\%Var is a variable name and is simply a string.}\\ A: & atomic state & = RECORD [m,en,v,x,forks] \\ \end{tabular}\\ Atomic states contain a memory and an environment which are used to map variables into locations and then into values. The value and exception in the state are used only for passing arguments and results. Atomic states also contains a {\em forks} element which is used for saving forked threads; forks were not discussed and will be treated at a later date. Although it is not required by the above, for simplicity we also assume that aliasing is not allowed. \subsection{Expressions} First we will give the meaning of expressions. Remember that an expression is a function from states to results: \begin{example} E : A \ra V|X V|X is a shorthand for UNION[V,X] \end{example} An expression is a function which takes in a state and produces a result. It must be a simple (mathematical) function: it can not be non-deterministic and it can not have side effects. To determine the meaning of an expression we define ME which takes an expression and returns a function from states to results: \begin{example} ME : E \ra A \ra V|X \end{example} For each kind of expression {\tt ME(e)} acts differently:\\ \begin{tabular}{ll} {\bf Expression} & {\bf Meaning}\\ {\em literal\/}, l: & constant function \\ {\em variable\/}, var:& {\tt (a.m (a.en(``var'')))}\\ & $ie.$: The meaning of a variable is the value stored in the\\ & memory location associated with that variable.\\ \\ {\em dereference\/}, e$\uparrow$: & {\tt a.m (ME(e)(a))} \ seems reasonable.\\ &This means find the meaning of $e$ in state $a$. This should\\ &be a location, and the meaning is the value stored at that location.\\ & But this doesn't take exceptions into account! \\ & As usual treating exceptions adds extra hair:\\ & \hspace{.1in} {\tt VAR vx := ME(e)(a);} \\ & \hspace{.1in} {\tt vx isX \Ra RET vx} \\ & \hspace{.1in} {\tt [] vx isV \Ra RET a.m(vx)} \\ \\ {\em invocation\/}, f(e): & Informally:\\ & If $f$ or $e$ returns an exception then return the exception;\\ & otherwise $f$ must be a function from values to results, so\\ & we want the meaning of $f$ applied to $e$.\\ & This will be discussed in more detail next class.\\ \end{tabular} \subsection{Statements} For determining the meaning of statements we define {\tt MS}, a function from states to atomic transition relations. (Remember that an ATr is a function which tells which state-outcome pairs are allowable.) \begin{example} MS: S \ra ATr where: ATr: (a,o) \ra BOOL \end{example} As with expressions, there are many different kinds of statements and {\tt MS} must treat each one differently. The {\tt MS} definition is one large case statement that branches based on the statement type. For each branch of the case statement there is much common code for handling exceptions and undefined expressions. This code is given in a boilerplate on page 19 so that it does not have to be repeated. For each statement type, this boilerplate needs a predicate that acts on a state, {\tt a}, and an outcome, {\tt o}, and is true only if that statement could produce outcome {\tt o} when applied to state {\tt a}. Page 20 gives the predicates for the possible statements. A few of these were discussed in class: \vspace{.1in} \begin{tabular}{lll} {\bf Statement} & {\bf Predicate} & {\bf Comments}\\ \\ SKIP & {\tt o=a} & A skip has no effect\\ HAVOC & true & HAVOC can produce any outcome\\ && from any state \\ RET e & {\tt o= a\{x:=retX v:= ME(e)(a)\}} & The outcome is the same as the \\ && original state except: \\ && {\tt x=retX} which means the call has\\ && returned normally.\\ && {\tt v} is set to the value being returned.\\ \end{tabular} \end{document}