\documentstyle[12pt,postscript,/nfs/thor/thor/6826/92/macros/times]{article}
%\documentstyle[12pt]{article}

\input{/nfs/thor/thor/6826/92/macros/lecture}
%\begin{figure}
%\PostscriptPicture{/nfs/thor/thor/6826/92/lectures/}
%\caption{}
%\end{figure}


% To latex this document, you need
% lect12.tex
% lect12.eps--encapsulated ps file for figure
% add consult to include figure
%\documentstyle[12pt,psbox]{article}

\Scribe{Susan Yeh\footnotemark[1]}\addtocounter{footnote}{1}\footnotetext{These notes are based on notes prepared by Bill Kruger in Fall 91}\addtocounter{footnote}{1}
\Lecturer{Bill Weihl}
\LectureNumber{12}
\LectureDate{October 26, 1992}

\begin{document}
\MakeScribeTop
\section{Lecture Overview}
Today's lecture begins a two-part discussion on transactions.  The topics 
covered are:
\begin{itemize}
\item{Sequential Transactions}
\item{Performance Considerations}
\item{Sequential Transactions with Caching}
\item{Recovery}
\end{itemize}

\section{Sequentials Transactions}
Transactions are a mechanism for making a sequence of operations atomic.  For 
example, a database cannot rely on disk atomic actions alone if updates are 
applied to more than one disk page.  Thus, a layer, called the transaction 
layer, is added around to disk to make an arbitrary sequence of
operations atomic.  (see Figure~1)

\begin{figure}
%\begin{center}
\PostscriptPicture{/nfs/thor/thor/6826/92/lectures/12/fig.eps}
%\centerline{\psfig{height=4in, figure=/mit/ssyeh/6826/lect12.eps}}
%\end{center}
\caption{Transaction Layer}
\end{figure}

\section{Performance}
The {\tt SequentialTransaction} spec and the {\tt LogRecovery} implementation 
were presented in Handout 10 and repeated in Handout 30. However, performance 
for this implementation is poor, and we introduce a more efficient implementation with caching.  The mechanisms for improving performance are:
\begin{itemize}
\item Cache information on the volatile state ({\tt vs}).  Since the
entire volatile state is probably too large, we construct {\tt vs} from
the stable state plus what is in the cache.  By caching frequently used parts of {\tt vs}, updates and reads can be faster.

\item Decouple commit from actually applying the updates written on the stable
log to the stable base. This reduces the latency of the commit. A fringe
benefit is that we can aggregate a series of updates to a single disk
block and pay only one disk write to record them. 
\end{itemize}

\section{Sequential Transactions with Caching}
The basic ideas for implementing sequential transactions with caching are:
\begin{itemize}

\item Since the cache will be of some finite size, we can't allow the cached
updates to grow without bound.  This means that we'll have to flush the
cached updates when it is determined that the cache is full.  Since the
updates are flushed independent of whether the containing transaction
has committed, the Stable Base may include updates not yet committed.

\item We want to defer updates for as long as possible, as we can aggregate
many updates of a single disk block and then just apply the most current one
to the Stable Base.  As mentioned above, however, by flushing the cache as
it fills, the Stable Base can contain updates not yet committed.  Recall that
the abstract Stable State must not contain uncommitted updates.
To restore the Stable State from the Stable Base (which contains committed and
uncommitted updates), we'll need to calculate an "undo" action for each action,
so that any uncommitted actions can be undone:

\begin{center}
{\tt ss} = {\tt sb} + All updates - uncommitted updates (undo)
\end{center}

\item We want to truncate the log of entries whose updates have been applied to
the stable base in order to reuse log space.  We'd like to truncate entries whether
or not the transaction responsible for them has committed (as we've decoupled commit
from the application of updates)- so long as they've been applied to the {\tt sb}, we can
remove them from the log.

\item To enable recovery of the base in case the disk gets corrupted, we'll need to save
discarded entries (those truncated in the above process) to a Permanent Log 
({\tt pl}).  The {\tt pl} is often kept on magnetic tape.
\end{itemize}

\subsection{Failure Types}

There are four levels of failure within this transaction subsystem, listed in order
of increasing severity:

\begin{itemize}
\item {\bf Transaction Abort}: the client aborts a transaction in progress.  It is not really a failure, since no work is lost except by request.

\item {\bf Crash}: this results in the loss of the volatile state and aborts all active
uncommitted transactions.  Committed transactions are recoverable from the Stable
Log and Undo Log.

\item {\bf Media Failure}: this results in the loss of the Stable Base, as well as the
Stable Log and Stable Undo Log.  The Stable Base is recoverable from the Permanent
Log.

\item {\bf Catastrophe}: the Permanent Log and the Stable Base are lost.
\end{itemize}

In practice, further steps are taken to handle failures, such as periodically taking
snapshots of the Stable Base and saving multiple copies to magnetic tape.

\subsection{Implementation: LogAndCache}

Handout 30 presents an implementation for sequential transactions with
logging and caching.  The rest of the lecture highlighted the important features of
the parts of the implementation.  The implementation has the following features:

\begin{itemize}
\item An action is turned into a sequence of updates that can be applied
atomically. Updates are applied to single disk blocks (which can be written atomically).

\item Undo Actions: for every action there exists a corresponding "undo" action which will
reverse the effect upon the base of that action.

\item The redo and the undo entries of an action are written to the logs
in one atomic step.

\item A cached update, or W, represents the new contents of a disk block.
\end{itemize}

The implementation is optimized for both crash recovery (due to logging) and for
speed of access (due to caching).  There are now two logs, each with two parts:

\begin{itemize}
\item Stable Log ({\tt  sl}) and Volatile Log ({\tt vl})

\item Stable Undo Log ({\tt sul}) and Volatile Undo Log ({\tt vul})
\end{itemize}

An action is converted to a sequence of updates (U), which are logged to the VL.
The updates are also applied directly and immediately
to the cache (C) before commit, so that the current state is immediately accessible.
For each action, an Undo Action is computed and written to the Volatile Undo Log 
({\tt vul}).  Note that the undo log contains the action, and not the sequence of updates into which the
action is converted- this is done for efficiency, as we normally don't expect to have to
undo actions, and so we can defer converting them to updates if and when we apply them.
Committing a transaction first moves its updates (U) from the {\tt vl} to the {\tt sl} and its undo
actions from the {\tt vul} to the {\tt sul}.  It then logs a commit
record in the {\tt sl} and empties the
{\tt sul} (since the action is committed, we no longer need to be able to undo it).


As noted in Handout 30, the U's and W's have the following properties:

\begin{itemize}

\item Adding an action's associated updates (Us) to the log is an atomic operation.

\item Applying a cached update (W) to the {\tt sb} is an atomic operation.

\item Converting a U to a W is cheap, and applying a W is cheap.

\item U's are idempotent (the hiccup property as discussed in Handout 10).

\item W's commute
\end{itemize}

We can write an abstraction function to {\it SequentialTransaction} from Handout 10:

\begin{center}
{\tt ss} = {\tt sb}+{\tt sl}-{\tt sul}
\end{center}
\begin{center}
{\tt vs} = {\tt sb}+{\tt sl}+{\tt vl}
\end{center}

\subsection{Implementation Routines}

The important aspects of many of the implementation routines were examined:
\begin{itemize}

\item {\bf DoOrUndoA}: We first construct the volatile state of the system (vs) from the 
{\tt sb} and the cache by applying all cached updates (W's) to the stable base.  We then
convert the action to its sequence of updates based upon the volatile state.  If the
action is not an undo action, then we compute an undo action.  If we were performing
an undo action, then we need to log the fact that this undo action has
actually been performed. The undo action is broken into updates, which
are logged in the redo log just as a normal action's updates would be. A
cancel record is logged in the undo log;
this provides idempotence in the undo log without requiring the undo
actions to be idempotent.

The action's updates are logged to the {\tt vl}, while an undo action is logged to the 
{\tt vul} and the
updates are converted to cache updates (W's) and applied to the cache.  As specified,
the entire procedure is done atomically.

\item {\bf Commit}: Commit first forces all updates from the {\tt vl} to the {\tt sl} and all undo
actions from the {\tt vul} to the {\tt sul}.  It then logs a "commit" marker to the {\tt sl} and
empties the {\tt sul}, these two operations done atomically.

\item {\bf Abort}: Abort undoes all uncommitted updates by applying the undo actions in the
undo logs.  In the implementation given in the handout, it forces all
updates in the {\tt vl} and {\tt vul} to the {\tt sl} and {\tt sul},
respectively. This is not necessary in general because in the event of a
crash, the transaction will be aborted anyway.

\item {\bf Crash}: The idea of Crash is that the volatile state has been lost.  This includes
the {\tt vl}, the {\tt vul} and the Cache (C).  Recovery then consists of replaying the {\tt sl} to reconstruct
the cache as it existed before the crash, and then applying any undo actions contained in the
{\tt sul} to undo all uncommitted updates.

\item {\bf Undo:} Undo concatenates the {\tt sul} with the {\tt vul} and then applies the undo actions
contained in this concatenated log {\it in reverse order}.  The undo actions must be
applied in order, and since each was computed based upon the volatile base that existed
after the application of the previous undo action's associated action, this requires a
reverse chronological order.  Undo skips over previously applied undo actions until one
is encountered which has not yet been applied.  It is then applied by invoking DoOrUndoA(),
which in addition to converting the undo action to a sequence of updates, also marks the
undo action as applied in the {\tt vul}.  In this way undo actions are only applied once, assuring
idempotence of the undo logs.

\item {\bf Redo}: Redo rebuilds the cache as it was before the crash, thus containing volatile
uncommitted updates.  This is faster than rebuilding and applying all updates to the {\tt sb}, and
uncommitted updates can be gotten rid of via Undo().

\item {\bf Flush}: flushes an update from the cache to the stable base,
to make room in the cache. The {\em write-ahead log} policy requires that the
corresponding update in the log (and the corresponding undo action) must be
forced to the disk {\em before} the update from the cache is allowed to
change the stable base. This ensures that in the event of a crash, the
update in the stable base will be undone.

Before applying a cached update to the {\tt sb}, Flush makes sure that the corresponding
action is in the {\tt sl}.

\item {\bf Truncate}: Truncate takes a contiguous portion of the {\tt sl} that has already been applied
to the {\tt sb} and removes it from the {\tt sl}.
\end{itemize}

\end{document}