\documentstyle[12pt,pocs-header]{article}
\Scribes{Kathy Knobe and Raymie Stata}
\Lecturer{Bill Weihl}
\LectureNumber{12}
\LectureDate{October 24, 1990}

\begin{document}
\MakeScribeTop

\section{Handouts}

\begin{center}
\begin{tabular}{|c|l|} \hline
\multicolumn{1}{|c|}{Handout} &
\multicolumn{1}{c|}{Title} \\ \hline
24 & Efficient Implementation \\
   & of Transactions \\
Notes & Lecture 10 \\
\hline
\end{tabular}
\end{center}


\section{The Problem}

The focus this week and next will be on efficient implementations of
transaction systems, which give ``all-or-nothing'' semantics to a
sequence of non-atomic operations.  Today's lecture is about {\it
sequential transactions}, i.e., transactions for systems with only one
client thread.  Next week we will deal with {\it concurrent
transactions} for systems with multiple client threads.

Handout~10 gave a specification for sequential transactions (it is
repeated in Handout~24).  The basic idea is to keep updates in
volatile storage until either a commit or an abort (crash is same as
abort), at which time the changes are either all installed atomically
or completely discarded.  The effect of this is to make long sequences
of operations atomic with respect to any crashes.

Handout~10 also gave an log-based implementation for sequential
transactions.  This implementation distinguished among {\it actions},
{\it transactions}, and {\it updates}.  An action is an operation that
the client can perform of the database.  A transaction is a sequence of
actions that are performed on the database with all-or-nothing
semantics.  An update is an atomic operation on the representation of
the database; when the user invokes an action, the implementation
breaks the action into a sequence of updates that perform the action.

To do an action, the implementation determines the current state by
applying the volatile log to the stable state, calculates a sequence
of atomic updates that will take the current state into the state
specified by the action, and appends this sequence to the volatile
log.  To commit, the implementation atomically copies the volatile log
to the stable log (this atomic copy is the ``commit point'' or
``decisive transition''), then calls {\bf Redo}.  {\bf Redo} applies
updates from stable log to the stable state one at a time until all
updates are done, then empties the stable log.  If a crash occurs
during {\bf Redo} (or at any other time), then crash recovery calls
{\bf Redo} and updates begin all over again from the beginning of the
log.  Since updates are idempotent, repeatedly applying prefixes of a
longer sequence before applying the whole sequence has the same effect
as applying the whole sequence only once.

This implementation has serious inefficiencies.  For each {\bf Do}
operation, the implementation applies the volatile log to the stable
state to find the current state, a major source of delay.  Another
major source of delay is commit and crash recovery, which apply all
updates to stable state, requiring lots of random disk access in the
process.  We would like to avoid these inefficiencies.


\section{A Solution}

The implementation of sequential transactions in Handout~24 avoids the
inefficiencies of the previous implementation by a combination of
caching and undo logging:
\begin{itemize}
\item The {\bf Do} operation will update the cache directly.  The 
current state won't have to be recalculated because it will always be
reflected in the cache.

\item To decouple the commit operation from the random disk access
inherent in updating stable storage, the commit operation will not
flush the cache.  Instead, commit will append the volatile log to the
stable log, which involves only fast, sequential disk I/O.

\item To decouple recovery from random disk~I/O, recovery will read
the stable log into the cache, but will not flush the cache to stable
storage.

\item Unlike before, the stable log may contain updates from more than
one transaction.  Since we don't want the log to get too long, we will
have a background process to trim the log after flushing the cache.

\item We want this background process to be able to flush cache
entries when convenient; in particular, we want to be able to flush
cache entries even if they contain updates that haven't been committed
yet.  Unlike before, then, stable storage can contain uncommitted
updates.  This implies that the stable log must contain an undo
operation: for each uncommitted action that has been flushed to disk,
we need a way to undo it if there is a crash or an abort.
\end{itemize}

The new implementation introduces considerable flexibility concerning
when cache entries are flushed to stable storage.  The entries can be
flushed long after commit, so stable storage can lag behind the stable
state.  In this case, a crash will require a redo operation to restore
the stable state from the stable log.  Entries can also be flushed
before commit, in which case stable storage gets ahead of the stable
state.  In this case, a crash or abort will require an undo operation
to remove the effect of aborted updates from stable storage.  (Loading
cache entries from stable storage occurs on demand when an operation
needs that data.)


\subsection{High-performance transaction systems}

A well designed, high-performance transaction system should be CPU
bound, not I/O bound.  Today's modified implementation of log-based
recovery is an important step towards realizing this goal.  Through
caching, we eliminate the need for most random disk accesses; by
establishing a a predominantly sequential disk access pattern, we use
the disk in its most efficient mode.

Another important step in making a transaction system CPU bound is
{\it disk striping}: partitioning a memory page into multiple segments
and concurrently writing each segment to a separate disk.  In the
right configuration, using $N$ disks in a stripped manner can lead to
an $N$-factor improvement in disk bandwidth.  Disk striping works
particularly well in systems with a sequential I/O pattern.

In concurrent transaction systems, a final technique for making the
system CPU bound is group commit: letting the commits pile up and
committing them at once to amortize the start-up cost of the commit.


\subsection{Types of failure}

The transaction system being described today deals with only two types
of failures: transaction aborts and crashes.  Real systems experience
a broader range of failures.  Typically, failures fall into four types:

\begin{itemize}
\item {\bf Transaction abort}: client software aborts active
transaction.  This is usually the most benign and easiest to deal
with.  In our system, a transaction abort require undo operations.

\item {\bf Crash}: the CPU, memory, I/O~channel or other hardware
fails, but media stay intact.  The second most benign failure.  In our
system, crash recovery requires both redo's and undo's.

\item {\bf Media failure}: disk or tape (or some other device used as 
``stable'' storage) looses data.  These failures occur often enough to
be a real concern to system designers, though our system doesn't deal
with it.  Typically, a ``permanent'' log stored on magnetic tape is
used to recover from media failures.  ``Disk mirroring'' is another
popular technique used in systems designed to transparently mask
single failures.

\item {\bf Catastrophes}: some {\it combination} of failures
that cause the system to loose data.  For example, both the primary
stable storage disk and the permanent log tape get destroyed in a
fire.  Multiple, off-site copies of the permanent log is an effective
guard against most catastrophes.
\end{itemize}
The more likely a particular failure, the faster recovery for that
failure should be.  The list above is ordered according to decreasing
likelihood.

In system design, an early step is to identify failure modes and
associate probabilities and to decide from what failures the system
must be able to recovery and how fast.


\section{Implementation}

Handout~24 describes the implementation abstractly.  {\bf S} is the
type of the abstract state of the system.  Actions take an abstract
state of the system to another and a return value.  Stable storage
isn't modeled in terms of disk pages, but rather as a ``stable base''
(type {\bf B}) which is an encoding of {\bf S}.  Atomic updates {\bf
U}'s are updates to the stable base, i.e., they take stable base
states to stable base states.  (In real life, {\bf U}'s might be
updates to single disk pages).  The cache is modeled as a set of
atomic updates to the stable storage.  The current state can be found
by applying updates in the cache to the stable base to get a new
stable base (the {\bf DoCache} routine does this).  In real life, it's
easy to implement caches that store the current state directly rather
than having to do this application.

We maintain separate undo (type {\bf UL}) and redo (type {\bf L})
logs.  The undo log uses {\it logical logging}: it logs logical
operations that operate on the abstract state.  The redo log uses
{\it physical logging}: it logs atomic updates to the representation.
As against physical logging, logical logging leads to much smaller
logs, allows for more concurrency, and is easier to prove correct.
However, logical logging makes flexible cache flushing difficult.
With physical logging, it's easy to atomically flush all the effects
of a logged update since updates will typically touch only one cache
entry.  With logical logging, on the other hand, the effects of a
single log entry could touch all kinds of cache entries.

For the redo log, then, we use physical logging to allow for the
flexible flushing we want.  For the undo log, we use logical logging
until the undo action actually has to affect the database (e.g., in an
abort), at which time we convert the undo entry into a sequence of
physical updates and insert those updates into the redo log.

We assume that the updates are idempotent and that they commute.  This
ensures the log idempotence property for the redo log described in
Handout~10.  This means that we can then redo a subsequence of updates
over and over with no ill effect.  Undos are not required to be
idempotent; therefore, it is important after a crash to make sure each
logical operation gets undone exactly once.

Before flushing uncommitted action to stable storage, it's crucial
that the corresponding undo log entry be stable.  Otherwise, if a
crash occurs between the point an action makes it to stable storage
and the point its undo entry makes it to stable storage, there would
be no way to undo the effect of the action.  This discipline is known
as the {\it write-ahead log rule}.

\end{document}