%     to get hardcopy of this lecture, you need the following files (plus 
%     the normal latex and tex base directories):
%     lecture23.tex:     latex source file
%     macros.tex:      figure macros
%     psfig.tex:       postscript figure (psfig) macro definition
%     pocs-header.sty: Principles of Computer Systems lecture latex 
%     header
%
% run the following programs:
% latex lecture23
%   -> Creates lecture23.dvi, plus latex intermediate files
% latex lecture23
%   -> Run a second time to get cross-references right
% dvi2ps lecture23 > lecture23.ps
%   -> Merges the figures with the text, can also pipe to lpr
%
\documentstyle[12pt,pocs-header]{article}
\Scribes{Steve Keckler and Beng-Hong Lim} 
\Lecturer{Bill Weihl}
\LectureNumber{23}
\LectureDate{December 5, 1990}
\include{macros} % used to include figures in this document

\def\dirty{{\bf DIRTY}}
\def\myvalid{{\bf VALID}}
\def\invalid{{\bf INVALID\@}}
\def\myshared{{\bf SHARED}}
\def\shared-dirty{{\bf SHARED-DIRTY}}
\def\valid-exclusive{{\bf VALID-EXCLUSIVE}}

\begin{document}
\MakeScribeTop

\rightline{Life's a tradeoff ~~{\it William E. Weihl}}

\section{Administrivia}

\begin{table}[h]
\centering
\begin{tabular}{|c|l|} \hline
\multicolumn{1}{|c|}{Handout} &
\multicolumn{1}{c|}{Title} \\ \hline
38 & {\em Cache Coherence Protocols:  Evaluation} \\
& {\em Using a Multiprocessor Simulation Model} \\
39 & Problem Set \#5 Solutions \\
Notes & Lecture Notes \#20 \\
\hline
\end{tabular}
\end{table}

\section{Overview}

This lecture covers further studies of cache coherency protocols 
including the following topics:
\begin{itemize}
\item Snoopy cache coherence protocols
\item Cache coherence protocols for arbitrary networks
\end{itemize}
Handout \#38, {\em Cache Coherence Protocols:  Evaluation Using a 
Multiprocessor Simulation Model} by James Archibald and Jean-Loup 
Baer, is the primary source for much of this lecture.

\section{Snoopy Caches}
This section focuses on bus-based multiprocessors and the mechanisms
in which cache coherency can be maintained.  In a bus based system,
the performance is often limited by bus bandwidth.  Therefore it is
desirable to provide local processor caches with some form of a {\em
write-back} policy such that a bus cycle is not necessary for every
memory reference.  The data is not copied back to main memory, and
does not incur a bus cycle, until that block is replaced in the cache.
The alternative, a {\em write-through} policy, requires that cache and
main memory copies of data be identical; thus a bus cycle would be
required for every write.

Snoopy cache coherency protocols allow for multiple copies of the data
to reside in different caches and relies on a smart cache controller
at each processor to monitor bus transactions and determine what, if
any, actions to take.  Each cache controller is charged with the task
of maintaining the consistency of those blocks of which it has copies.
A block in a cache is assigned to some local state which can be
changed by the controller depending upon what actions are taken upon
that block by other caches.  Other caches' actions will be visible on
the bus.



\subsection{Write Once}
This protocol was covered in detail in lecture 22.

\subsection{The Synapse Machine}
The Synapse machine protocol requires that all caches get their data from 
main memory.  There are no cache to cache transfers.

The internal states of a cache block are as follows:
\begin{itemize}
\item INVALID
\item VALID - The block is clean and possibly shared.
\item DIRTY - The block is modified and not shared.
\end{itemize}
In addition to these states, each block in main memory has a bit 
indicating whether or not it is DIRTY in some cache.  The cache that 
has a DIRTY block is called the {\em owner} of that block.  If a block is 
not DIRTY in some cache, then main memory is the owner.

The memory requests and their results are as follows:
\begin{itemize}
\item Read Hit:  A read hit is a read in which there is a local copy of 
the requested block and it is not in the INVALID state.  In this 
situation, no bus cycle is required.
\item Read Miss:  A read miss occurs if there is no local copy of the 
block, or if the block is in the INVALID state.  If another cache has a DIRTY 
copy, then the requestor gets a negative acknowledgement.  The 
owner of the DIRTY copy writes it back, resets the main memory DIRTY 
bit, and sets the local state to INVALID.  The requestor must retry and 
obtain the block from main memory.  When loaded in, the block is in 
the VALID state.
\item Write Hit:  If the block is DIRTY, then the block is written locally 
and no bus cycle is required.  If the block is INVALID or VALID, then a 
write miss is performed.
\item Write Miss:  A write miss is like a read miss, and the data must 
come from main memory.  Additionally, any other caches with that 
block in the VALID state, must set their local status to INVALID.  The 
block is loaded in the DIRTY state, and main memory reflects that it is 
not the owner.
\end{itemize}

\subsection{The Berkeley Machine}
The Berkeley scheme is similar to that of Synapse, except that it uses
cache to cache transfers on shared blocks and does not write back
DIRTY blocks to main memory when they become shared.

The internal states of a cache block are as follows:
\begin{itemize}
\item INVALID
\item VALID - The block is clean and possibly shared.
\item SHARED-DIRTY - The block is modified and possibly shared.
\item DIRTY - The block is modified and not shared.
\end{itemize}
A block cannot be DIRTY in more than one cache, but it can be
SHARED-DIRTY in one, and VALID in others.  The notion of ownership is
still used, and the last cache to write a block is responsible for
writing it back to main memory.

The memory requests and their results are as follows:
\begin{itemize}
\item Read Hit:  If the block is not in the INVALID state, no bus cycle is 
required.
\item Read Miss:  A read miss occurs if there is no local copy of the 
block, or the block is in the INVALID state.  If the block is DIRTY or
SHARED-DIRTY in another cache, that cache will supply the data and set
its local state to SHARED-DIRTY.  If it is not DIRTY, the block is
read from main memory.  Regardless, the block is loaded in the VALID
state.
\item Write Hit:  If the block is DIRTY, then the block is written locally 
and no bus cycle is required.  If the block VALID or SHARED-DIRTY, then 
an invalidation signal must be sent to all caches with this block.  The 
local state of the originator is then set to DIRTY.
\item Write Miss:  A write miss is like a read miss, and the data must 
come from the owner.  The originators local state for that block is set 
to DIRTY, and all other caches with the block set the state to INVALID.
\end{itemize}

\subsection{The Illinois Machine}
This scheme is also a cache to cache protocol, but it assumes that a 
missed block will always come from other caches, if possible, and 
only from main memory if no cache has a copy.  Thus each time a 
block is loaded, it can be determined whether or not is is shared.  
This has the advantage that write-hits on private blocks require 
no bus transaction.

The internal states of a cache block are as follows:
\begin{itemize}
\item INVALID
\item VALID-EXCLUSIVE - The block is clean and not shared.
\item SHARED - The block is clean and possibly shared.
\item DIRTY - The block is modified and not shared.
\end{itemize}
Again, a block cannot be DIRTY in more than one cache, and it is only 
written back to main memory upon replacement by the last 
processor to write it.

The memory requests and their results are as follows:
\begin{itemize}
\item Read Hit:  If the block is not in the INVALID state, no bus cycle is 
required.
\item Read Miss:  If another cache has a copy of the block, it puts it 
on the bus.  If the block is DIRTY, it is simultaneously written back to 
main memory.  All caches with copies of the block set the status to 
SHARED and the block is loaded by the originator in the SHARED state.  
If the block comes from main memory, meaning that no cache has a 
copy, it is loaded in the VALID-EXCLUSIVE state.
\item Write Hit:  If the block is DIRTY, then the block is written locally 
and no bus cycle is required.  If the block is SHARED, the write is 
delayed until all other caches have set that block to INVALID.  The 
originating cache then sets the block to DIRTY.
\item Write Miss:  A write miss is like a read miss, except that all 
other caches set this block to INVALID and the originator loads it in the 
DIRTY state.
\end{itemize}

\subsection{DEC's Firefly}
This again is a cache to cache protocol but here multiple writers are
permitted.  The data for writes to a shared block are transmitted to
other caches and to main memory when the write occurs.  Thus,
invalidation on a shared block never occurs and the INVALID state need
not be explicit.  In addition, there is a special bus called
SharedLine which is used to detect sharing of blocks among caches.

The internal states of a cache block are as follows:
\begin{itemize}
\item VALID-EXCLUSIVE - The block is clean and not shared.
\item SHARED - The block is clean and possibly shared.
\item DIRTY - The block is modified and not shared.
\end{itemize}
DIRTY blocks only are written back to main memory upon 
replacement.  

The memory requests and their results are as follows:
\begin{itemize}
\item Read Hit:  No bus cycle is required.
\item Read Miss:  If other caches have a copy of the block, they
simultaneously put it on the bus (the timing is set such that all 
caches respond in the same cycle).  All caches then set the status for 
that block to SHARED.  If the owning cache has the block in the DIRTY 
state, the block is also written back to main memory.  If no cache has 
a copy, which is indicated by the SharedLine not being raised, the 
block is loaded from main memory in the VALID-EXCLUSIVE state.
\item Write Hit:  If the block is DIRTY or VALID-EXCLUSIVE, then it can be 
written immediately, and the state is set to DIRTY.  If it is in the
SHARED state, the initiator must acquire the bus, and write to main
memory.  All other caches observe this write and update their local
copy simultaneously.  In addition, the updating caches raise the
SharedLine.  If the initiator sees the SharedLine raised, he keeps the
state of the block at SHARED.  Thus, the initiator has written the
data to all necessary locations but has not changed his local state of
the block.  If, however, the SharedLine is not raised, the initiator
can set his state to VALID-EXCLUSIVE.  Thus successive writes will not
need to be transmitted to other caches.
\item Write Miss:  A write miss is like a read miss plus a write hit.  
If the block comes from main memory, meaning that no other cache has a
copy, then the state is set to DIRTY, and local writes incur no
further overhead.  If the block is present in another cache, the initiator
sets the block state to SHARED and must then write the new block
to main memory.  As with a write hit, other caches will detect this
write and update their copies.
\end{itemize}

\subsection{Xerox Dragon}

States:
\begin{itemize}

\item VALID-EXCLUSIVE
\item SHARED-CLEAN
\item SHARED-DIRTY
\item DIRTY
\item INVALID

\end{itemize}

As with the Firefly, the invalid state is implicit.  However, since
there are effectively 5 states that need to be distinguished, at least
3 bits are needed to encode the state of a cache block whereas the
Firefly needs only 2 bits.

The Dragon protocol is like the Firefly's except that it does not have to
write through to main memory.  This is advantageous if we assume that
main memory is slower than the bus.  Archibald and Baer assume that
each memory transaction takes 4 bus cycles during which the bus is
held.

Archibald and Baer concluded that the Dragon and Firefly coherence
schemes performed the best with the Dragon being generally better.
The primary reason behind this is the ability to locally detect if
blocks are shared.  This is useful for unshared blocks.  For shared
blocks, 

\begin{enumerate}

\item The Dragon protocol does not incur a write to main memory.

\item Data is supplied by other caches and than marked as shared and dirty.

\item Upon a write, updating is used instead of invalidation.

\end{enumerate}

The performance advantage of item 3, however, is dependent on the
memory reference characteristic of the particuler load.

\subsection{General discussion}

A question was raised about the effect of the extra load placed on the
caches by having to monitor the cache transactions.  The more complex
the protocol, the more the load on the snooping caches, and this could
have an adverse effect on performance.

There was some discussion about the effect of technology on protocol
design.  Although a more complex protocol might provide better
performance, all else being equal, a simpler protocol which allows a
simpler implementation might have a faster cycle time, or might allow
more memory to be put on chip.  In general, a balance needs to be
acheived.

There is also some work on maintaining coherence in software by using
fence and flush instructions.  A fence operation blocks until all
buffered write transcations have completed.  There are 2 types of
flush operations: one invalidates the whole cache, and another
invalidates a particular block.  This type of software enforced cache
coherence complicates programming to a point that debugging becomes
next to impossible.  The programmer now has to keep track of the
states of each of the caches as well as the memory.

We then delved into some jargon concerning MIMD and SIMD models. 

\section{Cache coherence on general interconnection networks}

\begin{figure}
\centerline{\psfig{height=3.5in,figure=mp.idraw}}
\caption{Abstract view of a multiprocessor.}
\label{mp}
\end{figure}

If we limit ourselves to networks that rely on point to point links,
we cannot rely on a broadcast mechanism to maintain cache coherence.
This means that we can't use snoopy protocols because all the snoopy
protocols described above rely on a broadcast upon each cache
transaction.  To maintain coherence, we essentially need to find out
which caches contain a certain block so that we can update or
invalidate them upon a write.  We can do this by associating with each
block a list of the caches that contain a copy.  Coherence schemes
that use this idea are termed {\em directory-based}.

Two main issues influence the design of a directory-based cache
coherence scheme.  First, we have to decide where the directory
entries reside.  It is possible to keep the entries either on the
memory side or on the cache side.  Second, we have to decide on the
number of pointers that can be kept in the entry.  This determines how
many caches we can keep track of for each block.  Schemes that keep
the entries on the memory side and have a fixed number of entries are
termed {\sl limited} directories, while those that have as many
entries as there are caches are termed {\sl fullmap} directories.

The problem with fullmap directories is the memory requirements of the
system grows as $O(n^2)$.  This is because the number of cacheable
memory blocks grows as $O(n)$ and the size of each directory entry
also grows as $O(n)$.  This problem can be alleviated by using a
limited directory scheme, whose memory requirements only grow as
$O(n)$.  However, limited directories can only allow a fixed number of
caches to share a cache block, and some pointer need to be evicted to
make room for a new pointer when the directory entry overflows.  The
corresponding cache needs to invalidate its caches block.  This is a
performance hit if the number of caches that share a block is larger
than the number of available pointers.

A scheme that keeps the directory information on the cache side, and
allows an unlimited number of pointers is a chained directory scheme.
A linked list of caches that currently share a block is kept in the
cache.  Each cache contains a forwarding pointer to the next cache in
the list that has a copy of the block.  This scheme seems to be more
complex to implement than either the limited or fullmap directory
schemes.


% For GnuEmacs:
% Local variables:
% compile-command: "latex lecture23"
% End:

\end{document}