% to get hardcopy of this lecture,you need the following files (plus the
% normal latex and tex base directories):
%     lecture.tex:     latex source file
%     macros.tex:      figure macros
%     psfig.tex:       postscript figure (psfig) macro definition
%     figI.idraw:        figures created with idraw (I = 1,2,3,4)
%     pocs-header.sty: Principles of Comp Systems lecture latex header
%
% run the following programs:
% latex lecture
%   -> Creates lecture.dvi, plus latex intermediate files
% latex lecture
%   -> Run a second time to get cross-references right
% dvi2ps lecture |  egrep -v '^showpage$' > lecture.ps
%   -> Merges the figures with the text, can also pipe to lpr
%
\documentstyle[12pt,pocs-header]{article}
\Scribe{Atul Adya}
\Lecturer{Bill Weihl}
\LectureNumber{7}
\LectureDate{October 2, 1991}
\include{macros} % used to include figures in this document
\begin{document}
\MakeScribeTop

\section{Handouts}
	Handouts 15 ({\em Examples of Concurrency}), 16 ({\em Basics of
 Performance Analysis}) and 17 ({\em Problem Set 3}) were distributed.

\section{Overview}
	The lecture discussed a mathematical model (esentially
 a queueing network model which uses the operational approach\footnote
{Operational analysis is a framework which allows the analyst to test
 whether each assumption is met in a given system.}) for analyzing or
 evaluating the performance of a system. It also introduced the
 concept of concurrency and concurrent programs.


\section{The Queueing Model}

The model used for discussion was a central server model. In
 this model, device 1 is the CPU and devices 2,3,4 \ldots K are
 sub-servers. The assumptions for this model are :

\begin{itemize}
\item A job enters the system through the CPU queue. From there it is
 routed to various sub-servers (circulates in the network, waiting in
 queues and having service requests processed at variuos devices). When 
the job is done it exits the system or goes to the terminal sub-system (
 as explained later).
\item No job overlaps its use of different devices (although in real
 life situations I/O and CPU use overlaps but the error introduced by
 this assumption is insignificant).
\item A device is busy if a request is pending there --- no part of the
 system can block progress in another part. This assumption is not met
 in real systems, {\em e.g.} the CPU might be unable to continue if an
 I/O buffer is full.
\item A job is {\em in queue} at device {\em i} if it is waiting for
 or receiving service there. 

\end{itemize}

\begin{figure}
\centerline{\psfig{width=4in,height=3in,figure=fig1.idraw}}
\caption{Open Central Server System}
\label{fig1}
\end{figure}

There are two types of server systems --- open and closed.
\begin{itemize}
\item {\bf Open System} --- In this system, a job enters the system from
 the CPU queue and  exits when it is done. Fig 
~\ref{fig1} shows an example of an open system with a CPU and
 $K-1$ sub-servers.

\begin{figure}
\centerline{\psfig{width=4in,height=3in,figure=fig2.idraw}}
\caption{Terminal-driven System}
\label{fig2}
\end{figure}

\item {\bf Closed System} --- In a closed server system (Fig
 ~\ref{fig2}), there is a
 terminal sub-system along with the CPU and sub-server system. There is
 a user at each terminal (if all terminals are being used) who
 alternates between thinking and waiting. In the thinking state, the
 user is contemplating what job next to submit, and the central
 sub-system (CPU and the sub-servers) are performing no work for him.
On submitting a next job, the user enters the waiting state, where he
 remains until the central sub-system completes the job for him.

The mean time a user spends in a thinking interval is the {\em think
 time}, (denoted by Z) and the mean time the user spends in a waiting
 interval is the {\em response time of the sub-system} (denoted by R).
 Thus, $R+Z$ is the total cycle time for a user in the closed system,
 whereas in the open system R is the total time a job takes in the
 system. Since
 users think independently, the think time (Z) is independent of the
 numbers of users (M) in the system. A point to be noted is that a
 closed central server network can actually be modelled as an open
 server system in which the output has been connected back to the input.
\end{itemize}
\vspace {5mm}
\section{The Basic Operational Laws}

In this queuing model there are some basic parameters which can be
 measured. From these basic parameters we can derive certain other
 quantities which are useful for analyzing the performance of the system
\footnote{The parameters used  have been defined in handout 16 and we
 will use the same symbols for discussion.}. The derived parameters are
related to each other by the following laws of operational analysis :

\subsection{Principle of Job-Flow Balance} \vspace {1 mm}

\begin{center} \fbox{$\lambda _i = X_i$} \vspace {1 mm} \end{center}

The {\em Principle of Job-Flow Balance} states that for each device
 {\em i}, $X_i$ is the same as the total input rate ($\lambda _i$),
 {\em i.e.} in the observable time T, $A_i \approx C_i$.
This principle will give a good approximation for observation periods
 long enough that the difference between arrivals and completions,
 $A_i - C_i$ is small compared to $C_i$. \vspace {4 mm}

\subsection{Utilization Law} \vspace {1 mm}

\begin{center} \fbox{$U_i = X_i S_i$} \vspace {1 mm} \end{center}
			
This law states that the utilization ($U_i$) of the {\em ith} device
 is given by the completion rate ($X_i$) times the service time for
 each job ($S_i$) at that deivce {\em e.g.} if each job takes 20 msec 
($S_i$) at device {\em i} and the completion rate is 10 requests/sec
 ($X_i$) then the utilization is 20\%.
If there is {\em job-flow balance} then $U_i = \lambda _i S_i$.
 \vspace {4 mm}

\subsection{Little's Law} \vspace {1 mm}

\begin{center} \fbox{$\overline{N}_i =  X_i R_i$} \vspace {1 mm} \end{center}

This law states that the mean queue length at device {\em i} is product
 of the completion rate ($X_i$) and the response time ($R_i$).
Consider a new job A which has just arrived. It would be processed in
time $R_i$ and in this time $X_i R_i$ jobs would have gone out of
 this device. Since all these jobs were ahead of A in the queue, so
 $X_i R_i$ is the queue length when A had arrived.  \vspace {4 mm}

\subsection{General Response Time Law} \vspace {1 mm}

\begin{center} \fbox{$R = \sum_{i=1}^{n} V_i R_i$} \vspace {1 mm} \end{center}

This law states that response time of a system is the sum of the
 products of the visit ratios ($V_i$) and the individual response time
 ($R_i$) of each device {\em i.e.} every job on an average spends
 $V_i R_i$ time at device {\em i} and the response time is just the 
sum of the times spent at each device. Note that this law holds good
 even if the job flow is not balanced.  \vspace {4 mm}

\subsection{Interactive Response Time Formula} \vspace {1 mm}

\begin{center} \fbox{$M = (R+Z) \ X_0$} \vspace {1 mm} \end{center}
		
The mean time for a user to complete a think-wait cycle is $R+Z$. When
the job flow is balanced $X_0$ will denote the rate at which cycles
 are completed. By Little's Law, $(R+Z) \ X_0$ must be the mean number
 of users observed to be in a think-wait cycle. And the number of such
 users is M {\em i.e.} all the users in the system.  \vspace {4 mm}

\subsection{Forced Flow Law} \vspace {1 mm}

\begin{center} \fbox{$X_i = V_i X_0$} \vspace {1 mm} \end{center}
		
Actually, we can look at the formula as $V_i = C_i/C_0$.
Intuitively speaking, if $C_0$ jobs have been completed in T
 time and $C_i$ have been completed in the same time, then the number
 of times the job visited device {\em i} is $C_i/C_0$ on an average.
\vspace {7mm}

\section{Bottleneck Analysis}

This part deals with the asymptotic behaviour of throughput and the
 response time of systems as N (or M) increases. We will assume that the
 visit ratios and mean service times are invariant under changes in N.
Note that the ratio of completion rates for any two devices is equal to
 the ratio of their visit ratios :
\begin{center} $X_i / X_j = V _i / V _j$ \end{center} 
Since $U _i = X _i S _i$, a similiar property holds for utilizations :
\begin{center} $U _i / U _j = V _i S _i / V _j S _j$ \end{center}
Device {\em i} is saturated if its utilization is approximately 100\%.
 If $U _i = 1$ the utilization law implies that $X _i = 1 / S _i$.
In general, $U _i \leq 1$ and $X _i \leq 1 / S _i$. \\
Let the subscript {\em b} refer to any device capable of saturating as N
 becomes large. Such devices are called {\em bottlenecks} since they
 limit the overall performance of the system.
The device to achieve maximum utilization (near 100\%) is clearly the
 one which has the largest value of $V _i S _i$.
Therefore , if there are K such devices then
\begin{center} $V _b S _b =  max \{ V _1 S _1,V _2 S _2,...,V _K S _K \}$ \end{center}
Thus, when N becomes large we observe that
\begin{center}  $U _b = 1$ and $X _b = 1/S _b$ \end{center}
 Since $X _0 / X _b = 1 / V _b$ this implies
\begin{center} $X _0 = 1 / V _b S _b$ \end{center}
is the maximum possible value of system throughput. Since $V _i S _i$ is
 the total of  all service requests per job for device {\em i} , the sum
\begin{center} $R _{min} = V _1 S _1 + V _2 S _2 + \ldots + V _K S _K$ \end{center}
(which ignores  queueing delays) denotes the smallest possible value of
 the mean response time. In fact, $R _{min}$ is the response time when
 $N = 1$. This imples that $X _0 = 1 / R _{min}$ when $N = 1$ and in
 general,
\begin{center} $X _0 \leq N / R _{min}$ \end{center}

\begin{figure}
\centerline{\psfig{width=3in,figure=fig3.idraw}}
\caption{System Throughput Curve}
\label{fig3}
\end{figure}

The curve for $X _0$ can be seen in Figure ~\ref{fig3}. As a function of N, $X_0$ 
rises monotonically from $1 / R _{min}$ at $N=1$ to the asymptote
 $1 / V _b S _b$. It stays below the line of slope $1 / R _{min}$
 emanating from the origin.
Job interference via queueing when $N=k$ usually prevents throughput
 from achieving $k/R _{min}$.
Were we to hypothesize that {\em k} jobs always managed to avoid
 delaying each other in the network, so that $X _0 = k/R _{min}$, the
 saturation asymptote requires that
\begin{center}  $k/R _{min} = 1/V _b S _b$ , or in general \end{center}
\begin{center} $k \leq N^* = R _{min}/V _b S _b = (V _1 S _1 + ... + V _K S _K)/V _b S _b$ \end{center}
That is, if $k > N ^*$ then jobs would be queued up somewhere in the
 system. $N ^*$ is the point beyond which queuing is bound to be
 observed. Thus, we call $N ^*$ as the {\em saturation point} of the
 system.

\begin{figure}
\centerline{\psfig{width=3in,figure=fig4.idraw}}
\caption{Response Time Curve}
\label{fig4}
\end{figure}

For the response time, clearly by Little's Law, since $R _0 = N/X _0$
 and $X _0 \leq 1/V _b S _b$ , we have
\begin{center} $R _0 \geq N V _b S _b$ \end{center} 
Therefore, $R _0$ is bounded below by the curves $R _{min}$ and
 $N V _b S _b$.

In a closed system, $R _0 = M/X _0 - Z$ and we know that
 $X _0 \leq 1/V _b S _b$. Thus,
\begin{center} $R _0 \geq M V _b S _b - Z$ \end{center}
So $R _0$ is bounded by $R_{min}$ and $M V _b S _b - Z$ in a closed system
 which has essentially shifted the $R _0$ curve for an open system down
 by a value Z (see Figure ~\ref{fig4}).

But in actual practice the throughput ($X _0$) and response time (R) behave
 differently.  For example, the curve for the throughput ($X _0$) does not
 remain parallel to the asymptote $1/V _b S _b$. Instead after a
 saturation stage, the throughput starts going down (this phenomenon is
 known as {\em thrashing}). Also the response time increases very 
sharply after this saturation stage.
Such a behaviour is observed because in our
 model we have assumed thaat the queue length is independent of the
 service time and the number of visits to a device are independent of
 the load , which in general is not valid. \pagebreak

\section{Introduction to Concurrency}

Concurrent programs can get very hard to understand very quickly even if they are small.  Just as for sequential programs we need a way to:
\begin{itemize}
\item specify concurrent programs
\item implement current programs 
\item and prove them correct
\end{itemize}

These issues will be discussed over the next couple of lectures along
 with issues that effect performance.
Let us see a simple example that will illustrate the types of problems
that concurrency can cause.

\subsection{Incrementing a Register}

We begin with the first example in handout 15: {\em Incrementing a
Register}. Given a register that has a state s which is an integer, and
two operations:
\begin{itemize}
\item read, that returns the current state and 
\item write, that sets the current state
\end{itemize}

The operation we want to perform is an increment. In a sequential
system we could perform the increment in three steps:
\begin{itemize}
\item read the current value into a temporary
\item increment that value
\item write the updated value back into the register
\end{itemize}

as shown on page 1 of the handout.  Now consider what happens if we
have multiple processes accessing this register via the increment
procedure. If two processes are executing this procedure concurrently,
both could execute the read operation, each could then update the
temporary, then both could write back the updated version. The
register would only be incremented once instead of twice. This might
not be what we had in mind. This is what is commonly known as a case of
 {\em lost update}.
\subsection{Atomicity}

	To analyze this properly, we need to know the low level atomic
operations. For some operations Spec defines the level of atomicity,
for example, the assignment is atomic once the RHS has been computed.
We can infer from this that the Read and Write procedures appear to
act atomically by examining their bodies. It is possible, however, to
implement read and write as non-atomic operations, for example, by
viewing the register as an array of bits and operating on the bits one
at a time. If operations of two processes can be interleaved at the
level of the individual bits, the result could be practically
anything. \vspace {0.2 in}
 
	Therefore we need to know what the atomic operations are; What
are those operations that can be performed without any interleaving or
interference by any other processes. Without knowing this we cannot
understand a concurrent program.  The way to think about a concurrent
program is that the processes each take steps and that they are
running asynchronously.  The execution of the program is then as
follows: at each time step we pick a process and that process does the
next atomic step according to its own PC.  We then pick the next
process to run the next step more or less randomly.  The result is
interleaving at the level of the atomic operations. \vspace {0.2 in}

	A major issue in concurrent programming is how to make complex
operations (like increment) appear atomic. In Spec we have atomicity
brackets to define larger atomic operations from smaller ones. (Spec
uses double brackets. Often in the literature we will see single
brackets.) Assume code inside atomicity brackets executes atomically. \vspace {0.2 in}

In the example in handout 15, assume the read and write are
atomic. If the register started with value k and n processors are
concurrently executing the increment, the possible result values
include any integer from k+1 (if the all read the current value before
any of them update it) to k+n (if each processor completes the
increment before any other processor reads the current value) and
anything in between. Recall if the reads and writes are not atomic
then the possible results might be quite different.
This problem appears at all levels of concurrent systems, register
operations (as shown here) and at higher levels such as operating
systems and data bases etc.  \vspace {0.2 in}

The simplest way to make complex operations atomic is to enclose them
in atomicity brackets.  This feature is not something that most
languages provide (although there are some proposed languages that
do). One way to implement the atomicity bracket is to not allow any
other process to execute their next step while one process is
executing code within atomicity brackets.  On multiprocessors, this
wastes cycles on the idle processors and can have serious impact on
performance. Even on uniprocessors we wouldn't want to implement
atomicity brackets this way. Suppose that the process performs I/O,
the process would have to wait until the I/O is done before execution
proceeds. \vspace {0.2 in}

There are two types of atomicity that people tend to talk about :
\begin{itemize}
\item Atomicity with respect to failures: what are the operations that
once you start them are guaranteed to complete even in the presence of
crashes (or at least the effect will be that they totally completed or
appear as if unstarted)

\item Atomicity with respect to concurrency: what are the operations
that can not be interrupted by other processes.
\end{itemize}

But these two views are really the same if you think of crash as a
concurrent process that can wake up and crash the system between
atomic operations.  We need to know what operations will not be
interrupted.

Its clear the we can't just sprinkle atomicity brackets any where we
 want. If you are writing a
specification then atomicity brackets around large segments of code
are acceptable. However, if we write 
\begin{verbatim} <<disk.write; disk.write>> \end{verbatim}
then we can
reason about this as if it is atomic but we still have the task of
implementing this as atomic. If we are concerned only about
concurrency and not crashes then the implementation of large atomic
sections is straightforward. With crashes the problem is more complex.
At the start we must determine the model of failures in order to
decide what can be done atomically. Consider the appropriate level of
abstraction for your task. We can describe implementations at many
levels abstraction.

\section{Critical Sections}

Code that is supposed to run without interference from other processes
is called a {\em critical section}. This term arises because if that
 code is
interrupted or interleaved with code from another process, the state of
the system may become damaged. One safe way to implement a critical
section is to insure that nothing else runs concurrently with it. Such
an implementation is safe but may perform poorly. \vspace {0.2 in}

A problem arises only when two processes access common state
(variables, and values pointed to). If they don't access same state
then they can run concurrently. If both access the same state but both
are only reading that state there is no problem.  Only in the case
 that at least one
is writing and both are accessing do we have a problem. Hence, a lock is
associated with shared data. To access the data, a process has to get
 the lock before accessing the data and release it after done
accessing the shared data. If the performance is still not good enough,
partition the shared data into subsets and associate a lock with each
subset. Then prior to entering a critical section get all the locks
that are associated with the state referenced in the critical section.
When the critical section is complete the locks are released.  All
other methods boil down to this (These sections do not necessarily run
 atomically; they just appear to). It is easier to reason about atomic
 chunks. The larger the atomic chunks, the easier the program is to
 reason about.

\section{How Locks Work}

The approach presented on Page 2 of handout 15 is used in many systems
today. Mutex is a standard term in the literature. It stands for
mutual exclusion. This spec provides a way to create a new mutex (will
be more than one mutex). The state of the mutex is nil if no process
is holding a lock or it is the thread name (process id) of the process
\footnote{ There is a confusion between the term process (this
 has connotation of separate address space) and the term thread.  For
 this class the differences are not critical.} that is holding the
 lock.

This spec also provides for releasing and acquiring locks. {\em Release}
atomically sees if the process holding lock is the one asking to
release (need some magic to get name of current thread). If the caller
to release is the current holder it releases the mutex.

{\em Acquire} is more subtle. Atomically if current state of the mutex
 is nil it is set to the name of the current process. This must be done
atomically. If it is not done atomically race conditions could occur
in which two processes both find it nil and both acquire it.

Also if the guard is not satisfied then the thread waits at the
 beginning of the {\em Acquire} procedure since it cannot make an atomic
 transition to the final state (the semantics of specs are discussed
 in a later lecture).

The procedure {\em Increment} has been written using mutex on page 3 of
 handout 15. The use of mutex prevents concurrent calls of increment
 interfering with eaach other (this of course assumes that any thread
 accessing this register uses the same mutex to lock the register).
\end{document}