Computer Systems

Modelling Operational Analysis

Simon Crosby Part II, CST, 1996

Operational Analysis

Based on the idea of observation rather than a probablistic description of system behaviour

Operational analysis makes very weak assumptions about the system being modelled.

It is also concerned with quantities “directly related” to observables

Begin by examining some fundamental quantities and operational laws

We examine a system for some time recording the customer arrivals and departures, and define the quantities of interest • T is the length of time we observe the

system

• A is the number of arrivals observed

• C is the number of departures (or completions of service) observed.

If the system is a single resource, then we can also measure:

• B, the time for which the resource was busy.

From these quantities we can define the following quantities

• Arrival rate, λ =^{△} \frac{A}{T}

• Throughput, X =^{△} \frac{C}{T}. For a single resource, we can also define:

• Utilization, U =^{△} \frac{B}{T}

• Average service requirement, S =^{△} \frac{B}{C}

The Utilization Law

Our first “law” is just an algebraic identity

U =^{△} \frac{B}{T} = \frac{C}{T} \frac{B}{C} = XS

Our next law requires a further quantity to be observed

• W, the job-time product which is sum of the durations of all customers in the system over the observation period.

From this we can define

• Average number of requests in the

system, N =^{△} \frac{W}{T}

• Average residence time per request,

R =^{△} \frac{W}{C}

Again, simple manipulation gives the by now familiar Little’s Law

N =^{△} \frac{W}{T} = \frac{C}{T} \frac{W}{C} = XR

• very weak assumptions about the system

• applicable to a wide range of systems

• can be applied recursively to subsystems and to individual resources

An Example

Observe system for T = 10 sec 4 customers spend 10 s in the system One customer who spends 5 s Then we have W = 4 × 10 + 1 × 5 = 45 s. If also A = C = 5 then X = C/T = 5/10 = 0.5 customers per second λ = 5/10 = 0.5 customers per second N = 45/10 = 4.5 customers R = 45/5 = 9.0 s per customer

A Simple Interactive System

Fixed number, M, of users logged on

Customer is at the terminal whilst thinking

The think time, Z is the average time a user spends between receiving a prompt and responding

A customer not thinking is somewhere inside the central subsystem We can use Little’s Law to relate some observable quantities in the central subsystem:

• N is the number of customers in the central subsystem 0 ≤ N ≤ M

• X is the rate at which customers complete in the central subsystem

• R is the average time a customer spends in the central subsystem (intuitively equivalent to “response time”)

If we observe that system throughput is 0.5 interactions per second and we find on average 7.5 users in the subsystem then

R = \frac{N}{X} = \frac{7.5}{0.5} = 15 s

from Little’s Law

We can also “view” R as response time rather than residence time to get the interactive system version of Little’s Law

M = X(R + Z)

where R is the response time and Z is the think time

The residence time is R + Z (which we were calling R above)

With 10 users logged on, 5 s average think time and an average response time of 15 s,

X = \frac{M}{R + Z} = \frac{10}{15 + 5} = 0.5 interactions/sec

Under heavy load (M large or Z small)

U ≈ 1

Using the Utilzation Law the throughput X ≈ \frac{1}{S} and hence

R = \frac{M}{X} − Z ≈ \frac{1}{S}M − Z

⟹ The response time grows approximately linearly with the number of users M.

Visit Counts and Forced Flow

Extend our notation to allow the modelling of multiple i = 1, 2, …, K devices

e.g. Throughput at a device i =^{△} X_{i} etc

Assume that the service required by a customer is an inherent property of the customer not of the state of the system

A visit count for a device is the number of completions at that device for every completion from the system

V_{i} =^{△} \frac{C_{i}}{C}

Where C_{i} is the number of completions at device i.

The Forced Flow Law

Since X = \frac{C}{T}, we have

X_{i} = \frac{C_{i}}{T} = \frac{C_{i}}{C} \frac{C}{T} = V_{i}X

the Forced Flow Law

If devices are load independent, then define the service demand a customer makes on a device i,

D_{i} =^{△} V_{i}S_{i}

We can then determine the utilization at each device

U_{i} = X_{i}S_{i} = (XV_{i})S_{i} = X(V_{i}S_{i}) = XD_{i}

If we also know the N_{i}

N_{i} = R_{i}X_{i} ⟹ R_{i} = \frac{N_{i}}{X_{i}}

by applying Little’s Law at each device, where R_{i} is the residence time at device i.

We can infer X_{i} from U_{i} but can not find R_{i} without further assumptions

We can approximate it by R_{i}^{∗}

R_{i}^{∗} = N_{i}S_{i} + S_{i}
= R_{i}^{∗}X_{i}S_{i} + S_{i}
= R_{i}^{∗}U_{i} + S_{i}

Reducing this we get

R_{i}^{∗} = \frac{S_{i}}{1 − U_{i}}

and

N_{i} = R_{i}^{∗}X_{i}
= \frac{S_{i}X_{i}}{1 − U_{i}}
= \frac{U_{i}}{1 − U_{i}}

which looks remarkably like the M/M/1 queueing system.

Bottleneck Analysis

A bottleneck in a system is a hindrance to movement or progress.

Given the forced flow assumption, at high loads system performance is determined by the device with the highest utilization: the bottleneck

The ratio of the completion rates of any two devices is

\frac{X_{i}}{X_{j}} = \frac{V_{i}X}{V_{j}X} = \frac{V_{i}}{V_{j}}

Since U_{i} = X_{i}S_{i}, we have a similar property for utilizations

\frac{U_{i}}{U_{j}} = \frac{X_{i}S_{i}}{X_{j}S_{j}} = \frac{V_{i}S_{i}}{V_{j}S_{j}}

If: V_{i} are intrinsic properties of customers and S_{i} independent of the queue length at device i then the system has load independent parameters In such cases, the throughput and utilization ratios are the same for all loads This can be used to determine asymptotes for X and R: In general, U_{i} ≤ 1 and X_{i} ≤ \frac{1}{S_{i}} Device i is saturated if U_{i} → 1 Thus, as U_{i} → 1, we see X_{i} → \frac{1}{S_{i}} Use the subscript b to denote a device capable of saturating

Since the utilization ratios are fixed, the device i with the largest V_{i}S_{i} will be the first to achieve 100% utilization as N increases:

V_{b}S_{b} = max {V_{1}S_{1}, ..., V_{K}S_{K}}

so the bottleneck is determined by device and workload (the V_{i} and S_{i}) properties

As N increases, U_{b} → 1 and X_{b} → \frac{1}{S_{b}} Now, XV_{b} = X_{b} so that

X_{max} = \frac{X_{b}}{V_{b}} → \frac{1}{V_{b}S_{b}}

is the maximum possible system throughput as N increases.

The total service required by a customer in the system is

R_{min} = K ; ∑ ; i=1 V_{i}S_{i}

This also denotes the smallest possible value of mean response time

In fact, R_{min} is the response time when N = 1 This implies that X = \frac{1}{R_{min}} when N = 1 and that for N > 1, X ≤ \frac{N}{R_{min}} .

Assume that k ≤ K jobs always manage to avoid each other so that X = \frac{k}{R_{min}} , we would require that

\frac{k}{R_{min}} ≤ \frac{1}{V_{b}S_{b}}

Alternatively k ≤ N^{∗} where

N^{∗} = \frac{R_{min}}{V_{b}S_{b}} = \frac{∑_{i=1}^{K} V_{i}S_{i}}{V_{b}S_{b}}

N^{∗} represents a load beyond which queueing is certain to exist

System Throughput

System throughput, X, rises monotonically from \frac{1}{R_{min}} at N = 1 to the asymptote \frac{1}{V_{b}S_{b}} It stays below the straight line X = \frac{N}{R_{min}}

Interactive Response Time

• M terminals

• Average think time Z

• Mean response time

R = \frac{M}{X} − Z

When M = 1, we must have R = R_{min} The throughput, X, cannot exceed \frac{1}{V_{b}S_{b}}, so R ≥ MV_{b}S_{b} − Z ≥ MV_{i}S_{i} − Z, i = 1, ..., K by the choice of device b The response time asymptote intersects the x axis at

M_{b} = \frac{Z}{V_{b}S_{b}}

It intersects the minimum response time R_{min} at M_{b}^{∗} where

M_{b}^{∗}V_{b}S_{b} − Z = R_{min}

System Response Time

Thus

M_{b}^{∗} = \frac{R_{min} + Z}{V_{b}S_{b}} = N^{∗} + M_{b} , say

When there are more than M_{b}^{∗} terminals, queueing is certain to exist

Summary

• the largest of the V_{i}S_{i} determines the bottleneck b.

• the sum of these products determines the smallest possible response time R_{min}.

• queueing cannot be avoided when N exceeds N^{∗} = \frac{R_{min}}{V_{b}S_{b}}.

• queueing cannot be avoided in an interactive system when N^{∗} + \frac{Z}{V_{b}S_{b}} terminals are logged on

Summary

• Reducing V_{i}S_{i} for a non-bottleneck device will not move the \frac{1}{V_{b}S_{b}} asymptote

• It may improve R_{min}, but it will not improve response time at high loads

• Reducing V_{i}S_{i} for the bottleneck will

– raise the \frac{1}{V_{b}S_{b}} asymptote on the throughput graph

– lower the MV_{b}S_{b} − Z asymptote on the response time graph

– reduce R_{min}

• Asymptotes will remain relevant only as long as V_{b}S_{b} remains the largest of the V_{i}S_{i}

Example: interactive system

Suppose that Z = 20 s.

No. device  S_{i} (s)  V_{i} D_{i} = V_{i}S_{i}
1 CPU  0.05  20  1.00
2 disk  0.08  11  0.88
3 fast disk 0.04  8  0.32
R_{min}  2.20

Question: Is an 8 second response time feasible with 30 users logged on? If not, what changes are required?

The V_{i}S_{i} products are V_{1}S_{1} = 1 s, V_{2}S_{2} = 0.88 s and V_{3}S_{3} = 0.32 s

The minimum response time is 2.2 s (since 1 + 0.88 + 0.32 = 2.20) and the CPU is the bottleneck For M = 30, the response time asymptote requires that

R ≥ 30 × 1 − 20 = 10 s

So the answer is no

To make an 8 second response time feasible, we need to speed up the CPU, so that the new service time obeys

MV_{1}S′_{1} − Z ≤ 8

or

S′_{1} ≤ \frac{20 + 8}{30 × 20} = .047 s

which is a 7% speed up in the CPU.

Then V_{1}S′_{1} = 0.93 is still the largest product so the CPU is still the bottleneck.

Question: Is 10s response time feasible when 50 users are logged on? If not, how much CPU speedup is required?

If S_{1} → 0, the disk will become bottleneck:

R ≥ MV_{2}S_{2} − Z

For M = 50, this is

R ≥ 50 × 0.88 − 20 = 24 s

so a 10s response time is not feasible with M = 50 and no amount of CPU speedup is capable of achieving it.

Balanced System Bounds

Balanced system bounds provide tighter bounds at mid-range loads than bottleneck analysis

A system is balanced if for any load the utilizations of all devices are equal

Balanced systems exhibit the following important property that

U_{i}(N) = \frac{N}{N + K − 1}

So the system throughput is given by

X(N) = \frac{U_{i}}{D_{i}} = \frac{N}{N + K − 1} × \frac{1}{D_{i}}

For example,

\begin{matrix}N = 1 & K = 2 & U_{1} = U_{2} = \frac{1}{2} \\ N = 1 & K = 100 & U_{1} = … = U_{100} = \frac{1}{100} \\ N = 100 & K = 2 & U_{1} = U_{2} = \frac{100}{101} \\ N = 2 & K = 2 & U_{1} = U_{2} = \frac{2}{3} \end{matrix}

Look at the system and find • D_{max} – maximum demand at any device • D_{min} – minimum device demand

• D_{av} – average demand at each device • D = D_{tot} – total demand across all

devices So, we have D_{av} = D/K We can construct imaginary balanced systems related to our system

pess1: balanced system with K devices each with a demand of D_{max}

opt1: balanced system with K devices each with a demand of D_{min}

The throughput of system pess1 is

\frac{N}{N + K − 1} × \frac{1}{D_{max}}

The throughput of system opt1 is

\frac{N}{N + K − 1} × \frac{1}{D_{min}}

So for the system being modelled we have

\frac{N}{N + K − 1} ×\frac{1}{D_{max}} ≤ X(N) ≤ \frac{N}{N + K − 1} ×\frac{1}{D_{min}}

And since N = XR, (N + K − 1)D_{max} ≥ R(N) ≥ (N + K − 1)D_{min}

This will give tighter bounds on the mid-range performance than the bottleneck bounds We can do even better using the following line of argument

What is the best performance that the system can achieve?

Answer: spread the load out evenly among all the devices.

opt2: D_{av} at each of the K devices with throughput

\frac{N}{N + K − 1} × \frac{1}{D_{av}} = \frac{N}{D + (N − 1)D_{av}}

Now, what is the worst system subject to the constraints that D and D_{max} remain fixed?

Answer: place D_{max} at as many devices as possible and 0 at the rest pess2: D_{max} at each of the \frac{D}{D_{max}} devices with

throughput

\frac{N}{N + K − 1} × \frac{1}{D_{max}} = \frac{N}{D + (N − 1)D_{max}}

So that for the system being modelled

\frac{N}{D + (N − 1)D_{max}} ≤ X(N) ≤ \frac{N}{D + (N − 1)D_{av}}

And D + (N − 1)D_{max} ≥ R(N) ≥ D + (N − 1)D_{av} As N → ∞,

\frac{N}{D + (N − 1)D_{max}} → \frac{1}{D_{max}}

Note that at high loads the bottleneck bounds are the limiting high bound The asymptotic bottleneck bound \frac{1}{D_{max}} and the optimistic balanced bound intersect at N^{†} where

\frac{1}{D_{max}} = \frac{N^{†}}{D + (N − 1)D_{av}}

So that

N^{†} = \frac{D − D_{av}}{D_{max} − D_{av}}

System Measurement

We call devices which measure system performance monitors

Can be in hardware, software or hybrid software/hardware

Whatever form a monitor takes, we can describe it in terms of monitor artifact and monitor power.

Monitor artifact describes the side effects that operation of the monitor has on the system being observed

Software monitors will have artifact depending on processing requirements Monitor power has at least 5 dimensions:

• domain – the class of activities observable

• input rate

– the maximum frequency at which

events can be recognized and recorded • input width

– the amount of information the monitor can extract and process when a monitored event occurs

• recording capacity – the amount of memory available for

storing extracted information

– for hardware monitors (in particular):

∗ event activity can be measured by a single counter

∗ average duration can be derived from two values: accumulated duration of all occurrences of the activity and the total number of occurrences

∗ distributional information requires much more memory

• monitor resolution

– the resolution of the time clock from which the monitor derives timing information