Computer Systems Modelling

Simon Crosby

Twelve lectures (remainder of term) covering:

• Introduction to modelling: what is it, when is it useful ?

• Simple queueing theory: Stochastic processes, Markov chains.

• Birth/death processes, flow balance equations, queueing systems. The M/M/1 queue.

• Series and parallel stages, Multiple queue systems.

Computer Systems Modelling

• Operational analysis: balanced system bounds

• Queuing networks: Bottleneck analysis, improvements based on bottleneck analysis.

• Simulation modelling: algorithms, random variate generation, output analysis.

Computer Systems Modelling

• Measurement techniques: software and hardware monitors.

• Case studies. Recommended books:

• Kleinrock: Queueing Systems - Vol 1: Theory, Wiley 1975

• Lazowska et al: Quantitative System Performance, Prentice Hall 1984

• Leung: Quantitative Analysis of Computer Systems, Wiley 1988

Why model?

1. A manufacturer may have a range of compatible systems with different performances. A model might be used to determine the best configuration for a particular application.

2. If an existing system is performing inadequately a model can be used to determine how to improve the system and give an indication of the performance of the resulting system.

3. Fundamental design decisions may affect performance of a system. A model can be used as part of the design process to avoid bad decisions.

Introduction

• Begin with simple example: a canning line • Cans arrive at rate r = 1 can/s

• Filling takes 0.9 s/can

• Capacity c = 1/0.9 = 1.1 can/s

• What if r ≥ c ? CHAOS

• Mean capacity MUST exceed mean arrival

Introduction

• More general flow problems have random properties: arrival and service times

• This complicates the analysis considerably. Examples:

– arrival of keystrokes

– arrival of disk requests

– arrival and service of network packets • Single resource – difficult, network of

resources seems impossible

• Simplifying assumptions and powerful techniques needed

A toy problem

Database on type A system runs like a drain. Type B1 system available immediately, and a type B2 system in the future. What should the company do?

system CPU time disk time total
A  4.6  4.0  8.6
B1  5.1  1.9  7.0
B2  3.1  1.9  5.0

How can modelling help?

Typical performance questions we might ask include:

• How long will a database request wait before CPU service?

• What is the expected utilization of the resource (CPU, disk...)?

• How many requests will typically be queued ?

• What is the expected queue length at time t ?

These in general involve analysis using prob distributions of the time periods involved (arrival, service etc).

Analytical Techniques

More than 1 way to skin a cat...

• Queueing theory – stochastic processes, queueing networks

• Operational analysis – weaker assumptions but computationally expensive

• Back of the envelope – efficient but gives limited insight

Simulation Techniques

• Construct a computer program which emulates system under study

• The simulation measures its own performance

• Useful for awkward modelling problems with no analytical solution

• Problems include complexity, output analysis and stopping rules.

A Simple Queueing System

[figure: arrivals; queue; server; departures]

• Arrival process A(t) = ℙ ( inter-arrival time ≤ t)

• Service process B(x) = ℙ ( service time ≤ x)

Queueing Systems Analysis

Various properties of service system are important:

• Storage capacity available for waiting customers

• The number of servers available

• The types of the arriving customers (different demands)

• Queueing discipline used: FCFS, LCFS, SIRO, priority

• Defections, jockeying, balking, bribing...

Queueing Systems Notation

The Kendall notation describes a single queueing system using the notation A/B/m/k/l where:

• A is the inter-arrival time distribution of customers

• B is the service time distribution

• m is the number of parallel servers

• k is the limit on the queue length

• l is the the population size

Queueing Systems Notation

Typical values for A, B include:

• M – exponential distribution

• E_{r} – r stage Erlangian distribution

• H_{r} – r stage Hyperexponential distribution • D – Deterministic

• G – General Examples: M/M/1 – exponential inter-arrival, exponential service, single server M/E_{r}/1 – exponential inter-arrival, r stage Erlang service, single server

Probability Theory Refresher

We write the probability that an event will occur as ℙ (event )

ℙ (X ≤ 0.5) is the probability that a rv X is less than or equal to 0.5.

The joint probability of two events a, b is denoted ℙ (a, b)

The conditional probability of a given that event b has occurred is

ℙ (a|b) =^{△} \frac{ℙ (a, b)}{ℙ (b)}

To describe the distribution from which a rv X is drawn we use 2 functions: the CDF

F_{X}(x) =^{△} ℙ (X ≤ x)

and the pdf

f_{X}(x) =^{△} \frac{dF_{X}(x)}{dx}

with

F_{X}(x) = ∫_{−∞}^{x} f_{X}(y)dy , and ∫_{−∞}^{∞} f_{X}(y)dy = 1.

Thus for a random variable X uniformly distributed from 0 to 1, the CDF and pdf would be The expected value of X, 𝔼 (X) (sometimes called the average, or mean), is just

[figure: F_{X}(x); 0; 1; x; 0; 1; f_{X}(x); 0; 1; x; 0; 1]

𝔼 (X) =^{△} \overline{X} =^{△} ∫_{−∞}^{∞} xf_{X}(x)dx .

Note that 𝔼 (X + Y) = 𝔼 (X) + 𝔼 (Y). Also called the first moment of the distribution. The n^{th} moment is defined as

𝔼 (X^{n}) =^{△} \overline{X^{n}} =^{△} ∫_{−∞}^{∞} x^{n}f_{X}(x)dx ,

and the n^{th} central moment is

\overline{(X − \overline{X})^{n}} = ∫_{−∞}^{∞}(x − \overline{X})^{n}f_{X}(x)dx .

The second central moment is known as the variance,

σ_{X}^{2} =^{△} \overline{(X − \overline{X})^{2}} = \overline{X^{2}} − (\overline{X})^{2}.

The square root of the variance is σ_{X} and is called standard deviation. A dimensionless quantity, the coefficient of variation, is obtained by dividing the standard deviation by the average:

C_{X} =^{△} \frac{σ_{X}}{\overline{X}} .

Similar definitions apply to the case of discrete random variables taking values x_{i} (i ∈ I), say, such that ∑_{i∈I} P [X = x_{i}] = 1. The expected value of X is

𝔼 (X) = _{i∈I}^{∑} x_{i}P [X = x_{i}]

and similarly with the other moments the integration becomes a summation.

The notion of joint distributions is useful. The joint CDF of two random variables X and Y is

F_{XY} (x, y) =^{△} P [X ≤ x, Y ≤ y] ,

and the joint pdf is given by

f_{XY} (x, y) =^{△} \frac{d^{2}F_{XY} (x, y)}{dxdy} .

The random variables X and Y are said to be independent if the the joint pdf factors

f_{XY} (x, y) = f_{X}(x)f_{Y} (y) .

Little’s Result

Begin with a neat derivation of a very general result: Little’s Result

What is the relationship between the (mean) number of customers in the queue, the arrival rate of customers and the time spent in the queue? Let:

α(t) = number of arrivals in (0, t) δ(t) = number of departures in (0, t) Then N(t) = α(t) − δ(t) is the number in the system at t The area γ(t) between the curves α(t) and δ(t) represents the total time all customers have spent in system in (0, t)

Little’s Result

Let

λ_{t} = α(t)/t  (1)
T_{t} = γ(t)/α(t)  (2)
N̄_{t} = γ(t)/t  (3)

• the average arrival rate during (0, t)

• system time per customer averaged over all customers in (0, t)

• average number of customers in system during (0, t)

Little’s Result

Easily from the above

N̄_{t} = λ_{t}T_{t}.

Assume the following limits exist as t → ∞

λ = lim_{t→∞} λ_{t}  (4)
T = lim_{t→∞} T_{t}  (5)

If these exist then so does

N̄ = λT  (6)

In English: the average number in system = average arrival rate × average time in system This is Little’s result. The proof makes no assumptions about A(t), B(t), number of servers, or queueing discipline. First proved by Little in 1961.

Applications of Little’s Result

We can re-state this for any boundary of our queueing system: if we apply this to just the queue then we find that

N̄_{q} = λW

Where N̄_{q} is the average number in queue and W is the average waiting time

Applied to the server only

N̄_{s} = λx̄

Where N̄_{s} is average number in service, and x̄ = 𝔼 (B(x))

Note that T = x̄ + W

Utilization of the Server

For the single server clearly ρ = λx̄ is the average utilization of the server. For a m-server this simply becomes ρ = λx̄/m

ρ is the average number of busy servers, and we require that 0 ≤ ρ < 1 for system stability of G/G/1 queue

Let τ be a large interval. Then we expect with probability = 1 that the number of arrivals ≈ λτ

Let p_{0} = ℙ (server idle at some time ). Then server is busy for τ(1 − p_{0}) seconds and with probability 1 the number of customers served in τ is nearly (τ(1 − p_{0}))/x̄. Thus

λτ ≈ \frac{τ − τp_{0}}{x̄}

and for τ → ∞, λx̄ → (1 − p_{0}) giving

ρ = 1 − p_{0}  (7)

Stochastic Processes

A stochastic process is a function X(t) where t frequently denotes time and whose values are random variables.

{X(t)} → {(w, t)}

where w ∈ S and S is a sample space. The PDF for the process X(t) is given by

F_{X}(x, t) =^{△} ℙ (X(t) ≤ x)

The joint PDF, F_{X}(x, t), ∀t ∈ T is given by

F_{X_{1}X_{2}...X_{n}}(x_{1}x_{2} … x_{n}, t_{1}t_{2} … t_{n}) =

ℙ (X(t_{1}) ≤ x_{1}, X(t_{2}) ≤ x_{2}, …, X(t_{n}) ≤ x_{n})

We classify stochastic processes by 3 parameters:

• the state space (finite, infinite)

• the index parameter (time)

– countable → discrete time process (use X_{n} in place of X(t) )

– infinite → continuous process

• statistical dependencies between values of X(t) for various t ∈ T: examples follow

X(t) is stationary iff X(t) is invariant to time shifts. For any constant τ

F_{X}(x, t + τ) = F_{X}(x, t)

X(t) is wide sense stationary if 𝔼 (X(t)) is independent of t and 𝔼 (X(t)X(t + τ)) depends only on τ

Markov Processes

Stochastic process can have continuous or discrete state space A discrete space Markov process {X_{n} = X_{1}, X_{2}, … is called a Markov chain iff

ℙ (x_{n+1} = X_{n+1} |x_{1} = X_{1}, …, x_{n} = X_{n})  =
ℙ (x_{n+1} = X_{n+1} |x_{n} = X_{n})  (8)

In other words current state depends only on previous state

In continuous time state changes can occur at any time

This, together with (8) places strong conditions on description of time spent in a particular state

We cannot know how long process has been in a given state

Other types of Stochastic Process

• Birth Death Process: MC or MP in which transitions occur only between neighboring states

• Semi-Markov Process: relax memoryless condition for state residence time but Markov property holds at transition instants

• Random Walks: refer to Probability 1a notes

• Renewal Process: X(t) counts state transitions in (0, t)

The Memoryless Property

We show that for a continuous time MP the state residence time is exponentially distributed. For a MC it is geometrically distributed.

Let τ be a rv for the time spent in a state E_{i}. Given τ > s

P [τ > s + t | τ > s] = h(t) .

For some function h(t) not depending on s. Now

P [τ > s + t | τ > s] = \frac{P [τ > s + t, τ > s]}{P [τ > s]}
= \frac{P [τ > s + t]}{P [τ > s]}
= h(t) .

Therefore

P [τ > s + t] = P [τ > s] h(t) .

Now set s = 0, and since P [τ > 0] = 1, we have

P [τ > t] = h(t)

so that

P [τ > s + t] = P [τ > s]P [τ > t] .  (9)

Now the pdf of τ is given by

f_{τ}(t) = \frac{d}{dt}P [τ ≤ t]
= \frac{d}{dt} (1 − P [τ > t])
= −\frac{d}{dt}P [τ > t] .  (10)

Differentiating (9) with respect to s gives,

\frac{d}{ds}P [τ > s + t] = \frac{d}{ds}P [τ > s]P [τ > t]
= −f_{τ}(s)P [τ > t] .

Set s = 0 and divide by P [τ > t] to give,

\frac{dP [τ > t]}{P [τ > t]} = −f_{τ}(0)ds .

Now integrate from 0 to t,

log_{e}P [τ > t] = −f_{τ}(0)t
P [τ > t] = e^{−f_{τ}(0)t} .

From (10) above,

f_{τ}(t) = f_{τ}(0)e^{−f_{τ}(0)t}

It is customary to use λ for f_{τ}(0), giving the familiar exponential distribution

f_{τ}(t) = λe^{−λt} .

The mean of this distribution is

E[τ] = ∫_{0}^{∞} tλe^{−λt}dt
= \frac{1}{λ}

The variance is

E[τ^{2}] = ∫_{0}^{∞} t^{2}λe^{−λt}dt
= \frac{2}{λ^{2}}

and the coefficient of variation is

C_{τ} =^{△} \frac{σ_{τ}}{\overline{τ}}
= 1 .

We have shown that the memoryless distribution is necessarily the exponential distribution.

Exercise show for MC that the memoryless property requires the geometric distribution.

Note that the parameter λ can be interpreted as the rate of state transition.

The Poisson Process

• The memoryless arrivals (or inter-event) process is the well known Poisson process with parameter λ

• It can be defined as the dual to the exponential distribution: let o(Δt) be a quantity having an order of magnitude less than Δt

• Suppose ∃λ constant such that for any small time Δt,

ℙ (0 arrivals in t, t + Δt) = 1 − λΔt + o(Δt)

• events in the interval (t, t + Δt) are independent and

ℙ (1arrival in t, t + Δt) = λΔt + o(Δt)

• Let

A_{c}(t) = ℙ (time of next arrival, x > t_{0} + t)

Compute the inter-arrival time distribution A(t) between two events thus:

A_{c}(t + Δt) = ℙ (z > t + Δt) where z = x − t_{0}

or

A_{c}(t + Δt)  = ℙ (z > t) ℙ ( no arrivals in Δt)
= A_{c}(t)[1 − λΔt + o(Δt)]

using the assumption of independent events in disjoint intervals.

Rearranging etc.

lim_{Δt→0} \frac{A_{c}(t + Δt) − A_{c}(t)}{Δt} = −λA_{c}(t)

Thus

A′_{c}(t) = −λA_{c}(t)

Subject to boundary condition A_{c}(0) = 1 we have the following solution

A_{c}(t) = e^{−λt}, t ≥ 0

And hence

A(t) = 1 − e^{−λt}

Demonstrating that Poisson arrivals → exponential inter-arrivals.

How many arrivals in t?

• Indicate briefly the pmf f_{i}(t)(i = 0, 1, 2...) which specifies the probability of i arrivals in t

• Note that this probability is independent of where the interval begins

• Set t = mΔt

• Find probability of 1 arrival in i of the m intervals of length Δt and no arrival in the remaining m − i

f_{i}(t) = lim_{Δt→0} \binom{m}{i} [λΔt + o(Δt)] ^{i}
[1 − λΔt + o(Δt)] ^{m−i}

where m = \frac{t}{Δt} This can be reduced to

f_{i}(t) = \frac{(λt)^{i}}{i!} lim_{Δm→∞} \frac{m(m − 1) … (m − i + 1)}{m^{i}}
lim_{Δm→∞} [1 − \frac{λt}{m}]^{m}

From which we obtain

f_{i}(t) = \frac{(λt)^{i}}{i!} e^{−λt}  (11)

Equation (11) defines the Poisson distribution. Note that the inter-arrival time distribution is given by A(t) = 1 − f_{0}(t).

It is easily shown from Equation (11) that

f_{i}(Δt) = \{ \begin{matrix}1 − λΔt + o(Δt) & i = 0 \\ λΔt + o(Δt) & i = 1 \\ o(Δt) & i > 1 \end{matrix}

demonstrating the consistency of the definitions for the Poisson Process.

Birth Death Processes

The BDP is a special case of a CTMC in which we allow transitions only between neighboring states

The state space is the set of non-negative integers and we allow only a single birth or death per transition. Thus given X_{n} = i

X_{n+1} = \{ \begin{matrix}i + 1 & birth (or arrival) \\ i − 1 & death (or departure) \end{matrix}

We use λ_{i} to represent the birth rate in state i, and μ_{i} to represent the death rate in state n

[figure: 0; λ_{0}; μ_{1}; 1; λ_{1}; μ_{2}; 2; λ_{2}; μ_{3}; ...; λ_{i−1}; μ_{i}; i; λ_{i}; μ_{i+1}; i+1; λ_{i+1}; μ_{i+2}; ...]

Time Dependent Solution of BDP

We denote by P_{i}(t) the probability of being in state i at time t

Probability of a birth in an interval Δt when the system starts in state i is assumed to be λ_{i}Δt + o(Δt)

Probability of a death in Δt when the system starts in state i is μ_{i}Δt + o(Δt)

Probability of > 1 event in Δt is o(Δt)

o(Δt) denotes a quantity which becomes negligible when compared with Δt as Δt → 0

To solve for P_{i}(t) we write a set of difference equations called the Chapman Kolmogorov equations

Chapman Kolmogorov Equations

For i ≥ 1

P_{i}(t + Δt)  = P_{i}(t)(1 − λ_{i}Δt)(1 − μ_{i}Δt)
+ P_{i+1} (t)(μ_{i+1} Δt)(1 − λ_{n+1} Δt)
+ P_{i−1}(t)(λ_{i−1}Δt)(1 − μ_{i−1}Δt)
+ o(Δt)

For i = 0

P_{0}(t + Δt)  = P_{0}(t)(1 − λ_{0}Δt)
+ P_{1}(t)(μ_{1}Δt)(1 − λ_{1}Δt)
+ o(Δt)

We derive differential-difference equations from these by dividing through by Δt and taking the limit as Δt → 0:

\frac{dP_{i}(t)}{dt} = −(λ_{i} + μ_{i})P_{i}(t)
+ μ_{i+1} P_{i+1} (t)
+ λ_{i−1}P_{i−1}(t)

for i ≥ 1 and

\frac{dP_{0}(t)}{dt} = −λ_{0}P_{0}(t)
+ μ_{1}P_{1}(t)  (12)

The time dependent solution is extremely difficult for systems of interest, so we will study the stationary solution

Stationary Solution

We are interested in the long term probabilities after the system has reached an equilibrium.

These probabilities are independent of the initial conditions.

System reaches equilibrium if, for all i,

lim_{t→∞} P_{i}(t) = p_{i} exists .

The transitions into state i are births in state i − 1 and deaths in state i + 1.

Now dP_{i}(t)/dt is 0 and Equations (12) become

0 = −(λ_{i} + μ_{i})p_{i} + μ_{i+1} p_{i+1} + λ_{i−1}p_{i−1}
0 = −λ_{0}p_{0} + μ_{1}p_{1}  (13)

Rewriting gives

p_{i+1} = \frac{λ_{i} + μ_{i}}{μ_{i+1}} p_{i} − \frac{λ_{i−1}}{μ_{i+1}} p_{i−1}
p_{1} = \frac{λ_{0}}{μ_{1}}p_{0}  (14)

For i ≥ 1.

These equations can be solved iteratively (using induction). Proof is left as an exercise

Stochastic Balance

Provides short cut method to obtaining Equation (14) Require total flow into a state to equal total flow out of a state under steady-state conditions Intuitively the total flow is the product of the steady state probabilities and the flow rates (λ, μ) We enter state i at rate p_{i−1}λ_{i−1} + p_{i+1} μ_{i+1} . We exit state i at rate p_{i}λ_{i} + p_{i}μ_{i}.

The equation p_{i−1}λ_{i−1} + p_{i+1} μ_{i+1} = p_{i}λ_{i} + p_{i}μ_{i} for i > 0 . describes flow into state i. This is the so-called global balance equation the two equations:

p_{i}λ_{i} = p_{i+1} μ_{i+1} for i ≥ 0

describing flow from state i to state i + 1, and

p_{i}μ_{i} = p_{i−1}λ_{i−1}

describing flow from state i to state i − 1 are the detailed balance equations Rewriting gives

p_{i+1} = \frac{λ_{i}}{μ_{i+1}} p_{i}

which gives rise to the product solution

p_{k} = p_{0} k−1 ; ∏ ; i=0 \frac{λ_{i}}{μ_{i+1}} for k ≥ 0

Since the sum of state probabilities must be unity,

p_{0} + ∞ ; ∑ ; k=1 p_{k} = 1
p_{0} + ∞ ; ∑ ; k=1 p_{0} k−1 ; ∏ ; i=0 \frac{λ_{i}}{μ_{i+1}} = 1
p_{0} [1 + ∞ ; ∑ ; k=1 k−1 ; ∏ ; i=0 \frac{λ_{i}}{μ_{i+1}} ] = 1

so that

p_{0} = [1 + ∞ ; ∑ ; k=1 k−1 ; ∏ ; i=0 \frac{λ_{i}}{μ_{i+1}} ] ^{−1}
p_{k} = p_{0} k−1 ; ∏ ; i=0 \frac{λ_{i}}{μ_{i+1}}

which are known as the general flow balance equations. It may be shown that an equilibrium distribution p_{k} always exists if \frac{λ_{k}}{μ_{k}} < 1 for all k > K for some fixed K.

The M/M/1 Queue

The BDP maps well onto our domain of study – queueing systems

Births represent arrivals to queue, deaths represent departures as customers finish service

The M/M/1 queue is an infinite customer system, with infinite waiting room, and a state independent service rate

This means that λ_{i} = λ and μ_{i} = μ for all i and we can solve the balance equations:

p_{k} = p_{0} k−1 ; ∏ ; i=0 \frac{λ}{μ}
= p_{0} (\frac{λ}{μ})^{k} .

Writing ρ = \frac{λ}{μ}

p_{0} = \frac{1}{1 + ∑_{k=1}^{∞} ρ^{k}}
= \frac{1}{1 + ρ ∑_{k=0}^{∞} ρ^{k}}
= \frac{1}{1 + ρ (\frac{1}{1−ρ})}
= 1 − ρ

Recall that these results are the same as we derived in the first lecture for the G/G/1 queue: p_{0} is the probability that there is no customer in the system and 1 − p_{0} = ρ is the utilization of the server

Performance measures

The number in the system is geometrically distributed

p_{k} = (1 − ρ)ρ^{k}, k = 0, 1, 2, …

If ρ > 1, i.e. if λ > μ the system will not reach equilibrium

What is the average number of customers, \overline{N} in the system?

\overline{N} = ∞ ; ∑ ; k=0 kp_{k}
= ∞ ; ∑ ; k=0 k(1 − ρ)ρ^{k}

= (1 − ρ)ρ\frac{∂}{∂ρ} ( ∞ ; ∑ ; k=0 ρ^{k})
= (1 − ρ)ρ\frac{∂}{∂ρ} (\frac{1}{1 − ρ} )
= \frac{ρ}{1 − ρ}

An arriving customer will find, on average \overline{N} in the system, and will spend a time, say \overline{T}, in the system. During \overline{T} there will be, on average λ\overline{T} arrivals, leaving \overline{N} customers in the queue. Thus

\overline{N} = λ\overline{T}

which is Little’s result restated. In our case

\overline{T} = \frac{\overline{N}}{λ}
= \frac{\frac{1}{μ}}{1 − ρ}

which is the M/M/1 average response time.

Note that

• μ is the service rate

• \frac{1}{μ} is the average service time

• ρ is the utilization

Performance at high load

At high utilizations ρ approaches one and the response time is unbounded.

Response time

Consider the effect on response time by increasing the utilization by a constant load of 0.1.

utilization (ρ)  response time (\overline{T})
old new % ↑ old new % ↑
0.1 0.2 100.0 1.11 \frac{1}{μ} 1.25 \frac{1}{μ}  13
0.5 0.6 20.0  2 \frac{1}{μ} 2.5 \frac{1}{μ}  25
0.8 0.9 12.5  5 \frac{1}{μ}  10 \frac{1}{μ} 100

Predicting response times is very difficult at high loads.

Running systems at maximum utilization may please the providers, but it doesn’t please the users.

M/M/1 – an example

H.R Cutt barber shop, no appointment needed!

Customers served FCFS

On Saturday mornings he is very busy and is considering hiring a helper

He measures the (Poisson) arrival rate of customers to be 5 per hour

Customers are prepared to wait, and he spends on average 10 min per cut

What are the average number of customers in the shop, the average number of customers waiting? What percentage of time can a customer receive a cut without waiting?

He has 4 seats in his waiting room. What is the probability that an arriving customer will find no seat and have to stand?

Example Solution

The average number of customers in the shop: λ = 5 per hour, μ = 6 per hour So

ρ = 5/6, and hence N̄ = 5

Average number of customers waiting:

N̄_{q} = 4\frac{1}{6}

What percentage of time can a customer receive a cut without waiting?

p_{0} = 1 − ρ = \frac{1}{6}

No seat

ℙ (no seat ) = ℙ (N ≥ 5)
= ρ^{5}
≈ 0.402

M/M/m — m servers

This is just like the M/M/1 system, except that there are m servers

For all k, λ_{k} = λ, but now the service rate is a function of the number of customers in the system

μ_{k} = \{ \begin{matrix}kμ & if 0 < k ≤ m \\ mμ & if k ≥ m \end{matrix}

For an equilibrium distribution to exist we require that \frac{λ}{mμ} < 1.

[figure: 0; λ; μ; 1; λ; 2μ; 2; λ; 3μ; 3; λ; 4μ; ...; λ; (m − 1)μ; m−1; λ; mμ; m; λ; mμ]

M/M/m Performance Measures

Like the M/M/1 system, except that there are m servers As before, for all k, λ_{k} = λ, but

μ_{k} = \{ \begin{matrix}kμ & if 0 < k ≤ m \\ mμ & if k ≥ m \end{matrix}

We require that \frac{λ}{mμ} < 1. The difference equations are given by:

0 = μp_{1} − λp_{0}
0 = (k + 1)μp_{k+1} − (λ + kμ)p_{k} + λp_{k−1} if 0 < k ≤ m
0 = mμp_{k+1} − (λ + mμ)p_{k} + λp_{k−1} if k ≥ m

From which we may easily solve iteratively for the p_{k} as follows:

p_{1} = λ/μp_{0}
p_{2} = 1/2(λ/μ)^{2}p_{0}
⋮
p_{m} = 1/m!(λ/μ)^{m}p_{0}
p_{m+1} = 1/(mm!)(λ/μ)^{m+1} p_{0}
⋮

This gives us the general expression for p_{k}:

p_{k} = (λ/μ)^{k}\frac{p_{0}}{k!} if k ≤ m
= (λ/μ)^{k}\frac{p_{0}}{m!m^{k−m}} if k ≥ m

We also require that ∑_{k=0}^{∞} p_{k} = 1, we may solve for p_{0}:

p_{0} = [m−1 ; ∑ ; k=0 (\frac{λ}{μ})^{k} \frac{1}{k!} + \frac{m^{m}}{m!} ∞ ; ∑ ; k=m (\frac{λ}{mμ})^{k}] ^{−1}

= [m−1 ; ∑ ; k=0 \frac{(mρ)^{k}}{k!} + \frac{(mρ)^{m}}{(1 − ρ)m!} ] ^{−1} ,

making use of the fact that ρ = \frac{λ}{mμ} < 1, whence the second term in the equation for p_{0} converges:

∞ ; ∑ ; k=m ρ^{k} = ∞ ; ∑ ; k=0 ρ^{k} − m−1 ; ∑ ; k=0 ρ^{k}

= \frac{1}{1 − ρ} − \frac{1 − ρ^{m}}{1 − ρ}

= \frac{ρ^{m}}{1 − ρ}

Easiest to start with the derivation of N̄_{q}

N̄_{q} = ∞ ; ∑ ; n=m+1 (n − m)p_{n}

= ∞ ; ∑ ; n=m+1 \frac{(n − m)m^{m}ρ^{n}p_{0}}{m!}

= \frac{(mρ)^{m}p_{0}}{m!} ∞ ; ∑ ; n=m+1 (n − m)ρ^{n−m}

= \frac{(mρ)^{m}p_{0}}{m!} ρ ∞ ; ∑ ; k=1 kρ^{k−1}

= \frac{(mρ)^{m}p_{0}}{m!} ρ\frac{∂}{∂ρ} (\frac{1}{1 − ρ} )

= \frac{(mρ)^{m}p_{0}ρ}{m!(1 − ρ)^{2}}

= \frac{ρp_{m}}{(1 − ρ)^{2}}

With N̄_{q}, we can use Little’s formula to solve: N̄_{q} = T̄_{q}λ, and thence obtain performance measures for the queue using T̄ = T̄_{q} + \frac{1}{μ}.

M/M/1/K — finite capacity

If there are K customers in the system, any further arrivals are lost. Now for k ≥ K, λ_{k} = 0 and for k > K, p_{k} = 0 Using the equations from the M/M/1 queueing system, but limiting the summation, and again writing ρ = \frac{λ}{μ},

p_{k} = p_{0}ρ^{k} for k ≤ K

p_{0} = \frac{1}{1 + ∑_{k=1}^{K} ρ^{k}}
= \frac{1}{1 + \frac{ρ−ρ^{K+1}}{1−ρ}}
= \frac{1 − ρ}{1 − ρ^{K+1}}

Note that p_{0} is greater than in the M/M/1 case.

For this system with a finite state space an equilibrium distribution always exists whatever the arrival and departure rates.

M/M/1//N - finite population

Single server, unbounded queue capacity and a population of N customers.

We solve this system by modifying the λ_{k} to model the arrival rate.

Instead of having an arrival rate for the population as a whole, we assign an arrival rate to each customer, say λ.

If there are no customers in the system, then all of them are eligible to be born, so that

λ_{0} = Nλ .

As we have more customers in the system, we have less eligible to be born. So that,

λ_{k} = (N − k)λ, for 0 ≤ k ≤ N . With a single server the service rate is constant

μ_{k} = μ, for k ≥ 1 .

M/M/1//N – finite population

If there are no customers in the system, then all of them are eligible to be born, so that

λ_{0} = Nλ .

As we have more customers in the system, we have less eligible to be born. So that,

λ_{k} = (N − k)λ, for 0 ≤ k ≤ N .

With a single server the service rate is constant

μ_{k} = μ, for k ≥ 1 .

Solving the difference equations for this queue we get

p_{k} = p_{0} k−1 ; ∏ ; i=0 \frac{λ(N − i)}{μ} for 0 ≤ k ≤ N

= \{ \begin{matrix}p_{0}ρ^{k}\frac{N!}{(N − k)!} & if 0 < k ≤ N \\ 0 & if k > N \end{matrix}

and

p_{0} = [ N ; ∑ ; k=0 ρ^{k}\frac{N!}{(N − k)!} ]^{−1}

M/M/m/m - m server loss system

An application of this system is to model a link in a telephone network.

Such a link contains m circuits each of which carries a single call.

Suppose that calls arrive at the link according to a Poisson process of rate λ.

If there is a free circuit the call is connected and holds the circuit for an exponentially distributed length of time, with mean \frac{1}{μ}.

At the end of this holding period the call terminates and the circuit is freed.

If there are no free circuits then the call is lost.

The birth and death rates are

λ_{k} = \{ \begin{matrix}λ & k < m \\ 0 & k ≥ m \end{matrix}

and

μ_{k} = kμ for 1 ≤ k ≤ m .

[figure: 0; λ; μ; 1; λ; 2μ; 2; λ; 3μ; 3; λ; 4μ; ...; λ; (m − 1)μ; m−1; λ; mμ; m]

The flow balance equations give for k ≤ m

p_{k} = p_{0} k−1 ; ∏ ; i=0 \frac{λ_{i}}{μ_{i+1}}
= p_{0} k−1 ; ∏ ; i=0 \frac{λ}{(i + 1)μ}
= p_{0} (\frac{λ}{μ})^{k} \frac{1}{k!}

and p_{k} = 0 for k > m.

Solving for p_{0} gives

m ; ∑ ; k=0 p_{k} = 1
p_{0} m ; ∑ ; k=0 (\frac{λ}{μ})^{k} \frac{1}{k!} = 1

so that

p_{0} = [ m ; ∑ ; k=0 \frac{λ}{μ}^{k}\frac{1}{k!} ]^{−1} .

The probability that an arriving call finds all circuits occupied, p_{m}, is called the loss probability for the telephone link. Thus,

p_{m} = p_{0} (\frac{λ}{μ})^{m} \frac{1}{m!}
= (\frac{λ}{μ})^{m} \frac{1}{m!} [ m ; ∑ ; k=0 (\frac{λ}{μ})^{k} \frac{1}{k!} ]^{−1} .

This expression for the loss probability is known as Erlang’s B formula

Extensions to Birth Death Models

First we relax our constraints on the arrival process distribution.

We want to model systems in which the coefficient of variation of the interarrival time is less than one.

Consider a system in which a birth occurs in two stages.

Each stage has an exponentially distributed residence time.

[figure: 2λ; 2λ]

If the desired birth rate is λ, then let each stage have a rate 2λ.

The average time to get through the combined birth process will be

\overline{τ} = \frac{1}{2λ} + \frac{1}{2λ} = \frac{1}{λ} .

Since each stage has exponentially distributed residence times, the variance of each stage is

σ_{single}^{2} = \frac{1}{(2λ)^{2}} .

The two stages are independent, so the variance of τ, the time to get through both stages is

σ_{τ}^{2} = \frac{1}{(2λ)^{2}} + \frac{1}{(2λ)^{2}} = \frac{1}{2λ^{2}}

So the coefficient of variation is

C_{τ} = ^{\frac{√\frac{1}{2λ^{2}}}{\frac{1}{λ}}} = \frac{1}{√2} .

In general, if we use r stages each with rate rλ we get an average time through all stages of \frac{1}{λ} and a coefficient of variation of \frac{1}{√r}.

The distribution that describes this process is the Erlangian distribution denoted E_{r}.

Example, M/E_{2}/1

Allow the state of the process to represent the number of stages remaining to be served.

An arrival increases the number of stages remaining to be served by 2 and occurs at rate λ

A departure from a stage reduces the number of stages to be served by 1 and occurs at rate 2μ.

[figure: 0; λ; 2μ; 1; λ; 2μ; 2; 2μ; 3; λ; 2μ; ...]

Parallel Servers

Combining stages in series reduces the coefficient of variation.

If, instead, we combine them in parallel with a probability α_{i} of choosing the i^{th} parallel stage we get a service distribution with coefficient of variation larger than 1.

The coefficient of variation is given by

C_{τ} = \frac{2 ∑_{i=1}^{r} _{\frac{α_{i}}{μ_{i}^{2}}}}{(∑{r \atop i=1} \frac{α_{i}}{μ_{i}})2} − 1 ≥ 1 .

[figure: μ_{1}; μ_{2}; α_{1}; α_{2}]

Queueing Networks

We have seen the solution of several queueing systems of interest

In general we want to solve a system of such queues representing a real world performance problem e.g. a distributed computing system

We represent the system under study as a network of connected queues

Customers move (instantaneously) between service centres where they receive service

[figure: 1; 2; 3; 4; 5]

Queueing Networks

Depending on whether we allow external arrivals and departures, the queueing network is either open or closed

An open network is called feed-forward if no customer can visit the same node twice

It is of key importance that we understand the output process from each queue – this forms the input to the next queue

In general the inter-departure time distribution from a queue is different from the service time distribution

The point process represented by departures is in general very complex, but a special case exists...

The Output of an M/M/m Queue

We have studied this queue in some detail – in particular we know that the utilization of the server is

ρ = \frac{λ}{mμ}

Focus on the departure process during a small interval Δt, and compute the expected number of departures n:

• If the system is in state 1, the departure rate is μ → n = μΔt

• if the system is in state 1 < k ≤ m → n = kμΔt

• if the system is in state m ≤ k, → n = mμΔt

Burke’s Theorem

Averaging over the different states and their probabilities

n = μΔt ( m ; ∑ ; k=1 kp_{k} + _{k>m}^{∑} mp_{k})
= mμρΔt
= λΔt

Further, because the service distribution is exponential, events in different intervals are independent

We thus conclude that the output process from an M/M/m queue is Poisson with rate λ

Consequences

Burke’s Theorem implies that the input to each node in a feed-forward network is Poisson

This means that each queue can be solved in isolation

Although in general the number of customers at each node is independent, the waiting times of customers T̄_{q_{i}} are not

In spite of this, mean quantities may be obtained for the network by combining the individual means

Finally, terminal nodes may have general service time distributions

Open Queueing Networks

Allow customers to arrive from an external source, with rate γ

Customers leave the network on completion

Assume we have N nodes, each a M/M/m queue (our examples will use M/M/1) with infinite waiting room

Each server i has exponential service time with mean 1/μ_{i}, except possibly the terminal nodes

Example

Customers arrive (from outside) as a Poisson stream at node i at rate γ_{i} Suppose that the service rates of all queues are exponential μ_{i}, 1 ≤ i ≤ 5

The split between storage devices is equal

Determine the average response time for a message through the system From our earlier results, the mean response times at nodes 1 and 2 are

T̄_{1} = \frac{1}{(μ_{1} − γ_{1})}
T̄_{2} = \frac{1}{(μ_{2} − γ_{2})}

Similarly, given that the combined output from nodes 1 and 2 is Poisson with rate γ_{1} + γ_{2},

T̄_{3} = \frac{1}{(μ_{3} − (γ_{1} + γ_{2}))}

Finally, the inputs to nodes 4 and 5 is Poisson at rate 0.5(γ_{1} + γ_{2}) thus

T̄_{4,5} = \frac{1}{(μ_{4,5} − 0.5(γ_{1} + γ_{2}))}

Hence the mean response times for messages entering at nodes 1 and 2 respectively are

T̄′ = T̄_{1} + T̄_{3} + 0.5(T̄_{4} + T̄_{5})
T̄′′ = T̄_{2} + T̄_{3} + 0.5(T̄_{4} + T̄_{5})

Since the proportion of traffic entering at node i = 1, 2 is

\frac{γ_{i}}{(γ_{1} + γ_{2})}

The required result is

T̄ = \frac{(γ_{1} T̄′ + γ_{2} T̄′′)}{(γ_{1} + γ_{2})}

General Topologies

We might be tempted to think that a more general topology in which feedback is allowed will also have the convenient Poisson arrival process at each node

Unfortunately this is not the case – feedback introduces dependence on the past

Consider the case when p ≈ 1 and μ_{1} is high

The memoryless property is violated

Luckily, it can be shown that each node does behave as though it were subject to a Poisson input

Jackson Networks

Define a more general topology network, called a Jackson Network:

A customer completing at node i moves to node j with probability q_{ij} for (i, j = 1, 2, …, N)

Where

N ; ∑ ; j=1 q_{ij} ≤ 1

A job leaves the network from node i with probability

q_{i0} = 1 − N ; ∑ ; j=1 q_{ij} .

The probabilities q_{ij} are called the routing probabilities Written as an N × N matrix this is called the routing matrix Q = (q_{ij}) The system state is (k_{1}, k_{2}, …, k_{N}), where k_{i} is the number of jobs present at node i

Jackson’s Theorem

Essentially: In a Jackson Network each node behaves as if it were independent with Poisson input as far as the following are concerned:

• the average number of customers at the node, and

• the average arrival rate to each node j (which is the sum of the external and internal arrival rates)

Let λ_{j} be the average number of arrivals to node j per unit time

On average the departure count per unit time is therefore λ_{j}

A function q_{ji} go to node i The rate of traffic from node j to node i is thus λ_{j}q_{ji}

[figure: i; γ_{i}; λ_{1}q_{1i}; λ_{2}q_{2i}; λ_{N}q_{Ni}; λ_{i}q_{i0}; λ_{i}q_{i1}; λ_{i}q_{i2}; λ_{i}q_{iN}; ⋮; ⋮; λ_{i}; λ_{i}]

Solution of a Jackson Network

Adding together all contributions,

λ_{i} = γ_{i} + N ; ∑ ; j=1 λ_{j}q_{ji} i = 1, 2, …, N .

These are known as the traffic equations

A necessary and sufficient condition for the existence of an equilibrium distribution of customers in the network is that

ρ_{i} =^{△} \frac{λ_{i}}{μ_{i}} < 1

where λ_{i} is the solution of the traffic equations.

Let p(k_{1}, k_{2}, …, k_{N}) be the steady state equilibrium distribution. Then Jackson’s theorem states that

p(k_{1}, k_{2}, …, k_{N}) = p_{1}(k_{1})p_{2}(k_{2}) ⋯ p_{N}(k_{N})

where p_{i}(k_{i}) is the equilibrium distribution that there are k_{i} jobs in an M/M/m queue with traffic intensity ρ_{i}.

Implications

Jackson’s theorem has some important implications.

1. The numbers of jobs at the various nodes are independent.

2. Node i behaves as if subjected to a Poisson stream with rate λ_{i}.

Jackson’s theorem applies equally to the case where node i has n_{i} servers and so the nodes behave as independent M/M/n_{i} queues

Example – A Router of Sorts

Similar model – except that messages with errors are returned to the first node for correction, with probability p

Each of the 3 transmission lines has a different transmission rate μ_{i} and message sizes are exponentially distributed

Determine

• the average total delay for a packet

• the average total number of messages awaiting transmission

Solution

First determine input into each node using the traffic equations:

λ_{1} = γ_{1} + pλ_{2}
λ_{2} = λ_{1}
λ_{3} = λ_{2}(1 − p)/3
λ_{4} = λ_{2}(1 − p)/3
λ_{5} = λ_{2}(1 − p)/3

In other words

λ_{1} = λ_{2} = \frac{γ}{1 − p}
λ_{3} = λ_{4} = λ_{5} = \frac{γ}{3}

With the (obvious) check that

γ = 5 ; ∑ ; i=3 λ_{i}

From these we can calculate the response times

T̄_{1} = \frac{1}{(μ_{1} − γ/(1 − p))}

T̄_{2} = \frac{1}{(μ_{2} − γ/(1 − p))}

T̄_{i} = \frac{1}{(μ_{i} − γ/3)}

for i = 3, 4, 5

The number of times a message has to join the switching centre queue is geometrically distributed with

ℙ (message retransmitted n times ) = p^{n}(1−p) and with mean 1/(1 − p)

Thus the mean total delay through nodes 1 and 2 is

\frac{T̄_{1} + T̄_{2}}{1 − p}

The remaining average delay is ( 5 ; ∑ ; i=3 T̄_{i})/3

Thus the mean total delay which we wanted is

T̄ = \frac{T̄_{1} + T̄_{2}}{1 − p} + \frac{∑_{i=3}^{5} T̄_{i}}{3}

Next, the mean number of messages in queue i is, by Little’s law

N̄_{i} = λ_{i}T̄_{i}

Giving our final solution for the output queues of

5 ; ∑ ; i=3 N̄_{i} = \frac{γ}{3(1 − p)} 5 ; ∑ ; i=3 T̄_{i}

Closed Queueing Networks

Frequently used to model systems at high load or where a limited, constant number of jobs is admitted to service

No external arrivals or departures

Now the routing probabilities satisfy

N ; ∑ ; j=1 q_{ij} = 1 , i = 1, 2, …, N

The number of jobs in the system is always a constant, denoted by K

The states of the system, described by the vector (k_{1}, k_{2}, …, k_{N}), thus satisfy the constraint,

N ; ∑ ; i=1 k_{i} = K

The state space, S, is then finite. The number of states is

( \begin{matrix}K + N − 1 \\ N − 1 \end{matrix} )

The traffic equations become

λ_{i} = N ; ∑ ; j=1 λ_{j}q_{ji} , i = 1, 2, …, N

With a finite state space there always exists an equilibrium distribution, which is the solution to

p(k_{1}, k_{2}, …, k_{N}) N ; ∑ ; i=1 δ_{k_{i}−1}α_{i}(k_{i})μ_{i} =

N ; ∑ ; i=1 N ; ∑ ; j=1 δ_{k_{j}−1}α_{i}(k_{i}+1)μ_{i}q_{ij}p(…, k_{j}−1, .., k_{i}+1, …)

where δ_{k} is a unit step with value 1 if k ≥ 0 and 0 elsewhere and α_{i}(k_{i}) is the number of servers at node i which are busy when k_{i} customers are present

Analogous to Jackson’s theorem for the open network case it may be shown that

p(k_{1}, k_{2}, …, k_{N}) = \frac{1}{G}r_{1}(k_{1})r_{2}(k_{2}) ⋯ r_{N}(k_{N})

where r_{i}(k_{i}) is the probability that there are k_{i} jobs in an M/M/1 queue with traffic intensity given by a solution to the traffic equations

Open vs closed networks

The normalization constant G has to be determined by

G = _{s∈S}^{∑} r_{1}(k_{1})r_{2}(k_{2}) ⋯ r_{N}(k_{N})

obtained by summing over all states s = (k_{1}, k_{2}, …, k_{n}) ∈ S.

With closed networks need to compute the normalization constant G - a computationally hard problem

The constraint ∑_{i=1}^{N} k_{i} = K means that the numbers of jobs in the various queues are no longer independent.

For instance, consider the extreme case where all K jobs are at one node

The M/G/1 Queue

Analysis of the M/M/1 queue was made possible by the assumption of exponential interarrival and service times

The process X(t) describing the number in the system has the Markov property ie.

At t_{0} the future behavior of X(t) depends only on X(t_{0})

Can we honestly convince ourselves that the transmission time for IP traffic at a router conforms to an exponential distribution?

Even the Markovian M/H_{2}/1 queue might be unable to model the service process in this case.

The M/G/1 Queue

In the M/G/1 queue we retain the Poisson assumption for the arrivals process

We are interested in the analysis of X(t) assuming general service time distributions.

Now, if at t_{0} a job is in execution, then the behavior of X(t), t > 0, will depend not only on X(t_{0}) but on the amount of service already received.

Thus X(t) is no longer a Markov process.

A solution

Redefine X(t) so that its state space includes the values of a supplementary variable x that specifies the amount of service already received by the job in execution at t. The new process X′(t) is a pair (n, x), where

• n is the number in the system at time t • x is a continuous parameter expressing

the service time of the customer in service The new process now has the Markov property But the analysis is extremely difficult

A second approach

Key to solution is the recognition of the fact that although X(t) does not have the Markov property at every point in time …

there exist many embedded sequences of time points at which X(t) does have the Markov property.

Define the time instants (or epochs) t_{1}, t_{2}, … so that t_{i} is the instant immediately after the i^{th}job execution

Since the service and interarrival intervals are mutually independent, X(t_{i}) at the epochs t_{i} has the Markov property.

The discrete time process

X(t_{i}), i = 1, 2 …,

constitutes a Markov chain which is said to be an embedded Markov chain with respect to X(t).

The solution of the M/G/1 model is difficult

We simply present an intuitive derivation

M/G/1 Performance Measures

Let s̄ denote the mean service time

Begin with the queueing time of a customer before it receives service, which we all know is given by

T̄_{q} = N̄_{q}s̄ + x̄,

where x̄ is the remaining service time of the customer in service at the arrival of the given customer.

We also know from Little’s law that

N̄_{q} = λT̄_{q}

Substituting and solving for T̄_{q} gives

T̄_{q} = \frac{x̄}{(1 − λs̄)}

It turns out (using arguments from renewal theory) that

x̄ = \frac{1}{2}λs̄^{2}

Thus

T̄_{q} = \frac{λs̄^{2}}{2(1 − λs̄)}

= \frac{λs̄^{2}}{2(1 − ρ)}

where ρ = λs̄.

Writing C_{s} for the coefficient of variation of the service time gives us our solution:

T̄_{q} = \frac{s̄ρ(1 + C_{s}^{2})}{2(1 − ρ)}  (15)

Pollaczek-Khintchine Formula

From this we may derive directly the formula for T̄

T̄ = s̄ [1 + \frac{ρ(1 + C_{s}^{2})}{2(1 − ρ)} ]  (16)

This is the Pollaczek-Khintchine or PK formula.

From Little’s formula, equations for the other performance measures follow directly.

For example, the expected queue length is given by:

N̄ = ρ + \frac{ρ^{2}[1 + C_{s}^{2}]}{2(1 − ρ)}  (17)

Some Comments …

Equation (17) is the mean number of customers that are left in the system by a departing customer.

Since we observe the system only at departure times, during the service time of the departing customer more than one new arrival may have occured.

You might like to construct the Markov chain for this model and obtain our earlier results for the M/M/1 queue.

Note particularly that N̄ is determined not only by the mean interarrival and service times but also by s̄^{2}.

Also, N̄ increases linearly with s̄^{2} Considering constant service times, for example

N̄ = ρ + \frac{ρ^{2}}{2(1 − ρ)} = \frac{ρ(1 − ρ/2)}{(1 − ρ)} ,  (18)

compared with the expression ρ/(1 − ρ) for exponential service times

For small ρ the difference in the means is small, but as ρ → 1 the two differ by a factor of 2

M/G/1 queue length

What values of C_{s} = σ_{s}/s̄ is one likely to encounter?

• 0: This is the case of constant service times

• < 1: This ratio is better than the exponential case

⇒ M/M/1 performance measures are conservative

• ratio ≈ 1: This corresponds to exponential service time. e.g. packet switched networks with variable size packets, batch customers on a mainframe etc.

• ratio > 1: You need to use an M/G/1 model. e.g. IP traffic

M/G/1 queue length

96

And non-Poisson arrivals ?

The same considerations apply to the arrival rate

• For Poisson arrivals the ratio C_{a} ≈ 1

• If the observed ratio is < 1, arrivals tend to the evenly spaced

• If the ratio is > 1, arrivals tend to cluster

For the same λ, the performance of any queueing system in which arrivals are completely random will be worse than a system in which arrivals are completely predictable.

Leung gives the following rules of thumb

• 1 ≤ C_{a} ≤ 1.15 – Poisson is a reasonable approximation

• 0.85 ≤ C_{a} – Process becomes more constant

• C_{a} ≥ 1.15 – Arrivals tend to be bunched. The Poisson assumption over-estimates congestion for processes with C_{a} ≤ 1, and under-estimates for C_{a} > 1.

Example

An IP service provider has a bank of dial-in modems Each recieves messages according to a Poisson distribution with rate λ_{j} = 10j packets per second, for j = 1 … 5 Messages are multiplexed onto a single output link and transmitted in arrival order. Calculate

• the mean datagram response time

• the mean number in the system Given IP packets have a bimodal size distribution which gives them a mean service time of 5ms with a variance of 4ms

Solution

We recall that the sum of independent Poisson processes is again Poisson We also immediately recognize that the M/M/1 performance measures, were we to use them, would be conservative Here we get λ = ∑_{i=1}^{5} 10 × i = 150 messages per second Noticing that service times were given in ms, we convert to seconds thus 1/s̄ = 200 messages/second Immediately we see that ρ = λs̄ = 0.75 Similarly, C_{s}^{2} = 4/25 = 0.16 Using the PK formula for T̄ gives,

T̄ = 5 [1 + \frac{0.75 × (1 + 0.16)}{0.5} ]
= 13.7ms
N̄ = 0.75 + \frac{0.75^{2} × (1 + 0.16)}{0.5}
= 2.0551
N̄_{q} = 1.011 − 0.75
= 0.1.305

Applicability

The M/G/1 queueing model will apply to any system with a Poisson arrivals process and a service discipline which is :

• work conserving – work is neither created nor destroyed (customers cannot die/leave the queue, split in two etc),

• the service selection mechanism is not based on the service time of a particular customer, and

• scheduling of customers is non-preemptive.

For any system which obeys these constraints, the PK formula applies.

Response time is invariant to the scheduling rule BUT only to the mean value, and not to the distribution or its other moments.

In the case of N̄, however, the invariance applies to the distribution and the higher moments.

M_{i}/G_{i}/1 Non-preemptive

Priority

We will look at the case where a single server queue serves 2 priority classes

Analysis

The key is that the service is non-preemptive.

A low priority customer cannot be interrupted by the arrival of a high priority customer, and will complete service

Denote the two service queues by Q_{i}, i = 1, 2, where Q_{1} is the high priority queue.

The arrivals processes to Q_{i} are Poisson distributed at rates λ_{i}

The service processes are of general distribution with means s̄_{i} = 1/μ_{i} and variances σ_{s_{i}}^{2}, i = 1, 2.

The expected service time of the average customer, excluding the priority information is

s̄ = \frac{λ_{1}}{λμ_{1}} + \frac{λ_{2}}{λμ_{2}}

which follows directly from the definition.

Performance Measures

We commence with the derivation of the average time spent in the queue:

T̄_{q} = \frac{λ_{1}}{λ} T̄_{q_{1}} + \frac{λ_{2}}{λ} T̄_{q_{2}}

where

T̄_{q_{1}} = 𝔼 (Y_{0}) + 𝔼 (Y_{1})

and

Y_{0} = remaining service time of

the customer in service

Y_{1} = service time of the

customers ahead of the arrival

An arriving customer sees the sum of the remaining service of the customer currently receiving service plus the service time of the customers ahead of it

Clearly

𝔼 (Y_{1}) = \frac{N̄_{q_{1}}}{μ_{1}} = \frac{λ_{1} T̄_{q_{1}}}{μ_{1}} = ρ_{1} T̄_{q_{1}}

where we have defined ρ_{1} = λ_{1}/μ_{1}. Thus,

T̄_{q_{1}} = \frac{𝔼 (Y_{0})}{1 − ρ_{1}} .

Similarly, for arrivals to the second queue Q_{2},

T̄_{q_{2}} = 𝔼 (Y_{0}) + 𝔼 (Y_{1}) + 𝔼 (Y_{2}) + 𝔼 (Y_{1′})

where

Y_{2} = service time of customers in  Q_{2}
Y_{1′} = service time of customers joining
Q_{1}during the wait at Q_{2}

So

𝔼 (Y_{2}) = ρ_{2} T̄_{q_{2}}

and

𝔼 (Y_{1′}) = 𝔼 (number of arrivals at Q_{1}

during service of Q_{2})

× 𝔼 (service time of a customer in  Q_{1})
= λ_{1} T̄_{q_{2}} × 1/μ_{1}

Putting this all together gives

T̄_{q_{2}} = 𝔼 (Y_{0}) + ρ_{1} T̄_{q_{1}} + ρ_{2} T̄_{q_{2}} + ρ_{1} T̄_{q_{1}}

= \frac{𝔼 (Y_{0})}{(1 − ρ_{1})(1 − ρ_{1} − ρ_{2})}

where ρ_{1} + ρ_{2} < 1.

Similar to the derivation of the remaining time of a customer in service at the M/G/1 queue:

𝔼 (Y_{0}) = \frac{1}{2} 2 ; ∑ ; i=1 λ_{i}s̄_{i}^{2},

which permits the computation of T̄_{q}.

Using Little’s law:

N̄_{q_{i}} = λ_{i}T̄_{q_{i}}
N̄_{q} = λ 2 ; ∑ ; i=1 \frac{λ_{i}}{λ}T̄_{q_{i}}

= 2 ; ∑ ; i=1 N̄_{q_{i}}
T̄_{i} = T̄_{q_{i}} + 1/μ_{i}
N̄_{i} = N̄_{q_{i}} + ρ_{i}
N̄ = _{i}^{∑} N̄_{i}

These results may be generalized for arbitrary numbers of queues Q_{i} – see Leung or Kleinrock for the full details.

A second IP service provider who we are considering decides to queue the arriving traffic into two queues, on the basis of their sizes

Small datagrams take 2ms to transmit, on average, with standard deviation 2ms, and are given priority

Large datagrams take an average of 17ms to transmit, with standard deviation 2ms, and are given low priority

The arrivals process is as described previously, and small packets make up 80% of the traffic

Compare the performance of this system to one in which the priorities are reversed. Should we use this IP service provider (and why)?

Solution

As before λ = 150 packets/second The average service time is, as before

s̄ = 0.8 × 2 + 0.2 × 17 = 5ms

Now

T̄_{q} = 0.8T̄_{q_{1}} + 0.2T̄_{q_{2}}

First we calculate 𝔼 (Y_{0}):

𝔼 (Y_{0}) = \frac{1}{2} 2 ; ∑ ; i=1 λ_{i}s̄_{i}^{2}

= \frac{1}{2} [0.8 × 0.15 × 8 + 0.2 × 0.15 × 293]
= 4.875ms

Now

T̄_{q_{1}} = \frac{𝔼 (Y_{0})}{(1 − ρ_{1})}

= \frac{4.875}{1 − 0.24}
= 6.414
T̄_{q_{2}} = \frac{𝔼 (Y_{0})}{(1 − ρ_{1})(1 − ρ_{1} − ρ_{2})}

= \frac{4.875}{(0.76)(0.76 − 0.51)}
= 25.658

So, finally

T̄_{q} = 0.8 × 6.414 + 0.2 × 25.658
= 10.263

Also

N̄_{q} = λT̄_{q}
= 1.539
N̄ = N̄_{q} + ρ
= N̄_{q} + (\frac{λ_{1}}{λμ_{1}} + \frac{λ_{2}}{λμ_{2}} )
= 1.539 + 0.75
= 2.289