Performance Modelling of Computer Systems

Performance Modelling of Computer Systems Mirco Tribastone Institut für Informatik Ludwig-Maximilians-Universität München Fundamentals of Queueing Theory Tribastone (IFI LMU) Performance Modelling of Computer Systems Queueing Theory 1 / 30

Some Notable Infinite Series For any real x < 1, k=0 x k k! = ex, (1) x k = kx k = x 1 x, (2) x (1 x) 2. (3) Tribastone (IFI LMU) Performance Modelling of Computer Systems Queueing Theory 2 / 30

Birth-Death Processes Continuous-time Markov chain with states labelled 0, 1,..., k,... Jumps are only allowed between neighbouring states: State 0 may only make a transition to 1. A state k > 0 may make transitions to k 1 and k + 1. Population increases (births) happen at rate λ k > 0. Population decreases (deaths) happen at rate k > 0. Births and deaths are independent. Model assumptions P(exactly one birth in (t, t + t) X (t) = k) = λ k t + o( t), P(exactly zero births in (t, t + t) X (t) = k) = 1 λ k t + o( t), P(exactly one death in (t, t + t) X (t) = k) = k t + o( t), P(exactly zero deaths in (t, t + t) X (t) = k) = 1 k t + o( t). Tribastone (IFI LMU) Performance Modelling of Computer Systems Queueing Theory 3 / 30

Chapman-Kolmogorov Equations of Birth-Death Processes Denote By the law of total probability: p k (t) := P(X (t) = k), k 0. p 0 (t + t) = p 0 (t)(1 λ 0 t) + 1 p 1 (t) t + o( t), p k (t + t) = p k 1 (t)λ k 1 t + p k (t)(1 λ k t)(1 k t) Rearranging yields + p k+1 (t) k t + o( t) = p k 1 (t)λ k 1 t + p k (t) [ 1 k t λ k t + λ k k t 2] + p k+1 (t) k+1 t + o( t), for k > 0. p 0 (t + t) p 0 (t) = λ 0 p 0 (t) + 1 p 1 (t) + o( t), t t p k (t + t) p k (t) = λ k 1 p k 1 (t) (λ k + k )p k (t) t + k+1 p k+1 (t) + o( t), for k > 0. t Tribastone (IFI LMU) Performance Modelling of Computer Systems Queueing Theory 4 / 30

Chapman-Kolmogorov Equations of Birth-Death Processes p 0 (t + t) p 0 (t) = λ 0 p 0 (t) + 1 p 1 (t) + o( t), t t p k (t + t) p k (t) = λ k 1 p k 1 (t) (λ k + k )p k (t) t + k+1 p k+1 (t) + o( t), for k > 0. t Taking the limit t 0 yields dp 0 (t) = λ 0 p 0 (t) + 1 p 1 (t), dt dp k (t) = λ k 1 p k 1 (t) (λ k + k )p k (t) + k+1 p k+1 (t), k > 0. dt λ 0 λ 1 λ 2 λ k 1 λ k+1 0 1 2 k 1 2 3 k k+1 Tribastone (IFI LMU) Performance Modelling of Computer Systems Queueing Theory 5 / 30

A Generic Recursive Stationary Solution (1/2) λ 0 λ 1 λ 2 λ k 1 λ k+1 0 1 2 k 1 2 3 k k+1 dp 0 (t) = λ 0 p 0 (t) + 1 p 1 (t), dt dp k (t) = λ k 1 p k 1 (t) (λ k + k )p k (t) + k+1 p k+1 (t), k > 0. dt Setting dp k (t)/dt = 0 for all k 0 yields λ k 1 π k 1 (λ k + k )π k + k+1 π k+1 = 0, with λ i = i = 0 for all i < 0. Tribastone (IFI LMU) Performance Modelling of Computer Systems Queueing Theory 6 / 30

A Generic Recursive Stationary Solution (2/2) Rearranging λ k 1 π k 1 (λ k + k )π k + k+1 π k+1 = 0 gives Observe that λ k 1 π k 1 k π k = λ k π k k+1 π }{{ k+1. } g(k) g(k 1) = g(k), for all k, therefore g(k) must be constant with k. From dp 0 (t)/dt = 0 we get that g(k) = 0. Therefore, we obtain the recursive solution λ k π k+1 = π k = π k = π 0 k+1 k 1 λ i i+1 i=0 } {{ } product form type Question: How to compute π 0? 1 π 0 = 1 π k = 1 + k 1 i 0, k = 0, 1, 2,.... λ i i+1 Tribastone (IFI LMU) Performance Modelling of Computer Systems Queueing Theory 7 / 30

The Poisson Process Consider a pure-birth process with λ k = λ for k 0 and k = 0 for k > 0, and assume that p 0 (0) = 1 and p k (0) = 0 for all k > 0. The equations simplify to dp 0 (t) = λp 0 (t) dt dp k (t) = λp k 1 (t) λp k (t), k > 0. dt Solving the first equation p 0 (t) = e λt, by induction it is proven that p k (t) = e λt (λt) k, k 0. k! This is the Poisson process, a counting process with exponentially distributed increments with mean 1/λ. λ λ λ λ λ 0 1 2 k Tribastone (IFI LMU) Performance Modelling of Computer Systems Queueing Theory 8 / 30

Properties of Poisson Process Mean: E[N(t)] = kp k (t) = k=0 = e λt k=0 k=0 k e λt (λt) k k! (λt) k (k 1)! = e λt k=0 (λt) k (k 1)! = e λt (λt) k (λt) = e λt (λt)e λt = λt. k! Variance: Var[N(t)] = λt. Memoryless property: P(N(s, s + t) = k) = (λt)k e λt, k! where N(s, s + t) is defined as the number of arrivals between s and s + t. Tribastone (IFI LMU) Performance Modelling of Computer Systems Queueing Theory 9 / 30

Why Are Poisson Processes Relevant? In addition to mathematical tractability, they model many phenomena: Arrivals of calls in a telephone network; Decay of radioactive elements (gamma-ray emissions); Army soldiers killed due to being kicked by their horses; 1 A large number of independent renewal processes will tend to a Poisson process. The birth-death process may be interpreted as a queueing system where increments (with rate λ) denote arrivals of request and decrements (with rate ) are related to services. It is denoted in Kendall notation as the M/M/1 system: Exponentially distributed interarrival times; Exponentially distributed services; Single-server system. 1 Kleinrock, Queueing Systems: Volume I Theory, Wiley Interscience, NY, 1975. Tribastone (IFI LMU) Performance Modelling of Computer Systems Queueing Theory 10 / 30

Kendall s Notation A queueing system is often denoted by A/B/C/X /Y /Z, where: A gives the distribution of interarrival times (e.g., M, E, G, D,... ); B gives the service time distribution; C gives the service multiplicity (1, 2,..., ); X gives the system capacity; Y gives the customer population; Z gives the queue discipline (i.e., FIFO, LIFO, RANDOM, etc.). Tribastone (IFI LMU) Performance Modelling of Computer Systems Queueing Theory 11 / 30

Stationary Distribution of the M/M/1 System (1/2) Consider again the equations of motion: dp 0 (t) = λp 0 (t) + p 1 (t), dt dp k (t) = λp k 1 (t) (λ + )p k (t) + p k+1 (t), k > 0. dt We look for a probability vector π = [π 0, π 1,..., π k,...] such that λπ 0 + π 1 = 0, λπ k 1 (λ + )π k + π k+1 = 0, k > 0, π i = 1. i=0 From the first equation, π 1 = (λ/)π 0. For k = 1, λπ 0 (λ + )π 1 + π 2 = 0, from which π 2 = (λ/ + 1)π 1 (λ/)π 0 = (λ/) 2 π 0. Tribastone (IFI LMU) Performance Modelling of Computer Systems Queueing Theory 12 / 30

Stationary Distribution of the M/M/1 System (2/2) By induction one can prove that π k = (λ/) k π 0, for k > 0. ) π 0 = 1 π k = 1 (λ/) k π 0 = π 0 (1 + (λ/) k = 1. If λ < then the series converges. Therefore π 0 = Setting ρ = λ/ we obtain 1 1 + λ/ 1 λ/ = 1 λ/. π k = ρ k (1 ρ), for all k > 0. Tribastone (IFI LMU) Performance Modelling of Computer Systems Queueing Theory 13 / 30

Performance Metrics for the M/M/1 Queue (1/2) Mean queue length: average number of customers in the system. L = kπ k = kρ k (1 ρ) = (1 ρ)ρ kρ k 1 k=0 = (1 ρ)ρ = (1 ρ)ρ d dρ k=0 d dρ ρk = (1 ρ)ρ d dρ k=0 [ ] ρ 1 ρ k=0 ρ k = (1 ρ)ρ 1 ρ + ρ (1 ρ) 2 = ρ (1 ρ). Utilisation: probability that the server is busy. Formally, it may be defined as the expected value of a function of the random variable that underlies the stationary distribution. { 0 if X = 0, u(x ) = 1 otherwise. U := E[u(X )] = ρ k (1 ρ) = 1 π 0 = ρ. Tribastone (IFI LMU) Performance Modelling of Computer Systems Queueing Theory 14 / 30

Performance Metrics for the M/M/1 Queue (2/2) Average response time. We invoke Little s law, which states that for a system in steady state L = λw, where: L is the average number of users in the system; λ is the steady-state rate of arrivals into the system (which is equal to the throughput, i.e., the steady-state rate of departures from the system); W is the average response time. In the M/M/1 queue, W = L/λ = ρ (1 ρ)λ = 1 (1 ρ) = 1 λ. Tribastone (IFI LMU) Performance Modelling of Computer Systems Queueing Theory 15 / 30

M/M/1 Performance, Pictorially 100 80 60 L 40 20 0 0 0.2 0.4 0.6 0.8 1 Tribastone (IFI LMU) Performance Modelling of Computer Systems Queueing Theory 16 / 30

The M/M/ System Service capacity is proportional to the number of customers in the system. λ λ λ λ λ λ 0 1 2 k 1 k 2 3 (k 1) k λπ 0 = π 1 = π 1 = λ π 0 λπ 1 + π 1 = λπ 0 + 2π 2 = π 2 = 1 ( ) λ 2 π 0 2 By induction, π k = 1 k! (λ/)k π 0, k > 0. From the normalisation condition, ( ) [ 1 λ k ( ) ] 1 λ k ( ) 1 λ k 1=π 0 + π 0 =π 0 1 + =π 0 =π 0 e λ/ k! k! k! = π k = 1 k! k=0 ( ) λ k e λ/, k 0. Tribastone (IFI LMU) Performance Modelling of Computer Systems Queueing Theory 17 / 30

Performance of the M/M/ System Average queue length L = kπ k = k 1 ( ) λ k e λ/ = k 1 k! k! k=0 k=0 ( ) = e λ/ 1 λ k ( ) λ = e λ/ (k 1)! ( ) λ ( ) = e λ/ 1 λ n ( λ = e λ/ n! n=0 Average response time: W = 1/. Question: What is the utilisation? ( ) λ k e λ/ 1 (k 1)! ) e λ/ = λ. ( ) λ k 1 Tribastone (IFI LMU) Performance Modelling of Computer Systems Queueing Theory 18 / 30

The M/M/m System A multi-server system with finite capacity m. λ λ λ λ λ 0 1 2 k min(m, 2) min(m, 3) min(m, k) min(m, k + 1) If in the k-th state there are fewer clients than servers then the balance equations are as in the M/M/ system, thus yielding π k = 1 «k λ π 0, for all 1 k m. k! In state m, λπ m 1 + mπ m+1 = mπ m + λπ m leads to: π m+1 = 1 + λ «π m λ m m πm 1 = 1 + λ ««m 1 λ π 0 λ «m 1 1 λ π 0 = m 1 + λ m m! «1 m! λ «m π 0 1 m! m (m 1)! «m λ π 0 = λ 1 m m! «m λ π 0 = 1 m «m+1 λ π 0. Tribastone (IFI LMU) Performance Modelling of Computer Systems Queueing Theory 19 / 30

The M/M/m System In general, it holds that ( ) 1 λ k π 0, 1 k m, k! π k = ( ) λ k ( ) 1 1 n m π 0, k > m, m! m from which one obtains [ π 0 = 1 + m 1 1 k! ( ) λ k + 1 m! ( ) ] 1 λ m 1. 1 λ/(m) Performance Measures [ (λ/) m ] W = (m 1)!(m λ) 2 π 0 + 1, [ (λ/) m ] λ L = (m 1)!(m λ) 2 π 0 + λ. Tribastone (IFI LMU) Performance Modelling of Computer Systems Queueing Theory 20 / 30

Comparison = 1 20 15 M/M/1 M/M/2 M/M/ W 10 5 0 0.6 0.7 0.8 0.9 X: 0.93 Y: 1.276 Tribastone (IFI LMU) Performance Modelling of Computer Systems Queueing Theory 21 / 30

Finite Capacity: The M/M/1/K System π 0 + λ λ λ λ 0 1 2 K K π k = π 0 + = π 0 = λπ 0 = π 1, λπ k + π k = λπ k 1 + π k+1, 0 < k < K, λπ K 1 = π K. ( ) λ k π k = π 0, k 0. [ K k=0 K ( λ ) k π 0 = π 0 K k=0 ( ) λ k = 1 ( ) ] 1 { λ k (1 λ/)/(1 (λ/) K+1 ) if λ, = 1/(K + 1) if λ =. Tribastone (IFI LMU) Performance Modelling of Computer Systems Queueing Theory 22 / 30

Performance Measures for the M/M/1/K System From λ λ λ λ 0 1 2 K π 0 = ( 1 λ/ ) / ( 1 (λ/) K+1) and π k = and setting ρ = λ/, ( ) λ k π 0, k 0, 1 π 0 1 π K = 1 (1 ρ)/(1 ρk+1 ) 1 (1 ρ)ρ K /(1 ρ K+1 ) = (1 ρk+1 ) (1 ρ) (1 ρ K+1 ) (1 ρ)ρ K = ρ ρk+1 1 ρ K = ρ, which yields the relationship λ(1 π K ) = (1 π 0 ) [effective arrival rate = effective service rate]. Tribastone (IFI LMU) Performance Modelling of Computer Systems Queueing Theory 23 / 30

Measures as Rewards Consider the following function of a r.v. over the state space of the M/M/1/K system: { λ, if k K, X a (k) = 0, if k = K. Similarly, define E[X a ] = E[X s ] = K k=0 K 1 X a (k)π k = λ π k = λ(1 π K ). X s (k) = k=0 k=0 {, if k 0, 0, if k = 0. S K X s (k)π k = π k = (1 π 0 ) Tribastone (IFI LMU) Performance Modelling of Computer Systems Queueing Theory 24 / 30

Comparison 0.5 0.4 0.3 K 0.2 0.1 = 0.9 = 0.5 = 0.1 0 0 5 10 15 20 25 30 K Tribastone (IFI LMU) Performance Modelling of Computer Systems Queueing Theory 25 / 30

The Erlang Distribution Exponential service phases in tandem Begin End Each phase is defined by a r.v. Y with pdf f Y (y) = e y, y 0. The total service time is given by X = Y + Y, which has pdf f X (x) = = + x 0 f Y (y)f Y (x y)dy e y e (x y) dy x = 2 e x dy = 2 xe x, x 0. X is called the Erlang-2 distribution (E 2 ). 0 Tribastone (IFI LMU) Performance Modelling of Computer Systems Queueing Theory 26 / 30

Properties of the Erlang Distribution Mean and Variance E[X ] = + 0 Var[X ] = 2/ 2. + xf X (x)dx = 2 x 2 e x dx = 2/, Compare now an exponentially distributed r.v. with rate and an Erlang distribution with phase 2. 0 2 2 Mean : 1/ Variance : 1/ 2 Mean : 1/ Variance : 1/2 2 Generalisation: the Erlang-r distribution Mean : r/ Variance : r/ 2 Tribastone (IFI LMU) Performance Modelling of Computer Systems Queueing Theory 27 / 30

Pdfs of Some Erlang Distributions Mean = 1.0 4 3 r = 1 r = 5 r = 20 r = 100 2 1 0 0 1 2 3 4 5 Tribastone (IFI LMU) Performance Modelling of Computer Systems Queueing Theory 28 / 30

The M/E r /1 System Features: Poisson arrivals; Erlang-distributed service with r phases: if a customer is currently served at phase i, 1 i r then no other customer may be served in any of the other phases; Single-server model. Model description: a state is described by the pair (k, i) where k denotes the number of customer in the system, including the one in service; i denotes the phase of service being received by the customer, 0 i r; i = 0 when the queue is empty. Tribastone (IFI LMU) Performance Modelling of Computer Systems Queueing Theory 29 / 30

State Transition Diagram for the M/E r /1 System (0, 0) λ r (1, 1) r r (1, 2) r (1, r) r λ λ λ r r (2, 1) (2, 2) (2, r) r λ λ λ r λ λ λ r r (k, 1) (k, 2) (k, r) r λ λ λ Analysis techniques: 1. Quasi-birth-death (QBD) form for the transition rate matrix a B 00 B 01 0 0 0 B 10 A 1 A 2 0 0 0 A 0 A 1 A 2 0 Q = 0 0 A 0 A 1 A 2 0 0 0 A 0 A 1........ 2. Using z-transforms b a W.J. Stewart. Probability, Markov Chains, Queues, and Simulation, 2009, Princeton University Press. b Kleinrock, Queueing Systems: Volume I Theory, Wiley Interscience, NY, 1975. Tribastone (IFI LMU) Performance Modelling of Computer Systems Queueing Theory 30 / 30