CDA5530: Performance Models of Computers and Networks Chapter 4: Elementary Queuing Theory
Definition Queuing system: a buffer (waiting room), service facility (one or more servers) a scheduling policy (first come first serve, etc.) We are interested in what happens when a stream of customers (jobs) arrive to such a system throughput, sojourn (response) time, Service time + waiting time number in system, server utilization, etc. 2
A/B/c/K queue Terminology A - arrival process, interarrival time distr. B - service time distribution c - no. of servers K - capacity of buffer Does not specify scheduling policy 3
Standard Values for A and B M - exponential distribution (M is for Markovian) D - deterministic (constant) GI; G - general distribution M/M/1: most simple queue M/D/1: expo. arrival, constant service time M/G/1: expo. arrival, general distr. service time 4
Some Notations C n : custmer n, n=1,2,l a n : arrival time of C n d n : departure time of C n α(t): no. of arrivals by time t δ(t): no. of departure by time t N(t): no. in system by time t N(t)=α(t)- δ(t) 5
α(t) δ(t) Average arrival rate (from t=0 to now): λ t = α(t)/t 6
Little s Law γ(t): total time spent by all customers in system during interval (0, t) T t : average time spent in system during (0, t) by customers arriving in (0, t) T t = γ(t)/α(t) N t : average no. of customers in system during (0, t) N t = γ(t)/t For a stable system, N t = λ t T t Remmeber λ t = α(t)/t For a long time and stable system N = λ T Regardless of distributions or scheduling policy 7
Utilization Law for Single Server Queue X: service time, mean T=E[X] Y: server state, Y=1 busy, Y=0 idle ρ: server utilization, ρ = P(Y=1) Little s Law: N = λ E[X] While: N = P(Y=1) 1 + P(Y=0) 0 = ρ Thus Utilization Law: ρ = λ E[X] Q: What if the system includes the queue? 8
Internet Queuing Delay Introduction How many packets in the queue? How long a packet takes to go through? 9
The M/M/1 Queue An M/M/1 queue has Poisson arrivals (with rate λ) Exponential time between arrivals Exponential service times (with mean 1/μ, so μ is the service rate ). One (1) server An infinite length buffer The M/M/1 queue is the most basic and important queuing model for network analysis 10
State Analysis of M/M/1 Queue N : number of customers in the system (including queue + server) Steady state π n defined as π n =P(N=n) ρ=λ/µ: Traffic rate (traffic intensity) State transition diagram 11
we can use πq = 0 and π i = 1 We can also use balance equation 12
State Analysis of M/M/1 Queue # of transitions = # of transitions π n are probabilities: : prob. the server is working (why ρ is called server utilization ) 13
State Analysis of M/M/1 Queue N: avg. # of customers in the system 100 Packets in Queue 80 60 40 20 0 0 0.2 0.4 ρ 0.6 0.8 1 14
M/M/1 Waiting Time X n : service time of n-th customer, X n = st X where X is exponential rv W n : waiting time of n-th customer Not including the customer s service time T n : sojourned time T n =W n +X n When ρ < 1, steady state solution exists and X n, W n, T n X, W, T Q: E[W]? 15
State Analysis of M/M/1 Queue W: waiting time for a new arrival : service time of i-th customer : remaining service time of the customer in service Exponential r.v. with mean 1/µ due to memoryless property of expo. Distr. T: sojourn (response) time 16
Alternative Way for Sojourn Time Calculation We know that E[N] = ρ/(1-ρ) We know arrival rate λ Then based on Little s Law N = λ T E[T]=E[N]/λ = 1/(µ-λ) 17
M/M/1 Queue Example A router s outgoing bandwidth is 100 kbps Arrival packet s number of bits has expo. distr. with mean number of 1 kbits Poisson arrival process: 80 packets/sec How many packets in router expected by a new arrival? What is the expected waiting time for a new arrival? What is the expected access delay (response time)? What is the prob. that the server is idle? What is P( N > 5 )? Suppose you can increase router bandwidth, what is the minimum bandwidth to support avg. access delay of 20ms? 18
Sojourn Time Distribution T s pdf is denoted as f T (t), t 0 T = X 1 + X 2 +L +X n + X Given there are N=n customers in the system Then, T is sum of n+1 exponential distr. T is (n+1)-order Erlang distr. When conditioned on n, the pdf of X is denoted as f T N (t n) f T N (t n)= µ(µt)n e µt n! 19
Sojourn Time Distribution Remove condition N=n: Remember P(N=n) = π n = (1-ρ)ρ n f T (t)=f T 0 (t 0)π 0 +f T 1 (t 1)π 1 + f T (t)= n=0 (1 ρ)ρ nµ(µt)n e µt =(1 ρ)µe µt 20 n=0 =(µ λ)e µt e λt =(µ λ)e (µ λ)t n! (ρµt) n /n! Thus, T is exponential distr. with rate (µ-λ)
M/M/1/K Queue Arrival: Poisson process with rate λ Service: exponential distr. with rate µ Finite capacity of K customers Customer arrives when queue is full is rejected Model as B-D process N(t): no. of customers at time t State transition diagram 21
Calculation of π 0 Balance equation: π i =ρπ i-1 = ρ i π 0, i=0,l, K If λ µ: K i=0 If λ=µ: K i=0 π i =π 0 K 22 i=0 π i =1 π 0 = 1 ρ 1 ρ K+1 K i=0 π i =π 0 K ρ i =π 0 1 ρ K+1 ρ i =(K+1)π 0 i=0 π i =1/(K+1),i=0,,K 1 ρ
E[N] If λ µ: If λ=µ: E[N]= E[N]= K i=0 iπ i = 1 ρ 1 ρ K+1 K i=0 = 1 K+1 K i=0 ρ i iπ i = 1 K+1 K(K+1) 2 K i=0 i = K 2 23
Throughput Throughput? When not idle = µ When idle = 0 Throughput = (1-π 0 )µ +π 0 0 When not full = λ (arrive pass) When full = 0 (arrive drop) Prob. Buffer overflow = π K Throughput = (1-π K )λ +π K 0 24
Sojourn Time One way: T = X 1 +X 2 +L +X n if there are n customers in (n K) Doable, but complicated Another way: Little s Law The λ means actual N = λ T actual throughput E[T]= E[N] throughput = E[N] (1 π 0 )µ 25
M/M/c Queue c identical servers to provide service Model as B-D process, N(t): no.of customers State transition diagram: Balance equation: λπ i 1 =iµπ i, i c, λπ i 1 =cµπ i, i>c 26
Solution to balance equation: ρ i π i = i! π 0, 0 i c, ρ i c!c i cπ 0, c<i Prob. a customer has to wait (prob. of queuing) P(queuing)=P(wait)= π n n=c 27
M/M/ Queue Infinite server (delay server) Each user gets its own server for service No waiting time Balance equation: λπ i 1 =iµπ i, i=0,1, π i = ρi i! π 0 = ρi i! e ρ why? 28
E[N]= i=0 iπ i = =ρe ρ E[T]= E[N] λ i=1 = 1 µ i=1 iρ i e ρ i! ρ i 1 (i 1)! =ρ Why? 29
Machine Repairman Model c machines Each fails at rate λ (expo. distr.) Single repairman, repair rate µ (expo. distr.) Define: N(t) no. of machines working 0 N(t) c 30
π k 1 µ=kλπ k c i=0 π k = 1 k! (µ λ )k π 0 c π i =1 π 1 0 = k=0 1 k! (µ λ )k 31
Utilization rate? ρ=p(repairman busy) = 1-π c E[N]? We can use Complicated E[N]= c i=1 iπ i 32
E[N] Alternative: Little s Law Little s law: N =λ T Here: E[N] = arrival up-time Arrival rate: ρµ+(1 ρ) 0 Up time: expo. E[T]=1/λ Thus E[N]= ρµ λ 33
E[W]: waiting time in queue Y: up-time (working time), exp. λ X: repair time, exp. µ W: waiting time C: cycle time of a customer C=Y+W+X E[C] = E[W]+1/λ+1/µ Throughput ρ C? ρ = c/e[c] 34
Thus E[W]= c ρ (1 µ +1 λ ) Remember ρ=p(repairman busy) = 1-π c N q : avg. no. of machines waiting in the queue Use Little s Law: N=λ T E[N q ] =ρ E[W] = c -ρ(1/λ+1/µ) 35
PASTA property PASTA: Poisson Arrivals See Time Average Meaning: When a customer arrives, it finds the same situation in the queueing system as an outside observer looking at the system at an arbitrary point in time. N(t): system state at time t Poisson arrival process with rate λ M(t): system at time t given that an arrival occurs in the next moment in (t, t+ t) 36
P(M(t)=n)=P(N(t)=n arrival in(t,t+ t)) = P(N(t)=n,arrival in (t,t+ t)) P(arrival in (t,t+ t)) = P(N(t)=n)P(arrival in (t,t+ t)) P(arrival in (t,t+ t)) =P(N(t)=n) If not Poisson arrival, then not correct 37