Introducton to Contnuous-Tme Markov Chans and Queueng Theory
From DTMC to CTMC p p 1 p 12 1 2 k-1 k p k-1,k p k-1,k k+1 p 1 p 21 p k,k-1 p k,k-1 DTMC 1. Transtons at dscrete tme steps n=,1,2, 2. Past doesn t matter (Markov property) P(X n+1 X n,x n-1, X 1 ) = P(X n+1 X n ) 3. P(X n+1 = j X n = ) = p j (statonarty) CTMC 1. Transtons between states can happen any tme 2. Past stll doesn t matter 3. Transton stll ndependent of tme t 2
CTMC: A Generc Defnton A CTMC s contnuous tme stochastc process {X(t), t }, s.t. for all s,t, and any,j,x(u) P{X(t+s) = j X(s) =,X(u) = x(u), u s} = P{X(t+s) = j X(s) = } (Markov property) = P{X(t) = j X() = } (statonarty) τ : tme untl CTMC leaves state, f currently n Above def. mples: P{τ > t+s τ > s} = P{τ > t} Q: What does ths mply? A: τ s exponentally dstrbuted! 3
CTMC: Vewpont 1 exp(v ) p j p k j k p l l Step 1: When at state, wat an exponentally dstrbuted amount of tme denote rate out of: v Step 2: When the process leaves, t wll enter j wth probablty p j 4
CTMC: Vewpont 2 (more common) 1 1 exp( 1 ) 2 2 exp( 2 ) When at state : move to state j wth rate j (ndependently) Q: Tme to leave state? 3 exp( 3 ) frst move to any state frst phone rngs A: Exponental wth rate (v = 1 + 2 + + n = Σ j j ) Q: Probablty to go to state k, after leavng? j A: pj = j j 3 Vew 2 => Vew 1 5
A Sngle Server Example Posson () e.g. 3 jobs/sec servce: exp(μ) e.g. 5 jobs/sec μ 1 2 μ μ Let π j = lm t P j (t) : lmt probablty of beng n state j Q: How do we determne π j? how do we solve the chan? A: Assume an equvalent DTMC half-rgorous argument 6
CTMC DTMC wth tme unt δ CTMC DTMC (tme step δ ) δ+o(δ) δ+o(δ) 1 2 1 2 μ μ μ μδ+o(δ) μδ+o(δ) Dvde tme nto very small ntervals δ Prob. of an arrval n the next nterval δ : δ + o(δ) Prob. of a departure n the next nterval δ : μδ + o(δ) P{ +1} = δ (1-μδ) = δ + o(δ) P{ -1} = μδ (1- δ) = μδ + o(δ) (1 arrval + departure) ( arrval + 1 departure) P{ } = 1- δ -μδ +o(δ) (no event wth almost prob 1) we can gnore self-loops n the balance equatons 7
Solvng the Balance Equatons for a CTMC Methodology Step 1) Wrte balance equatons Step 2) Take lmt as δ δ+o(δ) δ+o(δ) 1 2 μδ+o(δ) μδ+o(δ) For state rate out = rate n ( δ + ο(δ) ) = π ( μδ ο(δ) ) π 1 + ο(δ) ο(δ) π + = π1 μ + δ δ ο(δ) ο(δ) lm π + = lm π1 μ + δ δ δ δ π = π1μ π1 = π μ For state 1 rate out = rate n ( δ + ο(δ) + μδ + ο(δ) ) = π ( δ + ο(δ) ) + π ( μδ ο(δ) ) π1 2 + ο(δ) + μ + = π δ ο(δ) + + π δ π1 2 π μ (π1 = π ) μ 1( + μ) = π + π2μ π μ + 2 π ( + μ) = π + π2μ π2 = μ ο(δ) δ 8
Solvng a CTMC Our frst Queueng Result! Lesson from prevous example: can smply solve the balance equatons, rate n = rate out See book (MHB, CH.13) for generalzaton Can also try local balance equatons Need to check for rreducblty (for π to exst) - What about perodcty?? Back to our queue) Posson () servce: exp(μ) Defne ρ = \μ (utlzaton) n n π (we derved earler) n = π = ρ π μ n π =1 (normalze to get π ) π = ρ ( 1 ρ) n= n Our frst contnuous tme queueng result!! n 9
Introducton to Queueng: M/M/1 Posson () servce: exp(μ) The example so far was an M/M/1 system M stands for memoryless --- both arrvals and departures 1 for sngle server π n : number of customers n system (n servce or queue) Q: What s the probablty that the server s empty? A: π =1-ρ Q: What s the expected number of customers? A: E[N] = = d = (1 = (1 = (1 d n n-1 n πn n ρ ρ) ρ ρ) n ρ ρ ρ) ρ n= n= n= ρ n= d 1 (1 ρ) dρ ρ 1 1 ρ (1 ρ) (1 ρ) ρ = 2 E[N] = ρ (1 ρ) n 1
M/M/1 contnued Q1: What happens when ρ 1? (.e. system very busy) ρ 1 ρ A: E[N] = E[N] (queue goes to nfnty!) ρ 1 Q2: What happens to the Markov Chan when ρ 1? A: All states become transent π = (for all ) Null-recurrent for ρ = 1 But ths means we cannot apply balance equatons (no balance!) Could we have guessed t from the MC?? (homework!) Q3: We know E[N]; What s the expected delay E[T]? A: Lttle s Law wll tell us that E[T] = E[N]/ 11
Introducton to Queueng Theory 12
Queueng Systems: Kendall s Notaton A queung system s descrbed by A/D/n/B/S A: the arrval process M: for Memoryless Posson arrvals D: Determnstc arrvals at fxed ntervals G: generc dstrbuton D: servce process M: Memoryless duraton of servce s exponentally dstrbuted D: Determnstc each servce last the same fxed amount of t G: generc dstrbuton (e.g. unform, pareto, gamma, etc.) n: number of servers B: sze of queue/buffer If queue s full addtonal arrvals are rejected No B n notaton nfnte queue (e.g. M/M/1, M/G/1, G/G/4) S: Schedulng polcy FCFS (default assumed), preemptve, random, etc. 13
Queue Stablty Posson () servce: exp(μ) For the system to be stable: < μ Q: What happens f > μ? A: Queue sze goes to nfnty (and so does delay!) Q: Can you gve the ntuton? A: Consder an nterval [,t] (large t) Number n system = (arrvals n [,t]) (departures n [,t]) Number n system = t μt = (- μ)t t -> queue sze -> 14
Throughput of a Queueng System Throughput X: jobs/mn leavng the system Q: Whch of the two systems has a hgher throughput? A: The throughput s the same for both: X = = 1/6 Consder a busy perod of the system Def: from tme an arrval fnds system empty, untl t emptes agan Assume A arrvals durng a busy perod Q: How many departures durng that busy perod? A: Exactly A! All arrved jobs should complete for the system to empty Open (stable) system: Throughput X = Input Rate 15
Lttle s Law (or the most famous Queueng Result) M/M/1: Posson () servce: exp(μ) E[N] : number of customers n the system (queue + server) E[T]: tme a customer spends n the system (queue+server) Lttle s Law: E[N] = E[T] M/M/1: E[T] = E[N] = ρ 1 ρ 1 = μ 1 (some) Intuton behnd Lttle s Law Customer X arrves at the end of a queue on average stays E[T] Q: how many customers have arrved whle X was n the system? A: E[T] => on average a departure leaves E[N] = E[T] customers n the system 16
Proof of Lttle s Law DEF: A s the area nsde boxes A(t): num of arrvals by t D(t): num of departures by t 1. Sum tmes horzontally D(t) = 1 T A A(t) = 1 T 2. Sum tmes vertcally: A t = N(s)ds N(t 2 )=3 N(t 1 )=2 area A t 1 t 2 D(t) = 1 T t N(s)ds A(t) = 1 T D(t) t A(t) T N(s)ds = 1 = 1 D(t) D(t) t t A(t) T D(t) = 1 t A(t) t T t t N(s)ds t TX N T A(t) = 1 t T ergodcty E[N] = E[T] 17
Generalty of Lttle s Law Q: Dd we assume anythng about the number of servers? A: No! Q: Dd we assume that schedulng s frst-come frst-serve (FCFS)? A: No! In fact, n the prevous example 3 fnshes before 2! Lttle s Law apples to ANY system and ANY schedulng Many servers, many queues, queueng network, schedulng polcy 18
Lttle s Law: Addtonal Results Q: What about E[T Q ]? number of jobs n queue not server(s) A: E[T Q ] = E[N Q ] (smlar proof) Q: What about ρ (number of customers n servce)? A: E[S] = ρ ρ = /μ Q: What s ρ? A: System utlzaton -> fracton of tme server(s) s busy (tme average) A: Ergodcty Prob{server s busy at a random tme} (ensemble average) 19
M/M/k/k loss system Consder a call center wth k total lnes E.g. an actual call center, or swtch wth k crcuts, or base staton wth k channels If all servers/lnes busy call/job s rejected (Key Queston): What s the probablty of blockng? 2
(contnuous-tme) Markov Chan for M/M/k/k Q: How many states do we need? A: k states Q:What s the rate from to +1? A: Q:What s the rate from to -1? A: kμ 21
Local balance equatons for M/M/k/k Q: Is the chan tme-reversble (to apply local balance)?.e. s rate from 1 2 = rate from 2 1? A: Yes, t s so, we can apply local balance - π = μ = π1μ => π1 π 2 1 1 = π22μ => π2 = π - π μ - π -1 = π μ => π = μ 2! 1! π π = 1 => Erlang B P π block = = π = μ k k = 1 1! μ k 1 = μ! k 1 k! 22
What about general servce (M/G/k/k)? Q: Job sze/duraton s not exponental? A: Turns out that the state probabltes π s as n M/M/k/k The block probablty s the same and gven by Erlang B P block = π k = μ k k 1 =! μ 1 k! Ths s an mportant nsenstvty result Result s nsenstve to servce dstrbuton (only depends on mean μ) Insenstvty results often arse when there s no queueng 23
M/M/k Dspatch to frst avalable server queue f all busy Exponental servce wth rate μ Posson () A very mportant model for server farms Amazon, Google, CNN, pretty much all major web stes Cloud computng 24
CTMC for M/M/k Q: How many states? A: Infnte states Q: What s the rate from +1 A: Q: What s the rate from -1 ( k) A: μ Q: What s the rate from -1 ( > k) A: kμ 25
Local Balance Equatons for M/M/k 26
M/M/k: Utlzaton and Queueng Q: What s the expected number of busy servers? A: /μ - Defnton: ρ = /kμ (utlzaton of a sngle server) Q: What s the probablty (P Q ) an arrvng job must queue? A: P Q = P{arrval fnds all servers busy} = P{arrval sees k jobs n system} = =k = Erlang C: P μ Q k π 1 k! π (from PASTA property) = k kμ k = ρ k k k k! π = k ρ k k k! k k = ρ π = k ( kρ) k 1( kρ) ( kρ) k 1 = π, where π = + = k!(1-ρ)! k!(1 ρ) ρ 27
M/M/k Queueng vs. M/M/k/k Blockng Q: For whch system (M/M/k and M/M/k/k) s the probablty that all k server are busy hgher? A: For the M/M/k Q: Why? A: In M/M/k jobs can arrve when all k servers are busy these do not dsappear but queue up more work for later In M/M/k/k these jobs are smply rejected Can confrm by comparng Erlang B and C (work the math as HW) ρ E[N] = PQ + kρ 1 ρ E[N] 1 E[T] = = PQ + kμ 1 μ 28
(Back to) Comparng Channel Allocatons or server organzatons or supermarket lnes Q: Whch s better n terms of E[T], FDM or M/M/1 (statstcal multplexng) A: M/M/1 E[T FDM ] = k/(kμ-) but E[T M/M/1 ] = 1/(kμ-) 29
(Back to) Comparng Channel Allocatons (cont d) or server organzatons or supermarket lnes Q: What about E[T M/M/1 ] and E[T M/M/k ]? A: E[T E[T M/M/k M/M/1 ] ] = P M/M/k Q + k(1 ρ) ρ ρ 1 E[T E[T E[T E[T M/M/k M/M/1 M/M/k M/M/1 ] k ] ] 1 ] 3