Structured Markov Chains

Structured Markov Chains Ivo Adan and Johan van Leeuwaarden Where innovation starts

Book on Analysis of structured Markov processes (arxiv:1709.09060) I Basic methods Basic Markov processes Advanced Markov processes Queues and transforms II Basic processes Birth-and-death processes Queueing networks Quasi-birth-and-death-processes Processes that are quasi-skip-free in one direction III Advanced processes Single-server priority systems Gated single-server systems Production systems Join the shortest queue 2

Markov chains in discrete time A stochastic process {X n, n = 0, 1, 2,...} with state space {0, 1, 2,...} is a Markov chain if for all i 0,..., i n+1 and n 0 Assumptions: P(X n+1 = i n+1 X 0 = i 0, X 1 = i 1,..., X n = i n ) = P(X n+1 = i n+1 X n = i n ) Time homogeneous: P(X n+1 = j X n = i) = p ij Irreducible: all states are reachable Aperiodic Equilibrium distribution p 0, p 1,... satisfy the equilibrium (or balance) equations p i = p j p ji, i = 0, 1,... j=0 If equilibrium distribution exists then it is equal to the limiting distribution lim n P(X n = i) = p i Vector-matrix notation: p = pp where p = (p 0, p 1,...) and transition probability matrix P = (p ij ) 3

Markov chains in continuous time A stochastic process {X(t), t 0} with state space {0, 1, 2,...} is a Markov chain if for all i, j, x(u) and s, t 0 Assumptions: P(X(t + s) = j X(u) = x(u), 0 u < s, X(s) = i) = P(X(t + s) = j X(s) = i) Time homogeneous: P(X(t + s) = j X(s) = i) = p ij (t) Irreducible: all states are reachable Transition rates q ij = lim h 0 p ij (h) h where we let q ii = 0 and q i = j=0 q ij Equilibrium distribution p 0, p 1,... satisfy the equilibrium (or balance) equations p i q i = p j q ji, i = 0, 1,... j=0 If equilibrium distribution exists then it is equal to the limiting distribution lim t P(X(t) = i) = p i 4 Vector-matrix notation: 0 = pq where p = (p 0, p 1,...) and generator Q with Q ij = q ij, i = j, Q ii = q i

Balance equations Global balance equations: or for subset A and its complement A c p i q i = p j q ji j=0 p i q ij = p j q ji j A c i A Partial balance equations: for subsets A and B Detailed (or local) balance equations: p i q ij = j A j B p j q ji p i q ij = p j q ji 5

Balance equations Global balance equations: Partial balance equations: Detailed (or local) balance equations: 6

Transforms Generating function of non-negative discrete random variable X with probability distribution p i = P(X = i) P X (z) = E(z X ) = p i z i, z 1 i=0 Laplace-Stieltjes transform a non-negative (continuous) random variable X with distribution function F X (x) X(s) = E(e sx ) = 0 e sx df X (x), Re(s) 0 Properties: Probability distribution p i is uniquely determined by generating function P X (z) Distribution function F X (x) is uniquely determined by Laplace-Stieltjes X(s) E(X(X 1) (X n + 1)) = P (n) X (1) and E(X n ) = ( 1) n X (n) (0) where (n) is nth derivative 7

Transforms: Example Markov chain {X n, n = 0, 1, 2,...} with state space {0, 1, 2,..., n + 1} and transition probabilities p ij State n + 1 is single absorbing state p 12 = 1 2 p 23 = 1 3 p 11 = 1 2 1 2 3 p 33 = 1 p 21 = 2 3 Random variable N i is number of visits to state i before absorption into n + 1 Time spent in state i during the kth visit is equal to α k 1 i where 0 < α i < 1 (learning curve) Random variable T is total time spent in states {1,..., n} before absorption in n + 1 What is E(T X 0 = 1)? 8

Transforms: Example Note so T = n ( ) 1 + α i + + α N i 1 i = i=1 E(T X 0 = 1) = n i=1 n i=1 1 α N i i 1 α i 1 E(α N i i X 0 = 1) 1 α i Introduce for i = 1,..., n the generating function P i (z) = E(z N 1 1 z N n n X 0 = i), z 1 1,..., z n 1, where z = (z 1,..., z n ) so P 1 (1,..., 1, α i, 1,..., 1) = E(α N i i X 0 = 1) 9

Transforms: Example One-step analysis: P i (z) = E(z N 1 1 z N n n X 0 = i) n+1 = p ij E(z N 1 1 z 1+N i i = = j=1 n+1 j=1 n+1 j=1 z i p ij E(z N 1 1 z N i i z i p ij P j (z), where P n+1 (z) = 1 (since N 1 = = N n = 0 if X 0 = n + 1) z N n n X 0 = j) z N n n X 0 = j) i = 1,..., n Hence P 1 (1,..., 1, α i, 1,..., 1) can be solved from above equations with z = (1,..., 1, α i, 1,..., 1) 10

Finite Markov chains (no structure) Markov process {X(t), t 0} with state space {0, 1,..., N} and transition rates q ij Global balance equations p i N q ij = N p j q ji, i = 0, 1,..., N j=0 j=0 11

Finite Markov chains (no structure) N q in q Nj q ij i j Numerical solution through embedding: Remove state N by embedding Markov chain on {0, 1,..., N 1} with transition rates q ij = q ij + q in Equilibrium probability p N follows from global balance q Nj q N0 + + q NN 1, i, j = 0,..., N 1 N 1 p N j=0 q Nj = N 1 j=0 p j q jn Repeat this for N 1, N 2,..., 1 and finally normalize p 0,..., p N 12 This is Gaussian elimination with multiplication, division and addition of positive numbers

Infinite Markov chains (with structure): Example of impatient customers θ λ µ Single server Poisson arrivals with rate λ Exponential service times with rate µ Exponential patience with rate θ: customer leaves if service does not start before patience expires Markov chain with states i where i is number in system Transition rates q i,i 1 = µ + (i 1)θ and q i 1,i = λ λ λ 0 1 i 1 i µ µ + (i 1)θ 13

Infinite Markov chains (with structure): Birth-and-Death processes λ 0 λ i 1 0 1 i 1 i µ 1 µ i Markov chain on {0, 1,...} with transitions to neighboring states with rates q i,i 1 = µ i and q i 1,i = λ i 1 Global balance p 0 λ 0 = p 1 µ 1 p i (λ i + µ i ) = p i 1 λ i 1 + p i+1 µ i+1, i = 1, 2,... Global balance for {0, 1,..., i 1} p i 1 λ i 1 = p i µ i, i = 1, 2,... 14

Infinite Markov chains (with structure): Birth-and-Death processes λ 0 λ i 1 0 1 i 1 i µ 1 µ i Iterating yields Normalization Assumption Example of impatient customers p i = p i 1 λ i 1 µ i = p i 2 λ i 2 λ i 1 µ i 1 µ i = = p 0 λ 0 λ i 1 µ 1... µ i p 0 = i=0 1 i=0 ij=1 λ j 1 µ j i j=1 λ j 1 µ j < 15 p i = p 0 µ(µ + θ) (µ + (i 1)θ) λ i

Infinite Markov chains (with more structure): Constant birth and death rates λ λ 0 1 i 1 i µ µ Global balance equations p 0 λ = p 1 µ p i (λ + µ) = p i 1 λ + p i+1 µ, i = 1, 2,... Assume λ µ < 0 and (without loss of generality) λ + µ = 1 Generator Q of the form λ λ 0 0... µ (λ + µ) λ 0 Q = 0 µ (λ + µ) λ 0 0 µ (λ + µ).... 16

Infinite Markov chains (with more structure): Constant birth and death rates λ λ 0 1 i 1 i µ µ Solution: Generating function P(z) = i=1 p i z i Multiply balance equations by z i and add all equations Solving this equation where ρ = λ/µ P(z) = Normalization P(1) = 1 so p 0 = 1 ρ and P(z)(λ + µ) p 0 µ = P(z)λz + (P(z) p 0 )µz 1 p 0 µ(1 z 1 ) (λ + µ) λz µz 1 = p 0µ(1 z 1 ) (1 z 1 )(µ λz) = p 0 1 ρz 17 P(z) = 1 ρ 1 ρz = (1 ρ)ρ i z i i=0

Infinite Markov chains (with more structure): Constant birth and death rates λ λ 0 1 i 1 i µ µ Solution: Geometric Ratio p i+1 /p i is expected time spent in i + 1 before first return to i, given initial state is i Transition structure implies that ratio p i+1 /p i does not depend on i p i+1 p i = α Hence p i = p 0 α i and substitution in global balance equation yields α i (λ + µ) = α i 1 λ + α i+1 µ Dividing by common power α i 1 α(λ + µ) = λ + α 2 µ This leads to α = λ µ = ρ 18

Infinite Markov chains (with more structure): Constant birth and death rates λ λ 0 1 i 1 i µ µ Solution: Difference equation Balance equations are second-order difference equation with constant coefficients Solution is linear combination of powers p i = α i Substitution of p i = α i in balance equation yields with roots α 1 = ρ and α 2 = 1 General solution where c 1 and c 2 are constants α(λ + µ) = λ + α 2 µ p i = c 1 α i 1 + c 2α i 2 = c 1ρ i + c 2 Convergence of p i implies c 2 = 0 and normalization yields c 1 = 1 ρ 19

Infinite Markov chains: M/G/1 structure b 0 a 1 b 1 0 1 i 1 i a 0 a 2 a 3 i + 1 i + 2 Transition probabilities P = (p ij ) P = b 0 b 1 b 2 b 3 a 0 a 1 a 2 a 3 0 a 0 a 1 a 2 0 0 a 0 a 1.... Example: Number in system just after departure in M/G/1 queue Let A(z) = i=0 a i z i and B(z) = i=0 b i z i and assume mean step size = A (1) (1) 1 < 0 Global balance equations p i = p i+1 a 0 + p i a 1 + + p 1 a i + p 0 b i, i = 0, 1, 2,... 20

Infinite Markov chains: M/G/1 structure b 0 a 1 b 1 0 1 i 1 i a 0 a 2 a 3 i + 1 i + 2 Solution: Recursion Global balance for {0, 1,..., i 1} p 0 b i + p 1 ā i 1 + + p i 1 ā 1 = p i a 0, i = 1, 2,... where b i = b i, ā i = j=i j=i a i Recursively calculate p 1, p 2,... starting with p 0 p 0 follows from (mean displacement is 0 in equilibrium) p 0 B (1) (1) + (1 p 0 )(A (1) (1) 1) = 0 21

Infinite Markov chains: M/G/1 structure b 0 a 1 b 1 0 1 i 1 i a 0 a 2 a 3 i + 1 i + 2 Solution: Generating function Let P(z) = i=0 p i z i, A(z) = i=0 a i z i, B(z) = i=0 b i z i Multiply balance equations by z i and add all equations P(z) = z 1 (P(z) p 0 )A(z) + p 0 B(z) Solving this equation Normalization P(1) = 1 so P(z) = p 0(B(z) z 1 A(z)) 1 z 1 A(z) p 0 = 1 A (1) (1) B (1) (1) + 1 A (1) (1) = p 0(zB(z) A(z)) z A(z) 22

Infinite Markov chains: G/M/1 structure a 1 b i a i 0 1 i 2 i 1 i a 3 a 2 a 0 i + 1 Transition probabilities P = (p ij ) P = b 0 a 0 0 0 b 1 a 1 a 0 0 b 2 a 2 a 1 a 0 b 3 a 3 a 2 a 1...., b i = 1 (a 0 + + a i ) = j=i+1 a i Example: Number in system just before arrival in G/M/1 queue Global balance equations 23 p i = p i 1 a 0 + p i a 1 + p i+1 a 2 + = p i 1+j a j, i = 1, 2,... j=0

Infinite Markov chains: G/M/1 structure a 1 b i a i 0 1 i 2 i 1 i a 3 a 2 a 0 i + 1 Solution: Geometric Ratio p i+1 /p i is expected time spent in i + 1 before first return to i, given initial state is i Transition structure implies that ratio p i+1 /p i does not depend on i p i+1 p i = α Hence p i = p 0 α i and substitution in global balance equation yields α = α j a j = A(α) j=0 24 α is the unique root on (0, 1) of α = A(α) provided mean step size 1 A (1) (1) < 0

Examples of Quasi-Birth-and-Death processes: Single server with setup Single server Poisson arrivals with rate λ wich need exponential service times with rate µ Server switched off when system is empty Server switched on when first customer arrives which requires exponential setup time with rate θ 25

Examples of Quasi-Birth-and-Death processes: Single server with setup Markov chain with states (i, j) with i number in system and j status of server: 0 means off, 1 means on Transition rate diagram λ λ λ 0, 1 1, 1 2, 1 i, 1 µ µ µ θ θ θ λ λ λ 0, 0 1, 0 2, 0 i, 0 Global balance equations p(0, 0)λ = p(1, 1)µ p(0, 1)λ = 0 p(i, 0)(λ + θ) = p(i 1, 0)λ p(i, 1)(λ + µ) = p(i 1, 1)λ + p(i, 0)θ + p(i + 1, 1)µ, i = 1, 2,... 26

Examples of Quasi-Birth-and-Death processes: Single server with setup Order states as (0, 0), (0, 1), (1, 0), (1, 1), (2, 0), (2, 1),... and partition into levels i = {(i, 0), (i, 1)} Generator Q of the form where B 1 = ( ) λ 0, B 0 λ 2 = ( ) 0 0, A µ 0 0 = Let p = (p 0, p 1,...) and p i = (p(i, 0), p(i, 1)) B 1 A 0 0 0... B 2 A 1 A 0 0 Q = 0 A 2 A 1 A 0 0 0 A 2 A 1.... ( ) ( ) λ 0 (λ + θ) θ, A 0 λ 1 =, A 0 (λ + µ) 2 = p 0 B 1 + p 1 B 2 = 0, p i 1 A 0 + p i A 1 + p i+1 A 2 = 0, i = 1, 2,... ( ) 0 0 0 µ 27

Examples of Quasi-Birth-and-Death processes: Single server with setup Solution: Matrix-geometric Global balance for level 0, 1,..., i p(i + 1, 1)µ = (p(i, 0) + p(i, 1))λ or in matrix form ( ) 0 λ with A 3 = 0 λ Substituting in balance equation p i+1 A 2 = p i A 3 28 Hence with 0 = p i 1 A 0 + p i A 1 + p i+1 A 2 = p i 1 A 0 + p i (A 1 + A 3 ) p i = p i 1 A 0 (A 1 + A 3 ) 1 = p i 1 R R = A 0 (A 1 + A 3 ) 1 = ( ) λ/(λ + θ) λ/µ 0 λ/µ

Examples of Quasi-Birth-and-Death processes: Single server with setup Solution: Matrix-geometric Iterating yields p i = p 0 R i, i = 0, 1, 2,... Vector p 0 follows from boundary equation p 0 B 1 + p 1 B 2 = p 0 (B 1 + RB 2 ) = 0 and normalization 1 = p(i, j) = i,j where I is identity matrix and e all-one vector p i e = p 0 R i e = p 0 (I R) 1 e i=0 i=0 29

Examples of Quasi-Birth-and-Death processes: Single server with setup Solution: Matrix-analytic Let g jk be probability of first passage to level i 1 in (i 1, k) for initial state (i, j) at level i ( ) 0 1 G = (g jk ) = 0 1 Then Substituting in balance equation p i+1 A 2 = p i A 0 G 0 = p i 1 A 0 + p i A 1 + p i+1 A 2 = p i 1 A 0 + p i (A 1 + A 0 G) Hence p i = p i 1 A 0 (A 1 + A 0 G) 1 = p i 1 R 30

Examples of Quasi-Birth-and-Death processes: Single server with setup Solution: Spectral expansion Seek solutions of balance equations p i 1 A 0 + p i A 1 + p i+1 A 2 = 0 of the form p i = y x i, i = 0, 1, 2,..., where y = (y(0), y(1)) is non-null vector and x < 1 Substitution and dividing by common powers of x yields y ( A 0 + xa 1 + x 2 ) A 2 = 0 Desired values of x are roots inside the unit circle of det(a 0 + xa 1 + x 2 A 2 ) = (λ (λ + θ)x)(µx λ)(x 1) = 0 31

Examples of Quasi-Birth-and-Death processes: Single server with setup Solution: Spectral expansion Roots with corresponding non-null vectors Set x 1 = λ λ + θ, x 2 = λ µ θx 1 y 1 = (1, λ (λ + µ)x 1 + µx1 2 ), y 2 = (0, 1) p i = c 1 y 1 x i 1 + c 2y 2 x i 2 where coefficients c 1 and c 2 follow from boundary equation p 0 B 1 + p 1 B 2 = 0 and normalization 1 = i,j p(i, j) = c 1y 1 e 1 x 1 + c 2y 2 e 1 x 2 32

Examples of Quasi-Birth-and-Death processes: Single server with setup Solution: Generating function Let P(z) = i=0 p i z i Multiply balance equations by z i and add all equations zp(z)a 0 + (P(z) p 0 )A 1 + z 1 (P(z) p 0 p 1 z)a 2 = 0 Solving this equation P(z)A(z) = p 0 (A 1 + z 1 A 2 ) + p 1 A 2 where A(z) = za 0 + A 1 + z 1 A 2 = ( λz (λ + θ) θ ) 0 λz (λ + µ) + µz 1 Vectors p 0, p 1 and P(1) follow from p 0 B 1 + p 1 B 2 = 0 P(1)A(1) = p 0 (A 1 + A 2 ) + p 1 A 2, P(1)e = 1 P(1)A (1) (1)e = p 0 A 2 e 33

Examples of Quasi-Birth-and-Death processes: Shortest queue with jockeying Two parallel exponential servers with rate µ, each with its own queue Customers arrive according to Poisson stream with rate λ and join shortest queue on arrival Jockeying: If difference in queues exceeds 1, then a customer jumps from longest to shortest queue 34

Examples of Quasi-Birth-and-Death processes: Shortest queue with jockeying Markov chain with states (i, j) with i number in shortest queue and j difference between queues Transition rate diagram 0, 1 1, 1 2µ i, 1 i+1, 1 λ µ λ 0, 0 1, 0 2µ λ λ i, 0 i+1, 0 35

Examples of Quasi-Birth-and-Death processes: Shortest queue with jockeying Partition into levels i = {(i, 0), (i, 1)} for i = 0, 1,... Generator Q of the form where B 1 = ( ) λ λ, A µ (λ + µ) 0 = Let p = (p 0, p 1,...) and p i = (p(i, 0), p(i, 1)) B 1 A 0 0 0... A 2 A 1 A 0 0 Q = 0 A 2 A 1 A 0 0 0 A 2 A 1.... ( ) ( ) 0 0 (λ + 2µ) λ, A λ 0 1 =, A 2µ (λ + 2µ) 2 = p 0 B 1 + p 1 A 2 = 0 p i 1 A 0 + p i A 1 + p i+1 A 2 = 0, i = 1, 2,... ( ) 0 2µ 0 0 36

Examples of Quasi-Birth-and-Death processes: Single server serving n customer types Single server serving n types of customers, numbered 1,..., n Type i customers arrive according to Poisson stream with rate λ i Type i customers require exponential service with rate µ i 37

Examples of Quasi-Birth-and-Death processes: Single server serving n customer types Markov chain with states (i, j) where i number waiting in the queue and j is customer type in service State (0, 0) is empty state Transition rate diagram where λ = λ 1 + + λ n and p i = λ i /λ λ λ 0, n µ n p n 1, n µ i, n n p n. λ µ n p 1 µ n p 1 0, 1 1, 1 i, 1 λ 1 µ 1 µ 1 p 1 µ 1 p 1 0, 0 38

Examples of Quasi-Birth-and-Death processes: Single server serving n customer types Partition into level 0 = {(0, 0),..., (0, n)} and levels i = {(i, 1),..., (i, n)} for i = 1, 2,... Generator Q of the form where B 1 B 0 0 0... B 2 A 1 A 0 0 Q = 0 A 2 A 1 A 0 0 0 A 2 A 1.... λ 0 (λ + µ 1 ) 0 µ 1 p 1 µ 1 p n A 0 =..., A 1 =..., A 2 =.. 0 λ 0 (λ + µ n ) µ n p 1 µ n p n Let p = (p 0, p 1,...), p 0 = (p(0, 0),..., p(0, n)) and p i = (p(i, 1),..., p(i, n)) 39 p 0 B 1 + p 1 B 2 = 0 p 0 B 0 + p 1 A 1 + p 2 A 2 = 0 p i 1 A 0 + p i A 1 + p i+1 A 2 = 0, i = 2, 3,...

Quasi-Birth-and-Death processes Markov chain with states (0, 0), (0, 1),..., (0, n), (1, 0), (1, 1),..., (1, n), (2, 0),... Partition into levels i = {(i, 0),..., (i, n)} for i = 0, 1,... Generator Q of the form where B 1, A 0, A 1 and A 2 are n + 1 n + 1 matrices Let p = (p 0, p 1,...) and p i = (p(i, 0), p(i, 1),..., p(i, n)) B 1 A 0 0 0... A 2 A 1 A 0 0 Q = 0 A 2 A 1 A 0 0 0 A 2 A 1.... p 0 B 0 + p 1 A 2 = 0 p i 1 A 0 + p i A 1 + p i+1 A 2 = 0, i = 1, 2,... 40

Quasi-Birth-and-Death processes Solution: Matrix-geometric Probability vector p is given by p i = p 0 R i, i = 0, 1, 2,... Rate matrix R is minimal nonnegative solution of A 0 + RA 1 + R 2 A 2 = 0 R jk is expected time spent in (i + 1, k) before first return to level i, given initial state is (i, j) Vector p 0 follows from boundary equation and normalization p 0 (B 1 + RA 2 ) = 0 p 0 (I R) 1 e = 1 Rate matrix R can be determined by successive substitution from fixed point equation starting with R = 0 R = (A 0 + R 2 A 2 )A 1 1 41

Quasi-Birth-and-Death processes Solution: Matrix-analytic Let g jk be probability of first passage to level i 1 in (i 1, k) for initial state (i, j) at level i Matrix G = (g jk ) is is minimal nonnegative solution of A 2 + A 1 G + A 0 G 2 = 0 Then p i+1 A 2 = p i A 0 G Substituting in balance equation 0 = p i 1 A 0 + p i A 1 + p i+1 A 2 = p i 1 A 0 + p i (A 1 + A 0 G) Probability vector p is given by p i = p i 1 A 0 (A 1 + A 0 G) 1 = p i 1 R with R = A 0 (A 1 + A 0 G) 1 42

Quasi-Birth-and-Death processes Solution: Matrix-analytic Vector p 0 follows from boundary equation and normalization Note RA 2 = A 0 G p 0 (B 0 + RA 2 ) = p 0 (B 0 + A 0 G) = 0 p 0 (I R) 1 e = 1 Matrix G can be determined by successive substitution from fixed point equation starting with G = 0 G = (A 2 + A 0 G 2 )A 1 1 43

Quasi-Birth-and-Death processes Solution: Spectral expansion Seek solutions of balance equations p i 1 A 0 + p i A 1 + p i+1 A 2 = 0 of the form p i = y x i, i = 0, 1, 2,..., where y = (y(0),..., y(n)) is non-null vector and x < 1 Substitution and dividing by common powers of x yields y ( A 0 + xa 1 + x 2 ) A 2 = 0 Suppose determinant equation det(a 0 + xa 1 + x 2 A 2 ) = 0 has exactly n + 1 roots x 0,..., x n with x k < 1 Let y 0,..., y n be the corresponding non-null vectors y k ( A0 + x k A 1 + x 2 k A 2) = 0, k = 0,..., n 44

Quasi-Birth-and-Death processes Solution: Spectral expansion Set n p i = c k y k xk i k=0 Coefficients c 0,..., c k follow from boundary equation p 0 B 1 + p 1 A 2 = n c k (y k B 1 + y k A 2 x k ) = 0 k=0 and normalization 1 = i,j p(i, j) = n c k y k e 1 x k k=0 45

Quasi-Birth-and-Death processes Solution: Generating function Let P(z) = i=0 p i z i Multiply balance equations by z i and add all equations zp(z)a 0 + (P(z) p 0 )A 1 + z 1 (P(z) p 0 p 1 z)a 2 = 0 Solving this equation P(z)A(z) = p 0 (A 1 + z 1 A 2 ) + p 1 A 2 = p 0 (A 1 B 1 + z 1 A 2 ) where A(z) = za 0 + A 1 + z 1 A 2 46

Quasi-Birth-and-Death processes Solution: Generating function Suppose determinant equation det(a(z)) = 0 has exactly n + 1 roots z 0 = 1, z 1,..., z n with z k < 1 for k = 1,..., n Let v 0 = e, v 1,..., v n be non-null vectors satisfying Vector p 0 follows from A(z k )v k = 0 p 0 (A 1 B 1 + z 1 k A 2)v k = 0, k = 1,..., n P(1)A(1) = p 0 (A 1 B 1 + A 2 ), P(1)e = 1 P(1)A (1) (1)e = p 0 A 2 e 47

Quasi-Birth-and-Death processes: More structure A 0, A 1 and A 2 are upper triangular: A 0 (i, j) = 0 if j < i Implication: R is also upper triangular Example: A 0 = ( ) ( ) ( ) a0 a 1 b0 b, A 0 a 1 = 1 c0 c, A 2 0 b 2 = 1, 2 0 c 2 Then where R = ( ) r0 r 1 0 r 2 a 0 + r 0 b 0 + r0 2 c 0 = 0 a 2 + r 2 b 2 + r2 2 c 2 = 0 a 1 + r 0 b 1 + r 1 b 2 + r0 2 c 1 + (r 0 r 1 + r 1 r 2 )c 2 = 0 48

Quasi-Birth-and-Death processes: More structure Transition rates A 2 from level i to level i 1 have the form 0 A 2 = rp = r. ( ) r 0 p 0 r 0 p n p 0 p n =.. r n r n p 0 r n p n Example: Single server with setup A 2 = ( ) ( ) 0 0 0 (0 ) = 1 0 µ µ Example: Shortest queue with jockeying A 2 = ( ) ( 0 2µ 2µ = 0 0 0 ) (0 1 ) 49 Example: Single server serving n customer types for which µ 1 p 1 µ 1 p n µ 1 A 2 =.. =. ( ) p 1 p n µ n p 1 µ n p n µ n

Quasi-Birth-and-Death processes: More structure Transition rates A 2 from level i to level i 1 have the form 0 A 2 = rp = r. ( ) r 0 p 0 r 0 p n p 0 p n =.. r n r n p 0 r n p n Interpretation: Rate of jumping from (i, j) to level i 1 is r j, this jump is to (i 1, k) with probability p k Important: Jump probability does not depend on j This implies and p 0 p n G =.. = ep p 0 p n R = A 0 (A 1 + A 0 G) 1 = A 0 (A 1 + A 0 ep) 1 50

Quasi-Birth-and-Death processes: More structure Transition rates A 0 from level i to level i + 1 have the form 0 A 0 = rp = r. ( ) r 0 p 0 r 0 p n p 0 p n =.. r n r n p 0 r n p n Example: Shortest queue with jockeying A 0 = ( ) ( ) 0 0 0 (1 ) = 0 λ 0 λ 51

Quasi-Birth-and-Death processes: More structure Transition rates A 0 from level i to level i + 1 have the form 0 A 0 = rp = r. ( ) r 0 p 0 r 0 p n p 0 p n =.. r n r n p 0 r n p n Interpretation: Rate of jumping from (i, j) to level i + 1 is r j, this jump is to (i + 1, k) with probability p k This implies for some vector q Hence 0 R = rq = r. ( ) r 0 q 0 r 0 q n q 0 q n =.. r n r n q 0 r n q n R i = (qr) i 1 R, p i = p 0 R i = (qr) i 1 p 1, i = 1, 2,... Number x = qr is spectral radius of R which is unique root on (0, 1) of 52 det(a 0 + xa 1 + x 2 A 2 ) = 0

Quasi-Birth-and-Death processes: Linear rates Transition rates have linear form to neighboring phases Rate from (i, j) to (i + k, j + 1) is a k (n j) Rate from (i, j) to (i + k, j) is b k (n j) + c k j Rate from (i, j) to (i + k, j 1) with rate d k j So A 0 is of tri-diagonal form b 1 n a 1 n 0 d 1 b 1 (n 1) + c 1 a 1 (n 1) 0 d 1 2 b 1 (n 2) + c 1 2 A 0 =......... b 1 2 + c 1 (n 2) a 1 2 0 d 1 (n 1) b 1 + c 1 (n 1) a 1 0 d 1 n c 1 n 53

Quasi-Birth-and-Death processes: Linear rates Transition rate diagram i, j + 1 a 1 (n j) a 1 (n j) i, j i + 1, j d 0 j b 1 j + c 1 (n j) i, j 1 54

Quasi-Birth-and-Death processes: Example with linear rates n parallel servers serving two customer types 1 and 2 Type 1 customers require exponential service with rate µ 1, type 2 with rate µ 2 Customers arrive as Poisson stream with rate λ, fraction p 1 is of type 1 and p 2 of type 2 States (i, j) with i number of customers in the queue and j number of type 1 customers in service Transition rate diagram i, j + 1 (n j)µ 2 p 1 i 1, j i, j i + 1, j jµ 1 p 1 + (n j)µ 2 p 2 λ jµ 1 p 2 i, j 1 Non-zero coefficients a 1 = µ 2 p 1, b 1 = c 1 = λ/n, b 1 = µ 2 p 2, c 1 = µ 1 p 1, d 1 = µ 1 p 2 55

Quasi-Birth-and-Death processes: Example with linear rates n parallel servers and Poisson arrivals with rate λ which need exponential service times with rate µ Server switched off when queue is empty Idle server switched on when customer arrives which requires exponential setup time with rate θ States (i, j) with i number of customers in the queue and j number of active servers Transition rate diagram i, j + 1 (n j)θ i 1, j i, j i + 1, j jµ λ i, j 1 Non-zero coefficients a 0 = θ, b 1 = c 1 = λ/n, c 1 = µ 56

Quasi-Birth-and-Death processes: Example with linear rates Solution: Spectral expansion Seek solutions of balance equations of the form p i = y x i, i = 0, 1, 2,..., where y = (y(0),..., y(n)) is non-null vector and x < 1 Substitution and dividing by common powers of x yields following equations for j = 0,, n where 0 = (j + 1)D(x)y(j + 1) + ((n j)b(x) + jc(x))y(j) ((n j)(a(1) + B(1)) + j(c(1) + D(1)))xy(j) + (n j + 1)A(x)y(j 1) A(x) = a 1 +a 0 x +a 1 x 2, B(x) = b 1 +b 0 x +b 1 x 2, C(x) = c 1 +c 0 x +c 1 x 2, D(x) = d 1 +d 0 x +d 1 x 2 Solve equations for y by generating function 57 Y (z) = n y(j)z j i=0

Quasi-Birth-and-Death processes: Example with linear rates Solution: Spectral expansion Multiply equations by z j and add all equations D(x)Y (z) + nb(x)y (z) + (C(x) B(x))zY (z) (A(1) + B(1))nxY (z) +(A(1) + B(1) C(1) D(1))xzY (z) + na(x)zy (z) A(x)z 2 Y (z) = 0 This equation can be rewritten as Y (z) Y (z) = = n[a(x)z + B(x) (A(1) + B(1))x] A(x)z 2 ((A(1) + B(1) C(1) D(1))x + C(x) B(x))z D(x) E(x) z z 1 (x) + n E(x) z z 2 (x) where z 1 (x) and z 2 (x) are the roots of the denominator and E(x) satisfies 2E(x) n = n B(x) + C(x) x (A(1) + B(1) + C(1) + D(1)), F (x) 2 + 4A(x)D(x) 58 where F (x) = (A(1) + B(1) C(1) D(1))x + C(x) B(x)

Quasi-Birth-and-Death processes: Example with linear rates Solution: Spectral expansion Differential equation Y (z) Y (z) = E(x) z z 1 (x) + n E(x) z z 2 (x) General solution Y (z) = C(z z 1 (x)) E(x) (z z 2 (x)) n E(x) Key observation: Y (z) is a polynomial in z, so exponent E(x) = k for k = 0,..., n For each k = 0,..., n the equation 2k n = n B(x) + C(x) x (A(1) + B(1) + C(1) + D(1)), F (x) 2 + 4A(x)D(x) has a unique solution x = x k in the interval (0, 1) 59

Quasi-Birth-and-Death processes: Example with linear rates Solution: Spectral expansion For each k = 0,..., n the equation 2k n = n B(x) + C(x) x (A(1) + B(1) + C(1) + D(1)), F (x) 2 + 4A(x)D(x) has a unique solution x = x k in the interval (0, 1) 6 4 2 0 x 4 x 3 x 1 x 2 x 0 1 2 4 6 60

Two-dimensional Markov chain: Re-entrant line (1) µ 1 µ 2 µ 3 Two stations, station 1 with two servers, labeled 1 and 3 and station 2 with single server, labeled server 2 Queue i is served at exponential rate µ i Queue 1 has infinite supply and customers subsequently go through queue 1, 2 and 3 61

Two-dimensional Markov chain: Re-entrant line (1) j µ 2 µ 1 µ 1 µ 3 µ 2 µ 3 µ 1 i Markov chain with states (i, j) where i number in queue 2 and j number in queue 3 Global balance equations p(i, j)(µ 1 + µ 2 + µ 3 ) = p(i 1, j)µ 1 + p(i, j + 1)µ 3 + p(i + 1, j 1)µ 2, i > 0, j > 0 p(i, 0)(µ 1 + µ 2 ) = p(i 1, 0)µ 1 + p(i, 1)µ 3, i > 0 p(0, j)(µ 1 + µ 3 ) = p(0, j + 1)µ 3 + p(1, j 1)µ 2, j > 0 p(0, 0)µ 1 = p 0,1 µ 3 62

Two-dimensional Markov chain: Re-entrant line (1) j µ 2 µ 1 µ 1 µ 3 µ 2 µ 3 µ 1 i Substitution of trial solution p(i, j) = α i β j in global balance equations αβ(µ 1 + µ 2 + µ 3 ) = βµ 1 + αβ 2 µ 3 + α 2 µ 2 α(µ 1 + µ 2 ) = µ 1 + αβµ 3 β(µ 1 + µ 3 ) = β 2 µ 3 + αµ 2 63 Equations solved by α = µ 1 µ 2 and β = µ 1 µ 3 Normalization yields p(i, j) = (1 α)α i (1 β)β j

Two-dimensional Markov chain: Re-entrant line (1) Global balance equations p(i, j)(µ 1 + µ 2 + µ 3 ) = p(i 1, j)µ 1 + p(i, j + 1)µ 3 + p(i + 1, j 1)µ 2, i > 0, j > 0 Partial balance Rate out of (i, j) due to arrival in queue 2 is equal to rate into (i, j) due to departure from queue 3 p(i, j)µ 1 = p(i + 1, j)µ 3 Rate out of (i, j) due to departure from queue 2 is equal to rate into (i, j) due to arrival in queue 2 p(i, j)µ 2 = p(i 1, j)µ 1 Rate out of (i, j) due to departure from queue 3 is equal to rate into (i, j) due to arrival in queue p(i, j)µ 3 = p(i + 1, j 1)µ 2 64 Partial balance yields candidate solution ( ) j µ2 p(i, j) = p(i + j, 0) = µ 3 ( µ2 ) j ( ) i+j µ1 p(0, 0) = µ 3 µ 2 ( µ1 ) i ( ) j µ1 p(0, 0) µ 2 µ 3

Two-dimensional Markov chain: Re-entrant line (2) µ 1 µ 2 µ 3 Two stations, both stations with single server Queue i is served at exponential rate µ i Queue 1 has infinite supply and customers subsequently go through queue 1, 2 and 3 Customers in queue 3 have preemptive priority over customers in queue 1 (Last Buffer First Served) Assumption µ 1 1 + µ 1 3 > µ 1 2 65

Two-dimensional Markov chain: Re-entrant line (2) j µ 2 µ 3 µ 2 µ 3 µ 1 i Markov chain with states (i, j) where i number in queue 2 and j number in queue 3 Global balance equations p(i, j)(µ 2 + µ 3 ) = p(i, j + 1)µ 3 + p(i + 1, j 1)µ 2, i > 0, j > 0 p(i, 0)(µ 1 + µ 2 ) = p(i 1, 0)µ 1 + p(i, 1)µ 3, i > 0 p(0, j)µ 3 = p(0, j + 1)µ 3 + p(1, j 1)µ 2, j > 0 p(0, 0)µ 1 = p(0, 1)µ 3 66

Two-dimensional Markov chain: Re-entrant line (2) j µ 2 µ 3 µ 2 µ 3 µ 1 i Substitution of trial solution p(i, j) = α i β j in global balance equations for i > 0 β(µ 2 + µ 3 ) = β 2 µ 3 + αµ 2 α(µ 1 + µ 2 ) = µ 1 + αβµ 3 Equations solved by α = µ 1 µ 2 (1 β) and β is the root on (0, 1) of β 2 µ 3 β(µ 1 + µ 2 + µ 3 ) + µ 1 = 0 β = µ 1 + µ 2 + µ 3 (µ 1 + µ 2 + µ 3 ) 2 4µ 1 µ 3 2µ 3 67

Two-dimensional Markov chain: Re-entrant line (2) j µ 2 µ 3 µ 2 µ 3 µ 1 i Substitution of p(i, j) = α i β j for i > 0 in global balance equations for i = 0 p(0, j)µ 3 = p(0, j + 1)µ 3 + αβ j 1 µ 2, j > 0 68 Equations are solved by p(0, j) = cβ j 1 where c = µ 1 µ 3 Normalization yields p(i, j) = µ 3 µ 1 +µ 3 (1 α), i = 0, j = 0 µ 1 µ 1 +µ 3 (1 α)β j 1, i = 0, j > 0 µ 3 µ 1 +µ 3 (1 α)α i β j, i > 0, j 0

Two-dimensional Markov chain: Single server priority queue Single server serving type 1 and type 2 customers Type i customers arrive according to Poisson stream with rate λ i Type i customers require exponential service with rate µ i Type 1 customers have preemptive priority over type 2 customers 69

Two-dimensional Markov chain: Single server priority queue j λ 2 λ 2 λ 1 µ 1 λ 1 µ 2 λ 2 µ 1 λ 1 i Markov chain with states (i, j) where i number of type 1 and j number of type 2 Global balance equations p(i, j)(λ 1 + λ 2 + µ 1 ) = p(i 1, j)λ 1 + p(i + 1, j)µ 1 + p(i, j 1)λ 2, i > 0, j > 0 p(i, 0)(λ 1 + λ 2 + µ 1 ) = p(i 1, 0)λ 1 + p(i + 1, 0)µ 1, i > 0 p(0, j)(λ 1 + λ 2 + µ 2 ) = p(1, j)µ 1 + p(0, j 1)λ 2 + p(0, j + 1)µ 2, j > 0 p(0, 0)(λ 1 + λ 2 ) = p(0, 1)µ 2 + p(1, 0)µ 1 70 Note that p(0, 0) = 1 ρ 1 ρ 2 where ρ i = λ i µ i

Two-dimensional Markov chain: Single server priority queue Solution: Difference equation Balance equations for j = 0 are homogeneous second-order difference equation p(i, 0)(λ 1 + λ 2 + µ 1 ) = p(i 1, 0)λ 1 + p(i + 1, 0)µ 1, i > 0 General solution p(i, 0) = c 0,0 x i 0 + c 0,1x i 1, i 0 where 0 < x 0 < 1 < x 1 are roots of quadratic equation Convergence of p(i, 0) implies c 0,1 = 0 so x(λ 1 + λ 2 + µ 1 ) = λ 1 + x 2 µ 1 p(i, 0) = c 0,0 x i 0 = (1 ρ 1 ρ 2 )x i 0, i 0 71

Two-dimensional Markov chain: Single server priority queue Solution: Difference equation Balance equations for j = 1 are inhomogeneous second-order difference equation General solution where p(i, 1)(λ 1 + λ 2 + µ 1 ) = p(i 1, 1)λ 1 + p(i + 1, 1)µ 1 + p(i, 0)λ 2 = p(i 1, 1)λ 1 + p(i + 1, 1)µ 1 + c 0,0 x i 0 λ 2, i > 0 p(i, 1) = c 1,0 x0 i + c 1,1(i + 1)x ( ) 0 i ( ) i + 1 i + 1 = c 1,0 x0 i 0 + c 1,1 x0 i 1, i 0 c 1,1 = Coefficient c 1,0 follows from balance equation in (0, 0) λ 2 c 0,0 λ 1 + λ 2 2µ 1 x 0 72 c 1,0 = λ 1 + λ 2 µ 1 x 0 µ 2 c 0,0 c 1,1

Two-dimensional Markov chain: Single server priority queue Solution: Difference equation Repeating procedure for balance equations for j = 2, 3,... leads to p(i, j) = j k=0 ( ) i + j c j,k x0 i k, i, j 0 where coefficients c j,k can be determined recursively starting with c 0,0 = 1 ρ 1 ρ 2 73

Two-dimensional Markov chain: Single server priority queue Solution: Generating function Let P(x, y) = i=0 j=0 p(i, j)x i y j for x, y 1 Multiply balance equations by x i y j and add all equations with h 1 (x, y)p(x, y) = h 2 (x, y)p(0, y) + h 3 (x, y)p(0, 0) h 1 (x, y) = λ 1 xy(1 x) + λ 2 xy(1 y) µ 1 y(1 x) h 2 (x, y) = µ 1 y(1 x) + µ 2 x(1 y) h 3 (x, y) = µ 2 x(1 y) and P(0, 0) = p(0, 0) = 1 ρ 1 ρ 2 74

Two-dimensional Markov chain: Single server priority queue Solution: Generating function For every y 1 let x = ξ(y) be the unique root with x 1 of the quadratic equation Then 0 = h 1 (x, y) = λ 1 xy(1 x) + λ 2 xy(1 y) µ 1 y(1 x) x = ξ(y) = λ 1 + µ 1 + λ 2 (1 y) (λ 1 + µ 1 + λ 2 (1 y)) 2 4λ 1 µ 1 2λ 1 Substituting x = ξ(y) and P(0, 0) = 1 ρ 1 ρ 2 in h 1 (x, y)p(x, y) = h 2 (x, y)p(0, y) + h 3 (x, y)p(0, 0) gives P(0, y) = h 3(ξ(y), y)(1 ρ 1 ρ 2 ) h 2 (ξ(y), y) 75

Two-dimensional Markov chain: Single server priority queue Solution: Matrix geometric Partition states space into levels i = {(i, 0), (i, 1),...} Generator Q of the form where (λ = λ 1 + λ 2 ) 1 A 0 = λ λ 1 B 1 A 0 0 0... A 2 A 1 A 0 0 Q = 0 A 2 A 1 A 0 0 0 A 2 A 1.... λ µ 1 λ 2, A 1 = λ µ 1 λ 2, A 2 =......... λ λ 2 B 1 = µ 2 λ µ 2 λ 2......... µ 1 µ 1... 76

Two-dimensional Markov chain: Single server priority queue Solution: Matrix geometric A 0, A 1 and A 2 are upper triangular and have repeating structure Implication: R is also upper triangular and of the form r 0 r 1 r 2... R = r 0 r 1... r 0...... Elements can be determined recursively from matrix equation A 0 + RA 1 + R 2 A 2 = 0 µ 1 r 2 0 (λ + µ 1)r 0 + λ 1 = 0 k µ 1 r k l r l (λ + µ 1 )r k + λ 2 r k 1 = 0, k 1 l=0 77

Two-dimensional Markov chain: Single server priority queue Solution: Matrix geometric Vector p 0 = (p(0, 0), p(0, 1),...) follows from boundary equations yielding the recursion p(0, 1)µ 2 = p(0, 0)(λ µ 1 r 0 ) 0 = p 0 B 1 + p 1 A 2 = p 0 (B 1 + RA 2 ) j p(0, j + 1)µ 2 = (λ + µ 2 )p(0, j) λ 2 p(0, j 1) µ 1 p(0, j k)r k, j 1 starting with p(0, 0) = 1 ρ 1 ρ 2 Vector p i+1 = (p(0, 0), p(0, 1),...) can be obtained from p i+1 = p i R yielding k=0 p(i + 1, j) = j p(i, j k)r k k=0 78

Two-dimensional Markov chain: Single server priority queue Solution: Matrix analytic A 0, A 1 and A 2 are upper triangular and have repeating structure Implication: G is also upper triangular and of the form g 0 g 1 g 2... G = g 0 g 1... g 0...... g k is probability of first passage to level i 1 in (i 1, j + k) for initial state (i, j) at level i One-step analysis g 0 = g k = µ 1 λ + µ 1 + λ 1 λ + µ 1 g 2 0 λ 2 λ + µ 1 g k 1 + λ 1 λ + µ 1 k g l g k l, k 1 l=0 79

Two-dimensional Markov chain: Single server priority queue Solution: Matrix analytic From p i+1 A 2 = p i A 0 G follows p(i + 1, j)µ 1 = j p(i, j k)λ 1 g k, i, j, 0 k=0 80

Two-dimensional Markov chain: Single server priority queue Solution: Matrix analytic Vector p 0 follows from boundary equations 0 = p 0 B 1 + p 1 A 2 = p 0 (B 1 + A 0 G) B 1 + A 0 G is generator of Markov chain embedded on level 0 (0, j + k) λ 2 λ 1 g k (0, j) µ 2 81 Global balance for {(0, 0), (0, 1),..., (0, j)} j p(0, j + 1)µ 2 = p(0, j)λ 2 + p(0, j k)λ 1 (1 k=0 ) k g l l=0

Two-dimensional Markov chain: Shortest queue λ µ Two parallel exponential servers with rate µ, each with its own queue Customers arrive according to Poisson stream with rate λ and join shortest queue on arrival µ 82

Two-dimensional Markov chain: Shortest queue µ µ λ µ λ 2µ λ Markov chain with states (i, j) where i number in shortest queue and j difference between queues Global balance equations p(i, j)(λ + 2µ) = p(i 1, j + 1)λ + p(i, j + 1)µ + p(i + 1, j 1)µ, i > 0, j > 1 p(i, 1)(λ + 2µ) = p(i 1, 2)λ + p(i, 2)µ + p(i, 0)λ + p(i + 1, 0)2µ, i > 0 p(i, 0)(λ + 2µ) = p(i 1, 1)λ + p(i, 1)µ, i > 0 p(0, j)(λ + µ) = p(0, j + 1)µ + p(1, j 1)µ, j > 1 83

Two-dimensional Markov chain: Shortest queue µ µ λ µ λ 2µ λ Markov chain with states (i, j) where i number in shortest queue and j difference between queues Global balance equations p(i, j)(λ + 2µ) = p(i 1, j + 1)λ + p(i, j + 1)µ + p(i + 1, j 1)µ, i > 0, j > 1 p(i, 1)(λ + 2µ) = p(i 1, 2)λ + p(i, 2)µ λ +(p(i 1, 1)λ + p(i, 1)µ) λ + 2µ + (p(i, 1)λ + p(i + 1, 1)µ) 2µ λ + 2µ, i > 0 p(0, j)(λ + µ) = p(0, j + 1)µ + p(1, j 1)µ, j > 1 84

Intermezzo: Method of images φ=0 φ=0 p q What is the potential Φ in point p? 85

Intermezzo: Method of images φ=0 φ=0 p q q What is the potential Φ in point p? Add image charge q: Potential Φ in point p is Φ(p) = q r 1 q r 2 where r 1 and r 2 are distances from point p to charge q and q 86

Intermezzo: Method of images φ=0 φ=0 p q q q q For two planes at right angles Φ(p) = q r 1 q r 2 + q r 3 q r 4 87

Intermezzo: Method of images A B a c b Φ=Φa Φ=0 What is the potential Φ in any point p outside the spheres A and B? 88

Intermezzo: Method of images A B a α 0 c b Φ=Φ Φ=0 a What is the potential Φ in any point p outside the spheres A and B? Add image charge α 0 in center of A with α 0 = aφ a : Then Φ(p) α 0 r 0 where r 0 is distance from p to charge α 0 (correct if c = ) Potential of α 0 does not vanish on B 89

Intermezzo: Method of images A B a c α 0 β 0 b Φ=Φ Φ=0 a Place new image charge β 0 in B to compensate for this error Φ(p) α 0 r 0 + β 0 r 1 where r 0 and r 1 are distances from p to charges α 0 and β 0 Charge β 0 alters potential on A 90

Intermezzo: Method of images A B a c α 0 α 1 β 0 b Φ=Φ Φ=0 a Place new image charge β 0 in B to compensate for this error Φ(p) α 0 r 0 + β 0 r 1 where r 0 and r 1 are distances from p to charges α 0 and β 0 Charge β 0 alters potential on A Place new image charge α 1 in A to compensate for this error Φ(p) α 0 r 0 + β 0 r 1 + α 1 r 2 91

Intermezzo: Method of images A B a c α 0 α 1 β 0 b Φ=Φ Φ=0 a Potential problem of two spheres is solved by infinite sequence of image charges in A and B Φ(p) = α 0 r 0 + β 0 r 1 + α 1 r 2 + β 1 r 3 + 92

Two-dimensional Markov chain: Shortest queue µ µ λ µ λ 2µ λ Image charges are products α i β j satisfying the global balance equations p(i, j)(λ + 2µ) = p(i 1, j + 1)λ + p(i, j + 1)µ + p(i + 1, j 1)µ, i > 0, j > 1 Substituting α i β j in this equation and dividing by common powers gives the curve αβ(λ + 2µ) = β 2 λ + αβ 2 µ + α 2 µ 93

Two-dimensional Markov chain: Shortest queue Quadratic curve αβ(λ + 2µ) = β 2 λ + αβ 2 µ + α 2 µ 94

Two-dimensional Markov chain: Shortest queue Unique product form satisfying global balance equations for i > 0, j > 1 and i > 0, j = 1: p(i, j) c 0 α0 i βj 0 where α 0 = ρ 2, β 0 = ρ 2 /(2 + ρ) with ρ = λ/2µ Product form c 0 α0 i βj 0 violates global balance equations for i = 0 95

Two-dimensional Markov chain: Shortest queue Add new product form c 1 α i 1 βj 1 such that sum satisfies balance equations for i = 0 Then β 1 = β 0 and α 1 is companion root of α 0 and p(i, j) c 0 α i 0 βj 0 + c 1α i 1 βj 1 c 1 = α 1 β 0 α 0 β 0 c 0 96 Product form c 1 α1 i βj 1 violates global balance equations for j = 1

Two-dimensional Markov chain: Shortest queue Add new product form c 2 α i 2 βj 2 such that sum satisfies balance equations for j = 1 p(i, j) c 0 α i 0 βj 0 + c 1α i 1 βj 1 + c 2α i 2 βj 2 Then α 2 = α 1 and β 2 is companion root of β 1 and (with ρ = λ/(2µ)) c 2 = (ρ + α 1)/β 2 (1 + ρ) (ρ + α 1 )/β 1 (1 + ρ) c 1 97 Product form c 2 α2 i βj 2 violates global balance equations for i = 0

Two-dimensional Markov chain: Shortest queue Iterating yields infinite sequence p(i, j) = c 0 α i 0 βj 0 + c 1α i 1 βj 1 + c 2α i 2 βj 2 + c 3α i 3 βj 3 + 98

Two-dimensional Markov chain: Random walk in quarter plane q i,j Equilibrium probabilities can be expressed as infinite sequence of product forms p(i, j) = c 0 α i 0 βj 0 + c 1α i 1 βj 1 + c 2α i 2 βj 2 + 99

Two-dimensional Markov chain: Random walk in quarter plane q i,j Equilibrium probabilities can be expressed as infinite sequence of product forms p(i, j) = c 0 α0 i βj 0 + c 1α1 i βj 1 + c 2α2 i βj 2 + Provided there are no transitions to North, North-East and East q 0,1 = q 1,1 = q 1,0 = 0 100

Two-dimensional Markov chain: Random walk in quarter plane Condition q 0,1 = q 1,1 = q 1,0 = 0 implies that curve for α and β is of the above form This form guarantees that α k 0 and β k 0 as k 101

Multi-skilled system Customer types C = {a, b,...} Independent Poisson arrivals with rates λ c, c C Exponential service requirements with mean 1 J servers, S = {m 1,..., m J } Server m j works at rate µ mj Skill-based service: Server m i can serve customer types C(m j ) Type c customer can be served by servers S(c) Service discipline combination of FCFS and ALIS: Server picks longest waiting compatible customer Customer assigned to longest idle compatible server 102

Multi-skilled system: Example b a c m 1 m 3 m 2 λ a, λ b, λ c a,b a,c a m 1 m 2 µ m1 µ m2 m 3 µ m3 C = {a, b, c} and S = {m 1, m 2, m 3 } Skill-based service: C(m 1 ) = {a, b}, C(m 2 ) = {a, c}, C(m 3 ) = {a} S(a) = {m 1, m 2, m 3 }, S(b) = {m 1 }, S(c) = {m 2 } 103

Multi-skilled system: Markov chain Markov chain with states (M 1, n 1, M 2,..., M i, n i, M i+1,..., M J ) i busy servers M 1,..., M i J i idle servers M i+1,..., M J with increasing idle times M 1,..., M J is a permutation of m 1,..., m J n j number of waiting jobs between server M j and M j+1 104

Multi-skilled system: Examples of states a/b b b b a/c b/c b/c b/c b/c a a/b/c a/b/c m 1 m 2 m 3 (i) All servers busy, servers move left to right, arrivals come from the right a a/b b b b * a/b b b b * * m 3 m 1 m 2 m 1 m 2 m 3 (ii) Two servers busy, no possible customers between m 3 and m 1 (iii) Two servers are idle, m 3 has been idle longest time (i) (m 1, 3, m 2, 4, m 3, 2) (ii) (m 3, 0, m 1, 3, m 2 ) (iii) (m 1, 3, m 2, m 3 ) 105

Multi-skilled system: N-system b a m 1 m 2 λ a, λ b a, b a m 1 m 2 µ 1 µ 2 Markov chain with states (m 1, n 1, m 2, n 2 ) with n 1, n 2 0 (m 2, 0, m 1, n 2 ) with n 2 0 (m 1, n 1, m 2 ) with n 1 0 (m 2, 0, m 1 ) (m 1, m 2 ) and (m 2, m 1 ) (empty system) Assume λ b < µ 1 and λ a + λ b < µ 1 + µ 2 106

Multi-skilled system: N-system n 2 λ a + λ b µ 1 (m 1, n 1, m 2, n 2 ) µ 2 p 1 µ 2 p 2 State transition from (m 1, n 1, m 2, n 2 ) to (m 1, n 1 + j 1, m 2, n 2 j) with rate µ 2 p j where p j = (1 γ )γ j 1 λ b γ = λ a + λ b n 1 107

Multi-skilled system: N-system n 2 λ a + λ b µ 1 (m 1, n 1, m 2, n 2 ) µ 2 p 1 µ 2 p 2 n 1 Global balance equation in (m 1, n 1, m 2, n 2 ) with n 2 > 0 (λ a + λ b + µ 1 + µ 2 )p(m 1, n 1, m 2, n 2 ) = (λ a + λ b )p(m 1, n 1, m 2, n 2 1) + µ 1 p(m 1, n 1 + 1, m 2, n 2 ) n 1 +µ 2 (1 γ ) γ j p(m 1, n 1 j, m 2, n 2 + j + 1) j=0 +µ 2 (1 γ )γ n 1 p(m 2, 0, m 1, n 1 + 1 + n 2 ) 108

Multi-skilled system: Partial balance (i) Rate out of state (M 1, n 1, M 2,..., M i, n i, M i+1,..., M J ) due to an arrival activating a server equals rate into that state due to a departure deactivating M i+1 (ii) Rate out of state (M 1, n 1, M 2,..., M i, n i, M i+1,..., M J ) due to an arrival that is queueing equals rate into that state due to a departure from a server staying active (iii) Rate out of state (M 1, n 1, M 2,..., M i, 0, M i+1,..., M J ) with n i = 0 due to a departure equals rate into that state due to an arrival activating M i (iv) Rate out of state (M 1, n 1, M 2,..., M i, n i, M i+1,..., M J ) with n i > 0 due to a departure equals rate into that state due to an arrival that is queueing 109

Multi-skilled system: N-system (iv) Rate out of state (m 1, n 1, m 2, n 2 ) with n i > 0 due to departure equals rate into that state due to an arrival that is queueing (λ a + λ b + µ 1 + µ 2 )p(m 1, n 1, m 2, n 2 ) = (λ a + λ b )p(m 1, n 1, m 2, n 2 1) + µ 1 p(m 1, n 1 + 1, m 2, n 2 ) n 1 +µ 2 (1 γ ) γ j p(m 1, n 1 j, m 2, n 2 + j + 1) j=0 +µ 2 (1 γ )γ n 1 p(m 2, 0, m 1, n 1 + 1 + n 2 ) 110

Multi-skilled system: N-system (ii) Rate out of state (m 1, n 1, m 2, n 2 ) due to an arrival that is queueing equals rate into that state due to a departure from a server staying active (λ a + λ b + µ 1 + µ 2 )p(m 1, n 1, m 2, n 2 ) = (λ a + λ b )p(m 1, n 1, m 2, n 2 1) + µ 1 p(m 1, n 1 + 1, m 2, n 2 ) n 1 +µ 2 (1 γ ) γ j p(m 1, n 1 j, m 2, n 2 + j + 1) j=0 +µ 2 (1 γ )γ n 1 p(m 2, 0, m 1, n 1 + 1 + n 2 ) 111

Multi-skilled system: N-system Partial balance dictates solution p(m 1, n 1, m 2, n 2 ) (iv) = p(m 1, n 1, m 2, n 2 1) λ a + λ b µ 1 + µ ( ) 2 (iv) λa + λ n2 b = p(m 1, n 1, m 2, 0) µ 1 + µ ( 2 (iii) λ a λa + λ b = p(m 1, n 1, m 2 ) µ 1 + µ 2 µ 1 + µ 2 (iv) = p(m 1, 0, m 2 ) ( ) n1 λb λa µ 1 (i) = p(m 1, m 2 ) λ a + λ b µ 1 µ 1 + µ 2 ( ) n1 λb λa µ 1 ) n2 ( λa + λ b µ 1 + µ 2 ) n2 µ 1 + µ ( 2 λa + λ b µ 1 + µ 2 ) n2 112

Multi-skilled system: N-system Candidate solution p(m 1, n 1, m 2, n 2 ) = p(m 1, m 2 ) λ a + λ b µ 1 ( ) n1 λb λa µ 1 p(m 2, 0, m 1, n 2 ) = p(m 1, m 2 ) λ ( a λa + λ b µ 2 µ 1 + µ 2 p(m 1, n 1, m 2 ) = p(m 1, m 2 ) λ a + λ b µ 1 p(m 2, 0, m 1 ) = p(m 1, m 2 ) λ a µ 2 λ a p(m2, m1) = p(m 1, m 2 ) λ a + λ b ( λb µ 1 ) n2 +1 ) n1 µ 1 + µ 2 ( λa + λ b µ 1 + µ 2 ) n2 113

Multi-skilled system: N-system Set C = p(m 1, m 2 )(λ a + λ b )λ a then candidate solution p(m 1, n 1, m 2, n 2 ) = C λn 1 b (λ a + λ b ) n 2 µ n 1+1 (µ 1 1 + µ 2 ) n 2+1 p(m 2, 0, m 1, n 2 ) = C 1 (λ a + λ b ) n 2 µ 2 (µ 1 + µ 2 ) n 2+1 p(m 1, n 1, m 2 ) = C λn 1 b 1 µ n 1+1 λ 1 a p(m 2, 0, m 1 ) = C 1 1 µ 2 λ a + λ b 1 1 p(m2, m1) = C λ a + λ b λ a + λ b 1 1 p(m1, m2) = C λ a + λ b λ a Checking partial balance equations proves that this is solution to the global balance equations 114

Multi-skilled system C is set of customer types {a, b,...} S is set of servers {m 1, m 1,..., m J } λ X = c X λ c where X C µ Y = M Y µ M where Y S S(X) is set of server types that can handle customers in X C(Y) is set of customer types that can be handled by servers in Y U(Y) is set of customer types that can be uniquely handled by servers in Y Assume (stability) λ U(Y) < µ Y for every subset of servers Y S 115

Multi-skilled system Solution to the global balance equations p(m 1, n 1,..., M i, n i, M i+1,..., M J ) = C i j=1 λ n j U({M 1,...,M j }) µ n j+1 {M 1,...,M j } J j=i+1 λ 1 C({M j,...,m J }) where C is a normalizing constant Proof by checking partial balance equations 116

Multi-skilled system: Alternative Markov chain Markov chain with states (C 1,..., C L, M 1,..., M K ) L customers waiting for service C 1,..., C L with decreasing waiting times (L 0) K idle servers M 1,..., M K with decreasing idle times (K J) Note that L = n 1 +... n i and M 1,..., M K correspond to M J,..., M i+1 Solution to the global balance equations p(c 1,..., C L, M 1,..., M K ) = C where C is a normalizing constant Proof by checking partial balance equations Note that L λ Cl µ S({C1,...,C l=1 l }) K µ Mk λ C({M1,...,M k=1 k }) 117 p(c 1,..., C L all servers busy) L λ Cl µ S({C1,...,C l=1 l })

FCFS matching queue Independent Poisson arrivals of customers with rates λ c, c C Independent Poisson arrivals of servers with rates µ mj, m j S Arriving server m j scans queue of customers Matches with the longest waiting compatible customer and the two leave the system immediately If no match is possible he leaves immediately without customer Note that this system is equivalent with a redundancy service system 118

FCFS matching queue: Markov chain Markov chain with states (C 1,..., C L ) L customers waiting for service C 1,..., C L with decreasing waiting times (L 0) Solution to the global balance equations p(c 1,..., C L ) = C L λ Cl µ S({C1,...,C l=1 l }) where C is a normalizing constant 119