Structured Markov Chains
|
|
- Giles Hart
- 5 years ago
- Views:
Transcription
1 Structured Markov Chains Ivo Adan and Johan van Leeuwaarden Where innovation starts
2 Book on Analysis of structured Markov processes (arxiv: ) I Basic methods Basic Markov processes Advanced Markov processes Queues and transforms II Basic processes Birth-and-death processes Queueing networks Quasi-birth-and-death-processes Processes that are quasi-skip-free in one direction III Advanced processes Single-server priority systems Gated single-server systems Production systems Join the shortest queue 2
3 Markov chains in discrete time A stochastic process {X n, n = 0, 1, 2,...} with state space {0, 1, 2,...} is a Markov chain if for all i 0,..., i n+1 and n 0 Assumptions: P(X n+1 = i n+1 X 0 = i 0, X 1 = i 1,..., X n = i n ) = P(X n+1 = i n+1 X n = i n ) Time homogeneous: P(X n+1 = j X n = i) = p ij Irreducible: all states are reachable Aperiodic Equilibrium distribution p 0, p 1,... satisfy the equilibrium (or balance) equations p i = p j p ji, i = 0, 1,... j=0 If equilibrium distribution exists then it is equal to the limiting distribution lim n P(X n = i) = p i Vector-matrix notation: p = pp where p = (p 0, p 1,...) and transition probability matrix P = (p ij ) 3
4 Markov chains in continuous time A stochastic process {X(t), t 0} with state space {0, 1, 2,...} is a Markov chain if for all i, j, x(u) and s, t 0 Assumptions: P(X(t + s) = j X(u) = x(u), 0 u < s, X(s) = i) = P(X(t + s) = j X(s) = i) Time homogeneous: P(X(t + s) = j X(s) = i) = p ij (t) Irreducible: all states are reachable Transition rates q ij = lim h 0 p ij (h) h where we let q ii = 0 and q i = j=0 q ij Equilibrium distribution p 0, p 1,... satisfy the equilibrium (or balance) equations p i q i = p j q ji, i = 0, 1,... j=0 If equilibrium distribution exists then it is equal to the limiting distribution lim t P(X(t) = i) = p i 4 Vector-matrix notation: 0 = pq where p = (p 0, p 1,...) and generator Q with Q ij = q ij, i = j, Q ii = q i
5 Balance equations Global balance equations: or for subset A and its complement A c p i q i = p j q ji j=0 p i q ij = p j q ji j A c i A Partial balance equations: for subsets A and B Detailed (or local) balance equations: p i q ij = j A j B p j q ji p i q ij = p j q ji 5
6 Balance equations Global balance equations: Partial balance equations: Detailed (or local) balance equations: 6
7 Transforms Generating function of non-negative discrete random variable X with probability distribution p i = P(X = i) P X (z) = E(z X ) = p i z i, z 1 i=0 Laplace-Stieltjes transform a non-negative (continuous) random variable X with distribution function F X (x) X(s) = E(e sx ) = 0 e sx df X (x), Re(s) 0 Properties: Probability distribution p i is uniquely determined by generating function P X (z) Distribution function F X (x) is uniquely determined by Laplace-Stieltjes X(s) E(X(X 1) (X n + 1)) = P (n) X (1) and E(X n ) = ( 1) n X (n) (0) where (n) is nth derivative 7
8 Transforms: Example Markov chain {X n, n = 0, 1, 2,...} with state space {0, 1, 2,..., n + 1} and transition probabilities p ij State n + 1 is single absorbing state p 12 = 1 2 p 23 = 1 3 p 11 = p 33 = 1 p 21 = 2 3 Random variable N i is number of visits to state i before absorption into n + 1 Time spent in state i during the kth visit is equal to α k 1 i where 0 < α i < 1 (learning curve) Random variable T is total time spent in states {1,..., n} before absorption in n + 1 What is E(T X 0 = 1)? 8
9 Transforms: Example Note so T = n ( ) 1 + α i + + α N i 1 i = i=1 E(T X 0 = 1) = n i=1 n i=1 1 α N i i 1 α i 1 E(α N i i X 0 = 1) 1 α i Introduce for i = 1,..., n the generating function P i (z) = E(z N 1 1 z N n n X 0 = i), z 1 1,..., z n 1, where z = (z 1,..., z n ) so P 1 (1,..., 1, α i, 1,..., 1) = E(α N i i X 0 = 1) 9
10 Transforms: Example One-step analysis: P i (z) = E(z N 1 1 z N n n X 0 = i) n+1 = p ij E(z N 1 1 z 1+N i i = = j=1 n+1 j=1 n+1 j=1 z i p ij E(z N 1 1 z N i i z i p ij P j (z), where P n+1 (z) = 1 (since N 1 = = N n = 0 if X 0 = n + 1) z N n n X 0 = j) z N n n X 0 = j) i = 1,..., n Hence P 1 (1,..., 1, α i, 1,..., 1) can be solved from above equations with z = (1,..., 1, α i, 1,..., 1) 10
11 Finite Markov chains (no structure) Markov process {X(t), t 0} with state space {0, 1,..., N} and transition rates q ij Global balance equations p i N q ij = N p j q ji, i = 0, 1,..., N j=0 j=0 11
12 Finite Markov chains (no structure) N q in q Nj q ij i j Numerical solution through embedding: Remove state N by embedding Markov chain on {0, 1,..., N 1} with transition rates q ij = q ij + q in Equilibrium probability p N follows from global balance q Nj q N0 + + q NN 1, i, j = 0,..., N 1 N 1 p N j=0 q Nj = N 1 j=0 p j q jn Repeat this for N 1, N 2,..., 1 and finally normalize p 0,..., p N 12 This is Gaussian elimination with multiplication, division and addition of positive numbers
13 Infinite Markov chains (with structure): Example of impatient customers θ λ µ Single server Poisson arrivals with rate λ Exponential service times with rate µ Exponential patience with rate θ: customer leaves if service does not start before patience expires Markov chain with states i where i is number in system Transition rates q i,i 1 = µ + (i 1)θ and q i 1,i = λ λ λ 0 1 i 1 i µ µ + (i 1)θ 13
14 Infinite Markov chains (with structure): Birth-and-Death processes λ 0 λ i i 1 i µ 1 µ i Markov chain on {0, 1,...} with transitions to neighboring states with rates q i,i 1 = µ i and q i 1,i = λ i 1 Global balance p 0 λ 0 = p 1 µ 1 p i (λ i + µ i ) = p i 1 λ i 1 + p i+1 µ i+1, i = 1, 2,... Global balance for {0, 1,..., i 1} p i 1 λ i 1 = p i µ i, i = 1, 2,... 14
15 Infinite Markov chains (with structure): Birth-and-Death processes λ 0 λ i i 1 i µ 1 µ i Iterating yields Normalization Assumption Example of impatient customers p i = p i 1 λ i 1 µ i = p i 2 λ i 2 λ i 1 µ i 1 µ i = = p 0 λ 0 λ i 1 µ 1... µ i p 0 = i=0 1 i=0 ij=1 λ j 1 µ j i j=1 λ j 1 µ j < 15 p i = p 0 µ(µ + θ) (µ + (i 1)θ) λ i
16 Infinite Markov chains (with more structure): Constant birth and death rates λ λ 0 1 i 1 i µ µ Global balance equations p 0 λ = p 1 µ p i (λ + µ) = p i 1 λ + p i+1 µ, i = 1, 2,... Assume λ µ < 0 and (without loss of generality) λ + µ = 1 Generator Q of the form λ λ µ (λ + µ) λ 0 Q = 0 µ (λ + µ) λ 0 0 µ (λ + µ)
17 Infinite Markov chains (with more structure): Constant birth and death rates λ λ 0 1 i 1 i µ µ Solution: Generating function P(z) = i=1 p i z i Multiply balance equations by z i and add all equations Solving this equation where ρ = λ/µ P(z) = Normalization P(1) = 1 so p 0 = 1 ρ and P(z)(λ + µ) p 0 µ = P(z)λz + (P(z) p 0 )µz 1 p 0 µ(1 z 1 ) (λ + µ) λz µz 1 = p 0µ(1 z 1 ) (1 z 1 )(µ λz) = p 0 1 ρz 17 P(z) = 1 ρ 1 ρz = (1 ρ)ρ i z i i=0
18 Infinite Markov chains (with more structure): Constant birth and death rates λ λ 0 1 i 1 i µ µ Solution: Geometric Ratio p i+1 /p i is expected time spent in i + 1 before first return to i, given initial state is i Transition structure implies that ratio p i+1 /p i does not depend on i p i+1 p i = α Hence p i = p 0 α i and substitution in global balance equation yields α i (λ + µ) = α i 1 λ + α i+1 µ Dividing by common power α i 1 α(λ + µ) = λ + α 2 µ This leads to α = λ µ = ρ 18
19 Infinite Markov chains (with more structure): Constant birth and death rates λ λ 0 1 i 1 i µ µ Solution: Difference equation Balance equations are second-order difference equation with constant coefficients Solution is linear combination of powers p i = α i Substitution of p i = α i in balance equation yields with roots α 1 = ρ and α 2 = 1 General solution where c 1 and c 2 are constants α(λ + µ) = λ + α 2 µ p i = c 1 α i 1 + c 2α i 2 = c 1ρ i + c 2 Convergence of p i implies c 2 = 0 and normalization yields c 1 = 1 ρ 19
20 Infinite Markov chains: M/G/1 structure b 0 a 1 b i 1 i a 0 a 2 a 3 i + 1 i + 2 Transition probabilities P = (p ij ) P = b 0 b 1 b 2 b 3 a 0 a 1 a 2 a 3 0 a 0 a 1 a a 0 a Example: Number in system just after departure in M/G/1 queue Let A(z) = i=0 a i z i and B(z) = i=0 b i z i and assume mean step size = A (1) (1) 1 < 0 Global balance equations p i = p i+1 a 0 + p i a p 1 a i + p 0 b i, i = 0, 1, 2,... 20
21 Infinite Markov chains: M/G/1 structure b 0 a 1 b i 1 i a 0 a 2 a 3 i + 1 i + 2 Solution: Recursion Global balance for {0, 1,..., i 1} p 0 b i + p 1 ā i p i 1 ā 1 = p i a 0, i = 1, 2,... where b i = b i, ā i = j=i j=i a i Recursively calculate p 1, p 2,... starting with p 0 p 0 follows from (mean displacement is 0 in equilibrium) p 0 B (1) (1) + (1 p 0 )(A (1) (1) 1) = 0 21
22 Infinite Markov chains: M/G/1 structure b 0 a 1 b i 1 i a 0 a 2 a 3 i + 1 i + 2 Solution: Generating function Let P(z) = i=0 p i z i, A(z) = i=0 a i z i, B(z) = i=0 b i z i Multiply balance equations by z i and add all equations P(z) = z 1 (P(z) p 0 )A(z) + p 0 B(z) Solving this equation Normalization P(1) = 1 so P(z) = p 0(B(z) z 1 A(z)) 1 z 1 A(z) p 0 = 1 A (1) (1) B (1) (1) + 1 A (1) (1) = p 0(zB(z) A(z)) z A(z) 22
23 Infinite Markov chains: G/M/1 structure a 1 b i a i 0 1 i 2 i 1 i a 3 a 2 a 0 i + 1 Transition probabilities P = (p ij ) P = b 0 a b 1 a 1 a 0 0 b 2 a 2 a 1 a 0 b 3 a 3 a 2 a 1...., b i = 1 (a a i ) = j=i+1 a i Example: Number in system just before arrival in G/M/1 queue Global balance equations 23 p i = p i 1 a 0 + p i a 1 + p i+1 a 2 + = p i 1+j a j, i = 1, 2,... j=0
24 Infinite Markov chains: G/M/1 structure a 1 b i a i 0 1 i 2 i 1 i a 3 a 2 a 0 i + 1 Solution: Geometric Ratio p i+1 /p i is expected time spent in i + 1 before first return to i, given initial state is i Transition structure implies that ratio p i+1 /p i does not depend on i p i+1 p i = α Hence p i = p 0 α i and substitution in global balance equation yields α = α j a j = A(α) j=0 24 α is the unique root on (0, 1) of α = A(α) provided mean step size 1 A (1) (1) < 0
25 Examples of Quasi-Birth-and-Death processes: Single server with setup Single server Poisson arrivals with rate λ wich need exponential service times with rate µ Server switched off when system is empty Server switched on when first customer arrives which requires exponential setup time with rate θ 25
26 Examples of Quasi-Birth-and-Death processes: Single server with setup Markov chain with states (i, j) with i number in system and j status of server: 0 means off, 1 means on Transition rate diagram λ λ λ 0, 1 1, 1 2, 1 i, 1 µ µ µ θ θ θ λ λ λ 0, 0 1, 0 2, 0 i, 0 Global balance equations p(0, 0)λ = p(1, 1)µ p(0, 1)λ = 0 p(i, 0)(λ + θ) = p(i 1, 0)λ p(i, 1)(λ + µ) = p(i 1, 1)λ + p(i, 0)θ + p(i + 1, 1)µ, i = 1, 2,... 26
27 Examples of Quasi-Birth-and-Death processes: Single server with setup Order states as (0, 0), (0, 1), (1, 0), (1, 1), (2, 0), (2, 1),... and partition into levels i = {(i, 0), (i, 1)} Generator Q of the form where B 1 = ( ) λ 0, B 0 λ 2 = ( ) 0 0, A µ 0 0 = Let p = (p 0, p 1,...) and p i = (p(i, 0), p(i, 1)) B 1 A B 2 A 1 A 0 0 Q = 0 A 2 A 1 A A 2 A ( ) ( ) λ 0 (λ + θ) θ, A 0 λ 1 =, A 0 (λ + µ) 2 = p 0 B 1 + p 1 B 2 = 0, p i 1 A 0 + p i A 1 + p i+1 A 2 = 0, i = 1, 2,... ( ) µ 27
28 Examples of Quasi-Birth-and-Death processes: Single server with setup Solution: Matrix-geometric Global balance for level 0, 1,..., i p(i + 1, 1)µ = (p(i, 0) + p(i, 1))λ or in matrix form ( ) 0 λ with A 3 = 0 λ Substituting in balance equation p i+1 A 2 = p i A 3 28 Hence with 0 = p i 1 A 0 + p i A 1 + p i+1 A 2 = p i 1 A 0 + p i (A 1 + A 3 ) p i = p i 1 A 0 (A 1 + A 3 ) 1 = p i 1 R R = A 0 (A 1 + A 3 ) 1 = ( ) λ/(λ + θ) λ/µ 0 λ/µ
29 Examples of Quasi-Birth-and-Death processes: Single server with setup Solution: Matrix-geometric Iterating yields p i = p 0 R i, i = 0, 1, 2,... Vector p 0 follows from boundary equation p 0 B 1 + p 1 B 2 = p 0 (B 1 + RB 2 ) = 0 and normalization 1 = p(i, j) = i,j where I is identity matrix and e all-one vector p i e = p 0 R i e = p 0 (I R) 1 e i=0 i=0 29
30 Examples of Quasi-Birth-and-Death processes: Single server with setup Solution: Matrix-analytic Let g jk be probability of first passage to level i 1 in (i 1, k) for initial state (i, j) at level i ( ) 0 1 G = (g jk ) = 0 1 Then Substituting in balance equation p i+1 A 2 = p i A 0 G 0 = p i 1 A 0 + p i A 1 + p i+1 A 2 = p i 1 A 0 + p i (A 1 + A 0 G) Hence p i = p i 1 A 0 (A 1 + A 0 G) 1 = p i 1 R 30
31 Examples of Quasi-Birth-and-Death processes: Single server with setup Solution: Spectral expansion Seek solutions of balance equations p i 1 A 0 + p i A 1 + p i+1 A 2 = 0 of the form p i = y x i, i = 0, 1, 2,..., where y = (y(0), y(1)) is non-null vector and x < 1 Substitution and dividing by common powers of x yields y ( A 0 + xa 1 + x 2 ) A 2 = 0 Desired values of x are roots inside the unit circle of det(a 0 + xa 1 + x 2 A 2 ) = (λ (λ + θ)x)(µx λ)(x 1) = 0 31
32 Examples of Quasi-Birth-and-Death processes: Single server with setup Solution: Spectral expansion Roots with corresponding non-null vectors Set x 1 = λ λ + θ, x 2 = λ µ θx 1 y 1 = (1, λ (λ + µ)x 1 + µx1 2 ), y 2 = (0, 1) p i = c 1 y 1 x i 1 + c 2y 2 x i 2 where coefficients c 1 and c 2 follow from boundary equation p 0 B 1 + p 1 B 2 = 0 and normalization 1 = i,j p(i, j) = c 1y 1 e 1 x 1 + c 2y 2 e 1 x 2 32
33 Examples of Quasi-Birth-and-Death processes: Single server with setup Solution: Generating function Let P(z) = i=0 p i z i Multiply balance equations by z i and add all equations zp(z)a 0 + (P(z) p 0 )A 1 + z 1 (P(z) p 0 p 1 z)a 2 = 0 Solving this equation P(z)A(z) = p 0 (A 1 + z 1 A 2 ) + p 1 A 2 where A(z) = za 0 + A 1 + z 1 A 2 = ( λz (λ + θ) θ ) 0 λz (λ + µ) + µz 1 Vectors p 0, p 1 and P(1) follow from p 0 B 1 + p 1 B 2 = 0 P(1)A(1) = p 0 (A 1 + A 2 ) + p 1 A 2, P(1)e = 1 P(1)A (1) (1)e = p 0 A 2 e 33
34 Examples of Quasi-Birth-and-Death processes: Shortest queue with jockeying Two parallel exponential servers with rate µ, each with its own queue Customers arrive according to Poisson stream with rate λ and join shortest queue on arrival Jockeying: If difference in queues exceeds 1, then a customer jumps from longest to shortest queue 34
35 Examples of Quasi-Birth-and-Death processes: Shortest queue with jockeying Markov chain with states (i, j) with i number in shortest queue and j difference between queues Transition rate diagram 0, 1 1, 1 2µ i, 1 i+1, 1 λ µ λ 0, 0 1, 0 2µ λ λ i, 0 i+1, 0 35
36 Examples of Quasi-Birth-and-Death processes: Shortest queue with jockeying Partition into levels i = {(i, 0), (i, 1)} for i = 0, 1,... Generator Q of the form where B 1 = ( ) λ λ, A µ (λ + µ) 0 = Let p = (p 0, p 1,...) and p i = (p(i, 0), p(i, 1)) B 1 A A 2 A 1 A 0 0 Q = 0 A 2 A 1 A A 2 A ( ) ( ) 0 0 (λ + 2µ) λ, A λ 0 1 =, A 2µ (λ + 2µ) 2 = p 0 B 1 + p 1 A 2 = 0 p i 1 A 0 + p i A 1 + p i+1 A 2 = 0, i = 1, 2,... ( ) 0 2µ
37 Examples of Quasi-Birth-and-Death processes: Single server serving n customer types Single server serving n types of customers, numbered 1,..., n Type i customers arrive according to Poisson stream with rate λ i Type i customers require exponential service with rate µ i 37
38 Examples of Quasi-Birth-and-Death processes: Single server serving n customer types Markov chain with states (i, j) where i number waiting in the queue and j is customer type in service State (0, 0) is empty state Transition rate diagram where λ = λ λ n and p i = λ i /λ λ λ 0, n µ n p n 1, n µ i, n n p n. λ µ n p 1 µ n p 1 0, 1 1, 1 i, 1 λ 1 µ 1 µ 1 p 1 µ 1 p 1 0, 0 38
39 Examples of Quasi-Birth-and-Death processes: Single server serving n customer types Partition into level 0 = {(0, 0),..., (0, n)} and levels i = {(i, 1),..., (i, n)} for i = 1, 2,... Generator Q of the form where B 1 B B 2 A 1 A 0 0 Q = 0 A 2 A 1 A A 2 A λ 0 (λ + µ 1 ) 0 µ 1 p 1 µ 1 p n A 0 =..., A 1 =..., A 2 =.. 0 λ 0 (λ + µ n ) µ n p 1 µ n p n Let p = (p 0, p 1,...), p 0 = (p(0, 0),..., p(0, n)) and p i = (p(i, 1),..., p(i, n)) 39 p 0 B 1 + p 1 B 2 = 0 p 0 B 0 + p 1 A 1 + p 2 A 2 = 0 p i 1 A 0 + p i A 1 + p i+1 A 2 = 0, i = 2, 3,...
40 Quasi-Birth-and-Death processes Markov chain with states (0, 0), (0, 1),..., (0, n), (1, 0), (1, 1),..., (1, n), (2, 0),... Partition into levels i = {(i, 0),..., (i, n)} for i = 0, 1,... Generator Q of the form where B 1, A 0, A 1 and A 2 are n + 1 n + 1 matrices Let p = (p 0, p 1,...) and p i = (p(i, 0), p(i, 1),..., p(i, n)) B 1 A A 2 A 1 A 0 0 Q = 0 A 2 A 1 A A 2 A p 0 B 0 + p 1 A 2 = 0 p i 1 A 0 + p i A 1 + p i+1 A 2 = 0, i = 1, 2,... 40
41 Quasi-Birth-and-Death processes Solution: Matrix-geometric Probability vector p is given by p i = p 0 R i, i = 0, 1, 2,... Rate matrix R is minimal nonnegative solution of A 0 + RA 1 + R 2 A 2 = 0 R jk is expected time spent in (i + 1, k) before first return to level i, given initial state is (i, j) Vector p 0 follows from boundary equation and normalization p 0 (B 1 + RA 2 ) = 0 p 0 (I R) 1 e = 1 Rate matrix R can be determined by successive substitution from fixed point equation starting with R = 0 R = (A 0 + R 2 A 2 )A
42 Quasi-Birth-and-Death processes Solution: Matrix-analytic Let g jk be probability of first passage to level i 1 in (i 1, k) for initial state (i, j) at level i Matrix G = (g jk ) is is minimal nonnegative solution of A 2 + A 1 G + A 0 G 2 = 0 Then p i+1 A 2 = p i A 0 G Substituting in balance equation 0 = p i 1 A 0 + p i A 1 + p i+1 A 2 = p i 1 A 0 + p i (A 1 + A 0 G) Probability vector p is given by p i = p i 1 A 0 (A 1 + A 0 G) 1 = p i 1 R with R = A 0 (A 1 + A 0 G) 1 42
43 Quasi-Birth-and-Death processes Solution: Matrix-analytic Vector p 0 follows from boundary equation and normalization Note RA 2 = A 0 G p 0 (B 0 + RA 2 ) = p 0 (B 0 + A 0 G) = 0 p 0 (I R) 1 e = 1 Matrix G can be determined by successive substitution from fixed point equation starting with G = 0 G = (A 2 + A 0 G 2 )A
44 Quasi-Birth-and-Death processes Solution: Spectral expansion Seek solutions of balance equations p i 1 A 0 + p i A 1 + p i+1 A 2 = 0 of the form p i = y x i, i = 0, 1, 2,..., where y = (y(0),..., y(n)) is non-null vector and x < 1 Substitution and dividing by common powers of x yields y ( A 0 + xa 1 + x 2 ) A 2 = 0 Suppose determinant equation det(a 0 + xa 1 + x 2 A 2 ) = 0 has exactly n + 1 roots x 0,..., x n with x k < 1 Let y 0,..., y n be the corresponding non-null vectors y k ( A0 + x k A 1 + x 2 k A 2) = 0, k = 0,..., n 44
45 Quasi-Birth-and-Death processes Solution: Spectral expansion Set n p i = c k y k xk i k=0 Coefficients c 0,..., c k follow from boundary equation p 0 B 1 + p 1 A 2 = n c k (y k B 1 + y k A 2 x k ) = 0 k=0 and normalization 1 = i,j p(i, j) = n c k y k e 1 x k k=0 45
46 Quasi-Birth-and-Death processes Solution: Generating function Let P(z) = i=0 p i z i Multiply balance equations by z i and add all equations zp(z)a 0 + (P(z) p 0 )A 1 + z 1 (P(z) p 0 p 1 z)a 2 = 0 Solving this equation P(z)A(z) = p 0 (A 1 + z 1 A 2 ) + p 1 A 2 = p 0 (A 1 B 1 + z 1 A 2 ) where A(z) = za 0 + A 1 + z 1 A 2 46
47 Quasi-Birth-and-Death processes Solution: Generating function Suppose determinant equation det(a(z)) = 0 has exactly n + 1 roots z 0 = 1, z 1,..., z n with z k < 1 for k = 1,..., n Let v 0 = e, v 1,..., v n be non-null vectors satisfying Vector p 0 follows from A(z k )v k = 0 p 0 (A 1 B 1 + z 1 k A 2)v k = 0, k = 1,..., n P(1)A(1) = p 0 (A 1 B 1 + A 2 ), P(1)e = 1 P(1)A (1) (1)e = p 0 A 2 e 47
48 Quasi-Birth-and-Death processes: More structure A 0, A 1 and A 2 are upper triangular: A 0 (i, j) = 0 if j < i Implication: R is also upper triangular Example: A 0 = ( ) ( ) ( ) a0 a 1 b0 b, A 0 a 1 = 1 c0 c, A 2 0 b 2 = 1, 2 0 c 2 Then where R = ( ) r0 r 1 0 r 2 a 0 + r 0 b 0 + r0 2 c 0 = 0 a 2 + r 2 b 2 + r2 2 c 2 = 0 a 1 + r 0 b 1 + r 1 b 2 + r0 2 c 1 + (r 0 r 1 + r 1 r 2 )c 2 = 0 48
49 Quasi-Birth-and-Death processes: More structure Transition rates A 2 from level i to level i 1 have the form 0 A 2 = rp = r. ( ) r 0 p 0 r 0 p n p 0 p n =.. r n r n p 0 r n p n Example: Single server with setup A 2 = ( ) ( ) (0 ) = 1 0 µ µ Example: Shortest queue with jockeying A 2 = ( ) ( 0 2µ 2µ = ) (0 1 ) 49 Example: Single server serving n customer types for which µ 1 p 1 µ 1 p n µ 1 A 2 =.. =. ( ) p 1 p n µ n p 1 µ n p n µ n
50 Quasi-Birth-and-Death processes: More structure Transition rates A 2 from level i to level i 1 have the form 0 A 2 = rp = r. ( ) r 0 p 0 r 0 p n p 0 p n =.. r n r n p 0 r n p n Interpretation: Rate of jumping from (i, j) to level i 1 is r j, this jump is to (i 1, k) with probability p k Important: Jump probability does not depend on j This implies and p 0 p n G =.. = ep p 0 p n R = A 0 (A 1 + A 0 G) 1 = A 0 (A 1 + A 0 ep) 1 50
51 Quasi-Birth-and-Death processes: More structure Transition rates A 0 from level i to level i + 1 have the form 0 A 0 = rp = r. ( ) r 0 p 0 r 0 p n p 0 p n =.. r n r n p 0 r n p n Example: Shortest queue with jockeying A 0 = ( ) ( ) (1 ) = 0 λ 0 λ 51
52 Quasi-Birth-and-Death processes: More structure Transition rates A 0 from level i to level i + 1 have the form 0 A 0 = rp = r. ( ) r 0 p 0 r 0 p n p 0 p n =.. r n r n p 0 r n p n Interpretation: Rate of jumping from (i, j) to level i + 1 is r j, this jump is to (i + 1, k) with probability p k This implies for some vector q Hence 0 R = rq = r. ( ) r 0 q 0 r 0 q n q 0 q n =.. r n r n q 0 r n q n R i = (qr) i 1 R, p i = p 0 R i = (qr) i 1 p 1, i = 1, 2,... Number x = qr is spectral radius of R which is unique root on (0, 1) of 52 det(a 0 + xa 1 + x 2 A 2 ) = 0
53 Quasi-Birth-and-Death processes: Linear rates Transition rates have linear form to neighboring phases Rate from (i, j) to (i + k, j + 1) is a k (n j) Rate from (i, j) to (i + k, j) is b k (n j) + c k j Rate from (i, j) to (i + k, j 1) with rate d k j So A 0 is of tri-diagonal form b 1 n a 1 n 0 d 1 b 1 (n 1) + c 1 a 1 (n 1) 0 d 1 2 b 1 (n 2) + c 1 2 A 0 = b c 1 (n 2) a d 1 (n 1) b 1 + c 1 (n 1) a 1 0 d 1 n c 1 n 53
54 Quasi-Birth-and-Death processes: Linear rates Transition rate diagram i, j + 1 a 1 (n j) a 1 (n j) i, j i + 1, j d 0 j b 1 j + c 1 (n j) i, j 1 54
55 Quasi-Birth-and-Death processes: Example with linear rates n parallel servers serving two customer types 1 and 2 Type 1 customers require exponential service with rate µ 1, type 2 with rate µ 2 Customers arrive as Poisson stream with rate λ, fraction p 1 is of type 1 and p 2 of type 2 States (i, j) with i number of customers in the queue and j number of type 1 customers in service Transition rate diagram i, j + 1 (n j)µ 2 p 1 i 1, j i, j i + 1, j jµ 1 p 1 + (n j)µ 2 p 2 λ jµ 1 p 2 i, j 1 Non-zero coefficients a 1 = µ 2 p 1, b 1 = c 1 = λ/n, b 1 = µ 2 p 2, c 1 = µ 1 p 1, d 1 = µ 1 p 2 55
56 Quasi-Birth-and-Death processes: Example with linear rates n parallel servers and Poisson arrivals with rate λ which need exponential service times with rate µ Server switched off when queue is empty Idle server switched on when customer arrives which requires exponential setup time with rate θ States (i, j) with i number of customers in the queue and j number of active servers Transition rate diagram i, j + 1 (n j)θ i 1, j i, j i + 1, j jµ λ i, j 1 Non-zero coefficients a 0 = θ, b 1 = c 1 = λ/n, c 1 = µ 56
57 Quasi-Birth-and-Death processes: Example with linear rates Solution: Spectral expansion Seek solutions of balance equations of the form p i = y x i, i = 0, 1, 2,..., where y = (y(0),..., y(n)) is non-null vector and x < 1 Substitution and dividing by common powers of x yields following equations for j = 0,, n where 0 = (j + 1)D(x)y(j + 1) + ((n j)b(x) + jc(x))y(j) ((n j)(a(1) + B(1)) + j(c(1) + D(1)))xy(j) + (n j + 1)A(x)y(j 1) A(x) = a 1 +a 0 x +a 1 x 2, B(x) = b 1 +b 0 x +b 1 x 2, C(x) = c 1 +c 0 x +c 1 x 2, D(x) = d 1 +d 0 x +d 1 x 2 Solve equations for y by generating function 57 Y (z) = n y(j)z j i=0
58 Quasi-Birth-and-Death processes: Example with linear rates Solution: Spectral expansion Multiply equations by z j and add all equations D(x)Y (z) + nb(x)y (z) + (C(x) B(x))zY (z) (A(1) + B(1))nxY (z) +(A(1) + B(1) C(1) D(1))xzY (z) + na(x)zy (z) A(x)z 2 Y (z) = 0 This equation can be rewritten as Y (z) Y (z) = = n[a(x)z + B(x) (A(1) + B(1))x] A(x)z 2 ((A(1) + B(1) C(1) D(1))x + C(x) B(x))z D(x) E(x) z z 1 (x) + n E(x) z z 2 (x) where z 1 (x) and z 2 (x) are the roots of the denominator and E(x) satisfies 2E(x) n = n B(x) + C(x) x (A(1) + B(1) + C(1) + D(1)), F (x) 2 + 4A(x)D(x) 58 where F (x) = (A(1) + B(1) C(1) D(1))x + C(x) B(x)
59 Quasi-Birth-and-Death processes: Example with linear rates Solution: Spectral expansion Differential equation Y (z) Y (z) = E(x) z z 1 (x) + n E(x) z z 2 (x) General solution Y (z) = C(z z 1 (x)) E(x) (z z 2 (x)) n E(x) Key observation: Y (z) is a polynomial in z, so exponent E(x) = k for k = 0,..., n For each k = 0,..., n the equation 2k n = n B(x) + C(x) x (A(1) + B(1) + C(1) + D(1)), F (x) 2 + 4A(x)D(x) has a unique solution x = x k in the interval (0, 1) 59
60 Quasi-Birth-and-Death processes: Example with linear rates Solution: Spectral expansion For each k = 0,..., n the equation 2k n = n B(x) + C(x) x (A(1) + B(1) + C(1) + D(1)), F (x) 2 + 4A(x)D(x) has a unique solution x = x k in the interval (0, 1) x 4 x 3 x 1 x 2 x
61 Two-dimensional Markov chain: Re-entrant line (1) µ 1 µ 2 µ 3 Two stations, station 1 with two servers, labeled 1 and 3 and station 2 with single server, labeled server 2 Queue i is served at exponential rate µ i Queue 1 has infinite supply and customers subsequently go through queue 1, 2 and 3 61
62 Two-dimensional Markov chain: Re-entrant line (1) j µ 2 µ 1 µ 1 µ 3 µ 2 µ 3 µ 1 i Markov chain with states (i, j) where i number in queue 2 and j number in queue 3 Global balance equations p(i, j)(µ 1 + µ 2 + µ 3 ) = p(i 1, j)µ 1 + p(i, j + 1)µ 3 + p(i + 1, j 1)µ 2, i > 0, j > 0 p(i, 0)(µ 1 + µ 2 ) = p(i 1, 0)µ 1 + p(i, 1)µ 3, i > 0 p(0, j)(µ 1 + µ 3 ) = p(0, j + 1)µ 3 + p(1, j 1)µ 2, j > 0 p(0, 0)µ 1 = p 0,1 µ 3 62
63 Two-dimensional Markov chain: Re-entrant line (1) j µ 2 µ 1 µ 1 µ 3 µ 2 µ 3 µ 1 i Substitution of trial solution p(i, j) = α i β j in global balance equations αβ(µ 1 + µ 2 + µ 3 ) = βµ 1 + αβ 2 µ 3 + α 2 µ 2 α(µ 1 + µ 2 ) = µ 1 + αβµ 3 β(µ 1 + µ 3 ) = β 2 µ 3 + αµ 2 63 Equations solved by α = µ 1 µ 2 and β = µ 1 µ 3 Normalization yields p(i, j) = (1 α)α i (1 β)β j
64 Two-dimensional Markov chain: Re-entrant line (1) Global balance equations p(i, j)(µ 1 + µ 2 + µ 3 ) = p(i 1, j)µ 1 + p(i, j + 1)µ 3 + p(i + 1, j 1)µ 2, i > 0, j > 0 Partial balance Rate out of (i, j) due to arrival in queue 2 is equal to rate into (i, j) due to departure from queue 3 p(i, j)µ 1 = p(i + 1, j)µ 3 Rate out of (i, j) due to departure from queue 2 is equal to rate into (i, j) due to arrival in queue 2 p(i, j)µ 2 = p(i 1, j)µ 1 Rate out of (i, j) due to departure from queue 3 is equal to rate into (i, j) due to arrival in queue p(i, j)µ 3 = p(i + 1, j 1)µ 2 64 Partial balance yields candidate solution ( ) j µ2 p(i, j) = p(i + j, 0) = µ 3 ( µ2 ) j ( ) i+j µ1 p(0, 0) = µ 3 µ 2 ( µ1 ) i ( ) j µ1 p(0, 0) µ 2 µ 3
65 Two-dimensional Markov chain: Re-entrant line (2) µ 1 µ 2 µ 3 Two stations, both stations with single server Queue i is served at exponential rate µ i Queue 1 has infinite supply and customers subsequently go through queue 1, 2 and 3 Customers in queue 3 have preemptive priority over customers in queue 1 (Last Buffer First Served) Assumption µ µ 1 3 > µ
66 Two-dimensional Markov chain: Re-entrant line (2) j µ 2 µ 3 µ 2 µ 3 µ 1 i Markov chain with states (i, j) where i number in queue 2 and j number in queue 3 Global balance equations p(i, j)(µ 2 + µ 3 ) = p(i, j + 1)µ 3 + p(i + 1, j 1)µ 2, i > 0, j > 0 p(i, 0)(µ 1 + µ 2 ) = p(i 1, 0)µ 1 + p(i, 1)µ 3, i > 0 p(0, j)µ 3 = p(0, j + 1)µ 3 + p(1, j 1)µ 2, j > 0 p(0, 0)µ 1 = p(0, 1)µ 3 66
67 Two-dimensional Markov chain: Re-entrant line (2) j µ 2 µ 3 µ 2 µ 3 µ 1 i Substitution of trial solution p(i, j) = α i β j in global balance equations for i > 0 β(µ 2 + µ 3 ) = β 2 µ 3 + αµ 2 α(µ 1 + µ 2 ) = µ 1 + αβµ 3 Equations solved by α = µ 1 µ 2 (1 β) and β is the root on (0, 1) of β 2 µ 3 β(µ 1 + µ 2 + µ 3 ) + µ 1 = 0 β = µ 1 + µ 2 + µ 3 (µ 1 + µ 2 + µ 3 ) 2 4µ 1 µ 3 2µ 3 67
68 Two-dimensional Markov chain: Re-entrant line (2) j µ 2 µ 3 µ 2 µ 3 µ 1 i Substitution of p(i, j) = α i β j for i > 0 in global balance equations for i = 0 p(0, j)µ 3 = p(0, j + 1)µ 3 + αβ j 1 µ 2, j > 0 68 Equations are solved by p(0, j) = cβ j 1 where c = µ 1 µ 3 Normalization yields p(i, j) = µ 3 µ 1 +µ 3 (1 α), i = 0, j = 0 µ 1 µ 1 +µ 3 (1 α)β j 1, i = 0, j > 0 µ 3 µ 1 +µ 3 (1 α)α i β j, i > 0, j 0
69 Two-dimensional Markov chain: Single server priority queue Single server serving type 1 and type 2 customers Type i customers arrive according to Poisson stream with rate λ i Type i customers require exponential service with rate µ i Type 1 customers have preemptive priority over type 2 customers 69
70 Two-dimensional Markov chain: Single server priority queue j λ 2 λ 2 λ 1 µ 1 λ 1 µ 2 λ 2 µ 1 λ 1 i Markov chain with states (i, j) where i number of type 1 and j number of type 2 Global balance equations p(i, j)(λ 1 + λ 2 + µ 1 ) = p(i 1, j)λ 1 + p(i + 1, j)µ 1 + p(i, j 1)λ 2, i > 0, j > 0 p(i, 0)(λ 1 + λ 2 + µ 1 ) = p(i 1, 0)λ 1 + p(i + 1, 0)µ 1, i > 0 p(0, j)(λ 1 + λ 2 + µ 2 ) = p(1, j)µ 1 + p(0, j 1)λ 2 + p(0, j + 1)µ 2, j > 0 p(0, 0)(λ 1 + λ 2 ) = p(0, 1)µ 2 + p(1, 0)µ 1 70 Note that p(0, 0) = 1 ρ 1 ρ 2 where ρ i = λ i µ i
71 Two-dimensional Markov chain: Single server priority queue Solution: Difference equation Balance equations for j = 0 are homogeneous second-order difference equation p(i, 0)(λ 1 + λ 2 + µ 1 ) = p(i 1, 0)λ 1 + p(i + 1, 0)µ 1, i > 0 General solution p(i, 0) = c 0,0 x i 0 + c 0,1x i 1, i 0 where 0 < x 0 < 1 < x 1 are roots of quadratic equation Convergence of p(i, 0) implies c 0,1 = 0 so x(λ 1 + λ 2 + µ 1 ) = λ 1 + x 2 µ 1 p(i, 0) = c 0,0 x i 0 = (1 ρ 1 ρ 2 )x i 0, i 0 71
72 Two-dimensional Markov chain: Single server priority queue Solution: Difference equation Balance equations for j = 1 are inhomogeneous second-order difference equation General solution where p(i, 1)(λ 1 + λ 2 + µ 1 ) = p(i 1, 1)λ 1 + p(i + 1, 1)µ 1 + p(i, 0)λ 2 = p(i 1, 1)λ 1 + p(i + 1, 1)µ 1 + c 0,0 x i 0 λ 2, i > 0 p(i, 1) = c 1,0 x0 i + c 1,1(i + 1)x ( ) 0 i ( ) i + 1 i + 1 = c 1,0 x0 i 0 + c 1,1 x0 i 1, i 0 c 1,1 = Coefficient c 1,0 follows from balance equation in (0, 0) λ 2 c 0,0 λ 1 + λ 2 2µ 1 x 0 72 c 1,0 = λ 1 + λ 2 µ 1 x 0 µ 2 c 0,0 c 1,1
73 Two-dimensional Markov chain: Single server priority queue Solution: Difference equation Repeating procedure for balance equations for j = 2, 3,... leads to p(i, j) = j k=0 ( ) i + j c j,k x0 i k, i, j 0 where coefficients c j,k can be determined recursively starting with c 0,0 = 1 ρ 1 ρ 2 73
74 Two-dimensional Markov chain: Single server priority queue Solution: Generating function Let P(x, y) = i=0 j=0 p(i, j)x i y j for x, y 1 Multiply balance equations by x i y j and add all equations with h 1 (x, y)p(x, y) = h 2 (x, y)p(0, y) + h 3 (x, y)p(0, 0) h 1 (x, y) = λ 1 xy(1 x) + λ 2 xy(1 y) µ 1 y(1 x) h 2 (x, y) = µ 1 y(1 x) + µ 2 x(1 y) h 3 (x, y) = µ 2 x(1 y) and P(0, 0) = p(0, 0) = 1 ρ 1 ρ 2 74
75 Two-dimensional Markov chain: Single server priority queue Solution: Generating function For every y 1 let x = ξ(y) be the unique root with x 1 of the quadratic equation Then 0 = h 1 (x, y) = λ 1 xy(1 x) + λ 2 xy(1 y) µ 1 y(1 x) x = ξ(y) = λ 1 + µ 1 + λ 2 (1 y) (λ 1 + µ 1 + λ 2 (1 y)) 2 4λ 1 µ 1 2λ 1 Substituting x = ξ(y) and P(0, 0) = 1 ρ 1 ρ 2 in h 1 (x, y)p(x, y) = h 2 (x, y)p(0, y) + h 3 (x, y)p(0, 0) gives P(0, y) = h 3(ξ(y), y)(1 ρ 1 ρ 2 ) h 2 (ξ(y), y) 75
76 Two-dimensional Markov chain: Single server priority queue Solution: Matrix geometric Partition states space into levels i = {(i, 0), (i, 1),...} Generator Q of the form where (λ = λ 1 + λ 2 ) 1 A 0 = λ λ 1 B 1 A A 2 A 1 A 0 0 Q = 0 A 2 A 1 A A 2 A λ µ 1 λ 2, A 1 = λ µ 1 λ 2, A 2 = λ λ 2 B 1 = µ 2 λ µ 2 λ µ 1 µ
77 Two-dimensional Markov chain: Single server priority queue Solution: Matrix geometric A 0, A 1 and A 2 are upper triangular and have repeating structure Implication: R is also upper triangular and of the form r 0 r 1 r 2... R = r 0 r 1... r Elements can be determined recursively from matrix equation A 0 + RA 1 + R 2 A 2 = 0 µ 1 r 2 0 (λ + µ 1)r 0 + λ 1 = 0 k µ 1 r k l r l (λ + µ 1 )r k + λ 2 r k 1 = 0, k 1 l=0 77
78 Two-dimensional Markov chain: Single server priority queue Solution: Matrix geometric Vector p 0 = (p(0, 0), p(0, 1),...) follows from boundary equations yielding the recursion p(0, 1)µ 2 = p(0, 0)(λ µ 1 r 0 ) 0 = p 0 B 1 + p 1 A 2 = p 0 (B 1 + RA 2 ) j p(0, j + 1)µ 2 = (λ + µ 2 )p(0, j) λ 2 p(0, j 1) µ 1 p(0, j k)r k, j 1 starting with p(0, 0) = 1 ρ 1 ρ 2 Vector p i+1 = (p(0, 0), p(0, 1),...) can be obtained from p i+1 = p i R yielding k=0 p(i + 1, j) = j p(i, j k)r k k=0 78
79 Two-dimensional Markov chain: Single server priority queue Solution: Matrix analytic A 0, A 1 and A 2 are upper triangular and have repeating structure Implication: G is also upper triangular and of the form g 0 g 1 g 2... G = g 0 g 1... g g k is probability of first passage to level i 1 in (i 1, j + k) for initial state (i, j) at level i One-step analysis g 0 = g k = µ 1 λ + µ 1 + λ 1 λ + µ 1 g 2 0 λ 2 λ + µ 1 g k 1 + λ 1 λ + µ 1 k g l g k l, k 1 l=0 79
80 Two-dimensional Markov chain: Single server priority queue Solution: Matrix analytic From p i+1 A 2 = p i A 0 G follows p(i + 1, j)µ 1 = j p(i, j k)λ 1 g k, i, j, 0 k=0 80
81 Two-dimensional Markov chain: Single server priority queue Solution: Matrix analytic Vector p 0 follows from boundary equations 0 = p 0 B 1 + p 1 A 2 = p 0 (B 1 + A 0 G) B 1 + A 0 G is generator of Markov chain embedded on level 0 (0, j + k) λ 2 λ 1 g k (0, j) µ 2 81 Global balance for {(0, 0), (0, 1),..., (0, j)} j p(0, j + 1)µ 2 = p(0, j)λ 2 + p(0, j k)λ 1 (1 k=0 ) k g l l=0
82 Two-dimensional Markov chain: Shortest queue λ µ Two parallel exponential servers with rate µ, each with its own queue Customers arrive according to Poisson stream with rate λ and join shortest queue on arrival µ 82
83 Two-dimensional Markov chain: Shortest queue µ µ λ µ λ 2µ λ Markov chain with states (i, j) where i number in shortest queue and j difference between queues Global balance equations p(i, j)(λ + 2µ) = p(i 1, j + 1)λ + p(i, j + 1)µ + p(i + 1, j 1)µ, i > 0, j > 1 p(i, 1)(λ + 2µ) = p(i 1, 2)λ + p(i, 2)µ + p(i, 0)λ + p(i + 1, 0)2µ, i > 0 p(i, 0)(λ + 2µ) = p(i 1, 1)λ + p(i, 1)µ, i > 0 p(0, j)(λ + µ) = p(0, j + 1)µ + p(1, j 1)µ, j > 1 83
84 Two-dimensional Markov chain: Shortest queue µ µ λ µ λ 2µ λ Markov chain with states (i, j) where i number in shortest queue and j difference between queues Global balance equations p(i, j)(λ + 2µ) = p(i 1, j + 1)λ + p(i, j + 1)µ + p(i + 1, j 1)µ, i > 0, j > 1 p(i, 1)(λ + 2µ) = p(i 1, 2)λ + p(i, 2)µ λ +(p(i 1, 1)λ + p(i, 1)µ) λ + 2µ + (p(i, 1)λ + p(i + 1, 1)µ) 2µ λ + 2µ, i > 0 p(0, j)(λ + µ) = p(0, j + 1)µ + p(1, j 1)µ, j > 1 84
85 Intermezzo: Method of images φ=0 φ=0 p q What is the potential Φ in point p? 85
86 Intermezzo: Method of images φ=0 φ=0 p q q What is the potential Φ in point p? Add image charge q: Potential Φ in point p is Φ(p) = q r 1 q r 2 where r 1 and r 2 are distances from point p to charge q and q 86
87 Intermezzo: Method of images φ=0 φ=0 p q q q q For two planes at right angles Φ(p) = q r 1 q r 2 + q r 3 q r 4 87
88 Intermezzo: Method of images A B a c b Φ=Φa Φ=0 What is the potential Φ in any point p outside the spheres A and B? 88
89 Intermezzo: Method of images A B a α 0 c b Φ=Φ Φ=0 a What is the potential Φ in any point p outside the spheres A and B? Add image charge α 0 in center of A with α 0 = aφ a : Then Φ(p) α 0 r 0 where r 0 is distance from p to charge α 0 (correct if c = ) Potential of α 0 does not vanish on B 89
90 Intermezzo: Method of images A B a c α 0 β 0 b Φ=Φ Φ=0 a Place new image charge β 0 in B to compensate for this error Φ(p) α 0 r 0 + β 0 r 1 where r 0 and r 1 are distances from p to charges α 0 and β 0 Charge β 0 alters potential on A 90
91 Intermezzo: Method of images A B a c α 0 α 1 β 0 b Φ=Φ Φ=0 a Place new image charge β 0 in B to compensate for this error Φ(p) α 0 r 0 + β 0 r 1 where r 0 and r 1 are distances from p to charges α 0 and β 0 Charge β 0 alters potential on A Place new image charge α 1 in A to compensate for this error Φ(p) α 0 r 0 + β 0 r 1 + α 1 r 2 91
92 Intermezzo: Method of images A B a c α 0 α 1 β 0 b Φ=Φ Φ=0 a Potential problem of two spheres is solved by infinite sequence of image charges in A and B Φ(p) = α 0 r 0 + β 0 r 1 + α 1 r 2 + β 1 r
93 Two-dimensional Markov chain: Shortest queue µ µ λ µ λ 2µ λ Image charges are products α i β j satisfying the global balance equations p(i, j)(λ + 2µ) = p(i 1, j + 1)λ + p(i, j + 1)µ + p(i + 1, j 1)µ, i > 0, j > 1 Substituting α i β j in this equation and dividing by common powers gives the curve αβ(λ + 2µ) = β 2 λ + αβ 2 µ + α 2 µ 93
94 Two-dimensional Markov chain: Shortest queue Quadratic curve αβ(λ + 2µ) = β 2 λ + αβ 2 µ + α 2 µ 94
95 Two-dimensional Markov chain: Shortest queue Unique product form satisfying global balance equations for i > 0, j > 1 and i > 0, j = 1: p(i, j) c 0 α0 i βj 0 where α 0 = ρ 2, β 0 = ρ 2 /(2 + ρ) with ρ = λ/2µ Product form c 0 α0 i βj 0 violates global balance equations for i = 0 95
96 Two-dimensional Markov chain: Shortest queue Add new product form c 1 α i 1 βj 1 such that sum satisfies balance equations for i = 0 Then β 1 = β 0 and α 1 is companion root of α 0 and p(i, j) c 0 α i 0 βj 0 + c 1α i 1 βj 1 c 1 = α 1 β 0 α 0 β 0 c 0 96 Product form c 1 α1 i βj 1 violates global balance equations for j = 1
97 Two-dimensional Markov chain: Shortest queue Add new product form c 2 α i 2 βj 2 such that sum satisfies balance equations for j = 1 p(i, j) c 0 α i 0 βj 0 + c 1α i 1 βj 1 + c 2α i 2 βj 2 Then α 2 = α 1 and β 2 is companion root of β 1 and (with ρ = λ/(2µ)) c 2 = (ρ + α 1)/β 2 (1 + ρ) (ρ + α 1 )/β 1 (1 + ρ) c 1 97 Product form c 2 α2 i βj 2 violates global balance equations for i = 0
98 Two-dimensional Markov chain: Shortest queue Iterating yields infinite sequence p(i, j) = c 0 α i 0 βj 0 + c 1α i 1 βj 1 + c 2α i 2 βj 2 + c 3α i 3 βj
99 Two-dimensional Markov chain: Random walk in quarter plane q i,j Equilibrium probabilities can be expressed as infinite sequence of product forms p(i, j) = c 0 α i 0 βj 0 + c 1α i 1 βj 1 + c 2α i 2 βj
100 Two-dimensional Markov chain: Random walk in quarter plane q i,j Equilibrium probabilities can be expressed as infinite sequence of product forms p(i, j) = c 0 α0 i βj 0 + c 1α1 i βj 1 + c 2α2 i βj 2 + Provided there are no transitions to North, North-East and East q 0,1 = q 1,1 = q 1,0 = 0 100
101 Two-dimensional Markov chain: Random walk in quarter plane Condition q 0,1 = q 1,1 = q 1,0 = 0 implies that curve for α and β is of the above form This form guarantees that α k 0 and β k 0 as k 101
102 Multi-skilled system Customer types C = {a, b,...} Independent Poisson arrivals with rates λ c, c C Exponential service requirements with mean 1 J servers, S = {m 1,..., m J } Server m j works at rate µ mj Skill-based service: Server m i can serve customer types C(m j ) Type c customer can be served by servers S(c) Service discipline combination of FCFS and ALIS: Server picks longest waiting compatible customer Customer assigned to longest idle compatible server 102
103 Multi-skilled system: Example b a c m 1 m 3 m 2 λ a, λ b, λ c a,b a,c a m 1 m 2 µ m1 µ m2 m 3 µ m3 C = {a, b, c} and S = {m 1, m 2, m 3 } Skill-based service: C(m 1 ) = {a, b}, C(m 2 ) = {a, c}, C(m 3 ) = {a} S(a) = {m 1, m 2, m 3 }, S(b) = {m 1 }, S(c) = {m 2 } 103
104 Multi-skilled system: Markov chain Markov chain with states (M 1, n 1, M 2,..., M i, n i, M i+1,..., M J ) i busy servers M 1,..., M i J i idle servers M i+1,..., M J with increasing idle times M 1,..., M J is a permutation of m 1,..., m J n j number of waiting jobs between server M j and M j+1 104
105 Multi-skilled system: Examples of states a/b b b b a/c b/c b/c b/c b/c a a/b/c a/b/c m 1 m 2 m 3 (i) All servers busy, servers move left to right, arrivals come from the right a a/b b b b * a/b b b b * * m 3 m 1 m 2 m 1 m 2 m 3 (ii) Two servers busy, no possible customers between m 3 and m 1 (iii) Two servers are idle, m 3 has been idle longest time (i) (m 1, 3, m 2, 4, m 3, 2) (ii) (m 3, 0, m 1, 3, m 2 ) (iii) (m 1, 3, m 2, m 3 ) 105
106 Multi-skilled system: N-system b a m 1 m 2 λ a, λ b a, b a m 1 m 2 µ 1 µ 2 Markov chain with states (m 1, n 1, m 2, n 2 ) with n 1, n 2 0 (m 2, 0, m 1, n 2 ) with n 2 0 (m 1, n 1, m 2 ) with n 1 0 (m 2, 0, m 1 ) (m 1, m 2 ) and (m 2, m 1 ) (empty system) Assume λ b < µ 1 and λ a + λ b < µ 1 + µ 2 106
107 Multi-skilled system: N-system n 2 λ a + λ b µ 1 (m 1, n 1, m 2, n 2 ) µ 2 p 1 µ 2 p 2 State transition from (m 1, n 1, m 2, n 2 ) to (m 1, n 1 + j 1, m 2, n 2 j) with rate µ 2 p j where p j = (1 γ )γ j 1 λ b γ = λ a + λ b n 1 107
108 Multi-skilled system: N-system n 2 λ a + λ b µ 1 (m 1, n 1, m 2, n 2 ) µ 2 p 1 µ 2 p 2 n 1 Global balance equation in (m 1, n 1, m 2, n 2 ) with n 2 > 0 (λ a + λ b + µ 1 + µ 2 )p(m 1, n 1, m 2, n 2 ) = (λ a + λ b )p(m 1, n 1, m 2, n 2 1) + µ 1 p(m 1, n 1 + 1, m 2, n 2 ) n 1 +µ 2 (1 γ ) γ j p(m 1, n 1 j, m 2, n 2 + j + 1) j=0 +µ 2 (1 γ )γ n 1 p(m 2, 0, m 1, n n 2 ) 108
109 Multi-skilled system: Partial balance (i) Rate out of state (M 1, n 1, M 2,..., M i, n i, M i+1,..., M J ) due to an arrival activating a server equals rate into that state due to a departure deactivating M i+1 (ii) Rate out of state (M 1, n 1, M 2,..., M i, n i, M i+1,..., M J ) due to an arrival that is queueing equals rate into that state due to a departure from a server staying active (iii) Rate out of state (M 1, n 1, M 2,..., M i, 0, M i+1,..., M J ) with n i = 0 due to a departure equals rate into that state due to an arrival activating M i (iv) Rate out of state (M 1, n 1, M 2,..., M i, n i, M i+1,..., M J ) with n i > 0 due to a departure equals rate into that state due to an arrival that is queueing 109
110 Multi-skilled system: N-system (iv) Rate out of state (m 1, n 1, m 2, n 2 ) with n i > 0 due to departure equals rate into that state due to an arrival that is queueing (λ a + λ b + µ 1 + µ 2 )p(m 1, n 1, m 2, n 2 ) = (λ a + λ b )p(m 1, n 1, m 2, n 2 1) + µ 1 p(m 1, n 1 + 1, m 2, n 2 ) n 1 +µ 2 (1 γ ) γ j p(m 1, n 1 j, m 2, n 2 + j + 1) j=0 +µ 2 (1 γ )γ n 1 p(m 2, 0, m 1, n n 2 ) 110
111 Multi-skilled system: N-system (ii) Rate out of state (m 1, n 1, m 2, n 2 ) due to an arrival that is queueing equals rate into that state due to a departure from a server staying active (λ a + λ b + µ 1 + µ 2 )p(m 1, n 1, m 2, n 2 ) = (λ a + λ b )p(m 1, n 1, m 2, n 2 1) + µ 1 p(m 1, n 1 + 1, m 2, n 2 ) n 1 +µ 2 (1 γ ) γ j p(m 1, n 1 j, m 2, n 2 + j + 1) j=0 +µ 2 (1 γ )γ n 1 p(m 2, 0, m 1, n n 2 ) 111
112 Multi-skilled system: N-system Partial balance dictates solution p(m 1, n 1, m 2, n 2 ) (iv) = p(m 1, n 1, m 2, n 2 1) λ a + λ b µ 1 + µ ( ) 2 (iv) λa + λ n2 b = p(m 1, n 1, m 2, 0) µ 1 + µ ( 2 (iii) λ a λa + λ b = p(m 1, n 1, m 2 ) µ 1 + µ 2 µ 1 + µ 2 (iv) = p(m 1, 0, m 2 ) ( ) n1 λb λa µ 1 (i) = p(m 1, m 2 ) λ a + λ b µ 1 µ 1 + µ 2 ( ) n1 λb λa µ 1 ) n2 ( λa + λ b µ 1 + µ 2 ) n2 µ 1 + µ ( 2 λa + λ b µ 1 + µ 2 ) n2 112
113 Multi-skilled system: N-system Candidate solution p(m 1, n 1, m 2, n 2 ) = p(m 1, m 2 ) λ a + λ b µ 1 ( ) n1 λb λa µ 1 p(m 2, 0, m 1, n 2 ) = p(m 1, m 2 ) λ ( a λa + λ b µ 2 µ 1 + µ 2 p(m 1, n 1, m 2 ) = p(m 1, m 2 ) λ a + λ b µ 1 p(m 2, 0, m 1 ) = p(m 1, m 2 ) λ a µ 2 λ a p(m2, m1) = p(m 1, m 2 ) λ a + λ b ( λb µ 1 ) n2 +1 ) n1 µ 1 + µ 2 ( λa + λ b µ 1 + µ 2 ) n2 113
114 Multi-skilled system: N-system Set C = p(m 1, m 2 )(λ a + λ b )λ a then candidate solution p(m 1, n 1, m 2, n 2 ) = C λn 1 b (λ a + λ b ) n 2 µ n 1+1 (µ µ 2 ) n 2+1 p(m 2, 0, m 1, n 2 ) = C 1 (λ a + λ b ) n 2 µ 2 (µ 1 + µ 2 ) n 2+1 p(m 1, n 1, m 2 ) = C λn 1 b 1 µ n 1+1 λ 1 a p(m 2, 0, m 1 ) = C 1 1 µ 2 λ a + λ b 1 1 p(m2, m1) = C λ a + λ b λ a + λ b 1 1 p(m1, m2) = C λ a + λ b λ a Checking partial balance equations proves that this is solution to the global balance equations 114
115 Multi-skilled system C is set of customer types {a, b,...} S is set of servers {m 1, m 1,..., m J } λ X = c X λ c where X C µ Y = M Y µ M where Y S S(X) is set of server types that can handle customers in X C(Y) is set of customer types that can be handled by servers in Y U(Y) is set of customer types that can be uniquely handled by servers in Y Assume (stability) λ U(Y) < µ Y for every subset of servers Y S 115
116 Multi-skilled system Solution to the global balance equations p(m 1, n 1,..., M i, n i, M i+1,..., M J ) = C i j=1 λ n j U({M 1,...,M j }) µ n j+1 {M 1,...,M j } J j=i+1 λ 1 C({M j,...,m J }) where C is a normalizing constant Proof by checking partial balance equations 116
117 Multi-skilled system: Alternative Markov chain Markov chain with states (C 1,..., C L, M 1,..., M K ) L customers waiting for service C 1,..., C L with decreasing waiting times (L 0) K idle servers M 1,..., M K with decreasing idle times (K J) Note that L = n n i and M 1,..., M K correspond to M J,..., M i+1 Solution to the global balance equations p(c 1,..., C L, M 1,..., M K ) = C where C is a normalizing constant Proof by checking partial balance equations Note that L λ Cl µ S({C1,...,C l=1 l }) K µ Mk λ C({M1,...,M k=1 k }) 117 p(c 1,..., C L all servers busy) L λ Cl µ S({C1,...,C l=1 l })
118 FCFS matching queue Independent Poisson arrivals of customers with rates λ c, c C Independent Poisson arrivals of servers with rates µ mj, m j S Arriving server m j scans queue of customers Matches with the longest waiting compatible customer and the two leave the system immediately If no match is possible he leaves immediately without customer Note that this system is equivalent with a redundancy service system 118
119 FCFS matching queue: Markov chain Markov chain with states (C 1,..., C L ) L customers waiting for service C 1,..., C L with decreasing waiting times (L 0) Solution to the global balance equations p(c 1,..., C L ) = C L λ Cl µ S({C1,...,C l=1 l }) where C is a normalizing constant 119
The shortest queue problem
The shortest queue problem Ivo Adan March 19, 2002 1/40 queue 1 join the shortest queue queue 2 Where: Poisson arrivals with rate Exponential service times with mean 1/ 2/40 queue 1 queue 2 randomly assign
More informationStatistics 150: Spring 2007
Statistics 150: Spring 2007 April 23, 2008 0-1 1 Limiting Probabilities If the discrete-time Markov chain with transition probabilities p ij is irreducible and positive recurrent; then the limiting probabilities
More informationP (L d k = n). P (L(t) = n),
4 M/G/1 queue In the M/G/1 queue customers arrive according to a Poisson process with rate λ and they are treated in order of arrival The service times are independent and identically distributed with
More informationPerformance Evaluation of Queuing Systems
Performance Evaluation of Queuing Systems Introduction to Queuing Systems System Performance Measures & Little s Law Equilibrium Solution of Birth-Death Processes Analysis of Single-Station Queuing Systems
More informationTCOM 501: Networking Theory & Fundamentals. Lecture 6 February 19, 2003 Prof. Yannis A. Korilis
TCOM 50: Networking Theory & Fundamentals Lecture 6 February 9, 003 Prof. Yannis A. Korilis 6- Topics Time-Reversal of Markov Chains Reversibility Truncating a Reversible Markov Chain Burke s Theorem Queues
More informationQueues and Queueing Networks
Queues and Queueing Networks Sanjay K. Bose Dept. of EEE, IITG Copyright 2015, Sanjay K. Bose 1 Introduction to Queueing Models and Queueing Analysis Copyright 2015, Sanjay K. Bose 2 Model of a Queue Arrivals
More informationIEOR 6711, HMWK 5, Professor Sigman
IEOR 6711, HMWK 5, Professor Sigman 1. Semi-Markov processes: Consider an irreducible positive recurrent discrete-time Markov chain {X n } with transition matrix P (P i,j ), i, j S, and finite state space.
More informationLecture 20: Reversible Processes and Queues
Lecture 20: Reversible Processes and Queues 1 Examples of reversible processes 11 Birth-death processes We define two non-negative sequences birth and death rates denoted by {λ n : n N 0 } and {µ n : n
More informationMarkov Chains. X(t) is a Markov Process if, for arbitrary times t 1 < t 2 <... < t k < t k+1. If X(t) is discrete-valued. If X(t) is continuous-valued
Markov Chains X(t) is a Markov Process if, for arbitrary times t 1 < t 2
More informationContinuous-Time Markov Chain
Continuous-Time Markov Chain Consider the process {X(t),t 0} with state space {0, 1, 2,...}. The process {X(t),t 0} is a continuous-time Markov chain if for all s, t 0 and nonnegative integers i, j, x(u),
More informationPart I Stochastic variables and Markov chains
Part I Stochastic variables and Markov chains Random variables describe the behaviour of a phenomenon independent of any specific sample space Distribution function (cdf, cumulative distribution function)
More informationM/G/1 and M/G/1/K systems
M/G/1 and M/G/1/K systems Dmitri A. Moltchanov dmitri.moltchanov@tut.fi http://www.cs.tut.fi/kurssit/elt-53606/ OUTLINE: Description of M/G/1 system; Methods of analysis; Residual life approach; Imbedded
More informationModelling Complex Queuing Situations with Markov Processes
Modelling Complex Queuing Situations with Markov Processes Jason Randal Thorne, School of IT, Charles Sturt Uni, NSW 2795, Australia Abstract This article comments upon some new developments in the field
More informationChapter 1. Introduction. 1.1 Stochastic process
Chapter 1 Introduction Process is a phenomenon that takes place in time. In many practical situations, the result of a process at any time may not be certain. Such a process is called a stochastic process.
More informationarxiv: v1 [math.pr] 11 May 2018
FCFS Parallel Service Systems and Matching Models Ivo Adan a, Igor Kleiner b,, Rhonda Righter c, Gideon Weiss b,, a Eindhoven University of Technology b Department of Statistics, The University of Haifa,
More informationLecture 7: Simulation of Markov Processes. Pasi Lassila Department of Communications and Networking
Lecture 7: Simulation of Markov Processes Pasi Lassila Department of Communications and Networking Contents Markov processes theory recap Elementary queuing models for data networks Simulation of Markov
More informationExercises Stochastic Performance Modelling. Hamilton Institute, Summer 2010
Exercises Stochastic Performance Modelling Hamilton Institute, Summer Instruction Exercise Let X be a non-negative random variable with E[X ]
More informationFigure 10.1: Recording when the event E occurs
10 Poisson Processes Let T R be an interval. A family of random variables {X(t) ; t T} is called a continuous time stochastic process. We often consider T = [0, 1] and T = [0, ). As X(t) is a random variable
More informationSTAT 380 Continuous Time Markov Chains
STAT 380 Continuous Time Markov Chains Richard Lockhart Simon Fraser University Spring 2018 Richard Lockhart (Simon Fraser University)STAT 380 Continuous Time Markov Chains Spring 2018 1 / 35 Continuous
More informationMatrix analytic methods. Lecture 1: Structured Markov chains and their stationary distribution
1/29 Matrix analytic methods Lecture 1: Structured Markov chains and their stationary distribution Sophie Hautphenne and David Stanford (with thanks to Guy Latouche, U. Brussels and Peter Taylor, U. Melbourne
More informationSolutions to Homework Discrete Stochastic Processes MIT, Spring 2011
Exercise 6.5: Solutions to Homework 0 6.262 Discrete Stochastic Processes MIT, Spring 20 Consider the Markov process illustrated below. The transitions are labelled by the rate q ij at which those transitions
More informationUNIVERSITY OF YORK. MSc Examinations 2004 MATHEMATICS Networks. Time Allowed: 3 hours.
UNIVERSITY OF YORK MSc Examinations 2004 MATHEMATICS Networks Time Allowed: 3 hours. Answer 4 questions. Standard calculators will be provided but should be unnecessary. 1 Turn over 2 continued on next
More informationA product form solution to a system with multi-type jobs and multi-type servers Visschers, J.W.C.H.; Adan, I.J.B.F.; Weiss, G.
A product form solution to a system with multi-type jobs and multi-type servers Visschers, JWCH; Adan, IJBF; Weiss, G Published: 01/01/2011 Document Version Publisher s PDF, also known as Version of Record
More informationECE-517: Reinforcement Learning in Artificial Intelligence. Lecture 4: Discrete-Time Markov Chains
ECE-517: Reinforcement Learning in Artificial Intelligence Lecture 4: Discrete-Time Markov Chains September 1, 215 Dr. Itamar Arel College of Engineering Department of Electrical Engineering & Computer
More informationStationary Analysis of a Multiserver queue with multiple working vacation and impatient customers
Available at http://pvamu.edu/aam Appl. Appl. Math. ISSN: 932-9466 Vol. 2, Issue 2 (December 207), pp. 658 670 Applications and Applied Mathematics: An International Journal (AAM) Stationary Analysis of
More informationQueuing Theory. Richard Lockhart. Simon Fraser University. STAT 870 Summer 2011
Queuing Theory Richard Lockhart Simon Fraser University STAT 870 Summer 2011 Richard Lockhart (Simon Fraser University) Queuing Theory STAT 870 Summer 2011 1 / 15 Purposes of Today s Lecture Describe general
More informationContents Preface The Exponential Distribution and the Poisson Process Introduction to Renewal Theory
Contents Preface... v 1 The Exponential Distribution and the Poisson Process... 1 1.1 Introduction... 1 1.2 The Density, the Distribution, the Tail, and the Hazard Functions... 2 1.2.1 The Hazard Function
More informationTime Reversibility and Burke s Theorem
Queuing Analysis: Time Reversibility and Burke s Theorem Hongwei Zhang http://www.cs.wayne.edu/~hzhang Acknowledgement: this lecture is partially based on the slides of Dr. Yannis A. Korilis. Outline Time-Reversal
More informationGI/M/1 and GI/M/m queuing systems
GI/M/1 and GI/M/m queuing systems Dmitri A. Moltchanov moltchan@cs.tut.fi http://www.cs.tut.fi/kurssit/tlt-2716/ OUTLINE: GI/M/1 queuing system; Methods of analysis; Imbedded Markov chain approach; Waiting
More informationLECTURE #6 BIRTH-DEATH PROCESS
LECTURE #6 BIRTH-DEATH PROCESS 204528 Queueing Theory and Applications in Networks Assoc. Prof., Ph.D. (รศ.ดร. อน นต ผลเพ ม) Computer Engineering Department, Kasetsart University Outline 2 Birth-Death
More informationIntroduction to Queuing Networks Solutions to Problem Sheet 3
Introduction to Queuing Networks Solutions to Problem Sheet 3 1. (a) The state space is the whole numbers {, 1, 2,...}. The transition rates are q i,i+1 λ for all i and q i, for all i 1 since, when a bus
More informationAssignment 3 with Reference Solutions
Assignment 3 with Reference Solutions Exercise 3.: Poisson Process Given are k independent sources s i of jobs as shown in the figure below. The interarrival time between jobs for each source is exponentially
More informationMarkov processes and queueing networks
Inria September 22, 2015 Outline Poisson processes Markov jump processes Some queueing networks The Poisson distribution (Siméon-Denis Poisson, 1781-1840) { } e λ λ n n! As prevalent as Gaussian distribution
More information1 IEOR 4701: Continuous-Time Markov Chains
Copyright c 2006 by Karl Sigman 1 IEOR 4701: Continuous-Time Markov Chains A Markov chain in discrete time, {X n : n 0}, remains in any state for exactly one unit of time before making a transition (change
More informationStochastic process. X, a series of random variables indexed by t
Stochastic process X, a series of random variables indexed by t X={X(t), t 0} is a continuous time stochastic process X={X(t), t=0,1, } is a discrete time stochastic process X(t) is the state at time t,
More informationDigital Workbook for GRA 6035 Mathematics
Eivind Eriksen Digital Workbook for GRA 6035 Mathematics November 10, 2014 BI Norwegian Business School Contents Part I Lectures in GRA6035 Mathematics 1 Linear Systems and Gaussian Elimination........................
More informationBIRTH DEATH PROCESSES AND QUEUEING SYSTEMS
BIRTH DEATH PROCESSES AND QUEUEING SYSTEMS Andrea Bobbio Anno Accademico 999-2000 Queueing Systems 2 Notation for Queueing Systems /λ mean time between arrivals S = /µ ρ = λ/µ N mean service time traffic
More informationA FAST MATRIX-ANALYTIC APPROXIMATION FOR THE TWO CLASS GI/G/1 NON-PREEMPTIVE PRIORITY QUEUE
A FAST MATRIX-ANAYTIC APPROXIMATION FOR TE TWO CASS GI/G/ NON-PREEMPTIVE PRIORITY QUEUE Gábor orváth Department of Telecommunication Budapest University of Technology and Economics. Budapest Pf. 9., ungary
More informationData analysis and stochastic modeling
Data analysis and stochastic modeling Lecture 7 An introduction to queueing theory Guillaume Gravier guillaume.gravier@irisa.fr with a lot of help from Paul Jensen s course http://www.me.utexas.edu/ jensen/ormm/instruction/powerpoint/or_models_09/14_queuing.ppt
More informationTHE QUEEN S UNIVERSITY OF BELFAST
THE QUEEN S UNIVERSITY OF BELFAST 0SOR20 Level 2 Examination Statistics and Operational Research 20 Probability and Distribution Theory Wednesday 4 August 2002 2.30 pm 5.30 pm Examiners { Professor R M
More informationMARKOV PROCESSES. Valerio Di Valerio
MARKOV PROCESSES Valerio Di Valerio Stochastic Process Definition: a stochastic process is a collection of random variables {X(t)} indexed by time t T Each X(t) X is a random variable that satisfy some
More informationQueueing Networks and Insensitivity
Lukáš Adam 29. 10. 2012 1 / 40 Table of contents 1 Jackson networks 2 Insensitivity in Erlang s Loss System 3 Quasi-Reversibility and Single-Node Symmetric Queues 4 Quasi-Reversibility in Networks 5 The
More informationMulti Stage Queuing Model in Level Dependent Quasi Birth Death Process
International Journal of Statistics and Systems ISSN 973-2675 Volume 12, Number 2 (217, pp. 293-31 Research India Publications http://www.ripublication.com Multi Stage Queuing Model in Level Dependent
More informationCPSC 531: System Modeling and Simulation. Carey Williamson Department of Computer Science University of Calgary Fall 2017
CPSC 531: System Modeling and Simulation Carey Williamson Department of Computer Science University of Calgary Fall 2017 Motivating Quote for Queueing Models Good things come to those who wait - poet/writer
More informationQueueing systems. Renato Lo Cigno. Simulation and Performance Evaluation Queueing systems - Renato Lo Cigno 1
Queueing systems Renato Lo Cigno Simulation and Performance Evaluation 2014-15 Queueing systems - Renato Lo Cigno 1 Queues A Birth-Death process is well modeled by a queue Indeed queues can be used to
More informationExercises Solutions. Automation IEA, LTH. Chapter 2 Manufacturing and process systems. Chapter 5 Discrete manufacturing problems
Exercises Solutions Note, that we have not formulated the answers for all the review questions. You will find the answers for many questions by reading and reflecting about the text in the book. Chapter
More informationIrreducibility. Irreducible. every state can be reached from every other state For any i,j, exist an m 0, such that. Absorbing state: p jj =1
Irreducibility Irreducible every state can be reached from every other state For any i,j, exist an m 0, such that i,j are communicate, if the above condition is valid Irreducible: all states are communicate
More informationM/M/1 Queueing System with Delayed Controlled Vacation
M/M/1 Queueing System with Delayed Controlled Vacation Yonglu Deng, Zhongshan University W. John Braun, University of Winnipeg Yiqiang Q. Zhao, University of Winnipeg Abstract An M/M/1 queue with delayed
More informationIEOR 6711: Stochastic Models I, Fall 2003, Professor Whitt. Solutions to Final Exam: Thursday, December 18.
IEOR 6711: Stochastic Models I, Fall 23, Professor Whitt Solutions to Final Exam: Thursday, December 18. Below are six questions with several parts. Do as much as you can. Show your work. 1. Two-Pump Gas
More informationA Heterogeneous Two-Server Queueing System with Balking and Server Breakdowns
The Eighth International Symposium on Operations Research and Its Applications (ISORA 09) Zhangjiajie, China, September 20 22, 2009 Copyright 2009 ORSC & APORC, pp. 230 244 A Heterogeneous Two-Server Queueing
More informationLecture Notes 7 Random Processes. Markov Processes Markov Chains. Random Processes
Lecture Notes 7 Random Processes Definition IID Processes Bernoulli Process Binomial Counting Process Interarrival Time Process Markov Processes Markov Chains Classification of States Steady State Probabilities
More informationContinuous Time Processes
page 102 Chapter 7 Continuous Time Processes 7.1 Introduction In a continuous time stochastic process (with discrete state space), a change of state can occur at any time instant. The associated point
More informationDerivation of Formulas by Queueing Theory
Appendices Spectrum Requirement Planning in Wireless Communications: Model and Methodology for IMT-Advanced E dite d by H. Takagi and B. H. Walke 2008 J ohn Wiley & Sons, L td. ISBN: 978-0-470-98647-9
More informationStochastic Models of Manufacturing Systems
Stochastic Models of Manufacturing Systems Ivo Adan Systems 2/49 Continuous systems State changes continuously in time (e.g., in chemical applications) Discrete systems State is observed at fixed regular
More informationPart II: continuous time Markov chain (CTMC)
Part II: continuous time Markov chain (CTMC) Continuous time discrete state Markov process Definition (Markovian property) X(t) is a CTMC, if for any n and any sequence t 1
More informationStability and Rare Events in Stochastic Models Sergey Foss Heriot-Watt University, Edinburgh and Institute of Mathematics, Novosibirsk
Stability and Rare Events in Stochastic Models Sergey Foss Heriot-Watt University, Edinburgh and Institute of Mathematics, Novosibirsk ANSAPW University of Queensland 8-11 July, 2013 1 Outline (I) Fluid
More informationAn M/M/1 Queue in Random Environment with Disasters
An M/M/1 Queue in Random Environment with Disasters Noam Paz 1 and Uri Yechiali 1,2 1 Department of Statistics and Operations Research School of Mathematical Sciences Tel Aviv University, Tel Aviv 69978,
More informationKendall notation. PASTA theorem Basics of M/M/1 queue
Elementary queueing system Kendall notation Little s Law PASTA theorem Basics of M/M/1 queue 1 History of queueing theory An old research area Started in 1909, by Agner Erlang (to model the Copenhagen
More information(b) What is the variance of the time until the second customer arrives, starting empty, assuming that we measure time in minutes?
IEOR 3106: Introduction to Operations Research: Stochastic Models Fall 2006, Professor Whitt SOLUTIONS to Final Exam Chapters 4-7 and 10 in Ross, Tuesday, December 19, 4:10pm-7:00pm Open Book: but only
More informationControl of Fork-Join Networks in Heavy-Traffic
in Heavy-Traffic Asaf Zviran Based on MSc work under the guidance of Rami Atar (Technion) and Avishai Mandelbaum (Technion) Industrial Engineering and Management Technion June 2010 Introduction Network
More informationStochastic Models in Computer Science A Tutorial
Stochastic Models in Computer Science A Tutorial Dr. Snehanshu Saha Department of Computer Science PESIT BSC, Bengaluru WCI 2015 - August 10 to August 13 1 Introduction 2 Random Variable 3 Introduction
More informationSince D has an exponential distribution, E[D] = 0.09 years. Since {A(t) : t 0} is a Poisson process with rate λ = 10, 000, A(0.
IEOR 46: Introduction to Operations Research: Stochastic Models Chapters 5-6 in Ross, Thursday, April, 4:5-5:35pm SOLUTIONS to Second Midterm Exam, Spring 9, Open Book: but only the Ross textbook, the
More information57:022 Principles of Design II Final Exam Solutions - Spring 1997
57:022 Principles of Design II Final Exam Solutions - Spring 1997 Part: I II III IV V VI Total Possible Pts: 52 10 12 16 13 12 115 PART ONE Indicate "+" if True and "o" if False: + a. If a component's
More informationMarkov Chains CK eqns Classes Hitting times Rec./trans. Strong Markov Stat. distr. Reversibility * Markov Chains
Markov Chains A random process X is a family {X t : t T } of random variables indexed by some set T. When T = {0, 1, 2,... } one speaks about a discrete-time process, for T = R or T = [0, ) one has a continuous-time
More informationPage 0 of 5 Final Examination Name. Closed book. 120 minutes. Cover page plus five pages of exam.
Final Examination Closed book. 120 minutes. Cover page plus five pages of exam. To receive full credit, show enough work to indicate your logic. Do not spend time calculating. You will receive full credit
More informationBirth and Death Processes. Birth and Death Processes. Linear Growth with Immigration. Limiting Behaviour for Birth and Death Processes
DTU Informatics 247 Stochastic Processes 6, October 27 Today: Limiting behaviour of birth and death processes Birth and death processes with absorbing states Finite state continuous time Markov chains
More informationM/G/1 queues and Busy Cycle Analysis
queues and Busy Cycle Analysis John C.S. Lui Department of Computer Science & Engineering The Chinese University of Hong Kong www.cse.cuhk.edu.hk/ cslui John C.S. Lui (CUHK) Computer Systems Performance
More informationQUEUING MODELS AND MARKOV PROCESSES
QUEUING MODELS AND MARKOV ROCESSES Queues form when customer demand for a service cannot be met immediately. They occur because of fluctuations in demand levels so that models of queuing are intrinsically
More informationChapter 4 Markov Chains at Equilibrium
Chapter 4 Markov Chains at Equilibrium 41 Introduction In this chapter we will study the long-term behavior of Markov chains In other words, we would like to know the distribution vector sn) when n The
More informationChapter 5. Continuous-Time Markov Chains. Prof. Shun-Ren Yang Department of Computer Science, National Tsing Hua University, Taiwan
Chapter 5. Continuous-Time Markov Chains Prof. Shun-Ren Yang Department of Computer Science, National Tsing Hua University, Taiwan Continuous-Time Markov Chains Consider a continuous-time stochastic process
More informationThe Transition Probability Function P ij (t)
The Transition Probability Function P ij (t) Consider a continuous time Markov chain {X(t), t 0}. We are interested in the probability that in t time units the process will be in state j, given that it
More informationA tandem queueing model with coupled processors
A tandem queueing model with coupled processors Jacques Resing Department of Mathematics and Computer Science Eindhoven University of Technology P.O. Box 513 5600 MB Eindhoven The Netherlands Lerzan Örmeci
More informationNon Markovian Queues (contd.)
MODULE 7: RENEWAL PROCESSES 29 Lecture 5 Non Markovian Queues (contd) For the case where the service time is constant, V ar(b) = 0, then the P-K formula for M/D/ queue reduces to L s = ρ + ρ 2 2( ρ) where
More informationEindhoven University of Technology BACHELOR. Queueing models involving cross-selling. Kisters, Thijs. Award date: Link to publication
Eindhoven University of Technology BACHELOR Queueing models involving cross-selling Kisters, Thijs Award date: 2017 Link to publication Disclaimer This document contains a student thesis bachelor's or
More informationAdvanced Queueing Theory
Advanced Queueing Theory 1 Networks of queues (reversibility, output theorem, tandem networks, partial balance, product-form distribution, blocking, insensitivity, BCMP networks, mean-value analysis, Norton's
More informationIntroduction to Markov Chains, Queuing Theory, and Network Performance
Introduction to Markov Chains, Queuing Theory, and Network Performance Marceau Coupechoux Telecom ParisTech, departement Informatique et Réseaux marceau.coupechoux@telecom-paristech.fr IT.2403 Modélisation
More information11 The M/G/1 system with priorities
11 The M/G/1 system with priorities In this chapter we analyse queueing models with different types of customers, where one or more types of customers have priority over other types. More precisely we
More informationOn Successive Lumping of Large Scale Systems
On Successive Lumping of Large Scale Systems Laurens Smit Rutgers University Ph.D. Dissertation supervised by Michael Katehakis, Rutgers University and Flora Spieksma, Leiden University April 18, 2014
More informationMulti-class, multi-server queues with non-preemptive priorities
Multi-class, multi-server queues with non-preemptive priorities Andrei Sleptcheno. EURANDOM, Eindhoven University of Technology, P.O. Box 53, 5600MB Eindhoven, The Netherlands. e-mail:sleptcheno@eurandom.tue.nl
More informationMarkov chains. 1 Discrete time Markov chains. c A. J. Ganesh, University of Bristol, 2015
Markov chains c A. J. Ganesh, University of Bristol, 2015 1 Discrete time Markov chains Example: A drunkard is walking home from the pub. There are n lampposts between the pub and his home, at each of
More informationElementary queueing system
Elementary queueing system Kendall notation Little s Law PASTA theorem Basics of M/M/1 queue M/M/1 with preemptive-resume priority M/M/1 with non-preemptive priority 1 History of queueing theory An old
More informationReadings: Finish Section 5.2
LECTURE 19 Readings: Finish Section 5.2 Lecture outline Markov Processes I Checkout counter example. Markov process: definition. -step transition probabilities. Classification of states. Example: Checkout
More informationCDA5530: Performance Models of Computers and Networks. Chapter 3: Review of Practical
CDA5530: Performance Models of Computers and Networks Chapter 3: Review of Practical Stochastic Processes Definition Stochastic ti process X = {X(t), t T} is a collection of random variables (rvs); one
More informationClass 11 Non-Parametric Models of a Service System; GI/GI/1, GI/GI/n: Exact & Approximate Analysis.
Service Engineering Class 11 Non-Parametric Models of a Service System; GI/GI/1, GI/GI/n: Exact & Approximate Analysis. G/G/1 Queue: Virtual Waiting Time (Unfinished Work). GI/GI/1: Lindley s Equations
More informationM/G/1 and Priority Queueing
M/G/1 and Priority Queueing Richard T. B. Ma School of Computing National University of Singapore CS 5229: Advanced Compute Networks Outline PASTA M/G/1 Workload and FIFO Delay Pollaczek Khinchine Formula
More informationExact Statistics of a Markov Chain trhough Reduction in Number of States: Marco Aurelio Alzate Monroy Universidad Distrital Francisco José de Caldas
Exact Statistics of a Markov Chain trhough Reduction in Number of States: A Satellite On-board Switching Example Marco Aurelio Alzate Monroy Universidad Distrital Francisco José de Caldas The general setting
More informationMarkov Chains and Stochastic Sampling
Part I Markov Chains and Stochastic Sampling 1 Markov Chains and Random Walks on Graphs 1.1 Structure of Finite Markov Chains We shall only consider Markov chains with a finite, but usually very large,
More informationCover Page. The handle holds various files of this Leiden University dissertation
Cover Page The handle http://hdlhandlenet/1887/39637 holds various files of this Leiden University dissertation Author: Smit, Laurens Title: Steady-state analysis of large scale systems : the successive
More informationProbability Distributions
Lecture : Background in Probability Theory Probability Distributions The probability mass function (pmf) or probability density functions (pdf), mean, µ, variance, σ 2, and moment generating function (mgf)
More informationMS&E 321 Spring Stochastic Systems June 1, 2013 Prof. Peter W. Glynn Page 1 of 10. x n+1 = f(x n ),
MS&E 321 Spring 12-13 Stochastic Systems June 1, 2013 Prof. Peter W. Glynn Page 1 of 10 Section 4: Steady-State Theory Contents 4.1 The Concept of Stochastic Equilibrium.......................... 1 4.2
More informationAsymptotics for Polling Models with Limited Service Policies
Asymptotics for Polling Models with Limited Service Policies Woojin Chang School of Industrial and Systems Engineering Georgia Institute of Technology Atlanta, GA 30332-0205 USA Douglas G. Down Department
More informationOther properties of M M 1
Other properties of M M 1 Přemysl Bejda premyslbejda@gmail.com 2012 Contents 1 Reflected Lévy Process 2 Time dependent properties of M M 1 3 Waiting times and queue disciplines in M M 1 Contents 1 Reflected
More informationModel reversibility of a two dimensional reflecting random walk and its application to queueing network
arxiv:1312.2746v2 [math.pr] 11 Dec 2013 Model reversibility of a two dimensional reflecting random walk and its application to queueing network Masahiro Kobayashi, Masakiyo Miyazawa and Hiroshi Shimizu
More informationQueueing Systems: Lecture 3. Amedeo R. Odoni October 18, Announcements
Queueing Systems: Lecture 3 Amedeo R. Odoni October 18, 006 Announcements PS #3 due tomorrow by 3 PM Office hours Odoni: Wed, 10/18, :30-4:30; next week: Tue, 10/4 Quiz #1: October 5, open book, in class;
More informationStochastic Processes
Stochastic Processes 8.445 MIT, fall 20 Mid Term Exam Solutions October 27, 20 Your Name: Alberto De Sole Exercise Max Grade Grade 5 5 2 5 5 3 5 5 4 5 5 5 5 5 6 5 5 Total 30 30 Problem :. True / False
More informationStochastic Models of Manufacturing Systems
Stochastic Models of Manufacturing Systems Ivo Adan Exponential closed networks 2/55 Workstations 1,..., M Workstation m has c m parallel identical machines N circulating jobs (N is the population size)
More informationProbability and Statistics Concepts
University of Central Florida Computer Science Division COT 5611 - Operating Systems. Spring 014 - dcm Probability and Statistics Concepts Random Variable: a rule that assigns a numerical value to each
More informationDynamic Control of a Tandem Queueing System with Abandonments
Dynamic Control of a Tandem Queueing System with Abandonments Gabriel Zayas-Cabán 1 Jungui Xie 2 Linda V. Green 3 Mark E. Lewis 1 1 Cornell University Ithaca, NY 2 University of Science and Technology
More informationChapter 2. Poisson Processes. Prof. Shun-Ren Yang Department of Computer Science, National Tsing Hua University, Taiwan
Chapter 2. Poisson Processes Prof. Shun-Ren Yang Department of Computer Science, National Tsing Hua University, Taiwan Outline Introduction to Poisson Processes Definition of arrival process Definition
More informationFluid Models of Parallel Service Systems under FCFS
Fluid Models of Parallel Service Systems under FCFS Hanqin Zhang Business School, National University of Singapore Joint work with Yuval Nov and Gideon Weiss from The University of Haifa, Israel Queueing
More information