A tandem queue under an economical viewpoint

A tandem queue under an economical viewpoint B. D Auria 1 Universidad Carlos III de Madrid, Spain joint work with S. Kanta The Basque Center for Applied Mathematics, Bilbao, Spain B. D Auria (Univ. Carlos III de Madrid) 2-queues tandem network January 19 th 2012 1 / 38

Outline 1 The Model 2 Fully-unobservable case 3 Fully-observable case 4 Partially-observable case Total number of customers Number of customers in the 1st queue Number of customers in the 2nd queue 5 Conclusions B. D Auria (Univ. Carlos III de Madrid) 2-queues tandem network January 19 th 2012 2 / 38

The two-queue tandem network n 1 m n tagged customer k = n + m 1 R - reward for joining the network C i - cost for unit waiting time at queue i λ - arrival rate µ i - service rate at queue i The expected net profit of the tagged customer that joins the system is P = R C 1 T 1 C 2 T 2 where T i = E[S i ] with S i be the sojourn time at queue i. B. D Auria (Univ. Carlos III de Madrid) 2-queues tandem network January 19 th 2012 3 / 38

The two-queue tandem network n 1 m n tagged customer When the tagged customer will k = n + m 1 enter? There will be an equilibrium R - reward for joining the network behaviour? C i - cost for unit waiting time at queue i λ - arrival rate µ i - service rate at queue i The expected net profit of the tagged customer that joins the system is P = R C 1 T 1 C 2 T 2 where T i = E[S i ] with S i be the sojourn time at queue i. B. D Auria (Univ. Carlos III de Madrid) 2-queues tandem network January 19 th 2012 3 / 38

References about game theory in queueing Pioneering works : Naor (1969) and Edelson and Hildebrand (1975), Main reference: the book of R. Hassin and M. Haviv (2003) Additional literature: Yechiali (1971, 1972), Mandelbaum and Shimkin (2000), Lin and Ross (2001), Armony and Haviv (2003), Economou and Kanta (2008), Economou et al. (2011) B. D Auria (Univ. Carlos III de Madrid) 2-queues tandem network January 19 th 2012 4 / 38

The M/M/1 queue: fully-unobservable case Assumption: All customer join with probability q. The arrival rate is qλ The stationary distribution is π q (n) = (1 ρ q )(ρ q ) n, with ρ q = q λ µ. The expected sojourn time of the tagged customer is if E[S] = n=0 n + 1 µ π q(n) = the average net profit is P(q) = R 1 µ(1 ρ q ) = 1 µ q λ C µ q λ P(q) > 0 the tagged customer will join w.p. 1 P(q) = 0 he is indifferent to join with any probability P(q) < 0 the tagged customer will balk The individual strategy is: as C µ < R < C µ λ Join if q µ λ + C λ R and Balk otherwise. B. D Auria (Univ. Carlos III de Madrid) 2-queues tandem network January 19 th 2012 5 / 38

The M/M/1 queue - The equilibrium strategies Individual optimization The individuals will behave as the rest only when P(q e ) = 0, i.e. 0 µ λ + C λ R 1 q e C/µ C/(µ λ) R Social optimization The net profit gained per unit of time is λ q P(q) = λ q (R and the q that maximizes it is q s = arg max {λ q P(q)} = µ µ q λ C λ 2 R Complete information With n customers in the queue the strategy is to join if P(n) = R n+1 µ R µ C 0, i.e. when n N 1 with N = C The equilibrium system is a M/M/1/N queue. C µ q λ ) B. D Auria (Univ. Carlos III de Madrid) 2-queues tandem network January 19 th 2012 6 / 38

The 2-queues tandem network Fully-unobservable case B. D Auria (Univ. Carlos III de Madrid) 2-queues tandem network January 19 th 2012 7 / 38

Fully-unobservable case: Individual Optimization Stability assumptions: Net profit function R > C 1 µ 1 + C 2 µ 2 and λ < min{µ 1, µ 2 } P(q) = R C 1 µ 1 λq C 2 µ 2 λq. Theorem The equilibrium strategy enter with probability q e is given by q e = R(µ 1 + µ 2 ) (C 1 + C 2 ) [R(µ 1 µ 2 ) (C 1 C 2 )] 2 + 4C 1 C 2 2λR B. D Auria (Univ. Carlos III de Madrid) 2-queues tandem network January 19 th 2012 8 / 38

Fully-unobservable case: Individual Optimization Stability assumptions: Net profit function R > C 1 µ 1 + C 2 µ 2 and λ < min{µ 1, µ 2 } P(q) = R C 1 µ 1 λq C 2 µ 2 λq. Theorem The unique equilibrium strategy enter with probability q e is given by q e = R(µ 1 + µ 2 ) (C 1 + C 2 ) [R(µ 1 µ 2 ) (C 1 C 2 )] 2 + 4C 1 C 2 2λR B. D Auria (Univ. Carlos III de Madrid) 2-queues tandem network January 19 th 2012 8 / 38

Fully-unobservable case: Individual Optimization Stability assumptions: R > C 1 + C 2 and λ < min{µ 1, µ 2 } µ 1 µ 2 This is usually the case Net profit function for the AWT (Avoid The Crowd) P(q) = R C 1systems µ 1 λq C 2 µ 2 λq. Theorem The unique equilibrium strategy enter with probability q e is given by q e = R(µ 1 + µ 2 ) (C 1 + C 2 ) [R(µ 1 µ 2 ) (C 1 C 2 )] 2 + 4C 1 C 2 2λR B. D Auria (Univ. Carlos III de Madrid) 2-queues tandem network January 19 th 2012 8 / 38

Fully-unobservable case: Individual Optimization Stability assumptions: Net profit function Theorem R > C 1 µ 1 + C 2 µ 2 and λ < min{µ 1, µ 2 } P(q) = R C 1 µ 1 λq C 2 Equilibrium strategy µ 2 λq. 0 q (0,1) 1 q e C 1 µ 1 + C 2 C 1 µ 2 µ 1 λ + C 2 µ 2 λ R The unique equilibrium strategy enter with probability q e is given by q e = R(µ 1 + µ 2 ) (C 1 + C 2 ) [R(µ 1 µ 2 ) (C 1 C 2 )] 2 + 4C 1 C 2 2λR B. D Auria (Univ. Carlos III de Madrid) 2-queues tandem network January 19 th 2012 8 / 38

Fully-unobservable case: Social Optimization Stability assumptions: Social net profit function Theorem R > C 1 µ 1 + C 2 µ 2 and λ < min{µ 1, µ 2 } P s (q) = λq ( R C 1 µ 1 λq C ) 2 µ 2 λq There is a unique strategy that maximizes the social benefit per time unit enter with probability q s, where q s is given by ( ) qs, if R C1 µ q s = 1 + C 2 µ µ 2, 1 C 1 + µ 2C 2 [ (µ 1 λ) 2 (µ 2 λ) ) 2 1, if R µ1 C 1 + µ 2C 2, + (µ 1 λ) 2 (µ 2 λ) 2 with q s (0, 1) unique solution of R = µ 1 C 1 (µ 1 λq s) 2 + µ 2 C 2 (µ 2 λq s) 2. B. D Auria (Univ. Carlos III de Madrid) 2-queues tandem network January 19 th 2012 9 / 38

Fully-unobservable case: Social Optimization Stability assumptions: R > C 1 µ 1 + C 2 µ 2 and λ < min{µ 1, µ 2 } Social net profit function ( P s (q) = λq R C 1 µ 1 λq C ) It can be shown that 2 µ 2 λq Theorem q s q e. There is a unique Like instrategy the M/M/1 that maximizes queue, thisthe cansocial be explained probability by theqfact s, where that customers q s is givenimpose by benefit per time unit enter with negative externalities ( on later arrivals ) qs, if R C1 µ q s = 1 + C 2 µ µ 2, 1 C 1 + µ 2C 2 [ (µ 1 λ) 2 (µ 2 λ) ) 2 1, if R µ1 C 1 + µ 2C 2, + (µ 1 λ) 2 (µ 2 λ) 2 with q s (0, 1) unique solution of R = µ 1 C 1 (µ 1 λq s) 2 + µ 2 C 2 (µ 2 λq s) 2. B. D Auria (Univ. Carlos III de Madrid) 2-queues tandem network January 19 th 2012 9 / 38

Fully-observable case B. D Auria (Univ. Carlos III de Madrid) 2-queues tandem network January 19 th 2012 10 / 38

Fully-observable case The net profit is given by P(n, m) = R C 1 T 1 (n, m) C 2 T 2 (n, m) = R (C 1 C 2 ) n C 2 T (n, m) µ 1 where T i (n, m) = E[S i Q 1 = n 1, Q 2 = m] with S i be the sojourn time at queue i and with T (n, m) = T 1 (n, m) + T 2 (n, m). B. D Auria (Univ. Carlos III de Madrid) 2-queues tandem network January 19 th 2012 11 / 38

Fully-observable case By first step analysis argument, we have that T (n, m) = 1 µ 1 + µ 2 + µ 1 µ 1 + µ 2 T (n 1, m + 1) + µ 2 µ 1 + µ 2 T (n, m 1), n, m > 0. that together with the boundary conditions T (0, m) = m µ 2 T (n, 0) = 1 µ 1 + T (n 1, 1) allows to recursively compute T (n, m). B. D Auria (Univ. Carlos III de Madrid) 2-queues tandem network January 19 th 2012 12 / 38

Fully-observable case To compute T (n, m) it is usefull to write it in the following way ( ) m µ2 T (n, m) = y(n, m) + n + m, µ 1 + µ 2 µ 2 where the function y(n, m) is given by y(n, m) = µ 1 µ 2 y(n 1, m + 1) + y(n, m 1) (µ 1 + µ 2 ) 2 y(n, 0) = 1 µ 1 + µ 2 µ 1 + µ 2 y(n 1, 1) y(0, m) = 0 and that admits the following recursive solution y(n, m) = 1 + µ 2 y(n 1, 1) + µ m 1 1 µ 2 µ 1 µ 1 + µ 2 (µ 1 + µ 2 ) 2 y(n 1, k + 2) k=0 B. D Auria (Univ. Carlos III de Madrid) 2-queues tandem network January 19 th 2012 13 / 38

Fully-observable case - an example The pictures below show the threshold policy for a tandem queue with the following parameters: R = 20, C 1 = 1, C 2 = 2, 6 5 10 4 8 3 6 2 4 1 2 0 n 0 n Figure: Thresholds (n, m) µ 1 = 1.2 and µ 2 = 0.7 Figure: Thresholds (n, m) µ 1 = 0.7 and µ 2 = 1.2 P(n, m) = R (C 1 C 2 ) n µ 1 C 2 T (n, m) B. D Auria (Univ. Carlos III de Madrid) 2-queues tandem network January 19 th 2012 14 / 38

Partially-observable case Total number of customers B. D Auria (Univ. Carlos III de Madrid) 2-queues tandem network January 19 th 2012 15 / 38

Partially-observable case - Total number of customers Arriving customers are informed about the total number of users already in the system, i.e. n + m Let denote QK = Q K,1 + Q K,2 the stationary number of customers in the system under the strategy K. At equilibrium, the stationary distribution is given by π K (n, m) = P(Q K,1 = n, Q K,2 = m) = c K ρ n 1 ρm 2, n + m K with c 1 K = n+m K ρn 1 ρm 2 and ρ i = λ/µ i. B. D Auria (Univ. Carlos III de Madrid) 2-queues tandem network January 19 th 2012 16 / 38

Partially-observable case - Total number of customers The net profit is given by P tot (k) = R (C 1 C 2 ) T K,1 (k) C 2 T K (k), 0 k K. with and T K,1 (k) = T K (k) = = k T 1 (n + 1, k n) P(QK,1 = n Q K = k) n=0 1 k + 1 µ k+1 2 µ 1 µ 2 µ 1 µ k+1 1 µ k+1 2 k T (n + 1, k n) P(QK,1 = n Q K = k) n=0 = (1 µ 2 µ 1 ) µ k+1 1 µ k+1 1 µ k+1 2 k T (n + 1, k n) n=0. ( ) n µ2 µ 1 B. D Auria (Univ. Carlos III de Madrid) 2-queues tandem network January 19 th 2012 17 / 38

Partially-observable case - Total number of customers Theorem The equilibrium strategy is given by the smallest integer K such that P tot (K ) 0. We prove something more, that is P tot (K ) is non increasing in K. Lemma The random variables QK,1, for K = 0, 1,... are stochastically ordered, i.e. QK,1 st Q K +1,1 B. D Auria (Univ. Carlos III de Madrid) 2-queues tandem network January 19 th 2012 18 / 38

Number of customers in the 1st queue B. D Auria (Univ. Carlos III de Madrid) 2-queues tandem network January 19 th 2012 19 / 38

Partially-observable case - Number of customers in the 1st queue The arriving customers can only see the waiting line of the first queue, and if they adopt the strategy N, the system can be seen as a tandem queue with the first queue with finite buffer N 1. Some matrix notations I and O - identity and zero matrix diag[x] - diagonal matrix with main diagonal given by x. e i - i + 1th vector of the canonical base in R n. U L and U R - left and right shift square matrices U k - the k projection matrix The elements of U L, U R and U k are given by (U L ) i,j = δ i 1,j, (U R ) i,j = δ i+1,j and (U k ) i,j = δ i,k δ j,k for 0 i, j K where δ i,j denotes the Kronecker delta. B. D Auria (Univ. Carlos III de Madrid) 2-queues tandem network January 19 th 2012 20 / 38

Proposition (in [Kroese, Scheinhardt and Taylor (2004)]) The stationary distribution π N (m) is given by π N (m) = π N (0) H m m 0, where H is the minimal non-negative solution of the equation A 0 + H A 1 + H 2 A 2 = O, with A 0 = µ 1 U R, A 1 = (λ + µ 1 + µ 2 ) I + µ 1 U 0 + λ(u N + U L ) and A 2 = µ 2 I are square matrices of size (N + 1). The value of π N (0) is given by π N (0) = y y v t, where v t = (I H) 1 1 t and y is a probability vector, satisfying the condition y v t <, that solves the equation y (A 1 + A 2 + H A 2 ) = 0. B. D Auria (Univ. Carlos III de Madrid) 2-queues tandem network January 19 th 2012 21 / 38

Partially-observable case - Number of customers in the 1st queue The net profit is given by with P 1st,N (n) = R (C 1 C 2 ) T N,1 (n) C 2 T N (n), 0 n N + 1. T N,1 (n) = n µ 1 T N (n) = T (n, m) P(QN,2 = m Q N,1 = n 1) m=0 Using the marginal generating function φ(n, z) = T (n, m) z m, m=0 we can write the T N (n) function in the following way T N (n) = π N(0) ( m=0 T (n, m) Hm) e t n 1 π N (0) ( m=0 Hm) e t n 1 = π N(0) φ(n, H) e t n 1 π N (0) (I H) 1 e t n 1. B. D Auria (Univ. Carlos III de Madrid) 2-queues tandem network January 19 th 2012 22 / 38

Partially-observable case - Number of customers in the 1st queue We define the marginal generating function of y(n, m) It follows that φ(n, z) = = ψ and we remind that m=0 ψ(n, z) = ( n, m=0 y(n, m) z m, ( ) µ2 z m y(n, m) + µ 1 + µ 2 µ 2 z µ 1 + µ 2 ) z + n (1 z) + µ 2 (1 z) 2, z + n (1 z) µ 2 (1 z) 2 y(n, m) = 1 + µ 2 y(n 1, 1) + µ m 1 1 µ 2 µ 1 µ 1 + µ 2 (µ 1 + µ 2 ) 2 y(n 1, k + 2) k=0 B. D Auria (Univ. Carlos III de Madrid) 2-queues tandem network January 19 th 2012 23 / 38

Partially-observable case - Number of customers in the 1st queue We define the marginal generating function of y(n, m) Using the recursive ψ(n, equation z) = for y(n, them) function z m, y(n, m) we can compute ψ(n, z) in them=0 following way It follows that ( ) µ2 z m ψ(n z + n (1 z) φ(n, + z) 1, = z) = µ 2 µ 1 ψ(n, z) (µ y(n, 1 + m) µ 2 ) 2 + µ z 1 + zµ 2 µ 2 µ 1 y(n, 0) (µ 1 + 2 µ µ 2 m=0 2 (1 ) 2 z z) 2 z 2 ( ) µ 2 µ 1 y(n, 1) y(n + 1, 0) µ 2 z n (1 z) = ψ (µ n, 1 + µ 2 ) 2 +, 1+ z 1 z µ 1 + µ 2 µ 2 (1 z) 2, using that ψ(0, z) = 0. and we remind that y(n, m) = 1 + µ 2 y(n 1, 1) + µ m 1 1 µ 2 µ 1 µ 1 + µ 2 (µ 1 + µ 2 ) 2 y(n 1, k + 2) k=0 B. D Auria (Univ. Carlos III de Madrid) 2-queues tandem network January 19 th 2012 23 / 38

Partially-observable case - Number of customers in the 1st queue Theorem The equilibrium strategy is given by the smallest integer N such that P 1st,N (N + 1) 0. At equilibrium the system is equivalent to a similar tandem network that has the first queue with finite room for only a maximum of N customers. B. D Auria (Univ. Carlos III de Madrid) 2-queues tandem network January 19 th 2012 24 / 38

Number of customers in the 2nd queue B. D Auria (Univ. Carlos III de Madrid) 2-queues tandem network January 19 th 2012 25 / 38

Partially-observable case - Number of customers in the 2nd queue The arriving customers can only see the waiting line of the second queue, and in this case the number of users in each queue can be any non negative integer value. B. D Auria (Univ. Carlos III de Madrid) 2-queues tandem network January 19 th 2012 26 / 38

Proposition (in [Leskelä and Resing (2007)]) The stationary distribution π M (n) is given by π M (n) = π M (0) H n n 0, where H is the minimal non-negative solution of H n A n = O, n=0 where the (M + 1)-square matrices A n are given by A 0 = λi A 1 = µ 2 U L (λ + µ1 + µ2)i + µ 2 U 0 A 2 = µ 1 (U R + κ 1 U M ) A n+1 = µ 1 κ n U M with κ n = 1/n ( 2n 2 n 1 ) (µ1 /(µ 1 + µ 2 )) n 1 (µ 2 /(µ 1 + µ 2 )) n. B. D Auria (Univ. Carlos III de Madrid) 2-queues tandem network January 19 th 2012 27 / 38

Proposition (in [Leskelä and Resing (2007)]) The value of π M (0) is given by π M (0) = λ + µ 2 x (I H) 1 e t x, 0 where the row vector x is the unique positive solution of x H n B n = O, n=0 µ 2 satisfying x v t = 1 with v t = (I H) 1 1 t. The (M + 1)-square matrices B n are given by B 0 = µ 2 U L (λ + µ2)i + µ 2 U 0 B 1 = µ 1 (U R + U M ) B n+1 = µ 1 (1 κ 1 κ n ) U M. B. D Auria (Univ. Carlos III de Madrid) 2-queues tandem network January 19 th 2012 27 / 38

Partially-observable case - Number of customers in the 2nd queue The net profit is given by P 2nd,M (m) = R (C 1 C 2 ) T M,1 (m) C 2 T M (m), m 0. with T M,1 (m) = = T M (m) = = T 1 (n + 1, m) P(QM,1 = n Q M,2 = m) n=0 n=0 n + 1 π M (n, m) µ 1 π M (, m) T (n + 1, m) P(QM,1 = n Q M,2 = m) n=0 n=0 T (n + 1, m) π M(n, m) π M (, m) B. D Auria (Univ. Carlos III de Madrid) 2-queues tandem network January 19 th 2012 28 / 38

Partially-observable case - Number of customers in the 2nd queue Again, using the marginal generating function φ(z, m) = T (n + 1, m) z n, n=0 we can write the previous quontities in this way T M,1 (m) = 1 π M (0) (I H) 2 e t m µ 1 π M (0) (I H) 1 e t m T M (m) = π M(0) φ(h, m) e t m π M (0) (I H) 1 e t m B. D Auria (Univ. Carlos III de Madrid) 2-queues tandem network January 19 th 2012 29 / 38

Partially-observable case - Number of customers in the 2nd queue We define the marginal generating function of y(n, m) It follows that φ(z, m) = = and we remind that ( µ2 ψ(z, m) = y(n + 1, m) z n, n=0 ) m y(n + 1, m) z n + µ 1 + µ 2 n=0 ) m ψ(z, m) + ( µ2 µ 1 + µ 2 1 + m (1 z) µ 2 (1 z) 2. 1 + m (1 z) µ 2 (1 z) 2 y(n, m) = 1 + µ 2 y(n 1, 1) + µ m 1 1 µ 2 µ 1 µ 1 + µ 2 (µ 1 + µ 2 ) 2 y(n 1, k + 2) k=0 B. D Auria (Univ. Carlos III de Madrid) 2-queues tandem network January 19 th 2012 30 / 38

Partially-observable case - Number of customers in the 2nd queue The function ψ(z, m) satisfies the following difference equation ψ(z, m + 1) = µ 2µ 1 z ψ(z, m + 2) + ψ(z, m) m 0 (µ 1 + µ 2 ) 2 1 ψ(z, 0) = µ 1 (1 z) + µ 2 z ψ(z, 1). µ 1 + µ 2 and the following result holds Theorem The function ψ(z, m) has the following expression ψ(z, m) = ( 1 1 µ ) 2 z 1 a m (z) µ 1 (1 z) µ 1 + µ 2 a(z) where a(z) = 1/2 + 1/4 z (µ 1 µ 2 )/(µ 1 + µ 2 ) 2. B. D Auria (Univ. Carlos III de Madrid) 2-queues tandem network January 19 th 2012 31 / 38

Partially-observable case - Number of customers in the 2nd queue Assumtion - a tagged customer that sees M customers in the second queue, does not know the macroscopic behavior of the rest of the population, and s/he easily believes that the rest of customers is willing to enter with any number of customers equals to or less than the ones s/he is observing. Formally this means that, according to his/her observation, s/he believes that the current policy is the threshold policy M. Under this assumption the equilibrium policy is given by the minimum M such that P 2nd,M (m) < 0 for any m > M. B. D Auria (Univ. Carlos III de Madrid) 2-queues tandem network January 19 th 2012 32 / 38

Partially-observable cases - Numerical example We use the following parameters: the arrival rate is λ = 0.5 the service rates are µ 1 = 0.7 and µ 2 = 1.2. the reward is R = 20 the waiting costs for unit of time are C 1 = 1 and C 2 = 2. 15 10 5 N K M P tot x P 1st,x x 1 P 2nd,x x 0 Figure: Profit functions for the partially-observable cases B. D Auria (Univ. Carlos III de Madrid) 2-queues tandem network January 19 th 2012 33 / 38

Summary We have shown how the economical considerations of the customers may modify the dynamics of a 2-queue tandem network As expected, the behaviour of the customers is affected by the level of information they know when deciding to join or balk Different levels of information require different tools of analysis, unfortunately the system quickly becomes too involved and intractable loooking at more complex topologies. B. D Auria (Univ. Carlos III de Madrid) 2-queues tandem network January 19 th 2012 34 / 38

Thank you for your attention. B. D Auria (Univ. Carlos III de Madrid) 2-queues tandem network January 19 th 2012 35 / 38

B. D Auria and S. Kanta Equilibrium strategies in a tandem queue under various levels of information Working Paper 11-33 (25) at UC3M, Spain, submitted. http://e-archivo.uc3m.es/handle/10016/12262 P. Naor The regulation of queue size by levying tolls Econometrica 37:15 24, 1969. N. M. Edelson and K. Hildebrand Congestion tolls for Poisson queueing processes. Econometrica 43:81 92, 1975. D. Kroese, W. Scheinhardt and P. Taylor Spectral properties of the tandem Jackson network, seen as a quasi-birth-and-death process The Annals of Applied Probability 14(4):2057 2089, 2004. B. D Auria (Univ. Carlos III de Madrid) 2-queues tandem network January 19 th 2012 36 / 38

L. Leskelä and J. Resing A tandem queueing network with feedback admission control In Network Control and Optimization - Proceedings of the First EuroFGI International Conference (NET-COOP), T. Chahed and B. Tuffin, editors, volume 4465 of Lecture Notes in Computer Science. Springer-Verlag, pp. 129 137, 2007. A.Economou and S. Kanta Optimal balking strategies and pricing for the single server Markovian queue with compartmented waiting space Queueing Systems 59:237 269, 2007 R. Hassin and M. Haviv To Queue or Not to Queue: Equilibrium Behavior in Queueing Systems Kluwer, Boston, 2003. B. D Auria (Univ. Carlos III de Madrid) 2-queues tandem network January 19 th 2012 36 / 38