Lectures on Stochastc Stablty Sergey FOSS Herot-Watt Unversty Lecture 5 Monotoncty and Saturaton Rule Introducton The paper of Loynes [8] was the frst to consder a system (sngle server queue, and, later, queues n tandem wth statonary-ergodc drver. The classcal recurson X n+ = (X n + ξ n + studed by Loynes s monotone. There s a varety of monotone models (ncludng queues n tandem, mult-server queues, Jackson-type networks, etc. for whch one can develop a unfed approach for stablty study. Here we provde a short survey, see the references for more detals. 2 Sngle-server queue revsted Consder a sngle server queue wth nterarrval tmes σ n and servce tmes t n. Assume that a sequence {(σ n, t n } s statonary wth fnte means b = Eσ and a = Et and satsfes the SLLN,.e. n σ = b and n t = a a.s. ( A suffcent condton for ( to hold (but not necessary n general see, e.g., a dscusson n [6] s that a sequence {(σ n, t n } s ergodc (see, e.g., Lecture for defntons. Assume further that b < a. Assume also, for smplcty, that customer arrves n an empty queue. Let W n be a watng tme of customer n. Then W = 0 and Also, {W n } satsfy the followng recurson n W n = max (σ t. n = W n+ = max(0, W n + σ n t n. (2
Note that W n concdes n dstrbuton wth W n θ = max + 0 = (σ t. The latter sequence ncreases a.s. and couples wth an a.s. fnte lmt W 0 = sup 0 = (σ t due to the SLLN (. One can also defne, for any m, Then {W m } s a statonary sequence and W m = sup (σ t. m = W m+ = max(0, W m + σ m t m. Thus, ths s a statonary soluton to recursve equaton (2. Exercse. Show that ths s the only statonary soluton. 3 Tandem of two sngle-server queues Tandem queue wll be our toy example. Consder an open network wth two sngle-server statons n tandem. Customers arrve to the frst staton wth nterarrval tmes {t n } and form a queue there. In ths example, t s convenent to assume that t n s an nterarrval tme between customers n and n. A server serves them n order of arrval wth servce tmes {σ n ( }. Upon servce completon, customers go to the second staton where are served also n order of arrval wth servce tmes {σ n (2 }. Assume that customer arrves n an empty system. Denote by Z n a soourn tme of customer n. Then Z = σ ( + σ (2 and, more generally, for any n, Z n = max k m n Exercse 2. Prove formula (3. ( m k σ ( + n m σ (2 n t It s known that f a sequence {(σ n (, σ n (2, t n } s statonary ergodc and f max(b (, b (2 a k+ (3 < (4 where b ( = Eσ ( and a = Et, then a dstrbuton of Z n converges to a lmtng statonary dstrrbuon n the total varaton norm. 2
Exercse 3. Show that f there s the opposte strct nequalty n (4, then a sequence {Z n } tends to nfnty a.s. In order to ntroduce a recursve scheme, let Z n ( be a soourn tme of customer n n the frst system (.e. a sum of a watng tme and of servce tme σ n (. Then Z ( = σ ( and we get the followng recursve relatons: Z ( n+ = max(0, Z ( n t n+ + σ ( n+, Z n+ = max(z ( n+, Z n t n+ + σ (2 n+. In other words, ntroduce a functon f : R 5 R 2 as follows: f(x, x 2, y, y 2, y 3 = (max(0, x y 3 + y, max(max(0, x y 3 + y, x 2 y 3 + y 2. Then f s monotone non-decreasng n (x, x 2, and we get a recurson where ξ n+ = (σ ( n+, σ(2 n+, t n+. (Z ( n+, Z n+ = f((z ( n, Z n, ξ n+ 4 General statements on monotone and homogeneous recursons There are many applcatons, especally n queueng networks, where monotoncty n the dynamcs can be exploted to prove exstence and unqueness of statonary solutons. Although the theory can be presented n the very general setup of a partally ordered state space (see Brandt et al. [6] we wll only focus on the case where the state s R d. Consder then the SRS X n+ = f(x n, ξ n+ =: ϕ n+ (X n and assume that ϕ 0 : R d + R d + s ncreasng and rght-contnuous, where the orderng s the standard component-wse orderng on R d +. Let θ be statonary and ergodc flow on (Ω, F, P and assume that ϕ n = ϕ 0 θ n, n Z. In other words, {ϕ n } s a statonary-ergodc sequence of random elements of the space of rght-contnuous ncreasng functons on R d +. We frst explan Loynes method. Defne Φ n := ϕ n ϕ. Thus, Φ n (Y s the soluton of the SRS at n 0 when X 0 = Y, a.s. Snce 0 s the least element of (R d +,, we have Φ n (0 Φ n (Y, a.s., for any R d + valued r.v. Y. Next consder Φ m+n (0 θ m = ϕ n ϕ m+ (0, n m, and nterpret Φ m+n (0 as the soluton of the SRS at tme n m, startng wth 0 at tme m. Clearly, Φ m+n (0 ncreases as m ncreases, because: Φ (m++n (0 θ (m+ = ϕ n ϕ m+ ϕ (m+ (0 = ϕ n ϕ m+ (ϕ (m+ (0 ϕ n ϕ m+ (0 = Φ m+n (0 θ m. 3
Fnally defne X n := lm m Φ m+n(0 θ m, n Z. The r.v. X n s ether fnte a.s., or s nfnte a.s., by ergodcty. Assumng that the frst case holds, we further have X n+ = lm m Φ m+(n+(0 θ m = lm m ϕ m+n+ϕ m+n ϕ (0 θ m = lm m ϕ n+ϕ n ϕ m+ (0 = lm m ϕ n+(ϕ n ϕ m+ (0 = lm m ϕ n+(φ m+n (0 θ m+ = ϕ n+ ( X n. Provdes then that we have a method for provng P(X 0 < > 0, Loynes technque results n the constructon of a statonary-ergodc soluton { X n } of the SRS. For a tandem queue, we get X n = (Z n (, Z n θ = σ ( 0 + max + 0 (σ ( = t, max + k 0 ( k σ ( + k σ (2 + t and clearly ths sequence ncreases a.s. Here, by conventon, r... = 0 s > r. Consder another example of a multserver queue. Example. A multserver queue G/G/s wth s servers and FCFS servce dscplne. Customers arrve wth nterarrval tmes {t n } and have servce tmes {σ n } (servce tmes are assocated wth customers, not wth servers. Upon arrval to the system, a customer s mmedately sent to a server (one of servers wth a mnmal workload. At each server, customers are served n order of arrval. Denote by V n, a workload of server ust before arrval of nth customer. Let R be an operator that orders coordnates of a vector n the non-decreasng order. Let W n = R(V n,,... V n,s. Then vectors {W n } satsfy the followng recursve equaton ( Kefer-Wolfowtz : W n+ = R(W n + e σ n t n+ + Here x + = max(x, 0 (coordnatewse, and e = (, 0,..., 0 and = (,,...,. Exercse 4. Show the monotoncty of the latter recurson. Return to the general settng. Wthout further assumptons and structure, not much can be sad. Assume next that, n addton, ϕ 0 s homogeneous,.e., ϕ 0 (x + c = ϕ 0 (x + c, 4
for all x R d + and all c R. Such s the case, e.g., wth the usual Lndley functon ϕ 0 : R + R +, wth ϕ 0 (x = max(x+ξ 0, 0. The homogenety assumpton s qute frequent n queueng theory. It s easy to see that ϕ 0 (x ϕ 0 (y x y, where x := max( x,..., x d. Suppose then that {X n }, {Y n } are two statonary solutons of the SRS. Then X n+ Y n+ = ϕ n+ (X n ϕ n+ (Y n X n Y n, (5 for all n, a.s., and snce { X n Y n, n Z} s statonary and ergodc, ths a.s. monotoncty may only hold f X n Y n = r, for some constant r 0. Thus, a necessary and suffcent condton for the two solutons to concde s that, P( ϕ (X 0 ϕ (Y 0 < X 0 Y 0 > 0. (6 Remark. Condtons (5 and (6 form the basc for the so-called contracton approach. A classcal example where, n general, (5 holds but (6 fals s the G/G/s queue, that s, the s-server queue wth statonary-ergodc data. Let λ, µ be the arrval and servce rates, respectvely. Here, there s a mnmal and a maxmal statonary soluton whch, provded that λ < sµ, may not concde. For detals see Brandt et al [6]. But condton (6 holds f the drvng sequences are..d. (or satsfy a weaker mxng condton. Also condton (6 holds for the tandem queue wth statonary and ergodc drvng sequences. Ths s Exercse 5 to you. 5 The Monotone-Homogeneous-Separable (MHS framework Consder a recurson of the form W n+ = f(w n, ξ n+, τ n+, where ξ n are general marks, and τ n 0. The nterpretaton s that τ n s the nterarrval tme between the n -th and n-th customer, and W n s the state ust after the arrval of the n-th customer. We consder arrval epochs {T n } such that T n+ T n = τ n. We wrte W m,n for the soluton of the recurson at ndex n when we start wth a specfc state, say 0, at m n. Fnally we consder a functons of the form X [m,n] = f m+n (W m,n ; T m,..., T n ; ξ m+,..., ξ n, whch wll be thought of as epochs of last actvty n the system. For nstance, when we have an s-server queue, X[m, n] represents the departure tme of the last customer when the queue s fed only by customers wth ndces from m to n. Correspondngly, we defne the quantty Z [m,n] := X [m,n] T n, the tme elapsed between the arrval of the last customer and the departure of the last customer. The framework s formulated n terms of the X [m,n], Z [m,n] and ther dependence on the {T n }. For c R, let {T n } + c = {T n + c}. For c > 0, let c{t n } = {ct n }. Defne {T n } {T n} f T n T n for all n. We requre a set of four assumptons: 5
(A Z [m,n] 0 (A2 {T n } {T n} X [m,n] X [m,n]. The frst assumpton s natural. In the second one, X [m,n] are the varables obtaned by replacng each T n by T n; t says that delayng the arrval epochs results n delayng of the last actvty epochs. (A3 {T n} = {T n } + c X [m,n] = X [m,n] + c. Ths s a tme-homogenety assumpton. (A4 For m l < l + n, X [m,l] T l+ X [m,n] = X [l+,n]. If the premse X [m,l] T l+ of the last assumpton holds, we say that we have separablty at ndex l. It means that the last actvty due to customers wth ndces n [m, l] happens pror to the arrval of the l + -th customer, and so the last actvty due to customers wth ndces n [m, n] s not nfluenced by those customers wth ndces n [m, l]. Basc consequences of the above assumptons are summarzed n: Lemma. ( The response Z [m,n] depends on T m,..., T n only through the dfferences τ m,..., τ n. ( Let a b be ntegers. Let T n = T n + Z [a,b] (n > b, T n = T n Z [a,b] (n b. And let X [m,n], X [m,n] be the correspondng last actvty epochs. Then both of them exhbt separablty at ndex b. ( The varables X [m,n], Z [m,n] ncrease when m decreases. (v For a b < b + c, Z [a,c] Z [a,b] + Z [b+,c]. Proof. ( Follows from the defnton Z [m,n] = X [m,n] T n and the homogenety assumpton (A3. ( Obvously, Z [a,b] τ b + Z [a,b], and so X [a,b] T b τ b + Z [a,b], whch mples X [a,b] T b+ + Z [a,b]. The rght-hand sde s T b+, by defnton. The left-hand sde s equal to X [a,b] because T n = T n for n b. So X [a,b] T b+ and ths s separablty at ndex b. Smlarly for the other varable. ( Let a = b = m n (. Snce we have separablty at ndex m, we conclude that X [m,n] = X [m+,n]. But T k = T k for k [m +, n] and so X [m+,n] = X [m+,n]. On the other hand, {T k } {T k} and so, by (A2, X [m,n] X [m,n]. Thus X [m,n] X [m+,n]. And so Z [m,n] Z [m+,n] also. (v Apply ( agan. Snce {T k } {T k }, (A2 gves X [a,c] X [a,c]. By separablty at ndex b, as proved n (, we have X [a,c] = X [b+,c]. Because T k = T k + Z [a,b] for all k [b +, c], we have, by (A3, X [b+,c] = X [b+,c] + Z [a,b]. Thus, X [a,c] X [b+,c] + Z [a,b]. Subtractng T c from both sdes gves the desred. Introduce next the usual statonary-ergodc assumptons. Namely, consder (Ω, F, P and a statonary-ergodc flow θ. Let ξ n = ξ 0 θ n, τ n = τ 0 θ n, set T 0 = 0, and suppose Eτ 0 = λ (0,, EZ 0,0 <. Stablty of the orgnal system can, n specfc but mportant cases, be translated n a stablty statement for Z [m,n]. Hence we shall focus on t. Note that Z [m,n] θ k = Z [m+k,n+k] for all k Z. For any c 0, ntroduce the epochs 6
c{t n } = {ct n } and let X [m,n] (c, Z [m,n] (c be the quanttes of nterest. The subaddtve ergodc theorem gves that γ(c := lm n n Z [, ](c = lm n n EZ [, ](c s a nonnegatve, fnte constant. The prevous lemma mples that γ(c γ(c when c > c. Smlarly, lm n X [,n] (c = γ(c + λ c, and the latter quantty ncreases as c ncreases. Monotoncty mples that Z [, ] (c ncreases as n ncreases, and let Z(c be the lmt. Ergodcty mples that P( Z(c < {0, }. Put Z = Z(. The stablty theorem s: Theorem. If λγ(0 < then P( Z < =. If λγ(0 > then P( Z < = 0. Proof. Assume frst that λγ(0 >. Fx n. Defne T k = T for all k Z. Hence X [,0] ( X [,0]( = Z [,0] (, by (A2. On the other hand, by (A3, X [,0] ( = X [,0] (0 + T = Z [,0] (0 + T. Thus, n Z [,0] ( n Z [,0] (0 + n T, and, takng lmts as n, we conclude lm nf n Z [,0] ( γ(0 λ > 0, a.s. Assume next that λγ(0 <. Let γ n (0 := EZ [+,0] (0/n. Snce γ(0 = lm n γ n (0 = nf n γ n (0, we can fnd an nteger K such that λγ K (0 <. Consder next an auxlary sngle server queue wth servce tmes σn := Z [ Kn+, K(n ] (0 and nterarrval tmes t n := K(n = Kn+ t. Notce that {(t n, s n, n Z} s a statonary sequence whch satsfes the SLLN. Consder the watng tme W n of ths auxlary system: W n+ = (W n +s n t n +. Snce Es n = γ K < λ = Et n, the auxlary queue s stable. Snce the separablty property holds, we have the followng domnaton: Z [K+,0] ( W n θ + s 0, a.s., where W n here s the watng tme of the n-th customer f the queue starts empty. By the Loynes scheme, W n θ converges (ncreases to an a.s. fnte random varable. Hence Z = lm n Z [K+,0] ( s also a.s. fnte. Example Consder a tandem queue and fnd γ(0. Let b = max(b (, b (2. Then ( m γ(0 = lm n n max σ ( 0 m n + σ (2 m = b. lm n max ( σ (, σ (2 From the other sde, assume that, say, b = b ( > b (2. Then ( γ(0 = lm σ ( + max (σ (2 n 0 m n σ ( lm n Ths s known as the saturaton rule σ ( + n sup m 0 m m (σ (2 σ ( 7
where the supremum n the RHS s fnte a.s. and does not depend on n. Therefore, the second term tends to 0 a.s. Thus, γ(0 = b. The same concluson holds f b ( < b (2 (by the symmetry and f b ( = b (2 (Why?? ths s Exercse 6 for you! Exercse 7. Usng Theorem, fnd stablty condtons for G/G/s queue. 6 Saturaton rule for large devatons The proposed constructon of an upper sngle-server queue may be of use not only for stablty study, but also for study of large devatons. We consder here only an example of a tandem queue. In ths secton, we assume that three drvng sequences {σ n ( }, {σ n (2 }, and {t n } are mutually ndependent and each of them conssts of..d.r.v. s. We assume also that the stablty condton a > b holds. We are nterested n the asymptotcs for P(Z > x as x. For the smplcty, we consder only the case when both dstrbutons of random varables σ ( and σ (2 are lght-taled, and we study only the logarthmc asymptotcs: log P(Z > x.... For =, 2, let ϕ ( (u = E exp(uσ ( and ϕ τ (u = E exp(ut. Let γ ( = sup{u : ϕ ( (uϕ τ ( u }. Theorem 2. If both γ ( and γ (2 are postve, then where γ = mn(γ (, γ (2. Sketch of Proof. Snce Z Z (, lm sup x log P(Z > x γx log P(Z > x x γ (. Smlarly, consder an auxlary system where servce tmes n the frst queue are replaced by zeros. Then we get a sngle server queue wth servce tmes {σ n (2 } and nterarrval tmes {t n }. If we denote by Z (2 a statonary servce tme n ths system, then Z (2 Z and, therefore, log P(Z > x lm sup γ (2. x x Thus, log P(Z > x lm sup γ. x To obtan the lower bound, we take a suffcently large K (see the proof of Theorem and an auxlary upper sngle server queue. If we let γ = sup{u : ϕ σ (uϕ t ( u }, then one can show that log P(Z>x (alm nf x γ, (b one can choose γ as close to γ as possble. 8
Exercse 8. Complete the proof of Theorem 2. Remark. The exact asymptotcs for P(Z > x n the heavy tal case have been found n [3]. References [] Baccell, F. and Brémaud, P. (2003 Elements of Queueng Theory. Sprnger, Berln. [2] Baccell, F. and Foss, S. (995 On the Saturaton Rule for the Stablty of Queues. J. Appl. Probab. 23, n.2, 494-507. [3] Baccell F, and Foss, S. (2004 Moments and tals n monotone-separable stochastc networks. Ann. of Appl. Probab. 4, 62-650. [4] Borovkov, A.A. (998 Ergodcty and Stablty of Stochastc Processes. Wley, New York. [5] Borovkov, A.A. and Foss, S.G. (992 Stochastcally recursve sequences and ther generalzatons. Sb. Adv. Math. 2, n., 6-8. [6] Brandt, A., Franken, P. and Lsek, B. (992 Statonary Stochastc Models. Wley, New York. [7] Dacons, P. and Freedman, P. (999 Iterated random functons. SIAM Revew 4 45-76. [8] Loynes, R.M. (962 The stablty of queues wth non-ndependent nterarrval and servce tmes. Proc. Cambrdge Phl. Soc. 58, 497-520. [9] Meyn, S. and Tweede, R. (993 Markov Chans and Stochastc Stablty. Sprnger, New York. [0] Whtt, W. (2002 Stochastc-Process Lmts. Sprnger, New York. 9