arxiv: v1 [cs.sy] 2 Sep 2014

Transcription

1 Noname manuscrp No. wll be nsered by he edor Sgnalng for Decenralzed Roung n a Queueng Nework Y Ouyang Demoshens Tenekezs Receved: dae / Acceped: dae arxv: v [cs.sy] Sep 04 Absrac A dscree-me decenralzed roung problem n a servce sysem conssng of wo servce saons and wo conrollers s nvesgaed. Each conroller s afflaed wh one saon. Each saon has an nfne sze buffer. Exogenous cusomer arrvals a each saon occur wh rae λ. Servce mes a each saon have rae µ. A any me, a conroller can roue one of he cusomers wang n s own saon o he oher saon. Each conroller knows perfecly he queue lengh n s own saon and observes he exogenous arrvals o s own saon as well as he arrvals of cusomers sen from he oher saon. A he begnnng, each conroller has a probably mass funcon PMF on he number of cusomers n he oher saon. These PMFs are common knowledge beween he wo conrollers. A each me a holdng cos s ncurred a each saon due o he cusomers wang a ha saon. The objecve s o deermne roung polces for he wo conrollers ha mnmze eher he oal expeced holdng cos over a fne horzon or he average cos per un me over an nfne horzon. In hs problem here s mplc communcaon beween he wo conrollers; whenever a conroller decdes o send or no o send a cusomer from s own saon o he oher saon communcaes nformaon abou s queue lengh o he oher saon. Ths mplc communcaon hrough conrol acons s referred o as sgnalng n decenralzed conrol. Sgnalng resuls n complex communcaon and decson Prelmnary versons of hs paper appeared n Ouyang and Tenekezs 03 and Ouyang and Tenekezs 04. Ths work was suppored n par by Naonal Scence Foundaon NSF Gran CCF-06 and NASA gran NNXAO54G. Y. Ouyang Deparmen of Elecrcal Engneerng and Compuer Scence, Unversy of Mchgan, Ann Arbor, MI E-mal: ouyangy@umch.edu D. Tenekezs Deparmen of Elecrcal Engneerng and Compuer Scence, Unversy of Mchgan, Ann Arbor, MI E-mal: eneke@umch.edu

2 Y Ouyang, Demoshens Tenekezs problems. In spe of he complexy of sgnalng nvolved, s shown ha an opmal sgnalng sraegy s descrbed by a hreshold polcy whch depends on he common nformaon beween he wo conrollers; hs hreshold polcy s explcly deermned. Keywords Decenralzed Sysem Non-classcal Informaon Srucure Sgnalng Queueng Neworks Common Informaon Inroducon Roung problems o parallel queues arse n many modern echnologcal sysems such as communcaon, ransporaon and sensor neworks. The majory of he leraure on opmal roung n parallel queues addresses suaons where he nformaon s cenralzed, eher perfec see Akgun e al 0; Davs 977; Ephremdes e al 980; Foley and McDonald 00; Hajek 984; Hordjk and Koole 990, 99; Ln and Kumar 984; Mench and Serfozo 99; Weber 978; Weber and Sdham Jr 987; Wh 986; Wnson 977 and references heren or mperfec see Beuler and Tenekezs 989; Kur and Kumar 995 and references heren. Very few resuls on opmal roung o parallel queues under decenralzed nformaon are currenly avalable. The auhors of Cogll e al 006 presen a heursc approach o decenralzed roung n parallel queues. In Abdollah and Khorasan 008; Manfred 04; Reddy e al 0; S e al 03; Yng and Shakkoa 0 and references heren, decenralzed roung polces ha sablze queueng neworks are consdered. The work n Pandels and Tenekezs 996 presens an opmal polcy o a roung problem wh a one-un delay sharng nformaon srucure. In hs paper we nvesgae a decenralzed roung problem n dscree me. We consder a sysem conssng of wo servce saons/queues, called Q and Q and wo conrollers, called C and C. Conroller C resp. C s afflaed wh servce saon Q resp. Q. Each saon has an nfne sze buffer. The processes descrbng exogenous cusomer arrvals a each saon are ndependen Bernoull wh parameer λ. The random varables descrbng he servce mes a each saon are ndependen geomerc wh parameer µ. A any me each conroller can roue one of he cusomers wang n s own queue o he oher saon. Each conroller knows perfecly he queue lengh n s own saon, and observes he exogenous arrvals n s own saon as well as he arrvals of cusomers sen from he oher saon. A he begnnng, conroller C resp. C has a probably mass funcon PMF on he number of cusomers n saon Q resp. Q. These PMFs are common knowledge beween he conrollers. A each me a holdng cos s ncurred a each saon due o he cusomers wang a ha saon. The objecve s o deermne decenralzed roung polces for he wo conrollers ha mnmze eher he oal expeced holdng cos over a fne horzon or he average cos per un me over an nfne horzon. Prelmnary versons of hs paper appeared n Ouyang and Tenekezs 03 for he fne horzon problem and

3 Sgnalng for Decenralzed Roung n a Queueng Nework 3 Ouyang and Tenekezs 04 for he nfne horzon average cos per un me problem. In he above descrbed roung problem, each conroller has dfferen nformaon. Furhermore, he conrol acons/roung decsons of one conroller affec he nformaon of he oher conroller. Thus, he nformaon srucure of hs decenralzed roung problem s non-classcal wh conrol sharng see Mahajan 03 for non-classcal conrol sharng nformaon srucures. Non-classcal nformaon srucures resul n challengng sgnalng problems see Ho 980. Sgnalng occurs hrough he roung decsons of he conrollers. Sgnalng s, n essence, a real-me encodng/communcaon problem whn he conex of a decson makng problem. By sendng or no sendng a cusomer from Q resp. Q o Q resp. Q conroller C resp. C communcaes a each me nsan a compressed verson of s queue lengh o C resp. C. For example, by sendng a cusomer from Q o Q a me, C may sgnal o C ha Q s queue lengh s above a pre-specfed hreshold l. Ths nformaon allows C o have a beer esmae of Q s queue lengh and, herefore, make beer roung decsons abou he cusomers n s own queue; he same argumens hold for he sgnals send hrough roung decsons from C o C. Thus, sgnalng hrough roung decsons has a rple funcon: communcaon, esmaon and conrol. Whn he conex of he problems descrbed above, here s enormous number of sgnalng possbles. For example, here s an arbrarly large number of choces of he sequences of pre-specfed hresholds l, l,..., l,... and hese choces are only a small subse of all he possble sequences of bnary parons of he se of non-negave negers ha descrbe all choces avalable o C and C. All hese possbles resul n hghly non-rval decson makng problems. I s he presence of sgnalng ha dsngushes he problem formulaed n hs paper from all oher roung problems n parallel queues nvesgaed so far. Some basc quesons assocaed wh he analyss of hs problem are: Wha s an nformaon sae suffcen sasc for each conroller? How s sgnalng ncorporaed n he evoluon/updae of he nformaon sae? Is here an explc descrpon of an opmal sgnalng sraegy? We wll answer hese quesons n Secon 3-6 and wll dscuss hem furher n Secon 7. Conrbuon of he paper The sgnalng feaure of our problem dsngushes from all prevous roung problems n parallel queues. In spe of he complexy of sgnalng, we show ha an opmal decenralzed sraegy s descrbed by a sngle hreshold roung polcy where he hreshold depends on he common nformaon beween he wo conrollers. We explcly deermne hs hreshold va smple compuaons.

4 4 Y Ouyang, Demoshens Tenekezs Saon Saon Fg. The Queueng Sysem Organzaon The res of he paper s organzed as follows. In Secon we presen he model for he queueng sysem and formulae he fne horzon and nfne horzon decenralzed roung problems. In Secon 3 we presen srucural resuls for opmal polces. In Secon 4 we presen a specfc decenralzed roung polcy, whch we call ĝ, and sae some feaures assocaed wh s performance. In Secon 5, we show ha when he nal queue lenghs n Q and Q are equal, ĝ s an opmal polcy for he fne horzon decenralzed roung problem. In Secon 6, we show ha ĝ s an opmal decenralzed roung polcy for he nfne horzon average cos per un me problem. We conclude n Secon 7. Noaon Random varables r.v.s are denoed by upper case leers, her realzaon by he correspondng lower case leer. In general, subscrps are used as me ndex whle superscrps are used o ndex servce saons. For me ndces, X : s he shor hand noaon for X, X +,..., X. For a polcy g, we use X g o denoe ha he r.v. X g depends on he choce of polcy g. We use vecors n R Z+ o denoe PMFs Probably Mass Funcons, where Z + denoes he se of non-negave negers. We also use a consan n Z + o denoe he corner PMF ha represens a consan r.v...e. a consan c Z + denoes he PMF whose enres are all zero excep he ch. Sysem Model and Problem Formulaon Sysem Model The queueng/servce sysem shown n Fgure, operaes n dscree me. The sysem consss of wo servce saons/queues, Q and Q wh nfne sze buffers. Conrollers C and C are afflaed wh queues Q and Q,

5 Sgnalng for Decenralzed Roung n a Queueng Nework 5 respecvely. Le X denoe he number of cusomers wang, or n servce, n Q, =,, a he begnnng of me. Exogenous cusomer arrvals a Q, =,, occur accordng o a Bernoull process {A, Z + } wh parameer λ. Servce mes of cusomers a Q, =, are descrbed by geomerc random varables wh parameer µ. We defne a Bernoull process {D, Z + } wh parameer µ. Then {D {X 0}, Z + } descrbes he cusomer deparure process from Q, =,. A any me, a conroller can roue one of he cusomers n s own queue o he oher queue. Le U denoe he roung decson of conroller C a =, ; f U = resp. 0 one cusomer resp. no cusomer s roued from Q o Q j j. A any me, each conroller C, =,, knows perfecly he number of cusomers X0:, =,, n s own queue; furhermore, observes perfecly he arrval sream A 0: o s own queue, and he arrvals due o cusomers roued o s queue from he oher servce saon up o me,.e. U j 0:, j. The order of arrvals A, deparures D and conroller decsons U concernng he roung of cusomers from one queue o he oher s shown n Fgure. The dynamc evoluon of he number of cusomers X, =, s descrbed by where for =,, X + = X U + U, X + = X U + U, X = X D + + A, 3 and x + := max0, x. We assume ha he nal queue lenghs X 0, X 0 and he processes {A, Z + }, {A, Z + }, {D, Z + }, {D, Z + } are muually ndependen and her dsrbuons are known by boh conrollers C and C. Le π 0 and π 0 be he PMFs on he nal queue lenghs X 0, X 0, respecvely. A he begnnng of me = 0, C resp. C knows X 0 resp. X 0. Furhermore C s resp. C s knowledge of he queue lengh X 0 resp. X 0 a he oher saon s descrbed by he PMF π 0 resp. π 0. The nformaon of conroller C, =,, a he momen makes he decson U, = 0,,..., s I := { } X0:, A 0:, X 0:, U0:, U0:, π0, π0, =,. 4 The conrollers roung decsons/conrol acons U are generaed accordng o where U = g I, =,, Z+, 5 g :Z + + {0, } + Z + + {0, } {0, } R Z+ R Z+ U. 6

6 6 Y Ouyang, Demoshens Tenekezs Fg. The order of varables The conrol acon space U a me depends on X. Specfcally U = { {0} when X = 0, {0, } oherwse. 7 Defne G d o be he se of feasble decenralzed roung polces; ha s G d = {g, g : g = g 0, g,..., g,..., =, and g s of form gven by 5-6}. 8 We sudy he operaon of he sysem defned n hs secon, frs over a fne horzon, hen over an nfne horzon.. The fne horzon problem For he problem wh a fne horzon T, we assume he holdng cos ncurred by he cusomers presen n Q a me = 0,,..., T s c X, =,, where c s a convex and ncreasng funcon. Then, he objecve s o deermne decenralzed roung polces g G d so as o mnmze J g T π 0, π 0 := E [ T c =0 X,g + c for any PMFs π 0, π 0 on he nal queue lenghs. X,g π 0, π 0 ] 9. The nfne horzon average cos per un me problem For he nfne horzon average cos per un me problem, we assume he holdng cos ncurred by he cusomers presen n Q a each me s a convex and ncreasng funcon c := c, =,. Then, he objecve s o The expecaon n all equaons appearng n hs paper s wh respec o he probably measure nduced by he polcy g G d.

7 Sgnalng for Decenralzed Roung n a Queueng Nework 7 deermne decenralzed roung polces g = g, g G d so as o mnmze J g π0, π0 := lm sup T T J g T π 0, π0 [ T = lm sup T T E c =0 X,g + c X,g π 0, π 0 ] 0 for any PMFs π 0, π 0 on he nal queue lenghs. 3 Qualave Properes of Opmal Polces In hs secon we presen a qualave propery of an opmal roung polcy for boh he fne horzon and he nfne horzon problem. For ha maer we frs nroduce he followng noaon. We denoe by Π and Π he PMFs on X and X, respecvely, condonal on all prevous decsons {U 0:, U 0: }. Π, =, s defned by Π x := P X = x U 0:, U 0:, x Z+. Smlarly, we defne he condonal PMFs Π, Π on X and X, respecvely, as follows. Π x := P X = x U0:, U0:, =,, x Z +. Noe ha for any polcy g G d all he above defned PMFs are funcons of {U 0:, U 0: }. Snce boh conrollers C and C know {U 0:, U 0: } a me, he PMFs defned by - are common knowledge Aumann 976 beween he conrollers. We ake X, =,, o be saon Q s sae a me. Combnng -3 we oban, for =,, X + = :=f X + U + U j D+ + A + X, U, U j, W, 3 where he random varables W := A +, D+, =,, = 0,,... are muually ndependen. The holdng cos a me, = 0,,... can be wren as ρ X, X, U, U :=c + X U + U + c + X U + U =c + X + + c+ X +. 4

8 8 Y Ouyang, Demoshens Tenekezs Noe ha for any me horzon T he oal expeced holdng cos due o 4 s equvalen o he oal expeced holdng cos defned by 9 snce for any polcy g G d =E =E =E J g T π 0, π 0 [ T c =0 [ T =0 X,g + c X,g ] c ] + X,g + + c + X,g + + E [ c 0 X 0 + c0 X 0 ] [ T =0 ρ X,g, X,g, U, U ] + E [ c 0 X 0 + c0 X 0 ]. 5 Wh he above noaon and defnon of sysem sae and nsananeous holdng cos, we have a dynamc eam problem wh non-classcal nformaon srucure where he common nformaon beween he wo conrollers a any me s her decsons/conrol acons up o me. Ths nformaon srucure s he conrol sharng nformaon srucure nvesgaed n Mahajan 03. Furhermore, he ndependen assumpon we made abou he exogenous arrvals and he servce processes s he same as he assumpons made abou he nose varables n Mahajan 03. Therefore, he followng Properes -3 hold by he resuls n Mahajan 03. Propery For each, and any gven gs., gs., s, we have P I =, I = U0:, U0: =P I = U0:, U0: P I = U0:, U0:. 6 Proof Same as ha of Proposon n Mahajan 03. Propery says ha he wo subsysems are ndependen condonal on pas conrol acons. Because of Propery and 3, each conroller C, =, can generae s decson a any me by usng only s curren local sae X and pas decsons of boh conrollers. Ths asseron s esablshed by he followng propery. Propery For he roung problems formulaed n Secon, whou loss of opmaly we can resrc aenon o roung polces of he form U = g X, U0:, U0:, 7 U = g X, U0:, U0:. 8 Proof Same as ha of Proposon n Mahajan 03.

9 Sgnalng for Decenralzed Roung n a Queueng Nework 9 Usng he common nformaon approach n Nayyar e al 03, we can refne he resul of Propery as follows. Propery 3 For he wo roung problems formulaed n Secon, whou loss of opmaly we can resrc aenon o roung polces of he form U = g X, Π, Π, 9 U = g X, Π, Π. 0 Proof Same as ha of Theorem n Mahajan 03. The resul of Propery 3 wll play a cenral role n he analyss of he decenralzed roung problems formulaed n hs paper. 4 The Decenralzed Polcy ĝ and Prelmnary resuls In hs secon, we specfy a decenralzed polcy ĝ and denfy an nformaon sae for each conroller. Furhermore, we develop some prelmnary resuls for boh he fne horzon problem and he nfne horzon problem formulaed n Secon. To specfy polcy ĝ, we frs defne he upper bound and lower bound on he suppor of he PMF, Π, =, as UB := maxx : Π x 0, LB := mnx : Π x 0. UB := maxub, UB, 3 LB := mnlb, LB. 4 Smlarly, we defne he bounds on he suppor of he PMF, Π, =, as where UB := maxx : Π x 0, 5 LB := mnx : Π x 0, 6 UB := maxub, UB, 7 LB := mnub, UB. 8 Usng he above bounds, we specfy he polcy ĝ := ĝ, ĝ as follows: { U = ĝ X, when X T H, UB, LB =, 0, when X 9 < T H, T H = UB + LB. 30

10 0 Y Ouyang, Demoshens Tenekezs Under ĝ, each conroller roues a cusomer o he oher queue when X, =,, he queue lengh of s own saon a he me of decson, s greaer han or equal o he hreshold gven by 30. Noe ha hs decenralzed roung polcy ĝ s ndeed of he form assered by Propery 3 snce he upper and lower bounds UB and LB are boh funcons of he PMFs Π, Π. Therefore, he hreshold T H, as a funcon of Π, Π, s common knowledge beween he conrollers. Usng he common nformaon, each conroller can compue he hreshold accordng o 30 ndvdually, and ĝ can be mplemened n a decenralzed manner. Under polcy ĝ, he evoluon of he bounds defned by 3-8 are deermned by he followng lemma. Lemma A any me we have UBĝ = UBĝ +, + LBĝ = LBĝ. 3 When U,ĝ, U,ĝ = 0, 0 When U,ĝ, U,ĝ =, When U,ĝ UBĝ + = T H, LBĝ + = LBĝ 3 UBĝ + = UBĝ,, U j,ĝ =, 0, =,, j UBĝ + = max UB,ĝ, T H LBĝ + = mn LB j,ĝ +, T H LBĝ + = T H 33 where x = maxmum neger x, and x = mnmum neger x. Proof See Appendx A Corollary below follows drecly form 3-35 n Lemma. Corollary Under polcy ĝ, UBĝ + LBĝ + { UBĝ LBĝ UBĝ LBĝ when U,ĝ, U,ĝ = 0, 0, oherwse. 36 Moreover, f UBĝ 0 LBĝ 0 for some me 0, hen UBĝ LBĝ for all 0. 37

11 Sgnalng for Decenralzed Roung n a Queueng Nework Corollary shows ha he dfference beween he hghes possble number of cusomers n Q or Q and he lowes possble number of cusomers n Q or Q s non-ncreasng under he polcy ĝ. Furhermore, he dfference s reduced by half when here s no cusomer roued from one queue o anoher one. 5 The fne horzon problem In hs secon, we consder he fne horzon problem formulaed n Secon., under he addonal condon X 0 = X 0 = x 0, where x 0 s arbrary bu fxed, and s common knowledge beween C and C. 5. Analyss The man resul of hs secon assers ha he polcy ĝ defned n Secon 4 s opmal. Theorem When X 0 = X 0 = x 0 and x 0 s common knowledge beween C and C, he polcy ĝ gven by 9-30 s opmal for he fne horzon decenralzed roung problem formulaed n Secon., ha s J ĝ T x 0, x 0 J g T x 0, x 0 38 for any feasble polcy g G d and any nal queue lengh x 0. Before provng Theorem, we noe ha when X 0 = X 0 = x 0 Corollary mples ha UBĝ LBĝ for all Equaon 39 says ha he dfference beween he hghes possble number of cusomers n Q or Q and he lowes possble number of cusomers n Q or Q s less han or equal o under polcy ĝ. Ths propery means ha ĝ conrols he lengh of he jon suppor of he PMFs Π, Π and balances he lenghs of he wo queues. A drec consequence of 39 s he followng corollary. Corollary A any me, we have X,ĝ + X,ĝ X,ĝ + X,ĝ = mn X,ĝ, X,ĝ, 40 = max X,ĝ, X,ĝ. 4

12 Y Ouyang, Demoshens Tenekezs As poned ou above, he polcy ĝ balances he lenghs of he wo queues. Ths balancng propery suggess ha he hroughpu of he sysem due o ĝ s hgh and he oal number of cusomers n he sysem s low. Ths s esablshed by he followng lemma. Lemma Under he assumpon X 0 = X 0 = x 0, where x 0 s common knowledge, for any polcy g of he form descrbed by 9-0, we have X,ĝ + X,ĝ s X,g + X,g, 4 where Z s Z means ha he r.v. Z s sochascally smaller han he r.v. Z, ha s, for any a R, PZ a PZ a see Marshall e al 00. Proof See Appendx B Usng Lemma, we now prove Theorem. Proof Proof of Theorem For any feasble polcy g, snce he funcons c, = 0,,..., T, are convex, we have a any me E E [c X,g + c X,g [ c X,g + X,g ] ] + c X,g + X,g. 43 Furhermore, usng Lemma and he fac ha c s ncreasng, we ge [ E [ c X,g + X,g E c X,ĝ + X,ĝ =E [c mnx,ĝ, X,ĝ + c =E [c + c X,ĝ X,ĝ + c X,g + X,g + c X,ĝ + X,ĝ ] maxx,ĝ, X,ĝ ] ] ]. 44 The nequaly n 44 s rue because X,g + X,g s X,ĝ + X,ĝ Lemma and c s ncreasng. The frs equaly n 44 follows from Corollary. Combnng 43 and 44 we oban, for any, E [c X,g + c X,g ] E [c X,ĝ The opmaly of polcy ĝ follows from 9 and c X,ĝ ]. 45

13 Sgnalng for Decenralzed Roung n a Queueng Nework 3 5. Comparson o he performance under cenralzed nformaon We compare now he performance of he opmal decenralzed polcy ĝ o he performance of he queueng sysem under cenralzed nformaon. The resuls of hs comparson wll be useful when we sudy he nfne horzon problem n Secon 6. Consder a cenralzed conroller who has all he nformaon I and I a each me. Then, he se G c of feasble roung polces of he cenralzed conroller s G c := {g, g : g = g 0, g,..., g,..., =, and U = g I, I }. 46 By he defnon, G d G c. Ths means ha he cenralzed conroller can smulae any decenralzed polcy g G d adoped by conrollers C and C. Therefore, for any nal PMFs π 0, π 0 nf g G c J g T π 0, π 0 nf g G d J g T π 0, π 0 47 nf g G c J g π 0, π 0 nf g G d J g π 0, π When X 0 = X 0 = x 0, Lemma and Theorem show ha he cos gven by ĝ s smaller han he cos gven by any polcy g G d. Furhermore we have: Lemma 3 Under he assumpon X 0 = X 0 = x 0, where x 0 s common knowledge, we have for any g G c, and for any g G c. X,ĝ + X,ĝ s X,g + X,g, 49 J ĝ T x 0, x 0 nf g G c J g T x 0, x Proof The proof of 49 s he same as he proof of Lemma, and he proof of 50 s he same as he proof of Theorem. Snce ĝ s a decenralzed polcy, 47 and Lemma 3 mply ha J ĝ T x 0, x 0 = nf g G d J g T x 0, x 0 = nf g G c J g T x 0, x 0. 5 Equaon 5 shows ha when X 0 = X 0 = x 0 and x 0 s common knowledge beween C and C, polcy ĝ acheves he same performance as any cenralzed opmal polcy.

14 4 Y Ouyang, Demoshens Tenekezs 5.3 The Case of Dfferen Inal Queue Lenghs When X 0 X 0, he polcy ĝ s no necessarly opmal for he fne horzon problem. Consder an example where he horzon T = wo-sep horzon, λ = 0., µ = 0.5 and ha s, P X 0 = 3 =, 5 P X 0 = = 0.9, P X 0 = 5 = 0., 53 π 0 =0, 0, 0,, 0, 0, 0,..., 54 π 0 =0, 0.9, 0, 0, 0, 0., 0,..., 55 where π 0, π 0 denoe he nal PMFs on he lenghs of he queues. Then, Π 0, Π 0 and he hreshold T H 0 are Π 0 = 0, 0, 0.5, 0.4, 0., 0, 0,..., 56 Π 0 = 0.45, 0.36, 0.09, 0, 0.05, 0.04, 0.0,..., 57 T H 0 = = Consder he cos funcons c 0 x = 0 and c x = x. Then, we have J g π0, π0 [ =E X,g + =E X,g ] [ X ] 0 U,g 0 + U,g 0 + X 0 U,g 0 + U,g Usng and he specfcaon of he polcy ĝ, we can compue he expeced cos due o ĝ. I s J ĝπ 0, π 0 = Consder now anoher polcy g descrbed below. For =,, j, [ ] U, g = g X, Π, Π, when X E X j Π j, = [ ] 0, when X < E X j Π j, Then, from and 6 we ge U, g 0 = U, g 0 = {, when X 0, 0, when X 0 <, {, when X 0.6, 0, when X 0 <.6,

15 Sgnalng for Decenralzed Roung n a Queueng Nework 5 Therefore, he expeced cos due o he polcy g s gven by J g π 0, π 0 = Snce J g π 0, π 0 = 8.8 < 8.48 = J ĝπ 0, π 0, polcy ĝ s no opmal. In hs example, each conroller has only one decson o make, he decson a me 0. As a resul, sgnalng does no provde any advanages o he conrollers, and ha s why he polcy ĝ s no he bes polcy. 6 Infne horzon We consder he nfne horzon decenralzed roung problem formulaed n Secon., and make he followng addonal assumpons. Assumpon µ > λ. Assumpon The nal PMFs π 0, π 0 are fnely suppored and common knowledge beween conrollers C and C..e. here exss M < such ha π 0x = π 0x = 0 for all x > M. Le g 0 denoe he open-loop polcy ha does no do any roung, ha s, a any me Assumpon 3 U,g0 = U,g0 = lm T T J g0 T π 0, π0 := J g0 < a.s., 66 where J g0 s a consan ha denoes he nfne horzon average cos per un me due o polcy g 0. Remark Due o polcy g 0, he queue lengh {X g0,, Z + }, =, s a posve recurren brh and deah chan wh arrval rae λ and deparure rae µ g {X 0,. Therefore, as T, he average cos per un me converges 0} o a consan a.s. f he expeced cos under he saonary dsrbuon of he process s fne see Bremaud, 999, chap. 3. Assumpon 3 s equvalen o he assumpon ha he expeced cos s fne under he saonary dsrbuon of he conrolled queue lenghs. We proceed o analyze he nfne horzon average cos per un me for he model of Secon under Assumpons -3.

16 6 Y Ouyang, Demoshens Tenekezs 6. Analyss When X 0 X 0, he polcy ĝ, defned n Secon 4, s no necessarly opmal for he fne horzon problem see he example n Secon 5.3. Neverheless, he polcy ĝ sll aemps o balance he queues. Gven enough me, polcy ĝ may be able o balance he queue lenghs even f hey are no nally balanced. In hs secon we show ha hs s ndeed he case. Specfcally, we prove he opmaly of polcy ĝ for he nfne horzon average cos per un me problem, as saed n he followng heorem whch s he man resul of hs secon. Theorem Under Assumpons -3, he polcy ĝ, descrbed by 9-30, s opmal for he nfne horzon average cos per un me problem formulaed n Secon.. To esablsh he asseron of Theorem we proceed n four seps. In he frs sep we show ha he nfne horzon average cos per un me due o polcy ĝ s bounded above by he cos of he unconrolled queues.e. he cos due o polcy g 0. In he second sep we show ha under polcy ĝ he queues are evenually balanced,.e. he queue lenghs can dffer by a mos one. In he hrd sep we derve a resul ha connecs he performance of polcy ĝ under he nal PMFs 0, 0 o he performance of he opmal polcy under any arbrary nal PMFs π 0, π 0 on queues Q and Q. In he forh sep we esablsh he opmaly of polcy ĝ based on he resuls of seps one, wo and hree. Sep We prove ha J ĝπ 0, π 0 J g0. To do hs, we frs esablsh some prelmnary resuls ha appear n Lemmas 4 and 5. Lemma 4 There exss processes {Y, Z + } and {Y, Z + } such ha {Y, Z + } has he same dsrbuon as {X,g0, Z + } 67 for =,, and for all mes Proof See Appendx C X,ĝ + X,ĝ Y + Y a.s., 68 max X,ĝ max Y a.s. 69

17 Sgnalng for Decenralzed Roung n a Queueng Nework 7 Lemma 4 means ha he unconrolled queue lenghs are longer han he queue lenghs under polcy ĝ n a sochasc sense. Noe ha 68 and 69 are no rue f Y, =,, s replaced by X,g0, =,, as he followng example shows. Example When X,g0 = 4, X,g0 = 6 and X,ĝ = X,ĝ = 5, he analogues of 68 and 69 where Y are replaced by X,g0, =, are X,ĝ + X,ĝ = X,g0 + X,g0 = 0, 70 max X,ĝ = 5 6 = max X,g0. 7 However, f A + =, A + = 0 and D + = 0, D + = we ge X,g0 X,g0 + = 5 and X,ĝ + = 6, X,ĝ + max X,ĝ + and he analogue of 69, when Y = 4, hen = 6 > 5 = max X,g0 + + =, 7 s replaced by X,g0, =,, does no hold. The sochasc domnance relaon assered by Lemma 4 mples ha he nsananeous cos under polcy ĝ s almos surely no greaer han he nsananeous cos due o polcy g 0. Ths mplcaon s made precse by he followng lemma. Lemma 5 The processes {Y, Z + } and {Y, Z + } defned n Lemma 4 are such ha a any me c X,ĝ Proof See Appendx C + c X,ĝ c Y + c Y a.s. 73 In order o apply he resul of Sep as he me horzon goes o nfny, we need he followng resul on he convergence of he cos due o {Y, Z + } and {Y, Z + }. Lemma 6 Le {Y, Z + } and {Y, Z + } be he processes defned n Lemma 4. Le W T denoe W T := T Under Assumpons and 3, T cy + cy. 74 =0 lm W T = J g0 a.s. 75 T Moreover, {W T, T =,,... } s unformly negrable, so also converges n expecaon.

18 8 Y Ouyang, Demoshens Tenekezs Proof See Appendx C A drec consequence of Lemmas 4, 5 and 6 s he followng. T Corollary 3 If lm T T =0 c X,ĝ + c X,ĝ converges a.s., hen, T T c X,ĝ + c X,ĝ J ĝπ 0, π0 76 =0 n expecaon and a.s. as T. Furhermore, Proof See Appendx C Sep J ĝπ 0, π 0 J g0 <. 77 We prove ha under polcy ĝ he queues are evenually balanced. For hs maer we frs esablsh some prelmnary resuls ha appear n Lemmas 7 and 8. Lemma 7 Le T 0 be a soppng me wh respec o he process {X,ĝ, X,ĝ, Z + }. Defne he process {S = Sĝ, T 0 + } as follows. S T0+ :=X,ĝ T 0+ + X,ĝ T S + :=S D D + A + A + {S=},ĝ {X =0} D D + D + {S=0} D + D 79 If µ > λ > 0, hen {S, T 0 + } s an rreducble posve recurren Markov chan. Proof See Appendx D Lemma 7 holds for arbrary soppng me T 0 wh respec o {X,ĝ, X,ĝ, Z + }. By appropraely selecng T 0 we wll show laer ha S s coupled wh X,ĝ +X,ĝ,.e. for all > T 0, S = X,ĝ +X,ĝ. Ths resul along wh he fac ha he process {S, T 0 + } s an rreducble posve recurren Markov chan wll allow us o analyze he cos due o polcy ĝ. Lemma 8 Under polcy ĝ, P U,ĝ, U,ĝ = 0, 0.o. =. 80

19 Sgnalng for Decenralzed Roung n a Queueng Nework 9 Proof See Appendx D Lemma 8 means ha he even { here exss 0 < such ha a leas one of he queue lenghs s above he hreshold defned by 30 for all > 0 } can no happen. The dea of Lemma 8 s he followng. If one of he queues, say Q, has lengh above he hreshold, hence above he lower bound LBĝ, hen, he lengh of Q does no decrease, because under polcy ĝ Q receves one cusomer from Q and has a mos one deparure a hs me. Therefore, boh queue lenghs a he nex me are bounded below by he curren lower bound LBĝ. When a leas one of he queue lenghs s above he hreshold for all > 0, he queue lenghs are bounded below by LBĝ 0 for all > 0. Ths knd of lower bound can no exs f he oal arrval rae λ o he sysem s less han he oal deparure rae µ from he sysem. Lemma 8 and Corollary n Secon 4 can be used o esablsh ha under polcy ĝ he queues are evenually balanced. Ths s shown n he corollary below. Corollary 4 Le Then T 0 := nf{ : UBĝ LBĝ }. 8 PT 0 < = 8 and UBĝ LBĝ for all T Sep 3 We compare he fne horzon cos J ĝ T 0, 0 respecvely, he nfne horzon cos J ĝ0, 0 due o polcy ĝ under nal PMFs 0, 0 o he mnmum fne horzon cos nf g Gd J g T π 0, π0 respecvely, he mnmum nfne horzon cos nf g Gd J g π0, π0 under arbrary nal PMFs π0, π0. Lemma 9 For any fne me T and any nal PMFs π 0, π 0. and J ĝ T 0, 0 = nf J g T 0, 0 nf J g T g G c g G π c 0, π0 nf J g T g G π 0, π0, 84 d J ĝ0, 0 = nf g G c J g 0, 0 nf g G c J g π 0, π 0 nf g G d J g π 0, π 0. 85

20 0 Y Ouyang, Demoshens Tenekezs Proof See Appendx E. Lemma 9 saes ha he mnmum cos acheved when he queues are nally empy s smaller han he mnmum cos obaned when he sysem s nal condon s gven by arbrary PMFs on he lenghs of queues Q and Q. Ths resul s esablshed hrough he use of he correspondng cenralzed nformaon sysem ha s dscussed n Secon 5.. Sep 4 Based on he resuls of Seps, and 3 we now esablsh he opmaly of polcy ĝ for he nfne horzon average cos per un me problem formulaed n Secon.. Frs, we oulne he key deas n he proof of Theorem, hen we presen s proof. Sep ensures ha polcy ĝ evenually n fne me balances he queues. Sep ensures ha he cos J ĝπ 0, π 0 s fne. These wo resuls ogeher mply ha he cos due o polcy ĝ s he same as he cos ncurred afer he queues are balanced. Furhermore, we show ha he cos of polcy ĝ s ndependen of he nal PMFs on he queue lenghs. Then, he resul of Sep 3 ogeher wh he resuls on he fne horzon problem esablsh he opmaly of polcy ĝ. Proof Proof of Theorem Defne T 0 o be he frs me when he lengh of he jon suppor of PMFs Π,ĝ, Π,ĝ s no more han. Tha s T 0 = nf{ : UBĝ LBĝ }. 86 The random varable T 0 s a soppng me wh respec o he process {X,ĝ, X,ĝ, Z + }. From Corollary 4 we have PT 0 < =, 87 UBĝ LBĝ for all T Furhermore, for all T 0 X,ĝ X,ĝ UBĝ LBĝ. 89 Consder he process {S, T 0 + } defned by 78 and 79 n Lemma 7. We clam ha for all T 0 + X,ĝ + X,ĝ = S. 90 We prove he clam n Appendx F. Suppose he clam s rue. Snce X,ĝ X,ĝ for all T 0 +, he nsananeous cos a me T 0 + s equal o c X,ĝ + c X,ĝ =c X,ĝ + X,ĝ + c X,ĝ + X,ĝ =c Sĝ + c Sĝ. 9

21 Sgnalng for Decenralzed Roung n a Queueng Nework Then, he average cos per un me due o polcy ĝ s gven by T = T T =0 T 0 =0 + T c c T =T 0+ Snce T 0 < a.s., we oban lm T T = lm T T T =0 T 0 =0 + lm T T = lm T = πĝs s=0 X,ĝ X,ĝ c c T =T 0+ T =T 0+ c + c + c X,ĝ X,ĝ c Sĝ + c Sĝ. 9 c X,ĝ X,ĝ s + c + c X,ĝ X,ĝ c Sĝ + c Sĝ Sĝ + c + c s Sĝ a.s. 93 where πĝs s he saonary dsrbuon of {S = Sĝ, T 0 + }. The second equaly n 93 holds because T 0 < a.s.; he las equaly n 93 follows by he Ergodc heorem for rreducble posve recurren Markov chans Bremaud, 999, chap. 3. Snce he sum T T =0 c 3 we have X,ĝ s=0 + c X,ĝ converges a.s., from Corollary J ĝπ T 0, π0 = lm c X,ĝ + c X,ĝ T T =0 = πĝs c s + c s. 94 Snce he rgh hand sde of equaon 94 does no depend on he nal PMFs π 0, π 0, we oban J ĝπ 0, π 0 = J ĝ0, 0. 95

22 Y Ouyang, Demoshens Tenekezs Combnng 95 and Lemma 9 we ge J ĝπ 0, π 0 = J ĝ0, 0 nf g G d J g π 0, π Thus, ĝ s an opmal roung polcy for he nfne horzon problem. 7 Dscusson and Concluson Based on he resuls esablshed n Secons 3-6, we now dscuss and answer he quesons posed n Secon. Conrollers C and C communcae wh one anoher hrough her conrol acons; hus, each conroller s nformaon depends on he decson rule/roung polcy of he oher conroller. Therefore, he queueng sysem consdered n hs paper has non-classcal nformaon srucure Wsenhausen 97. A key feaure of he sysem s nformaon srucure s ha a each me nsan each conroller s nformaon consss of one componen ha s common knowledge beween C and C and anoher componen ha s s own prvae nformaon. The presence of common nformaon allows us o use he common nformaon approach, developed n Nayyar e al 03, along wh specfc feaures of our model o denfy an nformaon sae/suffcen sasc for he fne and nfne horzon opmzaon problem. The denfcaon/dscovery of an approprae nformaon sae proceeds n wo seps: In he frs sep we use he common nformaon approach n parcular Mahajan 03 o denfy he general form of an nformaon sae namely X, Π, Π for conroller C, =,. and he correspondng srucure of an opmal polcy, Properes 3. In he second sep we ake advanage of he feaures of our sysem o furher refne/smplfy he nformaon sae; we dscover a smpler form of n- formaon sae, namely, X, { } The componen UB j, LB j j=, { UB j, LB j } j=, for conroller C, =,. of he above nformaon sae descrbes he common nformaon beween conrollers C and C a me, =,,.... Usng hs common nformaon we esablshed an opmal sgnalng sraegy ha s descrbed { by he hreshold } polcy ĝ specfed n Secon 4. The updae of UB j, LB j s descrbed by 3-35 and explcly j=, depends on he sgnalng polcy ĝ. Specfcally, f a cusomer s sen from Q o Q j j a me he lower bound on he queue lengh of Q ncreases because boh conrollers know ha he lengh of Q s above he hreshold T H a he me of roung; f no cusomer s sen from Q o Q j a me, he upper bound on he lengh of Q decreases because boh conrollers know ha he lengh of Q s below he hreshold T H a he me of roung. The updae of common nformaon ncorporaes he nformaon abou a conroller s prvae nformaon ransmed o he oher conroller hrough sgnalng. The sgnalng polcy ĝ communcaes nformaon n such a way ha evenually he dfference beween he upper bound and he lower bound on he

23 Sgnalng for Decenralzed Roung n a Queueng Nework 3 queue lenghs s no more han one. Thus, sgnalng hrough ĝ resuls n a balanced queueng sysem. A Proofs of he Resuls n Secon 4 Proof Proof of Lemma Snce here s one possble arrval o any queue and one possble deparure from any queue a each me nsan, 3 holds. When U,ĝ, U,ĝ = 0, 0, boh X,ĝ and X,ĝ are below he hreshold and no cusomers are roued form any queue. Therefore, he upper bound of he queue lenghs a + s UBĝ + = T H. 97 Moreover, he lower bound of he queue lenghs a + s he same as he lower bound of X,ĝ, X,ĝ. Tha s, LBĝ + =LBĝ. 98 When U,ĝ, U,ĝ =,, boh X,ĝ and X,ĝ are greaer han or equal o he hreshold. Snce he roung only exchanges wo cusomers beween he wo queues, he queue lenghs reman he same as he queue lenghs before roung. As a resul, he upper bound and lower bound of he queue lenghs a + are gven by UBĝ + =UBĝ. 99 LBĝ + = T H. 00 When U,ĝ, U j,ĝ =, 0, j, X,ĝ s greaer han or equal o he hreshold; X j,ĝ below he hreshold. Snce one cusomer s roued from Q o Q j, s X,ĝ + = X,ĝ, 0 X j,ĝ + = Xj,ĝ +. 0 Therefore, he upper bound of he queue lenghs a + becomes { } UBĝ + = max UB,ĝ, T H + { } = max UB,ĝ, T H, 03 and lower bound of he queue lenghs a + s gven by { } LBĝ + = mn T H, LB j,ĝ B Proofs of he Resuls n Secon 5 Proof Proof of Lemma The proof s done by nducon. A me = 0, X,ĝ 0 + X,ĝ 0 = X,g 0 + X,g 0 = x 0. Suppose he lemma s rue a me. A me +, from he sysem dynamcs -3 we ge, for any g, X,g + + X,g + = X,g + D + X,g + D + A + A. 05

24 4 Y Ouyang, Demoshens Tenekezs Therefore, suffces o show ha X,ĝ + D + X,ĝ + D s X,g + D + X,g + D. 06 Consder any realzaon X,g, X,g = x, x. If x, x > 0, hen x + x, x + x > 0. Therefore, X,g + D + X,g + D =x + x D D + + = x + x D + x + x D. 07 If x = 0 and x j j, hen x + x > 0 and x + x > 0. Therefore, X,g + D + X,g + D =x j D j x + x D D + + = x + x D + x + x D. 08 If x = 0 and x j = j, hen x + x = 0 and x + x =. Therefore, X,g + D + X,g + D = D j s D + + = x + x D + x + x D, 09 If x, x = 0, hen x + x, x + x = 0. Therefore, X,g + D + X,g + D =0 + + = x + x D + x + x D. 0 As a resul of 07-0, we oban X,g + D + X,g s X,g + X,g D + D X,g + X,g D.

25 Sgnalng for Decenralzed Roung n a Queueng Nework 5 Then, from, he nducon hypohess and Corollary we oban X,g + D + X,g + D + s X,g + X,g + D + X,g + X,g D + s X,ĝ + X,ĝ + D + X,ĝ + X,ĝ D = mnx,ĝ s X,ĝ D, X,ĝ + D + maxx,ĝ, X,ĝ + D + + X,ĝ + D. The frs and second sochasc nequales n follow from and he nducon hypohess, respecvely. The equaly n follows from Corollary. The las sochasc nequaly n s rue because D, D are..d. and ndependen of X,ĝ, X,ĝ. Thus, nequaly 06 s rue, and he proof of he lemma s complee. C Proofs of he Resuls Assocaed wh Sep of he Proof of Theorem Proof Proof of Lemma 4 The proof s done by nducon. A = 0, 67, 68 and 69 hold f we le Y0 = X,g 0 0 for =,. Assume he asseron of hs lemma s rue a me ; we wan o show ha he asseron s also rue a me +. For ha maer we clam he followng. Clam X,ĝ + + X,ĝ + = X,ĝ + X,ĝ a.s., 3 max X,ĝ + max X,ĝ a.s. 4 Clam There exss Y+, =, such ha P Y+ + Y0: 0: =P X,g0 + = y + X,g 0 0: = y 0: for all y 0:, 5 X,ĝ + X,ĝ Y+ + Y+ a.s., 6 max X,ĝ max Y + a.s. 7 We assume he above clams o be rue and prove hem afer he compleon of he proof of he nducon sep. For all y 0:+, from 5 and he nducon hypohess for 67 we ge for =, P Y0:+ = y 0:+ =P Y+ = y + Y0: = y 0: P Y = y,..., Y0 = y 0 =P X,g0 + = y + X,g 0 0: = y 0: P X,g 0 0: = y 0: =P X,g0 0:+ = y 0:+. 8 From 3 and 6 we oban X,ĝ + + X,ĝ + =X,ĝ + X,ĝ Y + + Y + a.s. 9

26 6 Y Ouyang, Demoshens Tenekezs Furhermore, combnaon of 4 and 7 gves max X,ĝ + max X,ĝ = max Y + a.s. 0 Therefore, he asserons 67, 68 and 69 of he lemma are rue a + by 8, 9 and 0, respecvely. We now prove clams and. Proof of Clam From he sysem dynamcs - X,ĝ + = X,ĝ U,ĝ + U j,ĝ, X,ĝ + = X,ĝ U,ĝ + U j,ĝ. Therefore, 3 follows by summng and. For 4, consder X,ĝ + he case of X,ĝ + follows from smlar argumens. When U,ĝ = 0, X,ĝ + = X,ĝ U,ĝ max X,ĝ. 3 When U,ĝ = U,ĝ =, X,ĝ + = X,ĝ max X,ĝ. 4 When U,ĝ = 0, U,ĝ =, X,ĝ s less han he hreshold and X,ĝ s greaer han or equal o he hreshold. Therefore, by, X,ĝ + = X,ĝ + T H Therefore, 4 follows from 3-5. Proof of Clam We se where Y X,ĝ max X,ĝ. 5 Y + := Y D + + Ã 6 sasfy he nducon hypohess, and Ã, D, =, are specfed as follows. Le M x =argmax {X,ĝ }, m x = argmn {X,ĝ } 7 M y =argmax {Y }, m y = argmn {Y }, 8 where M x =, m x = resp. M y =, m y = when {X,ĝ = X,ĝ } resp. {Y = Y }; defne y My ÃM, D, Ãmy my A Mx, D mx, A mx, D Mx n case,, D := 9 A Mx, D Mx, A mx, D mx n case, where he wo cases are : Case : {Y My = X Mx,ĝ = X mx,ĝ Case : All oher nsances. and A Mx, D Mx, A mx, D mx = 0,,, 0 or 0, 0,, }. Asseron: The random varables Y+, Y +, defned by 6-9 sasfy 5-7. As he proof of hs asseron s long, we frs provde a skech of s proof and hen we provde a full proof. Skech of he proof of he asseron

27 Sgnalng for Decenralzed Roung n a Queueng Nework 7 Equaon 9 mples he followng: In case we assocae he arrval o and he deparure from he longer queue M x o hose of he longer queue M y,.e. we se à My = A Mx My, D = D Mx. We do he same for he shorer queue m x, m y,.e. à my = A mx my, D = D mx. In case, we have he same assocaon for he arrvals as n case, ha s ÃMy = A Mx, Ãmy = A mx My, bu we reverse he assocaon of he deparures, ha s D = D mx my, D = D Mx. Therefore he arrvals Ã, and deparures D, have he same dsrbuon as he orgnal A, D, respecvely, =,. Then 5 follows from 6. To esablsh 6, we noe ha, because of 9, he sum of arrvals o respecvely, deparures from queues M y and m y equals o he sum of arrvals o respecvely, deparures from queues M x and m x. When X,ĝ, Y 0, =,, he funcon x d+ +a s lnear x, as x d + +a = x d+a. Then from 6, 9 and he nducon hypohess we oban Y+ + Y+ X,ĝ X,ĝ =Y + Y X,ĝ X,ĝ 0 30 and hs esablsh 6 when X,ĝ, Y 0, =,. In he full proof of he asseron, we show ha show ha 6 s also rue when X,ĝ, Y are no all non-zero. To esablsh 7 we consder he maxmum of he queue lenghs. In case, we show ha 6-9 ensure ha Y My + XMx,ĝ max, 3 X mx,ĝ ; 3 Y My +, Y my + hen 7 follows from 3-3. In case 7 s verfed by drec compuaon n he full proof. Proof of he asseron For all y 0:, we denoe by E y0: he even {Y0: = y 0:}. Le Z = y My ÃM, D, Ãmy my, D, hen for any realzaon z {0, } 4 of Z we have P Z = z E y0: =P Z = z, case E y0: When z 0,,, 0 or 0, 0,,, we ge P Z = z, case E y0: + P Z = z, case E y0:. 33 = 0, 34 and P Z = z, case E y0: =P A Mx, D Mx, A mx, D mx = z E y0: =P A, D, A, D = z, 35 where he las equaly n 35 holds because he random varables A Mx, D Mx, A mx, D mx are ndependen of Y 0, Y,..., Y and have he same dsrbuon as A, D, A, D. Therefore, combnng 34 and 35 we oban for z 0,,, 0 or 0, 0,, P Z = z E y0: = P A, D, A, D = z 36

28 8 Y Ouyang, Demoshens Tenekezs When z = 0,,, 0 or 0, 0,,, le E denoe he even {Y My = X Mx,ĝ hen we oban P Z = z, case E y0: =P A Mx, D mx, A mx, D Mx = z, E E y0: = X mx,ĝ }; =P A, D, A, D = z P E Ey0:, 37 and P Z = z, case E y0: =P A Mx, D Mx, A mx, D mx = z, E c E y0: =P A, D, A, D = z P E c E y0:, 38 where he las equaly n 37 and 38 follow by he fac ha he random varables A Mx, D Mx, A mx, D mx are ndependen of Y 0, Y,..., Y hence, he even E whch s generaed by Y 0, Y,..., Y and have he same dsrbuon as A, D, A, D. Therefore, combnng 37 and 38 we oban for z = 0,,, 0 or 0, 0,, P Z = z E y0: =P A, D, A, D = z P E Ey0: + P A, D, A, D = z P E c E y0: =P A, D, A, D = z, 39 where he las equaly n 39 s rue because A, D, A, D are ndependen and D has he same dsrbuon as D. As a resul of 36 and 39, for any z {0, } 4 we have P Z = z E y0: = P A, D, A, D = z. 40 Now consder any y 0:+. By 40 we have for = M y or m y P Y+ = y + E y0: =P y D + + à = y + E y0: y =P D + + A = y + =P X,g0 + = y + X,g 0 0: = y 0:. 4 whch s 5. Now consder he sum Y+ + Y +. From 9, we know ha Therefore, 4 mples à My D My + Ãmy + D my = A Mx = D Mx + A mx a.s., 4 + D mx a.s. 43 Y+ + Y + X,ĝ + X,ĝ + Y My My + D + Y my = X Mx,ĝ D Mx + X mx,ĝ my + D + D mx. 44

29 Sgnalng for Decenralzed Roung n a Queueng Nework 9 We proceed o show ha he rgh hand sde of 44 s posve. From he nducon hypohess for we have Y my Y My + Y My There are hree possbles: {Y My X Mx,ĝ, X mx > 0}. Frs consder {Y My Noe ha {Y My ha X mx,ĝ + X Mx,ĝ a.s., 45 X Mx,ĝ a.s. 46 = X Mx,ĝ }, {Y My = X Mx,ĝ }. By 45 we have Y my > X Mx,ĝ, X mx,ĝ = 0} and {Y My > X mx,ĝ a.s. 47 = X Mx,ĝ } belongs o case n 9. From case of 9 we also know D Mx Then, because of we ge If Y My If Y My > X Mx,ĝ > X Mx,ĝ X Mx,ĝ Y My = Y My and X mx,ĝ = 0 X Mx,ĝ = and X mx X Mx,ĝ = My D, D mx my = D D Mx + D Mx D My + + Y m y X mx,ĝ + + Y my + D Mx + + D Mx X Mx,ĝ Y My Y My My D > 0, hen X Mx,ĝ =X Mx,ĝ =X Mx,ĝ Y My + D mx D mx + X mx,ĝ + + Y my + D Mx + X mx,ĝ D Mx + X mx,ĝ + Y my Y My D My + X mx,ĝ D My D my D mx my D My my D D + + Y my + a.s D mx my + D 50 + D mx my + D 5 where he second equaly n 5 follows from 43 and he frs nequaly n 5 follows from he nducon hypohess for 68. The above resuls, namely 49-5, show ha he rgh hand sde of 44 s posve, and he proof for 6 s complee. I remans o show ha 7 s rue. We frs consder case. In case, we know from 9 ha y ÃM, D My, Ãmy, my D = A Mx, D Mx, A mx, D mx. 5

30 30 Y Ouyang, Demoshens Tenekezs Then, X Mx,ĝ = X Mx,ĝ = X Mx,ĝ Y My + D Mx + A M x D My D My + + Ã My + + Ã My =Y My +, 53 where he second equaly s a consequence of 5 and he nequaly follows from he nducon hypohess for 69. To proceed furher we noe ha n case here are hree possbles: {Y My {Y My X mx,ĝ } and {Y My > X Mx,ĝ, Y My < X mx,ĝ } If Y My If Y My = X Mx,ĝ X mx,ĝ If Y My > X Mx,ĝ we are n case, = X Mx,ĝ },, 47 s also rue. Followng smlar argumens as n 53 we oban X mx,ĝ and Y My A Mx, D Mx X mx,ĝ X mx,ĝ < X mx,ĝ, A mx, D mx Y my Y My Y My can only be Y My 0,,, 0. Therefore, = X Mx,ĝ = X mx,ĝ. Snce Then we ge A mx X mx,ĝ = D mx Y My = max max max max A Mx + D mx + A m x A mx A mx A mx, Y My, Y My D Mx D mx D Mx, Y My + Y my +, Y My + + A mx + A Mx. 57 Combnng 53, 54, 55 and 57 we ge 7 when case s rue. Now consder case. We have Y My = X Mx,ĝ = X mx,ĝ. When A Mx, D Mx, A mx, D mx = 0,,, 0, hen X Mx,ĝ = X Mx,ĝ + X mx =X mx = + Y My D mx + + A M x =Y My + 58

31 Sgnalng for Decenralzed Roung n a Queueng Nework 3 When A Mx, D Mx, A mx X Mx,ĝ, D mx = 0, 0,, we ge =X Mx,ĝ X mx,ĝ = max X mx,ĝ, = max Y My = max max D mx Y My +, Amx Y My +, Y my A M x, A mx. 59 Combnng 58 and 59 we oban 7 for case. As a resul, 7 holds for boh cases and. Remark: We noe ha we need he wo cases descrbed n 9 for he followng reasons. If we elmnae case and always assocae y ÃM, D My, Ãmy, D my wh A Mx, D Mx, A mx, D mx = 0,,, 0}, as n case, hen when {Y My and A Mx, D Mx, A mx, D mx he shorer queue m x ncreases by one cusomer, and he longer queue M y decreases by one cusomer; herefore X mx,ĝ = X mx,ĝ = Y My + + and 7 s no sasfed. Proof Proof of Lemma 5 From Lemma 4, a any me here exss Y such ha such ha hold. Adopng he noaons M x, m x and M y, m y n he proof of Lemma 4, we have a every me Furhermore, from 69 we have If X mx,ĝ If X mx,ĝ Y my X mx,ĝ X M X,ĝ a.s., 60 Y my X Mx,ĝ Y My a.s. 6, 6 and he fac ha c s ncreasng gve c X M X,ĝ + c X m X,ĝ c Y My > Y my, hen Y my < X mx,ĝ Snce c s convex, follows from 64 ha c Y My c X Mx,ĝ Y My X Mx,ĝ From 68 n Lemma 4 we know ha Y My Combnng 65 and 66 we ge c Y My X Mx,ĝ + c Y my Y My a.s. 6 X Mx,ĝ c X mx,ĝ X mx,ĝ + c Y my. 63 Y My. 64 X mx,ĝ c X Mx,ĝ + c c Y my Y my. 65 Y my. 66 X mx,ĝ. 67

32 3 Y Ouyang, Demoshens Tenekezs Proof Proof of Lemma 6 Le {Y, Z +} and {Y, Z +} be he processes defned n Lemma 4. Then {Y, Z +} has he same dsrbuon as {X,g 0, Z + } for =,. Snce µ > λ, he processes {Y, Z +}, =, are rreducble posve recurren Markov chans. Moreover, he wo processes {Y, Z +} and {Y, Z +} have he same saonary dsrbuon, denoed by π g 0. Under Assumpon, by Ergodc heorem of Markov chans see Bremaud, 999, chap. 3 we ge T lm T T =0 T cy = lm T T =0 cy = π g 0 xcx a.s. 68 Le WT Y 0:T := T T =0 cy, =,. We show ha {WT Y 0:T, T =,,... } s unformly negrable for =,. Tha s, x=0 [ ] sup E WT Y 0:T {W T T Y 0:T >N} 0 69 as N. Le p g 0x, y, x, y Z + be he ranson probables of he Markov chan. Noe ha he nal PMF of he process {Y, Z +}, =, s π0. From Assumpon we know ha π0 x = 0, =, for all x > M. π0 Leng R := max x x M π g 0 <, we oban for =, x [ ] E WT Y 0:T {W T Y 0:T >N} = WT y 0:T {W T y 0:T >N} PY 0:T = y 0:T y 0:T = WT y 0:T {W T y 0:T >N} π 0 y 0Π T = pg 0 y, y y 0:T R WT y 0:T {W T y 0:T >N} πg 0 y 0 Π T = pg 0 y, y y 0:T [ ] =RE W πg 0 T {W π g 0, 70 >N} T where W πg 0 T = T T =0 cy πg 0 and {Y πg 0, Z + } s he chan wh ranson probables p g 0x, y and nal PMF π g 0. Noe ha {Y πg 0, Z + } s saonary because he nal PMF s he saonary dsrbuon π g 0. From Brkhoff s Ergodc heorem we know ha {W πg 0 T, T =,,... } converges a.s. and n expecaon see Peersen and Peersen, 989, chap.. Therefore, {W πg 0 T, T =,,... } s unformly negrable, and he rgh hand sde of 70 goes o zeros unformly as N. Consequenly, {WT Y 0:T, T =,,... } s also unformly negrable for =,. Snce W T = WT Y 0:T +WT Y 0:T for all T =,,..., {W T, T =,,... } s unformly negrable. Proof Proof of Corollary 3 From Lemma 5, here exss {Y, Y, Z +} such ha 67 holds and c X,ĝ + c X,ĝ c Y + c Y a.s. 7

33 Sgnalng for Decenralzed Roung n a Queueng Nework 33 Le From 7 follows ha W T := T V T := T T =0 T =0 c Y + c Y, 7 c X,ĝ + c X,ĝ. 73 V T W T, T =,, From Lemmas 6, {W T, T =,,... } s unformly negrable, herefore {V T, T =,,... }, whch s bounded above by {W T, T =,,... } s also unformly negrable. From he propery of unformly negrably, f {V T, T =,,... } converges a.s., we know ha {V T, T =,,... } also converges n expecaon. Furhermore, J ĝ π 0, π 0 = lm sup T T E = lm sup E [V T ] T [ T =0 ] c Y + c Y lm sup E [W T ] = J g T D Proofs of he Resuls Assocaed wh Sep of he Proof of Theorem Proof Proof of Lemma 7 Frs we show ha {S, T 0 + } s a Markov chan. For s, P S + = s + S T0 +: = s T0 +: =P s D D + A + A = s+ S T0 +: = s T0 +: =P s D D + A + A = s+ S = s =P S + = s + S = s. 76 The frs and las equales n 76 follow from he consrucon of he process {S, T 0 + }. The second equaly n 76 s rue because T 0 s a soppng me wh respec o {X,ĝ, X,ĝ, Z + }, and A, D, =, are ndependen of all random varables before. Smlarly, for s = 0 we have, by argumens smlar o he above, P S + = s + S T0 +: = s T0 +: =P A + A = s + S T0 +: = s T0 +:, S = 0 =P A + A = s + S = 0 =P S + = s + S = The frs and las equaly n 77 follow from he consrucon of he process {S, T 0 + }. The second equaly n 77 s rue because A, D, =, are ndependen of all

34 34 Y Ouyang, Demoshens Tenekezs varables before. For s =, P S + = s + S T0 +: = s T0 +: =P s + { X,ĝ } D =0 D D + A + A = s + S T0 +: = s T0 +: =P D + A + A = s +, X,ĝ = 0 S T0 +: = s T0 +:, S = + P D + A + A = s +, X,ĝ = S T0 +: = s T0 +:, S = =P D + A + A = s +, X,ĝ = 0 S T0 +: = s T0 +:, S = + P D + A + A = s +, X,ĝ = S T0 +: = s T0 +:, S = =P D + A + A = s + S T0 +: = s T0 +:, S = =P D + A + A = s + S = =P S + = s + S = s. 78 The frs equaly n 78 follows from he consrucon of he process {S, T 0 + }. The second and forh equales follow from he fac ha X,ĝ can be eher 0 or. In he hrd equaly, D s replaced by D n he frs erm; hs s rue because D and D are dencally dsrbued and ndependen of X,ĝ and all pas random varables. The ffh equaly holds because T 0 s a soppng me wh respec o {X,ĝ, X,ĝ, Z + } and A, D, =, are ndependen of all pas random varables. The las equaly follows from he same argumens ha lead o he frs hrough he ffh equales. Therefore, he process {S, T 0 + } s a Markov chan. Snce λ, µ > 0, he Markov chan s rreducble. We prove ha he process {S, T 0 + } s posve recurren. Noe ha, for all s = 0,,,..., because of he consrucon of {S, T 0 + } Moreover, for all s, E [S + S = s] E [ S + A + A S = s] =s + λ <. 79 E [S + S = s] =E [ s D D + A + A S = s ] =s µ + λ < s. 80 Usng Foser s heorem see Bremaud, 999, chap. 5, we conclude ha he Markov chan {S, T 0 + } s posve recurren. Proof Proof of Lemma 8 Le Ω, F, P denoe he basc probably space for our problem. Defne evens E F, = 0,,... o be E ={ω Ω : U,ĝ ω, U,ĝ ω 0, 0 } 8 If he clam of hs lemma s no rue, we ge P E = P U,ĝ, U,ĝ = 0, 0.o. > 0. 8 =0 Therefore, here exs some 0 such ha PE 0 > 0. Snce 0 s a consan, s a soppng me wh respec o {X,ĝ, X,ĝ, Z + }.

35 Sgnalng for Decenralzed Roung n a Queueng Nework 35 Consder he process {S, = 0 +, 0 +,...} defned n Lemma 7 wh he soppng me 0. From Lemma 7 we know ha {S, 0 + } s an rreducble posve recurren Markov chan. Furhermore, along he sample pah nduced by any ω E 0, we clam ha for all 0 + S ω =X,ĝ ω + X,ĝ ω =X,ĝ ω + X,ĝ ω. 83 The clam s shown by nducon below. By he defnon of {S, 0 + } n Lemma 7, we have a me 0 + for any ω E 0 S 0 +ω =X,ĝ 0 + ω + X,ĝ 0 + ω =X,ĝ 0 ω + X,ĝ 0 ω, 84 where he las nequaly n 84 follows from he sysem dynamcs -3. Assume equaon 83 s rue a me 0 +. A me + we have, by -3, X,ĝ + + X,ĝ + =X,ĝ D + + X,ĝ D + + A + A =X,ĝ + X,ĝ D D + A + A + D { X,ĝ } + D =0 { X,ĝ }. 85 =0 Snce along he sample pah nduced by ω E 0, U,ĝ,ĝ ω, U ω 0, 0 and X,ĝ = X,ĝ U,ĝ j,ĝ +U, he even {X,ĝ = 0} E 0 = or mples ha X,ĝ =, U,ĝ = and U j,ĝ = 0. For hs case, X,ĝ = and U,ĝ = furher mply ha he hreshold s smaller han one. Then, he only possbly for U j,ĝ = 0 s Xj,ĝ = 0. Therefore, { X,ĝ = 0} E0 { } X,ĝ,ĝ j,ĝ =, U =, Xj,ĝ = 0 and U = 0 {S = }. 86 Consequenly, from 86, for any ω E 0 D ω { X,ĝ } + D ω=0 ω { X,ĝ } ω=0 = {S ω=} D ω{ X,ĝ } + D ω=0 ω{ X,ĝ }. ω=0 = {S ω=} } D ω D ω + D ω. 87 { X,ĝ ω=0 Moreover, U,ĝ,ĝ ω, U ω 0, 0 mples ha X,ĝ ω, X,ĝ ω 0, 0. Hence, S ω = X,ĝ ω + X,ĝ ω 0, 88