Markov chain. Markov Chain? Markov chain. Discrete-Time Markov Chain (DTMC) Memoryless (Markov) Property: Given the present state, Markov Chain

Transcription

1 Outle Marov chas ad some reewal theory Marov cha Revew: Dscrete Evet Radom Processes Hogwe Zhag htt:// Reewal rocesses, reewal reward rocesses, Marov reewal rocesses The excess dstrbuto Phase tye dstrbuto PASTA Level crossg aalyss Some mortat queueg models Reversblty of Marov chas ad Jacso Networ Outle Marov chas ad some reewal theory Marov cha Reewal rocesses, reewal reward rocesses, Marov reewal rocesses The excess dstrbuto Phase tye dstrbuto PASTA Level crossg aalyss Some mortat queueg models Reversblty of Marov chas ad Jacso Networ Marov cha Marov Cha Dscrete-Tme Marov Chas Calculatg Statoary Dstrbuto Global Balace Equatos Geeralzed Marov Chas Cotuous-Tme Marov Chas

2 Marov cha Marov Cha Dscrete-Tme Marov Chas Calculatg Statoary Dstrbuto Global Balace Equatos Geeralzed Marov Chas Cotuous-Tme Marov Chas Marov Cha? Stochastc rocess that taes values a coutable set Examle: {,,,,m}, or {,,, } Elemets rereset ossble states Cha trasts from state to state Memoryless (Marov) Proerty: Gve the reset state, future trastos of the cha are deedet of ast hstory Marov Chas: dscrete- or cotuous- tme Marov cha Marov Cha Dscrete-Tme Marov Chas Calculatg Statoary Dstrbuto Global Balace Equatos Geeralzed Marov Chas Cotuous-Tme Marov Chas Dscrete-Tme Marov Cha (DTMC) Dscrete-tme stochastc rocess {X :,,, } Taes values {,,, } Memoryless roerty: Also wrtte as P, P{ X X, X,..., X } P{ X X } + + P P{ X X } + Trasto robabltes P P, P Note: future ad ast are deedet gve the reset; but they are ot ucodtoally deedet. Trasto robablty matrx P[P ]

3 Comosto of DTMCs Gve two deedet DTMCs X,, o S ad Y,, o T wth trasto robablty matrces P ad Q; the Z (X, Y ) s a DTMC o S T wth Pr ( Z + ( s, t) Z ( s, t) ) s, sqt, t Multle mutually deedet DTMCs ca be comosed a smlar fasho Chama-Kolmogorov Equatos ste trasto robabltes Also wrtte as P, () P P{ X X },, m,, + m m How to calculate? Chama-Kolmogorov equatos + m m P P P,, m,, P s elemet (, ) matrx P Recursve comutato of state robabltes Thus, P ( ) P State Probabltes Statoary Dstrbuto State robabltes (tme-deedet) π P{ X }, π (π,π,...) I matrx form: P{ X } P{ X } P{ X X } π π P π π P π P... π P If tme-deedet dstrbuto coverges to a lmt π lmπ π πp π s called the statoary dstrbuto (or steady state dstrbuto) exstece deeds o the structure of Marov cha Irreducblty of DTMC States ad commucate: Bary relato s a equvalece (.e., reflexve, symmetrc, trastve); the equvalece classes duced by are called commucatg classes Irreducble Marov cha: all states commucate (ad thus form a sgle commucatg class) m, m : P >, P > 3 4 Deote as 3

4 Frst ht robabltes f, () Probablty of frst httg/vstg state at tme, whe startg state at tme f f ( ), (), ( X, X,..., X, X X ) Pr, ad for, f T : the frst assage tme from to Probablty of vstg state fte tme f startg state ( ) f, f, ( ), Aerodcty of DTMC Perod d of a state : Theorem: all the states a commucatg class of a DTMC have the same erod. State s aerodc f d d gcd ( ) ( ) { : f > } gcd{ : },, > Secal case: f, >, the s aerodc (why?) Aerodc Marov cha: oe of the states s erodc 3 4 Lmt Theorems Theorem a: Irreducble aerodc Marov cha For every state, the followg lmt π lm P{ X X },,,,... exsts ad s deedet of tal state N (): umber of vsts to state u to tme N ( ) P π lm X Exstece of Statoary Dstrbuto (or steady state dstrbuto) Theorem b: Irreducble aerodc Marov cha. There are two ossbltes for scalars: π lm P{ X X } lm P. π, for all states No statoary dstrbuto. π >, for all states π s the uque statoary dstrbuto Remar: If the umber of states s fte, case s the oly ossblty >π : frequecy the rocess vsts state 4

5 Postvty A state s ostve recurret f the rocess returs to state ftely ofte Formal defto: A state s absorbg f, A state s traset f f, < A state s recurret (or ersstet) f f, A recurret state s ostve f f < ; otherwse, t s ull, Note: ostve recurret > rreducble always hold, but rreducble > ostve recurret s guarateed to hold oly for fte MC Examle : a MC wth coutably fte state sace q q - q q q q q All states are ostve recurret f < ½, ull recurret f ½, ad traset f > ½ Decdg ostvty Theorem D.: for each commucatg class of a DTMC {X }, exactly oe of the followg holds: All the states the class are traset All the states the class are ull recurret All the states the class are ostve (recurret) Thus, a rreducble DTMC s ostve recurret f ay oe of ts state s ostve A commucatg class C s closed f, : C, C, Otherwse, the class s sad to be oe Theorem D.3: gve a DTMC, A oe commucatg class s traset A closed fte commucato class s ostve recurret 5

6 What about fte closed commucatg classes? Theorem D.4: a rreducble DTMC o state sace S s ostve recurret ff. ostve rob.dstrbuto π o Ss.t. π πp. where P s the state trasto matrx. Note: f such robablty π exsts, t s uque ad s called a varat robablty vector for the DTMC Alteratve aroach: drft aalyss of a sutable Lyauov fucto f(.) Theorem D.7: a rreducble DTMC X,, s recurret f oegatve fucto f(), S (state sace), s.t. f() as, ad a fte set A S s.t. + ( f ( X ) X ) f ( ) A : E If π s varat, ad f Pr(X ) π, the the DTMC so obtaed s a statoary radom rocess Α Theorem D.8: a rreducble DTMC X,, s traset f oegatve fucto f(), S, ad a set A S s.t. ad A s.t. + ( f ( X ) X ) f ( ) A : E A: f ( ) < f ( ) Theorem D.9: a rreducble DTMC X,, s ostve recurret f oegatve fucto f(), S, ad a fte set A S s.t. for some ε>, ad ( f ( X ) X ) f ( ε A: E + ) ( f ( X ) X ) B A E + : Α for some fte umber B. Α 6

7 Theorem D.: a rreducble DTMC X,, o {,,, } s ot ostve recurret f fte values K> ad B> s.t. : E K : E ( X X ) <, ad Bouded dowward drft + ( X X ), ad E( X X + ) X ) B + + Exercse Use Theorems D.7-9 to rove the results for Examle show earler I the cotext of Theorems D.7-9, Theorem D. s for Lyauov fucto f() Ths theorem s useful establshg stablty results E.g., for a queue wth fte # of servers where arrval rate s strctly greater tha the overall servce rate, X queue occuacy Covergece of ostve recurret DTMC Gve a rreducble, ostve DTMC wth erod d ad state sace S, Ergodcty A state s ergodc f t s aerodc ad ostve recurret S d,lm, dπ A MC s ergodc f every state s ergodc If the DTMC s aerodc (.e., d),, S,lm, π Ergodc chas have a uque statoary dstrbuto π /E(T ),,,, where T s the frst assage tme from to Note: Ergodcty Tme Averages Stochastc Averages 7

8 Marov cha Calculato of Statoary Dstrbuto Marov Cha A. Fte umber of states B. Ifte umber of states Dscrete-Tme Marov Chas Calculatg Statoary Dstrbuto Global Balace Equatos Geeralzed Marov Chas Cotuous-Tme Marov Chas Solve exlctly the system of equatos π π P,,,..., m coverges to a matrx wth rows equal to π m Sutable for a small umber of states Caot aly revous methods to roblem of fte dmeso Guess a soluto to recurrece: m π π P,,,..., π π Or, umercally from P whch (detaled) balace equatos ca hel the guess Examle: Fte Marov Cha Abset-mded rofessor uses two umbrellas whe commutg betwee home ad offce. If t ras ad a umbrella s avalable at her locato, she taes t. If t does ot ra, she always forgets to tae a umbrella. Let be the robablty of ra each tme she commutes. Q: What s the robablty that she gets wet o ay gve day? Marov cha formulato s the umber of umbrellas avalable at her curret locato Trasto matrx P Examle: Fte Marov Cha π ( )π P π πp π ( )π + π π,π,π π π π π π + π + π P{gets wet} π 3 8

9 Examle: Fte Marov Cha Tag.: π,,.3,.345, P Numercally determe lmt of P ( ) lm P ( 5) Effectveess deeds o structure of P Marov cha Marov Cha Dscrete-Tme Marov Chas Calculatg Statoary Dstrbuto Global Balace Equatos Geeralzed Marov Chas Cotuous-Tme Marov Chas Global Balace Equatos Global Balace Equatos (GBE) π P π P π P π P, P s the frequecy of trastos from to π Frequecy of Frequecy of trastos out of trastos to Ituto: ) vsted ftely ofte; ) for each trasto out of there must be a subsequet trasto to wth robablty Global Balace Equatos (cotd.) Alteratve Form of GBE π P π P, S {,,,... } S S S S If a robablty dstrbuto satsfes the GBE, the t s the uque statoary dstrbuto of the Marov cha Fdg the statoary dstrbuto: Guess dstrbuto from roertes of the system Verfy that t satsfes the GBE Secal structure of the Marov cha smlfes tas 9

10 Global Balace Equatos Proof Marov cha Frst form: π π P ad P π P π P π P π P Marov Cha Dscrete-Tme Marov Chas Secod form: π P π P π P π P S S π P + P πp + πp S S S S S S π P π P S S S S Calculatg Statoary Dstrbuto Global Balace Equatos Geeralzed Marov Chas Cotuous-Tme Marov Chas Geeralzed Marov Chas Marov cha o a set of states {,, }, that wheever eters state The ext state that wll be etered s wth robablty P Gve that the ext state etered wll be, the tme t seds at state utl the trasto occurs s a RV wth dstrbuto F {Z(t): t } descrbg the state of the cha at tme t: Geeralzed Marov cha, or Sem-Marov rocess Does GMC have the Marov roerty? Future deeds o ) the reset state, ad ) the legth of tme the rocess has set ths state Geeralzed Marov Chas (cotd.) T : tme rocess seds at state, before mag a trasto holdg tme Probablty dstrbuto fucto of T T : tme betwee successve trastos to X s the th state vsted. {X :,, } H ( t) P{ T t} P{ T t ext state } P F ( t) P Is a Marov cha: embedded Marov cha Has trasto robabltes P [ ] t dh ( t) E T Sem-Marov rocess rreducble: f ts embedded Marov cha s rreducble

11 Lmt Theorems Gve a rreducble sem-marov rocess w/ E[T ] < For ay state, the followg lmt lm P{ Z( t) Z() },,,,... t exsts ad s deedet of the tal state. E[ T ] E[ T ] T (t): tme set at state u to tme t T ( t) P lm Z() t t s equal to the roorto of tme set at state Occuacy Dstrbuto Gve a rreducble sem-marov rocess where E[T ] <, ad the embedded Marov cha s ergodc w/ statoary dstrbuto π the, wth robablty, the occuacy dstrbuto of the sem-marov rocess π E[ T ],,,... π E[ T ] π : roorto of trastos to state E[T ]: mea tme set at π π P, ; π Probablty of beg at s roortoal to π E[T ] Marov cha Marov Cha Dscrete-Tme Marov Chas Calculatg Statoary Dstrbuto Global Balace Equatos Geeralzed Marov Chas Cotuous-Tme Marov Chas Cotuous-Tme Marov Chas (def.?) Cotuous-tme rocess {X(t): t } tag values {,,, }. Wheever t eters state Tme t seds at state s exoetally dstrbuted wth arameter α Whe t leaves state, t eters state wth robablty P, where Σ P Cotuous-tme Marov cha s a sem-marov rocess wth Exoetal holdg tme > a cotuous-tme Marov cha has the Marov roerty α F, ( t) e t,,,,,...

12 CTMC: alteratve defto {X(t)} o state sace S s a cotuous tme Marov cha f t, s, S, Pr ( X ( t + s) X ( u), u s) Pr( X ( t + s) X ( s) ) Assume tme homogeety, we wrte ( X ( t + s) X ( s ), ( t) : Pr ) For a arbtrary tme t, tme to ext state trasto W(t) { s > : X ( t + s) X ( )} W ( t) f t Theorem D.: for a CTMC {X(t)}, S, t, u : Pr Soour tme at a state s exoetally dstrbuted wth arameter α that oly deed o A state S s called absorbg f α. ( W ( t) > u X ( t) ) e α u for some costat α. Jum cha/embedded rocess Let T, T, T, be the successve um stats (.e., stats whe state chages) of a CTMC, ad let X X(T ) Sequece T,, s called a sequece of embedded stats, ad X,, s called a um cha or a embedded rocess Theorem D.: gve a CTMC {X(t)} wth um stats T,, ad um cha X,, for,,, -,, S, t, t,, t, u, X,..., X, X, Pr X, T T > u e + +, T t,..., T t where,, ad f α >, the, S Soour tme at a state ad the ext state etered are deedet, ad oly deed o state,, α u Thus, the embedded rocess s a DTMC wth trasto robablty,

13 CTMC: trasece & recurrece A CTMC s Irreducble: f ts embedded Marov cha s rreducble, ad Regular: f umber of trastos a fte tme terval s fte wth robablty Theorem D.3: {X(t)} s a CTMC wth embedded DTMC {X } ad soour tme arameters α S, the If v s.t. α v for all, the {X(t)} s regular If {X } s recurret, the {X(t)} s regular Let τ, tme utl the rocess frst returs to after leavg t A state a CTMC s recurret f Pr(τ, <); otherwse, s traset. A recurret state s ostve f E(τ, )< ; otherwse, t s ull. Same as DTMC, the states of a rreducble CTMC are ether all traset, all ostve, or all ull A state s recurret CTMC ff. t s recurret the embedded DTMC; A rreducble CTMC s recurret ff. the embedded DTMC s recurret. Smlar results doest NOT hold for ostvty of CTMC states Trasto rate matrx Q For, S,, defe q, α, ; ca be terreted as, codtoal o beg at state, the rate of leavg to eter for S, q, -α Thus, the sum of each row of Q s Theorem D.4: a rreducble regular CTMC s ostve ff. ostve rob. vector π s.t. πq ad s uque. Note: the -th equato s S, Sπ. Whe such a π exsts, t π π α, meag the ucodtoal rate of eterg equals that of leavg q, 3

14 Examle If the ostve rob. vector π exsts, t s also a statoary rob. vector; that s, f Pr(X()) π, the M/M/ queue Pr(X(t)) π / α π τ lm, t, ( t) π No oto of erodcty for CTMC Basc Queueg Model Characterstcs of a Queue Buffer Server(s) Arrvals Queued I Servce Deartures b m A queue models ay servce stato wth: Oe or multle servers A watg area or buffer Customers arrve to receve servce A customer that uo arrval does ot fd a free server wats the buffer Number of servers m: oe, multle, fte Buffer sze b Servce dscle (schedulg) FCFS, LCFS, Processor Sharg (PS), etc Arrval rocess Servce statstcs 4

15 Arrval Process Servce-Tme Process + + τ s t t t τ : terarrval tme betwee customers ad + τ s a radom varable { τ, } s a stochastc rocess Iterarrval tmes are detcally dstrbuted ad have a commo mea E[ τ ] E[ τ ] /, where s called the arrval rate s : servce tme of customer at the server { s s a stochastc rocess, } Servce tmes are detcally dstrbuted wth commo mea E[ s, where s called the servce rate ] E[ s ] For acets, are the servce tmes really radom? Queue Descrtors Geerc descrtor: A/S/m/ A deotes the arrval rocess For Posso arrvals we use M (for Marova) S deotes the servce-tme dstrbuto M: exoetal dstrbuto D: determstc servce tmes G: geeral dstrbuto m s the umber of servers s the max umber of customers allowed the system ether the buffer or servce s omtted whe the buffer sze s fte Queue Descrtors: Examles M/M/: Posso arrvals, exoetally dstrbuted servce tmes, oe server, fte buffer M/M/m: same as revous wth m servers M/M/m/m: Posso arrvals, exoetally dstrbuted servce tmes, m server, o bufferg M/G/: Posso arrvals, detcally dstrbuted servce tmes follows a geeral dstrbuto, oe server, fte buffer */D/ : A costat delay system 5

16 Examle: M/M/ Queue Exoetal Radom Varables Arrval rocess: Posso wth rate Servce tmes: d, exoetal wth arameter Servce tmes ad terarrval tmes: deedet Sgle server Ifte watg room X(t): Number of customers system at tme t (state) + X: exoetal RV wth arameter Y: exoetal RV wth arameter X, Y: deedet The:. m{x, Y}: exoetal RV wth arameter +. P{X<Y} /(+) Proof: P{m{ X, Y} > t} P{ X > t, Y > t} P{ X > t} P{ Y > t} y XY y x y e e dx dy y y x P{ X < Y} f ( x, y) dx dy t e e e t + ) ( t ( + ) t P{m{ X, Y} t} e e e dx dy y y e ( e ) dy y ( + ) y e dy ( + ) e dy M/M/ Queue: Marov Cha Formulato Jums of {X(t): t } trggered by arrvals ad deartures P,+ P{X < Y } /(+), P,- P{Y < X } /(+) {X(t): t } ca um oly betwee eghborg states Assume rocess at tme t s state : N(t) X : tme utl the ext arrval exoetal wth arameter Y : tme utl the ext dearture exoetal wth arameter T m{x,y }: tme rocess seds at state T : exoetal wth arameter α + P, ad T s exoetal wth arameter {N(t): t } s a CTMC wth q q q q, +,,, α α,, +, -,, > 6

17 Outle πq has a ostve, summable (to ) soluto ff. < If <, Prob{queue s o-emty} -ρ, where ρ / π (- ρ)ρ,,,,, s the statoary dstrbuto Marov chas ad some reewal theory Marov cha Reewal rocesses, reewal reward rocesses, Marov reewal rocesses The excess dstrbuto Phase tye dstrbuto PASTA Level crossg aalyss Some mortat queueg models Reversblty of Marov chas ad Jacso Networ Reewal rocess Gve a sequece of mutually deedet r.v. s X,,,3,, s.t. X, > are..d., ad X ca have a ossbly dfferet dstrbuto, we defe the reewal stats, Z, >, as Z X The # of reewals tme (, t] s called a reewal rocess M(t) Examle: a CTMC B(t) wth B(), ad let s cosder vsts to state X : tme to frst vst X, >: tmes betwee subsequet vsts to M(t): # of vsts to u to tme t Reewal reward rocess To assocate a reward wth each reewal terval Formally: Gve a reewal rocess wth lfetmes X,, assocate X wth a reward R s.t. R,, are mutually deedet; R ca deed o X Examle: the CTMC B(t), defe R as the tme set at a secfc state durg the -th reewal terval 7

18 Marov reewal rocess (MRP) Geeralzed Marov Cha Let C(t) be the total reward accrued utl tme t, the the reward rate s C( t) lmt t [Reewal Reward Theorem D.5]: for E( R )< ad E( X )<, the followg hold: Wth robablty, lm E( C( t)) t t E( R ) E( X ) C( t) lm t t E( R ) E( X ) Note: geeral, E(R )/E(X ) E(R /X ) Let X,, be a radom sequece wth state sace S, ad let T T T be odecreasg sequece of radom tmes The radom sequece (X, T ),, s a Marov reewal rocess (MRP) f for,,, -,, S, t t t, u Pr X Pr + X,..., X, X,, T T u + T t,..., T t { X, T T u X } + + MRP s a geeralzato of CTMC: ) soour tme may ot be deedet of the ext state, ad ) soour tme may ot be exoetally dstrbuted { } Let lm Pr X, T T u X, assumg the, u + + lmt does ot deed o ; The, X,, s a DTMC o S wth trasto rob.,,, S Dstrbuto of soour tme gve the curret ad ext states: { T T u X X } H ( u) Pr,, + + Dstrbuto of soour tme: Mea soour tme at state : H u) H ( u) (,, S σ, where s the mea of H ( u) σ σ,,,, S Theorem D.6: Pr { T u, T T u,..., T T u } T H ( u ), Ideedet soour tmes gve the sequece of states at the ed ots 8

19 Outle Assocate a reward R wth the terval (T -, T ), for, s.t. R s deedet of aythg else gve (X -, X ) ad (T -T - ) Let r be the exected reward a terval that begs state Suose X,, s a ostve recurret DTMC o S wth statoary rob. S π σ < vector π. The uder the codtos that, C( t) Sπ r lm, wth robablty t t S π σ Marov chas ad some reewal theory Marov cha Reewal rocesses, reewal reward rocesses, Marov reewal rocesses The excess dstrbuto Phase tye dstrbuto PASTA Level crossg aalyss Some mortat queueg models Reversblty of Marov chas ad Jacso Networ Excess dstrbuto, or excess-lfe/resdual-lfe dstrbuto Gve a oegatve r.v. X wth dstrbuto F(.) ad fte mea EX ( F( u )) du, the excess dstrbuto s defed as Ca be terreted as the dstrbuto fucto of the resdual lfe see by a radom observer of a reewal rocess wth..d. lfetme X y F u du F y ( ( )) ( ) e EX Iterretato of F e (y) Cosder the reewal rocess wth..d. lfetmes X,, wth dstrbuto F(.); defe Y(t) as the resdual lfe or excess lfe at a radom tme t,.e., the tme utl the frst reewal (t, ) Cosder t lm I du : log-ru fracto of tme that the excess lfe s y t { Y u y} t ( ) t lm Pr( Y ( u) y) du t t The, by Theorem D.5, lm t t t w.. I { } du F ( y) Y ( u ) y e : tme average rob. that the excess lfe s y How? t lm Pr( Y( u) y) du F ( y) t e t 9

20 Outle Proof: defe reward fucto R m{x, y}, the t C( t) I... { Y ( u ) y } y E( R ) uf ( u) du R uf t E( C( t)) Pr( Y ( u) y) du du X y ( u) du + y( F( y)) ( F( u)) du Marov chas ad some reewal theory Marov cha Reewal rocesses, reewal reward rocesses, Marov reewal rocesses The excess dstrbuto Phase tye dstrbuto PASTA Level crossg aalyss Some mortat queueg models Reversblty of Marov chas ad Jacso Networ Phase tye dstrbuto For a CTMC X(t) o state sace {,,, M, a} s.t. states {,,, M} are all traset ad α s absorbg, the trasto rate matrx of X(t) s of the form Q q where Q s a M*M matrx, q s a colum vector of sze M; M robablty vector α of sze M (.e., α, α ) s.t., the CTMC starts state wth rob. α, ad the evolves to absorto state a The, the dstrbuto of the tme utl absorto s sad to be hase tye wth arameters (α, Q, q) Examle For - - T α (,,,), Q, q (,,, ) The hase tye dstrbuto s a Erlag dstrbuto of order 4, wth each stage beg exoetally dstrbuted wth mea / Whe the rocess s at state, t s sad to be at hase

21 Why hase-tye dstrbuto? Phase-tye dstrbuto ca be used to aroxmate arbtrarly closely ( the sese of covergece dstrbuto) ay dstrbuto Ths fact may ot always be useful for umercal aroxmato, due to the large # of hases requred for good aroxmato But t s very useful for theoretcal uroses: We ca ofte rove results usg hase tye dstrbutos thas to ther smle structure; the We ca rove that the result holds for ay dstrbuto by cosderg a sequece of hase tye dstrbutos covergg to the geeral dstrbutos Overflow rocess of M/M/c/c system The sequece of tmes at whch customers are deed servce forms a reewal rocess, ad the dstrbuto of these tmes s hase tye wth α (,,...,,), Q... T q (,,..., )... ( + ) ( + ) ( + ) Outle Marov chas ad some reewal theory Marov cha Reewal rocesses, reewal reward rocesses, Marov reewal rocesses The excess dstrbuto Phase tye dstrbuto PASTA Level crossg aalyss Posso arrvals see tme averages (PASTA) Observatos of a rocess X(t) at radom tme ots vs. Observatos of a rocess X(t) over all tme Some mortat queueg models Reversblty of Marov chas ad Jacso Networ

22 Motvatg examle Cosder a stable D/D/ queue where customers arrve erodcally at tervals of legth a ad requres a servce tme b<a. Let X(t) be the umber of customers the system at tme t The t b Average # of customers over all tme s lm X ( u) du t t a Now, observe x(t) at t a, (.e., what arrvals see o average) lm X ( t ) Pot observatos dffer from average behavors Formal characterzato Let X(t), t, be a radom rocess, ad B be a subset of the state sace of X(t); A(t) be a Posso arrval rocess wth rate, ad t,, be the arrval ots The t B t V ( t) I { X ( u ) B } du s the fracto of tme over (, t] that the rocess X(.) s B A( t ) B V ( t) I A A( t) { X ( t ) B} s the fracto of arrvals over (, t] that see the rocess X(.) B Beroull/Geometrc arrvals see tme averages (GASTA) Lac of atcato assumto: for all t, A(t+u)-A(t), u, s deedet of X(s), s t.e., for all t, future arrvals are deedet of the ast of X(.) Note: the assumto holds for deedet Posso arrval rocesses Theorem D.7: uder the lac of atcato assumto, B w V t.. B w ( ) V ff. V ( t).. V A B B.e., tme average ad arrval average are the same For queueg rocesses that evolve at dscrete tmes t T,,,,, let X deote the dscrete tme queue embedded at stats t Cosder a Beroull arrval rocess of rate,.e., at tmes t + a arrval occurs wth rob. Also called a Geometrc rocess sce ter-arrval tmes are geometrcally dstrbuted Due to lac of atcato, results smlar to PASTA holds for Beroull/geometrc arrvals ad ca be called GASTA

23 Outle Marov chas ad some reewal theory Marov cha Reewal rocesses, reewal reward rocesses, Marov reewal rocesses The excess dstrbuto Phase tye dstrbuto PASTA Level crossg aalyss Some mortat queueg models Reversblty of Marov chas ad Jacso Networ Level crossg aalyss (LCA) Whe drect dervato of statoary rob. dstrbutos (va ππp or other meas such as balace equatos) s dffcult, LCA may hel obta acllary equatos that rovde some formato about statoary dstrbuto Gve r.. X(t) o [, ) ad a x U-crossg rate U x (t): # of tmes that X(.) crosses the level x from below Dow-crossg rate D x (t): # of tmes that X(.) crosses the level x from above Outle Level crossg aalyss s based o the followg facts U ( t) D ( t), ad x lm U ( t) x t t lm t The above lmts ca usually be wrtte terms of the statoary dstrbuto of the r.. x D ( t), f ether lmt exsts x t Marov chas ad some reewal theory Marov cha Reewal rocesses, reewal reward rocesses, Marov reewal rocesses The excess dstrbuto Phase tye dstrbuto PASTA Level crossg aalyss Some mortat queueg models Reversblty of Marov chas ad Jacso Networ 3

24 Some mortat queueg models M/G/c/c queue Processor sharg queue Symmetrc queues M/G/c/c queue Posso arrvals wth fte rate Servce requremets are..d. ad geerally dstrbuted wth dstrbuto F(.) ad fte mea / Servce requremet s also called the holdg tme, sce a customer holds a dedcated server for the etre durato of ts servce Each arrvg customer s assged to a free server f oe exsts; otherwse, the arrvg customer s deed admsso ad t goes away Gve a examle of M/G/c/c queue? X(t): # of customers queue at tme t Let ρ / I M/G/c/c, ρ equals to the average # of ew arrvals durg the holdg tme of a customer (by Lttle s Theorem) [Exercse D.3]: f F(.) s a exoetal dstrbuto fucto, the X(t) s a ostve recurret CTMC o state sace {,,,c}, wth statoary dstrbuto ρ π! c ρ! Whe F(.) s ot a exoetal dstrbuto fucto, X(t) s ot Marova But (X(t), Y (t),, Y X(t) (t)) s a Marov rocess, where Y (t) deotes the resdual servce requremet of the -th customer the system; ad Pr( X ( t), Y y,..., Y y ) π ( ), F y e whereπ s as the case of exoetal hodlg tme, F (.)s the excess dstrbuto of the holdg tme dstrbuto F(.) e 4

25 Processor sharg queue: M/G/ PS Posso arrvals wth fte rate Servce requremets are..d. ad geerally dstrbuted wth dstrbuto F(.) ad fte mea / Overall servce rate: ut er secod (far) rocessor sharg rule: whe there customers system, the ufshed wor o the -th customer decreases at rate / Let ρ X(t) deotes the # of customers at tme t If F(.) s for a exoetal dstrbuto, the X(t) s a CTMC, ad t s ostve recurret ff. <, whch case the statoary dstrbuto of X(t) s gve by π ( ρ) ρ Soour tmes M/G/ PS Whe F(.) s ot a exoetal dstrbuto fucto, X(t) s ot Marova But (X(t), Y (t),, Y X(t) (t)) s a Marov rocess, where Y (t) deotes the resdual servce requremet of the -th customer the system; ad f ρ< Pr( X ( t), Y y,..., Y y ) ( ρ) ρ ( ), F y e where F (.)s the excess dstrbuto of F(.) e Note: the statoary dstrbuto of X(t) a M/G/ PS queue s the same as that a M/M/ queue, ad thus s sestve to the dstrbuto of F(.) (excet through ts mea) Soour tme W: amout of tme that a customer stays the system Sce π ( ρ) ρ, E(x) ρ/(- ρ); the, by Lttle s Theorem, E( S) E( W ),where E(S) ad s the mea servce requremet ρ s Moreover, E(W S s) ρ 5

26 Symmetrc queue Cosder the followg queue: A system state: Customers of class c, c C, arrve deedet Posso rocesses of rate c Customers of class c have a hase tye servce requremet wth c (t) φ (t) c 3 φ 3 c φ c ν((t )) φ arameter (α c, Q c ) ad mea / c A arrvg customer fdg (-) customers the system os osto l, l, wth rob. γ(, l) Whe there are customers the queue, the overall servce rate aled s ν(); ad a fracto δ(, l) of the servce effort s aled to the customer at osto l class hase A aforemetoed queueg system s sad to be a symmetrc queue f the fuctos δ(.,.) ad γ(.,.) are such that δ(, l) γ(, l) Postog mles rorty Examles M/PH/ queue wth last-come-frst-serve reemtve resume (LCFS-PR) dscle ν() costat ν M/PH/ queue? ν() ν γ(, l) δ(, l) /, for l γ(, ), ad γ(, ) for > δ(, ), ad δ(, ) for > M/PH/ rocessor sharg queue? ν() costat ν γ(, l) δ(, l) /, for l 6

27 Statoary dstrbuto c Let ρ c C c Theorem D.8: the statoary dstrbuto of the # of customers a symmetrc queue s gve by the rob. dstrbuto of system state x x ρ π ( x) G ν () ν ()... ν ( x ) where x # of G s the ormalzato costat Note: the dstrbuto s sestve to the servce requremet dstrbutos (excet for ther mea / c, c C) customers at state x, ad Outle Marov chas ad some reewal theory Marov cha Reewal rocesses, reewal reward rocesses, Marov reewal rocesses The excess dstrbuto Phase tye dstrbuto PASTA Level crossg aalyss Some mortat queueg models Reversblty of Marov chas ad Jacso Networ Tme Reversblty ad Bure s Theorem Tme-Reversal of Marov Chas Reversblty Trucatg a Reversble Marov Cha Bure s Theorem Queues Tadem Tme Reversblty ad Bure s Theorem Tme-Reversal of Marov Chas Reversblty Trucatg a Reversble Marov Cha Bure s Theorem Queues Tadem 7

28 Tme-Reversed Marov Chas {X :,, } rreducble aerodc Marov cha wth trasto robabltes P P,,,... Uque statoary dstrbuto (π > ) ff. GBE holds,.e., π π P,,,... Process steady state: Pr{ X } π lm Pr { } X X Starts at -, that s {X :,-,,, }, or Choose tal state accordg to the statoary dstrbuto Tme-Reversed Marov Chas Defe Y X τ-, for arbtrary τ> > {Y } s the reversed rocess. Proosto : {Y } s a Marov cha wth trasto robabltes: * π P P,,,,,... π {Y } has the same statoary dstrbuto π wth the forward cha {X } The reversed cha corresods to the same rocess, looed at the reversedtme drecto How does {X } loo reversed tme? Tme-Reversed Marov Chas Proof of Proosto : P P{ Y Y, Y, K, Y } * m m m m P{ X X, X, K, X } + τ τ m τ m+ τ m+ m P{ X X, X, K, X } P{ X, X +, X +, K, X + } P{ X, X, K, X } K P{ X,, X X, X } P{ X, X } P{ X, K, X X } P{ X } P{ X, X } P{ X } P{ X X } P{ X } Pπ P{ X } π + + π P * πp π π π P π P{ X X } P{ Y Y } + m m Tme Reversblty ad Bure s Theorem Tme-Reversal of Marov Chas Reversblty Trucatg a Reversble Marov Cha Bure s Theorem Queues Tadem 8

29 Reversblty Stochastc rocess {X(t)} s called reversble f (X(t ), X(t ),, X(t )) ad (X(τ-t ), X(τ-t ),, X(τ-t )) have the same robablty dstrbuto, for all τ, t,, t Proosto D.: f {X(t), t R} s statoary, the a tme reversed rocess s also statoary Proosto D.: a reversble rocess s statoary (ad cosequetly ay tme reversal of a reversble rocess s statoary). Marov cha {X } s reversble f ad oly f the trasto robabltes of forward ad reversed chas are equal,.e., P P * Detaled Balace Equatos Reversblty π P π P,,,,... Reversblty Dscrete-Tme Chas Theorem : If there exsts a set of ostve umbers {π }, that sum u to ad satsfy: π P π P,,,,... The:. {π } s the uque statoary dstrbuto. The Marov cha s reversble Examle: Dscrete-tme brth-death rocesses are reversble, sce they satsfy the DBE Examle: Brth-Death Process P + P P Oe-dmesoal Marov cha wth trastos oly betwee eghborg states: P, f - > Detaled Balace Equatos (DBE), + + +, Proof: GBE wth S {,,,} gve: S P, P, π P π P,,... π P π P π P π P P,, + + +, + + S c P, + P +, 9

30 Tme-Reversed Marov Chas (Revsted) Theorem : Irreducble Marov cha wth trasto robabltes P. If there exst: A set of trasto robabltes P *, wth P *,, ad A set of ostve umbers {π }, that sum u to, such that The: * π P π P,, P * are the trasto robabltes of the reversed cha, ad {π } s the statoary dstrbuto of the forward ad the reversed chas Remar: Used to fd the statoary dstrbuto, by guessg the trasto robabltes of the reversed cha eve f the rocess s ot reversble Cotuous-Tme Marov Chas {X(t): -< t <} rreducble aerodc Marov cha wth trasto rates q, Uque statoary dstrbuto ( > ) ff. Process steady state e.g., started at t -: If {π }, s the statoary dstrbuto of the embedded dscrete-tme cha: q q,,,... Pr{ X ( t) } lm Pr{ X ( t) X () } t π / ν, π / ν ν q,,,... Reversed Cotuous-Tme Marov Chas Reversed cha {Y(t)}, wth Y(t)X(τ-t), for arbtrary τ> Proosto :. {Y(t)} s a cotuous-tme Marov cha wth trasto rates: * q q,,,,...,. {Y(t)} has the same statoary dstrbuto { } wth the forward cha Remar: The trasto rate out of state the reversed cha s equal to the trasto rate out of state the forward cha q q q q ν,,,... * α, Reversblty Cotuous-Tme Chas Marov cha {X(t)} s reversble ff. the trasto rates of forward ad reversed chas are equal q q *, or equvaletly q q,,,,...,.e., Detaled Balace Equatos Reversblty Theorem 3: If there exsts a set of ostve umbers { }, that sum u to ad satsfy: q q,,,,..., The:. { } s the uque statoary dstrbuto. The Marov cha s reversble 3

31 Examle: Brth-Death Process + Trastos oly betwee eghborg states q, q, q, >, +, Detaled Balace Equatos + +,,,... Proof: GBE wth S {,,,} gve: M/M/, M/M/c, M/M/ q q S S c + Reversed Cotuous-Tme Marov Chas (Revsted) Theorem 4: Irreducble cotuous-tme Marov cha wth trasto rates q. If there exst: A set of trasto rates q *, wth q * q,, ad A set of ostve umbers { }, that sum u to, such that * q q,,, The: q * are the trasto rates of the reversed cha, ad { } s the statoary dstrbuto of the forward ad the reversed chas Remar: Used to fd the statoary dstrbuto, by guessg the trasto robabltes of the reversed cha eve f the rocess s ot reversble Reversblty: Trees Theorem 5: Irreducble Marov cha, wth trasto rates that satsfy q > q > Form a grah for the cha, where states are the odes, ad for each q >, there s a drected arc The, f grah s a tree cotas o loos the Marov cha s reversble Remars: q q Suffcet codto for reversblty q q 6 q q 6 Geeralzato of oe-dmesoal brth-death rocess 6 q 3 q 3 q 67 q Kolmogorov s Crtero (Dscrete Cha) Detaled balace equatos determe whether a Marov cha s reversble or ot, based o statoary dstrbuto ad trasto robabltes Should be able to derve a reversblty crtero based oly o the trasto robabltes! Theorem D.: A dscrete-tme Marov cha s reversble ff. P P LP P P P LP P 3 3 for every fte sequece of states:,,,, ad ay Ituto: Probablty of traversg ay loo s equal to the robablty of traversg the same loo the reverse drecto 3

32 Kolmogorov s Crtero (Cotuous Cha) Detaled balace equatos determe whether a Marov cha s reversble or ot, based o statoary dstrbuto ad trasto rates Should be able to derve a reversblty crtero based oly o the trasto rates! Theorem 7: A cotuous-tme Marov cha s reversble f ad oly f: q q Lq q q q Lq q 3 3 for ay fte sequece of states:,,,, ad ay Ituto: Product of trasto rates alog ay loo s equal to the roduct of trasto rates alog the same loo traversed the reverse drecto Kolmogorov s Crtero (roof) Proof of Theorem D.: Necessary: If the cha s reversble the DBE hold πp π P πp π 3 3P 3 M P P LP P P P P P L π P π P πp π P Suffcet: Fxg two states, ad ad summg over all states,, - we have Tag the lmt 3 3 P P LP P P P LP P P P P P,,,, 3 3 lm P P P lm P π P Pπ Examle: M/M/ Queue wth Heterogeeous Servers Theorem D.: A dscrete-tme Marov cha s reversble ff. q q Lq q q q Lq q 3 3 for every mmal, fte sequece of states:,,, α ( α) A B A B B A M/M/ queue. Servers A ad B wth servce rates A ad B resectvely. Whe the system emty, arrvals go to A wth robablty α ad to B wth robablty -α. Otherwse, the head of the queue taes the frst free server. 3 A + B A + B Need to ee trac of whch server s busy whe there s customer the system. Deote the two ossble states by: A ad B. Reversblty: we oly eed to chec the loo A B : q, Aq A,q,Bq B, α A B q,bq B,q,Aq A, ( α) B A Reversble f ad oly f α/. 3

33 Tme Reversblty ad Bure s Theorem Tme-Reversal of Marov Chas Reversblty Trucatg a Reversble Marov Cha Bure s Theorem Queues Tadem Trucato of a Reversble Marov Cha Theorem D.: {X(t)} reversble Marov rocess wth state sace S, ad statoary dstrbuto { : S}. Trucated to a set E S, such that the resultg cha {Y(t)} s rreducble. The, {Y(t)} s reversble ad has statoary dstrbuto: %, E Remar: Ths s the codtoal robablty that, steady-state, the orgal rocess s at state, gve that t s somewhere E Proof: Verfy that: E % q % q q q q q,, S; E E % E E E Examle Tme Reversblty ad Bure s Theorem Tme-Reversal of Marov Chas Reversblty Two deedet arrvals Jot rocess of queue legth (X (t), X (t)) s a CTMC For (a): π ftebuffers (a) ( a ) ( ρ ) ρ ( ρ ), (b) s a trucated verso of (a) the sese E {(+) : + K}, thus π, ( b ) ρ ( ρ ) ρ ( ρ ) ρ (, ) E ( ρ ) ρ ( ρ ) ρ buffer sze sk (b): same as (a) excet for that two Arrvals share a caacty-lmted buffer Trucatg a Reversble Marov Cha Bure s Theorem Brth-death rocesses: Posso deartures Queues Tadem 33

34 Brth-death rocess {X(t)} brth-death rocess wth statoary dstrbuto { } Arrval eochs: ots of crease for {X(t)} Dearture eoch: ots of decrease for {X(t)} {X(t)} comletely determes the corresodg arrval ad dearture rocesses Arrvals Deartures Forward & reversed chas of brth-death rocesses Posso arrval rocess:, for all Brth-death rocess called a (, )-rocess Examles: M/M/, M/M/c, M/M/ queues Posso arrvals LAA: for ay tme t, future arrvals are deedet of {X(s): s t} (, )-rocess at steady state s reversble: forward ad reversed chas are stochastcally detcal > Arrval rocesses of the forward ad reversed chas are stochastcally detcal > Arrval rocess of the reversed cha s Posso wth rate + the arrval eochs of the reversed cha are the dearture eochs of the forward cha > Dearture rocess of the forward cha s Posso wth rate Bure s Theorem t t t t Reversed cha: arrvals after tme t are deedet of the cha hstory u to tme t (LAA) Theorem : Cosder a (, )-rocess (e.g., those M/M/, M/M/c, or M/M/ systems). Suose that the system starts at steady-state. The:. The dearture rocess s Posso wth rate. At each tme t, the umber of customers the system s deedet of the dearture tmes ror to t > Forward cha: deartures ror to tme t ad future of the cha {X(s): s t} are deedet Fudametal result for study of etwors of M/M/* queues, where outut rocess from oe queue s the ut rocess of aother 34

35 Tme Reversblty ad Bure s Theorem Tme-Reversal of Marov Chas Reversblty Trucatg a Reversble Marov Cha Bure s Theorem Queues Tadem Sgle-Server Queues Tadem Customers arrve at queue accordg to Posso rocess wth rate. Servce tmes exoetal wth mea /. Assume servce tmes of a customer the two queues are deedet. Assume ρ / < Stato Stato Posso What s the ot statoary dstrbuto of N ad N umber of customers each queue? (, ) ( ρ ) ρ ( ρ ) ρ ( ) ( ) Result: steady state the queues are deedet Stato Stato Posso Q s a M/M/ queue. At steady state ts dearture rocess s Posso wth rate. Thus Q s also M/M/. Margal statoary dstrbutos: ( ) ( ρ ) ρ,,,... ( ) ( ρ ) ρ,,,... To comlete the roof: establsh deedece at steady state Q at steady state: at tme t, N (t) s deedet of deartures ror to t, whch are arrvals at Q u to t. Thus N (t) ad N (t) deedet: P{ N ( t), N ( t) } P{ N ( t) } P{ N ( t) } ( ) P{ N ( t) } Lettg t, the ot statoary dstrbuto (, ) ( ) ( ) ( ) ρ ρ ( ρ ) ρ Note: f N(t) s ot ts statoary verso, N (t) ad N(t) are NOT deedet. The asymtotc result, however, stll holds. Queues Tadem Theorem: Networ cosstg of K sgle-server queues tadem. Servce tmes at queue exoetal wth rate, deedet of servce tmes at ay queue. Arrvals at the frst queue are Posso wth rate. The statoary dstrbuto of the etwor s: K K ρ ρ (, K, ) ( ),,,...;,..., K At steady state the queues are deedet; the dstrbuto of queue s that of a solated M/M/ queue wth arrval ad servce rates ad ( ) ( ρ ) ρ,,,... Are the queues deedet f ot steady state? Are stochastc rocesses {N (t)} ad {N (t)} deedet? 35

36 Queues Tadem: State-Deedet Servce Rates Theorem : Networ cosstg of K queues tadem. Servce tmes at queue exoetal wth rate ( ) whe there are customers the queue deedet of servce tmes at ay queue. Arrvals at the frst queue are Posso wth rate. The statoary dstrbuto of the etwor s: K (, K, K ) ( ),,,...;,..., K where { ()} s the statoary dstrbuto of queue solato wth Posso arrvals wth rate Examles:./M/c ad./m/ queues If queue s./m/, the: ( / ) e ( ) /!,,,... Jacso Networs Oe Jacso Networs Networ Flows State-Deedet Servce Rates Networs of Trasmsso Les & Kleroc s Assumto Closed Jacso Networs Jacso Networs Oe Jacso Networs Networs of./m/ Queues Networ Flows γ r r State-Deedet Servce Rates Networs of Trasmsso Les & Kleroc s Assumto Closed Jacso Networs γ r Networ of K odes; Node s./m/-fcfs queue wth servce rate Exteral arrvals deedet Posso rocesses γ: rate of exteral arrvals at ode Marova routg: customer comletg servce at ode s routed to ode wth robablty r or exts the etwor wth robablty r - r Routg matrx R[r ] rreducble exteral arrvals evetually ext the system 36

37 Jacso Networ Defto: A Jacso etwor s the CTMC {N(t)}, wth N(t)(N (t),, N K (t)) that descrbes the evoluto of the revously defed etwor, where N (t) # of customers at ode Possble states: (,,, K ),,,,,,..,K Jacso s Theorem for Oe Networs : total arrval rate at ode K γ + r,,..., K Oe etwor: for some ode : γ > For ay state, defe the followg oerators: A + e arrval at D e dearture from T e + e trasto from to Trasto rates for the Jacso etwor: q(, A) γ whle q(,m) for all other states m q(, D ) r { > },,..., K q(, T ) r { > } e (,,,,,,) s ut vector wth legth K ad the -th osto beg Routg matrx s rreducble > Lear system has a uque soluto,,, K Theorem 3: Cosder a Jacso etwor, where ρ / <, for every ode. The statoary dstrbuto of the etwor s ( ) ( ),, K, K where for every ode,,,k K ( ) ( ρ ) ρ, γ r r γ r Jacso s Theorem (roof) Guess the reverse Marov cha ad use Theorem 4 Clam: The etwor reversed tme s a Jacso etwor wth the same servce rates, whle the arrval rates ad routg robabltes are r γ γ r, r, r * * * Verfy that for ay states ad m, * ( m) q ( m, ) ( ) q(, m) Need to rove oly for ma, D, T. We show the roof for the frst two cases the thrd s smlar q ( A, ) q ( A, D A ) r ( γ / ) ( A ) q ( A, ) ( ) q(, A ) ( A ) ( / ) ( ) ( A ) ( ) * * * γ γ ρ * * * * q ( D, ) q ( D, A D) γ r * ( D) q ( D, ) ( ) q(, D ) ( D ) r ( ) r { > } ρ ( D ) ( ){ > } Jacso s Theorem (roof cot.) Fally, verfy that for ay state : * q(, m) q (, m) m m q(, m) γ + r { > } + r { > } m, γ + [ r + r ] { > } γ + { > } q (, m) γ + { > } r + { > } * * m Thus, we eed to show that γ r r r r ( γ ) γ 37

38 Outut Theorem for Jacso Networs Theorem 4: The reversed cha of a statoary oe Jacso etwor s also a statoary oe Jacso etwor wth the same servce rates, whle the arrval rates ad routg robabltes are * * r * γ γ r, r, r Theorem 5: I a statoary oe Jacso etwor, the dearture rocess from the system at ode s Posso wth rate r. The dearture rocesses are deedet of each other, ad at ay tme t, ther ast u to t s deedet of the state of the system N(t). Remar: ) The total arrval rocess at a gve ode s ot Posso. The dearture rocess from the ode s ot Posso ether. However, the rocess of the customers that ext the etwor at the ode s Posso. ) I geeral, a oe Jacso etwor eed ot be reversble Arrval Theorem Oe Jacso Networs The comoste arrval rocess at ode a oe Jacso etwor has the PASTA roerty, although t eed ot be a Posso rocess Theorem 6: I a oe Jacso etwor at steady-state, the robablty that a comoste arrval at ode fds customers at that ode s equal to the (ucodtoal) robablty of customers at that ode: ( ) ( ρ ) ρ,,,..., K (Proof s omtted) Jacso Networs Oe Jacso Networs Networ Flows State-Deedet Servce Rates Networs of Trasmsso Les & Kleroc s Assumto Closed Jacso Networs No-Posso Iteral Flows Jacso s theorem: the umbers of customers the queues are dstrbuted as f each queue s a solated M/M/ wth arrval rate, deedet of all others Total arrval rocess at a queue, however, eed ot be Posso Loos allow a customer to vst the same queue multle tmes ad troduce deedeces that volate the Posso roerty Iteral flows are Posso acyclc etwors Smlarly. the dearture rocess from a queue s ot Posso geeral The rocess of deartures that ext the etwor at the ode s Posso accordg to the outut theorem 38

39 Examle # Examle: Sgle queue wth >>, where uo servce comleto a customer s fed bac wth robablty, og the ed of the queue The total arrval rocess does ot have deedet terarrval tmes: Posso If a arrval occurs at tme t, there s a very hgh robablty that a feedbac arrval wll follow (t, t+δ] Queue >> Posso At arbtrary t, the robablty of a arrval (t, t+δ] s small sce s small Jacso Networs Oe Jacso Networs Networ Flows State-Deedet Servce Rates Networs of Trasmsso Les & Kleroc s Assumto Closed Jacso Networs Arrval rocess cossts of bursts, each burst trggered by a sgle customer arrval State-Deedet Servce Rates Servce rate at ode deeds o the umber of customers at that ode: ( ) whe there are customers at ode But servce rate at does ot deed o the # of customers at other odes E.g,./M/c ad./m/ queues Theorem 7: The statoary dstrbuto of a oe Jacso etwor where the odes have state-deedet servce rates s K ( ) ( ),, K, K where for every ode,,,k ( ), G () L ( ) wth ormalzato costat G < () L ( ) (Proof follows detcal stes wth the roof of Theorem 3) Remar: The statoary dstrbuto has the roduct form; but f the etwor starts from some arbtrary tal state, the queues are ot deedet at ay fte tme Smlar to the examle of two M/M/ queues tadem (as dscussed at the ed of Aedx D.3. of boo R) 39

40 Jacso Networs Oe Jacso Networs Networ Flows State-Deedet Servce Rates Networs of Trasmsso Les & Kleroc s Assumto Networ of Trasmsso Les Real Networs: May trasmsso les (queues) teract wth each other Outut from oe queue eters aother queue, Mergg wth other acet streams deartg from the other queues > ) Iterarrval tmes at varous queues become strogly correlated wth acet legths; ) Servce tmes at varous queues are ot deedet Queueg models become aalytcally tractable Aalytcally Tractable Queueg Networs: Closed Jacso Networs Ideedece of terarrval tmes ad servce tmes Exoetally dstrbuted servce tmes Networ model: Jacso etwor Product-Form statoary dstrbuto Kleroc Ideedece Assumto. Iterarrval tmes at varous queues are deedet. Servce tme of a gve acet at the varous queues are deedet Legth of the acet s radomly selected each tme t s trasmtted over a etwor l 3. Servce tmes ad terarrval tmes: deedet Assumto has bee valdated wth exermetal ad smulato results Steady-state dstrbuto aroxmates the oe descrbed by Jacso s Theorems Good aroxmato whe: Posso arrvals at etry ots of the etwor Pacet trasmsso tmes early exoetal Several acet streams merged o each l Desely coected etwor Moderate to heavy traffc load Jacso Networs Oe Jacso Networs Networ Flows State-Deedet Servce Rates Networs of Trasmsso Les & Kleroc s Assumto Closed Jacso Networs 4

41 Closed Jacso Networs r 5 r Closed Jacso Networ (cotd.) Aggregate arrval rates K,,..., r K r 5 r M M Closed Networ: K odes wth exoetal servers No exteral arrvals (γ), o deartures (r ) Fxed umber M of crculatg customers Steady-state dstrbuto s of roduct-form tye (as show later) The arrval rates are relatve arrval rates vst ratos betwee states No uque soluto, ad ca oly be determed u to a multlcatve costat Use a addtoal equato to obta uque soluto to the above system, e.g. Set, for some ode Set, for some ode Set K Closed Jacso Networ (cotd.) Let x be the umber of customers at stato, at steady state Radom varables x, x,, x K are ot deedet ther sum must be equal to M The state x(x, x,, x K ) of the closed etwor ca tae values (,,, K ), wth K ad M Let F(M) deote the set of all such states Jacso s theorem for closed etwors gves the statoary dstrbuto P x P x K xk K ( ) { } {,, } Jacso s Theorem for Closed Networs Theorem : The statoary dstrbuto of a closed Jacso etwor s K ( ) ρ, for all F ( M ) { :, M} G( M ) where ρ / (ote, ths s o the actual utlzato factor of stato ), ad the ormalzato costat G(M) s a fucto of M G(M) guaratees that {()} s a vald robablty dstrbuto Ths statoary dstrbuto s sad to have a roduct-form However: at steady-state the queues are ot deedet ( ) G( M ) ρ F F M M ) ( ) ( { ( )}: margal statoary dstrbuto of queue K ( ) ( ) L ( ) K K 4

42 Jacso s Theorem for Closed Networs (roof) Theorem : The reversed cha of a statoary closed Jacso etwor s also a statoary closed Jacso etwor wth the same servce rates ad routg * robabltes: r r / Proof of Theorems & : Frst, show that for the corresodg forward ad reversed chas * ( m) q ( m, ) ( ) q(, m),, m F ( M ), m Need to rove oly for mt q(, T ) r { > } q ( T, ) q ( e + e, ) r { + > } ( r / ) * * * T q T q T T r r ( ) (, ) ( ) (, ) ( ) ( / ) ( ) { > } * ( T ) ρ ρ ( ){ > } Verfy, exactly as the oe-etwor case, that: F q(, m) * q (, m) { > }, ( M ) m m State-Deedet Servce Rates Theorem: The statoary dstrbuto of a closed Jacso etwor where the odes have state-deedet servce rates s ( ) K, for all ( ) { :, } G( M ) () ( ) F L K K where the ormalzato costat G(M) s a fucto of M, the fxed umber of customers the etwor Normalzato costat: K G( M ) ( ) () L ( ) F ( K ) F ( K ) Proof smlar to the oe for oe etwors Theorem D.7: a tme reversal of a closed Jacso etwor s also a closed Jacso etwor. Examle Closed etwor model for CPU (rate ) ad I/O (rate ) system. Uo servce comleto, customer routed to wth robablty -, or bac to wth robablty. M fxed umber of customers +,. Choose soluto: ad ρ, ρ Statoary dstrbuto: customers ad M- M ( M, ) ρ ρ ρ,,,..., M G( M ) G( M ) Normalzato costat M M ρ G( M ) ρ ρ Utlzato factor ad throughut of ode : M ρ G( M ) G( M ) U ( M ) (, M ) γ ( M ) U ( M ) G( M ) G( M ) G( M ) + 4

43 Closed Networs: Normalzato Costat Normalzato costat as a fucto of M ad K: All erformace measures of terest throughut, average delay ca be obtaed terms of G(M,K) Comutatoal comlexty s exoetal M ad K: M + K M + K terms thesummato K M Recursve methods ca be used to reduce comlexty Iteratve algorthm [due to Buze] K L K F ( ) + L+ M K M K G( M, K ) ρ ρ ρ ρ Normalzato costat wll be treated as a fucto of both M ad K ad deoted G(M,K) oly the cotext of the teratve algorthm Iteratve Comutato of G(M) For ay m ad (m,, M;,, K) defe: For a closed etwor of sgle-server queues G(M,K) ca be comuted teratvely usg the followg recursve relato: wth boudary codtos: F ( m) + L+ m G( m, ) ρ ρ ρ Lρ G( m, ) G( m, ) + ρ G( m, ) G( m,) ρ, m,,..., M m G(, ),,,..., K Iteratve Algorthm (roof) For m > ad > we slt the sum to two sums over dsot sets of states, corresodg to, ad >. G( m, ) ρ ρ Lρ + L+ M + L+ + L+ m m > ρ ρ Lρ + ρ ρ Lρ ρ ρ Lρ + ρ ρ Lρ + L+ + L+ m m > Note that the frst sum s G( m, ). For the secod sum, observg that >, we defe +, where. The: ρ ρ Lρ ρ ρ Lρ + L+ + L+ + m m > Therefore: G( m, ) G( m, ) + ρ G( m, ) + ρ ρ ρ Lρ ρ G( m, ) + L m + Iteratve Algorthm Examle r 5 r 53, 3 4 r r ρ ρ /, ρ 3 ρ 4 3 / ρ ρ / / 4 ρ / ρ, wth ρ / 5 5 Vst ratos determed u to a multlcatve costat Lettg, we have: Calculato of G(M,5) based o the teratve algorthm usg these values 5 ρ ρ, ρ 3 ρ 4, ρ 5 4 / ρ M 43

44 Iteratve Algorthm Examle m /ρ (6+4/ρ)(4/ρ) [3+(6+4/ρ)(4/ρ)](4/ρ) ρ ρ, ρ 3 ρ 4, ρ 5 4 / ρ Boudary codtos: G( m,) ρ, m,,..., M Iterato: m G(, ),,,..., K G( m, ) G( m, ) + ρ G( m, ) Examle: G(,) G(,) + ρ G(,) + G(,3) G(,) + ρ G(,3) + 4 G(,) G(,) + ρ G(,) Summary Marov chas ad some reewal theory Marov cha Reewal rocesses, reewal reward rocesses, Marov reewal rocesses The excess dstrbuto Phase tye dstrbuto PASTA Level crossg aalyss Some mortat queueg models Reversblty of Marov chas ad Jacso Networ 44