BOUNDS FOR THE COUPLING TIME IN QUEUEING NETWORKS PERFECT SIMULATION

Transcription

1 BOUNDS FOR THE COUPLING TIME IN QUEUEING NETWORKS PERFECT SIMULATION JANTIEN G. DOPPER, BRUNO GAUJAL AND JEAN-MARC VINCENT Abstract. In this aer, the duration of erfect simulations for Markovian finite caacity ueuing networks is studied. This corresonds to hitting time (or couling time) roblems in a Markov chain over the Cartesian roduct of the state sace of each ueue. We establish an analytical formula for the exected simulation time in the one ueue case which rovides simle bounds for acyclic networks of ueues with losses. These bounds corresond to sums on the couling time for each ueue and are either almost linear in the ueue caacities under light or heavy traffic assumtions or uadratic, when service and arrival rates are similar. Key words. Perfect simulation, Markov chain, Hitting time AMS subject classifications. 6J1, 6J22, 65C4, 65C2, 68U2 1. Introduction. Markov chains are an imortant tool in modelling systems. Amongst others, Markov chains are being used in the theory of ueueing systems, which itself is used in a variety of alications as erformance evaluation of comuter systems and communication networks. In modelling any ueueing system, one of the main oints of interest is samling the behavior of the system in the long run. For an irreducible, ergodic (i.e. aeriodic and ositive-recurrent) Markov chain with robability matrix P, this long run behavior follows the stationary distribution of the chain given by the uniue vector π which satisfies the linear system π = πp. However, it may be hard to comute this stationary distribution, esecially when the finite state sace is huge which is freuent in ueuing models. In that case, several aroaches have been roosed to get samles of the long run behavior of the system. The most classical methods are indirect. They consists in first comuting an estimation of π and then samle according to this distribution (by classical methods such as.d.f. inverse, rejection or aliasing). Estimating π can be done through efficient numerical iterative methods solving the linear system π = πp. [1]. Even if they converge fast, they do not scale when the state sace (and thus P ) grows. Another aroach to estimate π is to use a regenerative simulation [4, 7] based on the fact that on a trajectory of a Markov chain returning to its original state, the freuency of the visits to each state is steady state distributed. This techniue does not suffer from statistical biais but is very sensitive to the return time to the regenerative state. This means that the choice of the initial state is crucial but also that regenerative simulation comlexity increases fast with the state sace which is exonential in the number of ueues. This can be artially overcome by using imortance samling [3] or semi-regenerative simulation [2]. However, the simulation times still tyically exhibit a multilicative behavior with the number of ueues. There also exist direct techniues to samle states of Markov chain according to its stationary distribution. The classical method has been Monte Carlo simulation for many years. This method is based on the fact that for an irreducible aeriodic finite This work was artially suorted by the French ACI SurePath roject and SMS ANR Jantien Doer, Mathematical institute, Leiden University, NL (jgdoer@math.leidenuniv.nl) (this author was artially suorted by a grant from INRIA), Bruno Gaujal and Jean-Marc Vincent, Laboratoire ID-IMAG, Mescal Project INRIA-UJF-CNRS-INPG, 51, avenue Jean Kuntzmann, F Montbonnot, France, ({Bruno.Gaujal,Jean-Marc.Vincent}@imag.fr ) 1

2 2 J. Doer and B. Gaujal and J-M. Vincent Markov chain with initial distribution π (), the distribution π (n) of the chain at time n converges to π as n gets very large: lim n π(n) = lim n π() P n = π. After running the Markov chain long enough, the state of the chain will not deend on the initial state anymore. However, the uestion is how long is long enough? That is, when is n sufficiently large so that π (n) π ɛ for a certain ɛ >? Moreover, the samles generated by this method will always be biased. In 1996, Pro and Wilson[8] solved these roblems for Markov chain simulation by roosing an algorithm which returns exact samles of the stationary distribution very fast. The striking difference between Monte Carlo simulation and this new algorithm is that Pro and Wilson do not simulate into the future, but go backwards in time. The main idea is, while going backwards in time, to run several simulations, starting with all s S until the state at t = is the same for all of them. If the outut is the same for all runs, then the chain has couled. Because of this couling roerty and going backwards, this algorithm has been called Couling From The Past (from now on: CFTP). A more detailed descrition of this algorithm will be resented in section 2. When the couling from the ast techniue is alicable, one gets in a finite time one state with steady-state distribution. Then one can use either a one long-run simulation from this state avoiding the estimation of the initial transient roblem or relicate indeendently the CFTP algorithm to get a samle of indeendent steadystate distributed variables. The analysis of the choice could be done exactly as in [1]. The relication techniue has been alied successfully in finite caacity ueueing networks with blocking and rejection (very large state-sace) [12]. The efficiency of the simulation allows also the estimation of rare events (blocking robability, rejection rate) is done in [11]. The aim of this aer is to study the simulation time needed to get a stationary samle for finite caacity networks. We show that for monotone systems CFTP scales very well with the state sace exlosion accomanying the increase in the number of ueues. More recisely, we study the couling time τ of a CFTP algorithm (i.e. the number of stes needed to rovide one samle). Our main interest is setting bounds on the exected couling time. We first obtain exact analytical values for the exected simulation time for one M/M/1/C ueue which serves as a building block for the following). As for networks of ueues, we show how uer bounds on the mean simulation time can be obtained as sums of the couling times for all ueues. One of the main result of this aer is to show that for acyclic networks with rejection in case of overflow, Eτ i Λ C i + Ci 2, Λ i 2 where Λ is the global event rate in the network, Λ i is the rate of events affecting Queue i and C i is the caacity of Queue i. This result can be refined under light or heavy traffic assumtions to almost linear bounds in the caacities. All these bounds scale very well with the number of ueues. This exlains why erfect simulation of monotone ueueing networks is so fast, esecially when dealing with large scale

3 Bounds for the Couling Time 3 networks as in [11] where systems with u to 32 ueues of caacity 3 (the state sace is of size ) are samled over a classical deskto comuter is less than 2 milli-seconds. This is good enough to estimate rare event robabilities. The aer is organized as follows. We first introduce the couling from the ast algorithm in Section 2. Then we show general roerties of the couling time for oen Markovian ueueing networks in Section 3. We will investigate the M/M/1/c ueue in Section 4 roviding exact comutation for the exected couling time and the case of acyclic networks in Section 5 where bounds are derived, together with several exerimental tests assessing their uality. 2. Couling from the Past. Let {X n } n N be an irreducible and aeriodic discrete time Markov chain with a finite state sace S and a transition matrix P = ( i,j ). Let φ : S E S, encode the chain, which means that it verifies the roerty P (φ(i, e) = j) = i,j for every air of states (i, j) S and for any e, a random variable distributed on E. The function φ could be considered as a construction algorithm and e is the innovation for the chain. In the context of discrete event systems, e is an event and φ is the transition function. Now, the evolution of the Markov chain is described as a stochastic recursive seuence X n+1 = φ (X n, e n+1 ), (2.1) with X n the state of the chain at time n and {e n } n N an indeendent and identically distributed seuence of random variables. Let φ (n) : S E n S denote the function whose outut is the state of the chain after n iterations and starting in state s S. That is, φ (n) (s, e 1 n ) = φ (... φ (φ (s, e 1 ), e 2 ),..., e n ). (2.2) This notation can be extended to set of states. So for a set of states A S we note { } φ (n) (A, e 1 n ) = φ (n) (s, e 1 n ), s A. In the following, X denotes the size of set X. theorem 2.1 ([8]). Let φ be a transition function on S E. There exists an integer l such that lim φ (n) (S, e 1 n ) = l almost surely. n + The system coules if l = 1. Then the forward couling time τ f defined by τ f = min{n N; such that φ (n) (S, e 1 n ) = 1}, is almost surely finite. The couling roerty of a system φ deends only on the structure of φ. The robability measure on E does not affect the couling roerty, rovided that all events in E have a ositive robability. Moreover, the existence of some attern e 1 n that ensures couling, guarantees that τ f is stochastically uer bounded by a geometric distribution P(τ f k.n ) ( 1 (e 1).(e 2)... (e n ) ) k ; (2.3)

4 4 J. Doer and B. Gaujal and J-M. Vincent where (e) > is the robability of event e. At time τ f, all trajectories issued from all initial states at time have collased in only one trajectory. Unfortunately, the distribution of X τ f is not stationary. In [6] an examle is given that illustrates why it is not ossible to consider that this rocess has the stationary regime. In fact, this iteration scheme could be reversed in time as it is usually done in the analysis of stochastic oint rocesses. For that, one needs to extend the seuence of events to negative indexes and build the reversed scheme on sets by A n = φ (n) (S, e n+1 ). It is clear that the seuence of sets A n is non-decreasing (A n+1 A n ). Conseuently, the system coules if the seuence A n converges almost surely to a set with only one element. Almost surely, there exists a finite time τ b, the backward couling time, defined by τ b = min{n N; such that φ (n) (S, e n+1 ) = 1}. Proosition 2.2 ([13]). The backward couling time τ b and the forward couling time τ f have the same robability distribution. The main result of the backward scheme is the following theorem. theorem 2.3 ([8]). Provided that the system coules, the state when couling occurs for the backward scheme, is steady state distributed. From this fact, a general algorithm (1) samling the steady state can be constructed. Algorithm 1 Backward-couling simulation (general version) for all s S do y(s) s {choice of the initial value of the vector y, n = } end for reeat e Random event; {generation of e n+1 } for all s S do y(s) y(φ(s, e)); {y(s) state at time of the trajectory issued from s at time n + 1} end for until All y(x) are eual return y(x) The mean comlexity c φ of this algorithm is c φ = O(Eτ b. S ). The couling time τ b is of fundamental imortance for the efficiency of the samling algorithm. To imrove its comlexity, we could reduce the factor S and reduce the couling time. When the state sace is artially ordered by a artial order and the transition function is monotone for each event e, it is sufficient to simulate trajectories starting from the maximal and minimal states [8]. Denote by MAX and MIN the set of maximal, resectively minimal elements of S for the artial order. The monotone version of algorithm (1) is given by algorithm (2). In this case, we need to store the seuence of events in order to reserve the coherence between the trajectories driven from MAX MIN.

5 Bounds for the Couling Time 5 Algorithm 2 Backward-couling simulation (monotone version) n=1; R[n]=Random event;{array R stores the backward seuence of events } reeat n=2.n; for all s MAX MIIN do y(s) s {Initialize all trajectories at time n} end for for i=n downto n/2+1 do R[i]=Random event; {generates all events from time n + 1 to n 2 + 1} end for for i=n downto 1 do for all s MAX MIN do y(s) φ(y(s), R[i]) {y(s) is the state at time i of the trajectory starting in s at time n} end for end for until All y(s) are eual return y(s) The doubling scheme (first ste in the loo) leads to a mean comlexity c φ = O(Eτ b.( MAX + MIN )). (2.4) 3. Oen Markovian ueueing networks. Consider an oen network Q consisting of K ueues Q 1,..., Q K. Each ueue Q i has a finite caacity, denoted by C i, i = 1,... K. Thus the state sace of a single ueue Q i is S i = {,... C i }. Hence, the state sace S of the network is S = S 1 S K. The state of the system is described by a vector s = (s 1,..., s K ) with s i the number of customers in ueue Q i. The state sace is artially ordered by the comonent-wise ordering and there are a maximal state MAX when all ueues are full and a minimal state when all ueues are emty. The network evolves in time due to exogenous customer arrivals from outside of the network and to service comletions of customers. After finishing his service at a server, a customer is either directed to another ueue by a certain routing olicy or leaves the network. A routing olicy determines to which ueue a customer will go, taking into account the global state of the system. Moreover, the routing olicy also decides what haens with a customer if he is directed to a ueue with a buffer filled with C i customers. An event in this network is characterized by the movements of some clients between ueues, modeling the routing strategy and the Poisson rocess defines the occurrence rate of the events. For examle, consider the acyclic ueueing network of figure 3.1, made of 4 ueues and 6 events. Since the network is oen, clients are able to enter and leave the network. We assume that customers who enter from outside the network to a given ueue arrive according to a Poisson rocess. Furthermore, suose that the service times at server i are indeendent and exonentially distributed with arameter µ i.

6 6 J. Doer and B. Gaujal and J-M. Vincent C 1 λ 3 λ C λ 1 C 3 λ 2 C rate origin destination enabling condition routing olicy e λ Q 1 Q none rejection if Q is full e 1 λ 1 Q Q 1 s > rejection if Q 1 is full e 2 λ 2 Q Q 2 s > rejection if Q 2 is full e 3 λ 3 Q 1 Q 3 s 1 > rejection if Q 3 is full e 4 λ 4 Q 2 Q 3 s 2 > rejection if Q 3 is full e 5 λ 5 Q 3 Q 1 s 3 > none Fig Network with rejection 2 λ 4 λ 5 For examle, for event e 1 (fig 3.1) we get (s 1, s 1 + 1, s 2, s 3 ) if s 1 and s 1 < C 1 ; φ(., e 1 ) : (s, s 1, s 2, s 3 ) (s 1, s 1, s 2, s 3 ) if s 1 and s 1 = C 1 (Q 1 full); (s, s 1, s 2, s 3 ) if s = (Q emty). Definition 3.1. An event e is monotone if φ(x, e) φ(y, e) for every x, y in S with x y. It should be clear that event e 1 is monotone. Moreover usual events such as routing with overflow and rejection, routing with blocking and restart, routing with a index olicy rule (eg Join the shortest ueue) are monotone events [5, 12]. Denote by E = {e 1,..., e M } the finite collection of events of the network. With each event e i is associated a Poisson rocess with arameter λ i. If an event occurs which does not satisfy the enabling condition the state of the system is unchanged. To comlete the construction of the discrete-time Markov chain, the system is uniformized by a Poisson rocess with rate Λ = M i=1 λ i. Hence, one can see this Poisson rocess as a clock which determines when an event transition takes lace. To choose which secific transition actually takes lace, the collection E of events of the network is randomly samled with i = P (event e i occurs) = λ i Λ. By construction, the following roosition should be clear. Proosition 3.2. The uniformized Markov chain has the same stationary distribution as the ueueing network, and so does the embedded discrete time Markov chain. Provided that events are monotone, the CFTP algorithm can be alied on ueueing networks to build steady-state samling of the network. In our examle of Figure 3.1 we ran the CFTP algorithm and roduced samles of couling time. The arameters used for the simulation are the following. Queues caacity : for all i = 1,..., 4, C i = 1. Event rates: λ 1 = 1.4, λ 2 =.6, λ 3 =.8, λ 4 =.5 and λ 5 =.4. The global inut rate λ is varying. The number of samles used to estimate the mean couling time is 1. The result is dislayed in Figure 3.2.

7 Bounds for the Couling Time 7 τ λ Fig The mean couling time for the acyclic network of Figure 3.1 varies from 16 to 4 events when the inut rate ranges from to 4, with 95% confidence intervals. This tye of curve is of fundamental imortance because the couling time corresonds to the simulation duration and is involved in the simulation strategy (long run versus relication). These first results can be surrising because they exhibit a strong deendence on arameters values. The aim of this aer is now to understand more deely what are the critical values for the network and to build bounds on the couling time that are non-trivial. Let N i be the function from S to S i with N i (s 1,..., s K ) = s i. So N i is the number of customers in ueue Q i. As in section 2, τ b refers to the backward couling time of the chain, which is in case the couling time from the ast of the ueueing network. Definition 3.3. Let τi b denote the backward couling time on coordinate i of the state sace. Thus τi b is the smallest n for which ( ) {N i φ (n) (s, e n+1,..., e ), s S} = 1. Because coordinate s i refers to ueue Q i, the random variable τi b reresents the couling time from the ast of ueue Q i. Once all ueues in the network have couled, the CFTP algorithm returns one value and hence the chain has couled. Thus τ b = max 1 i K {τ b i } st K τi b. (3.1) By taking exectation and interchanging sum and exectation we get: E [ τ b] [ ] [ K ] = E max {τ K i b } E τi b = E [ τ b ] i 1 i K i=1 i=1 i=1 (3.2) It follows from Proosition 2.2 that τ b and τ f have the same distribution. The same holds for τ f i and τi b. Hence E [ ] [ ] τi b = E τ f i and Eτ b K i=1 [ ] E τ f i. (3.3) [ ] The bound given in Euation 3.3 is interesting because E τ f i is sometimes amenable to exlicit comutations, as shown in following sections. In order to de-

8 8 J. Doer and B. Gaujal and J-M. Vincent rive those bounds, one may rovide yet other bounds, by making the couling state exlicit. Definition 3.4. The hitting time h j k in a Markov chain X n is defined as h j k = inf N {n s.t. X n = k X = j} with j, k S. In the ueueing framework, h Ci reresents the number of stes it takes for ueue Q i to go from state to state C i. Now we consider ueue Q i out of the network and examine it indeendently. Suose that h Ci = n for the seuence of events e 1,... e n. Because of monotonicity of φ we have φ (n) (, e 1,..., e n ) φ (n) (s, e 1,..., e n ) φ (n) (C i, e 1,..., e n ) =, with s S i. Hence, couling has occurred. So h Ci is an uer bound on the forward couling of ueue Q i. The same argumentation holds for h Ci. Thus Hence, Eτ b K i=1 [ ] E τ f i by Jensen s Ineuality. i=1 [ ] E τ f i E [min{h Ci, h Ci }]. (3.4) K K E [min{h Ci, h Ci }] min(eh Ci, Eh Ci ), (3.5) 4. Couling time in a M/M/1/C ueue. The M/M/1/C ueue is well known and there is no need to run simulations to get the distribution of its stationary distribution. However, the comutation of hitting times rovided here is new and will serve as a building block for the following section on networks. In a M/M/1/C model, we have a single ueue with one server. Customers arrive at the ueue according to a Poisson rocess with rate λ and the service time is distributed according to an exonential distribution with arameter µ. In the ueue there is only lace for C customers. So the state sace S = {,..., C}. If a customer arrives when there are already C customers in the ueue, he immediately leaves without entering the ueue. After uniformization, we get a discrete time Markov chain which is governed by the events e a with robability = λ λ+µ and e d with robability = 1. Event e a reresents an arrival and event e d reresents an end of service with dearture of the customer. In order to estimate the exectation of the couling time from the ast E[τ b ] we use ineuality 3.5. Since there is only one ueue, the first two ineualities in 3.5 become eualities. Indeed, when alying forward simulation, the chain only can coule in state or state C. This follows since for r, s S with < r < s < C we have φ (r, e a ) = r + 1 < s + 1 = φ (s, e a ) and φ (r, e d ) = r 1 < s 1 = φ (s, e d ) So the chain cannot coule in a state s with < s < C. Furthermore we have φ(c, e a ) = C = φ(c 1, e a ) and φ(, e d ) = = φ(1, e d ). Hence, forward couling can only occur in or C: i=1 E [ τ b] = E [min{h C, h C }]. (4.1)

9 Bounds for the Couling Time 9,C Level 2,C 1 1,C Level 3,C 2 1,C 1 2,C Level 4,C 3 1,C 2 2,C 1 3,C Level 5,1 1,2 C 2,C 1 C 1,C Level C+1, C,C Level C+2 Fig Markov chain X() corresonding to H i,j 4.1. Exlicit calculation of E [ τ b]. In order to comute min{h C, h C } we have to run two coies of the Markov chain for a M/M/1/C ueue simultaneously.(whose states are x and y resectively). One coy starts in state and the other one starts in state C. We sto when either the chain starting in reaches state C or when the coy starting in state C reaches state. Therefore, let us rather consider a roduct Markov chain X() with state sace S S = {(x, y), x =,..., C, y =,..., C}. The Markov chain X() is also governed by the two events e a and e d and the function φ is: ψ ((x, y), e a ) = ((x + 1) C, (y + 1) C) ψ ((x, y), e d ) = ((x 1), (y 1) ). Without any loss of generality, we may assume that x y. This system corresonds with the yramid Markov chain X() dislayed in Figure 4.1. The rest of this section is devoted to establishing a formula for the exected exit time of the yramid. Although the techniue used here (one ste analysis) is rather classical, it is interesting to notice how this is related to random walks on the line (this also exlains the shifted indexes associated to the levels of the yramid). Since we can only coule in or C, this couling occurs as soon as the chain X() reaches states (, ) or (C, C). Define H i,j := number of stes to reach state (, ) or (C, C) starting from state (i, j) with (i, j) S S. By definition, min{h C, h C } = H,C. Now H i,j reresents the hitting time of the couling states (, ) and (C, C) (also called absortion time) in a roduct Markov chain. Using a one ste analysis, we get the following system of euations for E[H i,j ]: { E[Hi,j ] = 1 + E[H (i+1) C,(j+1) C ] + E[H (i 1),(j 1) ], i j, E[H i,j ] =, i = j (4.2) Two states (i, j) and (i, j ) are said to be at the same level if j i = j i. In Figure 4.1 we can distinguish C + 1 levels. Because of monotonicity of φ, j i

10 1 J. Doer and B. Gaujal and J-M. Vincent cannot increase. Hence, starting at a level with j i, the chain will gradually ass all intermediate levels to reach finally the level with j i = in state (, ) or (C, C). Thus, starting in state (, C), the chain will run through all levels to end u at the level with j i =. So, H,C = min{h C, h C }. To determine E[H,C ] we determine the mean time sent on each level and sum u over all levels. A state (i, j) belongs to level m if j i = C + 2 m. Then state (, C) belongs to level 2 and the states (, ) and C, C) belong to level C + 2. To get from (, C) into either (, ) or (C, C), the chain X() needs to cross all levels between the levels 2 and C + 2. Let T m denote time it takes to reach level m + 1, starting in level m. Then H,C = C+1 m=2 T m. (4.3) In order to determine T m, we associate to each level m a random walk R m on,..., m with absorbing barriers at state and state C. In the random walk, the robability of going u is and of going down is = 1. We have the following corresondence between the states of the random walk R m and the states of X() (see Figure 4.2). Level n,c n+2 1,C n+3 n 3,C 1 n 2,C,C n+1 n 1,C Corresonding random walk 1 2 n 2 n 1 n Fig Relationshi between level m and random walk R m. State of R m corresonds with state (, C m + 1) of X(), State i of R m corresonds with state (i 1, C m i) of the X(), 1 i m 1, State m of R m corresonds with state (m 1, C) of X(). Now the time sent on level m in X() is the same as the time sent in a random walk R m before absortion. Therefore, on can use the two following results on random walks in determining T m, which are known as ruin roblems (see for examle [9]). Let α m i denote the robability of absortion in state of the random walk R m starting in i. Then: αi m = a m a i a m 1, 1 2, m i m, = 1 2, (4.4) where a = /. Now, absortion occurs in R m once the state or C has been achieved. Lemma 4.1. Let T m i denote the mean absortion time of a random walk R m

11 Bounds for the Couling Time 11 starting in i. Then: i m(1 α E[ T m i ) i m, 1 2, ] = (4.5) i(m i), = 1 2. Now, let β m m). Then (res. β m m) denote the robability that absortion occurs in (res. β m = m αi P m (R m starts in state i), (4.6) i= and β m m = 1 β m. From the structure of the Markov chain X() and the corresondence between X() and the random walks, we have that (see Figure 4.2): P (enter level m + 1 at (, C m + 1)) = P (absortion in in R m ) = β m. Now one has: E [T m ] = E[ T 1 m ]β m 1 + E[ T m 1]β m m 1 m 1 = E[ T ( m 1] m + E[ T 1 m ] E[ T ) m 1] m β m 1. (4.7) Case = = 1/2. E[T m ] can be calculated exlicitly for = 1 2. Since the random walk is symmetric, we have β m = β m n = 1 2. So: Hence, E [T m ] = E[ T 1 m ]β m 1 + E[ T m 1]β m m 1 m 1 = m 1. (4.8) E [H,C ] = C+1 m=2 E [T m ] = C+1 m=2 m 1 = C2 + C. 2 Lemma 4.2. For a M/M/1/C with λ = µ, Eτ b = C2 +C Case 1 2. Since the random walks are not symmetric, one cannot aly the same reasoning as for the case = 1 2 to comute βm. Entering the random walk R m corresonds to entering level m in X(). Since we can only enter level m in the state (, C m + 2) or (m 2, C) this means we can only start the random walk in state 1 or m 1. Therefore (4.6) becomes: β m = m αi P m (R m starts in state i) i= = α m 1 P (R m starts in 1) + α m m 1 P (R m starts in m 1) = am a m 1 a m 1 + am 1 a a m 1 βm 1. This gives the recurrence: { β m = am a m 1 β 2 = 2. a m 1 + am 1 a a m 1 βm 1 m > 2; (4.9)

12 12 J. Doer and B. Gaujal and J-M. Vincent Thus we obtain, Proosition 4.3. For a M/M/1/C ueue, using the foregoing notations, Eτ b = E [H,C ] = C+1 m=2 E[ T ( m 1] m + E[ T 1 m ] E[ T ) m 1] m β m 1, (4.1) with β m defined by (4.9) and E[ T m m 1] and E[ T m 1 ] defined by (4.5) Comarison between the cases = 1/2 and 1/2. Proosition 4.4. The couling time in a M/M/1/C ueue is maximal when the inut rate λ and the service rate µ are eual. Proof. By definition, λ = µ corresonds to = = 1/2. The roof holds by induction on C. The result is obviously true when C =, because whatever, E [H,C ] =. For C + 1, let be an arbitrary robability with > 1/2 (the case < 1/2 is symmetric). We will comare the exected time for absortion of three Markov chains. The first one is the Markov chain X := X(1/2) dislayed in Figure 4.1, with = = 1/2. The second one is the Markov chain X = X() dislayed in Figure 4.1 and the last one X is a mixture between the two revious chains: The first C levels are the same as in X while the last level (C + 1) is the same as in X. The exected absortion time for the first C levels is the same for X and for X C : m=2 ET m = C m=2 ET m. By induction, this is larger than for X : we have C m=2 ET m = C m=2 ET m C m=2 ET m. Therefore, we just need to comare the exected exit times out of the last level, namely ET C+1, ET C+1 and ET C+1, to finish the roof. Let us first comare ET C+1 and ET C+1. In both cases, the Markov chain enters level C + 1 in state (, 1) with robability 1/2. Euation (4.8) says that ET C+1 = C and Euation (4.5) gives after straightforward comutations, ET C+1 = 1/2 1 C(1 αc 1 ) + 1/2 C 1 C(1 αc C 1 ) = C a C a 2 a C 1 C/(2) < C = ET C+1. In order to comare ET C+1 and ET C+1, let us first show that βm is larger than 1/2, for all m 2. This is done by an immediate induction on Euation (4.9). If β m 1 1/2, then βo m 2am a m 1 a 2(a m 1) Now, 2am a m 1 a 2(a m 1) 1/2 if 2a m a m 1 a a m 1, i.e. after recombining the terms, (a 1)(a m 1 1). This is true as soon as 1/2. To end the roof, it is enough to notice that for the chain X, time to absortion m m starting in 1, E T 1 is smaller that time to absortion starting in m 1, E T m 1 for m m all m. The difference E T m 1 E T 1 is ma m ma m 1 + ma m 2a m + 2 (a m 1) (a 1) by convexity of x a x. Finally, = ( ) a m(a 1) m a+ +am 1 m (a m, 1) (a 1) ET C+1 = β C+1 C+1 E T 1 + (1 β C+1 C+1 )E T C 1 C+1 E T C+1 E T C = ET 2 2 C+1.

13 Bounds for the Couling Time Exlicit Bounds. Euation (4.1) rovides a uick way to comute E [H,C ] using recurrence euation (4.9). However, it may also be interesting to get a simle closed form for an uer bound for E [H,C ]. This can be done using the last ineuality in Euation (3.5) that gives an uer bound for E [H,C ] amenable to direct comutations. E [H,C ] = E [min{h C, h C }] min{e [h C ], E [h C ]}. (4.11) The exact calculation of E [h C ] can be done using a one ste analysis. Let F i be the time to go from state i to. Then, h C = F C and for all i >, E[F i ] = 1 + E[F (i+1) C ] + E[F i 1 ]. (4.12) With an aroach derived from [9] one can condition on the next event. Let T i denote the time to go from state i to i + 1. Then E [h C ] = C 1 i= E [T i ]. (4.13) To get an exression for T i, with < i C, we condition on the first event. Therefore let E [T i e] denote the conditional exectation of T i knowing that the next event is e. Since E[T i e a ] = 1 and E[T i e d ] = 1 + E [T i 1 ] + E [T i ], conditioning delivers the following recurrent exression for the E [T i ]: { 1 E [T i ] = + E [T i 1], < i < C 1, i =. (4.14) By induction one can show that E [T i ] = 1 i k= from (4.13) it follows that E [h C ] = C 1 i= 1 ( ) i+1 = C ( ) k. Hence, E [Ti ] = 1 ( ) i+1 and ( (1 C) ) ( ) 2. (4.15) By reasons of symmetry, we have ( ) E [h C ] = C (1 C) ( ) 2 (4.16) The curves of E [h C ] and E [h C ] intersect in C 2 + C when =. If > then E [h C ] < E [h C ] and because of symmetry, if < then E [h C ] > E [h C ]. Since also C2 +C 2 is an uer bound corresonding to the critical case = on the mean couling time Eτ b, as shown in Proosition 4.4, one can state: Proosition 4.5. The mean couling time Eτ b of a M/M/1/C ueue with arrival rate λ and service rate µ is bounded using = λ/(λ + u) and = 1. Critical bound: [, 1], Eτ b C2 +C 2. Heavy traffic Bound: if > 1 2, Eτ b C (1 ( ) C ) ( ) 2. Light traffic bound: if < 1 2, Eτ b C (1 ( ) C ) ( ) 2.

14 14 J. Doer and B. Gaujal and J-M. Vincent Light traffic bound C + C 2 heavy traffic bound C+C Eτ b Fig Exected couling time in an M/M/1/1 ueue when varies from to 1 and the three exlicit bounds given in Proosition 4.5 Figure 4.3 dislays both the exact exected couling time for a ueue with caacity 1 as given by Euation (4.1) as well as the three exlicit bounds given in Proosition 4.5. Note that the bounds are very accurate under light or heavy traffic (.4 and >.6). In any case, the ratio is never larger than Couling in acyclic ueueing networks. This section is dedicated to the effective comutation of a bound of the couling time in acyclic networks. In the acyclic network given in Figure 3.1, the couling time has a eak at λ =.4, as one can see in Figure 3.2. This corresonds to the case when the inut rate and service rate in Queue 3 are eual. This should not be surrising regarding the result for a single ueue, which says that the couling time is maximal when the rates are eual. Then a second eak occurs around λ = 1.4 when couling in Queue is maximal. The rest of the curve shows a linear increase of the couling time which may suggest an asymtotic linear deendence in λ. In this art, an exlicit bound on the couling time which exhibits these two features will be derived. The first result concerns an extension of ineuality (3.5) to distributions. The second art shows how the results for a single M/M/1/C ueue can be used to get an effective comutation of bounds for acyclic networks on ueues. In the following, the ueues Q,... Q K are numbered according to the toological order of the network. Thus, no event occurring in ueue Q i has any influence on the state of ueue Q j as long as i > j Comutation of an uer bound on the couling time. Here, an acyclic network of./m/1/c ueues with an arbitrary toology and Bernoulli routings is considered. The events here are of only two tyes: exogenous arrivals (Poisson with rate γ i in ueue i) and routing of one customer from ueue i to ueue j after service comletion in ueue i (with rate µ ij ). Queue K + 1 is a dummy ueue reresenting exits: routing a customer to ueue K + 1 means that the customer exits the network forever. In case of overflow, the new customer trying to enter the full ueue is lost. The service rate at ueue i is also denoted µ i = K+1 i= µ ij. Let us introduce new random variables. τ b (s j = x) is the backward couling time of the network, over the set of all intial states with the j-th coordinate eual to x.

15 Bounds for the Couling Time 15 Namely, τ b (s j = x) = min { n s.t. φ (n) (S {s j = x}, e n+1,..., e ) = 1 }. τi b(s j = x) is the backward couling time on coordinate i given s j = x: τi b (s j = x) = min { n s.t. N i (φ (n) } (S {s j = x}, e n+1,..., e )) = 1. It should be obvious that τ b (s j = x) st τ b and for all i, τi b(s j = x) st τi b. We also have the same notions for forward couling times: τ f (s j = x) = min { n N; s.t. φ (n) (S {s j = x}, e 1 n ) = 1 }, τ f i (s j = x) being defined in the same manner, and for hitting times: h Ci (s j = x) = min{n N; s. t. φ (n) (S {s i = C i, s j = x}, e 1 n ) S {s i = }}. Now, sweeing the list of ueues in the toological order, one can construct a seuence of backward simulations in the following way. First simulate the ueueing system from the ast u to couling of ueue. The number of stes is by definition τ. b Queue Q has couled in a random state X. Then, run a second backward simulation u to couling of Queue Q 1 given s = X. This simulation takes τi b(s = X ) stes and the state at time t = is X1 1 for Q 1 and X 1 for Q. This construction goes on u to the backward simulation u to couling of Queue Q K given s = X K 1, s 1 = X1 K 1,..., s K 1 = X K 1 K 1. The last simulation takes τi b(s = X K 1, s 1 = X1 K 1,..., s K 1 = X K 1 K 1 ) stes and the couling state of Q K is XK K. Lemma 5.1. One has τ b K st i= τ i b(s = X i 1,..., s i 1 = X i 1 i 1 ), and for all i, (X, i..., Xi i) is steady state distributed for Q,..., Q i. Furthermore, for all i, τ b st K i= h Ci (s = X i 1,..., s i 1 = X i 1 ). (5.1) i 1 Proof. From the revious seuence of backward simulations one can construct a single one by aending them in the reverse order: the backward simulation for Queue Q K receded by the simulation of Q K 1, and so forth u to the simulation of Q. This is a backward simulation of the system (the last state is (X K,..., Xi K )). This construction is illustrated in the case of two ueues in tandem in Figure 5.1. A straightforward conseuence, using acyclicity, is that (X, i..., Xi i ) is steady state distributed for Q,..., Q i for all i. Furthermore, one gets in distribution τ b st st K i= K i= τ b i (s = X i 1,..., s i 1 = X i 1 i 1 ) = K i= h Ci (s = X i 1,..., s i 1 = X i 1 i 1 ), τ f i (s = X i 1,..., s i 1 = X i 1 i 1 )

16 16 J. Doer and B. Gaujal and J-M. Vincent C C 1 X 1 1 X X 1 τ b τ b 1 (s = X ) τ b 1 (s = X ) Time Fig The trajectories of the state in Q are in black while the the trajectories for Q 1 are in the lighter color. Starting at time τ b τ 1 b(s = X ), the state of Q has couled in X at time τ1 b(s = X ). From then on, Q stays couled and Q 1 coules at some time before. by indeendence of the variables, given the initial states X i 1. Let us now consider a new circuit with one difference from the original one: all ueues are relaced by infinite ueues, excet for ueue Q i which stays the same. In the following, all the notations related to this new network will be exressed by aending the symbol to all variables corresonding to this new circuit. The new circuit u to Queue i is roduct form and using Burke s Theorem, the inut stream in Queue i is Poisson. The rate of the inut stream in ueue i is given by l i, the solution of the flow euations: l i = j<i l j u ji µ j + γ i. The network is said to be stable for Queue i as soon as l i < µ i. We assume stability for all i in the following. One can construct a seuence of backward simulations for the new network in the same way as for the original network. This rovides the uantities X i 1 j, τi b(s = X i 1,..., s i 1 = X i 1 i 1 ), τ f i (s = X i 1,..., s i 1 = X i 1 i 1 ), and h Ci (s = X i 1,..., s i 1 = X i 1 i 1 ). The monotony roerty given above imlies that X j i st X j i and h Ci (s = X i 1,..., s i 1 = X i 1 i 1 ) st h Ci (s = X i 1,..., s i 1 = X i 1 i 1 ). The next ste is to build yet another model. This third model is made of a single M/M/1/C i ueue with three tyes of events, arrivals of customers with rate l i (rovided that the number of customers is smaller than C i ), deartures with rate µ i (rovided that the number of customers is ositive) and null events (with no effect on the ueue) with rate Λ l i µ i. For this isolated model, let us introduce the uniformizing robabilities = l i /Λ, = 1 and d = (Λ l i µ i )/Λ. Let F k be the time to go from state k to state

17 in the isolated system. A one ste analysis gives now 1 1 d factor of 1 1 d = Bounds for the Couling Time 17 E[F k ] = 1 + de[f k ] + l i Λ E[F (k+1) C i ] + µ i Λ E[F (k 1)] = 1 1 d + E[F (k+1) C i ] + E[F (k 1) ]. We get the same euation as (4.14) excet for the additional constant which is instead of 1, so that the solution is the same as before u to a multilicative Λ l i+µ i. Using Euation (4.16), one gets E[F Ci ] = Λ l i + µ i ( (1 ( ) 2 C i ) Ci). (5.2) Lemma 5.2. Under the foregoing notations and assumtions, h Ci (s = X i 1,..., s i 1 = X i 1 i 1 ) = F C i, in distribution. Proof. First, using Lemma 5.1 for the new network with infinite ueues (excet for Q i ), the state ( X i 1,..., X i 1 i 1 ) is steady state distributed. Using Burke s Theorem, this imlies that the inut stream in ueue Q i is Poisson with rate l i, when one runs a simulation starting in any state in S {s i = C i, s j = X i 1 j, j < i}. Now, during this simulation, one can coule the addition, subtraction et null events for ueue Q i in isolation and for Q i in the comlete network of infinite ueues, all of them having the same laws. This imlies that the state of ueue Q i in both systems is the same under that couling. Hence, they reach at the same time: h Ci (s = X i 1,..., s i 1 = X i 1 i 1 ) = F C i in distribution. We are ready to ut everything together in exectation. Eτ b st E[h Ci (s = X i 1,..., s i 1 = X i 1 )] (5.3) i i i 1 E[ h Ci (s = X i 1,..., s i 1 = X i 1 )] (5.4) i 1 i E[F Ci ]. (5.5) The seuence of ineualities may not hold in distribution because the variables X i and thus h Ci (s = X i 1,..., s i 1 = X i 1 i 1 ) are not indeendent. Using (5.2), Eτ b i ( ) Λ C (1 Ci) i l i + µ i ( ) 2. The result of this art is summarized in the following theorem. Theorem 5.3. In an acyclic stable network of K + 1./M/1/C i ueues with Bernoulli routing and losses in case of overflow, the couling time from the ast sat-

18 18 J. Doer and B. Gaujal and J-M. Vincent isfies in exectaction, ( ) K E[τ b Λ ] C (1 Ci) i l i + µ i K Λ ( ) 2 (C i + Ci 2 ). (5.6) l i + µ i i= i= Note that this bound on the exectation is ultimately linear in the rate of any event in the system. This behavior is also noticeable for E[τ b ] itself Some numerical exeriments. In the construction of the bound given in Theorem 5.3, several factors may be resonsible for the inaccuracy of the bound. 1. The first factor is the relacement of the max by the sum. We believe that it may be a hard task to get rid of this first aroximation because of the intricate deendencies between the ueues. Furthermore, exeriments reorted below show that this may not even be ossible in many cases (see Figure 5.2.b). 2. Another factor which may increase the inaccuracy of our bounds is the fact that most events change the states of several ueues at the same time, while the bound given here disregards this. In the network studied here, this may add a factor 2 between the true couling time and the bound given in Theorem The most imortant factor which jeoardizes the uality of the bound is the load issue. If one ueues has an heavy load (larger than 1), the bound rovided by Euation (5.6), also called the light traffic bound in Proosition 4.5 is very bad (as seen in Figure 4.3). So far we have not been able to come u with a better bound for ueues with large loads. However, when all ueues have a small load (smaller than one, and even more so when the load is smaller than 2/3), the bound tends to be more accurate. This is further verified in the exeriments reorted below. Comutations for the network dislayed in Figure 3.1 are reorted in Figure 5.2. We have used the following arameters. The inut rate is λ =.4. the rates of the other events are λ 1 = 1.4, λ 2 =.6, λ 3 =.8, λ 4 =.5. The number of simulation runs is 1. The caacity C is the same in all ueues, and we let it vary from 1 to 2. The service rate in the last ueue λ 5 takes three values, resectively.2,.6 and.4. In the first case (Fig. 5.2.a), λ 5 =.2 so that ueue Q 3 has a load larger than one. Figure 5.2 dislays the bound given by formula (5.6) as well as the mean couling time comuted over 1 simulation runs. As hinted before, the bound is indeed very bad for this system. A ratio larger than 1 w.r.t the true couling time is reached when C = 5. It should also be noticed that our bound is convex in C while the couling time does not seem to be so. In the second case (Fig. 5.2.b), λ 5 =.6, and all ueues have a load smaller that 2/3. Figure 5.2.b shows the bound rovided by (5.6) and the true couling time comuted by simulation runs. Both curves aear to be almost linear in C (this is true for the bound: when / is small, EH Ci, is almost linear in C i ) and the ratio is smaller than 1.3. In that case, the curve max i {,...K} EH Ci, is also dislayed and is below the actual couling time. This is to be related with the first item in the comments above. The last case (Fig. 5.2.c) is for λ =.4, so that Q 3 has load exactly one. This would corresond to the maximal couling time for Q 3 if it were alone. Figure 5.2.c dislays the backward couling time and the bound rovided by Euation (5.6). For ueue Q 3, we use a bound in C 3 + C3 2 which is a bad aroximation because of the loss of the factor 2 when comared with the bound for isolated ueues. Note that the total ga has a ratio which is almost 2. In that case both the couling time and the bound exhibit a convex behavior w.r.t. C.

19 Bounds for the Couling Time Uer bound 5 2 Max bound for each ueue C C C Fig The caacity C varies from 1 to 2 in all ueues. The uer left figure dislays the couling time (dots) with 95% confidence intervals, and the bound given by Euation (5.6) when Queue Q 3 is unstable (λ 5 = 2/1). In the uer right figure, are given the bound in Euation (5.6), the mean couling time (dots) with 95% confidence intervals and the maximum over Euations (5.6) for all ueues, when Queue Q 3 is stable (λ 5 = 6/1). The lower figure dislays the couling time (dots) with 95% confidence interval and the bound given by Euation (5.6) when Queue Q 3 is barely unstable (λ 5 = 4/1) A ratio smaller than 2 is indeed interesting because efficient erfect simulation algorithm use a doubling window techniue to reduce the comlexity and their running time (see Euation (2.4)) so that our bound gives a good estimation of the mean running time of the algorithms. One should also note that, on a ractical oint of view, most actual networks which reuire stationary erformance evaluations have small loads Extension to more general networks. Actually, extensive simulation runs over many examles show that the bound given in Theorem 5.3 is robust and also holds for more general networks with blocking and with circuits. While we have only been able to show that the light traffic bound holds for each ueue, we conjecture that the heavy traffic bound and the critical bound should also hold. This would yield an overall uadratic bound: E[τ b ] K Λ i= l i+µ i O(Ci 2 ), for any monotone Markovian network of ueues with a finite state sace. Furthermore under light or heavy traffic in all ueues, the bound should rather be linear: E[τ b ] K Λ i= l i+µ i O(C i ). To illustrate this conjecture, we have run simulations for the network dislayed in Figure 3.1 with the following arameters. The rates are λ =.4, λ 1 = 1.4, λ 2 =.6, λ 3 =.8, λ 4 =.5. The caacity is fixed to 1 in all ueues and we let λ 5 (the service rate in Q 3 ) vary from to 4. As long as λ 5 <.4, Q 3 is unstable and our roven bound (B 1 ) is oor. As soon as λ 5 is large enough our bound becomes accetable. In Figure 5.3, note that both the bound and the couling time τ have a linear asymtotic growth in λ 5. The Figure also dislays the heavy traffic bound B 2 and the critical bound B 3. Should these two bounds hold, the minimum of B 1, B 2, B 3 (in bold in the figure) would rovide a remarkable bound on the couling time, u to an additional constant. This issue is the subject of our current investigations.

20 2 J. Doer and B. Gaujal and J-M. Vincent B 2 (conjecture) B 1 (roven) B 3 (conjecture) B 1 B 2 B 3 Eτ b λ 5 Fig This figure dislays the actual couling time Eτ b for the network of Figure 3.1, when the service rate of the last ueue ranges from to 5, together with the roven light traffic bound B 1, the conjectured heavy traffic bound B 2, the conjectured critical bound B 3 and the minimum of the three bounds. Acknowledgments. The authors would like to thank Jérôme Vienne who artially designed the si2 erfect simulation software which was used to run all the simulations resented here. REFERENCES [1] C. Alexooulos and D. Goldsman, To batch or not to batch?, ACM Trans. Model. Comut. Simul., 14 (24), [2] J.M. Calvin, P.W. Glynn, and M.K. Nakayama, The semi-regenerative method of simulation outut analysis. submitted. [3], Imortance samling using the semi-regenerative method, in Proceedings of the Winter Simulation Conference, vol. 1, 21, [4] M. Crane and D.L. Iglehart, Simulating stable stochastic systems, iii: Regenerative rocesses and discrete-event simulation, Oeration Research, 23 (1975), [5] P. Glasserman and D.D. Yao, Monotone Structure in Discrete-Event Systems, Wiley Inter- Science, Series in Probability and Mathematical Statistics, [6] O. Häggström, Finite Markov Chains and Algorithmic Alications, Cambridge University Press, 22. [7] S.G. Henderson and P.W. Glynn, Regenerative steady-state simulation of discrete-event systems, ACM Trans. Model. Comut. Simul., 11 (21), [8] J. Pro and D. Wilson, Exact samling with couled Markov chains and alications to statistical mechanics, Random Structures and Algorithms, 9 (1996), [9] S. M. Ross, Probability models, Academic Press, 23. [1] W.J. Stewart, Introduction to the Numerical Solution of Markov Chains, Princeton, [11] J.-M. Vincent, Perfect simulation of monotone systems for rare event robability estimation, in Winter Simulation Conference, Orlando, dec 25. [12], Perfect simulation of ueueing networks with blocking and rejection, in Saint IEEE conference, Trento, 25, [13] J.-M. Vincent and C. Marchand, On the exact simulation of functionals of stationary markov chains, Linear Algebra and its Alications, 386 (24),