On Positive Harris Recurrence of Multiclass Queueing Networks: A Unified Approach Via Fluid Limit Models

Transcription

1 On Positive Harris Recurrence of Multiclass Queueing Networks: A Unified Approach Via Fluid Limit Models J. G. Dai School of Industrial and Systems Engineering and School of Mathematics Georgia Institute of Technology Atlanta, Georgia dai@isye.gatech.edu Abstract It is now known that the usual traffic condition (the nominal load being less than one at each station) is not sufficient for stability for a multiclass open queueing network. Although there has been some progress in establishing the stability conditions for a multiclass network, there is no unified approach to this problem. In this paper, we prove that a queueing network is positive Harris recurrent if the corresponding fluid it model eventually reaches zero and stays there regardless of the initial system configuration. As an application of the result, we prove that single class networks, multiclass feedforward networks and first-buffer-first-served preemptive resume discipline in a re-entrant line are positive Harris recurrent under the usual traffic condition. AMS 99 subject classification: Primary 6K25, 9B22; Secondary 6K2, 9B35. Key words and phrases: multiclass queueing networks, Harris positive recurrent, stability, fluid approimation Running title: Stability of multiclass networks First drafted in September 993 Revised on May 24, 994 Annals of Applied Probability, Vol. 5, (995) Research supported in part by NSF Grants DMS , DDM , and two grants from the Teas Instruments Corporation.

2 Introduction d. The We consider a network composed of d single server stations, which we inde by i network is populated by K classes of customers, and each class k has its own eogenous arrival process with interarrival times ξ k n n. We allow ξ k n for all n for some class k. In this case, the eternal arrival process for class k is null. We let denote the set of classes with non-null eogenous arrivals. Hereafter, whenever eternal arrival processes are under discussion, only classes with non-null eogenous arrivals are considered. Class k customers require service at station s k, and their service times are η k n n. Upon completion of service at station s k, a class k customer becomes a customer of class l with probability P kl and eits the network with probability l P kl, independent of all previous history. To be more precise about the last statement, let φ k n be the routing vector for the nth class k customer who finishes service at station s k. The lth component of φ k n is one if this customer becomes a class l customer and zero otherwise. Therefore, φ k n is a K-dimensional Bernoulli random variable with parameter Pk, where P k denotes the kth row of P P kl and primes denote transpose. (All vectors are envisioned as column vectors.) We assume that φ k φ k n n is iid, and φ φ K are independent and are independent of the arrival processes and service processes. Such a routing mechanism is often called Bernoulli routing. The transition matri P P kl is taken to be transient. That is, (.) I P P 2 is convergent. Condition (.) implies that the epected number of visits to class l by a class k customer is finite. Therefore, all customers eventually leave the network. Hence the networks we are considering are open queueing networks. We assume that the waiting buffer at each station has infinite capacity. To avoid trivial complications, we assume that no two events can happen simultaneously. Hereafter, we will refer to such a network as a multiclass open queueing network, or simply a multiclass network. If there is only one class, say class, that has non-null eogenous arrivals and the entries of the routing matri take the form P k k for k K and zero for all others, then the multiclass network is called a re-entrant line; see Kumar [32]. An eample of such network is given at the end of this section. So far, the sequences of interarrival times and service times are quite general. We need to put some assumptions on them. Throughout this paper, we make the following three assumptions on the network. We first assume that (.2) are iid sequences and they are mutually independent ξ ξ K η η K This independence assumption may be relaed for some networks. See the remark following Proposition 2.. Net, we put some moment assumptions on interarrival and service times. We assume that (.3) η k for k K and ξ k for k.

3 Finally, we assume that interarrival times are unbounded and spread out. That is, for each k, there eist some integer j k and some function p k on R with p k d, such that! (.4) ξ k for any, and (.5) Let C i incidence matri, k : s k!" a # j k b ξ k i %# b&' ( p i$ a k d for any # a b i. The set C i is called the constituency for station i. Let C be a d ) K " if C s k i, (.6) ik otherwise. + Let α k * ξ k and m k η k be the arrival rate and mean service time for class k customers, respectively. In light of assumption (.), I P, eists and Put λ (.7) I P, I P P 2 - I P., α. One interprets λ k as the effective arrival rate to class k. For each i d let ρ i λ k m k k/ C i We call ρ i the nominal workload for server i per unit of time. In vector form, we have ρ CMλ, where (.8) M diag m m K The main objective of this paper is to establish a sufficient condition for the stability of a multiclass open queueing network under variety of queueing disciplines. A queueing discipline at station i dictates which customer will be served net when server i completes a service. We say that a queueing discipline for a multiclass network is stable if the underlying Markov process describing the network dynamics is positive Harris recurrent, whose precise definition is given in Section 3. When there is no ambiguity in the underlying queueing discipline, we say that a queueing network is stable if the queueing discipline under discussion is stable. Our main theorem (Theorem 4.2) states that a queueing network is stable if the corresponding fluid it model is stable. The latter phrase means that the fluid it model eventually reaches zero and stays there regardless of the initial system configuration. To make the theorem practical, we give a systematic treatment of fluid it models. As an application of the theorem, we prove that single class networks, multiclass feedforward networks and first-buffer-first-served (FBFS) preemptive resume discipline in a re-entrant line are positive Harris recurrent under the usual traffic condition (.9) ρ i i d or in vector form ρ e, where ρ inequalities are interpreted componentwise.) ρ ρ d and e is the d-dimensional vector of ones. (Vector 2

4 Our theorem is a generalization of recent work by Rybko and Stolyar [47], who mainly considered the stability of multiclass Markovian networks with discrete state space. Their brilliant analysis, via fluid models, lead to the stability of a two station multiclass network under the first-in-first-out (FIFO) queueing discipline. They also showed that under a certain bad priority discipline, the two-station network was not stable under (.9). However, in their paper, convergence to a fluid model was never addressed. In other words, they did not consider the kind of fluid it that we do here. Rather, they worked with the formal fluid analog of the queueing network. Therefore, they needed a parallel analysis for the original network even if they proved a certain property for the fluid model. As far as the stability of a queueing network is concerned, our theorem is attractive in that readers can focus on the fluid it itself without eplicitly going back to the original queueing network. This work was initially motivated by recent work of Dupuis and Williams [7], where positive recurrence for d-dimensional reflecting Brownian motion in an orthant was investigated. By constructing a Lyapunov function, they showed that a semimartingale reflecting Brownian motion in an orthant is positive recurrent if all solutions of the corresponding deterministic Skorohod problem for a driving noiseless drift path are stable. In a similar philosophical vein, Malyshev has obtained conditions for positive recurrence of reflected random walks in the orthant by studying related deterministic dynamical systems, see [36] and the references there. Our description of a multiclass network is now quite standard. (The class of queueing networks described here is in fact an important special case of the setup in Harrison and Nguyen [24].) In his pioneering paper on queueing networks, Jackson [26] assumed that customers visiting or occupying any given station are essentially indistinguishable from one another, and that a customer completing service at station i will move net to station j with some fied probability P i j, independent of all previous history. Thus in Jackson s networks, each station serves a single customer class, hence these networks have been called single-class networks. Jackson s model was etended by Baskett, Chandy, Muntz and Palacios [2] and Kelly [3] to the multiclass setting. In both cases, they assumed either special types of service disciplines or eponential distributions for interarrival and service times. For a long time, research on stability for open queueing networks under general distributional assumptions has mostly been restricted to single class networks or generalized Jackson networks, see Borovkov [4], Sigman [5], Meyn and Down [38], Baccelli and Foss [] and Chang, Thomas and Kiang [7]. However, recently stability conditions for multiclass networks have received a lot of attention. In the deterministic setting, Kumar and his coauthors did the pioneering work on the stability of general multiclass networks under a variety of queueing disciplines, see [44, 34, 35, 32]. In particular, they proved that FBFS and last-buffer-first-served (LBFS) are stable under (.9) in a deterministic re-entrant line. They also found that ρ i for all i is not sufficient for stability under certain priority disciplines. Rybko and Stolyar [47] found that this phenomenon still eists for stochastic networks. Even more surprising, Bramson [6] found a two station multiclass stochastic network, where FIFO discipline is not stable even if (.9) holds. Seidman [48], independently, found a deterministic eample showing a similar phenomenon. These eamples demonstrate that to obtain a general criterion in terms of basic system parameters like (.9) for stability of a multiclass network is very difficult. On the positive side, Kumar and Meyn [33] recently gave some sufficient conditions for the stability of multiclass networks with Poisson arrivals and eponential service times. Their 3

5 2 2 2 Figure : A variant of Bramson s two station multiclass network. n 5 5 eponential deterministic Table : Simulation estimates of average throughput time T n. sufficient condition, in terms of the solution to a linear program, implies, among other things, that moments of some processes associated with the network converge as time t 3. More recently, while the paper is under review, Dai and Weiss [3] and Down and Meyn [6] showed that piecewise Lyapunov functions, initiated by Botvitch and Zamyatin [5], are powerful tools in characterizing stability regions for certain priority disciplines. For readers who are not familiar with the recent literature on multiclass networks, let us consider an eample of two station multiclass network pictured in Figure. It is a re-entrant line. There is one type of customers arriving at station one from the outside according to a Poisson process with rate. An arriving customer visits stations, 2, 2, 2, 2, and eits. We call customers in the kth stage of visit class k customers. The variants of this two station model were studied by Kumar [32], Whitt [5] and Bramson [6]. Obviously, ρ m m 6 ρ 2 m 2 m 3 m 4 m 5 Assume that all service times are eponential and m 6 899, m 2 897, m m 3 m 4 m 5. Therefore the nominal loads are ρ 9 and ρ 2 9. One would hope that such a network is stable under FIFO discipline and that the long run average queue lengths are finite. Let T n be the throughput time (total time in the network) of the nth customer. Let T n n We simulated this network starting from empty for one replication, which corresponds to some particular choice of seeds for random number generators. In this replication, we let n vary from to. The average throughput times (rounded to integers) are tabulated in row eponential of Table. It seems that as n increases, T n increases almost linearly. We conjecture that n i$ T i T n 3 almost surely 4

6 as n 3. If we replace all eponential random variables by constants (their respective means), a similar phenomenon occurs, as evidenced in row deterministic of Table. Figure 2 plots the queue lengths at both stations in and units of simulation time. Clearly, the queue lengths oscillate with increasing magnitude. Most of the time, only one of the two servers is working. This mutual blocking apparently causes the instability. Readers are referred to Bramson [6] for more insight. The following notation will be used in the rest of this paper. For a finite set A, 4 A4 denotes the cardinality of A. Let R. The set of nonnegative integers is 5. D R is the space of right continuous functions on R having left its on, endowed with the Skorohod topology, see Ethier and Kurtz [8]. Let C R be the subset of continuous paths in D R. The following fact is useful in this paper. Assume that f n D R for each n and f C R. Then f n 3 f in the Skorohod topology if and only if for each t, (.) sup 4 f n 6 s6 s 7 f s 8493 as n 3 t When (.) holds for all t, we say that f n 3 f uniformly on compact sets, or simply f n 3 f u.o.c. The symbol : denotes weak convergence or convergence in distribution. 2 Preinaries 2. State Descriptions We now define a Markov process X X t t which describes the dynamics of the queueing network. If all the interarrival time and service time distributions are eponential, then queue lengths, in addition to the order of customer arrivals in FIFO discipline, are sufficient for a state description. In general, we need to augment the previous state description with the remaining eogenous interarrival times and the remaining service times. Therefore, in general, the state X t at time t is X t <;= t U t V t where ;> t captures how customers are lined up at each station, U t U k t : k?@ A RBC+B and V t V t V K t 7 R K. For k?, U k t is the remaining time before the net class k customer will arrive from outside. For k K, V k t is the remaining service time for the class k customer that is in service, which is set to be zero if Q k t. Both U t and V t are taken to be right continuous. Let D be the set of all possible states that X can take. We use 4 ;= t E4 to denote the total queue length in the network at time t. For a u R K, the norm of u is 4 u4 K k$ 4 u k 4. We define the norm 4 X t 84 of X t to be the total queue length plus the total remaining interarrival time and total remaining service time at t. That is, 4 X t 84 4 ;F t 84G'4U t 84'4V t 84- The precise state description depends on a queueing discipline. Here we give some eamples. 5

7 FIFO Discipline Under the FIFO discipline, customers at each station are served on the first-in-first-out basis. For station i define (2.) ; i t k i k i 2 k i N ih ti where k i j is the class number for the jth customer at station i and N i t is the queue length (including possibly the one being served) at station i. (If N i t, ; i t is defined to be an empty list.) Then we can take ;> t <; t ; d t At each station i with N i t, there is only one customer that can receive partial service. Thus, X t KJ ;> t U t V kil H ti t i d GM 2 N K and 5 K be the set of finitely terminated sequences taking is the state at time t. Let 5 K values in 5 K. (Zero length sequence corresponds to the situation where there are no customers at the station.) It is evident that Hence DP ; i t O5 K 5 K d ) RBC+B d We assume that D is endowed with the natural induced topology from 5 K d ) RBC+B d. Priority Disciplines Under priority disciplines, customers at each station are ranked according to their class designations. Classes to be served at station i are ranked as,, 4C i 4. The class with rank has the highest priority and the class with rank 4C i 4 has the lowest priority. When the server finishes service of a customer, the server picks a customer from a nonempty class with the highest rank. If there is no customer at the station, the server stays idle. To be concrete, within each class, customers are served in FIFO order. In the case that ties among classes are allowed, all customers with the same rank are served in FIFO order. There are two different modes for a priority discipline. In nonpreemptive mode, the server has to finish service of the current customer being served before choosing another customer to work on. In preemptive resume mode, when an arriving customer has rank higher than the one currently being served, the customer in service is preempted immediately until all customers with higher ranks are served. At that point, the server recommences service of the preempted customer until service is completed or a higher priority customer arrives. For preemptive resume priority discipline, we can take ;= t as (2.2) ;= t Q t Q t Q K t and X t Q;> t U t V t R 6

8 where Q k t is the queue length for class k customers at time t. In this case we have DSPT5 K ) RBC+B K For nonpreemptive priority discipline, at each station there is at most one customer that can receive partial service. Thus, X t KJ Q t U t k i t V kih ti t i dmf where k i t is the class number of the customer being served at station i at time t. If there are no customers at station i at time t, k i t is set to be zero and V kih ti t. In the nonpreemptive case, Head-of-Line Processor Sharing Discipline DPT5 K ) RBC+B d )U5 d K With this discipline, the lead customer in all customer classes visiting a station simultaneously receives service. Hence, we can take X t <;= t U t V t where ;= t Q t as in (2.2). The state space Processor Sharing Discipline DSPT5 K ) RBC+B K With a processor-sharing discipline, all customers visiting a station simultaneously receive service. Let Ṽ i j t be the remaining service time for the jth customer at station i at time t. (If N i t, Ṽ i j t is taken to be zero. We can envision that customers at each station are lined up according to the FIFO discipline.) The state is X t <;= t U t Ṽ t where ;= t is defined in (2.), Ṽ t Ṽ t Ṽ d t and The state space Ṽ i t KJ Ṽ i t Ṽ i N ih ti t GMF DP 5 K d ) R K )WV R YX where R is the space of all finitely terminated sequences in R. The norm of X t is still the total queue length plus the total remaining interarrival time and the total remaining service time. Remark. It turns out that the processor sharing discipline or any disciplines in which the number of customers receiving simultaneous service is unbounded will slightly alter the fluid it model discussed in Section 4. Specifically, the statement and the proof of Theorem 4. need some modifications. However, all results in Section 3 are still valid without any modification. In order to keep a clean treatment in the paper, we shall not include these queueing disciplines in the discussion. However, etensions of Theorem 4.2 to cover these disciplines should be evident. d 7

9 !! 2.2 Strong Markov Property To establish that X X t t is a strong Markov process, we follow Sections 2.3 and 2.4 of Kaspi and Mandelbaum [28]. Because of the independence assumption (.2), we can check that X is Markov for each of the disciplines discussed in Section 2.. As time t goes on, U k t and V k (t) decrease while the remainder of the state remains constant. As one of these residual processes reaches zero, a jump occurs for X. Hence X X t t is a piecewise-deterministic Markov (PDM) process that conforms to Assumption 3. of Davis [5]. It follows from Davis [5, page 362] that Proposition 2. X X t t is a strong Markov process with state space D. Remark. Condition (.2) is mainly used to establish the Markov property for X. In certain networks, this condition can be relaed for eample, for the re-entrant line presented at the end of Section, we can weaken the assumption on the service times. For that network, it is enough to assume that η n η K n n is iid. For fied n, η n,, η K n may have arbitrary dependency. This feature is useful for certain applications, notably in computer communications and manufacturing systems. There the length of a computer message or the size of a manufacturing lot may be random. However, the service times in general are proportional to the message length or lot size, and therefore are positively correlated. Let Z\[ be the Borel σ-field of D. Having verified that X is a PDM, as in Section 2.4 of Kaspi and Mandelbaum [28], we may assume at our disposal the usual elements that constitute a Markovian environment for X. Formally, it will be assumed hereafter that Ω <], ] t X t, θ t P is a Borel right process on the measurable state space D^ Z [. In particular, X X t t has right-continuous sample paths; it is defined on Ω <] and is adapted to ] t t ;! =D_ are probability measures on Ω <] such that for all `D, and f X a θ τ b4 ] τ where τ is any ] t-stopping-time, X a θ τ ω X X H τi f X on τ c X τ ω R t ω t c -a.s. and f is any real-valued bounded measurable function; (the domain of f is the space of D -valued right-continuous functions on, equipped with the Kolmogorov σ-field generated by cylinders.) Let P t A, A OZ\[, be the transition probability of X. That is P t A! X t d A A nonzero measure π on D^ Z [ is invariant for X if π is σ-finite, and π A ( P t A π d S for all A OZ\[Y An invariant measure π is said to be unique if the only invariant measures for X are positive scalar multiples of π. We denote D `D :

10 !!!! 3 Harris Positive Recurrent Chains Let τ A inf t : X t A. The process X is Harris Recurrent if there eists some σ-finite measure µ on D^ Z [, such that whenever µ A and A OZ [, τ A fe It is well known that if X is Harris recurrent then an essentially unique invariant measure π eists, see for eample Getoor [2]. If the invariant measure is finite, then it may be normalized to a probability measure; in this case X is called positive Harris recurrent. Remark. Harris recurrence implies an apparently stronger condition that g ( A X s ds hie whenever µ A for some σ-finite measure µ. See Kaspi and Mandelbaum [29] or Meyn and Tweedie [4, Theorem.]. Remark 2. Assume that X X t t is positive recurrent with the unique stationary probability distribution π. For any measurable function f on D^ Z\[+, let π f ( [ f π d whenever the integral makes sense. It was proved in Dai and Meyn [2] that positive recurrence implies the following ergodic property: for every measurable f on D with π 4 f 4j %, tk t ( t f X s ds π f -a.s. for each `D. See also Theorem 7..(i) of Meyn and Tweedie [39] and Section 2.5C of Kaspi and Mandelbaum [28]. Take f to be the number of class k customers when the state is in. Because f is nonnegative, positive Harris recurrence for X implies that each customer class k, where Q k t tk t ( t Q k s ds 3 π f -a.s. for each `D, f X t is the class k queue length at time t. Note that in general π f may be infinite. Suppose that a is a general probability distribution on R, and define the Markov transition function K a as K a A e ( P t A a dt A non-empty set A lz\[ is called µ a -petite if µ a is a non-trivial measure on D^ Z\[ and a is a probability distribution on satisfying, K a m- % µ a m- for all A. The distribution a is called the sampling distribution for the petite set A, see Section 4. of Meyn and Tweedie [4]. The following result is from Meyn and Tweedie [42, Theorem 4.]. 9

11 ! p p Lemma 3. Let B be a closed petite set, suppose that! τ B Ye, and that for some δ (3.) sup / B τ B δ n where τ B δ inf t δ : X t d B. Then, X is positive Harris recurrent. Lemma 3.2 Under assumptions (.4) and (.5) on interarrival distributions, B closed petite set for any κ. 4 4o# κ is a PROOF. This lemma can be proved in a way similar to the proof of Lemma 3.7 of Meyn and Down [38]. Remark. This is the only place where conditions (.4) and (.5) is used. Theorem 3. If there eists δ such that (3.2) then (3.) holds for B 4 X B B k δ 84 =D : 4 4E# κ for some κ. In particular, X is positive Harris recurrent. Remark. Here we use X to denote X starting from. Note that 4 X t 84 can be alternatively written as 4 X t 84. PROOF. Let ε, for eample, * 2. From (3.2) there eists κ such that (3.3) 4 X δ 84q# ε for all such that 4 4 κ. Let B \D : 4 4r# κ. It follows from (.3) that, for some constant b, (3.4) for all. Let (3.5) 4 X 4 4 δ 84c# ε 84 4 b B " 4 4 δ if s B n δ if B Because κ, n % δ for all UD. It follows from (3.4) that 4 X n 84q# ε 84 4G b B #t4 4 ε δ n u b B for some b and all D. Proceeding eactly the same as in the proof of Theorem 2.(ii) of Meyn and Tweedie [43], we have for each `D, and Thus, recurrent. τ B δ o# δ ε V 4 4 bx δ δ sup τ / B B δ o# sup 4 4G bw ε v / B ε κ b τ B fe and it follows from Lemma 3. and Lemma 3.2 that X is positive Harris

12 p 4 Fluid Limit Models In order to make Theorem 3. practical, one must have a systematic method to check it (3.2). The fluid approimation developed in this section is a powerful tool to obtain such a it. In particular, we will prove in the main theorem (Theorem 4.2) of this paper that a queueing discipline in the queueing network is stable if the corresponding fluid it model is stable. In Sections 5 7, we will give eamples to demonstrate how one can check the stability of fluid it models. The following lemma maybe hidden in some tetbook. It is needed to prove Lemma 4.2. For completeness, we present a proof. Lemma 4. Let f n be a sequence of nondecreasing functions on R and f be a continuous function on R. Assume that f n t 3 f t for all rational t. Then f n 3 f u.o.c. PROOF. First, because f n is nondecreasing and f is continuous, one can easily check that f n t 73 f t for every t R. Net, suppose that f n does not converge to f uniformly on compact sets. Then there eist ε, t and t nl such that t nl # t and (4.) Because t nl is bounded, we may assume that t nl 3 than t δ. Hence for l large enough, Therefore, 4 f nl t nl 7 f t nl 84 ε for all l. t # t. Thus for any δ, t nl eventually is less f nl t nl A f t nl # f nl t δ 7 f t nl f nl t δ 7 f t δ u f t δ A f t u f t 7 f t nl sup lk Because f is continuous and δ is arbitrary, we have V f nl t nl A f t nl X # f t δ 7 f t sup lk When t, one can similarly prove that When t, Thus we have V f nl t nl A f t nl X # inf lk V f n l t nl 7 f t nl X inf lk V f n l t nl A f t nl X lk V f n l 7 f t nl X lk V f n l t nl 7 f t nl X which contradicts (4.). Hence the lemma is proved. Let <;=, U,V be the initial state of the network under a specified queueing discipline. In this section, we attach a superscript to a symbol to eplicitly denote the dependence on the initial

13 { state. In particular, Q k t is the queue length for class k customers at time t. For yzw and k K, define E{ t ma n : U{ R ξ{ u ξ{ n # tc t Sk t ma n : Ṽ k R η k u η k n d# tc t where Ṽ k V k if there is a class k customer that has non-zero remaining service time at time and is set to be a fresh class k service time η k otherwise, where η k is independent of η k n n and has the distribution as that of η k. (We make the convention that the maimum of an empty set is zero.) It is clear that E t is the total number of eogenous arrivals to class k by time t. For k K and each n, define Φ k n n φ k i i$ Note that Φ k n does not depend on. Now we present a functional strong law of large numbers for the processes defined. Lemma 4.2 Let n PTD with 4 n 483 as n 3. Assume that Then as n 3 nk, almost surely (4.2) (4.3) (4.4) U 4 n 4 Ū and nk 4 n 4 Φ k where t is the integer part of t and µ k * m k. 4 n 4 t} 3 Pk t u.o.c. V 4 n 4 V E n k n t 3 n α k t Ū k u.o.c. S n k n t 3 n µ k t V k u.o.c. Remark. The discussion in Section 2.3 of Chen and Mandelbaum [9] also outlined a proof for the lemma. PROOF. First (4.2) follows from the classical functional strong law of large numbers. To prove (4.3) it suffices to prove that (4.5) almost surely because Ek 4 n 4 4 n 4 t 3 α k t u.o.c. E 4 n 4 n k 4 n 4 t Ek 4 n 4 V 4 n 4 t U k *~4 n 4j To see (4.5), first by the strong law of large numbers for renewal process (see, for eample, Resnick [46, Theorem ()]), one has that almost surely tk E k t * t 3 α k 2 X

14 g It then follows that almost surely nk E k 4 n 4 t *~4 n 4 nk t E k 4 n 4 t * 4 n 4 t 3 α k t e f t for each t Because f t is continuous and Ek 4 n 4 t *~4 n 4 is nondecreasing for each n, it follows from Lemma 4. that (4.5) holds. Similarly, one can prove that (4.4) holds. p For <;= U V, let U k t Ek H ti ξ k i R U k 7 t i$ It is the residual life process associated with the delayed renewal process E k t t. Lemma 4.3 Let n PTD with 4 n 483 as n 3. Assume that (a) As n 3 almost surely nk U 4 n 4 k Ū k and nk U 4 n 4 n k 4 n 4 t 3 Ū k t u.o.c. and (b) For each fied t, V 4 n 4 k V k V 4 n 4 n k 4 n 4 t 3 V k t u.o.c. are uniformly integrable. U 4 n 4 n k 4 n 4 t 4 n 4 h and g 4 n 4 V n k 4 n 4 t 4 n 4 h PROOF. For each fied t, by the strong law of large numbers, we have (4.6) E n k H B nb ti 4 n 4 ξ k i 3 t Ū k i$ almost surely. Hence, (4.6) holds almost surely for all rational t. Because t 3 t Ū k is continuous, by Lemma 4., almost surely (4.6) holds u.o.c. Thus (a) is proved. To prove (b), note that n U 4 n 4 n k 4 n 4 t n ƒ E n k H B nb ti 4 n 4 ξ k i \ i$ U k 4 n 4 t By Wald s identity, the right term in the preceding equation is equal to 4 n 4 α k n ˆ E n k 4 n 4 t R o By the Elementary Renewal Theorem [46, Theorem 3.3.3], the preceding epression converges to t Ū k Ū k t. The proof of (b) then follows from Chung [, Theorem 4.5.4]. p 3 U k 4 n 4 t

15 ( ( v Now we describe a set of equations that capture most of the network dynamics. These equations were first derived by Harrison [23] to obtain Brownian models with scheduling capability. They were later refined by Harrison and Nguyen [24] for FIFO discipline. They are now quite standard in Brownian models of queueing networks. For notational convenience we assume that at most one customer within a customer class can have nonzero partial service time. Thus Sl t is the total number of service completions from class l if class l is given t units of service time. Let Tl t be the cumulative amount of service time that server s l has spent on class l customers by time t. Then Sl Tl t is the total number of service completions from class l by time t. A fraction of these customers, equal to Φ l k S l T l t of them, become class k customers. Thus, we have the following representation for the queue length processes. (4.7) Q k t For i d, let Q k u E k t R K Φ l k Sl Tl t A Sk Tk t k K l$ I i t t Tk t k/ C i Then I i t is the cumulative amount of time that server i has been idle by time t. We assume that all queueing disciplines considered in this paper are work-conserving. That is, server i is idle only when there is no customer at station i. Hence, we have (4.8) Put T t T t T d K t, I t Š k/ C i Q k t G di i t i d I t Id t and S T t S T t S T t For a fied work-conserving queueing discipline, in vector form, we have (4.9) (4.) (4.) (4.2) (4.3) (4.4) Q t Q u E t R Q t % T t is nondecreasing and T I t et CT t is nondecreasing K Φ l Sl Tl t 7 S T t l$ CQ t di t some additional conditions on Q m- T m- that are specific to the discipline, where e is the d-dimensional vector of ones, C is the constituency matri defined in (.6) and for a vector function f t f t f d t, ( f t di t ( f t di t ( f d t di d t w Conditions (4.9) (4.3) are satisfied for any work-conserving discipline. Eamples will be given for condition (4.4) for priority disciplines in Section 7. 4

16 ( Theorem 4. Consider a work-conserving queueing discipline. For almost all sample paths ω and any sequence of initial states n >PŒD with 4 n 4 3, there is a subsequence n j with 4 n j 4 3 such that (4.5) (4.6) 4 n j 4 Q n j U n j V n j Ž3 4 n j 4 Q n j 4 n j 4 t T n j 4 n j 4 t 3 Q Ū V Q t T t u.o.c. Furthermore, (4.7) (4.8) (4.9) (4.2) (4.2) (4.22) Q T satisfies the following set of equations. Q t Q u αt Ū I P M, T t 7 V Q t % T t is non-decreasing and starts from zero, Ī t et C T t is non-decreasing V C Q t X dī t some additional conditions on Q m- T m- that are specific to the discipline. Remark. This theorem, in slightly different form, was proved by Chen and Mandelbaum []. PROOF. Notice first that 4 Q 4 n 4 n 84c# 4U 4 n 4 n 84q# 4V 4 n 4 n 84# for all n. Therefore, there eists a subsequence n j such that (4.5) holds. For notational convenience, we assume that as n 3, 4 n 4 Q n U n V n 3 Q Ū V Let ω be a fied sample path such that (4.2) (4.4) hold. For `D and k K, it is easy to see that Tk t A Therefore there eists a subsequence n j such that for some process T t (4.) (4.2). By Lemma 4.2 and (4.9), we have Tk s %# t s for t s T n j 4 n j 4 k 4 n j 4 t 3 T k t u.o.c. as j 3 T t T K t. Furthermore, conditions (4.8) (4.2) follow easily from Q 4 n j 4 n j 4 n j 4 t 3 Q t 5

17 ( ( ( p where Q t satisfies (4.7). To prove (4.2), note first that (4.3) is equivalent to t Š k/ C i 4 n j 4 where a b is the minimum of a and b. Letting j 3 t Š Q n j k 4 n j 4 s ds for each t and i d and using Lemma 4.4 below, we have Q k s G ds for each t and i d k/ C i which is equivalent to (4.2). Additional condition (4.22) is justified for each discipline from (4.4) by passing through the iting procedure similar to the one for obtaining (4.2). p Lemma 4.4 Let z n y n be a sequence in D R b) C R. Assume that y n is nondecreasing and z n y n converges to z y C R ) C R u.o.c.then for any bounded continuous function f, t f z n s dy n s 3 ( PROOF. See Lemma 2.4 of Dai and Williams [4]. t f z s dy s u.o.c. Definition 4. Let a queueing discipline be fied. Any it Q m- T m- in (4.6) is a fluid it of the discipline. Any solution Q m- T m- to (4.7) (4.22) is called a fluid solution of the discipline. We say that fluid it model (resp. fluid model) of the queueing discipline is stable if there eists a constant δ that depends on α µ and P only, such that for any fluid it (resp. fluid solution) with 4 Q 84GŒ4Ū 4G'4 V 4, Q m δ +e. Now we can summarize our main result. Theorem 4.2 Let a queueing discipline be fied. Assume that (.2) (.4 ) hold. If the fluid it model of the queueing discipline is stable, then the Markov chain X describing the dynamics of the network under the discipline is positive Harris recurrent. PROOF. First notice that (4.23) 4 X t 84 K k$ Q k t u K _ Uk t R Vk t E k$ Assume that the fluid model is stable and δ is a constant as in Definition 4.. Let n >PD be any sequence of initial states with 4 n 483 as n 3. Because δ and the fact that 4Ū 4G'4 V 4 #, by Theorem 4. and Lemma 4.3 (a), almost surely there is a subsequence n j such that jk jk jk Q n j 4 n j 4 k 4 n j 4 δ Q i δ U n j 4 n j 4 k 4 n j 4 δ V n j 4 n j 4 k 4 n j 4 δ 6 δ Ū k Ū k δ δ V k V k δ

18 p p It follows from Lemma 4.5 and Lemma 4.3 (b) that (4.24) (4.25) jk jk 4 n j 4 4 n j 4 By (4.23) (4.25), we have ` Q n j k n j δ ` U n j k n j δ and jk jk F 4 n j 4 X n j 4 n j 4 δ V n j 4 n j 4 k 4 n j 4 δ Because n is an arbitrary sequence, we have that condition (3.2) holds, and hence X is positive Harris recurrent. Remark. Even if the fluid model is stable, the fluid solution in (4.7) (4.22) may not be unique. This is not surprising because the equations governing the fluid model did not specify how the fluid is configured initially. Remark 2. Under the standard assumptions (.2) (.4 ) the queueing network is stable if the corresponding fluid it model is stable. In the sequel, we always assume that these standard assumptions hold. Lemma 4.5 PROOF. Because B Q B 4 4 δ : `D with 4 4 is uniformly integrable. 4 Q δ 84q# Ek k/ δ u C it suffices to prove for each k \, (4.26) Ek t * t t is uniformly integrable. Because E k n * n 3 α k almost surely and Ek n }* n 3 α k by the Elementary Renewal Theorem [46, Theorem 3.3.3], it follows from Chung [, Theorem 4.5.4] that E n * n n is uniformly integrable. Note that for each t, there is a unique integer n such that n # t # n and Hence E k t * t # n Ek n n n E k t * t t is uniformly integrable, and the lemma is proved. 5 Generalized Jackson Network Consider a special case where 4C i 4 for i d. This is called a single class network because there is only one class of customers served at each station. In other words, all customers at a station are homogeneous in terms of service requirement and routing. This network is the same one as considered by Jackson [26] ecept that we allow general interarrival and service time distributions. Hence, this single class network is often called a generalized Jackson network. 7

19 Theorem 5. Assume that (.2) (.4) hold in a generalized Jackson network. Then each of the workconserving disciplines discussed in Section 2. is stable if (.9) holds. Remark. The stability question for this network was resolved by Borovkov [4], Sigman [5], Foss [2], Meyn and Down [38], Chang, Thomas and Kiang [7] and Baccelli and Foss [] under various assumptions on interarrival and service time distributions. Before proving the theorem, let us first introduce the dynamic complementarity problem (DCP). Let R I P M, Definition 5. Let : %3 R d be a continuous path with. The (continuous) dynamic complementarity problem for m- and R is DCP( m- R): given m-, find a pair of continuous paths z m- y m- such that z t t u Ry t for t z t for t y m- is nondecreasing, y and y i m- increases only at times t such that z i t, i d. The pair z m- y m- is a solution to the DCP( m- R). The term DCP is from Mandelbaum [37]. It is sometimes called the deterministic Skorohod problem. When the solution z m- y m- is unique for each m-, the mapping m- 3 z m- y m- is also called the reflection mapping with reflection matri R; see Harrison [22]. Harrison and Reiman [25] proved that given an m- with, there is a unique solution z m- y m- to the DCP( m- R). Mandelbaum [37], Bernard and El Kharroubi [3] and Kozyakin and Mandelbaum [3] studied the DCP for much more general reflection matri R. The reflection mapping has played an important role in the heavy traffic analysis of queueing networks, see Reiman [45], Johnson [27] and Chen and Mandelbaum [9]. A path m- is said to be regular at t if it is differentiable at t. We use ẋ t to denote the derivative of m- at a regular point t. The following lemma resembles a monotonicity result in Foss [9] and Shanthikumar and Yao [49]. Lemma 5. Let z m- y m- be the solution to the DCP( m- R) and s be fied. Assume that (.9) holds and (5.) where (5.2) Then, (5.3) and hence ẏ s # ρ if y m- is regular at s. z s R t s 7 s d θt for all t, θ α P µ µ y t s 7 y s # ρ t for all t, 8

20 ( p p PROOF. Take t θt for t and y t R, t ρ t Then y and y m- is nondecreasing by (.9). Let z m- ỹ m- be the unique solution to the DCP( m- R). Because z t and the elements of R, are nonnegative, we have ỹ t % y t for t. If follows from Propositions and 2 of Reiman [45] that z t e and ỹ t e y t. Note that z t s z s R t s 7 s R R y t s 7 y s e t u Rỹ t where t z s R t s 7 s and ỹ t y t s 7 y s. Assume that (5.) holds. Then, t b t for t. It follows again from Propositions and 2 of Reiman [45] that ỹ t y t s 7 y s d# y t ρ t for t Lemma 5.2 Let f : +3 be a nonnegative function that is absolutely continuous and κ be a constant. Suppose that for almost surely (with respect to Lebesgue measure on all regular points t, ḟ t #š κ whenever f t. Then f is nonincreasing and f t e for t f * κ. PROOF. The elementary proof is omitted. While this paper is under review, Chen [8] proved the following result. Lemma 5.3 Suppose that there is δ such that Q mn δ %e for any fluid solution Q m- T m- with 4 Q 84, Ū and V. Then the fluid model is stable. Proof of Theorem 5.. Without loss of generality, we assume that C i i. By Theorem 4.2 and Lemma 5.3 it suffices to show that any solution Q m- to (5.4) (5.5) (5.6) (5.7) (5.8) Q t Q R αt I P M, T t Q R αt I P µt I P M, Ī t Q t d T T t is nondecreasing, Ī t et T t is nondecreasing, Q t dī t is identically zero on δ for some δ that depends on α, µ and P only. At this point we could quote Chen and Mandelbaum [9] to finish the rest of the proof. Instead we provide a new approach to the proof. This approach was communicated to me by Meyn and Down. It follows that Q m- T m- is the solution to the DCP for Q R θt and R I P M,. One can check that condition (5.) is satisfied for each s. Therefore by Lemma 5. Ī s # ρ at regular point s of Ī m-. Let f t Ye e R, Q t 9 e M I P, Q t

21 p p It follows from (5.4) that f t f R d ρ i t Ī i t i$ Assume that t is a regular point for Q m- and Ī m-. If f t, then Q i t for some i. Hence, by (5.8), Ī i t. Hence ḟ t #œ ρ i It is clear that f is absolutely continuous and it follows from Lemma 4.2 that f t +e for t f * κ, where f e M I P, Q # m I P, e and κ ma 6 j6 d ρ j Put δ m I P, e* ma 6 j6 d ρ j. Then Q t e for t δ. 6 Multiclass Station and Feedforward Network Theorem 6. Let d. Any work-conserving queueing discipline discussed in Section 2. is stable under (.9). PROOF. Assume that (.9) holds. By Theorem 4.2 and Lemma 5.3, it is enough to show that the fluid model is stable when Ū and V. Let f t e M I P, Q t f u ρ t Ī t Let t be a regular point of Q m- and Ī m-. If f t, then K k$ Q k t. Therefore, it follows from (4.2) that Ī t, and thus ḟ t ρ. Therefore Q t e for t f * ρ b# m I P, e. A network is feedforward if stations can be numbered in such a way that P kl and l C j with i j. for any k C i Corollary 6.2 In a feedforward network, any work-conserving queueing discipline is stable under (.9). 7 First-Buffer-First-Served in a Reentrant Line Consider a re-entrant line where each customer follows one deterministic route. Customers in kth stage of service are called class k customers. The entries of the routing matri take the form P k k for k K and zero for all others. The deterministic version of this network was studied by Kumar [32]. Suppose that for any two customer classes k and l visiting the same station, class k customer priority is higher than class l s whenever k l. Then the queueing discipline is called first-buffer-first-served (FBFS) discipline. Theorem 7. For a re-entrant line, the first-buffer-first-served preemptive resume discipline is stable under (.9). 2

22 ( ( ( { { { { { Remark. FBFS is one of the priority disciplines considered in Dai and Weiss [3], where the stability for a class of priority queueing disciplines are studied. We present this special case here mainly to illustrate the effectiveness of using Theorem 4.2 to establish stability for some multiclass networks with feedback. A deterministic version of this theorem was proved by Kumar [32]. PROOF. Assume that α. Let H k denote the set of indices for all classes served at station s k which have priority greater than or equal to that of class k. That is, H k l : # l # k s l s k. Let T k t / H k T t I k t t T k t Q k t Q { t / H k Then T k t is the cumulative amount of service in t dedicated to customers whose classes are included in H k, and I k t is the total unused capacity that is available to serve customers whose class does not belong to H k. Note that Ii t is a station level quantity representing the total unused capacity in t by server i; whereas I k t is a class level quantity. The priority service discipline requires that for every k, all the service capacity of station s k is dedicated to classes in H k, as long as the workload present in these buffers is positive. Since we assume the preemptive resume queueing discipline, we have that I k m- increases only when Q k t. Thus we may epress additional condition (4.4) by the integral equation (7.) Q k Condition (7.) implies that for each UD, t di k t # k # K Q 4 4 k 4 4 t d I v 4 4 k 4 4 t Gw # k # K Taking the it as in Theorem 4. and using Lemma 4.4, we have (7.2) where Q k t dī k t # k # K T k t T{ / H k Ī k t t T k Q k t / H k Q{ t t t Therefore, for the priority discipline, the additional condition (4.22) takes the form (7.2). 2

23 ( Now we show that the fluid model determined by (4.7) (4.2) and (7.2) is stable under (.9). In light of Lemma 5.3, we can assume that Ū V. We use induction to show that for each k K, there eists t k such that Q{ t t k e for y k. For k, because Q t Q R µ t µ Ī t Q t % Ī t is nondecreasing, Q t dī t we have Q t Že for t Q * µ. Since 4 Q 84, we can take t * µ. Assume that at time t k, all the buffers k are empty and that they shall stay empty for t t k,. Let the content of buffer k be Q k t k,. This is bounded above by t k,. Then, since for t t k, buffers k remain empty, it follows that if Q k t for t t k, then it is the first nonempty buffer, and so by Proposition 4.2 of Dai and Weiss [3] the arrival rate to buffer k is a k t and the departure rate from buffer k is d k t { / H kžÿ k m{ µ k Therefore, Q k t µ k { / H k m{, and hence buffer k will be empty at time t k, where t k t k, Q k t k, m * k { / H k m{ and will stay empty at all times after t k. Therefore, by the induction argument the fluid model will reach Q t δ K k$ Š and remain zero thereafter no later than at l$ j/ H { m žgÿ j k l$ j/ H m j m k k, (See Dai and Weiss [3] for discussions on fluid models under other priority disciplines.) p. Acknowledgment. My interest in this research was initiated by a conversation with Ruth Williams during the 2st Conference on Stochastic Processes and their Applications in Toronto in the summer of 992. I am grateful to Ward Whitt for sending me a recent paper by Rybko and Stolyar, which provided further motivation for this research. Tom Kurtz helped to weaken condition (.3) to the current form. Sean Meyn made numerous comments that have greatly improved the eposition of this paper. In particular, he and a referee corrected mistakes in an earlier formulation of Lemma 3.2. I also benefited from conversations with Hong Chen, Peter Glynn, Avi Mandelbaum and Gideon Weiss, and from comments by two anonymous referees. References [] Baccelli, F. and Foss, S. Stability of Jackson-type queueing networks, I, Queueing Systems: Theory and Applications, to appear. [2] Baskett, F., Chandy, K. M., Muntz, R. R., and Palacios, F. G. Open, closed and mied networks of queues with different classes of customers. Journal of the ACM 22, (975). 22

24 [3] Bernard, A. and El Kharroubi, A. Régulations déterministes et stochastiques dans le premier orthant de R n. Stochastics and Stochastics Reports 34, 49 67, 99. [4] Borovkov, A. A. Limit theorems for queueing networks. Theory of Probability and its Applications 3, (986). [5] Botvitch, D. D. and Zamyatin A. A., Ergodicity of conservative communication networks. Rapport de recherche 772, INRIA, October 992. [6] Bramson, M. Instability of FIFO queueing networks. Annals of Applied Probability, to appear. [7] Chang, C. S., Thomas, J. A. and Kiang, S. H. On the stability of open networks: a unified approach by stochastic dominance. Queueing Systems: Theory and Applications 5, (994). [8] Chen, H. Fluid approimations and stability of multiclass queueing networks I: Workconserving disciplines. submitted for publication. [9] Chen, H. and Mandelbaum, A. Discrete flow networks: Bottlenecks analysis and fluid approimations. Mathematics of Operations Research 6, (99). [] Chen, H. and Mandelbaum, A. Open Heterogeneous fluid networks, with applications to multitype queues. Preprint. [] Chung, K. L. A Course in Probability. Wiley, 974. [2] Dai, J. G. and Meyn, S. P. Stability and convergence of moments for multiclass queueing networks via fluid it models. Submitted for publication. [3] Dai, J. G. and Weiss, G. Stability and instability of fluid models for certain re-entrant lines. Submitted for publication. [4] Dai, J. G. and Williams, R. J. Eistence and uniqueness of semimartingale reflecting Brownian motions in conve polyhedrons. Theory of Probability and its Applications (to appear). [5] Davis, M. H. A. Piecewise deterministic Markov processes: a general class of non-diffusion stochastic models. Journal of Royal Statist. Soc. series B (984). [6] Down D. and Meyn S. P., Piecewise linear test functions for stability of queueing networks, Proceedings of the 33rd Conference on Decision and Control, (994). submitted. [7] Dupuis, P. and Williams, R. J. Lyapunov functions for semimartingale reflecting Brownian motions. Annals of Probability, to appear. [8] Ethier, S. N. and Kurtz, T. G. Markov Processes: Characterization and Convergence. Wiley, New York,

25 [9] Foss, S. G. Some properties of open queueing networks. Problems of Information Transmission 25, (989). [2] Foss, S. G. Ergodicity of queueing networks. Sib. Math. J. 32, (99). [2] Getoor, R. K. Transience and recurrence of Markov processes. In Azéma, J. and Yor, M., editors, Séminaire de Probabilités XIV, pages , New York, 979. Springer-Verlag. [22] Harrison, J. M. Brownian motion and stochastic flow systems. Wiley, New York, 985. [23] Harrison, J. M. Brownian models of queueing networks with heterogeneous customer populations. Proceedings of the IMA Workshop on Stochastic Differential Systems (988). Springer- Verlag. [24] Harrison, J. M. and Nguyen, V. Brownian models of multiclass queueing networks: Current status and open problems. Queueing Systems: Theory and Applications 3, 5 4 (993). [25] Harrison, J. M. and Reiman, M. I. Reflected Brownian motion on an orthant. Annals of Probability 9, (98). [26] Jackson, J. R. Networks of waiting lines. Operations Research 5, (957). [27] Johnson, D. P. Diffusion Approimations for Optimal Filtering of Jump Processes and for Queueing Networks. PhD thesis, University of Wisconsin, 983. [28] Kaspi, H. and Mandelbaum, A. Regenerative closed queueing networks. Stochastics and S- tochastics Reports 39, (992). [29] Kaspi, H. and Mandelbaum, A. On Harris recurrence in continuous time. Mathematics of Operations Research 9, (994). [3] Kelly, F. P. Networks of queues with customers of different types. J. Appl. Probab. 2, (975). [3] Kozyakin, V. S. and Mandelbaum, A. Absolute stability and dynamic complementarity. submitted for publication. [32] Kumar, P. R. Re-entrant lines. Queueing Systems: Theory and Applications 3, 87 (993). [33] Kumar, P. R. and Meyn, S. P. Stability of queueing networks and scheduling policies. (Preprint). [34] Kumar, P. R. and Seidman, T. I. Dynamic instabilities and stabilization methods in distributed read-time scheduling of manufacturing systems. IEEE Transactions on Automatic Control AC- 35, (99). [35] Lu, S. H. and Kumar, P. R. Distributed scheduling based on due dates and buffer priorities. IEEE Transactions on Automatic Control 36, (99). 24