Dynamic Control of N-Systems with Many Servers: Asymptotic Optimality of a Static Priority Policy in Heavy Traffic

Transcription

1 Dynamic Contol of N-Systems with Many Seves: Asymptotic Optimality of a Static Pioity Policy in Heavy Taffic Tolga Tezcan Industial and Entepise Systems Engineeing Univesity of Illinois at Ubana-Champaign Ubana, Illinois ttezcan@uiuc.edu J. G. Dai H. Milton Stewat School of Industial and Systems Engineeing Geogia Institute of Technology Atlanta, Geogia dai@isye.gatech.edu Abstact: We conside a class of paallel seve systems that ae known as N-systems. In an N-system, thee ae two custome classes that ae cateed by seves in two pools. Seves in one of the pools ae coss-tained and can seve customes fom both classes wheeas all the seves in the othe pool can only seve one of the custome classes. A custome eneges fom his queue if his waiting time in the queue exceeds his patience. Ou objective is to minimize the total cost that includes a linea holding cost and a eneging cost. We pove that, when the sevice speed is pool-dependent, not class-dependent, a cµ-type geedy policy is asymptotically optimal in many-seve heavy taffic. The key to ou optimality poof is a state space collapse esult fo diffusion-scaled pocesses unde the geedy policy. We employ a famewok developed in Dai and Tezcan 25 to pove the state space collapse in many-seve heavy taffic. AMS 2 subject classifications: Pimay 6K25; seconday 9B22, 6F17. Keywods and phases: N-systems, Paallel Seve Systems, Asymptotic Optimality, Many seve limit, Heavy taffic limit, Halfin-Whitt Regime. 1. Intoduction Paallel seve systems have become a common tool fo the analysis of systems aising in diffeent settings such as communication netwoks, manufactuing and sevice systems; see, e.g., Betsekas and Gallaghe 1992, Buzacott and Shantikuma 1993, Gans et al. 23 and Yao In this pape we conside paallel seves systems with an N-design, o N-systems fo shot, with many seves. A topological classification of paallel seve systems can be found in Ganett and Mandelbaum 2. We pove that a cµ-type geedy outing policy is asymptotically optimal fo N-systems in a many-seve heavy taffic egime. An N-system consists of two custome classes and two seve pools and is simila to but moe geneal than those consideed in Haison 1998 and Bell and Williams 21. Unlike the models in Haison 1998 and Bell and Williams 21, ou N-system may have moe than one seve in each pool and customes enege fom queue if they wait too long. A moe detailed desciption of ou N-system with a schematic diagam is given in Section 2. Reseach suppoted in pat by National Science Foundation gant DMI-3599, by an IBM Faculty Awad 1

2 J. G. Dai and T. Tezcan/Dynamic Contol of N-Systems 2 Fo an N-system, the following decisions need to be made. a When an aiving custome finds his queue empty and thee ae idle seves capable of seving him, should the custome be outed to an idle seve immediately o be outed to queue? It may be beneficial sometimes to eseve some seves in anticipation of moe expensive custome aivals; b when an aiving custome can be seved immediately by seves fom eithe seve pool, which seve pool should be used to seve the custome? c when a seve in pool 1 finishes a sevice and thee ae customes waiting in queue 1, should the seve idle o pick a new custome to seve? d when a seve in pool 2 finishes a sevice, should the seve idles o seve a custome fom queue 1 if the queue is not empty o seve a custome fom queue 2 if the queue is not empty? A policy dictating these decisions is said to be a outing policy in the call cente liteatue; see, fo example, Gans et al. 23. The cost function we conside has two components, holding cost and eneging cost. We ae inteested in finding a outing policy to stochastically minimize the total cost of holding customes in the queue and abandonment ove a finite time inteval. We focus on the systems whee sevice times can be pool-dependent, but not class-dependent. Theefoe, the sevice time distibution only depends on the pool of the seve poviding sevice. We conside the following cµ type, geedy outing policy: a each seve is non-idling; b when a seve is eady to pick a next custome to wok on, he picks a custome fom the moe expensive queue; c when an aiving custome in class 1 is eady to pick a seve, he picks a faste seve. Ties ae assumed to be boken in favo of lowe indexed classes and pools. Fo futue efeence, this geedy policy is denoted by π. In this pape, we pove that the geedy policy π is asymptotically optimal. Unde ou asymptotic famewok, a sequence of N-models ae consideed, and the paametes of the sequence ae assumed to satisfy a many-seve heavy taffic condition. When this condition is satisfied, the sequence of systems is known to opeate in many-seve heavy taffic egime o the Halfin-Whitt asymptotic egime e.g. Gans et al. 23, Halfin and Whitt Unde the many-seve heavy taffic condition, aival ates and numbe of seves in each pool gow to infinity in such a way that the nominal load conveges to one; futhemoe, the total of numbe of seves in each system is equal to the total offeed load plus a constant multiple of the squae oot of the offeed load. This latte fact is in accodance with the squae-oot safety staffing ule as in Bost et al. 24 and Ata et al. 24. In addition, we focus on the setting whee seve pool 1 cannot handle all the load fom class 1 customes so that some of the class 1 customes have to be outed to the second seve pool. This assumption is simila to the complete esouce pooling assumption in Haison 1998 and Bell and Williams 21. Ou esult may come as a supise in light of the simulation study in Haison Thee, N-systems ae studied in the conventional heavy taffic egime when the numbe of seves in each seve pool is fixed as the offeed load and the sevice speed of each seve inceases. The simulation study in Haison 1998 shows that when each seve pool has a single seve, the system is unstable unde the outing policy π : queue 2 gows without bound even though the total offeed load is below the total system capacity. We pefom simulation expeiments in Section 1.1 to illustate the diffeences between these two heavy taffic egimes. Ou poof of the asymptotic optimality of policy π consists of two majo steps. In the fist step, we establish a lowe bound fo the total cost unde any policy. Citical in the lowe bound poof ae what we call a lowe bound map defined in Lemma 3.1 and an associated compaison esult in Lemma 3.2. The lowe bound map plays a simila ole in the many-seve limit setting to the one-dimensional Skoohod map in the single-seve setting. In the second step, we show that unde policy π, the total cost achieves the lowe bound. A key to poving this is the following state space

3 J. G. Dai and T. Tezcan/Dynamic Contol of N-Systems 3 collapse SSC esult: unde the diffusion scaling, a queue 1 is always empty; b pool 2 seves ae 1% busy; c pool 1 seves may idle only when queue 2 is empty. Pat c says that the system achieves the complete esouce pooling in diffusion limit unde policy π. To pove the SSC esult, we employ a famewok that is ecently developed in Dai and Tezcan 25. In addition to poving the optimality of π, this pape illustates in a concete setting, in the second step of the optimality poof, how the famewok in Dai and Tezcan 25 is used to pove SSC esults. This illustation constitutes anothe majo contibution of this pape. To use the famewok in Dai and Tezcan 25, we fist pove that policy π is efficient in many-seve fluid limits. This is poved by studying a fluid model that is defined via a set of deteministic fluid model equations see Section 6.2. We next pove that a hydodynamic model has the analogous SSC esult. The hydodynamic model is defined by a set of deteministic hydodynamic model equations. These equations ae simila to but diffeent fom the fluid model equations. The hydodynamic model equations ae satisfied by hydodynamic limits, a concept that is fully developed in Dai and Tezcan 25. The hydodynamic scaling that is used to define the hydodynamic limits is obtained by slowing down the time in the diffusion scaling. Among many diffeences, in the fluid model we keep tack of numbe of busy seves in each seve pool, wheeas in the hydodynamic model, we keep tack of numbe of idle seves in each seve pool. Following the famewok, the SSC esult fo the hydodynamic model implies a weak fom of the SSC esult, known as the multiplicative SSC, in diffusion limit. Finally, using the multiplicative SSC esult, we show a compact containment condition: unde policy π, the diffusion-scaled queue sizes and numbe of busy seves ae stochastically bounded. The compact containment condition allows us to conclude the SSC esult in the diffusion limit fom the multiplicative SSC. The famewok in Dai and Tezcan 25 is an extension of a simila famewok that was fist developed in Bamson 1998 fo multiclass queueing netwoks in the conventional heavy taffic setting. Ou assumption that sevice times may only depend on the seve pool is cucial in ou asymptotic optimality poof. In Section 7 we show how this assumption can be slightly elaxed to equie that the diffeence is asymptotically negligible as in Maglaas and Zeevi 23. The est of this pape is oganized as follows. As a pat of the intoduction, we fist pesent the esults of simulation expeiments that illustate the diffeence between the pefomances of N-systems unde two diffeent heavy taffic egimes. Then, in Section 1.2 we eview the elated liteatue. We pesent the notation used in the est of the pape in Section 1.3. In Section 2, we discuss the pecise details of the N-systems and the many-seve asymptotic famewok we conside. We pesent ou main esults in Section 3. In Section 4 we povide an analytical fomulation of N-systems and pesent an outline of the poofs of ou main esults. In Sections 5 and 6, we pove ou main esults. Extensions of ou main esults ae discussed in Section Simulation expeiments Fist, we epeat Haison s simulation study with slightly modified system paametes. Fo the simulation expeiments in this section we assume that customes neve abandon the system. We set the numbe of seves to N 1 = N 2 = 1, the aival ates to λ 1 = 26ρ, λ 2 = 8ρ, and the sevice ates to µ 1 = 14 and µ 2 = µ 3 = 2. Inteaival and sevice times ae assumed to be exponentially distibuted. We take the unit time to be one minute. Theoetically, if ρ < 1, then the system has enough capacity to pocess the offeed load. We set ρ = 95% and simulate this N-system fo 5 hous. The left figue in Figue 1 plots the queue lengths as functions of time. The solid cuve is fo

4 J. G. Dai and T. Tezcan/Dynamic Contol of N-Systems Queue 1 Queue Queue 1 Queue 2 No of Jobs No of Jobs Time Elapsed Time Elapsed Fig 1. Left: queue length plots in the N-system with one seve in each pool; ight: queue length plots in the N-system with 2 seves in each pool queue 1 and the dashed cuve is fo queue 2. It is clea fom this plot that queue 2 gows without bound. Since ou theoems suggest that the geedy policy π is nealy optimal in the many-seve heavy taffic egime, we simulate the many-seve vesion of the same system with the following modifications: 1 each seve is slowed down 2 times, and 2 each seve pool has 2 identical seves. Theefoe, the sevice capacity of each seve pool has not changed. One may expect that a policy could not pefom stikingly diffeent in these two systems. Howeve, this tuns out to be not tue fo π. The ight figue in Figue 1 again plots the queue lengths as functions of time. It is clea fom this plot that none of queues gows without bound. One wondes what makes the two systems pefom dastically diffeently. A close examination of the simulation study eveals that, in the single seve setting, the seve in pool 2 helps too often with class 1 customes that the seve in pool 1 is staved too much due to the lack of class 1 customes. See Ganett and Mandelbaum 2 fo moe simulation expeiments illustating this fact. Theefoe, pool 1 seve accumulates moe than his shae of idle time, causing the entie system to be unstable. In the many seve setting, we will pove in Section 6.2 that fo each fluid limit, seve pool 2 does not help any class 1 customes when it has waiting customes and seve pool 1 is not fully busy; see Using this fact, one can pove that seves in pool 1 do not idle when queue 2 is not empty. Theefoe, unde policy π, each fluid limit achieves what is known as the complete esouce pooling in the liteatue ; see Ata and Kuma 25, Bell and Williams 21, Haison 1998, Haison and López 1999, Laws In the single seve setting, fo each conventional fluid limit, one is not able to aticulate the concept that seve pool 1 is notfully-busy. Attempt to use queue 1 being empty as a poxy fails because pool 1 seve could be fully busy even though fluid level in queue 1 is empty. It is an open eseach poblem to find out the citical value of the offeed load fo system stability unde policy π. In Section 7, we show how policy π can be modified so that the esulting new policy is always stable and is still asymptotically optimal Liteatue eview In the conventional heavy taffic egime, Haison 1988 pioneeed Bownian contol models to study the dynamic contol of multiclass queueing netwoks. The Bownian contol models ae

5 J. G. Dai and T. Tezcan/Dynamic Contol of N-Systems 5 extended to stochastic pocessing netwoks in Haison 2. In the many-seve heavy taffic egime, a diffusion contol poblem is poposed in the liteatue. Haison and Zeevi 24 and Ata et al. 24 study V-systems and Ata 25a,b study tee-like paallel seve systems. They focus on a fomal deivation of the diffusion contol poblem and obtain optimal contol policies. Howeve, specification of the optimal policies uses the solution to a set of patial diffeential equations PDE s and this set of PDE s can only be solved numeically. Also, paametes of these PDE s depend on aival and sevice ates as well as othe system paametes. Hence, it is difficult to see how sensitive the contol policies ae to the changes in these paametes. In addition, appoximations to system pefomance measues ae difficult unde these policies since they ae specified numeically. As noted in Gans et al. 23; the asymptotic analysis of Haison and Zeevi 24 and Ata et al. 24 has so fa shed little qualitative light on the stuctue of optimal contols fo skills-based outing in the QED egime. In this study, we focus on a simple policy that is paamete independent and whose asymptotic pefomance can be expessed in analytical closed fom. N-systems and moe geneal paallel systems have been extensively studied in the liteatue, mostly in the conventional heavy taffic egime. In addition to discoveing the instability phenomenon discussed ealie in the intoduction, Haison 1998 devises the discete eview policies fo N-systems and shows that they asymptotically minimize the expected discounted, cumulative linea holding cost when the aival pocesses ae Poisson and the sevice times ae deteministic. Haison and López 1999 genealizes these policies fo geneal paallel systems without poving thei optimality. Fo the same N-systems, but with geneal aival pocesses and sevice time distibutions, Bell and Williams 21 devises static buffe pioity policies with thesholds and poves thei asymptotic optimality unde a linea holding cost. They genealize thei esults to geneal paallel seve systems in Bell and Williams 25. Mandelbaum and Stolya 24 popose a genealized cµ policy fo paallel seve systems. They pove that it is asymptotically optimal in minimizing a stictly convex holding cost. They also discuss how thei esults can be extended to a many-seve heavy taffic egime that is diffeent fom the egime we conside. They focus on the case whee almost all the customes wait befoe sevice. That enables them to extend thei esults fom the conventional heavy taffic analysis easily. This staightfowad extension is not possible in the many-seve heavy taffic egime we conside hee when only a faction of customes ae delayed in queue. Moe geneal systems that include paallel seve systems as a special case have also been studied in the liteatue, again in the conventional heavy taffic egime. Stolya 24 poves that MaxWeight policies asymptotically minimize the wokload pocesses fo a genealized switch model that belongs to one-pass systems in which each custome leaves the system afte being pocessed at one pocessing step. Ata and Kuma 25 extends the discete eview policies in Haison 1998 to stochastic pocessing netwoks and poves thei optimality unde a complete esouce pooling condition and a balanced, conventional heavy taffic condition that each seve is citically loaded. Dai and Lin 25 poves that maximum pessue polices ae asymptotically optimal fo stochastic pocessing netwoks unde the same esouce pooling condition but a elaxed heavy taffic condition. Motivated the by the simulation study in Haison 1998, Ganett and Mandelbaum 2 also pefom simulation expeiments on N-models to study the effect of the offeed load on the pefomance of system. They also simulate N-systems unde the theshold policy poposed by Bell and Williams 21. In the many-seve heavy taffic egime, thee have been some studies in the liteatue on the

6 J. G. Dai and T. Tezcan/Dynamic Contol of N-Systems 6 asymptotically optimal policies unde vaious objective functions. Guvich et al. 24 studies a V-paallel seve system with impatient customes. They show that a static buffe pioity policy with a theshold is asymptotically optimal. Amony 25 studies an inveted V-paallel seve system and shows that the faste-seve-fist FSF policy is asymptotically optimal. Amony and Maglaas 24a,b study an M/M/n system with two custome classes. Shumsky 24, Stanfod and Gassmann 2 and Gans and Zhou 26 have modeled N-systems with Poisson aival pocesses as continuous Makov chains. While the Makov chain analysis povides numeical solutions to some contol poblems, it is difficult to povide stuctual insights to the contol of these systems with a lage numbe of seves Notation We collect some of the notation that is used in the est of this pape. The set of non-negative integes is denoted by N. Fo an intege d 1, the d-dimensional Euclidean space is denoted by R d and R + denotes [,. Let x denote the max nom on R d given by x = max i=1,2...,d { x i }. Fo x R, x = x and x + = x. We use {x n } to denote a sequence whose nth tem is x n. Fo a function f : R R, we say that t is a egula point of f if f is diffeentiable at t and use ft to denote its deivative at t. Fo each positive intege d, D d [, denotes the d-dimensional Skoohod path space; see Ethie and Kutz Fo x, y D d [, and T > we set xt yt T = sup xt yt. t T The space D d [, is endowed with the J 1 topology and the weak convegence in this space is consideed with espect to this topology. The notation X X will denote the weak convegence of X to X in the appopiate topological space as. Fo a sequence of functions {x n } D d [,, the sequence is said to convege unifomly on compact sets to x D d [, as n, denoted by x n x u.o.c., if fo each T > x n t xt T as n. A sequence of andom vaiables {x } is said to satisfy the compact containment condition if lim C lim sup P { x > C} =. 1.1 A sequence of stochastic pocesses {X } is said to satisfy the compact containment condition if X t T satisfies the compact containment condition fo evey T >. 2. The queueing model and asymptotic famewok In this section, we pesent the mathematical details of N-systems and the many-seve asymptotic egime we conside. An N-system consists of two custome classes and two seve pools. See Figue 2 fo a schematic diagam of the system. The cicles in this figue epesent two seve pools which accommodate seves having the same capacity and capability woking in paallel. The open-ended ectangles epesent infinite capacity buffes whee customes wait fo sevice. Customes in each class aive at the system following a enewal pocess and thei sevice times ae assumed to be

7 J. G. Dai and T. Tezcan/Dynamic Contol of N-Systems 7 exponentially distibuted. The seves in the fist pool can only seve class 1 customes with aveage sevice time equal to 1/µ 1. The second pool can seve eithe class. Aveage sevice time of a class 1 custome by a seve in pool 2 is 1/µ 2 and fo a class 2 custome it is 1/µ 3. Ou assumption that sevice times may only depend on seve pool implies that µ 2 = µ 3. λ 1 λ 2 γ 1 γ 2 µ 1 µ 2 µ 3 N 1 N 2 Fig 2. An N-system Upon aival, a custome s sevice may stat immediately if thee is an idle seve that can handle that custome. Othewise, customes fom class k join queue k and stat waiting fo one of the seves handle thei sevice equest. We assume that customes enege fom the system if they wait too long in queue befoe the commencement of thei sevice. We model this custome behavio by assuming that with each custome thee is an associated exponentially distibuted patience time. If a custome s waiting time exceeds his patience he abandons the system. We denote the ate of the patience distibution fo class k by γ k. Note that γ k may be equal to. In that case customes in class k ae said to have infinite patience and so they neve abandon the system. We focus on a sequence of N-systems indexed by. The aival ate to each class in the th system is denoted by λ = λ 1, λ 2. We use N = N1, N 2 to denote the numbe of seves in each seve pool and to denote the total numbe of seves in the th system in this sequence. Fo notational convenience, we define K = {1, 2} the set of custome classes and J = {1, 2} the set of seve pools. We assume that the set of pools that can handle class k customes is fixed, hence does not depend on, and denote it by J k. Similaly, we assume that the set of queues that seves in pool j can handle is fixed and denote it by Kj. Hence, fo ou N-systems J 1 = {1, 2}, J 2 = {2}, K1 = {1}, and K2 = {1, 2}. Fo the sequence of N-systems we conside, we assume that aival ates and numbe of seves incease and the offeed load on the system appoaches to its capacity as gets lage. Fist, we equie that the aival ates and numbe of seves in each pool ae in the same ode; i.e., we assume that thee exist λ i >, k K and β j >, j J, such that λ k λ k, as fo k K and N j = β j fo j J. 2.1 We also assume that the total capacity of the seves in pool 1 is not enough to handle all class 1 customes. This condition is known as the esouce pooling condition in the conventional heavy taffic liteatue; see Haison and López 1999 and Bell and Williams 21, among othes.

8 J. G. Dai and T. Tezcan/Dynamic Contol of N-Systems 8 Specifically, we assume that thee exist x 21, x 22 > that satisfy x 21 + x 22 = 1, λ 1 = β 1 µ 1 + β 2 x 21 µ 2, and λ 2 = β 2 x 22 µ Simila to the squaed-oot staffing pinciple used in Halfin and Whitt 1981, Ganett et al. 22 and Bost et al. 24, the numbe of seves in each pool is assumed to satisfy λ 1 = µ 1 N 1 + µ 2 x 21 N 2 + θ 1 and λ 2 = µ 2 x 22 N 2 + θ fo θ 1, θ 2 R and we set θ = θ 1 + θ 2 and fo each N. We use h k to denote the holding cost fo each time unit a custome is held in queue k. We assume that a penalty of c k is incued each time a class k custome eneges fom queue. The cost function we conside is the total diffusion-scaled holding and eneging costs duing a finite time inteval. Befoe we can define the cost pocess, we need to intoduce some notation. Let Q k t denote the numbe of class k customes in queue at time t and Rk t denote the numbe of class k customes who have abandoned the system by time t. We define the diffusion scaling fo these pocesses by ˆQ k t = Q k t and ˆR k t = R k t fo t and k K. 2.4 The cost pocess ζ t is defined by ζ t = 2 k=1 h k ˆQ k sds + c k ˆR k t fo t. Ou objective is to find a outing policy that stochastically minimizes ζ T as fo any fixed T >. In geneal, a outing policy specifies how seves ae allocated to customes dynamically. Fo the est of this pape we use π to denote a geneic policy. The evolution of an N-system natually depends on the policy employed. Wheneve we need to make this dependence explicit, we append π to pocesses associated with the N-system. Optimal policies fo N-systems depend on system paametes. We assume fo the most pat of the pape that h 1 + c 1 γ 1 h 2 + c 2 γ 2 and γ 1 γ 2 µ 1 µ Othe cases ae handled in Section 7. Note that each custome in queue k incus a total cost of h k + c k γ k pe unit of time, k = 1, 2. Assumption 2.5 says that class 1 customes ae moe expensive to hold than class 2 customes, and they ae seved faste by seves in pool 2. Also, it implies that class 1 customes ae moe patient than class 2 customes. This latte assumption is in accodance with the empiical findings of Bown et al. 25. We note that, unde assumption 2.5, π dictates that class 1 customes have pioity ove class 2 customes and the second seve pool have pioity ove the fist seve pool. We estict ou attention to the class of admissible policies. A policy is said to be admissible if it is non-peemptive, head-of-the-line HL, and has the Makovian stuctue descibed below. Unde a non-peemptive policy, once the sevice of a custome stats it can not be inteupted befoe it is finished. A policy is said to be head-of-the-line if each seve can only seve one custome at any given time and the customes in the same queue ae seved on a Fist-in-Fist-out FIFO basis. Unde an admissible policy, the assignments of seves to customes can only be made at the time

9 J. G. Dai and T. Tezcan/Dynamic Contol of N-Systems 9 of a custome depatue o aival. Let Zjk t denote the numbe of class k customes in sevice in pool j at time t fo k K and j J k. Fo an event time t, unde an admissible policy, the new seve allocations at that instant ae made by only using the state of Zjk t and Q k t, fo k K and j J k. We equie that a policy has this Makovian stuctue fo technical easons elaboated in Remak 4.1. A detailed desciption of admissible policies can be found in Dai and Tezcan 25. The only diffeence between ou definition of an admissible policy and that in Dai and Tezcan 25 is that we allow an admissible policy to be idling; i.e., it may allow seves idle even when thee ae customes waiting in the queue that they can handle. In ode fo ζ,π to have a meaningful limit, the sequence of initial states must satisfy cetain conditions. Fist, it is necessay that Q /, Z / Q, Z a.s. 2.6 as, whee Q =, and Z = β 1, x 21 β 2, x 22 β 2. This condition implies that at time zeo the diffeence between the seve allocations and nominal allocations is o. Also, the sequence { ˆQ, Ẑ } must satisfy the following condition as fo a andom vecto ˆQ, Ẑ. ˆQ, Ẑ ˆQ, Ẑ 2.7 Remak 2.1. Condition 2.7 can be elaxed to assume that { ˆQ, Ẑ } is tight. In this case the asymptotic pefomance of two policies may be compaed along subsequences that { ˆQ, Ẑ } is convegent. Using ou main esults, one can show that π asymptotically minimizes the cost along each such subsequence. As discussed in Section 1, we assume that sevice times ae exponential distibuted. The aival pocess fo class k customes Ek is defined by m Ek t = sup{m : u k l λ k t} fo k K, whee {u k l : l = 1, 2,...} is a sequence of i.i.d. nonnegative andom vaiables with mean 1 and vaiance σ 2 k [, and u k s ae abitay nonnegative andom vaiables. By convention, empty sums ae set to be zeo. l= 3. Main esults Fist, we intoduce a pocess that we use to establish a lowe bound fo the total cost. We define the lowe bound map ψ a as follows that will be used below to build this lowe bound pocess. Fo a = a 1, a 2 R 2, we define ψ a : D[, D[, by whee ψ a x = y 3.1 yt = wt + a 1 ys ds a 2 ys + ds, 3.2 w D[,. Next, we establish the basic popeties of ψ a.

10 J. G. Dai and T. Tezcan/Dynamic Contol of N-Systems 1 Lemma 3.1. Fo each w D[, and a R 2 thee exists a unique y that satisfies 3.2. Futhemoe, ψ a is Lipschitz continuous on D[, with espect to the sup nom on finite intevals. The poof is placed in Appendix A and is simila to that of Theoem 11.4 in Mandelbaum et al Set ψ ψ µ1,γ 2. We define the pocess Y by whee ξ is a Bownian motion with dift θ, vaiance and initial distibution Let Y = ψξ, λ k σk 2 + λ k 3.4 k=1 ˆQ 1 + ˆQ 2 + Ẑ11 + Ẑ21 + Ẑ ζ t = h 2 + c 2 γ 2 Y s + ds. 3.6 The following esult shows that ζ povides a lowe bound fo the cumulative holding costs unde any admissible policy as. Theoem 3.2. Let π be an admissible policy and {X,π } be a sequence of N-systems woking unde π. Assume that 2.5 and hold. Then fo any T > and x > lim inf P{ζ,π T > x} P{ζ T > x}. 3.7 With Theoem 3.2 the following esult shows that π is asymptotically optimal. Theoem 3.3. Let {X,π } be a sequence of N-systems woking unde policy π. Assume that 2.5 and hold. Then, fo any T > and x > 4. Peliminaies and an outline of the poofs lim P{ζ,π T > x} = P{ζ T > x}. 3.8 In ode to ease the exposition, we fist povide the notation and basic popeties of N-systems that will be used extensively in the est of this pape. We then pesent the outline of the poofs Queueing model Recall that Ek t denotes the total numbe of class k aivals in the th system by time t. Let A k t denote the total numbe of customes who have enteed queue k by time t in the th system; a custome who eceives sevice without any wait does not ente a queue. We use A jk t to denote the total numbe of class k customes who stated eceiving sevice fom a seve in pool j without

11 J. G. Dai and T. Tezcan/Dynamic Contol of N-Systems 11 any wait by time t, and we use Bjk t to denote the total numbe of class k customes who have depated queue k and have commenced sevice at a seve in pool j by time t. Let Tjk t denote the total time spent seving class k customes by all N j seves of pool j in the th system until time t. Recall that we use Zjk t to denote the total numbe of class k customes that ae being seved by a seve in pool j and Q k t to denote the total numbe of customes in queue k at time t in the th system. Note that Fo notational simplicity we define Tjk t = Zjk sds. G k t = Q k sds, the total time spent in queue by class k customes until time t. We denote the total numbe of class k customes in the system at time t by Xk t and the total numbe of customes in the system at time t by Y t. Thus, X 1t = Q 1t + Z 11t + Z 21t, X 2t = Q 2t + Z 22t and Y t = X 1t + X 2t. Let S jk and F k be Poisson pocesses with ates µ jk and γ k, espectively, fo k K and j J k. Unde an admissible policy, the numbe of class k sevice completions in seve pool j and numbe of class k customes who abandoned the system until time t can be expessed as S jk T jk t and F k Q k sds, espectively see Remak 4.1 below. We efe Ek, F k, S jk, : k K, j J k as the pimitive pocesses fo the th system. Set X = A k, A jk, B jk, G k, T jk, Q k, X k, Y, Zjk : k K, j J k; it is efeed to as the pefomance pocess fo the th system. Wheeas the pimitive pocesses do not depend on policy π used fo the th system, the pefomance pocess X does depend on policy π. We use X,π to explicitly denote such dependency. We dop the subscipt π when the policy π is clea fom the context. Fo a given policy π, obseve that each component of A,π k, A,π jk, B,π jk, G,π k,,π jk is nondeceasing and each component of Q,π k, X,π k equations in Dai and Tezcan 25 that X,π π Ek t = A,π k t + j J k t = Q,π k + A,π k t Q,π k G,π k t = Z,π jk,π t = Z,π jk jk t = Q,π k, Y,π, Z,π jk is nonnegative. It follows fom satisfies the following equations: A,π jk t, fo k = 1, 2, 4.1 j J k B,π jk t F k G,π k t, fo k = 1, 2, 4.2 sds, k = 1, 2, A,π jk t + B,π jk t S jk,π jk t, fo k = 1, 2 and j J k, 4.4 Z,π jk sds, k = 1, 2 and j J k, 4.5 Additional equations associated with policy π. 4.6

12 J. G. Dai and T. Tezcan/Dynamic Contol of N-Systems 12 Remak 4.1. As discussed in Dai and Tezcan 25, the pocess X,π, which is called the petubed system thee, is pathwise diffeent fom the pefomance pocess of the undelying N-system descibed in Section 2. If π is an admissible policy it can be shown, simila to Theoem 2.1 in Tezcan 26, that X,π and the pefomance pocess associated with the N-system have the same distibution; see also Dai and Tezcan 25 fo moe details. The class of admissible policies can be extended to cove all policies unde which these two systems have the same distibution. Theoems 3.2 and 3.3 show that π is actually optimal among all policies in this extended class. Fo futue efeence, we define the fluid-scaled pimitive pocesses Ē k t = E k t/, F k t = F k t/, S jk t = S jk t/ and the fluid-scaled pefomance pocess X,π = Ā,π k, Ā,π,π jk, B k K, j J k by jk, Ḡ,π k, T,π jk, Q,π k, X,π, Ȳ,π, k Z,π jk : Ā,π k Ḡ,π k X,π k t = A,π k t/, Ā,π jk t = A,π jk t/, B,π jk t = B,π jk t/, t = G,π k t/ N,π,π, T jk t = Tjk t/, Q,π k t = Q,π k t/, t = X,π k t/, Ȳ,π t = Y,π t/, Z,π jk t = Z,π jk t/ fo k K and j J k. Define the diffusion-scaled pimitive pocesses by Êk t = Ēk t λ k t, ˆF k t = F jk t γ k t, Ŝjk t = S jk t µ jk t, and the diffusion scaling fo some components of the pefomance pocess by Ĝ,π k t = G,π k t/ N,π, ˆT jk t = N,π x jk β j t ˆT jk t, ˆQ,π k Ẑ,π jk t = Z,π jk t β jx jk ˆX,π 2 t = X,π 2 t β 2x 22 fo k K and j J k. It follows fom 4.2, 4.4 and the fact that t = Q,π k t/, ˆX,π 1 t = X,π 1 t β 1 + β 2 x 21 and Ŷ,π t = Ȳ,π t β 1 + β Ŷ = ˆX 1 + ˆX whee Ŷ,π t = ξ,π t µ 1 γ 1 Ẑ,π 11 sds µ 2 ˆQ,π 1 sds γ 2 Ẑ,π 21 s + Ẑ,π 22 s ds ˆQ,π 2 sds. 4.1 ξ,π t = Ê 1t + Ê 2t Ŝ 11 T,π 11 t Ŝ 21 Ŝ 22 T,π 22 t ˆF 1 Ḡ,π 1 t ˆF 2 T,π 21 t Ḡ,π 2 t + θt. 4.11

13 4.2. Lowe bound J. G. Dai and T. Tezcan/Dynamic Contol of N-Systems 13 It will be agued in Section 6 that fo each T and x > { lim inf P {ζ,π T > x} P h 2 + c 2 γ 2 lim inf T } Ŷ,π s + ds > x. Thus, to deive a lowe bound on the total cost ζ,π T, it is sufficient to deive a lowe bound fo Ŷ,π, the scaled total numbe of customes in the system. Obseve fom 4.8 and 4.9 that Ẑ,π 11 sds + Ẑ,π 21 s + Ẑ,π 22 s ds + ˆQ,π 1 sds + It follows fom Lemma 4.2 at the end of this section that on each sample path, Ŷ,π t ψξ,π t, fo t, whee ψ = ψ a with a = µ 1, γ 2. When policy π has the following fluid limit: fo evey t >, ˆQ,π 2 sds = Ŷ,π s ds.,π 11 t β 1t, T,π 21 t β 2x 21 t, T,π 22 t β 2x 22 t 4.12 almost suely as, ξ,π ξ as, whee ξ, defined as in 3.3, does not depend on π. Theefoe, fo each t and x >, } lim inf {Ŷ P,π t > x P {ψξ t > x}. When 4.12 is not satisfied, it will be agued in Section 5 that ζ,π T in some appopiate sense as. Thus, ψξ seves as an asymptotic lowe bound fo Ŷ,π as. Hence, we have fo each t > and x >. lim inf P {ζ,π T > x} P { h 2 + c 2 γ 2 T } ψξ t + dt > x = P {ζ T > x} The complete, igoous lowe bound analysis will be caied out in Section Optimality To pove that π is asymptotically optimal, it suffices to pove that 4.13 becomes an equality unde π. We fist show that 4.12 holds. Thus, as, ξ,π ξ. In Section 6, we study a fluid model fo the sequence of N-models. The fluid model is defined by a set of fluid model equations; these equations ae satisfied by each fluid limit, a limit point of X,π as. We pove 4.12 in Section 6 by poving that initial state of the fluid model given in 2.6 is an invaiant state of the fluid limit. Next, it follows fom 4.1 that Ŷ,π t = ξ,π t µ 1 Ŷ,π s ds γ 2 Ŷ,π s + ds + ɛ t 4.14

14 whee J. G. Dai and T. Tezcan/Dynamic Contol of N-Systems 14 ] ɛ t = µ 1 [Ŷ,π s ds Ẑ,π 11 s ds µ 2 γ 1 ˆQ,π 1 sds + γ 2 [ Ŷ,π s + Ẑ,π 21 s + Ẑ,π 22 s ds ˆQ,π 2 s ] ds. Thus, Ŷ,π = ψξ,π + ɛ fo each >. Assume that the following state space collapse esults hold unde policy π : as,,π Q 1, Ẑ,π 21 + Ẑ,π 22 T, T Ẑ,π 11 Ŷ +,π T It follows that ɛ as. Thus, Ŷ,π = ψξ,π + ɛ ψξ = Y, 4.16 as. Also, it will be poved in Section 6 that T + ζ,π T h 2 + c 2 γ 2 Y t,π dt as. Theefoe, ζ,π T ζ T as, poving that the total cost achieves the lowe bound unde policy π. The SSC esults 4.15 do not necessaily hold at time zeo because of assumption 2.7. Howeve, unde assumption 2.7, thee exists a sequence L = o such that 1 + L / N, T Q,π Ẑ,π 21 + L / N + Ẑ,π Ẑ,π 11 + L / N,π + Ŷ + L / N T as. We show in Section 6 that 4.17 and 4.15 imply ζ,π T ζ T 22 + L / N, T 4.17 as. We use the famewok in Dai and Tezcan 25 to pove The famewok consists of thee main steps. The fist step is to show that 4.12 holds unde π. This is poved in Poposition 6.2. We next pove that a hydodynamic model has the analogous SSC esult the hydodynamic model is intoduced in Section 6.3. By Theoem 5.4 in Dai and Tezcan 25 this implies the following multiplicative SSC esults hold fo diffusion-scaled pocesses;,π Q 1 + L / T B T L, Ẑ,π 21 + L / + Ẑ,π 22 + L / T B T L, 4.18 Ẑ,π 11 + L / + Ŷ,π + L / N T B T L

15 as, whee J. G. Dai and T. Tezcan/Dynamic Contol of N-Systems 15 B T L = sup L / t T t ˆQ Ẑ t 1. Finally, in the thid step, we pove that { ˆQ,π } and {Ẑ,π } ae stochastically bounded as in Section 6.4. Thus, the multiplicative SSC esults 4.18 imply the SSC esults We end this section with a lemma that is citical in poving the lowe bound. Lemma 4.2. Let b = b 1, b 2 R 2 and d = d 1, d 2 R 2 and b, d, satisfying the following condition max{d 1, d 2 } min{b 1, b 2 } Let w D[, be fixed. Assume that ỹ, u 1, u 2, v 1, v 2 D[, C 4 [, satisfies Then, ỹt = wt + b 1 u 1 t + b 2 u 2 t d 1 v 1 t d 2 v 2 t, t, 4.2 u i and v i ae nondeceasing and u i = and v i =, i = 1, 2, 4.21 u i is absolutely continuous, 4.22 v 1 s : t + v 2 s : t u 1 s : t u 2 s : t = s ỹsds, s < t ψ a xt ỹt, fo all t < 4.24 whee The poof is pesented in Appendix A. a = min{b 1, b 2 }, max{d 1, d 2 } Lowe bound In this section we pove Theoem 3.2. Fo the est of this pape, we set a = µ 1, γ 2 and define ψ a as in 3.1. Fo notational simplicity we omit a fom the notation and use ψ to denote this map. Fix an admissible policy π. Note that by ou definition of an admissible policy Z,π jk D[, a.s. fo each. Also Ẑ,π 11 t and Ẑ,π 21 t + Ẑ,π 22 t ae the diffusion-scaled numbe of idle agent,π,π,π in pools 1 and 2, espectively. Theefoe, ˆT 11 t and ˆT 21 t + ˆT 22 t give the diffusion-scaled total idle time expeienced by the seves in the fist and second seve pools, espectively. Obseve that and Ĝ,π satisfy the following conditions; fo all s t ˆT,π jk k N 1 t s/ N 2 t s/ Ĝ,π 1 ˆT,π 11 ˆT,π 21,π,π s : t ˆT 11 s : t ˆT 21 s : t = s Ĝ,π,π 2 s : t ˆT 22 s : t = s s : t, 5.1,π s : t + ˆT 22 s : t, 5.2 ˆX,π 1 udu, 5.3 ˆX,π 2 udu, 5.4

16 J. G. Dai and T. Tezcan/Dynamic Contol of N-Systems 16 The idea of the poof fo Theoem 3.2 is simila to that of Poposition 2 in Ata and Kuma 25. In the fist step we constuct the stochastic pocesses involved in ou system in a new pobability space by eplacing weak convegence of pimitive pocesses with a.s. convegence using the Skoohod epesentation theoem; see fo example Ethie and Kutz Then, we divide the sample paths into two sets; the fist set includes those sample paths that satisfy 4.12 and the second set includes the est of the sample paths. Fo the sample paths in the fist set we use Lemma 4.2 to show that ζ povides a lowe bound. Fo all the othe sample paths we show that the cost goes to infinity as. Fix T > and x >. Choose a subsequence l such that lim l P{ζl,π T > x} = lim inf P{ζ,π T > x} 1. Constuction of new pocesses Let {,π } be the sequence of allocation pocesses and {G,π } be the sequence of cumulative time in queue pocesses unde policy π. By Theoem A.1 in Dai and Tezcan 25 and Theoem 4.4 in Billingsley 1968, {Ê l, Ŝ l, ˆF l, l,π, Ḡ l,π } is tight and any weak limit of this sequence has continuous paths almost suely. In paticula, the limit is of the following fom Ê, Ŝ, ˆF, T π, Ḡπ, 5.5 whee Ê, Ŝ, and ˆF ae diftless Bownian motions of appopiate dimension, T π is a nondeceasing pocess with T π t T π s t se, fo s t a.s. fo all t, whee e = 1, 1, 1, and Ḡπ is absolutely continuous a.s. Let {,π, Ḡ,π } be a futhe subsequence of { l,π, Ḡ l,π } which conveges weakly to a limit as in 5.5. By appealing to the Skoohod epesentation theoem, we may choose an equivalent distibutional epesentation which we will denote by putting a above the symbols such that the sequence of andom pocesses Ê,π, Ŝ,π, ˆF,π,,π, Ḡ,π, as well as the limit Ê, Ŝ, ˆF, T π, Ḡ π ae defined on a new pobability space, say Ê,π, Ŝ,π, ˆF,π,,π, Ḡ,π Ω, F, P, so that P -a.s. Ê, Ŝ, ˆF, T π, Ḡ π 5.6 u.o.c. as. Fo the est of this poof we focus on the sample paths in Ω that satisfy 5.6.

17 J. G. Dai and T. Tezcan/Dynamic Contol of N-Systems 17 We can also assume without loss of geneality that thee exists a sequence of andom vectos in Q, Z in this new space that ae independent fom all stochastic pocesses in 5.6 such that Q, Z has the same distibution with Q, Z. In addition, Q /, Z / ˆQ, Ẑ a.s. whee ˆQ, Ẑ has the same distibution with ˆQ, Ẑ and independent of all the pocesses defined above on Ω. We define the following pocesses on this new pobability space: whee and Ē,π k t = 1 Ê,π k t + λ k t 5.7 S,π jk t = 1 Ŝ,π N jk t + µ jkt. 5.8 F,π k t = ˆ,π jk t = 1 ˆF,π N k t + γ k t. 5.9,π jk t + β ix jk t 5.1 Ĝ,π k t = Ḡ,π k t 5.11 ˆX,π 1 t = ˆX 1 + Ê,π 1 t Ŝ,π 11,π 11 t Ŝ,π 21,π 21 t ˆF,π 1 Ḡ,π 1 t + µ 1 ˆT,π 11 t + µ ˆ,π 2 21 t γ Ĝ 1,π 1 t 5.12 ˆX,π 2 t = ˆX 2 + Ê,π 2 t Ŝ,π 22,π 22 t ˆF,π 2 Ḡ,π 2 t + µ 2 ˆT,π 22 t γ Ĝ 2,π 2 t, 5.13 ˆX 1 = ˆQ 1 + Ẑ 11 + Ẑ 21, ˆX 2 = ˆQ 2 + Ẑ 22 ˆQ k = Q k / and Ẑ jk = Z jk x jk β j We note that pocesses defined by have the same joint distibution as the coesponding scaled pocesses in the oiginal pobability space fo each. Also, since ˆX,π,π 1, ˆX 2, ˆT,π, Ĝ,π and ˆX,π 1, ˆX,π 2,,π, Ḡ,π ae equal in distibution, we have P -a.s. and and,π is nondeceasing 5.14,π s : t t se fo s t, 5.15 Ḡ,π is continuous, nonnegative and inceasing 5.16,π = Ḡ,π =. 5.17

18 J. G. Dai and T. Tezcan/Dynamic Contol of N-Systems 18 Since ˆ,π and Ĝ,π satisfy ,,π and Ḡ,π satisfy the following equations. N1 t s/ ˆ,π 11 s : t, 5.18 N2 t s/ ˆ,π 21 s : t + ˆ,π 22 s : t, 5.19 Ĝ,π 1 s : t ˆ,π 11 s : t ˆ 21 s : t = Ĝ,π 2 s : t ˆ,π 22 s : t = s s ˆX,π 1 udu, 5.2 ˆX,π 2 udu We have by FSLLN that S,π, Ē,π, F,π µ, λ, γ 5.22 u.o.c. P -a.s., whee fo all t. We now define X,π 1 t = µt = µ 1 t, µ 2 t, µ 2 t, λt = λ 1 t, λ 2 t, and γt = γ 1 t, γ 2 t 1 N ˆX,π 1 t + β 1 + x 21 β 2 = X 1 + Ē,π 1 t S,π 11 T,π 11 t S,π 21 λ 1 t N + β 1µ 1 + x 21 β 2 µ 2 ˆX,π 2 t + x 22 β 2 T,π 21 t F,π 1 Ḡ,π 1 t X,π 2 t = 1 = X 2 + Ē,π 2 t S,π 22 T,π 22 t F,π 2 Ḡ,π 2 t λ 2 t N + x 22β 2 µ 2 Obseve that this scaling coesponds to the fluid scaling in the oiginal pobability space. By 5.6, 5.22 and Lemma 11 in Ata and Kuma 25 u.o.c. P -a.s., whee We note that P -a.s. by 5.18 and Also, by 2.7 X,π i Xπ i X 1 π t = X1 π π + λ 1 t µ 1 T 11 t µ 2 T 21 t γ 1 Ḡ π 1 t 5.23 X 2 π t = X2 π + λ 2 t µ 2 T 22 t γ 2 Ḡ π 2 t T π 11t β 1 t and T π 21 t + T π 22 t β 2 t 5.25 X 1 = β 1 + β 2 x 21 and X2 = β 2 x

19 P -a.s. Let J. G. Dai and T. Tezcan/Dynamic Contol of N-Systems 19 ξ,π t = Ŷ + Ê,π 1 t Ŝ,π 11,π 11 t Ŝ,π 21,π 21 t ˆF,π 1 Ḡ,π 1 t + ˆX 2 Ỹ,π t = ˆX,π 1 t + ˆX,π 2 t + Ê,π 2 t Ŝ,π 22,π 22 t ˆF,π 2 Ḡ,π 2 t, 5.27 = ξ,π t + µ 1 ˆT,π 11 t + µ 2 ˆT,π 21 t + ˆ,π 22 t γ 1 Ĝ,π 1 t γ 2 Ĝ,π 2 t 5.28 and and Define Ỹ by T 11 t = β 1 t, T22 t = x 21 β 2 t, T22 t = x 22 β 2 t, ξt = Ŷ + Ê 1 t Ŝ 11 T11 t Ŝ 21 T21 t + Ê 2 t Ŝ 22 T22 t. ζ t = Ỹ = ψ ξ h 2 + c 2 γ 2 Ỹ s + ds, t. Clealy, ζ has the same distibution with ζ defined by 3.6. Remak 5.1. We have by 5.2 and 5.21 that Ĝ,π,π 1 t + Ĝ 2 t ˆ,π 11 t ˆ 21 t ˆT,π 22 t = Ŷ,π udu Since ˆ,π 11 t and ˆ 21 t + ˆ,π 22 t ae Lipschitz continuous by 5.15 and theefoe absolutely continuous, one can show simila to A.3 that P -a.s. This with 5.29 implies that P -a.s. ˆ,π 11 t + ˆ 21 t + ˆ,π 22 t Ŷ,π u du + Ĝ,π,π 1 t + Ĝ 2 Ŷ t,π u du Sample Path Analysis Next, we divide the sample paths into two sets based on thei fluid limits. Define { V T = ω Ω : Ḡ π 1 T = Ḡ } π 2 T =. Hee and fo the est of this poof Ω is used to denote good sample paths that satisfy 5.6, and 5.3. We claim that fo ω V T and t T T π 11t = β 1 t, T π 21 t = x 21 β 2 t, T π 22 t = x 22 β 2 t. 5.31

20 To see this, obseve that by 5.23 and 5.23, Then, by 5.2 and 5.21, J. G. Dai and T. Tezcan/Dynamic Contol of N-Systems 2 T π 21t + T π 22 t = β 2 t and T π 11 t = β 1 t. T π 21t = x 21 β 2 t and T π 22 t = x 22 β 2 t on V T. Next, we analyze the sample paths on V T. Lemma 5.2. Fix T >. Fo all ω V T fo all t [, T ]. lim inf Poof. We use Lemma 4.2 to pove the esult. Fix and let u 1 = ˆ,π 11, u 2 = Ỹ,π t Ỹ t 5.32 ˆT,π 21 + ˆ,π 22 and 5.33 v 1 = Ĝ,π 1, v 2 = Ĝ,π Next, we check that the conditions of Lemma 4.2 ae satisfied by Ỹ,π, u i, v i ; i = 1, 2, whee Ỹ,π is defined by By 5.14, 5.15, 5.16 and 5.17, condition 4.21 holds. Also, by 5.15, u 1 and u 1 ae Lipschitz continuous, theefoe they ae absolutely continuous. By 5.28, 4.2 holds. By 5.29, 4.23 holds. Theefoe, by Lemma 4.2 fo. By 5.31 Ỹ,π ψ ξ,π, 5.35 ξ,π ξt 5.36 u.o.c. fo all ω V T. Theefoe, by continuity of the mapping ψ see Lemma 3.1, 5.35 and 5.36 imply that fo all t [, T ]. lim inf Ỹ,π t Ỹ t, Poof of Theoem 3.2. Let π be an admissible policy and {X,π } be a sequence of N-systems woking unde π. Assume that hold. Fix x >. Note that 2 T ζ,π T = h i ˆQ,π i sds + 1 c ifi G,π i T = = i=1 2 i=1 2 i=1 T h i + c i γ i h i + c i γ i Ĝ,π i T + ˆQ,π i sds + 2 i=1 2 i=1 c i ˆF i Ḡ,π i T c i ˆF i Ḡ,π i T 5.37

21 Let J. G. Dai and T. Tezcan/Dynamic Contol of N-Systems 21 η,π T = 2 h i + c i γ i i=1 Ĝ,π i T + 2 i=1 c i ˆF i Ḡ,π i 5.38 By 5.37 and fom the constuction of the pocesses on Ω { } P ζ,π T > x = P { } η,π T > x Define We have by 5.3 that T + ζ,π T = h 2 + c 2 γ 2 Ỹ,π s ds. η,π T ζ,π T + 2 i=1 c i ˆF i Ḡ,π i T 5.4 Obseve that on V T, as ˆF i Ḡ,π i T 5.41 by 5.22, since Ḡ,π i as u.o.c. Theefoe, fo all ω V T lim inf,π η T lim inf ζ,π T ζ T, 5.42 whee the fist inequality follows fom 5.4 and 5.41, and the second inequality follows fom Lemma 5.2. Fo all ω V c T, lim inf η,π T = 5.43 since lim inf 2 i=1 Ĝ,π i T = lim inf 2 N Ḡ,π i i=1 T = on V c T and by 5.6 and Lemma 11 in Ata and Kuma 25 lim inf 2 i=1 c i ˆF i Ḡ,π i T < P -a.s.

22 Theefoe J. G. Dai and T. Tezcan/Dynamic Contol of N-Systems 22 { } lim P ζ,π T > x { lim P = lim a Ẽ [ 1 } η,π T > x { η,π T > x }] Ẽ [ { lim inf 1 η,π T > x b P { ζ T > x }, } ] whee a follows fom Fatou s lemma and b follows fom 5.42 and The gives the desied esult since ζ T has the same distibution with ζ T. 6. Optimality of π In this section we pove Theoem 3.3. In Section 6.1, we fist establish additional queueing equations that must me satisfied by N-systems unde π in addition to These equations ae used in the subsequent sections in detemining fluid and hydodynamic model equations of π. In Section 6.2, we focus on fluid limits and show that 4.12 holds unde π. In Section 6.3 we pove multiplicative SSC esults given as in 4.18 and then in Section 6.4 we pove the SSC esults given as in Finally, in Section 6.5 we pove Theoem 3.3. A caeful eview of ou poof of 4.17 would eveal that assumption µ 2 = µ 3 is not used. Hence, the SSC esults in 4.17 still hold unde π even if µ 2 µ Policy dependent queueing equations fo π Recall that equations in Section 4 ae satisfied by N-systems unde any admissible policy. In this section, we pesent the additional equations that must be satisfied unde π in addition to these equations. We fist note that π is non-idling so X also satisfies the following non-idling condition Q k t N j Zjl t =, fo all k K, 6.1 j J k l Kj Recall that unde π class 1 customes have pioity ove class 2 customes in the second seve pool. Hence, a class 2 custome in the queue can stat eceiving sevice if thee ae no class 1 customes waiting in the queue fo sevice. Recall that Bjk t denote the numbe of class k customes who stated thei sevice in seve pool j befoe time t afte waiting in the queue. Theefoe B 22t can only incease when Q 1t =. 6.2 Since the second seve pool has pioity ove the fist seve pool, an aiving custome will stat his sevice in the fist seve pool only when all the seves in the second pool ae busy. Recall that

23 J. G. Dai and T. Tezcan/Dynamic Contol of N-Systems 23 A jk t denote the numbe of class k customes whose sevice stated immediately at the time of thei aival in seve pool j by time t. Hence A 11t and B 11t can only incease when Z 21t + Z 22t = N If thee ae class 2 customes waiting in the queue to eceive sevice then all the seves in the second pool should be busy at that instant since policy π is non-idling. It can be shown as in Lemma 2 in Appendix B of Tezcan 26 that the pobability that a class 1 aival and a sevice completion fom seve pool 2 take place at the same time instant is zeo. Fo the est of this pape we only conside sample paths that satisfy this condition. Theefoe, when a class 1 custome aives to the system at an instant when thee ae class 2 customes waiting in the queue, his sevice cannot stat in the second pool immediately. Hence, fo s < t If Q 2τ > fo all τ [s, t], then A 21t A 21s = a.s Fluid model of N-systems unde π The pupose of this section is to show that 4.12 holds unde π. We show that the limit of the initial states, given by 2.6, of fluid-scaled pocesses Q, Z is an invaiant state of the fluid model of π and then invoke Lemma A.3 in Dai and Tezcan 25 to complete the poof. In ode to study the popeties of fluid limits, we need to detemine the fluid model equations fo π. As it is the case fo the queueing model equations , some of the fluid model equations depend on the policy employed. Ou fist goal is to establish the additional fluid model equations associated with π. These equations ae then used to pove Poposition 6.2. Fluid limits of paallel seve systems in many-seve heavy taffic egime ae studied in Dai and Tezcan 25. In this section we epeat some of the definitions and basic esults fom that pape that will be used in ou analysis. We fist intoduce the fluid scaling and define fluid limits. Then, we pesent the fluid model equations that ae satisfied by all the fluid limits. Recall that the fluid scaling X is defined by X = X /. The pocess X is called a fluid limit of {X } if thee exists a sequence l, with l as l, and ω A such that X conveges u.o.c. to X, whee A is taken as in Theoem A.1 in Dai and Tezcan 25 and P A = 1. In Dai and Tezcan 25, the existence of fluid limits when { Q } is bounded a.s. is poved. It is also shown that they ae absolutely continuous and satisfy the following fluid model equations fo t ; λ k t = Ākt + Ā jk t, fo k = 1, 2, 6.5 j J k Q k t = Q k + Ākt Ḡ k t = j J k B jk t γ k Ḡ k t, fo k = 1, 2, 6.6 Q k sds, fo k = 1, 2, 6.7 Z jk t = Z jk + Ājkt + B jk t µ jk Tjk t, fo k = 1, 2 and j J k, 6.8 T jk t = Z jk sds fo k = 1, 2 and j J k, 6.9