arxiv: v2 [math.pr] 14 Jun 2016

Transcription

1 On the Contol of Fok-Join Netwoks Ehun Özkan1 and Amy R. Wad 2 axiv: v2 [math.pr] 14 Jun 2016 Abstact Netwoks in which the pocessing of jobs occus both sequentially and in paallel ae pevalent in many application domains, such as compute systems, healthcae, manufactuing, and poject management. The paallel pocessing of jobs gives ise to synchonization constaints that can be a main eason fo job delay. In compaison with feedfowad queueing netwoks that have only sequential pocessing of jobs, the appoximation and contol of netwoks that have synchonization constaints is less undestood. One well-known modeling famewok in which synchonization constaints ae pominent is the fok-join pocessing netwok. Ou objective is to find scheduling ules fo fok-join pocessing netwoks with multiple job types in which thee is fist a fok opeation, then activities that can be pefomed in paallel, and then a join opeation. The difficulty is that some of the activities that can be pefomed in paallel equie a shaed esouce. We solve the scheduling poblem fo that shaed seve that is, which type of job to pioitize each time the seve becomes available when that seve is in heavy taffic and pove an asymptotic optimality esult. Keywods: Fok-join pocessing netwok; Scheduling Contol; Diffusion Appoximation; Asymptotic Optimality. AMS Classification: Pimay 60K25, 90B22, 90B36, 93E20; Seconday 60F17, 60J70. 1 Intoduction Netwoks in which pocessing of jobs occus both sequentially and in paallel ae pevalent in many application domains, such as compute systems Xia et al. [2007], healthcae Amony et al. [2015], manufactuing Dalley and Geshwin [1992], poject management Adle et al. [1995], and the justice system Lason et al. [1993]. The paallel pocessing of jobs gives ise to synchonization constaints that can be a main eason fo job delay. Although delays in fok-join netwoks can be appoximated unde the common fist-come-fist-seved FCFS scheduling discipline Nguyen [1993, 1994], thee is no eason to believe FCFS scheduling minimizes delay. Ou objective in this pape is to devise contol policies that minimize delay o, moe geneally, holding costs in fok-join netwoks with multiple custome classes that shae pocessing esouces. Fo a concete motivating example, conside the patient-flow pocess associated with the emegency depatment at Saintemaie Univesity Hospital cf. Hublet et al. [2011]. An aiving patient is fist tiaged to detemine condition seveity, and then afte some potential waiting moves to the patient management phase befoe being dischaged. The patient management phase begins with the vital signs being taken and a fist evaluation. Then, depending on the condition, thee may be laboatoy tests and adiology exams. Simple laboatoy tests on the patient s blood and uine can be pefomed in paallel with the patient eceiving a adiology exam, such as a CT scan. The dischage decision - whethe the patient can etun home o should be admitted to the hospital - cannot be made until all test esults ae eceived. Roughly speaking, we can imagine a pocess flow diagam such as that in Figue 1, whee the patient type coesponds to the condition seveity detemined at tiage, the isolated opeations coespond to the laboatoy tests which ae necessaily associated with each individual patient, and the shaed opeation coesponds to the use of the CT scanne. The CT scanne is an expensive machine, and, as can be seen fom the case teaching note cf. Hublet et al. [2011], has a lage impact on patient wait time. This motivates us to study the poblem of how to schedule a shaed esouce that is used in paallel with othe esouces. 1 Mashall School of Business, Univesity of Southen Califonia, Ehun.Ozkan.2018@mashall.usc.edu. 2 Mashall School of Business, Univesity of Southen Califonia, amywad@mashall.usc.edu. 1

2 !"#$%! &'!"#$%" &'!"#$%#&' Figue 1: Colo online An example of a fok-join pocessing netwok with abitay numbe of job types, abitay numbe of foks associated with each job type, and a single shaed seve. The simple fok-join netwok shown in Figue 2 seves to illustate why fok-join netwok contol is difficult. In that netwok, thee ae two aiving job types a and b, seven seves numbeed 1 to 7, and ten buffes numbeed 1 to 10. We assume h a is the cost pe unit time to hold a type a job, and h b to hold a type b job. Type a b jobs ae fist pocessed at seve 1 2, then fok into two jobs, one that must be pocessed at seve 3 5 and the othe at seve 4, and finally join togethe to complete thei pocessing at seve 67. Thee is synchonization because the pocessing at seve 6 7 cannot begin until thee is at least one job in both buffes 7 and 8 9 and 10. Seve 4 pocesses both job types, but can only seve one job at a time. The contol decision is to decide which job type seve 4 should pioitize. This decision could be essentially ignoed by seving the jobs in the ode of thei aival egadless of type that is, implementing FCFS policy. Anothe option is to always pioitizing the moe expensive job type, in accodance with the well-known cµ-ule. Then, if h a µ A h b µ B whee µ A µ B is the ate at which seve 4 pocesses type a b jobs, seve 4 always pefes to wok on a type a job ove a type b job. Howeve, when thee ae multiple jobs waiting at buffes 8 and 10, and no jobs waiting at buffe 9, it may be pefeable to have seve 4 wok on a type b job instead of a type a job and especially if also no jobs ae waiting at buffe 7. This is because seve 4 can pevent the join seve 7 fom idling without being the cause of seve 6 s foced idling. Seve 3 is the cause. The fok-join netwok contol poblem is too difficult to solve exactly, and so we seach fo an asymptotic solution. We do this unde the assumption that seve 4 is in heavy taffic. Othewise, the scheduling contol in seve 4 has negligible impact on the delay of type a and type b jobs. We futhe assume the seves 6 and 7 ae in light taffic, which focuses attention on when the equied simultaneous pocessing of jobs at those seves foces idling. The seves 1, 2, 3, and 5 can all be in eithe light o heavy taffic. In the afoementioned heavy taffic egime, we fomulate and solve an appoximating diffusion contol poblem DCP. The DCP solution matches the numbe of jobs in buffe 4 to that in buffe 3, except when the total numbe of jobs waiting fo pocessing by seve 4 is too small fo that to be possible. The implication is that when seve 3 is in light taffic, so that buffe 3 is empty, buffe 4 2

3 +,-./!0123/4" # 5 +,-./$0123/4" % 5! $! $ " # % & & " ' & # % '!* & ' Figue 2: Colo online A fok-join pocessing netwok with two job types and a single shaed seve. is empty and all jobs waiting to be pocessed by seve 4 ae type b jobs. Othewise, when seve 3 is in heavy taffic, the contol at seve 4 must caefully pace its pocessing of type a jobs to pevent getting ahead of seve 3. Ou poposed policy is motivated by the obsevation fom the DCP solution that thee is no eason to have fewe type a jobs in buffe 4 than in buffe 3. If seve 3 can pocess jobs at least as quickly as seve 4 can pocess type a jobs, then the contol unde which seve 4 gives static pioity to type a jobs pefoms well. Othewise, we intoduce a slow depatue pacing SDP contol in which seve 4 slows its pocessing of type a jobs to match the depatue pocess of type a jobs fom buffe 4 to the one fom buffe 3. SDP is a obust idea that is applicable to the moe geneal netwok topology shown in Figue 1. To see this, we fomulate and solve appoximating DCPs fo the fok-join netwok in Figue 2 but with task dependent holding costs cf. Section 10.1, a fok-join netwok with an abitay numbe of foks cf. Figue 7 in Section 10.2, and fok-join netwoks with moe than two job types cf. Figue 8 in Section , Figue 1 and Remak 15 in Section , and Figue 9 in Section In each case, the DCP solutions suggest that, depending on the pocessing capacities of the seves and the netwok state, the seves in the netwok that pocess moe than one job type should eithe give static pioity to the moe expensive job types o slow the depatue pocess of these moe expensive jobs in ode to sometimes pioitize the less expensive jobs. This pevents unnecessay foced idling of the downsteam join seves that pocess the less expensive job types, without sacificing the speed at which the moe expensive job types depat the netwok. We pove that ou poposed policy is asymptotically optimal in heavy taffic fo the fok-join netwok in Figue 2. To do this, we fist show that the DCP solution povides a stochastic lowe bound on the holding cost unde any policy at evey time instant. This is a stong objective, in line with that in Ata and Kuma[2005], which implies asymptotic optimality fo minimizing the expected 3

4 total discounted cost o aveage cost ove a finite time hoizon but does not guaantee asymptotic optimality fo minimizing aveage cost ove an infinite time hoizon without a limit intechange esult. Then, we pove a weak convegence esult that implies the afoementioned lowe bound is achieved unde ou poposed policy. That igoous analysis suggests that ou tanslations of the DCP solutions elevant to the moe geneal topologies shown in Figues 1 and 9 should also pefom well. The weak convegence esult when the netwok opeates unde the SDP contol is a majo technical challenge fo the pape. This is because the SDP contol is a dynamic contol that depends on the netwok state. In ode to obtain the weak convegence, we must caefully constuct andom intevals on which we know the job type seve 4 is pioitizing. Although this idea is simila in spiit to the andom inteval constuction in Bell and Williams [2001], Ghamami and Wad [2013], the poof to show convegence on the intevals is much diffeent, due to the desied matching of the type a job depatue pocesses fom the seves 3 and 4. Moe specifically, the inteval constuction is detemined by tacking and compaing the job counts in buffes 3 and 4, because the SDP policy pioitizes type a jobs when the numbe of jobs in buffe 4 exceeds that in buffe 3, and pioitizes type b jobs othewise. Then, the keys to obtaining the desied weak convegence esult ae as follows. On the intevals on which type a jobs ae pioitized, we must bound the diffeence between two enewal pocesses having diffeent ates. On the intevals on which type b jobs ae pioitized, we equie a ate of convegence esult on a light taffic GI/GI/1 queue that is diffeent than any esult we find in the liteatue fo example, the one in Dai and Weiss [2002]. Finally, when we piece those intevals togethe, we see the DCP solution aise. The emainde of this pape is oganized as follows. We conclude this section with a liteatue eview and a summay of ou mathematical notation. Section 2 specifies ou model and Section 3 povides ou asymptotic famewok. We constuct and solve an appoximating DCP in Section 4. We intoduce the SDP contol in Section 5, and specify when the poposed policy is SDP and when it is static pioity. Section 6 poves that the DCP solution povides a lowe bound on the pefomance of any contol, and Sections 7 and 8 pove weak convegence unde the poposed policy. Section 9 povides extensive simulation esults. In Section 10, we constuct and solve appoximating DCPs fo a boade class of fok-join netwoks. Section 11 makes concluding emaks and poposes a futue eseach diection. We sepaate out ou ate of convegence esult fo a light taffic GI/GI/1 queue in the appendix, as that is a esult of inteest in its own ight. We also povide the poofs of the esults that use moe standad methodology as well as moe detailed simulation esults in the appendix. 1.1 Liteatue Review The inspiation fo this wok came fom the papes Nguyen [1993, 1994]. Nguyen [1993] establishes that a feedfowad FCFS fok-join netwok with one job type and single-seve stations in heavy taffic can be appoximated by a eflected Bownian motion RBM, and Nguyen [1994] extends this esult to include multiple job types. The dimension of the RBM equals the numbe of stations, and its state space is a polyhedal egion. In contast to the RBM appoximation fo feedfowad queueing netwoks Haison [1996], Haison and Van Mieghem [1997], Haison [1998, 2006], the effect of the synchonization constaints in fok-join netwoks is to incease the numbe of faces defining the state space. Also in contast to feedfowad queueing netwoks, that numbe is inceased futhe when moving fom the single job type to multiple job type scenaio. Although delay estimates fo fok-join netwoks follow fom the esults of Nguyen [1993, 1994], they leave open 4

5 the question of whethe and how much delays can be shotened by scheduling jobs in a non-fcfs ode. To solve the scheduling poblem, we follow the standad Bownian machiney poposed in Haison [1996]. This is typically done by fist intoducing a heavy-taffic asymptotic egime in which esouces ae almost fully utilized and the buffe content pocesses can be appoximated by a function of a Bownian motion, and second fomulating an appoximating Bownian contol poblem. Often, the dimension of the appoximating Bownian contol poblem can be educed by showing its equivalence with a so-called wokload fomulation Haison and Van Mieghem [1997], Haison [2006]. The intiguing diffeence when the undelying netwok is a fok-join netwok is that the join seves must be in light taffic to aive at an equivalent wokload fomulation. The issue is that othewise the appoximating poblem is non-linea. This light taffic assumption is asymptotically equivalent to the assumption that pocessing times ae instantaneous. Ou simulation esults suggest that ou poposed contol that is asymptotically optimal when the join seves ae in light taffic also pefoms vey well when the join seves ae in heavy taffic. The papes Pedasani et al. [2014a,b] ae some of the few studies we find that conside the contol of fok-join pocessing netwoks. In both papes, thee ae multiple job classes, but in Pedasani et al. [2014a] the seves can coopeate on the pocessing of jobs and in Pedasani et al. [2014b] they cannot. Thei focus is on finding obust polices in the discete-time setting that do not depend on system paametes and ae ate stable. They do not detemine whethe o not thei poposed policies minimize delay, which is ou focus. The pape Guvich and Wad [2014] seeks to minimize delay, but in the context of a matching queue netwok that has only joins and no foks. An essential question to answe when thinking about contols fo multiclass fok-join netwoks, as can be seen fom the papes Lu and Pang [2016, 2015], Ata et al. [2012], is whethe o not the jobs being joined ae exchangeable; that is, whethe o not a task foked fom one job can be late joined with a task foked fom a diffeent job. Exchangeability is geneally tue in the manufactuing setting, because thee is no diffeence between pats with the same specifications, and geneally false in healthcae settings, because all papewok and test esults associated with one patient cannot be joined with anothe patient. The papes Lu and Pang [2016, 2015] develop heavy taffic appoximations fo a non-exchangeable fok-join netwok with one aival steam that foks into aival steams to multiple many-seve queues, and then must be joined togethe to poduce one depatue steam. The heavy-taffic appoximation fo the non-exchangeable netwok is diffeent than fo the exchangeable netwok, and the non-exchangeability assumption inceases the poblem difficulty. Thei focus, diffeent than ous, is on the effect of coelation in the sevice times, and thee is no contol. The pape Ata et al. [2012] looks at a fok-join netwok in which thee is no contol decision if jobs ae exchangeable, and shows that the pefomance of the exchangeable netwok lowe bounds the pefomance of the non-exchangeable netwok. Then, they popose a contol fo the non-exchangeable netwok that achieves pefomance vey close to the exchangeable netwok. In compaison to the afoementioned papes, the exchangeability assumption is ielevant in ou case. This is because we assume head-of-line pocessing fo each job type, so that the exact same type a b jobs foked fom seve 1 2 ae the ones joined at seve Notation and Teminology Theset ofnonnegative integes isdenoted bynandtheset ofstictly positiveinteges aedenoted by N +. The k dimensional k N + Euclidean space is denoted by R k, R + denotes [0,+, and 0 k is the zeo vecto in R k. Fo x,y R, x y := max{x,y}, x y := min{x,y}, and x + := x 0. Fo any x R, x x denotes the geatest smallest intege which is smalle geate than o equal to x. The supescipt denotes the tanspose of a matix o vecto. Fo each k N +, D k denotes the the space of all ω : R + R k which ae ight continuous 5

6 with left limits. Let 0 D be such that 0t = 0 fo all t R +. Fo ω D and T R +, we let ω T := sup 0 t T ωt. WeconsideD k endowedwiththeusualskookhodj 1 topologycf. Chapte 3 of Billingsley [1999]. Let BD k denote the Boel σ-algeba on D k associated with Skookhod J 1 topology. By Theoem of Whitt [2002], BD k coincides with the Kolmogoov σ-algeba geneated by the coodinate pojections. Fo stochastic pocesses W, R +, and W whose sample paths ae in D k fo some k N +, W = W means that the pobability measues induced by W on D k,bd k weakly convege to the one induced by W on D k,bd k as. Fo x,y D, x y, x y, and x + ae pocesses in D such that x yt := xt yt, x yt := xt yt, and x + t := xt + fo all t R +. Fo x D, we define the mappings Ψ,Φ : D D such that fo all t 0, Ψxt := sup xs +, Φxt := xt+ψxt, 0 s t whee Φ is the one-sided, one-dimensional eflection map, which is intoduced by Skookhod [1961]. Let Z = {1,2,...,n} and X i be a pocess in D fo each i Z. Then X i,i Z denotes the pocess X 1,X 2,...,X n in D n. We denote e as the deteministic identity pocess in D such that et = t fo all t 0. We abbeviate the phase unifomly on compact intevals by u.o.c., almost suely by a.s.. We let a.s. denote almost sue convegence and = d denote equal in distibution. I denotes the indicato function and BM q θ,σ denotes a Bownian motion with dift vecto θ and covaiance matix Σ which stats at point q. The big-o notation is denoted by O, i.e., if x : R + R and y : R + R ae two functions, then xt = Oyt as t if and only if thee exist constants C and t 0 such that xt C yt fo all t t 0. Lastly, o p is the little-o in pobability notation, i.e., if {X n,n N} and {Y n,n N} ae sequences of andom vaiables, then X n = o p Y n if and only if X n / Y n conveges in pobability to 0. 2 Model Desciption We conside the contol of the fok-join pocessing netwok depicted in Figue 2. In this netwok, thee ae 2 job types, 7 seves, 10 buffes, and 8 activities. The set of job types is denoted by J, whee J = {a,b} and a and b denote the type a and type b jobs, espectively. The set of seves is denoted by S, whee S = {1,2,...,7}. The set of buffes is denoted by K, whee K = {1,2,...,10}, and the set of activities is denoted by A whee A = {1,2,3,A,B,5,6,7}. Except fo seve 4, each seve is associated with a single activity. Seve 4 is associated with two activities, denoted by A and B, which ae pocessing type a jobs fom buffe 4 and type b jobs fom buffe 5, espectively. Both seve 6 and seve 7 deplete jobs fom 2 diffeent buffes. Note that these two seves ae join seves and pocess jobs wheneve both of the coesponding buffes ae nonempty. Hence, both seve 6 and seve 7 ae associated with a single activity, namely activities 6 and 7, espectively. Let s : A S be a function such that sj denotes the seve in which activity j, j A takes place. Let f : K\{1,2} A be a function such that fk denotes the activity which feeds buffe k, k K\{1,2}. Lastly, let d : K A be a function such that dk denotes the activity which depletes buffe k, k K. Fo example, sa = sb = 4, f4 = 1 and d4 = A. 2.1 Stochastic Pimitives We assume that all the andom vaiables and stochastic pocesses ae defined in the same complete pobability space Ω,F,P, E denotes the expectation unde P, and PA,B := PA B. We associate the extenal aival time of each job and the pocess time of each job in the coesponding activities with the sequence of andom vaiables { v j i,j J A} i=1 and the stictly positive constants {λ j,j J} and {µ j,j A}. We assume that fo each j J A, { v j i} i=1 is a stictly positive, independent and identically distibuted i.i.d. sequence of andom vaiables mutually independent of { v k i} i=1 fo all k J A\{j}, E[ v j1] = 1, and the vaiance of v j 1, 6

7 denoted by Va v j 1, is σ 2 j. Fo j J, let v ji := v j i/λ j be the inteaival time between the i 1st and ith type j job. Then, λ j and σ j ae the aival ate and the coefficient of vaiation of the inteaival times of the type j jobs, whee j J. Fo j {1,3,A,6} j {2,B,5,7}, let v j i := v j i/µ j be the sevice time of the ith incoming type a b job associated with the activity j. Then, µ j and σ j ae the sevice ate and the coefficient of vaiation of the sevice times elated to activity j, j A. Fo each j J A, let V j 0 := 0 and V j n := n v j i n N +, S j t := sup{n N : V j n t}. 1 i=1 Then, S j is a enewal pocess fo each j J A. If j J, S j t counts the numbe of extenal type j aivals until time t; if j A, S j t counts the numbe of sevice completions associated with activity j until time t given that the coesponding seve woks continuously on this activity duing [0,t]. 2.2 Scheduling Contol and Netwok Dynamics Let T j t, j A, denote the cumulative amount of time seve sj devotes to activity j duing [0, t]. Then, a scheduling contol is defined by the two dimensional sevice time allocation pocess T A,T B. Although a scheduling contol indiectly affects T 6,T 7, since we do not have any diect contol on seves 6 and 7, we exclude T 6,T 7 fom the definition of the scheduling contol. Let, I sj t := t T j t, j A\{A,B}, I 4 t := t T A t T B t, 2a 2b denote the cumulative idle time of the seves duing the inteval [0,t]. Fo any j A, S j T j t denotes the total numbe of sevice completions elated to activity j in seve sj up to time t. Fo any k K, let Q k t be the numbe of jobs waiting in buffe k at time t, t 0, including the jobs that ae being seved. Then, fo all t 0, Q 1 t := S a t S 1 T 1 t 0, Q 2 t := S b t S 2 T 2 t 0, 3a Q k t := S fk T fk t S dk T dk t 0, k K\{1,2}. Fo simplicity, we assume that initially all buffes ae empty, i.e., Q k 0 = 0 fo all k K. Late, we elax this assumption in Remak 9. We have V j S j T j t T j t < V j S j T j t+1, fo all j A and t 0, 4 which implies that we conside only head-of-the-line HL policies, whee jobs ae pocessed in FCFS ode within each buffe. In this netwok, a task associated with a specific job cannot join a task oiginating in anothe job unde the HL policies; that is, ecalling ou liteatue eview cf. Section 1.1, the notion of exchangeability is not pesent. It is staightfowad to see that wok conseving policies ae moe efficient than the othes in this netwok. Hence, we ensue that all of the seves wok in a wok-conseving fashion by the following constaints: Fo all t 0, 3b I j is nondeceasing and I j 0 = 0 fo all j S, I sdk t inceases if and only if Q k t = 0, fo all k {1,2,3,6}, I 4 t inceases if and only if Q 4 t Q 5 t = 0, 5a 5b 5c 7

8 I 6 t inceases if and only if Q 7 t Q 8 t = 0, I 7 t inceases if and only if Q 9 t Q 10 t = 0. 5d 5e A scheduling policy T := T A,T B is admissible if it satisfies the following conditions: Fo any T i, i A, I j, j S, and Q k, k K satisfying 2, 3, 4, and 5, T j t F, t 0 and j {A,B}, T j is continuous and nondeceasing with T j 0 = 0, j {A,B}, I 4 is continuous and nondeceasing with I 4 0 = 0. 6a 6b 6c Conditions 6a 6c imply that we conside a wide ange of scheduling policies including the ones which can anticipate the futue. 2.3 The Objective A natual objective is to minimize the discounted expected total holding cost. Let h a and h b denote the holding cost ate pe job pe unit time fo a type a and b job, espectively; and δ > 0 be the discount paamete. Moeove, let Z a t := Q 3 t+q 4 t+q 7 t+q 8 t, Z b t := Q 5 t+q 6 t+q 9 t+q 10 t. Then Z a t and Z b t denote the total numbe of type a and b jobs in the system except jobs waiting in buffes 1 and 2. Since Q 1 t and Q 2 t ae independent of the scheduling policy, we exclude these pocesses fom the definitions of Z a t and Z b t. Then, the expected total discounted holding cost unde admissible policy T is [ ] J T = E e δt h a Z a t+h b Z b tdt, 7 0 and ou objective is to find an admissible policy which minimizes 7. Anothe natual objective is to minimize the expected total cost up to time t, t R +, which is [ t ] J T = E h a Z a s+h b Z b sds. 8 0 Yet anothe possible objective is to minimize the long-un aveage cost pe unit time, [ 1 t ] J T = limsup t t E h a Z a s+h b Z b sds. 9 We focus on a moe challenging objective which is minimizing 0 Ph a Z a t+h b Z b t > x, fo all t R + and x > It is possible to see that any policy that minimizes 10 also minimizes the objectives 7, 8, and 9. In this specific netwok, fo all t 0 Q 3 t+q 7 t = Q 4 t+q 8 t, Q 5 t+q 9 t = Q 6 t+q 10 t. 11 By 11, a policy is optimal unde the objective 10 if and only if it is optimal unde the objective of minimizing Ph a Q 3 t+q 7 t+h b Q 6 t+q 10 t > x, fo all t R + and x > We will focus on the objective 12 fom this point fowad. 8

9 3 Asymptotic Famewok It is vey difficult to analyze the system descibed in Section 2 exactly. Even if we can accomplish this vey challenging task, it is even less likely that the optimal contol policy will be simple enough to be expessed by a few paametes. Theefoe, we focus on finding an asymptotically optimal contol policy unde diffusion scaling and the assumption that seve 4 is in heavy taffic. We fist intoduce a sequence of fok-join systems and pesent the main assumptions done fo this study in Section 3.1. Then we fomally define the fluid and diffusion scaled pocesses and pesent convegence esults fo the diffusion scaled wokload facing seve 4 and the diffusion scaled queue length pocesses associated with seves 1, 2, 3, and 5 in Section 3.2. The question left open is to detemine what should be the elationship between the contol and the numbe of type a and b jobs in the wokload facing seve A Sequence of Fok-Join Systems We conside a sequence of fok-join systems indexed by whee though a sequence of values in R +. Each queueing system has the same stuctue defined in Section 2 except that the aival and sevice ates depend on. To be moe pecise, in the th system, we associate the extenal aival time of each job and the pocess time of each job in the coesponding activities with the sequence of andom vaiables { v j i,j J A} i=1, which we have defined in Section 2.1, and the stictly positive constants {λ j,j J} and {µ j,j A} such that v j i := v ji/λ j, j J is the inteaival time between the i 1st and ith type j job and vj i := v ji/µ j, j {1,3,A,6} j {2,B,5,7} is the sevice time of the ith incoming type a b job associated with the activity j in the th system. Theefoe, λ j, j J and µ j, j A ae the aival ates and sevice ates in the th system wheeas the coefficient of vaiations ae the same with the oiginal system defined in Section 2. Fom this point fowad, we will use the supescipt to show the dependence of the stochastic pocesses to the th queueing system. Next, we pesent ou assumptions elated to the system paametes. The fist one is elated to cost paametes. Assumption 1 Without loss of geneality, we assume that h a µ A h b µ B. This assumption implies that it is moe expensive to keep type a jobs than type b jobs in seve 4. Second, we make the following assumptions elated to the stochastic pimitives of the netwok. Assumption 2 Thee exists a non-empty open neighbohood, O, aound 0 such that fo all α O, E[e αv j1 ] <, fo all j J A. Assumption 2 is the exponential moment assumption fo the inteaival and sevice time pocesses. This assumption is common in the queueing liteatue, cf. Haison [1998], Bell and Williams [2001], Maglaas [2003], Meyn [2003]. Ou final assumption concens the convegence of the aival and sevice ates, and sets up heavy taffic asymptotic egime. Assumption 3 1. λ j λ j > 0 fo all j J as. 2. µ j µ j > 0 fo all j A as. 3. λ a /µ A +λ b /µ B = 1. 9

10 4. λ a/µ A +λ b /µ B 1 θ 4/µ A as, whee θ 4 is a constant in R. 5. As, 6. λ a < µ 6 and λ b < µ 7. λ a µ 1 θ 1 R { }, λ b µ 2 θ 2 R { }, 13a λ a µ 3 θ 3 R { }, λ b µ 5 θ 5 R { }. 13b 7. If h a µ A = h b µ B, thee exists an 0 R + such that h a µ A h bµ B fo all 0. Pats 3 and 4 of Assumption 3 ae the heavy taffic assumptions fo seve 4. Note that, if seve 4 was in light taffic, its pocessing capacity would be high, thus any wok-conseving contol policy would pefom well. Moeove, as explained in the emegency depatment example in Section 1, seve 4 epesents expensive and limited esouces e.g. a CT scanne which needs to pocess multiple job types, thus it is expected fom seve 4 to be in heavy taffic in such a setting. By Pats 1, 2, and 3, we have µ A > λ a and µ B > λ b. Pat 5 states that each of the seves 1, 2, 3, and 5 can be eithe in light o heavy taffic. On the one hand, if θ i = fo some i {1,2,3,5}, then seve i is in light taffic. On the othe hand, if θ i R, then seve i is in heavy taffic. Pat 6 states that the two join seves ae in light taffic. Note that, Ata et al. [2012], Guvich and Wad [2014], Lu and Pang [2016, 2015] assume that the sevice pocesses in the join seves ae instantaneous. Hence, Pat 6 of Assumption 3 extends the assumptions on the join seves made in the liteatue. Lastly, Pat 7 of Assumption 3 is done fo technical easons that occu only when h a µ A = h b µ B cf. Section 6.1. Fo simplicity, we assume that Q k 0 = 0, fo all k K and R +. Late, we elax this assumption in Remak 9. Assumptions 1, 2, and 3 ae assumed thoughout the pape. 3.2 Fluid and Diffusion Scaled Pocesses In this section, we pesent the fluid and diffusion scaled pocesses. Fo all t 0, the fluid scaled pocesses ae defined as S jt := 2 S j 2 t, j J A, T jt := 2 T j 2 t, j A, 14a Q jt := 2 Q j 2 t, j K, I jt := 2 I j 2 t, j S, 14b and the diffusion scaled pocesses ae defined as Ŝ j t := 1 S j 2 t λ j 2 t, j J, Ŝ j t := 1 S j 2 t µ j 2 t, j A, 15a T jt := 1 T j 2 t, j A, Î jt := 1 I j 2 t, j S, 15b Q jt := 1 Q j 2 t, j K. 15c By the functional cental limit theoem FCLT, cf. Theoem of Whitt [2002], we have the following weak convegence esult. Ŝ j,j J A = Sj,j J A, 16 10

11 whee S j is a one-dimensional Bownian Motion fo each j J A such that S j d = BM0 0,λ j σ 2 j fo j J, S j d = BM0 0,µ j σ 2 j fo j A and each S j is mutually independent of S i, i J A\{j}. Fo t 0, let us define the wokload pocess up to a constant scale facto W4 t := Q 4 t+ µ A µ Q 5 t. 17 B Then, W4 t/µ A is the expected time to deplete buffes 4 and 5 at time t, if no moe jobs aive afte time t in the th system. Let W 4 t := 2 W4 2 t and Ŵ 4 t := 1 W4 2 t denote the fluid and diffusion scaled wokload pocesses, espectively. Next, we pesent the convegence of the fluid scaled pocesses unde any wok-conseving policy. Poposition 1 Unde any wok-conseving policy cf. 5, as Q k,k K, W 4, T j,j A a.s. Q k,k K, W 4, T j,j A u.o.c., whee Q k = 0 fo all k K, W 4 = 0, and T j t = λ a /µ j t fo j {1,3,A,6} and T j t = λ b /µ j t fo j {2,B,5,7} fo all t 0. The poof of Poposition 1 is pesented in Appendix B.1. We will use Poposition 1 to pove convegence esults fo the diffusion scaled pocesses. Consideing Assumption 3 Pat 5, let H := {i {1,2,3,5} : θ i R}, i.e., H is the set of seves which ae in heavy taffic among the seves 1, 2, 3, and 5. Let H be the cadinality of the set H, and fo each i {1,2,3,5}, let { 1, if seve i is in heavy taffic, i.e., i H, χ i := 0, if seve i is in light taffic, i.e., i / H. Fo all t 0, let S 1t := χ 1 Ŝ 1 T 1t λ a µ 1t +1 χ 1 Ŝ a t Q 1t, 18a S 2t := χ 2 Ŝ 2 T 2t λ b µ 2t +1 χ 2 Ŝ b t Q 2t. 18b Then, by using 18, we define the so-called netput pocess fo each buffe as X k t := Ŝ l dk t Ŝ T dk t +λ l µ dk t, fo k,l {1,a,2,b}, 19a X k t := S l dk t Ŝ T dk t +λ i µ dk t, fo k,l,i {3,1,a,6,2,b}, 19b X k t := S l t Ŝ dk T dk t +µ dk λ i µ dk λ i µ dk X k t := Ŝ l t Q it Q jt Ŝ dk T dkt +λ l µ dk t, Let Q := Q 1 Q 2 Q 3 Q 6 Ŵ 4 t, fo k,l,i {4,1,a,5,2,b}, 19c fo k,l,i,j {7,a,1,3,8,a,1,4,9,b,2,5,10,b,2,6}. 19d, X := X 1 X 2 X 3 X 6 X 4 + µ A µ B X 5 11, Î := Î 1 Î 2 Î 3 Î 5 Î 4, 20

12 θ 1 µ θ 2 0 µ θ := θ 3 χ 1 θ 1 θ 5 χ 2 θ 2, R := χ 1 µ 1 0 µ 3 0 0, 21 0 χ 2 µ 2 0 µ 5 0 µ θ 4 χ 1 θ 1 χ A 2 µb θ 2 χ 1 µ µ 1 χ A 2 µ µ B µ A and let R be a 5 5 matix which is the component-wise limit of R. Then, we have Let us define Q = X +R Î, 22 Q k = X k +µ dkî sdk, k {7,8,9,10}. 23 Σ := λ a σa 2 +σ1 2 0 λ aσ1 2 0 λ a σ λ b σb 2 +σ2 2 0 λ bσ2 2 µ A µb λ b σ λ a σ1 2 0 Cov3,3 0 Cov3,4 6 0 λ b σ2 2 0 Cov6,6 Cov4,6 4 λ a σ1 2 µ A µb λ b σ2 2 Cov3,4 Cov4,6 Cov4,4 24 whee Cov3,3 := λ a χ1 σ χ 1σa 2 3 +σ2, Cov3,4 := λa χ1 σ χ 1σa 2, Cov4,6 := µ A λ b χ2 σ χ 2 σ 2 b, Cov6,6 := λb χ2 σ χ 2 σ 2 b µ +σ2 5, B Cov4,4 := λ a χ1 σ χ 1σa 2 2 µa +σ2 A + λ b χ2 σ χ 2σb 2 B +σ2. 25 Then, we have the following weak convegence esult. Poposition 2 Unde any wok-conseving policy cf. 5, Q 1, Q 2, Q 3, Q 4 6,Ŵ = Q1, Q 2, Q 3, Q 6, W 4, 26 whee Q i = 0 fo each i / H and Qi,i H, W4 is a semimatingale eflected Bownian motion SRBM associated with the data P H,θ H,Σ H,R H,0 H. P H is the nonnegative othant in R H ; θ H is an H -dimensional vecto deived fom the vecto θ cf. 21 by deleting the ows coesponding to each i, i / H; Σ H and R H ae H H -dimensional matices deived fom Σ cf. 24 and R cf. 21 by deleting the ows and columns coesponding to each i, i / H, espectively; and 0 H is the oigin in P H. The state space of the SRBM is P H ; θ H and Σ H ae the dift vecto and the covaiance matix of the undelying Bownian motion of the SRBM, espectively; R H is the eflection matix; and 0 H is the stating point of the SRBM. The fomal definition of an SRBM can be found in Definition 3.1 of Williams [1998b]. The poof of Poposition 2 is pesented in Appendix B.2. µ B 12

13 4 The Appoximating Diffusion Contol Poblem In this section, we constuct an appoximating diffusion contol poblemdcp with non-igoous mathematical aguments. Howeve, the solution of the DCP will help us to guess a heuistic contol policy. Then, we will pove the asymptotic optimality of this heuistic contol policy igoously in Sections 5, 6, 7, and 8. Paallel with the objective 12, conside the objective of minimizing P h a Q 3 t+ Q 7t +h b Q 6 t+ Q 10t > x, t 0, x > 0, 27 fo some. Motivated by Assumption 3 Pat 6, let us petend that the sevice pocesses at seves 6 and 7 happen instantaneously. Since the diffusion scaled queue length pocess weakly conveges to 0 in a light taffic queue, we believe that consideing instantaneous sevice ates in the downsteam seves will not change the behavio of the system in the limit. In this case, jobs can accumulate in buffe 7 8 only at the times buffe 8 7 is empty. Similaly, jobs can accumulate in buffe 10 9 only at the times buffe 9 10 is empty. By this fact and 11, Q 7 = Q 4 Q 3 +, Q 8 = Q 3 Q 4 +, Q 9 = Q 6 Q 5 +, Q 10 = Q 5 Q By 28, objective 27 is equivalent to minimizing P h a Q 3 Q4 t+ t Q + +h 3 t b Q 6 Q5 t+ t Q + 6 t > x, t 0, x > Fom the objective 29, we need appoximations fo Q 3, Q 4, Q 5, and Q 6 and we will achieve this goal by letting. By 15c, 17, and Poposition 2, we know that Q 3, Q 6, weakly conveges to Q3, Q 6 and Q 4 +µ A /µ B Q 5 weakly conveges to W 4. At this point, let us assume that Q 4, Q 5 = Q4, Q 5. Then we constuct the following DCP: Fo each x > 0 and t 0, min P h a Q 3 t+ Q4 t Q + 3 t +h b Q 6 t+ Q5 t Q + 6 t > x, s.t. Q4 t+ µ A µ B Q5 t = W 4 t, 30 Q k t 0, fo all k {4,5}. Intuitively, we want to minimize the total cost by splitting the total wokload in seve 4 to buffes 4 and 5 in the DCP 30. Now, we will conside DCP 30 path-wise. Let ω ω Ω denote a sample path of the pocesses in DCP 30 and fo any F : Ω D, Fω t denote the value of the pocess F at time t in the sample path ω. Then, conside the following optimization poblem fo each ω Ω and t 0. min h a Q4 ω t Q 3 ω t + +hb Q5 ω t Q 6 ω t +, 31a s.t. Q4 ω t + µ A Q5 ω t = µ W 4 ω t, B 31b Q k ω t 0, fo all k {4,5}. 31c 13

14 Note that, weexcludethetem h a Q3 ω t +h b Q6 ω t fomtheobjective function 31a because Q 3 ω t and Q 6 ω t ae independent of the decision vaiables Q 4 ω t and Q 5 ω t. Although the objective function 31a is nonlinea, the optimization poblem 31 is easy to solve because it has linea constaints. Moeove, Lemma 1, povides a closed-fom solution fo 31. Lemma 1 Conside the optimization poblem min h a q 4 q 3 + +h b q 5 q 6 +, s.t. q 4 + µ A µ B q 5 = w 4, q 4 0, q 5 0, whee q 4 and q 5 ae the decision vaiables, all of the paametes ae nonnegative, and h a µ A h b µ B. Then q 4 = q 3 w 4 and q 5 = µ B /µ A w 4 q 3 + is an optimal solution of this poblem. Pf: Replacing q 4 with w 4 µ A /µ B q 5 gives us the following equivalent optimization poblem which has only one decision vaiable. min h a w 4 µ + A q 5 q 3 +h b q 5 q 6 +, 32a µ B s.t. 0 q 5 µ B µ A w 4. 32b We will solve the optimization poblem 32 case by case. Fist, conside the case in which w 4 q 3. If q 5 = 0, then objective function value becomes 0, which is the lowest possible objective function value of the poblem 32. Theefoe, q 4 = w 4 and q 5 = 0 is an optimal solution when w 4 q 3. Second, conside the case whee w 4 > q 3. In this case, fist conside the following subcase: µ B /µ A w 4 q 3 q 6. In this case, if q 5 = µ B /µ A w 4 q 3, then the objective function value becomes 0. Thus, q 4 = q 3 and q 5 = µ B /µ A w 4 q 3 is an optimal solution when w 4 > q 3 and µ B /µ A w 4 q 3 q 6. Lastly, conside the subcase in which µ B /µ A w 4 q 3 > q 6. Figue 3 illustates the change of the objective function value as q 5 inceases fom 0 to its uppe bound, µ B /µ A w 4. Note that, when q 5 = q 6, the objective function value is h a w 4 µ A /µ B q 6 q 3. At this point, inceasing q 5 by a small amount, say ǫ, will change the objective function value by h a µ A /µ B +h b ǫ, which is a negative numbe because h a µ A h b µ B. When q 5 = µ B /µ A w 4 q 3, inceasing q 5 will incease the objective function value with the ate h b. Theefoe, as seen in Figue 3, q 4 = q 3 and q 5 = µ B /µ A w 4 q 3 is an optimal solution when w 4 > q 3 and µ B /µ A w 4 q 3 > q 6. Theefoe, by Lemma 1, we see that an optimal solution of the optimization poblem 31 is Q 4 ω t = W 4 ω t Q 3 ω t and Q 5 ω t = µ B /µ A W4 ω t Q 3 ω t + fo all ω and t 0. This esult and 28 imply the following poposition. Poposition 3 Q4, Q 5, Q 7, Q 8, Q 9, Q 10 = Q 3 W 4, µ B W4 µ Q +, 3 0, Q3 W +, 4 A Q 6 µ B W4 µ Q + + 3, A µb W4 µ Q Q6 A 33 is an optimal solution of the DCP

15 Objective function value - - Figue 3: Objective function value of the poblem 32 with espect to diffeent q 5 values. Note that the ight hand side of 33 is independent of the scheduling policies by Poposition 2. Theefoe, a contol policy in which the coesponding pocesses weakly convege to the ight hand side of 33 is a good candidate fo an asymptotically optimal policy. The DCP solution in Poposition 3 matches the content level of buffe 4 to that of buffe 3, except when the buffe 3 content level exceeds the total wok facing seve 4 that is, the combined contents of buffes 4 and 5. This ensues that seve 4 neve causes seve 6 to idle because of the join opeation, as is evidenced by the fact that buffe 7 is always empty wheeas buffe 8 sometimes has a positive content level. At the same time, seve 4 pevents any unnecessay idling of seve 7 by devoting its emaining effot to pocessing the contents of buffe 5. Sometimes, that effot is sufficient to keep up with seve 5 and pevent the contents of buffe 10 fom building and sometimes it is not. That is why sometimes buffe 9 has a positive content in the DCP solution and sometimes buffe 10 does but neve both simultaneously. In the next section, we fomally intoduce the poposed policy. 5 Poposed Policy Ou objective is to popose a policy unde which the diffusion scaled queue-length pocesses tack the DCP solution given in Poposition 3. This is because the DCP solution povides a lowe bound on the asymptotic pefomance of any admissible policy, as we will pove in Section 6 see Theoem 2. The key obsevation fom the DCP solution in Poposition 3 is that thee is no eason fo the depatue pocess of the moe expensive type a jobs fom seve 4 to exceed that of seve 3. Instead of eve letting seve 4 get ahead, it is pefeable to have seve 4 wok on type b jobs, so as to pevent as much foced idling at seve 7 due to the join opeation as possible. The only time thee should have been moe cumulative type a job depatues fom seve 4 than fom seve 3 is when the total numbe of jobs facing seve 4 is less than that facing seve 3. In that case, seve 4 can outpace seve 3 without focing additional idling at seve 7. The intuition in the peceding paagaph motivates the following depatue pacing policy, in which seve 4 gives pioity to type a jobs when the numbe of type a jobs in buffe 4 exceeds that 15

16 in buffe 3 and gives pioity to the type b jobs in buffe 5 othewise. Definition 1 Slow Depatue Pacing SDP Policy. The allocation pocess T A,T B satisfies IQ 3 t < Q 4 tdt T A t = 0, IQ 3 t Q 4 t,q 5 t > 0dt T B t = 0, IQ 3 t Q 4 t,q 4 t+q 5 t > 0dI 4 t = 0, 34a 34b 34c togethe with 2, 3, 4, and 5. It is possible to see that T A,T B that satisfies 34 also satisfies 6, and so is admissible. If Q 3 t < Q 4 t, 34a ensues that seve 4 gives pioity to buffe 4. If Q 3 t Q 4 t and Q 5 t > 0, 34b ensues that seve 4 gives pioity to buffe 5. 34c ensues a wok-conseving contol policy in seve 4 when Q 3 t Q 4 t. When µ 3 < µ A, so that seve 3 is the slowe seve, we use the slow depatue pacing policy to detemine when seve 4 can allocate effot to pocessing type b jobs without inceasing type a job delay. Othewise, when µ 3 µ A, thee is almost neve exta pocessing powe to allocate to type b jobs, and so a static pioity policy will pefom similaly to the slow depatue pacing policy see Remak 11 egading ou numeical esults. Definition 2 Static Pioity Policy. The allocation pocess T A,T B satisfies 0 0 IQ 4 t > 0dt T A t = 0, IQ 4 t+q 5 t > 0dI 4 t = 0, 35a 35b togethe with 2, 3, 4, and 5. It is possible to see that T A,T B that satisfies 35 also satisfies 6, and so is admissible. 35a ensues that seve 4 gives static pioity to buffe 4 and 35b ensues a wok-conseving contol policy in seve 4. The poposed policy is the SDP policy in Definition 1 when µ 3 < µ A and is the static pioity policy in Definition 2 when µ 3 µ A. We have the following weak convegence esult associated with the poposed policy. Theoem 1 Unde the poposed policy, Q k,k K, 4 Ŵ = Qk,k K, W 4, whee Q1, Q 2, Q 3, Q 6, W 4 is defined in Poposition 2 and Q4, Q 5, Q 7, Q 8, Q 9, Q 10 Poposition 3. is defined in The poof of Theoem 1 is pesented in Section 7. Theoem 1, the continuous mapping theoem see, fo example Theoem of Whitt [2002], and Theoem of Whitt [2002] establish the asymptotic behavio of the objective function 12 unde the poposed policy. 16

17 Coollay 1 Unde the poposed policy, fo all t 0 and x > 0, we have lim P h a Q 3 t+ Q 7 t +h b Q 6 t+ Q 10 t > x µb = P h a Q3 t+h b Q 6 t+ W4 t µ Q t Q6 t > x. 36 A Remak 1 The poposed policy is a peemptive policy. Howeve, it is often pefeed to use a nonpeemptive policy. To specify a non-peemptive policy, we must specify which type of job seve 4 chooses to pocess each time seve 4 becomes fee, and thee ae both type a and type b jobs waiting in buffes 4 and 5. The non-peemptive vesion of the SDP policy has seve 4 choose to seve a type a job when the numbe of jobs in buffe 4 exceeds that in buffe 3, and to seve a type b job othewise. The non-peemptive vesion of the static pioity policy has seve 4 always choose a type a job. We expect the pefomance of the non-peemptive vesion of ou poposed policy to be indistinguishable fom ou poposed policy in ou asymptotic egime, and we veify that the fome policy pefoms vey well by ou numeic esults in Section 9. Remak 2 The non-peemptive vesion of the poposed policy cf. Remak 1 is easy to implement in pactice. The system contolle does not need know any λ j, j J o µ j, j A but he needs to know whethe h a µ A h b µ B o not in ode to detemine the job type that needs pioity. Without loss of geneality, suppose that the system contolle knows that h a µ A h b µ B. Then, he needs to know whethe µ 3 < µ A o not in ode to detemine which policy to implement among the nonpeemptive vesions of the SDP and static pioity policies. Note that the simulation expeiments cf. Section 9.2 show that the nonpeemptive SDP policy pefoms well even when µ 3 µ A. Thus, the nonpeemptive SDP policy is a safe choice when it is not clea whethe µ 3 < µ A o not. In ode to implement this policy, the system contolle only needs to keep tack of the numbe of jobs in buffes 3 and 4 at the sevice completion epochs in seve 4. The implementation of the nonpeemptive static pioity policy is even moe tivial. The system contolle also needs to know whethe seves 6 and 7 ae in light taffic o not cf. Pat 6 of Assumption 3. Howeve, since the simulation expeiments cf. Section 9.2 show that the nonpeemptive vesion of the poposed policy pefoms well even when at least one of the downsteam seves ae in heavy taffic, the latte equied infomation is not vey cucial in the implementation of the this policy. Lastly, whethe seve 4 is in light o heavy taffic does not affect the pefomance of the non-peemptive vesion of the poposed policy, because any wok-conseving policy pefoms good when seve 4 is in light taffic and the poposed policy is asymptotically optimal when seve 4 is in heavy taffic cf. Section 6. Remak 3 In the classical open pocessing netwoks, if a seve is in light taffic, then the coesponding diffusion scaled buffe length pocess conveges to 0 see Theoem 6.1 of Chen and Mandelbaum [1991]. Howeve, we see in Theoem 1 that although seves 6 and 7 ae in light taffic, Q 9 and Q 10 ae non-zeo pocesses moeove Q 8 is a non-zeo pocess when seve 3 is in heavy taffic. Theefoe, even though a join seve has moe than enough pocessing capacity, significant amount of jobs can accumulate in the coesponding buffes due to the synchonization equiements between the jobs in diffeent buffes. This makes the contol of fok-join netwoks moe challenging than the one of classical open-pocessing netwoks. Remak 4 We will pove that the SDP and the static pioity policies ae asymptotically optimal when λ a µ 3 < µ A and µ 3 > λ a, espectively cf. Sections 7.1 and 7.2 and Remak 6. Hence, both SDP and static pioity policies ae asymptotically optimal when λ a < µ 3 < µ A and this implies 17

18 that thee ae many othe asymptotically optimal contol policies in addition to the one that we popose. Howeve, the simulation expeiments show that the static pioity policy pefoms pooly when λ a µ 3 but the SDP policy pefoms vey well even when µ 3 > µ A cf. Section 9.2. Theefoe, the pefomance of the poposed policy that we constuct is obust with espect to the paametes λ a, µ 3, and µ A, which has moe pactical appeal. In the following section, we fomally define asymptotic optimality and pove that the poposed policy is asymptotically optimal given Theoem 1 and Coollay 1. 6 Asymptotic Optimality In this section, we pove that the DCP solution cf. Poposition 3 is a lowe bound fo all admissible policies with espect to the objective function 12, i.e., we have the following esult. Theoem 2 Let {T, 0} be an abitay sequence of admissible policies. Then fo all t 0 and x > 0, we have lim inf P h a Q 3 t+ Q 7t P +h b Q 6 t+ Q 10t h a Q3 t+h b > x µb Q 6 t+ W4 t µ Q t Q6 t > x A Theoem 2 togethe with Coollay 1 state that the poposed policy is asymptotically optimal.. 37 Remak 5 In Section 2.2, we state that the objective 12 implies the objectives 7, 8, and 9. Although 36 and 37 imply asymptotic optimality of the sequences of contol policies with espect to the objectives 7 and 8, they do not necessaily imply the same esult elated to the objective 9. That is because we need to change the ode of the limits with espect to and t to get the desied esult. Howeve, fo that, we need additional esults such as unifom convegence of the elated pocesses. 6.1 Poof of Theoem 2 Let us conside the tem in the left hand side of 37. By 11 and the fact that Q k 0 fo all k K, we have Q 7 t Q 4 t Q 3 t+, Q 10 t Q 5 t Q 6 t+, t 0. Theefoe, it is enough to pove fo all t 0 and x > 0, lim inf h P a Q 3t+ Q 4 t Q + 3t +h b Q 6t+ Q 5 t Q + 6t h a Q3 t+h b P > x µb Q 6 t+ W4 t µ Q t Q6 t > x A. 38 By 16, Poposition 1, and Theoem of Whitt [2002] joint convegence when one limit is deteministic, we have Ŝ j,j J A, T i,i A = S j,j J A, T i,i A. 39 Now, we use Skookhod s epesentation theoem cf. Theoem of Whitt [2002] to obtain the equivalent distibutional epesentations of the pocesses in 39 fo which we use the same 18

19 symbols and call Skookhod epesented vesions such that all Skookhod epesented vesions of the pocesses ae defined in the same pobability space and the weak convegence in 39 is eplaced by almost sue convegence. Then we have Ŝ j,j J A, T a.s. i,i A Sj,j J A, T i,i A, u.o.c. as. 40 We will conside the Skookhod epesented vesions of these pocesses fom this point fowad and pove 38 with espect to these pocesses. By 3, 15, 17, 19, 22, 40, and Poposition 2, we have whee Q 3,Ŵ 4, Q 6 a.s. Q 3, W 4, Q 6, u.o.c., 41 Ŵ4 = Q 4 + µ A Q µ 5, a.s., 42 B and all of the pocesses in 41 and 42 have the same distibution with the oiginal ones. Then by Fatou s lemma, the tem in the left hand side of 38 is geate than o equal to P lim inf [h a Q 3 Q4 t+ t Q + +h 3 t b Q 6 Q5 t+ t Q ] + 6 t > x. 43 Fo each t 0 and sufficiently lage such that h a µ A h bµ B note that such an exists by Assumption 1 and Pats 2 and 7 of Assumption 3, we will find a path-wise lowe bound fo the tem [h a Q 3 Q4 t+ t Q + +h 3 t b Q 6 Q5 t+ t Q ] + 6 t. 44 Fom this point fowad, we will conside the sample paths in which Q k t, k {3,4,5,6} ae finite fo all and t. By 41 and 42, these sample paths occu with pobability one. Let ω be a sample path and ω t be defined as in Section 4. By 42, 44, and the fact that Q 3 t and Q 6 t ae independent of the contol policy, we constuct the following optimization poblem. Fo each ω in Ω except a null set and t 0 min h a Q 4 ω t Q 3 ω t + +hb Q 5 ω t Q 6 ω t +, 45a s.t. Q 4 ω t+ µ A Q 5 ω t = Ŵ 4 ω t, µ B Q 4 ω t 0, Q 5 ω t 0, 45b 45c whee Q 4 ω t and Q 5 ω t ae the decision vaiables. The optimization poblem 45 has the same stuctue with the one pesented in Lemma 1. Theefoe, we can use Lemma 1 to solve 45 and in the optimal solution Q 4 ω t = Q 3 ωt Ŵ 4 ω t, Q 5 ω t = µ B Ŵ µ 4 ω t Q +. 3 ω t 46 A Theefoe, by 46, a path-wise lowe bound on 44 unde the admissible policy T is µ h a Q 3 t+h b Q 6 t+ B Ŵ µ 4 t Q t Q 6 t. 47 A 19