Optimal Stochastic Scheduling in Multiclass Parallel Queues

Transcription

1 Optimal Stohasti Sheduling in Multilass Parallel Queues Jay Sethuraman Mark S. Squillante Operations Researh Center IBM Researh Division Massahusetts Institute of Tehnology Thomas J. Watson Researh Center Cambridge, MA Yorktown Heights, Y jay@mit.edu mss@watson.ibm.om Abstrat In this paper we onsider the problem of sheduling different lasses of ustomers on multiple distributed servers to minimize an objetive funtion based on per-lass mean response times. This problem arises in a wide range of distributed systems, networks and appliations. Within the ontext of our model, we observe that the optimal sequening strategy at eah of the servers is a simple stati priority poliy. Using this observation, we argue that the globally optimal sheduling problem redues to finding an optimal routing matrix under this sequening poliy. We formulate the latter problem as a nonlinear programming problem and show that any interior loal minimum is a global minimum, whih signifiantly simplifies the solution of the optimization problem. In the ase of Poisson arrivals, we provide an optimal sheduling strategy that also tends to minimize a funtion of the per-lass response time varianes. Applying our analysis to various stati instanes of the general problem leads us to rederive many results, yielding simple approximation algorithms whose guarantees math the best known results. 1 Introdution The fundamental problem of sheduling a set of distributed resoures among different lasses of ustomers to ahieve some performane objetive has reeived and ontinues to reeive onsiderable attention in the researh literature. This is motivated by problems arising in a wide range of distributed omputer appliations and system environments, as well as ommuniation network environments. A partiular reent instane of the general problem is motivated by salable Web server systems where inoming Web requests are immediately routed to one of a set of omputer nodes by a high-speed router, and eah node independently exeutes the ustomers assigned to it following a loal sequening algorithm [6,9]. We onsider the problem of sheduling different lasses of ustomers on multiple distributed heterogeneouservers to minimize an objetive funtion based on per-lass mean response times. This optimal sheduling problem onsists of two distint deisions: (i) the alloation of ustomers to the parallel servers; and (ii) the order of exeution for the ustomers at eah server. The first deision has the flavor of a global load-balaning optimization problem in whih the ustomers are distributed among the multiple heterogeneouservers PermIssion to make digltal or hard opies of all or part of this work for personal or lassroom use is granted without fee provided that opnes are not made or distributed for profit or ommerial advantage and that opies bear this notie and the full itation on the first page To opy otherwse, to republish, to post on Servers or to redwribute to lists, requws prior speifi permission and/or a fee. SIGMETRICS 99 5/99 Atlanta, Georgia. USA ACM X/99/0004..,$5.00 to minimize the response-time objetive funtion. The seond deision is loal in nature and onsists of solving an optimal sequening problem: given a mix of ustomers at a server, determine the best order of servie for the queued ustomers to satisfy the global objetive. In our present study we onsider the struture of the optimal solution to the general problem of interest under the following restritions on these two deisions: l Alloation: Customers are alloated to servers in a probebilisti manner; i.e., immediately upon arrival, a ustomer is assigned to a server based on a matrix of routing probabilities. This is often alled random splitring [26]. l Sequening: The sequening strategy is non-antiipative (i.e., does not require knowledge of the future), work-onserving (i.e., does not idle when there is work to do) and non-preemptive (i.e., the exeution of a ustomer annot be interrupted and subsequently resumed). The objetive onsidered in this paper is to globally minimize a linear funtion of the per-lass mean response times. Similar tehniques an be used to minimize a linear funtion of the per-lass mean waiting times. Throughout this paper we use the terms ustomer and server in order to be ompletely general and not restrited to any partiular appliation area. Our analysis of this optimal sheduling problem begins with the observation that the optimal sequening strategy at eah of the servers is a simple stati priority poliy. Using this observation, we argue that the globally optimal sheduling problem redues to finding an optimal routing matrix under this sequening poliy. We formulate the latter problem as a nonlinear programming problem and show that it has at most one solution in the interior of the feasible domain and that any loal minimum in the interior is a global minimum. This result signifiantly simplifies the solution of the general optimal sheduling problem. We first restrit our attention to Poisson arrivals, in whih ase we derive an optimal sheduling poliy that also tends to minimize a funtion of the per-lass response time varianes. We then onsider the ase of general arrivals by developing a fluid-model formulation of the optimization problem and deriving an analogous set of results. The use of fluid models as approximations for queueing systems, often within the ontext of optimal ontrol, has reeived and ontinues to reeive onsiderable attention in the researh literature; e.g., see [ 11, 4, 1, 131 and the referenes ited therein. Related sheduling problems have been examined in the researh literature. Our sheduling problem is onsistent with or a generalization of the problems onsidered in [2, 16, 3, 6, 9, 51 and the relevant referenes therein. A number of these studies [6, 9, 51 have analyzed the performane of speifi poliies, as opposed to obtaining the globally optimal solution. Borst [3] onsiders the globally 93

2 suboptimal sheduling problem of finding the optimal routing matrix under an FCFS sequening poliy at eah server, within the ontext of a system model similar to the Poisson arrival instane of the model assumed in our study. Our analysis addresses the globally optimal sheduling problem using different methods than those of Worst and yielding a sheduling strategy that also tends to have better per-lass response time variane properties. Furthermore, our analysis of the fluid version of the optimization problem establishes a orresponding set of results that are not restrited to Poisson arrivals. The alloation omponent of the optimal sheduling problem is somewhat related to a global load-balaning optimization problem that has reeived onsiderable attention in the literature; e.g., see [25, 26, 2, 161 and the referenes ited therein. Ross and Yao [16] onsider a problem that is similar to a single-lass instane of the problem studied in this paper, with the addition of a dediated independent stream of ustomer arrivals to eah server having non-preemptive priority over the other ustomers. Bonomi and umar [2] onsider a model similar to that in [ 161 but with additional restritions, and in both studies the objetive is to minimize the average response time taken over the two sets of ustomers where eah arrival stream is a Poisson proess. We note that additional linear equalities, whih inludes the dediated stream of arrivals in [2, 161, an be easily aommodated in our approah. Hene, the sheduling problem, the models and the lass of objetive funtions onsidered in our study are more general than those examined in [2, 161. Moreover, we use different methods than those proposed in [2, 161 to solve the general optimal sheduling problem, and in the ase of Poisson arrivals we further address some properties of per-lass response time variane. We also onsider various stati instanes of the general optimal sheduling problem where a finite set of ustomers arrive at time 0 and there are no other arrivals, in whih ases the stohasti problems redue to the orresponding deterministi sheduling problems (where the proessing times are replaed by their expeted values) without loss of generality. Following our solution approah for these speial ases leads us to rederive many results in a fairly elegant manner, yielding simple approximation algorithms whose guarantees math the best known results. Our approximation algorithms are based on the use of randomized rounding on a onvex relaxation, whih is the first use of suh a relaxation in the sheduling literature to our knowledge. We obtain an e-improvement over the previously known algorithm due to Shulz and Skutella [17]. For the speial ase in whih all the servers are idential, our analysis provides an optimal losed-form solution to a simpler onvex relaxation. A derandomized version of our algorithm for this ase also yields the algorithm due to awaguhi and yan [lo]. Furthermore, we believe that improvements in the approximation guarantees for some of the speial ases onsidered will be possible by exploiting the onvex programming tehniques of our approah. The alloation strategy onsidered in this paper is stati in the sense that the routing probabilities do not hange dynamially with time nor do they depend upon the server queue lengths. While dynami alloation poliies have the potential to outperform stati poliies [12, 7, 231, implementing a dynami poliy an be nontrivial and these poliies an inur onsiderable overheads. Stati poliies may therefore be preferable in ertain pratial situations, suh as the distributed environments motivating our present study [6, 9,5]. The use of our optimal sheduling solution in pratie an also onsist of periodi adjustments of the routing matrix of the alloation strategy with hanges in the system environment, suh as variations in the workload. Moreover, given an optimal routing matrix, one an use an equivalent deterministi version of the probabilisti routing sheme to obtain lower (response time) variane properties in a real system. This approah is onsistent with that taken in [6,9] where a deterministi implementation of a stati load-balaning poliy is used together with eah omputer node periodially informing the router of hanges in its load. The sequening strategy onsidered in this paper is restrited to non-preemptive poliies. While preemption has the potential to improve mean response times, it an involve onsiderable overhead in pratie. We note, however, that the results presented in this paper diretly hold for preemptive sequening strategies under exponen- tial servie time distributions. Furthermore, it an be established that the optimal preemptive sequening poliy is a dynami indexing sheme based on the remaining servie times for the ustomers [ 191. The rest of our results should then hold together with this sequening strategy, whih is the subjet of future work. The remainder of this paper is organized as follows. We first onsider the general stohasti sheduling problem under independent Poisson arrival streams. Then in Setion 3 we remove this assumption of Poisson arrivals and onsider a fluid-model formulation of the general stohasti sheduling problem. Setion 4 presents an analysis of stati instanes of the general problem, and our onluding remarks are provided in Setion 5. 2 Poisson Arrival Case In this setion we define more preisely the optimal sheduling problem of interest under the assumption of Poisson arrivals, for whih we derive an effiient solution. We first present the orresponding system model and define the linear mean response time objetive funtion onsidered in our study. An analysis of the sequening and random splitting aspets of the optimal sheduling problem is then derived in Setions 2.2 and 2.3, respetively. We end this setion by developing an equivalent optimal sheduling poliy that tends to also minimize a funtion of the per-lass response time varianes. 2.1 The Model We onsider a system model onsisting of independent ustomer lasses and heterogeneous parallel servers. Throughout this paper, we will use i to index the ustomer lasses and j to index the servers under the onstraints i = 1,2,..., and j = 1,2,...,, unless noted otherwise. Customers of lass i arrive to the system from a Poisson soure with rate Xi. The total ustomer arrival rate is given by X =,,, X;. Eah ustomer is routed to one of the servers immediately upon its arrival aording to a probability matrix P = hll<i<~l~j<~ independent of all else; i.e., a lass i ustomer arrivdisindependently routed to server j with probability pij. The rate of ustomer arrivals to server j is therefore given by F=, X;pij. The servie time of a lass i ustomer when exeuted on server j has a general distribution on lr.+ given by Fij (.), mutually independent of the arrival and routing proesses. We assume that the servers differ only in their speeds, and we use sj to denote the speed of server j. Hene, Fij(t) = Fi(sjt); i.e., the base servie-time distribution of lass i ustomers is F;(t) with server j having servie rate sj. One started, eah ustomer is exeuted to ompletion without interruption; i.e., there is no preemption. Most of our analysis requires only the first and seond moments of the base servie-time distributions F;(e), whih we respetively denote by xi and zi2). When a lass i ustomer is exeuted on server j, the expeted servie time is given by Zij = xi/sj and the seond moment of the servie time is given by 5::) = xj )/s;. Let pi = X%x; be the traffi intensity for lass i. The traffi intensity at server j an then be expressed as 4j = F=r pipij. Sine sj is the apaity of server j, neessary and suffiient onditions for stability (i.e., finite expeted response times) are given by 4j = F PiPij < Sj, j = 1,2,...,. (1) 94

3 The total traffi intensity of the system is p = j =, &, and the total apaity of the system is S =,,i sj. It then follows diretly from (I) that the system is stable as long as p < S, whih we assume to be the ase in what follows. Objetive Funtion. A positive weight ; is assoiated with ustomers of lass i. The sheduling objetive of interest in this paper is to minimize the linear funtion Xi*. A within the ontext of the above system model, where 2 ; is the expeted total amount of time spent in the system by lass i ustomers. We shall refer to Ti as the mean response time for lass i. 2.2 Optimal Sequening Let us initially suppose that we have obtained the routing matrix P for the globally optimal sheduling poliy. It follows from the properties of the lass of alloation poliies under onsideration that the servers are equivalent to multilass M/G/l queues. In partiular, server j is a -lass non-preemptive M/G/l queue with per-lass arrival rates XiPij. The mean per-lass response times will depend on the sequening strategy employed at the server. Our interest is in finding the sequening strategy that minimizes the objetive funtion in equation (2). Smith [21] showed that the optimal sequening strategy whih minimizes suh a linear funtion of expeted response times in a multilass M/G/l queue is a fixed priority poliy, often referred to as the p rule. Speifially, he established the following result. Proposition 2.1 Consider a -lass non-preemptive M/G/I queue, with lass i arrival rate A; and lass i servie time distribution G; having mean x;. Let Ti denote the expeted response time of lass i ustomers, let i be a positive onstant assoiated with lass i, and define X = F=, Xi. Then, the sheduling poliy JII that gives priority to lass i ustomers over lass k whenever Ci / 2; 2 k 1x1s minimizes CF=, i +Ti. Observe from Proposition 2.1 that the sheduling poliy II whih minimizes the response time objetive funtion in (2) is the same for all servers. This follows from the assumption that the servers are idential exept for their speed and thus no lass is given preferential treatment by any server. For onveniene, and without loss of generality, we assume that the ustomer lasses are labeled suh that Cl - Y2, l-5? z Xl x2 X This labeling ensures that under the optimal sheduling poliy the priority ordering of the ustomer lasses at any server follows the index order: lass k is given priority over lass e if k <!. Sine we assume that servers differ only in their speed, the optimal ordering <j for server j is independent of j and is denoted by <. Then, from standard queueing theory (e.g., refer to [I 11, noting that our priority ordering of the ustomer lasses is the opposite of what is onsidered in [l l]), the mean waiting time of lass k ustomers in the multilass M/G/l queue of Proposition 2.1 is given by (2) This simple analysis shows that our optimal sheduling problem is as hard as finding the optimal routing matrix P. Observe also that the optimal sequening poliy is idential for all servers and does not depend on P. The optimal sequening poliy instead depends only on the first moment of the servie time distributions of the ustomers. nowing the optimal sequening poliy enables us to express the expeted response times of the ustomer lasses in terms of P. Thus, the globally optimal sheduling problem redues to the problem of finding the optimal routing matrix under the optimal sequening poliy, and this problem an be posed as an optimization problem whih is addressed in the next setion. 2.3 Optimal Random Splitting Based on the above analysis, we formulate the problem of finding an optimal routing matrix as a nonlinear programming problem. Let eah server order all of the ustomers assigned to it aording to the priority rule of Proposition 2.1 in Setion 2.2. ote that the arrival rate of lass i ustomers to server j is Xipij. From equation (3), the mean waiting time of a lass i ustomer at server j an then be expressed as Wij = where pkj = Xkpkjxkj. Xi&j Wi = C TWij F= =, AkPkjxg) 2(1 - k:k<i Pkj)( - Ck:k<i Pkj) We therefore have from whih together with (4) we obtain Ti = C Pij(Wij + Xij) = Wi + Cx;,Pij, =, AkPkjxg) 2(1 - k<i pkj)(l - xk_<; pkj) + Using the analysis and observations of Setions 2.1 and 2.2 together with (1) and (6), our optimal sheduling problem redues to finding the optimal routing matrix P* = j~$]r<~<~,r<~<~ that solves the following optimization problem: (RS) min C Ci 2 C pij pij f= =, AkPkjxE) 2(1 - Ck:k<ipkj)( = 1, Vi, (5) - k:k<i - Pkj) fxij wk = 2(1 -,,, d1 -,,, Pf) and the orresponding mean lass-k response time an be expressed as Tk = wk+xk. (4) (3) 2 XiXipij < Sj, Vj, pij > 0, Vi, j. The optimization problem (RS) is a nonlinear programming problem and it appears to be diffiult to solve in general. However, we 95

4 establish the following theorem whih onsiderably simplifies the task of solving this optimization problem. Theorem 2.2 Any loal minimum ofproblem (RS) in the interior of the feasible domain is a global minimum. Proof Sketh: Let 3(p) denote our objetive funtion, and thus CiXiPij 1 (l - Ck:k<i Pkj)( - Ck:k~i pkj) + Ci Xi pij Xij. 22 With a hange of variables, setting?jii = CL=, pkj, our original variables pij an be written in terms of uij as follows: Furthermore, the term plifies to pij = "ij - (i-l)j XiXij. Ci Xipij izl (l - k:k<i Pkj)( - Ck:k<; Pkj) sim- 1 Ci G+l l-uij (i+l).i I \ ( G-- J. Using these simplifiations, we an express our objetive funtion 3 as a funtion of the uij variables. We then take the Lagrangian of 3 by relaxing the assignment onstraints. Using 6i to be the multiplier orresponding to the ith ustomer lass, we an write the Lagrangian L for this problem as L(U, 6) = 3(U) f 2 di 5 uij ~i~~~el j. We now differentiate L(u, 6) with respet to the new variables uij and set the derivatives equal to zero. After some algebrai manipulations, we obtain an equation of the form (1 - Uij)2 ZZ rij, where rij does not depend on uij and is linear in the other variables 6i. Suh an equation has at most one root in the interval (0,l). n As noted above, Theorem 2.2 signifiantly simplifies the solution of the nonlinear programming problem (RS). For example, starting from an interior point and applying standard gradient methods, we an find a loal minimum of the objetive funtion. If this happens to be in the interior of the domain, then we are done; otherwise, we iterately apply this algorithm starting from all of the lower-dimensional faes. While this property still does not guarantee polynomial-time solvability, it will often lead to effiient solutions for many problems in pratie. 2.4 Minimizing Response Time Variane Let T: denote the mean response time of lass i ustomers under the optimal routing matrix P obtained from the solution of the optimization problem (RS) based on Theorem 2.2, and under the priority rule of Proposition 2.1 at eah server. Our analysis in the previous setions establishes that the performane vetor T = (T;,T,,...,TiT) minimizes the objetive funtion in equation (2). However, there learly an be multiple sheduling poliies that ahieve T*. In this setion, we develop a sheduling method that realizes T* while tending to also minimize a funtion of the per-lass response time varianes. Squillante and Tsoukatos [24] onsider an optimal sequening strategy for minimizing a funtion of per-lass seond moment measures of response time within the ontext of the multilass nonpreemptive M/G/l queue of Proposition 2.1, whih is formulated as an optimization problem under appropriate onstraints and is solved by applying the uhn-tuker Theorem. They show that a strutural property of the optimal solution is to equalize a per-lass funtion of the response time for eah individual ustomer, over all ustomers and all lasses. One an then argue, as in [24], that an approah whih tends to exhibit this strutural property for a partiular instane of the objetive funtion is based on the use of general timebased funtions to ontrol the alloation of resoures to lasses of ustomers [8]. Time-funtion sheduling is in part a generalization of the linear time-dependent priority disipline [ 1 l] in whih the priority of eah ustomer inreases (linearly) aording to a per-lass funtion of its time in the system and the ustomer with the highest instantaneous priority value in the queue is seleted for exeution at eah sheduling epoh. This is based on the observation that, under linear time-dependent priorities, ustomers tend to be given the server one they reah priority values whih are fairly similar aross all of the ustomers. In partiular, it an be easily established in the heavy traffi limit as p^ + 1 that a linear time-funtion sheduling strategy will satisfy [ 141 Uk wk = 7 for all lasses k, where Uk is the slope of the lass / time-funtion, p^is the traffi intensity of the queue, and 7 > 0. We therefore derive a partiular instane of time-funtion sheduling for the servers that, together with the optimal routing matrix P, ahieves T and also tends to minimize a funtion of the perlass response time varianes. Consider a stable multilass nonpreemptive M/G/l queue with lass /Z arrival rate ik and lass k servie time distribution Gk having mean & and seond moment?$ ; throughout this setion we will use the lass index k under the onstraints k = 1,2,...,, unless noted otherwise. A linear timedependent queueing disipline is employed with per-lass priority funtion Slopes (Tk suh that (~1 > u U. It an be easily shown that the expeted waiting time for ustomers of lass k in this queueing system is given by [ 1 l] -, =,+, &z(l - Ui/Uk) 1 - f-; p^l(lmfjk/ui) where & = ik& and p^ = Cf= =, &. ote the very simple depen- - dene that vk has on the slope parameters, namely that these slopes only appear as ratios. Without loss of generality, let U = 1. Following [Is, 221, we define Qlk = Pk = l-cp?i 9 = iizi2) w -PI i=k+l k--1-2 FiGi, i=k+l (7) 96

5 Equation (7) an then be rewritten as whih upon substituting the relations vk = vk+r - &/ok and /&+I =,& - & from the above definitions and simplifying yields Uk = -(wk@k+l - ak) Z~I vy (wkpk+l - Qk) + dukwkvk+l 2wkvk+l fork=1,..., -l.sineul>uz>...>u=l,wehave Uk = -(wkpk+l - ak) - ak) + duki?kvk+l 2wkvk,1 (S) fork = l,..., - 1. This expression an then be used in a straightforward manner together with the above definitions and relations to reursively obtain the values U-~,..., ui of the linear time-funtion poliy that will ahieve the given performane vetor (t?l,@z )..., 5i5.f). Hene, by Setting XI, = &p;,, zk = Xkj, $ = ze) and??k = W~j, where the values of WG orresponding to the optimal performane vetor T are known from our analysis in the previous setions (see equation (.5)), we have the desired result for the equivalent linear time-funtion sequening poliy at server j. 3 General Arrival Case We now remove the assumption of Poisson arrivals by onsidering a fluid-model approximation of the optimal sheduling problem onsidered in Setion 2. A fluid approximation of a stohasti system is its deterministi, ontinuous analog that models the asymptoti behavior of the queueing system. We first provide a formal desription of the orresponding fluid ontrol problem. Then an analysis of the sequening and random splitting aspets of the fluid optimization problem is derived respetively in Setions 3.3 and Fluid Model Consider the system model of Setion 2.1 with the following modifiations. Customers arrive and depart in a ontinuous, deterministi fashion, and thus an be thought of as aflow of&id; in this setion we will use the terms fluid and ustomer interhangeably. Customers of lass i arrive to the system in a ontinuous manner with rate Xi, and require Zj time units of proessing. A fixed fration nii of the lass i fluid is routed to server i aording to the routina -i&ix P s [Pij]l<i<.l<j<, independent of all-else. The tot2 amount of time required io>r<ess one unit of lass i fluid on server j is xi/sj. For t 5 0, we iet Lij (t) be the amount of lass i fluid at server j at time t, where the set of initial queue lengths Lij(O) are assumed to be given. Sine the quantities L,j(O) will be used often we drop the time argument and use Lij instead. Customers of lass i inur holding osts at rate i. In other words, i is the ost inurred by a lass i ustomer in the system per unit time. Our sheduling objetive now is to minimize the total holding osts inurred until the system empties for the first time, starting from the given initial state L;j. To do this we need to find the routing matrix P and the loal sequening strategies at the servers that result in a shedule of minimum ost., 1 To put this problem in perspetive, it is helpful to make a few remarks about the objetive funtion onsidered in the fluid and the stohasti systems. In a stohasti system, we typially would like to find a sheduling poliy that minimizes a linear funtion of response times in steady-state. By Little s law, this is equivalent to minimizing a (different) linear funtion of expeted queue lengths in steady-state. In the fluid model, however, we are interested in minimizing the total holding osts inurred until the system empties starting from a speified state, whih is related to the queue length proesses of the system. Hene, the hope is that the solution of the fluid model, as a funtion of the initial state, will be useful in determining a near-optimal poliy for ontrolling the stohasti system of interest. For example, fluid-model formulations have been reently used to suessfully study general stohasti sheduling problems [4, 1, 131 (different from those onsidered here). In the rest of this setion we formulate and prove strutural properties for the fluid model of the stohasti system onsidered in Setion 2.1. We do not address in this paper the question of how a fluid ontrol an be translated to a ontrol for the stohasti system; refer to [4, 1, Formulation We now an provide a preise formulation of the fluid approximation orresponding to the stohasti system of Setion 2. eessary and suffiient onditions for stability (i.e., the Auid system empties in finite time) are given by equation (1), and thus the system is stable as long as p < S whih we assume in what follows. The fluid ontrol problem orresponding to the stohasti system then an be formulated as follows: (CTL) min ilij(t)dt Lj(t) = Xipij - Z;jlUij(t), CUijCt) L l, L(0) = given, J-J(t), w(t) 2 0, t 1 0, where Lij (t) denotes the derivative of Lii (t) with respet to t. In our formulation pij and Uij(t) are the deision variables, where pij is the fixed fration of lass i fluid routed to server j, and Uij (t) desribes the fration of server j apaity alloated to servie lass i fluid at time t. Observe that if Lij(O) = 0 for all i, j (i.e., if all of the initial queue lengths are zero), the fluid optimal solution will inur zero ost. This is an immediate onsequene of the stability ondition (1). 3.3 Optimal Sequening Let us initially suppose that we have obtained the optimal routing matrix P. It follows from the properties of the lass of alloation poliies onsidered that the servers are equivalent to multilass fluid queues. Reall that our interest is in finding the sequening strategy that provides the globally optimal sheduling solution. Avram, Bertsimas and Riard [I] showed that the optimal sequening strategy whih minimizes a linear funtion of expeted response times in a multilass fluid queue is a priority poliy, often referred to as the CCL rule. Interestingly, the optimal poliy is the same for the stohasti system as long as the interarrival times are exponentially distributed. Speifially, they proved the following proposition. 97

6 Proposition 3.1 Consider a multilass~uid queue with ustomer lasses. Let Xi be the lass i arrival rate and let xi be the servie time of lass i ustomers, where pi = Xixi. Let Li (t) be the amount of lass i ustomers in the system at time t. Then, the shedulingpoliy n that gives priority to lass i ustomers over lass k whenever i/xi 2 k/~g minimizes Jam Cl,, ili(t)dt. Observe from Proposition 3.1 that the sheduling poliy II whih minimizes the weighted sum of fluid queue lengths is the same for all servers. This follows from the assumption that servers are idential exept for their speed and thus no lass is given preferential treatment by any server. Without loss of generality, assume that ustomers lasses are labeled suh that ;/Zi 2 i+l/~i+l. Under the optimal poliy, ustomer lass i is given priority over ustomer lass k if i < k. We now evaluate the ost of lass i ustomers in the multilass fluid queue of Proposition 3.1. We emphasize that we are looking at a partiular multilass fluid queue, namely the one desribed in Proposition 3.1. Observe that the fluid system gives priority to lass i ustomers over lass k ustomers if i < k. Sine the ustomer arrivals are ontinuous and deterministi, the fluid system will always serve ustomers. To better understand the sequening poliy we desribe it in more detail. Initially the fluid will serve ustomers of lass one. Sine the effetive arrival rate of lass 1 ustomers is smaller than the rate at whih they are served, lass 1 ustomers will eventually deplete; at this point the server (in the fluid system) is ready to serve lass two ustomers. Observe, however, that if the server devotes its full apaity to serve lass 2 ustomers, lass 1 ustomers will start aumulating, and will regain priority. Thus, in an optimal sequening poliy, the server will devote some of its apaity to maintain higher priority lasses at zero levels, while working on a lower priority ustomer lass. In our example, the server will devote some apaity to maintain lass 1 ustomers at level zero, and devote the rest of the apaity to lass 2 ustomers. Generalizing this, when the server has depleted all lasses up to lass (i - l), an appropriate fration of its apaity will be devoted to keeping all the higher priority ustomer lasses at level zero, and the remaining apaity will be devoted to lass i ustomers. (Contrast this with the optimal sequening poliy in the stohasti system.) Clearly, the amount of effort required to maintain a lass at level zero an be omputed easily: the server has to ensure that suh ustomer lasses deplete exatly at the rate at whih they arrive. f-p T and ui (t) = pi for t >!I i. This immediately shows that the total amount of effort required by the server to keep lasses one through (i - 1) empty is C,:,.+ pk, and thus U;(t) = (1 -,:,,; pk) for E-1 < t 5 Ti. Using the fat that lass i ustomers arrive at rate Xi, we see that Ti = Ti-1 + Pi(l- Li + Ti-lXi C k:k<i Pk) - Xi. Solving these sets of linear equations, we obtain Ti = (1 - t;,$ PAI (i: Lkxk > Using equation (9) for Ti, we an find the ost inurred by lass i ustomers as follows. The total inventory of lass i ustomers is the area under the urve shown in Figure 1. The total inventory of lass i ustomers is i LiTi- + Ti(Li + AiT;-1) I and thus the total ost of lass i ustomers is given by LiTi- +Ti(Li + AiT;-1), where Ti is given by equation (9). This simple analysis shows that our optimal sheduling problem is as hard as finding the optimal routing matrix P*. otie also that the optimal sequening poliy is idential for all servers and does not depend on P The optimal sequening poliy instead depends only on the weights and the servie times of the ustomer lasses. nowing the optimal sequening poliy enables us to express our objetive (the total ost inurred) in terms of P. Thus, as in Setion 2, the problem of finding the optimal routing matrix an be posed as an (nonlinear) optimization problem, whih is onsidered in the next setion. 3.4 Optimal Random Splitting Based on the above analysis, we formulate the problem of finding an optimal routing matrix as a nonlinear programming problem. Let eah server order all of the ustomers assigned to it aording to the priority rule of Proposition 3.1. Sine we assume that servers differ only in their speed, the optimal ordering <j for server j is independent of j and is denoted by <. For onveniene, and without loss of generality, we assume that the ustomer lasses are relabeled suh that 1>2$!& Xl x2 Z This labeling ensures that the priority ordering of the ustomer lasses at any server is the index order: lass i is given priority over lass k if i < k. Let T;j be the depletion time of lass i ustomers at server j. From our analysis in Setion 3.3, we have, (9) 0 T-1 Ti t Figure 1: Inventory level of lass i Let T; be the time at whih lass i ustomers deplete from the fluid system, and let pi = 2;. By the definition of depletion times, and by our ordering of the ustomer lasses, u;(t) = 0 fort 5 I i-1, Tij = sj(l - Ck:k<i Pkj) (&kjxk), (10) where pkj = Xkpkjxkj. The total osts inurred by lass i ustomers at server j are then given by + T;j(Lij + XipijT(i-l)j). (11) 98

7 Using the analysis and observations in Setions 3.1 and 3.3 together with (1) and (1 l), our optimal sheduling problem redues to finding the optimal routing matrix P z [plj^]l<i<;15j< that -- - solves the following optimization problem: (FRS) min FF*ij 2 p, j = 1, Vi, PE', > 0, Vi, j. The optimization problem (PRS) is a nonlinear programming problem and it appears to be diffiult to solve in general. However, we establish the following theorem whih onsiderably simplifies the task of solving this problem (for reasons analogous to those given in Setion 2.3). Theorem 3.2 Any loal minimum of problem (FRS) in the interior of the feasible domain is a global minimum. Proof : The proof follows by an argument idential to the one used in the proof of Theorem 2.2. n 4 Stati Problems In this setion we turn our attention to stati instanes of the stohasti sheduling problems onsidered in Setions 2 and 3. When we restrit ourselves to these stati ases in whih a finite set of ustomers arrive at time 0 and there are no other arrivals, a formulation based on the above analysis leads us to rederive many results yielding simple approximation algorithms whose guarantees math the best known results. These results have been previously summarized in [ 181, and they were independently obtained by Skutella in [20]. We first desribe the stati problems onsidered and provide an exat nonlinear formulation, and then we present a onvex relaxation of this formulation in Setion 4.2. We desribe a simple (randomized) sheme that rounds a frational solution of this relaxation to an integer solution in Setion 4.3, where we also prove performane guarantees for eah of the problems onsidered. ote that in investigating these stati problems, we an restrit ourselves to deterministi sheduling problems without loss of generality; for the stati stohasti problems, the proessing times are replaed by their expeted values, and thus they redue to the deterministi ase. 4.1 Problem desription and formulation We first formulate the sheduling problem under onsideration as an integer program. Consider a system with ustomers and servers. Viewing eah ustomer as belonging to its own lass, we use i to index the ustomers, whereas j ontinues to index the servers. The proessing requirement of ustomer i on server j is.zij; for onveniene, define /Aij = 2lj-l. We emphasize that our formulation is general enough to handle the following three ases of interest: (a) Idential servers: Zij = zi and is independent of j. (b) Uniform servers: Z;j = Zi/sj, where z; is the proessing time requirement of ustomer i (reall that sj is the speed of server j). () Unrelated servers: aij depends on both i and j and is an arbitrary positive integer. For the most part we will work in the setting of unrelated servers, as it is the most general ase. Sine the problem under onsideration is a stati problem, the alloation question redues to finding an optimal assignment of ustomers to servers at time zero. Reall that a positive (integer) weight i is assoiated with ustomer i and our objetive is to minimize x7=, i!pi, where Ti is the ompletion time of ustomer i. Let pij be an indiator variable whih is one if ustomer i is assigned to be proessed at server j and zero otherwise. Suppose for the moment that we know the optimal assignment of ustomers to servers - this enables us to redue the server sheduling problem to independent single server sheduling problems. For the objetive under onsideration, the single server sheduling problem is solved by the C,Q rule [21]: at server j, all the ustomers assigned to server j are sheduled in suh a way that ustomer I is sheduled before ustomer e if and only if Ckpkj > Cfptj. Motivated by this sequening poliy, we assume that eah server orders all the ustomers in suh a way that ustomer k appears before ustomer fj in the ordering orresponding to server j (denoted by k <j a) if CkPkj 1 LP!j- ote that we know the optimal ompletion time of a ustomer given an assignment of ustomers to servers. Thus, we an formulate our problem as that of finding the alloation vetor whih minimizes weighted ompletion time. Spefially, / \ min z z ipij (% + gz P.i.i) 2 pij = 1, Vi, Pij E (0, I}, Vi, j. A few remarks about this formulation are in order. First, observe that an appliation of the p rule yields Ti=Pij Zij + and thus the optimal solution to our nonlinear integer program does yield an optimal solution to the sheduling problem we started with. Seond, this formulation has potentially two ompliating fators: (i) the integrality restritions on the assignment variables; and (ii) the nonlinearity of the objetive funtion. It is quite easy to see that (i) is not a serious problem: a straightforward (randomized) rounding sheme an be used to prove the integrality of the relaxation in whih the pij an assume any value between zero and one (a proof of this is embedded in the proof of Theorem 4.1 in Setion 4.3). Hene, the nonlinear optimization problem given by: (LPR) min g $fpij (in + gipkjzjg) 2 pij = 1, Vi, Pij L 0, Vi, j; is an exat nonlinear formulation of our sheduling problem in the sense that one an always find an integral optimal solution to this 99

8 nonlinear programming problem. Although this is an interesting property, it does not simplify our problem beause our relaxation is a nonlinear programming problem and thus is diffiult to solve. In the next setion we will provide a onvex relaxation whih an be used to design approximation algorithms - of ourse, the known hardness results of this problem suggest that our onvex relaxation is truly a relaxation and does not always provide the optimal solution to the sheduling problems onsidered. 4.2 Convex relaxations To larify the exposition, we fix a server j and assume that our ustomers are labeled suh that 1 <j 2 <j... <j. Under these assumptions, the ontribution Xj of server j to the total ost an be Written ES Xj = xi =, Cipij (Zij i- ~k,xpkjzkj). When the Pij are restrited to be ert er zero or one, we have p$ = Pij. Simple algebrai manipulations yield k-l Ckpkjzljplj = CrPljzkjPkj. (12) L=l L=k+l 4.3 Rounding We then an write two expressions for Xj using these observations: = k-l C=l Ckzkjdj CtpLjzkjpkj. (14) L=k+l Adding equations (13) and (14), we have kzkjpkj C!zkjpkjplj + L=k+l k-l \ CkzljPkjPtj + CkzkjpEj ). (15) L=l / To see why Xj given by equation (15) is onvex, we first find the Hessian of Xj. The Hessian of Xi, denoted by Hj, is a x matrix with its (k, Z)th entry given by (Hj)kl = Clzkj Ckzlj eyk elk TO prove that Xj is onvex, it suffies to show that Hj is a positive semidefinite matrix. This is immediate from the ordering of the ustomers; reall that all ustomers were ordered so that rplj 2 CZ/JZj 2.*. > C/.hj, whih readily yields a proof of the semidefiniteness of Hj. In fat, if there are no ties, the Hessian is positive definite. This simple hange of using Pt instead of Pij at appropriate plaes has helped us onvert a nononvex funtion to a onvex one. While this hange does not affet the integer program, it results in a weaker relaxation, whih fortunately is onvex. The onvex relaxation of the parallel server sheduling problem we started with is: (UPM) min exj l=k+l ClzkjPkjPlj, Jj, (16) Xj 2 C CiZijpij, Vj, (17) pij = 1, Vi 5 Pij L 0, VC.7. All of the onstraints in the onvex relaxation are straightforward and follow from our disussion. We have added equation (17) to the relaxation beause it is a linear onstraint that ertainly yields a lower bound on the optimal value for the integer problem. This onstraint does not make a differene to the quality of the relaxation in the ase of idential or uniform servers, but it strengthens the relaxation for the ase of unrelated servers. The onvex relaxation proposed in the previous setion an be solved effiiently using standard tehniques. Given a solution to the relaxation, we onsider the following straightforward rounding algorithm. ALGORITHM RR Step 1: Solve the onvex programming problem (UPM), and obtain the optimum routing matrix P. Step 2: Route ustomer i to server j with probability * Pij. Theorem 4.1 Algorithm RRprodues a shedule whose ost is within a fator of $ of the optimal ost for all three versions of the parallel server sheduling problem. Proof: Let the optimum solution to the onvex program (UPM) be denoted by P* with optimum value Y = y=, XT. The integer solution resulting from the randomized rounding algorithm is denoted by p with value Y = ~~=, Xj. We will analyze the on- tribution of server j (to the total ost) and show that E[Xj] Sine we do independent rounding, we have 5 $X;. E[ISkj&j] = JQ%jIEb~jI. (18) Moreover, the ontribution of server j to the total ost an be expressed as Xj = Cipij (Zij + zkjpkj). (19) Using equations (18) and (19), we an ompute the expeted ost E[Xj] due to server j as follows. i-l k1 E[xj] = E 5 Ci@ij(Zij + ~ ZkjlSkj) k1 1 = Ci&j(Zij + C Zkjpkj) i-l I Ckzkj(pkj +pij) + = E [C;Zij&j] + C E [Cifhjpkjzkj] 100

9 i-l = G Zijp,fj + C Ciptp;j Zkj CI 1 t k1 I i-l * * ZZ CiZzjPlj + Cipijpkjzkj. m However, from onstraints (16) and (17) we have due to awaguhi and yan [lo], who prove a guarantee of F using ompliated arguments. Interestingly, for this speial ase, an optimal solution to the simpler onvex relaxation, namely (IPM) min exj ekzkjbkj +PEj) + I=k+l k1 I=k+l ClZkjPkjPlj, v.i, We then obtain from equations (12) and (21) XT - f ~~kzkj(p;j) + i CkakjpEj 2 i-l (22) GZijp:j + iprjpijzkj. (23) Substituting equation (22) in the left hand side of (23) yields i-l CiZijpzj f x x ip,fjpljzkj. (24) The right hand side of equation (24) is simply E[Xj]. The left hand side of equation (24) simplifies to %XT - f= =, kzkj (p;j). oting that f= =, Ckzkj(~i~)~ is a non-negative quantity yields E[Xj] < ZXT, n Remarks (a) We have desribed our rounding sheme assuming that we an find an optimal solution to the onvex relaxation. In pratie, however, we an only find an e-approximate solution. The same rounding sheme an be used with an e-approximate solution while retaining the same performane guarantee. (b) Our rounding sheme an be derandomized using the method of onditional probabilities. This derandomized rounding algorithm an be oupled with the e-approximate solution to atually find an integer solution with value no worse than 3/2 times the optimum. This is an -improvement over the previously best known algorithm due to Shulz and Skutella [ 171. () The argument leading to equation (20) also establishes the integrality of the nonlinear programming relaxation (LPR) Idential Servers For the speial ase in whih all of the servers are idential, the guarantee we an prove is still only 3/2. However, just as in [17], the derandomized version of algorithm RR is exatly the algorithm 5 pij = 1, Vii, Pij 2 0, Vi, j; an be omputed in losed form: setting pij = & yields an optimal solution to (IPM). Moreover, we an still prove that algorithm RR applied to an optimal solution of (IPM) will yield an integer solution whih is no worse than 3/2 times the optimum. Theorem 4.2 The onvexprogramming problem (IPM) an be solved in losed form for the speial ase of idential parallel servers. in this ase, l;ij = &for all i, j is an optimal solution. Proof: We prove this by showing that the solution fiij = & satisfies the uhn-tuker onditions. (This suffies beause our optimization problem is onvex.) For onveniene, we set X = ~~=, Xj and hi(p) = ~~=, equations pij - 1. It then suffies to show that the set of Ax@) + &&+.@) = 0 an be solved for the ni. Simple omputations show that the (i, j) equation in this system (orresponding to the ustomer i at server j) is izi 1 2y I 2i ;z: zk + 2zi C~=i+l k + Iii = o, whih is independent of j and hene an be solved for 21;. 5 Conlusions In this paper we studied the problem of sheduling different lasses of ustomers on multiple distributed heterogeneous servers to minimize a general objetive funtion based on per-lass mean response times. This problem arises in a wide range of distributed omputer appliations and system environments, as well as ommuniation network environments. We first observed within the ontext of our model that the optimal sequening strategy at eah of the servers is a simple stati priority poliy. We then argued based on this observation that the global sheduling problem redues to finding an optimal routing matrix under this sequening poliy. We formulated the latter problem as a nonlinear programming problem and showed that any interior loal minimum is a global minimum, whih signifiantly simplifies the solution of the optimization problem. In the ase of independent Poisson arrival streams, we provided an optimal sheduling strategy that also tends to minimize a funtion of n 101

10 the per-lass response time varianes. We then onsidered the ase of general arrivals by developing a fluid-model formulation of the optimization problem and deriving an analogous set of results. Applying our analysis to various stati instanes of the general problem led us to rederive many results yielding simple approximation algorithms whose guarantees math the best known results. Referenes VI VI [31 [41 PI F. Avram, D. Bertsimas, and M. Riard. Fluid models of sequening problems in open queueing networks; an optimal ontrol approah. In E elly and R. Williams, editors, Stohasti etworks, volume IMA 71, pages , F. Bonomi and A. umar. Adaptive optimal load balaning in a nonhomogeneous multiserver system with a entral job sheduler. IEEE Transations on Computers, 39( 10): , Otober S. C. Borst. Optimal probabilisti alloation of ustomer types to servers. In Proeedings of the ACM Sigmetris Conferene on Measurement and Modeling of Computer Systems, pages , H. Chen and D. D. Yao. Dynami sheduling of a multilass fluid network. Operations Researh, 41(6):1104--l 115, M. E. Crovella, M. Harhol-Balter, and C. Murta. Task assignment in distributed systems: Improving performane by unbalaning load. In Proeedings of the ACMSigmetris Conferene on Measurement and Modeling of Computer Systems, pages , June [6] D. Dias, W. ish, R. Mukherjee, and R. Tewari. A salable and highly available Web server. In Proeedings of the 1996 IEEE Computer Conferene (COMPCO), February [7] D. L. Eager, E. D. Lazowska, and J. Zahorjan. Adaptive load sharing in homogeneous distributed systems. IEEE Transations on Software Engineering, SE-12(5): , May [8] L. L. Fong and M. S. Squillante. Time-Funtion Sheduling: A general approah to ontrollable resoure management. In Proeedings of the Symposium on Operating Systems Pnmipies (SOSP), page 230, Deember [9] G. Hunt, G. Goldszmidt, R. ing, and R. Mukherjee. etwork dispather: A onnetion router for salable Internet servies. In Proeedings of the 7th International World Wide Web Conferene, April [lo] T. awaguhi and S. yan. Worst ase bound of an LRF shedule for the mean weighted flow-time problem. SIAM Journal on Computing, 15(4):1119-l 129, [ 1 l] L. leinrok. Queueing Systems Volume II: Computer Appliations. John Wiley and Sons, [12] M. Livny and M. Melman. Load balaning in homogeneous broadast distributed systems. In Proeedings of the ACM Computer etwork Performane Symposium, pages 47-55, [ 131 C. Maglaras. A methodology for dynami ontrol poliy design for stohasti proessing networks via fluid models. In Proeedings of IEEE Conferene on Deision and Control, [ 141 R. D. elson. Heavy traffi response times for a priority queue with linear priorities. Operations Researh, 38(3): , [I51 [161. W. Ross and D. D. Yao. Optimal load balaning and sheduling in a distributed omputer system. Journal of the ACM, 38(3):67&690, July u71 U81 [I91 PO1 M. Skutella. Semidefinite relaxations for parallel mahine sheduling. In Proeedings of the IEEE Symposium on Foundations of Computer Siene, ovember WI WI [231 ~241 M. S. Squillante and. l? Tsoukatos. Fundamentals of timefuntion sheduling. Unpublished Draft, September A.. Tantawi and D. Towsley. Optimal stati load balaning in distributed omputer systems. Journal of the ACM, 32(2): , April WI R. D. elson. Invertible mapping of waiting times in a M/G/l queue with linear priorities. Unpublished Draft, June A. Shulz and M. Skutella. Random-based sheduling. Tehnial Report 549/1997, Department of Mathematis, TU Berlin, J. Sethuraman and M. S. Squillante. Optimal sheduling of multilass parallel mahines. In Proeedings of the Tenth Annual ACM-SIAM Symposium on Disrete Algorithms, January Sevik. A proof of the optimality of smallest rank sheduling. Journal of the ACM, 21:66-75, W. E. Smith. Various optimizers for single-stage prodution. aval Researh and Logistis Quarterly, 3:59-66, M. S. Squillante, L. L. Fong, S. Liu, and S.. Ryan. A ontrol study of time-funtion sheduling: Part I. Tehnial Report RC 19765, IBM Researh Division, September M. S. Squillante and R. D. elson. Analysis of task migration in shared-memory multiproessors. In Proeedings of the ACM Sigmetris Conferene on Measurement and Modeling of Computer Systems, pages , May Y. T. Wang and R. Morris. Load sharing in distributed systems. IEEE Transations on Computers, C-34(3): ,