Approximations and Optimal Control for State-dependent Limited Processor Sharing Queues

Transcription

1 Approxiations and Optial Control for State-dependent Liited Processor Sharing Queues Varun Gupta Booth School of Business University of Chicago Jiheng Zhang Departent of Industrial Engg. and Logistics Manageent The Hong Kong University of Science and Technology Abstract The paper studies approxiations and control of a processor sharing (PS server where the service rate depends on the nuber of jobs occupying the server. The control of such a syste is ipleented by iposing a liit on the nuber of jobs that can share the server concurrently, with the rest of the jobs waiting in a first-in-first-out (FIFO buffer. A desirable control schee should strike the right balance between efficiency (operating at a high service rate and parallelis (preventing sall jobs fro getting stuck behind large ones. We eploy the fraework of heavy-traffic diffusion analysis to devise near optial control heuristics for such a queueing syste. However, while the literature on diffusion control of statedependent queueing systes begins with a sequence of systes and an exogenously defined drift function, we begin with a finite discrete PS server and propose an axioatic recipe to explicitly construct a sequence of state-dependent PS servers which then yields a drift function. We establish diffusion approxiations and use the to obtain insightful and closed-for approxiations for the original syste under a static concurrency liit control policy. We extend our study to control policies that dynaically adjust the concurrency liit. We provide two novel nuerical algoriths to solve the associated diffusion control proble. Our algoriths can be viewed as average cost iteration: The first algorith uses binary-search on the average cost and can find an ɛ-optial policy in tie O ( (log 1 ɛ 2 ; the second algorith uses the Newton-Raphson ethod for root-finding and requires O ( log 1 ɛ log log 1 ɛ tie. Nuerical experients deonstrate the accuracy of our approxiation for choosing optial or near-optial static and dynaic concurrency control heuristics. 1 Introduction Consider an eergency roo where doctors, nurses, and diagnostic equipent ake up a shared resource for aditted patients. It has been epirically observed that the service rate of such service systes is state-dependent (e.g., [5]. Huan operators tend to speed up service when there is congestion. As another exaple, consider a typical web server or an online transaction processing syste. In such resource sharing systes, as the nuber of tasks (also called active threads concurrently sharing the server increases, the server throughput initially increases due to ore efficient utilization of resources. However, as the server switches fro one task to another, it needs to ake roo for the new task s data in its cache eory by evicting an older task s data (only to fetch it again later. Without a liit on the nuber of concurrent tasks, this contention for 1

2 the liited eory can lead to a phenoenon called thrashing which causes the syste throughput to drop drastically (e.g., [2, 7, 12, 13, 21, 4]. The resource sharing syste exaples we have described above fall into the category of the so-called State-dependent Liited Processor Sharing (Sd-LPS systes. To specify an Sd-LPS syste, we begin with a processor sharing (PS server whose service rate varies as a function of the nuber of jobs at the server. For exaple, µ(1 = 1, µ(2 = 1.5, µ(3 = 1.25, µ(4 = 1, µ(5 =.75,... (1 When there are n jobs at the PS server, each job gets served at a rate of µ(n n jobs/second. To ensure efficient operation, we ipose a liit on the axiu nuber of jobs that can be served in parallel. We call this the concurrency liit, K. Arriving jobs that find the server busy with K jobs wait in a first-coe-first-served (FCFS buffer. A static concurrency control policy is one where the concurrency liit is independent of the state. If the concurrency level can vary with the syste state (e.g., the queue length of the FCFS buffer, we call it a dynaic concurrency control policy. To understand the tradeoff involved in choosing the optial concurrency level, suppose there are 3 jobs in the syste described above. Even though the server is capable of serving at an aggregate rate of 1.5 jobs/second by liiting the concurrency level to 2, we ay choose to increase the concurrency level to 3 and operate below peak capacity. Why ight we want to do that? It is well known that if the job size distribution has high variability, then pure PS outperfors FCFS scheduling by allowing sall jobs to overtake large ones. Therefore, it ay be beneficial to increase the concurrency level beyond the peak efficiency even if soe capacity will be lost. Siilarly, for job size distributions with low variability, it ay be beneficial to operate at K = 1. Thus Sd-LPS systes are not work-conserving queueing systes. Contributions Naturally, our goal is to choose the best concurrency control policy. In this work we ai to develop a diffusion approxiation fraework for Sd-LPS queues, and to utilize the proposed diffusion approxiation to find concurrency control policies that iniize the ean sojourn tie. This iediately leads to the question: Given that we want to control the state-dependent PS server (exeplified by (1, what is a eaningful asyptotic scaling and diffusion approxiation? While there are soe works on heavy-traffic asyptotics for queues with state-dependent rates, (i they begin with a sequence of systes with exogenously given liiting drift functions whereas we begin with a discrete PS server of the kind shown in (1, (ii they are liited to odels where the server can only serve one job at a tie whereas ultiple jobs are processed in parallel by the PS server; and (iii they only analyze a Jackson network type of syste and do not solve a diffusion control proble. The present paper fills these gaps in the literature. Our ain contributions are as follows: 1. We propose an axioatic approach to reverse-engineer a sequence of Sd-LPS queueing systes starting with a discrete state-dependent PS server (Section 2. This sequence yields a liiting state-dependent drift function which we utilize to develop diffusion approxiations. All prior literaure on diffusion analysis of state-dependent queues assues that the drift function is given exogeneously. 2. We propose an approxiation for the distribution of the nuber of jobs in the Sd-LPS 2

3 syste for a static concurrency liit under a GI arrival process and GI job sizes. This approxiation is used to choose a near-optial static concurrency liit to iniize any cost that is a function of the nuber of jobs in the syste. 3. We extend our fraework by proposing a ore general scaling for developing dynaic (statedependent control policies. We present two nuerical algoriths for solving the resulting diffusion control proble to iniize the steady-state ean nuber of jobs in the syste. Our algoriths can be viewed as average cost iteration (as opposed to value function iteration or policy iteration and are novel to the best of our knowledge. In our siulation experients, the dynaic policies based on diffusion control perfor rearkably close to the true optial dynaic policies (for input distributions where the true optial policy can be coputed nuerically. Related work on control of LPS systes The literature on LPS-type systes has ostly focused on the constant rate LPS queue where the server speed is independent of the state. Yaazaki and Sakasegawa [42] show qualitatively the effect of increasing the concurrency level on the ean sojourn tie for NWU (New-Worse-than- Used and Erlang job size distributions. Avi-Itzhak and Halfin [4] derive an approxiation for the ean sojourn tie for the constant rate LPS queue with M/GI/ input process, while Zhang and Zwart [43] derive one for GI/GI/ input. Nair et al. [34] expose the power of LPS scheduling by analyzing the tail of sojourn tie under light-tailed and heavy-tailed job size distributions. They prove that with an appropriate choice of the concurrency level as a function of the load, LPS queues can achieve robustness to the distribution of job sizes (their tail to be precise. For Sd-LPS queues, Rege and Sengupta [38] derive expressions for the oents and distribution of the sojourn tie under M/M/ input. Gupta and Harchol-Balter [17] propose an approxiation for the ean sojourn tie for GI/GI/ input by approxiating the interarrival ties and job size distribution by the tractable degenerate hyperexponential distribution. They also propose heuristic dynaic adission control policies under M/GI/ input. In this paper, we propose the first diffusion approxiation for Sd-LPS queues with a GI/GI input and a static concurrency level. In addition, we propose the first heuristic dynaic adission control policies for Sd-LPS queues. Related work on control of queueing systes There is a considerable literature on the control of the arrival and service rates of queueing systes, but the ajority of this work focuses on control of M/M/1 or M/M/s systes via Markov decision process forulation, e.g., [1, 3, 16, 31]. Ward and Kuar [39] look at the diffusion control forulation for adission control in a GI/GI/1 with ipatient custoers. Our odel differs significantly fro those in the literature: in our odel, the space of actions is the nuber of jobs aditted to the PS server and is therefore state-dependent. The action space for control probles studied in the literature is usually either the probability of aditting jobs or the server speed, neither of which is state-dependent. The state-dependence of the action space eans that the value function ay not even be onotonic in the state. We establish this result for our proble and present a siple criterion under which onotonicity holds for general control probles with state-dependent action spaces (see proof of Proposition 3. In addition, the rather arbitrary nature of the service rate curve precludes elegant structural results for the optial value function which leads us to propose novel and efficient nuerical algoriths for solving the resulting diffusion control proble. 3

4 Related work on heavy-traffic analysis of systes with state-dependent rates Our heavy-traffic scaling is ost closely related to the recent work of Lee and Puhalskii [29], who analyze a queueing network of FCFS queues in the critically loaded regie and under non- Markovian arrival and service processes. Yaada [41] also analyzes Markovian state-dependent queueing networks under a siilar scaling of state-dependent service and arrival rates. Whereas [29, 41] assue an exogenously given liiting drift function, we propose a ethod to calculate it fro the finite queueing syste which is the object of the control proble. Further, the scheduling policy we consider is Processor Sharing. Other works on analysis of heavy-traffic asyptotics of state-dependent Markovian queues include Krichagina [27], Mandelbau and Pats [32], Janssen et al. [23]. Outline In Section 2 we present details of the Sd-LPS odel, introduce the notation used in the paper, and describe our approach towards arriving at the asyptotic regie for diffusion analysis. In Section 3, we present our results on diffusion approxiation for the Sd-LPS queue under a static concurrency control policy. We defer the proofs of convergence to the appendix. In Section 4 we turn to dynaic concurrency control policies for the Sd-LPS queue. We set up a diffusion control proble for the liiting diffusion-scaled syste, and propose two nuerical algoriths to solve the diffusion control proble. We ake our concluding rearks in Section 5. 2 Model and Diffusion Scaling 2.1 Stochastic odel and Notation We begin with a description of the Sd-LPS syste for which we want to find the optial control. Let X(t denote the total nuber of jobs in the syste at tie t. The control of such a syste is ipleented by iposing a concurrency liit K. Only Z(t = X(t K jobs are in service and server capacity of µ(z(t is shared equally aong the jobs. The reaining Q(t = (X(t K + jobs wait in a FCFS queue. A job, once in service, stays in service until copletion. The rate of the server µ(z(t is understood to be the speed at which it drains the workload. So the cuulative service aount a job in service can receive fro tie s to t is where S(s, t = ψ(z = s { µ(z z ψ(z(τdτ, (2, if z,, if z =. Without loss of generality, we assue that there is no intrinsic liit on the nuber of jobs the server can serve as we can set the service rate to to odel such a liit. Note that for the regular state-independent syste whose service rate µ( is a constant, say 1, µ(z z in the above will siply becoe 1/z. The state-dependent service rate akes the syste non-work-conserving, which brings a fundaental challenge to their study. Existing studies of PS or LPS systes crucially rely on the fact that the syste is work-conserving, which iplies that the workload process is equivalent to that of a siple G/G/1 queue. However, this is not the case for our Sd-LPS odel. The nuber of job arrivals in tie [, t] is denoted by Λ(t. We assue that Λ( is a renewal process with rate λ, and c 2 a denotes the squared coefficient of variation (SCV for the i.i.d. inter-arrival 4 (3

5 ties. The syste is allowed to be non-epty initially. We index jobs by i = X( + 1, X( + 2,...,, 1,.... The first X( jobs are initially in the syste, with jobs i = X(+1,..., Q( in service and jobs i = Q(+1,..., waiting in the queue. Arriving jobs are indexed by i = 1, 2,.... The size of the ith job is denoted by v i. We assue job sizes are i.i.d. rando variables with ean size (in the chosen unit easuring work and SCV c 2 s. Jobs leave the syste once the cuulative aount of service they have received fro the server exceeds their job sizes. In this study, we are interested in how the syste perforance (e.g., expected nuber of jobs in steady state depends on the state-dependent service rate function µ(, the paraeters (λ, c 2 a,, c 2 s of the stochastic priitives, and the concurrency level K, which is a decision variable we can control and optiize. Measure-valued state descriptor Analyzing the stochastic processes underlying the Sd-LPS odel with generally distributed service ties requires tracking of ore inforation about the syste state than just the nuber of jobs. Following the fraework in [44, 45], we introduce a easure-valued state descriptor to describe the full state of the syste. At any tie t and for any Borel set A (,, let Q(t(A denote the total nuber of jobs in the buffer whose job size belongs to A and Z(t(A denote the total nuber of jobs in service whose residual job size belongs to set A. Thus, Q( and Z( are easure-valued stochastic processes. Let δ a denote the Dirac easure of point a on R and A + y. = {a + y : a A}. By introducing the easure-valued processes, we can characterize the evolution of the syste via the following stochastic dynaic equations: Q(t(A = Λ(t i=b(t+1 Z(t(A = Z((A + S(, t + δ vi (A, (4 B(t i=b(+1 where τ i is the tie when the ith job starts to receive service and δ vi (A + S(τ i, t, (5 B(t = Λ(t Q(t, (6 which can be intuitively interpreted as the index of the last job to enter service by tie t. For any Borel easurable function f : R + R, the integral of this function with respect to a easure ν is denoted by f, ν. Then, both Q(t and Z(t can be represented using the easure-valued descriptors: Q(t = 1, Q(t, Z(t = 1, Z(t. Let W (t denote the workload of the syste at tie t which is defined as the su of the sizes of all jobs in queue and the reaining sizes of all jobs in service. Due to the varying service rate of the server, the dynaics of the workload process is represented by Λ(t W (t = W ( + v i i=1 µ(z(s1 {W (s>} ds. (7 Again, we can express the workload W (t in ters of the easure-valued descriptors: where χ denotes the identity function on R. W (t = χ, Q(t + Z(t, (8 5

6 2.2 Proposed Asyptotic Regie for Diffusion Approxiation of Sd-LPS systes We refer to the syste introduced in Section 2.1 as our original syste. We now propose an asyptotic regie where a sequence of Sd-LPS systes, paraetrized by r Z +, will be studied under an appropriate scaling. The objective is to a obtain a eaningful approxiation of the original syste with the goal of choosing the best concurrency control policy. This leads to the question: What is the appropriate scaling to analyze the Sd-LPS queue? That is, what asyptotic regie captures the entire service-rate curve of the original Sd-LPS syste, and thus can be used to find a near-optial concurrency liit? As we entioned earlier, the scaling we develop is very close to the scaling proposed by Yaada [41] and Lee et al. [29]. To provide an axioatic justification for why this is the appropriate scaling for Sd-LPS systes we begin by exaining two special cases of Sd-LPS systes: (i ultiserver systes, and (ii the constant rate LPS queue. The G/GI/k ultiserver syste A G/GI/k ultiserver syste with a service rate of µ jobs/second per server and a central buffer can be viewed as an Sd-LPS syste with µ(n = nµ and a concurrency liit of K = k. There is a rich literature on the question of whether having any slow servers is better than having a few fast servers (e.g., Bruelle [9], Daley and Rolski [11], which is siilar in spirit to the concurrency control proble. The work on diffusion approxiations for ultiserver systes started with Köllerströ [25] for the classical heavytraffic regie where k and µ are held constant while λ kµ. A ore refined heavy-traffic regie is the Halfin-Whitt regie (starting with [18] and ore recently [37], [14] where one fixes µ and creates a sequence of ultiserver systes paraetrized by r, where the nuber of servers grows according to k (r = rk while the ean arrival rate λ (r increases so that k (r µ λ (r k (r β. The constant β is chosen so that the probability that an arrival gets blocked converges to a non-degenerate liit (bounded away fro and 1. An extreely accurate diffusion approxiation for a given G/M/k syste can be obtained by atching the blocking probability under the Halfin-Whitt regie. State-independent (constant rate LPS queue In the state-independent LPS queue, the service rate of the server is a constant µ irrespective of the nuber of jobs at the server, and there is a fixed concurrency liit k. Recently, Zhang et al. [45] have proposed and analyzed a diffusion approxiation for the LPS syste where a sequence of LPS systes (paraetrized by r is devised so that the service rate reains fixed at µ, the concurrency liit increases according to k (r = rk and the arrival rate increases so that k (r (µ λ (r θ, a constant. As in the Halfin-Whitt regie for the ultiserver systes, under the proposed scaling for LPS systes the probability that an arrival finds all slots at the PS server occupied converges to a constant. In addition, the queue length scaled by 1 also converges to a non-degenerate k (r distribution, unlike Halfin-Whitt where the queue lengths are saller and ust be scaled by 1. k (r It is not obvious how either of these scalings can be extended to the Sd-LPS syste, but aong the chief desiderata is that the diffusion-scaled syste should in soe sense be a faithful proxy for the original syste. As an exaple of a regie that is not quite faithful enough, consider the following 6

7 exaple: We scale the concurrency liit as k (r = rk, leave the ean arrival rate λ unchanged, and stretch the service rate curve so that for the rth Sd-LPS syste, µ (r (rx = ˆµ(x where ˆµ( is a continuous interpolation of µ(. This would correspond to a fluid liit where the steady state gets stuck around x, where ˆµ(x = λ. Therefore this fluid regie cannot be used to devise a control policy for the original Sd-LPS syste. Instead, we adopt an axioatic approach to devising the asyptotic regie: Under soe reasonable non-trivial assuptions A, the behavior B of the diffusion-scaled syste should iic the original discrete syste we want to approxiate. The choice of A and B can be seen as the axios of our scaling which we will use to reverse-engineer a diffusion scaling. We now foralize our choice of the assuptions A and behavior B for Sd-LPS systes with static concurrency control. Later we will use the intuition gained fro this exercise to engineer a scaling for developing dynaic control policies. Axios for the Sd-LPS diffusion scaling We construct a sequence of Sd-LPS systes paraetrized by r Z + such that the rth syste has a concurrency level of k (r = rk. (9 Further, the sequence of service rate curves µ (r ( satisfies: (A, the assuptions under a Poisson arrival process with rate λ and i.i.d. Exponentially distributed job sizes with ean size (i.e., under M/M/ input, (B, the behavior the distribution of the scaled nuber of jobs in the syste (scaled by 1 k (r converges to a non-degenerate liit. Consequences of the scaling axios and an alternate characterization Since we start by fixing the concurrency levels for the sequence of Sd-LPS systes, the only design flexibility we have to satisfy the scaling axios is the choice of state-dependent service rate curves. Let us denote the service rate curve for the rth syste by µ (r (i, and the resulting distribution of the nuber of jobs in the rth syste under M/M/ input by F (r. Our goal is to find the sequence µ (r ( so that li F (r ( rx = ˆF (x x [, (1 r for soe distribution function ˆF (. This gives us our first way of deriving the scaling: Fix ˆF ( to be a sooth, strictly increasing interpolation of the distribution function of the original syste under M/M/ input and reverse-engineer the sequence of service rate functions µ (r (i. As we will show in the next section, the requisite service rate functions satisfy ( li r λ µ (r d log f(x ( rx = λ r dx x [,. (11 where f(x = d dx ˆF (x. Of course, by reverse-engineering the scaling, we guarantee ourselves a non-degenerate liit that is interesting in that it captures the effect of the entire µ( function. Further, it turns out we never really need to copute the service rate functions µ (r (! In Section 3, we will show that we can directly express the liiting steady-state quantities in ters of the distribution ˆF (, which can be easily obtained fro that of the original syste. 7

8 However, the ethod described above does not generalize to dynaic control policies since the distribution F ( (and its sooth interpolation ˆF ( was obtained under the assuption of a static concurrency liit of K. For this case, we propose a second way of deriving the scaling, that still guarantees (1: Begin with ˆµ( : R + R+ satisfying: 1. ˆµ( agrees with µ( at integer arguents: ˆµ(i = µ(i for i = {1, 2,...} 2. ˆµ( is continuous and sooth The sequence of service rate functions {µ (r ( } is chosen to satisfy ( li r λ µ (r ( rx = λ log λ r ˆµ(x x [,. (12 For either way of arriving at the diffusion scaling, we see that r(λ µ (r ( rx converges to a non-degenerate drift function θ(x. In the first case the θ(x function is reverse-engineered by fixing a liiting distribution, and in the second case it is obtained ore directly using a continuous extension of µ(i. The first is ore appropriate for finding static control policies, while the second is ore appropriate for coputing dynaic control policies. In both cases there is liited flexibility in extending a discrete function to a continuous sooth function. Intuitive explanation for the choice of service rate curves µ (r ( We begin by explaining our first choice of the asyptotic regie (11. Consider the rth Sd-LPS syste operating under M/M/ input. Let π (r (i be the probability ass function for the steadystate nuber of jobs in the rth syste. Flow-balance equations iply π (r ( rx + 1 π (r ( rx = λ µ (r ( rx. Since, by design, we want rπ (r ( rx to converge to the density function f(x, we should have: λ µ (r ( rx f Equivalently, r(λ µ (r d log f(x ( rx λ dx. To otivate the second proposal of µ (r (, note that if Or, ( x + 1 r f (x f(x r f(x. λ µ (r (r x 1 θ(x θ(x λr e λr, then π (r (ry π (r (rx e 1 y λ x θ(udu π(y π(x = λ Πy i=x+1 µ(i. 1 y θ(udu log π(r (ry π(y λ x π (r log (rx π(x = y i=x+1 log λ µ(i. Coparing the first and last expressions above gives us an approxiation for θ(u in ters of a continuous extension ˆµ of µ: r(λ µ (r ( rx λ log λ ˆµ(x. 8

9 Coparison with existing diffusion scalings For the two specific exaples of Sd-LPS systes we pointed out earlier we can now ask: What axios do the existing diffusion scalings satisfy? The ˇiplicit axios used by [18] posit that each syste in the sequence is itself a hoogeneous ultiserver syste, and further, under M/M/ input the blocking probability of the sequence converges to a non-degenerate liit (for exaple, to the blocking probability of the finite syste being approxiated. If we use our proposed scaling to approxiate a ultiserver syste, the sequence of Sd-LPS systes would not be a hoogeneous ultiserver syste. Indeed, r(λ µ (r (rx grows as log(x for sall x. On the positive side, this flexibility allows us to capture the entire distribution of nuber of jobs, not just the blocking probability. We should also point out that while the Halfin-Whitt scaling is ore useful for capacity provisioning, the goal of our proposed scaling is to solve the adission/concurrency control proble. For the constant rate LPS syste, our scaling atches the diffusion scaling of Zhang et al. [45], and thus can be seen as an extension of their scaling to Sd-LPS. Our convergence proofs follow their outline as well. 3 Diffusion approxiation for the Sd-LPS queue with a static concurrency level The goal of this section is to provide approxiations for the steady-state perforance of the Sd- LPS queue with a static concurrency level under the proposed scaling (11. In Section 3.1 we first suarize the results of this section by giving an approxiation for the ean nuber of jobs in an Sd-LPS syste under a static concurrency level (equation (13, and providing soe siulation results which show the utility of the approxiation for choosing a near-optial concurrency level. In Section 3.2, we prove process-level liits for diffusion-scaled workload and head count processes. In Section 3.3, we justify using the steady state of the liiting processes as an approxiation for the liit of the steady state of the diffusion-scaled processes by establishing the required interchange of liits. We also present closed-for forulae for these steady-state distributions. All the proofs for this section can be found in the appendix. 3.1 An approxiation and siulation results Let N denote the steady-state nuber of jobs in the Sd-LPS syste for a given static concurrency level K. Our ain result of this section yields the following siple approxiation forula for the expectation of N as a function of the concurrency level and other syste paraeters (see Proposition 2 for the foral stateent E[N] n= (n Kπ(n c 2 s +1 c 2 s +c2 a n= π(n c 2 s +1 c 2 s +c2 a ( c 2 + s + 1 n= (n K+ π(n 2 c 2 n= π(n s +1 c 2 s +c2 a c 2 s +1 c 2 s +c2 a, (13 where π(n denotes the steady-state probability of there being n jobs in the Sd-LPS syste under M/M/ input (that is, Poisson arrivals with ean rate λ and i.i.d. Exponentially distributed job sizes with ean size. Figure 1 shows a hypothetical service rate function for a PS server. The service rate has the functional for µ(i = 1.25 i2 15, and is onotonically decreasing in the concurrency level. Figure 2 shows the siulation results for the steady-state ean nuber of jobs as a function of the concurrency level K. The arrival process is Poisson with ean arrival rate shown below the figures. 9

10 1.2 Service rate µ(i Nuber of jobs at the PS server (i Figure 1: State-dependent service rate function used for siulation results We siulated three distributions, each with ean = 1 and SCV c 2 s = 19. The solid curve shows the diffusion approxiation (13 for the ean nuber of jobs. For each value of λ and each distribution, the optial concurrency level obtained via approxiation (13 atches the one obtained fro siulating the LPS syste. We should point out the caveat that while the proposed diffusion approxiation accurately captures the shape of E[N] versus the concurrency level curve and thus provides good guidance for concurrency control, the actual nuerical values for E[N] are not always very accurate for all values of K. E[N] Pareto 1.1 Lognoral Weibull Diff. approx. E[N] Pareto 1.1 Lognoral Weibull Diff. approx. E[N] Pareto 1.1 Lognoral Weibull Diff. approx Concurrency level (a λ = Concurrency level (b λ = Concurrency level (c λ =.9 Figure 2: Siulation results for ean nuber of jobs in the syste versus the concurrency level for the service rate function shown in Figure 1 for various job size distributions, all with ean = 1 and SCV c 2 s = 19. The arrival process is Poisson with indicated ean arrival rate λ. Also shown is the diffusion approxiation fro equation (13. The optial concurrency level for each curve is shown with a circle. 3.2 Diffusion analysis of Sd-LPS syste We now present the analysis of the Sd-LPS syste under the asyptotic regie described in (11. For generality and notational convenience, we present all the analysis in ters of the general drift function θ(x, and then translate the result into a for involving f(x (Proposition 2 for convenience. Consider the sequence of Sd-LPS systes indexed by r. We append a superscript (r to all the quantities associated with the rth syste. The concurrency level k (r is specified as (9. 1

11 Assue that the arrival process Λ (r ( satisfies Λ (r (r 2 t r 2 λt r M a (t, as r, (14 where M a ( is a Brownian otion with zero drift and variance c 2 a. Further, we assue that the sizes of arriving jobs follow distribution G which satisfies Introduce the drift function G is a continuous distribution function with ean. (15 θ (r (x = { r ( µ (r ( rx λ x >, x =. The above definition is only for technical convenience, since otherwise µ (r ( would be undefined. However, this does not atter since the server idles when there are no jobs in the syste. The heavy traffic condition is specified by θ (r (x u.o.c θ(x as n, (16 for soe locally Lipschitz continuous function θ( on (, satisfying θ(k >. (17 The notation u.o.c eans unifor convergence on copact sets, which is only required for technical reasons. Condition (17 ensures that the syste is stable (see the proof of Theore 2. As a quick reark, we ake a connection with the traditional single server syste where the server speed is constant, say µ (r ( 1, and the drift is created by constructing a sequence of λ (r which converges to λ at the rate of 1/r. The heavy traffic condition for this constant rate LPS syste then becoes ( r 1 λ (r θ >, as r. We are interested in the asyptotic behavior of the diffusion-scaled processes for the rth syste, defined as ˆX (r (t = 1 r X(r (r 2 t, Ŵ (r (t = 1 r W (r (r 2 t. (18 The diffusion scaling for other stochastic processes Q (r, Z (r, Z (r, Q (r and B (r is defined in the sae way. To obtain the diffusion liit of the head count process ˆX (r and workload process Ŵ (r, we need to carefully analyze the easure-valued processes introduced. The detailed analysis is presented in Appendix B. Since we need to work with the easure-valued process, let ν denote the probability easure associated with the probability distribution function G, and ν e denote the probability easure associated with the equilibriu distribution G e of G. That is, G e (x = 1 x [1 G(y]dy and the ean of G e is e = 1 + c2 s. 2 11

12 Let M denote the space of all non-negative finite Borel easures on [,. We need the following regularity assuptions on the initial state to rigorously prove the diffusion approxiation results. Assue there exists (ξ, µ M M such that ( ˆQ (r (, Ẑ(r ( (ξ, µ, (19 χ 1+p, ˆQ (r ( + Ẑ(r ( χ 1+p, ξ + µ for soe p >, (2 as r, and ( w (ξ, µ K e = ν, (w K e + ν e, (21 e where w = χ, ξ + µ. The above regularity assuptions (19 (21 basically require that the sequence of initial states is well behaved. These assuptions, together with the heavy traffic assuptions (14 (16, are ade throughout the rest of this paper. The first result we present is an asyptotic relationship, called State Space Collapse (SSC, between the workload process and the head count process. Define a ap K ( : R + R + by K (w = w K e + (w K e +. (22 e The SSC result states that the total nuber of jobs in the syste ˆX (r can be asyptotically represented using the workload Ŵ (r via the ap K, which is a bijective ap eaning that workload can also be represented using the total nuber of jobs. SSC is described as follows: Proposition 1 (State Space Collapse For the sequence of Sd-LPS systes paraetrized by r Z + and satisfying initial conditions (19-(21, as r, ( ˆX (r (t K e + ( ˆX (r (t K + Ŵ (r (t. (23 Note that sup t [,T ] 1 K (x = (x K e + (x K + is the inverse of the ap K (. A full version of the SSC, which deonstrates a bijective ap between the workload Ŵ (r and the easure-valued status ( ˆQ (r, Ẑ(r, is presented and proved in Appendix B. Roughly speaking, SSC reveals that the residual sizes of jobs in service follow the equilibriu distribution G e. The sipler SSC of Proposition 1 can be derived fro the full version proved in Appendix B. For the purpose of perforance analysis and for optial control in this paper, we only need the siple version of SSC. The next step is the analysis of the workload process defined in (7. The challenge here is that the evolution of workload depends on the nuber of jobs in service due to the state-dependent service rate. The siple SSC result allows us to overcoe this difficulty. The following theore establishes the diffusion liit of the workload process Ŵ (r (t and the nuber of jobs ˆX (r as reflected Brownian otion (RBM with state-dependent drifts. Theore 1 (Weak convergence to RBMs with state-dependent drift For the sequence of Sd-LPS systes paraetrized by r Z + satisfying (19-(21, as r, Ŵ (r W, (24 12

13 where W is an RBM with initial value W ( = w, drift θ ( K (W (t K and variance σ 2 = λ 2 (c 2 a + c 2 s. Moreover, as r, ˆX (r X = K (W. (25 The proof of Theore 1 is presented in Appendix B. 3.3 Steady State of the Diffusion Liit The entire goal of heavy traffic analysis is to obtain a tractable process, an RBM with statedependent drift, as an approxiation of the coplicated stochastic process underlying the original odel. That is, the steady state of the liiting RBM can be coputed. We begin with a basic result on the steady-state distribution of an RBM with state-dependent drift and variance (Lea 1. We then use this lea to derive the steady-state distribution and ean of the liiting workload and nuber of jobs (W and X in ters of the priitives of the original Sd-LPS odel. Lea 1 The stationary distribution of a one-diensional RBM, W, with state-dependent drift β( and state-dependent variance s( is given by w Pr[W ( w] = α e u β(v+ 2 1 s (v 1 dv w 1 u 2 s(v du = α s(u e where α is a noralization constant. β(v 1 2 s(v dv du, (26 The proof of this lea is presented in Appendix B. In our setting, the drift β(w = θ ( K (w and the variance s(w = σ 2 = λ 2 (c 2 a + c 2 s is a constant. Using Lea 1, we iediately have the closed-for expression for the steady-state W ( of the diffusion liit W in ters of the drift function θ( : Corollary 1 With W as defined in Theore 1, w α e 2 u θ(v/e σ 2 dv du w K e, Pr[W ( w] = ( w α e 2 Ke θ(v/e σ 2 dv e 2θ(K(u Ke σ 2 du w > K e. (27 To present the result for the liiting steady-state quantities in a for that is easier to apply in practice, we express the drift function θ( as follows: d log f(x θ(x = λ dx (28 where, recall, the function f( represents the derivative of a twice-differentiable interpolation of the distribution of steady-state nuber of jobs for the original Sd-LPS syste with concurrency level K under M/M/ input (see the discussion on the asyptotic regie in Section 2.2 preceding d log f(x equation (11. We assue that dx is a constant less than on the interval [K, (it is easy to verify that such an extension exists. It turns out that while we need f( to be differentiable to define θ(, the liiting steady-state distribution is well defined even without this condition. Finally, we obtain the following result on the steady-state distribution of workload and nuber of jobs in the syste: 13

14 Proposition 2 Let W and X be the workload and nuber of jobs for the liiting Sd-LPS syste (as defined in Theore 1. Let the drift function θ(x be given by θ(x = λ The steady-state distributions of W and X are given by Pr[W ( w] = α w e c 2 s +1 d log f(x dx. f(x c 2 s +c2 a dx, (29 Pr[X α c 2 x ( x] = f(u s +1 c 2 s +c2 a du x K, α K+(x K c 2 s +1 e f(u c 2 s +c2 a du x > K, where α is the noralization constant. The ean of the liiting scaled nuber of jobs is given by (3 E[X ( ] = x= (x Kf(x c 2 s +1 c 2 s +c2 a dx x= f (x c 2 s +1 c 2 s +c2 a dx + c2 s x= (x K+ f(x x= f (x c 2 s +1 c 2 s +c2 a dx c 2 s +1 c 2 s +c2 a dx. (31 We have thus obtained closed-for forulae (approxiations for steady-state quantities based on the liiting diffusion process. The approxiation (13 at the beginning of this section is obtained fro (31 by further using the probability ass function, π(, for the nuber of jobs corresponding to the original Sd-LPS syste in place of the density function f(. We now close the loop by translating the convergence at the process level to convergence of steadystate distributions in the following theore. The proof is presented in Section 2. This justifies the forulae in Proposition 2 as an approxiation for the steady state of the original Sd-LPS syste. The quality of the approxiation is deonstrated in the nuerical experient presented at the beginning of this section (see Figure 2. Theore 2 (Convergence of steady-state distributions For all large enough r, the stochastic process ˆX (r has a steady state, denoted by ˆX (r (. Moreover, Ŵ (r ( W (, ˆX (r ( X (, where W ( and X ( are characterized in (27 and (3. 4 Dynaic concurrency control for the Sd-LPS queue In Section 3, we established approxiations for the steady-state nuber of jobs and workload in an Sd-LPS syste operating under a static concurrency level. Our nuerical experients showed that the optial static level based on the approxiations yields near-optial perforance for the original Sd-LPS syste. In this section we go further by allowing a dynaically adjustable concurrency level. In Section 4.1 we first suarize our translation of the discrete state space control proble for the original syste to a continuous state space diffusion control proble, and the translation of the resulting control back to that for the original Sd-LPS syste. To deonstrate the efficacy of our approach, we present results of nuerical experients coparing the perforance of the proposed diffusion liit based control policies against the true optial dynaic control policy for a special non-trivial input process for which the true optial policy can be coputed nuerically. 14

15 In Section 4.2 we forulate the diffusion control proble and show how this can help solve the dynaic control proble for the original syste. We then describe two novel nuerical algoriths to solve the diffusion control proble: an algorith that iteratively refines its estiate of the average cost of the optial policy using binary search in Section 4.3, and an algorith that uses the Newton-Raphson root finding ethod to search for the average cost of the optial policy in Section Overview of our approach and Siulation results The following steps outline our approach to obtaining a heuristic dynaic control policy for the original Sd-LPS server: 1. Convert the discrete service rate vector µ(i for the original state-dependent PS server into a drift function according to (12: where ˆµ is a continuous extension of µ(i. θ(x. = λ log λ ˆµ(x, 2. Forulate a diffusion control proble to iniize the steady-state ean nuber of jobs. The action/control will be the concurrency level as a function of the state. For convenience, we frae the diffusion control proble with workload as the state variable since the variance of workload is a constant (i.e., independent of state or action, and the control affects the drift of the workload through θ(x. 3. Given k (w, the optial concurrency level as a function of the workload for the diffusion control proble, to obtain a control policy for the original (discrete Sd-LPS syste, we first obtain a control function k(w with discrete concurrency levels by rounding k(w to the nearest integer for all w. The control algorith is ipleented by using W (t = e Z(t + Q(t for the original syste as the proxy for the current workload, and then taking action to reach the concurrency level dictated by the diffusion control proble: k( W (t. In controling the original syste we only take actions upon job arrivals and departures, do not preept jobs once they enter service, and do not increase the concurrency level by ore than one in any arrival/departure event. The precise policy is given as follows: On arrival at t: Let W = e Z(t + (Q(t + 1, where (Z(t, Q(t denotes the syste state iediately before the event. If k( W (Z(t + 1 then adit one job to the server at t, otherwise do nothing. On departure at t: Let W = e (Z(t 1 + Q(t. Adit in {( k( W Z(t + 1 +, 2 } jobs at t. Siulation Results Table 1 shows experiental results coparing the perforance of the dynaic policies produced using the proposed diffusion scaling and the true optial dynaic policy. We focus on a special class of input processes: Poisson arrivals and a degenerate Hyperexponential job size distribution (a ix of a point ass at and an Exponential distribution. This allows us to copute optial dynaic policies using the algorith proposed by [17]. The dynaic policy for the diffusion control 15

16 proble was coputed using the Newton-Raphson ethod (Algorith 2, Section 4.4. The service rate curve is the one shown in Figure 1, which gives the drift function as θ(x. = λ log λ ˆµ(x. = λ log λ 1.25 x2 15. (32 We used MATLAB s ode45 function to solve the differential equations involved in Algorith 2. The perforance of the diffusion control policy was evaluated by siulating it for a Poisson arrival process and Hyperexponential job size distribution with the indicated c 2 s. For each of the six cases shown, the steady-state ean nuber of jobs for the diffusion control heuristic is within 2% of the optial dynaic policy, deonstrating the validity of our proposed scaling for coputing control policies for Sd-LPS systes across a range of traffic intensities. c 2 s = 4 c 2 s = 19 Steady-state ean nuber of jobs E[N] Opt. dynaic policy Diffusion control policy Suboptiality (% λ = λ = λ = λ = λ = λ = Table 1: Siulation results coparing the perforance of dynaic policies for Poisson arrivals with rate λ and a degenerate hyperexponential (H job size distribution with = 1 and SCV c 2 s. The first colun shows the steady-state ean nuber of jobs for the optial dynaic policy. The second colun shows the sae etric for the heuristic policy obtained fro the diffusion control proble. For each case, the diffusion control policy yields an E[N] of at ost 2% larger than the optial policy. 4.2 The diffusion control proble In this section we set up the diffusion control proble for dynaic concurrency control of the LPS server. We begin by generalizing the scaling of the concurrency liit k (r given in (9 so that it becoes a function of the workload in the syste: ( k (r (W (r W (r (t (t = rk, (33 r where k : R + R + and k(w w/ e for any w. The restriction k(w w/ e on the choice of concurrency level is driven by the state space collapse (see Conjecture 1. The objective is to find the optial state-dependent concurrency level function k(. For technical reasons, we restrict our consideration to the following faily of dynaic controls { K = k : R + R + k(w w/ e ; k is Lipschitz continuous; e } v θ(k(wdw dv <. (34 The Lipschitz continuity requireent is for technical reasons, and the last condition above is only to ensure that a stationary distribution for the diffusion-scaled workload under k(w exists. We use 16 v=

17 the sae heavy traffic regie as in Section 3.2 except that stability condition (17 is replaced by sup θ(x >, (35 x [,M] for soe M <. That is, a service rate strictly larger than the arrival rate is achievable at a finite concurrency liit and hence at a finite workload. In fact, we will ake a stronger assuption on sup x θ(x. Define ˆθ. = sup x R + θ(x ;. ˆk = arg ax {θ(k}. k Here ˆk denotes the ost efficient concurrency level, which we will assue to be finite. For any k K, define the apping k : R + R + by k (w = w k(w e + (w k(w e +. (36 e Note that we use K to denote the apping under a static concurrency level K, and k to denote the apping under a dynaic concurrency policy k K. Extending the diffusion liit result in Section 3.2, we have the following conjecture: Conjecture 1 (Diffusion liits under a dynaic policy For the sequence of Sd-LPS systes paraetrized by r Z + under the dynaic policy (33 for soe k K, as r, Ŵ (r W, (37 where W is an RBM with initial value W ( = w, drift θ ( K (W (t k(w (t and variance σ 2 = λ 2 (c 2 a + c 2 s. Moreover, as r, ˆX (r X = k (W. (38 In other words, we conjecture that the state space collapse result still holds and the state-dependent concurrency level function k( only plays a role in odifying the drift of the diffusion liit of the workload. The key to proving this conjecture is to extend the state space collapse result to allow a dynaic concurrency level and analyze the underlying fluid odel (as in [45]. Due to the technical intricacies involved, proving the conjecture is beyond the scope of this paper. Instead, we focus on utilizing the conjectured diffusion liit to identify a near-optial policy for the original LPS syste. As entioned earlier, we will forulate the diffusion control proble with the liiting workload process W as the state variable and the ap k ( as the state-dependent cost function (using the state space collapse conjecture (38. There are two reasons for choosing W over the head count process X as the state variable: (i the variance of X is state-dependent aking the coputation ore coplicated, while it is a constant for W, and (ii headcount does not carry enough inforation since two different states (Q 1, Z 1 and (Q 2, Z 2 along the state-space collapse trajectory ay have the sae head count but different workloads. Therefore the control is not uniquely obtained as a function of the nuber of jobs in the syste. Let V γ (w denote the discounted total cost (with discount rate γ for the liiting process W under a control policy k( when the workload starts in state w: [ ] V γ (w = E w e γt k (W (tdt. (39 17

18 Optiality Equations Consider a sall δ >. According to Itō calculus V γ (w = k (w + (1 γδe [V γ (W (δ] + o(δ [ = k (w + (1 γδe V γ (w + V γ(w(w (δ w + V γ ] (w (W (δ w 2 + o(δ + o(δ 2 [ = k (w + (1 γδ V γ (w + V γ(wθ(k(wδ + V γ ] (w σ 2 δ + o(δ, 2 where, recall, σ 2 function V γ : Letting γ, define. = λ 2 (c 2 s + c 2 a. We thus have the following relation for the discounted value γv γ (w = k (w θ(k(wv γ(w + σ2 2 V γ (w. (4 v = li γ γv γ (w, and G(w = li V γ γ(w, where v is the average cost of policy k(, and the value function gradient G(w solves the following ordinary differential equation (ODE: v = k (w θ(k(wg(w + σ2 2 G (w. (41 Above, we have provided a heuristic derivation to arrive at the average cost optial control proble as a liit of the discounted cost proble. For a foral treatent of the relation between discounted and average cost probles (i.e., by defining discounted relative cost functions h γ (w = V γ (w V γ ( w for soe positive recurrent state w, taking liit h(w = li γ h γ (w and v = li γ γv γ ( w, we refer readers to [22, Chapter 5], [6]. Proposition 3 The discounted value function V γ (w is non-decreasing in w for all γ, and hence G(w. Reark 1 For a given control policy k(w, equation (41 is a first order ODE for G(w. However, to solve G( we also need to know the average cost v. This is to be expected since we started fro a second order ODE where we would need two boundary conditions to copletely specify V γ. In our case, one boundary condition is easy to get hold of: since we have a reflecting boundary at w =, we ust have (see, for exaple, [33, page VIII]: V γ( = (42 and therefore, also G( =. This observation will be critical in the developent of our algoriths. Returning to equation (4, let Vγ denote the value function for the optial policy. Then Bellan s principle of optiality becoes: γv γ (w = in k [,w/ e] { } k (w θ(kvγ (w + σ2 2 V γ (w. (43 If we let γ, then v = } in { k (w θ(kg (w + σ2 k [,w/ e] 2 G (w, (44 18

19 where again, as rearked earlier, we have the boundary condition G ( =, leaving v the only unknown. Though any diffusion control probles addressed in the literature have a nice structure allowing a closed-for solution, e.g., [19, 2], the proble (44 is intrinsically difficult ainly due to the generality of the service rate curve. Thus we seek nuerical algoriths, which presents another challenge. For diffusion control probles where a closed-for solution can be found, one of the boundary conditions is iposed by setting the coefficient of the exponential ter in the solution of the second order ODE to zero. This captures the physical constraint that the optial value function should asyptotically grow at a polynoial rate and not exponentially. However, this trick cannot be applied when searching for a nuerical solution, which led us to develop the algoriths in Sections 4.3 and 4.4 to get around this obstacle. While the ajority of nuerical algoriths for solving diffusion control probles rely on the Markov chain ethod where tie and space are discretized and a probability transition atrix is engineered to satisfy local consistency requireents (e.g., [28], we directly work with the ODE in (44. Drift value (θ(x Nuber at PS server (x (a Drift function θ( Concurrency level (k(n Nuber in syste (n (b c 2 a = c 2 s = 1 Concurrency level (k(n Nuber in syste (n (c c 2 a = c 2 s =.3 Figure 3: A hypothetical drift function θ(x and the optial diffusion control policies for two choices of workload paraeters c 2 a, c 2 s. For an illustration of what an optial dynaic policy ight look like, see Figure 3. The first figure shows an illustrative exaple of the θ(x function for the PS server. As can be seen, the PS server is ost efficient when there are ˆk = 5 jobs at the server, and the speed drops on either side of this point. The second figure shows the optial dynaic policy (translated fro k(w to k(n, that is, as a function of the nuber of jobs in the syste, for clarity when c 2 s = c 2 a = 1. This corresponds to a workload that has significant variability, and the optial policy increases the concurrency level to approxiately 9 when the nuber of jobs in the syste is sall but scales it back when there is a long queue. The third figure shows the policy for c 2 s = c 2 a =.3. This is a low variability workload, and as the nuber of jobs in the syste increases, initially the PS server acts as an FCFS server and thus coproises speed to keep the concurrency level sall. At n.5, the syste switches to a controlled PS behavior by gradually increasing the concurrency level to increase service rate. At n 1.2 the syste switches to a pure PS behavior aditting everyone in queue, and finally at n = 3 it switches back to a controlled PS behavior, gradually increasing the concurrency level to ˆk = 5 as queue becoes longer. The graphs shown were produced using the Newton-Raphson average cost iteration algorith described in Section