Tail-robust Scheduling via Limited Processor Sharing

Transcription

1 Performane Evaluation Performane Evaluation ) 22 Tail-robust Sheduling via Limited Proessor Sharing Jayakrishnan Nair a, Adam Wierman b, Bert Zwart a Department of Eletrial Engineering, California Institute of Tehnology b Computing and Mathematial Sienes Department, California Institute of Tehnology CWI Amsterdam, VU University Amsterdam, Eurandom & Georgia Teh Abstrat From a rare events perspetive, sheduling disiplines that work well under light exponential) tailed workload distributions do not perform well under heavy power) tailed workload distributions, and vie-versa, leading to fundamental problems in designing shedulers that are robust to distributional assumptions on the job sizes. This paper shows how to exploit partial workload information system load) to design a sheduler that provides robust performane aross heavy-tailed and light-tailed workloads. Speifially, we derive new asymptotis for the tail of the stationary sojourn time under Limited Proessor Sharing LPS) sheduling for both heavy-tailed and light-tailed job size distributions, and show that LPS an be robust to the tail of the job size distribution if the multiprogramming level is hosen arefully as a funtion of the load. Keywords: GI/GI/ queue, sheduling, limited proessor sharing, large deviations, tail asymptotis, heavy-tailed job size, light-tailed job size, tail-robustness. Introdution In the study of sheduling poliies, muh of the fous has traditionally been on designing poliies that have good performane in expetation. For example, in order to minimize the expeted sojourn time a.k.a. response time, flow time) in a single server queue it is well known that the sheduler should give priority to jobs with small remaining sizes via Shortest Remaining Proessing Time SRPT) [], whih is optimal regardless of the job size distribution and arrival proess. However, providing good performane in expetation is not suffiient. It is also important for a sheduler to provide good distributional performane. For example, quality of servie guarantees in web appliations often rely on speifying guarantees about the tail of the sojourn time distribution, e.g., that 95% of requests will have sojourn time < s seonds. Resultantly, there has been a substantial amount of work in reent years studying the sojourn time distribution, P V > x) of sheduling poliies in a GI/GI/ setting. Due to the diffiulty of an exat distributional analysis, muh of this work fouses on understanding the sojourn time tail asymptotis, i.e., the behavior of P V > x) as x, whih provides a haraterization of the

2 Nair, Wierman and Zwart / Performane Evaluation ) 22 2 likelihood of large delays. From this work, whih we survey briefly in Setion 2, has emerged an understanding of how to optimally shedule for the sojourn time tail. Interestingly, unlike when optimally sheduling for the expeted sojourn time, prior work shows that there are two distint regimes: when the job size distribution is light-tailed, First Come First Served FCFS) sheduling minimizes the sojourn time tail [2], while if the job size distribution is heavy-tailed, SRPT, Proessor Sharing PS), and many other poliies e.g. all SMART poliies [3]) minimize up to a onstant) the sojourn time tail [4]. Interestingly, among the prior work, there are no poliies that are optimal aross both lighttailed and heavy-tailed job size distributions. In fat, Wierman & Zwart have reently proved an impossibility result [5], whih states that no work-onserving poliy that is non-learning i.e., does not learn information about the workload) an optimize the sojourn time tail aross both light-tailed and heavy-tailed job size distributions. Further, among the prior work, the poliies that produe the best possible sojourn time tail behavior under heavy-tailed job size distributions produe the worst possible sojourn time tail behavior under light-tailed job size distributions, and vie-versa. Indeed, there are no poliies that have been shown to maintain even better than worstase sojourn time tail performane aross both light-tailed and heavy-tailed job size distributions. So, at this stage, the literature understands how to design a sheduling poliy to be optimal for the sojourn time tail given a partiular workload, but annot design a sheduling poliy that is robust, even minimally so, aross both light-tailed and heavy-tailed job size distributions. This is in stark ontrast to the ase of sheduling for expeted sojourn time, where SRPT is optimal and robust. The lak of robustness when sheduling for the sojourn time tail is relevant from a pratial perspetive beause determining whether a partiular real-world workload is light-tailed or heavy-tailed is a diffiult task. For example, there is an unending debate over whether to model web file sizes as an unbounded heavy-tailed distribution or as a bounded distribution with a power-law body. Ideally, a sheduler design should be robust to suh assumptions. The goal of this paper is to present a sheduling poliy that is tail-robust, i.e., provides robust performane in terms of the sojourn time tail) aross both heavy-tailed and light-tailed job size distributions. The main ontribution of this work is to prove that Limited Proessor Sharing LPS-) an be designed to be tail-robust. Under LPS-, there is a limited multiprogramming level, whih determines the maximum number of jobs that the servie rate is shared among. Speifially, jobs are queued aording to the order of arrival and if there are n jobs in the system then the minn, ) jobs whih arrived earliest eah reeive a servie rate of / minn, ). LPS- is a natural andidate for our goal beause, as grows from to, LPS- transitions from FCFS, whih is optimal under light-tailed job sizes, to PS, whih is optimal under heavy-tailed job sizes. Our goal will be to determine how to hoose an intermediate suh that LPS- is tail-robust. It turns out that to ahieve tail-robustness, the hoie of must inorporate some information about the workload. We will prove that this an be hosen in suh a way that only information about the system load ρ is neessary, whih is not an unreasonable assumption as this information is also neessary to ahieve system stability. It is important to point out that LPS- is not a poliy that we artifiially onstruted to fit the goals of this paper. LPS- is a pratial poliy that is atually a more realisti version of both FCFS and PS in the ase of many omputer systems, where it is unrealisti to share the server among unboundedly many jobs or to devote the server entirely to a single job. Given its pratial importane, there have been a number of prior studies of LPS-: Avi-Itzhak & Halfin [6] propose an approximation for the mean response time assuming Poisson arrivals. A omputational analysis based on matrix geometri methods is performed in Zhang & Lipsky [7, 8]. Some stohasti ordering results are derived in Nuyens & van der Weij [9]. Zhang,

3 Nair, Wierman and Zwart / Performane Evaluation ) 22 3 Dai & Zwart [0,, 2] develop fluid, diffusion and heavy traffi approximations. Finally, Gupta & Harhol-Balter [3] onsider approximation methods and Markov deision tehniques to determine the optimal level when the system is not work-onserving. However, none of the prior work has foused on the sojourn time tail of LPS-. In order to understand how to design LPS- so that it is tail-robust, we first need to analyze the sojourn time tail asymptotis in both the ase of heavy-tailed and light-tailed job size distributions. We do this in Setions 3 and 4 respetively. In both ases our analysis reveals interesting insights. For example, for heavy-tailed job sizes we find that the behavior of LPS- is similar to that of the analogous GI/GI/ queue, where eah server works at rate /. However, this is not the ase for light tails, where quite a few qualitatively different senarios may lead to large sojourn times. In partiular, a large sojourn time may our through a ombined effet of a large baklog in the system upon arrival, a large servie time, and a higher than usual input during the sojourn of the ustomer under onsideration. Interestingly, this is in ontrast to poliies that have been analyzed up to this point, under whih one of these phenomena typially dominates. The sojourn time tail asymptotis of LPS- that we derive in Setions 3 and 4 also highlight a tension that must be resolved when attempting to design LPS- robustly. In partiular, when the job size distribution is light-tailed, reduing lightens the sojourn time tail; however, when the job size distribution is heavy-tailed, inreasing lightens the sojourn time tail. This highlights the tradeoff neessary between optimality and robustness. In Setion 5, we show that despite the onfliting demands on plaed by the light-tailed and heavy-tailed regimes, it is indeed possible to hoose so that LPS- is tail robust. In partiular, by hoosing = / ρ) + it is possible to guarantee that LPS- is tail-robust, i.e., that the sojourn time tail is better than worst-ase aross a large lass of heavy-tailed regularly varying) job size distributions and light-tailed phase-type) job size distributions. Further, this hoie of ensures that for large sublasses of heavy-tailed and light-tailed job size distributions the sojourn time tail is optimal see Corollary ). Additionally, this design is robust to estimation errors in ρ as long as the estimate of ρ that is used is an upper bound on the true ρ, this will still be tail-robust. Importantly, there is some freedom among the lass of tail-robust designs possible using LPS-. In partiular, Corollary 2 presents a parameterized design for that allows the designer to vary the importane plaed on optimality in the heavy-tailed and light-tailed regimes while still guaranteeing tail-robustness. However, in order to ensure that LPS- is tail robust, it is neessary to maintain / ρ) + to handle heavy-tailed job size distributions. The remainder of the paper is organized as follows. In Setion 2, we introdue the model and notation for the paper, and disuss prior work studying the sojourn time asymptotis of sheduling poliies. In Setions 3 and 4 we present our new results haraterizing the sojourn time asymptotis of LPS-. Then, in Setion 5 we present the main results of the paper showing how to design the multiprogramming level for LPS- to ensure tail-robust performane. Finally, we onlude in Setion Preliminaries 2.. Model and Notation Throughout this paper, our fous will be on the GI/GI/ queue. Jobs arrive aording to a renewal proess; let A denote a generi interarrival time. Eah job has an independent, identially distributed servie requirement size); let B denote a generi job size. The server speed is taken to be unity. We make the following standard assumptions: i) load ρ := E[B] E[A] 0, ), ii) P B > A) > 0 otherwise there would be no queueing). Denote α := E [A], β := E [B]. Let B e denote a random variable distributed as the ex-

4 Nair, Wierman and Zwart / Performane Evaluation ) 22 4 ess/residual lifetime of B, i.e., P B e > x) = β F x B t)dt for x 0. For funtions ϕx) and ξx), ϕx) the notation ϕx) ξx) means lim x ξx) =, ϕx) ξx) means lim inf x ϕx) ξx). The sojourn time response time) of a job refers to the time between its arrival and its departure. The waiting time delay) of a job refers to the time between its arrival and the instant it first reeives servie. V π and D π denote respetively random variables distributed as per the sojourn time and waiting time of a job in the stationary GI/GI/ queue operating under sheduling disipline poliy) π. In this paper, our interest is entered around the asymptoti behavior of the sojourn time tail, i.e., the behavior of P V π > x) as x. In our analysis of the tail behavior of the stationary sojourn time, we fous on the sojourn time of a tagged job, assumed to arrive into the stationary queue at time 0, with size B 0. W denotes the total work baklog) in the system just before the arrival of the tagged job. B i denotes the size of the i-th arrival after time 0. For i, A i denotes the time between the i )-st and i-th arrival. For x > 0, Nx) := max{n N : n i= A i x} is the number of arrivals into the system in time interval 0, x]. Ax) := Nx) i= B i is the total work entering the system in the interval 0, x] Heavy-tailed and light-tailed distributions For any non-negative random variable X, F X ) denotes the distribution funtion d.f.) of X, i.e., F X x) = P X x) ; Φ X ) denotes the moment generating funtion of X, i.e., Φ X s) = E [ e sx]. X or its d.f. F X ) is defined to be heavy-tailed if Φ X s) = for all s > 0. X or its d.f. F X ) is defined to be light-tailed if it is not heavy-tailed, i.e., if Φ X s) < for some s > 0. The following subsets of the lass of heavy-tailed distributions will be of interest to us. X PX>x+y) or its d.f. F X ) is said to be long-tailed denoted X L) if lim x PX>x) = for all y > 0. The lass of long-tailed distributions inludes most of the ommon heavy-tailed distributions, inluding the Pareto, the Lognormal and the heavy-tailed Weibull distribution [4]. X or its d.f. F X ) is said to be regularly varying with index θ > denoted X RVθ)) if P X > x) = x θ Lx), Lxy) where Lx) is a slowly varying funtion, i.e., Lx) satisfies lim x Lx) = y > 0. Note that all Pareto distributions are inluded in this lass. The lass of regularly varying distributions is a strit subset of the lass of long-tailed distributions, whih in turn is a strit subset of the lass of heavy-tailed distributions [4]. We desribe the logarithmi) asymptoti tail behavior of heavy-tailed X, using its tail index, defined as ΓX) := lim x log PX>x) logx), when the limit exists. Note that if ΓX) 0, ), then for arbitrarily small ɛ > 0, x ΓX)+ɛ) P X > x) x ΓX) ɛ) for large enough x. This means the tail index is useful for desribing the asymptoti tail behavior of distributions that exhibit a roughly power-law tail. Note that a smaller value of tail index implies a heavier tail. Setion 3 is devoted to the analysis of ΓV LPS ) when B RVθ). Similarly, we desribe the logarithmi) asymptoti tail behavior of light-tailed X, using its log PX>x) x deay rate, defined as γx) := lim x, when the limit exists. If γx) 0, ), then for arbitrarily small ɛ > 0, e ΓX)+ɛ)x P X > x) e ΓX) ɛ)x for large enough x. This means the deay rate is useful for desribing the asymptoti tail behavior of distributions that have a roughly exponential tail. Again, note that a smaller value of γx) implies a heavier tail. It is easy to prove that i) Φ X s) < for all s < γx), ii) if γx) <, then Φ X s) = for all s > γx). This implies that if we assume that the deay rate of X exists, then X is light-tailed if and only if γx) 0, ]. Setion 4 is devoted to the analysis of γv LPS ) for the ase γb) 0, ] Related literature We now review the sojourn time tail asymptotis for the following well known sheduling poliies: First Come First Served FCFS), Preemptive Last Come First Served PLCFS), Proessor Sharing PS), and Shortest Remaining Proessing Time SRPT). Note first, that for any work

5 Nair, Wierman and Zwart / Performane Evaluation ) 22 5 onserving sheduling poliy π, V π may be stohastially bounded as follows. B st V π st Z, where Z denotes the total time to emptyness of the queue in steady state, just after an arrival. We onsider separately the ase of heavy-tailed and light-tailed job sizes. Heavy-tailed job sizes: For the GI/GI/ queue with regularly varying job sizes, it is known that ΓV PS ) = ΓV S RPT ) = ΓV PLCFS ) = ΓB) = θ, ΓV FCFS ) = ΓZ ) = θ. See the survey by Boxma & Zwart [4] for details. Based on the bounds on V π desribed above, it is lear that PS, SRPT and PLCFS produe the optimal sojourn time tail index. The sojourn time tail under FCFS is one degree heavier; moreover, FCFS produes the worst possible sojourn time tail index. In fat, it turns out that all non-preemptive sheduling poliies produe this worst possible sojourn time tail index see the paper by Anantharam [5]). Light-tailed job sizes: In desribing the sojourn time asymptotis under light-tailed job sizes, the following funtion plays a key role. For s 0, Ψs) := Φ A Φ B s)). The following lemma gives an interpretation to this funtion. log E[e Lemma Mandjes-Zwart [6]). For s 0, lim sax) ] x x = Ψs). Further, Ψs) is stritly onvex and lower semi-ontinuous. For the GI/GI/ queue with light-tailed job sizes, γv FCFS ) = := sup{s 0 : Ψs) s 0}, γv PLCFS ) = γ L := sup{s Ψs)}; s 0 see Nuyens & Zwart [7]. From the strit onavity of s Ψs), and sine Ψ 0) = ρ, it is easy to show that γ L < ρ). This implies the stationary sojourn time tail under FCFS is lighter than that under PLCFS. It has been proved by Ramanan and Stolyar [2] that the sojourn time deay rate under FCFS is atually optimal. Moreover, it has been proved by Nuyens et al. [3] that γz ) = γ L, implying that PLCFS produes the worst possible sojourn time deay rate under light-tailed job sizes. Under mild regularity onditions, we additionally have γv PS ) = γv S RPT ) = γ L. The above diussion highlights the dihotomy desribed before; the poliies that produe the best possible sojourn time tail behavior under heavy-tailed job size distributions produe the worst possible sojourn time tail behavior under light-tailed job size distributions, and vie-versa Busy period deay rate as a funtion of server speed We onlude this setion with a brief disussion on the dependene of the busy period deay rate of a GI/GI/ queue on the server speed. This disussion will play a role in our analysis of the sojourn time deay rate under LPS- in Setion 4. Define, for r 0, gr) := sup s 0 [rs Ψs)]. gr) is the busy period deay rate, if the server speed equals r. It is easy to see that g ) is onvex inreasing; gr) = 0 for r ρ, g) = γ L. Suppose that γb) 0, ). In this ase, for all s > γb), Φ B s) =, implying Ψs) =. This means gr) = sup s [0,γB)] [rs Ψs)]. Now, sine Ψ ) is stritly onvex and lower semi-ontinuous, the supremum in the definition of g ) is uniquely ahieved; define ŝr) = arg max s 0 [rs Ψs)]. Lemma 2. If γb) 0, ), then gr) is ontinuously differentiable over r 0. Moreover, g r) = ŝr). Proof. That g r) = ŝr) follows by invoking an envelope theorem like Danskin s theorem; see Proposition B.25 in [8]. Sine g ) is onvex and differentiable, its derviative must be ontinuous; see Theorem 25.5 in [9].

6 Nair, Wierman and Zwart / Performane Evaluation ) Tail asymptotis under heavy-tailed job sizes We start our analysis by fousing on heavy-tailed job size distributions. In this setion, we desribe tail asymptotis for the sojourn time under LPS- for the ase of regularly varying job sizes. As we disussed in Setion 2, there is a signifiant amount of prior work deriving the sojourn time tail asymptotis for sheduling poliies in the heavy-tailed regime. This prior work has shown that: i) non-preemptive poliies e.g., FCFS) have a sojourn time tail that is one degree heavier than the job size distribution, whih is up to a onstant) as bad as possible; and ii) many preemptive poliies e.g., SRPT, PS) have sojourn time tails that are proportional to the tail of the job size distribution, whih is optimal up to a onstant). Interestingly, almost all poliies that have been studied have a sojourn time tail that falls into either ase i) or ase ii). Only reently was a poliy onstruted that has an intermediate sojourn time tail [20]. Our analysis shows that, in many settings, LPS- also has an intermediate sojourn time tail. For tehnial reasons, we must make the following assumption in our analysis. Assumption. ρ is not an integer, i.e., ρ < ρ. Under this assumption, we an state the sojourn time asymptotis of LPS- as follows. Theorem. Consider the GI/GI/ queue. Under Assumption, if B RVθ) for θ >, then ΓD LPS ) = lim log P D LPS > x) = θ ) ρ ), x logx) ) ΓV LPS ) = lim log P V LPS > x) = min {θ, θ ) ρ )}. x logx) 2) One natural way to view an LPS- queue is as a work-onserving version of a GI/GI/ queue, where eah server has speed /. In this view, this theorem an be interpreted as stating that LPS- has the same sojourn time tail as the GI/GI/ queue in the heavy-tailed regime. In fat, the proof relies on the parallels between these two queues. We prove an upper bound on the tail of delay in Appendix A., a lower bound on the tail of delay in Appendix A.2, and then ombine them to omplete the proof of Theorem in Appendix A.3. The upper bound follows immediately from bounding the LPS- queue by the GI/GI/ queue, while the lower bound proof uses a probabilisti argument that offer insight into how large waiting times our. The parallel between the GI/GI/ and LPS- also motivates us to define k = k := ρ. We refer to k as the number of spare slots. The name spare slots refers to the fat that k is the minimum number of infinite-sized jobs that must be added to the LPS- queue before the it beomes unstable. This definition of k parallels the notion of spare servers whih is used in the ontext of the GI/GI/ queue. In the GI/GI/ setting, the number of spare servers, k, has been shown to determine the moment onditions for delay see Appendix A.) [2], thus it is perhaps not surprising that k determines the weight of the sojourn time tail in the LPS- queue. However, we will see that the parallel between the LPS- queue and the GI/GI/ queue does not hold in light-tailed regime Setion 4). Another natural view of LPS- is as a hybrid version of FCFS = ) and PS ). In this view, Theorem highlights that the sojourn time tail transitions between the sojourn time tails of FCFS and PS as inreases. Speifially, reall that the sojourn time tail index for any workonserving sheduling poliy if it exists) lies between ΓV FCFS ) = θ and ΓV PS ) = ΓB) = θ. A remark about Assumption : This assumption ensures that in the presene of k infinitely sized jobs in the system, the queue is positive reurrent and not null reurrent). This assumption is also made in [2] for the analysis of the GI/GI/ queue.

7 Nair, Wierman and Zwart / Performane Evaluation ) 22 7 When = we have ΓV LPS ) = ΓV FCFS ), whih is the heaviest possible tail index. However, the weight of the sojourn time tail lightens monotonially as inreases and, as, the sojourn time tail index mathes that of the job size distribution, whih is optimal. Speifially, for all large enough that k > θ/θ ) we have that ΓV LPS ) = ΓV PS ) = θ. Thus, in the heavy-tailed setting, LPS- should be designed with large enough. Unfortunately, we will see the that the opposite is true in the light-tailed regime when job sizes are light-tailed, LPS- should be designed so that is small enough. This highlights the tension of designing LPS- so that it is tail-robust. Understanding this tension is the goal of Setion Tail asymptotis under light-tailed job sizes We now move to the light-tailed regime and again analyze the sojourn time asymptotis of the LPS- queue. As we disussed in Setion 2, there is a signifiant amount of prior work devoted to the sojourn time asymptotis of sheduling poliies in the light-tailed regime. From this prior work has evolved an understanding of what bad events lead to large delays under most ommon sheduling poliies. In partiular, a long delay will our beause of one or more) of the following three effets: i) a large baklog is at the server when the tagged job arrives, ii) the tagged job has a large size, iii) a large number of jobs enter the system during the tagged job s sojourn. Prior work has provided an understanding of whih ombination of these three effets is most likely to lead to a large delay under most ommon sheduling poliies. For example, under FCFS a long delay is most likely aused by i), while under SRPT and PS, a long delay is most likely aused by the ombination of ii) and iii). As disussed in Setion 2, FCFS produes the optimal largest possible) sojourn time deay rate whereas SRPT and PS produe the worst smallest) possible sojourn time deay rate. As in the heavy-tailed setting, there is a dihotomy in the previous analyses: all poliies that have been analyzed to the best of our knowledge) have sojourn time tail asymptotis that fall into two ategories: long delays are most likely aused by either i) or a ombination of ii) and iii). Like in the heavy-tailed setting, in some ases, LPS turns out to have intermediate tail-asymptotis where the most likely way a bad event an our is a workload-dependent) ombination of i), ii), and iii). Let us now state the main result for this setion. Throughout, we will assume that the deay rate of the job size distribution exists. Theorem 2. Consider the GI/GI/ queue. If γb) 0, ), γv LPS ) = lim log P V LPS > x) = min x x f a), 3) a [0,] [ a)γb) where f a) := a + + sup a)s ) ] Ψs). 4) s 0 Otherwise, if γb) =, then γv LPS ) = γv FCFS ). We prove 3) by providing mathing asymptoti lower and upper bounds on the tail of V LPS. The lower bound is proved in Appendix B., the upper bound is proved in Appendix B.2. We then prove the result for the ase of γb) = in Appendix B The ondition γb) = haraterizes very light-tailed job-size distributions that have a tail that deays faster than exponentially) and inludes all distributions with bounded support.

8 Nair, Wierman and Zwart / Performane Evaluation ) 22 8 Given the omplexity of the deay rate in Theorem 2, it is important to provide some interpretation of the theorem. To begin, note that, unlike in the heavy-tailed regime, the deay rate of LPS- does not parallel the deay rate of the GI/GI/ queue where servers have speed / see [22] for a derivation of the deay rate for the GI/GI/ queue). However, the deay rate does still highlight the fat that LPS- an be viewed as a hybrid of FCFS and PS. In partiular, in the ase of γb) 0, ), whih inludes all phase-type distributions, the tail asymptotis of LPS- an vary between the asymptotis of FCFS for small enough, whih is optimal, and the asymptotis of PS as, whih is pessimal. But, the omplexity of 3) hides muh of the behavior of the deay rate; thus we spend some time in the following setions interpreting and deriving important properties of the deay rate. 4.. Interpreting the deay rate under LPS- To build an understanding of 3) it is useful to begin by interpreting it in the ontext of effets i), ii), and iii) desribed above that ould lead to a long delay. Intuitively, effets i), ii) and iii) orrespond respetively to the first, seond and third term in 4). Further, the variable a [0, ] aptures the relative ontribution of these effets to a large sojourn time. If a is lose to, then effet i) dominates; if a is lose to 0, then to effets ii) and iii) dominate; and intermediate values of a represent different ombinations of all three effets. The minimization operation in 3) indiates that the most dominant ombination of effets i), ii) and iii) determines the deay rate, and whih ombination is dominant depends on, A, and B. Thus, one should interpret the value of a := arg min a [0,] f a) as providing a desription of how large sojourn times are aused. Informally, for large x, if the tagged job experienes a sojourn time V > x, it is most likely due to a) a baklog of the order of a x being present in the system when the job arrives, b) the tagged job having a size of the order of a )x, and ) work of the order of a ) ) x entering the queue in the interval 0, x) Properties of the deay rate under LPS- In this setion, we fous on the ase γb) 0, ), and try to provide insight into two questions: How does the deay rate of LPS- vary with? Can we provide a more expliit haraterization of a and, thus, γv LPS )? Additionally, we present some numeri examples to illustrate the points in our disussion. We start by studying the behavior of γv LPS ) as a funtion of. Given the view that LPS- is a hybrid of FCFS and PS, one expets that the deay rate of LPS- will transition monotonially between γv FCFS ), the optimal deay rate, and γv PS ), the pessimal deay rate, as grows from to. This is indeed what happens; the following lemma establishes the monotoniity of the sojourn time deay rate with respet to. Lemma 3. Consider the GI/GI/ queue. Assuming γb) 0, ), γv LPS ) is monotonially dereasing in. Moreover, lim γv LPS ) = γv PS ) = γ L. Lemma 3 implies that for light-tailed job sizes, the sojourn time tail under LPC- gets heavier with inreasing. In ontrast, for heavy-tailed job sizes, we proved in Setion 3 that the sojourn time tail gets lighter with inreasing. We prove Lemma 3 in Appendix B.4. Next, we provide a more expliit haraterization of a, and thus γv LPS ). To aomplish this, we must onsider two lasses of light-tailed workloads separately: < γb) and = γb). Reall the bakground provided in Setion 2.4 on the deay rate of the busy period. In light of that disussion, we may rewrite f ) as follows. a)γb) f a) = a + + g a) ) ). Moreover, f ) is ontinuously differentiable and onvex. Let f := min a [0,] f a).

9 Nair, Wierman and Zwart / Performane Evaluation ) 22 9 Case : < γb). Note that this ase inludes most ommon light-tailed job size distributions, e.g., all phasetype distributions. 3 To get a more expliit representation of a, begin by noting that f 0) = γb) + g ), f ) =. Next, Lemma 2 allows us to apture the derivative of f a) with respet to a. f a) = γb) ) ŝ ) ) a) f 0) = γb) ) ŝ ), f ) = γb). So, for γb), f ) 0, implying a = reall that f ) is onvex) and γv LPS ) =. Therefore, for small enough speifially, γb) ), the deay rate of LPC- mathes that of FCFS. Moreover, long delays are most likely aused by effet i). Consider now the ase > γb). In this ase, the funtion f a) is inreasing in a for a. This means γv LPS ) = min a 0 f a). This observation allows us to express the deay rate differently: [ [ a)γb) γv LPS ) = min a + + max a)s ) ]] Ψs) a [0,] s 0 [ a)γb) a + ) ] Ψs) = min max + a)s a 0 s 0 = γb) [ + min max s ) Ψs) a s ) γb) ))]. a 0 s 0 We may interpret the seond term above to be the dual of the onvex optimization problem [ max s ) ] Ψs), s [0,κ ] where κ := γb) Prop in [8]), we an rewrite the sojourn time deay rate as follows. = γb). Sine this optimization problem has zero duality gap see γv LPS ) = γb) [ + max s ) ] Ψs). 5) s [0,κ ] Note that the above form for the deay rate is more omputationally onvenient than that in the statement of Theorem 2. Additionally, it allows us to haraterize the value of a ; this is summarized in the following lemma. Lemma 4. Consider the GI/GI/ queue. If < γb), then for > γb), a is monotonially dereasing in. Moreover, there exists ĉ > γb) suh that for > ĉ, i) a = 0, ii) γv LPS ) = γb) + g ) > γvps ). From the standpoint of tail-robust sheduling using LPS, whih is the fous of Setion 5, the above lemma has the following important impliation: For the lass of workload distributions 3 If B is phase-type, then γb) 0, ) and lim s γb) Φ B s) =. This implies that lim s γb) Ψs) =, whih is suffiient to guarantee that < γb).

10 Nair, Wierman and Zwart / Performane Evaluation ) 22 0 γv LPS ) γ 0. F γ L 0 0 γb) a * γb) a) Figure : M/M/ example: B Exp), ρ = 0.9 b) γv LPS ) γ L 0 0 γb) γ F a * γb) a) Figure 2: M/Erlang-2/ example b) that satisfy < γb), for all, the sojourn time deay rate under LPS- is stritly better than worst-ase reall that PS has the smallest possible deay rate). The monotoniity of a with respet to implies that as inreases, the ontribution of effets ii) and iii) to a large sojourn time inreases relative to i). Moreover, for large enough, large sojourn times are most likely aused by effets ii) and iii). Interestingly, for intermediate values of, it is possible that a 0, ). In this ase, the bad event is a ombination of all three effets i)-iii); see Example 2 below. We give the proof of Lemma 4 in Appendix B.5. To illustrate the properties desribed above, we onsider a ouple of examples. Example : Consider first the M/M/ ase; A Expλ), B Expµ). 4 In this ase, γb) = ρ. Interestingly, it an be proved that for > γb), a = 0. Figure shows γv LPS ) and a as a funtion for the ase µ =, ρ = 0.9. Example 2: Next, we onsider an M/GI/ example, where A Expλ), and B has an Erlang-2 distribution, i.e., Φ B s) = µ 2 µ s). Figure 2 shows γvlps ) and a as a funtion for the ase µ =, ρ = 0.9. Note that for some intermediate values of, a 0, ). Case 2: = γb). This ase behaves fundamentally differently than Case above, however it is easier to haraterize. For =, it is obvious that γv LPS ) = and a =. On the other hand, for >, f 0) = ) )) γb) ŝ 0. This means a = 0 and γv LPS ) = f 0) = γb) + g ). Therefore, if = γb), for >, a large sojourn time is most likely aused by a ombination of effets ii) and iii). 4 A Expλ) means A is exponentially distributed with mean /λ.

11 Nair, Wierman and Zwart / Performane Evaluation ) Designing LPS robustly Now that we have derived the sojourn time asymptotis in both the light-tailed and heavytailed regimes, we an return to the question of designing a sheduling poliy that has robust performane aross both heavy-tailed and light-tailed job size distributions. Reall from our disussion in Setion 2 that there is a dihotomy in prior results showing that sheduling poliies that perform optimally in the heavy-tailed regime e.g., SRPT and PS) have the worst-ase sojourn time tail in the light-tailed regime and poliies that perform optimally in the light-tailed regime e.g., FCFS) have the worst-ase sojourn time tails in the heavy-tailed regime. In fat, a reent result by Wierman and Zwart [5] shows that this is a fundamental limit on all work-onserving poliies that do not learn the job size distribution. Speifially, no workonserving, non-learning poliy an be optimal in under heavy-tailed light-tailed) job sizes and better than worst-ase under light-tailed heavy-tailed) job sizes. Further, prior work provides no shedulers that are tail-robust, i.e., provide robust performane even a better than worst-ase sojourn time tail) aross both light-tailed and heavy-tailed job size distributions. This is problemati beause determining whether a workload is heavytailed or light-tailed is extremely diffiult if not impossible) and thus designing suh an assumption into a sheduler is undesirable. Ideally, a sheduler should provide performane that is robust to suh an assumption, and designing suh a sheduler is the goal of this setion. We show in this setion that by hoosing the multiprogramming level arefully, it is possible to design LPS- so that it provides tail-robust performane. Our results in Setion 3 and 4 highlight the tension in designing LPS- robustly. Reall that as grows the sojourn time tail gets heavier in the light-tailed regime while the sojourn time tail gets lighter in the heavy-tailed regime. Thus, an intermediate value of must be arefully hosen to provide robustness. Note that annot be hosen to be workload-independent; indeed, Theorem implies that with regularly varying job sizes, for any fixed, the sojourn time tail index mathes that under FCFS the worst-ase) as load ρ approahes. Our designs hoose as a funtion of ρ. Thus, these poliies must learn some information about the workload, but it only needs to learn expetations, whih an be aomplished quikly and is ertainly muh easier than learning the tail. In this setion we propose two possible hoies for that both provide tail-robust performane, but balane differently performane in the heavy-tailed and light-tailed regimes. Design. Our first proposed design guarantees better than worst-ase performane for a broad lass of light-tailed distribution phase type distributions) and heavy-tailed distributions regularly varying distributions). Further, it guarantees optimality under large sublasses of both heavy-tailed and light-tailed distributions. Corollary. Consider the GI/GI/ queue. Using LPS- with = ρ + ensures that the logarithmi) asymptoti tail behavior of the stationary sojourn time is i) better than worst-ase when the job size distribution is regularly varying with index θ >, i.e., ΓV LPS ) > ΓV FCFS ). ii) better than worst-ase when the job size distribution is phase-type, i.e., γv LPS ) > γv PS ). 5 iii) optimal when the job size distribution is regularly varying with index θ 2, i.e., ΓV LPS ) = ΓV PS ). iv) optimal when the job size distribution is light-tailed and satisfies γb) ρ + or γb) =, i.e., γv LPS ) = γv FCFS ). 5 Note that the sojourn time tail is atually better than worst-ase for a larger lass of light-tailed workloads, inluding workloads satisfying < γb).

12 Nair, Wierman and Zwart / Performane Evaluation ) 22 2 This orollary follows immediately from ombining Theorems and 2 with Lemma 4 see Appendix C). One important point about Design is that it is light-tailed entri, by whih we mean that the is hosen as the smallest possible that guarantees better than worst-ase performane under regularly varying job size distributions. Thus, sojourn time tail is the lightest possible in the light-tailed regime while still maintaining tail-robust performane. Additionally, a pratial remark about Design is that, though it requires learning the ρ, it does not atually require the exat ρ to be learned. If an upper bound on ρ is learned, then points i) and ii) remain true, and only the optimality regions hange. Thus, providing tail-robustness is possible even with quite inexat estimates of the load. Finally, it is worth disussing the sublasses of job size distributions where Design provides the optimal sojourn time tail. In the heavy-tailed regime, the sub-lass inludes all regularly varying distributions with finite variane. In the light-tailed regime, it is more diffiult to expliitly desribe the sublass. However, it is important to note that the exponential distribution is on the boundary. In partiular, for the M/M/ queue, γb)/ = ρ. Thus, it is impossible for LPS- to be designed with an optimal sojourn time tail for exponential job size distributions and a better than worst-ase sojourn time tail for all regularly varying job size distributions. Design 2. Our seond proposed design for provides a ontrast to Design in that it is heavytailed entri instead of light-tailed entri. Speifially, ompared to Design, Design 2 allows the lass of heavy-tailed job size distributions where the sojourn time tail index of LPS- is optimal to be enlarged, while still maintaining better than worst-ase performane under lighttailed job size distributions but shrinking the lass of light-tailed distributions where the sojourn time tail index is optimal. Θ Corollary 2. Consider the GI/GI/ queue. Let Θ, 2]. Using LPS- with = Θ ρ ensures that the logarithmi) asymptoti tail behavior of the stationary sojourn time is i) better than worst-ase when the job size distribution is regularly varying with index θ >, i.e., ΓV LPS ) > ΓV FCFS ). ii) better than worst-ase when the job size distribution is phase-type, i.e., γv LPS ) > γv PS ). 5 iii) optimal when the job size distribution is regularly varying with index θ Θ >, i.e., ΓV LPS ) = ΓV PS ). iv) optimal when the job size distribution is light-tailed and satisfies γb) γb) =, i.e., γv LPS ) = γv FCFS ). Θ Θ ρ + + or This orollary follows immediately from ombining Theorems and 2 with Lemma 4 see Appendix C). Design 2 provides a parameter, Θ, that allows the sheduling designer to tradeoff between the optimality guarantees in the light-tailed and heavy-tailed regimes, while at all times guaranteeing tail-robustness. Additionally, note that like Design, Design 2 is also robust against inexat estimates of ρ: as long as an upper bound on ρ is used Design 2 still guarantees properties i) and ii), though the sublasses of distributions defined in iii) and iv) hange depending on the estimation auray. 6. Conluding remarks The ontributions in this paper an be viewed along two axes. Firstly, we have derived GI/GI/ sojourn time tail asymptotis for the LPS- queue for both heavy-tailed and light-tailed job size distributions. These are the first results haraterizing the tail asymptotis of LPS-, whih is an important and pratial poliy that has reeived inreasing attention in reent years.

13 Nair, Wierman and Zwart / Performane Evaluation ) 22 3 Seondly, the results about LPS- illustrate that it is possible to design LPS- so that it is tailrobust, i.e., so that it provides a sojourn time tail that is robust aross heavy-tailed and light-tailed job size distributions. Prior to this work, there were no known poliies that had better than worstase sojourn time tails under both heavy-tailed and light-tailed job size distributions. Our results show that by hoosing = / ρ) +, LPS- is better than worst-ase aross large lasses of heavy-tailed and light-tailed job size distributions and even is optimal aross large sublasses of both heavy-tailed and light-tailed job size distributions. There are many interesting further researh questions that this work motivates along both of the diretions desribed above. First, with respet to the analysis of LPS-, it would be interesting to extend the asymptoti results presented to non-work-onserving ase where the servie rate is a funtion of the number of jobs in servie, as studied in [3]. This ase is important beause omputer systems have overheads that vary as a funtion of the multiprogramming level, usually in a unimodal fashion, and these variations an have a signifiant impat in the design of. We believe the analysis in the urrent paper should extend to this ase naturally, but there is not room in this paper to desribe the details. Seond, it would be quite interesting to study the performane of the suggested tail-robust designs for with respet to other performane metris, e.g., the expeted sojourn time. Finally, with respet to the design of tail-robust shedulers, it should be noted that our results provide the first example of a tail-robust poliy, and it would be interesting to understand if there are alternative designs that ahieve even better tradeoffs between robustness and optimality. Appendix A. Proofs for results in Setion 3 The goal of this appendix is to prove Theorem, whih desribes the logarithmi tail asymptotis of V LPS with regularly varying job sizes. We prove Theorem by proving mathing asymptoti) lower and upper bound on the tail of D LPS. The upper bound is established in Appendix A., and the lower bound is established in Appendix A.2. Finally, we use these bounds to omplete the proof of Theorem in Appendix A.3. Appendix A.. Upper bound The upper bound follows immediately from omparing LPS- with a FCFS GI/GI/ queue, where eah server has speed /. Speifially, let D GI/GI/ denote the stationary delay in the GI/GI/ system. It is easy to see that D LPS st D GI/GI/. A.) Therefore, we an obtain an asymptoti upper bound on the tail of D LPS from moment onditions for D GI/GI/, derived in [2]. The following lemma follows diretly from A.) and Theorem 2. in [2]. Lemma 5. Under Assumption, for η >, E [B η ] < E [ ] D kη ) LPS <. Appendix A.2. Lower bound We now prove our asymptoti lower bound on the waiting time tail in a GI/GI/ LPS- queue. Theorem 3. Under Assumption, if B e L 6 and E [B η ] < for some η >, then P D LPS > x) τ P B e > τ 2 x) k for positive onstants τ and τ 2. 6 B L B e L, but the onverse is not true; see Setion 3 of [4].

14 Nair, Wierman and Zwart / Performane Evaluation ) 22 4 To prove Theorem 3, we onstrut a bad event in whih a tagged job experienes a large waiting time. Intuitively, the bad event orresponds to k spare slots being filled by large jobs, whih auses the queue to beome overloaded and build a large baklog before the tagged job arrives. Assume the tagged job enters the system at time 0. Let D denote the waiting time of the tagged job and let S i, B i denote respetively the arrival instant and size of the ith job to enter the system before the tagged job. Before beginning the proof we need a bit of notation. For z > 0, denote B z) = BB z), β z) = E [ B z)], ρ z) = βz) ρ α. Sine ρ >, we an find a large enough y > 0 and small enough ɛ 0, β y) ) suh that Further, define, for x > 0, nx) = ρ > ρ y) = βy) α > βy) ɛ α + ɛ ρ x + y ρ ) β y) ɛ) α + ɛ) ρ Finally, define the following subsets of N k. For m N, > ρ. := ν x + ν 2. N m) = { n = n, n 2,, n k ) N k : m < n < n 2 < < n k }, N 2 m) = { } n = n, n 2,, n k ) N k : m n n 2 n k. Now, we are ready to build up the omponents of the bad event desribed above. First, define ) nx) ) Gx) = S nx) > nx)α + ɛ) B y) i > nx)β y) ɛ) := G x) G 2 x), i= where B y) i = B i B i y). Next, for n N nx)), define the event H n x) as follows. ) H n x) = S ni n i α + ɛ), n i α ɛ)), i =, 2,, k ) B ni > x + n i α + ɛ), i =, 2,, k ) B p x + pα + ɛ) p {nx) +, nx) + 2, } \ {n, n 2,, n k } := H n, x) H n,2 x) H n,3 x). Note that H n x) orresponds to k large jobs entering the system before the tagged job, with indies n i, i =, 2,, k. The job with index n i arrives in the interval n i α + ɛ), n i α ɛ)) and has size that exeeds x + n i α + ɛ). This implies that this job must remain in the system till time x. H n x) also implies that no other large arrivals our; this means for n, n 2 N nx)), the events H n and H n2 are mutually exlusive. Finally, our bad event is defined as follows: Ix) = Gx) ) n N nx)) H n x). The following lemma shows that the bad event does indeed ause the tagged job to experiene a large delay. Lemma 6. Ix) D > x. Proof. Sine Ix) H n x) for some n N nx)), Ix) implies k large jobs with indies stritly less than nx) remain in the system till time x. This means the nx) arrivals before the tagged job an reeive servie at a rate no greater than ρ till time x. This means that at time x, the

15 Nair, Wierman and Zwart / Performane Evaluation ) 22 5 work remaining in the system orresponding to the nx) arrivals before the tagged job having an original servie requirement bounded above by y stritly exeeds nx)β y) ɛ) ρ x + nx)α + ɛ)) = nx) β y) ɛ) ρ ) α + ɛ) x ρ y ρ ). This in turn implies the tagged job an reeive no servie until time x. All that remains is to bound P Ix)), and thus P D > x): P D > x) P Ix)) = P Gx) H n x) n N nx)) = P Gx) H n x) ) = P Gx) H n x) ) n N nx)) n N nx)) = P G x) G 2 x) H n, x) ) P H n,2 x) ) P H n,3 x) ) n N nx)) n N nx)) P G x) G 2 x) H n, x) ) P H n,2 x) ) P B p x + pα + ɛ) p N ) Using the weak law of large numbers, we see that the the probability of the events G x), G 2 x), and H n, x) approahes as x. Therefore, fixing δ 0, ), for large enough x, P G x) G 2 x) H n, x) ) δ. Also, invoking Lemma 7 stated and proved below), P B p x + pα + ɛ) p N ) α > 0 for large enough x. Therefore, for large enough x, k P D > x) α δ) P B > x + n i α + ɛ)). A.2) n N nx)) i= Define the bijetion ξ : N nx)) N 2 nx) + k) as follows. From A.2), PD > x) α δ) α δ) k! ξn, n 2,, n k )) = n + k, n 2 + k 2,, n k ). n N 2 nx)+k) i= n N k :n i nx)+k i= k P B > x + n i α + ɛ)) k P B > x + n i α + ɛ)) = α δ) k! n nx)+k P B > x + n α + ɛ)) α δ)βk α + ɛ) k k! P B e > x + α + ɛ)nx) + k))) k α The last inequality above uses the fat that for α, x > 0, i=0 β P B > x + i α) P B e > x). Finally, sine B e L, it is easy to see that P B e > x + α + ɛ)nx) + k)) P B e > + ν α + ɛ))x). P D > x) α δ)βk α+ɛ) k k! P B e > + ν α + ɛ))x)) k. This ompletes the proof. k

16 Nair, Wierman and Zwart / Performane Evaluation ) 22 6 Lemma 7. Assume B is a non-negative random variable satisfying E [ B +δ] < for some δ > 0. Then, for b > 0 satisfying F B b) > 0 and ã > 0, F B b + iã) = α b, ã) > 0. i= Proof. Define B = max{ B b ã, 0}. Clearly, E [ B +δ] < and for x > 0, F Bx) = F B b + xã). Pik δ 0, δ), ɛ > 0. Sine E [ B +δ] <, for large enough x, F Bx) { } + ɛ) exp x +δ The last inequality holds sine, for small enough y > 0, y e +ɛ)y. Let us say this inequality holds for x x 0. F B b + iã) = F Bi) F Bi)exp + ɛ) i +δ > 0. i= i= i< x 0 x +δ i= x 0 Appendix A.3. Proof of Theorem We an now omplete the proof of Theorem by ombining the upper and lower bounds derived in the preeding setions. Assume that B RVθ), θ >. Fix δ 0, ). Invoking Theorem 3, for large enough x, P D LPS > x) δ)τ P B e > τ 2 x) k lim sup log P D LPS > x) x logx) k lim log P B e > τ 2 x) = kθ ). A.3) x logx) For η, θ), E [B η ] <. Invoking Lemma 5, we onlude that E [ D kη ) LPS ] <. Therefore, for large enough x, P D LPS > x) x kη ) lim inf log P D LPS > x) kη ). x logx) Letting η θ, we get lim inf log P D LPS > x) kθ ). x logx) A.4) Finally, we omplete the proof of Theorem by noting that i) ) follows from A.3) and A.4), and ii) 2) follows from ) easily sine D LPS + B st V LPS st D LPS + B, where D LPS and B are independent. We omit the details due to spae limitations. Appendix B. Proofs for results in Setion 4 In this appendix, we prove the results stated in Setion 3. The first three setions are devoted to proving Theorem 2, whih desribes the deay rate V LPS with light-tailed job sizes. The proof for the ase γb) 0, ) is ompleted by establishing mathing asymptoti) lower and upper bounds on the tail of γv LPS ); this is done in Appendix B. and Appendix B.2 respetively. The proof for the ase γb) = is given in Appendix B.3. In Appendix B.4, we prove Lemma 3, whih establishes the monotoniity of γv LPS ) with respet to. Finally, we give the proof of Lemma 4 whih desribes properties of γv LPS )) in Appendix B.5.

17 Nair, Wierman and Zwart / Performane Evaluation ) 22 7 Appendix B.. Proof of Theorem 2: Lower bound for the ase γb) 0, ) In this setion, we prove the following asymptoti) lower bound on the tail of V LPS. Lemma 8. Assuming γb) 0, ), lim inf x x log P V LPS > x) min a [0,] f a). To begin the proof, note that sine f ) is ontinuous over [0, ], it suffies to prove that, for a 0, ), lim inf x x log P V LPS > x) f a). Fix a 0, ). Intuitively, to prove the above statement, we onstrut a bad event where the waiting time of a tagged job exeeds ax and its residene time exeeds a)x. Reall that we assume the tagged job has size B 0 and enters the stationary) system at time 0. We denote the sojourn time of the tagged job by V. We use a trunation-based argument. For z > 0, define B z) = BB z). Pik y > 0 large enough so that P B y) > A ) > 0. Consider a trunated system, in whih exept for the tagged job,) only jobs with size less than or equal to y are allowed to enter the system. Denote the total baklog in this trunated system just before the arrival of the tagged job by W y). The total work entering the trunated system in the time interval 0, u] is denoted by A y) u), i.e., Nu) A y) u) = B i B i y). i= For large x, small ɛ > 0, onsider the following event. Ix):= W y) > ax + )y ) A α) A y) u) > a) ) ) + ɛ)u A ) u A, x) :=I x) I 2 I 3 x). At the instant the tagged job begins servie, the maximum work remaining in the system orresponding to arrivals before time 0 is )y. Therefore, the event W y) > ax + )y and therefore event Ix)) implies the tagged job and subsequent arrivals wait for at least time ax before beginning servie. Moreover, at any time instant u ax, x), under Ix), the remaining work in the system orresponding to arrivals after time 0 exeeds ) ) a) + ɛ)u α) u ax) ) ) ) ) > a) + ɛ)u α) u au) = ɛ a) u α a) + ɛ) ) ) >ɛ a) ax α a) + ɛ) := ν x ν 2. Therefore, the number of jobs that arrived after time 0 and are still in the system at time u exeeds ν x ν 2 ) y > for large enough x. Therefore, under event Ix), the tagged job gets no servie until time ax and gets at most) servie at rate / in the interval ax, x). Therefore, B 0 > a)x ) Ix) V y) > x, where V y) denotes the sojourn time of the tagged job in the trunated system. Sine V y) st V, for large enough x, P V > x) P V y) > x ) P a)x B 0 > ) Ix) = P B 0 > ) a)x P I x)) P I 2 ) P I 3 x) I 2 ).