Analysis of Round-Robin Implementations of Processor Sharing, Including Overhead

Size: px

Start display at page:

Download "Analysis of Round-Robin Implementations of Processor Sharing, Including Overhead"

George Baker
5 years ago
Views:

1 Analysis of Round-Robin Implementations of Processor Sharing, Including Overhead Steve Thompson and Lester Lipsky Computer Science & Engineering University of Connecticut Storrs, CT Sarah Tasneem Math and Computer Science Eastern Connecticut State University Willimantic, CT 66 Feng Zhang Computer Science & Engineering University of Connecticut Storrs, CT Abstract It has been observed in recent years that in many applications service time demands are highly variable. Without foreknowledge of exact service times of individual jobs, processor sharing is an effective theoretical strategy for handling such demands. In practice, however, processor sharing must be implemented by time-slicing with a round-robin discipline. In this paper, we investigate how round-robin performs with the consideration of job switching overhead. Because of recent results, we assume that the best strategy is for new jobs to preempt the one in service. By analyzing time-slicing with overhead, we derive the effective utilization parameter, and give a good approximation regarding the lower bound of time-slice under a given system load and overhead. The simulation results show that for both exponential and non-exponential distributions, the system blowup points agree with what the effective utilization parameter tells us. Furthermore, with the consideration of overhead, an optimum time-slice value exists for a particular environment. I. INTRODUCTION In recent years it has been observed that many service time demands are highly variable [], [], [3], [4]. For example, the web and data servers were found to receive requests with processing times varying over several orders of magnitude [3], [4]. It is often the case that the majority of the jobs are short, while a tiny fraction of the largest flows constitutes more than half of the total load []. Such highly varied job demands can be modeled by some distribution. Specifically, let X be the random variable denoting the time needed for service by a single request, with the following definitions of F(x), the Probability Distribution Function (PDF), and probability density function (pdf), f(x). F(x) := Pr(X x), with f(x) := df(x) dx, Also let R(x) := F(x) (Complementary Distribution, or Reliability Function). then E[X l ] := with mean and variance o x l f(x)dx, x := E[X], σ = E[X ] ( x), We can then say that service demands are highly variable when C, where C σ := (E[X]) is the squared coefficient of variation. If the individual customer demands are known exactly, shortest-remaining-processing-time (SRP T ), which preempts the current customer being served if the newly arrived customer has less service time requirement, is optimal [], [6]. However, in many cases, no foreknowledge of exact service times of individual customers is available. In such a case, the default queueing discipline is first-come-first-served (FCFS). It is well known that FCFS at a single server yields a mean system time (total time for a customer, including waiting for service), given by the Pollaczek-Khinchine (P-K) formula: E[T s ] = x + x C, where the utilization is = λ x (λ being the arrival rate). Clearly, the larger C is, the longer is the system time. Because of this there has been an increased interest in processor sharing (PS), as this brings the mean system time to that of C =, i.e., E[T ps ] = x Clearly, if C < it would be foolish to use PS, since in this case E[T s ] < E[T ps ]. Therefore, hereafter we will always assume that C. It has been argued that FCFS is the fairest queueing discipline, and any attempt to let later arriving customers get earlier service is unfair. But for PS, the mean time for a job with service time x to complete service is x/( ). In other words the expected time spent getting and waiting for service is directly proportional to the amount of time needed. This too has a sense of fairness about it. Another scheduling scheme, Last Come first Served Preemptive Resume (LCFSPR) has also been considered because it produces the same mean system time as PS. Here an arriving customer preempts the customer presently being served and continues in this vein until he finishes or is himself preempted

2 by a newer arrival. The structure is a stack rather than a queue. But the system time is much more varied, and small jobs may actually get stuck in the queue for inordinately long times. Therefore, this scheme is considered unfair. Actually PS is an idealization, because it is virtually impossible to implement it on a single server. How can the server be dynamically partitioned into n pieces with each piece running independently? Instead one can use time-slicing with a round-robin discipline. A time interval, is chosen, and each job in the queue (actually, a ring) is allowed access to the processor for that time interval. As the discipline approaches PS. In any case, the job either finishes in that interval, or is interrupted and the processor is passed on to the next job. When a new job arrives several options are available. (A) It can be placed somewhere in the ring [(A) next in line, (A) last, or (A3) in a random slot], or (B) it can preempt the one presently in service. If the latter, then there are several options for the interrupted job after the newly arrived job finishes. Either it can (B) lose the rest of its time slice (the system moves to the next job in the ring); (B) receive the remaining time in its slice; or (B3) receive a whole new time slice. We have studied this in a previous paper [7] and found, by extensive simulations, that if option (A) is selected, the mean system time increases with increasing, approaching E[T s ], whereas (B) stays near x/( ), because with preemption the discipline approaches LCFSPR. In fact (B) and (B3) actually outperform PS over a wide range of values for. The shorter jobs get through even more quickly, while the longer jobs are only slightly delayed. (B) seems to outperform (B3) [(B3) would tend to favor longer jobs], so in what follows we will consider only (B) in our analysis. The rest of this paper is organized as follows: Section II summarizes some related work; Section III describes our analytical model for time-slicing with overhead; The simulation results are presented in Section IV; Section V concludes the paper. II. RELATED WORK Some analytical results have been given in the case of exponential service distribution with the consideration of overhead. For example, by assuming a constant swapping time overhead per time-slice, the exact expressions were presented in [8] for computing the queue size distribution and average waiting time for M/M/ using round-robin scheduling. The optimum quantum length was determined by considering a cost measure based on assigning priorities to jobs in a decreasing function of job service time. Similarly, this assumption was made in [9] for comparing round-robin and multi-quantum round-robin (i.e., an additional quantum being given to the currently executed job if a new job arrives during its execution) with exponential service distribution. However, the analysis of system performance for handling highly varied customer demands is more difficult if overhead is considered. So overhead is often assumed to be negligible. Cao et. al. [], presented an M/G/ K processor sharing queueing model of a multi-threaded web server. The model verification is accomplished by conducting experiments with one server (with Apache) and multiple clients. Server farm architecture is economic and flexible, which typically consists of a collection of servers and a front end high speed router. Analysis of different routing policies for FCFS server farms in the setting of high job size variability is reported in []. Gupta et. al. presented the first analysis of Join-Shortest-Queue for processor sharing farms []. They reported a single queue approximation to analyze a server farm by looking at just one queue in isolation from all other queues, but where the arrival rate into the queue is conditional (state dependent) on the number of jobs at that queue. This state dependent arrival rate is used to capture the effect of the other queues. A recent paper by Gupta [3] covered similar material to what we do here, but with the constraint that an arriving customer must wait for all customers already present to get a time slice before it gets its first. However, we have shown in a previous paper [7] that as long as C >, this is the worst strategy among those easily implementable in such systems. Therefore the performance formulas presented there are not applicable here. In [7] simulation results have been produced for several RR variants with negligible switching overhead, where a newly arrived job preempts the current active job. Our results demonstrated that significant improvement of mean system time can be achieved by allowing preemption in RR scheduling. Even if the newly arrived job is put at the front of the queue (without preemption) performance is better than putting it last. In this paper, we take the overhead into account. III. TIME-SLICING WITH OVERHEAD Since there must be some overhead cost in using timeslicing with a round-robin discipline, we now assume that each time slice costs α x units of time to implement, and a new job interrupt costs α x. For now we assume that α = α. Clearly, the smaller, the more interrupts occur and the greater the overhead. Fortunately we have found, for α =, [7] that minimum system time occurs somewhere in the range x/ < < x, depending on the distribution F(x) examined. That is, most jobs can finish in one time slice. Our goal here is to see where the minimum is when α >. But first it is necessary to know the range of values for which a simulation is meaningful. It is clear that overhead added to workload can overrun the server, causing system time to become unbounded. This depends on, α, and. To get some insight into the interrelationship of these parameters we first find the expected number of time intervals needed by a job. The probability that a job will require exactly n time slices of length is Pr[N( ) = n] = R([n ] ) R(n ), n.

3 Then the expected number of slices is given by npr[n( ) = n] = n= n [R([n ] ) R(n )]. n= After some manipulation this becomes R(n ) = R(n ) +, () n= where we have made use of the fact that R() =. The first sum (from ), when multiplied by, is nothing more than the left-hand-rule for numerical integration of R(x). The second sum (from ) is the right-hand-rule for numerical integration. But it is known that o R(x)dx = x, and since R(x) in a monotonically non-increasing function of x, it follows that E[N( )] = R(n ) This leads to or x R(n ) n= = E[N( )]. x + E[N( )] x, x + γ( ) () where γ( ). In Appendix A we derive explicit formulas for E[N( )] for hyper-exponential and Erlangiantype distributions. In all those cases, and small enough, we show that x/ + / is a good approximation to E[N( )]. In Appendix B we state and outline the proof for a theorem describing lim γ( ). We now claim that the total processor time needed by a job, on average, is x plus the amount of overhead it needs to handle the context switching. I.e., (the subscript e stands for effective ) x e = x + α xe[n( )], then the effective utilization parameter is e := λ x e = + αe[n( )] (3) For the system to be stable, it is required that e <. By using Equation (3) and e = we get a bounding relationship among α,, and. That is, for a given and α, the minimum time slice, min satisfies the non-linear equation αe[n( min )] =. (4) and > min for all acceptable time slices. If instead, and α are given, then for stability, < max := + αe[n( )]. () Finally, for given and, the overhead factor must satisfy α < α max := E[N( )]. As an example, in Appendix A we show that for exponential service time distributions, e / x. So for =., and α {.,.,.,.}, Equation () gives max {.,.97,.887,.7973}. The blow-up points in Figure correspond to these values. IV. SIMULATION RESULTS When e we expect the system to be unstable. To test this we ran some simulations of algorithm (B), with exponential service time distribution, with parameters =., and α {.,.,.,. }. Each run saw 9 arrivals. The results are shown in Figure. As expected, bigger α s caused the system time to blow up with smaller. We tested Equation (3) by plotting ( e ) Rt() versus, as shown in Figure. Here, all the curves are horizontal until e =, at which point the curves are meaningless. The height of each line is precisely x o. Given that our argument is correct, this result is to be expected. That is, all the curves in Figure satisfy Rt( α) = x e (6) e the same as the M/M/ and PS queues. Note, this is only true for exponential service times. Only in that case are time slices irrelevant, because every job, no matter how much service it has already received has the same expected time remaining (memoryless property). Figure 3 shows the same story even when the service times are non-exponential, in this case hyper-exponential- with x = and C =, namely, R(x) = p e xµ + ( p)e xµ. (7) We have chosen p =., µ =.36 and µ = Then p/µ + ( p)/µ =. Figure 3 looks like Figure. The blow-ups occur for the same values of, namely when e =. But when each curve is multiplied by ( e ), the difference between exponential and non-exponential service times becomes evident, as seen in Figure 4. Here the lines are not horizonal. In fact they have a negative slope, indicating, in this case, that response time is improving (getting smaller) relative to PS, as increases. Although we feel we have presented some interesting results, the practical application is in selecting (perhaps dynamically) the time-slice that is best for a particular working environment. Suppose that the workload and overhead are known (by knowing what R(x) and α are). Further, at some

4 4 4 3 α =. α =. Exponential Distribution = α =. α =. Hyper Exponential Distribution (p =., c =) =. 3 3 Rt() RT() Fig.. Round-robin Time-slicing queue with Exponential Service times. Rt() versus, with time slice, =.. Each time slice costs the server a factor of, α {α {.,.,.,. }. Fig. 3. Rt() versus, with Hyper-exponential Service times. =., C =, and α {.,.,.,. }. The blow-up points again occur at e =. RT()*( o ) Exponential Distribution =. α =. α = Fig.. Exponential Service times. Rt() ( e) versus, with =. The lines are horizontal, indicating that Equation (6) is correct, something to be expected for exponential service times. RT()*( o ) Hyper Exponential Distribution (p =., c =) α =. α =. = Fig. 4. Same Hyper-exponential Service times as Figure 3. But now Rt() ( e) versus is plotted. As in the previous figures, =.. Note that the lines have a negative slope, implying that Rt() is smaller than for pure PS. time, the arrival rate, λ has been measured. Then, can be computed, permitting the system to be studied as is varied. Figure is one such study, for hyper-exponential service times and =.7. Note that even for no overhead (α = ) timeslicing outperforms PS, with a minimum at about opt =.. That is, each time slice is larger than the mean service time. For non-negligible overhead, performance blows up if is too small, i.e., less than min according to Equation (4), but there is an optimum opt for each α. As α increases, so does opt. For very large, the system approaches LCFSPR, where each job needs x(+α) amount of service. After all, each job must interrupt the system once upon its arrival. V. CONCLUSIONS AND FUTURE WORK In this paper, we studied the performance of time-slicing with overhead. An analytical model was presented to calculate the job arrival rate as well as the lower bound time-slice value when the system will become unstable. The simulation study verifies the model. In presenting this study we have selected a specific algorithm for implementing time slicing. We have not discussed other algorithms, except a brief description in the introduction. We plan to do that in the future. In particular, the two alternatives labeled (B) and (B3) will be examined closely. Also, in creating our simulation, we assumed that if a job

5 Rt() Hyper Exponential Distribution (p =., c =) =.7 α =.. α =. 3 Fig.. Hyper-exponential Service times. Rt() versus for several values of α, and =.7. arrives while the processor is in overhead mode, then the new job interrupts the overhead and begins execution, starting with execution of the overhead needed for its initiation. When the new job s time slice is over (or the job completes), the interrupted overhead function resumes where it left off. In reality this is probably over optimistic, and we plan to run simulations where the overhead function must start over. We will also modify our analytic model to account for this as well (if we can). We also intend to look more deeply at the distribution of response times. APPENDIX A - EVALUATION OF E[N( )] In Equation () we established that R(n ). If the service time is exponentially distributed, then [ R(n ) = e n / x = e / x] n. Therefore the sum is a geometric series (where n= zn = /( z), yielding exp( / x) = x + + x +. Thus we see from its definition in Equation () that γ( ) / as long as < x. It follows directly that if the service time is hyper-exponential-, as given by Equation (7), then p e µ + p e µ x + + p µ where µ > µ. For hyper-exponential distributions in general, it is clear that γ( ) / as long as µ max <, where µ max = max{µ i }. The Erlangian- distribution has a reliability function of the form R(x) = ( + µx)e µx where E[X] = x = /µ. Then [ (e µ ) n ( + µ n e µ ) ] n After some manipulation, using n= nzn = z/( z), we get e µ + µ e µ ( e µ ). A Taylor series expansion yields E[N( )] x Once again, γ( ) / if < x. ( ) 3. x APPENDIX B - γ( ) FOR ANY DISTRIBUTION We have seen that for Distributons with exponential tails γ( ) /. Here we state as a theorem the behavior for all distributions, and then outline the proofs. Theorem: We have already seen that γ( ), defined by Equation (), is bounded by and. () If R(x) and f(x) are continuous in [, ) then lim γ( ) = /. () If R(x) and f(x) are continuous in (, ), but R( + ) = p then lim γ( ) = p (3) Even if R(x) has a finite number of kinks [f(x) has a finite number of discontinuities] the above equation is valid. If there are an infinite number of kinks then the limit still exists but depends on the particular function. (4) If R(x > ) has at least one discontinuity then the limit does not exist. Let r be the sum of these discontinuities, then γ( ) = p r + a( )r where a( ) varies cyclically between and as x/ goes from one integer value to the next with decreasing. Proof: () We have seen that γ( ) = R(n ) x. Consider the interval between n and (n + ). The area of the rectangle with height R(n ) minus the area bounded from above by R(x) is a triangular-like piece with height [R(n ) R((n + ) )]. If R (x) is continuous, then as gets smaller, R(x) approaches a straight line. The area of this triangle, then becomes [R(n ) R((n+) )]/+O( 3 ). The sum of these triangles equals γ( ), so γ( ) = [R(n ) R((n + ) )] + O( ) n= = R() + O( ) =.

6 () If there is an initial impulse [R( + ) = p < ] then the first interval has an initial rectangle of height ( p) on top of a triangle. This adds an area of ( p) to the sum of triangles, yielding γ( ) = ( p) + p p + O( ) =. (3) If R(x) has kinks then the intervals containing the kinks will not be triangles. But if there are only a finite number of kinks, then as becomes small enough, there will be exactly one interval for each kink (no two kinks will fall in the same interval). But the area of each is of order of (both the height and width get smaller with ) and since there are only a finite number of them, their contribution to the sum goes to. It is not clear what happens if here are an infinite number of kinks. (4) If R(x > ) has even one discontinuity [R(y ) R(y + ) > for some y > ] then the area contributed changes with. This differs from () because there the impulse always occurs at the beginning of the first interval, whereas here the impulse can occur anywhere in the interval. Consider the simplest example, the deterministic distribution. Here, R(x) = for x < x and R(x) = for x > x. Then by definition, γ( ) = E[N( )] x = x x. For some, x/ will be an integer, in which case, γ( ) =. But as soon as decreases slightly, the ceiling function, and thus γ( ), jumps by, gradually decreasing to as x/ approaches the next integer value. REFERENCES [] M. E. Crovella and A. Bestavros, Self-similarity in world wide web traffic: Evidence and possible causes, IEEE/ACM Transactions on Networking, vol., no. 6, pp , 997. [] M. Greiner, M. Jobmann, and L. Lipsky, The importance of powertail distributions for modeling queueing systems, Operations Research, vol. 47, no., pp , 999. [3] H. Han, S. Shakkottai, S. Member, C. V. Hollot, R. Srikant, and D. Towsley, Multi-path tcp: A joint congestion control and routing scheme to exploit path diversity on the internet, IEEE/ACM Transactions on Networking, vol. 4, no. 6, pp. 6 7, 6. [4] D. Starobinski and M. Sidi, Modeling and analysis of power-tail distributions via classical teletraffic methods, Queueing Syst. Theory Appl., vol. 36, no. -3, pp ,. [] N. Bansal and M. Harchol-Balter, Analysis of srpt scheduling: Investigating unfairness, in Proc. ACM Sigmetrics,. [6] L. Schrage and L. Miller, The Queue M/G/ with the Shortest Processing Remaining Time Discipline, Operations Research, vol. 4, pp , 966. [7] F. Zhang, S. Tasneem, L. Lipsky, and S. Thompson, Analysis of roundrobin variants: Favoring newly arrived jobs, in 4nd Annual Simulation Symposium (ANSS 9), 9. [8] P. J. Rasch, A queueing theory study of round-robin scheduling of time-shared computer systems, J. ACM, vol. 7, no., pp. 3 4, 97. [9] H. C. Heacox, Jr. and P. W. Purdom, Jr., Analysis of two time-sharing queueing models, J. ACM, vol. 9, no., pp. 7 9, 97. [] J. Cao, M. Andersson, C. Nyberg, and M. Kihl, Web server performance modeling using an m/g//k*ps queue, in th International Conference on Telecommunications (ICT3, 3, pp. 6. [] M. Harchol-Balter, M. E. Crovella, and C. D. Murta, On choosing a task assignment policy for a distributed server system, J. Parallel Distrib. Comput., vol. 9, no., pp. 4 8, 999. [] V. Gupta, M. Harchol Balter, K. Sigman, and W. Whitt, Analysis of join-the-shortest-queue routing for web server farms, Perform. Eval., vol. 64, no. 9-, pp. 6 8, 7. [3] V. Gupta, Finding the optimal quantum size: Sensitivity analysis of the m/g/ round-robin queue, in th Workshop on Mathematical Performance Modeling and Analysis (MAMA 8), 8.

Servers. Rong Wu. Department of Computing and Software, McMaster University, Canada. Douglas G. Down

Servers. Rong Wu. Department of Computing and Software, McMaster University, Canada. Douglas G. Down MRRK-SRPT: a Scheduling Policy for Parallel Servers Rong Wu Department of Computing and Software, McMaster University, Canada Douglas G. Down Department of Computing and Software, McMaster University,