Optimal Control of a Heterogeneous Two Server System with Consideration for Power and Performance

Transcription

1 Optimal Control of a Heterogeneos Two Server System with Consideration for Power and Performance by Jiazheng Li A thesis presented to the University of Waterloo in flfilment of the thesis reqirement for the degree of Master of Applied Science in Electrical and Compter Engineering Waterloo, Ontario, Canada, 2014 c Jiazheng Li 2014

2 I hereby declare that I am the sole athor of this thesis. This is a tre copy of the thesis, inclding any reqired final revisions, as accepted by my examiners. I nderstand that my thesis may be made electronically available to the pblic. ii

3 Abstract In this thesis we consider a system of two heterogeneos servers with a shared qee, and examine a schedling policy for the optimal control of sch a system. Previos reslts by Lin and Kmar [1] and Koole [2] fond that a threshold policy, i.e., refraining from assigning a job to a slow server ntil a certain threshold has been exceed in the job qee, is optimal when seeking only to minimize the mean sojorn time of a job in the system. We bild pon these reslts and generalise the analytical proof of the threshold policy s optimality to take into accont power consmption as another performance metric, in the setting where the faster server is more efficient. We also obtain preliminary reslts for a setting where the slower server is more efficient, nder the restriction of low arrival rates. We se experimental data from simlations to provide an assessment of the real world applicability of a threshold policy in this setting; a comparison between a threshold policy with optimal thresholds and a first-come-first-serve policy shows that it achieves a cost improvement of p to 29.19% over the naive policy. iii

4 Acknowledgements I wold like to thank Professors Shreyas Sndaram and Siddharth Garg for providing me with this opportnity for research, the gidance throghot the corse of my stdies, and ltimately the encoragement that allowed me to persevere. iv

5 Table of Contents List of Figres vii 1 Introdction Related Work Discrete Time Problem Formlation Discrete State Model Cost Modelling and Fixed Point Iteration Optimality of Threshold Policy When µ 1 > µ Bonds on Cost Difference Between Adjacent States Keep the Efficient Server Bsy When Possible Send Jobs to the Less Efficient Server When Qee Length Exceeds Threshold 24 4 Extension to µ 1 < µ 2 with Small λ J β (0, 1, 0) J β (0, 0, 1) Ftre Work to Show Optimality of Threshold Policy When µ 1 < µ v

6 5 Simlation Reslts Simlation Setp Threshold Policy Compared Conclsion 47 APPENDICES 49 A Detailed Proofs 50 A.1 Proof for Property 3 of Lemma A.2 J β (x) is increasing with λ A.3 The set of fnctions with properties listed in Proposition 1 is non-empty. 54 A.4 The set of fnctions with properties listed in Proposition 2 is non-empty. 54 B Simlator Sorce Code 56 References 74 vi

7 List of Figres 1.1 Qeeing system model Threshold policy with optimal thresholds compared against first-come-firstserve vii

8 Chapter 1 Introdction With the increasing biqity of mobile devices and web services in recent years, redcing power sage in compter systems has become an increasingly critical design objective. For mobile devices, battery capacity is often limited by constraints on the device s weight and physical dimensions, and redcing power consmption whenever possible is key to improving rn time performance. On the other side of the spectrm, the advent of web services and clod compting have necessitated large scale server farms with thosands of compting nodes to adeqately meet consmer needs. Server farms are highly energy intensive, and the operating costs incrred by energy consmption and removal of the resltant heat can rival the cost of the server hardware itself [3]; redcing the power sage in compting nodes redces operating costs, decreases the strain imposed on the electrical grid, and redces the emission of air polltants and greenhose gases [4]. In order to increase energy efficiency in hardware, varios researchers (sch as Cao et al. [5]) have proposed the se of asymmetric mlticore processors (AMP) which consists of a fast, high power core and a slow, low power core in order to optimize both performance 1

9 and energy sage based on the workload. In this thesis we examine a strategy for the optimal control of an AMP having two cores, one of which is more energy efficient than the other in terms of mean energy consmed per job dring service. The AMP is modelled as a heterogeneos qeeing system as shown in Figre??, consisting of two exponential servers with different service rates µ 1 and µ 2. Incoming jobs are modelled as a Poisson process with rate λ and stored in a common qee while waiting to be serviced. After each system event, which can be the arrival of a new job into the qee or the departre of a job which has completed service, an assignment decision is made by the schedling policy to decide if a job in the qee shold be assigned to a server for service. Performance is characterized here by the mean response time of the system, defined as the mean dration a job spends in the system from arrival ntil departre. Mean energy consmption is determined by charging a cost per nit time (E 1 and E 2 for server 1 and 2 respectively) for each time nit a server spends servicing a job. In a real-world setting, the fixed cost corresponds to the increase in power consmption of a server when it is serving a job compared to when it is idle. The goal of this thesis is to present an optimal control policy which minimizes a weighted sm of the mean response time and mean energy consmption, where varying the weight provides soltions on the Pareto bondary of mean response time and mean energy consmption. 1.1 Related Work Lin and Kmar [1] stdied the optimal control of the system shown in Figre?? with the goal of minimizing only the mean response time. They formlated a discrete-time problem by sampling the state of the system when a transition event occrs, i.e., when a job arrives 2

10 Arrivals λ Qee Server 1 µ 1 Departres Server 2 µ 2 Departres Figre 1.1: Qeeing system model at or departs from the system. The discrete-time problem is then modelled as a Markov decision process, and sing vale iteration they demonstrate that the faster server shold be sed whenever possible. Next, they se policy iteration to show that the optimal policy is a stationary policy of the threshold type, which keeps the faster server bsy whenever possible and makes se of the slower server only if the nmber of jobs in the qee exceeds a certain threshold. The intition behind this reslt is that when the arrival rate is low and not many jobs are in the qee, instead of sending a job to the slow server it may be better to wait for the fast server to finish service and service it there instead, reslting in a faster mean response time. Bilding pon Lin and Kmar s reslt that the fast server shold be kept bsy whenever possible, Koole [2] simplifies the rest of the proof by sing vale iteration instead of policy iteration. He shows that over the infinite horizon, the difference in expected cost of a policy which does not send a job to the slow server compared to one that does is monotonically increasing with the qee length; when the difference is negative, it is better not to send jobs to the slow server, and as qee length increases, the difference eventally becomes 3

11 positive, at which point it becomes better to send jobs to the slow server. This demonstrates the optimality of threshold behavior with the crossover point as the threshold vale. However, these proofs cannot be trivially extended to the setting where energy costs are taken into accont. Rykov [6] made se of vale iteration to show that the optimality of the threshold policy is preserved when additional qeeing and service penalties are specified by sing the minimm total average service cost as the criterion for server efficiency, nder the restriction that more efficient servers also have higher service rates than less efficient servers; the case where the more efficient server is slower remains an open problem [7], and is one of the scenarios we consider in this thesis. We will make se of lower bonds on the cost difference between adjacent states to demonstrate that the optimal policy has characteristics of a threshold policy for the setting where the faster server is more efficient. In the setting where the slower server is more efficient, we will provide partial analytical reslts in conjnction with simlation reslts which demonstrate the cost benefit of the threshold policy compared to a naive first-come-first-serve policy. 4

12 Chapter 2 Discrete Time Problem Formlation 2.1 Discrete State Model Adopting the strategy from [1], we begin by discretizing the continos time system described in Chapter 1 by sampling the system at the instants in time when an event occrs, which can be a job arriving in the qee or leaving either server. In order to ensre the probability of sampling is the same regardless of system state, we mst assme that if a server is idle then it is serving a dmmy job, and therefore has the same probability of being sampled as if it were serving a real job. We also normalize λ, µ 1, and µ 2 sch that λ + µ 1 + µ 2 = 1 while maintaining their proportion to each other, so that the probability of an event taking place, or eqivalently, the probability of the system being sampled dring the interval (t, t + dt) is (λ + µ 1 + µ 2 )dt = dt. The state of the discretized system at time-step k is x k = (x 0 k, x1 k, x2 k ) N {0, 1} {0, 1}, where x 0 k is the nmber of jobs in the qee, and x1 k and x2 k are the nmber of jobs being serviced on server 1 and 2 respectively. Servicing a job on server 1 and 2 also incrs 5

13 a cost of E 1 > 0 and E 2 > 0 respectively. Given the state of the system x k at all time instants, the mean response time can be written as: 1 T = lim t λt and the mean cost can be written as: 1 Ē = lim t t t k=1 t k=1 ( ) x 0 k + x 1 k + x 2 k, ( ) E1 x 1 k + E 2 x 2 k. The objective of the controller is to select control decisions which minimize T + ᾱē, where ᾱ > 0 is a pre-specified weight factor. In the rest of this paper, we will deal with the following eqivalent objective fnction: λ T + αē, where α = λᾱ. By Little s Law [8], λ T is eqal to the mean nmber of jobs in the system. As in the work by Lin and Kmar [1], we work with the disconted cost objective instead of the average cost objective: [ ] E β k g(x k ), (2.1) where β [0, 1) is a discont factor and k=0 g(x k ) = x 0 k + x 1 k + x 2 k + αe 1 x 1 k + αe 2 x 2 k is the stage cost at timestep k. Withot loss of generality, we designate server 1 as the more efficient server, i.e., 1 + αe 1 µ 1 < 1 + αe 2 µ 2. (2.2) Three system events are possible: an arrival into the system, a departre from server 1, or a departre from server 2. The state of the system after a new job arrival is indicated 6

14 by Ax k and the state after a departre from server i is indicated by D i x k (i {1, 2}); we define these states to be adjacent to x k, and vice-versa. The state mappings are as follows: A(x 0, x 1, x 2 ) = (x 0 + 1, x 1, x 2 ), D 1 (x 0, x 1, x 2 ) = (x 0, 0, x 2 ), D 2 (x 0, x 1, x 2 ) = (x 0, x 1, 0). Note that the departre of a dmmy cstomer does not change the state of the system. After each event, one of the following for possible control decisions is selected for the transition to the next state: hold the job at the head of the qee (P h ), assign a job to server 1 (P 1 ), assign a job to server 2 (P 2 ), or assign jobs to both servers (P b ). The notation x k Dom(P i ) (i {0, 1}) is sed to indicate that x 0 k > 0 and xi k = 0, i.e., a job is available in the qee for assignment and server i is available to service it. The notation x k Dom(P b ) is sed to indicate that x 0 k > 1, x1 k = 0 and x2 k = 0, i.e., there are at least two jobs in the qee and both servers are free to begin service. The state of the system after applying control P is indicated by P x k, where {h, 1, 2, b} and x k Dom(P ). The state mappings are as follows: P h (x 0, x 1, x 2 ) = (x 0, x 1, x 2 ), P 1 (x 0, x 1, x 2 ) = (x 0 1, x 1 + 1, x 2 ) defined on Dom(P 1 ), P 2 (x 0, x 1, x 2 ) = (x 0 1, x 1, x 2 + 1) defined on Dom(P 2 ), P b (x 0, x 1, x 2 ) = (x 0 2, x 1 + 1, x 2 + 1) defined on Dom(P b ). 7

15 2.2 Cost Modelling and Fixed Point Iteration Let J β (x) represent the minimm cost over all policies with initial state x. The stochastic Bellman eqation is J β (x) = g(x) + βλ min J β (P 0 Ax) + βµ 1 min J β (P 1 D 1 x) + βµ 2 min J β (P 2 D 2 x). Let F be the Banach space of all fnctions f : X R where the norm defined by f = sp f(x) x X max(g(x), 1), is finite, and define the dynamic programming operator T : F F as T f(x) = g(x) + βλ min f(p 0 Ax) + βµ 1 min Observation 1. For some n, T (n) is a contraction mapping [9]. f(p 1 D 1 x) + βµ 2 min f(p 2 D 2 x). By the Banach fixed-point theorem, for any f F, T n f will converge to a niqe fixed point w sch that T w = w [10]. Since the optimal cost fnction J β F cannot contract frther, it stands to reason that T J β = J β and therefore J β is the niqe fixed point to which lim n T n f converges. In the context of dynamic programming this techniqe is known as vale iteration. In the following chapter we will se vale iteration to show that there is a non-empty and closed set of fnctions G F having properties of a threshold policy and is invariant nder T. In other words, for any f G, T f is also G, therefore the fixed point lim n T n f = J β is in G. By extension, since the fixed point is niqe in F, any f F will also converge to the same fixed point, and being in G we can conclde that it has properties of a threshold policy. 8

16 We note that when α = 0, i.e., in the setting of [1], the stage costs satisfy g(p h x k ) = g(p 1 x k ) = g(p 2 x k ) for x Dom(P h ) Dom(P 1 ) Dom(P 2 ). The fact that the stage costs do not depend on the control decision simplifies the proof, since the the terms cancel ot when comparing control actions. However when α > 0, the proof becomes non-trivial as different control decisions incr different energy costs, and we mst show that the difference in ftre costs over the infinite time horizon balances ot the difference in immediate energy costs. 9

17 Chapter 3 Optimality of Threshold Policy When µ 1 > µ 2 In this chapter, we prove the optimality of threshold policies for the M/M/2 system described in Chapter 2, in the setting where the more power efficient server is also faster (i.e., µ 1 > µ 2 and 1+αE 1 µ 1 < 1+αE 2 µ 2.). In Section 3.1 we provide expressions for lower bonds on the difference in costs between adjacent states. In Section 3.2 we se the reslts from Section 3.1 to show the optimal policy has the first characteristic of a threshold policy, i.e., the more efficient server shold be kept bsy whenever possible. Finally in Section 3.3 we se the reslts obtained in the previos sections to show that the optimal policy has the second characteristic of a threshold type policy, i.e., jobs shold be sent to the less efficient server once the qee length exceeds some threshold. These two characteristics are sfficient to show that the optimal policy is of threshold type. 10

18 3.1 Bonds on Cost Difference Between Adjacent States In this section, lower bonds are shown for the change in the optimal vale fnction J β immediately after a system event, which can be an arrival or a departre from either server. Note that the reslts in this section do not depend on µ 1 > µ 2, and are eqally applicable to the setting where the slower server is more efficient. Lemma 1. There exists β [0, 1) sch that for all β [β, 1), the optimal vale fnction J β has the following properties for all x: 1. J β (x 0 + 1, x 1, x 2 ) J β (x 0, x 1, x 2 ) δ 1 where ( ) 1 δ 1 = min 1 β, δ 2, δ 3. (3.1) 2. J β (x 0, 1, x 2 ) J β (x 0, 0, x 2 ) δ 2 where δ 2 = 1 + αe 1 1 β(1 µ 1 ). (3.2) 3. J β (x 0, x 1, 1) J β (x 0, x 1, 0) δ 3 where δ 3 = 1 + αe 2 1 β(1 µ 2 ). (3.3) Proof. We first define G F as the set of fnctions f : X R which have the above properties. Note that G is non-empty; for example consider the fnction f 0 (x 0, x 1, x 2 ) = (x 0 + x 1 ) 1 + αe 1 µ 1 + x αe 2 µ 2. (3.4) 11

19 We have f 0 (x 0 + 1, x 1, x 2 ) f 0 (x 0, x 1, x 2 ) = 1 + αe 1 µ 1 δ 1, f 0 (x 0, 1, x 2 ) f 0 (x 0, 0, x 2 ) = 1 + αe 1 µ 1 δ 2, f 0 (x 0, x 1, 1) f 0 (x 0, x 1, 0) = 1 + αe 2 µ 2 δ 3, which shows that f 0 satisfies all of the properties in Lemma 1, and therefore f 0 G. Next consider a fnction f G. First we will show that min f(p x) G. Note that min a f(p a x a ) min b f(p b x b ) δ if, for any valid control decision P i nder x a, there exists a valid decision P j nder x b sch that f(p i x a ) f(p j x b ) δ, since f(p j x b ) min b f(p b x b ) and therefore f(p i x a ) min b f(p b x b ) δ as well. 1. Define x a = (x 0 +1, x 1, x 2 ), x b = (x 0, x 1, x 2 ). We now explore every possible action on x a and show that for each of them a valid action on x b exists sch that the difference δ 1. For P a = P h, we have For P a = P 1, we have For P a = P 2, we have For P a x 1 = 0, we have f(p h x a ) f(p h x b ) = f(x 0 + 1, x 1, x 2 ) f(x 0, x 1, x 2 ) δ 1. f(p 1 x a ) f(p h x b ) = f(x 0, 1, x 2 ) f(x 0, 0, x 2 ) δ 2 δ 1. f(p 2 x a ) f(p h x b ) = f(x 0, x 1, 1) f(x 0, x 1, 0) δ 3 δ 1. = P b, noting that x a Dom(P b ) implies x b Dom(P 1 ) since x 0 1 and f(p b x a ) f(p 1 x b ) = f(x 0 1, 1, 1) f(x 0 1, 1, 0) δ 3 δ 1. 12

20 2. Define x a = (x 0, 1, x 2 ) and x b = (x 0, 0, x 2 ). We explore all possible actions on x a noting that P 1 and P b are not valid decisions nder x a. For P a = P h, we have f(p h x a ) f(p h x b ) = f(x 0, 1, x 2 ) f(x 0, 0, x 2 ) δ 2. For P a = P 2, we have f(p 2 x a ) f(p 2 x b ) = f(x 0 1, 1, 1) f(x 0 1, 0, 1) δ Define x a = (x 0, x 1, 1) and x b = (x 0, x 1, 0). We explore all possible actions on x a noting that P 2 and P b are not valid decisions nder x a. For all P a = P h, we have f(p h x a ) f(p h x b ) = f(x 0, x 1, 1) f(x 0, x 1, 0) δ 3. For P a = P 1, we have f(p 1 x a ) f(p 1 x b ) = f(x 0 1, 1, 1) f(x 0 1, 1, 0) δ 3. The above shows that min f(p x) G for any fnction f G. Next we proceed with vale iteration to show that T f G. 1. We start by showing that Property 1 in the Lemma is invariant nder T. After 13

21 transformation, f(x a ) and f(x b ) can be written as: T f(x 0 + 1, x 1, x 2 ) = x x 1 + x 2 + αe 1 x 1 + αe 2 x 2 + βλ min f(p 0 A(x 0 + 1, x 1, x 2 )) + βµ 1 min f(p 1 D 1 (x 0 + 1, x 1, x 2 )) + βµ 2 min f(p 2 D 2 (x 0 + 1, x 1, x 2 )) = x x 1 + x 2 + αe 1 x 1 + αe 2 x 2 (3.5) + βλ min f(p 0 (x 0 + 2, x 1, x 2 )) + βµ 1 min f(p 1 (x 0 + 1, 0, x 2 )) + βµ 2 min f(p 2 (x 0 + 1, x 1, 0)), T f(x 0, x 1, x 2 ) = x 0 + x 1 + x 2 + αe 1 x 1 + αe 2 x 2 + βλ min f(p 0 A(x 0, x 1, x 2 )) + βµ 1 min f(p 1 D 1 (x 0, x 1, x 2 )) + βµ 2 min f(p 2 D 2 (x 0, x 1, x 2 )) = x 0 + x 1 + x 2 + αe 1 x 1 + αe 2 x 2 + βλ min f(p 0 (x 0 + 1, x 1, x 2 )) + βµ 1 min f(p 1 (x 0, 0, x 2 )) + βµ 2 min f(p 2 (x 0, x 1, 0)). (3.6) 14

22 Taking the difference between Eqation (3.5) and Eqation (3.6), T f(x 0 + 1, x 1, x 2 ) T f(x 0, x 1, x 2 ) = 1 + βλ(min f(p 0 (x 0 + 2, x 1, x 2 )) min f(p 0 (x 0 + 1, x 1, x 2 ))) + βµ 1 (min f(p 1 (x 0 + 1, 0, x 2 )) min f(p 1 (x 0, 0, x 2 ))) + βµ 2 (min f(p 2 (x 0 + 1, x 1, 0)) min f(p 2 (x 0, x 1, 0))) 1 + βλδ 1 + βµ 1 δ 1 + βµ 2 δ 1 = 1 + βδ 1. Since δ 1 can take the vale of 1 1 β, δ 2, or δ 3, we consider each case and show that 1 + βδ 1 δ 1 for any possible vale of δ 1. If δ 1 = 1 1 β, T f(x 0 + 1, x 1, x 2 ) T f(x 0, x 1, x 2 ) 1 + β 1 1 β = δ 1. If δ 1 = δ 2, we have T f(x 0 + 1, x 1, x 2 ) T f(x 0, x 1, x αe 1 ) 1 + β 1 β(1 µ 1 ) = 1 + β(µ 1 + αe 1 ) 1 β(1 µ 1 ). For β αe 1 µ 1 +αe 1, and therefore 1 + β(µ 1 + αe 1 ) 1 β(1 µ 1 ) 1 + αe 1 1 β(1 µ 1 ) = δ 2, T f(x 0 + 1, x 1, x 2 ) T f(x 0, x 1, x 2 ) δ 2 = δ 1. Similarly for δ 1 = δ 3 and β αe 2 µ 2 +αe 2, T f(x 0 + 1, x 1, x 2 ) T f(x 0, x 1, x 2 ) δ 3 = δ 1. 15

23 We have shown that T f(x 0 + 1, x 1, x 2 ) T f(x 0, x 1, x 2 ) δ 1 for all β [β, 1) where β = max( αe 1 αe µ 1 +αe 1, 2 µ 2 +αe 2 ), therefore the optimal vale fnction lim n T n f = J β has the same property. 2. Next we show that Property 2 in the Lemma is also invariant nder T. After transformation, T f(x a ) and T f(x b ) can be written as: T f(x 0, 1, x 2 ) = x x 2 + αe 1 + αe 2 x 2 + βλ min f(p 0 A(x 0, 1, x 2 )) + βµ 1 min f(p 1 D 1 (x 0, 1, x 2 )) + βµ 2 min f(p 2 D 2 (x 0, 1, x 2 )) = x x 2 + αe 1 + αe 2 x 2 + βλ min f(p 0 (x 0 + 1, 1, x 2 )) (3.7) + βµ 1 min f(p 1 (x 0, 0, x 2 )) + βµ 2 min f(p 2 (x 0, 1, 0)), T f(x 0, 0, x 2 ) = x 0 + x 2 + αe 2 x 2 + βλ min f(p 0 A(x 0, 0, x 2 )) + βµ 1 min f(p 1 D 1 (x 0, 0, x 2 )) + βµ 2 min f(p 2 D 2 (x 0, 0, x 2 )) = x 0 + x 2 + αe 2 x 2 + βλ min f(p 0 (x 0 + 1, 0, x 2 )) (3.8) + βµ 1 min f(p 1 (x 0, 0, x 2 )) + βµ 2 min f(p 2 (x 0, 0, 0)). Taking the difference between Eqation (3.7) and Eqation (3.8), T f(x 0, 1, x 2 ) T f(x 0, 0, x 2 ) = 1 + αe 1 + βλ(min f(p 0 (x 0 + 1, 1, x 2 )) min f(p 0 (x 0 + 1, 0, x 2 ))) + βµ 1 (min f(p 1 (x 0, 0, x 2 )) min f(p 1 (x 0, 0, x 2 ))) + βµ 2 (min f(p 2 (x 0, 1, 0)) min f(p 2 (x 0, 0, 0))) 1 + αe 1 + βλδ 2 + βµ 2 δ 2 = 1 + αe 1 + β(1 µ 1 )δ 2 = δ 2. 16

24 Hence we have proven that T f(x 0, 1, x 2 ) T f(x 0, 0, x 2 ) δ 2 for any β, and therefore the optimal vale fnction lim n T n f = J β has the same property. 3. Using the same techniqe sed to prove Property 2, we can show that T f(x 0, x 1, 1) T f(x 0, x 1, 0) δ 3 for any β, and therefore the optimal vale fnction lim n T n f = J β has the same property. 1 We have shown that G is non-empty and T f G for any fnction f G, therefore the optimal vale fnction lim n T n f = J β G. 3.2 Keep the Efficient Server Bsy When Possible In this section we show that it is optimal to make a schedling decision which keeps the more efficient server bsy whenever possible, i.e., ensre that a job is assigned to server 1 whenever x Dom(P 1 ) or x Dom(P b ), in the setting where the more efficient server is also faster. Proposition 1. When µ 1 > µ 2, there exists β [0, 1) sch that for all β [β, 1), J β has the following properties: 1. J β (P 1 x) J β (P 2 x) when x = (1, 0, 0), 2. J β (P 1 x) J β (P h x) for x Dom(P 1 ), and 3. J β (P b x) J β (P 2 x) for x Dom(P b ). 1 See A.1 for the complete proof. 17

25 Proof. Consider an f G which has the above properties. Again sch fnctions exist, for example f 0 in Eqation (3.4) 2. First we show that Property 1 (i.e., it is better to send the job to the more efficient server rather than the less efficient one when there is only one job in the qee) is invariant nder T. The state nder consideration is x = (1, 0, 0); possible actions in this state are P h, P 1, and P 2. We can eliminate the P h option from consideration since P 1 wold be a better choice in comparison, de to Property 2. The possibilities that remain are P 1 and P 2, and we will show that T f(p 1 x) T f(p 2 x) in this state. Taking the difference between the terms corresponding to the left hand side of the ineqality and the right, we get T f(0, 1, 0) T f(0, 0, 1) = αe 1 αe 2 + βλ(min f(p 0 (1, 1, 0)) min f(p 0 (1, 0, 1))) + βµ 1 (min f(p 1 (0, 0, 0)) min f(p 1 (0, 0, 1))) + βµ 2 (min f(p 2 (0, 1, 0)) min f(p 2 (0, 0, 0))) αe 1 αe 2 + βλ(f(0, 1, 1) f(0, 1, 1)) + βµ 1 (f(0, 0, 0) f(0, 0, 1)) + βµ 2 (f(0, 1, 0) f(0, 0, 0)) (3.9) = αe 1 αe 2 + β(µ 1 µ 2 )f(0, 0, 0) + βµ 2 f(0, 1, 0) βµ 1 f(0, 0, 1) = αe 1 αe 2 + β(µ 1 µ 2 )(f(0, 0, 0) f(0, 1, 0)) αe 1 αe 2 + β(µ 1 µ 2 )( δ 2 ) 1 + αe 1 = αe 1 αe 2 β(µ 1 µ 2 ) 1 β(1 µ 1 ). We make se of the fact that f(0, 1, 0) f(0, 0, 1) to pper bond the βµ 1 f(0, 0, 1) term with βµ 1 f(0, 1, 0) in order to factor ot β(µ 1 µ 2 ) between steps 3 and 4. 2 For proof see Appendix A.3. 18

26 In order to show that T f(0, 1, 0) T f(0, 0, 1), it is sfficient to show the pper bond 1 + αe 1 αe 1 αe 2 β(µ 1 µ 2 ) 1 β(1 µ 1 ) 0, which after algebraic maniplation yields the condition β(αe 1 αe 2 + µ 1 µ 2 + µ 1 αe 2 µ 2 αe 1 ) αe 1 αe 2. For convenience, we rewrite this in the form of β(a + B) A (3.10) where A = αe 1 αe 2 and B = µ 1 µ 2 + µ 1 αe 2 µ 2 αe 1. Note that B is always positive since we can derive µ 1 µ 2 + µ 1 αe 2 µ 2 αe 1 > 0 from Eqation (2.2). Several cases arise depending on the parameters of the system: If E 1 > E 2, both sides of Eqation (3.10) are positive, and we can rearrange it as β A A + B. Since A < A + B in this case, the right hand side is a vale [0, 1) and Eqation (3.10) holds for all β [ A A+B, 1). If E 1 E 2 and β(a + B) 0, Eqation (3.10) holds for all β [0, 1). On the other hand if β(a + B) < 0, rearranging Eqation (3.10) yields the condition In this case we can define A A+B β A A + B. > 1, so again the condition holds for all β [0, 1). Combining all cases, αe 1 αe 2 β αe = 1 αe 2 +µ 1 µ 2 +µ 1 αe 2 µ 2 αe 1, if E 1 > E 2 0, otherwise, 19

27 which gives s T f(0, 1, 0) T f(0, 0, 1), and by extension J β (0, 1, 0) J β (0, 0, 1) 0, for all β [β, 1), as reqired. Next we show that Property 2, i.e., sending a job to the more efficient server is better than holding it in the qee, is also invariant nder T. The setting nder consideration is x Dom(P 1 ), i.e., x = (x 0, 0, x 2 ), where x 0 1. After applying the transformation to f(p h x) and f(p 1 x), we get T f(p 1 x) = x 0 + x 2 + αe 1 + αe 2 x 2 + βλ min f(p 0 AP 1 x) + βµ 1 min f(p 1 D 1 P 1 x) + βµ 2 min f(p 2 D 2 P 1 x), T f(p h x) = x 0 + x 2 + αe 2 x 2 + βλ min f(p 0 AP h x) + βµ 1 min Taking the difference between them, we get f(p 1 D 1 P h x) + βµ 2 min f(p 2 D 2 P h x). T f(p 1 x) T f(p h x) = αe 1 + βλ(min f(p 0 AP 1 x) min f(p 0 AP h x)) (3.11) For the λ term, we have βλ(min + βµ 1 (min f(p 1 D 1 P 1 x) min f(p 1 D 1 P h x)) (3.12) + βµ 2 (min f(p 2 D 2 P 1 x) min f(p 2 D 2 P h x)). (3.13) f(p 0 AP 1 x) min f(p 0 AP h x)) = βλ(min f(p 0 (x 0, 1, x 2 )) min f(p 0 (x 0 + 1, 0, x 2 ))). (3.14) When x 2 = 0, the right hand side of Eqation (3.14) becomes βλ(min f(p 0 (x 0, 1, 0)) min f(p 0 (x 0 + 1, 0, 0))). While any P {P h, P 1, P 2, P b } wold be a valid action for the sbtrahend, the action which incrs the minimm cost can only be either P 1 or P b ; P h and P 2 are eliminated from 20

28 consideration since f(p 1 x) f(p h x), and f(p b x) f(p 2 x) by Property 3. In the former case, the sbtrahend evalates to f(x 0, 1, 0), and we have min f(p 0 (x 0, 1, 0)) f(x 0, 1, 0); in the latter case the sbtrahend evalates to f(x 0 1, 1, 1), and we have min f(p 0 (x 0, 1, 0)) f(x 0 1, 1, 1). In either case, the minend evalates to a qantity no larger than the vale of the sbtrahend, and ths the right hand side of Eqation (3.14) is non-positive. When x 2 = 1, Eqation (3.14) can be written as βλ(f(x 0, 1, 1) min f(p 0 (x 0 + 1, 0, 1))). The state in the sbtrahend, (x 0 + 1, 0, 1), is in Dom(P 1 ) and Dom(P h ); however we know that f(p 1 x) f(p h x), ths the action which incrs the minimm cost in this state mst be P 1. The sbtrahend therefore evalates to f(x 0, 1, 1), reslting in a difference of 0 in Eqation (3.14). In every case, and the entire λ term 0. For the µ 2 term, we have βµ 2 (min min f(p 0 (x 0, 1, x 2 )) min f(p 0 (x 0 + 1, 0, x 2 )), f(p 2 D 2 P 1 x) min f(p 2 D 2 P h x)) = βµ 2 (min f(p 2 (x 0 1, 1, 0)) min f(p 2 (x 0, 0, 0))). (3.15) 21

29 When x 0 = 1, this can be rewritten as βµ 2 (f(0, 1, 0) min f(p 2 (1, 0, 0))). Since we have f(0, 1, 0) f(0, 0, 1) and f(p 1 x) f(p h x), the action which incrs the minimm cost in the sbtrahend mst be P 1 ; the sbtrahend therefore evalates to f(0, 1, 0) as well, leaving a difference of 0. When x 0 2, we can define x 0 = x 0 1 and rewrite the above as βµ 2 (min f(p 2 ( x 0, 1, 0)) min f(p 2 ( x 0 + 1, 0, 0))), to which the same argments sed for the λ term can be applied to show that the entire µ 2 term 0. Lastly for the µ 1 term, we have βµ 1 (min f(p 1 D 1 P 1 x) min f(p 1 D 1 P h x)) = βµ 1 (min f(p 1 (x 0 1, 0, x 2 )) min f(p 1 (x 0, 0, x 2 ))), and from the proof of Lemma 1 we have min f(p 1 (x 0 + 1, x 1, x 2 )) min f(p 1 (x 0, x 1, x 2 )) δ 1. This prodces an pper bond of βµ 1 ( δ 1 ) for the µ 1 term. Combining the reslts of the λ, µ 1, and µ 2 terms, we can define the lower bond T f(p h x) T f(p 1 x) αe 1 + βµ 1 δ 1 for Eqation (3.11). Recall from Lemma 1 that δ 1 = min( 1 1 β, δ 2, δ 3 ). To facilitate a proof for the lower bond αe 1 + βµ 1 δ 1 0, we wold like to choose β sch that δ 1 = δ 2, i.e., δ β and δ 2 δ 3. After algebraic maniplation, the condition for δ β is β αe 1 αe 1 + µ 1, 22

30 and the condition for δ 2 δ 3 is identical to Eqation (3.10), ths the previos analytical reslts on the vales of β which satisfy this ineqality can be applied here as well. Combining the bonds on β for both conditions, we can define ( ) αe max 1 αe β αe = 1 +µ 1, 1 αe 2 αe 1 αe 2 +µ 1 µ 2 +µ 1 αe 2 µ 2 αe 1, if E 1 > E 2 otherwise, αe 1 αe 1 +µ 1, which gives s δ 1 = δ 2 for β [β, 1). The lower bond on T f(p h x) T f(p 1 x) becomes T f(p 1 x) T f(p h x) αe 1 β µ 1 δ 2 = αe 1 β µ αe 1 1 β (1 µ 1 ) = αe 1(1 β + β µ 1 ) β µ 1 (1 + αe 1 ) 1 β (1 µ 1 ) = αe 1 β (αe 1 + µ 1 ) 1 β (1 µ 1 ) αe 1 αe 1 αe 1 +µ 1 (αe 1 + µ 1 ) 1 β (1 µ 1 ) 0, which implies J β (P 1 x) J β (P h x) 0 for β [β, 1) as reqired. Finally we will show that Property 3, i.e., it is better to assign jobs to both servers rather than only the less efficient server when there are at least two jobs in the qee, is invariant nder T as well. For this property the setting is x Dom(P b ), i.e. x = (x 0, 0, 0), x 0 2. We begin by defining x = (x 0 1, 0, 1). The vale fnctions nder comparison are f(x 0 2, 1, 1) and f(x 0 1, 0, 1); after applying the transformation, we can rewrite the states sing x as T f(p b x) = T f(x 0 2, 1, 1) = T f(p 1 (x 0 1, 0, 1)) = T f(p 1 x ) 23

31 and T f(p 2 x) = T f(x 0 1, 0, 1) = T f(p h (x 0 1, 0, 1)) = T f(p h x ). We have shown above that T f(p 1 x) T f(p h x) for any x Dom(P 1 ) and β β. Noting that x Dom(P b ) = x Dom(P 1 ), we can write T f(p 1 x ) T f(p h x ) or eqivalently T f(p b x) T f(p 2 x), which implies that J β (P b x) J β (P 2 x) for β [β, 1) as reqired. We have shown that when there is only one job in the qee, it is always better to assign it to the more efficient server when possible, as J β (0, 1, 0) J β (0, 0, 1) and J β (P 1 (x)) J β (P h (x)). In the general case when there is more than one job in the qee, we have shown that J β (P b (x)) J β (P 2 (x)) in addition to J β (P 1 (x)) J β (P h (x)). A proof for J β (P 1 (x)) J β (P 2 (x)) for the general case is not necessary, as the ineqality is not reqired for sbseqent proofs, and P 1 and P b are the only viable actions both of which keep the more efficient server active whenever possible. 3.3 Send Jobs to the Less Efficient Server When Qee Length Exceeds Threshold To prove that the optimal policy has the second characteristic of a threshold type policy, i.e., jobs shold be schedle for the less efficient server once the qee length exceeds some threshold, it is sfficient to show [2] that when x Dom(P 2 ), the cost benefit of withholding a job from the less efficient server over sending it is monotonically decreasing with qee length. In other words, when x 0 = 0, J β (0, 0, 1) J β (0, 1, 0) J β (0, 1, 1) J β (1, 1, 0), (3.16) 24

32 and when x 0 1, J β (x 0 1, 1, 1) J β (x 0, 1, 0) J β (x 0, 1, 1) J β (x 0 + 1, 1, 0). (3.17) When there is only one job in the qee prior to the control decision and server 1 is idle, it is optimal to assign the job to server 1 by Proposition 1. When server 1 is occpied, the cost benefit of keeping server 2 idle is J β (x 0 1, 1, 1) J β (x 0, 1, 0); if this benefit is positive, it is better not to assign jobs to server 2. With increasing x 0 the cost benefit monotonically decreases, and if at x 0 = x 0 a there is no longer a positive cost benefit in keeping server 2 idle, i.e., J β (x 0 a 1, 1, 1) J β (x 0 a, 1, 0) 0, then the threshold has been exceeded and it becomes optimal to assign jobs to both servers for all x 0 x 0 a. We now show that J β has the properties pt forth in Proposition 2, nmber 1 and 2 of which are eqivalent to Eqations (3.17) and (3.16) respectively. The following proofs follow along the same lines as Koole s proof in [2], with necessary modifications to accont for the costs of sing each server. Proposition 2. There exists β [0, 1) sch that for all β [β, 1), the optimal vale fnction J β has the following properties: 1. J β (x 0, 1, 0) + J β (x 0, 1, 1) J β (x 0 + 1, 1, 0) + J β (x 0 1, 1, 1) for x J β (0, 1, 0) + J β (0, 1, 1) J β (1, 1, 0) + J β (0, 0, 1) 3. J β (x 0, 1, 0) + J β (x 0 1, 1, 1) J β (x 0 1, 1, 0) + J β (x 0, 1, 1) for x J β (0, 1, 0) + J β (0, 0, 1) J β (0, 0, 0) + J β (0, 1, 1). Proof. We take the set of fnctions G as defined in Section 3 and once more constrain it to fnctions with the above properties, noting that G is non-empty 3. 3 For proof see Appendix A.4. 25

33 Now, consider an f G. For convenience, we first derive three additional properties of f. Smming properties 1 and 3 yields 5. 2f(x 0, 1, 0) f(x 0 + 1, 1, 0) + f(x 0 1, 1, 0), smming properties 1 and 3 with x 0 replaced by x in 3 yields 6. 2f(x 0, 1, 1) f(x 0 + 1, 1, 1) + f(x 0 1, 1, 1), and smming properties 2 and 4 yields 7. 2f(0, 1, 0) f(1, 1, 0) + f(0, 0, 0). We begin by showing that min f(p (x)) G. As an aside, note that in the derivations in this chapter we make se of the properties from Proposition 1 that f(0, 1, 0) f(0, 0, 1) and f(x 0, 1, x 2 ) f(x 0 + 1, 0, x 2 ) to replace terms where the minimm cost action is taken from a state where x Dom(P 1 ), with the eqivalent term where the minimm cost action is taken after first assigning a job from the qee to server 1. In other words, and when x 0 1. min f(p (1, 0, 0)) = f(0, 1, 0), min f(p (x 0, 0, x 2 )) = min f(p (x 0 1, 1, x 2 )) 1. If min f(p (x 0 + 1, 1, 0)) = f(x 0 + 1, 1, 0), we have min f(p (x 0, 1, 0)) + min f(p (x 0, 1, 1)) f(x 0, 1, 0) + f(x 0, 1, 1) f(x 0 + 1, 1, 0) + f(x 0 1, 1, 1) = min f(p (x 0 + 1, 1, 0)) + min f(p (x 0 1, 1, 1)). 26

34 On the other hand if min f(p (x 0 + 1, 1, 0)) = f(x 0, 1, 1), we have In both cases, min f(p (x 0, 1, 0)) + min f(p (x 0, 1, 1)) f(x 0 1, 1, 1) + f(x 0, 1, 1) = min f(p (x 0 + 1, 1, 0)) + min f(p (x 0 1, 1, 1)). min f(p (x 0, 1, 0)) + min f(p (x 0, 1, 1)) 2. If min f(p (1, 1, 0)) = f(1, 1, 0), we have min f(p (x 0 + 1, 1, 0)) + min f(p (x 0 1, 1, 1)). min f(p (0, 1, 0)) + min f(p (0, 1, 1)) = f(0, 1, 0) + f(0, 1, 1) and if min f(p (1, 1, 0)) = f(0, 1, 1), we have f(1, 1, 0) + f(0, 0, 1) = min f(p (1, 1, 0)) + min f(p (0, 0, 1)) min f(p (0, 1, 0)) + min f(p (0, 1, 1)) = f(0, 1, 0) + f(0, 1, 1) f(0, 1, 1) + f(0, 0, 1) = min f(p (1, 1, 0)) + min f(p (0, 0, 1)), making se of f(0, 1, 0) f(0, 0, 1) to pper bond the right hand side in step 1. In both cases, min f(p (0, 1, 0)) + min f(p (0, 1, 1)) min f(p (1, 1, 0)) + min f(p (0, 0, 1)). 27

35 3. If min f(p (x 0 1, 1, 0)) = f(x 0 1, 1, 0), we have min f(p (x 0, 1, 0)) + min f(p (x 0 1, 1, 1)) f(x 0, 1, 0) + f(x 0 1, 1, 1) f(x 0 1, 1, 0) + f(x 0, 1, 1) = min f(p (x 0 1, 1, 0)) + min f(p (x 0, 1, 1)) and if min f(p (x 0 1, 1, 0)) = f(x 0 2, 1, 1), we have In both cases, min f(p (x 0, 1, 0)) + min f(p (x 0 1, 1, 1)) f(x 0 1, 1, 1) + f(x 0 1, 1, 1) f(x 0 2, 1, 1) + f(x 0, 1, 1) = min f(p (x 0 1, 1, 0)) + min f(p (x 0, 1, 1)). min f(p (x 0, 1, 0)) + min f(p (x 0 1, 1, 1)) min f(p (x 0 1, 1, 0)) + min f(p (x 0, 1, 1)). 4. This property is straightforward to show, since no actions are possible for any of the states nder consideration: min f(p (0, 1, 0)) + min f(p (0, 0, 1)) = f(0, 1, 0) + f(0, 0, 1) f(0, 0, 0) + f(0, 1, 1) Finally we proceed with vale iteration and show that T f G. 28 = min f(p (0, 0, 0)) + min f(p (0, 1, 1)).

36 1. We start by showing that the Property 1 in the Proposition is invariant nder T. T f(x 0, 1, 0) = x αe 1 + βλ min f(p 0 (x 0 + 1, 1, 0)) + βµ 1 min f(p 1 (x 0, 0, 0)) + βµ 2 min f(p 2 (x 0, 1, 0)), T f(x 0, 1, 1) = x αe 1 + αe 2 + βλ min f(p 0 (x 0 + 1, 1, 1)) + βµ 1 min f(p 1 (x 0, 0, 1)) + βµ 2 min f(p 2 (x 0, 1, 0)), T f(x 0 + 1, 1, 0) = x αe 1 + βλ min f(p 0 (x 0 + 2, 1, 0)) + βµ 1 min f(p 1 (x 0 + 1, 0, 0)) + βµ 2 min f(p 2 (x 0 + 1, 1, 0)), T f(x 0 1, 1, 1) = x αe 1 + αe 2 + βλ min f(p 0 (x 0, 1, 1)) + βµ 1 min f(p 1 (x 0 1, 0, 1)) + βµ 2 min f(p 2 (x 0 1, 1, 0)). Taking the difference between the terms corresponding to the left hand side of the ineqality and the right, we get T f(x 0, 1, 0) + T f(x 0, 1, 1) T f(x 0 + 1, 1, 0) T f(x 0 1, 1, 1) = βλ(min f(p 0 (x 0 + 1, 1, 0)) + min f(p 0 (x 0 + 1, 1, 1)) min f(p 0 (x 0 + 2, 1, 0)) min f(p 0 (x 0, 1, 1))) + βµ 1 (min f(p 1 (x 0, 0, 0)) + min f(p 1 (x 0, 0, 1)) min f(p 1 (x 0 + 1, 0, 0)) min f(p 1 (x 0 1, 0, 1))) + βµ 2 (min f(p 2 (x 0, 1, 0)) + min f(p 2 (x 0, 1, 0)) min f(p 2 (x 0 + 1, 1, 0)) min f(p 2 (x 0 1, 1, 0))). 29

37 The λ and µ 2 terms can be shown to be 0 sing Properties 1 and 5 respectively. For the expression within the µ 1 term, we have two cases depending on x 0 : if x 0 = 1, we have min f(p 1 (1, 0, 0)) + min f(p 1 (1, 0, 1)) min f(p 1 (2, 0, 0)) min f(p 1 (0, 0, 1)) = min f(p 1 (0, 1, 0)) + min f(p 1 (0, 1, 1)) min f(p 1 (1, 1, 0)) min f(p 1 (0, 0, 1)) 0 by Property 2. If x 0 2, we have min f(p 1 (x 0, 0, 0)) + min f(p 1 (x 0, 0, 1)) min f(p 1 (x 0 + 1, 0, 0)) min f(p 1 (x 0 1, 0, 1)) = min f(p 1 (x 0 1, 1, 0)) + min f(p 1 (x 0 1, 1, 1)) min f(p 1 (x 0, 1, 0)) min f(p 1 (x 0 2, 1, 1)) 0 by Property 1. In either case above, the right hand side 0 for all x 0 1 and β [β, 1) 4, therefore T f(x 0, 1, 0) + T f(x 0, 1, 1) T f(x 0 + 1, 1, 0) + T f(x 0 1, 1, 1) as reqired. 4 β being the vale of β above which properties from Lemma 1 and Proposition 1 hold. 30

38 2. Next we show that the Property 2 in the Proposition is invariant nder T. T f(0, 1, 0) = 1 + αe 1 + βλ min f(p 0 (1, 1, 0)) + βµ 1 min f(p 1 (0, 0, 0)) + βµ 2 min f(p 2 (0, 1, 0)) = 1 + αe 1 + βλ min f(p 0 (1, 1, 0)) + βµ 1 f(0, 0, 0) + βµ 2 f(0, 1, 0), T f(0, 1, 1) = 2 + αe 1 + αe 2 + βλ min f(p 0 (1, 1, 1)) + βµ 1 min f(p 1 (0, 0, 1)) + βµ 2 min f(p 2 (0, 1, 0)) = 2 + αe 1 + αe 2 + βλ min f(p 0 (1, 1, 1)) + βµ 1 f(0, 0, 1) + βµ 2 f(0, 1, 0), T f(1, 1, 0) = 2 + αe 1 + βλ min f(p 0 (2, 1, 0)) + βµ 1 min f(p 1 (1, 0, 0)) + βµ 2 min f(p 2 (1, 1, 0)) = 2 + αe 1 + βλ min f(p 0 (2, 1, 0)) + βµ 1 f(0, 1, 0) + βµ 2 min f(p 2 (1, 1, 0)), T f(0, 0, 1) = 1 + αe 2 + βλ min f(p 0 (1, 0, 1)) + βµ 1 min f(p 1 (0, 0, 1)) + βµ 2 min f(p 2 (0, 0, 0)) = 1 + αe 2 + βλ min f(p 0 (0, 1, 1)) + βµ 1 f(0, 0, 1) + βµ 2 f(0, 0, 0). 31

39 Taking the difference between the terms corresponding to the left hand side of the ineqality and the right, we get T f(0, 1, 0) + T f(0, 1, 1) T f(1, 1, 0) T f(0, 0, 1) = αe 1 + βλ(min f(p 0 (1, 1, 0)) + min f(p 0 (1, 1, 1)) min f(p 0 (2, 1, 0)) min f(p 0 (0, 1, 1))) + βµ 1 (f(0, 0, 0) + f(0, 0, 1) f(0, 1, 0) f(0, 0, 1)) + βµ 2 (f(0, 1, 0) + f(0, 1, 0) min f(p 2 (1, 1, 0)) f(0, 0, 0)) αe 1 βµ 1 δ 2 + βµ 2 (2f(0, 1, 0) min f(p 2 (1, 1, 0)) f(0, 0, 0)). For the µ 2 term, if min 2 f(p 2 (1, 1, 0)) = f(1, 1, 0), we have 2f(0, 1, 0)) f(1, 1, 0) + f(0, 0, 0) and if min 2 f(p 2 (1, 1, 0)) = f(0, 1, 1), we have In either case, This leaves s with the pper bond = min f(p 2 (1, 1, 0)) + f(0, 0, 0) 2f(0, 1, 0)) f(0, 1, 0) + f(0, 0, 1) f(0, 0, 0) + f(0, 1, 1) = min f(p 2 (1, 1, 0)) + f(0, 0, 0). 2f(0, 1, 0)) min f(p 2 (1, 1, 0)) + f(0, 0, 0). T f(0, 1, 0) + T f(0, 1, 1) T f(1, 1, 0) T f(0, 0, 1) αe 1 βµ 1 δ 2 = αe 1 βµ αe 1 1 β(1 µ 1 ) 32

40 For β αe 1 αe 1 +µ 1, This implies αe 1 βµ αe 1 1 β(1 µ 1 ) 0. T f(0, 1, 0) + T f(0, 1, 1) T f(1, 1, 0) + T f(0, 0, 1) for β [β, 1), where β is the maximm of 1 and Proposition 1 hold, as reqired. αe 1 αe 1 +µ 1 and the vale above which Lemma 3. Next we show that the Property 3 in the Proposition is invariant nder T. We have two cases depending on the vale of x 0. If x 0 2, T f(x 0, 1, 0) = x αe 1 + βλ min f(p 0 (x 0 + 1, 1, 0)) + βµ 1 min f(p 1 (x 0, 0, 0)) + βµ 2 min f(p 2 (x 0, 1, 0)) = x αe 1 + βλ min f(p 0 (x 0 + 1, 1, 0)) + βµ 1 min f(p 1 (x 0 1, 1, 0)) + βµ 2 min f(p 2 (x 0, 1, 0)), T f(x 0 1, 1, 1) = x αe 1 + αe 2 + βλ min f(p 0 (x 0, 1, 1)) + βµ 1 min f(p 1 (x 0 1, 0, 1)) + βµ 2 min f(p 2 (x 0 1, 1, 0)) = x αe 1 + αe 2 + βλ min f(p 0 (x 0, 1, 1)) + βµ 1 min f(p 1 (x 0 2, 1, 1)) + βµ 2 min f(p 2 (x 0 1, 1, 0)), T f(x 0 1, 1, 0) = x 0 + αe 1 + βλ min f(p 0 (x 0, 1, 0)) + βµ 1 min f(p 1 (x 0 1, 0, 0)) + βµ 2 min f(p 2 (x 0 1, 1, 0)) = x 0 + αe 1 + βλ min f(p 0 (x 0, 1, 0)) + βµ 1 min f(p 1 (x 0 2, 1, 0)) + βµ 2 min f(p 2 (x 0 1, 1, 0)), 33

41 T f(x 0, 1, 1) = x αe 1 + αe 2 + βλ min f(p 0 (x 0 + 1, 1, 1)) + βµ 1 min f(p 1 (x 0, 0, 1)) + βµ 2 min f(p 2 (x 0, 1, 0)) = x αe 1 + αe 2 + βλ min f(p 0 (x 0 + 1, 1, 1)) + βµ 1 min f(p 1 (x 0 1, 1, 1)) + βµ 2 min f(p 2 (x 0, 1, 0)). Taking the difference between the terms corresponding to the left hand side of the ineqality and the right, we get T f(x 0, 1, 0) + T f(x 0 1, 1, 1) T f(x 0 1, 1, 0) T f(x 0, 1, 1) = βλ(min f(p 0 (x 0 + 1, 1, 0)) + min f(p 0 (x 0, 1, 1)) min f(p 0 (x 0, 1, 0)) min f(p 0 (x 0 + 1, 1, 1))) + βµ 1 (min f(p 1 (x 0 1, 1, 0)) + min f(p 1 (x 0 2, 1, 1)) min f(p 1 (x 0 2, 1, 0)) min f(p 1 (x 0 1, 1, 1))) + βµ 2 (min f(p 2 (x 0, 1, 0)) + min f(p 2 (x 0 1, 1, 0)) min f(p 2 (x 0 1, 1, 0)) min f(p 2 (x 0, 1, 0))). The right hand side 0 since the expressions within each of the terms 0. However a special case emerges when x 0 = 1 de to the (x 0 2, 1, x 2 ) states, and the transformations mst be re-evalated as follows: T f(1, 1, 0) = 2 + αe 1 + βλ min f(p 0 (2, 1, 0)) + βµ 1 min f(p 1 (1, 0, 0)) + βµ 2 min f(p 2 (1, 1, 0)) = 2 + αe 1 + βλ min f(p 0 (2, 1, 0)) + βµ 1 min f(p 1 (0, 1, 0)) + βµ 2 min f(p 2 (1, 1, 0)), 34

42 T f(0, 1, 1) = 2 + αe 1 + αe 2 + βλ min f(p 0 (1, 1, 1)) + βµ 1 min f(p 1 (0, 0, 1)) + βµ 2 min f(p 2 (0, 1, 0)), T f(0, 1, 0) = 1 + αe 1 + βλ min f(p 0 (1, 1, 0)) + βµ 1 min f(p 1 (0, 0, 0)) + βµ 2 min f(p 2 (0, 1, 0)), T f(1, 1, 1) = 3 + αe 1 + αe 2 + βλ min f(p 0 (2, 1, 1)) + βµ 1 min f(p 1 (1, 0, 1)) + βµ 2 min f(p 2 (1, 1, 0)) = 3 + αe 1 + αe 2 + βλ min f(p 0 (2, 1, 1)) + βµ 1 min f(p 1 (0, 1, 1)) + βµ 2 min f(p 2 (1, 1, 0)). Taking the difference between the terms corresponding to the left hand side of the ineqality and the right, we get T f(1, 1, 0) + T f(0, 1, 1) T f(0, 1, 0) T f(1, 1, 1) = βλ(min f(p 0 (2, 1, 0)) + min f(p 0 (1, 1, 1)) min f(p 0 (1, 1, 0)) min f(p 0 (2, 1, 1))) + βµ 1 (min f(p 1 (0, 1, 0)) + min f(p 1 (0, 0, 1)) min f(p 1 (0, 0, 0)) min f(p 1 (0, 1, 1))) + βµ 2 (min f(p 2 (1, 1, 0)) + min f(p 2 (0, 1, 0)) min f(p 2 (0, 1, 0)) min f(p 2 (1, 1, 0))) 35

43 βλ(min f(p 0 (2, 1, 0)) + min f(p 0 (1, 1, 1)) min f(p 0 (1, 1, 0)) min f(p 0 (2, 1, 1))) + βµ 1 (f(0, 1, 0) + f(0, 0, 1) f(0, 0, 0) f(0, 1, 1)) + βµ 2 (min f(p 2 (1, 1, 0)) + min f(p 2 (0, 1, 0)) min f(p 2 (0, 1, 0)) min f(p 2 (1, 1, 0))). Again all the terms 0, therefore T f(x 0, 1, 0) + T f(x 0 1, 1, 1) T f(x 0 1, 1, 0) + T f(x 0, 1, 1) holds for any x 0 1 as reqired. 4. Finally we show that Property 4 in the Proposition is invariant nder T. T f(0, 1, 0) = 1 + αe 1 + βλ min f(p 0 (1, 1, 0)) + βµ 1 min f(p 1 (0, 0, 0)) + βµ 2 min f(p 2 (0, 1, 0)), T f(0, 0, 1) = 1 + αe 2 + βλ min f(p 0 (1, 0, 1)) + βµ 1 min f(p 1 (0, 0, 1)) + βµ 2 min f(p 2 (0, 0, 0)) = 1 + αe 2 + βλ min f(p 0 (0, 1, 1)) + βµ 1 min f(p 1 (0, 0, 1)) + βµ 2 min f(p 2 (0, 0, 0)), 36

44 T f(0, 0, 0) = βλ min f(p 0 (1, 0, 0)) + βµ 1 min f(p 1 (0, 0, 0)) + βµ 2 min f(p 2 (0, 0, 0)) = βλ min f(p 0 (0, 1, 0)) + βµ 1 min f(p 1 (0, 0, 0)) + βµ 2 min f(p 2 (0, 0, 0)), T f(0, 1, 1) = 2 + αe 1 + αe 2 + βλ min f(p 0 (1, 1, 1)) + βµ 1 min f(p 1 (0, 0, 1)) + βµ 2 min f(p 2 (0, 1, 0)). Taking the difference between the terms corresponding to the left hand side of the ineqality and the right, we get T f(0, 1, 0) + T f(0, 0, 1) T f(0, 0, 0) T f(0, 1, 1) = βλ(min f(p 0 (1, 1, 0)) + min f(p 0 (0, 1, 1)) min f(p 0 (0, 1, 0)) min f(p 0 (1, 1, 1))). The expression inside the λ term 0 by Property 3, and so the right hand side of the ineqality 0 for β [β, 1) 5. Therefore as reqired. T f(0, 1, 0) + T f(0, 0, 1) T f(0, 0, 0) + T f(0, 1, 1) We have shown that G is non-empty and that T f G for any fnction f G. This implies that the optimal vale fnction lim n T n f = J β G and therefore satisfies the 5 See footnote 4. 37

45 properties in Proposition 2, inclding Property 1 and 2 which are eqivalent to Eqation (3.17) and (3.16) respectively. This shows that jobs shold be sent to the less efficient server once the qee length exceeds some threshold, and this reslt in conjnction with the conclsion from Section 3.2 that the more efficient server shold be kept bsy whenever possible, we can conclde that the optimal policy is of threshold type in the setting where µ 1 > µ 2. This reslt generalizes the findings of Koole [2], i.e., the optimal policy is of threshold type when the goal is to minimize only the response time of a job throgh the system. 38

46 Chapter 4 Extension to µ 1 < µ 2 with Small λ In this chapter we extend or reslts from Chapter 3 to the setting where µ 1 < µ 2, i.e., the more efficient server is slower, with the additional restriction of small λ. In Section 4.1 we will show algebraically that J β (0, 1, 0) J β (0, 0, 1) in this setting. In Section 4.2 we discss the work reqired to extend the reslts from Chapter 3 to this setting. We were nable to obtain an analytical proof nder large λ, or a vale iteration proof of the ineqality. 4.1 J β (0, 1, 0) J β (0, 0, 1) In this section we will show that when there is only one job in the qee, it is better to send it to server 1 rather than 2. Lemma 2. When µ 1 < µ 2, for sfficiently small λ and fixed β [0, 1), J β (P 1 x) J β (P 2 x) when x = (1, 0, 0). 39

47 Proof. Rather than sing vale iteration, we will algebraically show that Lemma 2 is a property of the optimal cost fnction J β in this setting. We begin by noting that J β (x) is increasing with λ for all x X 1. We take the difference between the Bellman eqations for each resltant state: J β (0, 1, 0) J β (0, 0, 1) = αe 1 αe 2 + βλ(min J β (P 0 (1, 1, 0)) min J β (P 0 (1, 0, 1))) + βµ 1 (J β (0, 0, 0) J β (0, 0, 1)) + βµ 2 (J β (0, 1, 0) J β (0, 0, 0)) = αe 1 αe 2 + βλ(min J β (P 0 (1, 1, 0)) min J β (P 0 (1, 0, 1))) + β(µ 1 µ 2 )(J β (0, 0, 0) J β (0, 0, 1)) + βµ 2 (J β (0, 1, 0) J β (0, 0, 1)). Rearranging, we get J β (0, 1, 0) J β (0, 0, 1) ( 1 = αe 1 αe 2 + βλ(min J β (P 0 (1, 1, 0)) min J β (P 0 (1, 0, 1))) 1 βµ 2 ) + β(µ 1 µ 2 )(J β (0, 0, 0) J β (0, 0, 1)). (4.1) Note that the difference J β (0, 0, 0) J β (0, 0, 1) in the β(µ 1 µ 2 ) term can also be written as the difference between Bellman eqations as follows: J β (0, 0, 0) J β (0, 0, 1) = αe 2 + βλ(min J β (P 0 (1, 0, 0)) min J β (P 0 (1, 0, 1))) 1 For proof see Appendix A.2. + βµ 1 (J β (0, 0, 0) J β (0, 0, 1)). 40

48 Rearranging, we get J β (0, 0, 0) J β (0, 0, 1) = αe 2 + βλ(min 0 J β (P 0 (1, 0, 0)) min 0 J β (P 0 (1, 0, 1))) 1 βµ 1. (4.2) Sbstitting Eqation (4.2) into (4.1) yields J β (0, 1, 0) J β (0, 0, 1) ( 1 = (1 βµ 1 )(αe 1 αe 2 ) (1 βµ 1 )(1 βµ 2 ) = + (1 βµ 1 )βλ(min ( + β(µ 1 µ 2 ) 1 (1 βµ 1 )(1 βµ 2 ) ( + βλ J β (P 0 (1, 1, 0)) min J β (P 0 (1, 0, 1))) αe 2 + βλ(min J β (P 0 (1, 0, 0)) min ( αe 1 (1 βµ 1 ) αe 2 (1 βµ 2 ) min J β (P 0 (1, 1, 0)) min J β (P 0 (1, 0, 1)) J β (P 0 (1, 0, 1))) ) ) + βµ 1 (min J β (P 0 (1, 0, 0)) min J β (P 0 (1, 1, 0))) βµ 2 (min J β (P 0 (1, 0, 0)) min J β (P 0 (1, 0, 1))) Since we have αe 1 (1 βµ 1 ) αe 2 (1 βµ 2 ) 0 β [0, 1) and the βλ terms 0 as λ 0 since J β (x) is increasing in λ, for sfficiently small λ and fixed β, J β (0, 1, 0) J β (0, 0, 1). ) ). 41

49 4.2 Ftre Work to Show Optimality of Threshold Policy When µ 1 < µ 2 In this section we discss the additional reslt that wold have to be shown in order to prove optimality of a threshold policy for the setting where µ 1 < µ 2. The vale iteration proof for Property 1 of Proposition 1 reqires µ 1 > µ 2, since we cannot pper bond T f(0, 1, 0) T f(0, 0, 1) with αe 1 αe 2 + β(µ 1 µ 2 )( δ 2 ) as in Eqation 3.9 if β(µ 1 µ 2 ) is a negative coefficient. In its place we have Lemma 2, which shows the eqivalent reslt when µ 1 < µ 2 and λ is small. The rest of the proofs for Propositions 1 and 2 will hold nder µ 1 < µ 2 if one can show via vale iteration that J β (0, 1, 0) J β (0, 0, 1); the crrent proof of this in Lemma 2 ses a direct algebraic approach rather than vale iteration. 42

50 Chapter 5 Simlation Reslts 5.1 Simlation Setp In order to validate or system model as well as to evalate the practical applications of a power-aware threshold policy, we created a discrete event simlator in C++ 1 to realize or discrete state model from Chapter 2. The simlator consists of a system of two heterogeneos servers as described in Chapter 1, with system parameters µ 1 = 10, µ 2 = 100, E 1 = 5, E 2 = 100, ᾱ = 100, chosen based on response time and power sage measrements taken from real compters while satisfying the reqirement that server 1 more energy efficient than server 2, and ᾱ is chosen sch that the ineqality 1+αE 1 µ 1 1+αE 2 µ 2, where α = λᾱ, holds for all λ [1, µ 1 +µ 2 ). We also chose a setting where µ 1 < µ 2 in order to validate or reslts from Chapter 4. We implemented two schedling policies for or system: the naive first-come-first-serve 1 See Appendix B for fll sorce code listing. 43

51 policy, which assigns a job from the qee whenever a server is idle, beginning with server 1; and the threshold policy, which assigns a job to server 1 if possible, and server 2 only if the qee length has exceeded the threshold parameter m. For each policy, we sweep across a range of arrival rates from 1 p to bt not inclding µ 1 + µ 2, and serve a total of 10 million jobs at each arrival rate. In order to allow the system to stabilize, we first wait for 30% of the jobs to be completed before collecting any data. Workload is simlated by schedling exponentially distribted arrival events at rate λ; whenever a job arrives, the schedler pts it in a common qee, then makes a schedling decision depending on the policy in effect. Once a job starts being served, the server is set to a bsy state and the job s departre is schedled exponentially sing the server s service rate µ. When a departre occrs, the server s state is set to idle, data on the job s service and response times are recorded if the system is stable, and the schedler again checks if either server shold begin service on a job. The main performance metrics we collected dring simlation inclde the mean response time of a job throgh the system T and mean power consmption in the system Ē. These metrics are sed in calclating the cost J of each policy sing or cost fnction J = λ T + λᾱē, (5.1) where T is the mean response time and ᾱē is the weighted power sage. We ran simlation experiments sing each schedling policy from n-nified rates λ = 1 throgh µ 1 + µ 2 1; for the threshold policy, we adjst the threshold parameter m from 0 to 10 at each rate, completing a fll simlation rn of 10 million jobs for each parameter vale. For each arrival rate and parameter vale we record the mean response time and power sage, and compte the cost J with those parameters sing the cost fnction above. 44