c 2014 Siva Theja Maguluri

Transcription

1 c 2014 Siva Theja Maguluri

2 OPTIMA RESOURCE AOCATION AGORITHMS FOR COUD COMPUTING BY SIVA THEJA MAGUURI DISSERTATION Subitted in partial fulfillent of the requireents for the degree of Doctor of Philosophy in Electrical and Coputer Engineering in the Graduate College of the University of Illinois at Urbana-Chapaign, 2014 Urbana, Illinois Doctoral Coittee: Professor R. Srikant, Chair Professor Bruce Hajek Professor Praod Viswanath Assistant Professor Yi u Associate Professor ei Ying, Arizona State University

3 ABSTRACT Cloud coputing is eerging as an iportant platfor for business, personal and obile coputing applications. We consider a stochastic odel of a cloud coputing cluster, where jobs arrive according to a rando process and request virtual achines VMs), which are specified in ters of resources such as CPU, eory and storage space. The jobs are first routed to one of the servers when they arrive and are queued at the servers. Each server then chooses a set of jobs fro its queues so that it has enough resources to serve all of the siultaneously. There are any design issues associated with such systes. One iportant issue is the resource allocation proble, i.e., the design of algoriths for load balancing aong servers, and algoriths for scheduling VM configurations. Given our odel of a cloud, we define its capacity, i.e., the axiu rates at which jobs can be processed in such a syste. An algorith is said to be throughput-optial if it can stabilize the syste whenever the load is within the capacity region. We show that the widely-used Best-Fit scheduling algorith is not throughput-optial. We first consider the proble where the jobs need to be scheduled nonpreeptively on servers. Under the assuptions that the job sizes are known and bounded, we present algoriths that achieve any arbitrary fraction of the capacity region of the cloud. We then relax these assuptions and present a load balancing and scheduling algorith that is throughput optial when job sizes are unknown. In this case, job sizes durations) are odeled as rando variables with possibly unbounded support. Delay is a ore iportant etric then throughput optiality in practice. However, analysis of delay of resource allocation algoriths is difficult, so we study the syste in the asyptotic liit as the load approaches the boundary of the capacity region. This liit is called the heavy traffic regie. Assuing that the jobs can be preepted once after several tie slots, we present delay ii

4 optial resource allocation algoriths in the heavy traffic regie. We study delay perforance of our algoriths through siulations. iii

5 To Bhagavan Sri Raakrishna Oh ord! You are y other, father, relative and friend. You are y knowledge and y wealth. You are y all in all. iv

6 What is that which, being known, everything else is known? - Mundaka Upanishad v

7 ACKNOWEDGMENTS When I look back at y past self at the tie when I arrived here at UIUC as a fresh college graduate, I see how uch y experience at UIUC has helped in y growth and how uch I have learned here. I have been very fortunate to have had an opportunity to work in CS. The one person who played the ost iportant role in y experience here is obviously y advisor, Prof. Srikant. He is not only an aazing teacher but also a great entor. I thank hi for his constant guidance and support all through y grad school. He taught e the basics of how research is done, how it is counicated and above all how to think about research probles. He has always been there to support e. My heart-felt thanks for his support, care, and concern for y well-being. I thank Prof. ei Ying fro Arizona State University, who has been y coauthor for uch of the work that went into this dissertation. His suggestions and insights have been very iportant in this work. I thank Prof Bruce Hajek who was co-advisor during y aster s. I also thank Prof Bruce Hajek, Prof. ei Ying, Prof. Yi u and Prof. Praod Viswanath for serving on y coittee. I thank the anonyous reviewers who have taken tie to review y papers and have given their valuable feedback to iprove the results that went into this dissertation. Thanks are also due to Weina Wang for her iportant coents on one of the results. I thank UIUC for providing e with one of the best possible learning environents. In addition to an excellent research environent in CS, the university has provided e an opportunity to learn through the various courses that I took fro ECE, CS and Maths. These courses had soe of the best teachers and I have thoroughly enjoyed taking the. I thank y group ebers Rui, Chong, Javad and postdocs Chandraani and Joohwan who have always been ready for discussions. Thanks are due to Sreera who has been a great partner in course projects and for the vi

8 discussions we had on a variety of topics. I acknowledge the advice given by Vineet, Sreera and Siva Kuar on research, career and life. Conversations with Sachin on research probles, life, acadeia and everything else have been a highlight of y life at CS. Adinistrative staff in ECE and CS have ade life a lot soother. Special thanks to Peggy Wells, who has been our best office anager. She radiates cheerfulness and positive attitude in CS. I thank Jaie Hutchinson for carefully editing y thesis. I spent a very fruitful seester as an intern at Qualco, Bridgewater. I a very grateful to y entors Xinzhou Wu and Sundar Subraanian. They provided e with a great experience which was crucial in y decision to take up industry research as y career. Special thanks to Nilesh for his warth and friendship and ost iportantly for teaching e driving and thus epowering e. I thank y ala-ater IIT Madras, where I spent y forative years. My interactions with the professors there otivated e to pursue the PhD. At IITM, I ade lifelong friends, Vasi, Ranath, Aditya, Gaurav and Balu to nae a few. Our regular phone calls and annual reunions have been great fun and were very refreshing. I consider yself extreely fortunate to have known Swai Baneshananda, head of Vedanta-Gesellschaft, Frankfurt and Swai Atashraddhananda, editor of Vedanta Kesari. Swai Atashraddhananda has played a key role in defining y outlook towards life through his weekly lectures, personal conversations, regular pilgriages, and above all, the exaple set by his life. I always look forward to the annual visits of Swai Baneshananda to the US, every oent of which are filled with great fun and laughter. I can always go to the with any questions or dileas I face and can be assured of an answer. They have been y constant guiding force and I a grateful for their tie in spite of their extreely busy schedule. I feel blessed to have their unconditional love. The Vivekananda Vedanta Society of Chicago and the counity there has been y hoe away fro hoe. I thank Swai Chidananda, Swai Ishatananda and Swai Varadananda, who have always welcoed e at the society. Anjali, Pooja, Maya, Mithilesh, Jyoti Uncle, Manju Aunty, Manjusha Aunty, Rajini Aunty, Manish, Sachin and others have been like a faily to e. The town/city where I lived the longest so far has been vii

9 Chapaign-Urbana. If not for this faily, y life here, with the harsh winter, would have been very challenging. They have also provided e a sooth transition fro life in India to the US. Over the last year of y stay in Chabana, Vedanta Study Circle and y friends there have becoe the defining aspect of y life. I thank Chaitanya, Rasai, Srikanthan, Raghavendra, Suraj, Sasidhar and Kalyan for their copany, our weekly eetings, philosophical discussions, outings and relaxing dinners. In the last year of y grad school, I had to undergo a ajor surgery. It was one of the biggest challenges in y life and I could face it only with the support of so any of y friends and well-wishers. I express y heartfelt gratitude to Pooja, Chaitanya, Manju Aunty, Pushpa Aunty and Manish. Since y parents could not be around, y advisor, Prof Srikant assued that role. He drove in harsh winter weather to visit e and he constantly inquired about y well-being and progress. My sister, Hia Bindu, took a week off in the very first onth of her new job to attend to y needs. I should specially ention y friend Rasai in this context. He has been y constant copanion, caretaker and has attended to y every need. He has shown in practice, the philosophy of Kara Yoga Unselfish Action). I do not have words to express y gratitude to hi and so I quote Swai Vivekananda, the faous Indian onk. In happiness, in isery, in faine, in pain, in the grave, in heaven, or in hell who never gives e up is y friend. Is such friendship a joke? A an ay have salvation through such friendship. That brings salvation if we can love like that. I a fortunate to have found such a friend in Rasai. I thank y parents for their unconditional love, constant support and encourageent in all y endeavors. They have ade several sacrifices to give e the best in everything. Since y childhood, y other has otivated e to excel and has taken pride in y success. She has supported y decision to pursue the PhD in a far off country instead of a lucrative career in anageent closer to hoe, despite not copletely appreciating why. I thank y sister for her love and support. Most iportantly, I thank the ord who, out of His grace, has ore than provided for y every need. I thank hi for all the experiences he has put e through and for giving e the strength to go through the. He has given e the grad school experience which was a wonderful learning opportunity viii

10 not only acadeically and professionally, but also personally and spiritually. He put e in touch with great friends, teachers and well-wishers. I hubly dedicate this dissertation to Hi. ix

11 TABE OF CONTENTS 1 INTRODUCTION Best Fit Is Not Throughput Optial A Stochastic Model for Cloud Coputing THROUGHPUT OPTIMAITY: KNOWN JOB SIZES A Centralized Queueing Approach Resource Allocation with oad Balancing Sipler oad Balancing Algoriths Discussion and Siulations Conclusions UNKNOWN JOB SIZES Algorith Throughput Optiality - Geoetric Job Sizes Throughput Optiality - General Job Size Distribution ocal Refresh Ties Siulations Conclusion IMITED PREEMPTION Unknown Job Sizes Conclusion DEAY OPTIMAITY Algorith with iited Preeption - Known Job Sizes Throughput Optiality Heavy Traffic Optiality Power-of-Two-Choices Routing and MaxWeight Scheduling Conclusions CONCUSION Open Probles and Future Directions A PROOF OF EMMA B PROOF OF EMMA x

12 C PROOF OF CAIM REFERENCES xi

13 CHAPTER 1 INTRODUCTION Cloud coputing services are becoing the priary source of coputing power for both enterprises and personal coputing applications. A cloud coputing platfor can provide a variety of resources, including infrastructure, software, and services, to users in an on-deand fashion. To access these resources a cloud user subits a request for resources. The cloud provider then provides the requested resources fro a coon resource pool e.g., a cluster of servers) and allows the user to use these resources for a required tie period. Copared to traditional own-and-use approaches, cloud coputing services eliinate the costs of purchasing and aintaining the infrastructures for cloud users and allow the users to dynaically scale up and down coputing resources in real tie based on their needs. Several cloud coputing systes are now coercially available, including Aazon EC2 syste [1], Google s AppEngine [2], and Microsoft s Azure [3]. Coprehensive surveys on cloud coputing can be found in [4, 5, 6]. While cloud coputing services in practice provide any different services, we consider cloud coputing platfors that provide infrastructure as a service IaaS), in the for of virtual achines VMs), to users. We assue cloud users request VMs, which are specified in ters of resources such as CPU, eory and storage space. Each request is called a job. The type of a job refers to the type of VM the user wants. The aount of tie each VM or job is to be hosted is called its size. After receiving these requests, the cloud provider will schedule the VMs on physical achines, called servers. As an exaple, Table 1.1 lists three types of VMs called instances) available in Aazon EC2. A cloud syste consists of a nuber of networked servers. Each server in the data center has certain aount of resources. This iposes a constraint on the nuber of VMs of different types that can be served siultaneously depending on the aount of resources requested by each VM. This is illustrated 1

14 Table 1.1: Three representative instances in Aazon EC2 Instance Type Meory CPU Storage Standard Extra arge 15 GB 8 EC2 units 1,690 GB High-Meory Extra arge 17.1 GB 6.5 EC2 units 420 GB High-CPU Extra arge 7 GB 20 EC2 units 1,690 GB in the following exaple. Exaple 1.1. Consider a server with 30 GB eory, 30 EC2 coputing units and 4, 000 GB storage space. Then N = 2, 0, 0) and N = 0, 1, 1) are two feasible VM-configurations on the server, where N 1 is the nuber of standard extra-large VMs, N 2 is the nuber of high-eory extra-large VMs, and N 3 is the nuber of high-cpu extra-large VMs. N = 0, 2, 1) is not a feasible VM configuration on this server because it does not have enough eory and coputing units. Jobs with variable sizes arrive according to a stochastic process. These jobs need to be hosted on the servers for a requested aount of tie, after which they depart. We assue jobs are queued in the syste when the servers are busy. There are any design issues associated with such systes [7, 8, 9, 10, 11, 12]. One iportant issue is the resource allocation proble: When a job of a given type arrives, which server should it be sent to? We will call this the routing or load balancing proble. At each server, aong the jobs that are waiting for service, which subset of the jobs should be scheduled? Typically jobs have to be scheduled in a nonpreeptive anner. However, preeption once in a while is soeties allowable. We will call this the scheduling proble. We are interested in resource allocation algoriths with certain optiality properties. The siplest notion of optiality is throughput optiality. We say that an algorith is throughput optial if it can stabilize the syste when any other algorith can. oosely speaking, a throughput optial algorith can sustain the axiu possible rate at which jobs can be processed. Another notion of optiality of interest is delay optiality. delay optiality eans that the ean delay experienced by the jobs is iniized. We will study Delay optiality in the asyptotic liit when the load is close to the boundary of the capacity region. This is called the heavy traffic liit. 2

15 1.1 Best Fit Is Not Throughput Optial The resource allocation proble in cloud data centers has been well studied [7, 8]. Best Fit policy [13, 14] is a popular policy that is used in practice. According to this policy, whenever resources becoe available, the job which uses the largest aount of resources, aong all jobs that can be served, is selected for service. Such a definition has to be ade ore precise when a VM requests ultiple types of ultiple resources. In the case of ultiple types of resources, we can select one type of resource as reference resource, and define best fit with respect to this resource. If there is a tie, then best fit with respect to another resource is considered, and so on. Alternatively, one can consider a particular linear or nonlinear cobination of the resources as a eta-resource and define best fit with respect to the eta-resource. We now show that best fit is not throughput optial. Consider a siple exaple where we have two servers, one type of resource and two types of jobs. A type-1 job requests half of the resource and four tie slots of service, and a type-2 job requests the whole resource and one tie slot of service. Now assue that initially, the server 1 hosts one type-1 job and server 2 is epty; two type-1 jobs arrive once every three tie slots starting fro tie slot 3, and type-2 jobs arrive according to soe arrival process with arrival rate ɛ starting at tie slot 5. Under the best-fit policy, type-1 jobs are scheduled forever since type-2 jobs cannot be scheduled when a type-1 job is in a server. So the backlogged workload due to type-2 jobs will blow up to infinity for any ɛ > 0. The syste, however, is clearly stabilizable for ɛ < 2/3. Suppose we schedule type-1 jobs only in tie slots 1, 7, 13, 19,..., i.e., once every six tie slots. Then tie slots 5, 6, 11, 12, 17, 18,... are available for type-2 jobs. So if ɛ < 2/3, both queues can be stabilized under this periodic scheduler. The specific arrival process we constructed is not key to the instability of best-fit. Assue type-1 and type-2 jobs arrive according to independent Poisson processes with rates λ 1 and λ 2, respectively. Figure 1.1 is a siulation result which shows that the nuber of backlogged jobs blows up under bestfit with λ 1 = 0.7 and λ 2 = 0.1, but is stable under a MaxWeight-based policy with λ 1 = 0.7 and λ 2 = 0.5. This exaple raises the question as to whether there are any throughputoptial policies. To answer this question, we will first propose a stochastic 3

16 Figure 1.1: The nuber of backlogged jobs under the best-fit policy and a MaxWeight policy odel to study resource allocation probles in cloud coputing and then pose this question in a precise anner. 1.2 A Stochastic Model for Cloud Coputing The cloud data center consists of servers or achines. There are K different resources. Server i has C ik aount of resources of type k. There are M different types of VMs that the users can request fro the cloud service provider. Each type of VM is specified by the aount of different resources such as CPU, disk space, eory, etc.) that it requests. Type VM requests R k aount of resources of type k. For server i, an M-diensional vector N is said to be a feasible VMconfiguration if the given server can siultaneously host N 1 type-1 VMs, N 2 type-2 VMs,..., and N M type-m VMs. In other words, N is feasible at server i if and only if M N R k C ik =1 for all k. We let N ax denote the axiu nuber of VMs of any type that can be served on any server. 4

17 We consider a cloud syste which hosts VMs for clients. A VM request fro a client specifies the type of VM the client needs. We call a VM request a job. A job is said to be a type- job if a type- VM is requested. We assue that tie is slotted. We say that the size of the job is S if the VM needs to be hosted for S tie slots. We next define the concept of capacity for a cloud. First, as an exaple, consider the three servers defined in Exaple 1.1. Clearly this syste has an aggregate capacity of 90 GB of eory, 90 EC2 copute units and 12, 000 GB of storage space. However, such a crude definition of capacity fails to reflect the syste s ability to host VMs. For exaple, while = , = 86 90, = , it is easy to verify that the syste cannot host 4 high-eory extra-large VMs and 3 high-cpu extra-large VMs at the sae tie. Therefore, we have to introduce a VM-centric definition of capacity. et A t) denote the set of type- jobs that arrive at the beginning of tie slot t, and let A t) = A t), i.e., the nuber of type- jobs that arrive at the beginning of tie slot t. A t) is assued to be a stochastic process which is i.i.d. across tie and independent across different types. et λ = E[A t)] denote the arrival rate of type- jobs. Assue P A t) = 0) > ɛ A for soe ɛ A > 0 for all and t. For each job j, let S j denote its size, i.e., the nuber of tie slots required to serve the job. For each j, S j is assued to be a positive) integer valued rando variable independent of the arrival process and the sizes of all other jobs in the syste. The distribution of S j is assued to be identical for all jobs of sae type. In other words, for each type, the job sizes are i.i.d. We assue that each server aintains M different queues for different types of jobs. It then uses this queue length inforation in aking scheduling decisions. et Q denote the vector of these queue lengths where Q i is the nuber of type jobs at server i. Jobs are routed to the servers according to a load balancing algorith. et A i t) denote the nuber of type jobs that are routed to server i. Since 5

18 A t) denotes the total nuber of type job arrivals at tie t, routing is done so that i A it) = A t). In each tie slot, jobs are served at each server according to a scheduling algorith. et D i t) denote the nuber of type- jobs that finish service at server i in tie slot t. Then the queue lengths evolve as follows: Q i t + 1) = Q i t) + A i t) D i t). The cloud syste is said to be stable if the expected total queue length is bounded, i.e., li sup E t [ ] Q i t) <. i A vector of arrival rates λ and ean job sizes S is said to be supportable if there exists a resource allocation echanis under which the cloud syste is stable. et S ax = ax {S } and S in = in {S }. We first identify the set of supportable λ, S) pairs. et N i be the set of feasible VM-configurations on a server i. We define sets C and Ĉ as follows: C = { N R M + : N = } N i) and N i) ConvN i ), i=1 where Conv denotes the convex hull. Now define Ĉ = { λ, S) R M + R M + : λ S) C }, where λ S) denotes the Hadaard product or entrywise product of the vectors λ and S and is defined as λ S) = λ S. We use ˇλ to denote λ S so ˇλ C is sae as λ S) Ĉ We use int.) to denote interior of a set. We next use a siple exaple to illustrate the definition of C. Exaple 1.2. Consider a siple cloud syste consisting of three servers. Servers 1 and 2 are of the sae type i.e., they have the sae aount of resources), and server 3 is of a different type. Assue there are two types of VMs. The set of feasible VM configurations on servers 1 and 2 is assued to be N 1 = N 2 = {0, 0), 1, 0), 0, 1)}, i.e., each of these servers can at ost host either one type-1 VM or one type-2 VM. The set of feasible configurations on 6

19 N 1) 2 N 3) ConvN 1 ) ConvN 3 ) 0 1 N 1) N 3) 1 Figure 1.2: Regions ConvN 1 ) and ConvN 3 ) N 2 4 C 2, 2) 0 3 N 1 Figure 1.3: The region C server 3 is assued to be N 3 = {0, 0), 1, 0), 2, 0), 0, 1)}, i.e., the server can at ost host either two type-1 VMs or one type-2 VM. The regions ConvN 1 ) and ConvN 3 ) are plotted in Figure 1.2. Note that vector 0.75, 0.25) is in the region ConvN 1 ). While a type-1 server cannot host 0.75 type-1 VMs and 0.25 type-2 VM, we can host a type-1 VM on server 1 for 3/4 of the tie, and a type-2 VM on the server for 1/4 of the tie to support load 0.75, 0.25). Figure 1.3 shows the region calc. Capacity region for this siple cloud syste is then the set of all λ and S such that the total load λ S) is in the region C. This definition of the capacity of a cloud is otivated by siilar definitions in [15]. As in [15], it is easy to show the following result. Proposition 1.1. For any pair λ, S) such that λ, S) / Ĉ, li t E [ Q it)] =, i.e., the pair λ, S) is not supportable. In the next two chapters we will present algoriths that stabilize the systes as long as the arrival loads are within the region Ĉ. This shows that Ĉ is the capacity of the cloud. Such algoriths that stabilize the syste for any arrival load in the capacity region are said to be throughput optial. 7

20 Moreover, these algoriths do not require knowledge of the actual arrival rates. In the next chapter, we will consider the case when the job sizes are bounded and are known at arrival. We will also assue that preeption is not allowed. We will ake a connection to the scheduling proble in an ad hoc wireless network and propose an algorith inspired by the MaxWeight algorith for wireless networks. In Chapter 3, we will consider the case when the job sizes are not bounded and are known neither at arrival nor at the beginning of service and again present throughput optial resource allocation. algorith for this case. In Chapter 4, we will consider the case when jobs allowed to be preepted once in a while, and in Chapter 5 we will consider delay optiality in the heavy traffic liit. 8

21 CHAPTER 2 THROUGHPUT OPTIMAITY: KNOWN JOB SIZES In this chapter, we consider the resource allocation proble when preeption is not allowed. We assue that the job sizes are known at arrival and are bounded. We will first draw an analogy with scheduling in an ad hoc wireless network. We will show that the algoriths for ad hoc wireless, such as MaxWeight scheduling can be directly used here for a siplified syste. Nonpreeptive algoriths are ore challenging to study because the state of the syste in different tie slots is coupled. For exaple, a MaxWeight schedule cannot be chosen in each tie slot nonpreeptively. Suppose that there are certain unfinished jobs that are being served at the beginning of a tie slot. These jobs cannot be paused in the new tie slot. So, the new schedule should be chosen to include these jobs. A Maxweight schedule ay not include these jobs. Nonpreeptive algoriths were studied in literature in the context of input queued switches with variable packet sizes. One such algorith was studied in [16]. This algorith, however, uses the special structure of a switch and so it is not clear how it can be generalized for the case of a cloud syste. Reference [17] presents another algorith that is inspired by CSMA type algorith in wireless networks. One needs to prove a tie scale separation result to prove optiality of this algorith. This was done in [17] by appealing to prior work [18]. However, the result in [18] is applicable only when the Markov chain has finite nuber of states. However, since the Markov chain in [17] depends on the job sizes, it could have infinite states even in the special case when the job sizes are geoetrically distributed, so the results in [17] do not see to be iediately applicable to our proble. A siilar proble was studied in [19] for scheduling in an input queued switch. An algorith claied to be throughput optial was presented. However, the proof of optiality is incorrect. We present ore details about the algorith and the errors in the proof in the next chapter. 9

22 Since the job sizes are known at arrival, when the jobs are queued, one knows the total backlogged workload in the queue, i.e., the total aount of tie required to serve all the jobs in the queue. One can use this inforation in the resource allocation proble. et q i t) denote the total backlogged workload of type jobs at server i. In this chapter, we will first draw an analogy with scheduling in an ad hoc wireless network. We will show that the algoriths for ad hoc wireless can be directly used here for a siplified syste. Then, we present an alost throughput optial resource allocation algorith for the cloud coputing syste. The results in this chapter have been presented in [20] and [21]. 2.1 A Centralized Queueing Approach We now ake certain siplifying assuptions to gain intuition. Though soe of these are not practical, we first use the to present a very siple solution, which can then be generalized to the original cloud proble. We assue that jobs are queued in a centralized anner. So, for each type of job, there is a single queue at a centralized location as opposed to a separate queue at each server. So, there are M queues in all, one for each type of job. Recall that A t) is the set of type- jobs that arrive at the beginning of tie slot t, and A t) = A t), is the nuber of such jobs. We let a t) = j A S t) j be the total nuber of tie slots of service requested by the jobs that arrive at tie t, i.e., the total arrival of workload of type- jobs in tie slot t. Then, E[a t)] = λ S = ˇλ. et vara 2 t)) = σ 2 for each. et N t) denote the nuber of type- jobs that are served by the cloud at tie slot t. Note that the job size of each of these N t) jobs reduces by one at the end of tie slot t. We assue that a server can serve at ost N ax jobs at the sae tie. The total backlogged workload due to type- jobs is defined to be the su of the reaining job sizes of all jobs of type- in the syste. We let q t) denote the backlogged workload of type- jobs in the network at the beginning of tie slot t, before any other job arrivals. Then the dynaics of q t) can be described as q t + 1) = q t) + a t) N t)). 2.1) 10

23 A ink1 B 1 Interference C ink2 D 2 Interference E ink3 Figure 2.1: Interference constraints for six users and three links and the corresponding interference graph F 3 The resource allocation proble then reduces to the proble of choosing a vector Nt) = N 1 t), N 2 t),..., N M t)) that is a feasible configuration vector for the cloud. Here, we say that a vector N is a feasible vector for the cloud if can be written as N = i Ni) where Ni) is a feasible configuration for the server i Preeptive Algorith We first assue that all servers can be reconfigured at the beginning of each tie slot. This eans that a job can be interrupted at the beginning of each tie and put back in the centralized queue for that job type. In other words, we assue that coplete preeption is allowed. This proble is then identical to the proble of scheduling in an ad hoc wireless network. An ad hoc network consists of a collection of wireless nodes. A link in such a network refers to a transitter-receiver pair of nodes. Not all the links can be siultaneously active because of interference. These constraints are represented by an interference graph. Vertices in the interference graph correspond to the links. If there is an edge between two vertices, then the corresponding links interfere and so cannot transit at the sae tie. An exaple is shown in Figure 2.1. Packets arrive to be transitted over the links and are queued. Given the queue lengths at each link, a scheduling algorith has to choose a set of links that can transit at each given tie, without violating interference constraints. In other words, at any given tie, the scheduler should choose 11

24 an independent set fro the interference graph. In their seinal paper, Tassiulas and Ephriedes [15] presented the MaxWeight scheduling algorith for this proble and showed that it is throughput optial. Each link is associated with a weight which is a function of the queue length, usually the queue length itself, and a schedule with the axiu weight is chosen in each tie slot fro all possible schedules. Since the centralized and preeptive scheduling proble is identical to the wireless network scheduling proble, MaxWeight algorith is also throughput optial. However, the Server-by-server MaxWeight algorith Algorith 1) is equivalent to finding the axiu weight feasible vector for the cloud. Algorith 1 Server-by-server MaxWeight allocation for a centralized queueing syste with coplete preeption At the beginning of tie slot t, consider the i th server. If the set of jobs on the server are not finished, ove the back to the central queue. Find a VM-configuration N t) such that N i) t) arg ax N N i q t)n. At server i, we create up to N i) t) type- VMs depending on the nuber of jobs that are backlogged. et N i) t) be the actual nuber of VMs that were created. Then, we set ) q t + 1) = q t) + a t) i N i). Then, as in [15], we have the following proposition. Proposition 2.1. Consider the cloud syste with centralized queues and assue that coplete preeption of jobs is allowed. The server-by-server MaxWeight allocation presented in Algorith 1 is throughput optial, i.e., whenever ˇλ intc. li E t [ ] q t) < The proof is based on bounding the drift of a quadratic yapunov function. 12

25 The drift is shown to be negative outside a finite set and the Foster-yapunov theore is invoked to prove positive recurrence of the Markov chain corresponding to the syste as long as the arrivals are within the capacity region. We skip the proof of this proposition since it is uch sipler and is in the sae lines as that of Theore 2.1. One drawback of MaxWeight scheduling in wireless networks is that its coplexity increases exponentially with the nuber of wireless nodes. Moreover, it needs to be ipleented in a centralized policy. However, for the cloud syste the server by server ipleentation in Algorith 1 is has uch lower coplexity and can be ipleented in a distributed anner. Consider the following exaple. If there are servers and each server has S allowed configurations, then when each server coputes a separate MaxWeight allocation, it has to find a schedule fro S allowed configurations. Since there are servers, this is equivalent to finding a schedule fro S possibilities. However, for a centralized MaxWeight schedule, one has to find a schedule fro S schedules. Moreover, the coplexity of each server s scheduling proble depends only on its own set of allowed configurations, which is independent of the total nuber of servers. Typically the data center is scaled by adding ore servers rather than adding ore allowable configurations Nonpreeptive Algoriths One of the siplifying assuptions ade in the previous subsection is that jobs can be interrupted and reallocated later, possibly on different servers. In practice, a job ay not be interruptible or interrupting a job can be very costly the syste needs to store a snapshot of the VM to be able to restart the VM later). In the rest of this chapter and the next, we assue that jobs are not allowed to be interrupted. Nonpreeptive algoriths are ore challenging to study because the state of the syste in different tie slots is coupled. For exaple, a MaxWeight schedule cannot be chosen in each tie slot nonpreeptively. Suppose that there are certain unfinished jobs that are being served at the beginning of a tie slot. These jobs cannot be paused in the new tie slot, so the new schedule should be chosen to include these jobs. A Maxweight schedule ay not include these jobs. 13

26 Therefore, since one cannot choose MaxWeight schedule in every tie slot, a natural alternative is to soehow choose MaxWeight schedule every few tie slots and then perfor soe reasonable scheduling between these tie slots. It turns out that using MaxWeight schedule once in a while is good enough. Before we present the algorith, we outline the basic ideas first. We group T tie slots into a super tie slot, where T > S ax. At the beginning of a super tie slot, a configuration is chosen according to the MaxWeight algorith. When jobs depart a server, the reaining resources in the server are filled again using the MaxWeight algorith; however, we ipose the constraint that only jobs that can be copleted within the super slot can be served. So the algorith yopically without consideration of the future) uses resources, but is queue-length aware since it uses the MaxWeight algorith. This is described ore precisely in Algorith 2 Algorith 2 Myopic MaxWeight allocation: We group T tie slots into a super tie slot. At tie slot t, consider the i th server. et N i) t ) be the set of VMs that are hosted on server i at the beginning of tie slot t, i.e., these correspond to the jobs that were scheduled in the previous tie slot but are still in the syste. These VMs cannot be reconfigured due to our nonpreeption requireent. The central controller finds a new vector of configurations Ñ i) t) to fill up the resources not used by N i) t ), i.e., Ñ i) t) arg ax q t)n, N:N+N i) t ) N i The central controller selects as any jobs as available in the queue, up to a axiu of Ñ i) t) type- jobs at server i, and subject to the constraint that a type- job can only be served if its size S j T t od T ). et N i) t) denote the actual nuber of type- jobs selected. Server i then serves the N t) i) new jobs of type, and the set of jobs N i) t ) left over fro the previous tie slot. The queue length is updated as follows: q t + 1) = q t) + a t) i N i) t ) + N i) t) ). Note that this yopic MaxWeight allocation algorith differs fro the server-by-server MaxWeight allocation in two aspects: i) jobs are not interrupted when served and ii) when a job departs fro a server, new jobs 14

27 are accepted without reconfiguring the server. The following proposition characterizes the throughput achieved by the yopic MaxWeight. Proposition 2.2. Consider the yopic MaxWeight algorith in Algorith 2. Any job load that satisfies T T S ax ˇλ C is stabilizable under the yopic MaxWeight allocation. The proof of this proposition again uses the Foster-yapunov theore based on a quadratic yapunov function.however, instead of the one step drift, the drift over every super tie slot is shown to be negative outside a finite set). This then gives that a syste sapled at the beginning of every super tie slot is stable. Since the ean arriving workload in each tie slot is bounded, we then have stability of the original syste is also stable. We again skip the proof as ost of the sae ideas are presented in the proof of Theore 2.1. It is iportant to note that, unlike best fit, the yopic MaxWeight algorith can be ade to achieve any arbitrary fraction of the capacity region by choosing T sufficiently large. 2.2 Resource Allocation with oad Balancing In the previous section, we considered the case where there was a single queue for jobs of the sae type, being served at different servers. This requires a central authority to aintain a single queue for all servers in the syste. A ore distributed solution is to aintain queues at each server and route jobs as soon as they arrive. To the best of our knowledge, this proble does not fit into the scheduling/routing odel in [15]. However, we show that one can still use MaxWeight-type scheduling if the servers are load-balanced using a join-the-shortest-queue JSQ) routing rule. So, we now assue that each server aintains M different queues for different types of jobs. It then uses the inforation about backlogged workload in each of these queues in aking scheduling decisions. et q denote the vector of these backlogged workloads where q i is the backlogged workload of type jobs at server i. Routing and scheduling are perfored as described in Algorith 3. 15

28 Algorith 3 JSQ Routing and yopic Maxweight Scheduling 1. Routing Algorith JSQ Routing): All the type jobs that arrive in tie slot t are routed to the server with the shortest backlogged workload for type jobs i.e., the server i t) = arg inq i t). Therefore, i {1,2,,,} the arrivals to q i in tie slot t are given by {Â t) if i = i a i t) = t) 0 otherwise. 2.2) 2. Scheduling Algorith Myopic MaxWeight Scheduling) for each server i: T tie slots are grouped into a super tie slot. A MaxWeight configuration is chosen at the beginning of a super tie slot. So, for t = nt, configuration Ñ i) t) is chosen according to Ñ i) t) arg ax q i t)n. N N i For all other t, at the beginning of the tie slot, a new configuration is chosen as follows: Ñ i) t) arg ax q i t)n, N:N+N i) t ) N i where N i) t ) is the configuration of jobs at server i that are still in service at the end of the previous tie slot. As any jobs as available are selected for service fro the queue, up to a axiu of Ñ i) t) jobs of type, and subject to the constraint that a new type job is served only if it can finish its service by the end of the super tie slot, i.e., only if S j T t od T ). et N i) t) denote the actual nuber of type jobs selected at server i and define N i) t) = N i) t ) + N i) t). The queue lengths are updated as follows: q i t + 1) = q i t) + a i t) N i) t). 2.3) 16

29 The following theore characterizes the throughput perforance of the algorith. T Theore 2.1. Any job load vector that satisfies T S ax ˇλ C is stabilizable under the JSQ routing and yopic MaxWeight allocation as described in Algorith 3. Proof. The idea behind the proof is again to bound the drift of a quadratic yapunov function over a super tie slot. However, now the load balancing algorith also plays a role in the drift. et Y i t) denote the state of the queue for type- jobs at server i. If there are I such jobs, Y i t) is a vector of size I and Y j i t) is the backlogged) size of the j th type- job at server i. First, it is easy to see that Yt) = {Y i t)},i is a Markov chain under the yopic MaxWeight scheduling. Further define S = {y : PrYt) = y Y0) = 0) for soe t}, then Yt) is an irreducible Markov chain on state space S assuing Y0) = 0. This clai holds because i) any state in S is reachable fro 0 and ii) since Pra t) = 0) ɛ A for all and t, the Markov chain can ove fro Yt) to 0 in finite tie with a positive probability. Further q i t) = j Y j,i t), i.e., q i t) is a function of Y i t). We will first show that the increase of q i t)n i) t) is bounded within a super tie slot. For any t such that 1 t od T ) T S ax, for each server i, q i t)n i) t 1) = a) = b) + q i t)n i) t ) + q i t)n i) t ) + q i t) N i) t 1) N i) t ) ) q i t)ñ i) t) ) q i t)n i) t ) + q i t)ñ i) t) I qi t) S axn ax ) q i t)n i) t ) + q i t)ñ i) t) I qi t)<s axn ax q i t)n i) t) + MS ax N 2 ax, where the inequality a) follows fro the definition Ñ i) t); and inequality b) 17

30 holds because when q i t) S ax N ax, there are enough nuber of type- jobs to be allocated to the servers, and when 1 t od T ) T S ax, all backlogged jobs are eligible to be served in ters of job sizes. Now since q i t) q i t 1) = a i t 1) N i) t) a i t 1) + N ax, we have q i t 1)N i) t 1) β + q i t)n i) t) + a i t 1)N ax, 2.4) where β = MN 2 axs ax + 1). et V t) = qt) 2 be the yapunov function. et t = nt +τ for 0 τ < T. Then, E[V nt + τ + 1) V nt + τ) qnt ) = q] [ =E qi t) + a i t) N i) t) ) ] 2 q 2 i t) qnt ) = q i [ =E 2 q i t) a i t) N i) t) ) i + ai t) N i) t) ) ] 2 qnt ) = q i [ K 1 + 2E q i t)a i t) ] q i t)n i) t) qnt ) = q i i =K E[q i t)t)a t) qnt ) = q] [ ] 2E q i t)n i) t) qnt ) = q i =K ˇλ E[q i t)t) qnt ) = q] [ ] 2E q i t)n i) t) qnt ) = q i K ˇλ 2 τ + 2 ˇλ E[q i nt )nt ) qnt ) = q] [ ] 2E q i t)n i) t) qnt ) = q =K i ˇλ 2 τ + 2 ˇλ q i 2.5) 2.6) 2.7) 2.8) 2.9) 2.10) 18

31 [ ] 2E q i t)n i) t) qnt ) = q, 2.11) i where K 1 = MN 2 ax + ˇλ 2 + σ 2 + 2ˇλ N ax ) and i = i nt ) = arg inq i. Equation 2.8) follows fro the definition of a i in the routing i {1,2,,,} algorith in 5.2). Equation 2.9) follows fro the independence of the arrival process fro the queue length process. Inequality 2.10) follows fro the fact that E[q i t)t)] E[q i nt )t)] E[q i nt )nt ) + t 1 t =nt a t)] = E[q i nt )] + τ ˇλ. Now, applying 2.4) repeatedly for t [nt, n+1)t S ax ], and suing over i and using the fact that i a it) = a t), we get i q i t)n i) t) t nt )β i q i nt )N i) nt ) + n+1)t S ax 1 t =nt a t )N ax. 2.12) Since 1+ɛ)T 1+ɛ)T T S ax ˇλ intc), there exists ɛ > 0 s.t. T S ax ˇλ C, and so there exists {ˇλi } 1+ɛ)T such that i T S ˇλi ax ConvN i ) for all i and ˇλ = ˇλ i. According i to the scheduling algorith, for each i, we have that T 1 + ɛ) q i nt )ˇλ i T S ax q i nt )N i) nt ). 2.13) Thus, we get, i q i t)n i) t) t nt )β i q i nt )N i) nt ) + t nt )β 1 + ɛ)t q i nt )ˇλ i + T S ax i n+1)t S ax 1 t =nt a t )N ax n+1)t S ax 1 t =nt 2.14) a t )N ax. 2.15) 19

32 Substituting this in 2.11), for t [nt, n + 1)T S ax ], we get E[V nt + τ + 1) V nt + τ) qnt ) = q] K + 2τ ˇλ 2 + ˇλ N ax ) + 2t nt )β + 2 T ˇλ q i 21 + ɛ) q iˇλi T S. ax i 2.16) Note that ˇλ q i = i ˇλ i q i i ˇλ i q i. We will now use this relation and su the drift for τ fro 0 to T 1. Using 2.16) for τ [0, T S ax ], and 2.11) for the reaining τ, we get E[V n + 1)T ) V nt ) qnt ) = q] T K ˇλ 2 + ˇλ T 1 T S ax 1 N ax ) τ + 2β τ τ=0 τ=0 + 2T 1 + ɛ)t q iˇλi 2 q iˇλi T S T S ax ) i, ax i, K 2 2ɛT q iˇλi, i where K 2 = T K ˇλ 2 + ˇλ N ax ) T 1 τ=0 τ + 2β T S ax 1 τ=0 τ. et B = {Y : i q iy)ˇλ i K 2 /ɛt }. The set B is finite. This is because there are only a finite nuber of q Z M + vectors satisfying i q iˇλ i and for each q, there are a finite nuber of states Y such that q = qy). Clearly the drift E[V n + 1)T ) V nt ) qnt ) = q] is negative outside the finite set B. Then fro the Foster-yapunov theore [22, 23], we have that the sapled Markov chain Ỹn) YnT ) is positive recurrent and so li n E[ i q int )] <. For any tie t between nt and n + 1)T, we have [ ] [ E q i t) E q i nt ) + ] t 1 a i t ) i i i t =nt [ ] =E q i nt ) + T ˇλ. i 20

33 Therefore, we have li E n [ ] q i t) i li n E <. [ ] q i nt ) + T i ˇλ This proves the theore. Since the work load q i t) is always at least as uch as the nuber of jobs, Q i t), fro Proposition 1.1, we have that any arrival rate ˇλ / C is not supportable. Thus, we have that Algorith 3 is alost throughput optial. By this, we ean that given any arrival rate ˇλ C, we can choose the paraeter T so that the syste is stable. 2.3 Sipler oad Balancing Algoriths Though JSQ routing algorith is alost) throughput optial, the job scheduler needs the workload inforation fro all the servers. This iposes a considerable counication overhead as the arrival rates of jobs and nuber of servers increase. In this section, we present two alternatives which have uch lower routing coplexity Power-of-two-choices Routing and Myopic MaxWeight Scheduling An alternative to JSQ routing is the power-of-two-choices algorith [24, 25, 26], which is uch sipler to ipleent. When a job arrives, two servers are sapled at rando, and the job is routed to the server with the saller queue for that job type. In our algorith, in each tie slot t, for each type of job, two servers i 1 t) and i 2 t) are chosen uniforly at rando. The job scheduler then routes all the type job arrivals in this tie slot to the server with shorter backlogged workload aong these two, i.e., i t) = 21

34 arg in q i t) and so i {i 1 t),i 2 t)} a t) if i = i t) a i t) = 0 otherwise. Otherwise, the algorith is identical to the JSQ-Myopic MaxWeight algorith considered earlier. The following theore shows that the throughput perforance using the power-of-two-choices algorith is siilar to that of JSQ routing algorith when all the servers are identical. Theore 2.2. When all the severs are identical, any load vector that sat- T isfies T S ax ˇλ intc) is stabilizable under the power-of-two-choices routing and yopic MaxWeight allocation algorith. Proof. Again, we use V t) = qt) 2 as the yapunov function. Then, fro 2.7), we have [ ] E[V t + 1) V t) qnt ) = q] K 1 + 2E q i t)a i t) qnt ) = q i [ ]. 2E q i t)n i) t) qnt ) = q i 2.17) For fixed, let X t) be the rando variable which denotes the two servers that were chosen by the routing algorith at tie t for jobs of type. X t) is then uniforly distributed over all sets of two servers. Now, using the tower property of conditional expectation, we have [ ] E q i t)a i t) qnt ) = q i [ [ ] ] =E E q i t)a i t) qnt ) = q, X t) = {i, j } qnt ) = q i =E [E [in q i t), q j t)) a t) qnt ) = q, Xt) = {i, j }] qnt ) = q] 2.18) [ [ ] qi t) + q j t) E E a t) 2 qnt ) = q, Xt) = {i, j }] qnt ) = q [ ] 1 =E ) 1 q i t)ˇλ qnt ) = q 2.19) 2 2 i 22

35 = ˇλ [ ] E q i t) qnt ) = q i ˇλ [ ] E t 1 q i nt ) + a i t ) qnt ) = q i t =nt i ˇλ ) q i + τ ˇλ, 2.20) i where τ = t nt. Equation 2.18) follows fro the routing algorith and 2.19) follows fro the fact that X t) is uniforly distributed. Inequality 2.20 follows fro the fact that E[ i a it )] = E[a t )] = ˇλ. Since the scheduling algorith is identical to Algorith 3, the bound in 2.12) still holds i q i t)n i) t) t nt )β i 1+ɛ)T q i nt )N i) nt ) + n+1)t S ax 1 t =nt 1+ɛ)T a t )N ax. 2.21) Since T S ax ˇλ intc), there exists ɛ > 0 s.t. T S ax ˇλ C. We have assued that all the servers are identical, so C is obtained by suing 1+ɛ)T copies of ConvN ). Thus, T S ax ˇλ C, eans 1+ɛ)T ˇλ ConvN ) = T S ax ConvN i ) for all i. According to the scheduling algorith, for each i, we have that T 1 + ɛ) q i nt ) ˇλ T S ax q i nt )N i) nt ). Thus, we get i q i t)n i) t) t nt )β 1 + ɛ)t T S ax i q i nt ) ˇλ n+1)t S ax 1 + t =nt a t )N ax. 23

36 Now, substituting this and 2.20) in 2.17) and suing over t [nt, n + 1)T 1], as in the proof of Theore 2.1, we get E[V n + 1)T ) V nt ) qnt ) = q] T K T i, K 2 2ɛT i ˇλ 2 + ˇλ T 1 T S ax 1 N ax ) τ + 2β τ q i ˇλ τ=0 τ=0 1 + ɛ)t 2 q iˇλi T S T S ax ) ax ˇλ q i. This proof can be copleted by applying the Foster-yapunov theore [22, 23] and then observing that the workload does not change by uch within a i, supertie slot, as in the proof of Theore Pick-and-Copare Routing and Myopic MaxWeight Scheduling One drawback of the power-of-two-choices scheduling is that it is throughput optial only when all servers are identical. In the case of nonidentical servers, one can use pick-and-copare routing algorith instead of power-of-twochoices. The algorith is otivated by the pick-and-copare algorith for wireless scheduling and switch scheduling [27], and is as siple to ipleent as power-of-two-choices, and can be shown to be optial even if the servers are not identical. We describe this next. The scheduling algorith is identical to the previous case. Pick-and-copare routing works as follows. In each tie slot t, for each type of job, a server i t) is chosen uniforly at rando and copared with the server to which jobs were routed in the previous tie slot. The server with the shorter backlogged workload aong the two is chosen and all the type job arrivals in this tie slot are routed to that server. et i t) be the server to which jobs will be routed in tie slot t. Then, i t) = 24

37 arg in q i t) and so i {i t),i t 1)} a t) if i = i t) a i t) = 0 otherwise. Theore 2.3. Assue a t) a ax for all t and. Any job load vector T that satisfies T S ax ˇλ intc) is stabilizable under the pick-and-copare routing and yopic MaxWeight allocation algorith. Proof. Consider the irreducible Markov chain Yt) = Yt), {i t)} ) and the yapunov function V t) = qt) 2. Then, as in the proof of Theore 2.2, siilar to 2.17) for t nt, we have E[V t + 1) V t) qnt ) = q, i nt ) = i ] [ ] K 1 + 2E q i t)a i t) qnt ) = q, i nt ) = i i [ ] 2E q i t)n i) t) qnt ) = q, i nt ) = i. i 2.22) T Since T S ax ˇλ intc), there exists and ɛ > 0 such that, 1+ɛ) T T S ax ˇλ C and so there exists {ˇλi } T such that 1 + ɛ) i T S ˇλi ax ConvN i ) for all i and ˇλ = ˇλ i. This {ˇλi } can be chosen so that there is a κ so that ˇλ i κˇλ i. i This is possible because if ˇλ > 0 and ˇλ is not on the boundary of C, one can always find {ˇλi }i so that ˇλ i > 0. Since the scheduling part of the algorith is identical to Algorith 3, 2.15) still holds for t [nt, n + 1)T S ax ]. Thus, we have i q i t)n i) t) i t nt )β 1 + ɛ)t q i nt )ˇλ i T S, 2.23) ax where β = MN ax a ax + N ax ) + MS ax Nax. 2 We also need a bound on the increase in i q it)n i) t) over ul- 25