Response-Time-Optimized Distributed Cloud Resource Allocation

Transcription

1 Repone-Time-Optimized Ditributed Cloud Reoure Alloation Matthia Keller and Holger Karl the ruial quality metri i the uer-pereived repone time to a requet a the appliation need to quikly reat on uer interation. Large repone time impede uability, inreae uer rutration [7], [37], or prevent ommerial ue. An obviou olution to provide mall repone time would be to deploy an appliation at many ite o that eah uer ind one ite nearby. Thi, however, i ineaible a eah utilized ite inur additional ot. We are hene aed with the tak to deide where a uer requet hall be proeed, uing a ew ite a poible at a bet poible repone time. We reer to thi tak a the aignment problem. Thi problem trade-o between ot and quality i intuitive yet diiult to apture in a onrete problem tatement and olution. Thi diiulty lie in the nature o the repone time. It i determined by three part: The network lateny taken to end the requet rom the uer to the loud reoure and ending the anwer bak the round trip time; the atual proeing time o the requet; the queuing delay a requet inur at the loud reoure while other requet are urrently proeed at that reoure. In many appliation, we an onider the proeing time to be igniiantly maller than the round trip time. The round trip time depend on the hoie where a uer requet i proeed and, to a muh maller degree, on the network load along the way. The queuing delay, however, depend on the haring o a reoure among many uer and i not an eet immediately inluened by the deiion or a ingle uer; it depend on the joint deiion or all uer. In brie: The repone time RT i the um o the round trip time RTT, the queuing delay QD, and the proeing time PT (Fig. ). From queuing theory we know that, or a ixed utilization, the queuing delay i horter or higher ervie rate. Fig. 2 how queuing delay or dierent level o ytem utilizaarxiv:6.6262v [.NI] 23 Jan 26 Abtrat A urrent trend in networking and loud omputing i to provide ompute reoure over widely dipered plae exempliied by initiative like Network Funtion Virtualiation. Thi pave the way or a widepread ervie deployment and an improve ervie quality; a nearby erver an redue the uer-pereived repone time. But alway uing the nearet erver i a bad deiion i that erver i already highly utilized. Thi paper invetigate the optimal aignment o uer to widepread reoure a onvex apaitated aility loation problem with integrated queuing ytem. We determine the repone time depending on the number o ued reoure. Thi enable ervie provider to balane between reoure ot and the orreponding ervie quality. We alo preent a linear problem reormulation howing mall optimality gap and ater olving time; thi peed-up enable a wit reation to demand hange. Finally, we ompare olution by either onidering or ignoring queuing ytem and diu the repone time redution by uing the more omplex model. Our invetigation are baked by large-ale numerial evaluation. Index Term loud omputing; virtual network untion; network untion virtualiation; reoure management; plaement; aility loation; queueing model; lineariation; optimization INTRODUCTION. Challenge in Ditributed Cloud A urrent trend in networking and loud omputing i to provide ompute reoure over widely dipered plae. Computation will not only take plae on dektop or large data entre, but alo at maller entre or within the network itel, e.g., inide individual in-network erver rak loated near bakbone router. Thi trend i known under dierent label, or example, Carrier Cloud [7], [4], [5], Ditributed Cloud Computing [3], [6], [22], [46], or In- Network Cloud [29], [48], [5]. Thee In-Network Cloud tend to be le ot-eiient than onventional loud due to a wore eonomy o ale; they are hene oten geared toward peii network ervie (e.g., irewall, load balaner). Eaing a more lexible deployment o thee ervie beame popular a Network Funtion Virtualization [23] not only inide a data entre but alo beyond, in wide area network [8], [5], [52]. We onider not only exeuting network untion but more generially exeuting appliation at thoe In-Network Cloud yielding an important advantage [5], [43]: The reoure o thee loud are loer to end uer than thoe o onventional loud, have maller lateny between uer and loud reoure, and are thereore uitable or running highly interative appliation. Example or uh appliation are lateny-ritial appliation [8], [54], uer-utomized treaming ervie [9], [9], [3], or Cloud Gaming [37]; the omputing tak range rom proeing the requet, aggregating inoming data tream, up to rendering and enoding video tream. In uh appliation, Manuript reeived January xx,yy; revied January xx, yy. Thi work wa partially upported by the German Reearh Foundation (DFG) within the Collaborative Reearh Centre On-The-Fly Computing (SFB 9). The author are with the Univerity o Paderborn, Warburger Str., 3398 Paderborn, Germany. mkeller@upb.de, holger.karl@upb.de

2 2 Uer Server queue pu Figure : Repone time a the um o round trip time RTT, queuing delay QD, proeing time PT. queuing delay ¹ =: ¹ =: ¹ =: utilization ½ = ¹ Figure 2: Queuing delay QD or dierent ervie rate. tion ρ= λ /µ or three dierent ervie rate µ. For intane, web erver anwering imple requet have high ervie rate and oten negligible queuing delay. Thi explain why they are ommonly ignored in literature (Setion 2). We ou in thi paper on omputationally intenive appliation (exempliied beore). Suh an appliation proeing time i long, it ervie rate i low, the queuing delay beome large, and, hene, i a igniiant portion o the repone time. Shah et al. [49] urvey intruion detetion ytem and ite dierent meaurement o paket proeing time o up to m. Barker et al. [8] tudy game erver map load lating 2 m in their experiment. Ihii et al. [3] ondut experiment on AWS [] uing a parallel Data Proeing Appliation and oberve proeing time between 4 and 8 m. Lee et al. [37] and Claypool et al. [7] oberve drop in uer experiene when playing omputer game with artiiially inreaed lateny to larger than 2 m. We ou on appliation with long proeing time, m, and average internet round trip time, 6 6 m [36]. Longer proeing time, ay min, reult in queuing delay igniiantly larger than the round trip time rendering them le dominant; when ignoring round trip time the reulting problem beome a impler mapping problem. Smaller proeing time, ay. m, reult in queuing delay igniiantly maller than the round trip time rendering the queuing delay le dominant; when ignoring queuing delay the reulting problem beome a impler, non-onvex Faility Loation Problem. Setion 4.3 quantiie the repone time redution when onidering queuing delay v. ignoring them or dierent proeing time..2 Queuing Delay Eet How doe the queuing delay aet the repone time? A a toy example, let u onider the network rom Fig. 3 with three loation o interet: One lient and two poible aility loation a, b with ompute reoure to run the appliation. Thee reoure are equally at and an erve requet at rate µ= req. /. Aume the round trip time between and a a 6 m and between and b a 7 m. Requet enter the network at with arrival rate λ. With thi etup, the requet an be erved at only a, only b, or plit. Correponding ervie rate o, req./ o a high perormane weberver, e.g. Apahe, Nginx, delivering tati ontent. exp. rep. time [m] S 4 S S 2 S arrival rate [req/] Figure 3: The lower plot how repone time o dierent arrival rate λ or three dierent trategie or the topology. The upper plot how the ued topology. among a and b. Fig. 3 how, a a untion o the arrival rate, the reulting repone time RT = RTT + PT + QD or a ew imple trategie S i 2. The irt trategy S minimize only the requet round trip time: Requet are aigned to the nearet aility a and i it apaity µ i exeeded the remaining requet are aigned to b. The dramati repone time growth or λ>8 i the reult o too many requet aigned to aility a. Let λ a be the aigned requet to a, then a utilization ρ a i λ a /µ. To avoid too large utilization, trategie S 2 and S 3 limit them to a maximum value ˆρ, ρ a, ρ b ˆρ<; S 2 ue ˆρ=.9 and S 3 ue ˆρ=.8. On the one hand S 3 with a lower limit ha a horter RT (Fig. 3) than S 2 a the eond aility i ued earlier. But on the other hand S 2 an handle a higher arrival rate than S 3, 6<λ<8, beaue a higher limit enable handling more requet in total; ytem apaity i λ<2µˆρ. To relax uh a predeined upper bound, S 4 dynamially adjut the limit to the urrent ytem utilization, ˆρ= λ /2µ. With the ame reoure at both loation S 4 boil down to evenly plitting the load between the two ailitie. Compared to S..3 the reulting RT are on the one hand mall or λ>4 but on the other hand larger or λ<4. So ar, all aignment trategie ignore the reulting queuing delay. In ontrat, our lat trategy S 5 additionally redue the queuing delay o both reoure. The trategy requet aignment depend on the ditane between both reoureig. 3. The reulting RT are the lowet or all trategie S..5. In onluion, we were able to improve aignment by onidering queuing delay. Hereater, we lit the equation o the expeted repone time in Fig. 3: S to S 3 reult in (), S 4 in (2), and S 5 in (3). 6 + ( µ λ, ) ˆρµ µ,ˆρ (λ) := λ 6 + µ ˆρµ + ( λ ˆρµ λ 7 + g µ (λ) := ( 6+ ) + 2 µ λ /2 2 µ λ+ˆρµ S 5 i λ ˆρµ ), ele ( 7+ ) µ λ /2 2. Thi paper extended verion lit S i equation (whih one eay to derive) and provide detail about S 5. () (2)

3 3 aignment ratio o l = l =5 l = ¹ o a arrival rate [req/] l =2 l =4 l =8 Figure 4: Requet aignment to a with dierent ditane l between a and b. h µ (λ) := min λ [, λ] ( λ λ λ λ λ ) 6 + µ λ + ( 7 + µ λ+λ ) Fig. 4 how trategy S 5 requet aignment to reoure a a a ration o λ on the vertial axi; the remaining requet are aigned to b. The horizontal axi how an inreaing arrival rate λ. The dierent line orrepond to ditane l between a and b how muh longer requet tranportation take to end to b intead to a. Thi way the original toy example i line l = and the other line vary the round trip time to b. I the reoure have the ame round trip time, the aignment reult in an even plit. However, i one reoure i arer away, l >, at irt the nearer reoure i preerred and only with inreaing arrival do the aignment onverge to an even plit. Then, the queuing delay portion o the repone time i igniiantly larger than l. Fig. 5 how two olution or the ame network lie. The let (a) i obtained by ignoring queuing delay orrepond to trategy S and the right (b) i obtained by onidering queuing delay orrepond to trategy S 5. Reoure utilization i hown by a gray level; blak mean ull utilization. Comparing both, onidering queuing delay (b) avoid very high utilized reoure whih happen muh more requently when queuing delay are ompletely ignored like in trategy S. The downide o highly utilized reoure i longer queuing delay and, with it, longer repone time..3 Contribution Thi paper preent two optimization ormulation to aign requet to ompute reoure loation in a network that minimize the average requet repone time (Fig. ) or all uer (Setion 3.). One ormulation i exat; we how it onvexity and, or the irt time, obtain optimal 3 olution or larger network uing a onvex olver (Setion 3.3). The eond ormulation i a non-trivial lineariation o the irt with a mall optimality gap hown empirially (Setion 3.4). Finally, we evaluate three apet: Firt, we ompare the olution time and auray o both ormulation by olving oniguration baed on realiti network topologie (Setion 4.2). Seond, 3. Numerially obtaining olution by olver with a gap threhold o 6. (3) (a) Figure 5: Two olution obtained while ignoring and onidering time in ytem (a,b). The grey ale how utilization o loation, blak mean at apaity limit. we limit the ued reoure and invetigate the eet o uh limitation on the repone time (Setion 4.3). Third, we determine appliation and network propertie where the omplex problem ormulation preented in thi paper igniiantly redue the repone time ompared to a impler ormulation ignoring the queuing delay (Setion 4.4). Thi third evaluation extend our own previou work [33]. We ompare ive ator inluening queuing delay and in addition vary input randomly in order to veriy the tatitial relevane o our inding. In ummary, we olved,5 oniguration and analyed their olution. 2 RELATED WORK Aignment problem o the orm deribed above have been invetigated beore. We truture their omparion along our dimenion relevant to thi work: Their model omplexity, impliiation reduing the problem earh pae, optimization goal, and olution approahe. Finally, related ytem o geographial load balaning are ompared. 2. Model Complexity The implet model onider only the round trip time (RTT) when aigning uer to loud reoure. They equate repone time with RTT. Clearly, thi i a impliiation o reality, yet minimizing thi average RTT i equivalent to the well-known apaitated aility loation problem (FLP). I the problem i urther retrited to only ue p reoure, it beome a p-median FLP, whih i NP-hard [4]. A tep loer to reality i modelling alo the proeing time (PT) in addition to the RTT. But a long a PT i ontant, thi till tay a aility loation problem o the type deribed above. Thi an be eaily een by extending the original network topology by peudo-link (at the erver or uer ide) that repreent thee proeing time via their latenie; thi i a ommon rewriting tehnique or graph-baed problem inluding aility loation problem. (b)

4 4 The real hallenge our when we alo onider the queuing delay (QD). In thi ae, the additional time annot be expreed by rewriting the network topology a the QD depend on the aignment deiion: A higher load reult in a longer wait, poibly trading o againt a horter RTT. So ar, thi more general model ha been onidered only by ew work diued in the remaining o thi etion, mot ue impler aumption than our (Setion 3) rendering the problem eaier to olve. Vidyarthi et al. [53] allow the ame degree o reedom a we do. They approximate, imilar to u, the non-linear part o the objetive untion with a piee-wie linear untion. However, in ontrat to our work, they ued a utting plane tehnique whih iteratively reine the piee-wie untion a neeary; it remain unlear how large their lineariation error i. In ontrat, our evaluation how mall lineariation error; and thi i ahieved by uing a impler tehnique. 2.2 Simpliiation Other author invetigate lightly dierent enario, o that their problem ormulation are imilar, yet impler than our. Some author [55], [59] replae the non-linear QD part with a ontant upper bound and, onequently, the reulting problem beome impler to olve. But thi alo hide QD hange a a reult o aignment hange. For intane, in a ituation where load balaning would redue the QD, thi redution i not viible a the QD part i ontant. Conequently, the reulting olution ha urther potential or optimization. We exploit thi and do not onider upper bound or queuing delay. In another impliiation, the aignment are predeined by a rule. Some author [2], [55], [59] alway aign requet to the nearet loud reoure. Thi imple problem redue to jut inding the bet reoure loation and i eaier to olve. The aignment are then predetermined by the rule. However, balaning the aignment ould urther redue the QD but i not onidered. We do not ue any predeined aignment rule, o we have the reedom to hange aignment in order to urther redue the RT. Another group o author [], [2] ue a parametrized aignment rule alled the gravity rule: Weight determine how uer are aigned to loud reoure. Thee onigurable weight are ued to ontinuouly olve the ame problem with new weight releting the reoure utilization o the previou olution. Thi approah doe not guarantee to onverge, o the author propoe a heuriti that attenuate the hange in eah iteration, enoring onvergene with an unknown lineariation error. In ontrat, we olve the problem in one tep by uing all inormation to ind the global optimum. Liu et al. [39], Lin et al. [38], and Goudarzi et al. [26] preent a imilar Faility Loation Problem with onvex ot uh a queuing delay or reoure energy ot. In ontrat to our work, they relax the integer alloation deiion variable impliying the problem to the ot o a le aurate olution a a reult o rounding up the obtained ontinuou alloation. Our goal, in ontrat, i to prevent unexpeted expene by introduing an upper bound to the number o ued reoure. Continuouly relaxing our problem an aue any loation to be alloated a bit and, onequently, any ite i ued and paid. While the paper [38], [39] only onider queuing delay a a ot untion, thi paper diue a holiti queuing ytem integration and additionally onider plitting and joining (aigning) o the arrival proe. 2.3 Optimization Goal Exiting literature ue queuing delay in FLP with three optimization goal: lai FLP, min/max FLP, and overage FLP. Clai FLP are problem that minimize the average repone time, like our problem (Setion 3.) or other [], [2], [47], [53], [55], [59], allowing RT variation or individual uer. Aboolian et al. [2] min/max problem minimize the maximum repone time. Intuitively, uh problem improve epeially the uer RT with high RTT to loud reoure. However, i only one uh uer exit with reoure being ar away, aigning thi uer will negatively aet the aignment o other uer: Their aignment are now leretritedly ontrained by a relaxed upper bound and are likely wore than without the irt uer. In ontrat, laial FLP do not uer thi way rom a wore ae uer. Another type o problem i overage problem; the uer aignment repone time i upper bounded [4], [4], [42]. Struturally, a overage problem i a peial, impler ae o a min/max problem; the irt ha a predeined bound, whih i additionally minimized in the eond. Intuitively, uh problem an be applied in enario where ervie guarantee or a ertain maximal repone time will be provided and paid. In ontrat, laial FLP allow minimizing the average repone time below the lowet poible repone time bound. 2.4 Solution Approahe A ouple o heuriti were propoed olving related problem whih are variant o the NP-hard apaitated FLP [28]. No work o ar ued olver to obtain olution (or nonrelaxed problem) and ull enumeration are known or mall intane limited to open ive ailitie [2]. A greedy dropping heuriti ueively remove rom the et o andidate that reoure whih inreae the repone time by the mallet amount [55]. Greedy adding heuriti ueively add reoure, whih dereae the repone time by the larget amount [2], [], [2]. Another heuriti probabilitially elet et hange o ued reoure [2] or perorm a breath-irt-earh through neighbouring olution where two olution are neighbour i their et o ued reoure dier in one element [2], [2]. Suh heuriti an be toked in loal optima and to mitigate thi drawbak meta-heuriti are ued a a upertruture [2], [], [2], [47], [55]. Thee meta-heuriti typially reine previouly generated initial olution, whih are obtained randomly or by ombining exiting olution. The hope i that among the ound loal optima, one olution i very loe to the global optimum but without any guarantee. In ontrat, we obtain global optima. Thi i an important tep or heuriti development a only thi enable a lear judgement o heuriti auray; their olution gap to the global optimum.

5 5 Other [53], [55] may ahieve near optimal olution by uing optimization tehnique like branh-and-bound and utting plane but their olution have unknown optimality gab. In ummary, either optima or mall input or olution with unknown optimality gap are obtained. Thi motivated our work on inding near-optimal olution with a numerially very mall optimality gap. Liu et al. [39] and Wendell et al. [56] preent ditributed algorithm or their global Geographial Load Balaning problem by deompoing it into eparate ubproblem olved by all lient. Thee ubproblem onverge to the optimal olution only i they are exeuted in everal ynhronized round in whih aignment and utilization inormation are exhanged among all lient. Both paper tate that thi ditributed algorithm would obtain optimal olution ater than gathering everything to a entralied olver. However, we believe that eah round a ommuniation delay i introdued when ending update inormation among all lient; they had ignored thi delay in their evaluation. The reulting total delay over all round i likely larger than ommuniating with a entralied oordinator. In addition, our p-median Faility Loation Problem ha a global ontraint on the maximal ued reoure preventing it to be eaily eparated into ubproblem. We oberved that problem intane were olved only exemplary o ar [2], [], [2], [38], [39], [4], [47], [47], [56]. Conequently, the average perormane o thee olution approahe i hard to predit. We go beyond thi by undertaking a tatitial perormane evaluation. We randomly vary our input data and veriy the tatitial relevane o our inding. 2.5 Geographial Load Balaning A ytem or Geographial Load Balaning (GLB) omprie two part: The deiion part elet appropriate erver, ite, or Virtual Mahine or requet o a ertain origin the previou etion onider them. Thi etion oue on the realiation part, whih gather monitoring inormation and implement eletion. Dierent middleware had been propoed [24], [56], [57], [58] whih are hared between appliation. In thi way, eah appliation beneit rom intane o the other appliation running at divere ite by haring monitoring inormation uh a lateny to erver or to utomer. They realie requet aignment, e.g., to loe-by or low utilied erver by either oniguring the Domain Name Sytem (DNS) or are expliitly queried ahead a requet end. Slightly dierent, Cardellini et al. [5] propoe redireting requet to dierent ite to balane the load. Poliie range rom redireting all, only larget, or only group requet to eleting ite baed on round-robin, ite utiliation, or onnetion propertie. While our paper oue on olving our problem eiiently, GLB ytem an be enhaned by integrating our problem. 3 PROBLEM Thi etion irt ormalie our enario model and then detail on pratial realiation. Aterward thi etion diue problem onvexity and propoe a problem lineariation minimizing the maximal lineariation error. a) Figure 6: Bipartite graph o a aility loation problem (a); time-in-ytem untion at eah aility (b) and, alternatively, piee-wie linearied untion (). Table : Model variable Input ontant: G = (V, E) Bipartite graph with V = C F, C F = with lient node C and aility node F l R > Round trip time between and µ R > Servie rate a apaity at λ R > Arrival rate a demand at T µ R > Time in queuing ytem (TiS) α µ, β µ -th baepoint T µ (α )=β o T µ Deiion variable: x R > Aignment in demand unit y {, } Indiator i i opened (= ) z [, ] Weight o -th baepoint at Helper variable: Λ; Λ total Λ= λ ; at : Λ = x τ R > Suiient mall value ρ R > = λ /µ Sytem utilization 3. Model Our enario i ormalized a a apaitated p-median aility loation problem [2]. A bipartite graph G = (C F, E) ha two type o node: Client ( C) and ailitie ( F ). Client orrepond to loation where uer requet low enter the network. Failitie repreent andidate loation to exeute the appliation, e.g., data entre. More preiely, a (ompute) reoure reer to a hot at uh a data entre exeuting the appliation. Fig. 6a how uh a graph. The geographially 4 ditributed demand i modelled by the requet arrival rate λ or eah lient. Computing apaity i modelled a the requet erving rate µ or eah aility. The round trip time l i the time to end data rom to and bak. Table lit all variable. Our irt problem ormulation reapitulate the known p-median problem P(G, λ, µ, p): min x l (objetive) (4).t. x λ x = λ, (demand) (5) 4. More preiely, the requet arrival and ervie point are topologially ditributed; the round trip time o a path between two point only roughly mathe it geographially ditane. We ue geographially or a onvenient explanation. b) )

6 x y µ, (apaity) (6) y = p (limit) (7) The ormulation ontain two deiion variable: x R deribe whih part o requet rate λ i aigned to whih ; y {, } deribe i loation i ued or not. The objetive i to minimize the average repone time; but without modelling the queuing delay and ervie time at ailitie, the repone time only onit o the round trip time. The RTT i minimized while all demand i erved (5) and the apaity i not exeeded (6). In addition, exatly p loation are ued (7). Thi ontraint erve two propoe. Firt, by limiting the number o loation where the appliation i developed to, the expene or the appliation provider when leaing Cloud reoure i bound. In Faility Loation variant where aility opening ot are diretly integrated the reulting total ot are unure. Seond, tating the problem with thi bound allow u to invetigate the repone time trend while allowing more and more reoure (Setion 4.3). Sine 979 the problem without apaity i known to be NP-hard [28]. Thi problem i a generalization and, thu, alo NP-hard. Until now, the repone time ha only been the round trip time. To predit the queuing time, the model i extended by queuing ytem at eah aility (Fig. 6b). There, the ervie time are exponential ditributed. The inter-arrival time at eah node are deribed by a Poion proe. The requet an be aigned to multiple aility ( x ) and, there, the individual aignment rom dierent node are aggregated ( x ). The reulting proe i alo a Poion proe, beaue plitting and joining doe not hange the underlying random ditribution. A a reult, we have a M/M/-queuing model [2]. The untion or the time in queuing ytem (TiS) ompute the proeing time plu the queuing delay (Fig. ), T µ (λ)= µ λ. Putting everything together, the orreponding ormulation o thi queuing-extended p-median problem QP(G, λ, µ, p) i: min x, y.t. x l λ }{{} average RTT + ( x ) µ x λ }{{} average TiS (8) x = λ, (demand) (9) x < y µ, (apaity) () y = p (limit) () The new objetive (8) i to minimize the average repone time, whih i the um o the average round trip time and the average time in ytem (Fig. ). Contraint (9) i the ame a Contraint (5); all demand mut be erved. Contraint () aure the teady tate (λ<µ,.. [2]) or eah queuing ytem. Finally, Contraint () mandate to ue exatly p loation, jut like Contraint (7). 3.2 Sytem deign The preented optimiation problem QP i part o a large ytem whih dipathe requet o a ertain origin to ite a deided. Example o uh a ytem range rom Geographial Load Balaning ytem (Setion 2) to our own Appliation Deployment Toolkit [34]. They monitor trai, deide aignment, and reonigure the dipathing ubytem in time period. The average arrival rate λ i the averaged number o inoming requet at router or the lat period. By olving problem QP one a period, the requet aignment 5 are deided or the next period. The deiion i realied by oniguring the dipathing ubytem, e.g., DNS, and alloating Cloud reoure aordingly. The ytem i not meant to allow a ine grained aignment deiion or eah inoming requet, e.g. at line peed. On longer term, it deide whih ite are trategially ued and how inoming requet are roughly ditributed. 3.3 Convex Optimization Previou work (Setion 2) alo onidered our objetive untion (8) but did not olve the orreponding problem optimally, exept or mall graph via ull enumeration. Thi i beaue o the non-linearity o the objetive untion whih neeitate non-linear olver. There exit a ouple o non-linear olver with dierent peialization: quadrati, onvex, or non-onvex objetive untion. By determining the omplexity la o our objetive untion, we an hooe a uitable olver, to eiiently obtain a global optimum. Thi etion irt prove the objetive untion onvexity and how that it i not impler, e.g., quadrati. Aterward, it deribe how we ued a onvex olver. Deinition. A untion g i onvex i it domain dom(g) i a onvex et and i g (x) hold x dom(g) [3]. Lemma. Funtion g= i w ig i, g:r n R, i onvex, i i: g i :R n R, w i R >, and g i i onvex [3]. Theorem. The objetive untion (8) o QP i onvex with untion T µ omputing the ojourn time in an M/M/ queuing ytem. Proo: The domain o T µ (λ)= /µ λ i the interval λ<µ enored by ontraint (); an interval i alway a onvex et. The eond derivative T µ(λ)= 2 /(µ λ) 3 i alway larger within it domain. By Deinition, T µ (λ) i a onvex untion. For a ixed in the objetive untion, <Λ =λ<µ and T µ (Λ ) i onvex. Then, the nonnegative weighted um o onvex untion Λ T µ (Λ ), Λ = x i alo onvex (Lemma ). The term remain onvex ater /Λ> i multiplied. The let term o the objetive untion i linear and alo onvex. Sine the um o two onvex untion i onvex, the objetive untion (8) i onvex. With the knowledge o a onvex objetive untion, we an ignore le eiient olver or more general, non-onvex problem. The next more eiient olver la i quadrati, whih need objetive untion o the orm x T Mx with ymmetri matrix M R n. But our objetive untion i not o thi orm, making quadrati olver inappliable. Conequently, we have to ue a onvex olver. Implementation: We hooe the optimization ramework CVXOPT [6] rom the author o [3]. QP i a mixed integer 5. The aignment x i the requet rate dipathed rom to, in hort requet aignment. 6

7 7 problem, whih i not diretly upported by CVXOPT. Continuouly relaxing the problem i not poible (Setion 2). We deompoed QP into olving multiple ubet F o F with F = p: QP(G = (C F, E), λ, µ, p, τ) = min F F, F =p { PQP((C F, E), λ, µ, τ) } (2) with the purely onvex ub-problem PQP(G, λ, µ, τ): min x l x µ x λ + x λ (3).t. x = λ, (demand) (4) x µ τ, (apaity) (5) The deompoition optimally olve QP by olving a noninteger onvex ubproblem PQP everal time or dierent oniguration o the binary variable y q ; thee variable indiate whih aility i ued. Firt, problem PQP i olved with all aility ubet Q Q with Q =t. Then, one o all problem PQP olution i eleted that ha the minimal repone time (3). In thi olution, the deiion vetor x equal the QP deiion vetor x and problem QP deiion vetor y i repreented by ubet Q, q Q : y q =. In thi way, the optimal olution or problem QP i ound. CVXOPT olve PQP by heking the domain (ontraint) and iterating toward the optimum by uing the Jaobi and Heian matrix (irt and eond order derivative) o the objetive untion (3). Hardoding uh matrie i not eaible or a large number o parameter oniguration. We want to have an automated olution obtaining thee matrie at runtime. Algebra ytem like Maxima 6 an be ued, but need a detour through another ytem and omputing derivative o multi-dimenional untion take time; or mall input, more time than olving the problem. To obtain thee matrie ater, we ound, not too urpriingly, that the truture o (3) and it derivative are the ame or dierent C, F. Exploiting thi property, we were able to dedue a ontrution rule or both matrie. Uing thi rule, we ontrut our Jaobi and Heian matrie at runtime or dierent input without notable overhead. In detail, we ontruted the Jaobi and Heian matrie rom the objetive untion (3); here, reintrodued a a onvenient opy (6), (x,..., x ). (x) := l x + Λ, Λ Λ Λ µ Λ = λ, : Λ = x (6) The Jaobi matrix (7) or one untion i a vetor a i o partial derivative or eah variable x, i=. J (x) :=(a i) i C F, Λ a := Λ(µ Λ ) + 2 Λ(µ Λ ) + l Λ (7) 6. Maxima manuel : maxima.pd Struturally, (x) i a um o term, and dierentiating (x) an be done by dierentiating the term individually and aterward umming all term up. Two type o term exit with dierent derivative (8,9). The Jaobi matrie (7) ontain eah partial derivative a, whih i the um o all g (x) and g 2(x) term. g (x) = d l x dx ij Λ g 2(x) = d dx ij Λ Λ(Λ µ ) = { l Λ i ij= = ele { i j= Λ Λ(µ Λ ) 2 ele (8) (9) Similarly, the Heian matrix (2) ontain eond-order, partial derivative whih are irt derived in x, i = diretion (row) and then in x de, j = de diretion (olumn), d C, e F. H (x) :=(a i j ) i C F, j C F, 2Λ Λ(Λ µ ) a de := Λ(Λ µ ) i =e 2 ele (2) Eah ell a de (2) i the um o g (x) and g 2 (x) rom (2, 22). d l x (x) = = (2) dx ij dx de Λ d Λ 2 (x) = dx ij dx de Λ(Λ µ ) = { 2Λ 2 Λ(Λ µ ) + 3 Λ(Λ µ ) i j= e= 2 (22) ele g g 3.4 Linear Approximation While CVXOPT olve the problem optimally, it ha to tet all ubet F, whih take time. A an alternative, the onvex objetive untion i linearied. Thi way, well reearhed linear olver an be ued to obtain olution ater Piee-wie linear Any non-linear untion g(x):r R over a inite interval [α, α m ] R an be approximated by a piee-wie linear (PWL) untion g [25]. Thi untion onit o m baepoint α,.., α,.., α m, orreponding untion value β =g(α ), and i deined in (23) or α x α +. g(x):= (x α ) (β + β ) (α + α ) + β, α x α + [, m 2] (23) A an example, let u onider the part λt µ (λ) o (8) or µ=.. Then g(ρ):=λt (λ)= λ / λ i our example untion to linearie. Fig. 7a how g and two dierent lineariation g and g 2. The horizontal axi how the arrival rate and the vertial axi how the orreponding TiS. Fig. 7b how the abolute dierene between g and either lineariation g or g 2. Thee dierene denote the lineariation auray: The maller the dierene are, the tighter the PWL untion reemble the original untion. We ue the maximum o all abolute dierene ɛ g, deined in (24), to meaure the

8 8 TiS TiS g( ) = =T w : ( ) ~g ( ), imamoto ~g 2 ( ), uniorm ~g ( ) g( ), imamoto ~g 2 ( ) g( ), uniorm arrival rate Figure 7: The top plot how an example untion g with two poible lineariation with the ame number o baepoint. The bottom plot how abolute dierene between the lineariation and g. Imamoto lineariation ha a maller maximum dierene. lineariation auray. We eek baepoint α i that minimize thi error. ɛ g := max g(x) g(x) (24) x [α, α m ] Given a et o baepoint reulting in a ertain error, thi error i redued by plaing an additional baepoint at a point where the abolute dierene equal the error. However, more baepoint alo inreae the number o neeary variable or the optimization, whih inreae earh pae and olving runtime. Some untion are hard to approximate with linear egment, e.g., untion with large eond-order derivative value. I their value are large within the lineariation interval [α, α m ], the error will be large. The TiS untion aymptote lim λ µ T µ (λ)= approximated by linear egment reult in uh a larger error. One poible ontrol knob i to adjut the interval α m <µ. But thi alo introdue an artiiial apaity limit: Small value (e.g. α m =.8µ) reult in ewer requet erved than poible (.. Fig. 3b S 2 ). Conequently, the total arrival rate or whih olution are eaible to obtain i maller, λ /p α m <µ with p ued reoure. Both PWL untion in Fig. 7b, uniorm and imamoto, have the ame number o baepoint but at dierent poition. A hown, uniormly ditributing the baepoint an dramatially inreae the error ( g ). In ontrat, the grey baepoint have mall error ( g 2 ). Thoe baepoint were omputed by our algorithm detailed in Setion We evaluate the irt two ontrol knob, the number o baepoint and the lineariation interval upper bound, in Setion 4.. For the third ontrol knob, the baepoint poition, our algorithm determine baepoint with low error Lineariation algorithm Our algorithm obtain baepoint or onvex untion with low error. It i an extended verion o Imamoto algorithm [3]. Imamoto algorithm iteratively reine m baepoint by moving them individually along the abia to redue the error ɛ g. Eah baepoint adjutment along the abia, α neu =α alt +, i omputed rom the d baepoint irt-order derivative g(α dα alt alt ) and the interbaepoint ditane d =α α +. The paper [3] tatement i that the algorithm ompute baepoint whih have the maximal lineariation auray or the given number o ued baepoint. However, the algorithm run in numerial iue rendering the algorithm uele or ome onvex untion. When ixing 7 them, it annot be guaranteed any more that the reulting baepoint orm an lineariation with maximal auray (minimal error). But it i till very mall till a good and at option to linearie onvex untion. More in detail, we extend Imamoto algorithm [3] and ixed the ollowing two ae: Firt, the algorithm iteratively adjut the urrent et o baepoint o that the error i ueively redued. Thee adjutment are weighted in order to allow gradually iner hange o that the error ater eah iteration onverge to the minimum error in theory. In pratie, loating-point auray i limited and ometime value are too mall, hange not applied, and the algorithm iterate ininitely. We ixed that by additionally aborting i no urther baepoint hange are oberved. Seond, or peial untion the algorithm terminate with a diviion by zero. The aue i omputing a baepoint α adjutment depending on original untion derivative g (α )= d dα g(α ). The diviion by zero our i the dierene o two value g (α i ) numerially equal zero, i : g (α ) g (α i ) =. That i i g reemble a linear untion over ome interval. We ixed that by removing all baepoint α with g (α )=g (α i ), i< and inerting thee baepoint between baepoint whoe g (α) value dier rom eah other. Thi aure that the error never inreae or an be redued: For thoe interval o the untion whih are nearly linear, a lineariation over the whole interval yield a low error; thu, removing baepoint within thi interval ha little impat. Inerting thee baepoint at another non-linear part o the untion improve the lineariation auray a the PWL untion beome tighter Formulation o linearied problem Thi etion deribe the problem reormulation uing a PWL untion. From exiting alternative [45], we ued a Speial Ordered Set (SOSk) o type k = 2 (SOS2) []: In a et o ontinuou variable, at mot k o them, adjaent to eah other, may take non-zero value. Current linear olver diretly upport SOS2. A PWL untion ỹ= g(x) i repreented by a et o m ontinuou deiion variable z with a SOS2(z,.. z,.. z m ) ontraint and a onvex ombination = z. Thi way, two adjaent value um up to = z +z +. Thee value are then ued a weight or the baepoint (α, β ) obtained previouly by the lineariation proe (Setion 3.4.). Thi way, the weighted um o all baepoint reult in the piee-wie linear problem, x= z α, y= z β. 7. Thi paper extended verion detail our improvement o Imamoto algorithm.

9 9 Uing thi repreentation, we linearie the onvex part o the objetive untion (8) Λ T µ (Λ ), Λ = x, and ubtitute it by orreponding weighted baepoint um, the SOS2 ontraint, and a onvex ombination. Firt, we ou on one aility loation and then add indexe to model all loation. For loation, untion T µ ompute the TiS (25). With it linearied verion T µ (26) the onvex part o the objetive untion, Λ T µ (Λ ), beome Λ β z. A Λ depend on deiion variable x, multiplying x with z turn the replaement term to be quadrati; only untion T µ wa linearied, not the whole objetive untion. However, having a linear and not quadrati objetive untion would redue problem omplexity and peed up olving. The quadrati term Λ β z need to be replaed with an equivalent linear term. Thi i ahieved by moving the weight Λ into untion T µ, whih beome T w µ (27). Uing T w µ baepoint will tranorm the quadrati into the linear term β z ; the ordinate baepoint are now already weighted. Sine the other part o the objetive untion were linear, the whole objetive untion i now linear. T µ (x) = µ x = y, with x=λ (25) T µ : x = z α, ỹ = z β (26) T w µ (x) = xt µ (x) = x µ x = y, with x=λ (27) To model all loation, index i added or eah aility loation orming the deiion variable z and baepoint variable α, β. The linearied verion QP(G, λ, µ, p, α, β ) i hene: min x l + β z (28) x,y,z Λ Λ.t. x = λ, (demand) (29) x = α z, (apaity) (3) z =, SOS2(z,.. ), (pwl) (3) x y, (ore lip) (32) y = p (limit) (33) The demand-weighted TiS i repreented by term β z and the orreponding arrival rate at i α z = Λ = x ; the new apaity ontraint (3). Thi apaity ontraint alo impliitly aure that the queuing ytem i in a teady tate through the upper bound o the linearization interval α (m ) < µ; τ rom the old ontraint () beome obolete. The earh pae o QP onit o F binary and F C real variable. In addition QP ha m F real, retrited SOS variable. I both problem were linear we ould gue that olving the eond problem QP take longer than QP beaue the earh pae i larger. However, linear problem are uually olved ater than non-linear problem. Whih problem i olved ater? The anwer i not obviou, e.g., QP i linear but ha a larger earh pae. Their runtime are experimentally evaluated in Setion 4.2. The maximal lineariation error (34) o the objetive untion (28) depend on the error o the linearied part, whih i the um o ued reoure, y =, and their maximal lineariation error ɛ T w µ. Λ, y = ɛ T w µ p Λ max T w {ɛ T w} (34) The lineariation auray drop i more reoure are allowed to open (p). To maintain the ame lineariation auray while doubling p the lineariation error ɛ T w ha to be halved. Thi an be ahieved by uing more baepoint or the lineariation. Even i Fig. 8 indiate that le than twie the baepoint are neeary, inreaing the number o baepoint and, hene, inreaing the problem earh pae inreae the runtime. 4 EVALUATION Thi etion ha our part. Firt, it preent dierent TiS untion lineariation to ind a balane o two onliting goal: Small objetive untion approximation error and ew baepoint (m) or at omputation. Seond, olution o the onvex and linear problem (QP v. QP) are ompared or dierent real network. Third, the trade-o between the number o ued loation and reulting repone time i diued. Finally, appliation and network propertie are preented or whih onidering the QD yield better repone time than ignoring QD ( QP v. P). 4. Weighted TiS Linearization Thi etion deribe how we obtain the onrete baepoint or T w µ (λ) (27) in the evaluation. For thi, we how a impliiation with one et o baepoint adapted at runtime or dierent µ value. Aterward, we diu the trade-o between a at olving time and low approximation error. Funtion T w µ (λ) depend on µ and need individual linearization or dierent µ; let α µ, β µ be their baepoint (35). Alternatively, untion T w (ρ) (36) i independent o µ with orreponding baepoint α, β. Funtion T w µ an be rewritten a T w ( λ /µ)=t w µ (λ) (37) and the orreponding baepoint an be rewritten imilarly: : α µ =µα, β µ =β. T w µ (λ) = λ µ λ : T w (ρ) = ρ ρ : T w ( λ µ ) = λ /µ : λ /µ α µ z =λ, β µ z =T w µ (λ) (35) α z =ρ, β z =T w (ρ) (36) α λ z = µ, β z =T w ( λ µ ) (37) A the ordinate baepoint β remain unhanged, the baepoint approximation error i alo not aeted. With thi handy tranormation, we only need to preompute baepoint o T w intead o baepoint et o T w µ or eah dierent µ in the model, whih peed up the model etup proe. In the remaining etion, we invetigate the trade-o between a at olving time and a low-error lineariation. For the irt, we need to minimize the number o deiion variable z or, equivalently, the number o baepoint ued

10 ab. approx. error o ~T w m =:8 ( m =4:) m =:9 ( m =9:) m =:96 ( m =24:) m =:98 ( m =49:) m =:99 ( m =99:) number o baepoint (m) Figure 8: The error o lineariing T w µ i hown depending on the number o baepoint and dierent linearization interval [,.., α m ]. or the lineariation. For the eond, we invetigate two ontrol knob (Setion 3.4): many baepoint or mall interval end α m. Fig. 8 how the error o Tw depending on the number o baepoint m or dierent α m value. We need a mall error (down the vertial axi) with mall m (let on the horizontal axi) with large α m. The latter alo artiiially limit the reoure apaity and render olving an input ineaible that ould in at be olved with larger α m. For our evaluation, we et α m =.96 and m=6 with error ɛ T w µ =2.67 a a good ompromie between the number o deiion variable, approximation error, and artiiial apaity limit. 4.2 Comparion: Convex v. Linear We hooe the ollowing truturally dierent topologie rom SndLib [44]: ta2, zib54 with many node (around 5); yuan, bwin with ew node (around ); atlanta, norway or dene network (node:edge ratio :2). All topologie are onneted. We approximate the lateny between node by their geographial ditane [32]. We aume that data entre are built at well onneted node/router, o we eleted the node with the highet degree 8 to be data entre. We et a relatively low ervie rate o µ= req. / to relet our omputationally intenive example appliation [8], [37], [54]. The ervie rate were the ame or all data entre. Uer requet arrive at all node and the arrival rate λ or uer at ite i randomly generated: Eah value i uniormly drawn rom an [, ] interval and, aterward, all value are normalized to λ =47 req. /. Thi value, together with the ervie rate µ= req. /, enure eaibility or 5 or more ailitie. We randomly generated 5 dierent realization or all arrival rate. For eah o thee 5 et o arrival rate, we onidered p [5, ] ailitie, reulting in 3 oniguration per topology. Eah oniguration wa olved uing either QP or QP. Fig. 9 ompare the olution auray (a) and olving time (b). The horizontal axi lit the group o dierent topologie and the ued number o reoure or eah group. The vertial axi in (a) how the 95% onidene interval o the mean o the average repone time o the 5 realization or eah group. Similarly, the vertial axi in (b) how the 8. For ame degree the node id i the tie breaker. 95% onidene interval o the mean runtime or eah group. The linear olution ( QP) have a imilar quality a the onvex olution (QP) whih i a reult o our eort to ind a tight linear approximation. Looking at (b), obtaining onvex olution took longer than obtaining the linear olution. However, thoe value have to be interpreted with are. Our implementation o QP ha to proe all poible ombination (F ), wherea QP beneit rom the MIP olver branh-and-ut algorithm to redue the earh pae. To ompare thi truturally dierent problem, we retrit the number o andidate ailitie to and the number o poible ombination; the major aue o the higher runtime o QP. Neverthele, the abolute runtime QP are very hort or all group depite QP larger earh pae. In onluion, we ound that QP i a very good approximation or QP: at and aurate. 4.3 Repone Time Redution Fig. 9 alo how how repone time improve when adding a reoure. We ould veriy two eet dereaing the repone time: Firt, uing more loation allow better load balaning, whih redue the queuing delay. Thee redution are larger or highly utilized loation than or le utilized one. Seond, more loation allow nearer loation, reduing the round trip time. In onluion, the average repone time o QP(..., p) dereae monotonially in number o reoure. Then, ervie provider earning more money by onneting uer with lower repone time ae a diminihing return. At one breaking point p the ot or adding a reoure will exeed the additionally earned money. Thi point depend on the topology, ervie time, and ervie monetization. By uing QP with dierent p value, the ervie provider an determine p in advane to avoid proit lo. For a loer look, Fig. how not only the average repone time but alo the time in ytem and round trip time grouped along the horizontal axi the ame way like in Fig. 9. We an trae the two eet o repone time redution: Firt, load balaning aro more reoure redue the queuing delay and with it the time in queuing ytem. Seond, the round trip time i redued a nearer reoure are ued. 4.4 Conidering Queueing Delay Thi paper preent a reinement o the aignment problem (P in (4)) by onidering queuing delay. Thi reinement inreae the omplexity, auray, and runtime. Only a igniiantly lower repone time would make thee drawbak worthwhile; or intane, mall queuing delay ompared to large round trip time will render the reinement unnotieable. Thi etion invetigate multiple enario ator inluening queuing delay and judge the reinement gain by omparing olution repone time obtained by either ignoring or onidering queuing delay. For thi, we perorm a eond experiment with a lightly dierent oniguration a in Setion 4.2. The irt experiment howed that QP i a very good and at replaement or QP. Fouing on QP allow u to evaluate more enario variation in a reaonable time than would be poible with QP.

11 Repone time [m] mean runtime [] onvex linear onvex linear atlanta dn-bwin di-yuan norway ta2 zib54 Topology, number o reoure Figure 9: Solution quality (a) and olving time (b) or onvex QP and it linearization QP. Etimated mean onidene interval at 95% onidene level. Time [m] Repone time Time in ytem Round trip time atlanta dn-bwin di-yuan norway ta2 zib54 Topology, number o erver Figure : Round trip time, time in ytem, and round trip time or dierent topologie and number o ued erver. Etimated mean onidene interval at 95% onidene level Coniguration We vary oniguration by our ator 9 ˆµ, ˆD, ˆρ, Ĝ. The ervie rate ˆµ relet how omputationally intenive the appliation i. A mall ervie rate mean a high proeing time. For the ame arrival rate, queuing delay are higher or maller ervie rate. For larity, we aume homogeneou reoure, ame ervie time at eah loation, ˆµ=µ,. Dierent level o ˆµ repreent dierent appliation type ranging rom at web erver with a hort proeing time up to omputationally intenive appliation, ˆµ=,; ; ; ; req. /. Unlike the ervie rate, the arrival rate λ are not homogeneou but randomly ditributed. Thi enable u 9. The ˆ indiate the ator under invetigation.. We diard an additional minimization potential o heterogeneou reoure to impliy the omparion: When aigning demand, a arer but ater reoure enable trading o a larger RTT or a maller QD+PT. By doing o, the repone time i urther redued. to invetigate dierent pattern or patially ditributed load, e.g., lutuation or loal hot pot. Let ˆλ be the targeted mean arrival rate. For thi, we hooe three dierent random ditribution ˆD or our eond ator. Firt, a imilar load aro all node with mall lutuation i repreented by a narrow normal ditribution: N(mean=ˆλ, td.dev.=ˆλ/2)=n. Seond, a largely lutuating load around an average load per node i repreented by a wide normal ditribution: N(ˆλ, ˆλ)=N 2. Third, heavy variation auing loal hot pot are repreented by an exponential ditribution: Exp(ˆλ). For eah node the arrival rate λ i drawn rom ˆD, where negative value are apped to zero, λ = max{, X}, X ˆD {N, N 2, Exp}. We invetigate 5 dierent uh realization or eah topology. The third ator ˆρ r relet the average reoure utilization. Highly utilized reoure have high QD, and olution are very imilar whether or not the queuing delay i onidered.

12 2 5 ( ˆT =Colt) 5 ( ˆT =Forthnet) 4 4 repone time [m] 3 2 repone time [m] 3 2 Exp w Exp w/o N 2 w N 2 w/o N w N w/o 8% 5% 8% 5% 8% 5% 8% 5% 8% 5%..... Servie rate ˆµ, reoure utilization ˆρ r Exp w Exp w/o N 2 w N 2 w/o N w N w/o 8% 5% 8% 5% 8% 5% 8% 5% 8% 5%..... Servie rate ˆµ, reoure utilization ˆρ r Figure : Average repone time a a untion o ator ombination (ˆµ, ˆρ ) or dierent demand ditribution ( ˆD) with 95% onidene interval. Let (and right) omparion or topology ˆT =Colt (and Forthnet, repetively). To enore the ˆρ r level, we limit the number o utilied reoure p deined later. The lat ator, with dierent topologie Ĝ we repreent trutural dierene like the diameter or ratio between the round trip time and queuing delay, or intane, pare graph have a larger diameter and higher round trip time than dene graph. We eleted 4 out o 524 topologie rom dierent oure: ndlib [44], topology zooetion 4.4. [35], kingtrae 2 [27]. The eletion onider three ategorie or the number o node, edge, and diameter to eliminate roughly imilar topologie. In eah topology, the bet onneted node 3 were marked a andidate reoure loation F, F = min { N, }. Let p min F be the number o reoure at leat neeary to handle all demand. Having more demand uing more reoure mean le reedom or loation hoie: or intane, i p min = F all reoure are ully utilized and only thi deiion i poible. We et p min =.3 F to allow enough reedom. Then, ˆD target arrival rate ˆλ pmin i deined aordingly, ˆλ=ˆµ N. Atually uing p=p min reoure reult in a very high reoure utilization p r ; allowing more reoure redue the utilization. For our third ator, the reoure utilization ˆρ r =.97;.8;.67;.5;.375, we et orreponding p value ahieving (roughly) the targeted erver utilization, ˆp=.3 /ˆρ r F = a F, a=.3;.37;.45;.6;.8.. Round trip time were approximated by geographial ditane [32]. 2. A pare matrix peiie point to point latenie. Some were only available in one diretion. We aume the ame lateny or the oppoite diretion; otherwie thoe node had to be diarded. 3. Highet degree irt; node ID a the tie breaker Reult Summariing, eah 5 ombination o the our ator (ˆµ, ˆD, ˆρ, ˆT ) were randomied by 5 demand realiation, reulting in 52,5 dierent oniguration or whih problem P (without onidering queuing delay) and problem QP (with onidering queuing delay) are olved. The quality metri i the average repone time omputed by QP exat objetive untion (8). The dierene o the maller repone time obtained with QP than with P meaure the improvement when onidering queuing delay. A larger dierene mean QP aignment are uperior to P aignment. A P i a impliiation o QP, P repone time annot be maller than QP, and never wa. Fig. how repone time a a untion o two ator, ervie rate ˆµ and reoure utiliation ˆρ or two eleted topologie ˆT =Colt, Forthnet. Repone time obtained by P (by QP) are marked by a mall irle (by a ro, repetively). Repone time or the three ditribution have dierent olour. Eah ingle data point orrepond to the average repone time with 95% onidene interval o 5 realiation or one ator ombination. In general, Fig. how P repone time where never maller than QP repone time, ometime igniiant larger (the repone time axi ha logarithmi ale). Conidering queuing delay i beneiial under three ondition. Firt, or maller ervie rate ˆµ. the dierene grow: Small ervie rate and, equally, large ervie time reult in large queuing delay being large ompared to the round trip time. By onidering queuing delay, QP aign demand aro reoure where P aign a muh demand a poible to the nearet reoure, only minimiing the round trip time. On the other hand, or large ervie rate the oppoite i true; the queuing delay are very mall and, in omparion to the round trip time, marginally mall.

13 3 A a reult, in thee oniguration both olution dier only marginally. In onluion, QP olution are better or omputation-intenive appliation and queuing delay an be ignored or appliation with at proeing time. Seond, or higher reoure utiliation ˆρ.5, the dierene between P and QP grow: Highly utilied ailitie have long queuing delay, taking thee delay into aount allow hiting the load to le utilied ailitie. On the other hand, mall queuing delay redue the impat o uh load hit. In onluion, QP obtain better olution when uing ew reoure. Third, i demand trongly varie geographially ( ˆD {Exp, N 2 }) the dierene between P and QP are larger than or a moothly varying demand ( ˆD=N ). Without onidering queuing delay, loal demand hotpot reult in aignment to nearby reoure. Thoe reoure are ully loaded beore the next reoure arther away i ued. In onluion, QP ha better olution or large geographial demand variation. 5 CONCLUSION We extend previou work by optimally olving the aignment problem QP, a aility loation problem (FLP) with integrated queuing ytem. We propoed problem QP a a lineariation o QP. Solving QP yield mall optimality gap in a hort time. In our enario o adapting the reoure alloation at geographially ditributed ite, uh a wit reation i important. It allow to reat immediately to an ever hanging environment inluding demand lutuation or network ongetion. We howed that adding more and more reoure will at one point redue the uer expeted repone time only marginally. With our work, the appliation provider an determine thi point in advane and an alloate reoure aordingly. We perormed a large-ale experiment and traed down network and appliation propertie where integrating the queuing ytem into the FLP improve olution. Thi ould guide other reearher or appliation provider whether the omplex QP i neeary to apply or the impler P erve the need o their enario. The imple M/M/-queue model an be replaed with more ophitiated queuing model a long a (i) the interarrival time are deribed by a Poion proe and (ii) the queuing delay untion i onvex. For model with a dierent inter-arrival time proe, plitting and joining beome muh more ompliated. The linearied problem even upport non-onvex queuing delay untion, but or uh model the preented algorithm or obtaining baepoint i no more appliable. Finally, preented tehnique an be ued beyond our ue ae. The aignment problem QP i part o the broader amily o FLP with onvex ot untion. We think or mot, maybe all o them, a imilar good and at problem lineariation an be ormulated by reuing our piee-wie linear problem ormulation and determining the lineariation baepoint with the purpoed algorithm. REFERENCES [] Amazon Web Servie. [2] R. Aboolian, O. Berman, and Z. Drezner. The multiple erver enter loation problem. Ann. Oper. Re., 67(): , mar 29. [3] S. Agarwal, J. Dunagan, and N. Jain. Volley: Automated data plaement or geo-ditributed loud ervie. In 7th USENIX Con. Networked Syt. De. Implement. (NSDI ), 2. [4] A. N. Algorithmi, A. To, and N. Loation. An algorithmi approah to network loation problem. The p-median. SIAM J. Appl. Math., 37(3):539 56, 979. [5] M. Aliherry and T. Lakhman. Network aware reoure alloation in ditributed loud. In 22 Pro. IEEE INFOCOM, page IEEE, mar 22. [6] M. Anderen and L. Vandenberghe. CVxoPt - a ree otware pakage or onvex optimization. [7] M. Bagaa, T. Taleb, and A. Kentini. Servie-aware network untion plaement or eiient trai handling in arrier loud. In 24 IEEE Wirel. Commun. Netw. Con., page IEEE, apr 24. [8] S. Barker and P. Shenoy. Empirial evaluation o lateny-enitive appliation perormane in the loud. In t Annu. Con. Multimed. Syt., page 35. ACM Pre, 2. [9] M. Bauer, S. Braun, and P. P. Domhitz. Media Proeing in the Future Internet. In Pro. th Würzbg. Work. IP Viion Futur. Gener. Network, page 3 5, 2. [] E. Beale, M. Landowne, and J. A. Tomlin. Speial ailitie in a general mathematial programming ytem or non-onvex problem uing ordered et o variable. Oper. Re., 69( ):99, 97. [] O. Berman and Z. Drezner. The multiple erver loation problem. J. Oper. Re. So., 58():9 99, jan 26. [2] G. Bolh, S. Greiner, H. De Meer, and K. S. Trivedi. Queueing network and Markov hain: Modeling and Perormane Evaluation with Computer Sinee Appliation. Wiley-Interiene, 25. [3] S. Boyd and L. Vandenberghe. Convex optimization. Cambridge Univerity Pre, 24. [4] D. Cai and S. Natarajan. The Evolution o the Carrier Cloud Networking. In Seventh Int. Symp. Serv. Syt. Eng., page IEEE, mar 23. [5] V. Cardellini, M. Colajanni, and P. Yu. Geographi load balaning or alable ditributed Web ytem. In Pro. 8th Int. Symp. Model. Anal. Simul. Comput. Teleommun. Syt., page IEEE Comput. So, 2. [6] K. Churh, A. Greenberg, and J. Hamilton. On Delivering Embarraingly Ditributed Cloud Servie. In HotNet, 28. [7] M. Claypool and K. Claypool. Lateny and player ation in online game. Commun. ACM - Entertain. Netw., 49():4 45, 26. [8] A. Cázár, W. John, M. Kind, C. Meirou, G. Pongráz, D. Staeen, A. Taka, and F. J. Wetphal. Uniying loud and arrier network: Eu Fp7 Projet UniFy. In 6th Int. Con. Util. Cloud Comput., page IEEE/ACM, 23. [9] T. Cuinotta, K. Oberle, M. Stein, P. Domhitz, and S. Mullender. Run-time Support or Real-Time Multimedia in the Cloud. In 2nd Int. Work. Real-Time Ditrib. Comput. Emerg. Appl. (REACTION 23), number, 23. [2] T. Drezner and Z. Drezner. The gravity multiple erver loation problem. Comput. Oper. Re., 38(3):694 7, mar 2. [2] Z. Drezner and H. W. Hamaher. Faility loation: appliation and theory. Springer, 24. [22] P. Endo, A. de Almeida Palhare, N. Pereira, G. Gonalve, D. Sadok, J. Kelner, B. Melander, and J.-E. Mang. Reoure alloation or ditributed loud: onept and reearh hallenge. IEEE Netw., 25(4):42-46, 2. [23] A. Fiher, J. F. Botero, M. T. Bek, H. de Meer, and X. Heelbah. Virtual Network Embedding: A urvey. IEEE Commun. Surv. Tutorial, PP(99): 9, 23. [24] M. J. Freeman, K. Lakhminarayanan, and David Maziere. OAiS: Anyat or Any Servie. In Networked Syt. De. Implement., 26. [25] B. Geißler, A. Martin, A. Mori, and L. Shewe. Uing Pieewie Linear Funtion or Solving MinLp. In Mix. Integer Nonlinear Program., volume 54 o The IMA Volume in Mathemati and it Appliation, page Springer New York, 23. [26] H. Goudarzi and M. Pedram. Geographial Load Balaning or Online Servie Appliation in Ditributed Dataenter. In 6th Int. Con. Cloud Comput., page IEEE, 23. [27] K. P. Gummadi, S. Saroiu, and S. D. Gribble. King : Etimating Lateny between Arbitrary Internet End Hot. In 2nd ACM SIGCOMM Work. Internet Mea. (IMW 2), page p. 5 8, 22.

14 4 [28] O. Hakimi and S. L. Kariv. An Algorithmi Approah to Network Loation Problem. Ii: The p-median. SIAM J. Appl. Math., 37(3):539 56, eb 979. [29] F. Hao, T. V. Lakhman, S. Mukherjee, and H. Song. Enhaning dynami loud-baed ervie uing network virtualization. t ACM Work. Virtualized Inratrut. Syt. Arhit., 4():37, 29. [3] A. Imamoto and B. Tang. Optimal Pieewie Linear Approximation o Convex Funtion. In World Congr. Eng., page 22-25, 28. [3] A. Ihii and T. Suzumura. Elati Stream Computing with Cloud. In 4th Int. Con. Cloud Comput., page IEEE, jul 2. [32] S. Kaune, K. Puep, C. Leng, A. Kovaevi, G. Tyon, and R. Steinmetz. Modelling the internet delay pae baed on geographial loation. In Parallel, Ditrib. Network-baed Proe. 29 7th Euromiro Int. Con., page 3-3. IEEE, 29. [33] M. Keller and H. Karl. Repone Time-Optimized Ditributed Cloud Reoure Alloation. In Work. Ditrib. Cloud Comput., 24. [34] M. Keller, M. Peuter, C. Robbert, and H. Karl. A topology-aware Adaptive Deployment Framework or Elati Appliation. In 7th Int. Con. Intell. Next Gener. Network, page IEEE, ot 23. [35] S. Knight, H. X. Nguyen, N. Falkner, R. Bowden, and M. Roughan. The Internet Topology Zoo. IEEE J. Sel. Area Commun., 29(9): , ot 2. [36] M. Kwon. A tutorial on Network Lateny and it Meaurement. IGI Glob., 24. [37] Y. Lee, K. Chen, H. Su, and C. Lei. Are all game equally loud-gaming-riendly? an eletromyographi approah. In Pro. IEEE/ACM NetGame 22, page 4 9, 22. [38] M. Lin, Z. Liu, A. Wierman, and L. L. H. Andrew. Online algorithm or geographial load balaning. In Int. Green Comput. Con., page, 22. [39] Z. Liu, M. Lin, A. Wierman, S. Low, and L. Andrew. Greening Geographial Load Balaning. IEEE/ACM Tran. Netw., 23(2):657-67, 25. [4] V. Marianov and D. Serra. Probabiliti maximal overing loationalloation model with ontrained waiting time or queue length or ongeted ytem. J. Reg. Si., 38(3):4 424, 996. [4] V. Marianov and D. Serra. Loation Alloation o Multiple-Server Servie Center. Ann. Oper. Re., ( 4):35-5, 22. [42] M. Moghada and T. Kakhki. Maximal overing loation-alloation problem with M/m/k queuing ytem and ide ontraint. Iran. J. Oper. Re., 2(2): 6, 2. [43] K. Oberle, D. Cherubini, and T. Cuinotta. End-to-end ervie quality or loud appliation. In Eon. Grid, Cloud, Syt. Serv., volume 893 LNCS, page Springer, 23. [44] S. Orlowki, M. Pioro, A. Tomazewki, and R. Wealy. SNdlib Survivable Network Deign Library. Tehnial report, Konrad- Zue-Zentrum ür Inormationtehnik Berlin, 27. [45] M. Padberg. Approximating eparable nonlinear untion via mixed zero-one program. Oper. Re. Lett., 27(): 5, aug 2. [46] S. Pandey, A. Barker, K. K. Gupta, and R. Buyya. Minimizing Exeution Cot when Uing Globally Ditributed Cloud Servie. In 24th IEEE Int. Con. Adv. In. Netw. Appl., page Ieee, 2. [47] S. H. R. Paandideh and S. T. A. Niaki. Geneti appliation in a aility loation problem with random demand within queuing ramework. J. Intell. Manu., 23(3):65 659, may 2. [48] M. Shar, T. Voith, W. Roome, B. Gaglianello, M. Steiner, V. Hilt, and V. K. Gurbani. Monitoring and abtration or networked loud. In 6th Int. Con. Intell. Next Gener. Network, page IEEE, ot 22. [49] B. Shah and B. H. Trivedi. Improving Perormane o Mobile Agent Baed Intruion Detetion Sytem. Fith Int. Con. Adv. Comput. Commun. Tehnol., page , 25. [5] A. L. Stolyar and B. Lab. Shadow-Routing Baed Dynami Algorithm or Virtual Mahine Plaement in a Network Cloud. In INFOCOM, nd IEEE Int. Con. Comput. Commun., page , 23. [5] T. Taleb. Toward arrier loud: Potential, hallenge, and olution. IEEE Wirel. Commun., 2(3):8-9, jun 24. [52] T. Taleb and A. Kentini. Follow Me loud: Interworking ederated loud and ditributed mobile network. IEEE Netw., 27(5):2 9, 23. [53] N. Vidyarthi, S. Elhedhli, and E. Jewke. Repone time redution in make-to-order and aemble-to-order upply hain deign. IIE Tran., 4(5): , mar 29. [54] Z. Wan. Cloud Computing inratruture or lateny enitive appliation. In 2th Int. Con. Commun. Tehnol., page IEEE, nov 2. [55] Q. Wang, R. Batta, and C. M. Rump. Algorithm or a Faility Loation Problem with Stohati Cutomer Demand and Immobile Server. Ann. Oper. Re., (-4):7 34, mar 22. [56] P. Wendell, J. W. Jiang, M. J. Freedman, and J. Rexord. DOnaR : Deentralized Server Seletion or Cloud Servie. In SigComm, volume 4, page , 2. [57] B. Wong. CloetNode. om: an open ae, alable, hared geoat ervie or ditributed ytem. ACM SIGOPS Oper. Syt. Rev., page 62 64, 26. [58] B. Wong, A. Slivkin, and E. Sirer. Meridian: A lightweight network loation ervie without virtual oordinate. In ACM SIGCOMM Comput. Commun. Rev., volume 35, page ACM, 25. [59] Q. Zhang, Q. Zhu, M. F. Zhani, and R. Boutaba. Dynami Servie Plaement in Geographially Ditributed Cloud. In 2nd Int. Con. Ditrib. Comput. Syt., page IEEE, jun 22. Matthia Keller reeived hi diploma degree in omputer iene rom the Univerity o Paderborn, Germany. He i urrently working a a reearh aoiate in Computer Network group in Univerity o Paderborn. He oue on adaptive reoure alloation aro wide area network, deigned a ramework, and reated a prototype tetbed. Previouly, he worked at the Paderborn Centre or Parallel Computing a a reearh aoiate and at an international otware development ompany a a otware engineer. Holger Karl reeived hi PhD in 999 rom Humboldt Univerity Berlin; aterward he joined Tehnial Univerity Berlin. Sine 24, he i Proeor or Computer Network at Univerity Paderborn. He i alo reponible or the Paderborn Centre or Parallel Computing and ha been involved in variou European and national reearh projet. Hi main reearh interet are wirele ommuniation and arhiteture or the Future Internet.