TTIC/CMSC 350 Mathematical Toolkit Sprig 203 Madhur Tulsiai Lecture 5: April 7, 203 Scribe: Somaye Hashemifar Cheroff bouds recap We recall the Cheroff/Hoeffdig bouds we derived i the last lecture idepedet 0/ radom variables {X i } ad E [Z] = µ The we have [ e σ ] µ P [Z ( + σ)µ] ( + σ) (+σ) Let Z be a sum of Now, let s look at a large deviatio usig Cheroff bouds: P [Z eµ] ( e e e e = e µ ) µ Whe ( + σ) is eve larger ie ( + σ) 2e, usig Cheroff boud we would have P [Z ( + σ)µ] ( e ) (+σ)µ + σ 2 (+σ)µ 2 Permutatio routig i hypercube We ow cotiue the descriptio of the radomized routig scheme for the hypercube graph Recall that the -dimesioal biary hypercube graph is a graph with N = 2 odes where V = {0, } (x, y) E if x, y V ad x ad y differ i just oe bit Let π : {0, } {0, } be ay permutatio of the odes The goal is to sed a packet from ode x to ode π(x), simultaeously for all odes x Recall that we are workig i the sychroous model, where at each time istat we are allowed to sed at most oe packet across each edge of the hypercube If there are multiple packets waitig to cross a edge at time t, the we put them i a queue ad choose oe packet to cross that edge (the method for choosig a packet out of the queue ca be arbitrary) Also, we are iterested i oblivious routig schemes, such that the path of the packet from x to π(x) depeds oly o x ad π(x) ad ot o the destiatios of ay other packets We will prove the followig result due to Breber ad Valiat [BV8]
Theorem 2 ([BV8]) There is a radomized oblivious scheme to route packets from each x V to π[x], which takes time O() with probability 2 Ω() I the descriptio below, we idetify a packet with its source ad refer to the packet beig routed from x to π(x) as packet x We ow describe the routig scheme Scheme: The radomized routig scheme has two phases Phase : Packet x chooses a radom itermediate destiatio γ(x) {0, } ad goes to γ(x) usig the bit-fixig path Phase 2: Packet x goes from γ(x)to π(x) usig the bit-fixig path Recall that a bit-fixig path betwee i ad j is a path obtaied by flippig bits of i that are differet from j i a fixed sequece to reach j For example a bit-fixig path from i = 0000 to j = 00 i the hypercube {0, } 7 is i = 0000 000 00 000 = j A bit-fixig path is always a shortest path from i to j i the hypercube Basic idea: Sice π(x) is a permutatio, the two phases are symmetric It will be sufficiet to show that the time take i Phase is O() with high probability Let P x be the bit-fixig path from x to γ(x) ad T (x) be the total time take by packet x Also, let D(x) be the delay for packet goig from x to γ(x) ie, the time spet by packet x without movig whe it is waitig i a queue at some itermediate ode The T (x) + D(x) We wat to show that D(x) is small for every x The followig claim provides a upper boud o D(x) Claim 22 For all x {0, }, D(x) umber of paths itersectig P x = {y x : P y P x } We first aalyze the routig scheme assumig the above claim For each x, we defie the radom variable Z x Z x = {y x : P y P x } We wat to show that Z x is O() for each x We first derive a upper boud o Z x Let P x be the path (e,, e k ) The Z x x i= (# paths passig through e i ) Note that i the above descriptio the path (e,, e k ) ad its legth k are also radom Sice we do ot wat to compute ay bouds coditioed o the choice of P x, we will i fact show that with high probability, for every bit-fixig path P, the umber of paths P y itersectig P is O() Fix a arbitrary bit-fixig path P = (e,, e k ) ad defie N P = {y {0, } : P P y } i N ei, 2
Where a edge e i, N ei deotes the umber of paths passig through e i By symmetry, we must have that E [N e ] is the same for all e E The, [ ] [ ] E [N e ] = E N e = E P x = 2 2 e e Note that i the above we are coutig (x, y) ad (y, x) as differet edges Thus, the umber of edges is 2, which gives E [N e ] = /2 for all e Thus, we have for each path P, x E [N P ] = i E [N ei ] = P 2 2 Also, N P ca be writte as a sum of 2 idepedet radom variables I P,y where I P,y = { if P Py 0 otherwise Thus, ca ow boud the probability that N P is large usig Cheroff bouds Sice 2e 6, we have P [N P 3] 2 3 Note that the umber of bit-fixig paths is at most 2 2, sice the path ca be specified by specifyig a start ad a ed Thus, the probability that for ay P, N P is greater tha 3 is at most 2 2 2 3 = 2 Also, for ay x, we have that Z x N Px 3 Thus, with probability at least 2, the time take by all packets i phase is at most + 3 = 4 We ow prove Claim 22 Proof of Claim 22: First ote that, i the hypercube, whe two bit-fixig paths diverge they will ot come together agai; ie paths which itersect will itersect oly i oe cotiguous segmet Let P x = (e,,, e k ) For every packet y which goes through P x (at least for some time), defie the lag of packet y at the start the time step t is as t i, where e i is the ext edge that packet y wats to traverse Note that at time t, all packets waitig to cross a edge must have the same lag At each time step, oe of these packets crosses the edge ad its lag remais uchaged (or it exits the path) ad the lag of the remaiig packets icreases by The lag of the packet x goes from 0 to D(x) We will charge each icrease i the lag of packet x to a uique y such that P y P x Cosider the time step t whe the lag of packet x goes from L to L + ; suppose this happes whe it is waitig to traverse edge e i The x must be held up i the queue at e i (else its lag would ot icrease), so there exists at least oe other packet at e i with lag L, ad this packet actually moves at step t For ay such packet, at each step either it moves ad its lag remais L, or it waits a edge ad some other packet with lag L crosses the edge Now cosider the last time at which there exists a packet y with lag L This packet must exit the path or its lag will remai L We ca charge this packet y for icreasig the lag of packet x from L to L + Each packet y is charged at most oce, because it is charged oly whe it leaves P x which by the observatio at the start of the proof, happes oly oce 3
3 Balaced Allocatios We cosider the followig problem of allocatig jobs to servers: We are give a set of servers,, ad m jobs arrive oe by oe We seek to assig each job to oe of the servers so that the loads placed o all servers are as balaced as possible I developig simple, effective load balacig algorithms, radomizatio ofte proves to be a useful tool We cosider two approaches for this problem: Radom Choice: oe possible way to distribute the jobs is to simply place each job o a radom server, chose idepedetly of the previous allocatios Two Radom Choices: For each job, we choose two servers idepedetly ad uiformly at radom ad place the job o the server with less load (breakig ties arbitrarily) We will show that usig two radom choices sigificatly reduces the maximum load o ay server For the etire aalysis, we will work with the case whe m = The aalysis easily exteds to a arbitrary m, but it easier to appreciate the bouds whe m = O() (ad i particular whe m = ) It is coveiet to thik of the above i terms of the so called balls ad bis model Each job ca be thought of a s ball ad each server is a bi We thik of assigig job j to a server i as throwig the j th ball i bi i The load of a server is the same as the umber of balls i the correspodig bi 3 Radom choice Suppose Z i = umber of balls i bi i We ca write Z i = j { if ball j is throw i bi i X ij, where X ij = 0 otherwise The, we have that each Z i is a sum of m(= ) idepedet radom variables with E [Z i ] = Let K = By Cheroff/Hoeffdig bouds, we have that for each i, l l l P [Z i K] Thus, the probability that there exists a i such that Z i K is at most ( ) e K, K which is at most l l for the above value of K Hece, with probability at least, the maximum umber of balls i a bi is at most ( e K ) K l l l 32 The power of two radom choices We will ow show that two radom choices ca reduce the maximum load to O(l l ) The proof techique is due to Azar et al [ABKU94] ad various applicatios were explored by Mitzemacher i his thesis [Mitz96] We first provide the ituitio for the proof 4
For each i, let B i deote the umber of bis with at least i balls Suppose B i β i for some boud β i The B i+ is bouded above by a biomial radom variable correspodig to the umber of heads i idepedet coi tosses, where the probability of each toss beig heads is at most (β i /) 2 This is because for a ball to lad a bi such that the load of the bi becomes greater tha i, it must happe that both the radom bis which we chose to put it i, had load at least i This happes with probability at( most (β i /) 2 Thus, B i+ is upper bouded by the above radom ( ) ) 2 variable, which we deote as Bi, βi ( ) 2 This, E [B i+ ] βi ad Bi+ is at most e β2 i with high probability We ca the take β i+ to be e β2 i For the above sequece, the value of β i becomes less tha for i 0 = O(l l ), ad thus we ca boud the maximum load by i 0 The proof will follow this ituitio, except that for the last step, whe E [B i ] becomes very small, we will ot be able to use a Cheroff boud ad will have to resort to a slightly differet aalysis ( ) We first defie the values β i Let β 6 = 2e ad β i+ = e βi 2 β 6 = 2e ( ) 2 β 7 = e = 2e 4e = 2 2 e ( ) 2 β 8 = e = 4e 6e = ( ) 2 β 9 = e = 6e 256e = 2 22 e 2 23 e β i = 2 2i 6 e Let E i be the evet that B i β i Note that E 6 holds for sure sice there ca be at most /6 /2e bis with 6 or more balls We show that with high probability, if E i holds the E i+ holds provided βi 2 2 l Claim 3 Let i be such that β 2 i 2 l The, P [ E i+ E i ] 2 Proof: The proof follows from a direct calculatio We have P [ E i+ E i ] = P [ E i+ E i ] [ ( ( ) ) 2 P Bi, e (β i/) 2 β i 2 ( e β i ) 2 ] 5
whe β 2 2 l We ca the use iductio to show that for each i as above, the probability of the evet E i ot happeig is very low Claim 32 For all i such that β 2 i 2 l, we have P [ E i+ ] i + 2 Proof: We prove the claim by iductio o i We kow from the defiitio of β 6 that P [ E 6 ] = 0 Also, from the previous claim, we have that for ay i as above, P [ E i+ ] = P [ E i+ E i ] + P [ E i ] P [ E i+ E i ] 2 + i 2 i + 2 We will eed a slightly differet aalysis whe β 2 i < 2 l, which we will cover i the ext lecture Refereces [BV8] [ABKU94] [Mitz96] G Breber ad L Valiat, Uiversal Schemes for Parallel Commuicatio, Proceedigs of the 3th Aual ACM Symposium o Theory of Computig, 98, pp 263 277 Y Azar, A Z Broder, A R Karli ad E Upfal, Balaced Allocatios, Proceedigs of the 26th Aual ACM Symposium o Theory of Computig, 994, pp 593 602 M Mitzemacher, The Power of Two Choices i Radomized Load Balacig, PhD thesis, Uiversity of Califoria Berkeley, 996 6