MIT 6.854/8.45: Advaced Algorithms Sprig 206 Prof. Akur Moitra Lecture 2 February 8, 206 Scribe: Calvi Huag, Lih V. Nguye I this lecture, we aalyze the problem of schedulig equal size tasks arrivig olie to differet machies. This could easily be solved with a cetralized server that kows which machies are occupied ad assigs all tasks to uique machies. Istead, we wat to see if it s possible to distribute the tasks without without such coordiatio amog the tasks. I this problem, the metric we care about is the maximum load o ay give machie. We will cosider two such approaches: aive radom assigmet, ad the power of two choice. Radom Assigmet Oe approach might be that each task chooses a machie idepedetly at radom. This is the classic balls-i-bis ball = task, bi = machie. Let z i be the umber of balls tasks assiged to bi machie i. There are a couple ways we might try to put a boud o the maximum load.. Markov s iequality Theorem. Markov s iequality: If X is a oegative radom variable, the for ay a > 0 P[X a] E[X] a Usig Markov s iequality, sice E[z i ] = o average, each machie will receive task we get, for some k, Pz i k k, ad usig the uio boud P i : z i k k, which is ot very useful for all practical values of k, it tells us that the probability is at most..2 Chebyshev s iequality Theorem 2. Chebyshev s iequality: Let X be a radom variable with expectatio µ ad variace σ 2. The P[ X µ kσ] k 2 Let y i,j be radom idicator variable, defied so that y i,j = if the j-th ball is throw ito the i-th bi, ad y i,j = 0 otherwise. Note that y i,j = with probability of, ad Vary i,j = 2.
The for z i = j= y i,j, we get E[z i ] = ad Varz i = Vary i, j= = / / 2 = <. Usig Chebyshev s iequality ad a uio boud: Pz i k Pz i µ kσ k 2 P i : z i k + k 2 ad so we ca place a costat boud o the probability that ay bi has more tha k balls, if k >..3 Direct aalysis I a more direct aalysis, we take a uio boud over all possible subsets of k balls that might be set to machie i: e k Pz i k Pall balls i S set to machie i = k k 2 k S [], S =k Here, we used the fact that k e k. k The boud e k k is o the order of whe k = Θ log log log, so we ca put a costat boud o the probability that o bi will have more tha Θ balls. log log log.4 Cheroff boud Theorem 3. Upper tail of Cheroff boud: Let X i be idepedet Beroulli radom variables with E[ X i ] = µ. The e δ P Xi + δµ + δ +δ µ exp δ 2 µ 2 + δ I this case the X i are whether or ot each ball falls ito a particular bi, so µ =, k = +δ, ad so we get P X i k ek, which gives the same result as i direct aalysis P X k k i k = O whe k = Θ. log log log 2
2 Power of Two Choices This is a simple variatio o the fully radom method: istead of pickig oe machie at radom, each task picks two radom machies, ad assigs itself to the machie with the lower load. This simple chage will drastically lower the expected max load from Θ to Θlog log. log log log We ll start with a heuristic justificatio of this improvemet, before givig a more rigorous proof. 2. Heuristic Aalysis Cosider this origorous aalysis to help uderstad where the Θlog log boud arises. Let β i be a upper boud o the umber of bis with at least i balls at the ed of the process. Now, cosider the umber of bis which will cotai at least i + balls. Whe a ball arrives, i order for it to icrease a bi to i +, both radom bis picked must have at least i balls. The probability of a give radom choice edig up that way would be at most β i ; sice that must 2. happe twice idepedetly of such a icrease would be at most We d the expect, over 2 all arrivals, at most βi β = i 2 bis cotaiig at least i + balls, ad thus set β i+ = β2 i. Let β 6 = 2e < 6 must be true by pigeohole priciple, the by iductio β i the lowest i c such that β ic < will be Olog log. βi 2e 2i 6, therefore Note that this heuristic is ot a formal proof, sice it assumes that the β i bouds always hold while i reality there must be some small failure probability ad sice we bouded the expectatio of β i+ but we will wat a boud that holds with high probability. However, the full proof is coceptually essetially the same. 2.2 Rigorous Proof Theorem 4. Suppose that balls are distributed to bis, accordig to the power of two choices method. The i the ed, with high probability the most loaded bi cotais at most Olog log balls. If d 2 choices are used istead, the the load is at most O log log balls. Proof: We shall give the proof for geeral d. First we defie a few otatios. Let t be the time right after the t-th ball is placed. Defie ht, the height of the t-th ball be the umber of balls i the same bi as the tth ball immediately after it is placed icludig the tth ball itself. Defie β 6 = 2e ad β i+ = eβd i for i 6. Note that we ca boud β d i+ c β i d for some costat c, so with j = Olog log we have β j <. Let B, p be a Beroulli radom variable deotig the total umber of heads resultig from flippig cois, each flip with probability of heads p. 3
Let ν i t be the umber of bis with load of at least i, ad µ i t be the umber of balls of height at least i. Obviously ν i t µ i t. Let E i be the evet that ν i β i. Sice β 6 = 2e < 6, the evet E 6 holds with certaity. Fix a iteger i. Let Y i be a radom biary variable such that Y t = iff ht i + ad ν i t β i That is, Y t = iff the height of the t-th ball is at least i +, ad at time t there are at most β i bis of load at least i. Let ω j be the bi selected by the j-th ball. The P[Y t = ω,.., ω t ] βd i d := p i The proof will also use the cocept of first-order stochastic domiace. A radom variable x is said to stochastically domiate aother radom variable y if for all k, Px k Py k We use the followig theorem o stochastic domiace: Theorem 5. Stochastic Domiace: If X, X 2,... X are i.i.d. radom variables Y i = f i X, X 2,... X i, ad PY i = X, X 2,... X i p, the i Y i is stochastically domiated by B, p. Usig the Stochastic Domiace theorem P[ Y t k] P[B, p i k] t= With respect to E i it follows µ i+ = t= Y t. Thus P[ν i+ k E i ] P[µ i+ k E i ] = P[ By the Cheroff boud, with k = β i+ t= Y t k E i ] P[ t= Y t k] P[B, p i k] P[ν i+ β i+ E i ] P[B, p i ep i ] e p i If p i 2 log, we ca rewrite the above expressio as P[ E i+ E i ] 2 P[E i. That implies with ] high probability ν i β i for large eough i. However, if p i 2 log, we show with high probability that there is o ball at height i + 2. Let i be the smallest i such that βd i d We have P[ν i + 6 log E i ] 2 log. Sice β i+6 2 di, by iductio i P[B, 2 log / 6 log ] P[E i ] 4 2 P[E i ] log log + O.
also Apply the iequality we obtai P[µ i +2 µ i + 6 log ] P[B, 6 log /d ] P[µ i + 6 log ] P[ E i+ ] P[ E i+ E i ] + P[ E i ] P[µ i +2 ] 6 d + i + 2 = O That is, with high probability there s o ball at height i + 2 = O proof. 6 log /d P[µ i + 6 log ] log log. That cocludes the 5