Lecture Randomized Load Balancing strategies and their analysis. Probability concepts include, counting, the union bound, and Chernoff bounds.

Transcription

1 U.C. Berkeley CS273: Parallel and Dstrbuted Theory Lecture 1 Professor Satsh Rao August 26, 2010 Lecturer: Satsh Rao Last revsed September 2, 2010 Lecture 1 1 Course Outlne We wll cover a samplng of the followng topcs durng the semester. 1. Randomzed Load Balancng strateges and ther analyss. Probablty concepts nclude, countng, the unon bound, and Chernoff bounds. 2. Parallel Algorthms. Here, we dscuss basc technques for parallelzng algorthms n a model that only s concerned wth concurrency and not so much wth localty. (a) PRAM defnton. Basc algorthms: lst rankng, etc. Reductons to lst rankng. In parallel algorthms usng a black box algorthm that embodes the parallelsm s qute useful. Lst rankng s one wdely useful such black box and an llustraton of ths concept. (b) Defnton of NC. P-completeness. Reductons. Ths s classc complexty as appled to parallelsm and gves (condtonal) lmts on the nherent parallelsm avalable n certan computatons. (c) Advanced algorthms. Dstrbuted Colorng. Matchng. For Dstrbuted Colorng, the task s to break symmetry. The technques here wll also be useful for reducng communcaton costs as dscussed later n the course. The Matchng algorthm s beautful combnatorcs and randomzed analyss where the nfluental Isolaton Lemma was formulated and ntroduced to computer scence. 3. Routng/schedulng (a) Butterfly routng: off-lne. Benes Routng. (b) Valant routng n the Butterfly network. Randomzed strategy for avodng bad traffc patterns. Bounded queues. To prove results for bounded queues s qute an nterestng theoretcal endeavor whch leads to a somewhat counter ntutve though nterestng strategy for swtchng. (c) Path selecton n arbtrary networks. (Use lnear programmng, randomzed roundng for approxmaton.) There s a varant of the expert s algorthm here. (d) Schedulng n networks. (Lovasz local lemma.) Ths starts wth analyss of problems wth schedulng the flow of packets n a networks, ncludes a dscusson of on-lne strateges, and also ncludes the analyss of an approach that eventually yelds the optmal bounds.

2 Notes for Lecture 1: August 26, Dstrbuted Algorthms (a) Byzantne Generals. Ths shows how to reach agreement n a dstrbuted system n the presense of a rather large fracton of bad agents. (b) Algorthms for dstrbuted models. In partcular, effcent shortest route computatons. 5. Game Theory. Possbltes. (a) Nash s Theorem. Proof usng Sperner s lemma, and Brower s fxed pt theorem. (b) Congeston games. Prce of Anarchy. More modern stuff dscussng the cost of game theoretc behavor to the utlty of a system. (Possble..) (c) Arrows theorem. There s no votng methodology for rankng three or more optons that acheves three natural propertes: no dctator, wnner among two s ndependent of other optons, f one s preferred by all t comes frst. (d) On-lne auctons. 6. Parttonng Algorthms. (a) Spectral methods. The approach along wth ntuton about the classcal result called Cheeger s nequalty whch gves one provable bounds on the performance of these methods. (b) Lnear Programmng based methods. Approxmaton algorthm based on optmzaton algorthms for graph parttonng. 7. Object Locaton. (a) Dstrbuted Hash Tables. Chord, CANN. (b) Localty Based Solutons for specal graphs. Tapestry. (c) Graph Decomposton based solutons, Approxmate shortest path data structures. 2 Load Balancng You have m jobs to be dstrbuted nto n bns. How do you dstrbute them approxmately equally? Duh. Evenly spread the jobs, maxmum load s m/n wthn plus or mnus 1. Ok, let s examne algorthms that use (a lot) less nformaton. 1. Randomly select a bn. 2. Randomly select two bns, choose least loaded.

3 Notes for Lecture 1: August 26, What s an upper bound on the maxmum load for strategy 1? Strategy 2? (We assume for today that m = n.) Isn t n the upper bound!! Oh, yes. Not, what I mean. What bound s the max-load almost always under. Or, wth a very small probablty, the max-load s greater than what? A bt of probablty... Probablty (for today) s just countng. We start wth an experment. The experment gves a sample space whch s a set of outcomes along wth a set of probabltes for each one. An event s a subset of outcomes. An example s choosng a poker hand whch has the set of poker hands as outcomes where each s chosen wth the same probablty. An event of nterest may be the probablty of gettng a flush; a poker hand s a 5 card hand and t s a flush when all the cards are of the same sut. Ok, what s the probablty of a flush n a fve card poker hand. In ths case, we just see what fracton of the outcomes s a flush. The calculaton s as follows. 4 ( 13) 5 ( 52 ). 5 (Now that we have seen ( n k), we note that ( n k) (ne/k) k.) We wll use a couple of other facts repeatedly. For ndependent events, the probabltes multply. What s the probablty of 4 heads n a row wth an unbased con? (1/2) 4 Back to the calculuaton... So, now that we are experts wth probablty, let s upper bound the probablty that any bn has load greater than k. (We can just enumerate all the stuatons where any bn has load more than k and dvde by n n. How do we count these stuatons. Yuck...) OK, let s just look at one bn. What s the probablty that ts load s greater than k? k ( n ) ( 1 n ) ( 1 1 n ) n k ( ) ( n 1 n ) k ( ne ) ( 1 = n) e. k We can bound ths by 2(e/k) k. If we choose k > 2e log n, ths s bounded by 1/n 2. So, now we know any one bn s pretty unlkely to have load greater than 2e log n. But, we wanted a bound for all bns, not just for one bn. We use the followng smple dea. The probablty of events A or B s at most the probablty of A plus the probablty of B. Or, P r[ A ] P r[a ]. Ths s called the unon bound, and we use t often n computer scence. If we take A to be the event that bn has load greater than 2e log n, we see that the probablty of any bn has load greater than 2e log n (P r[ A ]) s n tmes the probablty than P r[a ] whch s at most 1/n 2.

4 Notes for Lecture 1: August 26, Thus, the probablty that any bn has load greater than 2e log n s at most 1/n. Queston 1. Show that wth hgh probablty that the max-load s at most O(log n/ log log n). Queston 2. Show that expected number of bns wth load Ω(log n/ log log n) s at least a constant. (Below s a dscusson of expectaton, whch we wll cover n more detal n lecture 3.) 3 A comment about wth hgh probablty. In the above analyss, appear to have arbtrarly used 1/n as an upper bound the probablty of falure. Specfcally, I found a value k for whch the maxmum load was more than k (ths s falure ) wth probablty at most 1/n. Why dd I choose 1/n? For example, I could have chosen 1% or.1% or some other number. In general, the noton s that we would lke the probablty of falure to be small. One noton of small s that t goes to 0. For theorsts, we tend to thnk of n as gettng large so 1/n tends to go to zero. Other expressons go to zero as well, e.g., 1/ log n, but somehow 1/n s convenent. Moreover, n ths case, wth respect to the bound on the maxmum load k, choosng 1/n rather than 1/ log n only affects the value of k by a constant factor (actually by a small order term f one were more careful n the analyss.) 4 Expectaton: Lnearty We consder a random varable X n some probablty space, (whch usng the above development s defned as a functon on a sample space.) The expectaton of the random varable s defned to be E[X] = a a P r[x = a ]. For example, n a stuaton where we throw a ball nto one of n bns. Let X be the 0 1 valued random varable that ndcates whether the ball fell nto bn 1. It s easy to see that E[X] = 1/n from the defnton of expectaton. On other hand, usng ths defnton for the stuaton of throwng n balls nto n bns, let s defne X as the load on bn 1. What s ts expectaton? Well, E[X] = P r[x = ] = ( ) n (1/n) (1 1/n) n. Yuck. We know t s 1. But, the formal reason s lnearty of expectaton, whch states that for a random varable X = Y + Z, E[X] = E[Y ] + E[Z]. Now, we can defne our random varable X whch s the number of balls n bn 1, as the X = X, where X s the 0 1 valued random varable that ndcates whether ball 1 chose bn 1. Now,

5 Notes for Lecture 1: August 26, E[X] = E[X ] = 1/n = 1.