CSE 202: Design and Analysis of Algorithms Lecture 16

Transcription

1 CSE 202: Desig ad Aalysis of Algorihms Lecure 16 Isrucor: Kamalia Chaudhuri

2 Iequaliy 1: Marov s Iequaliy Pr(X=x) Pr(X >= a) 0 x a If X is a radom variable which aes o-egaive values, ad a > 0, he Pr[X a] E[X] a Example: osses of a ubiased coi. X = #heads E[X] = /2. Le a = 3/4. By Marov s Iequaliy, Pr(X >= a) <= 2/3. Bu wha is i really? Pr[X 3 4 ] /4 3/4+1 2 (4e) /4 (e/4) /4 < (e/3) /4 2 2 /4 for large Summary: Marov s iequaliy ca be wea, bu i oly requires E[X] o be fiie! Fac: If >= e

3 Iequaliy 2: Chebyshev s Iequaliy Pr(X=x) Pr(X <= E[X] - a) Pr(X >= E[X] + a) 0 E[X] - a x E[X] + a If X is a radom variable ad a > 0, he Example: osses of a ubiased coi. X = #heads E[X] = /2 Pr[ X E[X] a] Var(X) a 2 Var[X] = /4 (how would you compue his?) From las slide, Pr(X >= 3/4) <= c /4 for some cosa c < 1, ad large eough Le a = /4, so ha we compue Pr(X >= 3/4). By Chebyshev, Pr(X >= 3/4) <= 4/ Summary: Chebyshev s iequaliy ca also be wea, bu oly requires fiie Var[X], E[X]

4 Iequaliy 3: Cheroff Bouds Pr(X=x) Pr(X<=(1-)E[X]) Pr(X>=(1+)E[X]) 0 x a Le X1,.., X be idepede 0/1 radom variables, ad X = X X. The, for ay >0, Moreover, for < 1, Pr(X (1 + )E[X]) e (1 + ) (1+) Pr(X (1 )E[X]) e E[X] E[X] Example: osses of a ubiased coi. X=#heads= X X where Xi=1 if oss i =head E[X] = /2. Pr[ X >= 3/4] = Pr[ X >= (1 + 1/2) E[X]), so = 1/2 Thus from Cheroff Bouds, Pr(X 3/4) e 1/2 (2/3) 3/2 /2 (0.88) /2 Summary: Sroger boud, bu eeds idepedece!

5 Cheroff Bouds: Simplified Versio Pr(X=x) Pr(X<=(1-)E[X]) Pr(X>=(1+)E[X]) 0 x a Le X1,.., X be idepede 0/1 radom variables, ad X = X X. The, for ay >0, Moreover, for < 1, Pr(X (1 + )E[X]) e (1 + ) (1+) Pr(X (1 )E[X]) e E[X] E[X] Simplified Versio: Le X1,.., X be idepede 0/1 radom variables, ad X = X X. The, for <2e -1, Pr(X >(1 + )E[X]) e 2 E[X]/4

6 Radomized Algorihms Coeio Resoluio Max 3-SAT Some Facs abou Radom Variables Global Miimum Cu Algorihm Radomized Selecio ad Sorig Three Coceraio Iequaliies Hashig ad Balls ad Bis

7 Hashig ad Balls--Bis Problem: Give a large se S of elemes x1,.., x, sore hem usig O() space s. i is easy o deermie wheher a query iem q is i S or o Table Lied lis of all xi s. h(xi) = 2 Popular Daa Srucure: A Hash able Algorihm: 1. Pic a compleely radom fucio h : U {1,...,} 2. Creae a able of size, iiialize i o ull 3. Sore xi i he lied lis a posiio h(xi) of able 4. For a query q, loo a he lied lis a locaio h(q) of able o see if q is here Wha is he query ime of he algorihm?

8 Hashig ad Balls--Bis Problem: Give a large se S of elemes x1,.., x, sore hem usig O() space s. i is easy o deermie wheher a query iem q is i S or o Table Algorihm: 1. Pic a compleely radom fucio h 2. Creae a able of size, iiialize i o ull 3. Sore xi i he lied lis a posiio h(xi) of able 4. For a query q, chec he lied lis a locaio h(q) Average Query Time: Suppose q is piced a radom s. i is equally liely o hash o 1,..,. Wha is he expeced query ime? Expeced Query Time = = 1 Pr[q hashes o locaio i] (legh of lis a T [i])] i=1 (legh of lis a T [i]) = 1 =1 i

9 Hashig ad Balls--Bis Problem: Give a large se S of elemes x1,.., x, sore hem usig O() space s. i is easy o deermie wheher a query iem q is i S or o Table Algorihm: 1. Pic a compleely radom fucio h 2. Creae a able of size, iiialize i o ull 3. Sore xi i he lied lis a posiio h(xi) of able 4. For a query q, chec he lied lis a locaio h(q) Wors Case Query Time: For ay q, wha is he query ime? (wih high probabiliy over he choice of hash fucios) Equivale o he followig Balls ad bis Problem: Suppose we oss balls u.a.r io bis. Wha is he max #balls i a bi wih high probabiliy? Wih high probabiliy (w.h.p) = Wih probabiliy 1-1/poly()

10 Balls ad Bis, agai Suppose we oss balls u.a.r io bis. Wha is he max load of a bi wih high probabiliy? Some Facs: 1. The expeced load of each bi is 1 2. Wha is he probabiliy ha each bi has load 1? Probabiliy = # permuaios # ways of ossig balls o bis =! 3. Wha is he expeced #empy bis? Pr[Bi i is empy] = E[# empy bis] = = Θ() ( (1-1/) lies bewee 1/4 ad 1/e for >=2 )

11 Balls ad Bis Suppose we oss balls u.a.r io bis. Wha is he max load of a bi wih high probabiliy? Le Xi = #balls i bi i Pr(X i ) 1 e 1 e 1 2 From Fac Would lie his for whp codiio Fac: If >= e Le = log e c log log log for cosa c = log = c log log log c 2 log 2 log, for c 4 (log c + log log log log log ) For large, his is 1 2 log log Therefore, w.p. 1/ 2, here are a leas balls i Bi i. Wha is Pr(All bis have <= balls)? Applyig Uio Boud, Pr(All bis have <= balls) >= 1-1/

12 Balls ad Bis Suppose we oss balls u.a.r io bis. Wha is he max load of a bi wih high probabiliy? Fac: W.p. 1-1/, he maximum load of each bi is a mos O(log /log log ) Fac: The max loaded bi has (log /3log log ) balls wih probabiliy a leas 1 - cos./ (1/3) Le Xi = #balls i bi i Pr(X i ) e 1 1 e A leas 1/e 1/3 for = log /3 log log Le Yi = 1 if bi i has load or more, Pr(Yi = 1) >= 1/e 1/3 = 0 oherwise E(Y) >= Y = Y1 + Y /3 /e Y Pr(Y = 0) = Pr(No bi has load or more) <= Pr( Y - E[Y] >= E[Y]) Usig Chebyshev, Pr( Y - E[Y] >= E[Y]) <= Var(Y)/E(Y) 2 Which coceraio boud o use?

13 Balls ad Bis Suppose we oss balls u.a.r io bis. Wha is he max load of a bi wih high probabiliy? Fac: W.p. 1-1/, he maximum load of each bi is a mos O(log /log log ) Fac: The max loaded bi has (log /3log log ) balls wih probabiliy a leas 1 - cos./ (1/3) Le Yi = 1 if bi i has load or more, = 0 oherwise Y = Y1 + Y Y Pr(Yi = 1) >= 1/e 1/3 Pr(Y = 0) = Pr(No bi has load >= ) <= Pr( Y - E[Y] >= E[Y]) <= Var(Y)/E(Y) 2 Var[Y] = Var[(Y Y) 2 ] = Var(Y i )+2 (E[Y i Y j ] E[Y i ]E[Y j ]) i=j i Chebyshev Now if i is o j, Yi ad Yj are egaively correlaed, which meas ha E[Y i Y j ] <E[Y i ]E[Y j ] Thus, Var(Y ) Var(Y i ) 1 i=1 E(Y) >= 2/3 /e Pr(Y = 0) Var(Y ) E(Y ) 2 e2 4/3 e2 1/3

14 The Power of Two Choices Problem: Give a large se S of elemes x1,.., x, sore hem usig O() space s. i is easy o deermie wheher a query iem q is i S or o Table Lied lis of all xi s. h(xi) = 2 Algorihm: 1. Pic wo compleely radom fucios h 1 : U {1,...,}, ad h 2 : U {1,...,} 2. Creae a able of size, iiialize i o ull 3. Sore xi a lied lis a posiio h1(xi) or h2(xi), whichever is shorer 4. For a query q, loo a he lied lis a locaio h1(q) ad h2(q) of able o see if q is here Equivale o he followig Balls ad Bis Problem: Toss balls io bis. For each ball, pic wo bis u.a.r ad pu he ball io he ligher of he wo bis. Wha is he wors case query ime? Aswer: O(log log ) (proof o i his class)