Desig ad Aalysis of Algorithms Probabilistic aalysis ad Radomized algorithms Referece: CLRS Chapter 5 Topics: Hirig problem Idicatio radom variables Radomized algorithms Huo Hogwei 1
The hirig problem Sceario You eed to hire a assistat. A employmet agecy seds you a cadidate each day. You are committed to havig the best cadidate, so that if you iterview a cadidate who is better tha your curret assistat, you hire the ew cadidate. Each time you hire a ew assistat you pay $c. Goal What is the price of this strategy? We are ot iterested i the ruig time of this algorithm. Istead we would like to kow the cost of employig the algorithm, perhaps switchig to a differet strategy if it achieves the same goal but with lower cost. Huo Hogwei 2
The hirig problem Assume that the cadidates are umbered 1 through. HIRE-ASSISTANT HIRE-ASSISTANT() 1 best 0 2 for i 1 to 3 if cadidate i is better tha best 4 the best i 5 hire cadicate i Best case Cadidate 1 is the best cadidate: Θ(1) Worst case Cadidate is the best cadidate: Θ(c h ) Probabilistic aalysis Sice we do t kow what order the cadidates appear i, we could assume that they show up i radom order, i. e., each of the! orderigs of the cadidates are equally likely. What is the expected value of the cost uder this distributio? Huo Hogwei 3
A radomized algorithm If we radomize the iput we ca impose a distributio o it (istead of assumig oe). For the hirig problem: if we had a list of the cadidates, each day we could pick oe uiformly at radom from the cadidates that have t bee iterviewed yet. [Exercise. Check that this iduces a uiform distributio o the! orderigs of the cadidates.] The executio of a radomized algorithm o the same iput chages from oe ru to the ext. We assume the existece of a fuctio RANDOM(a, b) that returs a uiformly radom iteger i the set {a,..., b}. Huo Hogwei 4
A radomized algorithm Give a evet A i a probability space (S, Pr), the idicator of A is defied as the radom variable I{A} = 1 if A occurs, 0 if A does ot occur. Lemma. E[X A ]= Pr[A]. Proof. E[X A ]= E[I{A}] = 1 Pr[A] + 0 (1- Pr[A]) = Pr[A]. This lemma (combied with liearity of expectatio) is useful for computig expectatios of radom variables that ca be writte as sums of idicators. Huo Hogwei 5
Example X := umber of heads i coi flips. What is EX? Method 1. E[X]= k Pr[X = k] k=0 = k 1 k 1 -k k 2 2 k=0 = 1 2 k k=1 k = 1-1 2 k=1 k -1 = 1 2-1 = 2 2 Huo Hogwei 6
Example X := umber of heads i coi flips. What is E[X]? Method2. Recogize that X = X i i=1 By liearity of expectatio, 1 E[X] = E X i = E[X i ] = 2 = i=1 i=1 Remark. Liearity of expectatio is applicable eve if the radom variable are depedet. (Not a issue i this example) i=1 2 Huo Hogwei 7
Hirig problem: probabilistic aalysis or radomized algorithm X = umber of times we hire a ew assistat. By the defiitio of expected value from C.19. E[X] = x Pr{X = x} x=1 The calculatio is cumbersome, we istead use idicator radom variable to simplify the calculatio Let X i be the idicator radom variable associated with the evet i which the ith cadidate is hired, i =1,,, X i = I{cadicate i is hired} = 1 if cadicate i is hired, 0 if cadicate i is ot hired, Huo Hogwei 8
Hirig problem: We ca compute E[X] E[X] = E X i i=1 = E[X i ] = i=1 i=1 1 i = l + O(1) By equatio A.7 Eve we iterview people, we oly actually hire approximately l of the o the average. Lemma: Assumig that the cadidate are preseted i a radom order, algorithm HIRE-ASSISTANT has a total hirig cost of O(c h l ). Huo Hogwei 9
Radom permutatio Goal. Produce a uiformly radom permutatio. (Each of the! permutatio is equally likely.) Idea. At the i th iteratio, swap A[i] with a radom elemet from A[i,..., ]. RANDOMIZE-IN-PLACE RANDOMIZE-IN-PLACE(A) 1 Legth[A] 2 for i 1 to 3 A[i] A[RANDOM(i,)] Huo Hogwei 10
Radom permutatio Goal. Produce a uiformly radom permutatio. (Each of the! permutatio is equally likely.) Idea. At the i th iteratio, swap A[i] with a radom elemet from A[i,..., ]. RANDOMIZE-IN-PLACE RANDOMIZE-IN-PLACE(A) 1 Legth[A] 2 for i 1 to 3 A[i] A[RANDOM(i,)] Loop ivariat. At the start of the for loop, A[1,..., i -1]cotais each of the (i -1)! C i -1 possible (i -1)-permutatios with equal probability. Iitializatio. i = 1. Trivial. Termiatio. i = + 1. A[1,..., ] cotais each of the! permutatios with equal probability. Huo Hogwei 11
Radom permutatio Maiteace. Cosider the ith iteratio. Assumig that whe the loop was etered A[1,..., i-1] cotai a uiformly radom (i -1)-permutatio of A, show that whe the loop is exited A[1,..., i] cotais a uiformly radom i -permutatio of A. Let <x 1,..., x i > deote the elemets i A[1,..., i] after the ith iteratio. It cosists of <x 1,..., x i-1 >, a (i-1)- permutatio followed by x i. E 1 := evet that the first i-1 iteratios produced <x 1,...,x i-1 >. E 2 := evet that the ith iteratio puts x i ito A[i]. Note that E 1 E 2 is the evet that we get the i-permutatio <x 1,...,x i >. The (-(i-1))! Pr[E 1 E 2 ] = Pr[E 1 E 2 ]Pr[E 1 ]= 1 (-i)! = -i+1!! Huo Hogwei 12
O-lie hirig problem Variat o the hirig problem. We ll settle for ot ecessarily the best office assistat i exchage for hirig exactly oce, which will make our cost O(1) [as opposed to O() i the worst- case or O(log ) o average]. Strategy. Look at the first k < cadidates. Reject each of them, otig the score of the best cadidate see so far. The, hire the first cadidate whose score exceeds this best score. (If o such cadidate is foud, hire the th cadidate.) Goal. Fid a k which maximizes the probability of hirig the best cadidate. Huo Hogwei 13
O-lie hirig problem Aalysis. S = evet that the best cadidate is foud, S i = evet that the best cadidate is foud by hirig the ith cadidate. The, S = i=1.. S i, ad sice the evets are idepedet ad we re discardig the first k cadidates, Pr[S] = Pr[S i ] = Pr[S i ] i=1 i=k+1 Notice that S i = B i O i where B i = evet that the ith cadidate is the best cadidate, O i = evet that oe of cadidates k + 1,..., i -1 are picked. Huo Hogwei 14
O-lie hirig problem Claim. The evets B i ad O i are idepedet. Proof. O i oly depeds o the relative order of the cadidates 1,..., i -1, i.e., kowig that the ith cadidate is the best does ot affect whether cadidates k+1,..., i-1 are goig to be picked by the strategy. Pr[S i ] = Pr[B i O i ] = Pr[B i ] Pr[O i ] = the last equality follows sice O i is the same evet as havig ay oe of the first k cadidates be the best amog the first i -1. Puttig everythig together k I order to maximize Pr[S], we should choose k/ to be a costat c < 1. (What would happe if we chose k to be a costat istead?) 1 k i-1 Pr[S i ] = = = [H -1 H k-1 ] i=k+1 1 i-1 k -1 1 k i=k i Huo Hogwei 15
O-lie hirig problem Heuristic argumet. (See the text for a precise argumet usig itegral approximatios to sums.) k Pr[S] [l lk] = c l c -1 which is maximized whe c = 1/ e 0. 37. Thus, if we choose k = 0. 37, with probability 0.37 we will pick the best cadidate. Huo Hogwei 16
Recap Probabilistic aalysis: assume distributio o the iputs ad examie expected value of cost. Radomized algorithms: iduce a distributio o the iputs by makig radom choices o determiistic iput. Tools: Idicators, liearity of expectatio,.... Huo Hogwei 17
For example oe who likes logic programmig - prolog ad who does ot like c, c++, java,... The first step is dowload (it is a free) xsb software package. The package has a lot of documetatio - prit ad start to read... The goal may be a implemetatio of the system to simulate da-computatio i the prolog programmig laguage... Somebody who is good i theory... ca prove equivalece betwee prolog computig model ad da computig model... The third who is good i compilers ca improve prolog laguage itroducig multyhead rules istead a:-b,c,d,... ad x:-b,c,... => a&x :- b,c,d Huo Hogwei 18