Data Structures and Algorithm Analysis (CSC317) Randomized algorithms

Transcription

1 Data Structures and Algorithm Analysis (CSC317) Randomized algorithms

2 Hiring problem We always want the best hire for a job! Using employment agency to send one candidate at a time Each day, we interview one candidate We must decide immediately if to hire candidate and if so, fire previous! Cost to interview (low); Cost to fire/hire (expensive)

3 Hire-Assistant(n) 1. best = 0 //least qualified candidate 2. for i = 1 to n 3. interview candidate i 4. if candidate i better than best 5. best = i 6. hire candidate i

4 Hiring problem Always want the best hire fire if better candidate comes along Cost to interview (low) c i Cost to fire/hire (expensive) c h c h > c i n m Total number candidates Total number hired O(c i n + c h m) Different type of cost not run time, but cost of hiring

5 Hiring problem c Cost to interview (low i ) Cost to fire/hire (expensive ) n Total number candidates m Total number hired c h O(c i n + c h m) Depends on order of candidates! Constant no matter order of candidates

6 Hiring problem and randomized algorithms O(c i n + c h m) Depends on order of candidates! Constant no matter order of candidates Randomized order can do us well on average (we will show!)

7 Hiring problem and randomized algorithms Random could be already inherent in the input distribution Or, as here, we implement randomization in the algorithm

8 Randomized hiring problem(n) 1. randomly permute the list of candidates 2. Hire-Assistant(n)

9 Analyzing randomized hiring We want to know cost on average of including randomization We will first discuss indicator random variables For this, we need to first review probability on the board

10 Probability review (brief summary of content on the board) Expectation or Expected Value of Random variable: = Average x E[X] = p(x = x) x Sum over each possible value, weighted by probability of that value

11 Probability review (brief summary of content on the board) Linearity of Expectation X 1,..X n random variables then: n E[ X ] = i i=1 n i=1 E[ X ] i Does not require independence of random variables

12 Probability review (brief summary of content on the board) Indicator Random Variable (indicating if occurs ) X = I { A} = A 1 if A occurs 0 if A does not occur Lemma: For event A { } E[X A ] = Pr A Pr = probability E[X A ] = 1Pr(A) + 0(1 Pr(A)) = Pr(A)

13 Probability review (brief summary of content on the board) Indicator Random Variable and Linearity of Expectation X = I { A} = i X = n X i i=1 1 if A occurs on trial i 0 otherwise Then by linearity of expectation: n E[ X ] = i i=1 n i=1 E[ X ] i

14 Back to hiring problem

15 Randomized hiring problem(n) 1. randomly permute the list of candidates 2. Hire-Assistant(n) Hire-Assistant(n) 1. best = 0 //least qualified candidate 2. for i = 1 to n 3. interview candidate i 4. if candidate i better than best 5. best = i 6. hire candidate i

16 Randomized hiring problem(n) Instead of worst case We would like to know the cost on average of hiring new applicants X random variable equal to number times new person hired X = X X = 1 if candidate i hired i i 0 otherwise i on the board

17 Randomized hiring problem(n) X random variable equal to number times new person hired X = E X i X i n i=1 [ ] = E X i = n i=1 E [ X ] i Linearity expecta2ons

18 Randomized hiring problem(n) X random variable equal to number times new person hired X = E X n i=1 X i n i=1 [ ] = E X i = n i=1 E [ X ] i E[ X ] i = Pr( i) = 1 i Prob candidate i hired E[ X] = n 1 =? i i=1

19 Randomized hiring problem(n) X random variable equal to number times new person hired X = n i=1 E[ X] = X i n 1 = 1 i n = i=1 ln(n) + O(1) Harmonic series

20 Randomized hiring problem(n) Cost of hiring new candidate C h So on average cost C h ln(n) What about worst case?

21 Randomized hiring problem(n) Cost of hiring new candidate C h So on average cost C h ln(n) Compare to worst case C h n

22 Quicksort Input: n numbers Output: sorted numbers, e.g., in increasing order O(n logn) on average Sorts in place, unlike Merge sort

23 Quicksort Quicksort(A, p, r) 1. If p<r 2. q = Partition(A,p,r) 3. Quicksort(A,p,q-1) 4. Quicksort(A,q+1,r)

24 Quicksort p r Quicksort(A, p, r) 1. If p<r 2. q = Partition(A,p,r) 3. Quicksort(A,p,q-1) 4. Quicksort(A,q+1,r) We can see it is a divide and conquer, but what is special?

25 Partition 1. Pick pivot element in array A (for now, we ll choose last element) 2. Rearrange A so that elements before pivot are smaller, and elements after pivot are larger Pivot Example: A = [ ]

26 Partition Example: A = [ ] After Partition: A = [ ] < pivot > pivot Pivot in its right spot example on the board

27 Partition Partition(A,p,r) 1. x=a[ r ] 2. i=p-1 3. for j = p to r-1 4. if A[ j ] <= x 5. i = I + 1; 6. swap A[ i ] with A[ j ] 7. swap A[ i+1 ] with A[ r ] 8. return i+1 p i j i keeps track of pivot location j keeps track of next element to chk r

28 Partition Partition(A,p,r) 1. x=a[ r ] 2. i=p-1 3. for j = p to r-1 4. if A[ j ] <= x 5. i = I + 1; 6. swap A[ i ] with A[ j ] 7. swap A[ i+1 ] with A[ r ] 8. return i+1 p i j i keeps track of pivot location j keeps track of next element to chk O(?) r

29 Partition Partition(A,p,r) p r 1. x=a[ r ] pivot 2. i=p-1 3. for j = p to r-1 4. if A[ j ] <= x 5. i = i swap A[ i ] with A[ j ] 7. swap A[ i+1 ] with A[ r ] 8. return i+1 O(n) Why?

30 Partition Partition(A,p,r) 1. x=a[ r ] pivot 2. i=p-1 3. for j = p to r-1 4. if A[ j ] <= x 5. i = i swap A[ i ] with A[ j ] 7. swap A[ i+1 ] with A[ r ] 8. return i+1 p i j i keeps track of pivot location j keeps track of next element to chk O(n) Why? For each j, at most one swap r

31 Partition Partition Summary: Partition is quick O(n) No extra memory needed Reduces problem size for Divide and Conquer Loop invariant?

32 Partition p i j r Loop invariant? i keeps track of pivot location j keeps track of next element to chk Before the for loop at given j 1. All elements in A[p..i] <= pivot 2. All elements in A[i+1..j-1] > pivot 3. A[r] = pivot

33 Partition p i j r Loop invariant? i keeps track of pivot location j keeps track of next element to chk p i j r p i j r A[r]: pivot A[j.. r 1]: not yet examined A[i+1.. j 1]: known to be > pivot A[p.. i]: known to be pivot

34 Quicksort p r Quicksort(A, p, r) 1. If p<r 2. q = Partition(A,p,r) 3. Quicksort(A,p,q-1) 4. Quicksort(A,q+1,r) When will Partition do a bad job?

35 Quicksort p q r Quicksort(A, p, r) p q r 1. If p<r 2. q = Partition(A,p,r) 3. Quicksort(A,p,q-1) 4. Quicksort(A,q+1,r) When will Partition do a bad job? When the partition is for one element versus n-1 T(n) = T(n 1) + Θ(n) = Θ(n 2 )

36 Quicksort p r Quicksort(A, p, r) 1. If p<r 2. q = Partition(A,p,r) 3. Quicksort(A,p,q-1) 4. Quicksort(A,q+1,r) When will Partition do a good job?

37 Quicksort p q r Quicksort(A, p, r) 1. If p<r 2. q = Partition(A,p,r) 3. Quicksort(A,p,q-1) 4. Quicksort(A,q+1,r) When will Partition do a good job? When the partition is roughly equal in terms of numbers smaller than and greater than pivot T(n) = 2T(n / 2) + Θ(n) = Θ(n logn) (as in Merge sort)

38 Quicksort p r Quicksort(A, p, r) 1. If p<r 2. q = Partition(A,p,r) 3. Quicksort(A,p,q-1) 4. Quicksort(A,q+1,r) How do we choose the pivot point?? So that good ON AVERAGE?

39 Quicksort p r Quicksort(A, p, r) 1. If p<r 2. q = Partition(A,p,r) 3. Quicksort(A,p,q-1) 4. Quicksort(A,q+1,r) Answer: Choose pivot RANDOMLY

40 Quicksort p r Quicksort(A, p, r) 1. If p<r 2. q = Partition(A,p,r) 3. Quicksort(A,p,q-1) 4. Quicksort(A,q+1,r) Why might a random choice be good enough? We saw that equal split good; What about a 3 to 1, or 9 to 1 split?

41 Quicksort p q r Quicksort(A, p, r) 1. If p<r 2. q = Partition(A,p,r) 3. Quicksort(A,p,q-1) 4. Quicksort(A,q+1,r) What about a 3 to 1, or 9 to 1 split?

42 Quicksort p q r Quicksort(A, p, r) 1. If p<r 2. q = Partition(A,p,r) 3. Quicksort(A,p,q-1) 4. Quicksort(A,q+1,r) What about a 9 to 1 split? T(n) = T(9n /10) + T(n /10) + Θ(n) =?

43 Quicksort n cn log 10 n 1 10 n 9 10 n 1 n 9 n 9 n 81 n cn cn log 10=9 n n n cn cn What about a 9 to 1 split? 1 cn O.nlg n/ T(n) = T(9n /10) + T(n /10) + Θ(n) = Θ(nlogn)

44 Quicksort p r Quicksort(A, p, r) 1. If p<r 2. q = Partition(A,p,r) 3. Quicksort(A,p,q-1) 4. Quicksort(A,q+1,r) Why might a random choice be good enough? Even a 9 to 1, or 3 to 1 split, is O(n log n)

45 Quicksort on average run time We ll prove that average run time with random pivots for any input array is O(n log n) Randomness is in choosing pivot Average as good as best case! To be continued