Chapter 2. Recurrence Relations. Divide and Conquer. Divide and Conquer Strategy. Another Example: Merge Sort. Merge Sort Example. Merge Sort Example

Transcription

1 Recurrence Relations Chapter 2 Divide and Conquer Equation or an inequality that describes a function by its values on smaller inputs. Recurrence relations arise when we analyze the running time of iterative or recursive algorithms. Ex: Divide and Conquer. T(n) = O(1) T(n) = a T(n/b) + D(n d ) if n c otherwise Solution Methods Substitution Method. Recursion-tree Method. Master Method. Divide and Conquer Strategy How does Divide and Conquer Strategy work? Another Example: Merge Sort Sorting Problem: Sort a sequence or n elements into non-decreasing order. Divide: Divide the n-element sequence to be sorted into two subsequences of n/2 elements each Conquer: Sort the two subsequences recursively using merge sort. Examples? Combine: Merge the two sorted subsequences to produce the sorted answer. Merge Sort Example Merge Sort Example Original Sequence Sorted Sequence

2 Analysis of Merge Sort Running time T(n) of Merge Sort: Divide: computing the middle takes O(1) Conquer: solving 2 sub-problems takes 2T(n/2) Combine: merging n elements takes O(n) Total: T(n) = O(1) if n = 1 T(n) = 2T(n/2) + O(n) if n > 1 T(n) =? Binary search Algorithm: if (no element in the array), return 1; else check K against the middle value of the array if (same), return position; else if (K is smaller) binary search the 1 st half of the array; else binary search the 2 st half of the array; T(n) = T(n/2) + 1; T(1) = 1;? T(n) =? Golden Ratio Search Records are sorted based on the time New record will have a higher chance to be searched again. Multiplication x= , y = x*y =? divide and conquer? Running Time Recurrence: T(n) = O(1) if n = 1 T(n) = 3T(n/2) + O(n) if n > 1 Quicksort T(n) =? 2

3 Recurrence Relations Equation or an inequality that describes a function by its values on smaller inputs. Recurrence relations arise when we analyze the running time of iterative or recursive algorithms. Ex: Divide and Conquer. T(n) = O(1) T(n) = a T(n/b) + D(n d ) Solution Methods Substitution Method. Recursion-tree Method. Master Method. if n c otherwise Substitution Method Guess the form of the solution, then use mathematical induction to show it correct. Substitute guessed answer for the function when the inductive hypothesis is applied to smaller values. Works well when the solution is easy to guess. No general way to guess the correct solution. Example Asymptotics To Solve: T(n) = 2T(n/2) + n Guess: T(n) = O(n lg n) Need to prove: T(n) lg n, for some c > 0. Hypothesis: T(k) ck lg k, for all k < n. Calculate: T(n) 2c n/2 lg n/2 + n c n lg (n/2) + n = c n lg n c n lg2 + n = c n lg n n (c lg 2 1) c n lg n (The last step is true for c 1 / lg2.) Exercises Solution of T(n) = 3T(n/3) + n is O(nlogn) Solution of T(n) = T(n/3) + 1 is O(lgn) Solve T(n) = 2T(n/2) + 1 O(n) Recursion-tree Method Making a good guess is sometimes difficult with the substitution method. Use recursion trees to devise good guesses. Recursion Trees Show successive expansions of recurrences using trees. Keep track of the time spent on the subproblems of a divide and conquer algorithm. Help organize the algebraic bookkeeping necessary to solve a recurrence. Recursion Tree Example Running time of Merge Sort: T(n) = Θ(1) if n = 1 T(n) = 2T(n/2) + Θ(n) if n > 1 Rewrite the recurrence as T(n) = c if n = 1 T(n) 2T(n/2) + if n > 1 c > 0: Running time for the base case and time per array element for the divide and combine steps. 3

4 Recursion Tree for Merge Sort For the original problem, we have a cost of, plus two subproblems each of size (n/2) and running time T(n/2). Cost of divide and merge. Each of the size n/2 problems has a cost of /2 plus two subproblems, each costing T(n/4). /2 /2 Recursion Tree for Merge Sort Continue expanding until the problem size reduces to 1. lg n /2 /2 /4 /4 /4 /4 T(n/2) T(n/2) Cost of sorting subproblems. T(n/4) T(n/4) T(n/4) T(n/4) c c c c c c Total : lgn+ Other Examples Use the recursion-tree method to determine a guess for the recurrences Multiplication x= , y = x*y =? divide and conquer? T(n) = T(n/1.618)+16 T(n) = 4T(n/4)+4 T(n) = T(n/2) + 2T(n/3) + n Running Time Recurrence: T(n) = O(1) if n = 1 T(n) = 3T(n/2) + O(n) if n > 1 T(n) =? The Master Method Based on the Master theorem. Cookbook approach for solving recurrences of the form T(n) = at(n/b) + f(n d ) a 1, b > 1 are constants. f(n) is asymptotically positive. n/b does not have to be an integer, but we ignore floors and ceilings. Why? Requires memorization of three cases. 4

5 Master Theorem Master Theorem Let a 1, b > 1 be constants, f(n) be a function. Let T(n) be defined on nonnegative integers by T(n) = at(n/b) + f(n), where we can replace n/b by n/b or n/b. Then T(n) can be bounded asymptotically in three cases: 1. If f(n) = O(n log ba ε ) for some ε > 0, then T(n) = Θ(n log ba ). 2. If f(n) = Θ(n log ba ), then T(n) = Θ(n log ba lg n). 3. If f(n) = Ω(n log ba+ε ) for some constant ε > 0, and if, for some constant c < 1 and all sufficiently large n, we have af(n/b) c f(n), then T(n) = Θ(f(n)). Master Method Examples T(n) = 16T(n/4)+n a = 16, b = 4, n log b a = n log 416 = n 2. f(n) = n = O(n log b a-ε ) = O(n 2-ε ), where ε = 1 Case 1. Hence, T(n) = Θ(n log ba ) = Θ(n 2 ). Multiplication x= , y = x*y =? divide and conquer? T(n) = T(3n/7) + 1 a = 1, b=7/3, and n log ba = n log 7/3 1 = n 0 = 1 f(n) = 1 = Θ(n log ba ) Case 2. Therefore, T(n) = Θ(n logba lg n) = Θ(lg n) Master Method Examples T(n) = 3T(n/2) + n? Exercise Which method? T(n) = 4T(n/3)+4 T(n) = 2T(n/2 + 17) + n T(n) = T(n/2) + 2T(n/4) + n T(n) = T(n-1)+n 5

6 Matrix Multiplications Matrix multiplications Example: AB=C r = ae + bg s = af + bh t = ce + dg u = cf + dh Let A,B,C be n n matrices. What s the cost to obtain C? (assuming n is a power of 2) Cost of the direct method It takes T(n/2) to obtain each of ae, bf, dh. It takes n/2 n/2 additions to obtain r, s, t, or u. Therefore: T(n) = 8T(n/2)+Θ(n 2 ) = 8T(n/2)+ 2 = 8( 8T(n/2 2 )+c(n/2) 2 )+ 2 =? Cost of the direct method It takes T(n/2) to obtain each of ae, bf, dh. It takes n/2 n/2 additions to obtain r, s, t, or u. Therefore: T(n) = 8T(n/2)+Θ(n 2 ) = Θ(n 3 ) Strassen s Algorithm Strassen s Algorithm Strassen s Algorithm: an undetermined coefficient method. It s based on that fact that A+B is much cheaper to calculate than AB. Outline of the proof: Let P i = A i B i, i=1,,7 where A i = (α i1 a+α i2 b+α i3 c+α i4 d), B i = (β i1 e+β i2 f+β i3 g+β i4 h), α ij, β ij {-1,0,1}. Try to determine α ij, β ij such that r, s, t, u = γ ij P i, γ ij {-1,0,1} Note: The hidden coefficients in front of n log 2 7 is larger than the one in front of n 3. Determine the coefficients α ij, β ij : A 1 = a, A 2 = a+b, A 3 = c+d, A 4 = d, A 5 = a+d, A 6 = b d, A 7 = a c B 1 = f h, B 2 = h, B 3 = e, B 4 = g e, B 5 = e+h, B 6 = g+h, B 7 = e+f r = P 5 +P 4 -P 2 +P 6 s = P 1 +P 2 t = P 3 +P 4 u = P 5 +P 1 P 3 P 7 Verify u= Cost for n=8? 6

7 Strassen s Algorithm Strassen s Algorithm n 3 Strassen s Algorithm: T(n) = 7T(n/2)+Θ(n 2 ) T(n)= Θ(n log 2 7 ) = O(n 2.8 ) n 2.8 Insertion sort and selection sort No divide and conquer algorithms Complexity? Recurrence: T(n) = O(1) if n = 1 T(n) = T(n-1) + O(n) if n > 1 T(n) =? 7