Divide and Conquer. CSE21 Winter 2017, Day 9 (B00), Day 6 (A00) January 30,

Transcription

1 Divide and Conquer CSE21 Winter 2017, Day 9 (B00), Day 6 (A00) January 30,

2 Merging sorted lists: WHAT Given two sorted lists a 1 a 2 a 3 a k b 1 b 2 b 3 b l produce a sorted list of length n=k+l which contains all their elements. Design a recursive algorithm to solve this problem

3

4 Last time: merging two sorted lists Similar to Rosen p. 369 A recursive algorithm Put the smallest element first, then recursively merge the rest. concatenate

5 Merging sorted lists: WHEN θ(1) θ(1) One recursive call If T(n) is the time taken by RMerge on input of total size n, T(0) = c T(n) = T(n-1) + c' where c, c' are some constants

6 Merging sorted lists: WHEN If T(n) is the time taken by RMerge on input of total size n, T(0) = c T(n) = T(n-1) + c' where c, c' are some constants What's a solution to this recurrence equation? A. B. C. D. E. None of the above.

7 Merge Sort: HOW "We split into two groups and organized each of the groups, then got back together and figured out how to interleave the groups in order."

8 Merge Sort: HOW A divide & conquer (recursive) strategy: Divide list into two sub-lists Recursively sort each sublist Conquer by merging the two sorted sublists into a single sorted list

9 Merge Sort: HOW Similar to Rosen p. 368 Use RMerge as subroutine

10 Merge Sort: WHY Claim that result is a sorted list containing all elements. Proof by strong induction on n: Why do we need strong induction? A. Because we're breaking the list into two parts. B. Because the input size of the recursive function call is less than n. C. Because we're calling the function recursively twice. D. Because we're using a subroutine, RMerge. E. Because the input size of the recursive function call is less than n-1.

11 Merge Sort: WHY Claim that result is a sorted list containing all elements. Proof by strong induction on n: Base case : Suppose n=0. Suppose n=1.

12 Merge Sort: WHY Claim that result is a sorted list containing all elements. Proof by strong induction on n: Base case : Suppose n=0. Then, in the else branch, we return the empty list, (trivially) sorted. Suppose n=1. Then, in the else branch, we return a 1, a (trivally) sorted list containing all elements.

13 Merge Sort: WHY Claim that result is a sorted list containing all elements. Induction step : Suppose n>1. Assume, as the strong induction hypothesis, that MergeSort correctly sorts all lists with k elements, for any 0<=k<n. Goal: prove that MergeSort(a 1,, a n ) returns a sorted list containing all n elements.

14 Merge Sort: WHY IH: MergeSort correctly sorts all lists with k elements, for any 0<=k<n Goal: prove that MergeSort(a 1,, a n ) returns a sorted list containing all n elements. Since n>1, in the if branch we return RMerge( MergeSort(L 1 ), MergeSort(L 2 ) ), where L 1 and L 2 each have no more than (n/2) + 1 elements and together they contain all elements. By IH, each of MergeSort(L 1 ) and MergeSort(L 2 ) are sorted and by the correctness of Merge, the returned list is a sorted list containing all the elements.

15 Merge Sort: WHEN θ(1) say θ(n) say θ(n) T Merge (n/2 + n/2) T MS (n/2) T MS (n/2) If T MS (n) is runtime of MergeSort on list of size n, T MS (0) = c 0 T MS (1) = c' T MS (n) = 2T MS (n/2) + T Merge (n) + c'' n where c 0, c', c'' are some constants

16 Merge Sort: WHEN θ(1) say θ(n) say θ(n) T Merge (n) is θ(n) T Merge (n/2 + n/2) T MS (n/2) T MS (n/2) If T MS (n) is runtime of MergeSort on list of size n, T MS (0) = c 0 T MS (1) = c' T MS (n) = 2T MS (n/2) + cn where c 0, c, c' are some constants

17 Merging sorted lists: WHEN If T MS (n) is runtime of MergeSort on list of size n, T MS (0) = c 0 T MS (1) = c' T MS (n) = 2T MS (n/2) + cn where c 0, c, c' are some constants Solving the recurrence by unraveling:

18 Solving the recurrence by unraveling: Merging sorted lists: WHEN What value of k should we substitute to finish unraveling (i.e. to get to the base case)? A. k B. n C. 2 n D. log 2 n E. None of the above.

19 Solving the recurrence by unraveling: Merging sorted lists: WHEN With k = log 2 n, T MS (n) = c'n + c n log 2 n which is θ(n log n)

20 Solving the recurrence by unraveling: Merging sorted lists: WHEN With k = log 2 n, T MS (n) = c'n + c n log 2 n which is θ(n log n) That was a lot of work!

21 Merge Sort In terms of worst-case performance, Merge Sort outperforms all other sorting algorithms we've seen. n n 2 n log n ~ ~ Divide and conquer wins big!

22 Divide & Conquer: General Strategy Divide the problem of size n into a subproblems of size n/b. Recursively solve each subproblem. Conquer the problem of size n by combining solutions of subproblems. Rosen 5.4 has more examples

23 Divide & Conquer: General Strategy Divide the problem of size n into a subproblems of size n/b. Recursively solve each subproblem. Conquer the problem of size n by combining solutions of subproblems. T(n) = time to solve problem of size n T(1)=constant g(n) = time to do the conquer step to solve problem of size n Recurrence for T(n)? A. T(n) = a*t(n/b) + a*g(n) C. T(n) = a*t(n/b) + g(n) B. T(n) = T(a*n/b) + g(n) D. T(n) = g(n)*a*t(n/b)

24 Solving Divide and Conquer Recurrence Relations Divide and conquer algorithms have recurrences of the form T(n) = a*t(n/b) + g(n). If g(n) is a polynomial, there is a nice theorem called the Master Theorem that allows us to quickly estimate the time complexity of many divide and conquer algorithms. T(n) = time to solve problem of size n T(1)=constant g(n) = time to do the conquer step to solve problem of size n

25 Master Theorem

26 Master Theorem a log b n Size 1

27 Master Theorem for Mergesort If T MS (n) is runtime of MergeSort on list of size n, T MS (0) = c 0 T MS (1) = c' T MS (n) = 2T MS (n/2) + cn where c 0, c, c' are some constants a=2, b=2, d=1 so a=b d O(n 1 log n)

28 Master Theorem for Mergesort If T MS (n) is runtime of MergeSort on list of size n, T MS (0) = c 0 T MS (1) = c' T MS (n) = 2T MS (n/2) + cn where c 0, c, c' are some constants Not much work! a=2, b=2, d=1 so a=b d O(n 1 log n)

29 Master Theorem for Binary Search Do one comparison to decide which half to search in. Then repeat on a list of half the size. T(0) = c 0 T(1) = c' T(n) = T(n/2) + c where c 0, c, c' are some constants a=1, b=2, d=0 so a=b d O(n 0 log n) = O(log n)

30 Divide & Conquer Wins Big Mergesort Dividing into two subproblems each with half the size is a big win over other sorting algorithms. Binary Search Dividing into one subproblem with half the size is a big win over linear search. Will this work in other contexts?

31 Multiplication: WHAT Given two n-digit (or bit) integers a = a n-1 a 1 a 0 and b = b n-1 b 1 b 0 return the decimal (or binary) representation of their product. Rosen p x

32 Multiplication: HOW Given two n-digit (or bit) integers a = a n-1 a 1 a 0 and b = b n-1 b 1 b 0 return the decimal (or binary) representation of their product. Rosen p x Compute partial products (using single digit multiplications), shift, then add. How many operations?

33 Multiplication: HOW Given two n-digit (or bit) integers a = a n-1 a 1 a 0 and b = b n-1 b 1 b 0 return the decimal (or binary) representation of their product. Rosen p x Compute partial products (using single digit multiplications), shift, then add. How many operations? O(n 2 )

34 Multiplication: HOW Divide and conquer? Divide n-digit numbers into two n/2-digit numbers. If a = and b = , we can write To multiply: a = (1234) * (5678) b = (2468) * (1357) ((1234) * (5678))((2468) * (1357))= (1234)(2468) * (1234)(1357) * (2468)(5678) * (1357)(5678)

35 One 8-digit multiplication Multiplication: WHEN ( )( )=((1234) * (5678))((2468) * (1357))= (1234)(2468) * (1234)(1357) * (2468)(5678) * (1357)(5678) Four 4-digit multiplications (plus some shifts, sums)

36 One 8-digit multiplication Multiplication: WHEN ( )( )=((1234) * (5678))((2468) * (1357))= (1234)(2468) * (1234)(1357) * (2468)(5678) * (1357)(5678) Four 4-digit multiplications (plus some shifts, sums) A. a = 4, b=4, d=0 C. a=4, b=2, d=1 B. a = 4, b=4, d=1 D. a=4, b=2, d=0

37 Multiplication: WHEN a=4, b=2, d=1 so a>b d O(n log_2(4) ) = O(n 2 )

38 Multiplication: WHEN a=4, b=2, d=1 so a>b d O(n log_2(4) ) = O(n 2 ) A. This is good news! B. This is bad news! C. I m not sure.

39 Enter Anatoly Karatsuba... Rosen p. 528 Insight: replace one (of the 4) multiplications by (linear time) subtraction

40 Karatsuba Multiplication: HOW Rosen p. 528 ( )( )=((1234) * (5678))((2468) * (1357))= (1234)(2468) * (1234)(1357) * (2468)(5678) * (1357)(5678) (1234)(2468) * ( ) + [(1234) - (5678)][ (1357)-(2468) ] * (1357)(5678) * ( ) Insight: replace one (of the 4) multiplications by (linear time) subtraction

41 Karatsuba Multiplication: WHEN Instead of T(n) = 4 T(n/2) + cn T K (n) = 3 T K (n/2) + pn with c a constant, now we have with p a constant. Rosen p. 528 a=3, b=2, d=1 so a>b d O(n log_2(3) ) = O(n )

42 Karatsuba Multiplication: WHEN Rosen p. 528 n 1.58 is better than n 2 Progress since then This is good news! 1963: Toom and Cook develop series of algorithms that are time O(n 1+ ). 2007: Furer uses number theory to achieve the best known time for multiplication. 2016: Still open whether there is a linear time algorithm for multiplication.

43 Monday 2/6 in class. Bring Student ID. Note sheet. No calculators. Covers through recursion (HWs 1, 2, 3). Announcements (Replacement) HW3 due Wednesday 11:59pm MT1 Review Session covering the Practice Midterm Date, Location TBD