Divide & Conquer. CS 320, Fall Dr. Geri Georg, Instructor CS320 Div&Conq 1

Transcription

1 Divide & Conquer CS 320, Fall 2017 Dr. Geri Georg, Instructor CS320 Div&Conq 1

2 Strategy 1. Divide the problem up into equal sized sub problems 2. Solve the sub problems recursively 3. Combine the sub problem solutions into a complete solution CS320 Div&Conq 2

3 Review what is recursion? Solving a problem by reducing it to instances of the same problem with a smaller input Properties: 1. One or more base cases where we can compute the solution directly without recursion 2. Every recursive call must be on a smaller instance of the same problem that will eventually lead to a base case CS320 Div&Conq 3

4 Recurrence Relation Equation that describes a function in terms of its value on smaller inputs Two parts: Base case for 0 or 1 Function for problem sizes of larger n CS320 Div&Conq 4

5 Merge Sort Recursion cn c(n/2) c(n/2) c(n/4) c(n/4) c(n/4) c(n/4) c(n / 2 k ) c c c c c c c c Divide:? Conquer:? Merge:? work done cn 2(cn/2) 4(cn/4) log 2 n... 2 k (cn / 2 k )... cn cn log 2 n + cn CS320 Div&Conq 5

6 Merge Sort Recursion cn c(n/2) c(n/2) c(n/4) c(n/4) c(n/4) c(n/4) c(n / 2 k ) c c c c c c c c Divide: d Conquer: 2T(n/2) Merge: cn work done cn 2(cn/2) 4(cn/4) log 2 n... 2 k (cn / 2 k )... cn cn log 2 n + cn CS320 Div&Conq 6

7 Merge Sort Recurrence c T( n) T solve left half n / 2 T n / 2 cn solve right half merging if n 1 otherwise CS320 Div&Conq 7

8 Rank Analysis CS320 Div&Conq 8

9 Work in groups of at least 3 1. As a group, write down your 5 favorite songs. Next, individually rank them with your favorite first. 2. Compare lists with the others in your groups. 3. Based on the preference lists in your group, what is one song that could make a good recommendation for you? Why did you choose this song? CS320 Div&Conq 9

10 What is Similar? Ranking: given a set of n songs, numbers from 1..n, preferences are permutations. Your preference: Friend s preference: Definition of similarity: The number of out of place rankings CS320 Div&Conq 10

11 Simplify the Problem Re number one ranking from 1 n, then modify the other based on the first s rankings: Your preference: Friend s preference: Re number your ranking: CS320 Div&Conq 11

12 Simplify the Problem Re number one ranking from 1 n, then modify the other based on the first s rankings: Your preference: Friend s preference: Re number your ranking: Re number your friend s ranking: What s inverted in your friend s ranking? (2,1) and (4,3) CS320 Div&Conq 12

13 Possible Inversions? Maximum number of inversions for a list of length n? Reverse it! E.g.: Maximum number of pairs to invert? n 2 n( n 1) 2 CS320 Div&Conq 15

14 Brute Force Method Check every pair r i and r j in the ranking of n items inv = 0 for iin 0 to n 1 for j in i+1 to n 1 compare r i and r j if inversion inv = inv + 1 Since there are n(n 1)/2 possible inversions and this counts every pair, it is O(n 2 ). Can we do better? CS320 Div&Conq 16

15 Merge Sort using Inversions Divide.. CS320 Div&Conq 17

16 Merge Sort using Inversions L R Algorithm: Take from left: no inversions Take from right: number of inversions = number of elements remaining in left Should be how many? L={6}, R={5} CS320 Div&Conq 18

17 Back in your groups of 3 or so. Do a merge sort counting inversions on the worst case ranking difference we identified: Using this algorithm to merge and count the number of inversions: If you take an element from the left subtree there are no inversions to count If you take an element from the right subtree the number of inversions = number of elements remaining in left subtree CS320 Div&Conq 19

18 Running Time Just like merge sort, the sort and count algorithm running time satisfies: T(n) = 2 T(n / 2) + cn Running time is therefore O(n log n) So we have a way to compare rankings and calculate similarities that is more efficient than pair wise comparison across the entire ranking CS320 Div&Conq 20

19 What s the recurrence relation for this algorithm? Base:? Arbitrary input size bigger than the base? T(n) c Tn/2 solve left half Tn/2 solve right half if n 1 cn otherwise merging CS320 Div&Conq 22

20 Solving the Recurrence T(n) c Tn/2 solve left half Tn/2 solve right half if n 1 cn otherwise merging We have this recurrence what does it tell us about complexity? We have to solve it to determine the bound CS320 Div&Conq 23

21 Enter the Master Theorem Many recurrences you will see have this form: f (n) a f (n /b) cn d With proper constraints so that we can just deal with integers: a is at least 1, b is at least 2, and c and d are real numbers with c >0 and d at least 0. f ( n) d d n if a b d d n log n if a b log a b d n if a b CS320 Div&Conq 24

22 Counting Inversions Merge Sort Recurrence Analysis T(n) c Tn/2 solve left half Tn/2 solve right half if n 1 cn otherwise merging Re write: T(n) = 2* T(n/2) + cn CS320 Div&Conq 25

23 Counting Inversions Merge Sort Recurrence Analysis T(n) = 2* T(n/2) + cn f (n) a f (n /b) cn d a = b = d = CS320 Div&Conq 26

24 Counting Inversions Merge Sort Recurrence Analysis f T(n) = 2* T(n/2) + cn f (n) a f (n /b) cn d ( n) d d n if a b d d n log n if a b log a b d n if a b a = 2 b = 2 d = 1 b d = 2 1 relation between a and b d is.. equal: T(n) = Θ(nlogn) CS320 Div&Conq 28

25 Maybe easier to remember f (n) a f (n /b) cn d We are comparing a and b d which is bigger? Take the log of both sides: log b a and log b b d log b a and d If these are not equal, then the answer is: T(n) = Θ(n max(log b a, d) ) If they are equal, think of there being a penalty : T(n) = Θ(n max(log b a, d) ) * log(n) CS320 Div&Conq 29

26 What about that last term? f ( n) a f ( n / b) d cn The cn d term is a simplifying assumption that is often seen. But there are cases where you have a recurrence that doesn t fit this pattern: Merge sort, but we sort the left and right subtrees and then let the top element sifts down to its correct position. This sift down can only go how many levels? logn CS320 Div&Conq 31

27 L R Recurrence: T(n) = 2T(n/2) + log n f (n) a f (n /b) cn d CS320 Div&Conq 34

28 Generalized Master Theorem T(n) = at(n/b) + f(n) where a 1, b > 1 We compare f(n) to a version of see which is bigger. There are 3 cases: to is larger: f(n) = O( ) T(n) = Θ( ) is smaller: f(n) = Ω( ) T(n) = Θ(f(n)) they are the same : f(n) = Θ( ) T(n) = Θ( log n) = Θ(f(n) log n) CS320 Div&Conq 35

29 Counting inversion merge sort We have: T(n) = 2* T(n/2) + cn Master Theorem: T(n) = at(n/b) + f(n): compare f(n) and a = 2 b = 2 f(n) = cn = n = = = n = f(n) which case? bigger/smaller/equal Therefore: T(n) = Θ(f(n) log n) = Θ(n log n) CS320 Div&Conq 38

30 Easier way to remember T ( n) at ( n / b) f ( n) We are comparing bigger? and f(n) which is If these are not equal, then the answer is: T(n) = Θ(max(n log b a, f(n))) If they are equal, think of there being a penalty : T(n) = Θ (max(n log b a, f(n))) * log(n) Since they are equal, we can simplify: T(n) = Θ(f(n) * log(n)) CS320 Div&Conq 39

31 You Try It Apply the generalized Master Theorem to the shift down building a heap. L R CS320 Div&Conq 40

32 Try these In groups of 2 or 3, solve the recurrences: Matrix multiplication: T(n) = 8T(n/2) + Θ(n 2 ) Strassen s method: T(n) = 7T(n/2) + Θ(n 2 ) CS320 Div&Conq 41

33 Image Credits triangulation: recursion tree: Prof. Wim Bohm applerecommendermusic: music vs spotify ?r=UK&IR=T spotifyreco, spotifyfriends: music vs spotify vs tidal vs google streaming wars/ CS320 Div&Conq 42