Recurrence Relations

Transcription

1 Recurrence Relations Analysis Tools S.V. N. (vishy) Vishwanathan University of California, Santa Cruz January 15, 2016 S.V. N. Vishwanathan (UCSC) CMPS101 1 / 29

2 Recurrences Outline 1 Recurrences 2 Substitution 3 Recursion-Tree Methods 4 Master Theorem S.V. N. Vishwanathan (UCSC) CMPS101 2 / 29

3 Recurrences Divide and Conquer: Basic Anatomy Divide the problem into a number of subproblems that are smaller instances of the same problem Conquer the subproblems by solving them recursively. If the subproblem sizes are small enough, however, just solve the subproblems in a straightforward manner. Combine the solutions to the subproblems into the solution for the original problem. S.V. N. Vishwanathan (UCSC) CMPS101 3 / 29

4 Recurrences Analysis One often needs to solve T (n) = at (n/b) + f (n) where a 1, b > 1, and f (n) is some function which bounds the time complexity of the merge step. S.V. N. Vishwanathan (UCSC) CMPS101 4 / 29

5 Recurrences Three Approaches Substitution: Guess a solution and then use induction to prove correctness Recursion Tree: Draw a recursion tree and add up work done at every level Master Theorem: Memorize a formula :) S.V. N. Vishwanathan (UCSC) CMPS101 5 / 29

6 Substitution Outline 1 Recurrences 2 Substitution 3 Recursion-Tree Methods 4 Master Theorem S.V. N. Vishwanathan (UCSC) CMPS101 6 / 29

7 Substitution Example 1 Suppose we want to solve T (n) = 2T (n/2) + n with T (1) = 1. Make a guess: T (n) = O(n log n) We will therefore need to prove that T (n) cn log n for some constant c > 0 and n n 0. S.V. N. Vishwanathan (UCSC) CMPS101 7 / 29

8 Substitution Induction Hypothesis Suppose that c 1 log(2) and T (m) cm log m for all n 0 m < n. Then We used c 1 log(2) T (n) = 2T (n/2) + n 2c(n/2)log(n/2) + n = cn log(n/2) + n = cn log n + n cn log(2) = cn log n + (1 c log(2))n cn log n for the last step. S.V. N. Vishwanathan (UCSC) CMPS101 8 / 29

9 Substitution Base Case T (1) = 1 T (2) = = 4 T (4) = = 12 Let n 0 = 4 and check that if we set c = 3 log 4 1 then T (4) = 12 c 4 log 4. S.V. N. Vishwanathan (UCSC) CMPS101 9 / 29

10 Substitution A Way to Mislead Yourself Suppose we want to solve T (n) = 2T (n/2) + n Make a guess: T (n) = O(n) We will therefore need to prove that T (n) cn for some constant c > 0 and n n 0. S.V. N. Vishwanathan (UCSC) CMPS / 29

11 Substitution A Way to Mislead Yourself - II Suppose that c > 0 and T (m) cm for all m < n. Then T (n) = 2T (n/2) + n 2cn/2 + n = cn + n = (c + 1)n Observing that T (n) is bounded by const. n completes the proof. What went wrong? S.V. N. Vishwanathan (UCSC) CMPS / 29

12 Substitution Another Subtle Example Suppose we want to solve T (n) = 2T (n/2) + 1 Make a guess: T (n) = O(n) We will therefore need to prove that T (n) cn for some constant c > 0 and n n 0. S.V. N. Vishwanathan (UCSC) CMPS / 29

13 Substitution Subtle Example Continued... Suppose that c > 0 and T (m) cm for all m < n. Then T (n) = 2T (n/2) + 1 2cn/2 + 1 = cn + 1 But this does not imply that T (n) cn :( S.V. N. Vishwanathan (UCSC) CMPS / 29

14 Substitution Subtle Example Continued... We will prove that T (n) cn d for some constant d 1. Suppose that for some c > 0 we have T (m) cm d for all m < n. Then T (n) = 2T (n/2) + 1 2(c(n/2) d) + 1 = cn 2d + 1 cn d S.V. N. Vishwanathan (UCSC) CMPS / 29

15 Recursion-Tree Methods Outline 1 Recurrences 2 Substitution 3 Recursion-Tree Methods 4 Master Theorem S.V. N. Vishwanathan (UCSC) CMPS / 29

16 Recursion-Tree Methods A Sample Problem Suppose we want to solve with T (1) = 1 T (n) = 3T (n/4) + cn 2 S.V. N. Vishwanathan (UCSC) CMPS / 29

17 Recursion-Tree Methods Recursion Tree S.V. N. Vishwanathan (UCSC) CMPS / 29

18 Recursion-Tree Methods Summing Over All Levels T (n) = cn ( ) 3 2 ( ) 3 log4 n 1 16 cn2 + cn cn 2 + Θ (n log 3) log 4 n 1 ( ) 3 i = cn 2 + Θ (n log 3) 4 i=0 i=0 ( ) i cn 2 + Θ (n log 3) 4 = cn 2 + Θ (n log 3) 4 16 = O(n 2 ). S.V. N. Vishwanathan (UCSC) CMPS / 29

19 Recursion-Tree Methods Second Sample Problem Suppose we want to solve T (n) = T (n/3) + T (2n/3) + O(n) S.V. N. Vishwanathan (UCSC) CMPS / 29

20 Recursion-Tree Methods Recursion Tree S.V. N. Vishwanathan (UCSC) CMPS / 29

21 Recursion-Tree Methods Summing Over All Levels T (n) = log 3/2 (n) i=1 cn cn log(n) S.V. N. Vishwanathan (UCSC) CMPS / 29

22 Master Theorem Outline 1 Recurrences 2 Substitution 3 Recursion-Tree Methods 4 Master Theorem S.V. N. Vishwanathan (UCSC) CMPS / 29

23 Master Theorem Recurrence Problem Suppose we want to solve T (n) = at (n/b) + f (n) S.V. N. Vishwanathan (UCSC) CMPS / 29

24 Master Theorem Master Theorem Let a 1 and b > 1 be constants, let f (n) be a function, and let T (n) be defined on the non-negative integers by the recurrence T (n) = at (n/b) + f (n). Then T (n) has the following asymptotic bounds 1 If f (n) = O(n log b a ɛ for some constant ɛ > 0, then T (n) = Θ(n log b a ) 2 If f (n) = θ(n log b a ), then T (n) = Θ(n log b a log n 3 If f (n) = O(n log b a+ɛ for some constant ɛ > 0, and af (n/b) cf (n) for some constant c < 1 and sufficiently large n, then T (n) = Θ(f (n)) S.V. N. Vishwanathan (UCSC) CMPS / 29

25 Master Theorem Examples Suppose we want to solve T (n) = 9T (n/3) + n Observe that a = 9, b = 3, and f (n) = n The function n log b a = n log 3 9 = n 2 We have f (n) = n = O(n log b a ɛ ) with ɛ = 1. Therefore T (n) = Θ(n 2 ) S.V. N. Vishwanathan (UCSC) CMPS / 29

26 Master Theorem Examples Suppose we want to solve T (n) = T (2n/3) + 1 Observe that a = 1, b = 3/2, and f (n) = 1 The function n log b a = n log 3/2 1 = n 0 = 1 We have f (n) = 1 = n log b a Therefore T (n) = Θ(log n) S.V. N. Vishwanathan (UCSC) CMPS / 29

27 Master Theorem Examples Suppose we want to solve T (n) = 3T (n/4) + n log n Observe that a = 3, b = 4, and f (n) = n log n The function n log b a = n log 4 3 = O(n ) We have f (n) = n log n > n ɛ for ɛ = 0.2 (say) af (n/b) = 3(n/4) log(n/4) (3/4)n log n = cf (n) with c = 3/4 Therefore T (n) = Θ(n log n) S.V. N. Vishwanathan (UCSC) CMPS / 29

28 Master Theorem A Case Where Master Theorem Does Not Apply Suppose we want to solve T (n) = 2T (n/2) + n log n Observe that a = 2, b = 2, and f (n) = n log n The function n log b a = n log 2 2 = n We have f (n) = n log n, but there is no ɛ such that n log n n 1+ɛ for all n S.V. N. Vishwanathan (UCSC) CMPS / 29

29 Master Theorem Questions? S.V. N. Vishwanathan (UCSC) CMPS / 29