Recurrence Relations Analysis Tools S.V. N. (vishy) Vishwanathan University of California, Santa Cruz vishy@ucsc.edu January 15, 2016 S.V. N. Vishwanathan (UCSC) CMPS101 1 / 29
Recurrences Outline 1 Recurrences 2 Substitution 3 Recursion-Tree Methods 4 Master Theorem S.V. N. Vishwanathan (UCSC) CMPS101 2 / 29
Recurrences Divide and Conquer: Basic Anatomy Divide the problem into a number of subproblems that are smaller instances of the same problem Conquer the subproblems by solving them recursively. If the subproblem sizes are small enough, however, just solve the subproblems in a straightforward manner. Combine the solutions to the subproblems into the solution for the original problem. S.V. N. Vishwanathan (UCSC) CMPS101 3 / 29
Recurrences Analysis One often needs to solve T (n) = at (n/b) + f (n) where a 1, b > 1, and f (n) is some function which bounds the time complexity of the merge step. S.V. N. Vishwanathan (UCSC) CMPS101 4 / 29
Recurrences Three Approaches Substitution: Guess a solution and then use induction to prove correctness Recursion Tree: Draw a recursion tree and add up work done at every level Master Theorem: Memorize a formula :) S.V. N. Vishwanathan (UCSC) CMPS101 5 / 29
Substitution Outline 1 Recurrences 2 Substitution 3 Recursion-Tree Methods 4 Master Theorem S.V. N. Vishwanathan (UCSC) CMPS101 6 / 29
Substitution Example 1 Suppose we want to solve T (n) = 2T (n/2) + n with T (1) = 1. Make a guess: T (n) = O(n log n) We will therefore need to prove that T (n) cn log n for some constant c > 0 and n n 0. S.V. N. Vishwanathan (UCSC) CMPS101 7 / 29
Substitution Induction Hypothesis Suppose that c 1 log(2) and T (m) cm log m for all n 0 m < n. Then We used c 1 log(2) T (n) = 2T (n/2) + n 2c(n/2)log(n/2) + n = cn log(n/2) + n = cn log n + n cn log(2) = cn log n + (1 c log(2))n cn log n for the last step. S.V. N. Vishwanathan (UCSC) CMPS101 8 / 29
Substitution Base Case T (1) = 1 T (2) = 2 1 + 2 = 4 T (4) = 2 4 + 4 = 12 Let n 0 = 4 and check that if we set c = 3 log 4 1 then T (4) = 12 c 4 log 4. S.V. N. Vishwanathan (UCSC) CMPS101 9 / 29
Substitution A Way to Mislead Yourself Suppose we want to solve T (n) = 2T (n/2) + n Make a guess: T (n) = O(n) We will therefore need to prove that T (n) cn for some constant c > 0 and n n 0. S.V. N. Vishwanathan (UCSC) CMPS101 10 / 29
Substitution A Way to Mislead Yourself - II Suppose that c > 0 and T (m) cm for all m < n. Then T (n) = 2T (n/2) + n 2cn/2 + n = cn + n = (c + 1)n Observing that T (n) is bounded by const. n completes the proof. What went wrong? S.V. N. Vishwanathan (UCSC) CMPS101 11 / 29
Substitution Another Subtle Example Suppose we want to solve T (n) = 2T (n/2) + 1 Make a guess: T (n) = O(n) We will therefore need to prove that T (n) cn for some constant c > 0 and n n 0. S.V. N. Vishwanathan (UCSC) CMPS101 12 / 29
Substitution Subtle Example Continued... Suppose that c > 0 and T (m) cm for all m < n. Then T (n) = 2T (n/2) + 1 2cn/2 + 1 = cn + 1 But this does not imply that T (n) cn :( S.V. N. Vishwanathan (UCSC) CMPS101 13 / 29
Substitution Subtle Example Continued... We will prove that T (n) cn d for some constant d 1. Suppose that for some c > 0 we have T (m) cm d for all m < n. Then T (n) = 2T (n/2) + 1 2(c(n/2) d) + 1 = cn 2d + 1 cn d S.V. N. Vishwanathan (UCSC) CMPS101 14 / 29
Recursion-Tree Methods Outline 1 Recurrences 2 Substitution 3 Recursion-Tree Methods 4 Master Theorem S.V. N. Vishwanathan (UCSC) CMPS101 15 / 29
Recursion-Tree Methods A Sample Problem Suppose we want to solve with T (1) = 1 T (n) = 3T (n/4) + cn 2 S.V. N. Vishwanathan (UCSC) CMPS101 16 / 29
Recursion-Tree Methods Recursion Tree S.V. N. Vishwanathan (UCSC) CMPS101 17 / 29
Recursion-Tree Methods Summing Over All Levels T (n) = cn 2 + 3 ( ) 3 2 ( ) 3 log4 n 1 16 cn2 + cn 2 +... + cn 2 + Θ (n log 3) 4 16 16 log 4 n 1 ( ) 3 i = cn 2 + Θ (n log 3) 4 i=0 i=0 ( 3 16 16 ) i cn 2 + Θ (n log 3) 4 = 1 1 3 cn 2 + Θ (n log 3) 4 16 = O(n 2 ). S.V. N. Vishwanathan (UCSC) CMPS101 18 / 29
Recursion-Tree Methods Second Sample Problem Suppose we want to solve T (n) = T (n/3) + T (2n/3) + O(n) S.V. N. Vishwanathan (UCSC) CMPS101 19 / 29
Recursion-Tree Methods Recursion Tree S.V. N. Vishwanathan (UCSC) CMPS101 20 / 29
Recursion-Tree Methods Summing Over All Levels T (n) = log 3/2 (n) i=1 cn cn log(n) S.V. N. Vishwanathan (UCSC) CMPS101 21 / 29
Master Theorem Outline 1 Recurrences 2 Substitution 3 Recursion-Tree Methods 4 Master Theorem S.V. N. Vishwanathan (UCSC) CMPS101 22 / 29
Master Theorem Recurrence Problem Suppose we want to solve T (n) = at (n/b) + f (n) S.V. N. Vishwanathan (UCSC) CMPS101 23 / 29
Master Theorem Master Theorem Let a 1 and b > 1 be constants, let f (n) be a function, and let T (n) be defined on the non-negative integers by the recurrence T (n) = at (n/b) + f (n). Then T (n) has the following asymptotic bounds 1 If f (n) = O(n log b a ɛ for some constant ɛ > 0, then T (n) = Θ(n log b a ) 2 If f (n) = θ(n log b a ), then T (n) = Θ(n log b a log n 3 If f (n) = O(n log b a+ɛ for some constant ɛ > 0, and af (n/b) cf (n) for some constant c < 1 and sufficiently large n, then T (n) = Θ(f (n)) S.V. N. Vishwanathan (UCSC) CMPS101 24 / 29
Master Theorem Examples Suppose we want to solve T (n) = 9T (n/3) + n Observe that a = 9, b = 3, and f (n) = n The function n log b a = n log 3 9 = n 2 We have f (n) = n = O(n log b a ɛ ) with ɛ = 1. Therefore T (n) = Θ(n 2 ) S.V. N. Vishwanathan (UCSC) CMPS101 25 / 29
Master Theorem Examples Suppose we want to solve T (n) = T (2n/3) + 1 Observe that a = 1, b = 3/2, and f (n) = 1 The function n log b a = n log 3/2 1 = n 0 = 1 We have f (n) = 1 = n log b a Therefore T (n) = Θ(log n) S.V. N. Vishwanathan (UCSC) CMPS101 26 / 29
Master Theorem Examples Suppose we want to solve T (n) = 3T (n/4) + n log n Observe that a = 3, b = 4, and f (n) = n log n The function n log b a = n log 4 3 = O(n 0.793 ) We have f (n) = n log n > n 0.793+ɛ for ɛ = 0.2 (say) af (n/b) = 3(n/4) log(n/4) (3/4)n log n = cf (n) with c = 3/4 Therefore T (n) = Θ(n log n) S.V. N. Vishwanathan (UCSC) CMPS101 27 / 29
Master Theorem A Case Where Master Theorem Does Not Apply Suppose we want to solve T (n) = 2T (n/2) + n log n Observe that a = 2, b = 2, and f (n) = n log n The function n log b a = n log 2 2 = n We have f (n) = n log n, but there is no ɛ such that n log n n 1+ɛ for all n S.V. N. Vishwanathan (UCSC) CMPS101 28 / 29
Master Theorem Questions? S.V. N. Vishwanathan (UCSC) CMPS101 29 / 29