Lecture 6: Recurrent Algorithms:

Transcription

1 Lecture 6: Recurrent Algorithms: Divide-and-Conquer Principle Georgy Gimel farb COMPSCI 220 Algorithms and Data Structures 1 / 20

2 1 Divide-and-conquer 2 Math induction 3 Telescoping 4 Examples 5 Pros and cons 2 / 20

3 Divide-and-Conquer Principle Divide-and-Conquer: Divide a large problem into smaller subproblems; Recursively solve each subproblem, then Combine solutions of the subproblems to solve the original problem. Running time: by a recurrence relation Recurrence relation combines: The size and the number of the subproblems and The cost of dividing the problem into the subproblems. 3 / 20

4 Recurrence Relation: A Simple Example F (n) = 2 n Implicit formula: F (n) = F (n 1) + F (n 1) ; F (0) = 1, or }{{}}{{} 2 n 1 2 n 1 F (n) = 2F (n 1); F (0) = 1; n = 1, 2, n F (n) Explicit formula with F (0) = 1: F (n) = 2 n 4 / 20

5 Let s Guess and Prove an Explicit, or Closed-Form F (n) Look at a series of results for the implicit recurrent formula: n F (n) Guess a closed-form formula F (n) = 2 n and prove it with Mathematical Induction Base inductive condition: F (0) = 2 0 = 1 Induction hypothesis: for any n 1, if F (n 1) = 2 n 1 holds for n 1, then F (n) = 2 n holds for n. Proof: F (n) = 2F (n 1) = 2 2 n 1 = 2 n. 5 / 20

6 Mathematical Induction Instead of directly proving an explicit relationship T (n), the math induction proves the following conditions: 1. Base condition: T (n base ), e.g. T (0) or T (1). 2. Induction hypothesis: For every n > n base, if T (n 1) holds for n 1, then T (n) holds for n; or 2. Strong induction: If T (k) holds for every k = n base,, n 1, then T (n) holds for n. 6 / 20

7 Example 1.25: Fibonacci Numbers 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, = = = = = 144 Implicit formula: F (n) = F (n 1) + F (n 2) Analysis: derive a closed-form formula for F (n) 7 / 20

8 Characteristic Equation Because F (n) > F (n 1) for all n 1, it holds that 2 n 1 < F (n) < 2 n. One may suggest that F (n) = cϕ n ; 1 < ϕ < 2. The implicit equation cϕ n = cϕ n 1 + cϕ n 2 leads to the quadratic characteristic equation for ϕ: ϕ 2 = ϕ + 1 with two solutions: ϕ 1,2 = 1 ( ) 2 1 ± 5. General solution: the linear combination F (n) = c 1 ϕ n 1 + c 2ϕ n 2 The coefficients c 1 and c 2 follow from the conditions F (1) = F (2) = 1, so that finally ( F (n) = ) n ( ) n / 20

9 Telescoping a Recurrence Given: An implicit recurrence relation and its base condition (i.e., the difference equation and its initial condition), for example: T (n) = 2T (n 1) + 1; T (0) = 0 Find: The closed-form (explicit) formula for T (n) by recursive substitution of the same implicit formula: T (n) = 2T (n 1) + 1 T (n 1) = 2T (n 2) + 1 T (2) = 2T (1) + 1 T (1) = 2T (0) + 1 = 1 9 / 20

10 Telescoping Substitution T (n) = 2T (n 1) + 1 Step 0: Initial recurrence 2T (n 1) = 2 2 T (n 2) + 2 Step 1: Substitute T (n 1) 2 2 T (n 2) = 2 3 T (n 3) Step 2: Substitute T (n 2) 2 n 1 T (1) = 2 n T (0) + 2 n 1 Step n 1: Substitute T (1) T (n) = n 1 = 2 n 1 10 / 20

11 Telescoping Substitution T (n) = 2T (n 1) + 1 Step 0: Initial recurrence 2T (n 1) = 2 2 T (n 2) + 2 Step 1: Substitute T (n 1) 2 2 T (n 2) = 2 3 T (n 3) Step 2: Substitute T (n 2) 2 n 1 T (1) = 2 n T (0) + 2 n 1 Step n 1: Substitute T (1) T (n) = n 1 = 2 n 1 10 / 20

12 Telescoping Substitution T (n) = 2T (n 1) + 1 Step 0: Initial recurrence 2T (n 1) = 2 2 T (n 2) + 2 Step 1: Substitute T (n 1) 2 2 T (n 2) = 2 3 T (n 3) Step 2: Substitute T (n 2) 2 n 1 T (1) = 2 n T (0) + 2 n 1 Step n 1: Substitute T (1) T (n) = n 1 = 2 n 1 10 / 20

13 Telescoping Substitution T (n) = 2T (n 1) + 1 Step 0: Initial recurrence 2T (n 1) = 2 2 T (n 2) + 2 Step 1: Substitute T (n 1) 2 2 T (n 2) = 2 3 T (n 3) Step 2: Substitute T (n 2) 2 n 1 T (1) = 2 n T (0) + 2 n 1 Step n 1: Substitute T (1) T (n) = n 1 = 2 n 1 10 / 20

14 Telescoping Substitution T (n) = 2T (n 1) + 1 Step 0: Initial recurrence 2T (n 1) = 2 2 T (n 2) + 2 Step 1: Substitute T (n 1) 2 2 T (n 2) = 2 3 T (n 3) Step 2: Substitute T (n 2) 2 n 1 T (1) = 2 n T (0) + 2 n 1 Step n 1: Substitute T (1) T (n) = n 1 = 2 n 1 10 / 20

15 Telescoping Substitution T (n) = 2T (n 1) + 1 Step 0: Initial recurrence 2T (n 1) = 2 2 T (n 2) + 2 Step 1: Substitute T (n 1) 2 2 T (n 2) = 2 3 T (n 3) Step 2: Substitute T (n 2) 2 n 1 T (1) = 2 n T (0) + 2 n 1 Step n 1: Substitute T (1) T (n) = n 1 = 2 n 1 10 / 20

16 Example 1.29: Textbook, p.23 The recurrence T (n) = T (n 1) + n; T (0) = 0, results in the closed-form (explicit) formula T (n) = n(n+1) 2. Telescoping the recurrence: T (n) = T (n 1) + n T (n 1) = T (n 2) + n 1 T (2) = T (1) + 2 T (1) = T (0) + 1 = 1 11 / 20

17 1.29: T (n) by Telescoping in More Detail Successive substitution: T (n) = T (n 1) + n = T (n 2) + (n 1) + n = T (n 3) + (n 2) + (n 1) + n = T (2) (n 2) + (n 1) + n = T (1) (n 2) + (n 1) + n = (n 2) + (n 1) + n = n(n+1) 2 12 / 20

18 1.29: T (n) by Guessing and Proving by Math Induction Numerical sequence: T (1) = = 1; T (2) = = 3; T (3) = = 6; T (4) = = 10; T (5) = = 15; Guessing: T (n) = n(n+1) 2? Base condition holds: T (1) = = 1. Induction hypothesis: If the guessed formula T (n) holds for n 1, then it holds also for n. The proof: T (n) = T (n 1) + n = (n 1)n 2 + n, i.e. T (n) = 1 2 ( n 2 n + 2n ) = 1 2 ( n 2 + n ) = n(n + 1) 2 Thus, the guessed formula for T (n) holds for all n / 20

19 Example 1.30, p.23 Repeated halving principle: halve the input in one step Recurrence (implicit formula): T (n) = T ( n 2 ) + 1; T (1) = 0. Closed-form (explicit) formula: T (n) log 2 n Telescoping (for n = 2 m ): T (2 m ) = T (2 m 1 ) + 1 T (2 m 1 ) = T (2 m 2 ) + 1 T (2 2 ) = T (2 1 ) + 1 T (2 1 ) = T (2 0 ) + 1 = 1 14 / 20

20 1.30: T (n) by Telescoping in More Detail T (2 m ) = T (2 m 1 ) + 1 = T (2 m 2 ) = T (2 m 3 ) = T (2 1 ) = T (2 0 ) = = m, or T (2 m ) = m For n = 2 m, T (n) = lg n, which is Θ(log n). For general n, the total number of halving steps cannot be greater than m = lg n, so T (n) lg n for all n. 15 / 20

21 Example 1.31, p.23 Scan and halve the input: Recurrence (implicit formula): T (n) = T ( n 2 ) + n; T (1) = 1. Closed-form (explicit) formula: T (n) 2n Telescoping (for n = 2 m ): T (2 m ) = T (2 m 1 ) + 2 m T (2 m 1 ) = T (2 m 2 ) + 2 m 1 T (2 2 ) = T (2 1 ) T (2 1 ) = T (2 0 ) T (2 0 ) = 2 0 = 1 16 / 20

22 1.31: T (n) by Telescoping in More Detail T (2 m ) = T (2 m 1 ) + 2 m = T (2 m 2 ) + 2 m m = T (2 m 3 ) + 2 m m m = T (2 1 ) m m m = T (2 0 ) m m m = m m m = 2 m+1 1 Therefore, T (2 m ) 2 2 m, or T (n) 2n. 17 / 20

23 Example 1.32, p.23 Divide-and-conquer prototype; n 2: Recurrence (implicit formula): T (n) = 2T ( n 2 ) + n; T (1) = 0. Closed-form (explicit) formula: T (n) n log 2 n Equivalent representation for telescoping : T (n) = 2T ( ) n 2 + n 1 n T (n) = 2 n T ( ) n T (n) n = T(n 2 ) n For n = 2 m, T (2m ) 2 m = T (2m 1 ) 2 m / 20

24 1.32: T (n) by Telescoping in More Detail T (2 m ) 2 m = T (2m 1 ) 2 m = T (2m 2 ) 2 m = T (2m 3 ) 2 m = T (21 ) = T (20 ) = = m Therefore, T (2 m ) = m 2 m, or T (n) n lg n. 19 / 20

25 Capabilities and Limitations Rough time complexity analysis cannot result immediately in an efficient program. But it helps to predict empirical running time of the program. Limitations of the Big-Oh / Theta / Omega analysis: It hides the constants (e.g. c and n 0 ) crucial for a practical task. It is unsuitable for small input. It is unsuitable if costs of access to input data items vary. It is unsuitable if there is lack of sufficient memory. But time complexity analysis provides ideas how to develop new and efficient algorithms. 20 / 20