Algorithm Design and Analysis

Transcription

1 Algorithm Design nd Anlysis LECTURE 12 Solving Recurrences Mster Theorem Adm Smith

2 Review Question: Exponentition Problem: Compute b, where b N is n bits long. Question: How mny multiplictions? Nive lgorithm: Θ(b) = Θ(2 n ) (exponentil in the input length!) Divide-nd-conquer lgorithm: b = b/2 b/2 if b is even; (b 1)/2 (b 1)/2 if b is odd. T(b) = T(b/2) + Θ(1) T(b) = Θ(log b) = Θ(n).

3 So fr: 2 recurrences Mergesort; Counting Inversions T(n) = 2 T(n/2) + Θ(n) = Θ(n log n) Binry Serch; Exponentition T(n) = 1 T(n/2) + Θ(1) = Θ(log n) Mster Theorem: method for solving recurrences.

4 The mster method The mster method pplies to recurrences of the form T(n) = T(n/b) +, where 1, b > 1, nd f is symptoticlly positive, tht is >0 for ll n > n 0.

5 Three common cses Compre with n log b : 1. = O(n log b ε ) for some constnt ε > 0. grows polynomilly slower thn n log b (by n n ε fctor). Solution: T(n) = Θ(n log b ).

6 Three common cses Compre with n log b : 1. = O(n log b ε ) for some constnt ε > 0. grows polynomilly slower thn n log b (by n n ε fctor). Solution: T(n) = Θ(n log b ). 2. = Θ(n log b lg k n) for some constnt k 0. nd n log b grow t similr rtes. Solution: T(n) = Θ(n log b lg k+1 n).

7 Three common cses (cont.) Compre with n log b : 3. = Ω(n log b + ε ) for some constnt ε > 0. grows polynomilly fster thn n log b (by n n ε fctor), nd stisfies the regulrity condition tht f (n/b) c for some constnt c < 1. Solution: T(n) = Θ( ).

8 Ide of mster theorem Recursion tree: f (n/b) f (n/b) f (n/b 2 ) f (n/b 2 ) f (n/b 2 ) f (n/b) Τ (1)

9 Ide of mster theorem Recursion tree: f (n/b) f (n/b) f (n/b 2 ) f (n/b 2 ) f (n/b 2 ) f (n/b) f (n/b) 2 f (n/b 2 ) Τ (1)

10 Ide of mster theorem h = log b n Recursion tree: f (n/b) f (n/b) f (n/b 2 ) f (n/b 2 ) f (n/b 2 ) f (n/b) f (n/b) 2 f (n/b 2 ) Τ (1)

11 Ide of mster theorem h = log b n Recursion tree: f (n/b) f (n/b) f (n/b 2 ) f (n/b 2 ) f (n/b 2 ) f (n/b) f (n/b) 2 f (n/b 2 ) Τ (1) #leves = h = log bn = n log b n log b Τ (1)

12 Ide of mster theorem h = log b n Recursion tree: f (n/b) f (n/b) f (n/b 2 ) f (n/b 2 ) f (n/b 2 ) f (n/b) f (n/b) 2 f (n/b 2 ) Τ (1) CASE 1: The weight increses geometriclly from the root to the leves. The leves hold constnt frction of the totl weight. n log b Τ (1) Θ(n log b )

13 Ide of mster theorem h = log b n Recursion tree: f (n/b) f (n/b) f (n/b 2 ) f (n/b 2 ) f (n/b 2 ) f (n/b) f (n/b) 2 f (n/b 2 ) Τ (1) CASE 2: (k = 0) The weight is pproximtely the sme on ech of the log b n levels. n log b Τ (1) Θ(n log b lg n)

14 Ide of mster theorem h = log b n Recursion tree: f (n/b) f (n/b) f (n/b 2 ) f (n/b 2 ) f (n/b 2 ) f (n/b) f (n/b) 2 f (n/b 2 ) Τ (1) CASE 3: The weight decreses geometriclly from the root to the leves. The root holds constnt frction of the totl weight. n log b Τ (1) Θ( )

15 Exmples EX. T(n) = 4T(n/2) + n = 4, b = 2 n log b = n 2 ; = n. CASE 1: = O(n 2 ε ) for ε = 1. T(n) = Θ(n 2 ).

16 Exmples EX. T(n) = 4T(n/2) + n = 4, b = 2 n log b = n 2 ; = n. CASE 1: = O(n 2 ε ) for ε = 1. T(n) = Θ(n 2 ). EX. T(n) = 4T(n/2) + n 2 = 4, b = 2 n log b = n 2 ; = n 2. CASE 2: = Θ(n 2 lg 0 n), tht is, k = 0. T(n) = Θ(n 2 lg n).

17 Exmples EX. T(n) = 4T(n/2) + n 3 = 4, b = 2 n log b = n 2 ; = n 3. CASE 3: = Ω(n 2 + ε ) for ε = 1 nd 4(n/2) 3 cn 3 (reg. cond.) for c = 1/2. T(n) = Θ(n 3 ).

18 Exmples EX. T(n) = 4T(n/2) + n 3 = 4, b = 2 n log b = n 2 ; = n 3. CASE 3: = Ω(n 2 + ε ) for ε = 1 nd 4(n/2) 3 cn 3 (reg. cond.) for c = 1/2. T(n) = Θ(n 3 ). EX. T(n) = 4T(n/2) + n 2 /lg n = 4, b = 2 n log b = n 2 ; = n 2 /lg n. Mster method does not pply. In prticulr, for every constnt ε > 0, we hve n ε = ω(lg n).

19 Notes Reference on Mster Th m to be posted on web Mster Th m generlized by Akr nd Bzzi to cover mny more recurrences: T (n) = f(n) + where h i (n) = O( k i=1 i T (b i n + h i (n)) n log 2 n ) See

20 Multiplying lrge integers Given n-bit integers, b (in binry), compute c=b n-1 n-2 0 Nïve (grde-school) lgorithm: b n-1 b n-2 b 0 Write,b in binry Compute n intermedite products Do n dditions Totl work: Θ(n 2 ) n bits n bits n bits 2n bit output

21 Multiplying lrge integers Divide nd Conquer (Attempt #1): Write = A 1 2 n /2 + A 0 b = B 1 2 n /2 + B 0 We wnt b = A 1 B 1 2 n + (A 1 B 0 + B 1 A 0 ) 2 n /2 + A 0 B 0 Multiply n/2 bit integers recursively T(n) = 4T(n/2) + Θ(n) Als! this is still Θ(n 2 ) (Mster Theorem, Cse 1)

22 Multiplying lrge integers Divide nd Conquer (Attempt #1): Write = A 1 2 n /2 + A 0 b = B 1 2 n /2 + B 0 We wnt b = A 1 B 1 2 n + (A 1 B 0 + B 1 A 0 ) 2 n /2 + A 0 B 0 Multiply Krtsub s n/2 ide: bit integers recursively T(n) (A 0 +A= 1 4T(n/2) ) (B 0 + B+ 1 ) Θ(n) = A 0 B 0 + A 1 B 1 + (A 0 B 1 + B 1 A 0 ) Als! We cn this get is wy still Θ(n with 2 ) 3. multiplictions! (in yellow) (Exercise: write out the recursion tree.) x = A 1 B 1 y = A 0 B 0 z = (A 0 +A 1 )(B 0 +B 1 ) Now we use b = A 1 B 1 2 n + (A 1 B 0 + B 1 A 0 ) 2 n /2 + A 0 B 0 = x 2 n + (z x y) 2 n /2 + y

23 Multiplying lrge integers MULTIPLY (n,, b) B nd b re n-bit integers B Assume n is power of 2 for simplicity 1. If n 2 then use grde-school lgorithm else 2. A 1 à div 2 n /2 ; B 1 à b div 2 n /2 ; 3. A 0 à mod 2 n /2 ; B 0 à b mod 2 n /2. 4. x à MULTIPLY(n/2, A 1, B 1 ) 5. y à MULTIPLY(n/2, A 0, B 0 ) 6. z à MULTIPLY(n/2, A 1 +A 0, B 1 +B 0 ) 7. Output x 2 n + (z x y)2 n /2 + y

24 Multiplying lrge integers The resulting recurrence T(n) = 3T(n/2) + Θ(n) Mster Theorem, Cse 1: T(n) = Θ (n log 23 ) = Θ(n 1.59 ) Note: There is Θ(n log n) lgorithm for multipliction (more on it lter in the course).