COMP26120: More on the Complexity of Recursive Programs (2018/19) Lucas Cordeiro

Transcription

1 COMP26120: More o the Complexity of Recursive Programs (2018/19) Lucas Cordeiro lucas.cordeiro@machester.ac.uk

2 Divide-ad-Coquer (Recurrece) Textbook: Algorithm Desig ad Applicatios, Goodrich, Michael T. ad Roberto Tamassia (chapter 8) Itroductio to Algorithms, Corme, Leiserso, Rivest, Stei (chapters 2 ad 4)

3 Iteded Learig Outcomes Review expoetials, logarithms, factorials, ad sums Solve recurreces usig iteratio method ad master method Describe various examples to aalyse dividead-coquer algorithms ad how to solve their recurreces

4 Expoetials For all real a > 0, m, ad, we have the followig idetities: a 0 1, a 1 a, a -1 1/a, (a m ) a m, a m a a m. For all ad a 1, the fuctio a is mootoically icreasig i

5 Logarithms For all real a > 0, b > 0, c > 0, ad, i each equatio above, logarithm bases are ot 1

6 Factorials The otatio! is defied for itegers 0 as A form of Stirlig s approximatio A weak upper boud o the factorial fuctio is! sice each of the terms i the factorial product is at most

7 Sums Give a sequece of a 1, a 2,, a of umbers, where is a oegative iteger, we ca write the fiite sum a 1 a 2 a as If 0, the value of the summatio is 0 Give a ifiite sequece of a 1, a 2,, a If the limit does ot exist, the series diverges; If the limit exists ad is fiite, the series coverges

8 Algebraic Regroupig Distributive: A commo factor ca be distributed over all the summads Additio: Add each pair of summads with the same idex (a k b k ) a k b k k K ca k c a k k K Permutatio: The value of a sum is uchaged by permutig the order of the summads k K k K k K a k a P (k ) k K p (k ) K

9 Arithmetic ad Geometric Series The summatio is a arithmetic series ad has the value For real x 1, the summatio is a geometric series ad has the value

10 The Perturbatio Method The iitial step of a perturbatio is to equate two expressios for S 1 by takig its first ad last terms Fid S o both sides of the equatio Isolate S to fid the formula for the sum What makes it iterestig is ot the theory behid it, but the fact that it works so effectively so ofte for sums that cotai expoetials

11 S Illustrative Example of the x k k 0 S 1 S x 1 Perturbatio Method Give our geometric series: 1 k 1 k 0 k 0 S 1 x 0 x k 1 x k 1 1 x x k 1 xs 1 xs S x 1 S x 1 1 x 1 Obtai the last term Obtai the first term

12 Approximatio by Itegrals Whe a summatio has the form, where f(k) is a mootoically icreasig fuctio, we ca approximate it by itegrals

13 Approximatio by Itegrals (Cot.) Whe f(k) is a mootoically decreasig fuctio, we ca use a similar method to provide the bouds First iequality which yields the boud Secod iequality

14 Solvig Recurreces I additio to the substitutio method, aother optio is what the literature calls the iteratio method Expad the recurrece Work some algebra to express as a summatio Evaluate the summatio We will show several examples

15 Example 1: Iteratio What is the complexity of this recurrece? s( ) c 0 s( 1) > 0 0

16 Example 1: Iteratio s() c s(-1) c c s(-2) c c c s(-3) 3c s(-3) kc s(-k) ck s(-k) s() c s(-1) s(-1) c s(-2) s(-2) c s(-3) s( ) c 0 s( 1) > 0 0

17 Example 1: Iteratio So far for > k we have s() ck s(-k) if k? s() c s(0) c The s() c s() O() s( ) c 0 s( 1) > 0 0

18 Example 2: Iteratio What is the complexity of this recurrece? s( ) 0 s( 1) > 0 0

19 Example 2: Iteratio k1 k2 s() s(-1) -1 s(-2) s( ) 0 s( 1) > 0 0 k s(-3) k s(-4) (k-1) s(-k) i i k 1 s( k)

20 So far for > k we have If k? The: ) ( 1 k s i k i (0) 1 1 i s i i i 2 1 ) ( s Example 2: Iteratio (Arithmetic series) s() O( 2 )

21 Example 3: Iteratio What is the complexity of this recurrece? T( ) 2T c 2 c 1 > 1

22 Example 3: Iteratio T() 2T(/2) c 2(2T(/2/2) c) c 2 2 T(/2 2 ) 2c c 2 2 (2T(/2 2 /2) c) 3c 2 3 T(/2 3 ) 4c 3c 2 3 T(/2 3 ) 7c 2 3 (2T(/2 3 /2) c) 7c 2 4 T(/2 4 ) 15c 2 k T(/2 k ) (2 k T( ) - 1)c c 2T c 2 1 > 1

23 Example 3: Iteratio So far for > 2 k we have T() 2 k T(/2 k ) (2 k - 1)c If klg? T() 2 log T(/2 log ) (2 log - 1)c T(/) ( - 1)c T(1) (-1)c c (-1)c (2-1)c a log b c c log b a T() O() T( ) c 2T c 2 1 > 1

24 Example 4: Iteratio What is the recurrece equatio? MergeSort(A, left, right) { if (left < right) { } } mid floor((left right) / 2); MergeSort(A, left, mid); MergeSort(A, mid1, right); Merge(A, left, mid, right); What is the complexity of this recurrece? T( ) at c b c 1 > 1

25 Example 4: Iteratio T() at(/b) c a(at(/b/b) c/b) c a 2 T(/b 2 ) ca/b c a 2 T(/b 2 ) c(a/b 1) T( ) at a 2 (at(/b 2 /b) c/b 2 ) c(a/b 1) a 3 T(/b 3 ) c(a 2 /b 2 ) c(a/b 1) a 3 T(/b 3 ) c(a 2 /b 2 a/b 1) a k T(/b k ) c(a k-1 /b k-1 a k-2 /b k-2 a 2 /b 2 a/b 1) c b c 1 > 1

26 Example 4: Iteratio So we have T() a k T(/b k ) c(a k-1 /b k-1... a 2 /b 2 a/b 1) For k log b b k T() a k T(1) c(a k-1 /b k-1... a 2 /b 2 a/b 1) a k c c(a k-1 /b k-1... a 2 /b 2 a/b 1) ca k c(a k-1 /b k-1... a 2 /b 2 a/b 1) c(a k / a k-1 /b k-1... a 2 /b 2 a/b 1) c(a k /b k a k-1 /b k-1... a 2 /b 2 a/b 1)

27 Example 4: Iteratio The, for k log b T() c(a k /b k... a 2 /b 2 a/b 1) If a b? T() c(k 1) c(log b 1) Θ( log b ) k

28 If a < b? Note that: We have: The: ( ) ( ) ( ) ( ) b a b a b a b a b a b a b a b a k k k k k k < ! T() c Θ(1) Θ() T() c(a k /b k... a 2 /b 2 a/b 1) ( ) x x x x x x x k k k k Example 4: Iteratio

29 Example 4: Iteratio If a > b? T() c(a k /b k... a 2 /b 2 a/b 1) a b k k a b k 1 k 1! a b 1 T() c Θ(a k / b k ) k ( a b) ( a b) Θ ( a b) k ) c Θ(a log b / b log b ) c Θ(a log b / ) sice a log b log b a c Θ( log b a / ) Θ(c log b a / ) Θ( log b a )

30 Example 4: Iteratio Therefore T () Θ( ) a < b Θ( log ) a b b ( Θ log b a ) a > b

31 The Master Theorem Give: a divide ad coquer algorithm A algorithm that divides the problem of size ito a subproblems, each of size /b Let the cost of each stage (i.e., the work to divide the problem D() combie solved subproblems C()) be described by the fuctio f()d()c() The master method provides a cookbook method for solvig recurreces of the form

32 The Master Theorem (cot.) if T() at(/b) f() the T () Θ( log b a ) Θ( log b a log) Θ( f ()) f () O ( log b a ε ) f () Θ( log b a ) f () Ω( log b a ε ) AND af ( / b) cf () for large ε > 0 c <1

33 The Master Theorem (cot.) Used for the solutio of this type of recurreces T ( ) atb ( / ) f ( ) a 1, b>1, ad f asymptotically positive; Sice T() is the executio time of the algorithm, we ca state that Sub-problems of size /b are solved recursively each oe i time T(/b) f() is the cost to divide the problem ad combie its results. I MergeSort we have: T ( ) 2 T ( /2) ( ) Θ a 2, b 2, f ( ) Θ( )

34 Cookbook Method Step 1: Idetify a, b e f () Step 2: Determie log b a Step 3: Compare f () to log b a asymptotically Step 4: Accordig to the case, apply the correspodig rule

35 Example 1 T() 9T(/3) Step 1: a9, b3, f() Step 2: log b a log Step 3: f() O( log ε ), O() where ε1 Step 4: case 1 applies: if f () O ( log b a ε ) for some costat ε > 0,the T () Θ ( log b a ) Therefore T() Θ( 2 )

36 Example 2 T() T(2/3) 1 Step 1: a 1, b 3/2, f() 1 Step 2: log b a log 3/ Step 3: f() Θ( log b a ) Θ(1) Step 4: case 2 applies: if f () Θ( log b a ), the T () Θ ( log b a log) Therefore T() Θ(log )

37 Example 3 T() 3T(/4) log Step 1: a 3, b 4, f() log Step 2: log b a log Step 3: f() Ω( log 4 3ε ), where ε 0.2 Step 4: case 3 applies: If f() Ω( log b aε ) for some costat ε > 0, ad if af(/b) cf() for some costat c < 1 ad all sufficietly large, the T() Θ(f())

38 Example 3 (Cot.) If f() Ω( log b aε ) for some costat ε > 0, ad if af(/b) cf() for some costat c < 1 ad all sufficietly large, the T() Θ(f()) For sufficietly large, we have that af(/b) 3(/4) log (/4) (3/4) log for c ¾ Therefore T() Θ( log )

39 Example 4 (Merge Sort) T ( ) 2 T ( / 2) Θ( ) a b Θ log b a log 2 2, 2; 2 ( ) Give f( ) Θ( ) We have case 2: if f () Θ( log b a ), the T () Θ ( log b a log) which leads to: T () Θ( log b a log ) Θ ( log )

40 Exercises What is the recurrece equatio of these programs? fac() if 1 retur 1 else retur * fac(-1) smallest(list, start, ed) if start ed-1 retur list[start] else mid (started)/2 small1 smallest(list, start, mid) small2 smallest(list, mid1, ed) retur smallest of small1 ad small2

41 Exercises What is the recurrece equatio of these programs? idex bisearch(umber, idex low, idex high, cost keytype S[], keytype x) if low high the mid (low high) / 2 if x S[mid] the retur mid elsif x < s[mid] the retur bisearch(, low, mid-1, S, x) else retur bisearch(, mid1, high, S, x) else retur 0 ed bisearch

42 Exercises Use the substitutio, iteratio or master method to solve the followig recurreces: a) T() T(-1) O(1) (Factorial) b) T() 2T(/2) O(1) (Smallest) c) T() T(/2) Θ(1) (Biary Search)

43 Summary May useful algorithms are recursive i structure: they call themselves recursively oe or more times to deal with closely related sub-problems We also aalysed recurreces usig the followig methods: substitutio Iteratio master We have demostrated the recurrece equatio ad its complexity for factorial, smallest, ad biary search