Algorithms ad Data Structures Lecture IV Simoas Šalteis Aalborg Uiversity simas@cs.auc.dk September 5, 00 1
This Lecture Aalyzig the ruig time of recursive algorithms (such as divide-ad-coquer) Writig ad solvig recurreces September 5, 00
Recurreces Ruig times of algorithms with Recursive calls ca be described usig recurreces A recurrece is a equatio or iequality that describes a fuctio i terms of its value o smaller iputs. For divide-ad-coquer algorithms: solvig_trivial_problem if = 1 T ( ) = um_pieces T ( / subproblem_size_factor) + dividig + combiig if > 1 Example: Merge Sort T ( ) Θ (1) if = 1 = T ( /) +Θ ( ) if > 1 September 5, 00 3
Solvig Recurreces Repeated (backward) substitutio method Expadig the recurrece by substitutio ad oticig a patter Substitutio method guessig the solutios verifyig the solutio by the mathematical iductio Recursio-trees Master method templates for differet classes of recurreces September 5, 00 4
Repeated Substitutio Let s fid the ruig time of the merge sort (let s assume that = b, for some b). 1 if = 1 T( ) = T( /) + if > 1 ( ) T( / 4 ) / expad T ( ) = T/ + substitute ( ) = + + = + T ( / 4) substitute = + + ( T ( /8) / 4) expad 3 T ( /8) 3 observe the patter = + T ( ) = i i T ( / ) + i = lg T ( / ) + lg = + lg September 5, 00 5
Repeated Substitutio Method The procedure is straightforward: Substitute Expad Substitute Expad Observe a patter ad write how your expressio looks after the i-th substitutio Fid out what the value of i (e.g., lg) should be to get the base case of the recurrece (say T(1)) Isert the value of T(1) ad the expressio of i ito your expressio September 5, 00 6
Repeated Substitutio (Ex. ) Let s fid a more exact ruig time of merge sort (let s assume that = b, for some b). T( ) if = 1 = T( /) + + 3 if > 1 ( ) T( ) ( ) T( ) T ( ) = T/ + + 3 substitute ( ) = / 4 + + 3 + + 3 expad = + + + T / 4 4 3 3 substitute ( ) = + + + + + / 8 / 3 4 3 3 expad ( ) ( ) = + + 3 3 1 0 T / 3 + + 3 observe the patter i 1 i i j T ( ) = T ( / ) + i+ 3 j= 0 = + + = + lg lg T ( / ) lg 3( 1) 5 lg 3 September 5, 00 7
Repeated Substitutio (Ex. 3) Let s fid the ruig time of the tromio tilig algorithm for a x board. T( ) 1 if = 1 = 4 T( 1) + 1 if > 1 T ( ) 44 ( T( ) 1) 1 expad 4 T( ) 4 1 substitute 4 ( 4T( 3) 1) 4 1 expad T ( ) = 4 1 + 1 substitute = + + = + + = + + + ( ) ( ) = + 3 1 0 4 T 3 4 +4 +4 observe the patter i 1 i T ( ) = 4 T ( i) + 4 j= 0 1 4 1 4 1 1 = 4 T (1) + = 4 1 3 j September 5, 00 8
Substitutio method Solve T ( ) = 4 T ( / ) + 1) Guess that T ( ) = O ( ), i.e., that Tof the form c 3 ) Assume Tk ( ) ck for k / ad 3 3) Prove T ( ) c by iductio 3 3 T ( ) = 4 T ( / ) + (recurrece) T 3 4c(/) (id. hypoth.) + 3 Thus ( ) ( )! c 3 = + (simplify) 3 c 3 = c (rearrage) 3 c if c ad 1 (satisfy) = O Subtlety: Must also check the base case (ot ecessarily T (1)) September 5, 00 9
Substitutio Method Achievig tighter bouds T Try to show ( ) ( ) Assume Tk ( ) ck T ( ) = 4 T ( /) + + 4( c/) = + c = O > c for o choice of c 0. September 5, 00 10
Substitutio Method () The problem? We could ot rewrite the equality as: T = ( ) c+ (somethig positive) T ( ) c i order to show the iequality we wated Sometimes to prove iductive step, try to stregthe your hypothesis T() (aswer you wat) - (somethig > 0) September 5, 00 11
Substitutio Method (3) Corrected proof: the idea is to stregthe the iductive hypothesis by subtractig lower-order terms! Assume Tk ( ) ck 1 ck for k< T ( ) = 4 T ( /) + 4( c ( / ) c ( / )) + = c c+ = c c ( c ) c cif c 1 1 1 1 1 September 5, 00 1
Recursio Tree A recursio tree is a coveiet way to visualize what happes whe a recurrece is iterated Good for guessig asymtotic solutios to recurreces T ( ) = T ( /4) + T ( /) + September 5, 00 13
Recursio Tree () September 5, 00 14
Master Method The idea is to solve a class of recurreces that have the form T ( ) = atb ( / ) + f ( ) a 1ad b > 1, ad f is asymptotically positive. Abstractly speakig, T() is the rutime for a algorithm ad we kow that a subproblems of size /b are solved recursively, each i time T(/b) f() is the cost of dividig the problem ad combiig the results. I merge-sort T ( ) = T ( /) +Θ( ) September 5, 00 15
Master Method () Split problem ito a parts at log b logb logba levels. There are a = leaves September 5, 00 16
Master Method (3) Number of leaves: a log Iteratig the recurrece, expadig the tree yields T ( ) = f ( ) + atb ( / ) The first term is a divisio/recombiatio cost (totaled across all levels of the tree) log The secod term is the cost of doig all b a subproblems of size 1 (total of all work pushed to leaves) b = log = f + af b + a T b ( ) ( / ) ( / ) = f + af b + a T b + ( ) ( / ) ( / )... logb 1 logb 1 logb + a f( / b ) + a T(1) Thus, logb 1 j j logb a ( ) = ( / ) +Θ( ) T afb j= 0 b a September 5, 00 17
MM Ituitio Three commo cases: Ruig time domiated by cost at leaves Ruig time evely distributed throughout the tree Ruig time domiated by cost at the root Cosequetly, to solve the recurrece, we eed oly to characterize the domiat term logb I each case compare f( ) with O ( a ) September 5, 00 18
MM Case 1 logb a ε f( ) = O( ) for some costat ε > 0 f() grows polyomially (by factor ) slower log tha b a The work at the leaf level domiates logb a Summatio of recursio-tree levels O ( ) logb a Cost of all the leaves Θ( ) logb a Thus, the overall cost Θ( ) ε September 5, 00 19
MM Case logb f( ) =Θ( a ) log ( ) ad b a are asymptotically the same f The work is distributed equally logb a throughout the tree T ( ) =Θ( lg ) (level cost) (umber of levels) September 5, 00 0
MM Case 3 logb a+ε f( ) =Ω( ) for some costat Iverse of Case 1 f() grows polyomially faster tha Also eed a regularity coditio log b a c < 1 ad > 0 such that af ( / b) cf ( ) > 0 0 The work at the root domiates T ( ) =Θ( f ( )) ε > 0 September 5, 00 1
Master Theorem Summarized Give a recurrece of the form 1. f( ) = O T ( ) =Θ. f( ) =Θ logb a ε ( ) logb a ( ) logb a ( ) logb a ( ) logb a+ε ( ) T ( ) =Θ lg ( f) T ( ) = atb ( / ) + f ( ) 3. f( ) =Ω ad af( / b) cf( ), for some c< 1, > T ( ) =Θ ( ) The master method caot solve every recurrece of this form; there is a gap betwee cases 1 ad, as well as cases ad 3 September 5, 00 0
Strategy Extract a, b, ad f() from a give recurrece Determie log b a Compare f() ad asymptotically Determie appropriate MT case, ad apply Example merge sort T ( ) = T ( /) +Θ( ) log b a a= b= = = =Θ Also f( ) =Θ( ) log b a log, ; ( ) ( log ) b a ( ) Case : T ( ) =Θ lg =Θ lg September 5, 00 3
Examples of MM T ( ) = T ( / ) + 1 a = b= = log1 1, ; 1 also f( ) = 1, f( ) =Θ(1) Case : T ( ) =Θ(lg ) Biary-search(A, p, p, r, r, s): s): q (p+r)/ if if A[q]=s the the retur q else else if if A[q]>s the the Biary-search(A, p, p, q-1, q-1, s) s) else else Biary-search(A, q+1, q+1, r, r, s) s) T ( ) = 9 T ( /3) + a = 9, b= 3; f = f = O ε= log3 9 ε ( ), ( ) ( ) with 1 Case 1: T ( ) =Θ ( ) September 5, 00 4
Examples () T ( ) = 3 T ( /4) + lg = 3, = 4; = 4 a b log 3 0.793 log4 3+ε ( ) lg, ( ) ( ) with 0. f = f =Ω ε Case 3: Regularity coditio af ( / b) = 3( /4)lg( / 4) (3/ 4) lg = cf ( ) for c = 3/ 4 T ( ) =Θ( lg ) T ( ) = T ( /) + lg =, = ; = a b log 1 1+ε f ( ) = lg, f( ) =Ω( ) with ε? also l 1 g / lg = either Case 3 or Case! September 5, 00 5
Examples: Multiplicatio (I) Multiplyig two -digit (or -bit) umbers costs digit multiplicatios usig a classical procedure. Observatio: 3*14 = ( 10 1 +3)*(1 10 1 +4) = (*1)10 + (3*1 + *4)10 1 + (3*4) To save oe multiplicatio we use a trick: (3*1 + *4) = (+3)*(1+4) - (*1) - (3*4) September 5, 00 6
Examples: Multiplicatio (II) To multiply a ad b, which are -digit umbers, we use a divide ad coquer algorithm. We split them i half: a = a 1 10 / + a 0 ad b = b 1 10 / + b 0 The: a *b = (a 1 *b 1 )10 + (a 1 *b 0 + a 0 *b 1 )10 / + (a 0 *b 0 ) Use a trick to save a multiplicatio: (a 1 *b 0 + a 0 *b 1 ) =(a 1 +a 0 )*(b 1 +b 0 ) - (a 1 *b 1 ) - (a 0 *b 0 ) September 5, 00 7
Examples: Multiplicatio (III) The umber of sigle-digit multiplicatios performed by the algorithm ca be described by a recurrece: T( ) Solutio: 1 if = 1 = 3 T( /) if > 1 T( ) = = log 3 1.585 September 5, 00 8
Next Week Sortig QuickSort HeapSort September 5, 00 9