Sectio IV.6: The Mster Method d Applictios Defiitio IV.6.1: A fuctio f is symptoticlly positive if d oly if there exists rel umer such tht f(x) > for ll x >. A cosequece of this defiitio is tht fuctio f is symptoticlly positive if d oly if the coefficiet of its domit term is positive. The domit term t is the oe with the lrgest growth rte, i. e. t > u for ll other terms i f, > defied s i sectio II.5. The mster method provides us strightforwrd d ÒcookookÓ method for solvig recurreces of the form T() = T( ) + f(), where 1 d > 1 re costts d f() is symptoticlly positive fuctio. This recurrece gives us the ruig time of lgorithm tht divides prolem of size ito suprolems of size. This recurrece is techiclly correct oly whe is iteger, so the ssumptio will e mde tht is either or sice such replcemet does ot ffect the symptotic ehvior of the recurrece. is positive iteger sice oe c hve oly whole umer of suprolems. Theorem IV.6.1: Mster Theorem: Let 1 d > 1 e costts d f() e fuctio, d let T() e defied o the oegtive itegers y T() = T( ) + f() where is treted s ove. The T() c e ouded symptoticlly s follows: (1). If f() = O( ε ) for some ε >, the T() = Θ( ). (). If f() = Θ( ), the T() = Θ ( log ). (3). If f() = Ω( +ε ) for some costt ε >, d if f( ) c f() for some costt c < 1 d ll sufficietly lrge, the T() = Θ(f()). Note: The reltio f( ) c f() is clled the regulrity coditio o f(). Ituitive expltio: I the Mster theorem oe compres d f(). If these fuctios re i the sme Θ clss (see sectio II.5), the we multiply y logrithmic fctor to get the ru time of T() (cse ). If f() is polyomilly smller th (y ε fctor of ) the T() is i the sme Θ clss s (cse 1). If f() is polyomilly lrger th, the T() is i the sme Θ clss s f() (cse 3). I cses 1 d 3, the fuctios must e polyomilly lrger or smller th. Exmple IV.6. illustrtes 1
cse where the fuctio is ot polyomilly lrger or smller th Mster Theorem does ot pply.. Here the Exmple IV.6.1: Get Θ estimte, coefficiet 1, for T() = 16 T( ) +. Solutio: Here = 16 d =, so log =. f() = = O() = O ( 1 ), so we c tke ε = 1. Therefore cse (1) of Mster Theorem holds, d, therefore, T() = Θ( ). Exmple IV.6.: Get Θ estimte, coefficiet 1, for T() = T 3 +. Solutio: Here = 1, = 3, d f() =. log = sice = 1. f() = = Θ(1) = Θ( ). Therefore, cse () of the Mster Theorem holds. So T() = Θ(1 log ) = Θ( log ). Exmple IV.6.3: Get Θ estimte, coefficiet 1, for T() = 3T +. Solutio: Here = 3, =, d f() =. log 3 =. 793. It is true tht. 793. f() = Ω( + ), where ε., sice. 993. Cse 3 of Mster Theorem pplies if we c fid c < 1 where 3f( ) c f(). 3 3 f( ) = 3( log ) = c f() for c =. Therefore, the regulrity coditio holds for f(). Cse 3 holds, so T() = Θ( ). Exmple IV.6.: Show tht the Mster Theorem cot e pplied to the fuctio T() = T( ) +. Solutio: Here =, =, d f() =. =. f() = is symptoticlly lrger th (fster growth rte) ut is ot polyomilly lrger th sice log < ε for y ε >. (Theorem II.5.6). CormÕs ook Itroductio to Algorithms gives detiled proof of the Mster Theorem. Here we give lemms d drw recursio tree to illustrte the cses of the Mster theorem. See CormÕs text for more i-depth pproch.
Lemm IV.6.1: Let 1 d > 1 e costts d f() e oegtive fuctio defied o exct powers of. Defie T() o exct powers of y the recurrece Θ( 1) if = 1 T() = i T( )+ f ( ) if = where i is positive iteger. The T() = Θ( j j ) + f( / ). Proof: Itertig the recurrece gives T () = f ( ) + T( ) = f ( ) + f ( ) + T( ) = ( ) ( log 1) log f ( ) + f + f ) + + f ( ) + T(1). k We stop the ove process whe ( ) 1, i.e., whe k log. Sice (*) log =, the lst term of the ove expressio ecomes log T(1) = Θ( ), usig the coditio T(1) = Θ(1). The remiig terms c e expressed s the sum j j f( ). Here we re summig cross the levels of the recursio tree elow. At the level j, there re j odes. log 1 j Therefore T() = Θ( j ) + f( / ), which completes the proof. Derivtio of (*): log = log log y the chge of se formul log log x = log log y log = x ( ) ( ) log log = x log y x log = log log = sice log fuctio is 1 to 1. 3
Below is drw recursio tree for T() = T( /) + f() Ech pret hs childre f() f(/) f(/) f(/) F(/ f(/ ) f(/ ) F(/ f(/ ) f(/ ) F(/ f(/ ) f(/ ) Θ(1) Θ(1) Θ(1) Θ(1) Θ(1) Θ(1) The height of the tree is log which is the umer of times we divide y efore gettig 1. Usig (*), it follows tht there re leves. Their sum is Θ( ). From the tree, oe c see tht the the sum of the vlues i the odes is Θ( j j ) + f( / ). This is true, sice s we go dow to the ext level of the tree, the umer of odes is multiplied y, d ew fctor of ppers i the deomitor of the rgumet of f. The recursio tree illustrtes the three cses of the Mster Theorem. I cse 1, the totl cost of the tree is domited y the cost of the leves, which is Θ( j j ). The cost of the iterl odes, cotriuted y f( / ) smller th tht of the leves, so y Theorem II.5.3, the growth rte of T is the mximum of the two costs, mely Θ( ). is polyomilly I cse, the totl cost is evely distriuted mog the levels of the tree. The lysis of this cost is geerliztio of tht of Merge Sort, foud i sectio IV.5. The cost t ech level of the tree is Θ( ), d there re log levels of the tree. So the totl cost is
Θ ( log ). Here we use Theorem II.5. (multiplictive property of Θ) d the fct tht the growth rte of log fuctio is idepedet of the se. I cse 3, the totl cost is domited y the root. The costrit f(/) c f() gives us coverget ifiite series i the derivtio of cost of T() d gurtees the domice of f(). The ext lemm, give without proof, with the previous lemm, leds directly to the Mster Theorem, for it defies g() = ehvior s g() i cses d 3 of the Mster Theorem. j j f( / ) d sserts tht T() hs the sme Lemm IV.6.: Let 1 d > 1 e costts d f() e oegtive fuctio defied o exct powers of. A fuctio g() defied over exct powers of y g() = j j f( / ) c e ouded symptoticlly for exct powers of s follows. (1) If f() = O( ) for some >, the g() = O( () If f() = Θ( ), the g() = Θ ( log ). ). (3) If f( ) c f() for some costt c < 1 d ll, the g() = Θ(f()). Clim: Lemms IV.6.1 d IV.6. directly led to the Mster Theorem. Proof of Clim: Cse 1: Sice g() = O( ) (y Lemm IV.6.), we hve T() = Θ( ) + g() (y Lemm IV.6.1) = Θ( ) s sserted i cse 1 of the Mster Theorem. Cse : Sice g() = Θ ( d = O( log ), we hve T() = Θ( ) + Θ ( = Θ ( log ) (y Lemm IV.6.) log ) (y Lemm IV.6.1) log ) s sserted i cse of the Mster Theorem. Cse 3: Sice f() is polyomilly lrger th (s stted i the Mster Theorem) d g() = Θ(f()) (y Lemm IV.6.), we hve T() = Θ( ) + Θ(f()) (y Lemm IV.6.1) = Θ(f()) s sserted i cse 3 of the Mster Theorem, sice f() is polyomilly lrger tht g(). 5
Note: CormÕs ook Itroductio to Algorithms, p.7 illustrtes fuctio etwee cse d cse 3 of the Mster Theorem. Here f() = Θ( k log ) d T() = Θ( log + ). k 1 Exercises: (1) Use the Mster theorem to get the Θ estimte with ledig coefficiet 1 for the followig recurreces. Clerly idicte which of the three cses hold, d, for cse 3, show tht the regulrity coditio holds. () T() = T( ) +. () T() = T( ) + (c) T() = T( ) + 3 () () Use the Mster theorem to show ru time of iry serch is Θ( log ). Here T() = T( ) + 1. () Use the Mster theorem to show tht the ru time of Merge Sort is Θ( log ) Here T() = T( ) +. (3) Use the Mster theorem to get the Θ estimte with ledig coefficiet 1 for T() = 3T( ) +. Your swer should e the sme s Exmple IV.5.5 i sectio IV.5. () Use the Mster theorem to get the Θ estimte with ledig coefficiet 1 for T() = 3T( ) +. Your swer should e the sme s Exmple IV.5.6 i sectio IV.5. (5) The ruig time of lgorithm A is descried y T() = 7T( ) +. Aother lgorithm A hs ruig time T () = T ( ) +. Wht is the lrgest vlue of such tht A is symptoticlly fster th A? Justify your swer, showig relevt clcultios. 6