Recursive Algorithms Recurreces Computer Sciece & Egieerig 35: Discrete Mathematics Christopher M Bourke cbourke@cseuledu A recursive algorithm is oe i which objects are defied i terms of other objects of the same type Advatages: Simplicity of code Easy to uderstad Disadvatages: Memory Speed Possibly redudat work Tail recursio offers a solutio to the memory problem, but really, do we eed recursio? Recursive Algorithms Aalysis We ve already see how to aalyze the ruig time of algorithms However, to aalyze recursive algorithms, we require more sophisticated techiques Specifically, we study how to defie & solve recurrece relatios Motivatig Factorial Recall the factorial fuctio { 1 if = 1! = ( 1)! if > 1 Cosider the followig (recursive) algorithm for computig!: Algorithm (Factorial) Iput Output :! 1 if = 1 the retur 1 3 ed : N 4 else 5 retur Factorial( 1) 6 ed Motivatig Factorial - Aalysis? Recurrece Relatios I How may multiplicatios M() does Factorial perform? Whe = 1 we do t perform ay Otherwise we perform 1 Plus how ever may multiplicatios we perform i the recursive call, Factorial( 1) This ca be expressed as a formula (similar to the defiitio of! M(0) = 0 M() = 1 + M( 1) This is kow as a recurrece relatio A recurrece relatio for a sequece {a } is a equatio that expresses a i terms of oe or more of the previous terms i the sequece, a 0, a 1,, a 1 for all itegers 0 where 0 is a oegative iteger A sequece is called a solutio of a recurrece relatio if its terms satisfy the recurrece relatio
Recurrece Relatios II The Fiboacci umbers are defied by the recurrece, F () = F ( 1) + F ( ) F (1) = 1 F (0) = 1 The solutio to the Fiboacci recurrece is ( f = 1 5 1 + 5 ) ( 1 5 1 ) 5 Recurrece Relatios III More geerally, recurreces ca have the form or T () = αt ( β) + f(), T () = αt T (δ) = c ( ) + f(), T (δ) = c β Note that it may be ecessary to defie several T (δ), iitial coditios (your book derives this solutio) Recurrece Relatios IV The iitial coditios specify the value of the first few ecessary terms i the sequece I the Fiboacci umbers we eeded two iitial coditios, F (0) = F (1) = 1 sice F () was defied by the two previous terms i the sequece Iitial coditios are also kow as boudary coditios (as opposed to the geeral coditios) From ow o, we will use the subscript otatio, so the Fiboacci umbers are f = f 1 + f f 1 = 1 f 0 = 1 Recurrece Relatios V Recurrece relatios have two parts: recursive terms ad o-recursive terms T () = T ( ) }{{} recursive + 10 }{{} o-recrusive Recursive terms come from whe a algorithm calls itself No-recursive terms correspod to the o-recursive cost of the algorithm work the algorithm performs withi a fuctio We ll see some examples later First, we eed to kow how to solve recurreces Solvig Recurreces Liear Homogeeous Recurreces There are several methods for solvig recurreces Characteristic Equatios Forward Substitutio Backward Substitutio Recurrece Trees Maple! A liear homogeeous recurrece relatio of degree k with costat coefficiets is a recurrece relatio of the form with c 1,, c k R, c k 0 a = c 1 a 1 + c a + + c k a k
Liear Homogeeous Recurreces s Solvig Liear Homogeeous Recurreces I s The Fiboacci sequece is a liear homogeeous recurrece relatio As are the followig a = 4a 1 + 5a + 7a 3 a = a + 4a 4 + 8a 8 How may iitial coditios do we eed to specify for these? As may as the degree, k = 3, 8 respectively So, how do we solve liear homogeeous recurreces? We wat a solutio of the form a = r where r is some (real) costat We observe that a = r is a solutio to a liear homogeeous recurrece if ad oly if r = c 1 r 1 + c r + + c k r k We ca ow divide both sides by r k, collect terms, ad we get a k-degree polyomial r k c 1 r k 1 c r k c k 1 r c k = 0 Solvig Liear Homogeeous Recurreces II Secod Order Liear Homogeeous Recurreces A secod order liear homogeeous recurrece is a recurrece of the form a = c 1 a 1 + c a r k c 1 r k 1 c r k c k 1 r c k = 0 This is called the characteristic equatio of the recurrece relatio The roots of this polyomial are called the characteristic roots of the recurrece relatio They ca be used to fid solutios (if they exist) to the recurrece relatio We will cosider several cases Theorem (Theorem 1, p414) Let c 1, c R ad suppose that r c 1 r c = 0 is the characteristic polyomial of a d order liear homogeeous recurrece which has two distict 1 roots, r 1, r The {a } is a solutio if ad oly if a = α 1 r 1 + α r for = 0, 1,, where α 1, α are costats depedet upo the iitial coditios 1 we discuss how to hadle this situatio later Secod Order Liear Homogeeous Recurreces Fid a solutio to a = 5a 1 6a with iitial coditios a 0 = 1, a 1 = 4 The characteristic polyomial is r 5r + 6 Usig the quadratic formula (or commo sese), the root ca be foud; r 5r + 6 = (r )(r 3) Secod Order Liear Homogeeous Recurreces Cotiued Usig the d-order theorem, we have a solutio, a = α 1 ( ) + α (3 ) Now we ca plug i the two iitial coditios to get a system of liear equatios a 0 = α 1 () 0 + α (3) 0 a 1 = α 1 () 1 + α (3) 1 1 = α 1 + α (1) 4 = α 1 + 3α () so r 1 =, r = 3
Secod Order Liear Homogeeous Recurreces Cotiued Solvig for α 1 = (1 α ) i (1), we ca plug it ito the secod 4 = α 1 + 3α 4 = (1 α ) + 3α 4 = α + 3α = α Substitutig back ito (1), we get α 1 = 1 Puttig it all back together, we have a = α 1 ( ) + α (3 ) = 1 + 3 Secod Order Liear Homogeeous Recurreces Aother Solve the recurrece a = a 1 + 15a with iitial coditios a 0 = 0, a 1 = 1 If we did it right, we have How ca we check ourselves? a = 1 8 (3) 1 8 ( 5) Sigle Root Case Recall that we ca oly apply the first theorem if the roots are distict, ie r 1 r If the roots are ot distict (r 1 = r ), we say that oe characteristic root has multiplicity two I this case we have to apply a differet theorem Theorem (Theorem, p416) Let c 1, c R with c 0 Suppose that r c 1 r c = 0 has oly oe distict root, r 0 The {a } is a solutio to a = c 1 a 1 + c a if ad oly if a = α 1 r 0 + α r 0 for = 0, 1,, where α 1, α are costats depedig upo the iitial coditios Sigle Root Case What is the solutio to the recurrece relatio a = 8a 1 16a with iitial coditios a 0 = 1, a 1 = 7? The characteristic polyomial is Factorig gives us so r 0 = 4 r 8r + 16 r 8r + 16 = (r 4)(r 4) Sigle Root Case Geeral Liear Homogeeous Recurreces By Theorem, we have that the solutio is of the form a = α 1 4 + α 4 Usig the iitial coditios, we get a system of equatios; a 0 = 1 = α 1 a 1 = 7 = 4α 1 + 4α Solvig the secod, we get that α = 3 4 Ad so the solutio is a = 4 + 3 4 4 There is a straightforward geeralizatio of these cases to higher order liear homogeeous recurreces Essetially, we simply defie higher degree polyomials The roots of these polyomials lead to a geeral solutio The geeral solutio cotais coefficiets that deped oly o the iitial coditios I the geeral case, however, the coefficiets form a system of liear iequalities We should check ourselves
Geeral Liear Homogeeous Recurreces I Distict Roots Geeral Liear Homogeeous Recurreces Ay Multiplicity Theorem (Theorem 3, p417) Let c 1,, c k R Suppose that the characteristic equatio r k c 1 r k 1 c k 1 r c k = 0 has k distict roots, r 1,, r k The a sequece {a } is a solutio of the recurrece relatio Theorem (Theorem 4, p418) Let c 1,, c k R Suppose that the characteristic equatio r k c 1 r k 1 c k 1 r c k = 0 if ad oly if a = c 1 a 1 + c a + + c k a k has t distict roots, r 1,, r t with multiplicities m 1,, m t a = α 1 r 1 + α r + + α k r k for = 0, 1,,, where α 1, α,, α k are costats Geeral Liear Homogeeous Recurreces Ay Multiplicity Theorem (Cotiued) The a sequece {a } is a solutio of the recurrece relatio if ad oly if a = c 1 a 1 + c a + + c k a k a = (α 1,0 + α 1,1 + + α 1,m1 1 m1 1 )r1 + (α,0 + α,1 + + α,m 1 m 1 )r + (α t,0 + α t,1 + + α t,mt 1 mt 1 )r t + For recursive algorithms, cost fuctios are ofte ot homogeous because there is usually a o-recursive cost depedig o the iput size Such a recurrece relatio is called a liear ohomogeeous recurrece relatio Such fuctios are of the form a = c 1 a 1 + c a + + c k a k + f() for = 0, 1,,, where α i,j are costats for 1 i t ad 0 j m i 1 Here, f() represets a o-recursive cost If we chop it off, we are left with a = c 1 a 1 + c a + + c k a k which is the associated homogeous recurrece relatio Every solutio of a liear ohomogeeous recurrece relatio is the sum of a particular solutio ad a solutio to the associated liear homogeeous recurrece relatio Theorem (Theorem 5, p40) If {a (p) } is a particular solutio of the ohomogeeous liear recurrece relatio with costat coefficiets a = c 1 a 1 + c a + + c k a k + f() the every solutio is of the form {a (p) + a (h) }, where {a (h) } is a solutio of the associated homogeous recurrece relatio a = c 1 a 1 + c a + + c k a k
Theorem (Theorem 6, p41) There is o geeral method for solvig such relatios However, we ca solve them for special cases I particular, if f() is a polyomial or expoetial fuctio (or more precisely, whe f() is the product of a polyomial ad expoetial fuctio), the there is a geeral solutio Suppose that {a } satisfies the liear ohomogeeous recurrece relatio a = c 1 a 1 + c a + + c k a k + f() where c 1,, c k R ad f() = (b t t + b t 1 t 1 + + b 1 + b 0 ) s where b 0,, b, s R I Theorem (Cotiued) Whe s is ot a root of the characteristic equatio of the associated liear homogeeous recurrece relatio, there is a particular solutio of the form (p t t + p t 1 t 1 + + p 1 + p 0 ) s Whe s is a root of this characteristic equatio ad its multiplicity is m, there is a particular solutio of the form m (p t t + p t 1 t 1 + + p 1 + p 0 ) s s (Rose): a = 3a 1 + Homogeous solutio form: a (h) = α3 Hetrogeeous form: p = c + d sice f() = is a liear fuctio Test: c + d = 3(c( 1) + d) + Collect terms, factor out a egatio: (c + ) + (d 3c) = 0 Total solutio: a = a (h) + a (p) = α3 + ( 3/) II a = 5a 1 6a + 7 Homogeous solutio form: a (h) = α 1 + α 3 Hetrogeeous form: p = c 7 sice f() = 7 is a expoetial fucito c7 = 5c7 1 6c7 + 7 (divide by 7 ad solve for c) Total solutio: a = a (h) + a (p) = α 1 + α 3 + 49 0 7 I Applicatio Recall that the umber of additios performed by the recursive fiboacci code ca be characterized as: A() = A( 1) + A( ) + 1 We ow recogize this as a liear, ohomogeeous recurrece relatio of degree Solvig the homogeous part we get: a = α 1 ( 61803) + α (161803) Note that φ = 161803 is the golde ratio To solve the o-homogeeous part (a costat fuctio), we assume it has a solutio of the form a = c ad so: c = c + c + 1
II Applicatio thus c = 1; altogether: a = a (h) + a (p) = α 1 ( φ + 1) + α (φ) 1 For A(0) = A(1) = 0 we get the regular solutio to the fiboacci recurrece mius 1: a = f 1 which is expoetial i Other Methods Whe aalyzig algorithms, liear homogeous recurreces of order greater tha hardly ever arise i practice We briefly describe two ufoldig methods that work for a lot of cases Backward substitutio this works exactly as its ame implies: startig from the equatio itself, work backwards, substitutig values of the fuctio for previous oes Recurrece Trees just as powerful but perhaps more ituitive, this method ivolves mappig out the recurrece tree for a equatio Startig from the equatio, you ufold each recursive call to the fuctio ad calculate the o-recursive cost at each level of the tree You the fid a geeral formula for each level ad take a summatio over all such levels Backward Substitutio Give a solutio to where T (1) = 5 T () = T ( 1) + We begi by ufoldig the recursio by a simple substitutio of the fuctio values Observe that T ( 1) = T (( 1) 1) + ( 1) = T ( ) + ( 1) Substitutig this ito the origial equatio gives us T () = T ( ) + ( 1) + Backward Substitutio Cotiued If we cotiue to do this, we get the followig T () = T ( ) + ( 1) + = T ( 3) + ( ) + ( 1) + = T ( 4) + ( 3) + ( ) + ( 1) + = T ( i) + i 1 j=0 ( j) Ie this is the fuctio s value at the i-th iteratio Solvig the sum, we get (i 1)(i 1 + 1) T () = T ( i) + (i 1) + + Backward Substitutio Cotiued We wat to get rid of the recursive term To do this, we eed to kow at what iteratio we reach our base case; ie for what value of i ca we use the iitial coditio, T (1) = 5? We ca easily see that whe i = 1, we get the base case Substitutig this ito the equatio above, we get T () = T ( i) + (i 1) i + i + = T (1) + ( 1 1) ( 1) + ( 1) + = 5 + ( ) ( + 1) + ( 1) + = + + 3 Recurrece Trees Whe usig recurrece trees, we graphically represet the recursio Each ode i the tree is a istace of the fuctio As we progress dowward, the size of the iput decreases The cotributio of each level to the fuctio is equivalet to the umber of odes at that level times the o-recursive cost o the size of the iput at that level The tree eds at the depth at which we reach the base case As a example, we cosider a recursive fuctio of the form ( ) T () = αt + f(), T (δ) = c β
Recurrece Trees Recurrece Trees Iteratio 0 1 T (/β) T () α T (/β) Cost f() ( ) α f β The total value of the fuctio is the summatio over all levels of the tree: log β ( ) T () = α i f β i i=0 T (/β ) α T (/β ) T (/β ) α T (/β ) ( ) α f β We cosider the followig cocrete example i log β ( ) α i f β i α log β T (δ) ( ) T () = T +, T (1) = 4 Recurrece Trees Cotiued Recurrece Trees Cotiued Iteratio 0 T () Cost 1 T (/4) T (/) T (/4) T (/4) T (/) T (/4) + 4 ( 4 ) The value of the fuctio the, is the summatio of the value of all levels We treat the last level as a special case sice its o-recursive cost is differet 3 8 ( 8 ) (log ) 1 T () = 4 + i = (log ) + 4 i i=0 i i i log log T (1) Smoothess Rule I Smoothess Rule II I the previous example we make the followig assumptio: that was a power of two; = k This was ecessary to get a ice depth of log ad a full tree We ca restrict cosideratio to certai powers because of the smoothess rule A fuctio f : N R is called smooth if it is mootoically odecreasig ad f() Θ(f()) Theorem For a smooth fuctio f() ad a fixed costat b Z such that b, f(b) Θ(f()) Thus the order of growth is preserved Most slow growig fuctios (logarithmic, polylogarithmic, polyomial) are smooth while expoetial fuctios are ot
How To Cheat With Maple I How To Cheat With Maple II Maple ad other math tools are great resources However, they are ot substitutes for kowig how to solve recurreces yourself As such, you should oly use Maple to check your aswers Recurrece relatios ca be solved usig the rsolve commad ad givig Maple the proper parameters The argumets are essetially a comma-delimited list of equatios: geeral ad boudary coditios, followed by the ame ad variable of the fuctio > rsolve({t() = T(-1) + *, T(1) = 5}, T()); ( ) 1 1 + ( + 1) + 1 You ca clea up Maple s aswer a bit by ecapsulatig it i the simplify commad: > simplify(rsolve({t() = T(-1) + *, T(1) = 5}, T())); 3 + +