Recurrece Relatios Aalysis of recursive algorithms, such as: it factorial (it ) { if (==0) retur ; else retur ( * factorial(-)); } Let t be the umber of multiplicatios eeded to calculate factorial(). The t = t - +. The equatio, t = t - +, is a example of a recurrece equatio or recurrece relatio. Recurrece relatios are usually accompaied by iitial coditio(s). For the factorial fuctio above, the iitial coditio would be writte as t 0 = 0; which idicates that o multiplicatios are calculated whe is 0. The first few values for the above recurrece relatio (begiig with the iitial coditio) are: t 0 = 0 t = t 0 + = 0 + = t = t + = + = t = t + = + = t 4 = t + = + = 4 From this patter we might guess that a o-recurrece expressio for t might be t =. Such a expressio is called a solutio to the recurrece relatio. May times we ca guess at the solutio of a recurrece relatio by lookig for a patter i the first few iteratios. If a potetial or cadidate solutio is foud by observatio, we still eed to prove that it does, ideed, solve the recurrece relatio. Such proofs ofte use a iductio style of proof. Whe provig somethig by iductio, oe first shows that cadidate solutio satisfies a base case. For istace, it is obvious that for = 0, it is true that t = 0. The ext step i a proof by iductio is to state the iductio hypothesis. I the iductio hypothesis, oe makes the assumptio that the cadidate solutio is true for the i th case (i 0). For example, for this problem oe would simply state the assumptio that t i = i. The fial step (sometimes called the iductio step ) is to show that assumig the iductio hypothesis is true (the solutio is true for the i th case) that the solutio is true for the i+ case. For our example problem, oe would show that: t i+ = t (i+)- + = t i + = i + DONE! Examples: a. Solve the recurrece relatio: t = t / +, where > is a power of ad t = (This recurrece relatio occurs with the worst case lookup usig a biary search.) t = t = t / + = t + = + = t 4 = t 4/ + = t + = + = t 8 = t 8/ + = t 4 + = + = 4 t 6 = t 6/ + = t 8 + = 4 + = From observatio, a cadidate solutio would be: t = log () + Usig proof by iductio: Base case: t = Iductio Hypothesis: Assume t i = log (i) + Iductio Step: t i = t i/ + = t i + = (log (i) + ) + = log (i) + log () + = log (i) + Doe! Page
b. Solve t = 7t /, where > ad is a power of ad t =. t = t = 7t / = 7t = 7 t 4 = 7t 4/ = 7t = 7*7 = 7 t 8 = 7t 8/ = 7t 4 = 7*7 = 7 t 6 =7 t 6/ = 7t 8 = 7*7 = 7 4 adidate: t = 7 log () Proof: Base: t = = 7 log () Iductio Hypothsis: t i = 7 log (i) Iductio Step: t i = 7t i/ = 7t i = 7 * 7 log (i) = 7 (+log (i)) = 7 (log () + log (i)) = 7 log (i) c. Solve t = t / +, where > ad is a power of ad t = 0. t = 0 t = t + = *0 + = t 4 = t + 4 = * + 4 = t 8 = t 4 + 8 = * + 8 = 7 t 6 = t 8 + 6 = *7 + 6 = 49 Solutio? (We ll retur to this problem later!) Homogeeous Liear Recurreces A homogeeous liear recurrece is ay recurrece that ca be put i the followig format: a 0 t + a t - + + a k t -k = 0 where each a i is a costat (i = 0... k) These are liear because each t i term appears oly to the st power (o t, t *t -, or t / ). They are homogeeous because the right-had side of the equatio is equal to 0. Example: Aalysis of the recursive fuctio to fid the th Fiboacci umber leads to a homogeeous liear recurrece relatio. it Fib(it ) { if (<=) retur ; else retur ( fib(-) + fib(-) ); } Let t = umber of additios eeded to fid Fib(). The t = t - + t -, with t 0 = 0 & t =. This ca be coverted to the format: t t - t - = 0, with t 0 = 0 & t =. The first few terms are: t = t + t 0 = + 0 = t = t + t = + = t 4 = t + t = + = t = t 4 + t = + = Page
Before we fid a solutio to this homogeeous liear recurrece relatio, let s try a easier problem. Let s solve the recurrece relatio: t t - + 6t - = 0, for > with t 0 = 0 ad t =. If we set t = r, the a root, r, of the polyomial: r r - + 6r - = 0, will give us a solutio to the recurrece relatio. r r - + 6r - = 0 r - ( r -r + 6) = 0 r - ( r-) (r-) = 0 Thus, the roots are r=0, r=, ad r=; ad some of the solutios to the recurrece relatio should be t = 0, t =, ad t = Verify: t = 0: 0 *0 +6*0 = 0 Verify: t = : * - + 6* - = * - +* - = -* - = = 0 Verify: t = : * - + 6* - = * - + * - = * - = = 0 t = 0 is a vacuous solutio that fails to satisfy the iitial coditios. We ll elimiate it. If t = ad t = are solutios, the so is t = +. a you verify this? I fact, t = + is a solutio for ay costats &. We ca use the iitial coditios (t 0 =0, t =) to fid the particular solutio that satisfies the iitial coditios: t = +, t 0 = 0 ad t = 0 = + ad = + Solvig these two equatios for the two ukows yields: = - ad = +. Thus, the particular solutio that satisfies the iitial coditios is: t = You should be able to verify that the equatio: t = - ( 0) geerates the same sequece as the recurrece relatio: t = t - - 6t - for > with t 0 = 0 ad t =. Defiitio: Give a homogeous liear recurrece relatio: a 0 t + a t - + + a k t -k = 0, the characteristic equatio is defied as a 0 r k + a r k- + + a k r 0 = 0 What are the characteristic equatios for the followig recurrece relatios? Recurrece Relatio haracteristic Equatio t 7t - + 6t - = 0 t t - 4t - = 0 t t - t - = 0 Theorem: If the characteristic equatio has k distict roots: r, r,, r k, the recurrece relatio has solutio: t = r + r + k r k Theorem: If the characteristic equatio has a root, r, of multiplicity = m, the t = r, t = r, t = r, t = r, t = m- r are all solutios to the recurrece relatio. Page
Examples: a) (Back to the Fiboacci example) t t - t - = 0, with t 0 = 0 ad t = haracteristic equatio: r r = 0 Roots: r = ( + ) / ad r = ( - ) / Geeral Solutio: t = + Applyig iitial coditios yields the followig equatios i ukows: 0 = + = + Which yields = ad = Specific Solutio: t = b) t 4t - + 4t - = 0, where > 0 ad t 0 = 0 & t = haracteristic Equatio: r 4r + 4 = 0 Roots: r = with multiplicity of Geeral Solutio: t = + Applyig iitial coditios: 0 = = + Which yields: = 0 & = ½ Specific Solutio: t = ½ = - Page 4
Nohomogeeous Liear Recurreces A homogeeous liear recurrece is ay recurrece that ca be put i the followig format: a 0 t + a t - + + a k t -k = f() where each a i is a costat (i = 0... k) ad f() is ay fuctio i. There is o kow geeral method for solvig ohomogeeous liear recurreces. However there is a method for the special case: a 0 t + a t - + + a k t -k = b p(), where b is a costat ad p() is a polyomial i. Examples: t t - = 4 t t - = 4 (+) The st example ca be made homogeeous by the applyig the followig trick: t t - = 4 t - t - = 4 - Also t t - = 4 t t = 4-4 4 Subtractig the st equatio from the d yields: t 7t + t - = 0 4 4 Multiplyig by 4: t 7t - + t - = 0 The characteristic equatio for this homogeeous recurrece is: r -7t + = 0 Which factors ito: (r-)(r-4) = 0 Thus the geeral solutio is: t = + 4 Usig the iitial coditios, t 0 = 0 ad t = 4, we get = -4 ad = 4 Thus, the specific solutio is: t = 4 + 4( ) Notice that the 4( ) term would have bee preset if the origial recurrece relatio had bee the homogeeous relatio: t t - = 0. The other term (4 + ) comes from the fuctio o the right-had side of the origial recurrece relatio. This leads us to the followig theorem: Theorem: A recurrece relatio i the form: a 0 t + a t - + + a k t -k = b p(), has a characteristic equatio i the form: (a 0 r k + a r k- + + a k )(r-b) d+ = 0, where b is the costat foud i the recurrece relatio ad d is the degree of p(). Thus the characteristic equatio is composed of i) the characteristic equatio correspodig to a homogeeous versio of the recurrece relatio, ad ii) (r-b) d+ which has a root of multiplicity d+. If there is more tha oe term like b p() o the right side, each oe cotributes a product such as (r-b) d+ to the characteristic equatio. Page
Examples: a) t t - = 4 (+) with iitial coditios t 0 = 0 ad t =. haracteristic Equatio: (r-)(r-4) = 0 Which has roots: r = ad r = 4 with multiplicity Geeral solutio: t = + 4 + 4 We eed to fid a value for t so that we ca derive equatios i ukows: t t = 4 (4+) = 6* =80 ad, thus, t = t + 80 = () + 80 = 6 Solvig for,, ad : 4 4 = 0, = -0, = 8 9 6 6 0 Thus, the specific solutio is: t = 0( ) 0(4 ) + 84 b) t t - = - where > 0 ad t 0 = 0 haracteristic Equatio: (r-)(r-) = 0 (r-) = 0 Which has roots: r = with multiplicity Geeral solutio: t = + + = + + Give t 0 = 0: t = t 0 + (-) = 0 + 0 = 0 t = t + (-) = 0 + = 0 Solvig for,, ad : 0 = 0, = -½, = ½ 4 Thus, the specific solutio is: t = - + = = ( ) c) t t - = + where > ad t = 0 haracteristic Equatio: (r-)(r-) (r-) = 0 Which has roots: r = with multiplicity ad r= with multiplicity Geeral solutio: t = + + + 4 = + + + 4 Give t = 0: t = t + + = 0 + + 4 = 6 t = t + + = *6 + + 8 = t 4 = t + 4 + 4 = * + 4 + 6 = 46 + 0 = 66 4 8 4 6 Solvig for,, ad : = -, = -, 8 4 4 = ½, 4 = 4 6 64 66 Thus, the specific solutio is: t = - + ½ ( )+ = - + Page 6 4 4 0
d) Let s retur to the example left ufiished o page : Solve t = t / +, where > ad is a power of ad t = 0. This is ot a liear recurrece relatio because of the t / term. However, by usig two substitutios, we ca covert this ito a liear recurrece relatio. First, sice is defied to always be a power of, we ll replace with k. Thus, the ew k k recurrece relatio is: t k t k = t k, where k > 0 ad t 0 = 0. Sice terms such as Let S k = t k. k t are tedious to work with, we ll make aother substitutio. We ow have the recurrece relatio: S k = S k- + k or S k S k- = k -, where k > 0 ad S 0 = 0, which is a liear ohomogeeous recurrece relatio. The right-had side of the equatio may be rewritte as k () + k (-), where () ad (-) are trivial fuctios i k. Thus usig the thereom for ohomogeeous recurrece relatios, the characteristic equatio is: (r ) (r ) (r ) = 0 The first factor is derived from the homogeeous cotributio while the other two factors follow directly from the thereom. Thus, the roots are ad (with multiplicity ). Geeral Solutio: t k = S k = k + k + k k Sice = k, k = log. Thus, the geeral solutio is: t = + + log, where ad is a power of. Solvig for,, ad : 4 8 0 =, = -, = Thus, the specific solutio is: t = + log or t = log ( ), where ad is a power of. You should verify that this solutio geerates the same sequece of values that is geerated by the origial recurrece relatio. Page 7
Master Theorem A very commo type of recurrece relatio occurs with Divide-ad-oquer type of algorithms. I fact, they are so commo that a master theorem has bee prove that addresses these problems ad provides a order of complexity formula. That is, if the recurrece relatio matches a commo patter, we ca fid the complexity of the solutio. Note, the theorem does ot provide the exact solutio; but, after all, most of the time all we really eed is the complexity. Master Theorem: Give a recurrece relatio of the form: t = at /b + f(), where f() is i ( d ) with d 0, the For example, let s oce agai cosider the recurrece relatio o the previous page: t = t / +. For this problem, a =, b =, ad sice - is i ( ), d =. Thus, a = b d ; therefore the solutio for the recurrece relatio belogs to ( log ) = ( log ). DONE! What are the order of growth for solutios of the followig recurrece relatios? a. t = 4t / +, t = b. t = 4t / +, t = c. t = 4t / +, t = Page 8