Recurrence Relations Reading: Gossett Sections 7.1 and 7.. Definition: Aone-wayinfinite sequence is a function from the natural numbers to some other set. E.g. fib(0) = 1, fib(1) = 1, fib() =,fib(3) = 3, fib(4) =, fib() = 8,. q(0) = 1,q(1) =,q() = 4,q(3) = 8,... pr(0) =,pr(1) = 3,pr() =,pr(3) = 7,... Definition: A recurrence relation is an equation that defines all members of a sequence past a certain point in terms of earlier members. That is an equation a(n) =F, for all n {n 1,n 1 +1, } where F is an expression combining only a(n 1), a(n ),..., a(0). Examples fib(n) =fib(n 1) + fib(n ), foralln q(n) = q(n 1), foralln 1 Discrete Math. for Engineering, 00. Notes 7. Recurrence Relations If we conjoin enough (n 1 ) base cases with the recurrence relation, then together they define a sequence. Examples: fib(0) = 1 fib(1) = 1 q(0) = 1 Typeset November 3, 00 1 Typeset November 3, 00
Substitute and simplify method Consider the sequence defined by a(0) = a(n) =a(n 1) 3, forn 1 We can reason (rather informally) that. a(n) =a(n 1) 3 =(a(n ) 3) 3 substitution =4a(n ) 3 3 simplify =4(a(n 3) 3) 3 3 substitution =8a(n 3) 4 3 3 3 simplify. = k a(n k) 3( k 1 + k + +1). = n a(0) 3 n 1 + n + +1 = n 3 ( n 1) since = n +3 = n+1 +3 Xn 1 i = n 1 Typeset November 3, 00 3 i=0 Discrete Math. for Engineering, 00. Notes 7. Recurrence Relations If there is any doubt, we can prove the result by induction. Unfortunately the substitute and simplify method does not always give an easy to simplify result. Typeset November 3, 00 4
Linear Homogeneous Recurrence Relations with Constant Coefficients of Degree k Definition: Alinear homogeneous recurrence relation with constant coefficients (LHRRCC) is a recurrence relation whose RHS is a sum of terms each of the form c b a(n j) where c C and j N are constants. Definition: The degree of a LHRRCC is the maximum b value for which the c valueisnot0.i.e.ifwehave a(n) =c 1 a(n 1) + c a(n ) + + c k a(n k) with c k 6=0, then the degree is k. Whenthedegreeis Considerthedegreecase.TheRRis a(n) =c 1 a(n 1) + c a(n ) (*) Typeset November 3, 00 Discrete Math. for Engineering, 00. Notes 7. Recurrence Relations Suppose there is a solution for the recurrence relation of the form a(n) =θr n,foralln N for some θ 6= 0and some r 6= 0. Then substituting into (*) we get (for any n N) θr n = c 1 θr n 1 + c θr n Thus r n c 1 r n 1 c r n =0 Thus r n (r c 1 r c )=0 So r is a root of the polynomial x c 1 x c (**) Conversely, if r is root of the polynomial x c 1 x c (**) then for any θ and any n N θr n = c 1 θr n 1 + c θr n so a(n) =θr n isasolutionto(*). The characteristic polynomial of (*) is (**). Typeset November 3, 00 6
We have proved the following theorem: Theorem: If r is a root of the characteristic polynomial x c 1 x c then, for any θ C, the sequence a(n), θr n, for all n N. is a solution to the equation a(n) =c 1 a(n 1) + c a(n ), for all n. Example. a(n) =a(n 1) + 6a(n ) The characteristic polynomial is x x 6 With roots 3 and. One root is r =3;pickingθ =1, we get a sequence 3 0, 3 1, 3, 3 4,... = h1, 3, 9, 7,...i So if the base cases are a(0) = 1 and a(1) = 3, we should pick r =3and θ =1. Discrete Math. for Engineering, 00. Notes 7. Recurrence Relations Trying the other root r = and picking θ =3we get 3 ( ) 0, 3 ( ) 1, 3 ( ), 3 ( ) 3,... h3, 6, 1, 4,...i So if the base cases are a(0) = 3 and a(1) = 6, we should pick r = and θ =3. Unfortunately the base cases may not allow such a simple solution. Consider a(0) = a(1) = 3 a(n) =a(n 1) + 6a(n ) The roots are r 1 =3and r =. Tryingr 1 we must find θ such that θ3 0 = and θ3 1 =3 Trying r we must find a θ such that θ ( ) 0 = and θ ( ) 1 =3 Neither root gives a solution that agrees with both base cases. Typeset November 3, 00 7 Typeset November 3, 00 8
Linear combinations of solutions Suppose that we have a recurrence relation: a(n) =c 1 a(n 1) + c a(n ), for all n Now suppose we know a particular sequence w solves the recurrence. I.e. w(n) =c 1 w(n 1) + c w(n ), for all n Then, for any constant θ, wecandefine a new sequence y defined by y(n), θ w(n), foralln N Thistoowillbeasolutionas,foralln y(n) =θ w(n) = θ(c 1 w(n 1) + c w(n )) = c 1 θw(n 1) + c θw(n ) = c 1 y(n 1) + c y(n ) So multiplying a solution by any constant gives a solution. Suppose that w and x are two solutions. I.e. w(n) =c 1 w(n 1) + c w(n ), for all n Typeset November 3, 00 9 Discrete Math. for Engineering, 00. Notes 7. Recurrence Relations and x(n) =c 1 x(n 1) + c x(n ), for all n Then we can build a sequence z defined by z(n), w(n)+x(n), for all n N Now z toowillbeasolutionas,foralln z(n) =w(n)+x(n) = c 1 w(n 1) + c w(n ) + c 1 x(n 1) + c x(n ) = c 1 (w(n 1) + x(n 1)) + c (w(n ) + x(n )) = c 1 z(n 1) + c (z(n ) So given any two solutions, x and w, any linear combination of them z(n), θ 1 w(n)+θ x(n), foralln N will also be a solution. A set that is closed under linear combinations is called a vector space. If the degree is, all solutions can be formed as linear combination of just (appropriately chosen) solutions. If there are two different roots, we have two solutions r1 n and r n, which will suffice. Typeset November 3, 00 10
Using both roots at once Consider, again, the LHRRCC of degree. a(n) =c 1 a(n 1) + c a(n ), foralln (*) with characteristic polynomial x c 1 x c =0 Let r 1 and r be the roots of this polynomial. Consider any two constants θ 1 and θ. θ 1 r1 n + θ r n isasolutionto(*) since it is a linear combination of the solutions r1 n and rn. Here is a direct proof: θ 1 r1 n + θ r n isasolutionto(*) iff θ 1 r1 n + θ r n = c 1 θ1 r1 n 1 + θ r n 1 + c1 θ1 r1 n + θ r n iff θ 1 r n 1 c 1 r1 n 1 c r1 n + θ r n c 1 r n 1 c r n =0 iff iff θ 1 r n 1 1 r 1 c 1 r 1 c + θ r n 1 r c 1 r c =0 T Typeset November 3, 00 11 Discrete Math. for Engineering, 00. Notes 7. Recurrence Relations This proves the following theorem Theorem: Given the LHRRCC of degree a(n) =c 1 a(n 1) + c a(n )., for all n> (*) If r 1 and r are the two roots of the characteristic polynomial x c 1 x c then for any θ 1 and θ isasolutionto(*). θ 1 r n 1 + θ r n Furthermore, if we know a(0) and a(1).then θ 1 + θ = a(0) θ 1 r 1 + θ r = a(1) With linear equations in unknowns we can solve for θ 1 and θ. This will succeed provided r 1 6= r. So, when the roots are distinct, we can find the unique values for θ 1 and θ that satisfy the two base cases. Typeset November 3, 00 1
Example: a(0) = a(1) = 3 a(n) =a(n 1) + 6a(n ) The characteristic polynomial x x 6=0 has roots r 1 =3and r =. Sowearelookingfora solution of the form θ 1 3 n + θ ( ) n We have θ 1 + θ = a(0) = 3θ 1 θ = a(1) = 3 So 3θ 1 ( θ 1 )=3 θ 1 +4 = 3 θ 1 = 1 Discrete Math. for Engineering, 00. Notes 7. Recurrence Relations and θ = θ 1 θ = 9 Procedure From the RR derive the characteristic polynomial Find roots r 1 and r of the characteristic polynomial. If r 1 6= r then look for a solution of the solution of the form θ 1 r n 1 + θ r n use the base cases to solve for θ 1 and θ Example: fib(0) = 1 fib(1) = 1 fib(n) =fib(n 1) + fib(n ) Form the characteristic polynomial x x 1=0 Typeset November 3, 00 13 Typeset November 3, 00 14
Find roots using the quadratic equation. The roots are 1+ =1.61803 = φ and 1 = 0.618 03 =1 φ = 1 φ We are looking for a solution of the form θ 1 φ n + θ (1 φ) n Using fib(0) = fib(1) = 1 we get θ 1 + θ =1 θ 1 φ + θ (1 φ) =1 Substitute θ =1 θ 1 into θ 1 φ + θ (1 φ) =1to get θ 1 φ +(1 θ 1 )(1 φ) =1 Now solve for θ 1. θ 1 φ +(1 θ 1 )(1 φ) =1 θ 1 φ +(1 φ) θ 1 (1 φ) =1 θ 1 (φ (1 φ)) = φ θ 1 (φ 1) = φ θ 1 =φ θ 1 = φ = 1+ = + 10 Discrete Math. for Engineering, 00. Notes 7. Recurrence Relations And then solve for θ θ =1 θ 1 = φ φ = = 1+ 1 = = 1 φ Sothesolutionis fib(n) = φ φ n (1 φ) (1 φ) n = 1 φ n+1 1 (1 φ) n+1 These ideas generalize to degrees larger than. Typeset November 3, 00 1 Typeset November 3, 00 16
Repeated roots for LHRRCCs with degree When r 1 = r then θ 1 r1 n + θ r n canbewrittenasθrn where θ = θ 1 + θ and r = r 1 = r. So the base cases may form an overdetermined system: equations and one unknown. Example a(0) = 1 a(1) = a(n) =6a(n 1) 9a(n ), forn The characteristic polynomial is x 6x +9 with root r =3. We look for a solution of the form θr n But θ =1 θr = is not solvable! Discrete Math. for Engineering, 00. Notes 7. Recurrence Relations Consider a quadratic with a repeated root x 4x +4 so r =.ThisischaracteristicoftheLHRRCC a(n) =4a(n 1) 4a(n ) adding a couple of base cases a(0) = 0 and a(1) = we get n a(n) 0 0 1 4 =8 3 4 8 4 =4 4 4 4 4 8=64 4 64 4 4 = 160 The solution is apparently n n. This suggests nr n is a potential solution in general. Note that nr n is r nr n 1 and that nr n 1 is the derivative of our basic solution r n. So we might do well to look at derivatives. Typeset November 3, 00 17 Typeset November 3, 00 18
Discrete Math. for Engineering, 00. Notes 7. Recurrence Relations When a polynomial has a repeated root, that root will also be a root of its derivative: 8 6 4 0 - -4-6 1 3 4 6 x Consider x 6x +9 = (x 3) and its derivative (x 3), 3 is a root of both. In general (for all r R) if p(x) =x c 1 x c =(x r) then r willalsobearootofp 0 (x) =x c 1 =(x r). Define p 0 (x), x n p(x) r will also be a root of p 0 since p 0 (r) =r n p(r) =0. r will be a root of p 0 0 since p 0 0(r) =r n p 0 (r)+(n )r n 3 p(r) =0. Theorem: If a LHRRCC of the form a(n) =c 1 a(n 1) + c a(n 1), foralln (*) has a characteristic polynomial with one root r then, nr n is a solution: Proof: Let p and p 0 be polynomials defined by p(x), x c 1 x c p 0 (x), x n p(x) As we just saw, since r isthesolerootofp, itisalsoa root of p 0 0. Typeset November 3, 00 19 Typeset November 3, 00 0
What is p 0 0? p 0 (x) =x n p(x) =x n c 1 x n 1 c x n p 0 0(x) =nx n 1 c 1 (n 1)x n c (n )x n 3 Now (for all n ) nr n is a solution of the RR (*) nr n = c 1 (n 1)r n 1 + c (n )r n nr n c 1 (n 1)r n 1 c (n )r n =0 r (nr n 1 c 1 (n 1)r n c (n )r n 3 )=0 r p 0 0(r) =0 T Since r is a root of p 0 0 Theorem: If a LHRRCC of the form a(n) =c 1 a(n 1) + c a(n 1), foralln (*) has a characteristic polynomial with one root r then, for any α 0,α 1 R, (α 0 + α 1 n)r n is a solution. Proof: (α 0 + α 1 n)r n is just a linear combination of the solutions r n and nr n. Wecanusethebasecasestocomputetheα 0 and α 1. Typeset November 3, 00 1 Discrete Math. for Engineering, 00. Notes 7. Recurrence Relations Back to the earlier example a(0) = 1 a(1) = a(n) =6a(n 1) 9a(n ), forn The root of the characteristic polynomial x 6x +9is 3. From the theorem, the solution is of the form (α 0 +α 1 n) 3 n Now solve α 0 =1 (α 0 + α 1 ) 3= so α 1 = 3 1= 1 3 Check: n a(n) (1 1 3n) 3n 0 1 (1 1 3 0) 30 =1.0 1 (1 1 3 1) 31 =.0 6 9 1=3 (1 1 3 ) 3 =3.0 3 6 3 9 =0 (1 1 3 3) 33 =0 4 6 0 9 3= 7 (1 1 3 4) 34 = 7.0 6 7 9 0= 16 (1 1 3 ) 3 = 16.0 Procedure for LHRRCC of degree From the RR derive the characteristic polynomial Typeset November 3, 00
Find roots r 1 and r of the characteristic polynomial. If r 1 6= r then look for a solution of the solution of the form θ 1 r n 1 + θ r n Use the base cases to solve for θ 1 and θ. If the sole root is r then look for a solution of the form (α 0 + α 1 n)r n Use the base cases to solve for α 0 and α 1. One can generalize these theorems and the resulting procedure to LHRRCCs with any degree and any number of repeated roots. See Gossett s book. Typeset November 3, 00 3