Applications. More Counting Problems. Complexity of Algorithms

Transcription

1 Recurrences

2 Applications More Counting Problems Complexity of Algorithms

3 Part I Recurrences and Binomial Coefficients

4 Paths in a Triangle P(0, 0) P(1, 0) P(1,1) P(2, 0) P(2,1) P(2, 2) P(3, 0) P(3,1) P(3, 2) P(3, 3) P(n 1,0) P(n 1,k 1) P(n 1,k) P(n 1,n 1) P(n, k) P(n, 0) P(n, n) P(n, k) = P(n 1, k 1) + P(n 1, k)

5 Taxi Cab Paths in a Rectangle n n 1 T (m 1, n) T (m, n) T (m, n 1) T (0, j) 2 1 start T (i, 0) m 1 m T (m, n) = T (m, n 1) +T (m 1, n)

6 Paths in a Rectangle a 1 a 3 a 5 a 2n 3 a 2n 1 a 2 a 4 a 6 a 2n 2 a 2n a 2n = a 2n 1 + a 2n 2

7 Path Problems Triangle P(n, k) = P(n 1, k 1) + P(n 1, k) C(n, k) = C(n 1, k 1) + C(n 1, k) P(n, k) = C(n, k) -- P(n, 0) = 1 = C(n, 0) Rectangle (Taxicab) T (m, n) = T (m, n 1) +T (m 1, n) C(m + n, n) = C(m + n 1, n 1) + C(m + n 1, n) T (m, n) = C(m + n, n) -- T (m, 0) = 1 = C(m, m) -- T (0, n) = 1 = C(n, n)

8 Taxicab Problem Solution T (m, n) = C(m + n, n) Examples T (8, 8) = C(16, 8) = 12, T (16,16) = C(32,16) = 601, 080, T (32, 32) = C(64, 32) = 1,832,624,140,942,590, Conclusion Searching even moderate size grids for optimal path is not feasible.

9 Pascal s Triangle and Fibonacci Numbers Definitions a n = C(n, 0) + C(n 1,1) + C(n 2, 2) + + C(n / 2, n / 2) a n 1 = C(n 1, 0) + C(n 2,1) + + C(n / 2, n / 2 1) + C ((n 1)/ 2, (n 1) / 2) Sum = C(n +1, 0) + C(n,1) + C(n 1, 2) + + C(n / 2 +1, n / 2) + C ((n +1) / 2,(n +1) / 2) Conclusion a n 1 + a n = a n+1

10 Indistinguishable Objects in Distinguishable Boxes Problem How many ways W (n, m) can you put n similar objects into m different boxes placing at least r j 0 objects into box j? Solution W (n, m) = W(n 1, m) + W (n, m 1) -- Either you put an object in the first box or you don t. C(m + n 1, m) = C(m + n 1, m 1) + C(m + n 1, m) W (n, m) = C(m + n 1, m 1) = C(m + n 1, n) -- W (n, 1) = 1 = C(n, 0)

11 Part II Linear Recurrences with Constant Coefficients

12 Recurrences with Constant Coefficients Linear Homogeneous Recurrence a n = c 1 a n c k a n k Linear Inhomogeneous Recurrence a n = c 1 a n c k a n k + f (n) Nonlinear Recurrence Some a j not to first power Will not study See homework for simple examples Problems Derive the recurrence Solve the recurrence

13 Fibonacci Sequences (Homogeneous) Permutations where each element either stays in place or changes places with an adjacent neighbor Examples: 1234, , 2143, 2134, 1324 a n = number of such permutations of n objects a 1 = 1, a 2 = 2, a 3 = 3, a 4 = 5, a n = a n 1 + a n 2 (either n changes places or it doesn t) 1, 2, 3, 5, 8, 13, Rabbits breed one pair/month after the second month f n = number of rabbit pairs after n months (no deaths) f 0 = f 1 = 1 f n = f n 1 + f n 2 (number last month + newborn) 1, 1, 2, 3, 5, 8, 13, 21,

14 More Fibonacci Sequences (Homogeneous) Bitstrings of length n that do NOT have two consecutive zeros Examples: 01 0, 011, 101, 110, 111 a 1 = 2 a 2 = 3 a n = a n 1 + a n 2 (sequence either ends in 1 or ends in 0) 2, 3, 5, 8, 13, 21, Pascal s Triangle Arrange binomial coefficients in a right triangle Sum along diagonal slant f n +1 = C(n, 0) + C(n 1,1) + + C(n / 2, n / 2) 1, 1, 2, 3, 5, 8, 13, 21,

15 Exponential Sequences (Homogeneous) Compound Interest r = interest rate A n = amount of money after n years -- A 0 = Initial deposit -- A n = (1 + r)a n 1 A n A n 1 = r A n 1 - A n = (1 + r) n A 0 Biological Growth r = yearly (hourly) rate of growth or decay A n = population after n years (hours) -- A 0 = Initial population -- A n = (1 ± r)a n 1 A n A n 1 = ±ra n 1 -- A n = (1 ± r) n A 0

16 Analogy to Differential Equations Differential Equations Difference Equations da dt = ra A n = A n A n 1 = ra n 1 A(t) = A 0 e rt A n = (1 ± r) n A 0

17 Inhomogeneous Equations Tower of Hanoi h n = number of moves needed to solve Tower of Hanoi -- h 1 = 1 -- h n = 2h n h n = 2 n 1 (by induction on n) Regions of Space r n = number of planar regions generated by n nonconcurrent lines -- r 1 = 2 -- r n = r n 1 + n (every crossing creates a new region) -- r n = n(n +1) 2 +1 (by induction on n or by inspection)

18 Backwards Difference Differences a n = a n a n 1 k +1 a n = k a n k a n 1 Examples a n = a n a n 1 2 a n = a n a n 1 = (a n a n 1 ) (a n 1 a n 2 ) = a n 2a n 1 + a n 2 3 a n = 2 a n 2 a n 1 = (a n 2a n 1 + a n 2 ) (a n 1 2a n 2 + a n 3 ) = a n 3a n 1 + 3a n 2 a n 3 Lemma: k a n = k j=0 ( 1) j ( k j )a n j Proof: By induction on k.

19 Lemma: k a n = k j=0 ( 1) j ( k j )a n j Proof: By induction on k. Base cases already verified for k = 1,2, 3 Inductive Step: Assume: k a n = k j=0 ( 1) j ( k j )a n j Must Show: k +1 a n = k +1 ( 1) j ( k +1 j )a n j j =0 Use the Inductive Definition: k +1 a n = k a n k a n 1

20 Must Show: k +1 a n = k +1 ( 1) j ( k +1 j )a n j j =0 But by definition: k +1 a n = k a n k a n 1, so k +1 a n = k +1 a n = k +1 a n = k +1 a n = k +1 a n = k j =0 k j =0 k j =0 k +1 j =0 k +1 k ( 1) j ( k j )a n j ( 1) j ( k j )a n 1 j j =0 k +1 ( 1) j ( k j )a n j ( 1) j 1 ( k j 1 )a n j {reindex j j 1} j=1 k +1 ( 1) j ( k j )a n j + ( 1) j ( k j 1)a n j {adjust signs} ( 1) j j=1 {( k ) ( j + k )} j 1 a n j ( 1) j ( k +1 j )a n j j =0

21 Backwards Differences and Recurrences Recurrence a n = c 1 a n c k a n k Differences a n = a n a n 1 k +1 a n = k a n k a n 1 Lemma: k a n = k j=0 ( 1) j ( k j )a n j Corollary 1: a n k can be written in terms of a n, n a,, k a n. Corollary 2: a n = b 1 a n + + b k k a n.

22 Characteristic Equation Explicit Solution for Linear Homogeneous Recurrence a n = c 1 a n c k a n k -- a n = r n -- rn = c 1 r n c k r n k Characteristic Equation r k c 1 r k 1 c k = 0

23 Analogy to Linear Homogeneous Differential Equations Difference Equation a n = b 1 a n + + b k k a n -- a n = r n (Exponential in n) -- r = solution of characteristic equation Differential Equation A(t) = b 1 da dt + + b k d k A dt k -- A(t) = e rt (Exponential in t) -- r = solution of characteristic equation

24 Solving Linear Homogeneous Recurrence Relations Hypotheses Recurrence Relation: a n = c 1 a n c k a n k Characteristic Poly: r k c 1 r k 1 c k = 0 Theorem If r 1,, r k are k distinct roots of the characteristic polynomial, then the only solutions of the recurrence are of the form: n a n = b 1 r n bk r k

25 Proof n a n = r j is a solution since n rj satisfies the characteristic polynomial. if a n = r(n) and a n = s(n) are two solutions of the recurrence, then by linearity so are -- a n = r(n) + s(n) -- a n = br(n) Given k initial conditions, we can solve for the constants b 1,, b k, since the coefficient matrix is a Vandermonde in r 1,, r k. n Therefore a n = b 1 r n bk r k is a solution that satisfies the initial conditions. Since the initial conditions specify a unique solution, all solutions must have this form.

26 Linearity Lemma: If a n = r(n) and a n = s(n) are two solutions of the recurrence, a n = c 1 a n c k a n k then so are i. a n = r(n) + s(n) ii. a n = br(n) Proof: If a n = r(n) and a n = s(n) are two solutions of the recurrence, then r(n) = c 1r(n 1) + + c k r(n k) s(n) = c 1s(n 1) + + c k s(n k) Therefore i. r(n) + s(n) = c 1 ( r(n 1) + s(n 1) ) + + c k ( r(n k)+ s(n k) ) ii. br(n) = c 1 ( b r(n 1) ) + + c k ( b r(n k) )

27 Initial Conditions and Vandermonde Matrices Initial Conditions 0 a 0 = b 1 r bk r 0 1 a 1 = b 1 r bk r k a k 1 = b 1 r 1 k bk r k k r 1 r 2 r k k 1 r 1 k 1 r 2 k 1 rk b 1 b 2 b k = a 0 a 1 a k 1 Vandermonde Determinant det r 1 r 2 r k r 1 k 1 r 2 k 1 r k k 1 0 provided r 1 r 1 r k

28 Vandermonde Determinants 2 2Vandermonde Determinant det 1 1 r 1 r 2 = r 2 r 1 0 if r 2 r 1 k k Vandermonde Determinant r 1 r 2 r k det k 1 k 1 k 1 r 1 r 2 r k 0 provided r 1 r 1 r k since a polynomial of degree k 1 has at most k-1 roots.

29 What if Question What happens if the roots r 1,, r k are not unique? Answer Given: a n = c 1 a n 1 + c 2 a n 2 and r 1 = r 2. Solutions: a n = b 1 r 1 n + b2 n r k n (Homework).

30 Solving Recurrences Recurrence Relation: a n = c 1 a n c k a n k n k Initial Conditions: a 0,, a k 1 Characteristic Poly: r k c 1 r k 1 c k = 0 Distinct Roots: r 1,, r k General Solution: a n = b 1 r 1 n + + bk r k n Constants from a 0 = b b k Initial Conditions: a 1 = b 1 r b k r k a k 1 = b 1 r 1 k bk r k k 1

31 Fibonacci -- Revisited Fibonacci Sequence f 0 = 0 f 1 = 1 f n = f n 1 + f n 2 Characteristic Polynomial r 2 r 1 = 0 r = (1 ± 5 )/ 2 (Golden Ratio)

32 Fibonacci -- Revisited (continued) Solution f n = b 1 2 n b 2 2 n Initial Conditions b 1 + b 2 = 0 (n = 0) b b = 1 (n = 1) -- b 1 = 1 5 b 2 = 1 5 Solution f n = n n

33 Polynomial Recurrences

34 Motivation Question What happens when r 1 = r 2 = = r k = 1 are the roots of the characteristic polynomial? Answer Recurrence is no longer an exponential. Recurrence is a polynomial. Characteristic Polynomial k (1 r) k = ( 1) k j ( k j )r k j = ( 1) k ( 1) j ( k j )r k j Recurrence j =0 k j =0 a n ( k 1 )a n 1 + ( k 2 )a n 2 + ( 1) k +1 a n k = 0

35 Backwards Difference Differences a n = a n a n 1 k +1 a n = k a n k a n 1 Examples a n = a n a n 1 2 a n = a n a n 1 = (a n a n 1 ) (a n 1 a n 2 ) = a n 2a n 1 + a n 2 3 a n = 2 a n 2 a n 1 = (a n 2a n 1 + a n 2 ) (a n 1 2a n 2 + a n 3 ) = a n 3a n 1 + 3a n 2 a n 3 Linearity k (a n + b n ) = k a n + k b n k (ca n ) = c k a n

36 Backwards Differences and Polynomial Recurrences Lemma: k a n = k j=0 ( 1) j ( k j )a n j Proof: By induction on k. Polynomial Recurrences a n ( k 1 )a n 1 + ( k 2 )a n 2 + ( 1) k +1 a n k = 0 k a n = 0 Examples See IQ-tests

37 Theorem: k +1 n k = 0 {k constant, n varies} Proof: By induction on k. Base Case: k = 0, n 0 = 1 n 0 = 1 1= 0. Inductive Step. Assume k n j = 0, j < k. Apply the binomial theorem. Must Show: k +1 n k = 0. k +1 n k = k n k k (n 1) k = k ( n k (n 1) k ) = k ( kn k 1 + lower order terms ) = 0 {by the inductive hypothesis}

38 Consequences Theorem: k +1 n k = 0 {k constant, n varies} Corollary 1: k +1 p = 0 if p is any polynomial of degree k. Corollary 2: k p = constant if p is any polynomial of degree k. Analogy: d k p dt = constant if p is any polynomial of degree k.

39 Theorem: The only solutions of the recurrence k +1 a n = 0 are of the form: a n = b k n k + b k 1 n k b 0 Proof: By the Corollary 1, a n = b k n k + b k 1 n k b 0 is a solution. Given k +1 initial conditions, we can solve for the constants b 0,, b k since the coefficient matrix is Vandermonde in 1, 2,, k. Therefore a n = b k n k + b k 1 n k b 0 is a solution that satisfies the initial conditions. Since the initial conditions specify a unique solution, all solutions must have this form.

40 Forward Differences Differences a n = a n +1 a n k +1 a n = k a n k a n 1 Examples a n = a n +1 a n 2 a n = a n+1 a n = (a n+2 a n +1 ) (a n +1 a n ) = a n +2 2a n+1 + a n 3 a n = 2 a n 2 a n 1 = (a n+3 2a n +2 + a n+1 ) (a n+2 2a n +1 + a n ) = a n +3 3a n+2 + 3a n+1 a n Lemma: k a n = k j=0 ( 1) k j ( k j )a n+ j Proof: By induction on k.

41 Forward Differences Differences a n = a n +1 a n k +1 a n = k a n k a n 1 Example ( ) = t(t 1) (t k 1) (t 1) (t k) = (t 1) (t k 1) ( t (t k) ) (t 1) (t k) = k(t 1) (t k 1) Similar to Differentiation Linearity k (a n + b n ) = k a n + k b n k (ca n ) = c k a n

42 Forward Differences and Polynomial Recurrences Lemma: k a n = k j=0 ( 1) k j ( k j )a n+ j Proof: By induction on k. Polynomial Recurrences a n +k ( k 1 )a n +k 1 + ( k 2 )a n+k 2 + ( 1) k +1 a n = 0 k a n = 0 Examples See IQ-Tests

43 IQ Tests Quadratic 2 a n : 6 6 6? a n : ? a n : ? p(t) =? Cubic 3 a n : 6 6? 2 a n : ? a n : ? a n : ? p(t) =?

44 IQ Tests -- continued Quadratic 2 a n : a n : a n : p(t) = 6(t 1)(t 2) 2 + 9(t 1) + 4 Cubic 3 a n : a n : a n : a n : p(t) = 6(t 1)(t 2)(t 3) 3! + 12(t 1)(t 2) 2 +11(t 1) + 5

45 IQ Tests -- Revisited Exponential 2 a n : a n : a n : de t dt = e t analogous to 2 n = 2 n Fibonacci 2 a n : a n : a n : a n = a n is the signature of an exponential

46 Polynomial Evaluation Evenly Spaced Points a n +1 = a n + h Differencing Calculate n evenly spaced values by Horner s method Build difference triangle Propagate the difference triangle Evaluate new points using only addition, no multiplication

47 Generating Functions

48 Applications Counting Solutions to Diophantine Equations Placing Indistinguishable objects into Indistinguishable boxes Making Change Combinatorial Identities Expectation of Binomial Distribution Vandermonde Identity Solving (In)Homogeneous Linear Equations Analogous to series solutions to differential equations

49 Methods Coding Information in Exponents Counting Solutions to Diophantine Equations Coding Information in Coefficients Combinatorial Identities Solving Linear Recurrences

50 Linear Equations with Unit Coefficients Unconstrained Solve x x k = n -- order matters # Solutions in non-negative integers = C(n + k 1, n) Constrained Solve x x k = n -- order matters Constraints: a 1 x 1 b 1 a k x k b k # Solutions -- Apply Inclusion/Exclusion (Old Method) -- Take Coeff of x n in (x a x b 1 ) (x a k + + x b k ) Equations n Indistinguishable objects into k Distinguishable boxes

51 Linear Equations with Unit Coefficients -- Constrained Example Solve x 1 + x 2 + x 3 = order matters Constraints: 1 x 1 2, 2 x 2 5, 4 x Indistinguishable Objects into 3 Distinguishable Boxes # Solution -- Coeff of x 10 in (x + x 2 ) (x 2 + x 3 + x 4 + x 5 ) (x 4 + x 5 + x 6 + x 7 ) = 7

52 Partitioning the Integers Partitions Find all partitions of n into non-negative integers -- ignore order Solve x x k = n -- ignore order # Solutions in non-negative integers is coeff of x n in a polynomial product Partitions with distinct summands -- (1+ x)(1 + x2 ) (1 + x n ) Partitions with repeated summands -- (1+ x + x2 + x n )(1 + x 2 + x 4 + ) (1 + x n ) Partitions n Indistinguishable objects into k Indistinguishable boxes

53 Examples Partitions of 6 with Distinct Summands Coeff of x 6 in -- (1+ x) (1 + x 2 ) (1 + x 3 ) (1 + x 4 ) (1 + x 5 ) (1 + x 6 ) 4 Solutions: (6, 0), (5,1), (4, 2), (3, 2,1) Partitions of 6 with Repeated Summands Coeff of x 6 in -- (1+ x + + x6 )(1 + x 2 + x 4 + x 6 ) (1+ x 3 + x 6 ) (1 + x 4 ) (1 + x 5 ) (1 + x 6 ) 11 Solutions: (6, 0), (5,1), (4, 2), (4,1,1), (3, 3), (3, 2,1), (3,1,1,1), (2, 2, 2), (2, 2,1,1), (2,1,1,1,1), (1,1,1,1,1,1)

54 # Solutions to Diophantine Equations Diophantine Equation -- Solutions in Non-Negative Integers a 1x a k x k = n a 1,, a k non-negative integers Coeff x n in (1+ x a 1 + x 2a 1 )(1 + x a 2 + x 2a 2 + ) (1 + x a k + x 2a k + ) Example: Change for a Dollar x 1 + 5x 2 +10x x x 5 = 100 Coeff x 100 in (1+ x + x x 100 )(1+ x 5 + x x 100 ) (1 + x 50 + x 100 )

55 Generating Functions Definition A = a 0, a 1, a 2, = sequence finite or infinite G(x) = a k x k = Generating Function of A k Examples a k = 1 0 k n G(x) = n x k k =0 = 1 xn+1 1 x a k = 1 0 k < G(x) = x k k =0 = 1 1 x

56 Generating Functions More Examples a k = r k 0 k < G(x) = r k x k = k =0 1 1 rx n a k = B k (t) = ( n k )t k (1 t) n k 0 k n G(x) = n n k B k (t)x = ( k n )t k x k (1 t) n k = (1 t) + t x k =0 n k =0 ( ) n

57 Binomial Distribution Generating Function n B k (t) = ( n k )t k (1 t) n k 0 k n n n k G(x) = B k (t)x = ( k n )t k x k (1 t) n k = (1 t) + t x k =0 n k =0 ( ) n G (x) = n k =0 kb k n (t)x k 1 = nt ((1 t) + t x) n 1 Expectation G (1) = n k =0 kb k n (t) = nt G ( j) (1) j! n n = C(k, j)b k (t) = C(n, j)t j k = j

58 Vandermonde s Identity Generating Function (1+ t) m +n = (1 + t) m (1+ t) n Binomial Theorem m +n C(n + m, k)t k m = C(m, i)t i k =0 i=0 n j =0 C(n, j)t j Identity C(m + n, k) = C(m, i)c(n, j) k min(m, n) i+ j=k

59 Algebra of Generating Functions Setup F(x) = a k x k k G(x) = b k x k k Addition F(x) + G(x) = (a k + b k )x k k Multiplication {Discrete Convolution} ( ) k b k j x k F(x) G(x) = a j j

60 Geometric Series Sums S = c + cr + cr2 + rs = cr + cr2 + cr 3 + S rs = c S = c 1 r Examples x + x 2 + x 3 = x 1 x 1+ ax + a 2 x 2 + = 1 1 ax

61 Solving Recurrence Relations -- Tower of Hanoi (Inhomogeneous) Tower of Hanoi (Inhomogeneous) h 0 = 0 h 1 = 1 h n = 2h n 1 +1 n 1 Generating Function h n x n = 2h n 1 x n + x n H(x) = h n x n = 2x h n 1 x n 1 + x n n =1 n =1 H(x) = 2xH(x) + x /(1 x) (Functional Equation) n=1

62 Solving Recurrence Relations -- Tower of Hanoi (continued) Generating Function (continued) H(x) = h n x n n =1 H(x) = 2xH(x) + x /(1 x) -- (1 2x)H (x) = x / (1 x) -- H(x) = x /(1 x)(1 2x) =1 /(1 2x) 1/ (1 x) H(x) = 2 n x n x n = (2 n n =0 n=0 n =0 1)x n Conclusion h n = 2 n 1

63 Solving Recurrence Relations -- Fibonacci Sequence (Homogeneous) Fibonacci Sequence f 0 = 0 f 1 = 1 f n = f n 1 + f n 2 n 2 Generating Function F(x) = f n x n n =1 f n x n = f n 1 x n + f n 2 x n f n x n = x f n 1 x n 1 + x 2 f n 2 n =2 n=2 n =2 x n 2 F(x) x = xf(x) + x 2 F(x) (Functional Equation)

64 Solving Recurrence Relations -- Fibonacci Sequence (continued) Generating Function (continued) F(x) x = xf(x) + x 2 F(x) -- (1 x x 2 )F(x) = x -- F(x) = x / (1 x x 2 ) = x / (1 ax)(1 bx) F(x) = ax 1 1 bx -- a = b = F(x) = 1 5 (a n b n )x n f n = an b n 5 n =0 Analogous to series solutions for ordinary differential equations.

65 Algorithm for Solving Linear Recurrences Algorithm 1. Multiply both sides of the recurrence by x n. 2. Sum over n. 3. Replace the power series by generating functions a. Find the functional equation 4. Solve the functional equation for the generating function -- explicit formula. 5. Convert the explicit formula back to a power series. 6. Read off the coefficients of the power series.

66 Fun Exercises 1. Prove that the generating function of a linear homogeneous recurrence is a rational function. 2. Prove that the roots of the denominator of the generating function are the reciprocals of the roots of the characteristic polynomial of the recurrence. 3. Prove that the numerator of the generating function for the recurrence a n = c 1 a n c k a n k is given by taking the product (a 0 + a 1 x + a k 1 x k 1 ) (c 1 x + + c k 1 x k 1 ) and truncating to the first k 1 powers of x.

67 Placing Objects into Boxes Objects Boxes Constraints Solution n type I k type D x j 1 C(n 1, k 1) n type I k type D x j 0 C(n + k 1, n) n type D k type D x j 1 ( 1) k C(n, k)(n k) m (Onto Functions) m type D n type I x j 1 n k =0 n k =0 ( 1) k C(n, k)(n k) m n! n type I k type I x j 0 Coeff x n (1+ x + + x n )(1+ x 2 + ) (1 + x n ) I = indistinguishable D = distinguishable

68 The Master Theorem and Algorithmic Complexity

69 The Master Theorem -- Divide and Conquer Binary Search f (n) = f (n / 2) + 2 Maxima and Minima f (n) = 2 f (n / 2) + 2 Divide and Conquer f (n) = a f (n /b) + c -- Divide problem into a subproblems, each of size n / b, using c additional operations. -- Conquer f (n) by solving the smaller problems

70 Master Theorem Theorem: Let f (n) = a f (n /b) + c Then i. f (n) = O log b (a) ii. ( ) a > 1 f (n) = O ( log(n) ) a = 1 Proof: f (n) = a f (n /b) + c { f (n / b) = a f (n / b 2 ) + c } = a 2 f (n / b 2 ) + ac + c { f (n / b 2 ) = a f (n / b 3 ) + c} = a 3 f (n / b 3 ) + a 2 c + ac + c = a k f (n / b k ) + a k 1 c + + ac + c (*)

71 Case 1. a > 1: a. Suppose n = b k. k = log b (n) a k = a log b (n) = n log b (a) (take log b of both sides) Then by (*) f (n) = a k f (1) + c ak 1 a 1 = f (1) + c a a 1 k b. Suppose b k < n < b k +1. Then since f is increasing c a 1 = O nlog b (a) ( ) f (n) < f (b k +1 ) = C 1 a k +1 + C 2 = (C 1 a)a k + C 2 < (C 1 a)a log b (n) + C 2 = (C 1 a)n log b (a) + C 2 = O( n log b (a) )

72 Case 2. a = 1: a. Suppose n = b k. Then by (*) f (n) = f (1) + kc = f (1) + c log b (n) = O ( log b (n)) b. Suppose b k < n < b k +1. Then since f is increasing f (n) < f (b k +1 ) = C 1 (k +1) + C 2 = (C 1 + C 2 ) + C 1 k = (C 1 + C 2 ) + C 1 log b (n) = O( log b (n)) QED

73 Examples Binary Search f (n) = f (n / 2) + 2 f (n) = O ( log(n) ) since a = 1 Maxima and Minima f (n) = 2 f (n / 2) + 2 f (n) = O(n) since a = b = 2

74 Theorem: Let f (n) = a f (n /b) + cn d, where a 1 Then i. f (n) = O(n d ) a < b d ii. iii. f (n) = O ( n d log(n) ) a = b d f (n) = O ( n log b (a) ) a > b d Proof: Homework