Discrete Mathematics. Kishore Kothapalli

Discrete Mathematics Kishore Kothapalli

2

Chapter 4 Advanced Counting Techniques In the previous chapter we studied various techniques for counting and enumeration. However, there are several interesting problems that have applications to Computer Science that are not nown to have a solution using just the techniques from the previous chapter. In this chapter, we expand our repertoire of techniques to handle these more complicated problems. Some of the techniques we study in this chapter are the principle of inclusion-exclusion, using recurrence relations, and generating functions. 4.1 The Principle of Inclusion-Exclusion Recall the sum rule from the basic counting techniques. For a disjoint collection of choices, we said that the total number of ways will be a sum of the ways for each item in the disjoint collection. The sum rule was silent on what if the collection is not disjoint. To handle this situation, we need a refined sum rule, which is called as the principle of inclusion-exclusion. To start with, let us compute the size of the set A B for two sets A and B. If A B = Φ, we now that A B = A + B. However, using Venn diagrams, it can be seen that if A B Φ, then the elements of A B get counted twice in A + B. They get counted as they belong to A, and for the second time as they belong to B. So, to get A B, we have to discount A B from A + B. Thus, A B = A + B A B. The above formula can be generalized to what is called as the principle of inclusion-exclusion. For three sets A, B, and C, we have that: A B C = A + B + C A B B C C A + A B C. The intuitive explanation for this is as follows. In A + B + C, we count elements that occur in exactly one of A, B, C once, the elements that occur in exactly two of the sets are counted twice, and the elements that occur in all three sets are counted thrice. To compensate for the over count of the elements that occur in exactly two of the sets, we subtract the cardinality of the intersections taen two sets at a time. But, it has the effect that elements that occur in all the three sets are not counted at all. These were counted three times in A + B + C but were discounted three times because of ( A B + B C + A C. To account for this under-counting, we have to add A B C at the end. Thus, we get the above expression. In a similar manner, we can generalize the expression for A 1 A 2 A n as follows. Theorem 4.1.1 (The Princple of Inclusion-Exclusion Let A 1, A 2,, A n be n finite sets. Then, A 1 A 2 A n = n i=1 A i 1 i<j n A i A j + 1 i<j< n A i A j A + ( 1 n+1 A 1 A 2 A n. 3

4 CHAPTER 4. ADVANCED COUNTING TECHNIQUES Proof. While a proof can be attempted using the principle of mathematical induction, following the flavor of the topic, we present a proof using combinatorics. We show the theorem by arguing that each element of A 1 A 2 A n is counted exactly once in the above formula. Consider an element x n i=1 A i. Let x belong to sets. Then, x is counted once in each of the sets, i.e., ( 1 times. It is discounted exactly ( ( 2 times during two-way intersections. It is then counted again for 3 times. So the number of times x is counted is: ( 1 ( + 2 ( ( + ( 1 n 3 Using nown identities about binomial coefficients, ( ( ( ( ( + + + ( 1 n = 0 0 1 2 3 So the above expression equals 1. Since the result is true for any x, and for any, the theorem holds. 4.1.1 Applications Another way to state the principle of inclusion-exclusion is to as for the number of elements in a set that do not have any of the properties P 1, P 2,, P n. Let N(P i1, P i2,, P i denote the number of elements that have all the properties P i1, P i2,, P i. Let the sets be A 1, A 2,, A n and let N be the total number of elements. We thus have that, N(P i1, P i2,, P i = A i1 A i2 A i It follows that we need to compute N(P 1, P 2,, P n which can be done as: N(P 1, P 2,, P n = N A 1 A 2 A n Applying the principle of inclusion-exclusion, we have: N(P 1, P 2,, P n = N A ( 1 A 2 A n = N 1 i n N(P i 1 i<j n N(P i, P j + + ( 1 n N(P 1 P 2 P n. There are several applications of the above formulation such as to the sieve of Eratosthenes, which we study below. Sieve of Eratosthenes The sieve of Eratosthenes is a procedure given by the Gree mathematician Eratosthenes to identify all primes not exceeding a certain positive integer n. The procedure starts by listing all the numbers 1 through n. Then, all the integers that are divisible by 2, except 2, are mared. Then, consider the smallest integer that is not mared, which is 3 in this case. Mar all integers that are divisible by 3, except 3. Continue this process of maring until no more integers can be mared. The unmared integers are the primes not exceeding n. Now, let us try to compute how many integers not exceeding n are prime. Notice that to see if any integer within n has any factors, then the factor is within n. Let n = 100. So we have to count the integers that are not divisible by any of 2, 3, 5, and 7. Let P i, for i = 1, 2, 3, and 4, denote the property that an integer is divisible by 2, 3, 5, and 7 respectively. We wish to compute 4 + N(P 1 P 2 P 3 P 4. Using the principle of inclusion-exclusion, we get: N(P 1 P 2 P 3 P 4 = n N(P 1 N(P 2 N(P 3 N(P 4 + N(P 1 P 2 + N(P 1 P 3 + N(P 1 P 4 + N(P 2 P 3 + N(P 2 P 4 + N(P 3 P 4 N(P 1 P 2 P 3 N(P 1 P 2 P 4 N(P 1 P 3 P 4 N(P 2 P 3 P 4 + N(P 1 P 2 P 3 P 4.

4.2. RECURRENCE RELATIONS 5 Now, N(P 1 = 100 2 = 50, similarly, N(P 2 = 100 3 = 33. N(P 2P 3 = 100/3 5 = 6. Computing other quantities similarly, we get that N(P 1P 2P 3P 4 = 99 50 33 20 14 + 16 + 10 + 7 + 6 + 4 + 2 3 2 1 0 + 0 = 21. The above number indicates the number of integers between 1 through 100 that are not a divisor of any of 2,3,5, and 7. So, the total number of primes less than 100 is 4 + 21 = 25. The list of these primes is 2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47, 53, 59, 61, 67, 71, 73, 79, 83, 89, and 97. Derangements We now see another application of the principle of inclusion-exclusion to a classical problem in counting. Recall that a permutation of size n is a mapping of {1, 2,, n} to {π(1, π(2,, π(n}. A derangement of {1, 2,, n} is a permutation such that π(i i for all i {1, 2,, n}. Using the principle of inclusion exclusion, we can compute the number of derangements as follows. Let N(P i denote the number of permutations such that π(i = i. We wish to compute N(P 1 P 2 P n. We have, N(P 1P 2 P n = N N(P i + N(P ij + ( 1 n N(P 1 P 2 P n. 1 i n 1 i<j n Now, N(P 1 = (n 1! as fixing one place of the permutation, there is a n 1 permutation. Similarly, N(P i P j = (n 2!, and in general N(P i1 P i2 P im = (n m!. Thus, N(P 1 P 2 P n = N n(n 1! + ( ( n 2 (n 2! n 3 (n 3! + + ( 1 n ( n n (n n! = n! n(n 1! + n(n 1 2 (n 2! n(n 1(n 2 3! (n 3! + + ( 1 n = n! [ 1 1 1! + 1 2! 1 3! + + ( 1n 1 ] n! The number of derangements of n elements is represented by D n. So the above expression gives the value of D n for n 1. Noticing that e 1 = 1 1 1! + 1 2! 1 3! + + ( 1n 1 n!, we get that D n = n!/e. Other problems that can be solved using the principle of inclusion exclusion are counting the number of onto functions, the Euler totient function, etc. 4.2 Recurrence Relations Studying the spread of viruses has been an active topic for hundreds of years. It is nown that viruses can spread rapidly and cause new infections at a rapid rate. One can as the time by which the infected population reaches a million mar. To answer such a question, one needs to understand the rate of growth of the virus. Imagine an hypothetical scenario where for a certain ind of virus there are 3 infected people and the infection spreads to twice as many people every hour. To answer the original question, let us denote by a n the number of infected people by n minutes. So we are given that a 0 = 3, and we infer that a n = 2a n 1. Such a relation is called as a recurrence relation, a formal definition for which shall be given soon. Notice, however, that the above solution does not express a n in closed form, i.e., the value of a n for a given n is not nown yet. One has to solve the above recurrence relation to get the closed form value for a n. In this section, we will study how to represent counting problems as recurrence relations, and also how to solve recurrence relations. Consider a sequence a 0, a 1,, a n. A recurrence relation for the sequence is an equation that defines a n in terms of one or more of the previous element of the sequence, a 0, a 1,, a n 1, for all positive integers n n 0, where n 0 is a nonnegative integer.

6 CHAPTER 4. ADVANCED COUNTING TECHNIQUES The notion of a recurrence relation is an oft-occurring theme in Computer Science, especially in the analysis of (recursive algorithms. Specifically, one can express the complexity of a recursive algorithm in terms of the complexity of the algorithm on smaller instances. Let us loo at some examples of recurrence relations. The relations a n = 2a n 1 with a 0 = 3, that we obtained earlier, is an example. Example. Suppose you invest in a scheme where you pay Rs. 50, 000 every year over 10 years at an interest of 9% per annum, compounded annually. What is the return at the end of 10 years. Solution. To solve this problem, we first recall the basics of compound interest. If a principal of P is deposited and earns an interest of r% p.a. compounded annually, then at the end of the first year, the available amount is P + (Pr/100 = P(1 + (r/100. At the end of the second year, this increases to P(1 + r/100 + P(1 + r/100 r/100 = P(1 + r/100 2. This can be generalized as follows. Let P(n denote the return at n years. Then, P(n = P(n 1 + P(n 1 r/100 = P(n 1(1 + r/100 In our problem, we can now see that the principal of Rs. 50, 000 will now be 50, 000 (1.09 10 = 1, 18, 000 at the end of 10 years. Another example that traces a classical problem is that of the growth rate of a colony of rabbits. The problem is introduced by Fibonacci. Example. Suppose that a young pair of rats is placed in a colony 4.2.1 Solving Recurrence Relations Let us now focus on how to solve recurrence relations. Here, we develop solution techniques by identifying classes of recurrence relations. Linear Homogeneous Recurrence Relations with Constant Coefficients A class of recurrence relations that are easy to solve is the linear recurrence relations that are homogeneous in nature with constant coefficients. A recurrence relation is said to be linear if the relation has only a sum of the previous terms. It is called homogeneous if all terms in the recurrence relation are of linear multiples of previous terms. The phrase Constant Coefficients refers to the condition that the coefficients are all constants, and not functions that depend on n. A formal definition is given below. Definition 4.2.1 A recurrence relation of the form a n = c 1 a n 1 + c 2 a n 2 + + c 1 a 1 + c 0 a 0 is called a linear homogeneous recurrence relation with constant coefficients of degree, where all c i IR and c 0. An example of a linear homogeneous recurrence relation with constant coefficients is a n = 3a n 1 + 5a n 2. This has a degree of 2. Notice that according to the above definition, the equation a n = a n 1 + n is not a linear homogeneous recurrence relation with constant coefficients. (It is not homogeneous. We show the following theorem that can be used to solve recurrence relations of the above ind.

4.2. RECURRENCE RELATIONS 7 4.2.2 Solving Recurrence Relations Let us study a solution for solving linear homogeneous recurrence relations with constant coefficients. Let a n = c 1 a n 1 + c 2 a n 2 + + c a n be a recurrence relation. We guess that a n = r n is a solution to the above recurrence relation if and only if r is a solution to the equation r c 1 r 1 c 2 r 2 c = 0. The above polynomial equation in r (of degree is called the characteristic equation of the recurrence relation. The solutions of this equation are called the characteristic roots of the recurrence relation. To show the above, we prove the theorem below. Theorem 4.2.2 Let a n = c 1 a n 1 + c 2 a n 2 + + c a n be a recurrence relation with c 1, c 2,, c be real numbers. Let the equation r c 1 r 1 c 2 r 2 c have distinct roots r 1, r 2,, r. Then, the recurrence relation has solution a n = α 1 r n 1 + α 2r n 2 + + α r n for n 0 where α 1, α 2,, α are constants. Proof. We will first show that a n = α 1 r1 n + α 2r2 n + + α r n is a solution to the given recurrence relation. For this, note that as r 1, r 2,, r are roots of r c 1 r 1 c 2 r 2 c = 0, we have that ri n = c 1 ri n 1 + c 2 ri n 2 + + c for i = 1, 2,,. Thus, c 1 a n 1 + c 2 a n 2 + + c a n = c 1 (α 1 r1 n 1 + α 2 r2 n 1 + + α r n 1 c 2 (α 1 r1 n 2 + α 2 r2 n 2 + + α r n 2 + + + + c (α 1 r1 n + α 2 r2 n + + α r n = α 1 (c 1 r1 n 1 + c 2 r1 n 2 + + c r1 n + α 2 (c 1 r2 n 1 + c 2 r2 n 2 + + c r2 n + + α (c 1 r n 1 + c 2 r n 2 = α 1 r1 n + α 2r2 n + + α r n + + c r n Now, we show that if the sequence {a n n 0} is a solution to the above recurrence relation, then also a n = α 1 r1 n + α 2r2 n + + α r n is a solution. This is done by establishing that there exist constants α 1, α 2,, α so that {a n } agrees with a n = α 1 r1 n + α 2r2 n + + α r n. Since the sequence {a n} n 0 satisfies the given recurrence relation, we have: C i = a i for i = 1, 2,, If a n = α 1 r n 1 + α 2r n 2 + + α r n is a solution, then we must have that: α 1 + α 2 + + α = C 0 α 1 r 1 + α 2 r 2 + + α r = C 1 α 1 r1 2 + α 2r2 2 + + α r 2 = C 2 α 1 r1 1 + α 2 r2 1 + + α r 1 = C The above system of equations can be solved for α i, for i = 1, 2,,, by representing the above equations in the following matrix form. 1 1 1 1 α 1 C 0 r 1 r 2 r 3 r α 2 C 1 r1 2 r2 2 r3 2 r 2 α 3 = C 2. r1 1 r2 1 r3 1 r 1. α. C

8 CHAPTER 4. ADVANCED COUNTING TECHNIQUES The above system of equations has a unique solution as the matrix is a form of matrix called the Vandermonde matrix. It is however required that the roots of the characteristic equation r 1, r 2,, r are unique. Let us now loo at the following examples that illustrate the usage of theorem. Example. Consider the recurrence relation corresponding to the Fibonacci series F n = F n 1 + F n 2 with F 0 = 0 and F 1 = 1. Find a solution for F n. Solution. The characteristic equation for the recurrence relation given is r 2 r 1 = 0. The characteristic roots are r = 1± 1+4 2. So the roots are r 1 = 1+ 5 2 and r 2 = 1 5 2. ( n ( So the solution to the given recurrence relation is of the form a n = α 1+ 5 1 2 + 1 5 n, α2 2 for two constants α 1, α 2. We find ( α 1 and α 2 as follows. Since F 0 = 0, we have that α 1 + α 2 = 0. Similarly, since F 1 = 1, we ( have, α 1+ 5 1 2 + α 1 5 1 2. Another example is given below. Example. Consider the recurrence relation a n = 3a n 1 2 n 2 with a 0 = 1 and a 1 = 2. The solution for this recurrence realtion can be obtained as follows. Solution. The characteristic equation for the given recurrence relation is r 2 3r + 2. The roots of this equation are r = 1 and r = 2. So, the solution is of the form a n = α 1 1 n + α 2 2 n for some constans α 1 and α 2. To find α 1, α 2, we note that a 0 = 1 = α 1 + α 2 and similarly, a 1 = 2 = α 1 + 2α 2. Solving, we get α 1 = 0 and α 2 = 1. Thus, a n = 2 n for n 0. Example. Consider the towers of Hanoi problem. There are three towers named Tower1, Tower2, and Tower3. On Tower1 n diss are placed so that the largest dis is on the bottom, the second largest is on top it, so on, till the smallest dis at the top. The goal is to arrange the n diss in the same order on Tower3 with the condition that a larger dis cannot be placed on top of a smaller dis at any time. What is the number of moves required for this rearrangement. Solution. While the exact number of moves is not nown, one solution is as follows. Suppose we move the n 1 diss, except the largest dis, onto Tower2. This can be done in a n 1 moves 4.2.3 Solving Linear Nonhomogenous Recurrence Relations with Constant Coefficients 4.2.4 A Master Theorem A particular ind of recurrence relations that occur often in analysis of algorithms, especially the divide and conquer ind of algorithms. Consider recurrence relations of the form T(n = at(n/b + f(n, where a, b are constants. Theorem 4.2.3 (Masters Theroem Let a 1, b > 1 be constants and f(n be a function. Let T(n be defined as T(n = at(n/b + f(n Then T(n can be bounded as follows.

4.3. GENERATING FUNCTIONS 9 1. If f(n = O(n log a b ǫ for some constant ǫ > 0, then T(n = O(n log a b. 2. If f(n = Θ(n log a b then T(n = Θ(n log a b log n. 3. If f(n = Ω(n log a b+ǫ for some constant ǫ > 0, and if af(n/b cf(n for some constant c < 1 all all sufficiently large n then T(n = Θ(f(n. 4.3 Generating Functions Another important tool in counting is the use of generating functions. When using generating functions, we encode the values of a sequence as the coefficients of powers of a variable x in a formal power series. This helps us in manipulating the formal series to find answers about the counting problem. Generating functions can be used to also solve for recurrence relations. Definition 4.3.1 Given a sequence a 0, a 1, a 2,,, of real numbers we define the generating function for the sequence as the infinite series G(x = a 0 + a 1 x + a 2 x 2 + = a x When the sequence is finite, so that a = 0 for n + 1 for some integer n, the generating function reduces to the polynomial a 0 + a 1 x + + a n x n of degree n. While using generating functions, and manipulating them to solve counting problems, one often ignores the convergence properties or uniqueness of the infinite series. Some examples of in order. Example. If a n = (1.11 n, then the generating function is G(x = =0 (1.11n x n. Example. For the sequence a = ( m for m, being positive integers, and 0 m, the generating function has the form (1 + x m. 1 Example. Since 1 x = 1 + x + x2 + x 3 +, G(x = 1 1 x is the generating function for the series 1, 1, 1,,. Notice however that the above expansion is valid for x < 1. We now define operations on generating functions that can be useful when applying them to counting problems or when proving combinatorial identities. Definition 4.3.2 Let F(x = i=0 a ix i and G(x = i=0 b ix i be two generating functions. Then, the generating function H(x = F(x + G(x is defined as =0 H(x = (a i + b i x i i=0 and the generating function K(x = F(x G(x is defined as i K(x = a j b i j x i i=0 j=0

10 CHAPTER 4. ADVANCED COUNTING TECHNIQUES To be able to apply generating functions to certain couting problems, we need to expand the binomial coefficients as follows. For n, positive integers and 0 n, we now that ( n = n!!(n!. We now extend the definition to ( x for x IR and a nonnegative integer as: ( { x x(x 1(x 2 (x +1 =! if > 0, 1 = 0. The above extension applies to also x being negative and fractional. For example, ( 1/2 (1/2(1/2 1(1/2 2 (1/2 + 1 =! Similalry, ( n = ( n( n 1( n 2 ( n +1! = ( 1 n(n+1(n+2 (n+ 1! = ( 1 (n+ 1(n+ 2 n! = ( 1 ( n+ 1 The above results can be used to define an extended binomial theorem. Theorem 4.3.3 For x IR and x < 1, and u IR, (1 + x u = =0 ( u x Notice that for u being a positive integer, since ( u = 0 for > u, the extended binomial theorem reduces to the standard binomial theorem in this case. Finally, we apply generating functions to solve the following counting problems. Example. One of the typical counting problems is to find the number of integral solutions to an equation of the form x 1 + x 2 + + n = n satisfying certain mild constraints on the x i s. When x i 0, we now that the solution is ( +n 1. If the constraints are nontrivial such that xi n i, we can use generating functions to solve the problem as follows. Solution. Let us consider = 3 and n = 20. Then we claim that the number of integral solutions that satisfy 2 x 1 8, 1 x 2 6, and 4 x 3 15, is the coefficient of x 20 in the expansion of (x 2 +x 3 + +x 8 (x+x 2 + +x 6 (x 4 +x 5 + +x 15. The argument is as follows. In the expansion, a term of x 20 is obtained by taing the sum of three integers i 1, i 2, and i 3 such that i 1 + i + 2 + i 3 = 20. Moreover i 1 is the power of the term x i 1 in the first sum, i 2 is the power of the term x i 2 in the second sum, and similarly using x i 3 from the third sum. While the earlier method of solving this problem did not give us an easy way to place such constraints, using generating functions allows us the ability to deal with complex constraints. If we specify no constraints apart from x i 0 for all i, we notice that the solution to x 1 + x 2 + + x = n is the coefficient of x n in the expansion of (1 + x + x 2 +. Now, ( n (1 + x + x 2 + = 1 1 x = (1 x = ( i=0 i ( x i = = i=0 i=0 ( 1i( +i 1 ( i +i 1 i x i ( x i

4.3. GENERATING FUNCTIONS 11 So, the number of solutions to x 1 + x 2 + + x n = is ( +n 1 n. The example below illustrates using generating functions to solve recurrence relations. Example. Consider the following recurrence relation with B(0 = 1. Solve for B(n. n 1 B(n = B(iB(n i 1 i=0 Solution. Let B(x = i 0 b ix i be the generating function for B(n. Then, the recurrence relation gives us that xb 2 (x = B(x 1. Solving for B(x gives us: B(x = 1 1 4x 2x The other possibility that B(x = 1 + 1 4x/2x can be ignored given that by convention we set B(0 = 1. It remains to simplify the above expression. Using binomial theorem for non-integral coefficients, we have that ( 1 ( 1/2 n 0 n ( 4x n = n 0 ( 1 n 2 2n+1 x m B(x = 1 2x ( 1/2 n+1 Thus, equating the coefficient of x m in both sides, we have that B(n = ( 1/2 n+1 ( 1 n 2 2n+1. On simplification, ( 1/2 (1/2(1/2 1(1/2 2 (1/2 n = = ( 1n 1 3 5 (2n 1 n + 1 (n + 1! 2 n (n + 1! Finally, B(n = 1 3 5 (2n 1 (n+1! 2 2n+1. On multiplying and dividing by n! we get that B(n = (2n! n!n! 1 n+1.