Mathematical Background

Transcription

1 Chapter 1 Mathematical Background When we analyze various algorithms in terms of the time and the space it takes them to run, we often need to work with math. That is why we ask you to take MA 2250 Math for CS. For example, when the following algorithm runs a two-layer for loop with loop variable k going from 1 through j, and another, j, going from 1 to n, for (j=1; j<=n; j++) for (; k<=j; k++) print k; we might want to know how many prints it has to do... 1

2 Let s find out For j [1, n] and k [1, j], the inner loop does one print, the total print statements the whole segment will carry out is j j=1 1 = j=1 j = n? = n(n + 1). 2 Assume each print takes a time unit, it takes n(n+1) 2 units to complete the above segment. In general, if such a general step takes f(j) steps, for j [1,n], the total time is j=1 f(j). Since we are still using the sequential platform a lot, the summation related analysis is quite useful. 2

3 More about summation Given a sequence a 1, a 2,..., a n, their finite sum a 1 + a 2 + +a n, n 0, can be written as the following: a k. When n = 0, the sum is defined to be 0. Another special case is the following: a k = lim n n a k. The sum operation has the linearity property, i.e., for any real number c and any finite sequences a 1, a 2,..., a n, and b 1, b 2,..., b n, (ca k + b k )? = c a k + b k. 3

4 Arithmetic series The arithmetic series is simply the following summation: k = n. We have already seen that k = n(n + 1). 2 We also have the following: and k 2 = n(n + 1)(2n + 1), 6 k 3 = n2 (n + 1) 2. 4 Question: Is there a pattern for n k m? Answer: n 1, n k m? = Θ(n m+1 ). 4

5 Harmonic series and more... The Harmonic series is also pretty useful in algorithm analysis, particularly in analyzing the Quicksort algorithm. For positive integers n ( 1), the n th harmonic number is H n = 1 k = n = ln n + O(1). The k th harmonic produced by a violin string is the fundamental tune produced by a string 1/k times as long. More generally, let ζ(z) = k 1 1 k z = k 1 k z, it is immediate that ζ(1) > H n. It is also well known that ζ(2) = k 1 1 k 2 = = π2 6. 5

6 We also have ζ(4) = k 1 ζ(6) = k 1 Just for fun... 1 k 4 = = π k 6 = = π The pattern continues with ζ(k), k is even, i.e., such a ζ(k), when k is even, equals a product of a rational number and π k, although itself is not since π is not rational. No such results hold for ζ(k), where k is odd, thus they can t be proved irrational by the same token. But, it has been established that ζ(3) is indeed irrational, and one of ζ(5), ζ(7), ζ(9), and ζ(11), is irrational as well, although it is uncertain which one is. 6

7 Geometric series By the geometric series, we mean the following: k=0 x k = 1 + x + + x n. When x = 1, we have that k=0 x k = n + 1. In general, when x 1, it is also easy to see that k=0 Finally, when x < 1, k=0 x k = lim n = 1 x 1 lim n x k? = x n+1 1 x 1. n k=0 ( x n+1 1 x k x n+1 1 = lim n x 1 ) = 1 1 x. 7

8 Telescoping series For any sequence a k0 1, a k0,..., a n, we have n 0 k=k 0 (a k a k 1 )? = a n0 a k0 1. Below is an example following the same reasoning. = n 1 n 1 1? ( 1 = k(k + 1) k 1 ) k + 1 ( ) ( ) ( n 1 1 ) n = 1 1 n. Question: How does the above telescope? Again, we will use the above result in analyzing the Quicksort algorithm. 8

9 A little about products Similarly, the product of a 1, a 2,..., a n can be represented as the following Π n a k. Again, if n = 0, we define the product to be 1. A quite useful case is the following: for any positive c, log c (a b) = log c a + log c b. In general, we have the following: log c Π n a k = log c a k. Log of the product is the sum of log. 9

10 Summation evaluation It is not always easy to evaluate summations. Thus, we are often happy with an upper bound of the summation. This is often all we need in estimating the running time of an algorithm: Find out an upper bound of an algorithm, then see if our system can handle it.... In particular, the upper bound of a sum of a series can be found by bounding each item of the series with, e.g., the largest item in the series. For example, k n = n 2. In general, if we let a max = max k [1,n] a k, then n a k na max. 10

11 A bit deeper... Given the summation n k=0 a k, if 0 < a k+1 /a k r < 1 for all k 0, then a k a 0 r k. Notice that the above really says that none of the a k equals 0. If some of the a k equals 0, we take them out. We then have the following k=0 a k? k=0 a 0 r k = a 0 using a result we got earlier. k=0 r k = a 0 1 r, 11

12 An example Given k 3 k = , we want to find out a simple upper bound of this sum. To apply the previous result, which starts with 0, we rewrite the above as the following: k 3 k = k 1 k 3 k = k+1 1 k k+1 = k=0 k k+1. Take a k = k+1 3 k+1, we have a 0 =

13 Moreover, for all k 0, the ratio of the two consecutive terms is r = ak+1 a k = (k + 2)3k+1 (k + 1)3 k+2 = 1 3 k + 2 k Thus, we immediately have that k 3 k = = a 0 1 r = 1 3 k=0 k k+1 a /3 = 1. r k k=0 Thus, this complicated sum is no more than 1. We note that computer programs, such as Maple or MATLAB, have been developed to find out closed-form solutions for summations if they exist. 13

14 Rules of sum and product You must have learned this in, e.g., MA2250 or MA3200. The rule of sum states that the number of ways to choose an element from one of two disjoint sets is the sum of the sizes of both. For example, if each position on a car s license place contains either a digit or a letter, there are 36 choices for that position. The rule of product states that the number of ways to choose an ordered pair is the number of ways to choose the first times the number of ways to choose the second. To continue the previous example, if there are four positions in a license, there will be 36 4 (= 1,679,616) different plates. Question: How about three letters followed by three digits, or seven digits? 14

15 Permutation In many cases, recursion provides a powerful problem solving tool. The generation of permutation provides a demonstrating example. The permutations of three items, e.g., a, b, and c are abc, acb, bac, bca, cab, and cba. Given n distinguishable objects: e 1, e 2,..., e n, we want to know 1) how many ways are there to line them up, i.e., the number of permutations, and 2) how to construct all these permutations. Question: If there are three burgers, four soft drinks, and two side orders, how many value meals can you make out of them? 15

16 Number of permutations There are n! permutations for n distinguishable objects. Clearly, there are n choices for the first position, n 1 for the second,..., and 1 for the last. Hence, by the rule of product, there are n (n 1) 1 = n! ways. Recursively, let M(n) be the number of permutations for n distinguishable objects, again, by applying the rule of the product, we have that M(1) = 1 M(n) = n M(n 1),n > 1. Solving the above, we have that M(n) = n!. Question: How? 16

17 A bunch of notations Let E = {e 1,..., e n } denote the set of n objects, and let E i be the set obtained by removing e i from E. For example, if E = {a, b,c}, i.e., e 1 = a, e 2 = b and e 3 = c, then E 1 = {b, c}, and E 2 = {a,c}, and E 3 = {a,b}. Let perm(x) be the collection of all the permutations based on X, and let e i perm(x) be all the permutations of X {e i } obtained by prefixing each and every permutation in X with e i, where stands for concatenation and is often omitted when it is clear. Thus, perm(e 1 ) = {bc, cb}, e 1 perm(e 1 ) = {abc,acb}, e 2 perm(e 2 ) = {bac,bca}, and e 3 perm(e 3 ) = {cab,cba}. 17

18 Permutation construction We will do it recursively. For the base case, when n = 1, E = {e 1 }, the only permutation is just e 1. In general, perm(e) = e 1 perm(e 1 ) e n perm(e n ). For example, if E = {a,b, c}, then perm(e) is a perm(e 1 ) plus those in b perm(e 2 ), and those in c perm(e 3 ). In other words, perm(e) = {abc,acb,bac, bca,cab, cba}. Question: How do we implement the above idea with a computer algorithm? 18

19 An algorithm Assume we store (a 0,..., a n 1 ) in list[n]. //The following generates all permutations where //list[0] through list[k-1] are fixed. void Perm(T list[], int k, int m){ int i; if (k == m) { //With only one choice for a[k], //we are done. Just print it out. for (i = 0; i <= m; i++) System.out.print(list[i]); System.out.println();} else for (i = k; i <= m; i++) { //Fix list[k] with something as well. //The first k+1 items are fixed. Swap(list[k], list[i]); //Generate the permutations //with this many items fixed. Perm(list, k+1, m); //Put it back and try next. Swap(list[k], list[i]); } } 19

20 What is going on? Let s check out an example of applying the procedure to (a, b, c). In general, when calling Perm(list, k, m) on an array a[0, m], what we want to get is all the permutations of the elements stored within a[0, m], where the prefix a[0, k-1] is fixed. In particular, Perm(a, 0, n-1) is to generate all the permutations of the elements of a[0, n-1] with nothing fixed. Notice that a[0, -1] is an empty segment. When k=m, there is only one choice of a[k] (=a[m]), we just print out the completed permutation. Otherwise, we extend the prefix with each and every one in a[k, m], then recursively generate all the permutations with this newly augmented prefix. Homework: Generate all the permutations of {a,b, c} by following the algorithm. 20

21 An example Let n = 4, let the four elements be {a,b, c, d}, and let k = 2. 1.a. We first fix the first two elements, position 0 and ( 1, by choosing two out of four elements in 4 2) = 6 ways: {a,b}, {a,c}, {a, d}, {b,c}, {b,d}, and {c, d}. 1.b. For each of these six choices, we arrange the two chosen elements in two (=2!) ways, e.g., ab and ba. 2. For position 2, there are two choices of placing an element there. For example, if we choose a and b in Step 1.a, we can place either c or d in position We finally have just one symbol left, e.g., if we choose a, b and c in Steps 1.a, 1.b, and 2, we have just d left for position 4. In total, we have = 24 = 4! permutations, which agrees with an earlier result. 21

22 How many in general? ( n k) 1. We choose k ( [1,n]) elements in ways, and then form a permutation using these many elements. Thus, there are ( ) n k k! ways to make this segment of k elements fixed. 2. We then have n k ways to select one out of n k elements to place it in position k Finally, there are (n k 1)! ways to arrange the remaining n k 1 elements. Thus, the total number of permutations we generate with the above Perm(a, 0, n-1) procedure is, by the rule of product, ( n) k! (n k) (n k 1)! = n!. k So, the algorithm works. 22

23 k Permutation Step 1 in the previous process is to form a list of k elements chosen from n, often called k- permutation. More generally, a k permutation of a finite set S, containing n elements, is an ordered sequence (Step 1.b) of k elements taken from S (Step 1.a). For example, if S is {a,b, c, d}, there are twelve 2-permutations of S, {ab,ac, ad,..., cd}. The number of the k permutations of S, containing n elements, P(n, k), is thus n(n 1) (n k + 1) = n! (n k)!. For example, when = 4 and k = 2, P(4,2) = 4! 2! = 24 2 =

24 Combination Notice that k-permutation is associated with an order, which sometimes we can do without. For example, a group, or a set, of things do not come with an order. This latter concept is captured with the notion of combination. Mathematically speaking, a k-combination of an n-set S is simply a subset of S. Question: With ten students, how many project teams of four can we form? As a start, there are twelve 2-permutations out of four symbols, including both ab and ba, both of which refer to the same subset {a,b}. Thus, the answer should be 6. On the other hand, there are indeed exactly six 2-combination of the 4-set {a,b, c, d} : {a, b}, {a, c} {a,d}, {b, d}, {b,c}, and {c, d}. 24

25 Combination formula Given an n set, to find out P(n, k), the number of all the k permutations, we start with C(n,k), the number of all the k combination. For each such k-combination, a set with k elements, we can generate k! k permutations. For example, {a, b}, a combination of a and b, corresponds to two different permutations: ab and ba. Thus, P(n, k) = C(n, k) k!, in other words, C(n, k) ( = ( n) ) = k P(n, k) k! = n! (n k)!k!. To find out how many project teams we can get out of ten people if each team contains four people, we have C(10,4) = P(10,4) 4! = 10! 4!6! = =

26 What about proofs? Besides doing calculations, we often need to prove statements, namely, providing mathematically solid arguments to show that something must be correct beyond any doubt. There are several proof techniques, including proof by induction, proof by contradiction, and proof by construction because of the mechanical nature of a computer. You must have done a bunch in MA2250. Each technique has its place in our study of algorithms. We will talk about the first two here, and deal with the constructive approach later. All such techniques, particularly the last one, will play a crucial role in CS

27 Proof by induction When a statement is involved with a natural number argument, n, we have infinite number of instances to prove. Proof by mathematical induction turns out to be quite effective in this case. The general approach is as follows: 1.Show that some statement, P(n), is true for a base case n 0, i.e., P(n 0 ). 2. Assume that P(n) is true for n, prove that P(n) is true for n + 1, i.e., n n 0, P(n) P(n + 1). We can then conclude that P(n) holds for all n n 0, i.e., n n 0, P(n). We also often use the following equivalent strong induction: Assume that P(k) holds for all k < n, and prove P(n) holds for n, then, n P(n). 27

28 An example For all n 1, n i=1 i 2 = n(n+1)(2n+1) 6. Proof by induction: When n = 1, clearly, LHS = 1 i=1 i 2 = 1 = = RHS. Assume that, for n 1, n i=1 i 2 = n(n+1)(2n+1) 6. LHS = n+1 i=1 i 2 = i=1 i 2 + (n + 1) 2 n(n + 1)(2n + 1) = + (n + 1) 2 6 (n + 1) = [n(2n + 1) + 6(n + 1)] 6 (n + 1) = (2n 2 + 7n + 6) 6 (n + 1) = (n + 2)(2n + 3) 6 (n + 1) = [(n + 1) + 1)[(2(n + 1) + 1)] 6 = RHS. 28

29 A stronger example Given the following Fibonacci series: F 0 = 1, F 1 = 1, F n = F n 1 + F n 2, n 2. Then, for all n 1, F n < ( 5 3 ) n. Proof: Assume that, for all k < n, F k < then we have ( ) 5 n 1 F n = F n 1 + F n 2 < + ( ) 3 5 n [ 3 = ] = 24 ( ) 5 n < ( ) 5 n. 3 ( 5 3) n 2 ( 53 ) k, Question: Where and how did we use the inductive assumption? 29

30 Happy Valentine s Day! Claim: The colors of all the flowers are the same. Proof: By induction on n, the number of flowers in a bouquet. The base case of n = 1 is trivial. Assume that the statement is true for n, and we now place n + 1 flowers into the bouquet. Take one, f 1, out. As there are only n flowers left, by the inductive assumption, they must be of the same color, say c. Now, take another one, f 2, out of the bouquet and put f 1 back. As the bouquet still has n flowers, those n flowers must also be of the same color, c. Particularly, the color of f 1 must also be c. Thus, the color of all of the n + 1 flowers is c. Therefore, by the inductive principle, the colors of all the flowers are the same. 30

31 What s wrong? When we have two flowers in the bouquet, i.e., n+1 = 2 (n 1 = 0), our implicit claim that the color of f 1 must be the same as that of those (n 1) flowers left in the bouquet does not hold up(?). Thus, F(1) F(2). As a result, the inductive part, for all n 1, F(n) F(n + 1), doesn t work. If we treat n = 2 as the base case, then, although we do have the inductive statement, for all n 2, F(n) F(n + 1), we can t prove the base case, i.e., the colors of any two flowers are the same. Thus, we still can t finish the whole proof. Lesson: To prove something inductively, both parts have to be checked out. 31

32 Proof by contradiction To prove some statement P is true, assume that it is false first, i.e., P is true. If this latter assumption leads to a contradiction to what has been known, or common sense, then that assumption, i.e., P is true, must be false. Thus, the original statement P must be true. For example, Either I am tired or sick. If I am sick, I go home. I am not going home. So, I am tired. Assume that I am not tired, then, it must be the case that I am sick. It follows that I go home. But, it is a contradiction to the given statement that I am not going home. Thus, the assumption is incorrect. As a result, I must be tired. 32

33 Another example It is easy (?) to prove that, for all x > 0, x + 4 x 4. Now let s assume that it is false, i.e., for some x > 0, x + 4 x < 4. (Notice that xp(x) x P(x). For example, It is not the case that I passed all the courses means that, for some course, I did not pass it. ) Since x > 0, we have that x + 4 x < 4 iff x2 + 4 < 4x iff x 2 4x + 4 < 0 iff (x 2) 2 < 0, which contradicts the fact that, for all x, (x 2) 2 0. Hence, it must be the case that, for all x > 0, x + 4 x 4. 33

34 Yet another example Let s prove that 2 is not a rational number. Just assume that 2 is a rational number, i.e., for some m and n, 2 = m /n. By dividing both m and n by all the factors common to both of them, we obtain 2 = m/n for some integers m and n, which share no common factors. Since 2 = m/n, m = n 2, i.e.,m 2 = 2n 2. Therefore, m 2 is even. Thus, m must be even,(?) i.e., for some k, m = 2k. Replace m with 2k, we have that 4k 2 = 2n 2, i.e., n 2 = 2k 2. By the same token, n is also even. Hence, both m and n has a common factor of 2. This is a contradiction. Hence, 2 must not be rational. 34