CPSC 121: Models of Computation. Module 9: Proof Techniques (part 2) Mathematical Induction

Transcription

1 CPSC 121: Models of Computation Module 9: Proof Techniques (part 2) Mathematical Induction

2 Module 9: Announcements Midterm #2: th Monday November 14, 2016 at 17:00 Modules 5 (from multiple quantifiers onwards) to 8 plus labs 4, 5, 6 and 7. Same locations as for midterm #1. You can bring one 8.5 x 11in (21.59 x 27.94cm) two sided sheet of paper. No textbook, calculator or other electronic equipment is allowed. 2

3 Module 9: Announcements Assignment #4 is due Thursday November 10th at 11:00. Pre-class quiz #10 is tentatively due on Monday November 21st at 19:00. Textbook sections: Epp, 4th edition: 6.1, 7.1 Epp, 3rd edition: 5.1, 6.1 Rosen, 6th edition: 2.1, 2.3 up to the top of page 136. Rosen, 7th edition: 2.1, 2.3 down to the bottom of page

4 Module 9: Mathematical Induction By the start of class, you should be able to Convert sequences to and from explicit formulas that describe the sequence. Convert sums to and from summation/σ notation. Convert products to and from product/π notation. Manipulate formulas in summation/product notation by adjusting their bounds, merging or splitting summations/products, and factoring out values. 4

5 Module 9: Mathematical Induction Pre-class quiz #9 Very well done. Being comfortable with summations is important not only for CPSC 121, but for following courses too! As usual, we will revisit the open-ended question shortly. 5

6 Module 9: Mathematical Induction If you haven t already done so, read the rest of the chapters on mathematical induction (Epp4, 5.2 to 5.4, etc). After this reading, you will be able to: Given a theorem to prove stated in terms of its induction variable (i.e., usually, in terms of n), write out the skeleton of an inductive proof including: the base case(s) that need to be proven, the induction hypothesis, and the inductive step that needs to be proven. 6

7 Module 9: Mathematical Induction By the end of this module, you should be able to: Establish properties of self-referential structures using inductive proofs that naturally build on those self-references. Critique formal inductive proofs to determine whether they are valid and where the error(s) lie if they are invalid. Prove properties of the non-negative integers (or a subset) with appropriate self-referential structure using weak or strong induction as needed. 7

8 Module 9: Mathematical Induction?? 1. How can we? convince ourselves that an algorithm? does what it's supposed to do?? 2. How do we determine whether or not one algorithm is better than? another one????useful tool Mathematical induction is a very? when proving the correctness or efficiency of an algorithm.?? We will see several examples of this.???? CPSC 121: the BIG questions: 8

9 Module 9: Mathematical Induction Module Summary Example: single-elimination tournaments. Defining and validating mathematical induction. More examples where we can use induction. A slightly different type of induction. 9

10 Module 9.1: Single-elimination tournaments Problem: single-elimination tournament Teams play one another in pairs The winner of each pair advances to the next round Los Angeles Vancouver Toronto St. Louis Los Angeles Toronto Los Angeles Montréal Pittsburgh New York Isl. Montréal Buffalo Round 2 Montréal New York Isl. Montréal Round 3 Round 4 10

11 Module 9.1: Single-elimination tournaments Question: if we have n rounds of playoffs, how many teams can participate? a) n b) 2n c) n2 d) 2n e) None of the above. 11

12 Module 9.1: Single-elimination tournaments How can we prove this result formally? We will use a technique called mathematical induction. We will convince ourselves that it is a valid proof technique. Many theorems involve summations, so it is important to be comfortable manipulating them. Hence the reading for pre-class quiz #9. 13

13 Module 9.1: Single-elimination tournaments First we look at the following question instead: If t teams can participate in a playoff with n rounds, how many teams can participate in a playoff with n+1 rounds? a) t+1 b) 2t c) t2 d) 2t e) None of the above. 14

14 Module 9.1: Single-elimination tournaments Proof (with holes): Consider an unspecified playoff with n rounds. Assume that We can think of a playoff with n+1 rounds as follows: Two playoffs with n rounds proceed in parallel. The two winners then Since each playoff with n rounds has 16

15 Module 9.1: Single-elimination tournaments How do we use this to prove what we want? Recall Universal Modus Ponens: P(a) x D, P(x) Q(x) Q(a) If we define P(t, n): Up to t teams can participate in a playoff with n rounds then we have proved t Z+ n Z+, P(t, n) P(2t, n+1) 17

16 Module 9.1: Single-elimination tournaments So We know that only 2 teams can play with 1 round. This is P(2, 1). Given P(2, 1) t Z+ n Z+, P(t, n) P(2t, n+1) We can deduce: P(4, 2) 18

17 Module 9.1: Single-elimination tournaments Given P(4, 2) t Z+ n Z+, P(t, n) P(2t, n+1) We can deduce: P(8, 3) Given P(8, 3) t Z+ n Z+, P(t, n) P(2t, n+1) We can deduce: P(16, 4) 19

18 Module 9.1: Single-elimination tournaments Given P(16, 4) t Z+ n Z+, P(t, n) P(2t, n+1) We can deduce: P(32, 5) Etc... Therefore we can show that n Z+, P(2n, n) 20

19 Module 9.1: Single-elimination tournaments Another way to think about this: Given an unspecified positive integer n. To prove P(2n+1, n+1): First we prove P(2n, n). Then we use the fact that t Z+ n Z+, P(t, n) P(2t, n+1) Hence induction is more or less the same as recursion! A fun example involving induction: 21

20 Module 9: Mathematical Induction Module Summary Example: single-elimination tournaments. Defining and validating mathematical induction. More examples where we can use induction. A slightly different type of induction. 22

21 Module 9.2: Defining and validating M.I. Define Q(n) by: Q(n): P(2n,n) In the previous example, we proved Q(1) n Z+, Q(n) Q(n+1) and deduced that n Z+, Q(n) Why is this a valid proof technique? 23

22 Module 9.2: Defining and validating M.I. Theorem: Q(1) ^ ( n Z+, Q(n) Q(n+1)) n Z+, Q(n) Proof: We use a proof by contradiction. Suppose that the premises hold, but that n Z+, Q(n) is false. There is at least one value of n for which Q(n) does not hold. Let y be the smallest such value. Clearly y > 1, so y 1 Z +. 24

23 Module 9.2: Defining and validating M.I. Proof (continued): Because y is the smallest value for which Q(n) is false, and y 1 Z+, Q(y - 1) is true. But then the 2nd premise implies that Q(y) is true, a contradiction (since we assumed Q(y) was false). Therefore the conclusion must be true, that is n Z+, Q(n) holds. 25

24 Module 9.2: Defining and validating M.I. So a proof by mathematical induction has One (or more) base case(s): Q(1) Usually very straightforward to prove. An induction step: n Z+, Q(n) Q(n+1) We can use any proof technique we know for it. Usually a direct proof will work well. 26

25 Module 9: Mathematical Induction Module Summary Example: single-elimination tournaments. Defining and validating mathematical induction. More examples where we can use induction. A slightly different type of induction. 27

26 Module 9.3: More induction examples Example 1: recall the exercises from our first class: Make group of 5 students. Then order each group from in order of day of birth by swapping pairs of adjacent students. We claimed the maximum number of swaps for n students is n(n-1)/2. 28

27 Module 9.3: More induction examples How many swaps do we need? Suppose we place students from left to right. The students already placed are ordered by day of birth. We swap each new student with his/her neighbour until he/she is at the right place. The ith student may be swapped with all previous i-1 students. n 1 So the total number of swaps is Hence we need to prove that n 1 i=0 i= i=0 i n(n 1) 2 29

28 Module 9.3: More induction examples Which fact(s) do we need to prove? 0 a) i=0 i=0 ( )( )( ) n n( n 1) (n+ 1) n + i= b) n ℤ i= 2 2 i=0 i= 0 n 1 n n( n 1) ( n+1) n + c) n ℤ +, i= n ℤ, i = 2 2 i= 0 i =0 ( n 1 d) Both (a) and (b) ) e) Both (a) and (c) 30

29 Module 9.3: More induction examples Proof: Base case: n = 1 0 Clearly, with only 1 student, no swap is needed: i =0 i=0 which is equal to 1 (1-1)/2 Induction step: Pick an unspecified n 1. Assume that when we have n students, we need at most n(n-1)/2 swaps. Equivalently we assume that n 1 n(n 1) i= 2 i=0 This is called the Induction Hypothesis. 32

30 Module 9.3: More induction examples Proof (continued) Induction step (continued) For n+1 students, we need at most swaps. n ( ) n 1 i= i + n i=0 i =0 By the induction hypothesis, this is equal to n(n 1) ( n+ 1) n + n= 2 2 swaps, as required for the induction step. Hence by the principle of M.I., the theorem holds. QED 33

31 Module 9.3: More induction examples Example 2: geometric series We will prove that for every value of a 0, 1: t t+ 1 a 1 i a = a 1 i=0 These summations occur frequently when we need to determine the running time of divide-and-conquer algorithms. 34

32 Module 9.3: More induction examples Proof: Base case: t = 0 1 a 1 0 =1 In this case the summation is a = 1, and a 1 Induction step: Pick an unspecified t 0. Assume that Now Hence by the principle of M.I., the theorem holds. QED 35

33 Module 9.3: More induction examples Example 3: prove that n 4, 2n < n! Rules for inequalities: Start from one side (say the left side) Work step by step towards the other. When dealing with <, you are allowed to make the expression larger, but never smaller. Example: if I am smaller than you, then I am still smaller than you when you stand on a bench. 36

34 Module 9.3: More induction examples Proof: by induction on n. Base case: n = Induction step: we want to prove that n, if 2n-1 < (n-1)! then 2n < n! Consider an unspecified n. Induction hypothesis: assume that 2n-1 < (n-1)! Then 2n = 2(2n-1) < 2(n-1)! < n(n-1)! = n! this is where we approximate the induction hypothesis is used here Hence by the principle of M.I., n 4, 2n < n! 37

35 Module 9.3: More induction examples Example 4: prove that for every n 1, n i2 n i=1 Note (this doesn t help with the proof; it s simply an interesting mathematical fact): 1 π2 i2 = 6 i=1 38

36 Module 9.3: More induction examples Example 5: consider the following DFA:

37 Module 9.3: More induction examples This DFA accepts the empty string, and which other strings? a) Strings whose length is divisible by 5. b) Unsigned binary integers that are prime numbers. c) Strings where every 1 is followed by 01. d) Unsigned binary integers that are divisible by 5. e) None of the above. 40

38 Module 9.3: More induction examples What should we prove (ignoring empty strings)? a) The DFA is in state 0 if and only if it has seen one of the strings 1010, 1111 or b) The DFA is in state 0 if and only if s is divisible by 5. c) The DFA is in state 0 if and only if s is terminated by the string 101. d) The DFA is in state 0 if and only if s contains the substring 101 followed by zero or more 0's. e) None of the above. 42

39 Module 9.3: More induction examples Theorem: The DFA ends up in state r after reading a string s if and only if q Z, s = 5q+r. Proof: by induction on the length n of s. Base case: n = 1 This can be verified easily by looking at the DFA. Induction step: consider an unspecified n > 1. Induction hypothesis: the theorem holds for strings with n 1 characters. Consider an unspecified string s with n bits. 44

40 Module 9.3: More induction examples Proof (continued): Suppose that s = bn-1bn-2...b2b1b0. After reading the bits bn-1bn-2...b2b1, the DFA is in state r. By the induction hypothesis, the binary integer bn-1bn-2...b2b1 = 5q + r for some integer q. Now, s = 2 (bn-1bn-2...b2b1) + b0 = 2(5q+r) + b0 = 10q + 2r + b0 and so s = 5 (2q) + (2r + b0). So the DFA should have the following transitions: r b0 2r+b0 r b0 2r+b

41 Module 9.3: More induction examples Proof (continued): It does, and hence after reading all n bits of s, the DFA ends up in state r if and only if q Z, s = 5q+r. Hence by the principle of mathematical induction, the DFA ends up in state r after reading a string s if and only if q Z, s = 5q+r. QED Corollary: the DFA is in state 0 if and only if s is divisible by 5. 46

42 Module 9.3: More induction examples Mathematical induction is how we analyze recursive algorithms: Example: (define (sum n) (if (= n 0) 0 (+ n (sum (- n 1))))) n ( ) n 1 i =n+ i i=0 i =0 The type of induction we discussed so far works when the recursive call is with (- n 1) 47

43 Module 9: Mathematical Induction Module Summary Example: single-elimination tournaments. Defining and validating mathematical induction. More examples where we can use induction. A slightly different type of induction. 48

44 Module 9.4: A slightly different type of induction How do we handle more general recursions? A recursive function/method has One or more base cases When it can compute the answer directly An induction step It computes recursively the solutions to one or more smaller sub-problems, and combines them to obtain its answer. Our induction proofs mimic this structure. It is similar to the templates you used in CPSC

45 Module 9.4: A slightly different type of induction We use a slightly different induction hypothesis. Instead of proving that x Z+, Q(x) Q(x+1) We prove that x Z+, (Q(1) ^ Q(2) ^... ^ Q(x)) Q(x+1) We can also show that this type of induction is a valid proof technique. The proof is the same proof by contradiction that we looked at earlier. 50

46 Module 9.4: A slightly different type of induction CPSC 110 review: A binary tree is a data structure that is defined recursively: it is either Empty, or A node with some data, and two children that are themselves trees

47 Module 9.4: A slightly different type of induction Example 6: consider the following function: (define (tree-size t) (if (null? t) 0 (+ 1 (tree-size (left-child t)) (tree-size (right-child t))))) How can we prove that it correctly computes the number of (non-null) nodes of the tree? 52

48 Module 9.4: A slightly different type of induction We prove this using mathematical induction on the size of the tree. Base case: t is null In this case t contains exactly 0 nodes. Induction step: Assume the algorithm works for trees that are smaller than t. Because the left sub-tree of t is smaller than t, the 1st recursive calls correctly returns the size of the left subtree of t. 53

49 Module 9.4: A slightly different type of induction Proof (continued) Induction step (continued) Similarly the right sub-tree of t is smaller than t, and so the 2nd recursive call correctly returns the size of the right sub-tree of t. But we return 1 + the sum of the values returned by the recursive calls. This is exactly the size of t (1 for the root, and the sum of the sizes of the two sub-trees). Hence by the principle of M.I., our algorithm computes correctly the size of every tree. QED 54

50 Module 9.4: A slightly different type of induction Example 7: every positive integer n greater than 1 can be written as a product of primes. What base case(s) is/are the most appropriate? a) n = 1 b) n = 2 c) n = 2, 3 or 5. d) n is prime. e) None of the above. 55

51 Module 9.4: A slightly different type of induction Proof: we prove the result by induction on n. Base case: n We can write n = n and n is a product of primes (with only 1 prime). Induction step: n Suppose that every value from 2 to n 1 is a product of primes. Since n We can write n = 57

52 Module 9.4: A slightly different type of induction Proof (continued) Since 2 a < n, Since 2 b < n, Then Hence by the principle of M.I., every positive integer larger than 1 can be written as a product of primes. QED 58

53 Module 9.4: A slightly different type of induction Example 8: we can find the ith smallest element in an unsorted list as follows: Pick a random element x of the list. Divide the list into three sublists: list-smaller: elements smaller than x list-equal: elements equal to x list-larger: elements larger than x Then search list-smaller if i length of list-smaller list-larger if i > length of list smaller + length of list-equal otherwise return x 59

54 Module 9.4: A slightly different type of induction Example 8: continued This algorithm is called randomized-quick-select. Racket code will be posted on the course web site. A student shows the expected number of steps S(n) of the algorithm on a list with n elements is: S(1) = 4c S(2) = 12c S(3) = 20c S(n) 2cn + S( 3n/4 ) when n 4 60

55 Module 9.4: A slightly different type of induction Example 8: continued Prove that for every n 1, S(n) 8cn. 61

56 Module 9.4: A slightly different type of induction Example 9: binary search Suppose we have something like a list, but whose ith element and length can be found in a single step. This structure is called a vector in Racket. It is similar to an ArrayList in Java. We assume that we have such a vector, sorted in increasing order. Examples: ( Anh, Charles, Dora, Gregor, Wei ). We want to find the position of a given element (for instance, Dora ). 62

57 Module 9.4: A slightly different type of induction The following algorithm (formerly known as B) works: (define (binary-search avector first-pos last-pos x) (if (> first-pos last-pos) false (if (= first-pos last-pos) (if (= (vector-ref avector first-pos) x) first-pos false) (local ((define mid-pos (quotient (+ first-pos last-pos) 2))) (if (= x (vector-ref avector mid-pos)) mid-pos (if (< x (vector-ref avector mid-pos)) (binary-search avector first-pos (- mid-pos 1) x) (binary-search avector (+ mid-pos 1) last-pos x))))))) 63

58 Module 9.4: A slightly different type of induction Prove that binary search makes at most log2 (size+1) comparisons if size 1. The proof is once again by induction on size. Base case: size = The algorithm uses comparison, which is Induction step: Consider an unspecified size Assume that. 64

59 Module 9.4: A slightly different type of induction Prove that binary search makes at most log2 (size+1) comparisons if size 1. The proof is once again by induction on size. Base case: size = 1 The algorithm uses 1 comparison, which is log2 2 Induction step: Consider an unspecified size > 1. Assume that for every integer k such that 1 k size 1, binary search makes at most log2 (k+1) comparisons for a vector with k elements. 65

60 Module 9.4: A slightly different type of induction We first make 1 comparison with the middle element. If it is the element we want, we make 0 more comparisons. Otherwise If size is odd, then we recurse on (size-1)/2 elements. size-1/2 size-1/2 If size is even, then we recurse on at most size/2 elements size/ size/2 In either case, the recursion is on at most (size 1)/ 2 elements 66

61 Module 9.4: A slightly different type of induction So the number of comparisons is at most 1 + number of comparisons for (size 1)/ 2 elements 1 + log2 ( (size 1)/ 2 + 1) by the ind. hypothesis. 1 + log2 ( (size 1)/ ) 1 + log2 ( (size + 1)/ 2 ) log2 (2 * (size + 1)/ 2 ) If size is odd, this is log2 (size + 1) If size is even, this is log2 (size + 2). Note, however, that log2 x and log2 (x + 1) are only different if x is a power of 2, which can t happen when x is odd. So log2 (size + 2) = log2 (size + 1) when size is even. 67

62 Module 9.4: A slightly different type of induction This completes the induction step and the proof of the theorem. 68