CS1800: Strong Induction. Professor Kevin Gold

CS1800: Strong Induction Professor Kevin Gold

Mini-Primer/Refresher on Unrelated Topic: Limits This is meant to be a problem about reasoning about quantifiers, with a little practice of other skills, too Intuitively, the limit is a number the sequence a n gets closer and closer to as n gets larger There exists x [the limit] such that for all ϵ [distances from the limit] we can find some n 0 [ point of no return ] such that for all n n 0, a n is no further away than ϵ. Any ϵ has a corresponding n0 ϵ n0 (3) x all ϵ or closer to x ϵ n0 (5) x ϵ or closer

Weak versus Strong Induction Both kinds of induction are used to argue that a statement involving some n keeps being true But they primarily differ in their inductive steps Weak induction: If it is true in the current step (n=k), it will be true for the next step (n=k+1) Strong induction: If it was true until now (1 < n < k), it will be true for the current step (n=k) It s a stronger assumption - a stronger inductive hypothesis - but if we can justify it, we can prove some things more easily

Brief Review of Proof By Weak Induction 1 2 3 4 5 Recall that proof by induction consists of two pieces: An inductive step - proving that if the statement is true for n=k, it is true for n=(k + 1) This basically proves that a statement will keep being true for n > k once it is true for some n=k A base case - proving the statement is definitely true for some k, thus starting off the induction chain

Practice - Proving it Slowly Together Prove 5 n -1 is divisible by 4 for n 1. First step: Base case. What does that look like?

Practice - Proving it Slowly Together Prove 5 n -1 is divisible by 4 for n 1. First step: Base case, for n=1. 5 1-1 = 4, which is clearly divisible by 4. Next: Inductive step. What are we supposing (in terms of k), and what will we try to show (also in terms of k)?

Practice - Proving it Slowly Together Prove 5 n -1 is divisible by 4 for n 1. First step: Base case, for n=1. 5 1-1 = 4, which is clearly divisible by 4. Next: Inductive step. Suppose 5 k -1 is divisible by 4. We will show 5 k+1-1 must be divisible by 4. By the inductive hypothesis, 5 k -1 = 4a for some integer a. How can we start turning this into a statement about 5 k+1-1? (This takes a few steps)

Practice - Proving it Slowly Together Prove 5 n -1 is divisible by 4 for n 1. First step: Base case, for n=1. 5 1-1 = 4, which is clearly divisible by 4. Next: Inductive step. Suppose 5 k -1 is divisible by 4. We will show 5 k+1-1 must be divisible by 4. By the inductive hypothesis, 5 k -1 = 4a for some integer a. Then 5 k = 4a+1 and 5 k+1 = 5(4a+1) = 20a+5, so 5 k+1-1 = 20a+4. All we need is to complete the inductive step. How can that happen?

Practice - Proving it Slowly Together Prove 5 n -1 is divisible by 4 for n 1. First step: Base case, for n=1. 5 1-1 = 4, which is clearly divisible by 4. Next: Inductive step. Suppose 5 k -1 is divisible by 4. We will show 5 k+1-1 must be divisible by 4. By the inductive hypothesis, 5 k -1 = 4a for some integer a. Then 5 k = 4a+1 and 5 k+1 = 5(4a+1) = 20a+5, so 5 k+1-1 = 20a+4. Since 20a + 4 is divisible by 4, 5 k+1-1 must be divisible by 4. Thus, if 5 k -1 is divisible by 4, 5 k+1-1 is divisible by 4.

Recall That This is Enough To Show the Argument Works for n 1 We ve given a general argument that if the statement 5 n -1 is divisible by 4" is true for n=k, it s true for n=k+1. If somebody didn t believe us about, say, n=3, they could just apply the inductive argument a few times from the base case, using specific values for k. If true for 1, true for 2: 5 1-1 = 4a, so 5 1 = 4a+1 and thus 5 2 = 5(4a+1) =20a+5 and 5 2-1 = 20a+4 so divisible by 4 If true for 2, true for 3: 5 2-1 = 4b, so 5 2 = 4b+1 and thus 5 3 = 5(4b+1) =20b+5 and 5 3-1 = 20b+4 so divisible by 4 And so on, ad infinitum. We don t need to explain this logic in a proof by induction - it s assumed.

The General Justification for Strong Induction Imagine actually applying the inductive step repeatedly to prove all values 1 < n < k. If true for 1, true for 2. If true for 2, true for 3. You haven t just proven the statement for one value. You ve also proven the statement for every value of n along the way. That means we can make arguments that rely on the truth of statements further back than the previous statement. 1 2 3 4 k-1 k

Currency Example A game issues in-game coins in $4 and $5 denominations. Show that this can hit any integer value $12. The slow way of proving every statement would be to give a way to make every value, one at a time. This proof would never finish, but can help us get a sense of the problem: Proof for $12: 4 + 4 + 4 = 12. Proof for $13: 4 + 4 + 5 = 13. Proof for $14: 4 + 5 + 5 = 14. Proof for $15: 5 + 5 + 5 = 15. Proof for $16: 4 + 4 + 4 + 4 = 16. Proof for $17: 4 + 4 + 4 + 5 = 17.

Currency Example A game issues in-game coins in $4 and $5 denominations. Show that this can hit any integer value $12. We can realize that the cycle is starting over again : the patterns for 16, 17, 18 are the patterns for 12, 13, 14 just with a $4 coin attached. Proof for $12: 4 + 4 + 4 = 12. Proof for $13: 4 + 4 + 5 = 13. Proof for $14: 4 + 5 + 5 = 14. Proof for $15: 5 + 5 + 5 = 15. Proof for $16: 4 + (4 + 4 + 4) = 16. Proof for $17: 4 + (4 + 4 + 5) = 17. Proof for $18: 4 + (4 + 5 + 5) = 18.

Currency Example A game issues in-game coins in $4 and $5 denominations. Show that this can hit any integer value $12. It seems like we have a general way to produce more values - take a combination we ve already got, and add a $4 coin. This is the basis of our proof by strong induction. Proof for $12: 4 + 4 + 4 = 12. Proof for $13: 4 + 4 + 5 = 13. Proof for $14: 4 + 5 + 5 = 14. Proof for $15: 5 + 5 + 5 = 15. Proof for $16: 4 + (4 + 4 + 4) = 16. Proof for $17: 4 + (4 + 4 + 5) = 17. Proof for $18: 4 + (4 + 5 + 5) = 18. +4 +4 +4

Currency Example: Inductive Step A game issues in-game coins in $4 and $5 denominations. Show that this can hit any integer value $12. Inductive step: Suppose the statement is true for 12 n < k, where k >= 16. We can argue that the statement must be true for k, because k-4 is in the range that we assume is achievable, and we can just add a $4 coin to that. Notice that this argument doesn t prove anything about n=12, 13, 14, or 15. Just use the value $4 less doesn t work for these. So these must be base cases. (The breakdowns given earlier, such as 4+4+4, prove them.) 12 13 14 15 16 17 k-4 k-3 k-2 k-1

Currency Example: Full Proof By Strong Induction A game issues in-game coins in $4 and $5 denominations. Show that this can hit any integer value $12. Base cases: 12=4+4+4, 13=4+4+5, 14=4+5+5, 15=5+5+5 Inductive step: Suppose the statement is true for 12 n < k, where k 16. We can argue that the statement must be true for k, because k-4 is in the range that we assume is achievable, and we can just add a $4 coin to that. 12 13 14 15 16 17 k-4 k-3 k-2 k-1

Fundamental Theorem of Arithmetic An example of needing to know the statement is true somewhere in the middle between 1 and k. To prove: Every integer 2 is the product of one or more primes (i.e. has a prime factorization ). Base case: 2 is the product of one prime - itself. Inductive step: Suppose the statement is true for all n, 2 n < k. Then k is either a prime - in which case the statement is clearly true, as it was for 2 - or not. If it is not, then there exist 2 numbers a,b where 1 < a,b < k and ab=k. By the inductive hypothesis, both a and b can be written as the products of primes. Then k could be written as the product of those factorizations. 2 a b k-1 k

Fundamental Theorem of Arithmetic * An example of needing to know the statement is true somewhere in the middle between 1 and k. To prove: Every integer 2 is the product of one or more primes (i.e. has a prime factorization ). Base case: 2 is the product of one prime - itself. Inductive step: Suppose the statement is true for all n, 2 n < k. Then k is either a prime - in which case the statement is clearly true, as it was for 2 - or not. If it is not, then there exist 2 numbers a,b where 1 < a,b < k and ab=k. By the inductive hypothesis, both a and b can be written as the products of primes. Then k could be written as the product of those factorizations. 2 a b k-1 k *Technically the fundamental theorem also guarantees uniqueness of the factorization, which we won t prove here.

n < k versus n k The inductive hypothesis for a strong induction is generally some form of Suppose the statement is true for all values of n up to But then, should we hypothesize it s true for n < k and prove for k, or draw the limit at n k and prove for n=k+1? This doesn t really matter; pick one. Even proofs by weak induction can be written either way. In any case, you re just proving the next case is true, however you number or name the cases I try to choose whichever way seems clearest for that particular proof - in particular, use what makes the inductive hypothesis easiest to say (hence often k to k+1 in weak induction)

Proving a Counting Pattern How many strings of n bits have no consecutive 1 bits? We can list some examples for different n; some will become base cases. n=1: 0 and 1 both work: 2 n=2: 00, 01, 10 [not 11]: 3 n=3: 000, 001, 010, [not 011], 100, 101, [not 11X]: 5 n=4: 0000,0001,0010,0100,0101,1000,1001,1010: 8

Proving a Counting Pattern How many strings of n bits have no consecutive 1 bits? There s a similarity here to the Fibonacci numbers - 0, 1, 1, 2, 3, 5, 8, Can we prove it? Why would each number be the sum of the two before? n=1: 0 and 1 both work: 2 n=2: 00, 01, 10 [not 11]: 3 n=3: 000, 001, 010, [not 011], 100, 101, [not 11X]: 5 n=4: 0000,0001,0010,0100,0101,1000,1001,1010: 8

Proving a Counting Pattern How many strings of n bits have no consecutive 1 bits? There s a similarity here to the Fibonacci numbers - 0, 1, 1, 2, 3, 5, 8, Can we prove it? Why would each number be the sum of the two before? n=1: 0 and 1 both work: 2 n=2: 00, 01, 10, but not 11: 3 n=3: 000, 001, 010, skip 011, 100, 101, skip11x: 5 n=4: 0000,0001,0010,0100,0101,1000,1001,1010: 8 Start with 0: all patterns of n-1 legal Start with a 1: need a 0, then n-2 left

Proving a Counting Pattern Prove: there are Fn+2 bitstrings of length n with no consecutive ones. Inductive step: Suppose the statement is true for 1 <= n < k. Then there are Fk+1 bitstrings of length k-1 and Fk bitstrings of length k-2. 0 k-1 bits + 10 k-2 bits Sketch of argument to come

Proving a Counting Pattern Prove: there are F n+2 bitstrings of length n with no consecutive ones. Inductive step: Suppose the statement is true for 1 n < k. Then there are F k+1 bitstrings of length k-1 and F k bitstrings of length k-2. The number of bitstrings of length k is equal to the number of bitstrings that start with 0 which must be equal to F k+1 by the inductive hypothesis, since any valid noconsecutive-1 s string of k-1 bits can follow plus the number of bitstrings that start with 1. To avoid consecutive 1 s, a 1 must be followed by a 0, so just the valid bitstrings of length k-2 are possible in the starts-with-1 case, and there are F k of them. 0 k-1 bits + 10 k-2 bits

Proving a Counting Pattern Prove: there are F n+2 bitstrings of length n with no consecutive ones. Base cases: n=1: F 3 = 2, and there are two 1-bit strings. n=2: F 4 = 3, and there are 3 2-bit strings besides 11. Inductive step: Suppose the statement is true for 1 n < k. Then there are F k+1 bitstrings of length k-1 and F k bitstrings of length k-2. The number of bitstrings of length k is equal to the number of bitstrings that start with 0 which must be equal to F k+1 by the inductive hypothesis, since any valid no-consecutive-1 s string of k-1 bits can follow plus the number of bitstrings that start with 1. To avoid consecutive 1 s, a 1 must be followed by a 0, so just the valid bitstrings of length k-2 are possible in the starts-with-1 case, and there are F k of them. Thus the total number of bitstrings of length k with no consecutive 1 s is F k + F k+1 = F k+2, which is what we wanted to prove about k-bit strings.

Proving Exponential Growth of Fibonacci Numbers There are some programming patterns that will result in running times that grow like the Fibonacci numbers with the size of the input. Is that bad? In fact, the Fibonacci numbers grow exponentially, so those running times are bad. Let r be the golden ratio We can notice r 2 = (1 + 2 5 + 5)/4 = (3 + 5)/2 = 1 + r, which we ll use later We ll prove F n r n-2 for n 2, and thus the Fibonacci numbers grow at least as fast as exponentially

Proving Exponential Growth of Fibonacci Numbers To prove: F n r n-2 for n 2. (F 0 = 0, F 1 = 1, F i = F i-1 + F i-2 ) Let s guess that since this is about Fibonacci numbers, the inductive step will need the previous two statements to be true Then, we need 2 base cases n=2: F 2 = 0, 1, 1 and r 2-2 = r 0 = 1. F 2 r 0. Check. n=3: F 3 = 1+1 = 2. r 3-2 = r = about 1.6. F 3 r 1. Check.

Proving Exponential Growth Of the Fibonacci Numbers To prove: Fn r n-2 for n 2. (F0 = 0, F1 = 1, Fi = Fi-1 + Fi-2) Base cases: n=2: F2 = 0, 1, 1 and r 2-2 = r 0 = 1. F2 r 0. Check. n=3: F3 = 1+1 = 2. r 3-2 = r = about 1.6. F3 r 1. Check. Inductive step. Suppose the statement is true for 2 n < k. Then Fk-1 r k-3 and Fk-2 r k-4. Since Fk = Fk-1 + Fk-2, Fk r k-3 + r k-4. When in doubt, try some algebra

Proving Exponential Growth Of the Fibonacci Numbers To prove: F n r n-2 for n 2. (F 0 = 0, F 1 = 1, F i = F i-1 + F i-2 ) Base cases: n=2: F 2 = 0, 1, 1 and r 2-2 = r 0 = 1. F 2 r 0. Check. n=3: F 3 = 1+1 = 2. r 3-2 = r = about 1.6. F 3 r 1. Check. Inductive step. Suppose the statement is true for 2 n < k. Then F k-1 r k-3 and F k-2 r k-4. Since F k = F k-1 + F k-2, F k r k-3 + r k-4 = r k-4 (r+1). Using the equation we mentioned earlier, 1+r = r 2, so r k-4 (r+1) = r k-2, which is the value we needed to compare to F k.

Approaching Strong Induction Problems For weak induction, you know exactly what to prove for the base case, and exactly where to start in the inductive step. For strong induction, it s not as clear - the main thing you know is what you want to prove (that the statement works for n=k). This suggests working backwards. Try to make the case for n=k, and figure out along the way how to make use of the inductive hypothesis. You will know which base cases you need only after you figure out your inductive step.

Practice Strong Induction: More Fib Facts Reminder: F0 = 0, F1 = 1, F2 = 1, Prove by strong induction: A Fibonacci number is even iff its index is divisible by 3. (So the pattern is even, odd, odd, even, odd, odd, ) You ll probably need to argue some different cases for k.

Practice Strong Induction: More Fib Facts Reminder: F 0 = 0, F 1 = 1, F 2 = 1, Prove by strong induction: A Fibonacci number is even iff its index is divisible by 3. Base cases: Illustrated above; F 0 is even, and the others are odd. Inductive step: Suppose the pattern holds up to n < k. Case 1: k divisible by 3. It is the sum of the previous two numbers, which by the inductive hypothesis are odd. The sum of odd numbers is even, which was to be proved. Case 2: k not divisible by 3. Then either k-1 is even and k-2 is odd, or vice versa. In either case, an even plus an odd is odd.

A Strong Induction on Trees We ve proven E = V -1 for trees already using weak induction, but it s interesting to see a contrast with a proof by strong induction. The weak argument required a clear way to get a tree exactly one vertex smaller. But with strong induction, we can make arguments about arbitrarily large or small pieces.

Strong Induction to Prove E = V -1 Induction on the number of vertices. Base case: V = 1. No edges; E = 0. Check. Inductive step: Suppose the formula holds for all smaller trees than k. We want to show a k-vertex tree must have V -1 edges. We re going to start by removing an arbitrary edge from our tree. We claim that the two subgraphs that remain must be connected and have no cycles, because these were true of the original tree, so these must be smaller trees. Subtree 1 Subtree 2

Strong Induction to Prove Induction on the number of vertices. E = V -1 Base case: V = 1. No edges; E = 0. Check. Inductive step: Suppose the formula holds for all smaller trees than k. We want to show a k-vertex tree must have V -1 edges. We re going to start by removing an arbitrary edge from our tree. We claim that the two subgraphs that remain must be connected and have no cycles, because these were true of the original tree, so these must be smaller trees. By the inductive hypothesis, the formula works for each of them. If they have a and b vertices, respectively, they have a-1 and b-1 edges. Subtree 1 Subtree 2

Proving All Degrees Even Implies Euler Cycle Early on in the semester we proved that if you didn t have even degree for all vertices, then you couldn t have an Euler cycle - a path that hit all edges once and ends where it started. That s not the same as claiming that if you do have even degree for all vertices, then you do have an Euler cycle. (These statements are the converse of each other) We didn t prove this because the proof required strong induction.

Proving All Degrees Even Implies Euler Cycle Claim: All connected graphs with at least 3 vertices have an Euler cycle if the degree of each vertex is even. Proof: By induction on the number of vertices. Base case: n=3. If the degree of each vertex is even, either the graph isn t connected or it s a triangle, and definitely has a cycle. Inductive step: Assume true for n < k. Now follow a path on the graph until you hit a vertex already in your path (continuing is always possible because of even degree for all vertices). Mark the cycle you found. The rest of the graph remains of even degree. By the inductive hypothesis, every connected component that remains besides your cycle has some Euler cycle.

Proving All Degrees Even Implies Euler Cycle Claim: All connected graphs with at least 3 vertices have an Euler cycle if the degree of each vertex is even. Proof: By induction on the number of vertices. begin detour! Base case: n=3. If the degree of each vertex is even, either the graph isn t connected or it s a triangle, and definitely has a cycle. resume Inductive step: Assume true for n < k. Now follow a path on the graph until you hit a vertex already in your path (continuing is always possible because of even degree for all vertices). Mark the cycle you found. The rest of the graph remains of even degree. By the inductive hypothesis, every connected component that remains besides your cycle has some Euler cycle. (Attached components must have at least 3 vertices - see diagram at right.) So the final Euler cycle consists of your arbitrary cycle, plus detours to complete every smaller Euler cycle attached to it. Even degree means at least 3 vertices in attached component

Another Example of the Euler Cycle Inductive Step

Another Example of the Euler Cycle Inductive Step Inductive hypothesis applies to both sides

Another Example of the Euler Cycle Inductive Step 2: inside start 3, 5: main cycle 1: outside 4: similar detour to 1,2 Inductive hypothesis applies to both sides

When Do I Need Strong Induction? Most proofs by induction in this course will be weak induction. It s a pretty good assumption for many algorithms. But there may come a time when you need the statement to be true for multiple previous instances, or for further back than one step, or something else. Especially if you re dealing with a recurrence (like the Fibonacci numbers). Your own argument dictates whether you need a strong inductive hypothesis. You can then just make the strong assumption instead, and use the cases that you need.

Induction Flowchart Did the instructions say prove this by induction? If not, consider proving it a different way. Did the instructions say prove using strong induction, or otherwise suggest strong induction (like with Fibonacci numbers)? If not, try weak induction first. Try proving that if the statement works for k, then it works for k+1. Need to assume that it works for more cases to make your argument? Switch to strong induction. If the statement seems algebraic, use algebra. If it s about graphs or some other structure, figure out how the truth of the smaller case helps you. Show the statement works for all the base cases - the cases your inductive step can t possibly work for (the first case if it s weak induction).

Time-Permitting-Exercise: Fish Fish Fish Sometimes in CS we think about parsing things - figuring out whether strings are meaningful or not. Fish can refer either to an animal (noun), or to the act of trying to catch that animal (verb). Thus, Fish fish fish is a valid English sentence that states that there are some cannibalistic fish out there. (noun-verb-noun) I can also make a noun phrase fish fish fish that is talking about the fish who are being hauled up by other fish (noun-noun-verb). A similar construction is mice cats chase for the mice that cats chase. Thus I can say fish fish fish fish meaning the fish who are being fished for by fish, actually go fishing themselves. Assuming it is always legal to replace a noun with a noun phrase, prove that n repetitions of the word fish is always a valid English sentence.

n Fish Solution Base cases: n=1. Fish! Parsed as a command. n=2. Fish fish. Noun-verb. n=3. Fish fish fish. Noun-verb-noun. Inductive step: Suppose the sentence is valid for 1 n < k. A sentence with k fish can be parsed the same way as a sentence with k-2 fish, but one of the nouns fish can be replaced with the noun phrase fish fish fish, which we said was always a legal transformation. Thus, if there is a valid parse of k-2 fish, there is a valid parse of k of them. Fish Fish Fish Fish Fish Fish Fish Fish N V N V N NP N N V

Induction/Strong Induction Summary Induction generally is a proof technique for arguing that something starts out working (base cases) and keeps working (inductive step) Both strong and weak induction work by reasoning from the fact that the statement has been true so far Weak just argues that each statement follows from the previous one Strong assumes the statement has been true for everything up to now (although it may only make use of particular cases) Strong induction is more common when dealing with complex entities like graphs that don t have an obvious previous step, or when reasoning about values that depend on multiple previous values (Fibonacci) Other proof strategies are often more intuitive, and therefore preferred