Computational Models #1 Handout Mode Nachum Dershowitz & Yishay Mansour March 13-15, 2017 Nachum Dershowitz & Yishay Mansour Computational Models #1 March 13-15, 2017 1 / 41
Lecture Outline I Motivation I Languages, words and alphabets I Finite automata and regular languages I Regular operations I Sipser, Chapter 1 Nachum Dershowitz & Yishay Mansour Computational Models #1 March 13-15, 2017 2 / 41
Time Magazine Put the right kind of software into a computer, and it will do whatever you want it to. There may be limits on what you can do with the machines themselves, but there are no limits on what you can do with software.
Chess Games: 10 10 50 Positions: 10 40 Positions: 20, 400, 5362, 71852, 809896(?), 9132484(?),...
states initial terminal
Part I Languages, words and alphabets Nachum Dershowitz & Yishay Mansour Computational Models #1 March 13-15, 2017 3 / 41
Languages, words and alphabets Definition 1 An alphabet is a finite set of letters. I ={a, b, c,...,z} the English alphabet. I ={,,,..., } the Greek alphabet. I ={0, 1} the binary alphabet. I ={0, 1,...,9} the digital alphabet. Definition 2 A word (i.e., string) over, is a finite sequence of letters from. The collection of all strings over is denoted by. For {0, 1}, the binary alphabet, ", 1, 0, 000000000, 1111111000 are all members of. Definition 3 A language over is a (possibly infinite) subset of. Nachum Dershowitz & Yishay Mansour Computational Models #1 March 13-15, 2017 4 / 41
Language Examples I Modern English. I Ancient Greek. I All prime numbers, written using digits. I A = {w 2{0, 1} : w has at most seventeen 0 s}. I B = {0 n 1 n : n 0}. I C = {w 2{0, 1} : w has an equal number of 0 s and 1 s}. Make sure you understand what the above notions stand for... Nachum Dershowitz & Yishay Mansour Computational Models #1 March 13-15, 2017 5 / 41
Part II Finite Automata Nachum Dershowitz & Yishay Mansour Computational Models #1 March 13-15, 2017 6 / 41
Example: A One-Way Automatic Door front pad rear pad door I open when person approaches I hold open until person clears I don t open when someone standing behind door Nachum Dershowitz & Yishay Mansour Computational Models #1 March 13-15, 2017 7 / 41
The Automatic Door as DFA REAR BOTH NEITHER closed FRONT open FRONT REAR BOTH NEITHER I States: I OPEN I CLOSED I Sensor: I FRONT: someone on front pad I REAR: someone on rear pad I BOTH: someone(s) on both pads I NEITHER no one on either pad. Nachum Dershowitz & Yishay Mansour Computational Models #1 March 13-15, 2017 8 / 41
The Automatic Door as DFA A DFA is Deterministic Finite Automata REAR BOTH NEITHER closed FRONT open FRONT REAR BOTH NEITHER neither front rear both closed closed open closed closed open closed open open open Nachum Dershowitz & Yishay Mansour Computational Models #1 March 13-15, 2017 9 / 41
DFA: Informal definition The machine M 1 : 0 1 1 0 q 1 q q 2 3 I States: q 1, q 2, and q 3. I Start state: q 1 (arrow from outside ). I Accept state: q 2 (double circle). I State transitions: arrows tagged with letters. 0,1 Nachum Dershowitz & Yishay Mansour Computational Models #1 March 13-15, 2017 10 / 41
DFA: Informal definition (cont.) 0 1 1 0 q 1 q q 2 3 0,1 I On an input string I DFA begins in start state q1 I after reading each symbol, DFA makes state transition with matching label. I After reading last symbol, DFA produces" output: I accept if DFA is an accepting state. I reject otherwise. Nachum Dershowitz & Yishay Mansour Computational Models #1 March 13-15, 2017 11 / 41
DFA: Informal definition (cont..) 0 1 1 0 q 1 q q 2 3 0,1 What happens on the following input strings: I 1101 I 0010 I 01100 I In general?! Nachum Dershowitz & Yishay Mansour Computational Models #1 March 13-15, 2017 12 / 41
DFA: Informal definition (cont...) 0 1 1 0 q 1 q q 2 3 This DFA accepts I All input strings that end with a 1 I All input strings that contain at least one 1, and end with an even number of 0 s I No other strings Proof:? 0,1 Nachum Dershowitz & Yishay Mansour Computational Models #1 March 13-15, 2017 13 / 41
DFA - Formal Definition Definition 4 A deterministic finite automaton (DFA) is a 5-tuple (Q,,,q 0, F), where I Q is a finite set called the states I is a finite set called the alphabet I : Q! Q is the transition function I q 0 2 Q is the start state I F Q is the set of accept states Nachum Dershowitz & Yishay Mansour Computational Models #1 March 13-15, 2017 14 / 41
Back to M 1 0 1 1 0 q 1 q q 2 3 0,1 M 1 =(Q,,,q 1, F) where I Q = {q 1, q 2, q 3 }, ={0, 1}, I the transition function I q 1 is the start state I F = {q 2 }. is 0 1 q 1 q 1 q 2 q 2 q 3 q 2 q 3 q 2 q 2 Nachum Dershowitz & Yishay Mansour Computational Models #1 March 13-15, 2017 15 / 41
Another Example a s b a q 1 r 1 b b a a b b q 2 r 2 a Nachum Dershowitz & Yishay Mansour Computational Models #1 March 13-15, 2017 16 / 41
Formal Model of Computation Definition 5 M =(Q,,,q 0, F) accepts w 2 if b M(q 0, w) 2 F. Definition 6 ( b ) b M : Q 7! Q is defined by b M (q, w) = ( b (q, w1,...,n 1 ), w n ), n = w 1 q, w = ".. I w 1,...,k stands for the word w 1,...,w k the k-letter prefix of w (w 1,...,0 = w ; = ") I Note that b M(q, )= (q, ) for 2. I We write b when M is clear from the context. Nachum Dershowitz & Yishay Mansour Computational Models #1 March 13-15, 2017 17 / 41
An equivalent definition Definition 7 (Equivalent definition) M =(Q,,,q 0, F) accepts w = w 1 w 2...w n, if 9r 0,...,r n 2 Q s.t., I r 0 = q 0. I (r i, w i+1 )=r i+1, for all 0 apple i < n. I r n 2 F. Nachum Dershowitz & Yishay Mansour Computational Models #1 March 13-15, 2017 18 / 41
The language of a DFA Definition 8 L(M), the language of a DFA M, is the set of strings that M accepts. We sometime say that M accepts the language L(M). I M may accept many strings I M accepts only one language. What language does M accept if it accepts no strings? Definition 9 A language is called regular, if some deterministic finite automaton accepts it. Nachum Dershowitz & Yishay Mansour Computational Models #1 March 13-15, 2017 19 / 41
Example: DFA M 2 I Q = {q 1, q 2 }, ={0, 1}, F = {q 2 }, =? I What is L(M 2 ):={w 2{0, 1} : b (q 1, w) =q 2 }? Theorem 10 L(M 2 )={w 2{0, 1} :# 1 (w) is odd}. w. I Proof by induction on the word length # 1 (w) number of ones in I What does it mean? The j th assumption: w 2{0, 1} j is in L(M) iff # 1 (w) is odd Nachum Dershowitz & Yishay Mansour Computational Models #1 March 13-15, 2017 20 / 41
Proving L(M 2 )=L 0 := {w 2 :# 1 (w) is odd} I Basis (length 0): " /2 L 0 and b (q 1, )=q 1 =) /2 L(M 2 ). I Step: assume hypothesis holds for words of length j 0. I Let x = y 2{0, 1} j+1 for 2{0, 1} (hence, y = j). I If # 1 (y) is even. By assumption b (q 1, y) =q 1. I = 1 =) 1. # 1 (x) is odd =) x 2L 0 2. b (q1, x) := ( b (q 1, y), )= (q 1, 1) =q 2 =) x 2L(M 2 ). I = 0 =) 1. # 1 (x) is even =) x /2 L 0 2. b (q1, x) := ( b (q 1, y), )= (q 1, 0) =q 1 =) x /2 L(M 2 ). I If # 1 (y) is odd... I We proved for x 2{0, 1} j+1 : x 2L(M 2 ) () x 2L 0 Nachum Dershowitz & Yishay Mansour Computational Models #1 March 13-15, 2017 21 / 41
The language of M 1 0 1 1 0 q 1 q q 2 3 Theorem 11 L(M 1 )={w10 2k : k 0, w 2{0, 1} } 0,1 Proof: Claim 12 (implies the theorem) Let L 0 i = {x 2{0, 1} : b (q 1, x) =q i } and let I L 1 = {0 k : k 0} I L 2 = {w10 2k : k 0, w 2{0, 1} } I L 3 = {w10 2k+1 : k 0, w 2{0, 1} } Then, L 0 i = L i for every i 2{1, 2, 3} Nachum Dershowitz & Yishay Mansour Computational Models #1 March 13-15, 2017 22 / 41
Proving Claim?? We need to prove that 8i 2{1, 2, 3}: x 2L i () x 2L 0 i. Recall that x 2L 0 i () b (x) =q i I Proof by induction on word length. I Induction basis: Easy to see that hypothesis holds for. I Induction step: Assume hypothesis holds for words of length j 0. I Let x = y 2{0, 1} j+1 for 2{0, 1}. I We prove the hypothesis for x, separately for each i 2{1, 2, 3} Nachum Dershowitz & Yishay Mansour Computational Models #1 March 13-15, 2017 23 / 41
L 1 = {0 k : k 0} Recall x = y 2{0, 1} j+1 for 2{0, 1}. Proving: x 2L 1 =) b (q 1, x) =q 1 ( =) x 2L 0 1 ). I x = 0 j+1, y = 0 j and = 0. I Since y 2L 1, by i.h. b (q 1, y) =q 1 I Therefore, b (q 1, x) = ( b (q, y), )= (q 1, 0) =q 1. Proving: b (q 1, x) =q 1 =) x 2L 1. I Let q y = b (q 1, y) (hence, b (q 1, x) = (q y, )=q 1 ) I q y = q 1 and = 0. (?) I By i.h. y = 0 j. I Hence, x = y = 0 j 0 = 0 j+1 2L 1. Nachum Dershowitz & Yishay Mansour Computational Models #1 March 13-15, 2017 24 / 41
L 2 = {w10 2k : k 0, w 2{0, 1} } Recall x = y 2{0, 1} j+1 for 2{0, 1}. Proving x 2L 2 =) b (q 1, x) =q 2. I Assume = 1 I Since (q i, 1) =q 2 for any i =) b (q 1, x) =q 2. I Assume x = w10 2k for k > 0 ( =) y = w10 2k 1 and = 0) I Hence, y 2L3. I By i.h. b (q1, y) =q 3 I Thus, b (q1, x) = (q 3, 0) =q 2. Proving b (q 1, x) =q 2 =) x 2L 2. I Assume = 1 =) x 2L 2.(?) I Assume = 0 I q y := b (q 1, y) =q 3 I By i.h. y = w10 2k+1 for some k 0 I Therefore x = y = w10 2k+1 0 2L 2. Nachum Dershowitz & Yishay Mansour Computational Models #1 March 13-15, 2017 25 / 41
L 3 = {w10 2k+1 : k 0, w 2{0, 1} } Recall x = y 2{0, 1} j+1 for 2{0, 1}. Proving x 2L 3 =) b (q 1, x) =q 3. I x = w10 2k+1, y = w10 2k and = 0 I y 2L 2 I By i.h. b (q 1, y) =q 2. I Therefore, b (q 1, x) = (q 2, 0) =q 3. Proving b (q 1, x) =q 3 =) x 2L 3. I Let q y = b (q 1, y) I Hence, q y = q 2 and = 0 (?) I By i.h. y = w10 2k I Therefore, x = y = w10 2k 0 2L 3. Nachum Dershowitz & Yishay Mansour Computational Models #1 March 13-15, 2017 26 / 41
Part III Regular Operations Nachum Dershowitz & Yishay Mansour Computational Models #1 March 13-15, 2017 27 / 41
Additional examples of regular languages Let ={0, 1}. I {w 2{0, 1} :# 1 (w) 0 mod 7}. I Sequence of 0 followed by sequence of 1, i.e., {0 m 1 n : m, n 0}. I Any finite language. All the above languages are regular Is there a simple proof? Nachum Dershowitz & Yishay Mansour Computational Models #1 March 13-15, 2017 28 / 41
The regular operations Let A and B be languages. The union operation: A[B = {x : x 2A_x 2B} The concatenation operation: AkB = {xy : x 2A^y 2B} The star operation: A = {x 1 x 2...x k : k 0 and each x i 2A} Nachum Dershowitz & Yishay Mansour Computational Models #1 March 13-15, 2017 29 / 41
The regular operations Examples Let A= {good, bad} and B = {boy, girl}. Union Concatenation A[B= {good, bad, boy, girl} AkB = {goodboy, goodgirl, badboy, badgirl} Star A = {", good, bad, goodgood, goodbad, badbad, badgood,...} Nachum Dershowitz & Yishay Mansour Computational Models #1 March 13-15, 2017 30 / 41
Closure under union Theorem 13 If L 1 and L 2 are regular languages, then so is L 1 [L 2. Approach to Proof: I Some DFA M 1 accepts L 1 I Some DFA M 2 accepts L 2 I Construct DFA M that accepts L 1 [L 2. Attempted Proof Idea: I first emulate M 1, and I if M 1 doesn t accept, then emulate M 2. What s wrong with this? Fix: Emulate both machines simultaneously. Nachum Dershowitz & Yishay Mansour Computational Models #1 March 13-15, 2017 31 / 41
Closure Under Union: Correct Proof Suppose I M 1 =(Q 1,, 1, q 1, F 1 ) accepts L 1, I M 2 =(Q 2,, 2, q 2, F 2 ) accepts L 2. Define M as follows (M will accept L 1 [L 2 ): I Q = Q 1 Q 2. I is the same. I For each (r 1, r 2 ) 2 Q and a 2, ((r 1, r 2 ), a) =( 1 (r 1, a), 2(r 2, a)) I q 0 =(q 1, q 2 ) I F = {(r 1, r 2 ): r 1 2 F 1 or r 2 2 F 2 }. I Formal proof (next slide) (hey, why not choose F = F 1 F 2?) Nachum Dershowitz & Yishay Mansour Computational Models #1 March 13-15, 2017 32 / 41
Correctness of the construction Claim 14 L(M) =L(M 1 ) [L(M 2 ). Follows by the next claim. Claim 15 b M ((q 1, q 2 ), x) =( b 1(q 1, x), b 2(q 2, x)). Proof: By induction on word length. DIY... Proving Claim??: I x 2L(M 1 ) =) b 1(q 1, x) =r 1 2 F 1. (similar if x 2L(M 2 ).) I By Claim??, b M((q 1, q 2 ), x) =(r 1, ) 2 F =) x 2L(M). I x 2L(M) =) b M((q 1, q 2 ), x) =(r 1, r 2 ) 2 F. I By Claim??, (r 1, r 2 )=( b 1(q 1, x), b 2(q 2, x)). I Hence, either r 1 2 F 1 or r 2 2 F 2 =) x 2L(M 1 ) [L(M 2 ). Nachum Dershowitz & Yishay Mansour Computational Models #1 March 13-15, 2017 33 / 41
What about concatenation? Theorem 16 If L 1, L 2 are regular languages, then so is L 1 kl 2. Example: L 1 = {good, bad} and L 2 = {boy, girl}. L 1 kl 2 = {goodboy, goodgirl, badboy, badgirl} This is much harder to prove. Idea: Simulate M 1 for a while, then switch to M 2. Problem: But when do you switch? This leads us into non-determinism, wait for next class... Nachum Dershowitz & Yishay Mansour Computational Models #1 March 13-15, 2017 34 / 41
Part IV Non-deterministic Finite Automata Nachum Dershowitz & Yishay Mansour Computational Models #1 March 13-15, 2017 35 / 41
NFA non-deterministic Finite Automata 0,1 0,1 0 0 q q q 1 2 3 1 q 4 I May have more than one transition labeled with the same symbol, I May have no transitions labeled with a certain symbol, I May have transitions labeled with ", the symbol of the empty string. Will deal with this latter Every DFA is also an NFA. Nachum Dershowitz & Yishay Mansour Computational Models #1 March 13-15, 2017 36 / 41
Non-deterministic computation 0,1 0,1 0 0 q q q 1 2 3 1 q 4 What happens when more than one transition is possible? I The machine splits into multiple copies I Each branch follows one possibility I Together, branches follow all possibilities. I If the input doesn t appear, that branch dies. I Automaton accepts if some branch accepts. Nachum Dershowitz & Yishay Mansour Computational Models #1 March 13-15, 2017 37 / 41
Computation on 1001 0,1 0,1 0 0 q q 1 q 2 3 1 q 4 symbol 1 q 1 0 q 1 0 q 1 q 2 1 q q q 1 2 3 q 1 q 4 Nachum Dershowitz & Yishay Mansour Computational Models #1 March 13-15, 2017 38 / 41
Why non-determinism? Theorem 17 (Informal, to be proved soon) Deterministic and non-deterministic finite automata, accept exactly the same set of languages. Q.: So why do we need NFA s? Design a finite automaton for the language L all binary strings with a 1 in their third-to-the-last position? Nachum Dershowitz & Yishay Mansour Computational Models #1 March 13-15, 2017 39 / 41
NFA for L 0,1 1 0,1 q q 1 q 2 3 0,1 q 4 I Guesses which symbol is third from the last, and I checks that indeed it is a 1. I If guess is premature, that branch dies, and no harm occurs. Nachum Dershowitz & Yishay Mansour Computational Models #1 March 13-15, 2017 40 / 41
DFA for L I Have 8 states, encoding the last three observed letters. I A state for each string in {0, 1} 3. I Add transitions on modifying the suffix, give the new letter. I Mark as accepting, the strings 1 0 0 q 000 0 0 q 100 q 010 q 110 1 1 1 0 0 0 q 001 q 101 1 q 011 1 q 111 1 1 DFA has few bugs... Nachum Dershowitz & Yishay Mansour Computational Models #1 March 13-15, 2017 41 / 41