Computational Models - Lecture 3 1

Transcription

1 Computational Models - Lecture 3 1 Handout Mode Iftach Haitner. Tel Aviv University. November 14, Based on frames by Benny Chor, Tel Aviv University, modifying frames by Maurice Herlihy, Brown University. Iftach Haitner (TAU) Computational Models, Lecture 3 November 14, / 43

2 Lecture highlight תן לי הכוח לשנות את מה שאני יכול לשנות, לקבל את מה שאינני יכול לשנות, ואת החוכמה להבדיל בין שניהם. Iftach Haitner (TAU) Computational Models, Lecture 3 November 14, / 43

3 Computational Models - Lecture 3 Non-regular languages: two approaches 1. Pumping Lemma 2. Myhill-Nerode Theorem (not in Sipser s book) Closure properties Algorithmic questions for NFAs Sipser, 1.4, 2.1, 2.2 Hopcroft and Ullman, 3.4 Iftach Haitner (TAU) Computational Models, Lecture 3 November 14, / 43

4 Proved last time Theorem 1 A language is described by a regular expression, iff it is regular. We have made a lot of progress understanding what finite automata can do, but what they cannot do? Iftach Haitner (TAU) Computational Models, Lecture 3 November 14, / 43

5 Negative results Is there a DFA that accepting the following languages (over {0, 1}). B = {0 n 1 n : n 0} C = {w : # 1 (w) = # 0 (w)} D = {w : # 01 (w) = # 10 (w)} # s (w) the number of times s appears in w. Consider B: DFA must remember how many 0 s it has seen Impossible with finite state. The others languages seem to be exactly the same... Question: Is this a proof? Answer: No, D is regular... Iftach Haitner (TAU) Computational Models, Lecture 3 November 14, / 43

6 Part I Pumping Lemma Iftach Haitner (TAU) Computational Models, Lecture 3 November 14, / 43

7 Regular languages can be pumped For every regular language L, there exists l > 0, the pumping length, s.t.: Every s L longer than l, can be pumped into a longer string in L. This is a powerful technique for showing that a language is not regular. Iftach Haitner (TAU) Computational Models, Lecture 3 November 14, / 43

8 The Pumping Lemma Lemma 2 For every regular language L, exists l > 0 (i.e., the pumping length) s.t.: every s L with s l, can be written as s = xyz with 1. xy i z L for every i 0, 2. y > 0, and 3. xy l. Remarks: Without the second condition, the theorem would be trivial. The third condition is technical and sometimes useful. Iftach Haitner (TAU) Computational Models, Lecture 3 November 14, / 43

9 Proving the Pumping Lemma Let M = (Q, Σ, δ, q 1, F) be a DFA accepting L, and let l = Q. Let s L be with s l, and consider the sequence of states M traverse as it reads s = s 1... s n : s 1 s 2 s 3 s 4 s 5 s 6... s n q 1 q 20 q 9 q 17 q 12 q 13 q 9 q 2 q 5 F By the pigeonhole principle, at least one of the states in the above sequence repeats. s = xyz Iftach Haitner (TAU) Computational Models, Lecture 3 November 14, / 43

10 Proving the Pumping Lemma, cont. s = xyz By inspection, M accepts xy k z for every k 0. y > 0, because the state q 9 is repeated. To ensure that xy l, pick first state repetition, which must occur no later than l + 1 states in sequence. Iftach Haitner (TAU) Computational Models, Lecture 3 November 14, / 43

11 Application # 1 Corollary 3 B = {0 n 1 n : n > 0} is not regular. Proof: By contradiction. Suppose B is regular and let l be its pumping length. Consider the string s = 0 l 1 l B. Let x, y, z be (one possible) strings guaranteed by the pumping lemma (i.e., s = xyz) 1. xy i z B for every i 0, 2. y > 0, and 3. xy l. If y is all 0, then xy 2 z has too many 0 s. If y is all 1, then xy 2 z has too many 1 s. If y is mixed, then xy 2 z is not of right form. We did not use the third property. Iftach Haitner (TAU) Computational Models, Lecture 3 November 14, / 43

12 Application # 2 Corollary 4 C = {w : # 1 (w) = # 0 (w)} is not regular. Proof: By contradiction. Suppose C is regular. Let l be the pumping length. Consider the string s = 0 l 1 l C. Let x, y, z be (possible) set of strings guaranteed by the pumping lemma (i.e., s = xyz) 1. xy i z B for every k 0, 2. y > 0, and 3. xy l. Since xy l, the string y is all 0 s. Thus, xy 2 z / C (more 0 s than 1 s). Could we have used s = (01) l? Iftach Haitner (TAU) Computational Models, Lecture 3 November 14, / 43

13 Application # 3 Corollary 5 E = {0 i 1 j : i > j} is not regular. Proof: By contradiction. Suppose E is regular. Let l be its pumping length. Consider the string s = 0 l 1 l 1 E. By pumping lemma, s = xyz, where xy k z E for every k 0, y > 0 and xy l. But xy 0 z = xz / E (at least as much 1 s as 0 s) Iftach Haitner (TAU) Computational Models, Lecture 3 November 14, / 43

14 Application # 4 Corollary 6 The language Primes 1 all strings whose length is a prime number is not regular. Proof: Suppose Primes is regular, and let l be its pumping length. Let s = 1 p Primes, where p l is a prime (?) By pumping lemma, s = xyz, where xy k z Primes for every k 0. Let y = m. Hence, xy p+1 z = 1 p+mp Primes but p(m + 1) is not prime... Iftach Haitner (TAU) Computational Models, Lecture 3 November 14, / 43

15 Another Example Consider the language L = {a i b n c n : n 0, i 1} {b n c m : n, m 0}. Any non-empty s L can be pumped: If s = a i b n c n with i > 0, then set x = ε and y = a. If s = b n c m with n > 0, then set x = ε and y = b. If s = c m with m > 0, then set x = ε and y = c. (in all cases z is set arbitrarily). Is L regular? No How can we prove it? Iftach Haitner (TAU) Computational Models, Lecture 3 November 14, / 43

16 Part II Characterization of Regular Languages Iftach Haitner (TAU) Computational Models, Lecture 3 November 14, / 43

17 The equivalence relation L Definition 7 For L Σ, define the equivalence relation L over words in Σ, by x L y if for every z Σ, it holds that xz L yz L. Easy to see that L is indeed an equivalence relation (reflexive, symmetric, transitive) on Σ. Hence, L partitions Σ into equivalence classes. For x Σ, let [x] Σ denote its equivalence class with respect to L How many equivalence classes does L induce? finite or infinite? Could be either (depends on L). Fact 8 (right invariance) If x L y, then xw L yw for every w Σ Proof: (xw)z L x(wz) L y(wz) L (yw)z L Iftach Haitner (TAU) Computational Models, Lecture 3 November 14, / 43

18 Three examples L 1 = {w : # 1 (w) mod 5 = 0} L 1 has finitely many equivalence classes. The equivalent classes are: [ε], [1], [11], [111], [1111]. Proof: Classes cover {0, 1} : for any x {0, 1} : x L 1 # 1(w) mod 5. xz L # 1 (xz) mod 5 = 0 (# 1 (x) mod 5) + # 1 (z) mod 5 = 0 1 # 1(x) mod 5 z L. Classes are disjoint: 1 i L 1 j for i j {0, 1, 2, 3, 4} L 2 = {0 n 1 n : n N} L 2 has infinitely many equivalence classes. [0] [0 2 ] [0 3 ]... L 3 = {a i b n c n : n 0, i 1} {b n c m : n, m 0} L 3 has infinitely many equivalence classes. [ab] [ab 2 ] [ab 3 ]... Iftach Haitner (TAU) Computational Models, Lecture 3 November 14, / 43

19 Myhill-Nerode Theorem Theorem 9 (Myhill-Nerode Theorem) L Σ is regular iff L finitely many equivalence classes. Hence L 1 = {w {0, 1} : # 1 (w) mod 4 = 0} is regular. L 2 = {0 n 1 n : n N} is not regular. L 3 = {a i b n c n : n 0, i 1} {b n c m : n, m 0} is not regular. Iftach Haitner (TAU) Computational Models, Lecture 3 November 14, / 43

20 Proving Myhill-Nerode Theorem = Let L be a regular language and let M = (Q, Σ, δ, q 0, F) be a DFA accepting it. Define the binary relation M by x M y if δ(q 0, x) = δ(q 0, y). M is an equivalence relation. x M y = xz M yz for every z Σ. = xz L iff yz L. Hence, x M y = x L y. Each equivalence class of L corresponds to union of classes of. M Namely, M is a refinement of. L (see drawing on board) Specifically, # of equivalence classes of L is less or equal than # of equivalence classes of M. M has finitely many equivalence classes. (?) Therefore, L has finitely many equivalence classes. Iftach Haitner (TAU) Computational Models, Lecture 3 November 14, / 43

21 Proving Myhill-Nerode theorem = Assume L has finitely many equivalence classes and let x 1,..., x n Σ be their representatives. We ll construct a DFA M = (Q, Σ, δ, q 0, F ) that accepts L. For x Σ, let C(x) be the index i {1,..., n} with x [x i ]. Q = {1,..., n}. δ(i, σ) = C(x i σ). q 0 = C(ε). F = {i : x i L}. Claim. For any y Σ : δ(q0, y) = C(y). Hence, y L(M) δ(q 0, y) F C(y) F x C(y) L y L. This is the optimal DFA, number of states wise, for L. (?) Proof: (of claim) By induction on word length. Let y = wσ Σ, and assume w [x i ] and y [x j ]. δ(i, σ) := C(x i σ) = (by right invariance) C(wσ) = C(y) = j. = δ(q 0, wσ) := δ( δ(q 0, w), σ) = (by i.h) δ(i, σ) = j. Iftach Haitner (TAU) Computational Models, Lecture 3 November 14, / 43

22 Example Construct a DFA for {w {0, 1} : # 1 (w) 0 (mod 5)}, via the latter proof method. Equivalent class representatives: {ε, 1, 11, 111, 1111} Q = {0, 1, 2, 3, 4} q 0 = 0 F = {0} δ(i, 0) = C(1 i 0) = i and δ(i, 1) = C(1 i 1) = i + 1 (mod 5) Iftach Haitner (TAU) Computational Models, Lecture 3 November 14, / 43

23 Part III Closure Properties of Regular Languages Iftach Haitner (TAU) Computational Models, Lecture 3 November 14, / 43

24 Simple closure properties Regular languages are closed under complement. 1. Let M = (Q, Σ, δ, q 0, F ) be a DFA that accepts L. 2. Then M = (Q, Σ, δ, q 0, Q \ F ) is a DFA that accepts L = Σ \ L. 3. NFA?! Regular languages are closed under intersection. 1. L 1 L 2 = L 1 L Proof with automata? Iftach Haitner (TAU) Computational Models, Lecture 3 November 14, / 43

25 Division For languages L 1, L 2 Σ, define L 1 /L 2 = {x Σ : y L 2, xy L 1 } Examples: L 1 = L(01 1) and L 2 = L(00). Then L 1 /L 2 = L 3 = L(a b c ) and L 4 = L(b). Then L 3 /L 4 = L(a b ) Iftach Haitner (TAU) Computational Models, Lecture 3 November 14, / 43

26 Closure under division Recall, L 1 /L 2 = {x : y L 2, xy L 1 } Theorem 10 Regular languages are closed under division with any language :L 1 is regular = L 1 /L 2 is regular. Proof: Let L 1 be a regular language and let L 2 be an arbitrary language. L 1 is a refinement of L 1 /L 2. Proof: Assume x L 1 y. For z Σ : xz L 1 /L 2 xzw L 1 yzw L 1 yz L 1 /L 2. = x L 1 /L 2 y. Hence, L 1 /L 2 has finite number of equivalent states, and thus regular. Another proof. Let M = (Q, Σ, δ, q 0, F) be a DFA for L 1. Let F = {q Q : y L 2, δ(q, y) F } The DFA M = (Q, Σ, δ, q 0, F ) accepts L 1 /L 2. F is well defined, but might be hard to compute non constructive proof. Iftach Haitner (TAU) Computational Models, Lecture 3 November 14, / 43

27 Homomorphism Definition 11 (Homomorphism) An homomorphism from alphabet to words over alphabet Σ, is a function h : Σ. For w, let h(w = w 1,..., w n ) = h(w 1 ) h(w n ). For L, let h(l) = {h(w): w L}. By definition, h(ε) = ε and h( ) =. Examples: Let h : {0, 1} {a, b} be defined by h(1) = aba and h(0) = aa. h(010) = aa aba aa. For L 1 = (01), h(l 1 ) = (aaaba). Let h(0) = a, h(1) = a. For L 2 = {0 n 1 n : n 0}, h(l 2 ) = {a 2n : n 0}. Iftach Haitner (TAU) Computational Models, Lecture 3 November 14, / 43

28 Closure under homomorphism Theorem 12 Regular languages are closed under homomorphism. Proof idea: Using regular expressions. Let L be regular language and let h : Σ. 1. h(l( )) =, h(l(ε)) = {ε}, and h(l(a)) = {h(a)} for any a. 2. h(l 1 L 2 ) = h(l 1 ) h(l 2 ), h(l 1 L 2 ) = h(l 1 ) h(l 2 ) and h(l ) = h(l) Let R be a regular expression with L(R) = L. The proof is by induction on R. R = 1. By Item 1, h(l) = h(l(r)) is regular. R > 1. Assume R = (R 1 R 2 ) (other cases are similar). By item 2, h(l) = h(l(r1 ) L(R 2 )) = h(l(r 1 )) h(l(r 2 )). By i.h., h(l(r1 ) and h(l(r 2 ) are regular. Thus, h(l) is regular. Iftach Haitner (TAU) Computational Models, Lecture 3 November 14, / 43

29 Closure under homomorphism, proof using Automata Let M = (Q,, δ, q 0, F) bea DFA for L. Define NFA N = (Q, Σ, δ, q 0, F ) for h(l) as follows: if δ(q i, σ) = q j and h(σ) = w 1,..., w t, then δ (diσ k, w i) = d k+1 iσ for all k {1,..., t 1}, letting diσ 1 = q i and diσ t = q j. Q includes Q and all new states. Claim 13 L(N) = h(l). Proof idea: h(l) L(N): w L = r 1,..., r w +1 s.t. r 1 = q 0, r w +1 F and r i+1 = δ(r i, w i+1 ).... = h(w) L(N). L(N) h(l): w L(N) = r 1,..., r w +1 s.t.r 1 = q 0, r w +1 F and r i+1 δ (r i, w i+1 ).... = w L with h(w ) = w. Iftach Haitner (TAU) Computational Models, Lecture 3 November 14, / 43

30 Inverse homomorphism Definition 14 (Inverse homomorphism) For homomorphism h : Σ, define its inverse homomorphism h 1 : Σ P( ), by h 1 (w) = {x : h(x) = w}. For L Σ, let h 1 (L) = x L h 1 (x) = {x : h(x) L} Example: h(0) = a, h(1) = b and h(2) = a. Then h 1 ({a n ba n : n 0}) = {{0, 2} n 1{0, 2} n : n 0}. Claim 15 For any h : Σ : 1. h(h 1 (L)) L, for any L Σ 2. L h 1 (h(l)), for any L Proof: 1. Immediate 2. Holds since w h 1 (h(w)) for any w Iftach Haitner (TAU) Computational Models, Lecture 3 November 14, / 43

31 Closure under inverse homomorphism Theorem 16 Regular languages are closed under inverse homomorphism. Proof idea: Let L be a regular language, let M = (Q, Σ, δ, q 0, F) be a DFA for L and let h : Σ. w h(l) h(w) L(M). Hence, to decide w simply emulate M(h(w))... More formally, Let M = (Q,, δ, q 0, F) with δ (q, a) = δ(q, h(a)). Easy to verify that δ (q, w) = δ M (q, h(w)) = w L(M ) h(w) L(M) w h 1 (L) Iftach Haitner (TAU) Computational Models, Lecture 3 November 14, / 43

32 Part IV Algorithmic Questions for NFAs Iftach Haitner (TAU) Computational Models, Lecture 3 November 14, / 43

33 Algorithmic Questions for NFAs Q: Given an NFA, N, and a string w, is w L(N)? Answer: Construct the DFA equivalent to N and run it on w. Q: Is L(N) =? Answer: This is a reachability question in graphs: Is there a path in the states graph of N from the start state to some accepting state? There are simple, efficient algorithms for this task. Iftach Haitner (TAU) Computational Models, Lecture 3 November 14, / 43

34 More Questions Q.: Is L(N) = Σ? Answer: Check if L(N) =. Q.: Given N 1 and N 2, is L(N 1 ) L(N 2 )? Answer: Check if L(N 2 ) L(N 1 ) =. Q.: Given N 1 and N 2, is L(N 1 ) = L(N 2 )? Answer: Check if L(N 1 ) L(N 2 ) and L(N 2 ) L(N 1 ). In the future, we will see that for stronger models of computations, many of these problems cannot be solved by any algorithm. Iftach Haitner (TAU) Computational Models, Lecture 3 November 14, / 43

35 Part V Summary Regular Languages Iftach Haitner (TAU) Computational Models, Lecture 3 November 14, / 43

36 Summary - Regular Languages So far we saw Finite automata, Regular languages, Regular expressions, Myhill-Nerode theorem and pumping lemma for regular languages. Did not do (see appendix): Given a DFA M, the minimal (in # of states) DFA M with L(M ) = L(M). Next class we introduce stronger machines and languages with more expressive power: pushdown automata, context-free languages, context-free grammars Iftach Haitner (TAU) Computational Models, Lecture 3 November 14, / 43

37 Part VII Appendix Not Taught in Class Iftach Haitner (TAU) Computational Models, Lecture 3 November 14, / 43

38 Section 1 Finding the Minimal Automata Iftach Haitner (TAU) Computational Models, Lecture 3 November 14, / 43

39 Finding the minimal automata Given a DFA M = (Q, Σ, δ, q 0, F ), find the minimal (with respect to # of states) DFA M with L(M ) = L(M). Definition 17 (Equivalent states ) For DFA (Q, Σ, δ, q 1, F), states q 1, q 2 Q are equivalent, if for all x 1, x 2 Σ with δ(q L 0, x i ) = q i, it holds that x 1 x 2. Idea: keep merging equivalent states in Q, until all states are non-equivalent. Actual idea: 1. Start with the two sets F and Q \ F. 2. Keep splitting the sets until all states in the same set are equivalent. To check whether states q and q are equivalent, check if δ(q, σ) and δ(q, σ) are in the same set, for all σ Σ.(?) 3. Merge all states in the same set. Is this a minimal automate? Yes, if M has no unreachable states. But we can assume that wlg. (?) Iftach Haitner (TAU) Computational Models, Lecture 3 November 14, / 43

40 Finding the minimal automata Algorithm 18 Input: DFA M = (Q, Σ, δ, q 0, F ) 1. Let T = {F, Q \ F }. 2. While S T, q 1, q 2 S and σ Σ s.t, δ(q 1, σ) S and δ(q 2, σ) / S for some S T : 2.1 Let S sp = {q S : δ(q, σ) S }. 2.2 Set T = T S sp (S \ S sp ) \ S. 3. Output DFA M = (Q, δ, q 0, F ), where Claim 19 Q = T q 0 = S 0 T, where q 0 S 0. F = {S T : S F} δ (S, σ) = S T, s.t. δ(q, σ) S for (every) q S. The above algorithm outputs the minimal automata for L(M). Iftach Haitner (TAU) Computational Models, Lecture 3 November 14, / 43

41 Example a s b a q 1 r 1 a b a b a b q 2 r 2 b On board... Iftach Haitner (TAU) Computational Models, Lecture 3 November 14, / 43

42 Section 2 Using Homomorphism Iftach Haitner (TAU) Computational Models, Lecture 3 November 14, / 43

43 Using homomorphism We know that L 1 = {0 n 1 n : n 1} is not regular, show that L 2 = {a n ba n : n 1} is not regular. We will prove using homomorphism and inverse homomorphism. Let h 1 (a) = a, h 1 (b) = b, h 1 (c) = a. (h 1 : {a, b, c} {a, b, c} ) h 2 (a) = 0, h 2 (b) = ɛ, h 2 (c) = 1. (h 1 : {a, b, c} {0, 1} ) We prove h 2 (h 1 1 (L 2) a b c ) = L 1. Thus, L 2 is not regular (?) h 1 1 (L 2) = L((a c) n ) b L((a c) n ) h 1 1 (L 2) a b c = {a n bc n : n 1} h 2 (h 1 1 (L 2) a b c ) = {0 n 1 n : n 1} Iftach Haitner (TAU) Computational Models, Lecture 3 November 14, / 43