Computational Models - Lecture 3 1

Transcription

1 Computational Models - Lecture 3 1 Handout Mode Iftach Haitner and Yishay Mansour. Tel Aviv University. March 13/18, Based on frames by Benny Chor, Tel Aviv University, modifying frames by Maurice Herlihy, Brown University. Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, / 43

2 Computational Models - Lecture 3 Non Regular Languages: Two Approaches (1) The Pumping Lemma (2) Myhill-Nerode Theorem (not in Sipser s book) Closure properties Algorithmic questions for NFAs Sipser s book, 1.4, 2.1, 2.2 Hopcroft and Ullman, 3.4 Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, / 43

3 Proved Last Time Theorem 1 A language, L, is described by a regular expression, R, if and only if L is regular. = construct an NFA accepting R. = Given a regular language, L, construct an equivalent regular expression We have made a lot of progress understanding what finite automata can do. But what can t they do? Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, / 43

4 Negative Results Is there a DFA that accepts B = {0 n 1 n : n 0} C = {w: w has an equal number of 0 s and 1 s} D = {w: w has an equal number of occurrences of 01 and 10 substrings} Consider B: DFA must remember how many 0 s it has seen impossible with finite state. The others are exactly the same... Question: Is this a proof? Answer: No, D is regular!??? Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, / 43

5 Part I Pumping Lemma Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, / 43

6 Pumping Lemma We will show that all regular languages have a special property. Suppose L is regular. If a string in L is longer than a certain critical length l (the pumping length), then it can be pumped to a longer string by repeating an internal substring any number of times. The longer string must be in L too. This is a powerful technique for showing that a language is not regular. Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, / 43

7 Pumping Lemma Lemma 2 If L is a regular language, then there is an l > 0 (the pumping length), where if s is any string in L of length s l, then s may be divided into three pieces s = xyz such that 1 for every i 0, xy i z L, 2 y > 0, and 3 xy l. Remarks: Without the second condition, the theorem would be trivial. The third condition is technical and sometimes useful. Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, / 43

8 Pumping Lemma Proof Let M = (Q,Σ,δ, q 1, F) be a DFA that accepts L. Let l be Q, the number of states of M. If s L has length at least l, consider the sequence of states M goes through as it reads s: s 1 s 2 s 3 s 4 s 5 s 6... s n q 1 q 20 q 9 q 17 q 12 q 13 q 9 q 2 q 5 F Since the sequence of states is of length s +1 > l, and there are only l different states in Q, at least one state is repeated (by the pigeonhole principle). Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, / 43

9 Pumping Lemma Proof (cont.) Write down s = xyz q 1 x q 9 y z q 5 By inspection, M accepts xy k z for every k 0. y > 0 because the state (q 9 in figure) is repeated. To ensure that xy l, pick first state repetition, which must occur no later than l+1 states in sequence. Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, / 43

10 Application # 1 Corollary 3 The language B = {0 n 1 n : n > 0} is not regular. Proof: By contradiction. Suppose B is regular. Let 2l be the pumping length. Consider the string s = 0 l 1 l. By pumping lemma s = xyz, where xy k z B for every k 0. If y is all 0, then xy k z has too many 0 s, for k 2. If y is all 1, then xy k z has too many 1 s, for k 2. If y is mixed, then xy k z is not of right form. Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, / 43

11 Using the Pumping Lemma How to prove that a language is not regular Given a language L An adversary sets the parameter l. Select a word s L. The word s would (usually) depend on l. The adversary selects a partition s = xyz, such that y 1 and xy l. Show an index k, such that xy k z L. Need to prove for any parameter l and any partition xyz. Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, / 43

12 Application # 2 Corollary 4 The language C = {w: # 1 (w) = # 0 (w)} is not regular. Proof: By contradiction. Suppose C is regular. Let l be the pumping length. Consider the string s = 0 l 1 l. By pumping lemma s = xyz, where xy k z C for every k 0. Since xy l then y is all 0, and xy k z has too many 0 s. Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, / 43

13 Reflections What about using (01) l C? What about D = {w: w has an equal number of occurrences of 01 and 10 substrings}? Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, / 43

14 Application # 3 Corollary 5 The language E = {0 i 1 j : i > j} is not regular. Proof: By contradiction. Suppose E is regular. Let l be its pumping length. Consider the string s = 0 l 1 l 1. By pumping lemma s = xyz, where xy k z E for every k 0, y > 0 and xy l But for k = 0 we have xz / E, contradiction. Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, / 43

15 Application # 4 Corollary 6 The language Primes {1}, which contains all strings whose length is a prime number, is not regular. Proof: By contradiction. Suppose Primes is regular, accepted by DFA M. Let l be the pumping length. Let s = 1 p Primes, where p l is a prime. By pumping lemma s = xyz, where xy k z Primes for every k. For k = p + 1 we have xy k z = 1 p+mp, where y = m. since p(m+1) is not prime, we have a contradiction. Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, / 43

16 Another Example Consider the language L = {a i b n c n : n 0, i 1} {b n c m : n, m 0}, For any word s L we can apply the pumping lemma: If s = a i b n c n, then set x = ε and y = a. If s = b n c m, then set x = ε and y = b. If s = c m, then set x = ε and y = c. Is L regular?! How can we prove it?! Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, / 43

17 Part II Characterization of Regular Languages Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, / 43

18 The Equivalence Relation L Let L Σ be a language. Define an equivalence relation L on pairs of strings: Let x, y Σ. We say that x L y if for every string z Σ, xz L if and only if yz L. It is easy to see that L is indeed an equivalence relation (reflexive, symmetric, transitive) on Σ. In addition, if x L y then for every string z Σ, xz L yz as well (this is called right invariance). Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, / 43

19 The Equivalence Relation L cont. Like every equivalence relation, L partitions Σ to (disjoint) equivalence classes. For every string x, let [x] Σ denote its equivalence class w.r.t. L (if x L y then [x] = [y] equality of sets). Question is, how many equivalence classes does L induce? L finite or infinite? Well, it could be either finite or infinite. This depends on the language L. Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, / 43

20 Three Examples Let L 1 {0, 1} contain all strings where the number of 1s is divisible by 4. Then L1 has finitely many equivalence classes. Let L 2 {0, 1} contain all strings of the form 0 n 1 n. Then L2 has infinitely many equivalence classes. Let L 3 = {a i b n c n : n 0, i 1} {b n c m : n, m 0}. Then L3 has infinitely many equivalence classes. (Proof on the board) Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, / 43

21 Myhill-Nerode Theorem Theorem 7 Let L Σ be a language. Then L is regular L has finitely many equivalence classes. Three specific consequences: L 1 {0, 1} contains all strings where the number of 1s is divisible by 4. Then L 1 is regular. L 2 {0, 1} contains all strings of the form 0 n 1 n. Then L 2 is not regular. Let L 3 = {a i b n c n : n 0, i 1} {b n c m : n, m 0}. Then L 3 is not regular. Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, / 43

22 Myhill-Nerode Theorem: Proof = Suppose L is regular. Let M = (Q,Σ,δ, q 0, F) be a DFA accepting it. The relation M on pairs of strings is defined as follows: x M y if δ(q 0, x) = δ(q 0, y). Clearly, M is an equivalence relation. Furthermore, if x M y, then xz M yz for every z Σ. Therefore, xz L if and only if yz L. This means that x M y = x L y (i.e., M is a refinement of L ). Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, / 43

23 Myhill-Nerode Theorem: Proof cont. = The equivalence relation M has finitely many equivalence classes (at most the number of states in M). We saw that x M y = x L y, so the number of equivalence classes of L is less or equal than the number of equivalence classes of M. Therefore, L has finitely many equivalence classes. Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, / 43

24 Myhill-Nerode Theorem: Proof cont. = Suppose L has finitely many equivalence classes. We ll construct a DFA M = (Q,Σ,δ, q 0, F) that accepts L. Let x 1,...,x n Σ be representatives for the finitely many equivalence classes of L. Q = {[x 1 ],...,[x n ]}. δ([x i ], a) = [x i a] for all a Σ. q 0 = [ε]. F = {[x i ]: x i L}. Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, / 43

25 Myhill-Nerode Theorem: Proof cont. = δ([ε], x) = [x] proof: Assume δ([ε], x) = [x] = [x i ]. By right invariance δ([ε], xa) = δ([x i ], a) = [x i a] = [xa] Therefore, M accepts x iff x L So L is accepted by DFA, hence L is regular. Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, / 43

26 Example Construct DFA (via the above method) for L 1 {0, 1} contains all strings where the number of 1s is divisible by 5. Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, / 43

27 Applications of the Proof Let L be a regular language and let M be a DFA accepting it The number of equivalence classes of L lowerbounds the number of equivalence classes of M, which equals the number of states in M. The equivalence relation M, is a refinement of L (each equivalence class of L correspond to a union of states). There is an automata whose number of states equals the number of equivalence classes of L Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, / 43

28 Minimizing Automata Input: An automata M Output: An automata M, such that L(M) = L(M ) and M has a minimal number of states. Let S 1 = F and S 2 = Q F. Set S = {S 1, S 2 }. While exists equivalence class S i S, q 1, q 2 S i and σ Σ such that, δ(q 1,σ) S j1 and δ(q 2,σ) S j2, j 1 j 2, then let S i,1 = {q S i : δ(q,σ) S j1 }, j 1 i. S = S S i S i,1 (S i S i,1 ) Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, / 43

29 Minimizing Automata Claim 8 Output M = (Q,δ, q 0, F ): where Q = S, q 0 = S 0 S such that q 0 S 0 F = {S 1,...,S k } S, such that S i F. δ (S i,σ) = S j if for q S i then δ(q,σ) S j. The algorithm terminates, and outputs a (in fact, the) minimal automata. Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, / 43

30 Example a s b a q 1 r 1 a b a b a b q 2 r 2 b Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, / 43

31 Part III Closure Properties of Regular Languages Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, / 43

32 Simple Closure Properties Regular languages are closed under complement. Let M = (Q,Σ,δ, q 0, F) be a DFA that accepts L. Then M = (Q,Σ,δ, q 0, Q F) is a DFA that accepts L = Σ \L. NFA?! Regular languages are closed under intersection. L 1 L 2 = L 1 L 2. Proof with automata?! Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, / 43

33 Division L 1 /L 2 = {x: y L 2, xy L 1 } Examples: L 1 = (01 1) and L 2 = 00, L 1 /L 2 =? L 1 /L 2 =. L 3 = a b c and L 4 = b, L 3 /L 4 =?. L 3 /L 4 = a b. Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, / 43

34 Division cont. Theorem 9 Regular languages are closed under division with any language. Proof: L 1 is a regular language, so it has a DFA M = (Q,Σ,δ, q 0, F). L 2 is an arbitrary language. For L 1 /L 2 we build M = (Q,Σ,δ, q 0, F ). F = {q: y L 2,δ(q, y) F}. F is well defined, but might be hard to compute non constructive proof". Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, / 43

35 Homomorphism Definition 10 Homomorphism: an assignment that replaces each letter with a word, i.e., h : Σ. For a word w = w 1,...,w n, then h(w) = h(w 1 ) h(w n ), ad for a language L, then h(l) = {h(x) : x L}. Example: h(1) = aba, h(0) = aa h(010) = aa aba aa L 1 = (01), h(l 1 ) = (aaaba). h(0) = a, h(1) = a L 2 = {0 n 1 n n 0}, h(l 2 ) = {a 2n n 0}. Theorem 11 Regular languages are closed under homomorphism. Proof: Using regular expressions. Using Automata Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, / 43

36 Inverse Homomorphism Definition 12 Inverse homomorphism: h 1 (w) = {x: h(x) = w}, h 1 (L) = {x: h(x) L} Example: h(1) = aba, h(0) = aa L 2 = (ab ba) a, h 1 (L 2 ) = {1}. Claim 13 h(h 1 (L)) L h 1 (h(l)). Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, / 43

37 Inverse Homomorphism cont. Theorem 14 Regular languages are closed under inverse homomorphism. Proof idea: Let M = (Q,Σ,δ, q 0, F) the automata for L, and h : Σ. for each letter a we advance in M using h(a). Formally, we define M = (Q,,δ, q 0, F), where δ (q, a) = δ(q, h(a)). Hence, δ (q, w) = δ(q, h(w)) w L(M ) h(w) L(M) Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, / 43

38 Using Homomorphism We know that L 1 = {0 n 1 n : n 1} is not regular. Show that L 2 = {a n ba n : n 1} is not regular We will prove using homomorphism and inverse homomorphism h 1 (a) = a, h 1 (b) = b, h 1 (c) = a. h 2 (a) = 0, h 2 (b) = ɛ, h 2 (c) = 1. h 2 (h 1 1 (L 2) a b c ) = L 1 h 1 1 (L 2) = (a c) k b(a c) k h 1 1 (L 2) a b c = {a n bc n : n 1} h 2 (h 1 1 (L 2) a b c ) = {0 n 1 n : n 1} Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, / 43

39 Part IV Algorithmic Questions for NFAs Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, / 43

40 Algorithmic Questions for NFAs Q.: Given an NFA, N, and a string w, is w L(N)? Answer: Construct the DFA equivalent to N and run it on w. Q.: Is L(N) =? Answer 1: Build a minimal size DFA. (Might be exponential time!) Answer 2: This is a reachability question in graphs: Is there a path in the states graph of N from the start state to some accepting state. There are simple, efficient algorithms for this task. Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, / 43

41 More Questions Q.: Is L(N) = Σ? Answer: Check if L(N) =. Q.: Given N 1 and N 2, is L(N 1 ) L(N 2 )? Answer: Check if L(N 2 ) L(N 1 ) =. Q.: Given N 1 and N 2, is L(N 1 ) = L(N 2 )? Answer: Check if L(N 1 ) L(N 2 ) and L(N 2 ) L(N 1 ). In the future, we will see that for stronger models of computations, many of these problems cannot be solved by any algorithm. Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, / 43

42 Part V Summary Regular Languages Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, / 43

43 Summary - Regular Languages So far we saw finite automata, regular languages, regular expressions, Myhill-Nerode theorem and pumping lemma for regular languages. Next class we introduce stronger machines and languages with more expressive power: pushdown automata, context-free languages, context-free grammars Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 3 March 13/18, / 43