Automata and Grammars

Transcription

1 Automata and Grammars TIN071 Marta Vomlelová September 19, 2017 Automata and Grammars Introduction 1 September 19, / 294

2 Organization Lecture: you meet more than on slides you may give me your feedbeck, influence the lecture Exercises: you construct automata, languages needed also for the exam to practice is very different than read you need the credit. Examination: Written exam with oral disscussion Understanding of lectures + ability to formalize Exercises as you met and simillar, Write a definition, theorem, the idea of a proof, algorithm. Automata and Grammars Introduction 1 September 19, / 294

3 The lecture goal Get used to abstract computer models, describe the model formally, understand small changes in the definition lead to significantly different classes get used to infinite objects Automata and Grammars two kinds of description finite automata regular grammars pushdown automata context free grammars lineary bounded automata context grammars Turing machines grammars of Type 0 Automata and Grammars Introduction 1 September 19, / 294

4 Literature J.E. Hopcroft, R. Motwani, J.D. Ullman: Introduction to Automata Theory, Languages, and Computations, Addison Wesley Exercises in Czech M. Chytil: Automaty a gramatiky, SNTL Praha, 1984 Automata and Grammars Introduction 1 September 19, / 294

5 A little bit of history By Alfred Edward Chalon - Science & Society Picture Library, Public Domain, At first the first formalization of algorithm : Ada, Countess of Lovelace 1852 more intensively with computers after 1925 What can machines do and what cannot? Church, Turing, Kleene, Post, Markov K. Čapek: RUR By Erlangen, ematisches after 1950 Copyright Konrad is MFO Forschungsinstitut Jacobs, - Math- Oberwol- fach, neural networks (1943) Finite Automata (Kleene 1956 neural networks FA) CC BY-SA 2.0 de, grammars (Chomsky) push down automata the formal theory of finite automata. Automata and Grammars Introduction 1 By September 19, / 294

6 Practical usage Justification of correctness of a program, an algorithm, a translator natural language processing lexical analyzer hardware design and verification digital circuits machines automata software implementation search of a word in text system verification applications in biology Automata and Grammars Introduction 1 September 19, / 294

7 Finite Automata Software for designing and checking the behavior of digital circuits. A Finite automaton modeling an on/off switch. off Lexical analyzer, web page analyzer. Push Push on A finite automaton modeling recognition of then. t h e n λ t th the then Automata and Grammars Introduction 1 September 19, / 294

8 Automata and Complexity Automata are essential for the study of the limits of computation. What can computer do at all? decidability. What can computer do efficiently? intractability. Time of calculation as a function of the size of the input. Growing faster than any polynomial is deemed to grow too fast. Automata and Grammars Introduction 1 September 19, / 294

9 Formal Proofs NAIL062 Propositional and Predicate Logic Deductive Proofs (axioms, statments, Modus Ponens) Contrapositive C H instead of H C Proof by Conradiction H and C implies falsehood. Counterexamples We prove that a statement is not true by prestenting an example where the statement does not hold. Inductive Proofs Prove the basis, usually i = 1 or single node tree. Prove the inductive step: For any n, if S(n) then also S(n + 1). Structural Induction on trees. Automata and Grammars Introduction 1 September 19, / 294

10 Definition (Deterministic Finite Automata) A deterministic finite automation (DFA) A = (Q, Σ, δ, q 0, F ) consists of: A finite set of states, often denoted Q. A finite set of input symbols, denoted Σ. A transition function Q Σ Q, denoted δ, represented by arcs. A start state q 0 Q. A set accepting states (final states) F Q. Convention: If some transitions are missing, we add a new state fail and make the transition δ total by adding edges to fail for any undefined pair q, s. If the set F is empty, we add to F and Q a new state final, with no transitions from other states, just staying in final for any s Σ: δ(final, s) = final. t h e n λ t th the then n n e h e e n t n e e n t h t h t h t h t hen final fail Automata and Grammars Introduction 1 September 19, / 294

11 Deterministic Finite Automata Description Example An automaton A that accepts L = {x01y : x, y {0, 1} }. State diagram (graph) Automaton A = ({q 0, q 1, q 2 }, {0, 1}, δ, q 0, {q 1 }) ,1 table q 0 0 q 1 2 q 1 δ 0 1 rows: states + transitions columns: letters from the input alphabet Σ q 0 q 1 q 2 q 2 q 1 q 2 q 0 q 1 q 1 State tree nodes = states edges = transitions only reachable states we need it only for nondeterministic FA. q0 0 q0 0 q0 1 q0 0 q0 1 q q1 (stuck) q1 1 q2 (stuck) q1 1 q2 Automata and Grammars Introduction 1 September 19, / 294

12 Alphabet, Words, Languages Definition (Word, λ,ɛ,σ,σ +, Language) Alphabet is a finite, nonempty set of symbols, denoted Σ (or X). String (=word) is a finite sequence of symbols s chosen from some alphabet Σ. For example ; empty string λ (or ɛ). The set of all strings in the alphabet is denoted Σ, the set of all non empty strings Σ + = Σ \ {λ}. A language L Σ is a set of strings in the alphabet Σ. Definition (concatenation, power, length) We define operations on strings from Σ : concatenation u.v or uv powers u n (u 0 = λ, u 1 = u, u n+1 = u n.u) length u ( λ = 0, auto = 4). number of occurences of s Σ in the string u is denoted u s ( zmrzlina z = 2). Automata and Grammars Introduction 1 September 19, / 294

13 Extended Transition Function Definition (Extended Transition Function to Strings) If δ is our transition function, then the extended transition function ˆδ, ˆδ : Q Σ Q (transitive closure of δ) takes a state q and a string w and returns a state p and is defined by induction: ˆδ(q, λ) = q. Let x Σ, w Σ. Then ˆδ(q, wx) = δ(ˆδ(q, w), x). Remark: Any time δ is applied to strings, δ was intended. δ (1100) = q 2, δ ( ) = q ,1 q 0 0 q 1 2 q 1 Automata and Grammars Introduction 1 September 19, / 294

14 Language of the DFA Definition (Language of the DFA) The language of the DFA A is L(A) = {w : ˆδ(q 0, w) F }. String w is accepted, recognized by the automaton A, iff w L(A). A language L can be recognized by a DFA, if there exists a DFA A such that L = L(A). Languages accepted by DFA s (and FA s) are called regular languages. The set of regular languages is denoted by F Example (regular languages) 0 1 L = {w w = xux, w {0, 1}, x {0, 1}, u {0, 1} } Automata and Grammars Introduction 1 September 19, / 294

15 Regular Languages Examples Example (Regular Language) b a a a b a L = {w w = ubaba, w {a, b}, u {a, b} }. a b b b Example (Regular Language) L = {w w {a, b} &w binary encoding of a number dividible by 5}. Example (A language that is not regular) L = {0 n 1 n w {0, 1}, n 1} is not regular Automata and Grammars Introduction 1 September 19, / 294

16 Congruence, Myhill Nerode Theorem Definition (congruence) Let us have a finite alphabet Σ and an equivalence relation on Σ (reflexive, symmetric, transitive). Then: is right congruence, iff u, v, w Σ u v uw vw. it has a finite index, iff the partition Σ / has finite number of classes. [u], [u] denotes the class of congruence containing string u. Theorem (Myhill Nerode Theorem) Let us have a languguage L Σ. The following statements are equivalent: a) L can be recognized by DFA, b) there exists a right congruence with a finite index over Σ such that L is a union of some classes of the partition Σ /. Automata and Grammars Introduction 1 September 19, / 294

17 Proof: Proof of the Myhill Nerode Theorem a) b); DFA right congruence with a finite index we define u v δ (q 0, u) = δ (q 0, v). it is equivalence (reflexive, symmetric, transitive) it is a right congruence (from the definition of δ ) it has finite index (finite number of states Q) L = {w δ (q 0, w) F } = q F {w δ (q 0, w) = q}. b) a);right congruence with a finite index DFA we take the alphabet Σ states Q are classes of the partition Σ / start state q 0 [λ] final states F = {c 1,..., c n }, where L = i=1,...,n c i transition function δ([u], x) = [ux] (is correct from the right congruence definition). L(A) = L w L w δ ([λ], w) = w i=1,...,n c i w c 1... w cn [w] = c 1... [w] = cn [w] F w L(A) Automata and Grammars Introduction 1 September 19, / 294

18 The Application of Myhill Nerode Theorem Automaton construction Example Construct an automaton that accepts the language L = {w w {a, b} & w a = 3k + 2, k N}. We define u v ( u a mod3 = v a mod3) equivalence classes are 0,1,2 L corresponds to the class 2 a transitions to the next class b transitions stay in the same class. b b b a a a Automata and Grammars Introduction 1 September 19, / 294

19 Myhill Nerode Theorem to Prove Non regularity of a Language Example (Proof that a language is not regular) Decide whether following language is regular L = {0 n 1 n n 1}. Assume it were regular there exists a right congruence L with a finite index m, L is union of some classes Σ / L we take the set of strings S = {0, 00, 000,..., 0 m,...}, m N for any i j there exists a string 1 i, such that 0 i 1 i L & 0 j 1 i / L no two elements of S may be in the same L congruence class the congruence L must have an infinite index contradiction with the finite index in Myhill Nerode theorem. Automata and Grammars Introduction 1 September 19, / 294

20 Summary Definition Deterministic Finite Automaton A = (Q, Σ, δ, q 0, F ) Language L Σ The Language of the DFA A: L(A) = {w w Σ & δ (q 0, w) F } the class of Regular Languages Mihyll Nerode theorem Example proof on non regularity of the language L = {w 0 i 1 i, i N} Examples of regular languages. Automata and Grammars Introduction 1 September 19, / 294

21 Example Example DFA accepting all and only strings with an even number of 0 s and an even number of 1 s. Transition Diagram Accepting even number of 1 and even number of 0. 1 Transition Table δ 0 1 q q 1 0 q 0 q 2 q 1 q 1 q 3 q 0 q 2 q 0 q 3 q 3 q 1 q 3 q 2 q 3 1 Automata and Grammars Introduction 1 September 19, / 294

22 Example of a (bad) electronic-money protocol Example Three parties: the customer, the store, the bank. Only one money file (for simplicity). Customer may decide to transfer money to the store, which will the redeem the file from the bank and ship goods to the customer. The customer has the option to cancel the file. Five events: Customer may pay. Customer may cancel. Store may ship the goods to the customer. Store may redeem the money. The bank may transfer the money by creating a new, suitably encrypted money file and sending it to the store. Automata and Grammars Introduction 1 September 19, / 294

23 (Incomplete) Finite Automata for the Bank Example pay redeem transfer a b d f ship ship ship c redeem e transfer g Store cancel pay cancel 2 1 redeem transfer 3 4 Customer Bank Automata and Grammars Introduction 1 September 19, / 294

24 An Edge for EACH Input Formally, the automaton should perform an action for any input. The automaton for the store needs an additional arc from each state to itself, labeled cancel. Customer must not kill the automaton by execution pay again, so loops with pay label are necessary. Similarly with other commands. pay,ship,redeem, cancel 2 cancel pay,redeem, cancel, ship 1 redeem transfer 3 4 pay,ship pay,redeem, cancel, ship Extended Automaton for the Bank Automata and Grammars Introduction 1 September 19, / 294

25 Product Automaton Product automaton of Bank and Store has states pairs B S. To construct the arc of the product automaton, we need to run the bank and store automata in parallel. If any automaton dies, the product dies too. pay redeem transfer a b d f ship ship ship redeem transfer c e g pay,ship,redeem, cancel 2 cancel pay,redeem, cancel, ship 1 redeem transfer 3 4 pay,ship pay,redeem, cancel, ship Automata and Grammars Introduction 1 September 19, / 294

26 Finite Automata, Regular Languages Deterministic Finite Automaton (DFA) A = (Q, Σ, δ, q 0, F ). Language accepted (recognized) by a DFA A = (Q, Σ, δ, q 0, F ) is the language L(A) = {w w Σ & δ (q 0, w) F } Language L is recognizable by a DFA, if there exists DFA A such that L = L(A) The class of languages recognizable by a DFA F is called regular languages. 0 Automata and Grammars Pumping Lemma, DFA minimization and equivalence 2 September 19, / 294

27 Is a given language regular? Is a given language regular? YES Construct an automaton. NO Find the contradiction with Myhill Nerode theorem or with the Pumping Lemma. Theorem (Myhill Nerode Theorem) Let us have a language L Σ. The following statements are equivalent: a) L can be recognized by DFA, b) there exists a right congruence with a finite index over Σ such that L is a union of some classes of the partition Σ /. Finite automaton encodes only finite information. It can recognize infinite languages. b b b a a a Automata and Grammars Pumping Lemma, DFA minimization and equivalence 2 September 19, / 294

28 Pumping Lemma For Regular Languages Theorem (Pumping Lemma For Regular Languages) Let L be a regular language. Then there exists a constant n N (which depends on L) such that for every string w such that w n, we can break w into three strings, w = xyz, such that: y λ. xy n. For all k 0, the string xy k z is also in L. Example abbbba = a(b)bbba; i 1; a(b) i bbba L(A). aaaaba = (aaa)aba; i 1; (aaa) i aba L(A). b b b a a a Automata and Grammars Pumping Lemma, DFA minimization and equivalence 2 September 19, / 294

29 Proof of the Pumping Lemma For Regular Languages Proof. Suppose L is regular, then L = L(A) for some DFA A with n states. Take any string w = a 1 a 2... a m of length m n, a i Σ. Define i p i = δ (q 0, a 1 a 2... a i ). Note p 0 = q 0. We have n + 1 p i s and n states, therefore there are i, j such that 0 i < j n : p i = p j. Define: x = a 1 a 2... a i, y = a i+1 a i+2... a j, z = a j+1 a j+2... a m. Note w = xyz. y = a i+1 a i+2... a j p 0 x = a 1 a 2... a i p i z = a j+1 a j+2... a m The loop above p i can be repeated any number of times and the input is also accepted. Automata and Grammars Pumping Lemma, DFA minimization and equivalence 2 September 19, / 294

30 Applications of the Pumping Lemma Example (The Pumping Lemma as an Adversarial Game) The language L eq = {w; w 0 = w 1 } of all strings with an equal number of 0 s and 1 s is not regular language. Proof. Suppose it is regular. Take n from the pumping lemma. Pick w = 0 n 1 n L eq. Break w = xyz as in the pumping lemma, y λ, xy n. Since xy n and it comes at front of w, it consists only of 0 s. The pumping lemma says: xy L eq (for k = 0). However, it has less 0 s and the same amount of 1 s as w, so one of them must not be in L eq. Example The language L = {0 i 1 i ; i 0} is not regular. Automata and Grammars Pumping Lemma, DFA minimization and equivalence 2 September 19, / 294

31 Applications of the Pumping Lemma 2 Example The language L pr of all strings of 1 s whose length is a prime is not a regular language. Proof. Suppose it were. Take a constant n from the pumping lemma. Consider some prime p n + 2, let w = 1 p. Break w = xyz by the pumping lemma, let y = m. Then xz = p m. xy p m z L pr by pumping lemma, but xy p m z = xz + (p m) y = p m + (p m)m = (m + 1)(p m) that is not a prime (none of two factors are 1). Example (Non regular language that can be pumped ) The language L = {u u = a + b i c i u = b i c j } is not regular (Myhill Nerode theorem), but the first symbol can be always pumped. Automata and Grammars Pumping Lemma, DFA minimization and equivalence 2 September 19, / 294

32 Pumping lemma and (in)finity of languages Theorem Regular language L is infinite iff there exists u L; n u < 2n, such that n is the number from the pumping lemma. Proof: If ( u L) n u < 2n, then we can pump u, ( i N) u i L, infinite number of elements L. Language L is infinite, it contains a string w such that n w. If w < 2n, we are done. Otherwise, from the Pumping lemma w = xyz and xz L, we have a shorter word. If 2n xz, we shorten xz further (Pumping lemma again). Each shortening cuts no more than n symbols therefore we meet the interval [n, 2n) (we do not jump over it). To check language (in)finity it is sufficient to check a finite number of strings {u: such that n u < 2 n}. Automata and Grammars Pumping Lemma, DFA minimization and equivalence 2 September 19, / 294

33 Definition (Reachable states) Let s have a DFA A = (Q, Σ, δ, q 0, F ) and q Q. The state q is reachable iff there exists w Σ such that δ (q 0, w) = w. Algorthm: Reachable States We search reachable states iteratively. Start: M 0 = {q 0 }. Repeat: M i+1 = M i {q q Q, ( p M i, x Σ) δ(p, x) = q} until: M i+1 M i. Proof: Correctness and completeness Correctness: M 0 M 1... Q and any M i contains only reachable states. Completeness: assume q is reachable, that is ( w Σ ) δ (q 0, w) = q we take the shortest w = x 1... x n such that δ (q 0, x 1... x n) = q obviously δ (q 0, x 1... x i) M i (more precisely M i \ M i 1) therefore δ (q 0, x 1... x n) M n, therefore q M n. Automata and Grammars Pumping Lemma, DFA minimization and equivalence 2 September 19, / 294

34 Equivalence of automata, Homomorphism Definition (Automata Equivalence) Finite automata A, B are equivalent iff they recognize the same language, that is L(A) = L(B). Definition (automata homomorfismus) Let A 1, A 2 are two DFA. A mapping h : Q 1 Q 2 is an (automata) homomorphism iff: h(q 01 ) = q 02 initial state maps to the initial state h(δ 1 (q, x)) = δ 2 (h(q), x) transition function correspondence q F 1 h(q) F 2 final states correspondence. Homomorfismus, that is a bijection, is called isomorphism. Theorem (Automata Equivalence Theorem) If there exists a homomorphism of automata A 1 and A 2, then A 1 and A 2 are equivalent. Automata and Grammars Pumping Lemma, DFA minimization and equivalence 2 September 19, / 294

35 Proof of the Automata Equivalence Theorem Theorem (Automata Equivalence Theorem) If there exists a homomorphism of automata A 1 and A 2, then A 1 and A 2 are equivalent. Proof: For any string w Σ we prove by finite iteration h(δ 1 (q, w)) = δ 2 (h(q), w) further: A 1 A 2 h q F1 q F2 w L(A 1 ) δ 1 (q 01, w) F 1 h(δ 1 (q 01, w)) F 2 δ 2 (h(q 01 ), w) F 2 w h w δ 2 (q 02, w) F 2 w L(A 2 ) q 01 q 02 Automata and Grammars Pumping Lemma, DFA minimization and equivalence 2 September 19, / 294

36 Language accepted by automata Ambiguity There exist different automata accepting the same language L. The language L = {w w {1} & w = 3k} Automata and Grammars Pumping Lemma, DFA minimization and equivalence 2 September 19, / 294

37 State Equivalence and its Transitivity Definition (State Equivalence) States p, q Q of a FA A are equivalent iff: For all strings w holds: δ (p, w) F iff δ (q, w) F. If states are not equivalent we call them distinguishable. Example Automaton in the figure right: C, G are distinguishable, δ (C, λ) F a δ (G, λ) / F. A,G: δ (A, 01) = C is accepting, δ (G, 01) = E is not accepting. A,E are equivalent λ, 1 obvious, 0 transits to non accepting states, 01 and 00 meet in the same state A B C D E F G H Theorem State equivalence is transitive. Automata and Grammars Pumping Lemma, DFA minimization and equivalence 2 September 19, / 294

38 Equivalent States Recognition Algorthm: Equivalent States Recognition for a DFA Following algorithm finds distinguishable states: Basis: If p F is accepting and q / F is not, the pair {p, q} is distinguishable. Induction: Let p, q Q, a Σ and the pair r, s; r = δ(p, a) and s = δ(q, a) is distinguishable. Then also {p, q} are distinguishable A B C D E F G H B x C x x D x x x E x x x F x x x x G x x x x x x H x x x x x x A B C D E F G The cross denotes distinguishable pairs. C is distinguishable immediatelly, other except {A, G}, {E, G} too. We can see three pairs of equivalent states. Automata and Grammars Pumping Lemma, DFA minimization and equivalence 2 September 19, / 294

39 Distinguishable states Accepting and non accepting states B C x x D x E x F x G x H x A B C D E F G 1 st step1: δ(q, 1) F for q {B, C, H} B x C x x D x x E x x F x x G x x H x x x x x x A B C D E F G 1 st step0δ(q, 0) F for q {D, F } B x C x x D x x x E x x x F x x x x G x x x x H x x x x x x A B C D E F G A B C D E F G H 0 1 B and G are distinguishable, δ(a, 0) = B, δ(g, 0) = H, therefoer A,G are distinguishable. Simillarly, δ(, 0) for E,G goes to distinguishable states H,G. B x C x x D x x x E x x x F x x x x G x x x x x x H x x x x x x A B C D E F G Three pairs of equivalent states remain Automata and Grammars Pumping Lemma, DFA minimization and equivalence 2 September 19, / 294

40 Theorem If two states are not distinguished by previous algorithm, they are equivalent. Proof: Algorithm Corectness Consider bad pairs of states that are distinguishable but not distinguish by the algorithm. Take the pair p, q that can be distinguished by the shortest string w = a 1... a k. States r = δ(p, a 1 ) and s = δ(q, a 1 ) are distinguishable by a shorter string a 2... a k, therefore r, s is not a bad pair. They must be distinguished by the algorithm. Then, in the next step, the algorithm distinguishes also p, q. The time is polynomial function of the number of states n. In one iteration, we consider all pairs, that is O(n 2 ). In each iteration we add a cross, that means no more than O(n 2 ) iterations. Together, O(n 4 ). The algorithm may be speeded up to O(n 2 ) by memorizing states that depend on the pair {r, s} and following the list backwards. Automata and Grammars Pumping Lemma, DFA minimization and equivalence 2 September 19, / 294

41 Testing the Equivalence of Regular Languages Algorthm: Testing the Equivalence of Regular Languages The equivalence of regular languages L, M may be tested as follows: Find DFA A L, A M recognizing them, L(A L ) = L, L(A M ) = M, Q L Q M =. Construct a DFA (Q L Q M, Σ, δ L δ M, q L, F L F M ) as a union of states and transitions; select one (any) initial state. Languages are equivalent iff initial states of original DFA s are equivalent. Example Language L = {λ} {0, 1} 0 accepting empty string and strings ending with 0. Two DFA s accepting L and a table of distinguishable states. 0 0 A C E 1 Automata and Grammars Pumping Lemma, DFA minimization and equivalence 2 September 19, / B 0 D B x C x D x E x x x A B C D

42 Minimal DFA Definition (minimum state DFA) A DFA is minimum state (reduced) DFA, iff has no non reachable states and no two states are equivalent. Theorem (DFA Minimization) For any DFA A there exist an equivalent DFA B that is minimum state (reduced) DFA D A B C not 0 reachable E F G H A,E G B,H F 0 C 1 Automata and Grammars Pumping Lemma, DFA minimization and equivalence 2 September 19, / 294

43 Algorithm for minimum automaton for DFA A Algorthm: Algorithm for minimum automaton for DFA A An algorithm to find a minimum state DFA equivalent with a DFA A Eliminate from DFA A all non reachable states. Find a partition of remaining states to equivalence classes. Construct DFA B with equivalence classes as states. We denote the transition function of B δ B. Let us have S Q B. For any q S we define δ B (S, a) = [δ A (q, a)] the equivalence class of δ A (q, a). This class is the same for all states a S since they are equivalent. Initial state q 0B is the class containing the initial state q 0A. The set of accepting states F B are equivalence classes containing accepting states F A. Automata and Grammars Pumping Lemma, DFA minimization and equivalence 2 September 19, / 294

44 Example of minimal state DFA construction A B C D not reachable E F G H 0 B x C x x E x x F x x x x G x x x x x H x x x x x A B C E F G G F 1 1 A,E B,H C 0 1 Equivalence classes: {A, E}, {B, H}, {C}, {F }, {G} Automata and Grammars Pumping Lemma, DFA minimization and equivalence 2 September 19, / 294

45 Non deterministic FA Example Example A non deterministic FA in Figure may be simplifyed by removing the state C. States {A, C} are distinguishable by the input 0, therefore the previous algorithm will not find this reduction. (We may use exhausive enumeration or reduce the NFA to DFA and find a minimal DFA for it.) 0,1 A 0 B 1 0 C Automata and Grammars Pumping Lemma, DFA minimization and equivalence 2 September 19, / 294

46 Summary Pumping lemma for DFA the use to prove a language is non regular there exist a language, that can be pumped and is not regular prove by Mihyll Nerode theorem to check (in)finity of a regular language is possible by checking a finite number of strings reachable states, an algorithm to identify them equivalent automata, states distinguishable states, an algorithm to identify them automata homomorphism Theorem: If there is a homomorphism from A 1 to A 2, they are equivalent. minimum state DFA, an algorithm to find equivalent minimum state DFA to a given DFA A Automata and Grammars Pumping Lemma, DFA minimization and equivalence 2 September 19, / 294

47 Finite Automata Regular languages are languages recognizable by finite automata. Finite automata are either deterministic or nondeterminitstic these can be in more states (in parallel). Both characterizes the same class of languages. The nondeterministic make the description simpler. There exists an algorithm transforming a NFA to a DFA. Another extension: λ transitions without reading any input symbol. Again, they accept only regular languages. Automata and Grammars NFA, λ transition, Two way DFA 3 September 19, / 294

48 Nondeterministic Finite Automata (NFA) A NFA can be in serveral states at once. It has an ability to guess something about input. A NFA accepting all strings that end in 01. 0,1 q 0 0 q 1 1 q 2 NFA processes input q 0 0 q 0 0 q 1 0 q 0 0 q 1 0 q q 1 (stuck) q 1 1 q 2 (stuck) q 1 1 q 2 Automata and Grammars NFA, λ transition, Two way DFA 3 September 19, / 294

49 Nondeterministic Finite Automata Definition (Nondeterministic Finite Automata) A nondeterministic finite automation (NFA) A = (Q, Σ, δ, q 0, F ) consists of: A finite set of states, often denoted Q. A finite set of input symbols, denoted Σ. A transition function δ : Q Σ P(Q) returns a subset of Q. A start state q 0 Q. A set accepting states (final states) F Q. Example The NFA from previous slide is A = ({q 0, q 1, q 2 }, {0, 1}, δ, q 0, {q 2 }). δ 0 1 q 0 {q 0, q 1 } {q 0 } q 1 {q 2 } q 2 Automata and Grammars NFA, λ transition, Two way DFA 3 September 19, / 294

50 Definition (Extended Transition Function to Strings) If δ is our transition function, then the extended transition function δ, δ : Q Σ P(Q) takes a state q and a string w and returns a set of states Q and is defined by induction: δ (q, λ) = {q}. Let w = ax, a Σ, x Σ, suppose δ (q, x) = {p 1,..., p k }. Let k i=1 δ(p i, a) = {r 1, r 2,..., r m }. Then δ (q, ax) = {r 1, r 2,..., r m }. First compute δ (q, x) and then follow any transitiohn from any of these states that is labeled a. 0,1 q 0 0 q 1 1 q 2 δ (q 0, λ) = ={q 0 } δ (q 0, 0) = δ(q 0, 0) ={q 0, q 1 } δ (q 0, 00) =δ(q 0, 0) δ(q 1, 0)={q 0, q 1 } δ (q 0, 001) =δ(q 0, 1) δ(q 1, 1)={q 0, q 2 } δ (q 0, 0010) =δ(q 0, 0) δ(q 2, 0)={q 0, q 1 } δ (q 0, 00101)=δ(q 0, 1) δ(q 1, 1)={q 0, q 2 } Automata and Grammars NFA, λ transition, Two way DFA 3 September 19, / 294

51 The Language of an NFA Definition (Language of an NFA) If A = (Q, Σ, δ, q 0, F ) is an NFA, then L(A) = {w δ (q 0, w) F } is the language accepted by NFA A. That is, L(A) is the set of strings w Σ such that δ (q 0, w) contains at least one accepting state. Example The NFA from previous slide accepts the language L = {w w ends in 01}. The proof is a mutual induction: δ (q 0, w) contains q 0 for every w. δ (q 0, w) contains q 1 iff w ends in 0. δ (q 0, w) contains q 2 iff w ends in 01. 0,1 q 0 0 q 1 1 q 2 Automata and Grammars NFA, λ transition, Two way DFA 3 September 19, / 294

52 Equivalence of Deterministic and Nondeterministic Finite Automata Definition (Subset Construction) The subset construction starts from an NFA N = (Q N, Σ, δ N, q 0, F N ). Its goal is the description of an DFA D = (Q D, Σ, δ D, {q 0 }, F D ) such that L(N) = L(D). Q D is the set of subsets of Q N, Q D = P(Q N ) (the power set). Inaccesible states can be thrown away so the number of states may be smaller. F D = {S : S P(Q N ) & S F N }, i.e. S include at least one accepting state of N. For each S Q N and for each input symbol a Σ, δ D (S, a) = p S δ N (p, a). Automata and Grammars NFA, λ transition, Two way DFA 3 September 19, / 294

53 Example of Subset Construction for language (0 + 1) 01 0,1 q 0 0 q 1 1 q {q 0 } {q 0, q 1 } {q 0 } {q 1 } {q 2 } {q 2 } {q 0, q 1 } {q 0, q 1 } {q 0, q 2 } {q 0, q 2 } {q 2 } {q 0, q 1, q 2 } {q 0, q 1 } {q 0, q 2 } 1 0 {q 0 } 0 {q 0, q 1 } 1 {q 0, q 2 } 0 1 Automata and Grammars NFA, λ transition, Two way DFA 3 September 19, / 294

54 Theorem (DFA for any NFA) If D = (Q D, Σ, δ D, {q 0 }, F D ) is the DFA constructed from NFA N = (Q N, Σ, δ N, q 0, F N ) by subset construction, then L(N) = L(D). Proof. By induction we prove: δ D ({q 0}, w) = δ N (q 0, w). Example (A Bad Case for the Subset Construction) A bad case for the subset construction is a language L(N) of all strings of 0 s and 1 s such that the nth symbol from the end is 1. Intuitively, a DFA must remember the last n symbols it has read. 0,1 1 0,1 0,1 0,1 0,1 {q 0 } q 1 q 2 q 3... q n Text search applications. Automata and Grammars NFA, λ transition, Two way DFA 3 September 19, / 294

55 Finite Automata With λ Transitions The new feature is that we allow a transition on λ, the empty string, that is without reading any input symbol. Example (λ transition NFA) (1) Any optional + or - sign, (2) a string of digits, (3) A decimal point, and (4) another string of digits. At least one of strings (2) and (4) must be nonempty. q 2 0,1,...,9. 0,1,...,9 λ, +, q 0 q 1 q 3. 0,1,...,9 0,1,...,9 q 4 λ q 5 Automata and Grammars NFA, λ transition, Two way DFA 3 September 19, / 294

56 Definition (λ-nfa) λ-nfa is E = (Q, Σ, δ, q 0, F ), where all components have their same interpretation as for NFA, except that δ is now a function that takes arguments Q (Σ {λ}). We require λ / Σ, so no confusion results. Example Previous λ-nfa is: E = ({q 0, q 1,..., q 5 }, {., +,, 0, 1,..., 9}, δ, q 0, {q 5 }), where λ +,-. 0,1,...,9 δ is: q 0 {q 1 } {q 1 } q 1 {q 2 } {q 1, q 4 } q 2 {q 3 } q 3 {q 5 } {q 3 } q 4 {q 3 } q 5 Automata and Grammars NFA, λ transition, Two way DFA 3 September 19, / 294

57 λ Closures Definition (λ-closure) For q Q we define the λ-closure λclose(q) recursively, as follows: State q is in λclose(q). If p λclose(q) and r δ(p, λ) then r λclose(q). For S Q we define λclose(s) = q S λclose(q). Automata and Grammars NFA, λ transition, Two way DFA 3 September 19, / 294

58 Extended Transitions and Languages for λ-nfa s Definition Suppose that E = (Q, Σ, δ, q 0, F ) is an λ-nfa. We define δ as follows: δ (q, λ) = λclose(q). Suppose w = xa where a Σ, x Σ. Let δ (q, x) = {p 1,..., p k }. Let k δ(pi, a) = {r1,..., rm}. i=1 Then δ (q, w) = λclose({r 1,..., r m}). Example δ (q, λ) = λclose(q 0 ) = {q 0, q 1 } δ (q, 5) =λclose( q δ (q,λ) δ(q, 5)) = λclose(δ(q 0, 5) δ(q 1, 5))= {q 1, q 4 } δ (q, 5.) = λclose(δ(q 1,.) δ(q 4,.)) ={q 2, q 3, q 5 δ (q, 5.6)= λclose(δ(q 2, 6) δ(q 3, 6) δ(q 5, 6)) = {q 3, q 5 } Automata and Grammars NFA, λ transition, Two way DFA 3 September 19, / 294

59 Definition (Eliminating λ-transition) Given any λ-nfa E = (Q E, Σ, δ E, q 0, F E ), we define a DFA D = (Q D, Σ, δ D, q D, F D ) that accepts the same language as E. Q D P(Q E ), S Q E : λclose(s) Q D. Note that may be in Q D. q D = λclose(q 0 ). F D = {S S is in Q D and S F E }. For S Q D, a Σ define δ D (S, a) = λclose( p S δ(p, a)). Theorem (Eliminating λ-transition) A language L is accepted by some λ-nfa if and only if L is regular. Automata and Grammars NFA, λ transition, Two way DFA 3 September 19, / 294

60 Proof. (IF) Just add δ(q, λ) =. (Only-if) Taking the D constructed above we prove L(D) = L(E) by induction on the length of w. w = 0, then w = λ, we know δe (q 0, λ) = λclose(q 0 ), q D = λclose(q 0 ). For DFA, p Q D : δd (p, λ) = p, therefore δd (q D, λ) = λclose(q 0 ) and δe (q 0, λ) = δd (q D, λ). Suppose w = xa, a Σ, x Σ, δ E (q 0, x) = δ D (q D, x). The recursive step is the same in definition of δ and in the definition of DFA eliminating λ transitions. Automata and Grammars NFA, λ transition, Two way DFA 3 September 19, / 294

61 Set operations on languages Definition (Set operations on languages) Let us have two languages L, M. We define following operations: (1) union L M = {w w L w M} Example: the language of words starting with a i or in the form b j c j. (2) intersection L M = {w w L & w M} Example: The language of words of even length that end by baa. (3) difference L M = {w w L&w / M} Example: the language of words of even length that do not end by baa. (4) complement L = L = {w w / L} = Σ L Example: the language of words that do not end by a Theorem (de Morgan rules) Proof from logic propeties of &,,. It holds: L M = L M L M = L M L M = L M. Automata and Grammars Kleene Theorem, Closure properties of R 4 September 19, / 294 L Σ M

62 Closure properties of regular languages (1) Theorem (Closure on set operations) Let us have regular languages L, M. Following languages are also regular: (1) the union L M (2) the intersection L M (3) the difference L M (4) the complement L = Σ L. Proof: RL closure on the complement Switch accepting and non accepting states of the DFA F = Q A F A. Example The language {w; w {0, 1} 01} 1 0 q 0 0 q 1 1 q Automata and Grammars Kleene Theorem, Closure properties of R 4 September 19, / 294

63 Product automaton Proof: Intersection, Union, Difference We construct the product automaton, Q = (Q 1 Q 2, Σ, δ((p 1, p 2 ), x) = (δ 1 (p 1, x), δ 2 (p 2, x)), (q 01, q 02 ), F ) intersection: F = F 1 F 2 union: F = (F 1 Q 2 ) (Q 1 F 2 ) difference: F = F 1 (Q 2 F 2 ). Product automaton example (intersection). Words containing 0,1, both. 1 1 p 0 q 0,1 pr 1 ps r 1 s 0,1 qr 1 qs 0,1 0 Automata and Grammars Kleene Theorem, Closure properties of R 4 September 19, / 294

64 Closure properties examples Example Construct an automaton accepting words, that contain 3k + 2 of 1 s and do not contain the substing 11. Example The direct construction is complicated. L 1 = {w w {0, 1} & w 1 = 3k + 2} L 2 = {w u, v {0, 1} &w = u11v} L = L 1 L 2. The language M containing non equal number of 0 s and 1 s is not regular. If M is regular, then M is also regular. We know M is not regular (pumping lemma). Automata and Grammars Kleene Theorem, Closure properties of R 4 September 19, / 294

65 String Operations on Languages Definition (String Operations on Languages) Let us have languages L, M. We define operations: concatenation of languages L.M = {uv u L&v M} L.x = L.{x} a x.l = {x}.l pro x Σ powers of languages L 0 = {λ} L i+1 = L i.l positive iteration L + = L 1 L 2... = i 1 Li (general) iteration L = L 0 L 1 L 2... = i 0 Li tedy L = L + {λ} reverse of the language L R = {u R u L} (=reverse) (x 1 x 2... x n ) R = x n x... x 2 x 1 left quotient L with M M \ L = {v uv L&u M} left derivation L with w w L = {w} \ L (right) quotient L with M L/M = {u uv L&v M} right derivation L with w w R L = L/{w}. Automata and Grammars Kleene Theorem, Closure properties of R 4 September 19, / 294

66 R Closed under Concatenation Theorem Assume L, M to be regular languages. The language L.M is also regular. Proof: We start with the DFA A 1 = (Q 1, Σ, δ 1, q 1, F 1 ) and A 2 = (Q 2, Σ, δ 2, q 2, F 2 ) such that L = L(A 1 ) and M = L(A 2 ). We define a nondeterministic NFA B = (Q {q 0 }, Σ, δ, q 0, F 2 ) where: Q = Q 1 Q 2 we assume different names of states, Q 1 Q 2 = we end after reading the word in the second language M δ(q 0, λ) = {q 1, q 2 } for q 1 F 1 that is λ L(A 1 ) δ(q 0, λ) = {q 1 } for q 1 / F 1 that is λ / L(A 1 ) δ(q 0, x) = for x Σ δ(q, x) = {δ 1 (q, x)} for q Q 1 & δ 1 (q, x) / F 1 we stay in A 1 = {δ 1 (q, x), q 2 } for q Q 1 & δ 1 (q, x) F 1 nondet. transit. from = {δ 2 (q, x)} for q Q 2 we stay in A 2. Then, L(B) = L(A 1 ).L(A 2 ). Automata and Grammars Kleene Theorem, Closure properties of R 4 September 19, / 294

67 R is Closed under Iteration Theorem Let us have a regular language L. The languages L, L + are also regular. Idea: repeated computation of A = (Q, Σ, δ, q 0, F ) that is: a nondeterministic decision whether restart or continue the evaluation a new state to accept λ L 0 (for L + without it or / F ). Proof: Proof for L Let us take a DFA A = (Q, Σ, δ, q 0, F ) such that L = L(A). We define nondeterministic NFA B = (Q {q B }, Σ, δ B, q B, F {q B }) where: δ B (q B, λ) = {q 0 } a new state q B to accept λ, we move to q 0 δ B (q B, x) = for x Σ δ B (q, x) = {δ(q, x)} if q Q & δ(q, x) / F inside A = {δ(q, x), q 0 } if q Q & δ(q, x) F restart is possible Then L(B) = L(A) (q B F B ), L(B) = L(A) + (q B / F B ). Automata and Grammars Kleene Theorem, Closure properties of R 4 September 19, / 294

68 R Closed under Reverse Theorem (R Closed under Reverse) Assume L is regular language. The language L R is also regular. Obviously (L R ) R = L therefore only one direction proof is needed. idea: reverse the edges in the state diagram; we get a NFA Proof: We start with the DFA A = (Q, Σ, δ, q 0, F ) such that L = L(A). We define a NFA B = (Q {q B }, Σ, δ B, q B, {q 0 }) where: δ B (q, x) = {p δ(p, x) = q} for q Q δ B (q B, λ) = F, δ B (q B, x) =. For any word w = x 1 x 2... x n q 0, q 1, q 2,..., q n is an accepting computation w in A q B, q n, q n 1,..., q 2, q 1, q 0 is an accepting computation w R in B. Remark. Sometimes L or L R has much simpler accepting automaton. Automata and Grammars Kleene Theorem, Closure properties of R 4 September 19, / 294

69 R Closed under Quotient Theorem If L, M are regular languages so is M \ L and L/M. Idea: A L, start in states reachable by a word in M Proof: We start with a DFA A L = (Q 1, Σ, δ 1, q 1, F 1 ) such that L = L(A 1 ). We define NFA B = (Q 1, Σ, δ 1, S, F 1 ) where: S = {q q Q 1 & u Mδ 1 (q 1, u)} can be found algorithmically (A q = (Q 1, Σ, δ 1, q 1, {q})), then q S L(A q) M ) v M \ L u Muv L u M q Q 1δ 1(q 1, u)&δ 1(q, v) F 1 q S&δ 1(q, v) F 1 v L(B) L/M = (M R \ L R ) R Automata and Grammars Kleene Theorem, Closure properties of R 4 September 19, / 294

70 Kleene Theorem, The Algebraic def. of Regular Languages Definition (RL Algebraic Description of Regular Languages) Assume Σ is a finite alphabet. We denote RL(Σ) the smallest set of languages that: contains the empty language for any letter x Σ RL contains the language {x} it is closed under union A, B RL(Σ) A B RL(Σ) it is closed under concatenation A, B RL(Σ) A.B RL(Σ) it is closed under iteration A RL(Σ) A RL(Σ) Specifically: {λ} RL(Σ) Σ RL(Σ) Σ RL(Σ) since {λ} = since X = x X {x} (finite union) Theorem (Kleene) Any language can be recognized by a deterministic finite automaton if and only if it is in RL (over the same alphabet). Automata and Grammars Kleene Theorem, Closure properties of R 4 September 19, / 294

71 Kleene Theorem Proof 1 Proof: RL recognizable by a DFA trivial languages are recognizable by a DFA regular languages are closed under operations union, concatenation, iteration (previous theorems). Automata and Grammars Kleene Theorem, Closure properties of R 4 September 19, / 294

72 Kleene Theorem Proof 2 Proof: Recognizable by a DFA RL We have an automaton A = (Q, Σ, δ, q 0, F ) accepting the language L(A). We ame to get L(A) from elementary languages and operations. we define R ij = {w Σ δ (q i, w) q j } strings moving the state q i to q j then L(A) = q f F R 0f words transferring q 0 to some accepting state q f RL is closed under union. We need to prove R ij RL(Σ). We define Rij k = strings transferring q i to j and avoiding states q m : m > k R 1 ij RL(Σ) (no intermediate states, strings of length max. 1) = Rij k R k i(k+1).(rk (k+1)(k+1) ).R k (k+1)j RL(Σ) (RL closed under union, concatenation, iteration) R k+1 ij R ij = R n ij where n is the number of states Q, therefore R ij RL(Σ) and L(A) = q f F R 0i RL(Σ). i k+1 j Automata and Grammars Kleene Theorem, Closure properties of R 4 September 19, / 294

73 Alternative Proof of Kleene Theorem Proof: Recognizable by a DFA RL We have an nondeterministic automaton A = (Q, Σ, δ, q 0, F ) accepting the language L(A). By the induction on the number of edges of A we prove L(A) RL(Σ). no edges only languages and {λ}, from the definition and. (n + 1) edges select one edge p a q, that is q δ(p, a) we construct four automata without this edge (δ ) A 1 = (Q, Σ, δ, S, F ) A 2 = (Q, Σ, δ, S, {p}) A 3 = (Q, Σ, δ, {q}, {p}) A 4 = (Q, Σ, δ, {q}, F ) 2 a 4 Then L(A) = L(A 1) (L(A 2).a).(L(A 3).a) L(A 4), languages L(A 1), L(A 2), L(A 3), L(A 4) RL(Σ) by induction hypothesis (n edges). 1 3 Automata and Grammars Kleene Theorem, Closure properties of R 4 September 19, / 294

74 Regular Expressions (RegE) Definition (Regular Expression (RegE), value of a RegE L(α)) Regular expressions α, β RegE(Σ) over a finite non empty alphabet Σ = {x 1, x 2,..., x n } and their value L(α) is defined by induction: Basis: expression α for value L(α) [α] λ empty string L(λ) = {λ} empty expression L( ) = {} a a Σ L(a) = {a}. Induction: expression value remark α + β L(α + β) = L(α) L(β) αβ L(αβ) = L(α)L(β). may be used α L(α ) = L(α) (α) L((α)) = L(α) brackets do not change the value. The class of regular expressions over Σ: RegE(Σ) is the smallest class closed under operations above. Automata and Grammars Regular Expressions, Moore and Mealy Machine, 2 way FA 5 September 19, / 294

75 Examples, Precedence Example (Regular Expressions) The language of alternating 0 s and 1 s may be written: either (01) + (10) + 1(01) + 0(10) or (λ + 1)(01) (λ + 0). The language L(( ) 0 ) = {w w {0, 1}, w 1 = 3k, k 0}. Definition (Precedence) The star is the operator with highest precedence, then concatenation., the lowest precedence has the union +. Theorem (RegE and DFA) Any language recognizable by a DFA can be expressed by a regular expression. Any language of a regular expression can be recognized by a λ-nfa (therefore also a DFA). Automata and Grammars Regular Expressions, Moore and Mealy Machine, 2 way FA 5 September 19, / 294

76 From DFA to Regular Expressions Compare to Kleene Theorem. R (k+1) ij = R (k) ij + R (k) i(k+1) (R(k) (k+1)(k+1) ) R (k) (k+1)j Finally, RegE = j FA R (n) 1j union over accepting states j. Automata and Grammars Regular Expressions, Moore and Mealy Machine, 2 way FA 5 September 19, / 294

77 From Regular Expression to an λ NFA automaton From Regular Expression to an λ NFA automaton By structural induction on R. In each step, we construct an λ-nfa E recognizing the language L(R) = L(E) with additional properties: Exactly one accepting state. No edges into the initial state. No edges from the accepting state. Union R + S: λ Basis: λ λ a empty string λ empty set a single letter a Σ a λ R λ INDUCTION: λ Concatenation RS: R S Closure R : λ S λ R λ λ Automata and Grammars Regular Expressions, Moore and Mealy Machine, 2 way FA 5 September 19, / 294

78 Converting Among Representations Converting NFA to DFA λ closure in O(n 3 ). Search n states multiplied by n 2 arcs for λ transitions. Subset construction, DFA with possibly 2 n states. For each state, O(n 3 ) time to compute transitions. Converting DFA to NFA λ NFA O(n) RE O(n 3 2 n ) O(n 3 4 n ) O(n) O(n 3 2 n ) DFA NFA O(n) Just modify transition table by putting set-brackets around states and adding column for λ in the case of λ NFA. Automaton to Regular Expression Conversion O(n 3 4 n ) (see construction in Section 3.2.1) RE to Automaton Conversion λ NFA in the time O(n). Automata and Grammars Regular Expressions, Moore and Mealy Machine, 2 way FA 5 September 19, / 294

79 Automata with the output Definition (Moore machine) Moore machine is a sixtuple A = (Q, Σ, Y, δ, µ, q 0 ) consisting of Q non empty set of states Σ finite nonempty set of symbols (input alphabet) Y finite nonempty set of symbols (output alphabet) δ a mapping Q Σ Q (transition function) µ a mapping Q Y (output function) q 0 Q (initial state) the output function may imitate final states F Q may be replaced by output function µ : Q {0, 1} as follows: µ(q) = 0 if q / F, µ(q) = 1 if q F. Automata and Grammars Regular Expressions, Moore and Mealy Machine, 2 way FA 5 September 19, / 294

80 Moore Machine Example Example (Tennis Game Score) A machine calculates the tenis score. Input alphabet: ID of the player who scored a point Output alphabet & states: the score ( Q = Y and µ(q) = q) State/output A B 00:00 15:00 00:15 15:00 30:00 15:15 15:15 30:15 15:30 00:15 15:15 00:30 30:00 40:00 30:15 30:15 40:15 30:30 30:30 40:30 30:40 15:30 30:30 15:40 00:30 15:30 00:40 40:00 A 40:15 40:15 A 40:30 40:30 A deuce 30:40 deuce B 15:40 30:40 B 00:40 15:00 B deuce A:40 40:B A:40 A deuce 40:B deuce B A 15:00 00:15 B 15:00 00:15 Automata and Grammars Regular Expressions, Moore and Mealy Machine, 2 way FA 5 September 19, / 294

81 Mealy machine Definition (Mealy machine) Mealy machine is a six tuple A = (Q, Σ, Y, δ, λ M, q 0 ) consisting of: Q non empty set of states Σ finite nonempty set of symbols (input alphabet) Y finte nonempty set of symbols (output alphabet) δ a mapping Q Σ Q (transition function) µ a mapping Q Σ Y (output function) q 0 Q (initial state) The output is determined by a state and the input symbol Mealy machine is more general then Moore The output function may be replaced as follows x Σ λ M (q, x) = µ(q) or x Σ λ M (q, x) = µ(δ(q, x)) Automata and Grammars Regular Expressions, Moore and Mealy Machine, 2 way FA 5 September 19, / 294

82 Mealy Machine Example Example (Mealy Machine) The automaton for integer division of the input by 8 (the reminder is discarded). Three bit move to the left we need to remember last three bits three bit dynamic memory. State\symbol /0 001/ /0 011/ /0 101/ /0 111/ /1 001/ /1 011/ /1 101/ /1 111/1 0/0 1/0 1/0 1/ / /0 0/1 0/ /0 1/1 1/0 After three steps calculates properly non regarding the initial state. 0/1 1/1 0/0 1/1 0/1 Automata and Grammars Regular Expressions, Moore and Mealy Machine, 2 way FA 5 September 19, / 294

83 Extended Output Function for any word in the input alphabet Σ we get a word in the output alphabet Y Moore machine output function µ : Q Y µ : Q Σ Y µ (q, λ) = λ (někdy µ (q, λ) = q) µ (q, wx) = µ (q, w).µ(δ (q, wx)) Example: µ (00:00,AABA)=(00:00.) 15:00. 30:00. 30:15. 40:15 Mealy machine output function λ M : Q Σ Y λ M : Q Σ Y λ M(q, λ) = λ λ M(q, wx) = λ M(q, w).λ M (δ (q, w), x) Example: λ M(000, )= a b c d x y u u v a b c d x y u u Automata and Grammars Regular Expressions, Moore and Mealy Machine, 2 way FA 5 September 19, / 294

84 Lemma (Moore and Mealy Machines Reductions) Proof. For any Moore machine there exists a Mealy machine mapping each input word to the same output word. For any Mealy machine there exists a Moore machine mapping each input word to the same output word. a/x q Mealy machine B = (Q, Σ, Y, δ, λ M, q 0 ) where λ M (q, x) = µ(δ(q, x)) We define states of the Moore machine Q Y, δ ([q, y], x) = [δ(q, x), λ(q, x)]), µ([q, y]) = y. b/y a q/x q/y b Automata and Grammars Regular Expressions, Moore and Mealy Machine, 2 way FA 5 September 19, / 294

85 Further Generalisation Finite automaton makes following actions: read a symbol changes its state moves its reading head to the right The head is not allowed to move to the left. What happens, if we allow the head to move left and right? The automaton does not write anything on the tape! Automata and Grammars Regular Expressions, Moore and Mealy Machine, 2 way FA 5 September 19, / 294

86 Two way finite automata Definition (Two way finite automata) Two way deterministic finite automaton is a five tuple A = (Q, Σ, δ, q 0, F ), where Q is a finite set of states, Σ is a finite set of input symbols transition function δ is a mapping Q Σ Q { 1, 0, 1} extended by head transitions q 0 Q initial state a set of accepting states F Q. We may represent it by a graph or a table. Automata and Grammars Regular Expressions, Moore and Mealy Machine, 2 way FA 5 September 19, / 294

87 Two way DFA computation Definition (Two way DFA computation) A string w is accepted by the two way DFA, iff: computation started in the initial state at the left most symbol of w the first transition from w to the right was in an accepting state the computation is not defined outside the word w (computation ends without accepting w). We may add special end symbols # / Σ to any word If L(A) = {#w# w L Σ } is regular, then also L is regular L = # R # (L(A) #Σ #) Automata and Grammars Regular Expressions, Moore and Mealy Machine, 2 way FA 5 September 19, / 294

88 Two way automaton example Example (Two way automaton example) Let A = (Q, Σ, δ, q 1, F ). We define a two way DFA B = (Q Q Q {q 0, q N, q F }, Σ, δ, q 0, {q F }) accepting the language L(B) = {#u# uu L(A)} (it is neither left nor right quotient!): δ x Σ # remark q 0 q N, 1 q 1, +1 q 1 is starting in A q p, +1 q, 1 p = δ(q, x) q q, 1 q, +1 q p, +1 q F, +1 q F, p = δ(q, x) q p, +1 q N, +1 q / F, p = δ(q, x) q N q N, +1 q N, +1 q F q N, +1 q N, +1 Theorem Languages accepted by two way DFA are exactly regular languages. Automata and Grammars Regular Expressions, Moore and Mealy Machine, 2 way FA 5 September 19, / 294

89 Two way DFA and Regular Languages Proof: DFA two way DFA To a DFA we add the move of the head to the right A = (Q, Σ, δ, q 0, F ) 2A = (Q, Σ, δ, q 0, F ), where δ (q, x) = (δ(q, x), +1). For the other direction, we need introduction. The influence of u Σ on the computation over v Σ the first time we leave u to the right we leave v to the left and return back v Automata and Grammars Regular Expressions, Moore and Mealy Machine, 2 way FA 5 September 19, / 294

90 Function f u describing computation two way DFA over u Algorthm: Function f u describing computation two way DFA over u We define f u : Q {q 0 } Q {0} f u (q 0 ) the state of the first transition to the right in case the computation begins left in the state q 0, f u (p); p Q the state of the right transition in case the computation begins right in p the symbol 0 denotes failure (a cycle or the head moves left from the initial symbol) We define similarity on strings: u w def f u = f w, strings are similar iff they define identical function f Languages recognized by two way DFA are regular Similarity is a right congruence with a finite index. According to Myhill Nerode theorem is the language L(A) regular. Automata and Grammars Regular Expressions, Moore and Mealy Machine, 2 way FA 5 September 19, / 294

91 Constructive proof We need the left right movement transcript to a linear computation. we are interested in accepting computations only. We focus on transitions in cuts between input symbols Observations: The direction of movement repeats (right, left) the first and the last transitions are to the right automaton is deterministic, any accepting computation is without cycles the first and the last cut contain only one state. Algorthm: 2DFA NFA Find all posssible cuts state sequences (its a finite number). Define non deterministic transition between cuts according to the input symbol. We re construct the computation by composing cuts like a puzzle. Automata and Grammars Regular Expressions, Moore and Mealy Machine, 2 way FA 5 September 19, / 294

92 Formal reduction two way DFA to NFA Algorthm: Formal reduction two way DFA to NFA Let A = (Q, Σ, δ, q 0, F )be a two way DFA. We define an equivalent NFA B = (Q, Σ, δ, (q 0 ), F ) as follows: Q all possible correct transition sequences sequences of states (q 1,..., q k ); q i Q with an even length (k = 2m + 1) no state repeats at odd nor at even position ( i j) (q 2i q 2j )&( i j) (q 2i+1 q 2j+1 ) F = {(q) q F } sequences of the length 1 δ (c, x) = {d d Q &c d is locally consistent transition for x} otočky: δ(c i, x) = (c i+1, 1) a δ(d i, x) = (d i+1, +1) zbytek c z = d z & ( i = 2m + 1 c ) δ(c i, x) = (d i, +1) & ( i = 2m c ) δ(d i, x) = (c i, 1). L(A) = L(B) Trajektory two way DFA A corresponds to cuts in NFA B, therefore L(A) = L(B). Automata and Grammars Regular Expressions, Moore and Mealy Machine, 2 way FA 5 September 19, / 294

93 Example Reduction Two way DFA to NFA Let us have two way DFA: a b p p,+1 q,+1 q q,+1 r,-1 r p,+1 r,-1 Possible cuts and their transitions leftwards only r all even positions r, that means only one even position possible cuts: (p), (q), (p, r, q), (q, r, p). a b (p) (p) (q) (q) (q),(q,r,p) (p,r,q) (q,r,p) (q) Non accepting computation example: a a b a a b a a b b p p p q q q r p q q q r r p q r p q.. Resulting NFA: a a a b (p) (q) (q,r,p) Automata and Grammars Regular Expressions, Moore and Mealy Machine, 2 way FA 5 September 19, / 294 b

94 Finite Automata Summary Finite Automata DFA, minimal DFA NFA 2 n, λ NFA, two way FA n n Regular Expressions Automata and Languages regular languages closed under set operations closed under string operations closed under substitution, homomorphism, inverse homomorphism all finite automata and regular expressions describe the same class of languges. Key theorems Mihyll Nerode theorem (kongruence) Kleene theorem (elementar languages and operations) Pumping lemma. Automata with the output Moore machine Mealy machine. Automata and Grammars Regular Expressions, Moore and Mealy Machine, 2 way FA 5 September 19, / 294

95 Pattern search in the text Static text: we create indexes rather than using RegE. RegE are useful in the dynamic text (like news). Example (Search for streets in addresses on the web) Street identification the name before house number all together We are missing: Bouleward, Place, Way Streen St\. Avenue Ave\. Road Rd\ [A-Z][a-z]*( [A-Z][a-z]*)* [0-9]+[A-Z]? [0-9]+[A-Z]? [A-Z][a-z]*( [A-Z][a-z]*)* Streen St\. Avenue Ave\. Road Rd\. Streets without any identifier (almost all Czech streets) Street names with numbers.... Automata and Grammars Regular Expressions, Moore and Mealy Machine, 2 way FA 5 September 19, / 294

96 String Substitution, (String) Homomorphism Definition (String Substitution, (String) Homomorphism) We have a finite alphabet Σ. For each x Σ we have σ(x) a language over the alphabet Y x. Further, we define: σ(λ) = {λ} σ(u.v) = σ(u).σ(v) The mapping σ : Σ P(Y ), kde Y = x Σ Y x is substitution. σ(l) = w L σ(w) e free, λ free substitution is a substitution where none σ(x) contains λ. For w = a 1... a n Σ n h(w) = h(a 1 )... h(a n ). Further, h(l) = {h(w) w L}. Example (substitution) σ(0) = {a i b j, i, j 0}, σ(1) = {cd} σ(010) = {a i b j ca k b l, i, j, k, l 0} Automata and Grammars Regular Expressions, Moore and Mealy Machine, 2 way FA 5 September 19, / 294

97 (String) Homomorphism Definition ((String) Homomorphism) Homomorphism h is a special case of a substitution where h(x) = w x x Σ. If x : w x λ is is e free (λ free) homomorphism. Inverse homomorphism h 1 (L) = {w h(w) L}. Example (homomorphism) The function h defined by: h(0) = ab, and h(1) = λ is a homomorphism. For example, h(0011) = abab. For L = 10 1 is h(l) = (ab). Theorem (Closure under homomorphism) If language L and all x Σ σ(x) are regular, so is also σ(l), h(l), h 1 (L). Automata and Grammars Regular Expressions, Moore and Mealy Machine, 2 way FA 5 September 19, / 294

98 Notes to Closure Properties Simplification of the automata design L. =.L = {λ}.l = L.{λ} = L (L ) = L (L 1 L 2 ) = L 1(L 2.L 1) = L 2(L 1.L 2) (L 1.L 2 ) R = L R 2.L R 1 w (L 1 L 2 ) = w (L 1 ) w (L 2 ) w (Σ L) = Σ w L h(l 1 L 2 ) = h(l 1 ) h(l 2 ) Proof of non regularity L = {w w {0, 1}, w 1 = w 2} is not regular since L {0 i 1 j i, j 0} = {0 i 1 i i 0} is not regular (pumping lemma). Automata and Grammars Regular Expressions, Moore and Mealy Machine, 2 way FA 5 September 19, / 294

99 Homomorphism preserves regularity Theorem If L is a regular language over alphabet Σ, and h is a homomorphism on Σ, then h(l) is also regular. Proof. Let L = L(R) for some regular expression R. The proof is done by structural induction on subexpressions E of R: we claim L(h(E)) = h(l(e)). Basis: h({λ}) = λ, h( ) =. If E = a then L(E) = {a}, so h(l(e)) = {h(a)}. Thus, L(h(E)) = {h(a)}. Induction: Union: L(h(F + G)) = L(h(F ) + h(g)) = L(h(F )) L(h(G)) and h(l(f + G)) = h(l(f ) L(G)) = h(l(f )) h(l(g)). Right sides are equal from inductive hypothesis therefore left sides also equal. concatenation, closure proofs are similar. Automata and Grammars Regular Expressions, Moore and Mealy Machine, 2 way FA 5 September 19, / 294

100 Inverse Homomorphism Definition (Inverse homomorphism) Suppose h is a homomorphism from some alphabet Σ to strings in another alphabet T. Then h 1 (L) h inverse of L is the set of strings w in Σ such that h(w) is in L. Example Let L = (00 + 1), h(a) = 01 and h(b) = 10. We claim h 1 (L) = (ba). Proof: h((ba) ) L is easy to see. Other w generates isolated 0 (4 cases to consider). A homomorphism applied in the forward and inverse direction. Automata and Grammars Regular Expressions, Moore and Mealy Machine, 2 way FA 5 September 19, / 294

101 Inverse Homomorphism DFA Theorem If h is a homomorphism from alphabet Σ to alphabet T, and L is a regular language over T, then h 1 (L) is also a regular language. Proof. The proof starts with a DFA A for L. We construct a DFA for h 1 (L). For A = (Q, T, δ, q 0, F ) we define B(Q, Σ, γ, q 0, F ) where γ(q, a) = δ (q, h(a)) (δ operates on strings). By induction on w, ˆγ(q 0, w) = δ (q 0, h(w)). Therefore, B accepts exactly those strings w that are in h 1 (L). The DFA for h 1 (L) applies h to ints input, and then simulates the DFA for L. Automata and Grammars Regular Expressions, Moore and Mealy Machine, 2 way FA 5 September 19, / 294

102 Visit every state example Example Suppose A = (Q, Σ, δ, q 0, F ) is an DFA. The language L of all strings w in Σ such that δ (q 0, w) is in F and for every state q Q there is some prefix x q of w such that δ (a 0, x q ) = q. This language is regular. M = L(A) the language accepted by DFA A in the usual way. We define a new alphabet T of triples {[paq]; p, q Q, a Σ, δ(p, a) = q}. We define the homomorphism h([paq]) = a for all p, q, a. Language L 1 = h 1 (M) is regular since M is regular (DFA and inverse homomorphism). h 1 (101) includes 2 3 = 8 strings, like [p1p][q0q][p1p] {[p1p], [q1q]}{[p0q], [q0q]}{[p1p], [q1q]}. We construct L from L 1 (next slide). 1 p 0 q 0,1 Automata and Grammars Regular Expressions, Moore and Mealy Machine, 2 way FA 5 September 19, / 294

103 Enforse start at q 0. Define E 1 = a Σ,q Q {[q 0aq]} = E 1 = {[q 0 a 1 q 0 ], [q 0 a 2 q 1 ],..., [q 0 a m q n ]}. Then, L 2 = L 1 L(E 1.T ). Adjacent states must equal. Define non-matching pairs E 2 = q r,p,q,r,s Q,a,b Σ {[paq][rbs]}. Define L 3 = L 2 L(T.E 2.T ), It ends in accepting state since we started from M language of accepting computations on the DFA A. All states. For each state q Q, define E q be the regular expression that is the sum of all the symbols in T such that q appears in neither its first or last position. We substract L(E q ) from L 3. L 4 = L 3 q Q {E q }. Remove states, leave symbols. L = h(l 4 ). We conclude L is regular. Constructing language L from language M by applying operations that preserve regularity of languages. Automata and Grammars Regular Expressions, Moore and Mealy Machine, 2 way FA 5 September 19, / 294

104 Decision Properties of Regular Languages Lemma (Testing Emptiness of Regular Languages) For finite automatons, it is a question of graph reachability of any final state from the initial one. Reachability is O(n 2 ). Lemma For regular expression, we can convert it to λ NFA in O(n) time and than check reachability. It can be done also by direct inspection: Basis: denotes empty language; λ and a are not empty. Induction: R = R 1 + R 2 is empty iff both L(R 1) and L(R 2) are empty. R = R 1R 2 is empty iff either L(R 1) or L(R 2) is empty. R = R 1 is never empty, in includets λ. R = (R 1) is empty iff R 1 is empty, since they are the same language. Automata and Grammars Regular Expressions, Moore and Mealy Machine, 2 way FA 5 September 19, / 294

105 Testing Membership in a Regular Language Given a string w; w = n and a regular language L, is w L? DFA: Run automaton; if w = n, suitable representation, constant time transitions, it is O(n). NFA with s states: running time O(ns 2 ).Each input symbol can be processed by taking the previous set of states, which numbers at most s states. λ NFA - first compute λ closure. Then, for each symbol proceed it and compute λ closure of the result. For a regular expression of size s we convert it to an λ NFA with at most 2s states and then simulate, taking O(ns 2 ). Automata and Grammars Regular Expressions, Moore and Mealy Machine, 2 way FA 5 September 19, / 294

106 Context-Free Grammars and Languages More powerful than finite automata. Used to describe document formats, via DTD - document-type definition used in XML (extensible markup language) We introduce context-free grammar notation parse tree. There exist an pushdown automaton that describes all and only the context-free languages. Will be described later. Automata and Grammars Grammars 6 September 19, / 294

107 Palindrome example A string the same forward and backward, like otto or Madam, I m Adam. w is a palindrome iff w = w R. The language L pal of palindromes is not a regular language. A context-free grammar for We use the pumping lemma. palindromes If L pal is a regular language, let n be the 1. P λ asssociated constant, and consider: 2. P 0 w = 0 n 10 n. 3. P 1 For regular L, we can break w = xyz such 4. P 0P0 that y consists of one or more 0 s from the first group. Thus, xz would be also in L pal 5. P 1P1 if L pal were regular. A context-free grammar (right) consists of one or more variables, that represent classes of strings, i.e., languages. Automata and Grammars Grammars 6 September 19, / 294

108 Definition (Grammar) A Grammar G = (V, T, P, S) consists of Finite set of terminal symbols (terminals) T, like {0, 1} in the previous example. Finite set of variables V (nonterminals,syntactic categories), like {P} in the previous example. Start symbol S is a variable that represents the language being defined. P in the previous example. Finite set of rules (productions) P that represent the recursive definition of the language. Each has the form: αaβ ω, A V, α, β, ω (V T ) notice the left side (head) contains at least one variable. The head - the left side, the production symbol, the body - the right side. Definition (Context free grammar CFG) Context free grammar (CFG) je G = (V, T, P, S) has only productions of the form A α, A V, α (V T ). Automata and Grammars Grammars 6 September 19, / 294

109 Chomsky hierarchy Grammar types according to productions allowed. Type 0 (recursively enumerable languages L 0 ) general rules α β, α, β (V T ), α contains at least one variable Type 1 (context languages L 1 ) productions of the form αaβ αωβ A V, α, β (V T ), ω (V T ) + with only exception S λ, then S does not appear at the right side of any production Type 2 (context free languages L 2 ) productions of the form A ω, A V, ω (V T ) Type 3 (regular (right linear) languages L 3 ) productions of the form A ωb, A ω, A, B V, ω T Automata and Grammars Grammars 6 September 19, / 294

110 Chomsky hierarchy The classes of languages are ordered later we show proper inclusions L 0 L 1 L 2 L 3 L 0 L 1 L 2 L 3 L 0 L 1 recursively enumerable contain context free productions αaβ αωβ have variable A in the head L 2 L 3 context free contain regular languages productions A ωb, A ω have in the body a string (V T ) L 1 L 2 context contain context free languages we have to eliminate rules A λ, we can do it (later). Automata and Grammars Grammars 6 September 19, / 294

111 Derivations Using a Grammar Definition (One step derivation) Suppose G = (V, T, P, S) is grammar. Let α, ω, η, ν (V T ). Let α ω be a production rule of G. Then one derivation step is: ηαν G ηων or just ηαν ηων. We extend to any number of derivation steps as follows. Definition (Derivation ) Let G = (V, T, P, S) is CFG. Basis: For any string α (V T ) it derives itself, α G α. Induction: If α G β and β G γ then α G γ. If grammar G is understood, then we use in place of G. Example (derivation E a (a + b00)) E E E I E a E a (E) a (E +E) a (I +E) a (a +E) a (a + I) a (a + I0) a (a + I00) a (a + b00) Automata and Grammars Grammars 6 September 19, / 294

112 The Language of a Grammar, Notation Convention for CFG Derivations a, b, c terminals A, B, C variables w, z strings of terminals X, Y either terminals or variables α, β, γ strings of terminals and/or variables. Definition (The Language of a Grammar) Let G = (V, T, P, S) is CFG. The language L(G) of G is the set of terminal strings that have derivations from the start symbol. L(G) = {w T S G w} Language of a variable A V is defined L(A) = {w T A G w}. Example (Not CFL example) L = {0 n 1 n 2 n n 1} is not context-free, there does not exist CFG grammar recognizing it. Automata and Grammars Grammars 6 September 19, / 294

113 Type 3 grammars and regular languages productions has the form A wb, A w, A, B V, w T an example of derivation: P : S 0S 1A λ, A 0A 1B, B 0B 1S S 0S 01A 011B 0110B 01101S Observations: each word contains exactly one variable (except the last one) the variable is always on the rightmost position the production A w is the last one of the derivation any step generates terminal string and (possibly) changes the variable The relation of the grammar and a finite automata variable = state of the finite automata productions = transition function Automata and Grammars Grammars 6 September 19, / 294

114 Example of the reduction FA to a grammar Example (G, FA binary numbers divisible by 5) L = {w w {a, b} &w binary numbers divisible by 5} 0 C 2 E 4 1 A 1B 0A λ B 0C 1D C 0E 1A D 0B 1C E 0D 1E A B D Derivation examples A 0A 0 (0) A 1B 10C 101A 101 (5) A 1B 10C 101A 1010A 1010 (10) A 1B 11D 111C 1111A 1111 (15) Automata and Grammars Grammars 6 September 19, / 294

115 FA to Grammar reduction Theorem (L RE L L 3 ) For any language recognized by a finite automata there exists a grammar Type 3 recognizing the language. Proof: FA to Grammar reduction L = L(A) for some automaton A = (Q, Σ, δ, q 0, F ). We define a grammar G = (Q, Σ, P, q 0 ), with productions P p aq, iff δ(p, a) = q p λ, iff p F Is L(A) = L(G)? λ L(A) q 0 F (q 0 λ) P λ L(G) a 1... a n L(A) q 0,..., q n Q tž. δ(q i, a i+1) = q i+1, q n F (q 0 a 1q 1... a 1... a nq n a 1... a n) is derivation of a 1... a n a 1... a n L(G) Automata and Grammars Grammars 6 September 19, / 294

116 We aim to construct Grammar to FA reduction Opposite direction production A ab are encoded to transition function productions A λ define the accepting states we rewrite productions A a 1... a nb, A a 1... a n with more terminals we introduce new variables H 2,..., H n define productions A a 1 H 2, H 2 a 2 H 3,..., H n a nb or A a 1 H 2, H 2 a 2 H 3,..., H n a n productions A B correspond to λ transitions Lemma For any Type 3 grammar there exist a Type 3 grammar with the same languages with all productions of the form: A ab, A B, A λ, A, B V, a T. Automata and Grammars Grammars 6 September 19, / 294

117 Standard form of a grammar Type 3 Lemma For any Type 3 grammar there exist a Type 3 grammar with the same languages with all productions of the form: A ab, A B, A λ, A, B V, a T. Proof. We define G = (V, T, S, P ), for each rule we introduce set of new variables Y 2,..., Y n, Z 1,..., Z n and define P P A ab A ab A λ A λ A a 1... a n B A a 1 Y 2, Y 2 a 2 Y 3,... Y n a n B Z a 1... a n Z a 1 Z 1, Z 1 a 2 Z 2,..., Z n 1 a n Z n, Z n λ we may eliminate also rules: A B transitive closure U(A) = {B B V &A B} A w for all Z U(A) and (Z w) P Automata and Grammars Grammars 6 September 19, / 294

118 Theorem (Reduction Type 3 grammar to a λ NFA) For any language L of a Type 3 grammar there exists a λ NFA recognizing the same language. Proof: Reduction Type 3 grammar to a λ NFA We take a grammar G = (V, T, P, S) with all productions of the form A ab, A B, A λ, A, B V, a T generating L (previous lemma) we define a non deterministic λ NFA A = (V, T, δ, {S}, F ) where: F = {A (A λ) P} δ(a, a) = {B (A ab) P} δ(a, λ) = {B (A B) P} L(G) = L(A) λ L(G) (S λ) P S F λ L(A) a 1... a n L(G) there exists a derivation (S a 1H 1... a 1... a nh n a 1... a n) H 0,..., H n V tak že H 0 = S, H n F H i+1 δ(h i, a k ) for the step a 1... a k 1 H i a 1... a k 1 a k H i+1 H i+1 δ(h i, λ) for the step a 1... a k H i a 1... a k H i+1 a 1... a n L(A) Automata and Grammars Grammars 6 September 19, / 294

119 Left (and right) linear grammars Definition (Left (and right) linear grammars) Type 3 grammars are also called right linear (the variable is always at right). A grammar G is left linear iff all production has the form A Bw, A w, A, B V, w T. Lemma Languages generated by left linear grammars are exactly regular languages. Proof: by reversing productions we get right linear grammar A Bw, A w we reduce to A w R B, A w R the new grammar generates L R we know regular languages are closed under reverse, L R is regular, so is L = (L R ) R any regular language can be expressed in this form (FA reverse right linear grammar left linear grammar) Automata and Grammars Grammars 6 September 19, / 294

120 Linear grammars and languages Left and right linear grammars together are stronger. Definition (linear grammar, language) A grammar is linear iff all productions have the form A ubw, A w, A, B V, u, w T (at most one variable in the body). Linear languages are languages generated by linear grammars. Obviously: regular languages linear languages. It is a proper subset. Example (linear, non regular language) The language L = {0 n 1 n n 1} is not regular but it is linear. It is generated by the grammar S 0S1 01. Observation: linar rules can be splitted to left linear and right linear rules: S 0A, A S1. Automata and Grammars Grammars 6 September 19, / 294

121 A context-free grammar for simple expressions Example (CFG for simple expressions) A grammar for (simple) expression is G = ({E, I}, {+,, (, ), a, b, 0, 1}, P, E) where P is the set of rules right. Rules 1-4 describe expression. Rules 5-10 describe identifiers I, correspond to regular expression (a + b)(a + b ). CFG for simple expressions 1. E I 2. E E + E 3. E E E 4. E (E) 5. I a 6. I b 7. I Ia 8. I Ib 9. I I0 10. I I1 Automata and Grammars Grammars 6 September 19, / 294

122 Leftmost and Rightmost Derivations Definition (Leftmost and Rightmost Derivation) Leftmost derivation lm, lm replaces at each step the leftmost variable by on of its production bodies. Rightmost derivation rm, rm replaces at each step the rightmost variable by on of its production bodies. Example (leftmost derivation) E lm E E lm I E lm a E lm a (E) lm a (E + E) lm a (I + E) lm a (a + E) lm lm a (a + I) lm a (a + I0) lm a (a + I00) lm a (a + b00) Example (rightmost derivation) E rm E E rm E (E) rm E (E + E) rm E (E + I) rm E (E + I0) rm E (E + I00) rm E (E + b00) rm rm E (I + b00) rm E (a + b00) rm I (a + b00) rm a (a + b00) Automata and Grammars Grammars 6 September 19, / 294

123 The Language of a Grammar Theorem L(G pal ), where G pal is the set of palindromes over {0, 1}. Proof. IF: Suppose w is a palindrome. Induction on w that w is in L(G pal ). BASIS: If w = 0 or w = 1, then w is λ, 0 or 1. We have production rules P λ,p 0,P 1, therefore P w in any basis case. INDUCTION: Suppose w 2. Since w = w R, so w = 0x0 or w = 1x1. Moreover, x must be a palindrome, x = x R. If w = 0x0, then we invoke the inductive hypothesis to claim P x. Then: P 0P0 0x0 = w. If w = 1x1, do it yourself. ONLY-IF We assume w L(G pal ), that is P w. We claim that w is a palindrome. Induction on the number of steps in a derivation of w from P. BASIS: λ, 0, 1 are palindromes. INDUCTION: Derivation P 0P0 0x0 = w. By inductive hypothesis x is a palindrome. Therefore, 0x0 and 1x1 are also palindromes. Automata and Grammars Grammars 6 September 19, / 294

124 Sentential Forms Definition (Sentential Forms) Let G = (V, T, P, S) is CFG. Any string α (V T ) such that S α is an sentential form. If S lm α, then α is a left sentential form, if S rm α, then α is a right sentential form. Example The string E (I + E) is an sentential form but is neither left nor right sentential form. Automata and Grammars Grammars 6 September 19, / 294

125 Parse Trees The tree is the data structure of choice to represent the source program in a compiler. The structure facilitates the translation into executable code. Definition (Parse Tree) Let us fix on a grammar G = (V, T, P, S). The parse trees for G are trees such that: Each interior node is labeled by a variable in V. Each leaf is labeled V T {λ}. If is labeled λ, it must be the only child of its parent. If an interior node is labeled A and its children X 1,..., X k then (A X 1,..., X k ) P is a production rule. The children of a node are ordered from the left. Notation (Tree terminology) Nodes, parent, child, root, interior nodes, leaves, descendants, ancestors. Automata and Grammars Grammars 6 September 19, / 294

126 Tree Examples, Yield Definition A parse tree of E I + E. A parse tree of P E P E + E 0 P 0 I 1 P 1 λ Definition (The Yield) The yield is a string of leaves of a parse tree concatenated from the left. Special importance has yield that: is a terminal string, the root is labeled by the start symbol. These yields are strings in the language of the underlying grammar. Automata and Grammars Grammars 6 September 19, / 294

127 Inference, Derivations, and Pare Trees Theorem Given a grammar G = (V, T, P, S), w T. The following are equivalent: The recursive inference procedure determines that terminal string w is in the language of variable A. A w. A lm w. A rm w. There is a parse tree with root A and yield w. Leftmost derivation Parse tree Derivation Rightmost derivation Recursive inference Proof directions: Automata and Grammars Grammars 6 September 19, / 294

128 From Inferences to Trees Theorem Let G = (V, T, P, S) be a CFG. If the recursive inference procedure tells us that terminal string w is in the language of variable A, then there is a parse tree with root A and yield w. Proof. A Induction on the number of steps used to infer w in L(A). w BASIS: A w P. INDUCTION: A X 1 X 2... X n P, we have parse trees for X i s from inductive hypothesis. X 1 w 1 X 2 w 2 A X n w n Automata and Grammars Grammars 6 September 19, / 294

129 From Trees to Derivations Example (Context free derivation) Assume following is a derivation: E I Ib ab. Then for any strings α, β following is also derivation: αeβ αiβ αibβ αabβ. Theorem Let G = (V, T, P, S) be a CFG, suppose there is a parse tree with root labeled by variable A and with yield w T. Then there is a leftmost derivation A lm w in grammar G. Automata and Grammars Grammars 6 September 19, / 294

130 Proof. Induction on the height of the tree. BASIS: height 1: Root A with children that read w. Since it is a parse tree, A w is a production, thus A lm w is a one step. INDUCTION: Height n > 1. Root A with children X 1, X 2,..., X k. If X i is a terminal, define w i to be a string consisting of X i alone. If X i is a variable, then by inductive hypothesis X i lm w i. We construct leftmost derivation, inductively on i = 1,..., k we show A lm w 1w 2... w i X i+1 X i+2... X k. If X i is a terminal, do nothing, just i + +. If X i is a variable, rewrite derivation: X i lm α 1 lm α 2... lm w i to w 1w 2... w i 1X ix i+1x i+2... X k lm w 1w 2... w i 1α 1X i+1x i+2... X k lm... lm w 1w 2... w i 1w ix i+1x i+2... X k. When i = k, the result is a leftmost derivation of w from A. Automata and Grammars Grammars 6 September 19, / 294

131 Leftmost Derivation from Parse Tree Example E E I a * E ( E ) E I + E I a I 0 I 0 Leftmost child of the root: E lm I lm a Rightmost child of the root: E lm (E) lm (E + E) lm (I + E) lm (a + E) lm (a+i) lm (a+i0) lm (a+i00) lm (a+b00) Root: E lm E E Leftmost integrated to root: E lm E E lm I E lm a E Full derivation: E lm E E lm I E lm a E lm lm a (E) lm a (E + E) lm a (I + E) lm lm a (a + E) lm a (a + I) lm a (a + I0) lm lm a (a + I00) lm a (a + b00). b Automata and Grammars Grammars 6 September 19, / 294

132 From Derivations to Recursive Inferences Lemma (Extracting X i derivation from A derivation) Suppose we have a derivation A X 1 X 2... X k w. Then we can break w into pieces w = w 1 w 2... w k such that X i w i. If X i is a variable, we can obtain the derivation of X i w i by starting with A w and stripping away: All the position of the sentential forms that are either to the left or right of the positions that are derived from X i, and All steps that are not relevant to the derivation of w i from X i. Example Consider derivation E E E E E + E I E + E I I + E I I + I a I + I a b + I a b + a. The derivation of central E from the third sentential form is: E I b. Automata and Grammars Grammars 6 September 19, / 294

133 From Derivations to Recursive Inference Theorem (From Derivations to Recursive Inference) Let G = (V, T, P, S) be a CFG, and suppose there is a derivation A w, w T. Then the recursive inference procedure applied to G determines what W is in the language of variable A. Proof. An induction on the length of the derivation A w. BASIS: One step, then A w must be a production. Since w T, w will be discovered in L(A) in the basis part of the recursive inference procedure. INDUCTION: n + 1 steps. Write the derivation as A X 1 X 2... X k w. Then, from the lemma we can break w into w = w 1 w 2... w k either X i T and X i = w i or X i V and X i w i has n or fewer steps. By inductive hypothesis w i is inferred to be in the language of X i. Now, we have production A X 1 X 2... X k. The next round of the recursive inference procedure shall discover that w 1 w 2... w k = w is in the L(A). Automata and Grammars Grammars 6 September 19, / 294

134 Applications of Context-Free Grammars Parsers Markup Languages (HTML) XML and Document Type Definitions (DTD) In the book chapter 5.3. Automata and Grammars Grammars 6 September 19, / 294

135 Ambiguity in Grammars Two derivations of the same expression: E E + E E + E E E E E E + E E E E E + E E * E E * E E + E Example The difference is important; left 1 + (2 3) = 7, right (1 + 2) 3 = 9. This grammar can be modified to be unambiguous. Different derivations may represent the same parse tree. Then, it is not a problem. 1. E E + E I + E a + E a + I a + b 2. E E + E E + I I + I I + b a + b. Automata and Grammars Grammars 6 September 19, / 294

136 Definition (ambiguous CFG) We say a CFG G = (V, T, P, S) is ambiguous if there is at least one string w T for which we can find two different parse trees, each with root labeled S and yield w. If each string has at most one parse tree in the grammar, then the grammar is unambiguous. E E E + E E * E Two derivation trees with yield a + a a shoving the ambiguity of the grammar. I E * E E + E I a I I I I a a a a a Automata and Grammars Grammars 6 September 19, / 294

137 Removing Ambiguity From Grammars There is no algorithm that can eve tell us whether a CFG is ambiguous. There are context-free languages that have nothing but ambiguous CFG s. There are some hints for removing ambiguity. There are two causes of ambiguity: The precedence of operators is not respected. A sequence of identical operators can group either from the left or from the right. Automata and Grammars Grammars 6 September 19, / 294

138 Enforcing Precedence The solution enforcing precedence is to introduce several different variables, each for one level of The sole parse tree fore a + a a. binding strength. Specifically: E A factor is an expression that cannot be broken by any operator. identifiers. Any parenthesized expression. A term is an expression that cannot be broken by + operator. An expression can be broken by either or +. E T F + T F T * F I An unambiguous expression grammar 1. I a b Ia Ib I0 I1 2. F I (E) 3. T F T F 4. I T E + T. I a I a a Automata and Grammars Grammars 6 September 19, / 294

139 Unambiguity of our grammar. Points why no string can have two different parse trees. A factor is either a single identifier or any parenthesized expression. Any string derived from T is a sequence of factors connected by. Because of two productions of T, the only parse tree breaks f 1 f 2... f n into a term f 1 f 2... f n 1 and a factor f n. Likewise, an expression is a sequence of terms connected by +. The production E E + T takes as the term always the last one. 1. I a b Ia Ib I0 I1 2. F I (E) 3. T F T F 4. I T E + T. T... T T * F T * F T * F * F F Automata and Grammars Grammars 6 September 19, / 294

140 Leftmost Derivations as a Way to Express Ambiguity Different leftmost derivations for different parse trees. E lm E + E lm I + E lm a + E lm a+e E lm a+i E lm a+a E lm a+a I lm a+a a E E lm E E lm E + E E lm I +E E lm a+e E lm a+i E lm a +a E lm a +a I lm a + a a E E + E E * E I E * E E + E I a I I I I a a a a a Automata and Grammars Grammars 6 September 19, / 294

141 Theorem For each grammar G = (V, T, P, S) and string w T, w has two distinct parse trees if and only if w has two distinct leftmost derivations from S. Proof. (Only-if) Wherever the two parse trees first have a node at which different productions are used, the leftmost derivations constructed will also use different productions and thus be different derivations. (If) Start constructing a tree only the root, labeleds. From eahc production used determine the node is being changed and what the children of this node should be. If there are two distinct derivations, then at the first step where the derivations differ, the nodes being constructed will get different lists of children, and this difference guarantees that the parse trees are distinct. Automata and Grammars Grammars 6 September 19, / 294

142 Inherent Ambiguity Definition (Inherent Ambiguity) A context-free language L is said to be inherently ambiguous if all its grammars are ambiguous. If one grammar for L is unambiguous, then L is an unambiguous language. Example (Inherently ambiguous language) An example of an inherently ambiguous language: 1. S AB C 2. A aab ab 3. B cbd cd 4. C acd add 5. D bdc bc. L = {a n b n c m d m n 1, m 1} {a n b m c m d n n 1, m 1}. This grammar is ambiguous. For example, the string aabbccdd has the two leftmost derivations: 1. S lm AB lm aabb lm aabbb lm aabbcbd lm aabbccdd 2. S lm C lm acd lm aaddd lm aabdcdd lm aabbccdd Automata and Grammars Grammars 6 September 19, / 294

143 Two parse trees for aabbccdd. S S A B C a A b c B d a C d a b c d a C d b D c D No matter what modifications we make to the basic grammar, it will generate at least some of the strings of the form a n b n c n d n in the two ways that the grammar presented. b c Automata and Grammars Grammars 6 September 19, / 294

144 Summary of Chapter 5 Context-Free Grammars G = (V, T, P, S), P recursive rules called productions. Derivations and Languages Beginning with S we repeatedly replace variable by the body. The language is the set of terminal strings we can so derive. Leftmost, Rightmost Derivations. Sentential Forms any step of derivation. Parse Trees Interior nodes are labeled by variables, and leaves are labeled by terminals or λ. For internal node, there must be a production justifying the node-children relation. Equivalence of Parse Trees and Derivations A terminal string is in the language of a grammar iff it is the yield of at least one parse tree. The existence of leftmost derivations and parse trees define exactly the strings in the language of a CFG. Ambiguous Grammars For some CFG s, it is possible to find a terminal string with more than one parse tree. Eliminating Ambiguity For many useful grammars it is possible to find an unambiguous grammar that generates the same language. Automata and Grammars Grammars 6 September 19, / 294

145 Pushdown Automata Pushdown automata is an extension of the λ NFA. The additional feature is the stack. Stack can be read, pushed, and popped only at the top. It can remember an infinite amount of information. Pushdown automata define context-free languages. Deterministic pushdown automata accept only a proper subset of the CFL s. A pushdown automaton. Automata and Grammars Pushdown Automata 7 September 19, / 294

146 In one transition, the pushdown automaton: Example Consumes from the input zero or one symbol. (λ transitions for zero input.) Goes to a new state. Replaces the symbol at the top of the stack by any string (λ corresponds to pop, replace top symbol, push more symbols). PDA for the language ww R : L wwr = {ww R w (0 + 1) }. A PDA accepting L wwr : Start q 0 represents a guess that we have not yet seen the middle. At any time, non deterministically guess Stay q 0 (not yet in the middle). Spontaneously go to state q 1 (we have seen the middle). In q 0, read the input symbol and push it onto the stack. In q 1, compare the input symbol with the one on top of the stack. If they match, consume the input symbol and pop the stack. If we empty the stack, w accept the input that was read up to this point. Automata and Grammars Pushdown Automata 7 September 19, / 294

147 Pushdown Automata (PDA) Definition (Pushdown Automata) A pushdown automaton (PDA) is P = (Q, Σ, Γ, δ, q 0, Z 0, F ), where Q A finite set of states. Σ A finite set of input symbols. Γ A finite stack alphabet. q 0 Z 0 δ The transition function. δ : Q (Σ {λ}) Γ P FIN (Q Γ ), (q, a, X) = (p, γ) where p is the new state and γ a string of stack symbols that replace X on top of the stack. The start state. The start symbol. The only symbol on the stack at the beginning. F The set of accepting (final) states. Automata and Grammars Pushdown Automata 7 September 19, / 294

148 PDA for L wwr Example (PDA for L wwr ) PDA for L wwr can be described P = ({q 0, q 1, q 2, {0, 1}, {0, 1, Z 0 }, δ, q 0, Z 0, {q 2 }}) where δ is defined: δ(q 0, 0, Z 0 ) = {(q 0, 0Z 0 )} Push the input on stack, leave the start symbol there. δ(q 0, 1, Z 0 ) = {(q 0, 1Z 0 )} δ(q 0, 0, 0) = {q 0, 00} δ(q 0, 0, 1) = {q 0, 01} Stay in q 0, read the input and push it onto stack. δ(q 0, 1, 0) = {q 0, 10} δ(q 0, 1, 1) = {q 0, 11} δ(q 0, λ, Z 0 ) = {q 1, Z 0 } δ(q 0, λ, 0) = {q 1, 0} Spontaneous transition to q 1, no change on stack. δ(q 0, λ, 1) = {q 1, 1} δ(q 1, 0, 0) = {q 1, λ} State q 1 matches the input and the stack symbols. δ(q 1, 1, 1) = {q 1, λ} δ(q 1, λ, Z 0 ) = {q 2, Z 0 } We have found ww R and go to the accepting state. Automata and Grammars Pushdown Automata 7 September 19, / 294

149 A Graphical Notation for PDA s Definition (Transition diagram for PDA) A transition diagram for PDA contains: The nodes correspond to the states of the PDA. The first arrow indicates the start state, and doubly circled states are accepting. The arc correspond to transitions of the PDA. An arc labeled a, X α from state q to p means that δ(q, a, X) (p, α). Conventionally, the start stack symbol is Z 0. Labels: input_symbol, stack_symbol string_to_push 0, Z 0 0Z 0 1, Z 0 1Z 0 0, , , , , 0 λ 1, 1 λ q 0 q 1 q 2 λ, Z 0 Z 0 λ, Z 0 Z 0 λ, 0 0 λ, 1 1 Automata and Grammars Pushdown Automata 7 September 19, / 294

150 Definition (PDA configuration) We represent the configuration of a PDA by a triple (q, w, γ), where q is the state w is the remaining input and γ is the stack contents (top on the left). Such a tripple is called an instantaneous description (ID) of the pushdown automaton. Definition (, Sequences of instantaneous descriptions) Let P = (Q, Σ, Γ, δ, q 0, Z 0, F ) be a PDA. Define P or just as follows. Suppose δ(q, a, X) (p, α). Then for all strings w Σ and β Γ : (q, aw, Xβ) (p, w, αβ). We also use the symbol P or to represent zero or more moves of the PDA, i.e. I I for any ID I I J if there exists some ID K such that I K and K J. Automata and Grammars Pushdown Automata 7 September 19, / 294

151 ID s of the PDA on input 1111 (q 0, 1111, Z 0 ) (q 0, 111, 1Z 0 ) (q 1, 1111, Z 0 ) (q 0, 11, 11Z 0 ) (q 1, 111, 1Z 0 ) (q 2, 1111, Z 0 ) (q 0, 1, 111Z 0 ) (q 1, 11, 11Z 0 ) (q 1, 11, Z 0 ) (q 0, λ, 1111Z 0 ) (q 1, 1, 111Z 0 ) (q 1, 1, 1Z 0 ) (q 2, 11, Z 0 ) (q 1, λ, 1111Z 0 ) (q 1, λ, 11Z 0 ) (q 1, λ, Z 0 ) (q 2, λ, Z 0 ) Automata and Grammars Pushdown Automata 7 September 19, / 294

152 Data P never looks at cannot affect its computation. Theorem If P = (Q, Σ, Γ, δ, q 0, Z 0, F ) is a PDA, and (q, x, α) P (p, y, β), then for any strings w Σ and γ Γ, it is also true that (q, xw, αγ) P (p, yw, βγ). Specially, it is true for γ = λ and/or w = λ. Proof. Induction on the number of steps in the sequence of ID s that take (q, xw, αγ) to (p, yw, βγ). Each of the moves (q, x, α) P (p, y, β) is justified without sing w and/or γ in any way. Therefore, each move is still justified when these strings are sitting on the input and stack. Theorem If P = (Q, Σ, Γ, δ, q 0, Z 0, F ) is a PDA, and (q, xw, α) P (p, yw, β), then it is also true that (q, x, α) P (p, y, β). Remark The same with the stack does not hold. The PDA may use γ on the stack and push it back again. Automata and Grammars Pushdown Automata 7 September 19, / 294

153 Notation Convention for PDA s a, b, c symbols of the input alphabet q, p states w, z strings of input symbols X, Y stack symbols α, β, γ strings stack symbols. Automata and Grammars Pushdown Automata 7 September 19, / 294

154 The Languages of a PDA Definition (PDA language accepted by final state) Let P = (Q, Σ, Γ, δ, q 0, Z 0, F ) be a PDA. Then L(P), the language accepted by F by final state, is {w (q 0, w, Z 0 ) P (q, λ, α) for some q F and any stack string α. Example The PDA example for L wwr accepts the language. (IF) For any x = ww R, we have a accepting computation (q 0, ww R, Z 0 ) (q 0, w R, w R Z 0 ) (q 1, w R, w R Z 0 ) (q 1, λ, Z 0 ) (q 2, λ, Z 0 ). (Only If) The only way to enter q 2 is from q 1 and Z 0 at the top of the stack. Any accepting computation strats in q 0, changes to q 1 and never returns to q 0. We prove (q 0, x, Z 0) (q 1, λ, Z 0) exactly for the strings of the form x = ww R. Proof by induction on x in the book p.235. Automata and Grammars Pushdown Automata 7 September 19, / 294

155 Accdeptance by Empty Stack Definition (PDA language accepted by empty stack) For each PDA P = (Q, Σ, Γ, δ, q 0, Z 0, F ) we define N(P),the language accepted by F by empty stack N(P) = {w (q 0, w, Z 0 ) P (q, λ, λ)} for any q Q. The set of accepting states is in this case irrelevant, we shall sometimes leave it off and state P as a six-tuple P = (Q, Σ, Γ, δ, q 0, Z 0 ). Example In the previous example, we change the transition δ(q 1, λ, Z 0 ) = {(q 2, Z 0 )} to pop the last symbol: δ(q 1, λ, Z 0 ) = {(q 2, λ)}. Now, L(P ) = N(P ) = L wwr. Automata and Grammars Pushdown Automata 7 September 19, / 294

156 From Empty Stack to Final State Theorem (From Empty Stack to Final State) If L = N(P N ) for some PDA P N = (Q, Σ, Γ, δ N, q 0, Z 0 ), then there is a PDA P F such that L = L(P F ). p0 λ, X0 Z0X0 q0 PN λ, X0 λ λ, X0 λ pf λ, X0 λ Proof. P F = (Q {p 0, p f }, Σ, Γ {X 0 }, δ F, p 0, X 0, {p F }), where δ F is λ, λ δ F (p 0, λ, X 0 ) = {(q 0, Z 0 X 0 )} (start). (q Q, a Σ {λ}, Y Γ), δ F (q, a, Y ) = δ N (q, a, Y ). In addition, δ F (q, λ, X 0 ) (p f, λ) for every q Q. We must show that w L(P N ) iff w L(P F ). (If) P F accepts as follows: (p 0, w, X 0 ) PF (q 0, w, Z 0 X 0 ) P F = N F (q, λ, X 0 ) PF (p f, λ, λ). (Only if) No other way to go to p f than that above. Automata and Grammars Pushdown Automata 7 September 19, / 294

157 If-Else Example Example (Accepts by empty stack) We design a PDA that stops at the first error on if (i) and else (e) error, i.e. we have more else s then if s. P N = ({q}, {i, e}, {Z}, δ N, q, Z) where δ N (q, i, Z) = {(q, ZZ)} push. δ N (q, e, Z) = {(q, λ)} pop. λ, Z λ i, Z ZZ q Example (Accepts by final state) P F = ({p, q, r}, {i, e}, {Z, X 0 }, δ F, p, X 0, {r}) where δ F (p, λ, X 0 ) = {(q, ZX 0 )} start. δ F (q, i, Z) = {(q, ZZ)} push. δ F (q, e, Z) = {(q, λ)} pop. δ F (q, λ, X 0 ) = {(r, λ)} accept. λ, Z λ i, Z ZZ λ, X 0 ZX 0 λ, X 0 λ p q r Automata and Grammars Pushdown Automata 7 September 19, / 294

158 From Final State to Empty Stack Theorem Let L = L(P F ) for some PDA P F = (Q, Σ, Γ, δ F, q 0, Z 0, F ). There is a PDA P N such that L = L(P N ). p0 λ, X0 Z0X0 q0 PF λ, any λ λ, any λ p λ, any λ Proof. Let P N = (Q {p 0, p}, Σ, Γ {X 0 }, δ N, p 0, X 0 ), where δ N (p 0, λ, X 0 ) = {(q, Z 0 X 0 )} start. (q Q, a Σ {λ}, Y Γ) δ N (q, a, Y ) = δ F (q, a, Y ) simulate. (q F, Y Γ {X 0 }), δ N (q, λ, Y ) (q, λ) i.e. accept if P F accepts. (Y Γ {X 0 }), δ N (p, λ, Y ) = {(p, λ)} clean the stack. The proof w N(P n ) iff w L(P F ) is similar as before. Automata and Grammars Pushdown Automata 7 September 19, / 294

159 Equivalence of PDA s and CFG s Lemma Following three classes are equivalent: The context-free languages,i.e. the languages defined by CFG s. The languages that are accepted by final state by some PDA. The languages that are accepted by empty stack by some PDA. Organization of constructions showing equivalence of three ways of defining the CFL s. Grammar PDA by empty stack PDA by final state Automata and Grammars Pushdown Automata 7 September 19, / 294

160 From Grammars to Pushdown Automata Construction of PDA from a CFG G. Let G = (V, T, P, S) be a CFG. Construct the PDA P = ({q}, T, V T, δ, q, S). (1) For each variable A V, δ(q, λ, A) = {(q, β) A β is a production of G}. (2) For each terminal a T, δ(q, a, a) = {(q, λ)}. Given a CFG G, we construct a PDA that simulates the leftmost derivations on G. Any not terminal left-sentential form can be written xaα, A the leftmost nonterminal,x is a string of terminals. We call Aα to be a tail, it is λ for terminals only sentential form. The tail of each sentential form appears on the stack. Suppose ID (q, y, Aα) representing left-sentential form xaα. Production guesses a rule A β. The PDA replaces A on the stack by β, entering ID (q, y, βα). Remove all terminals from the left of βα comparing them against the input. Then, either the stack is empty or we reach the left-sentential form w and iterate. Automata and Grammars Pushdown Automata 7 September 19, / 294

161 Example Let us convert the grammar: I a b Ia Ib I0 I1 E I E E E + E (E). The set of input symbols for the PDA is Σ = {a, b, 0, 1, (, ), +, }, Γ = Σ {I, E}, the transition δ: δ(q, λ, I) = {(q, a), (q, b), (q, Ia), (q, Ib), (q, I0), (q, I1)}. δ(q, λ, E) = {(q, I), (q, E E), (q, E + E), (q, (E))}. For all s Σ is δ(q, s, s) = {(q, λ)}, for example δ(q, +, +) = {(q, λ)}. Also, δ is empty otherwise. Theorem (PDA empty stack acceptance from a CFG.) If PDA P if constructed from CFG G by the construction above, then N(P) = L(G). Leftmost derivation: E E E I E a E a I a b PDA: (q, a b, E) (q, a b, E E) (q, a b, I E) (q, a b, a E) (q, b, E) (q, b, E) (q, b, I) (q, b, b) (q, λ, λ) Automata and Grammars Pushdown Automata 7 September 19, / 294

162 Proof. We shall prove w N(P) iff w L(G). (Only If) Suppose w L(G), w has leftmost derivation S = γ 1 lm γ 2 lm... lm γ n = w. By induction on i: (q, w, S) P (q, y i, α i ) where γ i = x i α i is the left-sentential form and x i y i = w. (If) We need to prove: If (q, x, A) P (q, λ, λ), then A G x. Proof is an induction on the number of moves taken by P. n = 1 move: The only possibility the production (A λ) G, in construction only for x = λ. n > 1 moves; First move type (1), A replaced by Y 1Y 2... Y k from a production A Y 1Y 2... Y k. We split x = x 1x 2... x k as on the Figure and use inductive hypothesis on each i = 1,..., k: (q, x ix i+1... x k, Y i) (q, x i+1... x k, λ) to get Y i x i. Together, A Y 1Y 2... Y k x 1Y 2... Y k... x 1x 2... x k. Automata and Grammars Pushdown Automata 7 September 19, / 294

163 From PDA to Grammars The key event: PDA pops one symbol off the stack. The state before and after the popping may be different. Grammar variables are composite symbols [pxq], PDA started p, popped X and moved to state q and a special new start symbol S. On Figure, [p 0 Y 1 p 1 ], [p 1 Y 2 p 2 ],..., [p k 1 Y k p k ]. Theorem (Grammar for a PDA) Let P = (Q, Σ, Γ, δ, q 0, Z 0 ) be a PDA. Then there is a context-free grammar G such that L(G) = N(P). Productions will be: p Q: S [q 0 Z 0 p], i.e. ques the end symbol and run the PDA to (q 0, w, Z 0 ) (p, λ, λ). For all pairs (r, Y 1 Y 2... Y k ) in all δ(q, a, X) create a production [qxr k ] a[ry 1 r 1 ][r 1 Y 2 r 2 ]... [r k 1 Y k r k ]. Automata and Grammars Pushdown Automata 7 September 19, / 294

164 Proof. The proof that: [qxp] if and only if(q, w, X) (p, λ, λ) is done in both direction by induction (number of moves, number of steps in the derivation.) Example ({0 n 1 n ; n > 0}) δ Productions S [pzp] [pzq] (1) δ(p, 0, Z) (p, A) [pzp] 0[pAp] (2) [pzq] 0[pAq] (3) δ(0, A) (q, AA) [pap] 0[pAp][pAp] (4) [pap] 0[pAq][qAp] (5) [paq] 0[pAp][pAq] (6) [paq] 0[pAq][qAq] (7) δ(p, 1, A) (q, λ) [paq] 1 (8) δ(q, 1, A) (q, λ) [qaq] 1 (9) 0, Z A 0, A AA p 1, A λ 1, A λ Derivation of 0011 S (1) [pzq] (3) 0[pAq] (6) 00[pAq][qAq] (8) 001[qAq] (9) 0011 q Automata and Grammars Pushdown Automata 7 September 19, / 294

165 Deterministic Pushdown Automata (DPDA) Definition (Deterministic Pushdown Automata (DPDA)) A PDA P = (Q, Σ, γ, δ, q 0, z 0, F ) is deterministic PDA iff obht: δ(q, a, X) has at most one member q Q, a Σ {λ} and X Γ. If δ(q, a, X) is nonempty, for some a Σ, then δ(q, λ, X) must be empty. Example (A deterministic PDA accepting L wcwr ) The language L wwr before is CFL that has no DPDA. The second condition assure that there is no choice between λ transition and consumption of the input symbol. By putting a center-mark c in the middle L wcwr = {wcw R w (0 + 1) } it is recognizable by DPDA. 0, Z 0 0Z 0 1, Z 0 1Z 0 0, , , , , 0 λ 1, 1 λ q 0 q 1 q 2 c, Z 0 Z 0 λ, Z 0 Z 0 c, 0 0 c, 1 1 Automata and Grammars Pushdown Automata 7 September 19, / 294

166 Regular Langs and Deterministic PDA s, Prefix Property Theorem RL L(P DPDA ) CFL N(P DPDA ). If L is a regular language, then L = L(P) for some DPDA P. Proof. Essentially, a DPDA can simulate a deterministic FA and ignores the stack (keeping Z 0 there). Definition (prefix property) Say that a language L has the prefix property if there are no two different strings x, y L such that x is a prefix of y. Example The language L wcwr has the prefix property. On the other hand, L = {0} does not have the prefix property. Automata and Grammars Pushdown Automata 7 September 19, / 294

167 Theorem (L N(P DPDA ) iff prefix property L and L L(P DPDA ) ) A language L is N(P) for some DPDA P iff L has the prefix property and L is L(P ) for some DPDA P. Lemma The language L wcwr is accepted by a DPDA but it is not regular. To prove non-regularity use pumping lemma on the string 0 n c0 n. Lemma The language L wwr is CFL but not accepted by any DPDA. Formal proof is complex. The idea is that words 0 n 110 n 0 n 110 n, 0 n 110 n 0 m 110 m are both accepted or both rejected since in the middle the stack is empty and there is no other way to remember any number n than on the stack. The first word is in L wwr, the second is not. Automata and Grammars Pushdown Automata 7 September 19, / 294

168 DPDA s and Ambiguous Grammars Theorem (L = N(P DPDA ) L has an unambiguous CFG.) Let L = N(P) for some DPDA P. Then L has an unambiguous CFG. Let L = L(P) for some DPDA P. Then L has an unambiguous CFG. The opposite is not true since L wwr has an unambiguous grammar S 0S0 1S1 λ but is not a DPDA language. Proof. N(P): The construction of a CFG from a PDA accepting by empty stack applied to NPDA yields an unambiguous CFG G. L(P): Construct a language that has the prefix property by adding a new symbol $ at the end of any word w L. Construct a DPDA P for that L = N(P ). Construct a grammar G generating the language N(P ). Construct G such that L(G) = L. Just get rid of $ keeping it as a variable and adding a production $. All other productions remain the same as in G. G is unambiguous since G is unambiguous and we did not add ambiguity. Automata and Grammars Pushdown Automata 7 September 19, / 294

169 L 3 regular lang. {0, 00} {010} prefix prop & DPDA {0 n 1 n ; n > 0} deterministic PDA {0 n 1 m ; 0 < n m} context (=CL) L 1 {a i b i c i i = 0, 1,...} recursively enumerable L 0 context free (=CFL) {ww R w {0, 1}} L 2 Automata and Grammars Pushdown Automata 7 September 19, / 294

170 Other Characterization of Context Free Languages Until we only PUSH to the stack we may remember the last stack symbol finite automaton can simulate this. we need to POP the stack we need an infinite memory for this PUSH and POP is not arbitrary it is a stack, that is LIFO (last in, first out) structure we put the stack actions into a linear structure X PUSH(X) X 1 POP(X) PUSH(X) and POP(X) make a pair ZZ }{{ 1 } B AA 1 1 }{{}} CC {{} B 1 }{{} that behaves as a bracket in the sequence Automata and Grammars Pushdown Automata 7 September 19, / 294

171 Dyck Langulages Definition (Dyck Language) Dyck Language D n is defined on the alphabet Z n = {a 1, a 1,..., a n, a n} by the grammar: S λ SS a 1 Sa 1... a nsa n. Observations: Dyck language is context free Dyck language D n describes correctly formed formulas with n pairs of brackets we may describe any context free language in the following form homomorphism; erases not necessary symbols L = h(d R) Dyck language; selects correct sequences of moves regular language; all steps of the PDA Automata and Grammars Pushdown Automata 7 September 19, / 294

172 Theorem (Dyck Languages) For any context free language L there exists a regular language R so that L = h(d R) for some Dyck language D and a homomorphism h. Proof: we have a PDA P such that L = N(P) (by empty stack) we transform instructions to δ(q, a, Z) (p, w), w 2 we split longer PUSH sequences by additional states regular language R contains expressions q 1 aa 1 Z 1 BAp for transition δ(q, a, Z) (p, AB) similarly δ(q, a, Z) (p, A), δ(q, a, Z) (p, λ) for a = λ we leave aa 1 out we define R as follows: Z 0 q 0 (R ) Q 1 Dyck language is defined over the alphabet Σ Σ 1 Q Q 1 Γ Γ 1 D Z 0 q 0 (R ) Q 1 describes correct sequences of PDA moves {}}{{}}{{}}{{}}{ Z 0 q 0 q 1 0 aa 1 Z 1 0 }{{} B A pp 1 bb 1 A }{{ 1 qq } 1 cc 1 B 1 rr 1 }{{} homomorphism h select the input word, that is h(a) = a for input symbols h(y) = λ for any other symbols Automata and Grammars Pushdown Automata 7 September 19, / 294

173 Summary of Chapter 6 Pushdown automata A PDA is a non-deterministic FA couplet with a stack. Moves of a Pushdown Automata: δ(p, a, X) = {(q, β)}, p, q Q, a Σ {λ}, X Γ, β Γ. Acceptance by Pushdown Automata: by an empty stack N(P) or by final states L(P). instantaneous Descriptions: ID = state, remaining input and the stack. A transition between ID s represents single moves of a PDA. Pushdown Automata and Grammars For (non-deterministic) PDA s, the languages accepted either by final state or by empty stack, are exactly the context-free languages. Deterministic Pushdown Automata: It never has a choice of mover for a given state, input symbol(including λ) and the stack symbol, nor between move using a true input symbol and a move using λ symbol. Acceptance by DPDA s: L N(P DPDA ) iff prefix property L and L L(P DPDA ). The Languages Accepted by DPDA s: RL L(P DPDA ) CFG languages with unambiguous grammars, i.e. L(P DPDA ) CFG. Automata and Grammars Pushdown Automata 7 September 19, / 294

174 Properties of context-free Languages We simplify CFL s. Chomsky Normal Form Greibach Normal Form We prove pumping lemma for CFL s. We study closure properties and decision properties. Some of them remain, some not. Automata and Grammars Normal Forms, Pumping Lemma 8 September 19, / 294

175 Normal Forms for Context-Free Grammars Chomsky Normal Form: all production are of the form A BC or A a, A, B, C where are variables, a is a terminal. Every CFL (without λ) is generated by a CFG in Chomsky Normal Form. To get there, we perform simplifications Eliminate useless symbols eliminate λ-productions A λ for some variable A eliminate unit productions A B for variables A, B. Automata and Grammars Normal Forms, Pumping Lemma 8 September 19, / 294

176 Eliminating Useless symbols Definition (useful symbol) A symbol X V T is useful for a grammar G = (V, T, P, S) if there is some derivation of the form S αxβ w where w T, X (V T ). If X is not useful, we say it is useless. X is generating iff X w for some terminal string w. Note always w w by zero steps. X is reachable iff the is S αxβ for some α, β. We aim eliminate non generating and not reachable symbols. Example Consider the grammar: S AB a A b Eliminate B (nongenerating): S a A b. Eliminate A (not reachable): S a. Automata and Grammars Normal Forms, Pumping Lemma 8 September 19, / 294

177 Theorem (Eliminating useless symbols) Let G = (V, T, P, S) be a CFG, and assume that L(G). Let G 1 = (V 1, T 1, P 1, S) is obtained: Eliminate nongenerating symbols and all productions involving them. Eliminate all symbols that are not reachable after previous step. Then G 1 has no useless symbols, and L(G 1 ) = L(G). Generating symbols: BASIS: Every a T is generating. INDUCTION: For any production A α and every symbol of α is generating. Then A is generating. (This includes A λ). Reachable symbols: BASIS: S is surely reachable. INDUCTION: If A is reachable, for all production with A in the head, all symbols of the bodies are also reachable. Theorem The algorithms above find all and only the generating / reachable symbols. Automata and Grammars Normal Forms, Pumping Lemma 8 September 19, / 294

178 Eliminating λ-productions Without λ-production, λ / L. We aim to prove: L has a CFG, then L {λ} has a CFG without λ-productions. Definition (nullable variable) A variable A is nullable if A λ. For nullable variables in the body B CAD, we create two versions of the production - with and without this variable. An algorithm to find all nullable symbols of G: If A λ is a production of G, then A is nullable. If B C 1... C k where each C i is nullable, then B is nullable (note terminal C i T is not nullable). Automata and Grammars Normal Forms, Pumping Lemma 8 September 19, / 294

179 Construction of a grammar without λ-productions from G = (V, T, P, S). Determine nullable symbols. For each production A X 1... X k P, k 1 suppose m of X i s are nullable. The new grammar G 1 will have 2 m versions of this production with/without each nullable symbol except λ in case m = k. Example Consider the grammar: S AB A aaa λ B bbb λ S AB A B A aaa aa aa a B bbb bb bb b Final grammar: S AB A B A aaa aa a B bbb bb b. Automata and Grammars Normal Forms, Pumping Lemma 8 September 19, / 294

180 Eliminating Unit Productions Definition (unit production) A unit production is A B P where both A, B are variables. Example I a b Ia Ib I0 I1 F I (E) T F T F E T E + T Expanding T in E T E F T F Expanding E F E I (E) Expanding E I E a b Ia Ib I0 I1 Together: E a b Ia Ib I0 I1 (E) T F E + T. We have to avoid possible cycles. Definition (unit pair) A pair A, B V such that A B using only unit productions is called a unit pair. Automata and Grammars Normal Forms, Pumping Lemma 8 September 19, / 294

181 Unit pairs identification BASIS: (A, A) for A V is a unit pair. INDUCTION: If (A, B) is a unit pair and B C P, then (A, C) is a unit pair. Example (Unit pairs from previous grammar) (E, E), (T, T ), (F, F ), (I, I), (E, T ), (E, F ), (E, I), (T, F ), (T, I), (F, I). To eliminate unit productions from G = (V, T, P, S): Find all unit pairs of G. For each unit pair (A, B) new grammar all productions A α where B α P and B α is not a unit production. Example I a b Ia Ib I0 I1 F I (E) T F T F E T E + T I a b Ia Ib I0 I1 F (E) a b Ia Ib I0 I1 T T F (E) a B Ia Ib I0 I1 E E + T T F (E) a b Ia Ib I0 I1 Automata and Grammars Normal Forms, Pumping Lemma 8 September 19, / 294

182 Normal Form of CFGs Theorem (Normal Form of CFGs) If G is a CFG, L(G) {λ}, then there is a CFG G 1 such that L(G 1 ) = L(G) {λ} and G 1 has no λ-productions, unit productions, or useless symbols. Proof. Proof outline: Start by eliminating λ-productions. Eliminate unit productions. This does not introduce λ productions. Eliminate useless symbols. This does not introduce any new production. Automata and Grammars Normal Forms, Pumping Lemma 8 September 19, / 294

183 Chomsky Normal Form Definition (Chomsky Normal Form) A CFG grammar G = (V, T, P, S) that has no useless symbols and all productions are in on of two forms: A BC, A, B, C V, A a, A V, a T, is said to be in Chomsky Normal Form (ChNF). To put a grammar to ChNF, we need two additional steps: Bodies of length 2 or more consist only of variables. Break bodies of length 3 or more to bodies of two variables. For every terminal a create a new variable, say A, add one production rule A a, Use A in place of a everywhere a appears in a body of length 2 or more. For a production A B 1... B k introduce k 2 variables C i And productions A B 1 C 1,C 1 B 2 C 2,...,C k 2 B k 1 B k. Automata and Grammars Normal Forms, Pumping Lemma 8 September 19, / 294

184 ChNF Example I a b IA IB IZ IU F LER a b Ia IB IZ IU T TMF LER a b IA IB IZ IU E EPT TMF LER a b IA IB IZ IU A a B b Z 0 U 1 P + M L ( R ) F LC 3 a b IA IB IZ IU T TC 2 LC 3 a b IA IB IZ IU E EC 1 TC 2 LC 3 a b IA IB IZ IU C 1 PT C 2 MF C 3 ER I, A, B, Z, U, P, M, L, R as before Theorem (ChNF) If G is a CFG, L(G) {λ}, then there is a grammar G 1 in Chomsky Normal Form such that L(G 1 ) = L(G) {λ}. Automata and Grammars Normal Forms, Pumping Lemma 8 September 19, / 294

185 The Size of the Parse Tree Theorem (The Size of Parse Trees) Suppose we have a parse tree according the a ChNF grammar G = (V, T, P, S) that yield a terminal string w. If the length of the longest path is n, then w 2 n 1. Proof. By induction on n, BASIS: a = 1 = 2 0, INDUCTION: 2 n n 2 = 2 n 1. Theorem ((Consequence)) Let us have a parse tree for a ChNF grammar G = (V, T, P, S) that yield a terminal string w; w > 2 n 1. Then the longest path of the tree has more than n edges. Automata and Grammars Normal Forms, Pumping Lemma 8 September 19, / 294

186 Pumping Lemma for Context Free Languages Theorem (Pumping Lemma for Context Free Languages) Let L be a CFL. Then there exists constants p, q N such that any z L, z > p can be written z = uvwxy subject to: vwx q. vx λ. i 0, uv i wx i y L. A 1 T 1 A 2 T 2 u v w x y v w x Proof Idea: take the parse tree for z find the longest path there must be two equal variables these variables define two subtrees the subtrees define partition of z = uvwxy we can move the tree T 1 (i > 1) or replace T 1 by T 2 (i = 0) Automata and Grammars Normal Forms, Pumping Lemma 8 September 19, / 294

187 Proof: z > p : z = uvwxy, vwx q, vx λ, i 0uv i wx i y L we take the grammar in Chomsky NF (for L = {λ} and aside). Let V = k. We set p = 2 k 1, q = 2 k. For z L, z p, the parse tree has a path z of length > k we denote the terminal of the longest path t At least two of the last k variables on the path to t are equal we take the couple A 1, A 2 closest to t (it defines subtrees T 1, T 2 ) the path from A 1 to t is the longest in T 1 and the length is maximally k + 1 the yield of T 1 is no longer than 2 k (so vwx q) there are two paths from A 1 (ChNF), one to T 2 other to the rest of vx ChNF not nullable, so vx λ derivation of the word (A 1 va 2 x, A 2 w) S ua 1 y uva 2 xy uvwxy if we move A 2 to A 1 (i = 0) S ua 2 y uwy if we move A 1 to A 2 (i = 2, 3,...) S ua 1 y uva 1 xy uvva 2 xxy uvvwxxy Automata and Grammars Normal Forms, Pumping Lemma 8 September 19, / 294

188 Applications of the Pumping Lemma for CFL s "Adversary game" as for regular languages: Pick a language L that is not CFL. Our adversary gets to pik n, which we do not know. We get to pick z, and we may use n as a parameter. Our adversary gets to berak z into uvwxy, subject vwx n and vx λ. We win the game, if by picking i and showing uv i wx i y is not in L. Lemma (Not CFL) Following languages are not CFL: {0 n 1 n 2 n n 1} {0 i 1 j 2 i 3 j i 1&j 1} {ww w is in {0, 1} } Automata and Grammars Normal Forms, Pumping Lemma 8 September 19, / 294

189 Pumping Lemma Usage Example (non CFL) Following language is not CFL {0 n 1 n 2 n n 1} assume it were CFL we get p, q from the Pumping Lemma select k = max(p, q), then 0 k 1 k 2 k > p the word is not longer then q we pump at most two different symbols the equality of symbols is violated CONTRADICTION. Example (not a CFL) Following language is not CFL {0 i 1 j 2 k 0 i j k} assume it were CFL we get p, q from the Pumping Lemma select n = max(p, q), then 0 n 1 n 2 n > p the word is not longer then q we pump at most two different symbols in the case of a (or b), pump up CONTRADICTION i j (or j k) if c (or b), pump down CONTRADICTION j k (or i j) Automata and Grammars Normal Forms, Pumping Lemma 8 September 19, / 294

190 Použití lemma o vkládání Example (ne bezkontextový jazyk) Následující jazyk není bezkontextový {0 i 1 j 2 i 3 j i, j 1} důkaz sporem: předpokládejme bezkontextovost z lemmatu o vkládání máme p, q zvolme k = max(p, q), potom 0 k 1 k 2 k 3 k > p pumpovací slovo není delší než q tj. vždy lze pumpovat maximálně dva různé sousední symboly poruší se rovnost počtu symbolů 0 a 2 nebo 1 a 3 SPOR Example (ne bezkontextový jazyk) Následující jazyk není bezkontextový {ww w is in {0, 1} } důkaz sporem: předpokládejme bezkontextovost z lemmatu o vkládání máme p, q zvolme n = max(p, q), potom 0 n 1 n 0 n 1 n > p pumpovací slovo není delší než q tj. vždy lze pumpovat maximálně dva různé sousední symboly poruší se buď rovnost nul či jedniček. Automata and Grammars Normal Forms, Pumping Lemma 8 September 19, / 294

191 Infinity of Context Free Languages Lemma For any CFL L there exists two numbers m, n N such that L is infinite iff z L : m < z < n. Proof: We have from Pumping lemma p, q, set m = p, n = p + q p < z, therefore we may pump z L is infinite L is infinite z L : p = m < z. we take the shortest z; z > p and prove z n = p + q by contradiction Assume p + q < z, z may be pumped down, therefore z < z we remove part no longer than q, therefore p < z CONTRADICTION z was shortest. Faster algorithm: take reduced grammar G in ChNF so that L = L(G) create an oriented graph nodes = variables, edges = {(A, B), (A, C)for(A BC) P G } we look for an oriented cycle (if it exists the language is infinite). Automata and Grammars Normal Forms, Pumping Lemma 8 September 19, / 294

192 Generalizations of the Pumping Lemma Pumping lemma is only an implication. Example (non CFL, that can be pumped) L = {a i b j c k d l i = 0 j = k = l} is not a CFL but it can be pumped. i = 0 : b j c k d l i > 0 : a i b n c n d n can be pumped in any letter can be pumped in a A solution? generalisations (Ogden lemma and others) pumping marked symbols closure properties. Automata and Grammars Normal Forms, Pumping Lemma 8 September 19, / 294

193 Greibach Normal Form It would be nice to know which rule to select expecially dificult is the left recursion A Aα Definition Greibach Normal Form or a CFG We say a grammar G is in Greibach Normal Form iff all rules are in the form A aβ, where a T, β V (a string of variables). the terminal in the body of the rule helps to select appropriate rule expecially if we have a unique rule with this terminal. Theorem (Greibach Normal Form) For each CFL language L there exists a CFG grammar G in Greibach Normal Form such that L(G) = L {λ}. Automata and Grammars Normal Forms, Pumping Lemma 8 September 19, / 294

194 Join the rules Lemma (Joining the rules) Let us have a rule A αbβ in grammar G and B ω 1,..., B ω k are all rules for B. If we replace the rule A αbβ by rules A αω 1 β,..., A αω k β we get an equivalent grammar. Proof: A αbβ α Bβ α ω i β A αω i β α ω i β in the original grammar in the new grammar Automata and Grammars Normal Forms, Pumping Lemma 8 September 19, / 294

195 Left Recursion Lemma (Left Recursion) Let A Aω 1,..., A Aω k be all productions with left recursion in the grammar G for A and A α 1,..., A α m be all other rules for A, Z is a new variable. Then, by replaceing these rules by the rules: 1. A α i, A α i Z, Z ω j, Z ω j Z, nebo 2. A α i Z, Z ω j Z, Z λ we get an equivalent grammar. Proof: A Aω in... Aω i1... ω in α j ω i1... ω in (G) A α j Z α j ω i1 Z... α j ω i1... ω in 1 Z α j ω i1... ω in (1) A α j Z α j ω i1 Z... α j ω i1... ω in Z α j ω i1... ω in (2) Theorem (Greibach Normal Form) For each CFL language L there exists a CFG grammar G in Greibach Normal Form such that L(G) = L {λ}. Automata and Grammars Normal Forms, Pumping Lemma 8 September 19, / 294

196 Proof: Greibach Normal Form join rules and remove left recursion we enumerate all variables {A 1,..., A n } we allow recursion only in the form A i A j ω, where i < j we iterate i from 1 to n A i A j ω for j < i removed by joining the rules for j = i remove left recursion we get rules in the form A i A j ω (i < j), A i aω (a T ), Z i ω rules with A i (original variables) only in the form A i aω iteratively join the rules for i from n to 1 (for n already holds) rules with Z i (new variables) only in the form Z i aω In none rule for Z i the body begins with Z j either it is in required form or we join it with the rule A j aω we remove terminals inside rules. Automata and Grammars Normal Forms, Pumping Lemma 8 September 19, / 294

197 Reduction to GreibachNF Example Original Grammar E E + T T T T F F F (E) a (almost) Greibach Normal Form E (E) a (E)T at (E)E ae (E)T E at E E +T + TE T (E) a (E)T at T F FT F (E) a Left Recursion Removed E T TE E +T + TE T F FT T F FT F (E) a Greibach Normal Form E (EP a (EPT at (EPE ae (EPT E at E E +T + TE T (EP a (EPT at T F FT F (EP a P ) Automata and Grammars Normal Forms, Pumping Lemma 8 September 19, / 294

198 Closure Properties of CFLs Theorem (CFLs are closed on substitution) If L is a CFL over Σ, and s is a substitution on Σ such that s(a) is a CFL for each a Σ, then s(l) is CFL. Proof. Idea: Replace each a by the start symbol of a CFG for language s(a). First, rename variables to be unique in all G = (V, T, P, S), G a = (V a, T a, P a, S a ), a Σ. We construct a new grammar G = (V, T, P, S) for s(l): V = V a Σ Va T = a Σ Ta P = Pa {p P with all a replaced by Sa}. a Σ G generates the language s(l). Automata and Grammars Normal Forms, Pumping Lemma 8 September 19, / 294

199 Cocke-Younger-Kasami algorithm for testing a membership in a CFL Inefficient: exponential in w (check all derivation trees of appropriage depth of ChNF grammar for L. Definition (CYG Algorithm, time O(n 3 )) The input is ChNF grammar G = (V, T, P, S) for language L and a string w = q 1 a 2... a n T. Produce triangular table (right), horizontal axis is w X ij is the set of variables A such that A a ia i+1... a j. Fill the table upwards. X ij = {A BC; B X ik, C X k+1,j } X 15 X 14 X 25 X 13 X 24 X 35 X 12 X 23 X 34 X 45 X 11 X 22 X 33 X 44 X 55 a 1 a 2 a 3 a 4 a 5 BASIS: X ii = {A; A a i P} Automata and Grammars Normal Forms, Pumping Lemma 8 September 19, / 294

200 {S, A, C} - {S, A, C} - {B} {B} {S, A} {B} {S, C} {S, A} {B} {A, C} {A, C} {B} {A, C} b a a b a Automata and Grammars Normal Forms, Pumping Lemma 8 September 19, / 294

201 Cocke-Younger-Kasami algorithm for membership in CFL Exponentially to w : try all parse trees of the appropriate length for L. Algorthm: CYK algorithm, in time O(n 3 ) Let us have a ChNF grammar G = (V, T, P, S) generating the language L and a word w = a 1 a 2... a n T. We create triangular table (right), horizontal axis is w X ij are sets of variables A so that A a ia i+1... a j. Basis: X ii = {A; A a i P} Induction: X ij = {A BC; B X ik, C X k+1,j } We fill the table bottom up. If S X 1,n then w L(G). X 15 X 14 X 25 X 13 X 24 X 35 X 12 X 23 X 34 X 45 X 11 X 22 X 33 X 44 X 55 a 1 a 2 a 3 a 4 a 5 Automata and Grammars CYK algorithm, Closure properties of CFLs 9 September 19, / 294

202 Example (CYK algoritmus) Gramatika S AB BC A BA a B CC b C AB a Filled from the bottom up: {S, A, C} - {S, A, C} - {B} {B} {S, A} {B} {S, C} {S, A} {B} {A, C} {A, C} {B} {A, C} b a a b a Automata and Grammars CYK algorithm, Closure properties of CFLs 9 September 19, / 294

203 Substitution and homomorphism These definitions already appeared: Definition (substitution, homomorphism, inverse homomorphism) Let us have a language L over the alphabet Σ. Substitution σ; a Σ : σ(a) = L a language over the alphabet Σ a, tj. σ(a) Σ a maps words to languages: σ(λ) = {λ}, σ(a 1... a n ) = σ(a 1 ).....σ(a n ) (concatenation), that is σ : Σ P(( a Σ Σ a) ) σ(l) = w L σ(w). homomorphism h, a Σ : h(a) Σ a maps words to words h(λ) = {λ}, h(a 1... a n ) = h(a 1 ).....h(a n ) (concatenation), that is h : Σ ( a Σ Σ a) h(l) = {h(w) w L}. Inverse homomorphism maps words backwards h 1 (L) = {w h(w) L}. Automata and Grammars CYK algorithm, Closure properties of CFLs 9 September 19, / 294

204 Closure properties of Context Free languages Theorem (CFLs are closed under substitution) Let s have a CFL L over Σ and a substitution σ on Σ such that σ(a) is CFL a Σ. The also σ(l) is context free language (CFL). Proof: Idea: leaves in the parse tree generate further trees. We rename all variables to unique names in all grammars G = (V, T, P, S), G a = (V a, T a, P a, S a ), a Σ. We construct a new grammar G = (V, T, P, S) for σ(l): V = V a Σ Va T = a Σ Ta P = Pa {p P with all a replaced by Sa}. a Σ G generates the language σ(l). Automata and Grammars CYK algorithm, Closure properties of CFLs 9 September 19, / 294

205 Substitution on CFL s Example (substitution) L = {a i b j 0 i j} S asb Sb λ σ(a) = L 1 = {c i d i i 0} S 1 cs 1 d λ σ(b) = L 2 = {c i i 0} S 2 cs 2 λ σ(l): S S 1 SS 2 SS 2 λ, S 1 cs 1 d λ, S 2 cs 2 λ Theorem (homomorphism) Context free languages are closed under homomorphism. Proof: Direct consequence: homomorphism is a special case of the substitution. Each terminal a in the parse tree replace by h(a). Automata and Grammars CYK algorithm, Closure properties of CFLs 9 September 19, / 294

206 CFL are closed under inverse homomorphism Theorem (CFL are closed under inverse homomorphism) Let us have a CFL language L an a homomorphism h. Then h 1 (L) is also context free. If L is deterministic CFL, h 1 (L) is also deterministic CFL. Idea: use PDA recognizing the language L read a letter a, place h(a) into an inner buffer simulate M, the input is taken from the buffer if the buffer is empty, read next letter from the real input the input w is accepted iff the buffer is empty and M is in an accepting state! buffer is finite therefore it can be modeled in the finite state (states are tuples, the state and the buffer). Automata and Grammars CYK algorithm, Closure properties of CFLs 9 September 19, / 294

207 Proof: for L we have a PDA M = (Q, Σ, Γ, δ, q 0, Z 0, F ) (by final state) h : Ξ Σ we define PDA M = (Q, Ξ, Γ, δ, [q 0, λ], Z 0, F {λ}) where Q = {[q, u] q Q, u Σ, (a Ξ) (v Σ )h(a) = vu} u is the buffer δ ([q, u], λ, Z) = {([p, u], γ) (p, γ) δ(q, λ, Z)} {([p, v], γ) (p, γ) δ(q, b, Z), u = bv} reads the buffer δ ([q, λ], a, Z) = {([q, h(a)], Z)} fills the buffer For a deterministic PDA M is M also deterministic. Automata and Grammars CYK algorithm, Closure properties of CFLs 9 September 19, / 294

208 Further closure properties Theorem (CFL are closed under union, concatenation, closure, reverse) CFL are closed under union, concatenation, closure ( ), positive closure (+), reverse w R. Proof: Union: if V 1 V 2 rename variable to unique names, add new variable S new and the rule S new S 1 S 2 concatenation L 1.L 2 S new S 1S 2 (pro V 1 V 2 =,otherwise rename variables) iteration L = i 0 Li S new SS new λ positive iteration L + = i 1 Li S new SS new S reverse L R = {w R w L} X ω R reverse the right side of all rules. Automata and Grammars CYK algorithm, Closure properties of CFLs 9 September 19, / 294

209 Quotient with a regular language Lemma Context free languages are closed under left and right quotient with regular languages. Proof: FA A 1 = (Q 1, Σ, δ 1, q 1, F 1 ) PDA M 2 = (Q 2, Σ, Γ, δ 2, q 2, Z 0, F 2 ) R \ L = {w u R uw L}, L/R = {u w R uw L} Idea: PDA and FA run in parallel, do not read the input if FA is in an accepting state, PDA begins to read the real input we define PDA M = (Q, Σ, Γ, δ, (q 1, q 2 ), Z 0, F 2 ) where Q = (Q 1 Q 2 ) Q 2 couples of states for parallel run δ((p, q), λ, Z) = {((p, q ), γ) (a Σ)p δ 1 (p, a)&(q, γ) δ 2 (q, a, Z) {((p, q ), γ) (q, γ) δ 2 (q, λ, Z)} {(q, Z) p F 1 } δ(q, a, Z) = δ 2 (q, a, Z), a Σ {λ}, q Q 2, Z Γ obviously L(M) = L(A 1 ) \ L(M 2 ). Right quotient from closure properties of reverse and the left quotient L/M = (M R \ L R ) R. Automata and Grammars CYK algorithm, Closure properties of CFLs 9 September 19, / 294

210 Intersection of context free languages Example (CFL are not closed under intersection) The language L = {0 n 1 n 2 n n 1} = {0 n 1 n 2 i n, i 1} {0 i 1 n 2 n n, i 1} is not CFL, even though both members of the intersection are CFL s, they are also {0 deterministic CFLs. n 1 n 2 i n, i 1} {S AC, A 0A1 λ, C 2C λ} {0 i 1 n 2 n n, i 1} {S AB, A 0A λ, B 1B2 λ} the intersection is not CFL from the pumping lemma. two PDAs in parallel FA unit can be joined (as in the finite automata) reading the input can be joined (one automaton can wait) two stacks is not possible to join to one stack two general stacks =Turing machine = recursively enumerable languages L 0 Automata and Grammars CYK algorithm, Closure properties of CFLs 9 September 19, / 294

211 Intersection of an CFL an a regular language Theorem (Both CFL and DCFL are closed under intersection with a regular language) Let s have a CFL L and a regular language R. Then L R is context free. Let s have a deterministic CFL L and a regular language R. Then L R is deterministic CFL. Proof: PDA and FA can be joined FA A 1 = (Q 1, Σ, δ 1, q 1, F 1) PDA accepts by the final state M 1 = (Q 2, Σ, Γ, δ 2, q 2, Z 0, F 2) new automaton M = (Q 1 Q 2, Σ, Γ, δ, (q 1, q 2 ), Z 0, F 1 F 2 ) ((r, s), α) δ((p, q), a, Z) iff a λ: r = δ 1(p, a)&(s, α) δ 2(q, a, Z)... automata read the input a = λ: (s, α) δ 2(q, λ, Z) PDA changes the stack r = p FA does not move obviously L(M) = L(A 1 ) L(M 2 ) automata run in parallel. Automata and Grammars CYK algorithm, Closure properties of CFLs 9 September 19, / 294

212 Not context free prove from closure properties Example The language L = {0 i 1 j 2 k 3 l i = 0 j = k = l} is not CFL. Proof: Proof by contradiction: Assume L to be CFL L 1 = {01 j 2 k 3 l i, j, k 0} is a regular language {S 0B, B 1B C, C 2C D, D 3D λ} L L 1 = {01 i 2 i 3 i i 0} is not CFL CONTRADICTION L is context language S B 1 0A B 1 1B 1 C 1, C 1 2C 1 D 1, D 1 3D 1 λ A 0A P P 1PCD λ DC CD we rewrite to {DC XC, XC XY, XY CY, CY CD} 1C 12, 2C 22, 2D 23, 3D 33 Automata and Grammars CYK algorithm, Closure properties of CFLs 9 September 19, / 294

213 Difference and the complement Theorem (Difference with a regular language) Let s have a CFL language L and a regular language R. Then: Proof. L R is CFL. L R = L R, R is regular. Theorem (CFL are not closed under difference nor complement) The class of the context free languages is not closed under the difference nor the complement. CFL are not closed under difference nor complement. Let us have CFL languages L, L 1, L 2, and a regular language R. Then: L is not necessary context free. L 1 L 2 = L 1 L 2. L 1 L 2 is not necessary context free. CFL. Σ L not necessary a Automata and Grammars CYK algorithm, Closure properties of CFLs 9 September 19, / 294

214 Closure properties of deterministic CFL s Lemma Good programming languages are deterministic CFL s. Deterministic context free languages are not closed under intersection are closed under intersection with a regular language are closed under inverse homomorphism. The complement of a deterministic CFL is a deterministic CFL. Proof: idea: accept iff q Q F undefined steps catched by new bottom of the stack Z B cycle can be recognized by a counter when reading the input is finished check transition through an accepting state (may be more λ transitions) if an accepting state was visited, original PDA accepted, do not accept. Automata and Grammars CYK algorithm, Closure properties of CFLs 9 September 19, / 294

215 Deterministic CFL s are not closed under union Example (DCFL are not closed under union) The language L = {a i b j c k i j j k i k} is a CFL, but not a deterministic CFL. Proof. Due to closer of DCFL s under complement would be deterministic CFL also L a b c = {a i b j c k i = j = k}, and we know it is not a CFL (pumping lemma) Example (DCFL are not closed under homomorphism) Languages L 1 = {a i b j c k i = j}, L 2 = {a i b j c k j = k} are deterministic context free. The language 0L 1 1L 2 is deterministic context free The language 1L 1 1L 2 is not a deterministic CFL let be h(0) = 1 h(x) = x for other symbols h(0l 1 1L 2 ) = 1L 1 1L 2. Automata and Grammars CYK algorithm, Closure properties of CFLs 9 September 19, / 294

216 Closure properties in brief language regular (RL) context free deterministic CFL union YES YES NO intersection YES NO NO with RL YES YES YES complement YES NO YES homomorphism YES YES NO inverse hom. YES YES YES Automata and Grammars CYK algorithm, Closure properties of CFLs 9 September 19, / 294

217 Chomsky hierarchy Grammar types according to productions allowed. Type 0 (recursively enumerable languages L 0 ) general rules α β, α, β (V T ), α contains at least one variable Type 1 (context languages L 1 ) productions of the form αaβ αωβ A V, α, β (V T ), ω (V T ) + with only exception S λ, then S does not appear at the right side of any production Type 2 (context free languages L 2 ) productions of the form A ω, A V, ω (V T ) Type 3 (regular (right linear) languages L 3 ) productions of the form A ωb, A ω, A, B V, ω T Automata and Grammars CYK algorithm, Closure properties of CFLs 9 September 19, / 294

218 Context Grammars productions in the form αaβ αωβ A V, α, β (V T ), ω (V T ) + with one exception S λ, in this case S does not appear in the body (right side) of any rule variable A rewrites only in the context α, β S λ is used only to add λ to the language Example (a context language) L = {a n b n c n n 1} is a context language that is not context free. Grammar: S asbc abc CB BC (*) is not a context rule, will be rewriten bb bb bc bc cc cc (*) CB BC we replace this rule by CB XB, XB XY, XY BY, BY BC Automata and Grammars CYK algorithm, Closure properties of CFLs 9 September 19, / 294

219 Separated grammars Definition (Separated grammar) A grammar is separated, iff all productions are in the form α β, where: Lemma either α, β V + (non empty sequences of variables) or α V and β T {λ}. For any grammar G there exists equivalent separated grammar G. Proof: Let G = (V, T, P, S) for each terminal a V we introduce new variable A. in productions in P we replace terminals by corresponding variables we add rules A a Resulting grammar is separated and obviously L(G) = L(G ). Automata and Grammars CYK algorithm, Closure properties of CFLs 9 September 19, / 294

220 From monotony to context productions Definition (monotone grammar) A grammar is monotone, iff for each rule (α β) P holds α β. Monotone grammar never shortens sentential form during the derivation. Lemma To any monotone grammar there exists an equivalent context grammar. Proof: first, find an equivalent separated grammar this does not violate the monotony (and rules A a are context rules) remaining productions A 1... A m B 1... B n, m n replace by productions with new variables C A 1A 2... A m C 1A 2... A m C 1C 2... C m B 1C 2... C m C 1A 2... A m C 1C 2... A m B 1C 2... C m B 1B 2... C m.... C 1... C m 1A m C 1... C m 1C m B 1... B m 1C m B 1... B m 1B m... B n Automata and Grammars CYK algorithm, Closure properties of CFLs 9 September 19, / 294

221 Context language example Example The language L = {a i b j c k 1 i j k} is a context language, that is not a context free language. Proof: S asbc abc generate sybols a B BBC adds symbols B C CC adds symbols C CB BC ordering of B s and C s (*) ab ab start rewriting of B to b bb bb continue rewriting B to b bc bc start rewriting C to c cc cc continue rewriting C to c (*) CB BC is not a context rule, we replace it with CB XB, XB XY, XY BY, BY BC Automata and Grammars CYK algorithm, Closure properties of CFLs 9 September 19, / 294

222 Introduction to Turing Machines General model of any computer We aim to show problems undecidable by any computer. Automata and Grammars Turing Machines 10 September 19, / 294

223 Turing Machine Definition (Turing Machine) Turing Machine (TM) is the 7 tuple M = (Q, Σ, Γ, δ, q 0, B, F ) with the components: Q The finite set of states of the finite control. Σ The finite set of input symbols. Γ The complete set of tape symbols. Always Γ Σ, Q Γ =. δ The partial transition function (Q F ) Γ Q Γ {L, R}. δ(q, x) = (p, Y, D), where: q Q is the current state. X Γ is the current tape symbol. p is the next state, p Q. Y Γ is written in the cell being scanned, replacing anything there. D {L, R} is a direction in which the head moves (left, right). q 0 Q is the start state. B Γ \ Σ. It appears initially in all but the finite number of initial cells that hold input symbols. F Q The set of final or accepting states. Note there are no transitions for accepting states. Automata and Grammars Turing Machines 10 September 19, / 294

224 Instantaneous Description for TM Definition (Instantaneous Description (ID) for TM) Instantaneous Description (ID) for a Turing machine is a string X 1 X 2... X i 1 qx i X i+1... X n where q is the state of the Turing machine. The tape head is scanning the ith symbol from the left. X 1... X n is the portion of the tape between the leftmost and the rightmost nonblank. As an exception, if the head is at any end one blank is added on that side. Definition (Moves of a TM) We describe moves of a TM M by the M, M, notation as used for PDA s. For δ(q, X i ) = (p, Y, R) X 1 X 2... X i 1 qx i X i+1... X n M X 1 X 2... X i 2 px i 1 YX i+1... X n For δ(q, X i ) = (p, Y, R) X 1 X 2... X i 1 qx i X i+1... X n M X 1 X 2... X i 1 YpX i+1... X n. Automata and Grammars Turing Machines 10 September 19, / 294

225 A TM for {0 n 1 n ; n 1} Definition (TM accepts the language) A TM M = (Q, Σ, Γ, δ, q 0, B, F ) accepts the language L(M) = {w Σ : q 0 w M αpβ, p F, α, β Γ }. Example A TM M = ({q 0, q 1, q 2, q 3, q 4 }, {0, 1}, {0, 1, X, Y, B}, δ, q 0, B, {q 4 }) with δ in the table accepts the language {0 n 1 n ; n 1}. State 0 1 X Y B q 0 (q 1, X, R) (q 3, Y, R) q 1 (q 1, 0, R) (q 2, Y, L) (q 1, Y, R) q 2 (q 2, 0, L) (q 0, X, R) (q 2, Y, L) q 3 (q 3, Y, R) (q 4, B, R) q 4 Automata and Grammars Turing Machines 10 September 19, / 294

226 Recursively enumerable languages (RE) Definition (Recursively enumerable languages (RE)) The set of languages that we can accept by a Turing machine. Definition Turing Machine halts A TM halts if it enters a state q, scanning a tape symbol X, and there is no move in this situation, i.e., δ(q, X) is undefined. We assume that a TM always halt when it is in an accepting state. We can require that a TM halts even if it does not accept only for recursive languages, a proper subset of recursively enumerable languages. Automata and Grammars Turing Machines 10 September 19, / 294

227 Transition Diagrams for Turing Machines Definition (Transition diagram) A transition diagram consists of a set of nodes corresponding to the states of the TM. Any arc q p is labeled the list of items X/YD for all δ(q, X) = (p, Y, D), D {, }. We assume that the blank symbol is B unless we state otherwise. State 0 1 X Y B q 0 (q 1, X, R) (q 3, Y, R) q 1 (q 1, 0, R) (q 2, Y, L) (q 1, Y, R) q 2 (q 2, 0, L) (q 0, X, R) (q 2, Y, L) q 3 (q 3, Y, R) (q 4, B, R) q 4 Automata and Grammars Turing Machines 10 September 19, / 294

228 A TM for {0 n 1 n ; n 1} Word 0011 q Xq X0q 1 11 Xq 2 0Y 1 q 2 X0Y 1 Xq 0 0Y 1 XXq 1 Y 1 XXYq 1 1 XXq 2 YY Xq 2 XYY XXq 0 YY XXYq 3 Y XXYYq 3 B XXYYBq 4 B Word 0010 q Xq X0q 1 10 Xq 2 0Y 0 q 2 X0Y 0 Xq 0 0Y 0 XXq 1 Y 0 XXYq 1 0 XXY 0q 1 B ends up with a failure since there is no instruction for q 1, 0. Automata and Grammars Turing Machines 10 September 19, / 294

229 A TM with Output A TM that computes monus, proper substraction m. n = max(m n, 0). M = ({q 0, q 1, q 2, q 3, q 4, q 5, q 6 }, {0, 1}, {0, 1, B}, δ, q 0, B), accepting set omited (TM used for output, not acceptance). Start tape 0 m 10 n. M halts with the tape 0 m. n surrounded by blanks. Find leftmost 0, replace it by a blank. Search right, looking for a 1; continue, find 0 and replace it by 1. Return left. End if no 0 found, either left or right; right: replace all 1 by B. left: m < n: replace all 1 and 0 by B, leave the tape blank. Automata and Grammars Turing Machines 10 September 19, / 294

230 Storage in the FA unit Storage in the State Consider state as a tuple M = ({q 0, q 1 } {0, 1, B}, {0, 1}, {0, 1, B}, δ, [q 0, B], B, {[q 1, B]}) L(M) = ( ), δ 0 1 B [q 0, B] ([q 1, 0], 0, R) ([q 1, 1], 1, R) [q 1, 0] ([q 1, 0], 1, R) ([q 1, B], B, R) [q 1, 1] ([q 1, 1], 0, R) ([q 1, B], B, R) [q 1, B] Automata and Grammars Turing Machines 10 September 19, / 294

231 Multiple Tracks L wcw = {wcw w (0 + 1) + }, M = ({q 0,..., q 9 } {0, 1, B}, {[B, 0], [B, 1], [B, c]}, {B, } {0, 1, B, c}, δ, [q 1, B], [B, B], {[q 9, B]}) δ is defined as (a, b {0, 1}): δ([q 1, B], [B, a]) = ([q 2, a], [, a], R) picks up the symbol a δ([q 2, a], [B, b]) = ([q 2, a], [B, b], R) move right, look for c, δ([q 2, a], [B, c]) = ([q 3, a], [B, c], R) continue right, the state changed, δ([q 3, a], [, b]) = ([q 3, a], [, b], R) continue right, δ([q 3, a], [B, a]) = ([q 4, B], [, a], L) check correct, drop memory and go left, δ([q 4, B], [, a]) = ([q 4, B], [, a], L) go left, δ([q 4, B], [B, c]) = ([q 5, B], [B, c], L) c found, continue left, decide whether all inputs left and right are checked, branch adequately δ([q 5, B], [B, a]) = ([q 6, B], [B, a], L) left symbol unchecked, δ([q 6, B], [B, a]) = ([q 6, B], [B, a], L) proceed left, δ([q 6, B], [, a]) = ([q 1, B], [, a], R) start again, δ([q 5, B], [, a]) = ([q 7, B], [, a], R) symbol left from c checked, go right, δ([q 7, B], [B, c]) = ([q 8, B], [B, c], R) proceed right, δ([q 8, B], [, a]) = ([q 8, B], [, a], R) proceed right, δ([q 8, B], [B, B]) = ([q 8, B], [B, B], R) accept. Automata and Grammars Turing Machines 10 September 19, / 294

232 Theorem (Recursively enumerable languages are of the type L 0 ) Any recursively enumerable language is of the type 0. Proof: From a Turing Machine to the Grammar for any given Turing Machine T we construct a grammar G, L(T ) = L(G) first, we generate the tape and a copy of the input word then, we simulate TM T (states are inside the word) at the end, we erase the tape leaving only the input word w wb n w R q 0B m where B i is an empty space for TM work 1) S DQ 0 E D xdx E generates the word and a reverse copy E BE B generates empty space 2) XPY QX Y for δ(p, x) = (q, x, R) XPY X Y Q for δ(p, x) = (q, x, L) 3) P C pro p F CA C,AC C cleaning A Σ {B} C λ end of derivation Automata and Grammars Turing Machines 10 September 19, / 294

233 From TM to Grammar Ještě L(T ) = L(G)? w L(T ) there exists a finite accepting sequence of steps T (finite space) the grammar generates sufficient space it simulates the moves and cleans non input symbols w L(G) the rules need not be in our order we may sort the derivation steps to ordering 1),2),3). underlined symbols were erased, therefore accepting state must be generated. Grammar after simplification Example q 0, B q F, B, R q 0, a q 1, a, R q 1, a q 0, a, R S Dq 0 D ada B Bq 0 C aq 0 q 1 a aq 1 q 0 a Ca C C λ Automata and Grammars Turing Machines 10 September 19, / 294

234 From grammars to TM Theorem Any Type 0 language is recursively enumerable. Proof: the idea: TM generates sequentially all possible derivations we code S ω 1... ω n = w as a word #S#ω 1 #... #w# we can construct a TM accepting words #α#β# where α β we can construct a TM accepting words #ω 1 #... #ω k #, kde ω 1 ω k we can construct a TM generating sequentially all possible words. generate (next) word word represents a derivation yes derivation ends with w yes no no Automata and Grammars Turing Machines 10 September 19, / 294

235 TM extension: Multitape TM Definition (Multitape Turing Machine) Initial posisiton the input word on the first tape, other tapes blank the first head left from the input, others anywhere FA control in the initial state One step of a multitape TM FA control moves to the new state on each tape we write the appropriate new symbol each head moves independently left or right. Multitape TM Theorem (Multitape TM) Any language accepted by a multitape TM is accepted also by some (one tape) Turing machine TM. Automata and Grammars Turing Machines 10 September 19, / 294

236 Proof: Multitape TM TM we construct a TM M the tape will have 2k tracks odd tracks: i th head position even tracks: symbols on the i th tape to simulate one step we wisit all heads we remember in the FA control state the number of heads left i symbol under i th head then we know enough to simulate one step (visit all heads again) 2 tape TM simulation The simultation of a k tape machine with n steps can be performed in O(n 2 ) (one step simulation takes 4n + 2k, heads are no further than 2n read, write, move the head position marks). Automata and Grammars Turing Machines 10 September 19, / 294

237 Extension: Nondeterministic Turing Machines Definition (Nondeterministic TM) Nondeterministic Turing machine is a 7 tuple M = (Q, Σ, Γ, δ, q 0, B, F ) with Q, Σ, Γ, q 0, B, F as TM and δ : (Q F ) Γ P(Q Γ {L, R}). A word w Σ is accepted by the nondeterminisic TM M iff there exists accepting sequence of moves q 0 w αpβ, p F. Theorem (Nondeterministic TM) To any nondeterministic Turing macnine M N there exist a deteministic TM M D such that L(M N ) = L(M D ). The main idea breath search all calculations of M N derived in k steps no more that m k IDs (instantaneous descriptions) where m = max δ(q, x) the maximum number of choices in M N Automata and Grammars Turing Machines 10 September 19, / 294

238 Proof: proof idea the tape is infinite subset construction is not possible breath search of all possible calculations of M N we simulate by a two tape TM first tape: a sequence of IDs current position marked (by a cross) left already evaluated we may erase right the current ID and a waiting list second tape: scratch tape to evaluate one ID means read the state and the head symbol of current ID if in the accepting state F, accept and halt write ID on the scratch tape for each step δ (stored in the head M D ) perform the step and write the new ID to the end of the first tape visit the marked ID, erase the mark and move it 1 ID to the right repeat Automata and Grammars Turing Machines 10 September 19, / 294

239 Summary The Turing Machine Acceptance by a Turing Machine: Start: finite length string of tape symbols, the rest is blank. The TM accepts its input if it ever enters an accepting state. Recursively Enumerable Languages (RE): the languages accepted by a TM. Instantaneous Descriptions of a TM: All tape symbols between the leftmost and the rightmost nonblank. The state and the position of the head are shown by placing the state within the sequence of tape symbols, just to the left of the cell scanned. Storage in the Finite Control: The state has two or more components, one for the control, others the storage. Multiple Tracks The tape symbols are vectors with a fixed number of components. Multitape TM An extended TM model has some fixed number of tapes greater than one. Although able to recognize certain languages faster than the conventional TM, it cannot recognize any language that is not RE. Nondeterministic Turing Machines: The NTM has a finite number of choices of next move. It accepts an input if any sequence of choices leads to an ID with an accepting state. It does not recognize any language that is not RE. Automata and Grammars Turing Machines 10 September 19, / 294

240 Subroutines Multiplication: Input: 0 m 10 n 1, Output: 0 mn. Strategy: On the tape generally 0 i 10 n 10 kn In one basic step, change a 0 in the first group to B and add n 0 s to the last group, giving us the string of the form 0 i 1 10 n 10 (k+1)n. When finished, change the leading 10 n 1 to blanks. Copy Multiply Automata and Grammars Turing Machines 10 September 19, / 294

241 Restricted Turing Machines X 0 X 1 X 2... * X 1 X 2... Theorem (Semi infinite Tape, Never Writes a Blank) Every language accepted by a TM M 2 is also accepted by a TM M 1 with the following restrictions: M 1 s head never moves left of its initial position. M 1 never writes a blank. Automata and Grammars Turing Machines 10 September 19, / 294

242 TM extension: Multitape TM Definition (Multitape Turing Machine) Initial posisiton the input word on the first tape, other tapes blank the first head left from the input, others anywhere FA control in the initial state One step of a multitape TM FA control moves to the new state on each tape we write the appropriate new symbol each head moves independently left or right. Multitape TM Theorem (Multitape TM) Any language accepted by a multitape TM is accepted also by some (one tape) Turing machine TM. Automata and Grammars Linear Bounded Automata 11 September 19, / 294

243 Proof: Multitape TM TM we construct a TM M the tape will have 2k tracks odd tracks: i th head position even tracks: symbols on the i th tape to simulate one step we wisit all heads we remember in the FA control state the number of heads left i symbol under i th head then we know enough to simulate one step (visit all heads again) 2 tape TM simulation The simultation of a k tape machine with n steps can be performed in O(n 2 ) (one step simulation takes 4n + 2k, heads are no further than 2n read, write, move the head position marks). Automata and Grammars Linear Bounded Automata 11 September 19, / 294

244 Extension: Nondeterministic Turing Machines Definition (Nondeterministic TM) Nondeterministic Turing machine is a 7 tuple M = (Q, Σ, Γ, δ, q 0, B, F ) with Q, Σ, Γ, q 0, B, F as TM and δ : (Q F ) Γ P(Q Γ {L, R}). A word w Σ is accepted by the nondeterminisic TM M iff there exists accepting sequence of moves q 0 w αpβ, p F. Theorem (Nondeterministic TM) To any nondeterministic Turing macnine M N there exist a deteministic TM M D such that L(M N ) = L(M D ). The main idea breath search all calculations of M N derived in k steps no more that m k IDs (instantaneous descriptions) where m = max δ(q, x) the maximum number of choices in M N Automata and Grammars Linear Bounded Automata 11 September 19, / 294

245 Proof: proof idea the tape is infinite subset construction is not possible breath search of all possible calculations of M N we simulate by a two tape TM first tape: a sequence of IDs current position marked (by a cross) left already evaluated we may erase right the current ID and a waiting list second tape: scratch tape to evaluate one ID means read the state and the head symbol of current ID if in the accepting state F, accept and halt write ID on the scratch tape for each step δ (stored in the head M D ) perform the step and write the new ID to the end of the first tape visit the marked ID, erase the mark and move it 1 ID to the right repeat Automata and Grammars Linear Bounded Automata 11 September 19, / 294

246 Restricted Turing Machines X 0 X 1 X 2... * X 1 X 2... Theorem (Semi infinite Tape, Never Writes a Blank) Every language accepted by a TM M 2 is also accepted by a TM M 1 with the following restrictions: M 1 s head never moves left of its initial position. M 1 never writes a blank. Automata and Grammars Linear Bounded Automata 11 September 19, / 294

247 Context Grammars productions in the form αaβ αωβ A V, α, β (V T ), ω (V T ) + with one exception S λ, in this case S does not appear in the body (right side) of any rule variable A rewrites only in the context α, β S λ is used only to add λ to the language Example (a context language) L = {a n b n c n n 1} is a context language that is not context free. Grammar: S asbc abc CB BC (*) is not a context rule, will be rewriten bb bb bc bc cc cc (*) CB BC we replace this rule by CB XB, XB XY, XY BY, BY BC Automata and Grammars Linear Bounded Automata 11 September 19, / 294

248 Separated grammars Definition (Separated grammar) A grammar is separated, iff all productions are in the form α β, where: Lemma either α, β V + (non empty sequences of variables) or α V and β T {λ}. For any grammar G there exists equivalent separated grammar G. Proof: Let G = (V, T, P, S) for each terminal a V we introduce new variable A. in productions in P we replace terminals by corresponding variables we add rules A a Resulting grammar is separated and obviously L(G) = L(G ). Automata and Grammars Linear Bounded Automata 11 September 19, / 294

249 From monotony to context productions Definition (monotone grammar) A grammar is monotone, iff for each rule (α β) P holds α β. Monotone grammar never shortens sentential form during the derivation. Lemma To any monotone grammar there exists an equivalent context grammar. Proof: first, find an equivalent separated grammar this does not violate the monotony (and rules A a are context rules) remaining productions A 1... A m B 1... B n, m n replace by productions with new variables C A 1A 2... A m C 1A 2... A m C 1C 2... C m B 1C 2... C m C 1A 2... A m C 1C 2... A m B 1C 2... C m B 1B 2... C m.... C 1... C m 1A m C 1... C m 1C m B 1... B m 1C m B 1... B m 1B m... B n Automata and Grammars Linear Bounded Automata 11 September 19, / 294

250 Context language example Example The language L = {a i b j c k 1 i j k} is a context language, that is not a context free language. Proof: S asbc abc generate sybols a B BBC adds symbols B C CC adds symbols C CB BC ordering of B s and C s (*) ab ab start rewriting of B to b bb bb continue rewriting B to b bc bc start rewriting C to c cc cc continue rewriting C to c (*) CB BC is not a context rule, we replace it with CB XB, XB XY, XY BY, BY BC Automata and Grammars Linear Bounded Automata 11 September 19, / 294

251 Automata and Grammars Chomsky hierarchy ( bottom up ) finite automata regular grammars pushdown automata context free grammars lineary bounded automata context grammars Turing machines grammars Type 0 Grammars Type 1 (context languages L 1 ) all productions in the form αaβ αωβ A V, α, β (V T ), ω (V T ) + with one exception S λ, in this case S does not appear in the body of any rule Automata and Grammars Linear Bounded Automata 11 September 19, / 294

252 Linear Bounded Automata Definition (Linear Bounded automaton (LBA)) Linear Bounded Automaton LBA is a nondeterministic TM, with left and right endmark on the tape l, r. These symbols are not possible to rewrite and cannot be stepped out during evaluation (left from l and right from r). A string w is accepted by a LBA,= iff q 0 lwr αpβ, p F. The space is defined by the input string and the LBA cannot use more space monotone (context) derivations do not need more space no sentential form in the derivation is longer then the resulting word Automata and Grammars Linear Bounded Automata 11 September 19, / 294

253 From context languages to LBA Theorem Each context language can be accepted by a LBA. Proof: from a context grammar to LBA derivation in the grammar will be simulated by a LBA we use two track tape upper track: the string w, lower track: S (start symbol of the grammar followed by blanks) l S w Aplikace pravidla αxβ αγβ u α X β v u α γ β v we rewrite the sentential form on the second tape by productions in G we nondeterministically choose the part to rewrite we rewrite appropriate part (the right side may be moved to the right) if only terminals on the second track, we compare it with the first track accept or reject according the result of the comparison r Automata and Grammars Linear Bounded Automata 11 September 19, / 294

254 From LBA to context languages Theorem LBA s accept only context languages. Proof: From LBA to context grammars we reduce LBA to monotone grammar the grammar cannot use more space than the resulting string of terminals. we hide the steps into two tracks variables 1) generate string in the form (a 0, [q 0, l, a 0 ]), (a 1, a 1 ),..., (a n, [a n, r]) w q 0, l, a 0 a n, r 2) simulate LBA steps on the second track (similar to TM simulation) pro δ(p, x) = (q, x, R): PX X Q pro δ(p, x) = (q, x, L): Y PX QY X if in the accepting state, errase the second track acceptance of the empty string: If LBA accepts λ, we add the special rule. Automata and Grammars Linear Bounded Automata 11 September 19, / 294

255 Two way finite automata Definition (Two way finite automata) Two way deterministic finite automaton is a five tuple A = (Q, Σ, δ, q 0, F ), where Q is a finite set of states, Σ is a finite set of input symbols transition function δ is a mapping Q Σ Q { 1, 0, 1} extended by head transitions q 0 Q initial state a set of accepting states F Q. We may represent it by a graph or a table. Automata and Grammars Linear Bounded Automata 11 September 19, / 294

256 Two way DFA computation Definition (Two way DFA computation) A string w is accepted by the two way DFA, iff: computation started in the initial state at the left most symbol of w the first transition from w to the right was in an accepting state the computation is not defined outside the word w (computation ends without accepting w). We may add special end symbols # / Σ to any word If L(A) = {#w# w L Σ } is regular, then also L is regular L = # R # (L(A) #Σ #) Automata and Grammars Linear Bounded Automata 11 September 19, / 294

257 Two way automaton example Example (Two way automaton example) Let A = (Q, Σ, δ, q 1, F ). We define a two way DFA B = (Q Q Q {q 0, q N, q F }, Σ, δ, q 0, {q F }) accepting the language L(B) = {#u# uu L(A)} (it is neither left nor right quotient!): δ x Σ # remark q 0 q N, 1 q 1, +1 q 1 is starting in A q p, +1 q, 1 p = δ(q, x) q q, 1 q, +1 q p, +1 q F, +1 q F, p = δ(q, x) q p, +1 q N, +1 q / F, p = δ(q, x) q N q N, +1 q N, +1 q F q N, +1 q N, +1 Theorem Languages accepted by two way DFA are exactly regular languages. Automata and Grammars Linear Bounded Automata 11 September 19, / 294