Computability and Complexity

Computability and Complexity Push-Down Automata CAS 705 Ryszard Janicki Department of Computing and Software McMaster University Hamilton, Ontario, Canada janicki@mcmaster.ca Ryszard Janicki Computability and Complexity 1 / 45

Pushdown Automata (PDAs): A Preview a1 a2 a3 a4 a5 a6 a7 a8 an left to right, read only Q Finite Control push/pop A B C B Stack Ryszard Janicki Computability and Complexity 2 / 45

Pushdown Automata (PDAs): A Preview A nondeterministic pushdown automaton (NPDA) is like a nondeterministic nite automaton, except it has a stack Its input head is read-only and may only move right The stack is used in a last-in-rst-out (LIFO) fashion It can push symbols onto the top of the stack or pop them o the top of the stack In each step, the machine pops the top symbol o the stack Ryszard Janicki Computability and Complexity 3 / 45

Pushdown Automata (PDAs): A Preview Based on this symbol, the input symbol it is currently reading, and its current state, it can push a sequence of symbols onto the stack, move its read head one cell to the right, and enter a new state, according to the transition rules of the machine It allows also ɛ-transitions in which it can pop and push without reading the next input symbol or moving its read head Ryszard Janicki Computability and Complexity 4 / 45

Nondeterministic Pushdown Automaton Formal Denition A nondeterministic PDA is a 7-tuple M = (Q, Σ, Γ, δ, s 0,, F ), where Q is a nite set (the states), Σ is a nite set (the input alphabet), Γ is a nite set (the stack alphabet), δ : Q (Σ {ε}) Γ 2 Q Γ, is a function (the transition function), s 0 Q (the start state), Γ (the initial stack symbol), and F Q (the nal or accept states). Ryszard Janicki Computability and Complexity 5 / 45

Nondeterministic Pushdown Automaton Example (q, B 1 B 2 B k ) δ(p, a, A) means whenever the machine is in state p reading input symbol a on the input tape and A on the top of the stack then it can pop A o the stack, push B 1 B 2 B k onto the stack (B k rst and B 1 last), leave its read head where it is, and enter state q. Ryszard Janicki Computability and Complexity 6 / 45

Nondeterministic Pushdown Automaton Example (q, B 1 B 2 B k ) δ(p, ε, A) means whenever the machine is in state p and A on the top of the stack then it can pop A o the stack, push B 1 B 2 B k onto the stack (B k rst and B 1 last), leave its read head where it is, and enter state q. Ryszard Janicki Computability and Complexity 7 / 45

Congurations Denition A conguration of the machine M is an element of Q Σ Γ describing the current state, the portion of the input yet unread, and the current stack contents. A conguration gives complete information about the global state of M at some point during a computation In general, the set of congurations is innite The start conguration on input x is (s, x, ) Ryszard Janicki Computability and Complexity 8 / 45

Congurations The conguration might describe the following (p, baaabba, ABAC ) a b a b b a a a b b a p A B A C Stack Ryszard Janicki Computability and Complexity 9 / 45

Congurations The next conguration relation 1 describes how the M machine can move from one conguration to another in one step It is dened formally as follows: if (q, γ) δ(p, a, A) then y Σ, β Γ. (p, ay, Aβ) and if ((p, ɛ, A), (q, γ)) δ then y Σ, β Γ. (p, y, Aβ) 1 (q, y, γβ) M 1 (q, y, γβ) M Ryszard Janicki Computability and Complexity 10 / 45

Congurations Denition For C, D, E conguration of M, we dene the relations M as follows: n M and C 0 D def M C = D C n+1 D def E. C n E E 1 M M M D C M D def n 0. C n M D M is the reexive transitive closure of 1 M Ryszard Janicki Computability and Complexity 11 / 45

Acceptance There are two alternative denitions of acceptance in common use: by empty stack and by nal state It turns out that it doesn't matter which denition we use, since each kind of machine can simulate the other. Formally, M accepts x by nal state if (s, x, ) M (q, ε, γ) for some q F and γ Γ. Let L F (M) denote the language of all sequences accepted by M by nal state. Formally, M accepts x by empty stack if (s, x, ) M (q, ε, ε) for some q Q. Let L E (M) denote the language of all sequences accepted by M by empty stack. Ryszard Janicki Computability and Complexity 12 / 45

Acceptance: Example 1 Example A nondeterministic pushdown automaton that accepts the set of balanced strings of parentheses [ ] by empty stack. Q = {q}, Σ = {[, ]}, Γ = {, [}, start state = q, initial stack symbol = and δ is dened as follows: (i) δ(q, [, ) = {(q, [ )}; (ii) δ(q, [, [) = {(q, [[)}; (iii) δ(q, ], [) = {(q, ε)}; (iv) δ(q, ε, ) = {(q, ɛ)}. (v) δ(,, ) = for all other cases (usually dened by default) Ryszard Janicki Computability and Complexity 13 / 45

Acceptance: Example 1 A sequence of congurations leading to the acceptance of the balanced string [ [ [] ] [] ] [] Ryszard Janicki Computability and Complexity 14 / 45

Acceptance: Example 2 The pushdown automaton M below accepts the language L E (M) = {xcx R x {a, b} } with empty stack, where (a 1 a 2... a n ) R = a n a n 1... a 1. Ryszard Janicki Computability and Complexity 15 / 45

Acceptance: Example 3 The pushdown automaton M below accepts the language L E (M) = {xx R x {a, b} } with empty stack. Ryszard Janicki Computability and Complexity 16 / 45

Graphical Notation: meaning of a, b c We write a, b c to signify that when the automaton is reading an a from the input, it may replace the symbol b on the top of the stack with a c. Any of a, b, and c may be ε. If a is ε, the automaton may make this transition without reading any symbol from the input. If b is ε, the automaton may make this transition without reading and popping any symbol from the stack. If c is ε, the automaton does not write any symbol on the stack when going along this transition. Ryszard Janicki Computability and Complexity 17 / 45

Graphical Notation: Example 1 The pushdown automaton M below accepts the language L F (M) = {0 n 1 n n 0} with F = {q 1, q 4 }. Ryszard Janicki Computability and Complexity 18 / 45

Graphical Notation: Example 2 The pushdown automaton M below accepts the language L F (M) = {a i b j c k i, j, k 0 and i = j or i = k} with F = {q 4, q 7 }. Ryszard Janicki Computability and Complexity 19 / 45

From Final States To Empty Stack Theorem For every pushdown automaton M there is a pushdown automaton M such that: L F (M) = L E (M ). Proof. (sketch). We just have to pop stuck until empty when in nal state. Let M = (Q, Σ, Γ, δ, s 0,, F ). Dene M = (Q {s e, s 0 }, Σ, Γ { }, δ, s 0,, ), where 1 δ (s 0, ε, ) = {(s 0, )} 2 q Q, a Σ {ε}, Z Γ. δ (q, a, Z) = δ(q, a, Z) 3 q F, Z Γ. δ (q, ε, Z) = δ(q, ε, Z) {(s e, ε)} 4 Z Γ { }. δ (s e, ε, Z) = {(s e, ε)}. Clearly L F (M) = L E (M ). Ryszard Janicki Computability and Complexity 20 / 45

From Empty Stack To Final States Theorem For every pushdown automaton M there is a pushdown automaton M such that: L E (M) = L M (M ). Proof. (sketch). We just have to jump to nal state when stack is `empty'. Let M = (Q, Σ, Γ, δ, s 0,, ). Dene M = (Q {s f, s 0 }, Σ, Γ { }, δ, s 0,, {s f }), where 1 δ (s 0, ε, ) = {(s 0, )} 2 q Q, a Σ {ε}, Z Γ. δ (q, a, Z) = δ(q, a, Z) 3 q Q {s f, s 0 }. δ (q, ε, ) = {(s f, ε)}. Clearly L E (M) = L F (M ). Ryszard Janicki Computability and Complexity 21 / 45

Denition (Deterministic Pushdown Automata) A pushdown automaton M = (Q, Σ, Γ, δ, s 0,, F ) is deterministic if and only if 1 q Q. Z Γ. δ(q, ε) = ( a Σ. δ(q, a, Z) = ) 2 q Q. Z Γ. a Σ {ε}. δ(q, a, Z) 1. Theorem L = {ww R w {0, 1} }, (where (a 1 a 2... a n ) R = a n a n 1... a 1 ). There is no deterministic pushdown automaton M such that L F (M) = L or L E (M) = L. The language {ww R w {0, 1} } is given by the following nondeterministic pushdown automaton: Ryszard Janicki Computability and Complexity 22 / 45

From CF-Grammars To Pushdown Automata We will show how to convert a given CF-grammar to an equivalent nondeterministic pushdown automaton Suppose we are given a CF-grammar G = (V, T, P, σ). We wish to construct a nondeterministic pushdown automaton M such that L(M) = L(G) Without loss of generality we can that all productions of G are of the Greibach normal form where c Σ {ε} and k 0. A cb 1 B 2 B k Ryszard Janicki Computability and Complexity 23 / 45

From CF-Grammars To Pushdown Automata We construct from G = (V, T, P, σ) an equivalent nondeterministic pushdown automaton M with only one state that accepts by empty stack. Let M = ({q}, T, V, δ, q, σ, ), where q is the only state, T,the set of terminals of G, is the input alphabet of M, V, the set of variables of G, is the stack alphabet of M, δ is the transition function dened below, q is the start state, σ, the start symbol of G, is the initial stack symbol of M,, the null set, is the set of nal states. The transition function δ is dened as follows. For each production in P, we have A cb 1 B 2... B k (q, B 1 B 2... B k ) δ(q, c, A). Ryszard Janicki Computability and Complexity 24 / 45

Theorem Let G be a context-free grammar and M G be a nondeterministic pushdown automaton derived from G using the procedure from page 24. Then L(G) = L E (M G ). Example Consider the set of nonnull balanced strings of parentheses [ ] Transformation: S [B [SB [BS [SBS B ] (i) S [BS (q, BS) δ(q, [, S) (ii) S [B (q, B) δ(q, [, S) (iii) S [SB (q, SB) δ(q, [, S) (iv) S [SBS (q, SBS) δ(q, [, S) (v) B ] (q, ε) δ(q, ], B) In other words δ(q, [, S) = {(q, BS), (q, B), (q, SB), (q, SBS)}, δ(q, ], S) = {(q, ε)} and δ(,, ) = for all other cases. Ryszard Janicki Computability and Complexity 25 / 45

From CF-Grammars To Pushdown Automata Example (Continued) For the input x = [[[]][]], we have Ryszard Janicki Computability and Complexity 26 / 45

From Pushdown Automata To CF-Grammars Theorem Let L = L E (M) for some pushdown automaton M. Then L is a context-free language. Proof.(sketch). Let M = (Q, Σ, Γ, δ, s 0,, ). We construct G M = (V, T, P, σ), where V = (Q Γ Q) {σ}, (with elements of V \ {σ} written as [p, A, q]) T = Σ, P is the following set of productions: 1 σ [s 0,, q] for each q Q, 2 [s, A, q m+1 ] a[q 1, B 1, q 2 ][q 2, B 2, q 3 ]... [q m, B m, q m+1 ] for each s, q 1,..., q m+1 Q, each a Σ {ε} and A, B 1,..., B m Γ, such that (q 1, B 1 B 2... B m ) δ(s, a, A). If m = 0, the the production is [s, A, q 1 ] a. Ryszard Janicki Computability and Complexity 27 / 45

From Pushdown Automata To CF-Grammars Proof.(sketch) (continuation). Explanation: The intuition is that a variable [q, A, p] derive x if and only if x causes M to erase an A from its stack by some sequence of moves, beginning in q and ending in p. It may be proved that: [q, A, p] G x T (q, x, A) M (p, ε, ε). Ryszard Janicki Computability and Complexity 28 / 45

Example Let M = ({q 0, q 1 }, {a, b}, {X, }, δ, q 0,, ), where δ is given by: δ(q 0, a, ) = {(q 0, X )} δ(q 0, a, X ) = {(q 0, XX )} δ(q 0, b, X ) = {(q 1, ε)} δ(q 1, b, X ) = {(q 1, ε)} δ(q 1, ε, X ) = {(q 1, ε)} δ(q 1, ε, ) = {(q 1, ε)} Dene G M = (V, T, P, σ) where: V = {σ, [q 0, X, q 0 ], [q 0, X, q 1 ], [q 1, X, q 0 ], [q 1, X, q 1 ], [q 0,, q 0 ], [q 0,, q 1 ], [q 1,, q 0 ], [q 1,, q 1 ]}, T = {a, b}, and P is of the following } form: σ [q 0,, q 0 ] since σ [q σ [q 0,, q 1 ] 0,, q] for each q Q [q 0,, q 0 ] a[q 0, X, q 0 ][q 0,, q 0 ] [q 0,, q 0 ] a[q 0, X, q 1 ][q 1,, q 0 ] [q 0,, q 1 ] a[q 0, X, q 0 ][q 0,, q 1 ] [q 0,, q 1 ] a[q 0, X, q 1 ][q 1,, q 1 ] since δ(q 0, a, ) = {q 0, X )} Ryszard Janicki Computability and Complexity 29 / 45

Example (continued) [q 0, X, q 0 ] a[q 0, X, q 0 ][q 0, X, q 0 ] [q 0, X, q 0 ] a[q 0, X, q 1 ][q 1, X, q 0 ] since δ(q [q 0, X, q 1 ] a[q 0, X, q 0 ][q 0, X, q 1 ] 0, a, X ) = {q 0, XX )} [q 0, X, q 1 ] a[q 0, X, q 1 ][q 1, X, q 1 ] [q 0, X, q 1 ] b since δ(q 0, b, X ) = {q 1, ε)} [q 1,, q 1 ] ε since δ(q 1, ε, ) = {q 1, ε)} [q 1, X, q 1 ] ε since δ(q 1, ε, X ) = {q 1, ε)} [q 1, X, q 1 ] b since δ(q 1, b, X ) = {q 1, ε)} Some optimization: There are no productions for the variables [q 1, X, q 0 ] and [q 1,, q 0 ]. All productions from [q 0, X, q 0 ] and [q 0,, q 0 ] have [q 1, X, q 0 ] or [q 1,, q 0 ] on the right, i.e. no terminal string can be derived from [q 0, X, q 0 ] or [q 0,, q 0 ]. This means the variables [q 0, X, q 0 ], [q 0,, q 0 ], [q 1, X, q 0 ] and [q 1,, q 0 ] are useless, i.e. we can remove all productions involving these symbols. Ryszard Janicki Computability and Complexity 30 / 45

Example (continued) The result is: V = {σ, [q 0, X, q 1 ], [q 1, X, q 1 ], [q 0,, q 1 ], [q 1,, q 1 ]} T = {a, b} P = σ [q 0,, q 1 ] [q 0,, q 1 ] a[q 0, X, q 0 ][q 0,, q 1 ] [q 0, X, q 1 ] a[q 0, X, q 1 ][q 1, X, q 1 ] [q 0, X, q 1 ] b [q 1,, q 1 ] ε [q 1, X, q 1 ] ε [q 1, X, q 1 ] b Ryszard Janicki Computability and Complexity 31 / 45

Automata and CF-Lanuages Theorem The following statements are equivalent: 1 L L CF, 2 L = L E (M) for some pushdown automaton M, 3 L = L F (M) for some pushdown automaton M. Remark. We may prove the above theorem without using Greibach Normal Form. Ryszard Janicki Computability and Complexity 32 / 45

Theorem Let L L CF. There is a pushdown automaton M such that L = L E (M). Proof. (sketch). Let G = (V, T, P, σ) be a CG-grammar. Dene M = (Q, Σ, Γ, δ, s 0,, ) as follows: Q = {s 0, q}, Σ = T, Γ = V T, = σ, and δ is dened as follows: 1 A Γ. δ(q, ε, A) = {(q, x) A x P}, { 2 {(q, ε)} if a = X a σ. X Γ. δ(q, a, X ) = if a X { 3 {(q, σ), (q, ε)} if ε L δ(s 0, ε, σ) = {(q, σ)} if ε / L 4 δ(,, ) = for all remaining cases. One can prove that L E (M) = L(G). Ryszard Janicki Computability and Complexity 33 / 45

Deterministic Context-Free Languages Denition A context-free language L is called deterministic if there exists a deterministic pushdown automaton M such that L = L(M). Let L DCF denotes the class of all deterministic context-free languages. Theorem L R L DCF L CF. Proof. By the denition L DCF L CF and Theorem from page 22 states that L DCF L CF, so L DCF L CF. Every deterministic nite automaton can be transformed into deterministic pushdown automaton by adding one element stack and not using it at all. Hence L R L DCF. The language from page 13 is deterministic context free but not regular, so L R L DCF. Ryszard Janicki Computability and Complexity 34 / 45

Pumping Lemma for CF-Languages Lemma Let L Σ be a context-free language. Then there is a constant n, depending only on L, such that if z L and z n then there are u, v, w, x, y Σ such that z = uvwxy and 1 vx 1, 2 vwx n, 3 i 0. uv i wx i y L. Using the above Pumping Lemma, one may easily show that {a n b n c n n 0} / L CF. Ryszard Janicki Computability and Complexity 35 / 45

Closures Theorem Let L 1, L 2 L CF. Then 1 L 1 L 2 L CF. 2 L 1 L 2 L CF. 3 L 1 L CF. Proof. Let G i = (V i, T i, P i, σ i ) be CF-grammars such that L I = L(G i ) for i = 1, 2. We may assume that V 1 V 2 =. New grammars: 1 σ L1 L 2 σ 1 σ 2 plus P 1 P 2, 2 σ L1 L 2 σ 1 σ 2 plus P 1 P 2, 3 σ L 1 σ L 1 σ 1 ε plus P 1. Ryszard Janicki Computability and Complexity 36 / 45

Intersection and Subtraction Theorem If L 1, L 2 L CF, the language L 1 L 2 may not be context-free. Proof. Take L 1 = {a n b n c k n, k 0} and L 2 = {a k b n c n n, k 0}. The language L 1 L 2 = {a n b n c n n 0} / L CF. Corollary If L Σ and L L CF, the language Σ \ L may not be context-free. Ryszard Janicki Computability and Complexity 37 / 45

Intersection with a Regular Language Theorem Let L L CF and R L R, i.e. L is context-free and R is regular. Then L R L CF, i.e. L R is context-free. Proof. (sketch). Let M L be a pushdown automaton such that L = L(M L ), and let M R be a deterministic nite state automaton such that R = L(M R ). Let M be constructed from M)L and M R in such a way that the state part of M is a product of states of M L and M R, and the stack part is the same as for M L. Then we have L(M) = L R. Corollary L = {x # a (x) = # b (x) = # c (x)} / L CF. Proof. Suppose L is context-free. Clearly a b c is regular, so L a b c L CF. But L a b c = {a n b n c n n 0}, a contradiction. Ryszard Janicki Computability and Complexity 38 / 45

Decision Algorithms for CF-Languages Theorem There are algorithms to determine if a CF-language is 1 empty 2 nite 3 innite Proof. (1) The test of Lemma A for useless symbols (page 36 of Lecture Notes 4) determines if a variable generates any string of terminals. Hence: L(G) σ generates some string of terminals. (2) and (3) We may assume that a grammar is in Chomsky Normal Form (we have a algorithm for appropriate transformation). We dene a directed graph with a vertex for each variable and an edge from A to B if there is a production of the form A BC or A CB for any C. Then L(G) is nite if and only if this graph has no cycles. Otherwise it is innite. Ryszard Janicki Computability and Complexity 39 / 45

Membership Theorem There is an algorithm to decide if x L for each L L CF. Proof. Theorem about grammars without ε-productions gives an algorithm to decide if ε L. Let x ε. We may assume that we have a grammar G in Greibach Normal Form such that L = L(G). Let x = a 1...a n. If x L then σ G a 1 y 1 G a 1 a 2 y 2 G... G a 1...a n 1 y n 1 }{{ G a 1...a n = x. } n steps, n = x There is a nite number of derivations in G starting from σ and doing n steps. If no variable in G jas more than k productions, we have: {y σ n y y (V T ) } k n. More ecient G algorithms, as O(n 3 ) do exist. Ryszard Janicki Computability and Complexity 40 / 45

Undecidability of CF-Languages The concept of Undecidability will be discussed later, here we will just mention undecidable properties of CF-grammars and CF-Languages. Theorem The following properties are undecidable for CF-grammars over sets of terminals with more than one element. 1 L(G 1 ) L(G 2 ) L CF 2 Is G ambiguous? 3 L(G) L R 4 L(G 1 ) L(G 2 ) 5 L(G 1 ) = L(G 2 ) 6 For L L CF, is L inherently ambiguous? Ryszard Janicki Computability and Complexity 41 / 45

Context-Sensitive/Monotone Languages Lemma The language L = {a n b n c n n 1} is context-sensitive or monotone, i.e. L L CS = L M. Proof. The following two dierent monotone grammars generate L: σ aσx ay YX byc cx Xc Y bc σ aσbc abc CB BC ab ab bb bb bc bc cc cc Ryszard Janicki Computability and Complexity 42 / 45

Context-Sensitive/Monotone Languages - Properties Theorem L CF L CS = L M Proof. The theorem about removal of ε-productions implies L CF L CS and the lemma form page 42 gives L CF L CS. Theorem If L 1, L 2 L CS = L M then: 1 L 1 L 2 L CS 2 L 1 L 2 L CS 3 L 1 L 2 L CS 4 L 1 L CS 5 Σ \ L 1 L CS (if L 1 Σ ) Ryszard Janicki Computability and Complexity 43 / 45

Membership for Context-Sensitive Languages Theorem There is an algorithm to decide if x L for each L L CS. Proof. From the denition of context-sensitive grammar we immediately know if ε is in L or not. Let L = L(G) for some context-sensitive (or monotone) grammar. If x ε, then σ = x 0 x 1... x n = x, where x i (V T ) for i = 1,..., n and x i x i+1 for i = 0,..., n 1. We construct a graph whose vertices are the strings in (V T ) of length n or less. We put an arc from the vertex α to the vertex β if α β. Then paths in the graph correspond to derivations in the grammar G, and w L(G) if and only if there is a path from the vertex σ to the vertex x. Any path nding techniques can be used. Ryszard Janicki Computability and Complexity 44 / 45

This is the end of GRAMMARS THEORY Ryszard Janicki Computability and Complexity 45 / 45