Computational Models: Class 5

Computational Models: Class 5 Benny Chor School of Computer Science Tel Aviv University March 27, 2019 Based on slides by Maurice Herlihy, Brown University, and modifications by Iftach Haitner and Yishay Mansour. Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 1 / 80

Class 5: Outline Push Down Automata Closure Properties of CFLs Non determinism adds power to PDAs (not in book) Algorithmic Aspects of PDAs and CFLs Equivalence of PDAs and CFLs Chomsky normal form construction (high level only) Sipser s book, 2.2 & 2.3 Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 2 / 80

Part I Push Down Automata Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 3 / 80

A Finite Automaton (reminder) Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 4 / 80

A PushDown Automaton (PDA) (reminder) We add a memory device with restricted access: A stack (last in, first out; push/pop). (ignore the $ sign). Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 5 / 80

The language accepted by a PDA A pushdown automaton M accepts a string w, if there is a computation of M on w that leads to an accepting state when all of the input is read. The language accepted by M, denoted L(M), is the set of all strings w Σ accepted by M. A (non-deterministic) PDA may have many computations on a single string. To accept, it suffices that one of them leads to acceptance. To reject, none of them should lead to acceptance. Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 6 / 80

PDA: Variants Instead of pushing in one move a single letter (or nothing) to the stack, push a string from Γ. Instead of acceptance by being in an accept state after reading all the input, define acceptance as having an empty stack after reading all the input. Instead of acceptance by being in an accept state after reading all the input, define acceptance as having an empty stack and being in an accept state after reading all the input. These variants give rise to PDAs with the same computing power as our PDAs (not hard to show). For different languages, we may choose a variant that is easier to use. Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 7 / 80

The Transition Function Denote input alphabet by Σ and stack alphabet by Γ. the domain of the transition function δ is current state: Q next input, if any: Σ ε (=Σ {ε}) stack symbol popped, if any: Γ ε (=Γ {ε}) and its range is new state: Q stack symbol pushed, if any: Γ ε non-determinism: P(of two components above) δ : Q Σ ε Γ ε P(Q Γ ε ) Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 8 / 80

PDA: Formal Definitions (reminder) A pushdown automaton (PDA) is a 6-tuple (Q, Σ, Γ, δ, q 0, F), where Q is a finite set called the states, Σ is a finite set called the input alphabet, Γ is a finite set called the stack alphabet, δ : Q Σ ε,$ Γ ε P(Q Γ ε ) is the transition function, q 0 Q is the start state, and F Q is the set of accept states. Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 9 / 80

Model of Computation The following corresponds to M = (Q, Σ, Γ, δ, q 0, F). Definition 1 (δ ) For w Σ let δ(q, w, s) be all pairs (q, s ) Q Γ for which exist w 1,..., w m Σ ε, states r 1,..., r m Q and strings s 0, s 1,... s m Γ s.t.: 1 w = w 1,..., w m, r 0 = q, r m = q, s 0 = s and s m = s 2 For every i {0,..., m 1} exist a, b Γ ε and t Γ s.t.: 1 (r i+1, b) δ(r i, w i+1, a) 2 s i = at and s i+1 = bt Namely, (q, s ) δ(q 0, w, ε) if after reading w (possibly with in-between ε moves), M can find itself in state q and stack value s. M accepts w Σ if q F such that (q, t) δ(q 0, w, ε) for some t. Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 10 / 80

Diagram Notation When drawing the automata diagram, we use the following notation Transition a, b c from state q to q means (q, c) δ(q, a, b), and informally means the automata read a from input pop b from stack push c onto stack Meaning of ε transitions (informally): a = ε : don t read an input symbol b = ε: don t pop any symbol c = ε: don t push any symbol Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 11 / 80

Conventions It will be convenient to be able to know when the stack is empty, but there is no built-in mechanism to do that. Solution: Start by pushing $ onto stack. When you see it again, stack is empty. Question: When is input string exhausted? Answer: When $ is read. Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 12 / 80

A PDA for L 1 = {0 n 1 n : n 0} Example 2 0011 L(P). Formally, for 0011, the PDA goes through following sequence of symbols, w i, stack contents, s i, and states, q i. w 1 = ε w 2 = 0 w 3 = 0 w 4 = 1 w 5 = 1 w 6 = ε s 0 = ε s 1 = $ s 2 = 0$ s 3 = 00$ s 4 = 0$ s 5 = $ s 6 = ε r 0 = q 1 r 1 = q 2 r 2 = q 2 r 3 = q 2 r 4 = q 3 r 5 = q 3 r 6 = q 4 Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 13 / 80

A PDA for L 1 = {0 n 1 n : n 0} We want to show that L(P) = L 1 = {0 n 1 n : n 0} After understanding the intuition, we formally prove this in the following manner: Claim 3 δ(q 1, ε, ε) = {(q 1, ε), (q 2, $)}. δ(q 1, 0 k, ε) = {(q 2, 0 k $)}, for k 1. δ(q 1, 0 k 1 i, ε) = {(q 3, 0 k i $)}, for k > i 1. δ(q 1, 0 k 1 k, ε) = {(q 3, $), (q 4, ε)}, for k 1. δ(q 1, w, ε) =, for w {0 k 1 i k i 0}. Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 14 / 80

A Second Example A PDA that accepts {a i b j c k i, j, k > 0 and i = j or i = k} Informally: read and push a s either pop and match with b s or else ignore b s. Pop and match with c s a non-deterministic choice! Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 15 / 80

A PDA for L 2 = {a i b j c k : i = j i = k} Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 16 / 80

A PDA for L 2 = {a i b j c k : i = j i = k}, cont. Non-determinism is essential here! Unlike finite automata, non-determinism does add power. But how do we prove that non-determinism adds power?. Does not seem trivial or immediate. Another example: L = {x n y n : n 0} {x n y 2n : n 0} is accepted by a non-deterministic PDA, but not by a deterministic one. We will prove this claim In a few slides. Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 17 / 80

Example 3 Palindrome A palindrome is a string w satisfying w = w R. Madam I m Adam Dennis and Edna sinned Red rum, sir, is murder Able was I ere I saw Elba In girum imus nocte et consumimur igni (Latin: "we go into the circle by night, we are consumed by fire.) νιψoν ανoµηµατ α µη µoναν oψιν Palindromes also appear in nature. For example as DNA restriction sites short genomic strings over {A, C, T, G}, being cut by (naturally occurring) restriction enzymes. What is the difference from {ww R }? Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 18 / 80

A PDA for Palindromes Algorithm 4 Input: x Σ 1 Start pushing x into stack. 2 At some point, guess that the mid point of x was reached. 3 Pops and compares to input, letter by letter. 4 Reject if mismatch found. 5 Accept If stack is empty together with end of input. This PDA accepts palindromes of even length over the alphabet. Taking care of odd and even lengths is an easy modification. Again, non-determinism (at which point to make the switch) seems necessary. Q.: Which CFG generates palindromes? Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 19 / 80

PDA Languages The Push-Down Automata Languages, L PDA, is the set of all languages that can be described by some PDA: L PDA = {L : PDA M such that L[M] = L} We already know L DFA L PDA, since every DFA can be viewed as a PDA that ignores the stack. Likewise, we define L CFL = {L : CFG G such that L[G] = L} Questions: L CFG L PDA? L PDA L CFG? Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 20 / 80

Equivalence Theorem Theorem 5 A language is context free if and only if some pushdown automaton accepts it. This time, both the if part and the only if part are non-trivial. We will present a high level view of the proof (not all details) later today. Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 21 / 80

Part II (Non) Closure Properties of Context Free Languages Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 22 / 80

CFL Closure Properties We saw that Context-Free Languages are closed under union, concatenation, and star. It is time we resolve closure with respect to complementation and intersection. Let us start with closure under intersection. Possible approach: Can we intersect two context free languages to get the non CFL {0 n 1 n 2 n }? Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 23 / 80

CFL Closure Properties So, are the context free languages closed under intersection? S 1 A 1 B 1 S 2 A 2 B 2 A 1 0A 1 1 01 A 2 0A 2 ε B 1 2B 1 ε B 2 1B 2 2 12 L 1 = 0 n 1 n 2 L 2 = 0 1 n 2 n L 1 L 2 = 0 n 1 n 2 n L 1 is a context free language, L 2 is a context free language, but L 1 L 2 is not a context free language. Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 24 / 80

CFL Closure Properties The fact that CFLs are not closed under intersection but are closed under union implies they are not closed under complementation, as L 1 L 2 = L 1 L 2. Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 25 / 80

CFL Closure Properties Can we give a simple, specific example, where L is not CFL but L is? Take L = {ww w {0, 1} }. In homework assignment 3, you will be asked to describe a context free grammar for L. We sketch a PDA for L (and recall the CFG-PDA equivalence): For any y L, either y s length is odd. y s length is even, 2l, and there is an i 1 such that y i y l+i. PDA non-deterministically tries to verify one of the options. For the second, harder, option, PDA employs stack for matching locations. Accepts only on a successful branch (we recommend you fill in the PDA details!). Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 26 / 80

CFL Closure Properties Are the context free languages context free languages closed under intersection with a regular language? That is, if L 1 is context free languages, and L 2 is regular, must L 1 L 2 be context free languages? Yes! Run PDA L 1 and DFA L 2 in parallel (much like the intersection of two regular languages). Formal details omitted (but you should be able to figure them out). Why does intersection work here but does not work for two PDAs? Because the DFA does not try to access the stack, while the two PDAs may impose conflicting actions on the stack! Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 27 / 80

CFL Closure Properties: Another Example Let L = {w {0, 1, 2} : # 0 (w) = # 1 (w) = # 2 (w)}. Is this a context free language? L 0 1 2 = {0 n 1 n 2 n : n 0} is not context free. 0 1 2 is regular. Context free languages intersected with a regular languages are context free. So L is not a context free language This could also be established using the pumping lemma, but proof above is more elegant. Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 28 / 80

Part III Non Determinism Adds Power to PDAs Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 29 / 80

Non-Determinism Adds Power Theorem: There are context free languages that are accepted only by non-deterministic push down automata. Comment: Recall that all regular languages are accepted by deterministic finite automata. For finite automata, non-determinism does not add power, even though it may make life more convenient. Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 30 / 80

Non-Determinism Adds Power: Proof Theorem: Let M be a PDA that accepts Then M is non-deterministic. L = {x n y n n 0} {x n y 2n n 0}. Proof : Suppose, by way of contradiction, that M is deterministic. Create two copies of this PDA, denoted by M 1 and M 2, respectively. Two states in M 1 and M 2 are called cousins if they are copies of the same state in the original PDA. modified from www.cs.may.ie/ jpower/courses/parsing/node38.html, but this URL is now (Nov. 2014) broken. Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 31 / 80

Non-Determinism Adds Power (cont.) We now modify these PDA copies to make them into one PDA, M 0, over the alphabet {x, y, z}. The stack alphabet is not changed. States of the new M 0 are those of M 1 union M 2. Start state of the new M 0 is the start state of M 1. The accepting states of the new M 0 are the accepting states of M 2. Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 32 / 80

Non-Determinism Adds Power (cont.) Modifications: Erase all x transitions of M 2 (formally, on input letter x and any stack option, M 2 moves to a new dead end state). Replace every existing y transition of M 2 by a new z transition (how is this implemented formally?). At this point M 2 got only z transition (so while in M 2, the letters x and y lead immediately to rejection). Erase all x transitions out of accepting states of M 1. Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 33 / 80

Non-Determinism Adds Power (cont.) The surgery is almost done, but if we don t connect the two halves of its brain, the patient will not function coherently. Replace every existing y transition leading out of an accepting state of M 1 to some state q, by a new z transition, and redirect it to the cousin of q in M 2. Surgery over. Patient (a deterministic PDA) still alive. Let us now diagnose what, if anything, it can do. Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 34 / 80

Non-Determinism Adds Power (cont.) What language does M 0 recognizes? If M 0 accepts a string, it must be of the form (x y) z. Surely not all strings of that form are accepted by M 0. For example, the prefix in (x y) must be accepted by the original M. Otherwise there will be no switch to M 2, and no acceptance by M 0. So the prefix of an accepted string is either of the form x n y n or x n y 2n. And the whole string is of the form x n y n z i or x n y 2n z j. Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 35 / 80

Non-Determinism Adds Power (cont.) What can we say about the z i part? First, i must be greater than 0 for a transition to take place. By construction, M 2 on z i imitates the actions of M on y i from the same starting point. This means that if M 0 accepts x n y n z i then M accepts x n y n+i. Which is only possible if i = n, so M 0 accepts x n y n z n, n > 0. Notice that M 0 cannot accepts x n y 2n z j, because M does not accepts any string of the form x n y 2n+j, j > 0. Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 36 / 80

Conclusion of Proof We just showed that the PDA M 0 accepts the language {x n y n z n n 1}. By the equivalence theorem, the language that a PDA accepts is a context free language. By the so called uvxyz pumping lemma, {x n y n z n n 1} is not a CFL. Contradiction. This means that our initial supposition that the language {x n y n } {x n y 2n } is accepted by a deterministic PDA, does not hold. While thinking about the proof, where would it fail if the original M were non-deterministic? Specifically, why is it true that for deterministic M, L(M 0 ), but for non deterministic M, L(M 0 ) = is possible? Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 37 / 80

Part IV Algorithmic Questions for CFG/CFL/PDA Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 38 / 80

Algorithmic Questions Regarding CFGs Given a CFG, G, and a string w, does G generate w? Initial Idea: Design an algorithm that tries all derivations. Problem: If G does not generate w, we ll never stop. Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 39 / 80

Algorithmic Questions for CFGs (2) Lemma 6 If G is in Chomsky normal form, w = n, and w is generated by G, then w has a derivation of length 2n 1 or less. The proof of the lemma relies on the fact that a long derivation which produces a string of terminals must produce long string (try proving it at home!). Algorithm s idea for "does G generate w": First, convert G to Chomsky normal form (see last slides in presentation for details). Now need only consider a bounded number of derivations those of length 2n 1 or less. Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 40 / 80

Algorithmic Questions for CFGs (3) Theorem 7 There is an algorithm (that halts on every inputs) A, that on inputs G and w, decides if G generates w. On input G, w, where G is a grammar and w a string, 1. Convert G to Chomsky normal form. 2. List all derivations with 2n 1 steps, were n = w. 3. If any generates w, accept, otherwise reject. Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 41 / 80

Algorithmic Questions for CFGs (4) Theorem 8 There is an algorithm (that halts on every inputs) A, that on inputs G and w, decides if G generates w. Remarks: Related to problem of compiling prog. languages. Would you want to use this algorithm at work? There is a more efficient algorithm, which employs dynamic programming. You will see it in the recitation. Every theorem about CFLs is also about PDAs. Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 42 / 80

Emptiness of CFGs Question 9 Given a CFG, G, is L(G) =? In other words, is there a string generated by G? Theorem 10 There is an algorithm that solves this problem (and always halts). Possible approaches for a proof: Bad Idea: We know how to test whether w L(G) for any string w, so just try it for each w... Better Idea: Can the start variable generate a string of terminals? Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 43 / 80

Removing redundant variables and terminals of a CFG 1 Mark all terminal symbols in G. 2 Repeat until no new variable become marked: Mark any A where A U 1 U 2... U k and all U i have already been marked. 3 Remove all unmarked variables, and any rule they appear in. 4 If S is removed, then L(G) =. A simple example on board. Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 44 / 80

Checking Emptiness Algorithm 11 Input a CFG G 1 Remove redundant variables and terminals to get G. 2 Return TRUE (i.e., L(G ) = ) iff S is removed Correctness? Two directions: S not removed implies L(G ). S removed implies L(G ) =. Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 45 / 80

CFGs Fullness Question 12 Given a CFG G, is L(G) = Σ? Fact 13 We just saw an algorithm to determine, given a CFG G, whether L(G) = L(G) = Σ iff L(G) =. Why not modify the algorithm so it determines emptiness of the complement? Unfortunately, CFGs are not closed under complement. There is no algorithm to solve CFG fullness. We are not prepared to prove this remarkable fact (yet). Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 46 / 80

Finiteness of CFGs Question 14 Given a CFG G, is L(G) finite? 1 Remove redundant variables and terminals. 2 Turn into a CNF form 3 Create a graph C whose nodes are variables and its directed edges are derivations. 4 Return TRUE iff C has a no cycles. Correctness? Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 47 / 80

CFGs Inherent Ambiguity Question 15 Given a CFG G, is L(G) inherently ambiguous? This means that for any CFG that generates L(G), there is a word in the language with two different parse trees. Fact 16 There is no algorithm to solve CFG inherent ambiguity. We will not prove this fact, yet we want you to know it, to put things in context. Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 48 / 80

When Are Two CFGs equivalent? Question 17 Given two CFG G 1 and G 2, test if L(G 1 ) = L(G 2 ). Is there an algorithm to solve this problem? Fact 18 There is no algorithm to solve Two CFG equivalence. We will prove this result. But only in the computability part of the course (in approx. 5 weeks). Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 49 / 80

Part V CFG PDA Equivalence Theorem Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 50 / 80

The CFG PDA Equivalence Theorem Theorem 19 L PDA = L CFG : A language is context free if and only if some pushdown automata accepts it. This time (unlike the regular expression vs. regular languages theorem), both the proof if part and of the only if part are non trivial. Proof sketch follows. Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 51 / 80

PDA = CFL Lemma 20 L PDA L CFG : If a PDA accepts a language then it is context free. We prove the lemma by constructing a CFG G for a language L accepted by a PDA P Let P = (Q, Σ, Γ, δ, q 0, F). We assume wlg. that: A single accepting state qa F. P empties the stack before accepting Each transition either pops or pushes How can we justify the above? Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 52 / 80

Defining G = (V, Σ, R, S) V = {A pq : p, q Q} Idea: The variable A pq will generate all strings that take P from p with an empty stack, to q with an empty stack S = A q0,q a Initially the set of rules R is assigned. We then add rules, as following: 1 Add {A pq A p,r A r,q : p, q, r Q} to R 2 Add {A qq ε: q Q} to R 3 For all p, r, s, q Q, a, b Σ ε and t Γ such that 1 (r, t) δ(p, a, ε) and 2 (q, ε) δ(s, b, t) add A pq aa r,s b to R Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 53 / 80

Example PDA to CFG A q1,q 4 A q2,q 3 A q2,q 3 0A q2,q 3 1 A q2,q 3 0A q2,q 2 1. A q2,q 2 ε. Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 54 / 80

PDA Computation corresponding to A pq A p,r A r,q Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 55 / 80

PDA Computation corresponding to A pq aa r,s b Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 56 / 80

Claim: L(G) = L(P) Claim 21 A pq w Σ iff (q, ε) δ(p, w, ε) Proof by induction on the number of derivation rules/ transitions Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 57 / 80

CFL = PDA Lemma 22 L CFG L PDA : If a language is context free, then some PDA accepts it. Let L be a context-free language, and let G = (V, Σ, R, S) be context-free grammar for L We build a PDA P = (Q, Σ, Γ, δ, q 0, F), such that on input w it figures out" if there is a derivation of w using G. Question 23 How does P figure out which substitution to make? Answer: It guesses. Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 58 / 80

Simplifying Assumptions 1 In a single move, a PDA can push a whole word (from some fixed set) into the stack (first letter at the top) Can we justify it? 2 When deriving a word form a CFL, we always substitute the left most variable 3 Does this change the derived language? Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 59 / 80

Informal Description of P Algorithm 24 (P) 1 Push S$ on stack 2 While top of the stack t is not $: 3 If t is variable A, (non-deterministically) select rule A α and substitute. 4 If t is a terminal a, read next input and compare; Reject if different. 5 Accept if end of input and stack is empty Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 60 / 80

State Diagram for P Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 61 / 80

Example consider the CFG: S 0S1 ε. The related PDA: Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 62 / 80

Claim: L(P) = L(G) Claim 25 S α iff α = α 1 α 2 such that (q loop, α 2 $) δ(q loop, α 1, ε). Recall S α. Next, we will see a proof of this claim. The claim implies that L(P) = L(G) (why?) Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 63 / 80

S α = α = α 1 α 2 such that (q loop, α 2 $) δ(q loop, α 1, $) Proof by induction on the number of derivations steps used to yield α from S. 0 derivation steps: hence α = S. Since (q loop, S$) δ(q loop, ε, $), the proof follows for α 1 = ε and α 2 = S. Assume S α in k > 0 derivation steps, and let α be the string derived by the first (k 1) steps. By induction hypothesis, α = α 1 α 2 such that (q loop, α 2 $) δ(q loop, α 1, $) Write α 2 = w 1Aw 2 where A is the left most variable in α 2. The k th derivation step replaces this occurrence of A with a string t (why?) It is easy to see that (q loop, tw 2 $) δ(q loop, α 1 w 1, $). To complete the proof take α 1 = α 1 w 1 and α 2 = tw 2. Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 64 / 80

α = α 1 α 2 such that (q loop, α 2 $) δ(q loop, α 1, $) = S α Proof by induction on the number of steps used by P to process α 1. A single step: α 1 = ε and α 2 = S$, and the proof follows since S S. Assume α 1 was processed in k > 1 steps, and let α 1 and α 2 be the input string read and the stack value before the last step Note that (q loop, α 2 $) δ(q loop, α 1, $). By i.h S α = α 1 α 2. In case the k th move of P is reading an input character, then α 1 α 2 = α 1 α 2, and therefore S α 1 α 2 Otherwise, α 1 = α 1, α 2 = Aw and α 2 = tw for some rule A t R Hence S α 1 α 2 This completes the proof that CFL = PDA. Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 65 / 80

Part VI Chomsky Normal Form (CNF) Linguist and political commentator Noam Chomsky (born Dec. 1928). Photo taken on May 2017. Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 66 / 80

Chomsky Normal Form A simplified, canonical form of context free grammars. Elegant by itself, useful (but not crucial) in proving equivalence theorem. Can also be used to slightly simplify proof of pumping lemma. Every rule has the form A BC A a S ε where S is the start symbol, A, B and C are any variable, except B and C not the start symbol, and A can be the start symbol. Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 67 / 80

Theorem: Any context-free language is generated by a context-free grammar in Chomsky normal form. Proof (basic idea): Add new start symbol S 0. Eliminate all ε rules of the form A ε. Eliminate all unit rules of the form A B. At each step, patch up rules so that grammar generates the same language. Convert remaining long rules to proper form. Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 68 / 80

Proof Add new start symbol S 0 and rule S 0 S. Guarantees that new start symbol does not appear on right-hand-side of a rule. Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 69 / 80

Proof Eliminating ε rules. Repeat: remove some A ε. for each R uav, add rule R uv. and so on: for R uavaw add R uvaw, R uavw, and R uvw. for R A add R ε, except if R ε has already been removed. until all ε-rules not involving the original start variable have been removed. Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 70 / 80

Proof Eliminate unit rules. Repeat: remove some A B. for each B u, add rule A u, unless this is previously removed unit rule. (u is a string of variables and terminals.) until all unit rules have been removed. Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 71 / 80

Proof Finally, convert long rules. To replace each A u 1 u 2... u k (for k 3), introduce new non-terminals N 1, N 2,..., N k 1 and rules A u 1 N 1 N 1 u 2 N 2. N k 3 u k 2 N k 2 N k 2 u k 1 u k Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 72 / 80

Conversion Example Initial Grammar: S ASA ab A B S B b ε (1) Add new start state: S 0 S S ASA ab A B S B b ε Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 73 / 80

Conversion Example (2) S 0 S S ASA ab A B S B b ε (2) Remove ε-rule B ε: S 0 S S ASA ab a A B S ε B b ε Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 74 / 80

Conversion Example (3) S 0 S S ASA ab a A B S ε B b (3) Remove ε-rule A ε: S 0 S S ASA ab a AS SA S A B S ε B b Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 75 / 80

Conversion Example (4) S 0 S S ASA ab a AS SA S A B S B b (4) Remove unit rule S S S 0 S S ASA ab a AS SA S A B S B b Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 76 / 80

Conversion Example (5) S 0 S S ASA ab a AS SA A B S B b (5) Remove unit rule S 0 S: S 0 S ASA ab a AS SA S ASA ab a AS SA A B S B b Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 77 / 80

Conversion Example (6) S 0 ASA ab a AS SA S ASA ab a AS SA A B S B b (6) Remove unit rule A B: S 0 ASA ab a AS SA S ASA ab a AS SA A B S b B b Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 78 / 80

Conversion Example (7) S 0 ASA ab a AS SA S ASA ab a AS SA A S b B b Remove unit rule A S: S 0 ASA ab a AS SA S ASA ab a AS SA A S b ASA ab a AS SA B b Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 79 / 80

Conversion Example (8) S 0 ASA ab a AS SA S ASA ab a AS SA A b ASA ab a AS SA B b (8) Final simplification treat long rules: S 0 AA 1 UB a SA AS S AA 1 UB a SA AS A b AA 1 UB a SA AS A 1 SA U a B b Benny Chor (Tel Aviv University) Computational Models Class 5: Plan March 27, 2019 80 / 80