Computational Models - Lecture 5 1

Computational Models - Lecture 5 1 Handout Mode Iftach Haitner and Yishay Mansour. Tel Aviv University. April 10/22, 2013 1 Based on frames by Benny Chor, Tel Aviv University, modifying frames by Maurice Herlihy, Brown University. Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 April 10/22, 2013 1 / 50

Outline Push Down Automata (PDA) Equivalence of CFGs and PDAs Closure properties for CFL Sipser s book, 2.1, 2.2 & 2.3 Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 April 10/22, 2013 2 / 50

Part I Push-Down Automata Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 April 10/22, 2013 3 / 50

String Generators and String Acceptors Regular expressions are string generators they tell us how to generate all strings in a language L Finite Automata (DFA, NFA) are string acceptors they tell us if a specific string w is in L CFGs are string generators Are there string acceptors for CFLs? YES! Push-down automata Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 April 10/22, 2013 4 / 50

A Finite Automaton Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 April 10/22, 2013 5 / 50

A PushDown Automaton (ignore the $ sign) Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 April 10/22, 2013 6 / 50

Example 1 PDA for L 1 = {0 n 1 n : n 0} Informally: 1 Read input symbols 1 Push each read 0 on the stack 2 Pop a 0 for each read 1 2 Accept if stack is empty after last symbol read, and no 0 appears after 1 Recall that L 1 is not regular Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 April 10/22, 2013 7 / 50

Example 2 PDA for L 2 = {a i b j c k : i = j i = k} Informally: Read and push a s Either pop and match with b s or pop and match with c s A non-deterministic choice Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 April 10/22, 2013 8 / 50

Pushdown Automaton (PDA) Formal Definition A PDA is a 6-tuple (Q,Σ,Γ,δ, q 0, F), where Q is a finite set called the states, Σ is a finite set called the input alphabet, Γ is a finite set called the stack alphabet, δ : Q Σ ε Γ ε P(Q Γ ε ) is the transition function, 2 q 0 Q is the starting state, and F Q is the set of accepting states. The symbol ε above does not stand for the empty string (rather for a character not in Σ Γ). In the following we interpret the symbol ε as a character or as the empty string according to the context; specifically, ε is taken as the empty string only if the relevant alphabet does not contain the symbol ε. 2 X ε := X {ε}. Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 April 10/22, 2013 9 / 50

The language accepted by a PDA A pushdown automaton (PDA) M accepts a string w, if there is a computation" of M on w (see next slide) that leads to an accepting state. The language accepted by M, denoted L(M), is the set of all strings w Σ accepted by M. A (non-deterministic) PDA may have many computations on a single string Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 April 10/22, 2013 10 / 50

Model of Computation The following is with respect to M = (Q,Σ,Γ,δ, q 0, F). Definition 1 (δ,ne ) For w = (w 1,..., w n ) Σ n ε let δ,ne (w) be all pairs (q, s) Q Γ for which exist states r 1,...,r n Q and strings s 0, s 1,... s n Γ s.t.: 1 r 0 = q 0, r n = q, s 0 = ε and s n = s (here ε stands for the empty string) 2 For every i {0,..., n 1} exist a, b Γ ε and t Γ s.t.: 1 (r i+1, b) δ(r i, w i+1, a) 2 s i = at and s i+1 = bt (here a = ε, or b = ε, is taken as the empty string). Namely, (q, s) δ,ne (w) if after reading w, M can find itself in state q and stack value s. δ (w) = w (ε w 1 ε...w nε ) δ (w ) M accepts w Σ if q F such that (q, t) δ (w) for some t. Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 April 10/22, 2013 11 / 50

Diagram Notation When drawing the automata diagram, we use the following notation Transition a, b c from state q to q means (q, c) δ(q, a, b), and informally means the automata read a from input pop b from stack push c onto stack Meaning of ε transitions ((informally): a = ε : don t read input b = ε: don t pop any symbol c = ε: don t push any symbol Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 April 10/22, 2013 12 / 50

A PDA for L 1 = {0 n 1 n : n 0} Claim 2 0011 L(P). Proof: take w 1 = ε w 2 = 0 w 3 = 0 w 4 = 1 w 5 = 1 w 6 = ε s 0 = ε s 1 = $ s 2 = 0$ s 3 = 00$ s 4 = 0$ s 5 = $ s 7 = ε r 0 = q 1 r 1 = q 2 r 3 = q 2 r 4 = q 2 r 5 = q 3 r 6 = q 3 r 7 = q 4 Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 April 10/22, 2013 13 / 50

Knowing when stack is empty It is convenient to be able to know when the stack is empty, but there is no built-in mechanism to do that. Solution 1 Start by pushing $ onto stack. 2 When you see it again, stack is empty. Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 April 10/22, 2013 14 / 50

A PDA for L 2 = {a i b j c k : i = j i = k} Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 April 10/22, 2013 15 / 50

A PDA for L 2 = {a i b j c k : i = j i = k}, cont. Non-determinism is essential here! Unlike finite automata, non-determinism does add power. But we saw deterministic algorithm to deicide any CFL (and as we see later, CFLs are exactly the languages decided by DFAs)! How to prove that non-determinism adds power?. Does not seem trivial or immediate. Another example: L = {x n y n : n 0} {x n y 2n : n 0} is accepted by a non-deterministic PDA, but not by a deterministic one. Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 April 10/22, 2013 16 / 50

PDA Languages The Push-Down Automata Languages,L PDA, is the set of all languages that can be described by some PDA: L PDA = {L(M): M is a PDA} It is immediate that L PDA L DFA : every DFA is just a PDA that ignores the stack. L CFG L PDA? L PDA L CFG? L PDA = L CFG!!! Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 April 10/22, 2013 17 / 50

Part II Equivalence Theorem Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 April 10/22, 2013 18 / 50

The CFG PDA Equivalence Theorem Theorem 3 L PDA = L CFG : A language is context free if and only if some pushdown automata accepts it. This time (unlike the regular expression vs. regular languages theorem), both the proof if part and of the only if part are non trivial. Proof sketch follows. Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 April 10/22, 2013 19 / 50

CFL = PDA Lemma 4 L CFG L PDA : If a language is context free, then some PDA accepts it. Let L be a context-free language, and let G = (V,Σ, R, S) be context-free grammar for L We build a PDA P = (Q,Σ,Γ,δ, q 0, F), such that on input w it figures out" if there is a derivation of w using G. Question 5 How does P figure out which substitution to make? Answer: It guesses. Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 April 10/22, 2013 20 / 50

Simplifying Assumptions 1 In a single move, a PDA can push a whole word (from some fixed set) into the stack (first letter at the top) Can we justify it? 2 When deriving a word form a CFL, we always substitute the left most variable Does it change the derived language? Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 April 10/22, 2013 21 / 50

Informal Description of P Algorithm 6 (P) 1 Push S$ on stack 2 While top of the stack t is not $: 3 If t is variable A, (non-deterministically) select rule A α and substitute. 4 If t is a terminal a, read next input and compare; Reject if different. 5 Accept if end of input and stack is empty Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 April 10/22, 2013 22 / 50

State Diagram for P Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 April 10/22, 2013 23 / 50

Claim: L(P) = L(G) Claim 7 S α iff α = α 1 α 2 such that (q loop,α 2 $) δ (α 1 ). Does the above yields that L(P) = L(G)? Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 April 10/22, 2013 24 / 50

S α = α = α 1 α 2 such that (q loop,α 2 $) δ (α 1 ) Proof by induction on the number of derivations steps used to yield α from S. 0 derivation steps: hence α = S. Since (q loop, S$) δ (ε), the proof follows for α 1 = ε and α 2 = S. Assume S α in k > 0 derivation steps, and let α be the string derived by the first (k 1) steps. By i.h α = α 1 α 2 such that (q loop,α 2 $) δ (α 1 ) Write α 2 = w 1Aw 2 where A is the left most variable in α 2. The k th derivation step replaces this occurrence of A with a string t (why?) It is easy to see that (q loop, tw 2 $) δ (α 1 w 1) To complete the proof take α 1 = α 1 w 1 and α 2 = tw 2. Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 April 10/22, 2013 25 / 50

α = α 1 α 2 such that (q loop,α 2 $) δ (α 1 ) = S α Proof by induction on the number of steps used by P to process α 1. A single step: α 1 = ε and α 2 = S$, and the proof follows since S S. Assume α 1 was processed in k > 1 steps, and let α 1 and α 2 be the input string read and the stack value before the last step Note that (q loop,α 2 $) δ (α 1 ). By i.h S α = α 1 α 2. In case the k th move of P is reading an input character, then α 1 α 2 = α 1 α 2, and therefore S α 1 α 2 Otherwise, α 1 = α 1, α 2 = Aw and α 2 = tw for some rule A w R Hence S α 1 α 2 Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 April 10/22, 2013 26 / 50

PDA = CFL Lemma 8 L PDA L CFG : If a PDA accepts a language then it is context free. We prove the lemma by constructing a CFG G for a language L accepted by a PDA P Let P = (Q,Σ,Γ,δ, q 0, F). We assume wlg. that: A single accepting state qa F. P empties the stack before accepting Each transition either pops or pushes Can we justify the above? Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 April 10/22, 2013 27 / 50

Defining G = (V,Σ, R, S) V = {A pq : p, q Q} Idea: A pq will generate all strings that take P from p with an empty stack, to q with an empty stack S = A q0,q a Initially R = and 1 Add {A pq A p,r A r,q : p, q, r Q} to R 2 Add {A qq ε: q Q} to R 3 For all p, r, s, q Q, a, b Σ ε and t Γ such that 1 (r, t) δ(p, a,ε) and 2 (q,ε) δ(s, b, t) add A pq aa r,s b to R Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 April 10/22, 2013 28 / 50

PDA Computation corresponding to A pq A p,r A r,q Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 April 10/22, 2013 29 / 50

PDA Computation corresponding to A pq aa r,s b Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 April 10/22, 2013 30 / 50

Claim: L(G) = L(P) For p Q, w Σ and t Γ ε, let δ (p, w, t) be the natural generalization of the single-input δ where the computation starts from state p and stack t (and not necessarily from q 0 and ε). Claim 9 A pq w Σ iff (q,ε) δ (p, w,ε) Proof by induction on the number of derivation rules/ transitions Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 April 10/22, 2013 31 / 50

Part III Closure Properties Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 April 10/22, 2013 32 / 50

Simple Closure Properties of Context-Free Languages CFL s are closed under Union:S S1 S 2 Concatenation:S S1 S 2 Star:Snew ε S old S new S new What about complement and intersection? Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 April 10/22, 2013 33 / 50

Intersection S 1 A 1 B 1 S 2 A 2 B 2 A 1 0A 1 1 ε A 2 0A 2 ε B 1 2B 1 ε B 2 1B 2 2 ε L 1 = 0 n 1 n 2 L 2 = 0 1 n 2 n L 1 L 2 = 0 n 1 n 2 n L 1 and L 2 are CFLs (why?), But L 1 L 2 is not a CFL. But can t we run two PDA s in parallel, and accept iff both accept?? What about intersection of a CFL with a regular language? Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 April 10/22, 2013 34 / 50

When CFL Intersects Regular Language Are the context free languages closed under intersection with a regular language? That is, if L 1 is context free languages, and L 2 is regular, must L 1 L 2 be context free languages? Run PDA L 1 and DFA L 2 in parallel (just like the intersection of two regular languages). Formal details omitted (but you should be able to figure them out). Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 April 10/22, 2013 35 / 50

Applications Is L = {w {0, 1, 2} : # 0 (w) = # 1 (w) = # 2 (w)} context free? L 0 1 2 = {0 n 1 n 2 n : n 0} is not context free. 0 1 2 is regular. Context free languages intersected with a regular languages are context free. So L is not a context free language This could also be established using pumping lemma, but proof above is more elegant. Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 April 10/22, 2013 36 / 50

Complementation The fact that CFLs are not closed under intersection, but are closed under union, implies they are not closed under complementation, as L 1 L 2 = L 1 L 2. We give a simple example where L is not CFL but L is. Take L = {ww : w {0, 1} }. L is not a CFL (why?) We prove that L is a CFL Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 April 10/22, 2013 37 / 50

Complementation cont. For any y L, either y s length is odd, or y s length is even, 2l, and i 1 such that yi y l+i. It suffice to construct a PDA/CFG for L even the even length members of L (why?) Let L σ even = {{0, 1}k σ{0, 1} j {0, 1} k σ{0, 1} j : k, j 0} Note that L even = L 0 even L1 even and that L σ even = {{0, 1} k σ{0, 1} k {0, 1} j σ{0, 1} j : k, j 0} CFG for L 0 even S AB A CAC 0 B CBC 1 C 0 1 Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 April 10/22, 2013 38 / 50

A PDA for L 0 even Idea: Guess k, j 0, and accept w if it is of the form: {0, 1} k 0{0, 1} k {0, 1} j 1{0, 1} j Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 April 10/22, 2013 39 / 50

Homomorphism and Inverse Homomorphism Homomorphism: replaces each letter with a word Example: h(1) = aba, h(0) = aa h(010) = aa aba aa L 1 = (01), h(l 1 ) = (aaaba). Claim: Assuming that L is a CFL, then so is h(l) Proof? use the grammar Inverse homomorphism: h 1 (w) = {x: h(x) = w}, h 1 (L) = {x: h(x) L} Example: L 2 = (ab ba) a, h 1 (L 2 ) = {1}. Claim: Assuming that L is a CFL, then so is h 1 (L) Proof? use the automata (see next) Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 April 10/22, 2013 40 / 50

CFL s are Closed Under Inverse Homomorphism Idea: let P be a PDA for L. On input word w, emulate P(h(w)) But we cannot afford to store h(w)! Solution, compute h(w) on demand": Algorithm 10 Input w: 1 Initialized a buffer" Buff to h(a), where a is the first letter of w 2 Emulate a running of P with Buff as its input string. Each time Buff is fully read by P, set Buff = h(a), where a is the next letter in w (if exists) 3 Accept iff P does How do we implement Buff? Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 April 10/22, 2013 41 / 50

Emptiness of CFGs Question 11 Given a CFG, G, is L(G) =? In other words, is there a string generated by G? Theorem 12 There is an algorithm that solves this problem (and always halts). Possible approaches for a proof: Bad Idea: We know how to test whether w L(G) for any string w, so just try it for each w... Better Idea: Can the start variable generate a string of terminals? A more holistic approach: Can a particular variable generate a string of terminals? Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 April 10/22, 2013 42 / 50

Removing redundant variables and terminals of a CFG 1 Mark all terminal symbols in G. 2 Repeat until no new variable become marked: Mark any A where A U 1 U 2...U k and all U i have already been marked. 3 Remove all unmarked variables, and any rule they appear in. 4 If S is removed, then L(G) =. 5 Remove any variable A not reachable from S. 6 Remove any terminal which does not appear in some rule. Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 April 10/22, 2013 43 / 50

Checking Emptiness Algorithm 13 Input a CFG G 1 Remove redundant variables and terminals to get G. 2 Return TRUE (i.e., L(G ) = ) iff S is removed Correctness? Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 April 10/22, 2013 44 / 50

CFGs Fullness Question 14 Given a CFG G, is L(G) = Σ? Fact 15 We just saw an algorithm to determine, given a CFG G, whether L(G) = L(G) = Σ iff L(G) =. Why not modify the algorithm so it determines emptiness of the complement? Unfortunately, CFGs are not closed under complement. There is no algorithm to solve CFG fullness. We are not prepared to prove this remarkable fact (yet). Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 April 10/22, 2013 45 / 50

Finiteness of CFGs Question 16 Given a CFG G, is L(G) finite? 1 Remove redundant variables and terminals. 2 Turn into a CNF form 3 Create a graph C whose nodes are variables and its directed edges are derivations. 4 Return TRUE iff C has a no cycles. Correctness? Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 April 10/22, 2013 46 / 50

CFGs Inherent Ambiguity Question 17 Given a CFG G, is L(G) inherently ambiguous? This means that for any CFG that generates L(G), there is a word in the language with two different parse trees. Fact 18 There is no algorithm to solve CFG inherent ambiguity. We will not prove this fact, yet you want to know it to put things in context. Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 April 10/22, 2013 47 / 50

When Are Two CFGs equivalent? Question 19 Is there an algorithm to solve this problem? Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 April 10/22, 2013 48 / 50

A Short Summary Regular Languages Finite Automata. Context Free Languages Push Down Automata. Closure properties of regular languages and of CFLs. Most algorithmic problems for finite automata are solvable. Some algorithmic problems for finite automata are not solvable. Pumping lemmata for both classes of languages. There are additional languages out there. Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 April 10/22, 2013 49 / 50

The View Over The Horizon enumerable decidable context free regular Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 5 April 10/22, 2013 50 / 50