This lecture covers Chapter 7 of HMU: Properties of CFLs Chomsky Normal Form Pumping Lemma for CFs Closure Properties of CFLs Decision Properties of CFLs Additional Reading: Chapter 7 of HMU.
Chomsky Normal Form (CNF) for CF Chomsky Normal Forms A normal or canonical form (be it in algebra, matrices, or languages) is a standardized way of presenting the object (in this case, languages). A normal form for CFs provides a prescribed structure to the grammar without compromising on its power to define all context-free languages. Every non-empty language L with ɛ / L has Chomsky Normal Form grammar = (V, T, P, S) where every production rule is of the form: A BC for A, B, C V, or A a for A V and a T. CNF disallows: A ɛ [ɛ-productions]. A B for A, B V. [Unit productions]. A B 1 B k, A V, B i V T for k 2 [Complex productions]. 2 / 27
Chomsky Normal Form (CNF) for CF Towards CNF [Step 1: Remove ɛ-productions] ɛ-production: A ɛ for some A V. Let us call a variable A V as nullable if A ɛ. We can identify nullable variables as follows: Basis: A V is nullable if A ɛ is a production rule in P. Induction: B V is nullable if B A 1 A k is in P, and each A i is nullable. Procedure to Eliminate ɛ-productions iven = (V, T, P, S) define no-ɛ = (V, T, P no-ɛ, S) as follows: 1. Start with P no-ɛ = P. Find all nullable variables of. 3. For each production rule in P do the following: If the body contains k > 0 nullable variables, add 2 k productions to P no-ɛ obtained by choosing a subset of nullable variables and replacing each by ɛ 4. Delete any production in P no-ɛ of the form Y ɛ for any Y V. For example, suppose that in a given grammar, B, D are nullable and C is not. If A BCD is a rule in P, then A BCD CD BC C are rules in P no-ɛ. Similarly, if A BD is a rule in P, then A BD B D are rules in P no-ɛ. 3 / 27
Chomsky Normal Form (CNF) for CF Towards CNF [Step 1: Remove ɛ-productions] An Example Suppose = ({A, B, C}, {0, 1}, P, A) with P: A BC; B 0B ɛ; C C11 ɛ. B and C are nullable since B ɛ and C ɛ. Then, A is also nullable. Define no-ɛ = ({A, B, C}, {0, 1}, P no-ɛ, A) with P no-ɛ containing A BC B C ɛ B 0A ɛ C C11 ɛ Theorem 7.1.1 The above induction procedure described in Slide 1 identifies all nullable variables. Theorem 7.1.2 L( no-ɛ) = L() \ {ɛ}. a a Proof in the Additional Proofs Section at the end 4 / 27
Chomsky Normal Form (CNF) for CF Towards CNF [Step 2: Remove Unit Productions] iven a grammar and variables A, B V, we say (A, B) form a unit pair if A using unit productions alone. We can identify unit pairs as follows: Basis: For each A V, (A, A) is a unit pair (since A A). Induction: If (A, B) is a unit pair, and B C is a production in P, then (A, C) is a unit pair. Suppose A BC and C ɛ are productions then A unit pair. Procedure to Eliminate Unit Productions B, but (A, B) is not a iven = (V, T, P, S) define no-unit = (V, T, P no-unit, S) as follows: 1. Start with P no-unit = P. Find all unit pairs of. 2. For every unit pair (A, B) and non-unit production rule B α, add rule A α to P no-unit. 3. Delete all unit production rules in P no-unit. B 5 / 27
Chomsky Normal Form (CNF) for CF Towards CNF [Step 2: Remove Unit Productions] An Example Suppose = ({A, B, C, D}, {a, b}, P, A) with P: A B ac; B A bd; C ac ɛ; D bd ɛ. (A, B) and (B, A) are the only two pairs of unit variables. Define no-unit = ({A, B, C, D}, {a, b}, P no-unit, A) with P no-unit containing A ac bd B B bd ac A C ac ɛ D bd ɛ Note: Rules with B being the head can never be used. Theorem 7.1.3 The induction procedure on Slide 5 identifies all unit pairs. Theorem 7.1.4 L( no-unit ) = L(). b b Outline of the proof is given in the Additional Proofs Section at the end 6 / 27
Chomsky Normal Form (CNF) for CF Towards CNF [Step 3: Remove Useless Variables] A symbol X V T is said to be generating if X w for some w T ; reachable if S αx β for some α, β (V T ) ; and useful if S αx β w for some w T and α, β (V T ). (Useful Reachable + enerating) iven a grammar, we can identify generating variables as follows: Basis: For each s T, s s. So s is generating Induction: If A α, and every symbol of α is generating, so is A. iven a grammar, we can identify reachable variables as follows: Basis: S S so S is reachable. Induction: If A α, and A is reachable, so is every symbol of α. 7 / 27
Chomsky Normal Form (CNF) for CF Towards CNF [Step 3: Remove Useless Variables] Procedure to Eliminate Useless Variables iven = (V, T, P, S) define = (V, T, P, S) as follows: Find all generating symbols of V is the set of all generating variables. P is the set of production rules involving only generating symbols. Now, define R = (V R, T R, P R, S) as follows: Find all reachable symbols of V R is the set of all reachable variables. P R is the set of production rules involving only reachable symbols. The Order of Eliminating Variables is Important! Consider = ({A, B, S}, {0, 1}, P, S) with P : S AB 0; A 1A; B 1. A is not generating. Removing A and the rules S AB and A 1A results in B being unreachable. Removing B and B 1 yields R = ({S}, {0}, S 0, S). Reversing the order, we first see that all symbols are reachable; removing then the non-generating symbol A and production rules S AB and A 1A yields R = ({B, S}, {0}, S 0 and B 0, S). But B is unreachable now! 8 / 27
Chomsky Normal Form (CNF) for CF Towards CNF [Step 3: Remove Useless Variables] Theorem 7.1.5 The induction procedure on Slide7 identifies all generating variables. Theorem 7.1.6 The induction procedure on Slide7 identifies all reachable variables. Theorem 7.1.7 (1) L() = L( R ); and (2) Every symbol in R is useful. c c Proof in the Additional Proofs Section at the end 9 / 27
Chomsky Normal Form (CNF) for CF Towards CNF [Step 4: Remove Complex Productions] Procedure to Eliminate Complex Productions iven = (V, T, P, S), define Ĝ = ( ˆV, T, ˆP, S) as follows: Start with Ĝ = and do the following operations. For every terminal a T that appears in the body of length 2 or more, introduce a new variable A and a new production rule A a. Replace the occurrence all such terminals in the body of length 2 or more by the introduced variables. Replace every rule A B 1 B k for k > 2, by introducing k 2 variables D 1,..., k 2, and by replacing the rule by the following k 1 rules: A B 1D 1 D 2 B 3 D 3 D k 2 B k 1 B k D 1 B 2D 2 D k 3 B k 2 D k 2 Note: Each introduced variable appears in the head exactly once. Theorem 7.1.8 L() = L(Ĝ).d d Outline of the proof is given in the Additional Proofs Section at the end 10 / 27
Chomsky Normal Form (CNF) for CF The Chomsky Normal Form Theorem 7.1.9 For every context-free language L containing a non-empty string, there exists a grammar in Chomsky Normal Form such that L \ {ɛ} = L(). Proof Since L is a CFL, it must correspond to some CF. Eliminate ɛ productions (Step 1) to derive a grammar 1 from such that L( 1) = L() \ {ɛ}. Eliminate unit productions (Step 2) to derive a grammar 2 from 1 such that L( 2) = L( 1). Eliminate useless variables (Step 3) to derive a grammar 3 from 2 such that L( 3) = L( 2). Eliminate complex productions (Step 4) to derive a grammar 4 from 3 such that L( 4) = L( 3). 4 contains no ɛ-productions, no unit productions, no useless variables, and no productions with body consisting of 3 or more symbols; Hence 4 is in CNF. 11 / 27
Pumping Lemma for CFLs Pumping Lemma Theorem 7.2.1 Let L = be a CFL. Then there exists n > 0 such that for any string z L with z n, (1) z = uvwxy; (2) vx ɛ; (3) vwx n; uv i wx i y L for any i 0. Proof Since the claim only pertains to non-empty strings, we can show the claim for L \ {ɛ}. Let CNF grammar generate L \ {ɛ}. Choose n = 2 m where m = V in. Pick any z with z n. Depth d m + 1. {z } At most two children per node A! BC. apple 2. apple 2 2 one child A! s apple 2 3 apple 2 d-1 apple 2 d-1 yield = z and z =2 m 12 / 27
Pumping Lemma for CFLs Proof Since depth D = m + 1 or more, there must be a path with m + 1 edges in the tree. There must be two labels that match! The claim follows from the following pictorial argument. S X... S X... ) u w y S X X {z} {z } {z } {z } {z } u v w x y ). u v X X... x y v w x 13 / 27
Pumping Lemma for CFLs Uses of Pumping Lemma Pumping lemma can be used to argue that some langauges are not CFLs. Proof that L = {0 n 1 n 2 n : n 0} is Not Context-Free Suppose it were. There exists an n such that for strings z longer than n pumping lemma applies. Applying pumping lemma to z = 0 n 1 n 2 n, we see that z = uvwxy such that vwx n. n n n z } { z } { z } { 0 0 1 1 2 2 vwx vwx 2 possible options for vwx vwx cannot contain both zeros and twos. Two cases arise: Case (a): Suppose vwx contains no 2s. Then uwx contains fewer 1s or 0s than 2s. Such a string is not in L. Case (b): Suppose vwx contains no 1s. Then uwx contains fewer 1s or 2s than 1s. Such a string is not in L. 14 / 27
Closure Properties Substitution of Symbols with Languages Let L be a CFL on Σ 1, and let h be a substitution, i.e., for each a Σ 1, h(a) is a language over some alphabet Σ a. We can extend the substitution to words by concatenation, i.e., h(s 1 s k ) = h(s 1)h(s 2) h(s k ). One can then extend the substitution to languages by unioning, i.e., h(l) := h(s 1 s l ) = h(s 1) h(s l ) s 1 s l L s 1 s l L i.e., h(l) is the language formed by substituting each symbol in a string in the language L by a corresponding language. An Example Suppose L = {a n b n : n 0} and h(a) = {0} and h(b) = {1, 11}. Then, h(l) = {0 n 1 m : n m 2n} Theorem 7.3.1 If L is a CFL over Σ 1 and h(s) is a CFL for every s Σ 1, then h(l) is also a CFL. 15 / 27
Closure Properties Substitution of Symbols with Languages Proof of Theorem 7.3.1 Let = (V, Σ 1, P, S) be a grammar that generates L. Let for a Σ 1, let a = (V a, Σ a, P a, S a) be a grammar that generates h(a). WLO, assume that V V a = for each a Σ 1. Now define Ĝ = (V, {Sa : a Σ1}, ˆP, S) by Every rule of ˆP is a rule of P obtained by replacing each a Σ 1 by S a. For example, X axb in P will correspond to X S axs b in ˆP if a, b Σ 1. Let sub = (V ( a Σ1 V a), a Σ1 Σ a, ˆP ( a Σ1 P a), S) Claim: sub generates h(l). Note that w h(l) can be written as w a1 w al for w ai h(a i ) for each i, and for some a 1 a l L. S ) a 1 a (in ) For i =1;:::;, S ai ) wai (in ai ) + + S ) S a1 S a (in Ĝ as well as sub ) S ai ) wai (in sub ) + z } { z } { S ) S a1 S a ) wa1 S a2 S a ) wa1 w a2 S a3 S a ) ) wa1 w a + 16 / 27
Closure Properties Closure under substitution means... Closure under (Finite) Union: Let L = {1, 2,..., k} and h(i) = L i be a CFL for each i = 1,..., k. By Theorem 7.3.1, h(l) = L 1 L k is a CFL. (FInite) Concatenation: Let L = {a 1a 2 a k } and h(a i ) = L ai be a CFL for each i = 1,..., k. By Theorem 7.3.1, h(l) = L a1 L ak is a CFL. Kleene- closure: Let L = {a} and h(a) = L a be a CFL. By Theorem 7.3.1, h(l) = (L a) is a CFL. Positive closure: Let L = {a} + := {a n : n 1} and h(a) = L a be a CFL. By Theorem 7.3.1, h(l) = L a(l a) is a CFL. Homomorphism: Let L be a CFL and g be a homomorphism (i.e., h maps symbols to strings of symbols over some alphabet). Define h(a) = {s(a)}, which is a regular/cf language.then, h(l) = s(l) and by Theorem 7.3.1, it is a CFL. 17 / 27
Closure Properties Some More Closure Properties - 1 Theorem 7.3.2 If L is CFL, then so is L R. Proof of Theorem 7.3.2 If = (V, T, P, S) generates L, then R = (V, T, P R, S) generates L R where A X 1 X l in P A (X 1 X l ) R = X l X l 1 X 1 in P R (1) Theorem 7.3.3 If L is a CFL, R is a regular language, then L R is a CFL. Proof of Theorem 7.3.3 Product PDA Approach: Run the PDA accepting L 1 and DFA accpeting L 2 in parallel. Accept input string iff both machines are in one of their respective final states. Corollary If L is a CFL, and R is regular, then L \ R = L R c is regular 18 / 27
Closure Properties Some More Closure Properties - 2 Theorem 7.3.4 If L is a CFL, h 1 (L) = {w : h(w) L} is also a CFL. A Coarse Outline of Proof of Theorem 7.3.4 ::: h ::: PDA P accepting L Yes/No Note that editing the input tape is not a valid PDA operation. To fix that, we need to alter the state of the PDA P to store h(a) in the state itself. Let L be a language over {0, 1} and h(0) = aa and h(1) = bbb. Let the states of PDA P be q 0,..., q k. Then, the PDA that accepts h 1 (L) has 6k states, namely (q i, aa), (q i, a), (q i, ɛ), (q i, bbb), (q i, bb), and (q i, b). The transition between states of P is defined as if the second component is the input tape. When the second component is empty, the PDA has the choice to read an input symbol a and move into a state with h(a) as the second component. 19 / 27
Closure Properties Some Non-Closure Properties CFLs are not closed under intersection. Let L 1 = {0 n 1 n 2 m : n, m 0}, L 2 = {0 n 1 m 2 n : n, m 0}. Both are CFLs. However, L 1 L 2 = {0 n 1 n 2 n : n 0} is not a CFL. CFLs are not closed under complementation. Suppose CFLs are closed under complementation. Let L 1, L 2 be the aforementioned CFLs. Then L 1 L 2 = (L c 1 L c 2) c must be a CFL, but it is not. (See Slide 14). Hence, CFLs cannot be closed under complementation. Note: There exist CFLs L such that L c is a CFL as well. 20 / 27
Decision Properties Emptiness and Membership Conversion of a grammar to a corresponding PDA, PDA to a corresponding grammar, and a grammar to CNF can each be achieved in polynomial time. Emptiness of a CFL L Let a grammar = (V, T, P, S) generating the language L be given. (If PDA is given, convert it to a grammar ). is non-empty S is generating. 21 / 27
Decision Properties Emptiness and Membership Emptiness of a CFL L iven = (V, T, P, S) and w = a 1 a l we identify l(l + 1)/2 sets E i,j 1 i j n E i,j corresponds to all variables that can derive a i a j. E i,j s are identify from bottom to top, left to right by the following induction. Basis: For each i = 1,..., l, E i,i contains all variables X such that X a i. Induction: For each i = 1,..., l and j > i, E ij contains X if: (1) X YZ (2) Y E i,i and Z E i +1,j for some i i < j. S E 1,l w L(). E 1; E 1; 1 E 2;..... E 1;3 E 2;4 E 2; E 1;2 E 2;3 E 3;4 E 1; E 1;1 E 2;2 E 3;3 E ; a 1 a 2 a 3 a 22 / 27
Decision Properties Some Undecidable Questions about CFs and CFLs Is a given grammar unambiguous/ambiguous? Is a given CFL inherently ambiguous? Is the intersection of two CFLs empty? Are two CFLs identical? Is a given CFL equal to Σ? 23 / 27
Additional Proofs Additional Proofs Proof of Theorem 7.1.2 Construct a parse tree with yield w L() \ {ɛ}. Identify a maximal subtree, rooted at say X, whose yield is ɛ. Delete X and its subtree. Repeat until no such subtrees remain. In this illustrative example below, suppose that there is only one subtree with ɛ yield; let B be its label and let A BCD be the production that introduced B. Now, delete B and its subtree. This new subtree is a parse tree for no-ɛ with yield w since A CD is a valid production rule in P no-ɛ [Why? B is nullable]. Construct a parse tree with yield w L( no-ɛ). Identify production rules (used in the tree) that are not in P. For each such rule, find an appropriate rule by appending nullable variables. To the parse tree, add the corresponding nullable variable(s) and a zero-yield subtrees to transform it to a parse tree for. In the example, the portion of the parse tree in yellow corresponds to the rule A CD; then there must be some rule in P (namely A BCD) such that the added variable(s) (B in this case) is nullable. So we add a child node with label B to the node with label A and append a sub-tree of yield ɛ rooted at B. This is now a parse tree for with yield w. no- S S A! CD A A! BCD A C D () B C D yield: w.. yield: w. sub-yield = z } {. 24 / 27
Additional Proofs Additional Proofs Outline of Proof of Theorem 7.1.4 L( no-unit ) L(): By definition, A γ in P no-unit iff there exists a B V such that A B and B γ in P. Thus, every production rule A γ of P no-unit is effectively a derivation A Hence, every derivation of no-unit is a derivation of. L() L( no-unit ): Consider a derivation of w L() from S. α in. Argue that every run of unit productions in P that are used in this derivation must be followed by a non-unit production rule in P. Each such run of unit productions in P followed by a non-unit production can be condensed to a single production in P no-unit. [See definition of P no-unit ] The condensed derivation is then a derivation of w using rules in P no-unit. 25 / 27
Additional Proofs Additional Proofs Proof of Theorem 7.1.7 (1) L( R ) L()) since the alphabets and the rule of R are subsets of those of. Suppose w L(). Then, there must be such a derivation of w from S: S γ 1 γ 2 γ 3 γ k = w. Rule: R 1 R 2 R 3 R k Since every variable symbol that appears in this derivation is generating, they and the production rules R 1,..., R k used in this derivation will be present in. Every variable in this derivation is reachable; consequently, the variables that appear and the rules R 1,..., R k will be present in R. Then, w L( R ). (2) A straightforward exercise in verifying the definition on Slide 7. Note that the remaining symbols have to be shown to be useful in R, and not in! 26 / 27
Additional Proofs Additional Proofs Outline of Proof of Theorem 7.1.8 L() L(Ĝ) because every production rule of Ĝ has a corresponding equivalent derivation of α from A in Ĝ. Consider the parse tree of w L(Ĝ). If there are no introduced variables, then this is also the parse tree of w in and hence w L(). If there are introduced variables, replace them by the complex production in that introduced them in the first place (such replacements are always possible). The resultant tree is a parse tree for w in, and hence w. Ĝ A S. S. B 1 B 2 D 1 D2 B 3... D k 2 ) B 1 B 2 B 3 B k yield: w... B k 1 B k yield: w. 27 / 27