Recitation 4: Converting Grammars to Chomsky Normal Form, Simulation of Context Free Languages with Push-Down Automata, Semirings 11-711: Algorithms for NLP October 10, 2014
Conversion to CNF Example grammar G: S ba ab A baa as a B abb bs b Task: Convert G to Chomsky Normal Form (Perhaps we re using a CYK parser, more info in future lectures) CNF allows productions in the form: A BC A a
Conversion to CNF Requirements before converting: No unit productions: A B No epsilon productions: A ɛ Algorithms exist to factor out, usually straightforward process.
Conversion to CNF G S ba S ab A baa A as A a B abb B bs B b Leave unchanged: Productions: A a
Conversion to CNF G S ba S ab A baa A as A a B abb B bs B b Leave unchanged: Productions: A a
Conversion to CNF G S ba S ab A baa A as A a B abb B bs B b For productions: A X 1...X m, m 2, where X i is a terminal a Replace a with new variable C a and add production: C a a
Conversion to CNF G (modified) S C b A S C a B A C b AA A C a S A a B C a BB B C b S B b C a a C b b What modified rules now fit CNF?
G (modified) S C b A S C a B A C b AA A C a S A a B C a BB B C b S B b C a a C b b Conversion to CNF
G (modified) S C b A S C a B A C b AA A C a S A a B C a BB B C b S B b C a a C b b For productions: A B 1 B 2...B m, m 3 Replace with: Conversion to CNF A B 1 D 1 D 1 B 2 D 2... D m 2 B m 1 B m
G (modified) S C b A S C a B A C b D 1 D 1 AA A C a S A a B C a BB B C b S B b C a a C b b Conversion to CNF
G (modified) S C b A S C a B A C b D 1 D 1 AA A C a S A a B C a D 2 D 2 BB B C b S B b C a a C b b Conversion to CNF
G (CNF) S C b A S C a B A C b D 1 D 1 AA A C a S A a B C a D 2 D 2 BB B C b S B b C a a C b b Conversion to CNF
Equivalence of CFGs and PDAs Given a CFG G = (V, T, P, S), where the production rules in P are: S Ab ASb AbS ASbS A a 1. Construct a PDA M that accepts by empty stack such that L(M) = L(G). 2. Show the computation of M and the corresponding derivation in G for the string aabaabbb.
Constructing M
Constructing M M = { Q =
Constructing M M = { Q = {q} // States Σ =
Constructing M M = { Q = {q} // States Σ = T = {a, b} // Input symbols Γ =
Constructing M M = { Q = {q} // States Σ = T = {a, b} // Input symbols Γ = V T = {S, A, a, b} // Stack symbols q 0 =
Constructing M M = { Q = {q} // States Σ = T = {a, b} // Input symbols Γ = V T = {S, A, a, b} // Stack symbols q 0 = q // Start state z 0 =
Constructing M M = { Q = {q} // States Σ = T = {a, b} // Input symbols Γ = V T = {S, A, a, b} // Stack symbols q 0 = q // Start state z 0 = S // Start stack symbol δ(q, ɛ, S) =
Constructing M M = { Q = {q} // States Σ = T = {a, b} // Input symbols Γ = V T = {S, A, a, b} // Stack symbols q 0 = q // Start state z 0 = S // Start stack symbol δ(q, ɛ, S) = {(q, Ab), (q, ASb), (q, AbS), (q, ASbS)} δ(q, ɛ, A) =
Constructing M M = { Q = {q} // States Σ = T = {a, b} // Input symbols Γ = V T = {S, A, a, b} // Stack symbols q 0 = q // Start state z 0 = S // Start stack symbol δ(q, ɛ, S) = {(q, Ab), (q, ASb), (q, AbS), (q, ASbS)} δ(q, ɛ, A) = {(q, a)} // Transitions for nonterminals δ(q, a, a) =
Constructing M M = { Q = {q} // States Σ = T = {a, b} // Input symbols Γ = V T = {S, A, a, b} // Stack symbols q 0 = q // Start state z 0 = S // Start stack symbol δ(q, ɛ, S) = {(q, Ab), (q, ASb), (q, AbS), (q, ASbS)} δ(q, ɛ, A) = {(q, a)} // Transitions for nonterminals δ(q, a, a) = {(q, ɛ)} // Transitions for terminals δ(q, b, b) =
Constructing M M = { Q = {q} // States Σ = T = {a, b} // Input symbols Γ = V T = {S, A, a, b} // Stack symbols q 0 = q // Start state z 0 = S // Start stack symbol δ(q, ɛ, S) = {(q, Ab), (q, ASb), (q, AbS), (q, ASbS)} δ(q, ɛ, A) = {(q, a)} // Transitions for nonterminals δ(q, a, a) = {(q, ɛ)} // Transitions for terminals δ(q, b, b) = {(q, ɛ)} }
Computation / Derivation Grammar PDA δ Applied S (q, aabaabbb, S)
Computation / Derivation Grammar PDA δ Applied S (q, aabaabbb, S) ASb (q, aabaabbb, ASb) δ(q, ɛ, S) (q, ASb)
Computation / Derivation Grammar PDA δ Applied S (q, aabaabbb, S) ASb (q, aabaabbb, ASb) δ(q, ɛ, S) (q, ASb) asb (q, aabaabbb, asb) δ(q, ɛ, A) (q, a)
Computation / Derivation Grammar PDA δ Applied S (q, aabaabbb, S) ASb (q, aabaabbb, ASb) δ(q, ɛ, S) (q, ASb) asb (q, aabaabbb, asb) δ(q, ɛ, A) (q, a) (q, abaabbb, Sb) δ(q, a, a) (q, ɛ)
Computation / Derivation Grammar PDA δ Applied S (q, aabaabbb, S) ASb (q, aabaabbb, ASb) δ(q, ɛ, S) (q, ASb) asb (q, aabaabbb, asb) δ(q, ɛ, A) (q, a) (q, abaabbb, Sb) δ(q, a, a) (q, ɛ) aabsb (q, abaabbb, AbSb) δ(q, ɛ, S) (q, AbS)
Computation / Derivation Grammar PDA δ Applied S (q, aabaabbb, S) ASb (q, aabaabbb, ASb) δ(q, ɛ, S) (q, ASb) asb (q, aabaabbb, asb) δ(q, ɛ, A) (q, a) (q, abaabbb, Sb) δ(q, a, a) (q, ɛ) aabsb (q, abaabbb, AbSb) δ(q, ɛ, S) (q, AbS) aabsb (q, abaabbb, absb) δ(q, ɛ, A) (q, a) (q, baabbb, bsb) δ(q, a, a) (q, ɛ)
Computation / Derivation Grammar PDA δ Applied S (q, aabaabbb, S) ASb (q, aabaabbb, ASb) δ(q, ɛ, S) (q, ASb) asb (q, aabaabbb, asb) δ(q, ɛ, A) (q, a) (q, abaabbb, Sb) δ(q, a, a) (q, ɛ) aabsb (q, abaabbb, AbSb) δ(q, ɛ, S) (q, AbS) aabsb (q, abaabbb, absb) δ(q, ɛ, A) (q, a) (q, baabbb, bsb) δ(q, a, a) (q, ɛ) (q, aabbb, Sb) δ(q, b, b) (q, ɛ)
Computation / Derivation Grammar PDA δ Applied S (q, aabaabbb, S) ASb (q, aabaabbb, ASb) δ(q, ɛ, S) (q, ASb) asb (q, aabaabbb, asb) δ(q, ɛ, A) (q, a) (q, abaabbb, Sb) δ(q, a, a) (q, ɛ) aabsb (q, abaabbb, AbSb) δ(q, ɛ, S) (q, AbS) aabsb (q, abaabbb, absb) δ(q, ɛ, A) (q, a) (q, baabbb, bsb) δ(q, a, a) (q, ɛ) (q, aabbb, Sb) δ(q, b, b) (q, ɛ) aabasbb (q, aabbb, ASbb) δ(q, ɛ, S) (q, ASb)
Computation / Derivation Grammar PDA δ Applied S (q, aabaabbb, S) ASb (q, aabaabbb, ASb) δ(q, ɛ, S) (q, ASb) asb (q, aabaabbb, asb) δ(q, ɛ, A) (q, a) (q, abaabbb, Sb) δ(q, a, a) (q, ɛ) aabsb (q, abaabbb, AbSb) δ(q, ɛ, S) (q, AbS) aabsb (q, abaabbb, absb) δ(q, ɛ, A) (q, a) (q, baabbb, bsb) δ(q, a, a) (q, ɛ) (q, aabbb, Sb) δ(q, b, b) (q, ɛ) aabasbb (q, aabbb, ASbb) δ(q, ɛ, S) (q, ASb) aabasbb (q, aabbb, asbb) δ(q, ɛ, A) (q, a)
Computation / Derivation Grammar PDA δ Applied aabasbb (q, aabbb, asbb)
Computation / Derivation Grammar PDA δ Applied aabasbb (q, aabbb, asbb) (q, abbb, Sbb) δ(q, a, a) (q, ɛ)
Computation / Derivation Grammar PDA δ Applied aabasbb (q, aabbb, asbb) (q, abbb, Sbb) δ(q, a, a) (q, ɛ) aabaabbb (q, abbb, Abbb) δ(q, ɛ, S) (q, Ab)
Computation / Derivation Grammar PDA δ Applied aabasbb (q, aabbb, asbb) (q, abbb, Sbb) δ(q, a, a) (q, ɛ) aabaabbb (q, abbb, Abbb) δ(q, ɛ, S) (q, Ab) aabaabbb (q, abbb, abbb) δ(q, ɛ, A) (q, a) (q, bbb, bbb) δ(q, a, a) (q, ɛ)
Computation / Derivation Grammar PDA δ Applied aabasbb (q, aabbb, asbb) (q, abbb, Sbb) δ(q, a, a) (q, ɛ) aabaabbb (q, abbb, Abbb) δ(q, ɛ, S) (q, Ab) aabaabbb (q, abbb, abbb) δ(q, ɛ, A) (q, a) (q, bbb, bbb) δ(q, a, a) (q, ɛ) (q, bb, bb) δ(q, b, b) (q, ɛ)
Computation / Derivation Grammar PDA δ Applied aabasbb (q, aabbb, asbb) (q, abbb, Sbb) δ(q, a, a) (q, ɛ) aabaabbb (q, abbb, Abbb) δ(q, ɛ, S) (q, Ab) aabaabbb (q, abbb, abbb) δ(q, ɛ, A) (q, a) (q, bbb, bbb) δ(q, a, a) (q, ɛ) (q, bb, bb) δ(q, b, b) (q, ɛ) (q, b, b) δ(q, b, b) (q, ɛ)
Computation / Derivation Grammar PDA δ Applied aabasbb (q, aabbb, asbb) (q, abbb, Sbb) δ(q, a, a) (q, ɛ) aabaabbb (q, abbb, Abbb) δ(q, ɛ, S) (q, Ab) aabaabbb (q, abbb, abbb) δ(q, ɛ, A) (q, a) (q, bbb, bbb) δ(q, a, a) (q, ɛ) (q, bb, bb) δ(q, b, b) (q, ɛ) (q, b, b) δ(q, b, b) (q, ɛ) (q, ɛ, ɛ) δ(q, b, b) (q, ɛ)
Semirings S = (R,,, 0, 1) R is a set and are binary operations R R R, and 0, 1 R
Semiring Laws a, b, c R: (a b) c = a (b c) (associative ) (a b) c = a (b c) (associative ) a b = b a (commutative ) 0 a = a 0 = a ( 0 is id for ) 1 a = a 1 = a ( 1 is id for ) a (b c) = (a b) (a c) (left distributive) (a b) c = (a c) (b c) (right distributive) 0 a = a 0 = 0 ( 0 is annihilator for )
Semiring Examples (N, +,, 0, 1) (whole numbers)
Semiring Examples (N, +,, 0, 1) (whole numbers) ([0, 1], +,, 0, 1) (probabilities)
Semiring Examples (N, +,, 0, 1) (whole numbers) ([0, 1], +,, 0, 1) (probabilities) (Reg Σ, +,,, ɛ) (Regular expressions)
Semiring Examples (N, +,, 0, 1) (whole numbers) ([0, 1], +,, 0, 1) (probabilities) (Reg Σ, +,,, ɛ) (Regular expressions) (Z p, + (mod p), (mod p), 0, 1) (integers modulo some prime p)
Semirings Built from Other Semirings If S and T are semirings, then so are: (S n n, +,, 0, I ) (n by n matrices over S)
Semirings Built from Other Semirings If S and T are semirings, then so are: (S n n, +,, 0, I ) (n by n matrices over S) (S[X ], +,, 0, 1) (polynomials over S)
Semirings Built from Other Semirings If S and T are semirings, then so are: (S n n, +,, 0, I ) (n by n matrices over S) (S[X ], +,, 0, 1) (polynomials over S) (WFST Σ Σ (S),,,, ) (WFSTs with weights from S)
Semirings Built from Other Semirings If S and T are semirings, then so are: (S n n, +,, 0, I ) (n by n matrices over S) (S[X ], +,, 0, 1) (polynomials over S) (WFST Σ Σ (S),,,, ) (WFSTs with weights from S) (S T,?,?,?, 1 S 1 T ) (pairs of S and T )
Semirings Built from Other Semirings If S and T are semirings, then so are: (S n n, +,, 0, I ) (n by n matrices over S) (S[X ], +,, 0, 1) (polynomials over S) (WFST Σ Σ (S),,,, ) (WFSTs with weights from S) (S T,?,?,?, 1 S 1 T ) (pairs of S and T ) (Option[S],?,?, None, Some( 1)) (optional Ss)
Semirings Built from Other Semirings If S and T are semirings, then so are: (S n n, +,, 0, I ) (n by n matrices over S) (S[X ], +,, 0, 1) (polynomials over S) (WFST Σ Σ (S),,,, ) (WFSTs with weights from S) (S T,?,?,?, 1 S 1 T ) (pairs of S and T ) (Option[S],?,?, None, Some( 1)) (optional Ss) (GL(S), +,, 0, id) (linear functions S S)