Context-Free Parsing: CKY & Earley Algorithms and Probabilistic Parsing
|
|
|
- Adelia Barrett
- 5 years ago
- Views:
Transcription
1 Context-Free Parsing: CKY & Earley Algorithms and Probabilistic Parsing Natural Language Processing CS 4120/6120 Spring 2017 Northeastern University David Smith with some slides from Jason Eisner & Andrew McCallum
2 From Shift-Reduce to CKY Shift-reduce parsing can make wrong turns, needs backtracking Shift-reduce must pop the top of the stack, but how many items to pop? Time-space tradeoff Chomsky normal form
3 Chomsky Normal Form Any CFL can be generated by an equivalent grammar in CNF Rules of three types X Y Z X, Y, Z nonterminals X a X nonterminal, a terminal S ε S the start symbol NB: the derivation of a given string may change
4 CNF Conversion Create new start symbol Remove NTs that can generate epsilon Remove NTs that can generate each other, (unary rule cycles) Chain rules with RHS > 2 Related topic: rule Markovization (later)
5 CKY Algorithm Input: string of n words Output (of recognizer): grammatical or not Dynamic programming in a chart: rows labeled 0 to n-1 columns labeled 1 to n cell [i,j] lists possible constituents spanning words between i and j
6 CKY Algorithm! for i := 1 to n! Add to [i-1,i] all (part-of-speech) categories for the i th word! for width := 2 to n! for start := 0 to n-width! Define end := start + width! for mid := start+1 to end-1! for every constituent X in [start,mid]! for every constituent Y in [mid,end]! for all ways of combining X and Y (if any)! Add the resulting constituent to [start,end] if it s not already there.
7 CKY Algorithm! for i := 1 to n! Add to [i-1,i] all (part-of-speech) categories for the i th word! for width := 2 to n! for start := 0 to n-width! Define end := start + width! for mid := start+1 to end-1! for every constituent X in [start,mid]! for every constituent Y in [mid,end]! for all ways of combining X and Y (if any)! Add the resulting constituent to [start,end] if it s not already there. Time complexity?
8 CKY Algorithm! for i := 1 to n! Add to [i-1,i] all (part-of-speech) categories for the i th word! for width := 2 to n! for start := 0 to n-width! Define end := start + width! for mid := start+1 to end-1! for every constituent X in [start,mid]! for every constituent Y in [mid,end]! for all ways of combining X and Y (if any)! Add the resulting constituent to [start,end] if it s not already there. Time complexity? O(Gn 3 )
9 CKY Algorithm! for i := 1 to n! Add to [i-1,i] all (part-of-speech) categories for the i th word! for width := 2 to n! for start := 0 to n-width! Define end := start + width! for mid := start+1 to end-1! for every constituent X in [start,mid]! for every constituent Y in [mid,end]! for all ways of combining X and Y (if any)! Add the resulting constituent to [start,end] if it s not already there. Time complexity? O(Gn 3 ) Space complexity?
10 CKY Algorithm! for i := 1 to n! Add to [i-1,i] all (part-of-speech) categories for the i th word! for width := 2 to n! for start := 0 to n-width! Define end := start + width! for mid := start+1 to end-1! for every constituent X in [start,mid]! for every constituent Y in [mid,end]! for all ways of combining X and Y (if any)! Add the resulting constituent to [start,end] if it s not already there. Time complexity? O(Gn 3 ) Space complexity? O(Tn 2 )
11 time 1 flies 2 like 3 an 4 arrow 5 NP 3 Vst 3 0 NP! time Vst! time NP! flies VP! flies P! like V! like Det! an N! arrow NP 4 VP 4 P 2 V 5 Det 1 N 8 1 S! NP VP 6 S! Vst NP 2 S! S PP 1 VP! V NP 2 VP! VP PP 1 NP! Det N 2 NP! NP PP 3 NP! NP NP 0 PP! P NP
12 time 1 flies 2 like 3 an 4 arrow 5 NP 3 Vst NP 4 VP 4 1 S! NP VP 6 S! Vst NP 2 S! S PP 1 VP! V NP 2 VP! VP PP P 2 V 5 Det 1 N 8 1 NP! Det N 2 NP! NP PP 3 NP! NP NP 0 PP! P NP
13 time 1 flies 2 like 3 an 4 arrow 5 NP 3 Vst 3 NP NP 4 VP 4 1 S! NP VP 6 S! Vst NP 2 S! S PP 1 VP! V NP 2 VP! VP PP P 2 V 5 Det 1 N 8 1 NP! Det N 2 NP! NP PP 3 NP! NP NP 0 PP! P NP
14 time 1 flies 2 like 3 an 4 arrow 5 NP 3 Vst 3 NP 10 S NP 4 VP 4 1 S! NP VP 6 S! Vst NP 2 S! S PP 1 VP! V NP 2 VP! VP PP P 2 V 5 Det 1 N 8 1 NP! Det N 2 NP! NP PP 3 NP! NP NP 0 PP! P NP
15 time 1 flies 2 like 3 an 4 arrow 5 0 NP 3 Vst 3 NP 10 S 8 S 13 1 NP 4 VP 4 1 S! NP VP 6 S! Vst NP 2 S! S PP 1 VP! V NP 2 VP! VP PP P 2 V 5 Det 1 N 8 1 NP! Det N 2 NP! NP PP 3 NP! NP NP 0 PP! P NP
16 time 1 flies 2 like 3 an 4 arrow 5 0 NP 3 Vst 3 NP 10 S 8 S 13 1 NP 4 VP 4 _ 1 S! NP VP 6 S! Vst NP 2 S! S PP 1 VP! V NP 2 VP! VP PP P 2 V 5 _ Det 1 N 8 1 NP! Det N 2 NP! NP PP 3 NP! NP NP 0 PP! P NP
17 time 1 flies 2 like 3 an 4 arrow 5 0 NP 3 Vst 3 NP 10 S 8 S 13 1 NP 4 VP 4 _ 1 S! NP VP 6 S! Vst NP 2 S! S PP 1 VP! V NP 2 VP! VP PP P 2 V 5 _ Det 1 NP 10 N 8 1 NP! Det N 2 NP! NP PP 3 NP! NP NP 0 PP! P NP
18 time 1 flies 2 like 3 an 4 arrow 5 0 NP 3 Vst 3 NP 10 S 8 S 13 1 NP 4 VP 4 _ 1 S! NP VP 6 S! Vst NP 2 S! S PP 1 VP! V NP 2 VP! VP PP P 2 V 5 _ Det 1 NP 10 N 8 1 NP! Det N 2 NP! NP PP 3 NP! NP NP 0 PP! P NP
19 time 1 flies 2 like 3 an 4 arrow 5 NP 3 NP 10 _ Vst 3 S 8 S NP 4 VP 4 1 S! NP VP 6 S! Vst NP 2 S! S PP 1 VP! V NP 2 VP! VP PP P 2 V 5 _ Det 1 PP 12 NP 10 N 8 1 NP! Det N 2 NP! NP PP 3 NP! NP NP 0 PP! P NP
20 time 1 flies 2 like 3 an 4 arrow 5 NP 3 NP 10 _ Vst 3 S 8 S NP 4 VP 4 1 S! NP VP 6 S! Vst NP 2 S! S PP 1 VP! V NP 2 VP! VP PP P 2 V 5 _ Det 1 PP 12 VP 16 NP 10 N 8 1 NP! Det N 2 NP! NP PP 3 NP! NP NP 0 PP! P NP
21 time 1 flies 2 like 3 an 4 arrow 5 NP 3 NP 10 _ Vst 3 S 8 S NP 4 VP 4 1 S! NP VP 6 S! Vst NP 2 S! S PP 1 VP! V NP 2 VP! VP PP P 2 V 5 _ Det 1 PP 12 VP 16 NP 10 N 8 1 NP! Det N 2 NP! NP PP 3 NP! NP NP 0 PP! P NP
22 time 1 flies 2 like 3 an 4 arrow 5 NP 3 NP 10 Vst 3 S 8 S NP 4 VP 4 NP 18 1 S! NP VP 6 S! Vst NP 2 S! S PP 1 VP! V NP 2 VP! VP PP P 2 V 5 _ Det 1 PP 12 VP 16 NP 10 N 8 1 NP! Det N 2 NP! NP PP 3 NP! NP NP 0 PP! P NP
23 time 1 flies 2 like 3 an 4 arrow 5 NP 3 NP 10 Vst 3 S 8 S NP 4 VP 4 NP 18 S 21 1 S! NP VP 6 S! Vst NP 2 S! S PP 1 VP! V NP 2 VP! VP PP P 2 V 5 _ Det 1 PP 12 VP 16 NP 10 N 8 1 NP! Det N 2 NP! NP PP 3 NP! NP NP 0 PP! P NP
24 time 1 flies 2 like 3 an 4 arrow 5 NP 3 NP 10 Vst 3 S 8 S NP 4 VP 4 NP 18 S 21 VP 18 1 S! NP VP 6 S! Vst NP 2 S! S PP 1 VP! V NP 2 VP! VP PP P 2 V 5 _ Det 1 PP 12 VP 16 NP 10 N 8 1 NP! Det N 2 NP! NP PP 3 NP! NP NP 0 PP! P NP
25 time 1 flies 2 like 3 an 4 arrow 5 NP 3 NP 10 Vst 3 S 8 S NP 4 VP 4 NP 18 S 21 VP 18 1 S! NP VP 6 S! Vst NP 2 S! S PP 1 VP! V NP 2 VP! VP PP P 2 V 5 _ Det 1 PP 12 VP 16 NP 10 N 8 1 NP! Det N 2 NP! NP PP 3 NP! NP NP 0 PP! P NP
26 time 1 flies 2 like 3 an 4 arrow 5 NP 3 NP 10 NP 24 Vst 3 S 8 S NP 4 VP 4 NP 18 S 21 VP 18 1 S! NP VP 6 S! Vst NP 2 S! S PP 1 VP! V NP 2 VP! VP PP P 2 V 5 _ Det 1 PP 12 VP 16 NP 10 N 8 1 NP! Det N 2 NP! NP PP 3 NP! NP NP 0 PP! P NP
27 time 1 flies 2 like 3 an 4 arrow 5 NP 3 NP 10 NP 24 Vst 3 S 8 S 22 S NP 4 VP 4 NP 18 S 21 VP 18 1 S! NP VP 6 S! Vst NP 2 S! S PP 1 VP! V NP 2 VP! VP PP P 2 V 5 _ Det 1 PP 12 VP 16 NP 10 N 8 1 NP! Det N 2 NP! NP PP 3 NP! NP NP 0 PP! P NP
28 time 1 flies 2 like 3 an 4 arrow 5 NP 3 NP 10 NP 24 Vst 3 S 8 S 22 S 13 S NP 4 VP 4 NP 18 S 21 VP 18 1 S! NP VP 6 S! Vst NP 2 S! S PP 1 VP! V NP 2 VP! VP PP P 2 V 5 _ Det 1 PP 12 VP 16 NP 10 N 8 1 NP! Det N 2 NP! NP PP 3 NP! NP NP 0 PP! P NP
29 time 1 flies 2 like 3 an 4 arrow 5 NP 3 NP 10 NP 24 Vst 3 S 8 S 22 S 13 S NP 4 VP 4 NP 18 S 21 VP 18 1 S! NP VP 6 S! Vst NP 2 S! S PP 1 VP! V NP 2 VP! VP PP P 2 V 5 _ Det 1 PP 12 VP 16 NP 10 N 8 1 NP! Det N 2 NP! NP PP 3 NP! NP NP 0 PP! P NP
30 time 1 flies 2 like 3 an 4 arrow 5 NP 3 NP 10 NP 24 Vst 3 S 8 S 22 S 13 S 27 0 NP 24 1 NP 4 VP 4 NP 18 S 21 VP 18 1 S! NP VP 6 S! Vst NP 2 S! S PP 1 VP! V NP 2 VP! VP PP P 2 V 5 _ Det 1 PP 12 VP 16 NP 10 N 8 1 NP! Det N 2 NP! NP PP 3 NP! NP NP 0 PP! P NP
31 time 1 flies 2 like 3 an 4 arrow 5 NP 3 NP 10 NP 24 Vst 3 S 8 S 22 S 13 S 27 0 NP 24 1 NP 4 VP 4 S 27 NP 18 S 21 VP 18 1 S! NP VP 6 S! Vst NP 2 S! S PP 1 VP! V NP 2 VP! VP PP P 2 V 5 _ Det 1 PP 12 VP 16 NP 10 N 8 1 NP! Det N 2 NP! NP PP 3 NP! NP NP 0 PP! P NP
32 time 1 flies 2 like 3 an 4 arrow 5 NP 3 NP 10 NP 24 Vst 3 S 8 S 22 S 13 S 27 0 NP 24 1 NP 4 VP 4 S 27 S 22 NP 18 S 21 VP 18 1 S! NP VP 6 S! Vst NP 2 S! S PP 1 VP! V NP 2 VP! VP PP P 2 V 5 _ Det 1 PP 12 VP 16 NP 10 N 8 1 NP! Det N 2 NP! NP PP 3 NP! NP NP 0 PP! P NP
33 time 1 flies 2 like 3 an 4 arrow 5 NP 3 NP 10 NP 24 Vst 3 S 8 S 22 S 13 S 27 0 NP 24 1 NP 4 VP 4 S 27 S 22 S 27 NP 18 S 21 VP 18 1 S! NP VP 6 S! Vst NP 2 S! S PP 1 VP! V NP 2 VP! VP PP P 2 V 5 _ Det 1 PP 12 VP 16 NP 10 N 8 1 NP! Det N 2 NP! NP PP 3 NP! NP NP 0 PP! P NP
34 Follow backpointers S time 1 flies 2 like 3 an 4 arrow 5 NP 3 NP 10 NP 24 Vst 3 S 8 S 22 S 13 S 27 0 NP 24 S 27 S 22 1 NP 4 VP 4 S 27 NP 18 S 21 1 S! NP VP 6 S! Vst NP 2 S! S PP P 2 V 5 _ Det 1 VP 18 PP 12 VP 16 NP 10 N 8 1 VP! V NP 2 VP! VP PP 1 NP! Det N 2 NP! NP PP 3 NP! NP NP 0 PP! P NP
35 S time 1 flies 2 like 3 an 4 arrow 5 NP VP NP 3 NP 10 NP 24 Vst 3 S 8 S 22 S 13 S 27 0 NP 24 S 27 S 22 1 NP 4 VP 4 S 27 NP 18 S 21 1 S! NP VP 6 S! Vst NP 2 S! S PP P 2 V 5 _ Det 1 VP 18 PP 12 VP 16 NP 10 N 8 1 VP! V NP 2 VP! VP PP 1 NP! Det N 2 NP! NP PP 3 NP! NP NP 0 PP! P NP
36 S time 1 flies 2 like 3 an 4 arrow 5 NP VP NP 3 Vst 3 NP 10 S 8 NP 24 S 22 VP PP S 13 S 27 0 NP 24 S 27 S 22 1 NP 4 VP 4 S 27 NP 18 S 21 1 S! NP VP 6 S! Vst NP 2 S! S PP P 2 V 5 _ Det 1 VP 18 PP 12 VP 16 NP 10 N 8 1 VP! V NP 2 VP! VP PP 1 NP! Det N 2 NP! NP PP 3 NP! NP NP 0 PP! P NP
37 S time 1 flies 2 like 3 an 4 arrow 5 NP VP NP 3 Vst 3 NP 10 S 8 NP 24 S 22 VP PP 0 S 13 S 27 NP 24 P NP S 27 S 22 1 NP 4 VP 4 S 27 NP 18 S 21 1 S! NP VP 6 S! Vst NP 2 S! S PP P 2 V 5 _ Det 1 VP 18 PP 12 VP 16 NP 10 N 8 1 VP! V NP 2 VP! VP PP 1 NP! Det N 2 NP! NP PP 3 NP! NP NP 0 PP! P NP
38 S time 1 flies 2 like 3 an 4 arrow 5 NP VP NP 3 Vst 3 NP 10 S 8 NP 24 S 22 VP PP 0 S 13 S 27 NP 24 P NP S 27 S 22 Det N 1 NP 4 VP 4 S 27 NP 18 S 21 1 S! NP VP 6 S! Vst NP 2 S! S PP P 2 V 5 _ Det 1 VP 18 PP 12 VP 16 NP 10 N 8 1 VP! V NP 2 VP! VP PP 1 NP! Det N 2 NP! NP PP 3 NP! NP NP 0 PP! P NP
39 Treebank Grammars What rules would you extract from this tree? What probabilities would you assign them? Det The NP NOM Noun S Verb read VP Det NP NOM man this Noun book
40 Treebank Grammars Penn Treebank Lots of rules have high fanout (flat phrases) Lots of unary cycles How should we evaluate? What are the consequences of CNF conversion?
41 Parsing as Deduction CKY as inference rules CKY as Prolog program But Prolog is top-down with backtracking i.e., backward chaining, CKY is forward chaining Inference rules as Boolean semiring
42 Probabilistic CFGs Generative process (already familiar) It s context free: Rules are applied independently, therefore we multiply their probabilities How to estimate probabilities? Supervised and unsupervised
43 Questions for PCFGs What is the most likely parse for a sentence? (parsing) What is the probability of a sentence? (language modeling) What rule probabilities maximize the probability of a sentence? (parameter estimation)
44 Algorithms for PCFGs Exact analogues to HMM algorithms Parsing: Viterbi CKY Language modeling: inside probabilities Parameter estimation: inside-outside probabilities with EM
45 Parsing as Deduction A, B, C N,W V,0 i, j, k m constit(b, i, j) constit(c, j, k) A BC constit(a, i, k) word(w, i) A W constit(a, i, i + 1) In Prolog: constit(a, I1, I) :- word(i, W), (A ---> [W]), I1 is I - 1. constit(a, I, K) :- constit(b, I, J), constit(c, J, K), (A ---> [B, C]). But Prolog uses top-down search with backtracking...
46 Parsing as Deduction A, B, C N,W V,0 i, j, k m constit(b, i, j) constit(c, j, k) A BC constit(a, i, k) word(w, i) A W constit(a, i, i + 1) constit(a, i, k) = _ B,C,j constit(b,i,j) ^ constit(c, j, k) ^ A! BC constit(a, i, j) = _ W word(w, i, j) ^ A! W
47 Parsing as Deduction constit(a, i, k) = _ B,C,j constit(b,i,j) ^ constit(c, j, k) ^ A! BC constit(a, i, j) = _ W word(w, i, j) ^ A! W score(constit(a, i, k)) = max B,C,j score(constit(b,i,j)) score(constit(a, i, j)) = max W score(constit(c, j, k)) score(a! BC) score(word(w, i, j)) score(a! W ) And how about the inside algorithm?
48 Inside & Viterbi Algorithms Let A(i, j) = p(constit(a, i, j)) NB: index between words; M&S index words = p(w ij nonterminal A from i to j) A(i, k) = B,C,j B(i, j) C(j, k) p(a BC) Let A(i, j) = p best (constit(a, i, j)) A(i, k) = max B,C,j B (i, j)) C(j, k) p(a BC) S(0,n) =? S(0,n) =?
49 Forward-Backward Algorithm NN αnn(2) a[vb NN]b[interest VB] a[nn VB]b[rates NN] βnn(4) NNS αnns(2) a[vb NNS]b[interest VB] a[nns VB]b[rates NNS] βnns(4) NNP αnnp(2) a[vb NNP]b[interest VB] a[nnp VB]b[rates NNP] βnnp(4) VB αvb(2) a[vb VB]b[interest VB] a[vb VB]b[rates VB] βvb(4) VBZ αvbz(2) a[vb VBZ]b[interest VB] a[vbz VB]b[rates VBZ] βvbz(4) Fed raises interest rates
50 Inside & Outside constit(a, i, j) p(words 0-i, words j-n, constit) w(0, 1) w(i-1, i) w(j, j+1) w(n-1, n)
51 Inside & Outside constit(a, i, j) p(words 0-i, words j-n, constit) Inside: p(words i-j constit) w(0, 1) w(i-1, i) w(j, j+1) w(n-1, n)
52 Inside & Outside constit(a, i, j) p(words 0-i, words j-n, constit) Inside: p(words i-j constit) w(0, 1) w(i-1, i) w(j, j+1) w(n-1, n)
53 Inside & Outside constit(a, i, j) Inside x Outside = Inside probability of the whole tree p(words 0-i, words j-n, constit) Inside: p(words i-j constit) w(0, 1) w(i-1, i) w(j, j+1) w(n-1, n)
54 Outside Algorithm A(i, j) =p(w 0,i,A i,j,w j,n ) Uses inside probs. A(i, j) = S(0,n) =? PP(0,n) =? n B,C,k=j + i B,C,k=0 B(i, k) C (j, k) p(b AC) B(k, j) C (k, i) p(b CA) Some resemblance to derivative product rule
55 Problems with Inside-Outside EM Each sentence at each iteration takes O(m 3 n 3 ) Local maxima even more problematic than for HMMs: Charniak (1993) found a different maximum for each of 300 trials More NTs needed to learn a good model NTs don t correspond to intuitions: HMMs are easier to constrain with tag dictionaries
56 Top-Down/Bottom-Up Top-down parsers Can get caught in infinite loops Take exponential time backtracking CKY Needs Chomsky normal form Builds all possible constituents
57 Earley Parser (1970) Nice combination of dynamic programming incremental interpretation avoids infinite loops no restrictions on the form of the context-free grammar. A! B C the D of causes no problems O(n 3 ) worst case, but faster for many grammars Uses left context and optionally right context to constrain search.
58 Earley s Overview Finds constituents and partial constituents in input A! B C. D E is partial: only the first half of the A A B C D E D + = A B C D E A! B C. D E A! B C D. E
59 Earley s Overview Proceeds incrementally left-to-right Before it reads word 5, it has already built all hypotheses that are consistent with first 4 words Reads word 5 & attaches it to immediately preceding hypotheses. Might yield new constituents that are then attached to hypotheses immediately preceding them E.g., attaching D to A! B C. D E gives A! B C D. E Attaching E to that gives A! B C D E. Now we have a complete A that we can attach to hypotheses immediately preceding the A, etc.
60 The Parse Table Columns 0 through n corresponding to the gaps between words Entries in column 5 look like (3, NP! NP. PP) (but we ll omit the! etc. to save space) Built while processing word 5 Means that the input substring from 3 to 5 matches the initial NP portion of a NP! NP PP rule Dot shows how much we ve matched as of column 5 Perfectly fine to have entries like (3, VP! is it. true that S)
61 The Parse Table Entries in column 5 look like (3, NP! NP. PP) What will it mean that we have this entry? Unknown right context: Doesn t mean we ll necessarily be able to find a VP starting at column 5 to complete the S. Known left context: Does mean that some dotted rule back in column 3 is looking for an S that starts at 3. So if we actually do find a VP starting at column 5, allowing us to complete the S, then we ll be able to attach the S to something. And when that something is complete, it too will have a customer to its left In short, a top-down (i.e., goal-directed) parser: it chooses to start building a constituent not because of the input but because that s what the left context needs. In the spoon, won t build spoon as a verb because there s no way to use a verb there. So any hypothesis in column 5 could get used in the correct parse, if words 1-5 are continued in just the right way by words 6-n.
62 Earley s as a Recognizer Add ROOT!. S to column 0. For each j from 0 to n: For each dotted rule in column j, (including those we add as we go!) look at what s after the dot: If it s a word w, SCAN: If w matches the input word between j and j+1, advance the dot and add the resulting rule to column j+1 If it s a non-terminal X, PREDICT: Add all rules for X to the bottom of column j, wth the dot at the start: e.g. X!. Y Z If there s nothing after the dot, ATTACH: We ve finished some constituent, A, that started in column I<j. So for each rule in column j that has A after the dot: Advance the dot and add the result to the bottom of column j. Output yes just if last column has ROOT! S. NOTE: Don t add an entry to a column if it s already there!
63 Earley s Summary Process all hypotheses one at a time in order. (Current hypothesis is shown in blue.) This may add new hypotheses to the end of the to-do list, or try to add old hypotheses again. Process a hypothesis according to what follows the dot: If a word, scan input and see if it matches If a nonterminal, predict ways to match it (we ll predict blindly, but could reduce # of predictions by looking ahead k symbols in the input and only making predictions that are compatible with this limited right context) If nothing, then we have a complete constituent, so attach it to all its customers
64 A (Whimsical) Grammar S! NP VP NP! Papa NP! Det N N! caviar NP! NP PP N! spoon VP! V NP V! ate VP! VP PP P! with PP! P NP Det! the Det! a An Input Sentence Papa ate the caviar with a spoon.
65 0 initialize 0 ROOT. S Remember this stands for (0, ROOT!. S)
66 0 0 ROOT. S 0 S. NP VP predict the kind of S we are looking for Remember this stands for (0, S!. NP VP)
67 0 0 ROOT. S 0 S. NP VP 0 NP. Det N 0 NP. NP PP 0 NP. Papa predict the kind of NP we are looking for (actually we ll look for 3 kinds: any of the 3 will do)
68 0 0 ROOT. S 0 S. NP VP 0 NP. Det N 0 NP. NP PP 0 NP. Papa 0 Det. the 0 Det. a predict the kind of Det we are looking for (2 kinds)
69 0 0 ROOT. S 0 S. NP VP 0 NP. Det N 0 NP. NP PP 0 NP. Papa 0 Det. the 0 Det. a predict the kind of NP we re looking for but we were already looking for these so don t add duplicate goals! Note that this happened when we were processing a left-recursive rule.
70 0 Papa 1 0 ROOT. S 0 S. NP VP 0 NP. Det N 0 NP. NP PP 0 NP. Papa 0 Det. the 0 Det. a 0 NP Papa. scan: the desired word is in the input!
71 0 Papa 1 0 ROOT. S 0 S. NP VP 0 NP. Det N 0 NP. NP PP 0 NP. Papa 0 Det. the 0 Det. a 0 NP Papa. scan: failure
72 0 Papa 1 0 ROOT. S 0 S. NP VP 0 NP. Det N 0 NP. NP PP 0 NP. Papa 0 Det. the 0 Det. a 0 NP Papa. scan: failure
73 0 Papa 1 0 ROOT. S 0 NP Papa. 0 S. NP VP 0 S NP. VP 0 NP. Det N 0 NP NP. PP 0 NP. NP PP 0 NP. Papa 0 Det. the 0 Det. a attach the newly created NP (which starts at 0) to its customers (incomplete constituents that end at 0 and have NP after the dot)
74 0 Papa 1 0 ROOT. S 0 NP Papa. 0 S. NP VP 0 S NP. VP 0 NP. Det N 0 NP NP. PP 0 NP. NP PP 1 VP. V NP 0 NP. Papa 1 VP. VP PP 0 Det. the 0 Det. a predict
75 0 Papa 1 0 ROOT. S 0 NP Papa. 0 S. NP VP 0 S NP. VP 0 NP. Det N 0 NP NP. PP 0 NP. NP PP 1 VP. V NP 0 NP. Papa 1 VP. VP PP 0 Det. the 1 PP. P NP 0 Det. a predict
76 0 Papa 1 0 ROOT. S 0 NP Papa. 0 S. NP VP 0 S NP. VP 0 NP. Det N 0 NP NP. PP 0 NP. NP PP 1 VP. V NP 0 NP. Papa 1 VP. VP PP 0 Det. the 1 PP. P NP 0 Det. a 1 V. ate predict
77 0 Papa 1 0 ROOT. S 0 NP Papa. 0 S. NP VP 0 S NP. VP 0 NP. Det N 0 NP NP. PP 0 NP. NP PP 1 VP. V NP 0 NP. Papa 1 VP. VP PP 0 Det. the 1 PP. P NP 0 Det. a 1 V. ate predict
78 0 Papa 1 0 ROOT. S 0 NP Papa. 0 S. NP VP 0 S NP. VP 0 NP. Det N 0 NP NP. PP 0 NP. NP PP 1 VP. V NP 0 NP. Papa 1 VP. VP PP 0 Det. the 1 PP. P NP 0 Det. a 1 V. ate 1 P. with predict
79 0 Papa 1 ate 2 0 ROOT. S 0 NP Papa. 1 V ate. 0 S. NP VP 0 S NP. VP 0 NP. Det N 0 NP NP. PP 0 NP. NP PP 1 VP. V NP 0 NP. Papa 1 VP. VP PP 0 Det. the 1 PP. P NP 0 Det. a 1 V. ate 1 P. with scan: success!
80 0 Papa 1 ate 2 0 ROOT. S 0 NP Papa. 1 V ate. 0 S. NP VP 0 S NP. VP 0 NP. Det N 0 NP NP. PP 0 NP. NP PP 1 VP. V NP 0 NP. Papa 1 VP. VP PP 0 Det. the 1 PP. P NP 0 Det. a 1 V. ate 1 P. with scan: failure
81 0 Papa 1 ate 2 0 ROOT. S 0 NP Papa. 1 V ate. 0 S. NP VP 0 S NP. VP 1 VP V. NP 0 NP. Det N 0 NP NP. PP 0 NP. NP PP 1 VP. V NP 0 NP. Papa 1 VP. VP PP 0 Det. the 1 PP. P NP 0 Det. a 1 V. ate 1 P. with attach
82 0 Papa 1 ate 2 0 ROOT. S 0 NP Papa. 1 V ate. 0 S. NP VP 0 S NP. VP 1 VP V. NP 0 NP. Det N 0 NP NP. PP 2 NP. Det N 0 NP. NP PP 1 VP. V NP 2 NP. NP PP 0 NP. Papa 1 VP. VP PP 2 NP. Papa 0 Det. the 1 PP. P NP 0 Det. a 1 V. ate 1 P. with predict
83 0 Papa 1 ate 2 0 ROOT. S 0 NP Papa. 1 V ate. 0 S. NP VP 0 S NP. VP 1 VP V. NP 0 NP. Det N 0 NP NP. PP 2 NP. Det N 0 NP. NP PP 1 VP. V NP 2 NP. NP PP 0 NP. Papa 1 VP. VP PP 2 NP. Papa 0 Det. the 1 PP. P NP 2 Det. the 0 Det. a 1 V. ate 2 Det. a 1 P. with predict (these next few steps should look familiar)
84 0 Papa 1 ate 2 0 ROOT. S 0 NP Papa. 1 V ate. 0 S. NP VP 0 S NP. VP 1 VP V. NP 0 NP. Det N 0 NP NP. PP 2 NP. Det N 0 NP. NP PP 1 VP. V NP 2 NP. NP PP 0 NP. Papa 1 VP. VP PP 2 NP. Papa 0 Det. the 1 PP. P NP 2 Det. the 0 Det. a 1 V. ate 2 Det. a 1 P. with predict
85 0 Papa 1 ate 2 0 ROOT. S 0 NP Papa. 1 V ate. 0 S. NP VP 0 S NP. VP 1 VP V. NP 0 NP. Det N 0 NP NP. PP 2 NP. Det N 0 NP. NP PP 1 VP. V NP 2 NP. NP PP 0 NP. Papa 1 VP. VP PP 2 NP. Papa 0 Det. the 1 PP. P NP 2 Det. the 0 Det. a 1 V. ate 2 Det. a 1 P. with scan (this time we fail since Papa is not the next word)
86 0 Papa 1 ate 2 the 3 0 ROOT. S 0 NP Papa. 1 V ate. 2 Det the. 0 S. NP VP 0 S NP. VP 1 VP V. NP 0 NP. Det N 0 NP NP. PP 2 NP. Det N 0 NP. NP PP 1 VP. V NP 2 NP. NP PP 0 NP. Papa 1 VP. VP PP 2 NP. Papa 0 Det. the 1 PP. P NP 2 Det. the 0 Det. a 1 V. ate 2 Det. a 1 P. with scan: success!
87 0 Papa 1 ate 2 the 3 0 ROOT. S 0 NP Papa. 1 V ate. 2 Det the. 0 S. NP VP 0 S NP. VP 1 VP V. NP 0 NP. Det N 0 NP NP. PP 2 NP. Det N 0 NP. NP PP 1 VP. V NP 2 NP. NP PP 0 NP. Papa 1 VP. VP PP 2 NP. Papa 0 Det. the 1 PP. P NP 2 Det. the 0 Det. a 1 V. ate 2 Det. a 1 P. with
88 0 Papa 1 ate 2 the 3 0 ROOT. S 0 NP Papa. 1 V ate. 2 Det the. 0 S. NP VP 0 S NP. VP 1 VP V. NP 2 NP Det. N 0 NP. Det N 0 NP NP. PP 2 NP. Det N 0 NP. NP PP 1 VP. V NP 2 NP. NP PP 0 NP. Papa 1 VP. VP PP 2 NP. Papa 0 Det. the 1 PP. P NP 2 Det. the 0 Det. a 1 V. ate 2 Det. a 1 P. with
89 0 Papa 1 ate 2 the 3 0 ROOT. S 0 NP Papa. 1 V ate. 2 Det the. 0 S. NP VP 0 S NP. VP 1 VP V. NP 2 NP Det. N 0 NP. Det N 0 NP NP. PP 2 NP. Det N 3 N. caviar 0 NP. NP PP 1 VP. V NP 2 NP. NP PP 3 N. spoon 0 NP. Papa 1 VP. VP PP 2 NP. Papa 0 Det. the 1 PP. P NP 2 Det. the 0 Det. a 1 V. ate 2 Det. a 1 P. with
90 0 Papa 1 ate 2 the 3 caviar 4 0 ROOT. S 0 NP Papa. 1 V ate. 2 Det the. 3 N caviar. 0 S. NP VP 0 S NP. VP 1 VP V. NP 2 NP Det. N 0 NP. Det N 0 NP NP. PP 2 NP. Det N 3 N. caviar 0 NP. NP PP 1 VP. V NP 2 NP. NP PP 3 N. spoon 0 NP. Papa 1 VP. VP PP 2 NP. Papa 0 Det. the 1 PP. P NP 2 Det. the 0 Det. a 1 V. ate 2 Det. a 1 P. with
91 0 Papa 1 ate 2 the 3 caviar 4 0 ROOT. S 0 NP Papa. 1 V ate. 2 Det the. 3 N caviar. 0 S. NP VP 0 S NP. VP 1 VP V. NP 2 NP Det. N 0 NP. Det N 0 NP NP. PP 2 NP. Det N 3 N. caviar 0 NP. NP PP 1 VP. V NP 2 NP. NP PP 3 N. spoon 0 NP. Papa 1 VP. VP PP 2 NP. Papa 0 Det. the 1 PP. P NP 2 Det. the 0 Det. a 1 V. ate 2 Det. a 1 P. with
92 0 Papa 1 ate 2 the 3 caviar 4 0 ROOT. S 0 NP Papa. 1 V ate. 2 Det the. 3 N caviar. 0 S. NP VP 0 S NP. VP 1 VP V. NP 2 NP Det. N 2 NP Det N. 0 NP. Det N 0 NP NP. PP 2 NP. Det N 3 N. caviar 0 NP. NP PP 1 VP. V NP 2 NP. NP PP 3 N. spoon 0 NP. Papa 1 VP. VP PP 2 NP. Papa 0 Det. the 1 PP. P NP 2 Det. the 0 Det. a 1 V. ate 2 Det. a 1 P. with attach
93 0 Papa 1 ate 2 the 3 caviar 4 0 ROOT. S 0 NP Papa. 1 V ate. 2 Det the. 3 N caviar. 0 S. NP VP 0 S NP. VP 1 VP V. NP 2 NP Det. N 2 NP Det N. 0 NP. Det N 0 NP NP. PP 2 NP. Det N 3 N. caviar 1 VP V NP. 0 NP. NP PP 1 VP. V NP 2 NP. NP PP 3 N. spoon 2 NP NP. PP 0 NP. Papa 1 VP. VP PP 2 NP. Papa 0 Det. the 1 PP. P NP 2 Det. the 0 Det. a 1 V. ate 2 Det. a 1 P. with attach (again!)
94 0 Papa 1 ate 2 the 3 caviar 4 0 ROOT. S 0 NP Papa. 1 V ate. 2 Det the. 3 N caviar. 0 S. NP VP 0 S NP. VP 1 VP V. NP 2 NP Det. N 2 NP Det N. 0 NP. Det N 0 NP NP. PP 2 NP. Det N 3 N. caviar 1 VP V NP. 0 NP. NP PP 1 VP. V NP 2 NP. NP PP 3 N. spoon 2 NP NP. PP 0 NP. Papa 1 VP. VP PP 2 NP. Papa 0 S NP VP. 0 Det. the 1 PP. P NP 2 Det. the 1 VP VP. PP 0 Det. a 1 V. ate 2 Det. a 1 P. with attach (again!)
95 0 Papa 1 ate 2 the 3 caviar 4 0 ROOT. S 0 NP Papa. 1 V ate. 2 Det the. 3 N caviar. 0 S. NP VP 0 S NP. VP 1 VP V. NP 2 NP Det. N 2 NP Det N. 0 NP. Det N 0 NP NP. PP 2 NP. Det N 3 N. caviar 1 VP V NP. 0 NP. NP PP 1 VP. V NP 2 NP. NP PP 3 N. spoon 2 NP NP. PP 0 NP. Papa 1 VP. VP PP 2 NP. Papa 0 S NP VP. 0 Det. the 1 PP. P NP 2 Det. the 1 VP VP. PP 0 Det. a 1 V. ate 2 Det. a 4 PP. P NP 1 P. with
96 0 Papa 1 ate 2 the 3 caviar 4 0 ROOT. S 0 NP Papa. 1 V ate. 2 Det the. 3 N caviar. 0 S. NP VP 0 S NP. VP 1 VP V. NP 2 NP Det. N 2 NP Det N. 0 NP. Det N 0 NP NP. PP 2 NP. Det N 3 N. caviar 1 VP V NP. 0 NP. NP PP 1 VP. V NP 2 NP. NP PP 3 N. spoon 2 NP NP. PP 0 NP. Papa 1 VP. VP PP 2 NP. Papa 0 S NP VP. 0 Det. the 1 PP. P NP 2 Det. the 1 VP VP. PP 0 Det. a 1 V. ate 2 Det. a 4 PP. P NP 1 P. with 0 ROOT S. attach (again!)
97 0 Papa 1 ate 2 the 3 caviar 4 0 ROOT. S 0 NP Papa. 1 V ate. 2 Det the. 3 N caviar. 0 S. NP VP 0 S NP. VP 1 VP V. NP 2 NP Det. N 2 NP Det N. 0 NP. Det N 0 NP NP. PP 2 NP. Det N 3 N. caviar 1 VP V NP. 0 NP. NP PP 1 VP. V NP 2 NP. NP PP 3 N. spoon 2 NP NP. PP 0 NP. Papa 1 VP. VP PP 2 NP. Papa 0 S NP VP. 0 Det. the 1 PP. P NP 2 Det. the 1 VP VP. PP 0 Det. a 1 V. ate 2 Det. a 4 PP. P NP 1 P. with 0 ROOT S.
98 0 Papa 1 ate 2 the 3 caviar 4 0 ROOT. S 0 NP Papa. 1 V ate. 2 Det the. 3 N caviar. 0 S. NP VP 0 S NP. VP 1 VP V. NP 2 NP Det. N 2 NP Det N. 0 NP. Det N 0 NP NP. PP 2 NP. Det N 3 N. caviar 1 VP V NP. 0 NP. NP PP 1 VP. V NP 2 NP. NP PP 3 N. spoon 2 NP NP. PP 0 NP. Papa 1 VP. VP PP 2 NP. Papa 0 S NP VP. 0 Det. the 1 PP. P NP 2 Det. the 1 VP VP. PP 0 Det. a 1 V. ate 2 Det. a 4 PP. P NP 1 P. with 0 ROOT S. 4 P. with
99 0 Papa 1 ate 2 the 3 caviar 4 0 ROOT. S 0 NP Papa. 1 V ate. 2 Det the. 3 N caviar. 0 S. NP VP 0 S NP. VP 1 VP V. NP 2 NP Det. N 2 NP Det N. 0 NP. Det N 0 NP NP. PP 2 NP. Det N 3 N. caviar 1 VP V NP. 0 NP. NP PP 1 VP. V NP 2 NP. NP PP 3 N. spoon 2 NP NP. PP 0 NP. Papa 1 VP. VP PP 2 NP. Papa 0 S NP VP. 0 Det. the 1 PP. P NP 2 Det. the 1 VP VP. PP 0 Det. a 1 V. ate 2 Det. a 4 PP. P NP 1 P. with 0 ROOT S. 4 P. with
100 0 Papa 1 ate 2 the 3 caviar 4 with 5 0 ROOT. S 0 NP Papa. 1 V ate. 2 Det the. 3 N caviar. 4 P with. 0 S. NP VP 0 S NP. VP 1 VP V. NP 2 NP Det. N 2 NP Det N. 0 NP. Det N 0 NP NP. PP 2 NP. Det N 3 N. caviar 1 VP V NP. 0 NP. NP PP 1 VP. V NP 2 NP. NP PP 3 N. spoon 2 NP NP. PP 0 NP. Papa 1 VP. VP PP 2 NP. Papa 0 S NP VP. 0 Det. the 1 PP. P NP 2 Det. the 1 VP VP. PP 0 Det. a 1 V. ate 2 Det. a 4 PP. P NP 1 P. with 0 ROOT S. 4 P. with
101 0 Papa 1 ate 2 the 3 caviar 4 with 5 0 ROOT. S 0 NP Papa. 1 V ate. 2 Det the. 3 N caviar. 4 P with. 0 S. NP VP 0 S NP. VP 1 VP V. NP 2 NP Det. N 2 NP Det N. 4 PP P. NP 0 NP. Det N 0 NP NP. PP 2 NP. Det N 3 N. caviar 1 VP V NP. 0 NP. NP PP 1 VP. V NP 2 NP. NP PP 3 N. spoon 2 NP NP. PP 0 NP. Papa 1 VP. VP PP 2 NP. Papa 0 S NP VP. 0 Det. the 1 PP. P NP 2 Det. the 1 VP VP. PP 0 Det. a 1 V. ate 2 Det. a 4 PP. P NP 1 P. with 0 ROOT S. 4 P. with
102 0 Papa 1 ate 2 the 3 caviar 4 with 5 0 ROOT. S 0 NP Papa. 1 V ate. 2 Det the. 3 N caviar. 4 P with. 0 S. NP VP 0 S NP. VP 1 VP V. NP 2 NP Det. N 2 NP Det N. 4 PP P. NP 0 NP. Det N 0 NP NP. PP 2 NP. Det N 3 N. caviar 1 VP V NP. 5 NP. Det N 0 NP. NP PP 1 VP. V NP 2 NP. NP PP 3 N. spoon 2 NP NP. PP 5 NP. NP PP 0 NP. Papa 1 VP. VP PP 2 NP. Papa 0 S NP VP. 5 NP. Papa 0 Det. the 1 PP. P NP 2 Det. the 1 VP VP. PP 0 Det. a 1 V. ate 2 Det. a 4 PP. P NP 1 P. with 0 ROOT S. 4 P. with
103 0 Papa 1 ate 2 the 3 caviar 4 with 5 0 ROOT. S 0 NP Papa. 1 V ate. 2 Det the. 3 N caviar. 4 P with. 0 S. NP VP 0 S NP. VP 1 VP V. NP 2 NP Det. N 2 NP Det N. 4 PP P. NP 0 NP. Det N 0 NP NP. PP 2 NP. Det N 3 N. caviar 1 VP V NP. 5 NP. Det N 0 NP. NP PP 1 VP. V NP 2 NP. NP PP 3 N. spoon 2 NP NP. PP 5 NP. NP PP 0 NP. Papa 1 VP. VP PP 2 NP. Papa 0 S NP VP. 5 NP. Papa 0 Det. the 1 PP. P NP 2 Det. the 1 VP VP. PP 5 Det. the 0 Det. a 1 V. ate 2 Det. a 4 PP. P NP 5 Det. a 1 P. with 0 ROOT S. 4 P. with
104 0 Papa 1 ate 2 the 3 caviar 4 with 5 0 ROOT. S 0 NP Papa. 1 V ate. 2 Det the. 3 N caviar. 4 P with. 0 S. NP VP 0 S NP. VP 1 VP V. NP 2 NP Det. N 2 NP Det N. 4 PP P. NP 0 NP. Det N 0 NP NP. PP 2 NP. Det N 3 N. caviar 1 VP V NP. 5 NP. Det N 0 NP. NP PP 1 VP. V NP 2 NP. NP PP 3 N. spoon 2 NP NP. PP 5 NP. NP PP 0 NP. Papa 1 VP. VP PP 2 NP. Papa 0 S NP VP. 5 NP. Papa 0 Det. the 1 PP. P NP 2 Det. the 1 VP VP. PP 5 Det. the 0 Det. a 1 V. ate 2 Det. a 4 PP. P NP 5 Det. a 1 P. with 0 ROOT S. 4 P. with
105 0 Papa 1 ate 2 the 3 caviar 4 with 5 0 ROOT. S 0 NP Papa. 1 V ate. 2 Det the. 3 N caviar. 4 P with. 0 S. NP VP 0 S NP. VP 1 VP V. NP 2 NP Det. N 2 NP Det N. 4 PP P. NP 0 NP. Det N 0 NP NP. PP 2 NP. Det N 3 N. caviar 1 VP V NP. 5 NP. Det N 0 NP. NP PP 1 VP. V NP 2 NP. NP PP 3 N. spoon 2 NP NP. PP 5 NP. NP PP 0 NP. Papa 1 VP. VP PP 2 NP. Papa 0 S NP VP. 5 NP. Papa 0 Det. the 1 PP. P NP 2 Det. the 1 VP VP. PP 5 Det. the 0 Det. a 1 V. ate 2 Det. a 4 PP. P NP 5 Det. a 1 P. with 0 ROOT S. 4 P. with
106 0 Papa 1 ate 2 the 3 caviar 4 with 5 0 ROOT. S 0 NP Papa. 1 V ate. 2 Det the. 3 N caviar. 4 P with. 0 S. NP VP 0 S NP. VP 1 VP V. NP 2 NP Det. N 2 NP Det N. 4 PP P. NP 0 NP. Det N 0 NP NP. PP 2 NP. Det N 3 N. caviar 1 VP V NP. 5 NP. Det N 0 NP. NP PP 1 VP. V NP 2 NP. NP PP 3 N. spoon 2 NP NP. PP 5 NP. NP PP 0 NP. Papa 1 VP. VP PP 2 NP. Papa 0 S NP VP. 5 NP. Papa 0 Det. the 1 PP. P NP 2 Det. the 1 VP VP. PP 5 Det. the 0 Det. a 1 V. ate 2 Det. a 4 PP. P NP 5 Det. a 1 P. with 0 ROOT S. 4 P. with
107 e 2 the 3 caviar 4 with 5 a 6 1 V ate. 2 Det the. 3 N caviar. 4 P with. 1 VP V. NP 2 NP Det. N 2 NP Det N. 4 PP P. NP P 2 NP. Det N 3 N. caviar 1 VP V NP. 5 NP. Det N 2 NP. NP PP 3 N. spoon 2 NP NP. PP 5 NP. NP PP P 2 NP. Papa 0 S NP VP. 5 NP. Papa 2 Det. the 1 VP VP. PP 5 Det. the 2 Det. a 4 PP. P NP 5 Det. a 0 ROOT S. 4 P. with 5 Det a.
108 e 2 the 3 caviar 4 with 5 a 6 1 V ate. 2 Det the. 3 N caviar. 4 P with. 1 VP V. NP 2 NP Det. N 2 NP Det N. 4 PP P. NP P 2 NP. Det N 3 N. caviar 1 VP V NP. 5 NP. Det N 2 NP. NP PP 3 N. spoon 2 NP NP. PP 5 NP. NP PP P 2 NP. Papa 0 S NP VP. 5 NP. Papa 2 Det. the 1 VP VP. PP 5 Det. the 2 Det. a 4 PP. P NP 5 Det. a 0 ROOT S. 4 P. with 5 Det a. 5 NP Det. N
109 e 2 the 3 caviar 4 with 5 a 6 1 V ate. 2 Det the. 3 N caviar. 4 P with. 1 VP V. NP 2 NP Det. N 2 NP Det N. 4 PP P. NP P 2 NP. Det N 3 N. caviar 1 VP V NP. 5 NP. Det N 2 NP. NP PP 3 N. spoon 2 NP NP. PP 5 NP. NP PP P 2 NP. Papa 0 S NP VP. 5 NP. Papa 2 Det. the 1 VP VP. PP 5 Det. the 2 Det. a 4 PP. P NP 5 Det. a 0 ROOT S. 4 P. with 5 Det a. 5 NP Det. N 6 N. caviar 6 N. spoon
110 e 2 the 3 caviar 4 with 5 a 6 1 V ate. 2 Det the. 3 N caviar. 4 P with. 1 VP V. NP 2 NP Det. N 2 NP Det N. 4 PP P. NP P 2 NP. Det N 3 N. caviar 1 VP V NP. 5 NP. Det N 2 NP. NP PP 3 N. spoon 2 NP NP. PP 5 NP. NP PP P 2 NP. Papa 0 S NP VP. 5 NP. Papa 2 Det. the 1 VP VP. PP 5 Det. the 2 Det. a 4 PP. P NP 5 Det. a 0 ROOT S. 4 P. with 5 Det a. 5 NP Det. N 6 N. caviar 6 N. spoon
111 e 2 the 3 caviar 4 with 5 a 6 spoon 7 1 V ate. 2 Det the. 3 N caviar. 4 P with. 5 Det a. 1 VP V. NP 2 NP Det. N 2 NP Det N. 4 PP P. NP 5 NP Det. N P 2 NP. Det N 3 N. caviar 1 VP V NP. 5 NP. Det N 6 N. caviar 2 NP. NP PP 3 N. spoon 2 NP NP. PP 5 NP. NP PP 6 N. spoon P 2 NP. Papa 0 S NP VP. 5 NP. Papa 2 Det. the 1 VP VP. PP 5 Det. the 2 Det. a 4 PP. P NP 5 Det. a 0 ROOT S. 4 P. with 6 N spoon.
112 e 2 the 3 caviar 4 with 5 a 6 spoon 7 1 V ate. 2 Det the. 3 N caviar. 4 P with. 5 Det a. 1 VP V. NP 2 NP Det. N 2 NP Det N. 4 PP P. NP 5 NP Det. N P 2 NP. Det N 3 N. caviar 1 VP V NP. 5 NP. Det N 6 N. caviar 2 NP. NP PP 3 N. spoon 2 NP NP. PP 5 NP. NP PP 6 N. spoon P 2 NP. Papa 0 S NP VP. 5 NP. Papa 2 Det. the 1 VP VP. PP 5 Det. the 2 Det. a 4 PP. P NP 5 Det. a 0 ROOT S. 4 P. with 6 N spoon. 5 NP Det N.
113 e 2 the 3 caviar 4 with 5 a 6 spoon 7 1 V ate. 2 Det the. 3 N caviar. 4 P with. 5 Det a. 1 VP V. NP 2 NP Det. N 2 NP Det N. 4 PP P. NP 5 NP Det. N P 2 NP. Det N 3 N. caviar 1 VP V NP. 5 NP. Det N 6 N. caviar 2 NP. NP PP 3 N. spoon 2 NP NP. PP 5 NP. NP PP 6 N. spoon P 2 NP. Papa 0 S NP VP. 5 NP. Papa 2 Det. the 1 VP VP. PP 5 Det. the 2 Det. a 4 PP. P NP 5 Det. a 0 ROOT S. 4 P. with 6 N spoon. 5 NP Det N. 4 PP P NP. 5 NP NP. PP
114 0 Papa 1 ate 2 the 3 caviar 4 with a spoon 7 0 ROOT. S 0 NP Papa. 1 V ate. 2 Det the. 3 N caviar. 0 S. NP VP 0 S NP. VP 1 VP V. NP 2 NP Det. N 2 NP Det N. 0 NP. Det N 0 NP NP. PP 2 NP. Det N 3 N. caviar 1 VP V NP. 0 NP. NP PP 1 VP. V NP 2 NP. NP PP 3 N. spoon 2 NP NP. PP 0 NP. Papa 1 VP. VP PP 2 NP. Papa 0 S NP VP. 0 Det. the 1 PP. P NP 2 Det. the 1 VP VP. PP 0 Det. a 1 V. ate 2 Det. a 4 PP. P NP 1 P. with 0 ROOT S. 4 P. with 6 N spoon. 5 NP Det N. 4 PP P NP. 5 NP NP. PP 2 NP NP PP. 1 VP VP PP.
115 0 Papa 1 ate 2 the 3 caviar 4 with a spoon 7 0 ROOT. S 0 NP Papa. 1 V ate. 2 Det the. 3 N caviar. 0 S. NP VP 0 S NP. VP 1 VP V. NP 2 NP Det. N 2 NP Det N. 0 NP. Det N 0 NP NP. PP 2 NP. Det N 3 N. caviar 1 VP V NP. 0 NP. NP PP 1 VP. V NP 2 NP. NP PP 3 N. spoon 2 NP NP. PP 0 NP. Papa 1 VP. VP PP 2 NP. Papa 0 S NP VP. 0 Det. the 1 PP. P NP 2 Det. the 1 VP VP. PP 0 Det. a 1 V. ate 2 Det. a 4 PP. P NP 1 P. with 0 ROOT S. 4 P. with 6 N spoon. 5 NP Det N. 4 PP P NP. 5 NP NP. PP 2 NP NP PP. 1 VP VP PP. 7 PP. P NP
116 0 Papa 1 ate 2 the 3 caviar 4 with a spoon 7 0 ROOT. S 0 NP Papa. 1 V ate. 2 Det the. 3 N caviar. 0 S. NP VP 0 S NP. VP 1 VP V. NP 2 NP Det. N 2 NP Det N. 0 NP. Det N 0 NP NP. PP 2 NP. Det N 3 N. caviar 1 VP V NP. 0 NP. NP PP 1 VP. V NP 2 NP. NP PP 3 N. spoon 2 NP NP. PP 0 NP. Papa 1 VP. VP PP 2 NP. Papa 0 S NP VP. 0 Det. the 1 PP. P NP 2 Det. the 1 VP VP. PP 0 Det. a 1 V. ate 2 Det. a 4 PP. P NP 1 P. with 0 ROOT S. 4 P. with 6 N spoon. 5 NP Det N. 4 PP P NP. 5 NP NP. PP 2 NP NP PP. 1 VP VP PP. 7 PP. P NP 1 VP V NP. 2 NP NP. PP
117 0 Papa 1 ate 2 the 3 caviar 4 with a spoon 7 0 ROOT. S 0 NP Papa. 1 V ate. 2 Det the. 3 N caviar. 0 S. NP VP 0 S NP. VP 1 VP V. NP 2 NP Det. N 2 NP Det N. 0 NP. Det N 0 NP NP. PP 2 NP. Det N 3 N. caviar 1 VP V NP. 0 NP. NP PP 1 VP. V NP 2 NP. NP PP 3 N. spoon 2 NP NP. PP 0 NP. Papa 1 VP. VP PP 2 NP. Papa 0 S NP VP. 0 Det. the 1 PP. P NP 2 Det. the 1 VP VP. PP 0 Det. a 1 V. ate 2 Det. a 4 PP. P NP 1 P. with 0 ROOT S. 4 P. with 6 N spoon. 5 NP Det N. 4 PP P NP. 5 NP NP. PP 2 NP NP PP. 1 VP VP PP. 7 PP. P NP 1 VP V NP. 2 NP NP. PP 0 S NP VP. 1 VP VP. PP
118 0 Papa 1 ate 2 the 3 caviar 4 with a spoon 7 0 ROOT. S 0 NP Papa. 1 V ate. 2 Det the. 3 N caviar. 0 S. NP VP 0 S NP. VP 1 VP V. NP 2 NP Det. N 2 NP Det N. 0 NP. Det N 0 NP NP. PP 2 NP. Det N 3 N. caviar 1 VP V NP. 0 NP. NP PP 1 VP. V NP 2 NP. NP PP 3 N. spoon 2 NP NP. PP 0 NP. Papa 1 VP. VP PP 2 NP. Papa 0 S NP VP. 0 Det. the 1 PP. P NP 2 Det. the 1 VP VP. PP 0 Det. a 1 V. ate 2 Det. a 4 PP. P NP 1 P. with 0 ROOT S. 4 P. with 6 N spoon. 5 NP Det N. 4 PP P NP. 5 NP NP. PP 2 NP NP PP. 1 VP VP PP. 7 PP. P NP 1 VP V NP. 2 NP NP. PP 0 S NP VP. 1 VP VP. PP 7 P. with
119 0 Papa 1 ate 2 the 3 caviar 4 with a spoon 7 0 ROOT. S 0 NP Papa. 1 V ate. 2 Det the. 3 N caviar. 0 S. NP VP 0 S NP. VP 1 VP V. NP 2 NP Det. N 2 NP Det N. 0 NP. Det N 0 NP NP. PP 2 NP. Det N 3 N. caviar 1 VP V NP. 0 NP. NP PP 1 VP. V NP 2 NP. NP PP 3 N. spoon 2 NP NP. PP 0 NP. Papa 1 VP. VP PP 2 NP. Papa 0 S NP VP. 0 Det. the 1 PP. P NP 2 Det. the 1 VP VP. PP 0 Det. a 1 V. ate 2 Det. a 4 PP. P NP 1 P. with 0 ROOT S. 4 P. with 6 N spoon. 5 NP Det N. 4 PP P NP. 5 NP NP. PP 2 NP NP PP. 1 VP VP PP. 7 PP. P NP 1 VP V NP. 2 NP NP. PP 0 S NP VP. 1 VP VP. PP 7 P. with
120 0 Papa 1 ate 2 the 3 caviar 4 with a spoon 7 0 ROOT. S 0 NP Papa. 1 V ate. 2 Det the. 3 N caviar. 0 S. NP VP 0 S NP. VP 1 VP V. NP 2 NP Det. N 2 NP Det N. 0 NP. Det N 0 NP NP. PP 2 NP. Det N 3 N. caviar 1 VP V NP. 0 NP. NP PP 1 VP. V NP 2 NP. NP PP 3 N. spoon 2 NP NP. PP 0 NP. Papa 1 VP. VP PP 2 NP. Papa 0 S NP VP. 0 Det. the 1 PP. P NP 2 Det. the 1 VP VP. PP 0 Det. a 1 V. ate 2 Det. a 4 PP. P NP 1 P. with 0 ROOT S. 4 P. with 6 N spoon. 5 NP Det N. 4 PP P NP. 5 NP NP. PP 2 NP NP PP. 1 VP VP PP. 7 PP. P NP 1 VP V NP. 2 NP NP. PP 0 S NP VP. 1 VP VP. PP 7 P. with
121 0 Papa 1 ate 2 the 3 caviar 4 with a spoon 7 0 ROOT. S 0 NP Papa. 1 V ate. 2 Det the. 3 N caviar. 0 S. NP VP 0 S NP. VP 1 VP V. NP 2 NP Det. N 2 NP Det N. 0 NP. Det N 0 NP NP. PP 2 NP. Det N 3 N. caviar 1 VP V NP. 0 NP. NP PP 1 VP. V NP 2 NP. NP PP 3 N. spoon 2 NP NP. PP 0 NP. Papa 1 VP. VP PP 2 NP. Papa 0 S NP VP. 0 Det. the 1 PP. P NP 2 Det. the 1 VP VP. PP 0 Det. a 1 V. ate 2 Det. a 4 PP. P NP 1 P. with 0 ROOT S. 4 P. with 6 N spoon. 5 NP Det N. 4 PP P NP. 5 NP NP. PP 2 NP NP PP. 1 VP VP PP. 7 PP. P NP 1 VP V NP. 2 NP NP. PP 0 S NP VP. 1 VP VP. PP 7 P. with 0 ROOT S.
122 0 Papa 1 ate 2 the 3 caviar 4 with a spoon 7 0 ROOT. S 0 NP Papa. 1 V ate. 2 Det the. 3 N caviar. 0 S. NP VP 0 S NP. VP 1 VP V. NP 2 NP Det. N 2 NP Det N. 0 NP. Det N 0 NP NP. PP 2 NP. Det N 3 N. caviar 1 VP V NP. 0 NP. NP PP 1 VP. V NP 2 NP. NP PP 3 N. spoon 2 NP NP. PP 0 NP. Papa 1 VP. VP PP 2 NP. Papa 0 S NP VP. 0 Det. the 1 PP. P NP 2 Det. the 1 VP VP. PP 0 Det. a 1 V. ate 2 Det. a 4 PP. P NP 1 P. with 0 ROOT S. 4 P. with 6 N spoon. 5 NP Det N. 4 PP P NP. 5 NP NP. PP 2 NP NP PP. 1 VP VP PP. 7 PP. P NP 1 VP V NP. 2 NP NP. PP 0 S NP VP. 1 VP VP. PP 7 P. with 0 ROOT S.
123 0 Papa 1 ate 2 the 3 caviar 4 with a spoon 7 0 ROOT. S 0 NP Papa. 1 V ate. 2 Det the. 3 N caviar. 0 S. NP VP 0 S NP. VP 1 VP V. NP 2 NP Det. N 2 NP Det N. 0 NP. Det N 0 NP NP. PP 2 NP. Det N 3 N. caviar 1 VP V NP. 0 NP. NP PP 1 VP. V NP 2 NP. NP PP 3 N. spoon 2 NP NP. PP 0 NP. Papa 1 VP. VP PP 2 NP. Papa 0 S NP VP. 0 Det. the 1 PP. P NP 2 Det. the 1 VP VP. PP 0 Det. a 1 V. ate 2 Det. a 4 PP. P NP 1 P. with 0 ROOT S. 4 P. with 6 N spoon. 5 NP Det N. 4 PP P NP. 5 NP NP. PP 2 NP NP PP. 1 VP VP PP. 7 PP. P NP 1 VP V NP. 2 NP NP. PP 0 S NP VP. 1 VP VP. PP 7 P. with 0 ROOT S.
124 0 Papa 1 ate 2 the 3 caviar 4 with a spoon 7 0 ROOT. S 0 NP Papa. 1 V ate. 2 Det the. 3 N caviar. 0 S. NP VP 0 S NP. VP 1 VP V. NP 2 NP Det. N 2 NP Det N. 0 NP. Det N 0 NP NP. PP 2 NP. Det N 3 N. caviar 1 VP V NP. 0 NP. NP PP 1 VP. V NP 2 NP. NP PP 3 N. spoon 2 NP NP. PP 0 NP. Papa 1 VP. VP PP 2 NP. Papa 0 S NP VP. 0 Det. the 1 PP. P NP 2 Det. the 1 VP VP. PP 0 Det. a 1 V. ate 2 Det. a 4 PP. P NP 1 P. with 0 ROOT S. 4 P. with 6 N spoon. 5 NP Det N. 4 PP P NP. 5 NP NP. PP 2 NP NP PP. 1 VP VP PP. 7 PP. P NP 1 VP V NP. 2 NP NP. PP 0 S NP VP. 1 VP VP. PP 7 P. with 0 ROOT S.
125 0 Papa 1 ate 2 the 3 caviar 4 with a spoon 7 0 ROOT. S 0 NP Papa. 1 V ate. 2 Det the. 3 N caviar. 0 S. NP VP 0 S NP. VP 1 VP V. NP 2 NP Det. N 2 NP Det N. 0 NP. Det N 0 NP NP. PP 2 NP. Det N 3 N. caviar 1 VP V NP. 0 NP. NP PP 1 VP. V NP 2 NP. NP PP 3 N. spoon 2 NP NP. PP 0 NP. Papa 1 VP. VP PP 2 NP. Papa 0 S NP VP. 0 Det. the 1 PP. P NP 2 Det. the 1 VP VP. PP 0 Det. a 1 V. ate 2 Det. a 4 PP. P NP 1 P. with 0 ROOT S. 4 P. with 6 N spoon. 5 NP Det N. 4 PP P NP. 5 NP NP. PP 2 NP NP PP. 1 VP VP PP. 7 PP. P NP 1 VP V NP. 2 NP NP. PP 0 S NP VP. 1 VP VP. PP 7 P. with 0 ROOT S.
126 Left Recursion Kills Pure Top-Down Parsing VP Andrew McCallum, UMass Amherst
127 Left Recursion Kills Pure Top-Down Parsing VP VP PP Andrew McCallum, UMass Amherst
128 Left Recursion Kills Pure Top-Down Parsing VP VP PP VP PP Andrew McCallum, UMass Amherst
129 Left Recursion Kills Pure Top-Down Parsing VP VP PP VP PP makes new hypotheses ad infinitum before we ve seen the PPs at all VP PP hypotheses try to predict in advance how many PP s will arrive in input Andrew McCallum, UMass Amherst
130 but Earley s Alg is Okay! VP 1 VP!. VP PP VP PP (in column 1) Andrew McCallum, UMass Amherst
131 but Earley s Alg is Okay! VP 1 VP!. VP PP VP PP (in column 1) VP 1 VP! V NP. V NP ate the caviar (in column 4) Andrew McCallum, UMass Amherst
132 but Earley s Alg is Okay! VP 1 VP!. VP PP VP PP (in column 1) attach VP 1 VP! VP. PP VP PP V NP ate the caviar (in column 4) Andrew McCallum, UMass Amherst
133 but Earley s Alg is Okay! VP 1 VP!. VP PP VP PP (in column 1) VP 1 VP! VP PP. VP PP with a spoon V NP ate the caviar (in column 7) Andrew McCallum, UMass Amherst
134 but Earley s Alg is Okay! VP VP PP (in column 1) 1 VP!. VP PP can be reused VP 1 VP! VP PP. VP PP with a spoon V NP ate the caviar (in column 7) Andrew McCallum, UMass Amherst
135 but Earley s Alg is Okay! VP VP PP (in column 1) 1 VP!. VP PP can be reused attach VP 1 VP! VP. PP VP PP VP PP with a spoon V NP ate the caviar (in column 7) Andrew McCallum, UMass Amherst
136 but Earley s Alg is Okay! VP VP PP (in column 1) 1 VP!. VP PP can be reused VP VP PP in his bed VP PP with a spoon V NP ate the caviar (in column 10) 1 VP! VP PP. Andrew McCallum, UMass Amherst
137 but Earley s Alg is Okay! VP VP PP (in column 1) 1 VP!. VP PP can be reused again VP VP PP in his bed VP PP with a spoon V NP ate the caviar (in column 10) 1 VP! VP PP. Andrew McCallum, UMass Amherst
138 but Earley s Alg is Okay! VP VP PP (in column 1) 1 VP!. VP PP can be reused again attach VP VP VP PP in his bed VP PP with a spoon V NP ate the caviar (in column 10) 1 VP! VP. PP PP Andrew McCallum, UMass Amherst
139 0 Det. the 0 Det. a 0 Papa 1 ate 2 the 3 caviar 4 with a spoon 7 0 ROOT. S 0 S. NP VP 0 NP. Det N 0 NP. NP PP 0 NP. Papa 0 NP Papa. 0 S NP. VP 0 NP NP. PP 1 VP. V NP 1 VP. VP PP 1 PP. P NP 1 V. ate 1 P. with 1 V ate. 1 VP V. NP 2 NP. Det N 2 NP. NP PP 2 NP. Papa 2 Det. the 2 Det. a 2 Det the. 2 NP Det. N 3 N. caviar 3 N. spoon 3 N caviar. 2 NP Det N. 1 VP V NP. 2 NP NP. PP 0 S NP VP. 1 VP VP. PP 4 PP. P NP 0 ROOT S. 4 P. with completed a VP in col 4 col 1 lets us use it in a VP PP structure 6 N spoon. 5 NP Det N. 4 PP P NP. 5 NP NP. PP 2 NP NP PP. 1 VP VP PP. 7 PP. P NP 1 VP V NP. 2 NP NP. PP 0 S NP VP. 1 VP VP. PP 7 P. with 0 ROOT S.
140 0 Det. the 0 Det. a 0 Papa 1 ate 2 the 3 caviar 4 with a spoon 7 0 ROOT. S 0 S. NP VP 0 NP. Det N 0 NP. NP PP 0 NP. Papa 0 NP Papa. 0 S NP. VP 0 NP NP. PP 1 VP. V NP 1 VP. VP PP 1 PP. P NP 1 V. ate 1 P. with 1 V ate. 1 VP V. NP 2 NP. Det N 2 NP. NP PP 2 NP. Papa 2 Det. the 2 Det. a 2 Det the. 2 NP Det. N 3 N. caviar 3 N. spoon 3 N caviar. 2 NP Det N. 1 VP V NP. 2 NP NP. PP 0 S NP VP. 1 VP VP. PP 4 PP. P NP 0 ROOT S. 4 P. with completed that VP = VP PP in col 7 col 1 would let us use it in a VP PP structure can reuse col 1 as often as we need 6 N spoon. 5 NP Det N. 4 PP P NP. 5 NP NP. PP 2 NP NP PP. 1 VP VP PP. 7 PP. P NP 1 VP V NP. 2 NP NP. PP 0 S NP VP. 1 VP VP. PP 7 P. with 0 ROOT S.
141 Beyond Recognition So far, we ve described an Earley recognizer Note what we did when we tried to create entries that already existed What should we do when combining items? How to derive outside algorithm?
142 Parsing Tricks
143 Left-Corner Parsing Technique for 1 word of lookahead in algorithms like Earley s (can also do multi-word lookahead but it s harder)
144 Basic Earley s Algorithm 0 Papa 1 0 ROOT. S 0 NP Papa. 0 S. NP VP 0 S NP. VP 0 NP. Det N 0 NP NP. PP 0 NP. NP PP 0 NP. Papa 0 Det. the 0 Det. a attach
145 0 Papa 1 0 ROOT. S 0 NP Papa. 0 S. NP VP 0 S NP. VP 0 NP. Det N 0 NP NP. PP 0 NP. NP PP 1 VP. V NP 0 NP. Papa 1 VP. VP PP 0 Det. the 0 Det. a predict
146 0 Papa 1 0 ROOT. S 0 NP Papa. 0 S. NP VP 0 S NP. VP 0 NP. Det N 0 NP NP. PP 0 NP. NP PP 1 VP. V NP 0 NP. Papa 1 VP. VP PP 0 Det. the 1 PP. P NP 0 Det. a predict
147 0 Papa 1 0 ROOT. S 0 NP Papa. 0 S. NP VP 0 S NP. VP 0 NP. Det N 0 NP NP. PP 0 NP. NP PP 1 VP. V NP 0 NP. Papa 1 VP. VP PP 0 Det. the 1 PP. P NP 0 Det. a 1 V. ate 1 V. drank 1 V. snorted predict
148 0 Papa 1 0 ROOT. S 0 NP Papa. 0 S. NP VP 0 S NP. VP 0 NP. Det N 0 NP NP. PP 0 NP. NP PP 1 VP. V NP 0 NP. Papa 1 VP. VP PP 0 Det. the 1 PP. P NP 0 Det. a 1 V. ate 1 V. drank 1 V. snorted predict
149 0 Papa 1 0 ROOT. S 0 NP Papa. 0 S. NP VP 0 S NP. VP 0 NP. Det N 0 NP NP. PP 0 NP. NP PP 1 VP. V NP 0 NP. Papa 1 VP. VP PP 0 Det. the 1 PP. P NP 0 Det. a 1 V. ate 1 V. drank 1 V. snorted predict!.v makes us add all the verbs in the vocabulary!! Slow we d like a shortcut.
150 0 Papa 1 0 ROOT. S 0 NP Papa. 0 S. NP VP 0 S NP. VP 0 NP. Det N 0 NP NP. PP 0 NP. NP PP 1 VP. V NP 0 NP. Papa 1 VP. VP PP 0 Det. the 1 PP. P NP 0 Det. a 1 V. ate 1 V. drank 1 V. snorted predict
151 0 Papa 1 0 ROOT. S 0 NP Papa. 0 S. NP VP 0 S NP. VP 0 NP. Det N 0 NP NP. PP 0 NP. NP PP 1 VP. V NP 0 NP. Papa 1 VP. VP PP 0 Det. the 1 PP. P NP 0 Det. a 1 V. ate 1 V. drank 1 V. snorted predict
152 0 Papa 1 0 ROOT. S 0 NP Papa. 0 S. NP VP 0 S NP. VP 0 NP. Det N 0 NP NP. PP 0 NP. NP PP 1 VP. V NP 0 NP. Papa 1 VP. VP PP 0 Det. the 1 PP. P NP 0 Det. a 1 V. ate 1 V. drank 1 V. snorted predict! Every.VP adds all VP " rules again.! Before adding a rule, check it s not a duplicate.! Slow if there are > 700 VP " rules
153 0 Papa 1 0 ROOT. S 0 NP Papa. 0 S. NP VP 0 S NP. VP 0 NP. Det N 0 NP NP. PP 0 NP. NP PP 1 VP. V NP 0 NP. Papa 1 VP. VP PP 0 Det. the 1 PP. P NP 0 Det. a 1 V. ate 1 V. drank 1 V. snorted 1 P. with predict
154 0 Papa 1 0 ROOT. S 0 NP Papa. 0 S. NP VP 0 S NP. VP 0 NP. Det N 0 NP NP. PP 0 NP. NP PP 1 VP. V NP 0 NP. Papa 1 VP. VP PP 0 Det. the 1 PP. P NP 0 Det. a 1 V. ate 1 V. drank 1 V. snorted 1 P. with predict!.p makes us add all the prepositions
155 1-word lookahead would help 0 Papa 1 ate 0 ROOT. S 0 NP Papa. 0 S. NP VP 0 S NP. VP 0 NP. Det N 0 NP NP. PP 0 NP. NP PP 1 VP. V NP 0 NP. Papa 1 VP. VP PP 0 Det. the 1 PP. P NP 0 Det. a 1 V. ate 1 V. drank 1 V. snorted 1 P. with
156 1-word lookahead would help 0 Papa 1 0 ROOT. S 0 NP Papa. 0 S. NP VP 0 S NP. VP 0 NP. Det N 0 NP NP. PP 0 NP. NP PP 1 VP. V NP 0 NP. Papa 1 VP. VP PP 0 Det. the 1 PP. P NP 0 Det. a 1 V. ate 1 V. drank 1 V. snorted 1 P. with ate No point in adding words other than ate
157 1-word lookahead would help 0 Papa 1 0 ROOT. S 0 NP Papa. 0 S. NP VP 0 S NP. VP 0 NP. Det N 0 NP NP. PP 0 NP. NP PP 1 VP. V NP 0 NP. Papa 1 VP. VP PP 0 Det. the 1 PP. P NP 0 Det. a 1 V. ate 1 V. drank 1 V. snorted 1 P. with ate In fact, no point in adding any constituent that can t start with ate Don t bother adding PP, P, etc. No point in adding words other than ate
158 With Left-Corner Filter 0 Papa 1 ate 0 ROOT. S 0 NP Papa. 0 S. NP VP 0 S NP. VP 0 NP. Det N 0 NP NP. PP 0 NP. NP PP 0 NP. Papa 0 Det. the 0 Det. a attach
159 With Left-Corner Filter 0 Papa 1 0 ROOT. S 0 NP Papa. 0 S. NP VP 0 S NP. VP 0 NP. Det N 0 NP NP. PP 0 NP. NP PP 0 NP. Papa 0 Det. the 0 Det. a ate attach PP can t start with ate
160 With Left-Corner Filter 0 Papa 1 0 ROOT. S 0 NP Papa. 0 S. NP VP 0 S NP. VP 0 NP. Det N 0 NP NP. PP 0 NP. NP PP 0 NP. Papa 0 Det. the 0 Det. a ate attach PP can t start with ate Pruning now we won t predict 1 PP. P NP 1 PP. ate
161 With Left-Corner Filter 0 Papa 1 0 ROOT. S 0 NP Papa. 0 S. NP VP 0 S NP. VP 0 NP. Det N 0 NP NP. PP 0 NP. NP PP 0 NP. Papa 0 Det. the 0 Det. a ate attach PP can t start with ate Pruning now we won t predict 1 PP. P NP 1 PP. ate either!
162 With Left-Corner Filter 0 Papa 1 0 ROOT. S 0 NP Papa. 0 S. NP VP 0 S NP. VP 0 NP. Det N 0 NP NP. PP 0 NP. NP PP 0 NP. Papa 0 Det. the 0 Det. a ate attach PP can t start with ate Pruning now we won t predict 1 PP. P NP 1 PP. ate either!
163 0 Papa 1 0 ROOT. S 0 NP Papa. 0 S. NP VP 0 S NP. VP 0 NP. Det N 0 NP NP. PP 0 NP. NP PP 1 VP. V NP 0 NP. Papa 1 VP. VP PP 0 Det. the 0 Det. a ate predict
164 0 Papa 1 ate 0 ROOT. S 0 NP Papa. 0 S. NP VP 0 S NP. VP 0 NP. Det N 0 NP NP. PP 0 NP. NP PP 1 VP. V NP 0 NP. Papa 1 VP. VP PP 0 Det. the 1 V. ate 0 Det. a 1 V. drank 1 V. snorted predict
165 0 Papa 1 ate 0 ROOT. S 0 NP Papa. 0 S. NP VP 0 S NP. VP 0 NP. Det N 0 NP NP. PP 0 NP. NP PP 1 VP. V NP 0 NP. Papa 1 VP. VP PP 0 Det. the 1 V. ate 0 Det. a 1 V. drank 1 V. snorted predict
166 0 Papa 1 ate 0 ROOT. S 0 NP Papa. 0 S. NP VP 0 S NP. VP 0 NP. Det N 0 NP NP. PP 0 NP. NP PP 1 VP. V NP 0 NP. Papa 1 VP. VP PP 0 Det. the 1 V. ate 0 Det. a 1 V. drank 1 V. snorted predict
167 Merging Right-Hand Sides Grammar might have rules X A G H P X B G H P Could end up with both of these in chart: (2, X A. G H P) in column 5 (2, X B. G H P) in column 5 But these are now interchangeable: if one produces X then so will the other To avoid this redundancy, can always use dotted rules of this form: X... G H P
168 Merging Right-Hand Sides Similarly, grammar might have rules X A G H P X A G H Q Could end up with both of these in chart: (2, X A. G H P) in column 5 (2, X A. G H Q) in column 5 Not interchangeable, but we ll be processing them in parallel for a while Solution: write grammar as X A G H (P Q)
169 Merging Right-Hand Sides Combining the two previous cases: X A G H P X A G H Q X B G H P X B G H Q becomes X (A B) G H (P Q) And often nice to write stuff like NP (Det ε) Adj* N
170 Merging Right-Hand Sides
171 Merging Right-Hand Sides X (A B) G H (P Q) NP (Det ε) Adj* N
172 Merging Right-Hand Sides X (A B) G H (P Q) NP (Det ε) Adj* N These are regular expressions!
173 Merging Right-Hand Sides X (A B) G H (P Q) NP (Det ε) Adj* N These are regular expressions! Build their minimal DFAs:
174 Merging Right-Hand Sides X (A B) G H (P Q) NP (Det ε) Adj* N These are regular expressions! Build their minimal DFAs: A P X B G H Q
175 Merging Right-Hand Sides X (A B) G H (P Q) NP (Det ε) Adj* N These are regular expressions! Build their minimal DFAs: A P X B G H Q Adj Det NP N Adj N
176 Merging Right-Hand Sides X (A B) G H (P Q) NP (Det ε) Adj* N These are regular expressions! Build their minimal DFAs: A P X B G H Q Adj! Automaton states Det replace dotted NP N rules (X A G. H P) Adj N
177 Merging Right-Hand Sides Indeed, all NP rules can be unioned into a single DFA! NP ADJP ADJP JJ JJ NN NNS NP ADJP DT NN NP ADJP JJ NN NP ADJP JJ NN NNS NP ADJP JJ NNS NP ADJP NN NP ADJP NN NN NP ADJP NN NNS NP ADJP NNS NP ADJP NPR NP ADJP NPRS NP DT NP DT ADJP NP DT ADJP, JJ NN NP DT ADJP ADJP NN NP DT ADJP JJ JJ NN NP DT ADJP JJ NN NP DT ADJP JJ NN NN
178 Merging Right-Hand Sides Indeed, all NP rules can be unioned into a single DFA! NP ADJP ADJP JJ JJ NN NNS ADJP DT NN ADJP JJ NN ADJP JJ NN NNS ADJP JJ NNS ADJP NN ADJP NN NN ADJP NN NNS ADJP NNS ADJP NPR ADJP NPRS DT DT ADJP DT ADJP, JJ NN DT ADJP ADJP NN DT ADJP JJ JJ NN DT ADJP JJ NN DT ADJP JJ NN NN etc.
Context-Free Parsing: CKY & Earley Algorithms and Probabilistic Parsing
Context-Free Parsing: CKY & Earley Algorithms and Probabilistic Parsing Natural Language Processing! CS 6120 Spring 2014! Northeastern University!! David Smith! with some slides from Jason Eisner & Andrew
Context-Free Parsing: CKY & Earley Algorithms and Probabilistic Parsing
Context-Free Parsing: CKY & Earley Algorithms and Probabilistic Parsing Natural Language Processing CS 4120/6120 Spring 2016 Northeastern University David Smith with some slides from Jason Eisner & Andrew
Parsing. Based on presentations from Chris Manning s course on Statistical Parsing (Stanford)
Parsing Based on presentations from Chris Manning s course on Statistical Parsing (Stanford) S N VP V NP D N John hit the ball Levels of analysis Level Morphology/Lexical POS (morpho-synactic), WSD Elements
Probabilistic Context-free Grammars
Probabilistic Context-free Grammars Computational Linguistics Alexander Koller 24 November 2017 The CKY Recognizer S NP VP NP Det N VP V NP V ate NP John Det a N sandwich i = 1 2 3 4 k = 2 3 4 5 S NP John
CKY & Earley Parsing. Ling 571 Deep Processing Techniques for NLP January 13, 2016
CKY & Earley Parsing Ling 571 Deep Processing Techniques for NLP January 13, 2016 No Class Monday: Martin Luther King Jr. Day CKY Parsing: Finish the parse Recognizer à Parser Roadmap Earley parsing Motivation:
Parsing with CFGs L445 / L545 / B659. Dept. of Linguistics, Indiana University Spring Parsing with CFGs. Direction of processing
L445 / L545 / B659 Dept. of Linguistics, Indiana University Spring 2016 1 / 46 : Overview Input: a string Output: a (single) parse tree A useful step in the process of obtaining meaning We can view the
Parsing with CFGs. Direction of processing. Top-down. Bottom-up. Left-corner parsing. Chart parsing CYK. Earley 1 / 46.
: Overview L545 Dept. of Linguistics, Indiana University Spring 2013 Input: a string Output: a (single) parse tree A useful step in the process of obtaining meaning We can view the problem as searching
Natural Language Processing CS Lecture 06. Razvan C. Bunescu School of Electrical Engineering and Computer Science
Natural Language Processing CS 6840 Lecture 06 Razvan C. Bunescu School of Electrical Engineering and Computer Science bunescu@ohio.edu Statistical Parsing Define a probabilistic model of syntax P(T S):
LECTURER: BURCU CAN Spring
LECTURER: BURCU CAN 2017-2018 Spring Regular Language Hidden Markov Model (HMM) Context Free Language Context Sensitive Language Probabilistic Context Free Grammar (PCFG) Unrestricted Language PCFGs can
Parsing with Context-Free Grammars
Parsing with Context-Free Grammars CS 585, Fall 2017 Introduction to Natural Language Processing http://people.cs.umass.edu/~brenocon/inlp2017 Brendan O Connor College of Information and Computer Sciences
Review. Earley Algorithm Chapter Left Recursion. Left-Recursion. Rule Ordering. Rule Ordering
Review Earley Algorithm Chapter 13.4 Lecture #9 October 2009 Top-Down vs. Bottom-Up Parsers Both generate too many useless trees Combine the two to avoid over-generation: Top-Down Parsing with Bottom-Up
Advanced Natural Language Processing Syntactic Parsing
Advanced Natural Language Processing Syntactic Parsing Alicia Ageno ageno@cs.upc.edu Universitat Politècnica de Catalunya NLP statistical parsing 1 Parsing Review Statistical Parsing SCFG Inside Algorithm
CMPT-825 Natural Language Processing. Why are parsing algorithms important?
CMPT-825 Natural Language Processing Anoop Sarkar http://www.cs.sfu.ca/ anoop October 26, 2010 1/34 Why are parsing algorithms important? A linguistic theory is implemented in a formal system to generate
10/17/04. Today s Main Points
Part-of-speech Tagging & Hidden Markov Model Intro Lecture #10 Introduction to Natural Language Processing CMPSCI 585, Fall 2004 University of Massachusetts Amherst Andrew McCallum Today s Main Points
Statistical Machine Translation
Statistical Machine Translation -tree-based models (cont.)- Artem Sokolov Computerlinguistik Universität Heidelberg Sommersemester 2015 material from P. Koehn, S. Riezler, D. Altshuler Bottom-Up Decoding
Statistical Methods for NLP
Statistical Methods for NLP Stochastic Grammars Joakim Nivre Uppsala University Department of Linguistics and Philology joakim.nivre@lingfil.uu.se Statistical Methods for NLP 1(22) Structured Classification
Parsing with Context-Free Grammars
Parsing with Context-Free Grammars Berlin Chen 2005 References: 1. Natural Language Understanding, chapter 3 (3.1~3.4, 3.6) 2. Speech and Language Processing, chapters 9, 10 NLP-Berlin Chen 1 Grammars
Handout 8: Computation & Hierarchical parsing II. Compute initial state set S 0 Compute initial state set S 0
Massachusetts Institute of Technology 6.863J/9.611J, Natural Language Processing, Spring, 2001 Department of Electrical Engineering and Computer Science Department of Brain and Cognitive Sciences Handout
Parsing. Probabilistic CFG (PCFG) Laura Kallmeyer. Winter 2017/18. Heinrich-Heine-Universität Düsseldorf 1 / 22
Parsing Probabilistic CFG (PCFG) Laura Kallmeyer Heinrich-Heine-Universität Düsseldorf Winter 2017/18 1 / 22 Table of contents 1 Introduction 2 PCFG 3 Inside and outside probability 4 Parsing Jurafsky
In this chapter, we explore the parsing problem, which encompasses several questions, including:
Chapter 12 Parsing Algorithms 12.1 Introduction In this chapter, we explore the parsing problem, which encompasses several questions, including: Does L(G) contain w? What is the highest-weight derivation
Grammar formalisms Tree Adjoining Grammar: Formal Properties, Parsing. Part I. Formal Properties of TAG. Outline: Formal Properties of TAG
Grammar formalisms Tree Adjoining Grammar: Formal Properties, Parsing Laura Kallmeyer, Timm Lichte, Wolfgang Maier Universität Tübingen Part I Formal Properties of TAG 16.05.2007 und 21.05.2007 TAG Parsing
Context- Free Parsing with CKY. October 16, 2014
Context- Free Parsing with CKY October 16, 2014 Lecture Plan Parsing as Logical DeducBon Defining the CFG recognibon problem BoHom up vs. top down Quick review of Chomsky normal form The CKY algorithm
Probabilistic Context-Free Grammars. Michael Collins, Columbia University
Probabilistic Context-Free Grammars Michael Collins, Columbia University Overview Probabilistic Context-Free Grammars (PCFGs) The CKY Algorithm for parsing with PCFGs A Probabilistic Context-Free Grammar
Natural Language Processing : Probabilistic Context Free Grammars. Updated 5/09
Natural Language Processing : Probabilistic Context Free Grammars Updated 5/09 Motivation N-gram models and HMM Tagging only allowed us to process sentences linearly. However, even simple sentences require
Probabilistic Context Free Grammars. Many slides from Michael Collins and Chris Manning
Probabilistic Context Free Grammars Many slides from Michael Collins and Chris Manning Overview I Probabilistic Context-Free Grammars (PCFGs) I The CKY Algorithm for parsing with PCFGs A Probabilistic
Attendee information. Seven Lectures on Statistical Parsing. Phrase structure grammars = context-free grammars. Assessment.
even Lectures on tatistical Parsing Christopher Manning LA Linguistic Institute 7 LA Lecture Attendee information Please put on a piece of paper: ame: Affiliation: tatus (undergrad, grad, industry, prof,
Chapter 14 (Partially) Unsupervised Parsing
Chapter 14 (Partially) Unsupervised Parsing The linguistically-motivated tree transformations we discussed previously are very effective, but when we move to a new language, we may have to come up with
Multilevel Coarse-to-Fine PCFG Parsing
Multilevel Coarse-to-Fine PCFG Parsing Eugene Charniak, Mark Johnson, Micha Elsner, Joseph Austerweil, David Ellis, Isaac Haxton, Catherine Hill, Shrivaths Iyengar, Jeremy Moore, Michael Pozar, and Theresa
A Supertag-Context Model for Weakly-Supervised CCG Parser Learning
A Supertag-Context Model for Weakly-Supervised CCG Parser Learning Dan Garrette Chris Dyer Jason Baldridge Noah A. Smith U. Washington CMU UT-Austin CMU Contributions 1. A new generative model for learning
Introduction to Computational Linguistics
Introduction to Computational Linguistics Olga Zamaraeva (2018) Based on Bender (prev. years) University of Washington May 3, 2018 1 / 101 Midterm Project Milestone 2: due Friday Assgnments 4& 5 due dates
CISC4090: Theory of Computation
CISC4090: Theory of Computation Chapter 2 Context-Free Languages Courtesy of Prof. Arthur G. Werschulz Fordham University Department of Computer and Information Sciences Spring, 2014 Overview In Chapter
CMSC 723: Computational Linguistics I Session #5 Hidden Markov Models. The ischool University of Maryland. Wednesday, September 30, 2009
CMSC 723: Computational Linguistics I Session #5 Hidden Markov Models Jimmy Lin The ischool University of Maryland Wednesday, September 30, 2009 Today s Agenda The great leap forward in NLP Hidden Markov
Soft Inference and Posterior Marginals. September 19, 2013
Soft Inference and Posterior Marginals September 19, 2013 Soft vs. Hard Inference Hard inference Give me a single solution Viterbi algorithm Maximum spanning tree (Chu-Liu-Edmonds alg.) Soft inference
CS 6120/CS4120: Natural Language Processing
CS 6120/CS4120: Natural Language Processing Instructor: Prof. Lu Wang College of Computer and Information Science Northeastern University Webpage: www.ccs.neu.edu/home/luwang Assignment/report submission
Lecture 12: Algorithms for HMMs
Lecture 12: Algorithms for HMMs Nathan Schneider (some slides from Sharon Goldwater; thanks to Jonathan May for bug fixes) ENLP 17 October 2016 updated 9 September 2017 Recap: tagging POS tagging is a
Natural Language Processing. Lecture 13: More on CFG Parsing
Natural Language Processing Lecture 13: More on CFG Parsing Probabilistc/Weighted Parsing Example: ambiguous parse Probabilistc CFG Ambiguous parse w/probabilites 0.05 0.05 0.20 0.10 0.30 0.20 0.60 0.75
Lecture 12: Algorithms for HMMs
Lecture 12: Algorithms for HMMs Nathan Schneider (some slides from Sharon Goldwater; thanks to Jonathan May for bug fixes) ENLP 26 February 2018 Recap: tagging POS tagging is a sequence labelling task.
A* Search. 1 Dijkstra Shortest Path
A* Search Consider the eight puzzle. There are eight tiles numbered 1 through 8 on a 3 by three grid with nine locations so that one location is left empty. We can move by sliding a tile adjacent to the
Parsing. Left-Corner Parsing. Laura Kallmeyer. Winter 2017/18. Heinrich-Heine-Universität Düsseldorf 1 / 17
Parsing Left-Corner Parsing Laura Kallmeyer Heinrich-Heine-Universität Düsseldorf Winter 2017/18 1 / 17 Table of contents 1 Motivation 2 Algorithm 3 Look-ahead 4 Chart Parsing 2 / 17 Motivation Problems
Unit 2: Tree Models. CS 562: Empirical Methods in Natural Language Processing. Lectures 19-23: Context-Free Grammars and Parsing
CS 562: Empirical Methods in Natural Language Processing Unit 2: Tree Models Lectures 19-23: Context-Free Grammars and Parsing Oct-Nov 2009 Liang Huang (lhuang@isi.edu) Big Picture we have already covered...
Processing/Speech, NLP and the Web
CS460/626 : Natural Language Processing/Speech, NLP and the Web (Lecture 25 Probabilistic Parsing) Pushpak Bhattacharyya CSE Dept., IIT Bombay 14 th March, 2011 Bracketed Structure: Treebank Corpus [ S1[
Syntax-Based Decoding
Syntax-Based Decoding Philipp Koehn 9 November 2017 1 syntax-based models Synchronous Context Free Grammar Rules 2 Nonterminal rules NP DET 1 2 JJ 3 DET 1 JJ 3 2 Terminal rules N maison house NP la maison
Part of Speech Tagging: Viterbi, Forward, Backward, Forward- Backward, Baum-Welch. COMP-599 Oct 1, 2015
Part of Speech Tagging: Viterbi, Forward, Backward, Forward- Backward, Baum-Welch COMP-599 Oct 1, 2015 Announcements Research skills workshop today 3pm-4:30pm Schulich Library room 313 Start thinking about
Natural Language Processing
SFU NatLangLab Natural Language Processing Anoop Sarkar anoopsarkar.github.io/nlp-class Simon Fraser University September 27, 2018 0 Natural Language Processing Anoop Sarkar anoopsarkar.github.io/nlp-class
Midterm sample questions
Midterm sample questions CS 585, Brendan O Connor and David Belanger October 12, 2014 1 Topics on the midterm Language concepts Translation issues: word order, multiword translations Human evaluation Parts
Administrivia. Test I during class on 10 March. Bottom-Up Parsing. Lecture An Introductory Example
Administrivia Test I during class on 10 March. Bottom-Up Parsing Lecture 11-12 From slides by G. Necula & R. Bodik) 2/20/08 Prof. Hilfinger CS14 Lecture 11 1 2/20/08 Prof. Hilfinger CS14 Lecture 11 2 Bottom-Up
LING 473: Day 10. START THE RECORDING Coding for Probability Hidden Markov Models Formal Grammars
LING 473: Day 10 START THE RECORDING Coding for Probability Hidden Markov Models Formal Grammars 1 Issues with Projects 1. *.sh files must have #!/bin/sh at the top (to run on Condor) 2. If run.sh is supposed
CSCI 5832 Natural Language Processing. Today 2/19. Statistical Sequence Classification. Lecture 9
CSCI 5832 Natural Language Processing Jim Martin Lecture 9 1 Today 2/19 Review HMMs for POS tagging Entropy intuition Statistical Sequence classifiers HMMs MaxEnt MEMMs 2 Statistical Sequence Classification
Probabilistic Context Free Grammars. Many slides from Michael Collins
Probabilistic Context Free Grammars Many slides from Michael Collins Overview I Probabilistic Context-Free Grammars (PCFGs) I The CKY Algorithm for parsing with PCFGs A Probabilistic Context-Free Grammar
Lecture 3: ASR: HMMs, Forward, Viterbi
Original slides by Dan Jurafsky CS 224S / LINGUIST 285 Spoken Language Processing Andrew Maas Stanford University Spring 2017 Lecture 3: ASR: HMMs, Forward, Viterbi Fun informative read on phonetics The
Empirical Methods in Natural Language Processing Lecture 11 Part-of-speech tagging and HMMs
Empirical Methods in Natural Language Processing Lecture 11 Part-of-speech tagging and HMMs (based on slides by Sharon Goldwater and Philipp Koehn) 21 February 2018 Nathan Schneider ENLP Lecture 11 21
Hidden Markov Models
CS769 Spring 2010 Advanced Natural Language Processing Hidden Markov Models Lecturer: Xiaojin Zhu jerryzhu@cs.wisc.edu 1 Part-of-Speech Tagging The goal of Part-of-Speech (POS) tagging is to label each
CS460/626 : Natural Language
CS460/626 : Natural Language Processing/Speech, NLP and the Web (Lecture 23, 24 Parsing Algorithms; Parsing in case of Ambiguity; Probabilistic Parsing) Pushpak Bhattacharyya CSE Dept., IIT Bombay 8 th,
DT2118 Speech and Speaker Recognition
DT2118 Speech and Speaker Recognition Language Modelling Giampiero Salvi KTH/CSC/TMH giampi@kth.se VT 2015 1 / 56 Outline Introduction Formal Language Theory Stochastic Language Models (SLM) N-gram Language
Natural Language Processing 1. lecture 7: constituent parsing. Ivan Titov. Institute for Logic, Language and Computation
atural Language Processing 1 lecture 7: constituent parsing Ivan Titov Institute for Logic, Language and Computation Outline Syntax: intro, CFGs, PCFGs PCFGs: Estimation CFGs: Parsing PCFGs: Parsing Parsing
CS838-1 Advanced NLP: Hidden Markov Models
CS838-1 Advanced NLP: Hidden Markov Models Xiaojin Zhu 2007 Send comments to jerryzhu@cs.wisc.edu 1 Part of Speech Tagging Tag each word in a sentence with its part-of-speech, e.g., The/AT representative/nn
CS 301. Lecture 18 Decidable languages. Stephen Checkoway. April 2, 2018
CS 301 Lecture 18 Decidable languages Stephen Checkoway April 2, 2018 1 / 26 Decidable language Recall, a language A is decidable if there is some TM M that 1 recognizes A (i.e., L(M) = A), and 2 halts
Probabilistic Context Free Grammars
1 Defining PCFGs A PCFG G consists of Probabilistic Context Free Grammars 1. A set of terminals: {w k }, k = 1..., V 2. A set of non terminals: { i }, i = 1..., n 3. A designated Start symbol: 1 4. A set
{Probabilistic Stochastic} Context-Free Grammars (PCFGs)
{Probabilistic Stochastic} Context-Free Grammars (PCFGs) 116 The velocity of the seismic waves rises to... S NP sg VP sg DT NN PP risesto... The velocity IN NP pl of the seismic waves 117 PCFGs APCFGGconsists
Decoding and Inference with Syntactic Translation Models
Decoding and Inference with Syntactic Translation Models March 5, 2013 CFGs S NP VP VP NP V V NP NP CFGs S NP VP S VP NP V V NP NP CFGs S NP VP S VP NP V NP VP V NP NP CFGs S NP VP S VP NP V NP VP V NP
Grammars and introduction to machine learning. Computers Playing Jeopardy! Course Stony Brook University
Grammars and introduction to machine learning Computers Playing Jeopardy! Course Stony Brook University Last class: grammars and parsing in Prolog Noun -> roller Verb thrills VP Verb NP S NP VP NP S VP
Roger Levy Probabilistic Models in the Study of Language draft, October 2,
Roger Levy Probabilistic Models in the Study of Language draft, October 2, 2012 224 Chapter 10 Probabilistic Grammars 10.1 Outline HMMs PCFGs ptsgs and ptags Highlight: Zuidema et al., 2008, CogSci; Cohn
CS : Speech, NLP and the Web/Topics in AI
CS626-449: Speech, NLP and the Web/Topics in AI Pushpak Bhattacharyya CSE Dept., IIT Bombay Lecture-17: Probabilistic parsing; insideoutside probabilities Probability of a parse tree (cont.) S 1,l NP 1,2
Lecture VII Part 2: Syntactic Analysis Bottom-up Parsing: LR Parsing. Prof. Bodik CS Berkley University 1
Lecture VII Part 2: Syntactic Analysis Bottom-up Parsing: LR Parsing. Prof. Bodik CS 164 -- Berkley University 1 Bottom-Up Parsing Bottom-up parsing is more general than topdown parsing And just as efficient
Lecture 13: Structured Prediction
Lecture 13: Structured Prediction Kai-Wei Chang CS @ University of Virginia kw@kwchang.net Couse webpage: http://kwchang.net/teaching/nlp16 CS6501: NLP 1 Quiz 2 v Lectures 9-13 v Lecture 12: before page
Sequence Labeling: HMMs & Structured Perceptron
Sequence Labeling: HMMs & Structured Perceptron CMSC 723 / LING 723 / INST 725 MARINE CARPUAT marine@cs.umd.edu HMM: Formal Specification Q: a finite set of N states Q = {q 0, q 1, q 2, q 3, } N N Transition
CSE 105 THEORY OF COMPUTATION
CSE 105 THEORY OF COMPUTATION Spring 2016 http://cseweb.ucsd.edu/classes/sp16/cse105-ab/ Today's learning goals Sipser Ch 2 Define push down automata Trace the computation of a push down automaton Design
Everything You Always Wanted to Know About Parsing
Everything You Always Wanted to Know About Parsing Part V : LR Parsing University of Padua, Italy ESSLLI, August 2013 Introduction Parsing strategies classified by the time the associated PDA commits to
Parsing -3. A View During TD Parsing
Parsing -3 Deterministic table-driven parsing techniques Pictorial view of TD and BU parsing BU (shift-reduce) Parsing Handle, viable prefix, items, closures, goto s LR(k): SLR(1), LR(1) Problems with
Context Free Grammars
Automata and Formal Languages Context Free Grammars Sipser pages 101-111 Lecture 11 Tim Sheard 1 Formal Languages 1. Context free languages provide a convenient notation for recursive description of languages.
Spectral Unsupervised Parsing with Additive Tree Metrics
Spectral Unsupervised Parsing with Additive Tree Metrics Ankur Parikh, Shay Cohen, Eric P. Xing Carnegie Mellon, University of Edinburgh Ankur Parikh 2014 1 Overview Model: We present a novel approach
Computers playing Jeopardy!
Computers playing Jeopardy! CSE 392, Computers Playing Jeopardy!, Fall 2011 Stony Brook University http://www.cs.stonybrook.edu/~cse392 1 Last class: grammars and parsing in Prolog Verb thrills VP Verb
Aspects of Tree-Based Statistical Machine Translation
Aspects of Tree-Based Statistical Machine Translation Marcello Federico Human Language Technology FBK 2014 Outline Tree-based translation models: Synchronous context free grammars Hierarchical phrase-based
Algorithms for Syntax-Aware Statistical Machine Translation
Algorithms for Syntax-Aware Statistical Machine Translation I. Dan Melamed, Wei Wang and Ben Wellington ew York University Syntax-Aware Statistical MT Statistical involves machine learning (ML) seems crucial
Topics in Natural Language Processing
Topics in Natural Language Processing Shay Cohen Institute for Language, Cognition and Computation University of Edinburgh Lecture 5 Solving an NLP Problem When modelling a new problem in NLP, need to
Natural Language Processing Prof. Pushpak Bhattacharyya Department of Computer Science & Engineering, Indian Institute of Technology, Bombay
Natural Language Processing Prof. Pushpak Bhattacharyya Department of Computer Science & Engineering, Indian Institute of Technology, Bombay Lecture - 21 HMM, Forward and Backward Algorithms, Baum Welch
S NP VP 0.9 S VP 0.1 VP V NP 0.5 VP V 0.1 VP V PP 0.1 NP NP NP 0.1 NP NP PP 0.2 NP N 0.7 PP P NP 1.0 VP NP PP 1.0. N people 0.
/6/7 CS 6/CS: Natural Language Processing Instructor: Prof. Lu Wang College of Computer and Information Science Northeastern University Webpage: www.ccs.neu.edu/home/luwang The grammar: Binary, no epsilons,.9..5
Parsing. Unger s Parser. Laura Kallmeyer. Winter 2016/17. Heinrich-Heine-Universität Düsseldorf 1 / 21
Parsing Unger s Parser Laura Kallmeyer Heinrich-Heine-Universität Düsseldorf Winter 2016/17 1 / 21 Table of contents 1 Introduction 2 The Parser 3 An Example 4 Optimizations 5 Conclusion 2 / 21 Introduction
Lecture 15. Probabilistic Models on Graph
Lecture 15. Probabilistic Models on Graph Prof. Alan Yuille Spring 2014 1 Introduction We discuss how to define probabilistic models that use richly structured probability distributions and describe how
Data-Intensive Computing with MapReduce
Data-Intensive Computing with MapReduce Session 8: Sequence Labeling Jimmy Lin University of Maryland Thursday, March 14, 2013 This work is licensed under a Creative Commons Attribution-Noncommercial-Share
Maschinelle Sprachverarbeitung
Maschinelle Sprachverarbeitung Parsing with Probabilistic Context-Free Grammar Ulf Leser Content of this Lecture Phrase-Structure Parse Trees Probabilistic Context-Free Grammars Parsing with PCFG Other
A Context-Free Grammar
Statistical Parsing A Context-Free Grammar S VP VP Vi VP Vt VP VP PP DT NN PP PP P Vi sleeps Vt saw NN man NN dog NN telescope DT the IN with IN in Ambiguity A sentence of reasonable length can easily
Final exam study sheet for CS3719 Turing machines and decidability.
Final exam study sheet for CS3719 Turing machines and decidability. A Turing machine is a finite automaton with an infinite memory (tape). Formally, a Turing machine is a 6-tuple M = (Q, Σ, Γ, δ, q 0,
Maschinelle Sprachverarbeitung
Maschinelle Sprachverarbeitung Parsing with Probabilistic Context-Free Grammar Ulf Leser Content of this Lecture Phrase-Structure Parse Trees Probabilistic Context-Free Grammars Parsing with PCFG Other
Probabilistic Context-Free Grammar
Probabilistic Context-Free Grammar Petr Horáček, Eva Zámečníková and Ivana Burgetová Department of Information Systems Faculty of Information Technology Brno University of Technology Božetěchova 2, 612
CS460/626 : Natural Language
CS460/626 : Natural Language Processing/Speech, NLP and the Web (Lecture 27 SMT Assignment; HMM recap; Probabilistic Parsing cntd) Pushpak Bhattacharyya CSE Dept., IIT Bombay 17 th March, 2011 CMU Pronunciation
CMSC 330: Organization of Programming Languages. Pushdown Automata Parsing
CMSC 330: Organization of Programming Languages Pushdown Automata Parsing Chomsky Hierarchy Categorization of various languages and grammars Each is strictly more restrictive than the previous First described
Automata & languages. A primer on the Theory of Computation. Laurent Vanbever. ETH Zürich (D-ITET) October,
Automata & languages A primer on the Theory of Computation Laurent Vanbever www.vanbever.eu ETH Zürich (D-ITET) October, 5 2017 Part 3 out of 5 Last week, we learned about closure and equivalence of regular
Part 3 out of 5. Automata & languages. A primer on the Theory of Computation. Last week, we learned about closure and equivalence of regular languages
Automata & languages A primer on the Theory of Computation Laurent Vanbever www.vanbever.eu Part 3 out of 5 ETH Zürich (D-ITET) October, 5 2017 Last week, we learned about closure and equivalence of regular
Hidden Markov Models
CS 2750: Machine Learning Hidden Markov Models Prof. Adriana Kovashka University of Pittsburgh March 21, 2016 All slides are from Ray Mooney Motivating Example: Part Of Speech Tagging Annotate each word
CS 545 Lecture XVI: Parsing
CS 545 Lecture XVI: Parsing brownies_choco81@yahoo.com brownies_choco81@yahoo.com Benjamin Snyder Parsing Given a grammar G and a sentence x = (x1, x2,..., xn), find the best parse tree. We re not going
Part A. P (w 1 )P (w 2 w 1 )P (w 3 w 1 w 2 ) P (w M w 1 w 2 w M 1 ) P (w 1 )P (w 2 w 1 )P (w 3 w 2 ) P (w M w M 1 )
Part A 1. A Markov chain is a discrete-time stochastic process, defined by a set of states, a set of transition probabilities (between states), and a set of initial state probabilities; the process proceeds
Introduction to Bottom-Up Parsing
Introduction to Bottom-Up Parsing Outline Review LL parsing Shift-reduce parsing The LR parsing algorithm Constructing LR parsing tables 2 Top-Down Parsing: Review Top-down parsing expands a parse tree
INF4820: Algorithms for Artificial Intelligence and Natural Language Processing. Hidden Markov Models
INF4820: Algorithms for Artificial Intelligence and Natural Language Processing Hidden Markov Models Murhaf Fares & Stephan Oepen Language Technology Group (LTG) October 27, 2016 Recap: Probabilistic Language
Parsing Algorithms. CS 4447/CS Stephen Watt University of Western Ontario
Parsing Algorithms CS 4447/CS 9545 -- Stephen Watt University of Western Ontario The Big Picture Develop parsers based on grammars Figure out properties of the grammars Make tables that drive parsing engines
Parsing VI LR(1) Parsers
Parsing VI LR(1) Parsers N.B.: This lecture uses a left-recursive version of the SheepNoise grammar. The book uses a rightrecursive version. The derivations (& the tables) are different. Copyright 2005,
INF4820: Algorithms for Artificial Intelligence and Natural Language Processing. Language Models & Hidden Markov Models
1 University of Oslo : Department of Informatics INF4820: Algorithms for Artificial Intelligence and Natural Language Processing Language Models & Hidden Markov Models Stephan Oepen & Erik Velldal Language
This kind of reordering is beyond the power of finite transducers, but a synchronous CFG can do this.
Chapter 12 Synchronous CFGs Synchronous context-free grammars are a generalization of CFGs that generate pairs of related strings instead of single strings. They are useful in many situations where one
Introduction to Bottom-Up Parsing
Introduction to Bottom-Up Parsing Outline Review LL parsing Shift-reduce parsing The LR parsing algorithm Constructing LR parsing tables Compiler Design 1 (2011) 2 Top-Down Parsing: Review Top-down parsing
NPDA, CFG equivalence
NPDA, CFG equivalence Theorem A language L is recognized by a NPDA iff L is described by a CFG. Must prove two directions: ( ) L is recognized by a NPDA implies L is described by a CFG. ( ) L is described