Natural Language Processing. Lecture 13: More on CFG Parsing

Transcription

1 Natural Language Processing Lecture 13: More on CFG Parsing

2 Probabilistc/Weighted Parsing

3 Example: ambiguous parse

4 Probabilistc CFG

5 Ambiguous parse w/probabilites P(lef) = 2.2 *10^-6 P(right) = 6.1 *10^-7

6 Review: Context-Free Grammars Vocabulary of terminal symbols, Σ Set of nonterminal symbols (a.k.a. variables), N Special start symbol S N Producton rules of the form X α where X N α (N Σ)* (in CNF: α N Σ)

7 Probabilistc Context-Free Grammars Vocabulary of terminal symbols, Σ Set of nonterminal symbols (a.k.a. variables), N Special start symbol S N Producton rules of the form X α, each with a positve weight p(x α), where X N α (N Σ)* (in CNF: α N 2 Σ) X N, α p(x α) = 1

8 CKY Algorithm: Review for i = 1... n C[i-1, i] = { V V w i } for l = 2... n // width for i = 0... n - l // lef boundary k = i + l // right boundary for j = i k 1 // midpoint C[i, k] = C[i, k] { V V YZ, Y C[i, j], Z C[j, k] } return true if S C[0, n]

9 Weighted CKY Algorithm for i = 1... n, V N C[V, i-1, i] = p(v w i ) for l = 2... n // width of span for i = 0... n - l // lef boundary k = i + l // right boundary for j = i k 1 // midpoint for each binary rule V YZ C[V, i, k] = max{ C[V, i, k], C[Y, i, j] C[Z, j, k] p(v YZ) } return true if S C[,0, n]

10 CKY Algorithm: Review

11 Weighted CKY Algorithm

12 P-CKY algorithm from book

13

14 Parsing as (Weighted) Deducton

15 Earley s Algorithm

16

17 Example Grammar (same for CKY)

18 Earley Parsing Allows arbitrary CFGs Top-down control Fills a table (or chart) in a single sweep over the input Table is length N+1; N is number of words Table entries represent Completed consttuents and their locatons In-progress consttuents Predicted consttuents 02/26/2019 Speech and Language Processing - Jurafsky and Martn 18

19 States The table-entries are called states and are represented with doted-rules. S. VP NP Det. Nominal VP V NP. A VP is predicted An NP is in progress A VP has been found 02/26/2019 Speech and Language Processing - Jurafsky and Martn 19

20 States/Locatons S. VP [0,0] NP Det.Nominal [1,2] VP V NP. [0,3] A VP is predicted at the start of the sentence An NP is in progress; the Det goes from 1 to 2 A VP has been found startng at 0 and ending at 3 02/26/2019 Speech and Language Processing - Jurafsky and Martn 20

21 Earley top-level As with most dynamic programming approaches, the answer is found by looking in the table in the right place. In this case, there should be an S state in the final column that spans from 0 to N and is complete. That is, S α. [0,N] If that s the case, you re done. 02/26/2019 Speech and Language Processing - Jurafsky and Martn 21

22 Earley top-level (2) So sweep through the table from 0 to N New predicted states are created by startng topdown from S New incomplete states are created by advancing existng states as new consttuents are discovered New complete states are created in the same way. 02/26/2019 Speech and Language Processing - Jurafsky and Martn 22

23 Earley top-level (3) More specifically 1. Predict all the states you can upfront 2. Read a word 1. Extend states based on matches 2. Generate new predictons 3. Go to step 2 3. When you re out of words, look at the chart to see if you have a winner 02/26/2019 Speech and Language Processing - Jurafsky and Martn 23

24 Earley code: top-level

25 Earley code: 3 main functons

26 Extended Earley Example Book that fight We should find: an S from 0 to 3 that is a completed state 02/26/2019 Speech and Language Processing - Jurafsky and Martn 26

27

28

29

30

31

32 Earley s Algorithm in equatons We can look at this from the declaratve programming point of view too. ROOT S [0,0] goal: ROOT S [0,n] book the fight through Chicago

33 Earley s Algorithm: PREDICT Given V α Xβ [i, j] and the rule X γ, create X γ [j, j] ROOT S [0,0] S VP S VP [0,0] ROOT S [0,0] S VP [0,0] S NP VP [0,0]... VP V NP [0,0]... NP DT N [0,0]... book the fight through Chicago

34 Earley s Algorithm: SCAN Given V α Tβ [i, j] and the rule T w j+1, create T w j+1 [j, j+1] VP V NP [0,0] V book V book [0,1] ROOT S [0,0] S VP [0,0] S NP VP [0,0]... VP V NP [0,0]... NP DT N [0,0]... V book [0, 1] book the fight through Chicago

35 Earley s Algorithm: COMPLETE Given V α Xβ [i, j] and X γ [j, k], create V αx β [i, k] VP V NP [0,0] V book [0,1] VP V NP [0,1] ROOT S [0,0] S VP [0,0] S NP VP [0,0]... VP V NP [0,0]... NP DT N [0,0]... V book [0, 1] VP V NP [0,1] book the fight through Chicago

36

37 Thought Questons Runtme? O(n 3 ) Memory? O(n 2 ) Can we make it faster? Recovering trees?

38 Make it an Earley Parser

39 Parsing as Search, Again

40 Implementng Recognizers as Search Agenda = { state0 } while(agenda not empty) s = pop a state from Agenda if s is a success-state return s // valid parse tree else if s is not a failure-state: generate new states from s push new states onto Agenda return nil // no parse!

41 Agenda-Based Probabilistc Parsing Agenda = { (item, value) : inital updates from equatons } // items take the form [X, i, j]; values are reals while(agenda not empty) u = pop an update from Agenda if u.item is goal return u.value // valid parse tree else if u.value -> Chart[u.item] store Chart[u.item] u.value if u.item combines with other Chart items: generate new updates from u and items stored in Chart push new updates onto Agenda return nil // no parse!

42 Catalog of CF Parsing Algorithms Recogniton/Boolean vs. parsing/probabilistc Chomsky normal form/cky vs. general/earley s Exhaustve vs. agenda