Context- Free Parsing with CKY. October 16, 2014
|
|
- Melanie Watkins
- 6 years ago
- Views:
Transcription
1 Context- Free Parsing with CKY October 16, 2014
2 Lecture Plan Parsing as Logical DeducBon Defining the CFG recognibon problem BoHom up vs. top down Quick review of Chomsky normal form The CKY algorithm CKY as dynamic programming Complexity analysis
3 Algorithms for CFGs Given a CFG G and a string s: RecogniBon: Is s L(G)? Equivalently, find some derivabon that proves s is in G's language. Parsing: What are (all of) G's derivabons of s? What is the correct derivabon of s under G? The same core algorithms actually provide solubons to all of these!
4 Important DisBncBon DeterminisBc grammars give much faster recognibon and parsing algorithms! LR languages are parseable in linear Bme; you may have learned about these in a course on programming languages or compilers. For us, the best known algorithms are O( N n ), but these are not pracbcal. Today you will learn an O( N 3 n 3 ) solubon.
5 Parsing Strategies
6 Parsing as Search S top- down bohom- up? x1 x2... xn Trees break into pieces (parbal trees), which can be used to define a search space.
7 Top- Down Parsing (Recursive Descent) (S) s = Book that flight
8 Top- Down Parsing (Recursive Descent) (S) (S (NP) (VP)) (S Aux (NP) (VP)) (S (VP)) x = Book that flight
9 Top- Down Parsing (Recursive Descent) (S) (S (NP) (VP)) (S Aux (NP) (VP)) (S (VP)) (S (NP Pronoun) (VP)) (S (NP ProperNoun) (VP)) (S (NP Det Nominal) (VP)) x = Book that flight
10 Top- Down Parsing (Recursive Descent) (S) (S (NP) (VP)) (S Aux (NP) (VP)) (S (VP)) (S (NP Pronoun) (VP)) (S (NP ProperNoun) (VP)) (S (NP Det Nominal) (VP)) (S Aux (NP Pronoun) (VP)) (S Aux (NP ProperNoun) (VP)) (S Aux (NP Det Nominal) (VP)) x = Book that flight
11 Top- Down Parsing (Recursive Descent) (S) (S (NP) (VP)) (S Aux (NP) (VP)) (S (VP)) (S (NP Pronoun) (VP)) (S (NP ProperNoun) (VP)) (S (NP Det Nominal) (VP)) (S Aux (NP Pronoun) (VP)) (S Aux (NP ProperNoun) (VP)) (S Aux (NP Det Nominal) (VP)) (S (VP (VP) (PP))) (S (VP Verb)) (S (VP Verb (NP))) (S (VP Verb (NP) (PP))) (S (VP Verb (PP))) x = Book that flight
12 Top- Down Parsing (Recursive Descent) Never wastes Bme exploring ungrammabcal trees! Inefficiency: most search states (parbal trees) could never lead to a derivabon of our sentence. One other problem...
13 Lef Recursion! (S) (S (VP)) (S (VP (VP) (PP))) (S (VP (VP (VP) (PP)) (PP))) (S (VP (VP (VP (VP) (PP)) (PP)) (PP))) (S (VP (VP (VP (VP (VP) (PP)) (PP)) (PP)) (PP))) (S (VP (VP (VP (VP (VP (VP) (PP)) (PP)) (PP)) (PP)) (PP))) x = Book that flight
14 Top- Down RecogniBon Don't need to store the tree! Can collapse states that funcbon the same way. Store unexpanded nonterminals (in sequence) only, along with the number of words covered so far. Start state: (S, 0) Scan: If you're at (xj+1 β, j), you can get to (β, j + 1). Predict: If you're at (Zβ, j) and Z γ, you can get to (γβ, j). Final ( success ) state: (ε, n) This should remind you of genera-ng from a CFG.
15 BoHom- Up Parsing book that flight
16 BoHom- Up Parsing (Verb book) (Det that) (Noun flight) (Noun book) (Det that) (Noun flight) book that flight
17 BoHom- Up Parsing (Nominal (Noun book)) (Det that) (Nominal (Noun flight)) (Verb book) (Det that) (Noun flight) (Noun book) (Det that) (Noun flight) book that flight
18 BoHom- Up Parsing (Verb book) (Det that) (Nominal (Noun flight)) (Nominal (Noun book)) (Det that) (Nominal (Noun flight)) (Verb book) (Det that) (Noun flight) (Noun book) (Det that) (Noun flight) book that flight
19 BoHom- Up Parsing Nominal (Noun book)) (NP (Det that) (Nominal (Noun flight))) (Verb book) (Det that) (Nominal (Noun flight)) (Nominal (Noun book)) (Det that) (Nominal (Noun flight)) (Verb book) (Det that) (Noun flight) (Noun book) (Det that) (Noun flight) book that flight
20 BoHom- Up Parsing Never generates trees that are inconsistent with the sentence. Generates parbal trees that have no hope of geong to S.
21 Shif- Reduce (A BoHom- Up Method) Don't need to store the tree! Can collapse states that funcbon the same way. Store only the nodes that are parentless. Start state: (ε, 0) Shi/: If you're at (α, j), you can get to (αxj+1, j + 1). Reduce: If you're at (αγ, j) and Z γ, you can get to (αz, j). Final ( success ) state: (S, n)
22 ImplemenBng Recognizers as Search Agenda = { state0 } while(agenda not empty) s = pop a state from Agenda if s is a success- state return s // valid parse tree else if s is not a failure- state: generate new states from s push new states onto Agenda return nil // no parse!
23 Parsing as Search Basically, you need a strategy for deciding what to pursue next (how to priori-ze the agenda). Examples: depth- first, breadth- first, bohom- up, top- down, lef- to- right, right- to- lef,... The problems: Memory! Ambiguity leads to lots of bad paths In a depth- first model, this results in lots of wasted effort in backtracking. One solubon: greedy parsing. Only works if you have a really smart agenda!
24 Backtracking
25 Ambiguity Redux A sentence may have many parses. Even if a sentence has only one parse, finding it may be difficult, because there are many misleading paths you could follow. BoHom- up: fragments that can never have a home in any S Top- down: fragments that never get you to x What to do when there are many parses... how to choose? Return them all?
26 So Far... Two ways to think of parsing as search Problems: Lef- recursion (in top- down parsers) Efficiency (backtracking, memory) Ambiguity
27 CKY Kasami, 1965 Younger, 1967 Cocke and Schwartz, 1970
28 Chomsky Normal Form G = (Σ, N, S, R) Σ : Vocabulary of terminal symbols N: set of nonterminal symbols S N: special start symbol R: ProducBon rules of the form X α where X N (a nonterminal symbol) and α N 2 Σ
29 CKY Schema Data structure: chart. In each cell, store the possible phrase types. Visit each cell once. Start with narrow spans, progress to longer ones. book the flight through Chicago
30 CKY Example boo k the flight through Chicago
31 CKY Schema book Noun Verb VP Nominal S S, VP, X2 S, VP Det NP NP the Nominal Noun Nominal flight Prep PP through ProperN NP Chicago
32 CKY Algorithm (RecogniBon) input: x, G (in CNF) output: true iff x is in L(G) initialize all cells of C to for i = 1... n C[i-1, i] = {A A xi} for l = 2... n for i = 0... n - l k = i + l for j = i k 1 for each X YZ s.t. Y C[i, j] & Z C[j, k] C[i, k] = C[i, k] {X} return true if S C[0, n]
33 Visualizing CKY X 1 5 Y 1 5 Z n
34 Visualizing CKY How do we fill in C(0,2)? n
35 Visualizing CKY How do we fill in C(0,2)? Put together C(0,1) and C(1,2) n
36 Visualizing CKY How do we fill in C(0,3)? n
37 Visualizing CKY How do we fill in C(0,3)? One way n
38 Visualizing CKY How do we fill in C(0,3)? One way Another way n
39 Visualizing CKY How do we fill in C(0,n)? n
40 Visualizing CKY How do we fill in C(0,n)? n - 1 ways! n
41 Ordering in CKY n
42 CKY Algorithm for CF RecogniBon input: x, G (in CNF) output: true iff x is in L(G) initialize all cells of C to for i = 1... n C[i-1, i] = {A A xi} for l = 2... n for i = 0... n - l k = i + l for j = i k 1 for each X YZ s.t. Y C[i, j] & Z C[j, k] C[i, k] = C[i, k] {X} return true if S C[0, n]
43 CKY Algorithm input: x, G (in CNF) output: true iff x is in L(G) initialize all cells of C to for i = 1... n for each A s.t. A xi C[A, i-1, i] = 1 for l = 2... n for i = 0... n - l k = i + l for j = i k 1 for each X YZ C[X, i, k] = C[X, i, k] (C[Y, i, j] C[Z, j, k]) return C[S, 0, n]
44 CKY OperaBons given... for every add to - A x i A C[i - 1, i] Y C[i, j] Z C[j, k] X Y Z X C[i, k]
45 Visualizing CKY Operations Y Z O(n) B x i B O(n 3 ) X Y Z k X j j i i-1 i k i size of N nonterminal start end length of sentence length of sentence
46 CKY Algorithm AsymptoBcs Space: O(n 2 ) RunBme: O(n 3 ) Both also depend on grammar size
47 Parsing as Logical DeducBon
48 (Weighted) Logic Programming Parsing algorithms can be expressed as (weighted) logic programs I 1 I 2 I k I Shieber, Schabes, & Pereira (1995). Principles and ImplementaBon of DeducBve Parsing. J. Logic Programming
49 (Weighted) Logic Programming Parsing algorithms can be expressed as (weighted) logic programs I 1 I 2 I k I I 1 : w 1 I 2 : w 2 I k : w k I : w N k i=1 w i w, Shieber, Schabes, & Pereira (1995). Principles and ImplementaBon of DeducBve Parsing. J. Logic Programming
50 Understanding Logic Programs Antecedents I 1 I 2 I k I Consequent Side condibons
51 Understanding Logic Programs Items I Axioms A I Inference rules Goals G I I 1 I 2 I k I
52 Item form [N,N 1, N 1 ] CKY Algorithm
53 CKY Algorithm Item form [N,N 1, N 1 ] Goals [S, 1, x + 1]
54 CKY Algorithm Item form [N,N 1, N 1 ] Goals [S, 1, x + 1] Axioms [X, i, i + 1] (X! x i ) 2 N
55 CKY Algorithm Item form [N,N 1, N 1 ] Goals [S, 1, x + 1] Axioms [X, i, i + 1] (X! x i ) 2 N Inference rules [X, i, k] [Z, i, j] [Y,k,j] (Z! XY) 2 R
56 Top- Down Parsing Item form [(N [ ), N] Goals [", x ] Axioms [S, 0] Inference rules [x j+1,j] [,j + 1] [Z,j] [,j] (Z! ) 2 R Scan Predict
57 CKY Algorithm: Discussion Top- down or bohom- up? Parsing as deducbon: visual notabon Horn clauses, used to prove x L(G).
58 Dynamic Programming Examples: WFSA best path, Viterbi, Earley, CKY Solve sub- problems, store results in a table. Reuse work you ve done already (runbme) Pack sub- problems efficiently (memory) For parsing, many sub- parses can be used repeatedly in many larger sub- parses! Don t build the same sub- parse more than once.
59 DP ExecuBon Model Clever ordering to ensure items are built exactly once Viterbi: lef to right CKY: narrow to wide MemoizaBon Agenda DP logic execubon model
60 ImplemenBng Recognizers as Search Agenda = { state0 } while(agenda not empty) s = pop a state from Agenda if s is a success-state return s // valid parse tree else if s is not a failure-state: generate new states from s push new states onto Agenda return nil // no parse!
61 CKY as Search Agenda = [{ [A, i 1, i] A x i } while(agenda not empty) [N, i, j] = pop from Agenda if [N, i, j] = [S, 0, n] return true else: for each visited [Y, j, k] and X N Y push [X, i, k] onto Agenda for each visited [Z, k, i] and X Z N push [X, k, i] onto Agenda return false
62 What Else? So far, this only does recogni-on. Can we recover parse trees? (Need to store some more informabon.) All trees Best tree
Natural Language Processing. Lecture 13: More on CFG Parsing
Natural Language Processing Lecture 13: More on CFG Parsing Probabilistc/Weighted Parsing Example: ambiguous parse Probabilistc CFG Ambiguous parse w/probabilites 0.05 0.05 0.20 0.10 0.30 0.20 0.60 0.75
More informationCKY & Earley Parsing. Ling 571 Deep Processing Techniques for NLP January 13, 2016
CKY & Earley Parsing Ling 571 Deep Processing Techniques for NLP January 13, 2016 No Class Monday: Martin Luther King Jr. Day CKY Parsing: Finish the parse Recognizer à Parser Roadmap Earley parsing Motivation:
More informationParsing with CFGs L445 / L545 / B659. Dept. of Linguistics, Indiana University Spring Parsing with CFGs. Direction of processing
L445 / L545 / B659 Dept. of Linguistics, Indiana University Spring 2016 1 / 46 : Overview Input: a string Output: a (single) parse tree A useful step in the process of obtaining meaning We can view the
More informationParsing with CFGs. Direction of processing. Top-down. Bottom-up. Left-corner parsing. Chart parsing CYK. Earley 1 / 46.
: Overview L545 Dept. of Linguistics, Indiana University Spring 2013 Input: a string Output: a (single) parse tree A useful step in the process of obtaining meaning We can view the problem as searching
More informationReview. Earley Algorithm Chapter Left Recursion. Left-Recursion. Rule Ordering. Rule Ordering
Review Earley Algorithm Chapter 13.4 Lecture #9 October 2009 Top-Down vs. Bottom-Up Parsers Both generate too many useless trees Combine the two to avoid over-generation: Top-Down Parsing with Bottom-Up
More informationParsing with Context-Free Grammars
Parsing with Context-Free Grammars Berlin Chen 2005 References: 1. Natural Language Understanding, chapter 3 (3.1~3.4, 3.6) 2. Speech and Language Processing, chapters 9, 10 NLP-Berlin Chen 1 Grammars
More informationIn this chapter, we explore the parsing problem, which encompasses several questions, including:
Chapter 12 Parsing Algorithms 12.1 Introduction In this chapter, we explore the parsing problem, which encompasses several questions, including: Does L(G) contain w? What is the highest-weight derivation
More informationParsing with Context-Free Grammars
Parsing with Context-Free Grammars CS 585, Fall 2017 Introduction to Natural Language Processing http://people.cs.umass.edu/~brenocon/inlp2017 Brendan O Connor College of Information and Computer Sciences
More informationIntroduction to Computational Linguistics
Introduction to Computational Linguistics Olga Zamaraeva (2018) Based on Bender (prev. years) University of Washington May 3, 2018 1 / 101 Midterm Project Milestone 2: due Friday Assgnments 4& 5 due dates
More informationCMPT-825 Natural Language Processing. Why are parsing algorithms important?
CMPT-825 Natural Language Processing Anoop Sarkar http://www.cs.sfu.ca/ anoop October 26, 2010 1/34 Why are parsing algorithms important? A linguistic theory is implemented in a formal system to generate
More informationNatural Language Processing CS Lecture 06. Razvan C. Bunescu School of Electrical Engineering and Computer Science
Natural Language Processing CS 6840 Lecture 06 Razvan C. Bunescu School of Electrical Engineering and Computer Science bunescu@ohio.edu Statistical Parsing Define a probabilistic model of syntax P(T S):
More informationHandout 8: Computation & Hierarchical parsing II. Compute initial state set S 0 Compute initial state set S 0
Massachusetts Institute of Technology 6.863J/9.611J, Natural Language Processing, Spring, 2001 Department of Electrical Engineering and Computer Science Department of Brain and Cognitive Sciences Handout
More informationParsing. Based on presentations from Chris Manning s course on Statistical Parsing (Stanford)
Parsing Based on presentations from Chris Manning s course on Statistical Parsing (Stanford) S N VP V NP D N John hit the ball Levels of analysis Level Morphology/Lexical POS (morpho-synactic), WSD Elements
More informationContext-Free Parsing: CKY & Earley Algorithms and Probabilistic Parsing
Context-Free Parsing: CKY & Earley Algorithms and Probabilistic Parsing Natural Language Processing CS 4120/6120 Spring 2017 Northeastern University David Smith with some slides from Jason Eisner & Andrew
More informationA* Search. 1 Dijkstra Shortest Path
A* Search Consider the eight puzzle. There are eight tiles numbered 1 through 8 on a 3 by three grid with nine locations so that one location is left empty. We can move by sliding a tile adjacent to the
More informationLECTURER: BURCU CAN Spring
LECTURER: BURCU CAN 2017-2018 Spring Regular Language Hidden Markov Model (HMM) Context Free Language Context Sensitive Language Probabilistic Context Free Grammar (PCFG) Unrestricted Language PCFGs can
More informationGrammar formalisms Tree Adjoining Grammar: Formal Properties, Parsing. Part I. Formal Properties of TAG. Outline: Formal Properties of TAG
Grammar formalisms Tree Adjoining Grammar: Formal Properties, Parsing Laura Kallmeyer, Timm Lichte, Wolfgang Maier Universität Tübingen Part I Formal Properties of TAG 16.05.2007 und 21.05.2007 TAG Parsing
More informationCS 6120/CS4120: Natural Language Processing
CS 6120/CS4120: Natural Language Processing Instructor: Prof. Lu Wang College of Computer and Information Science Northeastern University Webpage: www.ccs.neu.edu/home/luwang Assignment/report submission
More informationHarvard CS 121 and CSCI E-207 Lecture 12: General Context-Free Recognition
Harvard CS 121 and CSCI E-207 Lecture 12: General Context-Free Recognition Salil Vadhan October 11, 2012 Reading: Sipser, Section 2.3 and Section 2.1 (material on Chomsky Normal Form). Pumping Lemma for
More informationCISC4090: Theory of Computation
CISC4090: Theory of Computation Chapter 2 Context-Free Languages Courtesy of Prof. Arthur G. Werschulz Fordham University Department of Computer and Information Sciences Spring, 2014 Overview In Chapter
More informationEven More on Dynamic Programming
Algorithms & Models of Computation CS/ECE 374, Fall 2017 Even More on Dynamic Programming Lecture 15 Thursday, October 19, 2017 Sariel Har-Peled (UIUC) CS374 1 Fall 2017 1 / 26 Part I Longest Common Subsequence
More informationSyntactical analysis. Syntactical analysis. Syntactical analysis. Syntactical analysis
Context-free grammars Derivations Parse Trees Left-recursive grammars Top-down parsing non-recursive predictive parsers construction of parse tables Bottom-up parsing shift/reduce parsers LR parsers GLR
More informationOutline. CS21 Decidability and Tractability. Machine view of FA. Machine view of FA. Machine view of FA. Machine view of FA.
Outline CS21 Decidability and Tractability Lecture 5 January 16, 219 and Languages equivalence of NPDAs and CFGs non context-free languages January 16, 219 CS21 Lecture 5 1 January 16, 219 CS21 Lecture
More informationStatistical Machine Translation
Statistical Machine Translation -tree-based models (cont.)- Artem Sokolov Computerlinguistik Universität Heidelberg Sommersemester 2015 material from P. Koehn, S. Riezler, D. Altshuler Bottom-Up Decoding
More informationTop-Down Parsing and Intro to Bottom-Up Parsing
Predictive Parsers op-down Parsing and Intro to Bottom-Up Parsing Lecture 7 Like recursive-descent but parser can predict which production to use By looking at the next few tokens No backtracking Predictive
More informationCS 301. Lecture 18 Decidable languages. Stephen Checkoway. April 2, 2018
CS 301 Lecture 18 Decidable languages Stephen Checkoway April 2, 2018 1 / 26 Decidable language Recall, a language A is decidable if there is some TM M that 1 recognizes A (i.e., L(M) = A), and 2 halts
More informationTree Adjoining Grammars
Tree Adjoining Grammars TAG: Parsing and formal properties Laura Kallmeyer & Benjamin Burkhardt HHU Düsseldorf WS 2017/2018 1 / 36 Outline 1 Parsing as deduction 2 CYK for TAG 3 Closure properties of TALs
More informationSyntactic Analysis. Top-Down Parsing
Syntactic Analysis Top-Down Parsing Copyright 2015, Pedro C. Diniz, all rights reserved. Students enrolled in Compilers class at University of Southern California (USC) have explicit permission to make
More information1 Rules in a Feature Grammar
27 Feature Grammars and Unification Drew McDermott drew.mcdermott@yale.edu 2015-11-20, 11-30, 2016-10-28 Yale CS470/570 1 Rules in a Feature Grammar Grammar rules look like this example (using Shieber
More informationParsing. Unger s Parser. Laura Kallmeyer. Winter 2016/17. Heinrich-Heine-Universität Düsseldorf 1 / 21
Parsing Unger s Parser Laura Kallmeyer Heinrich-Heine-Universität Düsseldorf Winter 2016/17 1 / 21 Table of contents 1 Introduction 2 The Parser 3 An Example 4 Optimizations 5 Conclusion 2 / 21 Introduction
More informationMA/CSSE 474 Theory of Computation
MA/CSSE 474 Theory of Computation CFL Hierarchy CFL Decision Problems Your Questions? Previous class days' material Reading Assignments HW 12 or 13 problems Anything else I have included some slides online
More informationAdministrivia. Test I during class on 10 March. Bottom-Up Parsing. Lecture An Introductory Example
Administrivia Test I during class on 10 March. Bottom-Up Parsing Lecture 11-12 From slides by G. Necula & R. Bodik) 2/20/08 Prof. Hilfinger CS14 Lecture 11 1 2/20/08 Prof. Hilfinger CS14 Lecture 11 2 Bottom-Up
More informationProbabilistic Context-free Grammars
Probabilistic Context-free Grammars Computational Linguistics Alexander Koller 24 November 2017 The CKY Recognizer S NP VP NP Det N VP V NP V ate NP John Det a N sandwich i = 1 2 3 4 k = 2 3 4 5 S NP John
More informationParsing Algorithms. CS 4447/CS Stephen Watt University of Western Ontario
Parsing Algorithms CS 4447/CS 9545 -- Stephen Watt University of Western Ontario The Big Picture Develop parsers based on grammars Figure out properties of the grammars Make tables that drive parsing engines
More informationDecoding and Inference with Syntactic Translation Models
Decoding and Inference with Syntactic Translation Models March 5, 2013 CFGs S NP VP VP NP V V NP NP CFGs S NP VP S VP NP V V NP NP CFGs S NP VP S VP NP V NP VP V NP NP CFGs S NP VP S VP NP V NP VP V NP
More informationComputing if a token can follow
Computing if a token can follow first(b 1... B p ) = {a B 1...B p... aw } follow(x) = {a S......Xa... } There exists a derivation from the start symbol that produces a sequence of terminals and nonterminals
More informationComputational Models - Lecture 4
Computational Models - Lecture 4 Regular languages: The Myhill-Nerode Theorem Context-free Grammars Chomsky Normal Form Pumping Lemma for context free languages Non context-free languages: Examples Push
More informationParsing. Unger s Parser. Introduction (1) Unger s parser [Grune and Jacobs, 2008] is a CFG parser that is
Introduction (1) Unger s parser [Grune and Jacobs, 2008] is a CFG parser that is Unger s Parser Laura Heinrich-Heine-Universität Düsseldorf Wintersemester 2012/2013 a top-down parser: we start with S and
More informationContext-Free Parsing: CKY & Earley Algorithms and Probabilistic Parsing
Context-Free Parsing: CKY & Earley Algorithms and Probabilistic Parsing Natural Language Processing! CS 6120 Spring 2014! Northeastern University!! David Smith! with some slides from Jason Eisner & Andrew
More informationContext Free Grammars
Automata and Formal Languages Context Free Grammars Sipser pages 101-111 Lecture 11 Tim Sheard 1 Formal Languages 1. Context free languages provide a convenient notation for recursive description of languages.
More informationA parsing technique for TRG languages
A parsing technique for TRG languages Daniele Paolo Scarpazza Politecnico di Milano October 15th, 2004 Daniele Paolo Scarpazza A parsing technique for TRG grammars [1]
More informationCS 545 Lecture XVI: Parsing
CS 545 Lecture XVI: Parsing brownies_choco81@yahoo.com brownies_choco81@yahoo.com Benjamin Snyder Parsing Given a grammar G and a sentence x = (x1, x2,..., xn), find the best parse tree. We re not going
More informationCSCI 1010 Models of Computa3on. Lecture 17 Parsing Context-Free Languages
CSCI 1010 Models of Computa3on Lecture 17 Parsing Context-Free Languages Overview BoCom-up parsing of CFLs. BoCom-up parsing via the CKY algorithm An O(n 3 ) algorithm John E. Savage CSCI 1010 Lect 17
More informationComputational Models - Lecture 4 1
Computational Models - Lecture 4 1 Handout Mode Iftach Haitner and Yishay Mansour. Tel Aviv University. April 3/8, 2013 1 Based on frames by Benny Chor, Tel Aviv University, modifying frames by Maurice
More informationFinite Automata Theory and Formal Languages TMV027/DIT321 LP4 2018
Finite Automata Theory and Formal Languages TMV027/DIT321 LP4 2018 Lecture 14 Ana Bove May 14th 2018 Recap: Context-free Grammars Simplification of grammars: Elimination of ǫ-productions; Elimination of
More informationParsing. Weighted Deductive Parsing. Laura Kallmeyer. Winter 2017/18. Heinrich-Heine-Universität Düsseldorf 1 / 26
Parsing Weighted Deductive Parsing Laura Kallmeyer Heinrich-Heine-Universität Düsseldorf Winter 2017/18 1 / 26 Table of contents 1 Idea 2 Algorithm 3 CYK Example 4 Parsing 5 Left Corner Example 2 / 26
More informationContext-Free Parsing: CKY & Earley Algorithms and Probabilistic Parsing
Context-Free Parsing: CKY & Earley Algorithms and Probabilistic Parsing Natural Language Processing CS 4120/6120 Spring 2016 Northeastern University David Smith with some slides from Jason Eisner & Andrew
More informationIntroduction to Bottom-Up Parsing
Introduction to Bottom-Up Parsing Outline Review LL parsing Shift-reduce parsing The LR parsing algorithm Constructing LR parsing tables Compiler Design 1 (2011) 2 Top-Down Parsing: Review Top-down parsing
More informationComputational Models - Lecture 5 1
Computational Models - Lecture 5 1 Handout Mode Iftach Haitner. Tel Aviv University. November 28, 2016 1 Based on frames by Benny Chor, Tel Aviv University, modifying frames by Maurice Herlihy, Brown University.
More informationCISC 4090 Theory of Computation
CISC 4090 Theory of Computation Context-Free Languages and Push Down Automata Professor Daniel Leeds dleeds@fordham.edu JMH 332 Languages: Regular and Beyond Regular: Captured by Regular Operations a b
More informationCSE 105 THEORY OF COMPUTATION
CSE 105 THEORY OF COMPUTATION Spring 2017 http://cseweb.ucsd.edu/classes/sp17/cse105-ab/ Review of CFG, CFL, ambiguity What is the language generated by the CFG below: G 1 = ({S,T 1,T 2 }, {0,1,2}, { S
More informationSoft Inference and Posterior Marginals. September 19, 2013
Soft Inference and Posterior Marginals September 19, 2013 Soft vs. Hard Inference Hard inference Give me a single solution Viterbi algorithm Maximum spanning tree (Chu-Liu-Edmonds alg.) Soft inference
More informationParsing. Probabilistic CFG (PCFG) Laura Kallmeyer. Winter 2017/18. Heinrich-Heine-Universität Düsseldorf 1 / 22
Parsing Probabilistic CFG (PCFG) Laura Kallmeyer Heinrich-Heine-Universität Düsseldorf Winter 2017/18 1 / 22 Table of contents 1 Introduction 2 PCFG 3 Inside and outside probability 4 Parsing Jurafsky
More informationNotes for Comp 497 (Comp 454) Week 10 4/5/05
Notes for Comp 497 (Comp 454) Week 10 4/5/05 Today look at the last two chapters in Part II. Cohen presents some results concerning context-free languages (CFL) and regular languages (RL) also some decidability
More informationCSE 105 THEORY OF COMPUTATION
CSE 105 THEORY OF COMPUTATION Spring 2016 http://cseweb.ucsd.edu/classes/sp16/cse105-ab/ Today's learning goals Sipser Ch 2 Define push down automata Trace the computation of a push down automaton Design
More informationEXAM. CS331 Compiler Design Spring Please read all instructions, including these, carefully
EXAM Please read all instructions, including these, carefully There are 7 questions on the exam, with multiple parts. You have 3 hours to work on the exam. The exam is open book, open notes. Please write
More informationSyntax Analysis Part I
1 Syntax Analysis Part I Chapter 4 COP5621 Compiler Construction Copyright Robert van Engelen, Florida State University, 2007-2013 2 Position of a Parser in the Compiler Model Source Program Lexical Analyzer
More informationIntroduction to Bottom-Up Parsing
Outline Introduction to Bottom-Up Parsing Review LL parsing Shift-reduce parsing he LR parsing algorithm Constructing LR parsing tables Compiler Design 1 (2011) 2 op-down Parsing: Review op-down parsing
More informationCMSC 330: Organization of Programming Languages. Pushdown Automata Parsing
CMSC 330: Organization of Programming Languages Pushdown Automata Parsing Chomsky Hierarchy Categorization of various languages and grammars Each is strictly more restrictive than the previous First described
More informationFoundations of Informatics: a Bridging Course
Foundations of Informatics: a Bridging Course Week 3: Formal Languages and Semantics Thomas Noll Lehrstuhl für Informatik 2 RWTH Aachen University noll@cs.rwth-aachen.de http://www.b-it-center.de/wob/en/view/class211_id948.html
More informationFollow sets. LL(1) Parsing Table
Follow sets. LL(1) Parsing Table Exercise Introducing Follow Sets Compute nullable, first for this grammar: stmtlist ::= ε stmt stmtlist stmt ::= assign block assign ::= ID = ID ; block ::= beginof ID
More informationIntroduction to Bottom-Up Parsing
Introduction to Bottom-Up Parsing Outline Review LL parsing Shift-reduce parsing The LR parsing algorithm Constructing LR parsing tables 2 Top-Down Parsing: Review Top-down parsing expands a parse tree
More informationSection 1 (closed-book) Total points 30
CS 454 Theory of Computation Fall 2011 Section 1 (closed-book) Total points 30 1. Which of the following are true? (a) a PDA can always be converted to an equivalent PDA that at each step pops or pushes
More informationPredictive parsing as a specific subclass of recursive descent parsing complexity comparisons with general parsing
Plan for Today Recall Predictive Parsing when it works and when it doesn t necessary to remove left-recursion might have to left-factor Error recovery for predictive parsers Predictive parsing as a specific
More informationHarvard CS 121 and CSCI E-207 Lecture 10: CFLs: PDAs, Closure Properties, and Non-CFLs
Harvard CS 121 and CSCI E-207 Lecture 10: CFLs: PDAs, Closure Properties, and Non-CFLs Harry Lewis October 8, 2013 Reading: Sipser, pp. 119-128. Pushdown Automata (review) Pushdown Automata = Finite automaton
More informationCSE 355 Test 2, Fall 2016
CSE 355 Test 2, Fall 2016 28 October 2016, 8:35-9:25 a.m., LSA 191 Last Name SAMPLE ASU ID 1357924680 First Name(s) Ima Regrading of Midterms If you believe that your grade has not been added up correctly,
More informationContext Free Grammars: Introduction
Context Free Grammars: Introduction Context free grammar (CFG) G = (V, Σ, R, S), where V is a set of grammar symbols Σ V is a set of terminal symbols R is a set of rules, where R (V Σ) V S is a distinguished
More informationProbabilistic Context-Free Grammars. Michael Collins, Columbia University
Probabilistic Context-Free Grammars Michael Collins, Columbia University Overview Probabilistic Context-Free Grammars (PCFGs) The CKY Algorithm for parsing with PCFGs A Probabilistic Context-Free Grammar
More informationCS481F01 Prelim 2 Solutions
CS481F01 Prelim 2 Solutions A. Demers 7 Nov 2001 1 (30 pts = 4 pts each part + 2 free points). For this question we use the following notation: x y means x is a prefix of y m k n means m n k For each of
More informationIntroduction to Bottom-Up Parsing
Outline Introduction to Bottom-Up Parsing Review LL parsing Shift-reduce parsing he LR parsing algorithm Constructing LR parsing tables 2 op-down Parsing: Review op-down parsing expands a parse tree from
More informationIntro to Theory of Computation
Intro to Theory of Computation LECTURE 7 Last time: Proving a language is not regular Pushdown automata (PDAs) Today: Context-free grammars (CFG) Equivalence of CFGs and PDAs Sofya Raskhodnikova 1/31/2016
More informationLecture VII Part 2: Syntactic Analysis Bottom-up Parsing: LR Parsing. Prof. Bodik CS Berkley University 1
Lecture VII Part 2: Syntactic Analysis Bottom-up Parsing: LR Parsing. Prof. Bodik CS 164 -- Berkley University 1 Bottom-Up Parsing Bottom-up parsing is more general than topdown parsing And just as efficient
More informationAdvanced Natural Language Processing Syntactic Parsing
Advanced Natural Language Processing Syntactic Parsing Alicia Ageno ageno@cs.upc.edu Universitat Politècnica de Catalunya NLP statistical parsing 1 Parsing Review Statistical Parsing SCFG Inside Algorithm
More informationSyntax Analysis Part I
1 Syntax Analysis Part I Chapter 4 COP5621 Compiler Construction Copyright Robert van Engelen, Florida State University, 2007-2013 2 Position of a Parser in the Compiler Model Source Program Lexical Analyzer
More informationParsing. Left-Corner Parsing. Laura Kallmeyer. Winter 2017/18. Heinrich-Heine-Universität Düsseldorf 1 / 17
Parsing Left-Corner Parsing Laura Kallmeyer Heinrich-Heine-Universität Düsseldorf Winter 2017/18 1 / 17 Table of contents 1 Motivation 2 Algorithm 3 Look-ahead 4 Chart Parsing 2 / 17 Motivation Problems
More informationCISC 4090 Theory of Computation
CISC 4090 Theory of Computation Context-Free Languages and Push Down Automata Professor Daniel Leeds dleeds@fordham.edu JMH 332 Languages: Regular and Beyond Regular: a b c b d e a Not-regular: c n bd
More informationProperties of Context-Free Languages
Properties of Context-Free Languages Seungjin Choi Department of Computer Science and Engineering Pohang University of Science and Technology 77 Cheongam-ro, Nam-gu, Pohang 37673, Korea seungjin@postech.ac.kr
More informationCYK Algorithm for Parsing General Context-Free Grammars
CYK Algorithm for Parsing General Context-Free Grammars Why Parse General Grammars Can be difficult or impossible to make grammar unambiguous thus LL(k) and LR(k) methods cannot work, for such ambiguous
More informationContext Free Grammars: Introduction. Context Free Grammars: Simplifying CFGs
Context Free Grammars: Introduction CFGs are more powerful than RGs because of the following 2 properties: 1. Recursion Rule is recursive if it is of the form X w 1 Y w 2, where Y w 3 Xw 4 and w 1, w 2,
More informationPushdown Automata (2015/11/23)
Chapter 6 Pushdown Automata (2015/11/23) Sagrada Familia, Barcelona, Spain Outline 6.0 Introduction 6.1 Definition of PDA 6.2 The Language of a PDA 6.3 Euivalence of PDA s and CFG s 6.4 Deterministic PDA
More informationChapter 14 (Partially) Unsupervised Parsing
Chapter 14 (Partially) Unsupervised Parsing The linguistically-motivated tree transformations we discussed previously are very effective, but when we move to a new language, we may have to come up with
More informationNPDA, CFG equivalence
NPDA, CFG equivalence Theorem A language L is recognized by a NPDA iff L is described by a CFG. Must prove two directions: ( ) L is recognized by a NPDA implies L is described by a CFG. ( ) L is described
More informationHarvard CS 121 and CSCI E-207 Lecture 10: Ambiguity, Pushdown Automata
Harvard CS 121 and CSCI E-207 Lecture 10: Ambiguity, Pushdown Automata Salil Vadhan October 4, 2012 Reading: Sipser, 2.2. Another example of a CFG (with proof) L = {x {a, b} : x has the same # of a s and
More informationSequentially Indexed Grammars
Sequentially Indexed Grammars Jan van Eijck CWI and ILLC, Amsterdam, Uil-OTS, Utrecht Draft, January 2005 Abstract This paper defines the grammar class of sequentially indexed grammars. Sequentially indexed
More informationNote: In any grammar here, the meaning and usage of P (productions) is equivalent to R (rules).
Note: In any grammar here, the meaning and usage of P (productions) is equivalent to R (rules). 1a) G = ({R, S, T}, {0,1}, P, S) where P is: S R0R R R0R1R R1R0R T T 0T ε (S generates the first 0. R generates
More informationParser Combinators, (Simply) Indexed Grammars, Natural Language Parsing
Parser Combinators, (Simply) Indexed Grammars, Natural Language Parsing Jan van Eijck jve@cwi.nl Essex, April 23, 2004 Abstract Parser combinators [4, 6] are higher order functions that transform parsers
More informationCSCI Compiler Construction
CSCI 742 - Compiler Construction Lecture 12 Cocke-Younger-Kasami (CYK) Algorithm Instructor: Hossein Hojjat February 20, 2017 Recap: Chomsky Normal Form (CNF) A CFG is in Chomsky Normal Form if each rule
More informationLanguages. Languages. An Example Grammar. Grammars. Suppose we have an alphabet V. Then we can write:
Languages A language is a set (usually infinite) of strings, also known as sentences Each string consists of a sequence of symbols taken from some alphabet An alphabet, V, is a finite set of symbols, e.g.
More informationSyntax Analysis Part I. Position of a Parser in the Compiler Model. The Parser. Chapter 4
1 Syntax Analysis Part I Chapter 4 COP5621 Compiler Construction Copyright Robert van ngelen, Flora State University, 2007 Position of a Parser in the Compiler Model 2 Source Program Lexical Analyzer Lexical
More informationCSE302: Compiler Design
CSE302: Compiler Design Instructor: Dr. Liang Cheng Department of Computer Science and Engineering P.C. Rossin College of Engineering & Applied Science Lehigh University February 27, 2007 Outline Recap
More informationCS5371 Theory of Computation. Lecture 7: Automata Theory V (CFG, CFL, CNF)
CS5371 Theory of Computation Lecture 7: Automata Theory V (CFG, CFL, CNF) Announcement Homework 2 will be given soon (before Tue) Due date: Oct 31 (Tue), before class Midterm: Nov 3, (Fri), first hour
More informationHarvard CS 121 and CSCI E-207 Lecture 9: Regular Languages Wrap-Up, Context-Free Grammars
Harvard CS 121 and CSCI E-207 Lecture 9: Regular Languages Wrap-Up, Context-Free Grammars Salil Vadhan October 2, 2012 Reading: Sipser, 2.1 (except Chomsky Normal Form). Algorithmic questions about regular
More informationCS20a: summary (Oct 24, 2002)
CS20a: summary (Oct 24, 2002) Context-free languages Grammars G = (V, T, P, S) Pushdown automata N-PDA = CFG D-PDA < CFG Today What languages are context-free? Pumping lemma (similar to pumping lemma for
More informationFrom EBNF to PEG. Roman R. Redziejowski. Concurrency, Specification and Programming Berlin 2012
Concurrency, Specification and Programming Berlin 2012 EBNF: Extended Backus-Naur Form A way to define grammar. EBNF: Extended Backus-Naur Form A way to define grammar. Literal = Decimal Binary Decimal
More informationComputability Theory
CS:4330 Theory of Computation Spring 2018 Computability Theory Decidable Problems of CFLs and beyond Haniel Barbosa Readings for this lecture Chapter 4 of [Sipser 1996], 3rd edition. Section 4.1. Decidable
More informationProbabilistic Context Free Grammars
1 Defining PCFGs A PCFG G consists of Probabilistic Context Free Grammars 1. A set of terminals: {w k }, k = 1..., V 2. A set of non terminals: { i }, i = 1..., n 3. A designated Start symbol: 1 4. A set
More informationMaschinelle Sprachverarbeitung
Maschinelle Sprachverarbeitung Parsing with Probabilistic Context-Free Grammar Ulf Leser Content of this Lecture Phrase-Structure Parse Trees Probabilistic Context-Free Grammars Parsing with PCFG Other
More informationEinführung in die Computerlinguistik
Einführung in die Computerlinguistik Context-Free Grammars (CFG) Laura Kallmeyer Heinrich-Heine-Universität Düsseldorf Summer 2016 1 / 22 CFG (1) Example: Grammar G telescope : Productions: S NP VP NP
More informationAutomata & languages. A primer on the Theory of Computation. Laurent Vanbever. ETH Zürich (D-ITET) October,
Automata & languages A primer on the Theory of Computation Laurent Vanbever www.vanbever.eu ETH Zürich (D-ITET) October, 5 2017 Part 3 out of 5 Last week, we learned about closure and equivalence of regular
More informationPart 3 out of 5. Automata & languages. A primer on the Theory of Computation. Last week, we learned about closure and equivalence of regular languages
Automata & languages A primer on the Theory of Computation Laurent Vanbever www.vanbever.eu Part 3 out of 5 ETH Zürich (D-ITET) October, 5 2017 Last week, we learned about closure and equivalence of regular
More information