Quasi-Second-Order Parsing for 1-Endpoint-Crossing, Pagenumber-2 Graphs
|
|
|
- Oswin James Fields
- 6 years ago
- Views:
Transcription
1 Quasi-Second-Order Parsing for 1-Endpoint-Crossing, Pagenumber-2 Graphs Junjie Cao, Sheng Huang, Weiwei Sun, Xiaojun Wan Institute of Computer Science and Technology Peking University September 5, of 41
2 Overview The Problem First-order Algorithm Second-order Algorithm Experiments 2 of 41
3 Outline The Problem First-order Algorithm Second-order Algorithm Experiments 3 of 41
4 Semantic dependency parsing Example arg1 arg1 arg1 arg1 arg1 The company that Mark wants to buy Predicate argument analysis, bi-lexical relations Long-distance dependencies Graph-structured representations, many crossing arcs Not a tree: single-headed ( ), cycle-free ( ) 4 of 41
5 Semantic dependency parsing Example arg1 arg1 arg1 arg1 arg1 The company that Mark wants to buy Predicate argument analysis, bi-lexical relations Long-distance dependencies Graph-structured representations, many crossing arcs Not a tree: single-headed ( ), cycle-free ( ) 4 of 41
6 Semantic dependency parsing Example arg1 arg1 arg1 arg1 arg1 The company that Mark wants to buy Predicate argument analysis, bi-lexical relations Long-distance dependencies Graph-structured representations, many crossing arcs Not a tree: single-headed ( ), cycle-free ( ) 4 of 41
7 Semantic dependency parsing Example arg1 arg1 arg1 arg1 arg1 The company that Mark wants to buy Predicate argument analysis, bi-lexical relations Long-distance dependencies Graph-structured representations, many crossing arcs Not a tree: single-headed ( ), cycle-free ( ) 4 of 41
8 Semantic dependency parsing Example arg1 arg1 arg1 arg1 The company that Mark wants to buy Predicate argument analysis, bi-lexical relations Long-distance dependencies Graph-structured representations, many crossing arcs Not a tree: single-headed ( ), cycle-free ( ) 4 of 41
9 Maximum Subgraph Input A directed graph G = (V, A) Output Subgraph G = (V, A A) with maximum total weight such that G belongs to G G (s) = arg max ScorePart(s, p) H G(s,G ) p H Example When G is tree, Maximum Subgraph = Maximum Spanning Tree Complexity G and the order of ScorePart determine the complexity of inference. 5 of 41
10 Complexity G O Algo Arbitrary 1 O(n 2 ) Arbitrary 2 NP-hard (Du et al., 2015) Acyclic 1 NP-hard (Kuhlmann and Jonsson, 2015) Noncrossing 1 O(n 3 ) (Kuhlmann and Jonsson, 2015) Noncrossing 2 O(n 4 ) (Sun et al., 2017) 1-endpoint-crossing 1 O(n 5 ) Ongoing work 1-endpoint-crossing 1 O(n 5 ) (Cao et al., 2017) pagenumber-2 1-endpoint-crossing 1 O(n 4 ) (Cao et al., 2017) pagenumber-2, C-free 1-endpoint-crossing 2 O(n 4 ) This paper pagenumber-2, C-free 6 of 41
11 1-Endpoint-Crossing Graphs Definition A dependency graph is 1-Endpoint-Crossing if for any edge e, all edges that cross e share an endpoint p named pencil point. 7 of 41
12 1-Endpoint-Crossing Graphs Definition A dependency graph is 1-Endpoint-Crossing if for any edge e, all edges that cross e share an endpoint p named pencil point. arg1 arg1 arg1 arg1 arg1 The company that Mark wants to buy 7 of 41
13 1-Endpoint-Crossing Graphs Definition A dependency graph is 1-Endpoint-Crossing if for any edge e, all edges that cross e share an endpoint p named pencil point. arg1 arg1 The company that Mark wants to buy 7 of 41
14 1-Endpoint-Crossing Graphs Definition A dependency graph is 1-Endpoint-Crossing if for any edge e, all edges that cross e share an endpoint p named pencil point. arg1 The company that Mark wants to buy 7 of 41
15 1-Endpoint-Crossing Graphs Definition A dependency graph is 1-Endpoint-Crossing if for any edge e, all edges that cross e share an endpoint p named pencil point. arg1 The company that Mark wants to buy 7 of 41
16 1-Endpoint-Crossing Graphs Definition A dependency graph is 1-Endpoint-Crossing if for any edge e, all edges that cross e share an endpoint p named pencil point. arg1 The company that Mark wants to buy 7 of 41
17 Pagenumber-K Graphs Definition A dependency graph G is a pagenumber-k graph if G consists at most K subgraphs called pages. Each page contains all vertices, but only a subset of arcs that are not crossed with other arcs in this page. 8 of 41
18 Pagenumber-K graph Example The company that Mark wants to buy arg1 arg1 arg1 arg1 arg1 A Pagenumber-2 Graph 9 of 41
19 Pagenumber-K graph Example The company that Mark wants to buy arg1 arg1 arg1 arg1 arg1 A Pagenumber-3 Graph 9 of 41
20 Coverage PN 2 1EC EnjuBank DeepBank PCEDT CCGBank Yes Both 99.53% 99.69% 98.39% 98.09% Both Yes 97.28% 97.67% 97.53% 95.73% Yes Yes 97.28% 97.67% 97.53% 95.68% No Yes 0.0% 0.0% 0.0% 0.05% Yes No 2.25% 2.02% 0.86% 2.41% Sentences 100% 100% 100% 100% 10 of 41
21 Coverage PN 2 1EC EnjuBank DeepBank PCEDT CCGBank Yes Both 99.53% 99.69% 98.39% 98.09% Both Yes 97.28% 97.67% 97.53% 95.73% Yes Yes 97.28% 97.67% 97.53% 95.68% No Yes 0.0% 0.0% 0.0% 0.05% Yes No 2.25% 2.02% 0.86% 2.41% Sentences 100% 100% 100% 100% Most semantic dependency graphs are 1EC/P2 graphs. 10 of 41
22 Coverage PN 2 1EC EnjuBank DeepBank PCEDT CCGBank Yes Both 99.53% 99.69% 98.39% 98.09% Both Yes 97.28% 97.67% 97.53% 95.73% Yes Yes 97.28% 97.67% 97.53% 95.68% No Yes 0.0% 0.0% 0.0% 0.05% Yes No 2.25% 2.02% 0.86% 2.41% Sentences 100% 100% 100% 100% Most semantic dependency graphs are 1EC/P2 graphs. Theorem The pagenumber of a 1EC graph is at most of 41
23 Previous Work (1) G O Algo Arbitrary 1 O(n 2 ) Arbitrary 2 NP-hard (Du et al., 2015) Acyclic 1 NP-hard (Kuhlmann and Jonsson, 2015) Noncrossing 1 O(n 3 ) (Kuhlmann and Jonsson, 2015) Noncrossing 2 O(n 4 ) (Sun et al., 2017) 1-endpoint-crossing 1 O(n 5 ) Ongoing work 1-endpoint-crossing 1 O(n 5 ) (Cao et al., 2017) pagenumber-2 1-endpoint-crossing 1 O(n 4 ) (Cao et al., 2017) pagenumber-2, C-free 1-endpoint-crossing 2 O(n 4 ) This paper pagenumber-2, C-free 11 of 41
24 Previous Work (2) Key observation Every subgraph of a 1EC/P2 graph is still a 1EC/P2 graph. A dynamic programming algorithm gchsw In each construction step, usually more than one arcs are allowed to be constructed. Whether or not such arcs are created depends on their arc-weights. We are able to get a maximal 1EC/P2 graph, but just choose a subgraph of it with all positive arcs. 12 of 41
25 Challenge of High-order Factorization (1) A single step in gchsw i l k j e (i,k),e (l,j) and e (i,j) can be created at the same time. Eisner s algorithm 13 of 41 In a single step, which arc is created is deterministic!
26 Challenge of High-order Factorization (2) It is very difficult to enumerate all high-order features for crossing arcs. 14 of 41
27 Challenge of High-order Factorization (2) It is very difficult to enumerate all high-order features for crossing arcs. x r x i r i k j l j 14 of 41
28 Challenge of High-order Factorization (2) It is very difficult to enumerate all high-order features for crossing arcs. x r x i r i k j l j It is hard to cover sibling features between e (x,k) and e (x,rx ). 14 of 41
29 Challenge of High-order Factorization (3) Pitler (2014) It is still possible to build accurate tree parsers by considering only higher-order features of noncrossing arcs. 15 of 41
30 Challenge of High-order Factorization (3) Pitler (2014) It is still possible to build accurate tree parsers by considering only higher-order features of noncrossing arcs. arg1 arg1 arg1 arg1 arg1 The company that Mark wants to buy 15 of 41
31 Challenge of High-order Factorization (3) Pitler (2014) It is still possible to build accurate tree parsers by considering only higher-order features of noncrossing arcs. arg1 arg1 arg1 arg1 arg1 The company that Mark wants to buy 15 of 41
32 Challenge of High-order Factorization (3) Pitler (2014) It is still possible to build accurate tree parsers by considering only higher-order features of noncrossing arcs. arg1 arg1 arg1 arg1 arg1 The company that Mark wants to buy 15 of 41
33 Challenge of High-order Factorization (3) Pitler (2014) It is still possible to build accurate tree parsers by considering only higher-order features of noncrossing arcs. arg1 arg1 arg1 arg1 arg1 The company that Mark wants to buy 15 of 41
34 Challenge of High-order Factorization (3) Pitler (2014) It is still possible to build accurate tree parsers by considering only higher-order features of noncrossing arcs. arg1 arg1 arg1 arg1 arg1 The company that Mark wants to buy Good news: Most of arcs are noncrossing even in crossing graphs. 15 of 41
35 Previous Work (3) O[s, e] s e C [s, e, l] s e s e = s + 1 e s e = s k + k e 16 of 41
36 Outline The Problem First-order Algorithm Second-order Algorithm Experiments 17 of 41
37 Sub-problem of C-free 1EC/P2 Int O [i, j] LR[i, j, x] N O [i, j, x] L O [i, j, x] R O [i, j, x] i j x i j x i j x i j x i j 18 of 41
38 Sub-problem of C-free 1EC/P2 Int O [i, j] LR[i, j, x] N O [i, j, x] L O [i, j, x] R O [i, j, x] i j x i j x i j x i j x i j Int C [i, j] N C [i, j, x] L C [i, j, x] R C [i, j, x] i j x i j x i j x i j 18 of 41
39 Sub-problem of C-free 1EC/P2 Int O [i, j] LR[i, j, x] N O [i, j, x] L O [i, j, x] R O [i, j, x] i j x i j x i j x i j x i j Int C [i, j] N C [i, j, x] L C [i, j, x] R C [i, j, x] i j x i j x i j x i j Open-structure can be transformed to close-structure if red arc exists. 18 of 41
40 Decomposition of Int C Decompose Int C considering farthest arc from i 1 No arc 2 Noncrossing edge 3 Crossing edge with outer pencil point 4 Crossing edge with inner pencil point 19 of 41
41 Decomposition of Int C (a) i j = i + 1 j If there is no arc from i to (i, j). 20 of 41
42 Decomposition of Int C (b) i k j = i k + k j If there is a noncrossing arc from i to (i, j). 21 of 41
43 Decomposition of Int C (c) i k Dashed edge exist? x j For a crossing arc e (i,k) with outer pt(i,k) = x 22 of 41
44 Decomposition of Int C (c) i k Dashed edge exist? x j (c.1) i k x j = i k x + k x + k x j For a crossing arc e (i,k) with outer pt(i,k) = x 22 of 41
45 Decomposition of Int C (c) i k Dashed edge exist? x j (c.1) i k x j = i k x + k x + k x j (c.2) i k x j = i k x + k x + x j For a crossing arc e (i,k) with outer pt(i,k) = x 22 of 41
46 Decomposition of Int C Dashed edge exist? i x k j For a crossing arc e (i,k) with inner pt(i,k) = x 23 of 41
47 Decomposition of Int C Dashed edge exist? i (d.1) i x x k k j = i x + i x k + x k j j For a crossing arc e (i,k) with inner pt(i,k) = x 23 of 41
48 Decomposition of Int C Dashed edge exist? i (d.1) i x (d.2) i x x k k k j j = i x + i x k + x k j j = i x k + x k + x k j For a crossing arc e (i,k) with inner pt(i,k) = x 23 of 41
49 C-free LR Decomposition x i j 24 of 41
50 C-free LR Decomposition x i j x i = j x i k + x k j If there exists k dividing [i,j] into two independent spans 24 of 41
51 C-free LR Decomposition For each vertex k, there must be edges from [i,k) to (k,j]. x i b 1 a 1 b 2 a 2 j, b 3 b 3 = j, there exists only e x,b1 or e x,a2. 25 of 41
52 C-free LR Decomposition For each vertex k, there must be edges from [i,k) to (k,j]. x i b 1 a 1 b 2 a 2 j, b 3 b 3 = j, there exists only e x,b1 or e x,a2. x i b 1 a 1 b 2 a 2 b 3 j, a 3 a 3 = j, there exists both e x,b1 and e x,b3. 25 of 41
53 Example The company that Mark wants to buy Int O [1, 7] 26 of 41
54 Example The company that Mark wants to buy Int O [1, 7] = Int C [1, 2] + Int O [2, 7] 26 of 41
55 Example The company that Mark wants to buy Int O [2, 7] = Int C [2, 7] 26 of 41
56 Example The company that Mark wants to buy Int c [2, 7] = Int c [2, 3] + Int O [3, 7] 26 of 41
57 Example The company that Mark wants to buy Int O [3, 7] = R O [3, 4; 5] + Int O [4, 5] + L O [5, 7; 4] 26 of 41
58 Example The company that Mark wants to buy R O [3, 4; 5] = Int O [3, 4] 26 of 41
59 Example The company that Mark wants to buy Int O [4, 5] = Int C [4, 5] 26 of 41
60 Example The company that Mark wants to buy L O [5, 7; 4] = L C [5, 7; 4] 26 of 41
61 Example The company that Mark wants to buy L C [5, 7; 4] = Int O [5, 6] + L O [6, 7; 5] 26 of 41
62 Example The company that Mark wants to buy L O [6, 7; 5] = L C [6, 7; 5] 26 of 41
63 Example The company that Mark wants to buy L C [6, 7; 5] = Int O [6, 7] = Int C [6, 7] 26 of 41
64 Example The company that Mark wants to buy Get All Arcs 26 of 41
65 Spurious Ambiguity A cross-type subproblem allows to build crossing arcs, but does not necessarily create crossing arcs. 27 of 41
66 Spurious Ambiguity A cross-type subproblem allows to build crossing arcs, but does not necessarily create crossing arcs. a b c d e 27 of 41
67 Spurious Ambiguity A cross-type subproblem allows to build crossing arcs, but does not necessarily create crossing arcs. a b c d e Int C [a, e] Int C [a, c] + Int O [c, e]. 27 of 41
68 Spurious Ambiguity A cross-type subproblem allows to build crossing arcs, but does not necessarily create crossing arcs. a b c d e Int C [a, e] LR[a, c, d] + Int O [k, d] + L O [d, e, c]; LR[a, c; d] L O [a, b; d] + R O [b, c, d] Int O [a, b] + Int O [b, c]. 27 of 41
69 Outline The Problem First-order Algorithm Second-order Algorithm Experiments 28 of 41
70 Crossing-sensitive Single-side Second-order algorithm G (s) = arg max G e Edge(G ) Score 1 (e) + s Sib(G ) max(score 2 (s), 0) 29 of 41
71 Crossing-sensitive Single-side Second-order algorithm G (s) = arg max G e Edge(G ) Score 1 (e) + s Sib(G ) max(score 2 (s), 0) Both sibling arcs are noncrossing 29 of 41
72 Second-order Factorization s e = s + 1 e 1 s e = s r s + rs e 1 s e = s + 1 l e + le e s e = s + 1 l e + le e 30 of 41
73 Second-order Factorization Noncrossing sibling features can only be captured by decomposing Int C 31 of 41
74 Second-order Factorization Noncrossing sibling features can only be captured by decomposing Int C (a.1) (b.1) (c.1) i j = i + 1 j 1 i j = i j 1 i j = + i ri ri j 1 (a.2) i j = i + 1 j (b.2) i j = i rj rj j (c.2) i j = + i ri ri j (a.3) i j = i + 1 lj + lj j (b.3) i j = + i rj rj j (c.3) i j = i ri + ri lj + lj j 31 of 41
75 Example The company that Mark wants to buy of 41
76 Example The company that Mark wants to buy Int c [2, 7] = Int c [2, 3] + Int O [3, 7] + sib(e (2,7), e (2,3) ) 32 of 41
77 Spurious Ambiguity (1) This model is somehow inadequate given that the second-order score function cannot penalize a bad factor. When a negative score is assigned to a second-order factor, it will be taken as 0 by our algorithm. 33 of 41
78 Spurious Ambiguity (1) This model is somehow inadequate given that the second-order score function cannot penalize a bad factor. When a negative score is assigned to a second-order factor, it will be taken as 0 by our algorithm. a b c d e 33 of 41
79 Spurious Ambiguity (1) This model is somehow inadequate given that the second-order score function cannot penalize a bad factor. When a negative score is assigned to a second-order factor, it will be taken as 0 by our algorithm. a b c d e Int C [a, e] Int C [a, c] + Int O [c, e] + S sib (e (a,e), e (a,c) ). 33 of 41
80 Spurious Ambiguity (1) This model is somehow inadequate given that the second-order score function cannot penalize a bad factor. When a negative score is assigned to a second-order factor, it will be taken as 0 by our algorithm. a b c d e Int C [a, e] LR[a, c, d] + Int O [k, d] + L O [d, e, c]; LR[a, c; d] L O [a, b; d] + R O [b, c, d] Int O [a, b] + Int O [b, c]. 33 of 41
81 Spurious Ambiguity (2) G (s) = arg max G e Edge(G ) Score 1 (e) + s Sib(G ) max(score 2 (s), 0) Score 2 (s) 0 Our algorithm selects the derivation that takes s into account since it increases the total score. Score 2 (s) < 0 Our algorithm avoids including s by selecting other paths. In other words, our algorithm treats this score as of 41
82 Outline The Problem First-order Algorithm Second-order Algorithm Experiments 35 of 41
83 Results 92 Without Tree 90 F-Score DM PAS CCG PCEDT First Second 36 of 41
84 Results 94 Syntax Tree F-Score DM PAS CCG PCEDT First Second 37 of 41
85 Conclusion Our contributions A new dynamic programming algorithm for first-order parsing to 1-endpiont-crossing, pagenumber-2, C-free graphs. A new quasi-second-order extension. Lesson learned Crossing-sensitive second-order features are helpful. 38 of 41
86 Game Over 39 of 41
87 Game Over QUESTIONS? COMMENTS? 39 of 41
88 References (1) Junjie Cao, Sheng Huang, Weiwei Sun, and Xiaojun Wan Parsing to 1-endpoint-crossing, pagenumber-2 graphs. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages Association for Computational Linguistics, Vancouver, Canada. URL Yantao Du, Weiwei Sun, and Xiaojun Wan A data-driven, factorization parser for CCG dependency structures. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages Association for Computational Linguistics, Beijing, China. URL Marco Kuhlmann and Peter Jonsson Parsing to noncrossing dependency graphs. Transactions of the Association for Computational Linguistics, 3: Emily Pitler A crossing-sensitive third-order factorization for dependency parsing. TACL, 2: URL 40 of 41
89 References (2) Weiwei Sun, Junjie Cao, and Xiaojun Wan Semantic dependency parsing via book embedding. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages Association for Computational Linguistics, Vancouver, Canada. URL 41 of 41
Bringing machine learning & compositional semantics together: central concepts
Bringing machine learning & compositional semantics together: central concepts https://githubcom/cgpotts/annualreview-complearning Chris Potts Stanford Linguistics CS 244U: Natural language understanding
Graph-based Dependency Parsing. Ryan McDonald Google Research
Graph-based Dependency Parsing Ryan McDonald Google Research ryanmcd@google.com Reader s Digest Graph-based Dependency Parsing Ryan McDonald Google Research ryanmcd@google.com root ROOT Dependency Parsing
A Polynomial Time Algorithm for Parsing with the Bounded Order Lambek Calculus
A Polynomial Time Algorithm for Parsing with the Bounded Order Lambek Calculus Timothy A. D. Fowler Department of Computer Science University of Toronto 10 King s College Rd., Toronto, ON, M5S 3G4, Canada
Advanced Graph-Based Parsing Techniques
Advanced Graph-Based Parsing Techniques Joakim Nivre Uppsala University Linguistics and Philology Based on previous tutorials with Ryan McDonald Advanced Graph-Based Parsing Techniques 1(33) Introduction
Driving Semantic Parsing from the World s Response
Driving Semantic Parsing from the World s Response James Clarke, Dan Goldwasser, Ming-Wei Chang, Dan Roth Cognitive Computation Group University of Illinois at Urbana-Champaign CoNLL 2010 Clarke, Goldwasser,
Computational Linguistics
Computational Linguistics Dependency-based Parsing Clayton Greenberg Stefan Thater FR 4.7 Allgemeine Linguistik (Computerlinguistik) Universität des Saarlandes Summer 2016 Acknowledgements These slides
Lab 12: Structured Prediction
December 4, 2014 Lecture plan structured perceptron application: confused messages application: dependency parsing structured SVM Class review: from modelization to classification What does learning mean?
Introduction to Computational Linguistics
Introduction to Computational Linguistics Olga Zamaraeva (2018) Based on Bender (prev. years) University of Washington May 3, 2018 1 / 101 Midterm Project Milestone 2: due Friday Assgnments 4& 5 due dates
Computational Linguistics. Acknowledgements. Phrase-Structure Trees. Dependency-based Parsing
Computational Linguistics Dependency-based Parsing Dietrich Klakow & Stefan Thater FR 4.7 Allgemeine Linguistik (Computerlinguistik) Universität des Saarlandes Summer 2013 Acknowledgements These slides
Penn Treebank Parsing. Advanced Topics in Language Processing Stephen Clark
Penn Treebank Parsing Advanced Topics in Language Processing Stephen Clark 1 The Penn Treebank 40,000 sentences of WSJ newspaper text annotated with phrasestructure trees The trees contain some predicate-argument
Language Learning Problems in the Principles and Parameters Framework
Language Learning Problems in the Principles and Parameters Framework Partha Niyogi Presented by Chunchuan Lyu March 22, 2016 Partha Niyogi (presented by c.c. lyu) Language Learning March 22, 2016 1 /
Generalized Pigeonhole Properties of Graphs and Oriented Graphs
Europ. J. Combinatorics (2002) 23, 257 274 doi:10.1006/eujc.2002.0574 Available online at http://www.idealibrary.com on Generalized Pigeonhole Properties of Graphs and Oriented Graphs ANTHONY BONATO, PETER
Tree Decompositions and Tree-Width
Tree Decompositions and Tree-Width CS 511 Iowa State University December 6, 2010 CS 511 (Iowa State University) Tree Decompositions and Tree-Width December 6, 2010 1 / 15 Tree Decompositions Definition
S NP VP 0.9 S VP 0.1 VP V NP 0.5 VP V 0.1 VP V PP 0.1 NP NP NP 0.1 NP NP PP 0.2 NP N 0.7 PP P NP 1.0 VP NP PP 1.0. N people 0.
/6/7 CS 6/CS: Natural Language Processing Instructor: Prof. Lu Wang College of Computer and Information Science Northeastern University Webpage: www.ccs.neu.edu/home/luwang The grammar: Binary, no epsilons,.9..5
Polyhedral Outer Approximations with Application to Natural Language Parsing
Polyhedral Outer Approximations with Application to Natural Language Parsing André F. T. Martins 1,2 Noah A. Smith 1 Eric P. Xing 1 1 Language Technologies Institute School of Computer Science Carnegie
Advanced Natural Language Processing Syntactic Parsing
Advanced Natural Language Processing Syntactic Parsing Alicia Ageno ageno@cs.upc.edu Universitat Politècnica de Catalunya NLP statistical parsing 1 Parsing Review Statistical Parsing SCFG Inside Algorithm
Relations Graphical View
Introduction Relations Computer Science & Engineering 235: Discrete Mathematics Christopher M. Bourke cbourke@cse.unl.edu Recall that a relation between elements of two sets is a subset of their Cartesian
Parameterized Domination in Circle Graphs
Parameterized Domination in Circle Graphs Nicolas Bousquet 1, Daniel Gonçalves 1, George B. Mertzios 2, Christophe Paul 1, Ignasi Sau 1, and Stéphan Thomassé 3 1 AlGCo project-team, CNRS, LIRMM, Montpellier,
FIRST ORDER SENTENCES ON G(n, p), ZERO-ONE LAWS, ALMOST SURE AND COMPLETE THEORIES ON SPARSE RANDOM GRAPHS
FIRST ORDER SENTENCES ON G(n, p), ZERO-ONE LAWS, ALMOST SURE AND COMPLETE THEORIES ON SPARSE RANDOM GRAPHS MOUMANTI PODDER 1. First order theory on G(n, p) We start with a very simple property of G(n,
Marrying Dynamic Programming with Recurrent Neural Networks
Marrying Dynamic Programming with Recurrent Neural Networks I eat sushi with tuna from Japan Liang Huang Oregon State University Structured Prediction Workshop, EMNLP 2017, Copenhagen, Denmark Marrying
Cases Where Finding the Minimum Entropy Coloring of a Characteristic Graph is a Polynomial Time Problem
Cases Where Finding the Minimum Entropy Coloring of a Characteristic Graph is a Polynomial Time Problem Soheil Feizi, Muriel Médard RLE at MIT Emails: {sfeizi,medard}@mit.edu Abstract In this paper, we
Decoding and Inference with Syntactic Translation Models
Decoding and Inference with Syntactic Translation Models March 5, 2013 CFGs S NP VP VP NP V V NP NP CFGs S NP VP S VP NP V V NP NP CFGs S NP VP S VP NP V NP VP V NP NP CFGs S NP VP S VP NP V NP VP V NP
Learning Dependency-Based Compositional Semantics
Learning Dependency-Based Compositional Semantics Semantic Representations for Textual Inference Workshop Mar. 0, 0 Percy Liang Google/Stanford joint work with Michael Jordan and Dan Klein Motivating Problem:
Combinatorial Optimization
Combinatorial Optimization Problem set 8: solutions 1. Fix constants a R and b > 1. For n N, let f(n) = n a and g(n) = b n. Prove that f(n) = o ( g(n) ). Solution. First we observe that g(n) 0 for all
UNIT II REGULAR LANGUAGES
1 UNIT II REGULAR LANGUAGES Introduction: A regular expression is a way of describing a regular language. The various operations are closure, union and concatenation. We can also find the equivalent regular
Acyclic and Oriented Chromatic Numbers of Graphs
Acyclic and Oriented Chromatic Numbers of Graphs A. V. Kostochka Novosibirsk State University 630090, Novosibirsk, Russia X. Zhu Dept. of Applied Mathematics National Sun Yat-Sen University Kaohsiung,
Causal Belief Decomposition for Planning with Sensing: Completeness Results and Practical Approximation
Causal Belief Decomposition for Planning with Sensing: Completeness Results and Practical Approximation Blai Bonet 1 and Hector Geffner 2 1 Universidad Simón Boĺıvar 2 ICREA & Universitat Pompeu Fabra
Cycle Double Cover Conjecture
Cycle Double Cover Conjecture Paul Clarke St. Paul's College Raheny January 5 th 2014 Abstract In this paper, a proof of the cycle double cover conjecture is presented. The cycle double cover conjecture
Natural Language Processing CS Lecture 06. Razvan C. Bunescu School of Electrical Engineering and Computer Science
Natural Language Processing CS 6840 Lecture 06 Razvan C. Bunescu School of Electrical Engineering and Computer Science bunescu@ohio.edu Statistical Parsing Define a probabilistic model of syntax P(T S):
Proof of Theorem 1. Tao Lei CSAIL,MIT. Here we give the proofs of Theorem 1 and other necessary lemmas or corollaries.
Proof of Theorem 1 Tao Lei CSAIL,MIT Here we give the proofs of Theorem 1 and other necessary lemmas or corollaries. Lemma 1 (Reachability) Any two trees y, y are reachable to each other. Specifically,
Automata Theory, Computability and Complexity
Automata Theory, Computability and Complexity Mridul Aanjaneya Stanford University June 26, 22 Mridul Aanjaneya Automata Theory / 64 Course Staff Instructor: Mridul Aanjaneya Office Hours: 2:PM - 4:PM,
Learning Goals of CS245 Logic and Computation
Learning Goals of CS245 Logic and Computation Alice Gao April 27, 2018 Contents 1 Propositional Logic 2 2 Predicate Logic 4 3 Program Verification 6 4 Undecidability 7 1 1 Propositional Logic Introduction
On the Sizes of Decision Diagrams Representing the Set of All Parse Trees of a Context-free Grammar
Proceedings of Machine Learning Research vol 73:153-164, 2017 AMBN 2017 On the Sizes of Decision Diagrams Representing the Set of All Parse Trees of a Context-free Grammar Kei Amii Kyoto University Kyoto
Min-max model for the network reduction problem
Min-max model for the network reduction problem tefan PE KO University of šilina, Slovakia Mathematical Methods in Economics - Jihlava, Czech Republic 11-13 September 2013 Network reduction problem We
Decomposing planar cubic graphs
Decomposing planar cubic graphs Arthur Hoffmann-Ostenhof Tomáš Kaiser Kenta Ozeki Abstract The 3-Decomposition Conjecture states that every connected cubic graph can be decomposed into a spanning tree,
CKY & Earley Parsing. Ling 571 Deep Processing Techniques for NLP January 13, 2016
CKY & Earley Parsing Ling 571 Deep Processing Techniques for NLP January 13, 2016 No Class Monday: Martin Luther King Jr. Day CKY Parsing: Finish the parse Recognizer à Parser Roadmap Earley parsing Motivation:
The Mixed Chinese Postman Problem Parameterized by Pathwidth and Treedepth
The Mixed Chinese Postman Problem Parameterized by Pathwidth and Treedepth Gregory Gutin, Mark Jones, and Magnus Wahlström Royal Holloway, University of London Egham, Surrey TW20 0EX, UK Abstract In the
Parsing. Based on presentations from Chris Manning s course on Statistical Parsing (Stanford)
Parsing Based on presentations from Chris Manning s course on Statistical Parsing (Stanford) S N VP V NP D N John hit the ball Levels of analysis Level Morphology/Lexical POS (morpho-synactic), WSD Elements
An introduction to PRISM and its applications
An introduction to PRISM and its applications Yoshitaka Kameya Tokyo Institute of Technology 2007/9/17 FJ-2007 1 Contents What is PRISM? Two examples: from population genetics from statistical natural
Handout: Proof of the completeness theorem
MATH 457 Introduction to Mathematical Logic Spring 2016 Dr. Jason Rute Handout: Proof of the completeness theorem Gödel s Compactness Theorem 1930. For a set Γ of wffs and a wff ϕ, we have the following.
Graphical Model Inference with Perfect Graphs
Graphical Model Inference with Perfect Graphs Tony Jebara Columbia University July 25, 2013 joint work with Adrian Weller Graphical models and Markov random fields We depict a graphical model G as a bipartite
Undirected Graphical Models
Outline Hong Chang Institute of Computing Technology, Chinese Academy of Sciences Machine Learning Methods (Fall 2012) Outline Outline I 1 Introduction 2 Properties Properties 3 Generative vs. Conditional
1. For the following sub-problems, consider the following context-free grammar: S AA$ (1) A xa (2) A B (3) B yb (4)
ECE 468 & 573 Problem Set 2: Contet-free Grammars, Parsers 1. For the following sub-problems, consider the following contet-free grammar: S $ (1) (2) (3) (4) λ (5) (a) What are the terminals and non-terminals
NLU: Semantic parsing
NLU: Semantic parsing Adam Lopez slide credits: Chris Dyer, Nathan Schneider March 30, 2018 School of Informatics University of Edinburgh alopez@inf.ed.ac.uk Recall: meaning representations Sam likes Casey
Maschinelle Sprachverarbeitung
Maschinelle Sprachverarbeitung Parsing with Probabilistic Context-Free Grammar Ulf Leser Content of this Lecture Phrase-Structure Parse Trees Probabilistic Context-Free Grammars Parsing with PCFG Other
Maschinelle Sprachverarbeitung
Maschinelle Sprachverarbeitung Parsing with Probabilistic Context-Free Grammar Ulf Leser Content of this Lecture Phrase-Structure Parse Trees Probabilistic Context-Free Grammars Parsing with PCFG Other
Dual Decomposition for Inference
Dual Decomposition for Inference Yunshu Liu ASPITRG Research Group 2014-05-06 References: [1]. D. Sontag, A. Globerson and T. Jaakkola, Introduction to Dual Decomposition for Inference, Optimization for
Introduction to Semantic Parsing with CCG
Introduction to Semantic Parsing with CCG Kilian Evang Heinrich-Heine-Universität Düsseldorf 2018-04-24 Table of contents 1 Introduction to CCG Categorial Grammar (CG) Combinatory Categorial Grammar (CCG)
Notes. Relations. Introduction. Notes. Relations. Notes. Definition. Example. Slides by Christopher M. Bourke Instructor: Berthe Y.
Relations Slides by Christopher M. Bourke Instructor: Berthe Y. Choueiry Spring 2006 Computer Science & Engineering 235 Introduction to Discrete Mathematics Sections 7.1, 7.3 7.5 of Rosen cse235@cse.unl.edu
Jointly Extracting Event Triggers and Arguments by Dependency-Bridge RNN and Tensor-Based Argument Interaction
Jointly Extracting Event Triggers and Arguments by Dependency-Bridge RNN and Tensor-Based Argument Interaction Feng Qian,LeiSha, Baobao Chang, Zhifang Sui Institute of Computational Linguistics, Peking
A* Search. 1 Dijkstra Shortest Path
A* Search Consider the eight puzzle. There are eight tiles numbered 1 through 8 on a 3 by three grid with nine locations so that one location is left empty. We can move by sliding a tile adjacent to the
CS 188: Artificial Intelligence Spring Announcements
CS 188: Artificial Intelligence Spring 2011 Lecture 16: Bayes Nets IV Inference 3/28/2011 Pieter Abbeel UC Berkeley Many slides over this course adapted from Dan Klein, Stuart Russell, Andrew Moore Announcements
CS626: NLP, Speech and the Web. Pushpak Bhattacharyya CSE Dept., IIT Bombay Lecture 14: Parsing Algorithms 30 th August, 2012
CS626: NLP, Speech and the Web Pushpak Bhattacharyya CSE Dept., IIT Bombay Lecture 14: Parsing Algorithms 30 th August, 2012 Parsing Problem Semantics Part of Speech Tagging NLP Trinity Morph Analysis
NLP Homework: Dependency Parsing with Feed-Forward Neural Network
NLP Homework: Dependency Parsing with Feed-Forward Neural Network Submission Deadline: Monday Dec. 11th, 5 pm 1 Background on Dependency Parsing Dependency trees are one of the main representations used
Lecture 9: Decoding. Andreas Maletti. Stuttgart January 20, Statistical Machine Translation. SMT VIII A. Maletti 1
Lecture 9: Decoding Andreas Maletti Statistical Machine Translation Stuttgart January 20, 2012 SMT VIII A. Maletti 1 Lecture 9 Last time Synchronous grammars (tree transducers) Rule extraction Weight training
Preliminaries. Introduction to EF-games. Inexpressivity results for first-order logic. Normal forms for first-order logic
Introduction to EF-games Inexpressivity results for first-order logic Normal forms for first-order logic Algorithms and complexity for specific classes of structures General complexity bounds Preliminaries
Lecture 3: Decidability
Lecture 3: Decidability January 11, 2011 Lecture 3, Slide 1 ECS 235B, Foundations of Information and Computer Security January 11, 2011 1 Review 2 Decidability of security Mono-operational command case
Models of Adjunction in Minimalist Grammars
Models of Adjunction in Minimalist Grammars Thomas Graf mail@thomasgraf.net http://thomasgraf.net Stony Brook University FG 2014 August 17, 2014 The Theory-Neutral CliffsNotes Insights Several properties
arxiv: v3 [cs.dm] 18 Oct 2017
![arxiv: v3 [cs.dm] 18 Oct 2017](/thumbs/76/73498857.jpg "arxiv: v3 [cs.dm] 18 Oct 2017")
Decycling a Graph by the Removal of a Matching: Characterizations for Special Classes arxiv:1707.02473v3 [cs.dm] 18 Oct 2017 Fábio Protti and Uéverton dos Santos Souza Institute of Computing - Universidade
Probabilistic Context-free Grammars
Probabilistic Context-free Grammars Computational Linguistics Alexander Koller 24 November 2017 The CKY Recognizer S NP VP NP Det N VP V NP V ate NP John Det a N sandwich i = 1 2 3 4 k = 2 3 4 5 S NP John
Language Technology. Unit 1: Sequence Models. CUNY Graduate Center. Lecture 4a: Probabilities and Estimations
Language Technology CUNY Graduate Center Unit 1: Sequence Models Lecture 4a: Probabilities and Estimations Lecture 4b: Weighted Finite-State Machines required hard optional Liang Huang Probabilities experiment
Relation between Graphs
Max Planck Intitute for Math. in the Sciences, Leipzig, Germany Joint work with Jan Hubička, Jürgen Jost, Peter F. Stadler and Ling Yang SCAC2012, SJTU, Shanghai Outline Motivation and Background 1 Motivation
Foundations of Informatics: a Bridging Course
Foundations of Informatics: a Bridging Course Week 3: Formal Languages and Semantics Thomas Noll Lehrstuhl für Informatik 2 RWTH Aachen University noll@cs.rwth-aachen.de http://www.b-it-center.de/wob/en/view/class211_id948.html
A Syntax-based Statistical Machine Translation Model. Alexander Friedl, Georg Teichtmeister
A Syntax-based Statistical Machine Translation Model Alexander Friedl, Georg Teichtmeister 4.12.2006 Introduction The model Experiment Conclusion Statistical Translation Model (STM): - mathematical model
Geometric Steiner Trees
Geometric Steiner Trees From the book: Optimal Interconnection Trees in the Plane By Marcus Brazil and Martin Zachariasen Part 3: Computational Complexity and the Steiner Tree Problem Marcus Brazil 2015
15.1 Matching, Components, and Edge cover (Collaborate with Xin Yu)
15.1 Matching, Components, and Edge cover (Collaborate with Xin Yu) First show l = c by proving l c and c l. For a maximum matching M in G, let V be the set of vertices covered by M. Since any vertex in
A Tabular Method for Dynamic Oracles in Transition-Based Parsing
A Tabular Method for Dynamic Oracles in Transition-Based Parsing Yoav Goldberg Department of Computer Science Bar Ilan University, Israel yoav.goldberg@gmail.com Francesco Sartorio Department of Information
Learning from Sensor Data: Set II. Behnaam Aazhang J.S. Abercombie Professor Electrical and Computer Engineering Rice University
Learning from Sensor Data: Set II Behnaam Aazhang J.S. Abercombie Professor Electrical and Computer Engineering Rice University 1 6. Data Representation The approach for learning from data Probabilistic
Natural Language Processing
Natural Language Processing Spring 2017 Unit 1: Sequence Models Lecture 4a: Probabilities and Estimations Lecture 4b: Weighted Finite-State Machines required optional Liang Huang Probabilities experiment
HOMEWORK #2 - MATH 3260
HOMEWORK # - MATH 36 ASSIGNED: JANUARAY 3, 3 DUE: FEBRUARY 1, AT :3PM 1) a) Give by listing the sequence of vertices 4 Hamiltonian cycles in K 9 no two of which have an edge in common. Solution: Here is
Logic: Propositional Logic Truth Tables
Logic: Propositional Logic Truth Tables Raffaella Bernardi bernardi@inf.unibz.it P.zza Domenicani 3, Room 2.28 Faculty of Computer Science, Free University of Bolzano-Bozen http://www.inf.unibz.it/~bernardi/courses/logic06
Attendee information. Seven Lectures on Statistical Parsing. Phrase structure grammars = context-free grammars. Assessment.
even Lectures on tatistical Parsing Christopher Manning LA Linguistic Institute 7 LA Lecture Attendee information Please put on a piece of paper: ame: Affiliation: tatus (undergrad, grad, industry, prof,
Chapter 3: Propositional Calculus: Deductive Systems. September 19, 2008
Chapter 3: Propositional Calculus: Deductive Systems September 19, 2008 Outline 1 3.1 Deductive (Proof) System 2 3.2 Gentzen System G 3 3.3 Hilbert System H 4 3.4 Soundness and Completeness; Consistency
Median orders of tournaments: a tool for the second neighbourhood problem and Sumner s conjecture.
Median orders of tournaments: a tool for the second neighbourhood problem and Sumner s conjecture. Frédéric Havet and Stéphan Thomassé Laboratoire LaPCS, UFR de Mathématiques, Université Claude Bernard
Tasks of lexer. CISC 5920: Compiler Construction Chapter 2 Lexical Analysis. Tokens and lexemes. Buffering
Tasks of lexer CISC 5920: Compiler Construction Chapter 2 Lexical Analysis Arthur G. Werschulz Fordham University Department of Computer and Information Sciences Copyright Arthur G. Werschulz, 2017. All
Outline. Logical Agents. Logical Reasoning. Knowledge Representation. Logical reasoning Propositional Logic Wumpus World Inference
Outline Logical Agents ECE57 Applied Artificial Intelligence Spring 007 Lecture #6 Logical reasoning Propositional Logic Wumpus World Inference Russell & Norvig, chapter 7 ECE57 Applied Artificial Intelligence
CS Lecture 29 P, NP, and NP-Completeness. k ) for all k. Fall The class P. The class NP
CS 301 - Lecture 29 P, NP, and NP-Completeness Fall 2008 Review Languages and Grammars Alphabets, strings, languages Regular Languages Deterministic Finite and Nondeterministic Automata Equivalence of
Probabilistic Graphical Models (I)
Probabilistic Graphical Models (I) Hongxin Zhang zhx@cad.zju.edu.cn State Key Lab of CAD&CG, ZJU 2015-03-31 Probabilistic Graphical Models Modeling many real-world problems => a large number of random
Dependency Parsing. Statistical NLP Fall (Non-)Projectivity. CoNLL Format. Lecture 9: Dependency Parsing
Dependency Parsing Statistical NLP Fall 2016 Lecture 9: Dependency Parsing Slav Petrov Google prep dobj ROOT nsubj pobj det PRON VERB DET NOUN ADP NOUN They solved the problem with statistics CoNLL Format
Automata, Logic and Games: Theory and Application
Automata, Logic and Games: Theory and Application 1. Büchi Automata and S1S Luke Ong University of Oxford TACL Summer School University of Salerno, 14-19 June 2015 Luke Ong Büchi Automata & S1S 14-19 June
CSCI 1010 Models of Computa3on. Lecture 17 Parsing Context-Free Languages
CSCI 1010 Models of Computa3on Lecture 17 Parsing Context-Free Languages Overview BoCom-up parsing of CFLs. BoCom-up parsing via the CKY algorithm An O(n 3 ) algorithm John E. Savage CSCI 1010 Lect 17
Polynomial Space. The classes PS and NPS Relationship to Other Classes Equivalence PS = NPS A PS-Complete Problem
Polynomial Space The classes PS and NPS Relationship to Other Classes Equivalence PS = NPS A PS-Complete Problem 1 Polynomial-Space-Bounded TM s A TM M is said to be polyspacebounded if there is a polynomial
Rough Sets. V.W. Marek. General introduction and one theorem. Department of Computer Science University of Kentucky. October 2013.
General introduction and one theorem V.W. Marek Department of Computer Science University of Kentucky October 2013 What it is about? is a popular formalism for talking about approximations Esp. studied
Natural Language Processing
SFU NatLangLab Natural Language Processing Anoop Sarkar anoopsarkar.github.io/nlp-class Simon Fraser University September 27, 2018 0 Natural Language Processing Anoop Sarkar anoopsarkar.github.io/nlp-class
Properties of context-free Languages
Properties of context-free Languages We simplify CFL s. Greibach Normal Form Chomsky Normal Form We prove pumping lemma for CFL s. We study closure properties and decision properties. Some of them remain,
Transition-based dependency parsing
Transition-based dependency parsing Daniël de Kok Overview Dependency graphs and treebanks. Transition-based dependency parsing. Making parse choices using perceptrons. Today Recap Transition systems Parsing
GENERALIZED PIGEONHOLE PROPERTIES OF GRAPHS AND ORIENTED GRAPHS
GENERALIZED PIGEONHOLE PROPERTIES OF GRAPHS AND ORIENTED GRAPHS ANTHONY BONATO, PETER CAMERON, DEJAN DELIĆ, AND STÉPHAN THOMASSÉ ABSTRACT. A relational structure A satisfies the n k property if whenever
Easy Shortcut Definitions
This version Mon Dec 12 2016 Easy Shortcut Definitions If you read and understand only this section, you ll understand P and NP. A language L is in the class P if there is some constant k and some machine
Computing if a token can follow
Computing if a token can follow first(b 1... B p ) = {a B 1...B p... aw } follow(x) = {a S......Xa... } There exists a derivation from the start symbol that produces a sequence of terminals and nonterminals
Latent Semantic Indexing (LSI) CE-324: Modern Information Retrieval Sharif University of Technology
Latent Semantic Indexing (LSI) CE-324: Modern Information Retrieval Sharif University of Technology M. Soleymani Fall 2016 Most slides have been adapted from: Profs. Manning, Nayak & Raghavan (CS-276,
Inference in Graphical Models Variable Elimination and Message Passing Algorithm
Inference in Graphical Models Variable Elimination and Message Passing lgorithm Le Song Machine Learning II: dvanced Topics SE 8803ML, Spring 2012 onditional Independence ssumptions Local Markov ssumption
The inefficiency of equilibria
The inefficiency of equilibria Chapters 17,18,19 of AGT 3 May 2010 University of Bergen Outline 1 Reminder 2 Potential games 3 Complexity Formalization Like usual, we consider a game with k players {1,...,
arxiv: v3 [cs.ds] 24 Jul 2018
![arxiv: v3 [cs.ds] 24 Jul 2018](/thumbs/87/95878176.jpg "arxiv: v3 [cs.ds] 24 Jul 2018")
New Algorithms for Weighted k-domination and Total k-domination Problems in Proper Interval Graphs Nina Chiarelli 1,2, Tatiana Romina Hartinger 1,2, Valeria Alejandra Leoni 3,4, Maria Inés Lopez Pujato
Unit 1: Sequence Models
CS 562: Empirical Methods in Natural Language Processing Unit 1: Sequence Models Lecture 5: Probabilities and Estimations Lecture 6: Weighted Finite-State Machines Week 3 -- Sep 8 & 10, 2009 Liang Huang
NP-Completeness. Until now we have been designing algorithms for specific problems
NP-Completeness 1 Introduction Until now we have been designing algorithms for specific problems We have seen running times O(log n), O(n), O(n log n), O(n 2 ), O(n 3 )... We have also discussed lower
arxiv: v1 [cs.ds] 26 Feb 2016
![arxiv: v1 [cs.ds] 26 Feb 2016](/thumbs/71/65629444.jpg "arxiv: v1 [cs.ds] 26 Feb 2016")
On the computational complexity of minimum-concave-cost flow in a two-dimensional grid Shabbir Ahmed, Qie He, Shi Li, George L. Nemhauser arxiv:1602.08515v1 [cs.ds] 26 Feb 2016 Abstract We study the minimum-concave-cost
Bayes Nets: Independence
Bayes Nets: Independence [These slides were created by Dan Klein and Pieter Abbeel for CS188 Intro to AI at UC Berkeley. All CS188 materials are available at http://ai.berkeley.edu.] Bayes Nets A Bayes
Hamiltonian paths in tournaments A generalization of sorting DM19 notes fall 2006
Hamiltonian paths in tournaments A generalization of sorting DM9 notes fall 2006 Jørgen Bang-Jensen Imada, SDU 30. august 2006 Introduction and motivation Tournaments which we will define mathematically
Learning Bayesian networks
1 Lecture topics: Learning Bayesian networks from data maximum likelihood, BIC Bayesian, marginal likelihood Learning Bayesian networks There are two problems we have to solve in order to estimate Bayesian
Logic: Top-down proof procedure and Datalog
Logic: Top-down proof procedure and Datalog CPSC 322 Logic 4 Textbook 5.2 March 11, 2011 Lecture Overview Recap: Bottom-up proof procedure is sound and complete Top-down Proof Procedure Datalog 2 Logical