Multi-dimensional Dependency Grammar as Multigraph Description

Transcription

1 Multi-dimensional Dependency Grammar as Multigraph Description Ralph Debusmann and Gert Smolka Programming Systems Lab Universität des Saarlandes Postfach 0 0 Saarbrücken, Germany {rade,smolka}@ps.uni-sb.de Abstract Extensible Dependency Grammar (XDG) is new, modular grammar formalism for natural langue. An XDG analysis is a multi-dimensional dependency graph, where each dimension represents a different aspect of natural langue, e.g. syntactic function, predicate-argument structure, information structure etc. Thus, XDG brings togeer two recent trends in computational linguistics: e increased application of ideas from dependency grammar and e idea of multi-layered linguistic description. n is paper, we tackle one of e stumbling blocks of XDG so far its incomplete formalization. We present e first complete formalization of XDG, as a description langue for multigraphs based on simply typed lambda calculus. ntroduction Extensible Dependency Grammar (XDG) (Debusmann et al. 00) brings togeer two recent trends from computational linguistics:. dependency grammar. multi-layered linguistic description Firstly, e ideas of dependency grammar, lexicalization, e head-dependent asymmetry, valency etc., have become more and more popular in computational linguistics. Most of e popular grammar formalisms like Combinatorial Categorial Grammar (CCG) (Steedman 000), Headdriven Phrase Structure Grammar (HPSG) (Pollard & S 99), Lexical Functional Grammar (LFG) (Bresnan 00) and Tree Adjoining Grammar (TAG) (Joshi 987) have already adopted ese ideas. Moreover, e most successful approaches statistical parsing crucially depend on notions from dependency grammar (Collins 999), and new treebanks based on dependency grammar are being developed for various langues, e.g. e Prue Dependency Treebank (PDT) for Czech and e TiGer Dependency Bank for German. Secondly, many treebanks such as e Penn Treebank, e TiGer Treebank and e PDT are continuously being extended wi additional layers of annotation in addition to e syntactic layer, i.e. ey become more and more multilayered. For example, e PropBank (Kingsbury & Palmer Copyright c 00, American Association for Artificial ntelligence ( All rights reserved. 00) (Penn Treebank), e SALSA project (Erk et al. 00) (TiGer Treebank) and e tectogrammatical layer (PDT) add a layer of predicate-argument structure. Oer added layers concern information structure (PDT) and discourse structure as in e Penn Discourse Treebank (Webber et al. 00). These additional layers of annotation are often dependencylike, i.e. could be straightforwardly represented in a framework for dependency grammar which is multi-layered. XDG is such a framework. t has already been successfully applied to model a relational syntax-semantics interface (Debusmann et al. 00) and to model e relation between prosodic structure and information structure in English (Debusmann, Postolache, & Traat 00). We hope to soon be able to employ XDG to directly make use of e information contained in e new multi-layered treebanks, e.g. for e automatic induction of multi-layered grammars for parsing and generation. To achieve is goal, XDG still needs to overcome a number of weaknesses. The first is e lack of a polynomial parsing algorim so far, we only have a parser based on constraint programming (Debusmann, Duchier, & Niehren 00), which is fairly efficient, given at e parsing problem is NP-hard, but does not scale up to large-scale grammars. The second major stumbling block of XDG so far is e lack of a complete formalization. The latter is what we will change in is paper: we will present a formalization of XDG as a description langue for multigraphs based on simply typed lambda calculus (Church 90; Andrews 00). To give a hint of e expressivity of XDG, we additionally present a proof at e parsing problem of (unrestricted) XDG is NP-hard. We begin e paper wi introducing e notion of multigraphs. Multigraphs Multigraphs are motivated by dependency grammar, and in particular by its structures: dependency graphs. Dependency Graphs Dependency graphs such as e one in Figure typically represent e syntactic structure of sentences in natural langue. They have e following properties:. Each node (round circle) is associated wi a word (, Peter, wants etc.), which is connected to e corresponding node by a dotted vertical line called projection edge, 70

2 for ese lines signify e projection of e nodes onto e sentence.. Each node is additionally associated wi an index (,, etc.) signifying its position in e sentence, and displayed above e words.. The nodes are connected to each oer by labeled and directed edges, which express syntactic relations. n e example, ere is an edge from node to node labeled adv, expressing at wants is modified by e adverb, and one from to labeled subj, expressing at Peter is e subject of wants. wants also has e infinitival verbal complement (edge label vinf), which has e particle (part) to and e object (obj) sphetti. adv syntax Peter wants to subj vinf semantics part obj pat sphetti Peter wants to sphetti adv subj vinf part obj Figure : Multigraph Peter wants to sphetti Figure : Dependency Graph (syntactic analysis) Dependency graphs are not restricted to describing syntactic structures alone. As an example in point, Figure shows a dependency graph which describes e semantic structure of our example sentence, where e adverb is e root and has e eme (edge label ) wants, which in turn has e ent () Peter and e eme. has e ent Peter and e patient (pat) sphetti. Node (e particle to) has no meaning and us remains unconnected. Peter wants to pat sphetti Figure : Dependency Graph (semantic analysis) Multigraphs A multigraph is a multi-dimensional dependency graph consisting of an arbitrary number of dependency graphs called dimensions. All dimensions share e same set of nodes. Multigraphs can significantly simplify linguistic modeling by modularizing bo e structures and eir description. We show an example multigraph in Figure, where we simply draw e dimensions of syntax (upper half, from Figure ) and semantics (lower half, from Figure ) as individual dependency graphs for clarity, and indicate e node sharing by arranging shared nodes in e same columns. The multigraph simultaneously describes e syntactic and semantic structures of e sentence, and expresses e.g. at Peter (node ), e subject of wants, syntactically realizes bo e ent of wants and of (node ). Formalization We formalize multigraphs as tuples (V, Dim, Word, W, Lab, E, Attr, A) of a finite set V of nodes, a finite set Dim of dimensions, a finite set Word of words, e node-word mapping W V Word, a finite set Lab of edge labels, a set E V V Dim Lab of edges, and in addition a finite set Attr of attributes and e node-attributes mapping A V Dim Attr to equip e nodes wi extra information (not used in e example above). The set of nodes V must be a finite interval of e natural numbers starting from, i.e., given n nodes, V = {,..., n}, which induces a strict total order on V. Relations We associate each dimension d Dim wi four relations: ) e labeled edge relation ( d ), ) e edge relation ( d ), ) e dominance relation ( + d ), and ) e precedence relation ( ). Labeled Edge. Given two nodes v and v and a label l Lab, e labeled edge relation v d v holds iff ere is an edge from v to v labeled l on dimension d: l d = {(v, v, l) (v, v, d, l) E} () Edge. Given two nodes v and v, e edge relation v d v holds iff ere is an edge from v to v on dimension d labeled by any label in Lab: d = {(v, v ) l Lab : v l d v } () Dominance. The (strict, non-reflexive) dominance relation + d is e transitive closure of e edge relation d. Precedence. Given two nodes v and v, e precedence relation v v holds iff v < v, where < is e total order on e natural numbers. Note at here, formally, a dimension is just a name identifying a particular dependency graph in e multigraph. Conceptually, a dimension denotes not just e name but e dependency graph itself. 7

3 A Description Langue for Multigraphs We proceed wi formalizing Extensible Dependency Grammar (XDG) as a description langue for multigraphs based based on simply typed lambda calculus. Types Definition. We define e types T Ty given a set At of atoms (arbitrary symbols): a At T Ty ::= B boolean V node T T function {a,..., a n } finite domain {a : T,..., a n : T n } record () where n and for finite domains and records, a,..., a n are pairwise distinct. Following (Church 90; Andrews 00), we adopt e classical semantics for lambda calculus and us forbid empty finite domain types. nterpretation. B as {0, } We interpret: V as a finite interval of e natural numbers from T T as e set of all functions from e interpretation of T to e interpretation of T {a,..., a n } as e set {a,..., a n } {a : T,..., a n : T n } as e set of all functions f wi. Dom f = {a,..., a n }. for all i n, f a i is an element of e interpretation of T i Multigraph Type Multigraphs can be distinguished according to eir dimensions, words, edge labels and attributes. This leads us to e definition of a multigraph type, which we define given e types Ty as a tuple M = (dim, word, lab, attr), where. dim Ty is a finite domain of dimensions. word Ty is a finite domain of words. lab dim Ty is a function from dimensions to label types (finite domains), i.e. e type of e edge labels on at dimension. attr dim Ty is a function from dimensions to attributes types (any type in Ty), i.e. e type of e attributes on at dimension Writing M T for e interpretation of type T over M, a multigraph M = (V, Dim, Word, W, Lab, E, Attr, A) has multigraph type M = (dim, word, lab, attr) iff. The dimensions are e same:. The words are e same: Dim = M dim () Word = M word (). The edges in E have e right edge labels for eir dimension (according to lab): (v, v, d, l) E : l M (lab d) (). The nodes have e right attributes for eir dimension (according to attr): v V : d Dim : (A v d) M (attr d) (7) Terms The terms of XDG augment simply typed lambda calculus wi atoms, records and record selection. Given a set of constants Con, we define: a At c Con t ::= x variable c constant λx : T.t abstraction t t application a atom {a = t,..., a n = t n } record t.a record selection where for records, a,..., a n are pairwise distinct. Signature An XDG signature is determined by a multigraph type M = (dim, word, lab, attr), and consists of two parts: e logical constants and e multigraph constants. Logical Constants. The logical constants include e type constant B and e following term constants: 0 : B false : B true : B B negation : B B B disjunction. = T : T T B equality T : (T B) B existential quantification For convenience, we introduce e usual logical constants,,,, T and T as notation. Multigraph Constants. The multigraph constants include e type constant V, and e following term constants: d : V V lab d B labeled edge d : V V B edge + d : V V B dominance : V V B precedence (word ) : V word word (d ) : V attr d attributes (0) where we interpret d as e labeled edge relation on dimension d. d as e edge relation on d. (8) (9) 7

4 + d as e dominance relation on d. as e precedence relation (word ) as e word (d ) as e attributes on d. Grammar An XDG grammar G = (M, P ) is defined by a multigraph type M and a set P of formulas called principles. Each principle must be formulated according to e signature M. Models The models of a grammar G = (M, P ) are all multigraphs which. have multigraph type M. satisfy all principles P Recognition Problem We distinguish two kinds of recognition problems:. e universal recognition problem. e fixed recognition problem Universal Recognition Problem. Given an XDG grammar G wi words word and a string s = w... w n in word +, e universal recognition problem (G, s) is e problem of determining wheer ere is a model of G such at:. ere are as many nodes as words: V = {,..., n} (). e concatenation of e words of e nodes yields s: (word )... (word n) = s () Fixed Recognition Problem. The fixed recognition problem (G, s) poses e same question as e universal recognition problem, but wi e restriction at for all input strings s, e grammar G must remain fixed. Complexity What is e complexity of e two kinds of recognition problems? n is section, we prove at e fixed recognition problem is NP-hard. The purpose of e proof is to give e reader a feeling for e expressivity of XDG. Fixed Recognition Problem Proof. To prove at e fixed string membership problem is NP-hard, we opt for e reduction of e NP-complete problem of SAT (satisfiability), which is e problem of deciding wheer a formula in propositional logic has an assignment at evaluates to true. We restrict ourselves to formulas f, which is already sufficient to cover full propositional logic: f ::= X, Y, Z,... variable 0 false f f implication The reduction of SAT proceeds as follows: (). n ree steps, we transform e propositional formula into a string suitable as input to e fixed string membership problem. For example, given e formula (X 0) Y () e transformation goes along as follows: (a) To avoid ambiguity, we transform e formula into prefix notation: X 0 Y () (b) A propositional formula can contain an arbitrary number of variables, yet e domain of words of an XDG grammar must be finite. To overcome is limitation, we adopt a unary encoding for variables, where we encode e first variable from e left of e formula (here: X) as var, e second (here: Y ) var etc.: var 0 var () (c) To clearly distinguish e string from e original propositional formula, we replace all implication symbols wi e word impl: impl impl var 0 var (7). We introduce e Propositional Logic dimension (abbreviation: ) to model e structure of e propositional formula. On e dimension, e example formula () is analyzed as in Figure. arg arg bar arg arg impl impl var 0 var { } { } { }{ } { }{ } { } { } tru= tru=0 tru= tru=0 tru=0 tru= tru=0 tru=0 bars= bars= bars= bars= bars= bars= bars= bars= bar bar Figure : analysis of (X 0) Y The edge labels are arg and arg for e antecedent and e consequent of an implication, respectively, and bar for connecting e bars (word ) of e unary variable encoding. Below e words of e nodes, we display eir attributes, which are of e following type: { } tru : B (8) bars : V where tru represents e tru value of e node and bars e number of bars below e next node to its right plus. The bars attribute will become crucial for establishing coreferences between variables. ts type is V for two reasons: (a) There are always less (or equally many) variables in a formula an ere are nodes, since every encoded formula contains less (or equally many) variables an words. Hence, V always suffices to distinguish em. (b) We require e precedence predicate to implement incrementation (cf. 9. below), which is defined only on V

5 . We require at e models on are trees. n XDG, we can express is as follows: (a) ere are no cycles: (v + v) (9) (b) each node has at most one incoming edge: v v v v v. = v (0) (c) ere is precisely one node wi no incoming edge (e root): v : v : v v (). We describe e structure of e tree by, for each node, depending on its word, constraining ) its incoming edges, ) its outgoing edges and ) its order wi respect to its daughters and e order among e daughters: (a) A node wi word impl ) can eier be e antecedent or e consequent of an implication (its incoming edge must be labeled eier arg or arg), ) requires precisely one outgoing edge labeled arg (for e antecedent) and one labeled arg (for e consequent) and must have no oer outgoing edges, and ) must precede its arg-daughter, which in turn must precede e arg-daughter. We illustrate is below in (), where e question mark? marks optional and e exclamation mark! obligatory edges: arg?, arg? impl arg! arg! We can express ese ree constraints in XDG as: (word v) =. impl ) v l v l =. arg l =. arg ) v : v arg v v : v arg v () v : v bar v ) v arg v v arg v v v v v () (b) A node wi word 0 can ) eier be e antecedent or e consequent of an implication and ) must not have any outgoing edges: arg?, arg? 0 () (c) A node wi word var ) can eier be e antecedent or e consequent of an implication, ) requires only precisely one outgoing edge labeled bar for e first bar below it, and ) precedes its bar-daughter: arg?, arg? var bar! () (d) A node wi word ) must have an incoming edge labeled bar, ) can have only at most one outgoing edge labeled bar, and ) precedes its potential bar-daughter: bar! bar? (). Just ordering e nodes is not enough: to precisely capture e context-free structure of e propositional formula and e unary variable encoding, we must ensure at e models are projective, i.e. at no edge crosses any of e projection edges. Oerwise, we are faced wi wrong analyses as in Figure, where e rightmost bar (node 8) is incorrectly taken over by e first bar (node ) of e left variable (node ). arg arg arg arg bar impl impl var 0 var { } { } { }{ } { }{ } { } { } tru= tru=0 tru= tru=0 tru=0 tru=0 tru=0 tru=0 bars= bars= bars= bars= bars= bars= bars= bars= Figure : Non-projective analysis of (X 0) Y We express e projectivity constraint as follows in XDG: for each node v and each daughter v, all nodes v between v and v must be below v: v v v v v : v v v v v + v v v v v v : v v v v v + v (7). We must ensure at e root of analysis always has tru value, i.e. at e propositional formula is true: v : v v ( v).tru. = (8) 7. Nodes wi word 0 have tru value false. Their bar count is not relevant, hence we can pick an arbitrary value and set it to : (word v) =. 0 ( v).tru =. 0 ( v).bars =. (9) 8. The tru value of nodes wi word impl equals e implication of e tru value of its arg-daughter (e antecedent) and its arg-daughter (e consequent). Again e bar count is not relevant and we set it to : (word v) =. impl (v arg v v arg v ( v).tru =. (( v ).tru ( v ).tru)) ( v).bars =. (0) Note at while XDG dimensions can be constrained to be projective, ey need not be. bar bar 7 8 7

6 9. The tru value of bars (word ) is not relevant, and hence we can safely set it to 0. The bar value is eier for e bars not having a daughter, or else one more an its daughter: (word v) =. ( v).tru =. 0 v : v v ( v).bars =. (v bar v ( v ).bars ( v).bars v : ( v ).bars v v ( v).bars) () Notice at e latter constraint actually increments e bar value, even ough XDG does not provide us wi any direct means to do at. The trick is to emulate incrementing using e precedence predicate. 0. The tru value of variables is only constrained by e overall propositional formula. Their bar value is e same as at of eir bar daughter. (word v). = var v bar v ( v).bars. = ( v ).bars (). We can now establish coreferences between e variables. To is end, we stipulate at for each pair of nodes v and v wi word var, if ey have e same bar values, en eir tru values must be e same: (word v). = var (word v ). = var ( v).bars. = ( v ).bars ( v).tru. = ( v ).tru () The described reduction is polynomial, which concludes e proof at e fixed string membership problem is NP-hard. Universal Recognition Problem Proof. Since e fixed recognition problem is a specialization of e universal recognition problem, e NP-hardness result carries over: e universal recognition problem of XDG is also NP-hard. Conclusion n is paper, we have presented e first complete formalization of XDG as a description langue for multigraphs based on simply typed lambda calculus. We also showed at e recognition problem of XDG as it stands is NP-hard. But all is not lost: on e one hand, we already have an implementation of XDG as a constraint satisfaction problem which is fairly efficient for handcrafted grammars at least. Also, oer grammar formalisms such as LFG (Barton, Berwick, & Ristad 987) and HPSG (wi significant restrictions) (Trautwein 99) are also NP-hard but still used for parsing in practice. On e oer hand, e formalization is meant to be a starting point for future research on e formal aspects of XDG, wi e eventual goal to find more tractable, polynomial frments, wiout losing e potential to significantly modularize and us improve e modeling of natural langue. Acknowledgments We would like to ank e oer members of e CHO- RUS project (Alexander Koller, Marco Kuhlmann, Manfred Pinkal, Stefan Thater) for discussion, and also Denys Duchier and Joachim Niehren. This work was funded by e DFG (SFB 78) and e nternational Post-Graduate College in Langue Technology and Cognitive Systems (GK) in Saarbrücken. References Andrews, P. B. 00. An ntroduction to Maematical Logic and Type Theory: To Tru Through Proof. Kluwer Academic Publishers. Barton, G. E.; Berwick, R.; and Ristad, E. S Computational Complexity and Natural Langue. MT Press. Bresnan, J. 00. Lexical Functional Syntax. Blackwell. Church, A. 90. A Formulation of e Simple Theory of Types. Journal of Symbolic Logic (): 8. Collins, M Head-Driven Statistical Models for Natural Langue Parsing. Ph.D. Dissertation, University of Pennsylvania. Debusmann, R.; Duchier, D.; Koller, A.; Kuhlmann, M.; Smolka, G.; and Thater, S. 00. A Relational Syntax- Semantics nterface Based on Dependency Grammar. n Proceedings of COLNG 00. Debusmann, R.; Duchier, D.; and Niehren, J. 00. The XDG Grammar Development Kit. n Proceedings of e MOZ0 Conference, volume 89 of Lecture Notes in Computer Science, Charleroi/BE: Springer. Debusmann, R.; Postolache, O.; and Traat, M. 00. A Modular Account of nformation Structure in Extensible Dependency Grammar. n Proceedings of e CCLNG 00 Conference. Mexico City/MX: Springer. Erk, K.; Kowalski, A.; Pado, S.; and Pinkal, M. 00. Towards a Resource for Lexical Semantics: A Large German Corpus wi Extensive Semantic Annotation. n Proceedings of ACL 00. Joshi, A. K An ntroduction to Tree-Adjoining Grammars. n Manaster-Ramer, A., ed., Maematics of Langue. Amsterdam/NL: John Benjamins. 87. Kingsbury, P., and Palmer, M. 00. From Treebank to PropBank. n Proceedings of LREC-00. Pollard, C., and S,. A. 99. Head-Driven Phrase Structure Grammar. Chico/US: University of Chico Press. Steedman, M The Syntactic Process. Cambridge/US: MT Press. Trautwein, M. 99. The complexity of structure sharing in unification-based Grammars. n Daelemans, W.; Durieux, G.; and Gillis, S., eds., Computational Linguistics in e Neerlands 99, 79. Webber, B.; Joshi, A.; Miltsakaki, E.; Prasad, R.; Dinesh, N.; Lee, A.; and Forbes, K. 00. A Short ntroduction to e Penn Discourse TreeBank. Technical report, University of Pennsylvania. 7