Context-free grammars for natural languages

Transcription

1 Context-free grammars Basic definitions Context-free grammars for natural languages Context-free grammars can be used for a variety of syntactic constructions, including some non-trivial phenomena such as unbounded dependencies, extraction, extraposition etc. However, some (formal) languages are not context-free, and therefore there are certain sets of strings that cannot be generated by context-free grammars. The interesting question, of course, involves natural languages: are there natural languages that are not context-free? Are context-free grammars sufficient for generating every natural language? c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 1 / 300

2 Context-free grammars Basic definitions A context-free grammar, G 0, for E 0 Example A context-free grammar, G 0, for E 0 S NP VP VP V VP V NP NP D N NP Pron NP PropN D the, a, two, every,... N sheep, lamb, lambs, shepherd, water... V sleep, sleeps, love, loves, feed, feeds, herd, herds,... Pron I, me, you, he, him, she, her, it, we, us, they, them PropN Rachel, Jacob,... c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 2 / 300

3 Context-free grammars Basic definitions Context-free grammars for natural languages There are two major problems with this grammar. 1 it ignores the valence of verbs: there is no distinction among subcategories of verbs, and an intransitive verb such as sleep might occur with a noun phrase complement, while a transitive verb such as love might occur without one. In such a case we say that the grammar overgenerates: it generates strings that are not in the intended language. 2 there is no treatment of subject verb agreement, so that a singular subject such as the cat might be followed by a plural form of verb such as smile. This is another case of overgeneration. Both problems are easy to solve. c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 3 / 300

4 Context-free grammars Basic definitions Problems of G 0 Over-generation (agreement constraints are not imposed): Rachel feed the sheep The shepherds feeds the sheep Rachel feeds Jacob loves she Them herd the sheep c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 4 / 300

5 Context-free grammars Basic definitions Problems of G 0 Over-generation (subcategorization constraints are not imposed): the lambs sleep Jacob loves Rachel the lambs sleep the sheep Jacob loves c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 5 / 300

6 Context-free grammars Basic definitions Problems of G 0 Example (Over-generation) S NP VP D N V NP Pron the lambs sleeps they c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 6 / 300

7 Context-free grammars Basic definitions Verb valence To account for valence, we can replace the non-terminal symbol V by a set of symbols: Vtrans, Vintrans, Vditrans etc. Example We must also change the grammar rules accordingly: VP Vintrans VP Vtrans NP VP Vditrans NP NP Vintrnas sleep, sleeps Vtrans love, loves Vditrans give, gives c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 7 / 300

8 Context-free grammars Basic definitions Agreement To account for agreement, we can again extend the set of non-terminal symbols such that categories that must agree reflect in the non-terminal that is assigned for them the features on which they agree. In the very simple case of English, it is sufficient to multiply the set of nominal and verbal categories, so that we get Dsg, Dpl, Nsg, Npl, NPsg, NPpl, Vsg, Vlp, VPsg, VPpl etc. We must also change the set of rules accordingly: c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 8 / 300

9 Context-free grammars Basic definitions Agreement Example Nsg lamb Nsg sheep Vsg sleeps Vsg smiles Vsg loves Vsg saw Dsg a Npl lambs Npl sheep Vpl sleep Vpl smile Vpl love Vpl saw Dpl two c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 9 / 300

10 Context-free grammars Basic definitions Agreement Example S NPsg VPsg NPsg Dsg Nsg VPsg Vsg VPsg VPsg NP S NPpl VPpl NPpl Dpl Npl VPpl Vpl VPpl VPpl NP c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 10 / 300

11 Context-free grammars Basic definitions Methodological properties of the CFG formalism 1 Concatenation is the only string combination operation 2 Phrase structure is the only syntactic relationship 3 The terminal symbols have no properties 4 Non-terminal symbols (grammar variables) are atomic 5 Most of the information encoded in a grammar lies in the production rules 6 Any attempt of extending the grammar with a semantics requires extra means. c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 11 / 300

12 Context-free grammars Basic definitions Alternative methodological properties 1 Concatenation is not necessarily the only way by which phrases may be combined to yield other phrases. 2 Even if concatenation is the sole string operation, other syntactic relationships are being put forward. 3 Modern computational formalisms for expressing grammars adhere to an approach called lexicalism. 4 Some formalisms do not retain any context-free backbone. However, if one is present, its categories are not atomic. 5 The expressive power added to the formalisms allows also a certain way for representing semantic information. c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 12 / 300

13 Feature structures Introduction Feature structures Motivated by the violations of the context-free grammar G 0, we would like to extend the CFG formalism with additional mechanisms that will facilitate the expression of information that is missing in G 0 in a uniform and compact way. The core idea is to incorporate into the grammar properties of symbols, in terms of which the violations of G 0 were stated. Properties are represented by means of feature structures. c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 13 / 300

14 Feature structures Introduction Overview An overview of feature structures, motivating their use as a representation of linguistic information Four different views of these entities: feature graphs feature structures attribute-value matrices (AVMs) Feature structures in a broader context. c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 14 / 300

15 Feature structures Motivation Motivation Words in natural languages have properties We want to model these properties in the lexicon We would like to associate with words not just atomic symbols, as in CFGs, but rather structural information that reflects their properties. c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 15 / 300

16 Feature structures Motivation A simple lexicon Example (A simple lexicon) lamb: I: dreams: [ ] num : sg pers : third [ ] num : sg pers : first [ ] num : sg pers : third lambs: sheep: dreams: [ ] num : pl pers : third [ ] num : [ ] pers : third [ ] num : sg pers : third c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 16 / 300

17 Feature structures Motivation Feature structures Feature structures map features into values, which are themselves feature structures A special case of feature structures are atoms, which represent structureless values. For example, to deal with number (and impose its agreement), we use a feature num, and a set of atomic feature structures {sg,pl} as its values, representing singularity and plurality, respectively. When a value is not atomic, it is complex. A complex value is, recursively, a feature structure consisting of features and values. c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 17 / 300

18 Feature structures Motivation A complex feature structure Example (A complex feature structure) loves: vtype : transitive agr : [ num : sg pers : third ] c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 18 / 300

19 Feature structures Motivation Grouping features Deciding how to group features is up to the grammar designer, and is intended to capture syntactic generalizations. If number and person go together in formulating restrictions, it is more appropriate to group them as in this example. Moreover, such a grouping might be beneficial when feature structures are being modified. Processes of derivation and parsing (the application of grammar rules) are able to manipulate feature structures to reflect application of such constraints. When the properties of some feature structure are changed, it is possible to change the value of only one feature, namely agr, rather than specify two separate changes for each subfeature. c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 19 / 300

20 Feature structures Motivation Grouping features In the example lexicon, the lexical ambiguity of sheep is represented by an empty feature structure as the value of the num feature. This is interpreted as the value of this feature being unconstrained. However, it would have been useful to be able to state that the only possible values for this feature are, say, sg and pl. There are at least two different ways to specify such information: by listing a set of values for the feature; or by restricting its value to a certain type of permissible values. We do not explore the former solution here. The latter solution is employed by typed feature structure formalisms. c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 20 / 300

21 Feature structures Motivation Adding features to phrases Words are not the only linguistic entities that have properties; words are combined into phrases, and those also have properties which can be modeled by feature:value pairs. For example, the noun phrase a sheep has the value sg for the num feature, while two sheep has the value pl for num. Consequently, grammar non-terminals, too, must be decorated with features, representing the endowment of phrases of this category with that feature. c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 21 / 300

22 Feature structures Feature graphs Feature graphs The informal discussion of feature structures above depicted them using a representation, called attribute-value matrices (AVMs), which is common in the linguistic literature. We begin the discussion of feature structures by defining the concept of feature graphs, using well-known concepts of graph theory. A graph view of feature structures facilitates computational processing because so many properties of graphs are well understood and because graphs lend themselves to efficient processing. We will return to AVMs and discuss their correspondence with feature graphs later on. c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 22 / 300

23 Feature structures Feature graphs Definitions Feature graphs are defined over a signature consisting of non-empty, finite, disjoint sets Feats of features and Atoms of atoms. Features are used to encode properties of (linguistic) objects, such as number, gender etc. Atoms are used for the (atomic) values of such features, as in plural, feminine etc. We use a convention of depicting features in small capitals and atoms in italics. c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 23 / 300

24 Feature structures Feature graphs Signature Definition (Signature) A signature is a structure S = Atoms, Feats, where Atoms is a finite set of atoms and Feats is a finite set of features. We assume some fixed signature throughout this presentation. Meta-variables f,g (with or without subscripts or superscripts) range over features, and a,b, etc. over atoms. We usually assume that both Feats and Atoms are non-empty (and sometimes even assume that they include more than one element each). c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 24 / 300

25 Feature structures Feature graphs Feature graphs Definition (Feature graphs) A feature graph A = Q A, q A,δ A,θ A is a finite, directed, connected, labeled graph consisting of a finite, nonempty set of nodes Q A (such that Q A Feats = Q A Atoms = ), a root q A Q A, a partial function δ A : Q A Feats Q A specifying the arcs such that every node q Q A is accessible from q A, and a partial function, marking some of the sinks: θ A : Q S Atoms, where Q S = {q Q A δ A (q,f ) for every f }. Given a signature of features Feats and atoms Atoms, let G(Feats,Atoms) be the set of all feature graphs over the signature. c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 25 / 300

26 Feature structures Feature graphs Feature graphs Example (Feature graphs) The graph displayed below is Q, q,δ,θ, where Q = {q 0,q 1,q 2,q 3 }, q = q 0,δ(q 0,agr) = q 1,δ(q 1,num) = q 2,δ(q 1,pers) = q 3,Q S = {q 2,q 3 },θ(q 2 ) = pl,θ(q 3 ) = third. num q 0 agr q 1 pers q 2 pl q 3 third c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 26 / 300

27 Feature structures Feature graphs Feature graphs The arcs of a feature graph are thus labeled by features. The root is a designated node from which all other nodes are accessible (through δ); note that nothing prevents the root from having incoming arcs. Sink nodes (nodes with no outgoing edges) can be marked by an atom, but can also be unmarked. c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 27 / 300

28 Feature structures Feature graphs Feature graphs We use meta-variables A, B (with or without subscripts) to refer to feature graphs. We use Q, q,δ,θ, to refer to constituents of feature graphs. When displaying feature graphs, the root is depicted as a grey-colored node, usually at the top or the left side of the graph. The identities of the nodes are arbitrary, and we use generic names such as q 0,q 1 etc. to refer to them. c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 28 / 300

29 Feature structures Feature graphs Feature graphs Example (Feature graphs) In the following graph, the leaves q 2 and q 3 bear no marking; in other words, the marking function θ is undefined for the two sinks in its domain. q 2 num agr q 0 q 1 pers q3 The graph displayed above is Q, q,δ,θ, where Q = {q 0,q 1,q 2,q 3 }, q = q 0,δ(q 0,agr) = q 1,δ(q 1,num) = q 2,δ(q 1,pers) = q 3,Q S = {q 2,q 3 }, and θ is undefined for its entire domain. c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 29 / 300

30 Feature structures Feature graphs Feature graphs A feature graph is empty if it consists of a single unmarked node with no arcs. A feature graph is atomic if it consists of a single marked node with no arcs. c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 30 / 300

31 Feature structures Feature graphs Empty and atomic feature graphs Example (Empty and atomic feature graphs) A, an empty feature graph: q 0 B, an atomic feature graph: q 0 pl c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 31 / 300

32 Feature structures Feature graphs Paths The concept of paths is natural when graphs are concerned. A path (over Feats) is a finite sequence of features, and the set Paths = Feats is the collection of all paths. Meta-variables π,α (with or without subscripts) range over paths. ǫ is the empty path, denoted also by. The length of a path π is denoted π. For example, if Feats = {a, b} then Paths includes ǫ, a, b, a,b,a, b,b,b,b,a,b, etc. While a path is a purely syntactic notion (every sequence of features constitutes a path), interesting paths are those that can be interpreted as actual paths in some graph, leading from the root to some node. c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 32 / 300

33 Feature structures Feature graphs Paths The definition of δ is therefore extended to paths: given a feature graph A = Q A, q A,δ A,θ A, define ˆδ A : Q A Paths Q A as follows: ˆδ A (q,ǫ) = q ˆδ A (q,f π) = ˆδ A (δ A (q,f ),π) (defined only if δ A (q,f ) ) Since for every node q Q A and every feature f Feats, δ A (q,f) = ˆδ A (q, f ), we identify ˆδ with δ in the future and use only the latter. When the index (A) is clear from the context, it is omitted. When δ A (q,π) = q we say that π leads (in A) from q to q. c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 33 / 300

34 Feature structures Feature graphs Paths Definition (Paths) The paths of a feature graph A are Π(A) = {π Paths δ A ( q A,π) }. c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 34 / 300

35 Feature structures Feature graphs Paths Example (Paths) Consider the following feature graph, A: num q 0 agr q 1 pers q 2 pl q 3 third Its paths are Π(A) = {ǫ, agr, agr num, agr pers } c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 35 / 300

36 Feature structures Feature graphs Path values Of particular interest are paths which lead from the root of a feature graph to some node in the graph. For such paths we define the notion of a value, which is the sub-graph whose root is the node at the end of the path. It would have been possible to define as value the node itslef, rather than the sub-graph is induces; the choice is a matter of taste, as moving from one view of values to another is trivial. c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 36 / 300

37 Feature structures Feature graphs Path values Definition (Path value) For a feature graph A = Q A, q A,δ A,θ A and a path π Π(A), the value val A (π) of π in A is a feature graph B = Q B, q B,δ B,θ B, over the same signature as A, where: q B = δ A ( q A,π) Q B = {q Q A for some π, δ A ( q B,π ) = q } (Q B is the set of nodes reachable from q B ) for every feature f and for every q Q B, δ B (q,f) = δ A (q,f ) (δ B is the restriction of δ A to Q B ) for every q Q B, θ B (q ) = θ A (q ) (θ B is the restriction of θ A to Q B ) c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 37 / 300

38 Feature structures Feature graphs Paths Example (Paths) Consider the following feature graph, A: num q 0 agr q 1 pers q 2 pl q 3 third Its paths are Π(A) = {ǫ, agr, agr num, agr pers } c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 38 / 300

39 Feature structures Feature graphs Path values Example (Path values) The value of the path agr in A is: val A ( agr ) = q 1 num pers q 2 pl q 3 third and the value of the path agr num in A is: val A ( agr num ) = q 2 pl Note that, for example, the value of agr pers num in A is undefined. c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 39 / 300

40 Feature structures Feature graphs Reentrancy The definition of path values raises the question of when two paths have equal values. We distinguish between paths which lead to one and the same node, and those whose values are isomorphic but not identical. The former case is called reentrancy. c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 40 / 300

41 Feature structures Feature graphs Reentrancy Definition (Reentrancy) Let A = Q, q,δ,θ be a feature graph. Two paths π 1,π 2 Π(A) are reentrant in A, denoted π 1 A π2, iff δ( q,π 1 ) = δ( q,π 2 ), implying val A (π 1 ) = val A (π 2 ). A feature graph A is reentrant iff there exist two distinct paths π 1,π 2 Π(A) such that π 1 A π2. c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 41 / 300

42 Feature structures Feature graphs Reentrancy Example (A reentrant feature graph) This feature graph, A, is reentrant because δ A (q 0, agr ) = δ A (q 0, subj,agr ) num agr q 0 q 1 subj q 4 agr pers q 2 pl q 3 third The (single) value of the (different) paths agr and subj agr in A is: q 1 num pers q 2 pl q 3 third c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 42 / 300

43 Feature structures Feature graphs Reentrancy The notion of reentrancy touches on the issue of the distinction between type- and token-identity. Two feature graphs are token identical if their components (i.e., their sets of nodes, roots, transition functions and atom marking functions) are identical. They are type-identical if they are isomorphic, not necessarily requiring their nodes to be identical. We will discuss feature graph isomorphism later. c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 43 / 300

44 Feature structures Feature graphs Cicles Early feature structure based formalisms used to employ only acyclic feature graphs. However, modern ones usually allow (or even require) feature structures to be possibly cyclic. While the linguistic motivation for cyclic feature structures is limited, there is good practical motivation for allowing them: when implementing a system for manipulating feature graphs, it is usually easier to support cycles than to guarantee that all the graphs in a system are acyclic. The reason is that unification, which is the major operation defined on feature graphs, can yield a cyclic graph even when its operands are acyclic. c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 44 / 300

45 Feature structures Feature graphs Cicles Definition (Cycles) A feature graph A = Q A, q A,δ A,θ A is cyclic if two paths π 1,π 2 Π(A), where π 1 is a proper subsequence of π 2, are reentrant: π 1 A π2. A is acyclic otherwise. Note that cyclicity is a special case of reentrancy (every cyclic feature graph is reentrant, but not vice versa). A corollary of the definition is that when a feature graph is cyclic, it has at least one node q such that δ(q,α) = q for some non-empty path α. c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 45 / 300

46 Feature structures Feature graphs Cicles Example (A cyclic feature graph) Following is a cyclic feature graph, C: h f q 0 q 1 q 2 g a The value of the path f in C, as well as the values of the (infinitely many) paths f h n, for n 0, is the same feature graph: h q 1 g q 2 a c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 46 / 300

47 Feature structures Feature graph subsumption Feature graph isomorphism Since feature graphs are just a special case of directed, labeled graphs, we can adapt the well-defined notion of graph isomorphism to feature graphs. Informally, two graphs are isomorphic when they have the same structure; the identites of their nodes may differ without affecting the structure. In our case, we require also that the labels of sink nodes be identical in order for two graphs to be considered isomorphic. c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 47 / 300

48 Feature structures Feature graph subsumption Feature graph isomorphism Definition (Feature graph isomorphism) Two feature graphs A = Q A, q A,δ A,θ A and B = Q B, q B,δ B,θ B are isomorphic, denoted A B, iff there exists a one-to-one and onto mapping i : Q A Q B, called an isomorphism, such that: i( q A ) = q B ; for all q 1,q 2 Q A and f Feats, δ A (q 1,f ) = q 2 iff δ B (i(q 1 ),f ) = i(q 2 ); and for all q Q A, θ A (q) = θ B (i(q)) (either both are undefined, or both are defined and equal). c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 48 / 300

49 Feature structures Feature graph subsumption Feature graph subsumption Definition (Subsumption) Let A 1 = Q 1, q 1,δ 1,θ 1 and A 2 = Q 2, q 2,δ 2,θ 2 be two feature graphs. A 1 subsumes A 2 (denoted by A 1 A 2 ) iff there exists a total function h : Q 1 Q 2, called a subsumption morphism, such that h( q 1 ) = q 2 for every q Q 1 and for every f such that δ 1 (q,f ), h(δ 1 (q,f )) = δ 2 (h(q),f ) for every q Q 1, if θ 1 (q) then θ 1 (q) = θ 2 (h(q)). If A 1 A 2 then A 1 is said to subsume, or be more general than A 2 ; A 2 is subsumed by, or is more specific than, A 1. c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 49 / 300

50 Feature structures Feature graph subsumption Subsumption The morphism h associates with every node in Q 1 a node in Q 2 ; if an arc labeled f connects q with q, then such an arc connects h(q) with h(q ). In other words, δ and h commute, as depicted in the following diagram, where δ-arcs are depicted using solid lines, whereas h-mappings are depicted using dashed lines: h δ : f f h c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 50 / 300

51 Feature structures Feature graph subsumption Subsumption In addition, if a node q Q 1 is marked by an atom, then its image h(q) must be marked by the same atom (recall that only sinks can be thus marked). Note that if a sink in Q 1 is not marked, there is no constraint on its image (in particular, it can be a non-sink). c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 51 / 300

52 Feature structures Feature graph subsumption Subsumption morphism Example (Subsumption morphism) A 1 A 2 q h h( q) q f h h(q) f h q h(q ) c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 52 / 300

53 Feature structures Feature graph subsumption Subsumption morphism Example (Subsumption) agr A : q0 A q1 A num pers q A 3 q A 2 third agr B : q0 B q1 B num q B 2 pl subj q4 B agr pers q3 B third c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 53 / 300

54 Feature structures Feature graph subsumption Subsumption morphism Indeed, B can and does have nodes that do not correspond to nodes in A: such is q4 B in the example. In addition, while the sink q2 A is not marked by an atom (that is, it is a variable), its image in B, q2 B, is marked as pl. Notice that no subsumption morphism can be defined from Q B to Q A, since there is no node into which q4 B can be mapped. In particular, it cannot be mapped to the root of A since this would necessitate an arc from q0 A to itself (as the root of A would be the image of both q4 B and qb 0 ). Trying to take h 1 as an inverse subsumption morphism will fail both because of q4 B and because it would map qb 2 to qa 2, violating the last clause of the subsumption relation (a marked sink must be mapped to a sink with the same mark). We conclude that B A. c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 54 / 300

55 Feature structures Feature graph subsumption Subsumption Given a feature structure, what modifications can be made to it in order for it to become more specific? Three different kinds of modifications are possible: 1 Adding arcs; 2 Adding reentrancies; 3 Marking unmarked sinks by some atom. c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 55 / 300

56 Feature structures Feature graph subsumption Subsumption Example (Subsumption as an order on information) num pl adding arcs num num num 1 num 2 num pl adding atomic marks sg num sg adding arcs per third sg num 1 num 2 sg adding reentrancies sg c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 56 / 300

57 Feature structures Feature graph subsumption Subsumption Lemma If A B then Π(A) Π(B). c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 57 / 300

58 Feature structures Feature graph subsumption Subsumption Lemma If A B then for each π Π(A), if θ A (δ A ( q A,π)) then θ B (δ B ( q B,π)) and θ A (δ A ( q A,π)) = θ B (δ B ( q B,π)). c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 58 / 300

59 Feature structures Feature graph subsumption Subsumption Lemma If A B and π 1,π 2 are reentrant in A (that is, π 1 A π2 ) then π 1,π 2 are reentrant in B (that is, π 1 B π2 ). c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 59 / 300

60 Feature structures Feature graph subsumption Subsumption Corollary If A B, then: Π(A) Π(B) for each π Π(A), if θ A (δ A ( q A,π)) then θ B (δ B ( q B,π)) and θ A (δ A ( q A,π)) = θ B (δ B ( q B,π)) for each π 1,π 2 Π(A), if π 1 A π2 then π 1 B π2 (and, therefore, if A is reentrant/cyclic then so is B). c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 60 / 300

61 Feature structures Feature graph subsumption Subsumption Theorem If A is an atomic feature graph and A B, then A B. c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 61 / 300

62 Feature structures Feature graph subsumption Subsumption Theorem Subsumption has a least element: there exists a feature graph A such that for all feature graph B, A B. Proof. Consider the (empty) feature graph A = {q 0 },q 0,δ,θ, where δ and θ are undefined for their entire domains. For every feature graph B, A B by mapping (through h) the root q 0 to the root of B, q B. The two clauses of the definition of subsumption hold vacuously. c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 62 / 300

63 Feature structures Feature graph subsumption Subsumption Theorem Subsumption is reflexive: for every feature graph A, A A. Proof. Take h to be the identity function that maps every node in A to itself. c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 63 / 300

64 Feature structures Feature graph subsumption Subsumption Theorem Subsumption is transitive: if A B and B C then A C. c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 64 / 300

65 Feature structures Feature graph subsumption Subsumption Theorem Subsumption is not antisymmetric: if A B and B A then not necessarily A = B. Proof. Consider the feature graphs A = { q A }, q A,δ,θ and B = { q B }, q B,δ,θ, where δ and θ are undefined for their entire domains, and where q A q B. Trivially, both A B and B A, but A B. c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 65 / 300

66 Feature structures Feature graph subsumption Subsumption Thus, feature graph subsumption forms a partial pre-order on feature graphs. It is a pre-order since it is not antisymmetric; it is partial as there are feature graphs that are incomparable with respect to subsumption. c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 66 / 300

67 Feature structures Feature graph subsumption Subsumption Example (Feature graph subsumption is a partial relation) Feature graphs can be incomparable due to inconsistency (contradicting information) or to complementary information. sg pl num sg num pl num pers c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 67 / 300

68 Feature structures Feature graph subsumption Subsumption There is a clear connection between feature graph isomorphism and feature graph subsumption: Theorem A B iff A B and B A. c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 68 / 300

69 Feature structures Attribute-value matrices AVMs We now return to attribute-value matrices (AVMs). This is the view that we will adopt for depicting feature structures (and grammars based on them), both because they are easy to present on paper and because of their centrality in existing literature. Like feature graphs, AVMs are defined over a signature of features and atoms, which we fix below. In addition, AVMs make use of variables, also called tags below. Meta-variables X, Y, Z, etc. range over over variables. Variables are used to encode sharing of values, as will be clear presently. When AVMs are concerned, we follow the convention of the linguistic literature by which variables are natural numbers, depicted in boxes, e.g., 3. c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 69 / 300

70 Feature structures Attribute-value matrices AVMs Definition (AVMs) Given a signature S, the set Avms(S) of AVMs over S is the least set satisfying the following two clauses: 1 M = Xa Avms(S) for any a Atoms and X Tags; M is said to be atomic and X is the tag of M, denoted tag(m) = X. 2 M = X[f 1 : M 1,...,f n : M n ] Avms(S) for n 0, X Tags, f 1,...,f n Feats and M 1,...,M n Avms(S), where f i f j if i j. M is said to be complex, and X is the tag of M, denoted tag(m) = X. If n = 0, M = X[] is an empty AVM. Note that two AVMs which differ only in their tag are distinct: if X Y, X [ ] Y [ ]. In particular, there is no unique empty AVM. Note also that the same variable can be used more than once in an AVM. c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 70 / 300

71 Feature structures Attribute-value matrices AVMs Example (AVMs) Consider a signature consisting of Atoms = {a} and Feats = {f,g}. Then M 1 = 4a is an AVM by the first clause of the definition, M 2 = 2 [ ] is an empty AVM by the second clause, M 3 = [ 3 f : 4a ] is an AVM by the second clause (using M 1 as the value of f, so that fval(m 3,f) = M 1 ), and [ [ ]] g : 3 f : 4a M 4 = 2 f : 2 [ ] is an AVM by the second clause, as is [ [ ]] g : 3 f : 4a M 5 = 4 f : 2 [ ] c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 71 / 300

72 Feature structures Attribute-value matrices AVMs Meta-variables M, with or without subscripts, range over Avms; the parameter S is omitted when it is clear from the context. The domain of an AVM M, denoted dom(m), is undefined when M is atomic, and {f 1,...,f n } when M is complex (hence, dom(m) is empty for an empty AVM). The value of some feature f Feats in M, denoted fval(m,f ), is defined if f = f i dom(m), in which case it is M i, and undefined otherwise. c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 72 / 300

73 Feature structures Attribute-value matrices Sub-AVMs Definition (Sub-AVMs) Given an AVM M, its sub-avms are SubAVM(M), defined as: 1 SubAVM(Xa) = {Xa} 2 SubAVM(X[f 1 : M 1,...,f n : M n ]) = X[f 1 : M 1,...,f n : M n ] 1 i n SubAVM(M i ) c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 73 / 300

74 Feature structures Attribute-value matrices AVMs Definition (Tags) Given an AVM M, its tags Tags(M) are defined as: 1 Tags(Xa) = {X } 2 Tags(X[f 1 : M 1,...,f n : M n ]) = X 1 i n Tags(M i ) Definition (Tagset) The tagset of an AVM M and a tag X Tags(M) is the set of sub-avms of M (including M itself) which are tagged by X: TagSet(M,X) = {M SubAVM(M) tag(m ) = X }. c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 74 / 300

75 Feature structures Attribute-value matrices AVMs Example (AVMs) Let: M 4 = 2 [ [ ]] g : 3 f : 4a f : 2 [ ] fval(m 4,f) = 2 [ ]. Observe that Tags(M 4 ) = {2, 3, 4 }. Also, TagSet(M 4, 4) is {4a}, TagSet(M 4, 3) is [ {3 f : 4a ] } and TagSet(M 4, 2) is {M 4, 2 [ ]}. Trivially, tag(m 4 ) = 2. c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 75 / 300

76 Feature structures Attribute-value matrices AVMs Example (AVMs) Let: [ [ ]] g : 3 f : 4a M 5 = 4 f : 2 [ ] Similarly, fval(m 5,f) = 2 [ ], whereas fval(m 5,g) = [ 3 f : 4a ]. Observe that Tags(M 5 ) = {2, 3, 4 }.Also TagSet(M 5, 2) = {2 [ ]}, TagSet(M 5, 3) = [ {3 f : 4a ] } and TagSet(M 5, 4) = {M 5, 4a}. Trivially, tag(m 5 ) = 4. c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 76 / 300

77 Feature structures Attribute-value matrices AVMs Example (AVMs) As another example, consider the AVM M 6 = [ 1 f : [ 1 f : [ 1 f : 1 [ ] ]]] Here, Tags(M 6 ) = {1}, and TagSet(M 6, 1) is: {M 6, [ 1 f : [ 1 f : 1 [ ] ]], [ 1 f : 1 [ ] ], 1 [ ]} Of course, tag(m 6 ) = 1. c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 77 / 300

78 Feature structures Attribute-value matrices Well-formed AVMs [ ] f1 : 2M Consider some AVM M = 1 1 where M f 2 : 2M 1 M 2. 2 Both M 1 and M 2 are sub-avms of M, and both have the same tag, although they are different. In other words, the recursive definition of AVMs allows two different, contradicting AVMs to be in the TagSet of the same variable. To eliminate such cases, we define well-formed AVMs as follows: Definition (Well-formed AVMs) An AVM M is well-formed iff for every variable X Tags(M), TagSet(M,X) includes at most one non-empty AVM. c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 78 / 300

79 Feature structures Attribute-value matrices Reentrancy Example (A reentrant AVM) The following AVM is reentrant but not cyclic: [ ] num : 2pl 0 agr : 1 pers : 3 third subj : [ 4 agr : ] 1 c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 79 / 300

80 Feature structures Attribute-value matrices Conventions We introduce three conventions regarding the depiction of well-formed AVMs, motivated by the fact that variables are used primarily to indicate value sharing. If a variable occurs more than once then its value is explicated only once; where this value is explicated (i.e., next to which occurrence of the variable) is immaterial. Variables which occur only once can be omitted. The empty AVM is sometimes omitted when it is associated with a variable. The first convention is crucial in the case of cyclic AVMS: there is no finite representation of cyclic AVMs unless this convention is adopted. c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 80 / 300

81 Feature structures Attribute-value matrices Conventions Example (Shorthand notation for AVMs) Consider the following AVM: f : 3 ] 6 g : [ 4 h : 3a ] h : 2 [ ] Notice that it is well-formed, since the only variable occurring more than once (3) is associated with a non-empty value (a) only once. We can therefore leave only one occurrence of the value explicit The tag 2 is associated with the empty feature structure, which can be omitted Finally, the tags 4 and 6 occur only once, so they can be omitted This is the conventional form of the AVM. c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 81 / 300

82 Feature structures Attribute-value matrices AVM equivalence Example (AVM equivalence) M 1 and M 2 differ only in the instance of 0 whose value is explicated: [ ] num : 2pl M 1 = 0 agr : 1 pers : 3 third subj : [ 4 agr : ] 1 agr : 1 M 2 = 0 subj : 4 Then M 1 M 2 and M 2 M 1. [ [ num : 2pl agr : 1 pers : 3 third ]] c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 82 / 300

83 Feature structures Attribute-value matrices AVM equivalence Example (AVM renamings) The following two AVMs are renamings of each other: [ ] num : 2pl M 1 = 0 agr : 1 pers : 3 third subj : [ 4 agr : ] 1 agr : 11 M 2 = 10 subj : 14 [ [ num : 12pl agr : 11 pers : 13 third ]] c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 83 / 300

84 Feature structures The correspondence between feature graphs and AVMs The correspondence between feature graphs and AVMs AVMs are the entities that the linguistic literature employs to depict feature structures; feature graphs are well-understood mathematical entities to which various results of graph theory can be applied. We define the relationship between these two views. c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 84 / 300

85 Feature structures The correspondence between feature graphs and AVMs From AVMs to feature graphs Example (AVM to graph mapping) A reentrant AVM and its feature graph image: [ ] num : 2pl M = 0 agr : 1 pers : 3 third subj : [ 4 agr : ] 1 φ(m) = num 2 pl agr 0 1 third subj 4 agr pers 3 c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 85 / 300

86 Feature structures Feature structures in a broader context Feature structures in a broader context Feature structures are utilized by many grammatical formalisms to encode different kinds of linguistic information: they serve in representing phonological, morphological, syntactic and semantic knowledge. But the use of feature structures is not limited to computational linguistics; indeed, they are present in other areas of computer science as well. c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 86 / 300

87 Feature structures Feature structures in a broader context Feature structures in a broader context A somewhat degenerate form of feature structures is utilized by many programming languages: records (as in Pascal, known as structures in C). There are some major differences between records and feature structures. The notion of sharing that is central to feature structures is less significant for records. The values of record fields are not necessarily other records different data types can be freely used; hence transfer of values is mediated through explicit assignments, not unifications. Unification-based formalisms usually do not allow such a diversity of operations to apply to feature structures as programming languages allow to records. In particular, arithmetic operations are usually not applicable to feature structures values, while they are very natural to numeric records fields. c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 87 / 300

88 Feature structures Feature structures in a broader context Feature structures in a broader context Logic programming languages such as Prolog manipulate first-order terms (FOTs), which might be viewed as a special case of feature structures. There are some important differences between feature structures and FOTs. FOTs are essentially trees, with possibly shared leaves, whereas feature structures allow reentrancies to occur in every level of the structure. Feature structures can be cyclic, in contrast to (ordinary) FOTs. FOTs use positional encoding of argument structures, with no features. Two FOTs are unifiable only if they have the same functor and the same arity, while two feature structures might be unifiable even if they have different number of features. c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 88 / 300

89 Unification Motivation Unification The subsumption relation compares the information content of feature structures. Unification combines the information that is contained in two (compatible) feature structures. We use the term unification to refer to both the operation and its result. Whenever two feature structures are related, they are assumed to be over the same signature. c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 89 / 300

90 Unification Motivation Unification The mathematical interpretation of combining two members of a partially ordered set is to take the least upper bound of the two operands with respect to the partial order; in our case, subsumption. Indeed, feature structure unification is exactly that. However, since subsumption is antisymmetric for feature structures and AFSs but not for feature graphs and AVMs, a unique least upper bound cannot be guaranteed for all four views. c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 90 / 300

91 Unification Feature structure unification Feature structure unification Definition (Feature structure unification) Two feature structures fs 1 and fs 2 are consistent if they have an upper bound (with respect to subsumption), and inconsistent otherwise. If fs 1 and fs 2 are consistent, their unification, denoted fs 1ˆ fs 2, is their least upper bound with respect to subsumption. If two feature structures have an upper bound, they have a (unique) least upper bound. c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 91 / 300

92 Unification Feature graph unification Feature graph unification While the definition of unification as least upper bound is useful mathematically, it does not tells us how to compute the unification of two given feature structures. To this end, we provide a constructive definition in terms of feature graphs, which induces an algorithm for computing unification. For reasons that will be clear presently, we require that the two feature graphs be node-disjoint. c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 92 / 300

93 Unification Feature graph unification Feature graph unification Definition Let A = Q A, q A,δ A,θ A and B = Q B, q B,δ B,θ B with Q A Q B = be two feature graphs. Let u be the least equivalence relation on Q A Q B such that: q A u qb for every q 1,q 2 Q A Q B and f Feats, if q 1 u q2,(δ A δ B )(q 1,f ) and (δ A δ B )(q 2,f ), then (δ A δ B )(q 1,f ) u (δ A δ B )(q 2,f ) c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 93 / 300

94 Unification Feature graph unification Feature graph unification The u relation partitions the nodes of Q A Q B to equivalence classes such that both roots are in the same class, and if some feature is defined for two nodes in one class, then the two nodes this feature leads to are also in one (possibly different) class. Clearly, the number of equivalence classes (called the index of u ) is finite. The requirement that Q A and Q B be disjoint is essential here: we would want two nodes to be in the same equivalence class with respect to u only if they comply with the above definition; if we allowed a non-empty intersection of nodes, u could have been a different relation. c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 94 / 300

95 Unification Feature graph unification The u relation g A : q0 A q1 A num f q2 A sg B : q0 B f q1 B pers q2 B 3rd c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 95 / 300

96 Unification Feature graph unification The u relation g A : q0 A f q1 A q2 A h q B 0 B : q f 1 B g c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 96 / 300

97 Unification Feature graph unification Type-respecting relation Definition A binary relation over the nodes of two feature structures Q A Q B is said to be type respecting iff for every node q Q A Q B, if (θ A θ B )(q) and (θ A θ B )(q) = a, then for every node q such that q q, q is a sink and either (θ A θ B )(q ) or (θ A θ B )(q ) = a. c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 97 / 300

98 Unification Feature graph unification Type-respecting relation When is u not type respecting? The above condition can hold for a node q Q A Q B only if (θ A θ B )(q) ; that is, q must be a sink in either A or B. The type respecting condition requires that all nodes that are equivalent to q be sinks, either unmarked or marked by the same atom. Since this is the only requirement, the relation is not type respecting if it maps two nodes, one of which is a marked sink and the other of which is either a non-sink or a sink with a different label, to the same equivalence class. A non-type respecting u is the only source for unification failure. c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 98 / 300

99 Unification Feature graph unification Type respecting u relation g A : q0 A q1 A num f q2 A sg B : q0 B f q1 B pers q2 B 3rd c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 99 / 300

100 Unification Feature graph unification Feature graph unification Lemma If A and B have a common upper bound C, such that A C through the morphism h A and B C through the morphism h B, and if q A Q A and q B Q B are such that q A u qb, then h A (q A ) = h B (q B ). c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 100 / 300

101 Unification Feature graph unification Feature graph unification Definition (Feature graph unification) Let A and B be two feature graphs such that Q A and Q B are disjoint. The unification of A and B, denoted A B, is defined only if u is type respecting, in which case it is the feature graph Q, q, δ, θ, where: Q = {[q] u q (Q A Q B )} q = [ q 1 ] u (= [ q 2 ] u ) { [q ] u δ([q] u, f ) = if there exists q [q] u s.t. (δ A δ B )(q, f ) = q undef. if (δ A δ B )(q, f ) for all q [q] u { (θa θ B )(q ) if there exists q [q] u θ([q] u ) = s.t. (θ A θ B )(q ) undefined if (θ A θ B )(q ) for all q [q] u If u is not type respecting, A and B are inconsistent. c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 101 / 300

102 Unification Feature graph unification Feature graph unification g A : q0 A q1 A num q2 A f sg B : q0 B f q1 B pers q2 B 3rd c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 102 / 300

103 Unification Feature graph unification Unification To see that the result of unification is indeed a feature graph, observe that Q, q, δ, θ is connected because both A and B are connected; it is finite since both A and B are (and hence the number of equivalence classes is finite); and θ labels only sinks, since u is type respecting. c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 103 / 300

104 Unification Feature graph unification Unification Example (Unification combines information) num num q 0 q 1 q 3 = q 6 q 7 sg sg pers q 5 3rd pers q 8 3rd c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 104 / 300

105 Unification Feature graph unification Unification Example (Unification is absorbing) num num num q 0 q 1 q 3 q 4 = q 6 q 7 sg sg sg pers q 5 3rd pers q 8 3rd c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 105 / 300

106 Unification Feature graph unification Unification with reentrancies subj obj num sg subj obj num pers sg pers 3rd 3rd subj obj c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 106 / 300

107 Unification Feature graph unification Unification Theorem If A and B are inconsistent, they do not have a common upper bound. Otherwise, C = A B is a minimal upper bound of A and B with respect to (feature graph) subsumption. c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 107 / 300

108 Unification Feature graph unification Unification The previous theorem connects feature graph unification with feature structure unification. In order to compute fs = fs 1ˆ fs 2, simply compute A = A 1 A 2, where A 1 fs 1 and A 2 fs 2, and take fs = [A]. Theorem For all feature graphs A 1,A 2, if A = A 1 A 2 then [A] = [A 1 ] ˆ [A 2 ]. c Shuly Wintner (University of Haifa) Unification Grammars c Copyrighted material 108 / 300