Ling 5801: Lecture Notes 7 From Programs to Context-Free Grammars

Transcription

1 Ling 5801: Lecture otes 7 From Programs to Context-Free rammars 1. The rules we used to define programs make up a context-free grammar A Context-Free rammar is a tuple C,X,S,R, where: C is a finite set of type or category symbols X C is a set of terminal symbols (e.g. if,:,...) S C is a set of start symbols (e.g. program ) R C C C is a set of rewrite rules of the formc c c (or c,c,c ) The symbol to the left of each arrow may be called the parent The symbols to the right of each arrow may be called the children rammars allowing only two children per rule are called binary-branching For example: cattoy = {S, P, VP, the, cat, hits, toy}, {the, cat, hits, toy}, {S}, {S P VP, P the cat, P the toy, VP hits P} 2. Languages accepted by a CF For any CF = C,X,S,R : L() = {x 1..x T c S s.t. c x 1..x T } wherecyieldsx i..x j if and only if there is a treeτ w. root c, all leaves inx i..x j, branches inr (defining trees τ as sets of constituents c,i,j of categorycspanning inputitoj): { ifc ifi =j : =x i : True c ifc x i x i..x j iff : False ifi τ w. root c,i,j c,i,j τ <j : R(c d e ) k,d,e s.t. d,i,k, e,k +1,j τ For example, the cat hits the toy is inl( cattoy ) because this treeτ exists: S P VP the cat hits P the toy with constituents: τ = { S,1,5, P,1,2, the,1,1, cat,2,2, VP,3,5, hits,3,3, P,4,5, the,4,4, toy,5,5 } 1

2 We can now move one branch conjunct outside the disjunction over trees (distributive axiom): { ifc=xi : True ifi=j : ifc x i : False c x i..x j iff ifi<j : d e) k,d,er(c {... τ w. root d,i,k c,i,j τ and identify parenthesized terms as recursive instances ofc x i..x j : { ifc=xi : True ifi=j : c ifc x i : False x i..x j iff ifi<j : ) ) d e) (d k,d,er(c x i..x k (e x k+1..x j More simply: τ w. root e,k+1,j c,i,j τ {... c x i..x j iff { ifi = j : c = xi ifi < j : k,d,e s.t. c de R and d x i..x k and e x k+1..x j And the languages recognizable by a CF are: L(CF) = {L() CF } 3. L(CF) L(CF 2 children per branch ), so L(CF) = L(CF 2 children per branch ) CFs which have rules R containing more than two children can be reduced to binaryr : R = {c c 0 c 1... c B c c 0... c B R} {c b... c B c b c b+1... c B c c 0... c B R, 1 b < B} C = C {c b... c B c c 0... c B R, 1 b < B} For example: VP pick P up,... would become: VP pick P up, P up P up, L(CF 2 children per branch ) L(CF 2 children per branch ), so L(CF) = L(CF 2 children per branch ) CFs which have rules R containing fewer than two children can be reduced to binary R : (a) First, obtain reflexive transitive closurer ( C ) of unary rules in R (likeǫ-transitions in FSA): R (0) = {c c c C} R (k) = {c c c c R (k 1), c c R} (b) Then add these closure expansions to children of each binary expansion: R = {c d e c d e R, d d R ( C ), e e R ( C ) } 2

3 (c) Then add these closure expansions to the set of start symbols: S = S {c c S, c c R ( C ) } For example, adding the following unary rule to our cat-toy grammar: P Meowy would result in the addition of the following binary rules: S Meowy VP, VP hits Meowy 5. A weak but intuitive claim: L(RE) L(CF) We can write a CF (ρ) implementing any RE ρ, concatenated to a start symbol. First, remove epsilons from Kleene star: ρ = (ǫ ρ + ) (ǫ ρ 1 )ρ 2 = (ρ 2 ρ 1 ρ 2 ) ρ 1 (ǫ ρ 2 ) = (ρ 1 ρ 1 ρ 2 ) Then map regular expression syntax to CF rules: R (ρ1 ρ 2 ) = R (ρ1 ) R (ρ2 ) {c ρ1 ρ 2 c ρ1 c ρ2 } R (ρ1 ρ 2 )= R (ρ1 ) R (ρ2 ) {c ρ1 ρ 2 c ρ1, c ρ1 ρ 2 c ρ2 } R (ρ + 1 ) = R (ρ1 ) {c ρ + c ρ 1 1 c ρ +, c 1 ρ + c ρ 1 1 } 6. A strong claim: L(RE) L(CF) There is a language that cannot be recognized by a RE: a n b n (proof by pumping lemma)...which can be recognized by a CF: (proof by existance: {S, a, b},{a, b},{s},{s a S b, S a b} ) 7. We can use CFs to model natural languages like English Do natural languages have natural category types that we can use in CF rules? Yes! First, observe that natural languages use different argument structures for different verbs: They sleep. one argument ahead (intransitive) They find pets. one argument ahead and one argument behind (transitive) They give people pets. one argument ahead, two arguments behind (ditransitive) ext, observe natural languages coordinate conjunctions (combine like types): α α and α. [ β [ β They find people] and [ β they find pets]]. sounds ok (β is sentence) They find [ γ [ γ people] and [ γ pets]]. sounds ok (γ is noun phrase) *They find [ γ [ β they find people] and [ γ pets]]. sounds wrong; conjuncts must match 3

4 ow, allowable conjunctions give us insight into the category structure of language: They [ δ [ δ sleep] and [ δ find pets]]. sounds ok (δ is verb phrase) They [ η [ η find] and [ η give people]] pets. sounds ok (but what sη?) Transitive verbs (find) match type with ditransitive verb + indirect object (give people)! Both lack argument ahead and behind it seems types are defined by missing arguments! Formalize set of categoriesc as follows clauses with various unmet requirements: (a) every U is inc, for some set U of primitive categories; (b) every C O C is inc, for some set O of type-combining operators; (c) nothing else is inc Define primitive categories U = {, V}: : noun-headed category with no missing arguments (noun phrase) V: verb-headed category with no missing arguments (sentence) Define type-combining operators O = {-a, -b}: α-aβ : α lacking β argument ahead (e.g. V-a for intransitiveδ above) α-bβ : α lackingβ argument behind (e.g. V-a-b for transitiveη above) ow we can define CF rules R over these categories: α β α-aβ : argument attachment ahead α α-bβ β : argument attachment behind α α and α : conjunction These three rules model all of the above sentences: V V V-a V-a They V-a and V-a They V-a-b sleep V-a-b V-a-b and V-a-b pets find pets find V-a-b-b give people Also note that the parents in these rules all have simpler types than the children. This means for any lexicon (constraining types at tree leaves), the set of categoriesc is finite. 4

5 8. Limits of CFs atural languages may also use non-local dependencies. In English, these show up in topicalization, which seem to use a gap at one argument: These pets, you say they found. These coordinate as well, but our test shows categories with gaps differ from those without: These pets, you [ δ [ δ say they found ] and [ δ think gave people joy]]. sounds ok *These pets, you [ δ [ V-a say they found pets] and [ δ think gave people joy]]. wrong We can model this by adding a new type-combining operator for non-local dependencies: α-gβ : α lacking non-localβ argument (e.g. V-a-g for intransitiveδ above) and adding rules to introduce non-local dependencies: α-gβ α-aβ : introduce non-local dependency to argument ahead α-gβ α-bβ : introduce non-local dependency to argument behind and adding rules to attach non-local dependencies: α β α-gβ : non-local dependency attachment and modifying existing rules to propagate non-local dependencies ψ m {-g} C: αψ 1..M βψ 1..m α-aβψ m+1..m : argument attachment ahead, with propagation αψ 1..M α-bβψ 1..m βψ m+1..m : argument attachment behind, with propagation Here s the analysis: V V-g These pets V-a-g you V-a-g and V-a-g V-a-bV V-g V-a-bV V-g say V-a-g think V-a they V-a-b V-a-b found V-a-b-b joy gave people 5

6 ote that M above is unbounded, so our rules no longer guarantee a finite set of categories. (Any number of arguments may be extracted and propagated up from children.) Some use evidence like this to argue language isn t context-free but mildly context-sensitive [Shieber, 1985, Joshi, 1985, Steedman, 2000]. In practice, though, we can just constrain category sets to combinations seen in training data. 9. atural language grammar: English = C,X,S,R (generatively sound, not complete) (a) Categories C defined from set of primitives U and set of type-constructing operators O: i. Primitive categoriesu different ways to say the same thing: V finite / declarative sentence (e.g. they knew it ) I infinitive clause (e.g. them to know it, as in we expect... ) B base-form clause (e.g. them know it, as in we let... ) L participle clause (e.g. them known it complete form not used) A adjectival/predicative/ small clause (e.g. them knowing it, it known, as in we consider... ) R adverbial clause (e.g. them knowingly complete form not used) gerund clause (e.g. them knowing it ) accus. noun phrase / nominal clause (e.g. their knowledge of it, her, him, Kim ) 1S nom. 1st-pers. sing. noun phrase (e.g. I ) 1P nom. 1st-pers. plur. noun phrase (e.g. we ) 2 nom. 2nd-pers. noun phrase (e.g. you ) 3S nom. 3rd-pers. sing. noun phrase (e.g. their knowledge of it, she, he, Kim ) 3P nom. 3rd-pers. plur. noun phrase (e.g. they, the risks ) D determiner (clause) (e.g. their knowledge of it s, the, their ) O non-possessive genitive (clause) (e.g. of their knowledge of it ) C complementized clause (e.g. that they know it, as in we think... ) E embedded infinitive clause (e.g. for them to know it, as in we hope... ) Q (polar) question (e.g. did they know it ) S start symbol / sentence / utterance (e.g. know it, yeah, duh ) ii. Type-constructing operators O these allow specification of unmet requirements: -a lacking initial argument (precedes lexical head / main word, e.g. subject) -b lacking final argument (follows lexical head / main word, e.g. object) -c lacking initial conjunct (precedes conjunction) -d lacking final conjunct (follows conjunction) -g lacking gap argument (missing at any position in clause) -h lacking right-node-raised argument (right-node raising) -i lacking interrogative pronoun referent -r lacking relative pronoun referent iii. Define categories C as follows clauses with various unmet requirements: A. everyu is inc B. everyc O C is inc C. nothing else is in C English needs categories for interrog. and relative pronouns, clauses, and gapped clauses: 6

7 V-g gapped finite clause / bare relative B-g gapped base-form clause -i interrogative pronoun Q-i content question V-i embedded question -r relative pronoun V-r relative clause... (and others) and categories for constituents within clauses: (e.g. Kim gave ǫ to Chris ) (e.g. Kim give ǫ to Chris ) (e.g. what ) (e.g. what did Kim giveǫto Chris ) (e.g. what Kim gave ǫ to Chris ) (e.g. which ) (e.g. which Kim gave ǫ to Chris ) -ad, 3S-aD, 3P-aD common noun, lacking determiner (e.g. risk ) V-a3S intransitive verb, lacking subject (e.g. sleeps ) V-a3S-b transitive verb, lacking subject, object (e.g. finds ) V-a3S-b-b ditransitive verb (e.g. gives ) V-a3S-bC bridge verb (e.g. thinks ) V-a3S-b(I-a) raising verb (e.g. seems ) V-a3S-b(B-a) auxiliary verb (e.g. would ) V-a3S-bE (e.g. hopes ) V-a3S-b(Q-i) (e.g. wonders ) A-a adjective / predicative form of verb (e.g. asleep, knowing, known ) A-a-b adjectival preposition (e.g. in, finding ) A-a-b(I-a-g) tough adjective (e.g. tough ) R-a adverb (e.g. knowingly ) R-a-b adverbial preposition (e.g. into ) R-a-bV discourse connective (e.g. after )... (and others) (b) Observation symbols: X = {the,a,and,car,...} (c) Start symbols: S = {S} (d) Rules in R: i. lexical items:... (e.g. D the) ii. arguments: forc U ({-a, -b, -c, -d} C), d C: c d c-ad, c c-bd d e.g.: V 3P V-a3P babies cry iii. modifiers: forc U ({-a, -b, -c, -d} C), u U: c c u-a, c u-ac e.g.: 3P 3P people A-a here 7

8 iv. hypothesize gaps: forc C, d C: c-gd c-ad c-gd c-bd c-gd c v. propagate non-local arguments: fora b c R, ψ {-g, -h, -i, -r} C: aψ bcψ, aψ bψ c, aψ bψ cψ e.g.:... something V-g 2 you V-a2-g V-a2-b ate vi. connect gaps to modificands or fillers: forc C, d C, e C: e e c-rd, c-re d-re c-gd e e c-gd, c-ie d-ie c-gd e.g.: something 2 you V-g V-a2-g V-a2-b ate vii. conjuncts: forc C, d C: c d c-cd, c-cd d c-cd, c c-dd d e.g.: -c lions -c tigers -c-d and bears viii. elided arguments (e.g. determiners): forc C, d C: c c-ad c c-bd e.g.: -ad people ix. sentential forms: 8

9 For example: S V declarative sentence (e.g. Kim slept ) S B-a imperative sentence (e.g. sleep! ) S Q polar question (e.g. did Kim sleep? ) S Q-i content question (e.g. what did Kim drive? ) S V-g topicalized sentence (e.g. that park I like ) S S V-gS topicalized sentence (e.g. I ate she said ) S Q-i -i Q-g what Q-b(B-a) B-a-g Q-b(B-a)-b2 2 B-a-b(I-a) I-a-g do you want I-a-b(B-a) B-a-g to B-a-g R-a B-a-b(A-a)-g A-a R-a-b B-a-b(A-a)-b white in D -ad paint the den 10. Parsing as deduction: Can also express rules as inferences in natural deduction: c de d e c Parse trees are then proofs: paint the B-a-b(A-a)-bwhite in D B-a-b(A-a)-g A-aR-a-b to B-a-g R-a do you want I-a-b(B-a) B-a-g what -i Q-b(B-a)-b2 2 B-a-b(I-a) Q-b(B-a) I-a-g B-a-g Q-g Q-i S den -ad 9

10 References [Joshi, 1985] Joshi, A. K. (1985). How much context sensitivity is necessary for characterizing structural descriptions: Tree adjoining grammars. In D. Dowty, L. K. and Zwicky, A., editors, atural language parsing: Psychological, computational and theoretical perspectives, pages Cambridge University Press, Cambridge, U.K. [Shieber, 1985] Shieber, S. (1985). Evidence against the context-freeness of natural language. Linguistics and Philosophy, 8: [Steedman, 2000] Steedman, M. (2000). The syntactic process. MIT Press/Bradford Books, Cambridge, MA. 10