Sharpening the empirical claims of generative syntax through formalization

Transcription

1 Sharpening the empirical claims of generative syntax through formalization Tim Hunter University of Minnesota, Twin Cities ESSLLI, August 2015

2 Part 1: Grammars and cognitive hypotheses What is a grammar? What can grammars do? Concrete illustration of a target: Surprisal Parts 2 4: Assembling the pieces Minimalist Grammars (MGs) MGs and MCFGs Probabilities on MGs Part 5: Learning and wrap-up Something slightly different: Learning model Recap and open questions

3 Sharpening the empirical claims of generative syntax through formalization Tim Hunter ESSLLI, August 2015 Part 3 MGs and MCFGs

4 Where we re up to We ve seen: Now: Later: MGs with operations defined that manipulated trees that the structure that really matters (e.g. for recursion) can be boiled down to funny-looking derivation trees (with things like t, -k at the non-leaf nodes) A way to think of how these derivation trees relate to surface strings (without going via trees) In some ways not totally necessary for the rest of the course, but helpful Adding probabilities to MGs: in a way that sort of works, and does some good stuff, but doesn t do everything we d want Adding probabilities to MGs: in an even better way 102 / 201

5 Outline 9 A different perspective on CFGs 10 Concatenative and non-concatenative operations 11 MCFGs 12 Back to MGs 103 / 201

6 Outline 9 A different perspective on CFGs 10 Concatenative and non-concatenative operations 11 MCFGs 12 Back to MGs 104 / 201

7 Trees S the boy likes cake :: S NP VP the boy :: NP likes cake :: VP D the N boy V likes NP N cake the :: D boy :: N likes :: V cake :: NP cake :: N 105 / 201

8 Trees S the boy likes cake :: S NP VP the boy :: NP likes cake :: VP D the N boy V likes NP N cake the :: D boy :: N likes :: V cake :: NP cake :: N How to think of a tree: less as a picture of a string more as a graphical representation of how a string was constructed, with the string at the top node 105 / 201

9 Two sides of a CFG rule A rule like S NP VP says two things: What combines with what: An NP and a VP can combine to form an S How to produce a string of the new category: Put the NP-string to the left of the VP-string More explicitly: st :: S s :: NP t :: VP 106 / 201

10 Example: X-bar theory Japanese XP Spec X X Comp X English XP Spec X X X Comp 107 / 201

11 Example: X-bar theory Japanese XP Spec X X Comp X English XP Spec X X X Comp Japanese st :: XP s :: Spec t :: X st :: X s :: Comp t :: X English st :: XP s :: Spec t :: X ts :: X s :: Comp t :: X 107 / 201

12 Example: X-bar theory Japanese XP Spec X X Comp X English XP Spec X X X Comp Japanese st :: XP s :: Spec t :: X st :: X s :: Comp t :: X English st :: XP s :: Spec t :: X ts :: X s :: Comp t :: X John-ga Mary-o mita :: VP John saw Mary :: VP John-ga :: Spec Mary-o mita :: V John :: Spec saw Mary :: V Mary-o :: Comp mita :: V Mary :: Comp saw :: V 107 / 201

13 Outline 9 A different perspective on CFGs 10 Concatenative and non-concatenative operations 11 MCFGs 12 Back to MGs 108 / 201

14 Concatenative and non-concatenative operations Concatenative morphology: play + ed played play + ing playing play + s plays Non-concatenative morphology: (k,t,b) + (i,aa) kitaab ( book ) (k,t,b) + (aa,i) kaatib ( writer ) (k,t,b) + (ma,uu) maktuub ( written ) (k,t,b) + (a,i,a) katiba ( document ) 109 / 201

15 Concatenative and non-concatenative operations Concatenative morphology: play + ed played play + ing playing play + s plays Non-concatenative morphology: (k,t,b) + (i,aa) kitaab ( book ) (k,t,b) + (aa,i) kaatib ( writer ) (k,t,b) + (ma,uu) maktuub ( written ) (k,t,b) + (a,i,a) katiba ( document ) Concatenative syntax: plays + tennis plays tennis plays + soccer plays soccer John + plays soccer John plays soccer Mary + plays soccer Mary plays soccer 109 / 201

16 Concatenative and non-concatenative operations Concatenative morphology: play + ed played play + ing playing play + s plays Non-concatenative morphology: (k,t,b) + (i,aa) kitaab ( book ) (k,t,b) + (aa,i) kaatib ( writer ) (k,t,b) + (ma,uu) maktuub ( written ) (k,t,b) + (a,i,a) katiba ( document ) Concatenative syntax: plays + tennis plays tennis plays + soccer plays soccer John + plays soccer John plays soccer Mary + plays soccer Mary plays soccer Non-concatenative syntax: seems + (John, to be tall) John seems to be tall seems + (Mary, to be intelligent) Mary seems to be intelligent did + (John see, who) who did John see did + (Mary meet, who) who did Mary meet 109 / 201

17 Non-concatenative morphology kitaab :: A kaatib :: A kutub :: A k,t,b :: B i,aa :: C k,t,b :: B aa,i :: C k,t,b :: B u,u :: C stuvw :: A s, u, w :: B t, v :: C 110 / 201

18 Non-concatenative morphology kitaab :: A kaatib :: A kutub :: A k,t,b :: B i,aa :: C k,t,b :: B aa,i :: C k,t,b :: B u,u :: C stuvw :: A s, u, w :: B t, v :: C gespielt :: E gekauft :: E gemacht :: E spiel :: A ge,t :: D kauf :: A ge,t :: D mach :: A ge,t :: D stu :: E t :: A s, u :: D 110 / 201

19 Non-concatenative morphology stuvw :: A s, u, w :: B t, v :: C stu :: E t :: A s, u :: D 111 / 201

20 Non-concatenative morphology stuvw :: A s, u, w :: B t, v :: C stu :: E t :: A s, u :: D gekitaabt :: E kitaab :: A ge,t :: D k,t,b :: B i,aa :: C 111 / 201

21 Non-concatenative morphology stuvw :: A s, u, w :: B t, v :: C stu :: E t :: A s, u :: D ts, u :: D t :: F s, u :: D gekitaabt :: E ausgekitaabt :: E kitaab :: A ge,t :: D kitaab :: A ausge,t :: D k,t,b :: B i,aa :: C k,t,b :: B i,aa :: C aus :: F ge,t :: D 111 / 201

22 Non-concatenative morphology stuvw :: A s, u, w :: B t, v :: C stu :: E t :: A s, u :: D ts, u :: D t :: F s, u :: D gekitaabt :: E ausgekitaabt :: E kitaab :: A ge,t :: D kitaab :: A ausge,t :: D k,t,b :: B i,aa :: C k,t,b :: B i,aa :: C aus :: F ge,t :: D If our goal is to characterize the array of well-formed/derivable objects not to pronounce them then all we care about is what s built out of what : A B C E A D D F D 111 / 201

23 Outline 9 A different perspective on CFGs 10 Concatenative and non-concatenative operations 11 MCFGs 12 Back to MGs 112 / 201

24 Multiple Context-Free Grammars (MCFGs) st :: S s :: NP t :: VP An MCFG generalises to allow yields to be tuples of strings. t 2st 1 :: Q s :: NP t 1, t 2 :: VPWH This rule says two things: We can combine an NP with a VPWH to make a Q. The yield of the Q is t 2st 1, where s is the yield of the NP and t 1, t 2 is the yield of the VPWH. 113 / 201

25 Multiple Context-Free Grammars (MCFGs) st :: S s :: NP t :: VP An MCFG generalises to allow yields to be tuples of strings. t 2st 1 :: Q s :: NP t 1, t 2 :: VPWH This rule says two things: We can combine an NP with a VPWH to make a Q. The yield of the Q is t 2st 1, where s is the yield of the NP and t 1, t 2 is the yield of the VPWH. which girl the boy says is tall :: Q the boy :: NP says is tall,which girl :: VPWH 113 / 201

26 Some technical details Each nonterminal has a rank n, and yields only n-tuples of strings. So given this rule: t 2st 1 :: Q s :: NP t 1, t 2 :: VPWH we know that anything producing a VPWH must produce a 2-tuple....,... :: VPWH... and that anything producing an NP must produce a 1-tuple:... :: NP... (Seki et al. 1991, Weir 1988, Vijay-Shanker et al. 1987) 114 / 201

27 Some technical details Each nonterminal has a rank n, and yields only n-tuples of strings. So given this rule: t 2st 1 :: Q s :: NP t 1, t 2 :: VPWH we know that anything producing a VPWH must produce a 2-tuple....,... :: VPWH... and that anything producing an NP must produce a 1-tuple:... :: NP... The string-composition functions cannot copy pieces of their arguments. OK s t :: VP s :: V t :: NP OK t s himself :: S s :: V t :: NP Not OK t s t :: S s :: V t :: NP (Seki et al. 1991, Weir 1988, Vijay-Shanker et al. 1987) 114 / 201

28 Some technical details Each nonterminal has a rank n, and yields only n-tuples of strings. So given this rule: t 2st 1 :: Q s :: NP t 1, t 2 :: VPWH we know that anything producing a VPWH must produce a 2-tuple....,... :: VPWH... and that anything producing an NP must produce a 1-tuple:... :: NP... The string-composition functions cannot copy pieces of their arguments. OK s t :: VP s :: V t :: NP OK t s himself :: S s :: V t :: NP Not OK t s t :: S s :: V t :: NP Essentially equivalent to linear context-free rewriting systems (LCFRSs). (Seki et al. 1991, Weir 1988, Vijay-Shanker et al. 1987) 114 / 201

29 Beyond context-free t 1t 2 :: S t 1, t 2 :: P t 1u 1, t 2u 2 :: P t 1, t 2 :: P u 1, u 2 :: E ɛ, ɛ :: P a, a :: E b, b :: E { ww w {a, b} } aabaaaba :: S aaba,aaba :: P aab,aab :: P a,a :: E aa,aa :: P b,b :: E a,a :: P a,a :: E ɛ, ɛ :: P a,a :: E Unlike in a CFG, we can ensure that the two halves are extended in the same ways without concatenating them together. 115 / 201

30 For comparison t 1st 2 :: S t 1 :: A s :: S t 2 :: A t 1st 2 :: S t 1 :: B s :: S t 2 :: B ɛ :: S a :: A b :: B abaaaaba :: S a :: A baaaab :: S a :: A b :: B aaaa :: S b :: B a :: A aa :: S a :: A a :: A ɛ :: S a :: A 116 / 201

31 Outline 9 A different perspective on CFGs 10 Concatenative and non-concatenative operations 11 MCFGs 12 Back to MGs 117 / 201

32 Merge and move < =f α f β merge α β > merge =f α f β β α > +f α move β α -f β 118 / 201

33 What matters in a (derived) tree This tree: x -f -g becomes a tuple of categorized strings: s :: x, t :: -f, u :: -g or, equivalently, a tuple-of-strings, categorized by a tuple-of-categories: s, t, u :: x, -f, -g 0 0 (Michaelis 2001, Stabler and Keenan 2003) 119 / 201

34 Remember MG derivation trees? We can tell that this tree represents a well-formed derivation, by checking the feature-manipulations at each step. How can we work out which string it derives? Build up a tree according to merge and move rules, and read off leaves of the tree. But there s a simpler way. t +k t, -k will :: =v +k t =subj v think :: =t =subj v v, -k t Mary :: subj -k +k t, -k will :: =v +k t v, -k =subj v John :: subj -k eat :: =obj =subj v cake :: obj 120 / 201

35 Producing a string from a derivation tree t +k t, -k What do we need to have computed at the +k t, -k node, in order to compute the final string at the t node? Mary will think John will eat cake 121 / 201

36 Producing a string from a derivation tree t +k t, -k What do we need to have computed at the +k t, -k node, in order to compute the final string at the t node? Mary will think John will eat cake This tree would do: < will :: +k t > Mary :: -k think :: < John :: > > But all we actually need to know is: What s the string corresponding to the part that s going to move to check -k? What s the string corresponding to the leftovers? will :: > < These questions are answered by the tuple will think John will eat cake, Mary eat :: cake :: 121 / 201

37 Producing a string from a derivation tree t +k t, -k will :: =v +k t v, -k What do we need to have computed at the v, -k node, in order to compute the desired tuple will think John will eat cake, Mary at the +k t, -k node? 122 / 201

38 Producing a string from a derivation tree t +k t, -k will :: =v +k t v, -k What do we need to have computed at the v, -k node, in order to compute the desired tuple will think John will eat cake, Mary at the +k t, -k node? This tree would do: > Mary :: -k think :: v < > But all we actually need to know is: What s the string corresponding to the part that s going to move to check -k? John :: > What s the string corresponding to the leftovers? will :: > < These questions are answered by the tuple think John will eat cake, Mary eat :: cake :: 122 / 201

39 Producing a string from a derivation tree t will :: =v +k t +k t, -k =subj v v, -k Mary :: subj -k What do we need to have computed at the =subj v node, in order to compute the desired tuple think John will eat cake, Mary at the v, -k node? / 201

40 Producing a string from a derivation tree t will :: =v +k t +k t, -k =subj v v, -k Mary :: subj -k What do we need to have computed at the =subj v node, in order to compute the desired tuple think John will eat cake, Mary at the v, -k node? This tree would do: < think :: =subj v John :: > will :: > > < But all we actually need to know is: What s the string corresponding to the entire tree? (The leftovers after no movement.) This question is answered by the string think John will eat cake eat :: cake :: 123 / 201

41 What matters in a (derived) tree This tree: x -f -g becomes a tuple of categorized strings: s :: x, t :: -f, u :: -g or, equivalently, a tuple-of-strings, categorized by a tuple-of-categories: s, t, u :: x, -f, -g 0 0 (Michaelis 2001, Stabler and Keenan 2003) 124 / 201

42 MCFG rules t +k t, -k will :: =v +k t v, -k =subj v Mary :: subj -k t 2t 1 :: t t 1, t 2 :: +k t, -k Mary will think John will eat cake :: t will think John will eat cake, Mary :: +k t, -k st 1, t 2 :: +k t, -k s :: =v +k t t 1, t 2 :: v, -k will think John will eat cake, Mary :: +k t, -k will :: =v +k t think John will eat cake, Mary :: v, -k s, t :: v, -k s :: =subj v t :: subj -k think John will eat cake, Mary :: v, -k think John will eat cake :: =subj v Mary :: subj -k 125 / 201

43 One slightly annoying wrinkle We know that this is a valid derivational step: α =f α f What is the corresponding MCFG rule? Selected thing on the right? st :: α s :: =f α t :: f Selected thing on the left? ts :: α s :: =f α t :: f 126 / 201

44 One slightly annoying wrinkle We know that this is a valid derivational step: α =f α f What is the corresponding MCFG rule? Selected thing on the right? st :: α s :: =f α t :: f < p Selected thing on the left? ts :: α s :: =f α t :: f with :: p John :: with :: =d p John :: d 126 / 201

45 One slightly annoying wrinkle We know that this is a valid derivational step: α =f α f What is the corresponding MCFG rule? Selected thing on the right? st :: α s :: =f α t :: f < p Selected thing on the left? ts :: α s :: =f α t :: f > v with :: p John :: John :: < with :: =d p John :: d eat :: v cake :: < =d v John :: d eat :: =d v cake :: 126 / 201

46 One slightly annoying wrinkle Each type needs to record not only the unchecked features, but also whether the expression is lexical. I ll write lexical types as... 1 and non-lexical types as So types of the form =f α 1 act slightly differently from those of the form =f α 0. st :: α 0 s :: =f α 1 t :: f n with John :: p 0 with :: =d p 1 John :: d 1 ts :: α 0 s :: =f α 0 t :: f n John eat cake :: v 0 eat cake :: =d v 0 John :: d / 201

47 Context-free structure =subj v =q =subj v q q +wh q, -wh +wh q, -wh =t +wh q t, -wh 128 / 201

48 Context-free structure =subj v =q =subj v q q +wh q, -wh +wh q, -wh =t +wh q t, -wh General schemas for merge steps (approximate): γ, α 1,..., α j, β 1,..., β k =fγ, α 1,..., α j f, β 1,..., β k γ, α 1,..., α j, δ, β 1,..., β k =fγ, α 1,..., α j fδ, β 1,..., β k General schemas for move steps (approximate): γ, α 1,..., α i 1, α i+1,..., α k +fγ, α 1,..., α i 1, -f, α i+1,..., α k γ, α 1,..., α i 1, δ, α i+1,..., α k +fγ, α 1,..., α i 1, -fδ, α i+1,..., α k 128 / 201

49 Context-free structure =subj v =q =subj v q q +wh q, -wh +wh q, -wh =t +wh q t, -wh General schemas for merge steps (approximate): γ, α 1,..., α j, β 1,..., β k =fγ, α 1,..., α j f, β 1,..., β k γ, α 1,..., α j, δ, β 1,..., β k =fγ, α 1,..., α j fδ, β 1,..., β k General schemas for move steps (approximate): γ, α 1,..., α i 1, α i+1,..., α k +fγ, α 1,..., α i 1, -f, α i+1,..., α k γ, α 1,..., α i 1, δ, α i+1,..., α k +fγ, α 1,..., α i 1, -fδ, α i+1,..., α k move steps change something without combining it with anything Compare with unary CFG rules, or type-raising in CCG, or / 201

50 Three schemas for merge rules: st, t 1,..., t k :: γ, α 1,..., α k 0 s :: =fγ 1 t, t 1,..., t k :: f, α 1,..., α k n ts, s 1,..., s j, t 1,..., t k :: γ, α 1,..., α j, β 1,..., β k 0 s, s 1,..., s j :: =fγ, α 1,..., α j 0 t, t 1,..., t k :: f, β 1,..., β k n s, s 1,..., s j, t, t 1,..., t k :: γ, α 1,..., α j, δ, β 1,..., β k 0 s, s 1,..., s j :: =fγ, α 1,..., α j n t, t 1,..., t k :: fδ, β 1,..., β k n Two schemas for move rules: s is, s 1,..., s i 1, s i+1,..., s k :: γ, α 1,..., α i 1, α i+1,..., α k 0 s, s 1,..., s i,..., s k :: +fγ, α 1,..., α i 1, -f, α i+1,..., α k 0 s, s 1,..., s i,..., s k :: γ, α 1,..., α i 1, δ, α i+1,..., α k 0 s, s 1,..., s i,..., s k :: +fγ, α 1,..., α i 1, -fδ, α i+1,..., α k / 201

51 Part 1: Grammars and cognitive hypotheses What is a grammar? What can grammars do? Concrete illustration of a target: Surprisal Parts 2 4: Assembling the pieces Minimalist Grammars (MGs) MGs and MCFGs Probabilities on MGs Part 5: Learning and wrap-up Something slightly different: Learning model Recap and open questions

52 References I Billot, S. and Lang, B. (1989). The structure of shared forests in ambiguous parsing. In Proceedings of the 1989 Meeting of the Association of Computational Linguistics. Chomsky, N. (1965). Aspects of the Theory of Syntax. MIT Press, Cambridge, MA. Chomsky, N. (1980). Rules and Representations. Columbia University Press, New York. Ferreira, F. (2005). Psycholinguistics, formal grammars, and cognitive science. The Linguistic Review, 22: Frazier, L. and Clifton, C. (1996). Construal. MIT Press, Cambridge, MA. Gärtner, H.-M. and Michaelis, J. (2010). On the Treatment of Multiple-Wh Interrogatives in Minimalist Grammars. In Hanneforth, T. and Fanselow, G., editors, Language and Logos, pages Akademie Verlag, Berlin. Gibson, E. and Wexler, K. (1994). Triggers. Linguistic Inquiry, 25: Hale, J. (2006). Uncertainty about the rest of the sentence. Cognitive Science, 30:643 Âŋ672. Hale, J. T. (2001). A probabilistic earley parser as a psycholinguistic model. In Proceedings of the Second Meeting of the North American Chapter of the Association for Computational Linguistics. Hunter, T. (2011). Insertion Minimalist Grammars: Eliminating redundancies between merge and move. In Kanazawa, M., Kornai, A., Kracht, M., and Seki, H., editors, The Mathematics of Language (MOL 12 Proceedings), volume 6878 of LNCS, pages , Berlin Heidelberg. Springer. Hunter, T. and Dyer, C. (2013). Distributions on minimalist grammar derivations. In Proceedings of the 13th Meeting on the Mathematics of Language.

53 References II Koopman, H. and Szabolcsi, A. (2000). Verbal Complexes. MIT Press, Cambridge, MA. Lang, B. (1988). Parsing incomplete sentences. In Proceedings of the 12th International Conference on Computational Linguistics, pages Levy, R. (2008). Expectation-based syntactic comprehension. Cognition, 106(3): Michaelis, J. (2001). Derivational minimalism is mildly context-sensitive. In Moortgat, M., editor, Logical Aspects of Computational Linguistics, volume 2014 of LNCS, pages Springer, Berlin Heidelberg. Miller, G. A. and Chomsky, N. (1963). Finitary models of language users. In Luce, R. D., Bush, R. R., and Galanter, E., editors, Handbook of Mathematical Psychology, volume 2. Wiley and Sons, New York. Morrill, G. (1994). Type Logical Grammar: Categorial Logic of Signs. Kluwer, Dordrecht. Nederhof, M. J. and Satta, G. (2008). Computing partition functions of pcfgs. Research on Language and Computation, 6(2): Seki, H., Matsumara, T., Fujii, M., and Kasami, T. (1991). On multiple context-free grammars. Theoretical Computer Science, 88: Stabler, E. P. (2006). Sidewards without copying. In Wintner, S., editor, Proceedings of The 11th Conference on Formal Grammar, pages , Stanford, CA. CSLI Publications. Stabler, E. P. (2011). Computational perspectives on minimalism. In Boeckx, C., editor, The Oxford Handbook of Linguistic Minimalism. Oxford University Press, Oxford. Stabler, E. P. and Keenan, E. L. (2003). Structural similarity within and among languages. Theoretical Computer Science, 293:

54 References III Vijay-Shanker, K., Weir, D. J., and Joshi, A. K. (1987). Characterizing structural descriptions produced by various grammatical formalisms. In Proceedings of the 25th Meeting of the Association for Computational Linguistics, pages Weir, D. (1988). Characterizing mildly context-sensitive grammar formalisms. PhD thesis, University of Pennsylvania. Yngve, V. H. (1960). A model and an hypothesis for language structure. In Proceedings of the American Philosophical Society, volume 104, pages