Why Synchronous Tree Substitution Grammars?

Transcription

1 Why ynchronous Tr ustitution Grammars? Andras Maltti Univrsitat Rovira i Virgili Tarragona, pain andras.maltti@urv.cat Los Angls Jun 4, 2010 Why ynchronous Tr ustitution Grammars? A. Maltti 1

2 Motivation ynchronous Tr ustitution Grammars Wight: 1 Why ynchronous Tr ustitution Grammars? A. Maltti 2

3 Motivation ynchronous Tr ustitution Grammars CONJ wa Wight: Why ynchronous Tr ustitution Grammars? A. Maltti 2

4 Motivation ynchronous Tr ustitution Grammars NP 1 VP CONJ V NP 2 wa V NP 1 NP 2 Wight: Why ynchronous Tr ustitution Grammars? A. Maltti 2

5 Motivation ynchronous Tr ustitution Grammars NP 1 VP CONJ V NP 2 wa V NP 1 NP 2 saw ra aa Wight: Why ynchronous Tr ustitution Grammars? A. Maltti 2

6 Motivation ynchronous Tr ustitution Grammars NP VP CONJ DT N V NP wa V NP NP th saw ra aa N Wight: Why ynchronous Tr ustitution Grammars? A. Maltti 2

7 Motivation ynchronous Tr ustitution Grammars NP VP CONJ DT N V NP wa V NP NP th oy saw ra aa N atfl Wight: Why ynchronous Tr ustitution Grammars? A. Maltti 2

8 Motivation ynchronous Tr ustitution Grammars NP VP CONJ DT N V NP wa V NP NP th oy saw DT N ra aa N N th atfl Wight: Why ynchronous Tr ustitution Grammars? A. Maltti 2

9 Motivation ynchronous Tr ustitution Grammars NP VP CONJ DT N V NP wa V NP NP th oy saw DT N ra aa N N th door atfl ala Wight: Why ynchronous Tr ustitution Grammars? A. Maltti 2

10 Motivation ynchronous Tr ustitution Grammars (cont d) Advantags simpl and natural modl asy to train (from linguistic rsourcs) symmtric (Ovious) Disadvantags computs joint-proaility ( gnrativ story) no stat havior ( local havior) Implmntation xtndd top-down tr transducr in TIBURON [MAY, KNIGHT 06] Why ynchronous Tr ustitution Grammars? A. Maltti 3

11 Motivation ynchronous Tr ustitution Grammars (cont d) Advantags simpl and natural modl asy to train (from linguistic rsourcs) symmtric (Ovious) Disadvantags computs joint-proaility ( gnrativ story) no stat havior ( local havior) Implmntation xtndd top-down tr transducr in TIBURON [MAY, KNIGHT 06] Why ynchronous Tr ustitution Grammars? A. Maltti 3

12 Motivation ynchronous Tr ustitution Grammars (cont d) Advantags simpl and natural modl asy to train (from linguistic rsourcs) symmtric (Ovious) Disadvantags computs joint-proaility ( gnrativ story) no stat havior ( local havior) Implmntation xtndd top-down tr transducr in TIBURON [MAY, KNIGHT 06] Why ynchronous Tr ustitution Grammars? A. Maltti 3

13 Motivation ynchronous Tr ustitution Grammars (cont d) ynchronous tr sustitution grammar rul: NP 1 VP V NP 2 w V NP 1 NP 2 Corrsponding xtndd top-down tr transducr rul: x 1 VP x 2 x 3 w V x 2 NP x 1 NP x 3 Why ynchronous Tr ustitution Grammars? A. Maltti 4

14 Motivation Extndd Top-down Tr Transducr Advantags input-drivn modl (can asily comput conditional proaility) stat havior Disadvantags (also of TG) not inarizal [AHO, ULLMAN 72; ZHANG, HUANG, GILDEA, KNIGHT 06] infficint input/output rstriction (BAR-HILLEL construction) [M., ATTA 10] not composal [ARNOLD, DAUCHET 82] Why ynchronous Tr ustitution Grammars? A. Maltti 5

15 Motivation Extndd Top-down Tr Transducr Advantags input-drivn modl (can asily comput conditional proaility) stat havior Disadvantags (also of TG) not inarizal [AHO, ULLMAN 72; ZHANG, HUANG, GILDEA, KNIGHT 06] infficint input/output rstriction (BAR-HILLEL construction) [M., ATTA 10] not composal [ARNOLD, DAUCHET 82] Why ynchronous Tr ustitution Grammars? A. Maltti 5

16 Motivation Extndd Bottom-up Tr Transducr Top-down tr transducr rul: x 1 VP x 2 x 3 w V x 2 NP x 1 NP x 3 Corrsponding xtndd ottom-up tr transducr rul: NP x 1 VP V NP x 2 x 3 w x 2 x 1 x 3 Why ynchronous Tr ustitution Grammars? A. Maltti 6

17 Motivation Extndd Bottom-up Tr Transducr (cont d) Thorm For vry TG w can construct an uivalnt xtndd ottom-up tr transducr in linar tim. Qustion Do thy hav ttr proprtis? Why ynchronous Tr ustitution Grammars? A. Maltti 7

18 Motivation Extndd Bottom-up Tr Transducr (cont d) Thorm For vry TG w can construct an uivalnt xtndd ottom-up tr transducr in linar tim. Qustion Do thy hav ttr proprtis? Why ynchronous Tr ustitution Grammars? A. Maltti 7

19 Roadmap Extndd Multi Bottom-up Tr Transducrs 1 Motivation 2 Extndd Multi Bottom-up Tr Transducrs 3 BAR-HILLEL Construction 4 Composition Construction Why ynchronous Tr ustitution Grammars? A. Maltti 8

20 yntax Extndd Multi Bottom-up Tr Transducrs Dfinition Wightd xtndd multi ottom-up tr transducr (XMBOT) is a systm (Q, Σ,, F, R) with Q rankd alphat of stats Σ and rankd alphats of input and output symols F Q 1 final stats w R finit st of ruls l r with w R 0, linar l T Σ (Q(X)), linar r Q(T (X)) such that var(l) = var(r) Why ynchronous Tr ustitution Grammars? A. Maltti 9

21 yntax (cont d) Extndd Multi Bottom-up Tr Transducrs Dfinition XMBOT (Q, Σ,, F, R) is propr if {l, r} Q(X) for vry l w r R. Why ynchronous Tr ustitution Grammars? A. Maltti 10

22 yntax (cont d) Extndd Multi Bottom-up Tr Transducrs Dfinition XMBOT (Q, Σ,, F, R) is propr if {l, r} Q(X) for vry l w r R. Exampl Disallowd rul for proprnss: p w x 1 x 2 x 3 x 2 x 1 x 3 Why ynchronous Tr ustitution Grammars? A. Maltti 10

23 Extndd Multi Bottom-up Tr Transducrs yntax An Exampl Exampl Q = {f (1), (2) } and F = {f } Σ = {a (1), (1), (0) } and = Σ { (2) } th following ruls a x 1 x 2 1 a x 1 a x 2 1 x 1 x 2 1 x 1 x 2 f 1 x 1 x 2 x 1 x 2 Why ynchronous Tr ustitution Grammars? A. Maltti 11

24 mantics Extndd Multi Bottom-up Tr Transducrs p x 1 x 2 w x 3 x 4 δ x 1 x 2 x 4 x 3 Why ynchronous Tr ustitution Grammars? A. Maltti 12

25 mantics Extndd Multi Bottom-up Tr Transducrs p t u 3 u 4 w = M δ t u 1 u 1 u 2 u 2 u 4 u 3 Why ynchronous Tr ustitution Grammars? A. Maltti 12

26 mantics Extndd Multi Bottom-up Tr Transducrs p t u 3 u 4 w = M δ t u 1 u 1 u 2 u 2 u 4 u 3 mantics wt(ξ 1 w 1 = M wn 1 = M ξ n ) = w 1... w n 1 wt(t, u) = F,d : t= M(u) wt(d) Why ynchronous Tr ustitution Grammars? A. Maltti 12

27 Extndd Multi Bottom-up Tr Transducrs mantics An Exampl Usd rul (, ) Exampl a Why ynchronous Tr ustitution Grammars? A. Maltti 13

28 Extndd Multi Bottom-up Tr Transducrs mantics An Exampl Usd rul (, ) Exampl a a = M Why ynchronous Tr ustitution Grammars? A. Maltti 13

29 Extndd Multi Bottom-up Tr Transducrs mantics An Exampl Usd rul ((x 1, x 2 )) ((x 1 ), (x 2 )) Exampl a a a = M = M Why ynchronous Tr ustitution Grammars? A. Maltti 13

30 Extndd Multi Bottom-up Tr Transducrs mantics An Exampl Usd rul ((x 1, x 2 )) ((x 1 ), (x 2 )) Exampl a a a a = M = M = M Why ynchronous Tr ustitution Grammars? A. Maltti 13

31 Extndd Multi Bottom-up Tr Transducrs mantics An Exampl Usd rul a((x 1, x 2 )) (a(x 1 ), a(x 2 )) Exampl a a a a a a = M = M = M = M Why ynchronous Tr ustitution Grammars? A. Maltti 13

32 Extndd Multi Bottom-up Tr Transducrs mantics An Exampl Usd rul (x 1, x 2 ) f ((x 1, x 2 )) Exampl a = M a = M a = M a = M a a = M f a a Why ynchronous Tr ustitution Grammars? A. Maltti 13

33 Roadmap BAR-HILLEL Construction 1 Motivation 2 Extndd Multi Bottom-up Tr Transducrs 3 BAR-HILLEL Construction 4 Composition Construction Why ynchronous Tr ustitution Grammars? A. Maltti 14

34 BAR-HILLEL Construction On-ymol Normal Form Dfinition XMBOT (Q, Σ,, F, R) is in on-symol normal form if xactly on input or output symol occurs in ach rul. Thorm For vry propr XMBOT thr xists an uivalnt XMBOT in on-symol normal form. It can constructd in linar tim. Corollary For vry propr XMBOT th transition from joint-distriution to conditional-distriution is linar tim. Why ynchronous Tr ustitution Grammars? A. Maltti 15

35 BAR-HILLEL Construction On-ymol Normal Form Dfinition XMBOT (Q, Σ,, F, R) is in on-symol normal form if xactly on input or output symol occurs in ach rul. Thorm For vry propr XMBOT thr xists an uivalnt XMBOT in on-symol normal form. It can constructd in linar tim. Corollary For vry propr XMBOT th transition from joint-distriution to conditional-distriution is linar tim. Why ynchronous Tr ustitution Grammars? A. Maltti 15

36 BAR-HILLEL Construction On-ymol Normal Form Dfinition XMBOT (Q, Σ,, F, R) is in on-symol normal form if xactly on input or output symol occurs in ach rul. Thorm For vry propr XMBOT thr xists an uivalnt XMBOT in on-symol normal form. It can constructd in linar tim. Corollary For vry propr XMBOT th transition from joint-distriution to conditional-distriution is linar tim. Why ynchronous Tr ustitution Grammars? A. Maltti 15

37 BAR-HILLEL Construction On-ymol Normal Form (cont d) Rul not in on-symol normal form: w x 1 x 2 x 1 x 2 Rplacmnt ruls for this rul: x 1 x 2 w 1 x 1 x x 1 x 2 x x 2 1 x 1 x 2 x 1 x 2 Why ynchronous Tr ustitution Grammars? A. Maltti 16

38 BAR-HILLEL Construction Binarization Dfinition An XMBOT is fully inarizd if ach rul contains at most 3 stats. ( 2 in ach lft-hand sid) Thorm Evry propr XMBOT can fully inarizd in linar tim. Proof. First inariz th trs in th ruls and thn transform into on-symol normal form. Why ynchronous Tr ustitution Grammars? A. Maltti 17

39 BAR-HILLEL Construction Binarization Dfinition An XMBOT is fully inarizd if ach rul contains at most 3 stats. ( 2 in ach lft-hand sid) Thorm Evry propr XMBOT can fully inarizd in linar tim. Proof. First inariz th trs in th ruls and thn transform into on-symol normal form. Why ynchronous Tr ustitution Grammars? A. Maltti 17

40 Binarization (cont d) BAR-HILLEL Construction x 1 x 2 x 3 x 4 w x 3 x 1 x 4 x 2 Comparison In gnral, TG cannot inarizd, ut popl try... [ZHANG, HUANG, GILDEA, KNIGHT 06; DENERO, PAUL, KLEIN 09] Why ynchronous Tr ustitution Grammars? A. Maltti 18

41 BAR-HILLEL Construction BAR-HILLEL Construction Dfinition Th input product of a wightd tr transformation τ : T Σ T with a powr sris ϕ: Σ is τ (s, t) = τ(s, t) ϕ(yd(s)). s 1 s 2 s 1 s 3 s 1 s 2 s 1 s 2 s 2 s 3 wt(s 1, γ, s 2 ) s 1 γ s 2 Why ynchronous Tr ustitution Grammars? A. Maltti 19

42 BAR-HILLEL Construction BAR-HILLEL Construction (cont d) Thorm Th input product of an XMBOT M with a WA can computd in tim O( M 3 ). Not Th output product of an XMBOT M with a WA can computd in tim O( M 2 rk(m)+2 ). Comparison Th input/output product of an TG M with a WA can computd in tim O( M 2 rk(m)+5 ). [M., ATTA 10] Why ynchronous Tr ustitution Grammars? A. Maltti 20

43 BAR-HILLEL Construction BAR-HILLEL Construction (cont d) Thorm Th input product of an XMBOT M with a WA can computd in tim O( M 3 ). Not Th output product of an XMBOT M with a WA can computd in tim O( M 2 rk(m)+2 ). Comparison Th input/output product of an TG M with a WA can computd in tim O( M 2 rk(m)+5 ). [M., ATTA 10] Why ynchronous Tr ustitution Grammars? A. Maltti 20

44 Roadmap Composition Construction 1 Motivation 2 Extndd Multi Bottom-up Tr Transducrs 3 BAR-HILLEL Construction 4 Composition Construction Why ynchronous Tr ustitution Grammars? A. Maltti 21

45 Composition Construction Composition of TG t 1 δ t 2 t 3 δ t n 4 t n 3 δ = t 1 t 2 t 3 t 4 t n 3 δ t 2 t 1 δ = t 4 t 3 δ t n 2 t n 3 t n 2 t n 1 t n t n 2 t n 1 t n Conclusion TGs ar not composal! t n 1 t n Why ynchronous Tr ustitution Grammars? A. Maltti 22

46 Composition Construction Composition of TG t 1 δ t 2 t 3 δ t n 4 t n 3 δ = t 1 t 2 t 3 t 4 t n 3 δ t 2 t 1 δ = t 4 t 3 δ t n 2 t n 3 t n 2 t n 1 t n t n 2 t n 1 t n Conclusion TGs ar not composal! t n 1 t n Why ynchronous Tr ustitution Grammars? A. Maltti 22

47 Composition Construction Composition of TG t 1 δ t 2 t 3 δ t n 4 t n 3 δ = t 1 t 2 t 3 t 4 t n 3 δ t 2 t 1 δ = t 4 t 3 δ t n 2 t n 3 t n 2 t n 1 t n t n 2 t n 1 t n Conclusion TGs ar not composal! t n 1 t n Why ynchronous Tr ustitution Grammars? A. Maltti 22

48 Composition Construction Composition Construction Dfinition for XMBOT M = (Q, Σ, Γ, F, R) and N = (Q, Γ,, G, P) construct with thr typs of ruls: M ; N = (Q(Q ), Σ,, F(G), R ) 1 input-consuming ruls constructd from input-consuming ruls of R (with thir wight) 2 psilon ruls constructd from psilon-ruls of P 3 psilon ruls constructd from an psilon rul of R followd y an input consuming rul of P (product of th wights) Why ynchronous Tr ustitution Grammars? A. Maltti 23

49 Composition Construction Composition construction (cont d) Exampl Input consuming rul of R and rsulting rul: 1 x 1 x 2 1 p 1 p 2 2 x 1 x 2 2 w w x 2 p 2 x 1 x 2 Why ynchronous Tr ustitution Grammars? A. Maltti 24

50 Composition Construction Composition construction (cont d) Exampl Epsilon rul of P and rsulting rul: p 1 p w α 1 p 1 p 2 x 1 x 2 w p α 1 p 2 x 1 x 2 Why ynchronous Tr ustitution Grammars? A. Maltti 24

51 Composition Construction Composition construction (cont d) Exampl Epsilon rul of R and input consuming of P and rsulting rul: 1 w 1 x 1 x 2 γ x 2 γ p 2 x 1 x 2 p w 2 x 2 1 p 1 p 2 x 1 x 2 w 1 γ p 2 x 1 x 2 w 2 p x 2 Why ynchronous Tr ustitution Grammars? A. Maltti 24

52 Composition Construction Composition construction (cont d) Not Th constructd XMBOT might non-propr. Thorm For all propr XMBOTs M and N such that M has no cyclic input psilon ruls or N has no cyclic output psilon ruls, thn thr xists a propr XMBOT that computs th composition of th transformations computd y M and N. Why ynchronous Tr ustitution Grammars? A. Maltti 25

53 Composition Construction Composition construction (cont d) Not Th constructd XMBOT might non-propr. Thorm For all propr XMBOTs M and N such that M has no cyclic input psilon ruls or N has no cyclic output psilon ruls, thn thr xists a propr XMBOT that computs th composition of th transformations computd y M and N. Why ynchronous Tr ustitution Grammars? A. Maltti 25

54 ummary Composition Construction Algorithm \ Dvic TG XMBOT Binarization O( M ) Input product O( M 2 rk(m)+5 ) O( M 3 ) Output product O( M 2 rk(m)+5 ) O( M 2 rk(m)+2 ) Composition O( M 1 M 2 rk(m1)+1 ) Why ynchronous Tr ustitution Grammars? A. Maltti 26

55 ummary Composition Construction Algorithm \ Dvic TG XMBOT Binarization O( M ) Input product O( M 2 rk(m)+5 ) O( M 3 ) Output product O( M 2 rk(m)+5 ) O( M 2 rk(m)+2 ) Composition O( M 1 M 2 rk(m1)+1 ) Why ynchronous Tr ustitution Grammars? A. Maltti 26

56 ummary Composition Construction Algorithm \ Dvic TG XMBOT Binarization O( M ) Input product O( M 2 rk(m)+5 ) O( M 3 ) Output product O( M 2 rk(m)+5 ) O( M 2 rk(m)+2 ) Composition O( M 1 M 2 rk(m1)+1 ) Why ynchronous Tr ustitution Grammars? A. Maltti 26

57 ummary Composition Construction Algorithm \ Dvic TG XMBOT Binarization O( M ) Input product O( M 2 rk(m)+5 ) O( M 3 ) Output product O( M 2 rk(m)+5 ) O( M 2 rk(m)+2 ) Composition O( M 1 M 2 rk(m1)+1 ) Why ynchronous Tr ustitution Grammars? A. Maltti 26

58 ummary Composition Construction Algorithm \ Dvic TG XMBOT Binarization O( M ) Input product O( M 2 rk(m)+5 ) O( M 3 ) Output product O( M 2 rk(m)+5 ) O( M 2 rk(m)+2 ) Composition O( M 1 M 2 rk(m1)+1 ) Rvrsal O( M ) Prs. of REC Why ynchronous Tr ustitution Grammars? A. Maltti 26

59 ummary Composition Construction Algorithm \ Dvic TG XMBOT Binarization O( M ) Input product O( M 2 rk(m)+5 ) O( M 3 ) Output product O( M 2 rk(m)+5 ) O( M 2 rk(m)+2 ) Composition O( M 1 M 2 rk(m1)+1 ) Rvrsal O( M ) Prs. of REC Why ynchronous Tr ustitution Grammars? A. Maltti 26

60 Rfrncs (1/2) Composition Construction AHO, ULLMAN: Th thory of parsing, translation, and compiling. Prntic Hall ARNOLD, DAUCHET: Morphisms t imorphisms d arrs. Thort. Comput. ci. 20(1):33 93, 1982 CHIANG, KNIGHT: An introduction to synchronous grammars. Tutorial at ACL DENERO, PAUL, KLEIN: Asynchronous inarization for synchronous grammars. In ACL, p , 2009 ENGELFRIET: Bottom-up and top-down tr transformations a comparison. Math. ystms Thory 9(3), 1975 ENGELFRIET, LILIN, MALETTI: Extndd multi ottom-up tr transducrs composition and dcomposition. Acta Inf. 46(8): , 2009 GÉCEG, TEINBY: Tr Automata. Akadémiai Kiadó, Budapst 1984 Why ynchronous Tr ustitution Grammars? A. Maltti 27

61 Composition Construction Rfrncs (2/2) LILIN: Un généralisation ds transducturs d états finis d arrs: ls -transducturs. Univrsité d Lill LILIN: Propriétés d clôtur d un xtnsion d transducturs d arrs détrminists. In CAAP, LNC 112, p , 1981 MALETTI, ATTA: Parsing and translation algorithms asd on wightd xtndd tr transducrs. In ATANLP 2010 MAY, KNIGHT: TIBURON a wightd tr automata toolkit. In CIAA, LNC 4094, p , 2006 ZHANG, HUANG, GILDEA, KNIGHT: ynchronous inarization for machin translation. In NAACL-HLT, p , 2006 Thank you for your attntion! Why ynchronous Tr ustitution Grammars? A. Maltti 28