ynchronous Grammars
ynchronous grammars are a way of simultaneously generating pairs of recursively related strings ynchronous grammar w wʹ
ynchronous grammars were originally invented for programming language compilation ynchronous grammar for i := 1 to 10 do begin n := n + i end mov ax, 1 loop: add bx, ax cmp ax, 10 jle loop
ynchronous grammars have been proposed as a way of doing semantic interpretation ynchronous grammar I open the box open (me, box )
ynchronous grammars have been used for syntax-based machine translation ynchronous grammar I open the box watashi wa hako wo akemasu
ynchronous grammars can do much fancier transformations than finite-state methods The boy stated that the student said that the teacher danced shoonen ga gakusei ga sensei ga odotta to itta to hanashita boy student teacher danced that said that stated
ynchronous grammars can do much fancier transformations than finite-state methods that John saw Peter help the children swim dat Jan Piet de kinderen zag helpen zwemmen John Peter the children saw help swim
Overview Definitions Properties Algorithms Extensions
Definitions
Context-free grammars NP P NP I NP the box P NP open NP P I NP open the box
Context-free grammars NP P NP watashi wa NP hako wo P NP akemasu NP P watashi wa NP hako wo akemasu
ynchronous CFGs NP 1 P 2 NP 1 P 2 NP I NP watashi wa NP the box NP hako wo P 1 NP 2 P NP 2 1 open akemasu
ynchronous CFGs NP 1 P 2, NP 1 P 2 NP I, watashi wa NP the box, hako wo P 1 NP 2, NP 2 1 open, akemasu
ynchronous CFGs 1 1 NP 2 P 3 NP 2 P 3 I 4 NP 5 watashi wa NP 5 4 open the box hako wo akemasu
Adding probabilities 0.3 NP 1 P 2, NP 1 P 2 0.1 NP I, watashi wa 0.6 NP the box, hako wo 0.5 P 1 NP 2, NP 2 1 0.2 open, akemasu
ynchronous CFGs 1 1 NP 2 P 3 NP 2 P 3 I 4 NP 5 watashi wa NP 5 4 open the box hako wo akemasu Derivation probability: 0.3 0.1 0.5 0.6 0.2
Other notations P ( 1 NP 2, NP 2 1 ) yntax directed translation schema (Aho and Ullman; Lewis and tearns) (P 1 NP 2, P NP 2 1 ) P NP Inversion transduction grammar (Wu) P [1,2] ( NP ) [2,1] NP Multitext grammar (Melamed)
Properties
Chomsky normal form X Y Z X a
Chomsky normal form A (((B C) D) E) F rank 5
Chomsky normal form A [[[B C] D] E] F rank 5 A 1 F 1 2 E 2 3 D rank 2 3 B C
A hierarchy of synchronous CFGs 1-CFG 2-CFG = 3-CFG = 4-CFG = 1-CFG 2-CFG = 3-CFG 4-CFG = = ITG (Wu, 1997)
ynchronous CNF? A (B 1 [C 2 D 3 ], [C 2 D 3 ] B 1 ) rank 3
ynchronous CNF? A (B 1 [C 2 D 3 ], [C 2 D 3 ] B 1 ) rank 3 A (B 1 1 2, 1 2 B 1 ) 1 (C 1 D 2, C 1 D 2 ) rank 2
ynchronous CNF? A (B 1 C 2 D 3 E 4, C 2 E 4 B 1 D 3 ) rank 4 A ([B 1 C 2 ] D 3 E 4, [C 2 E 4 B 1 ] D 3 ) A (B 1 [C 2 D 3 ] E 4, [C 2 E 4 B 1 D 3 ]) A (B 1 C 2 [D 3 E 4 ], C 2 [E 4 B 1 D 3 ])
ynchronous CNF? 1 2 3 A (B 1 C 2 D 3, C 2 D 3 B 1 ) 1 B 2 C 3 D 1 2 3 4 1 B A (B 1 C 2 D 3 E 4, C 2 E 4 B 1 D 3 ) 2 C 3 D 4 E
A hierarchy of synchronous CFGs 1-CFG 2-CFG = 3-CFG = 4-CFG = 1-CFG 2-CFG = 3-CFG 4-CFG = = ITG (Wu, 1997)
Algorithms
Overview Translation Bitext parsing
Review: CKY NP P NP I NP the box P NP open NP I open the box
Review: CKY NP P NP I NP the box P NP open NP NP I open the box
Review: CKY NP P NP I NP the box P NP open NP P NP I open the box
Review: CKY NP P NP I NP the box P NP open NP P I open the box
Review: CKY NP P NP I NP the box P NP open I open the box
Review: CKY 2 NP 3 2 P 4 open the box O(n 3 ) ways of matching
Translation NP 1 P 2 I 3 NP 4 open the box O(n 3 ) I open the box
Translation O(n) NP 1 P 2 NP 1 P 2 I 3 NP 4 watashi wa NP 4 3 open the box hako wo akemasu O(n 3 ) O(n) I open the box watashi wa hako wo akemasu
Translation What about A B C D E O(n 5 ) ways of combining?
1 A Translation A B C D E 2 B C D E flatten O(n) translate O(n) A C E D B parse O(n 3 )
Bitext parsing NP 1 P 2 NP 1 P 2 I 3 NP 4 watashi wa NP 4 3 open the box hako wo O(n? ) akemasu I open the box watashi wa hako wo akemasu
Bitext parsing NP 1 NP 1 3 NP 4 NP 4 3 I open the box watashi wa hako wo akemasu Consider rank-2 synchronous CFGs for now
Bitext parsing NP 1 P 2 NP 1 P 2 3 NP 4 NP 4 3 I open the box watashi wa hako wo akemasu
Bitext parsing NP 1 P 2 NP 1 P 2 I open the box watashi wa hako wo akemasu
Bitext parsing I open the box watashi wa hako wo akemasu
Bitext parsing P 2 P 2 3 NP 4 NP 4 3 open the box hako wo akemasu O(n 6 ) ways of matching
Bitext parsing A A B 1 C 2 D 3 E 4 C 2 E 4 B 1 D 3 O(n 10 ) ways of combining!
Algorithms Translation: easy Bitext parsing: polynomial in n but worst-case exponential in rank
Algorithms Translation with an n-gram language model Offline rescoring Intersect grammar and LM (Wu 1996; Huang et al. 2005): slower Hybrid approaches (Chiang 2005; Zollman and enugopal 2006)
Extensions
Limitations of synchronous CFGs NP P NP P John NP Marie PP misses Mary manque P NP à Jean
One solution 1 NP 2 misses NP 3 1 NP 3 manque à NP 2 John Mary Marie Jean
ynchronous tree substitution grammars NP 1 NP P NP 2 P John NP 2 PP NP Jean misses manque P NP 1 NP NP à Mary Marie
ynchronous tree substitution grammars NP 1 P NP 2 P John NP 2 Marie PP misses Mary manque P NP 1 à Jean
Limitations of synchronous TGs dat Jan Piet de kinderen zag helpen zwemmen that John saw Peter help the children swim This pattern extends to n nouns and n verbs
Limitations of synchronous TGs NP 1 P NP 1 P John 2 Jan 2 saw? zag? Piet de kinderen helpen zwemmen Peter help the children swim
Tree-adjoining grammar NP P zwemmen NP P helpen de kinderen Piet * t t
zwemmen NP P zwemmen NP P helpen de kinderen t NP P helpen Piet Piet * NP P t t de kinderen t
ynchronous TAG 1 1 NP 2 P 3 NP 2 P 3 zwemmen the children de kinderen swim t
1 1 NP 2 P 3 NP 2 P 3 zwemmen the children de kinderen swim t 1 1 NP 2 P 3 NP 2 P 3 helpen Peter * Piet * help t
1 1 zwemmen NP 2 P 3 NP 2 P 3 helpen Peter Piet help NP 4 P 5 NP 4 P 5 t the children de kinderen swim t
1 NP 2 P 3 Peter 1 zwemmen help NP 4 P 5 NP 2 P 3 helpen the children Piet swim NP 4 P 5 t 1 de kinderen 1 NP 2 P 3 John saw * t NP 2 P 3 Jan * t zag
1 NP 2 P 3 zwemmen John 1 helpen saw NP 4 P 5 NP 2 P 3 zag Peter Jan help NP 6 P 7 NP 4 P 5 t the children swim Piet NP 6 P 7 t de kinderen t
ummary ynchronous grammars are useful for various tasks including translation ome rules in the wild (Chiang, 2005): X (de, s) X (X 1 de X 2, the X 2 of X 1) X (X 1 de X 2, the X 2 that) X (zai X 1 xia, under X 1 ) X (zai X 1 qian, before X 1 ) X (X 1 zhiyi, one of X 1 )
ummary ynchronous context-free grammars vary in power depending on rank Translation is easy; bitext parsing is exponential in rank
ummary Beyond synchronous CFGs, synchronous TGs allow multilevel rules synchronous TAGs allow discontinuous constituents