How to train your multi bottom-up tree transducer

Transcription

1 How to trin your multi bottom-up tree trnsducer Andres Mletti Universität tuttgrt Institute for Nturl Lnguge Processing Azenbergstrße 7074 tuttgrt Germny Abstrct The locl multi bottom-up tree trnsducer is introduced nd relted to the (non-contiguous) synchronous tree sequence substitution grmmr. It is then shown how to obtin weighted locl multi bottom-up tree trnsducer from bilingul nd biprsed corpus. Finlly the problem of non-preservtion of regulrity is ddressed. Three properties tht ensure preservtion re introduced nd it is discussed how to djust the rule extrction process such tht they re utomticlly fulfilled. Introduction A (forml) trnsltion model is t the core of every mchine trnsltion system. Predominntly sttisticl processes re used to instntite the forml model nd derive specific trnsltion device. Brown et l. (990) discuss utomticlly trinble trnsltion models in their seminl pper. However the IBM models of Brown et l. (993) re stringbsed in the sense tht they bse the trnsltion decision on the words nd their surrounding context. Contrry in the field of syntx-bsed mchine trnsltion the trnsltion models hve full ccess to the syntx of the sentences nd cn bse their decision on it. A good exposition to both fields is presented in (Knight 007). In this pper we del exclusively with syntxbsed trnsltion models such s synchronous tree substitution grmmrs (TG) multi bottom-up tree trnsducers (MBOT) nd synchronous tree-sequence substitution grmmrs (TG). Ching (006) gives good introduction to TG which originte from the syntx-directed trnsltion schemes of Aho nd Ullmn (97). Roughly speking n TG hs rules in which two linked nonterminls re replced (t the sme time) by two corresponding trees contining terminl nd nonterminl symbols. In ddition the nonterminls in the two replcement trees re linked which cretes new linked nonterminls to which further rules cn be pplied. Henceforth we refer to these two trees s input nd output tree. MBOT hve been introduced in (Arnold nd Duchet 98; Lilin 98) nd re slightly more expressive thn TG. Roughly speking they llow one replcement input tree nd severl output trees in single rule. This chnge nd the presence of sttes yields mny lgorithmiclly dvntgeous properties such s closure under composition efficient binriztion nd efficient input nd output restriction [see (Mletti 00)]. Finlly TG which hve been derived from rtionl tree reltions (Roult 997) hve been discussed by Zhng et l. (008) Zhng et l. (008b) nd un et l. (009). They re even more expressive thn the locl vrint of the multi bottom-up tree trnsducer (LMBOT) tht we introduce here nd cn hve severl input nd output trees in single rule. In this contribution we restrict MBOT to form tht is prticulrly relevnt in mchine trnsltion. We drop the generl stte behvior of MBOT nd replce it by the common loclity tests tht re lso present in TG TG nd TAG (hieber nd chbes 990; hieber 007). The obtined device is the locl MBOT (LMBOT). Mletti (00) rgued the lgorithmicl dvntges of MBOT over TG nd proposed MBOT s n implementtion lterntive for TG. In prticulr the trining procedure would trin TG; i.e. it would not utilize the dditionl expressive power 85 Proceedings of the 49th Annul Meeting of the Assocition for Computtionl Linguistics pges Portlnd Oregon June c 0 Assocition for Computtionl Linguistics

2 of MBOT. However Zhng et l. (008b) nd un et l. (009) demonstrte tht the dditionl expressivity gined from non-contiguous rules gretly improves the trnsltion qulity. In this contribution we ddress this seprtion nd investigte trining procedure for LMBOT tht llows non-contiguous frgments while preserving the lgorithmic dvntges of MBOT. To this end we introduce rule extrction nd weight trining method for LMBOT tht is bsed on the corresponding procedures for TG nd TG. However generl LMBOT cn be too expressive in the sense tht they llow trnsltions tht do not preserve regulrity. Preservtion of regulrity is n importnt property for efficient representtions nd efficient lgorithms [see (My et l. 00)]. Consequently we present 3 properties tht ensure tht n LMBOT preserves regulrity. In ddition we shortly discuss how these properties could be enforced in the rule extrction procedure. Nottion The set of nonnegtive integers is N. We write [k] for the set {i i k}. We tret functions s specil reltions. For every reltion R A B nd A we write R() = {b B : ( b) R} R = {(b ) ( b) R} where R is clled the inverse of R. Given n lphbet Σ the set of ll words (or sequences) over Σ is Σ of which the empty word is ε. The conctention of two words u nd w is simply denoted by the juxtposition uw. The length of word w = σ σ k with σ i Σ for ll i [k] is w = k. Given i j k the (i j)- spn w[i j] of w is σ i σ i+ σ j. The set T Σ of ll Σ-trees is the smllest set T such tht σ(t) T for ll σ Σ nd t T. We generlly use bold-fce chrcters (like t) for sequences nd we refer to their elements using subscripts (like t i ). Consequently tree t consists of lbeled root node σ followed by sequence t of its children. To improve redbility we sometimes write sequence t t k s t... t k. The positions pos(t) N of tree t = σ(t) re inductively defined by pos(t) = {ε} pos(t) where pos(t) = {ip p pos(t i )}. i t Note tht this yields n undesirble difference between pos(t) nd pos(t) but it will lwys be cler from the context whether we refer to single tree or sequence. Note tht positions re ordered vi the (stndrd) lexicogrphic ordering. Let t T Σ nd p pos(t). The lbel of t t position p is t(p) nd the subtree rooted t position p is t p. Formlly they re defined by { σ if p = ε t(p) = t(ip) = t i (p) t(p) otherwise { t if p = ε t p = t ip = t i p t p otherwise for ll t = σ(t) nd i t. As demonstrted these notions re lso used for sequences. A position p pos(t) is lef (in t) if p / pos(t). Given subset NT Σ we let NT (t) = {p pos(t) t(p) NT p lef in t}. Lter NT will be the set of nonterminls so tht the elements of NT (t) will be the lef nonterminls of t. We extend the notion to sequences t by NT (t) = {ip p NT (t i )}. i t We lso need substitution tht replces subtrees. Let p... p n pos(t) be pirwise incomprble positions nd t... t n T Σ. Then t[p i t i i n] denotes the tree tht is obtined from t by replcing (in prllel) the subtrees t p i by t i for every i [k]. Finlly let us recll regulr tree lnguges. A finite tree utomton M is tuple (Q Σ δ F ) such tht Q is finite set δ Q Σ Q is finite reltion nd F Q. We extend δ to mpping δ : T Σ Q by δ(σ(t)) = {q (q σ q) δ i [ t ]: q i δ(t i )} for every σ Σ nd t TΣ. The finite tree utomton M recognizes the tree lnguge L(M) = {t T Σ δ(t) F }. A tree lnguge L T Σ is regulr if there exists finite tree utomton M such tht L = L(M). 86

3 VBD signed PV twl -OBJ DET-NN AltwqyE -BJ PV -OBJ -BJ Figure : mple LMBOT rules. 3 The model In this section we recll prticulr multi bottomup tree trnsducers which hve been introduced by Arnold nd Duchet (98) nd Lilin (98). A detiled (nd English) presenttion of the generl model cn be found in Engelfriet et l. (009) nd Mletti (00). Using the nomenclture of Engelfriet et l. (009) we recll vrint of liner nd nondeleting extended multi bottom-up tree trnsducers (MBOT) here. Occsionlly we will refer to generl MBOT which differ from the locl vrint discussed here becuse they hve explicit sttes. Throughout the rticle we ssume sets Σ nd of input nd output symbols respectively. Moreover let NT Σ be the set of designted nonterminl symbols. Finlly we void weights in the forml development to keep it simple. It is strightforwrd to dd weights to our model. Essentilly the model works on pirs t u consisting of n input tree t T Σ nd sequence u T of output trees. Ech such pir is clled pre-trnsltion nd the rnk rk( t u ) the pre-trnsltion t u is u. In other words the rnk of pre-trnsltion equls the number of output trees stored in it. Given pre-trnsltion t u T Σ T k nd i [k] we cll u i the i th trnsltion of t. An lignment for the pre-trnsltion t u is n injective mpping ψ : NT (u) NT (t) N such tht (p j) ψ( NT (u)) for every (p i) ψ( NT (u)) nd j [i]. In other words n lignment should request ech trnsltion of prticulr subtree t most once nd if it requests the i th trnsltion then it should lso request ll previous trnsltions. Definition A locl multi bottom-up tree trnsducer (LMBOT) is finite set R of rules such tht every rule written l ψ r contins pre-trnsltion l r nd n lignment ψ for it. The component l is the left-hnd side r is the right-hnd side nd ψ is the lignment of rule l ψ r R. The rules of n LMBOT re similr to the rules of n TG (synchronous tree substitution grmmr) of Eisner (003) nd hieber (004) but right-hnd sides of LMBOT contin sequence of trees insted of just single tree s in n TG. In ddition the lignments in n TG rule re bijective between lef nonterminls wheres our model permits multiple lignments to single lef nonterminl in the left-hnd side. A model tht is even more powerful thn LMBOT is the non-contiguous version of TG (synchronous tree-sequence substitution grmmr) of Zhng et l. (008) Zhng et l. (008b) nd un et l. (009) which llows sequences of trees on both sides of rules [see lso (Roult 997)]. Figure displys smple rules of n LMBOT using grphicl representtion of the trees nd the lignment. Next we define the semntics of n LMBOT R. To void difficulties we explicitly exclude rules like l ψ r where l NT or r NT ; i.e. rules where the left- or right-hnd side re only lef nonterminls. We first define the trditionl bottom-up semntics. Let ρ = l ψ r R be rule nd p NT (l). The p-rnk rk(ρ p) of ρ is rk(ρ p) = {i N (p i) ψ( NT (r))}. Definition The set τ(r) of pre-trnsltions of n LMBOT R is inductively defined to be the smllest set such tht: If ρ = l ψ r R is rule t p u p τ(r) is pre-trnsltion of R for every p NT (l) nd rk(ρ p) = rk( t p u p ) l(p) = t p (ε) nd Actully difficulties rise only in the weighted setting. 87

4 IN for N erbi PREP En NN-PROP rbya... VBD signed IN for PV N erbi PV -OBJ DET-NN PREP twl AltwqyE En NN-PROP rbya -OBJ... VBD twl signed IN DET-NN PREP for N AltwqyE En NN-PROP erbi rbya Figure : Top left: () Initil pre-trnsltion; Top right: (b) Pre-trnsltion obtined from the left rule of Fig. nd (); Bottom: (c) Pre-trnsltion obtined from the right rule of Fig. nd (b). r(p ) = u p (i) with ψ(p ) = (p i) for every p NT (r) then t u τ(r) where t = l[p t p p NT (l)] nd u = r[p (u p ) i p ψ (p i)]. In plin words ech nonterminl lef p in the left-hnd side of rule ρ cn be replced by the input tree t of pre-trnsltion t u whose root is lbeled by the sme nonterminl. In ddition the rnk rk(ρ p) of the replced nonterminl should mtch the rnk rk( t u ) of the pre-trnsltion nd the nonterminls in the right-hnd side tht re ligned to p should be replced by the trnsltion tht the lignment requests provided tht the nonterminl mtches with the root symbol of the requested trnsltion. The min benefit of the bottomup semntics is tht it works exclusively on pretrnsltions. The process is illustrted in Figure. Using the clssicl bottom-up semntics we simply obtin the following theorem by Mletti (00) becuse the MBOT constructed there is in fct n LMBOT. Theorem 3 For every TG n equivlent LMBOT cn be constructed in liner time which in turn yields prticulr MBOT in liner time. Finlly we wnt to relte LMBOT to the TG of un et l. (009). To this end we lso introduce the top-down semntics for LMBOT. As expected both semntics coincide. The top-down semntics is introduced using rule compositions which will ply n importnt rule lter on. Definition 4 The set R k of k-fold composed rules is inductively defined s follows: R = R nd l ϕ s R k+ for ll ρ = l ψ r R nd ρ p = l p ψp r p R k such tht rk(ρ p) = rk( l p r p ) l(p) = l p (ε) nd r(p ) = r p (i) with ψ(p ) = (p i) for every p NT (l) nd p NT (r) where l = l[p l p p NT (l)] s = r[p (r p ) i p ψ (p i)] nd ϕ(p p) = p ψ p (ip) for ll positions p ψ (p i) nd ip NT (r p ). The rule closure R of R is R = i Ri. The top-down pre-trnsltion of R is τ t (R) = { l r l ψ r R NT (l) = }. 88

5 b b b Figure 3: Composed rule. b The composition of the rules which is illustrted in Figure 3 in the second item of Definition 4 could lso be represented s ρ(ρ... ρ k ) where ρ... ρ k is n enumertion of the rules {ρ p p NT (l)} used in the item. The following theorem is esy to prove. Theorem 5 The bottom-up nd top-down semntics coincide; i.e. τ(r) = τ t (R). Ching (005) nd Grehl et l. (008) rgue tht TG hve sufficient expressive power for syntxbsed mchine trnsltion but Zhng et l. (008) show tht the dditionl expressive power of treesequences helps the trnsltion process. This is mostly due to the fct tht smller (nd less specific) rules cn be extrcted from bi-prsed word-ligned trining dt. A detiled overview tht focusses on TG is presented by Knight (007). Theorem 6 For every LMBOT n equivlent TG cn be constructed in liner time. 4 Rule extrction nd trining In this section we will show how to utomticlly obtin n LMBOT from bi-prsed word-ligned prllel corpus. Essentilly the process hs two steps: rule extrction nd trining. In the rule extrction step n (unweighted) LMBOT is extrcted from the corpus. The rule weights re then set in the trining procedure. The two min inspirtions for our rule extrction re the corresponding procedures for TG (Glley et l. 004; Grehl et l. 008) nd for TG (un et l. 009). TG re lwys contiguous in both the left- nd right-hnd side which mens tht they (completely) cover single spn of input or output words. On the contrry TG rules cn be noncontiguous on both sides but the extrction procedure of un et l. (009) only extrcts rules tht re contiguous on the left- or right-hnd side. We cn djust its st phse tht extrcts rules with (potentilly) non-contiguous right-hnd sides. The djustment is necessry becuse LMBOT rules cnnot hve (contiguous) tree sequences in their left-hnd sides. Overll the rule extrction process is sketched in Algorithm. Algorithm Rule extrction for LMBOT Require: word-ligned tree pir (t u) Return: LMBOT rules R such tht (t u) τ(r) while there exists mximl non-lef node p pos(t) nd miniml p... p k pos(u) such tht t p nd (u p... u pk ) hve consistent lignment (i.e. no lignments from within t p to lef outside (u p... u pk ) nd vice vers) do : dd rule ρ = t p ψ (u p... u pk ) to R with the nonterminl lignments ψ // excise rule ρ from (t u) 4: t t[p t(p)] u u[p i u(p i ) i {... k}] 6: estblish lignments ccording to position end while The requirement tht we cn only hve one input tree in LMBOT rules indeed might cuse the extrction of bigger nd less useful rules (when compred to the corresponding TG rules) s demonstrted in (un et l. 009). However the stricter rule shpe preserves the good lgorithmic properties of LMBOT. The more powerful TG rules cn cuse nonclosure under composition (Roult 997; Rdmcher 008) nd prsing to be less efficient. Figure 4 shows n exmple of biprsed ligned prllel text. According to the method of Glley et l. (004) we cn extrct the (miniml) TG rule displyed in Figure 5. Using the more liberl formt of LMBOT rules we cn decompose the TG rule of Figure 5 further into the rules displyed in Figure. The method of un et l. (009) would lso extrct the rule displyed in Figure 6. Let us reconsider Figures nd. Let ρ be the top left rule of Figure nd ρ nd ρ 3 be the 89

6 -BJ NML N VBD JJ N Voislv signed IN ugoslv President for N erbi AltwqyE DET-NN twl rbya En NN-PROP Alr}ys AlywgwslAfy fwyslaf PREP DET-NN DET-ADJ NN-PROP PV -OBJ -BJ Figure 4: Biprsed ligned prllel text. -BJ VBD signed PV twl -OBJ DET-NN AltwqyE Figure 5: Miniml TG rule. -BJ left nd right rule of Figure respectively. We cn represent the lower pre-trnsltion of Figure by ρ 3 ( ρ (ρ )) where ρ (ρ ) represents the upper right pre-trnsltion of Figure. If we nme ll rules of R then we cn represent ech pretrnsltion of τ(r) symboliclly by tree contining rule nmes. uch trees contining rule nmes re often clled derivtion trees. Overll we obtin the following result for which detils cn be found in (Arnold nd Duchet 98). Theorem 7 The set D(R) is regulr tree lnguge for every LMBOT R nd the set of derivtions is lso regulr for every MBOT. VBD signed IN for PV twl DET-NN PREP En AltwqyE Figure 6: mple TG rule. Moreover using the input nd output product constructions of Mletti (00) we obtin tht even the set D tu (R) of derivtions for specific input tree t nd output tree u is regulr. ince D tu (R) is regulr we cn compute the inside nd outside weight of ech (weighted) rule of R following the method of Grehl et l. (008). imilrly we cn djust the trining procedure of Grehl et l. (008) which yields tht we cn utomticlly obtin weighted LMBOT from bi-prsed prllel corpus. Detils on the run-time cn be found in (Grehl et l. 008). 5 Preservtion of regulrity Clerly LMBOT re not symmetric. Although the bckwrds ppliction of n LMBOT preserves regulrity this property does not hold for forwrd ppliction. We will focus on forwrd ppliction here. Given set T of pre-trnsltions nd tree lnguge 830

7 L T Σ we let T c (L) = {u i (u... u k ) T (L) i [k]} which collects ll trnsltions of input trees in L. We sy tht T preserves regulrity if T c (L) is regulr for every regulr tree lnguge L T Σ. Correspondingly n LMBOT R preserves regulrity if its set τ(r) of pre-trnsltions preserves regulrity. As mentioned n LMBOT does not necessrily preserve regulrity. The rules of n LMBOT hve only lignments between the left-hnd side (input tree) nd the right-hnd side (output tree) which re lso clled inter-tree lignments. However severl lignments to single nonterminl in the left-hnd side cn trnsitively relte two different nonterminls in the output side nd thus simulte n intrtree lignment. For exmple the right rule of Figure reltes PV nd n -OBJ node to single node in the left-hnd side. This could led to n intr-tree lignment (synchroniztion) between the PV nd -OBJ nodes in the right-hnd side. Figure 7 displys the rules R of n LMBOT tht does not preserve regulrity. This cn esily be seen on the lef (word) lnguges becuse the LMBOT cn trnslte the word x to ny element of L = {wcwc w { b} }. Clerly this word lnguge L is not context-free. ince the lef lnguge of every regulr tree lnguge is context-free nd regulr tree lnguges re closed under intersection (needed to single out the trnsltions tht hve the symbol t the root) this lso proves tht τ(r) c (T Σ ) is not regulr. ince T Σ is regulr this proves tht the LMBOT does not preserve regulrity. Preservtion of regulrity is n importnt property for number of trnsltion model mnipultions. For exmple the bucket-brigde nd the on-the-fly method for the efficient inference described in (My et l. 00) essentilly build on it. Moreover regulr tree grmmr (i.e. representtion of regulr tree lnguge) is n efficient representtion. More complex representtions such s context-free tree grmmrs [see e.g. (Fujiyoshi 004)] hve worse lgorithmic properties (e.g. more complex prsing nd problemtic intersection). In this section we investigte three syntctic restrictions on the set R of rules tht gurntees tht the obtined LMBOT preserves regulrity. Then we shortly discuss how to djust the rule extrction lgorithm so tht the extrcted rules utomticlly hve these property. First we quickly recll the notion of composed rules from Definition 4 becuse it will ply n essentil role in ll three properties. Figure 3 shows composition of two rules from Figure 7. Mind tht R might not contin ll rules of R but it contins ll those without lef nonterminls. Definition 8 An LMBOT R is finitely collpsing if there is n N such tht ψ : NT (r) NT (l) {} for every rule l ψ r R n. The following sttement follows from more generl result of Roult (997) which we will introduce with our second property. Theorem 9 Every finitely collpsing LMBOT preserves regulrity. Often the simple condition finitely collpsing is fulfilled fter rule extrction. In ddition it is utomticlly fulfilled in n LMBOT tht ws obtined from n TG using Theorem 3. It cn lso be ensured in the rule extrction process by introducing collpsing points for output symbols tht cn pper recursively in the corpus. For exmple we could enforce tht ll extrcted rules for cluse-level output symbols (ssuming tht there is no recursion not involving cluse-level output symbols) should hve only output tree in the right-hnd side. However finitely collpsing is rther strict property. Finitely collpsing LMBOT hve only slightly more expressive power thn TG. In fct they could be clled TG with input desynchroniztion. This is due to the fct tht the lignment in composed rules estblishes n injective reltion between lef nonterminls (s in n TG) but it need not be bijective. Consequently there cn be lef nonterminls in the left-hnd side tht hve no ligned lef nonterminl in the right-hnd side. In this sense those lef nonterminls re desynchronized. This feture is illustrted in Figure 8 nd such n LMBOT cn compute the trnsformtion {(t ) t T Σ } which cnnot be computed by n TG (ssuming tht T Σ is suitbly rich). Thus TG with input desynchroniztion re more expressive thn TG but they still compute clss of trnsformtions tht is not closed under composition. 83

8 x c c b b Figure 7: Output subtree synchroniztion (intr-tree). t 3 Figure 8: Finitely collpsing LMBOT. Theorem 0 For every TG we cn construct n equivlent finitely collpsing LMBOT in liner time. Moreover finitely collpsing LMBOT re strictly more expressive thn TG. Next we investigte weker property by Roult (997) tht still ensures preservtion of regulrity. Definition An LMBOT R hs finite synchroniztion if there is n N such tht for every rule l ψ r R n nd p NT (l) there exists i N with ψ ({p} N) {iw w N }. In plin terms multiple lignments to single lef nonterminl t p in the left-hnd side re llowed but ll lef nonterminls of the right-hnd side tht re ligned to p must be in the sme tree. Clerly n LMBOT with finite synchroniztion is finitely collpsing. Roult (997) investigted this restriction in the context of rtionl tree reltions which re generliztion of our LMBOT. Roult (997) shows tht finite synchroniztion cn be decided. The next theorem follows from the results of Roult (997). Theorem Every LMBOT with finite synchroniztion preserves regulrity. MBOT cn compute rbitrry compositions of TG (Mletti 00). However this no longer remins true for MBOT (or LMBOT) with finite synchroniztion. In Figure 9 we illustrte trnsltion tht cn be computed by composition of two TG but tht cnnot be computed by n MBOT (or LMBOT) with finite synchroniztion. Intuitively when processing the chin of s of the trnsformtion depicted in Figure 9 the first nd second suc- This ssumes strightforwrd generliztion of the finite synchroniztion property for MBOT.. t t Z t t t 3 Figure 9: Trnsformtion tht cnnot be computed by n MBOT with finite synchroniztion. cessor of the Z -node t the root on the output side must be ligned to the -chin. This is necessry becuse those two mentioned subtrees must reproduce t nd t from the end of the -chin. We omit the forml proof here but obtin the following sttement. Theorem 3 For every TG we cn construct n equivlent LMBOT with finite synchroniztion in liner time. LMBOT nd MBOT with finite synchroniztion re strictly more expressive thn TG nd compute clsses tht re not closed under composition. Agin it is strightforwrd to djust the rule extrction lgorithm by the introduction of synchroniztion points (for exmple for cluse level output symbols). We cn simply require tht rules extrcted for those selected output symbols fulfill the condition mentioned in Definition. Finlly we introduce n even weker version. Definition 4 An LMBOT R is copy-free if there is n N such tht for every rule l ψ r R n nd p NT (l) we hve (i) ψ ({p} N) N or (ii) ψ ({p} N) {iw w N } for n i N. Intuitively copy-free LMBOT hs rules whose right hnd sides my use ll lef nonterminls tht re ligned to given lef nonterminl in the lefthnd side directly t the root (of one of the trees 83

9 Z Figure 0: Composed rule tht is not copy-free. in the right-hnd side forest) or group ll those lef nonterminls in single tree in the forest. Clerly the LMBOT of Figure 7 is not copy-free becuse the second rule composes with itself (see Figure 0) to rule tht does not fulfill the copy-free condition. Theorem 5 Every copy-free LMBOT preserves regulrity. Proof sketch: Let n be the integer of Definition 4. We replce the LMBOT with rules R by the equivlent LMBOT M with rules R n. Then ll rules hve the form required in Definition 4. Moreover let L T Σ be regulr tree lnguge. Then we cn construct the input product of τ(m) with L. In this wy we obtin n MBOT M whose rules still fulfill the requirements (dpted for MBOT) of Definition 4 becuse the input product does not chnge the structure of the rules (it only modifies the stte behvior). Consequently we only need to show tht the rnge of the MBOT M is regulr. This cn be chieved using decomposition into relbeling which clerly preserves regulrity nd deterministic finite-copying top-down tree trnsducer (Engelfriet et l. 980; Engelfriet 98). Figure shows some relevnt rules of copyfree LMBOT tht computes the trnsformtion of Figure 9. Clerly copy-free LMBOT re more generl thn LMBOT with finite synchroniztion so we gin cn obtin copy-free LMBOT from TG. In ddition we cn djust the rule extrction process using synchroniztion points s for LMBOT with finite synchroniztion using the restrictions of Definition 4. Theorem 6 For every TG we cn construct n equivlent copy-free LMBOT in liner time. Figure : Copy-free LMBOT for the trnsformtion of Figure 9. Copy-free LMBOT re strictly more expressive thn LMBOT with finite synchroniztion. 6 Conclusion We hve introduced simple restriction of multi bottom-up tree trnsducers. It bstrcts from the generl stte behvior of the generl model nd only uses the loclity tests tht re lso present in TG TG nd TAG. Next we introduced rule extrction procedure nd corresponding rule weight trining procedure for our LMBOT. However LMBOT llow trnsltions tht do not preserve regulrity which is n importnt property for efficient lgorithms. We presented 3 properties tht ensure tht regulrity is preserved. In ddition we shortly discussed how these properties could be enforced in the presented rule extrction procedure. Acknowledgements The uthor grtefully cknowledges the support by KEVIN KNIGHT who provided the inspirtion nd the dt. JONATHAN MA helped in mny fruitful discussions. The uthor ws finncilly supported by the Germn Reserch Foundtion (DFG) grnt MA / 4959 /

10 References Alfred V. Aho nd Jeffrey D. Ullmn. 97. The Theory of Prsing Trnsltion nd Compiling. Prentice Hll. André Arnold nd Mx Duchet. 98. Morphismes et bimorphismes d rbres. Theoret. Comput. ci. 0(): Peter F. Brown John Cocke tephen A. Dell Pietr Vincent J. Dell Pietr Fredrick Jelinek John D. Lfferty Robert L. Mercer nd Pul. Roossin A sttisticl pproch to mchine trnsltion. Computtionl Linguistics 6(): Peter F. Brown tephen A. Dell Pietr Vincent J. Dell Pietr nd Robert L. Mercer Mthemtics of sttisticl mchine trnsltion: Prmeter estimtion. Computtionl Linguistics 9():63 3. Dvid Ching A hierrchicl phrse-bsed model for sttisticl mchine trnsltion. In Proc. ACL pges Assocition for Computtionl Linguistics. Dvid Ching An introduction to synchronous grmmrs. In Proc. ACL. Assocition for Computtionl Linguistics. Prt of tutoril given with Kevin Knight. Json Eisner impler nd more generl minimiztion for weighted finite-stte utomt. In Proc. NAACL pges Assocition for Computtionl Linguistics. Joost Engelfriet Grzegorz Rozenberg nd Gior lutzki Tree trnsducers L systems nd two-wy mchines. J. Comput. ystem ci. 0():50 0. Joost Engelfriet Eric Lilin nd Andres Mletti Composition nd decomposition of extended multi bottom-up tree trnsducers. Act Inform. 46(8): Joost Engelfriet. 98. The copying power of one-stte tree trnsducers. J. Comput. ystem ci. 5(3): Akio Fujiyoshi Restrictions on mondic contextfree tree grmmrs. In Proc. CoLing pges Assocition for Computtionl Linguistics. Michel Glley Mrk Hopkins Kevin Knight nd Dniel Mrcu Wht s in trnsltion rule? In Proc. HLT-NAACL pges Assocition for Computtionl Linguistics. Jonthn Grehl Kevin Knight nd Jonthn My Trining tree trnsducers. Computtionl Linguistics 34(3): Kevin Knight Cpturing prcticl nturl lnguge trnsformtions. Mchine Trnsltion (): 33. Eric Lilin. 98. Propriétés de clôture d une extension de trnsducteurs d rbres déterministes. In Proc. CAAP volume of LNC pges pringer. Andres Mletti. 00. Why synchronous tree substitution grmmrs? In Proc. NAACL pges Assocition for Computtionl Linguistics. Jonthn My Kevin Knight nd Heiko Vogler. 00. Efficient inference through cscdes of weighted tree trnsducers. In Proc. ACL pges Assocition for Computtionl Linguistics. Frnk G. Rdmcher An utomt theoretic pproch to rtionl tree reltions. In Proc. OFEM volume 490 of LNC pges pringer. Jen-Clude Roult Rtionl tree reltions. Bull. Belg. Mth. oc. imon tevin 4(): turt M. hieber nd ves chbes ynchronous tree-djoining grmmrs. In Proc. CoLing volume 3 pges Assocition for Computtionl Linguistics. turt M. hieber ynchronous grmmrs s tree trnsducers. In Proc. TAG+7 pges Vncouver BC Cnd. imon Frser University. turt M. hieber Probbilistic synchronous treedjoining grmmrs for mchine trnsltion: The rgument from bilingul dictionries. In Proc. T pges Assocition for Computtionl Linguistics. Jun un Min Zhng nd Chew Lim Tn A noncontiguous tree sequence lignment-bsed model for sttisticl mchine trnsltion. In Proc. ACL pges Assocition for Computtionl Linguistics. Min Zhng Hongfei Jing Aiti Aw Hizhou Li Chew Lim Tn nd heng Li A tree sequence lignment-bsed tree-to-tree trnsltion model. In Proc. ACL pges Assocition for Computtionl Linguistics. Min Zhng Hongfei Jing Hizhou Li Aiti Aw nd heng Li. 008b. Grmmr comprison study for trnsltionl equivlence modeling nd sttisticl mchine trnsltion. In Proc. CoLing pges Assocition for Computtionl Linguistics. 834