Tree transformations and dependencies

Transcription

1 Tree transformations and deendencies Andreas Maletti Universität Stuttgart Institute for Natural Language Processing Azenbergstraße 12, Stuttgart, Germany Abstract Several tree transformation devices that are relevant in natural language rocessing are resented with a focus on the deendencies that they are able to cature In many cases, the consideration of the deendencies alone can be used to rovide a high-level exlanation of the short-comings of tree transformation devices and allows surrising insights into their structure 1 Motivation In the subfield of machine translation [31], which is concerned with the automatic translation of natural language texts, it was recently realized that string-based systems [50] cannot easily comute certain imortant translations [48, 1, 49, 9] and that the structural information rovided by modern and reliable arsers [11, 28, 27, 6] actually hels the translation rocess [53] This develoment created renewed interest in tree automata [5, 8] and tree transducers [32, 14], which are finite-state devices that comute tree languages and tree transformations, resectively However, there does not exist a tree translation model that is universally acceted and used in the machine translation task On the contrary, many different models with different exressive ower are used The first formal tree transducer model was the to-down tree transducer investigated by Thatcher [46] and Rounds [41] Several other models such as bottom-u tree transducers [47], attributed tree transducers [20, 30] and ebble tree transducers [38], macro tree transducers [12, 18] and modular tree transducers [19], monadic second-order logic tree transducers [16, 7], and tree bimorhisms [3] and various models with synchronization [40] were introduced later and have been investigated in the theory of formal languages In general, tree transducers rocess an inut tree and nondeterministically generate an outut tree In the rocess, they can move comlete subtrees or decide to rocess subtrees differently based on an internal state Shieber [42] and others have argued that to-down tree transducers are generally inadeuate for linguistic tasks In this survey we will focus on three models that received attention from the machine translation community, which are: Suorted by the German Research Foundation (DFG) grant MA/4959/1-1

2 extended to-down tree transducers [13, 2, 29, 26], extended multi bottom-u tree transducers [33, 3, 21, 22, 15, 35], and synchronous tree-seuence substitution grammars [40, 51, 52, 45] We review these three models and investigate their exressive ower from a very abstract viewoint by looking only at the tye of deendencies that they create It turns out that the models, which indeed have successively more exressive ower, can already be distinguished easily based on the tyes of deendencies that they can comute Informally, a deendency records that a certain art of the outut tree was created in accordance with a articular art of the inut tree (or vice versa) This influence is informally called deendence, synchronization, or contribution [17] Formally, we establish deendencies using the derivation mechanism, which is tyically term rewriting [4] However, for our uroses synchronous substitution is much more suitable since the synchronization links are an exlicit reresentation of our deendencies Thus, we adjust the derivation rocess to use synchronous substitution keeing all synchronization links during the whole derivation We investigate the tye of deendencies each of our three mentioned tree transformation models can comute It shows that all of them enjoy a certain hierarchy roerty that can be used to uickly show that transformations that have crossing deendencies cannot be comuted by any of our models This allows us, for examle, to set all three models aart from synchronous tree-adjoining grammars [44, 42, 43], which can comute crossing deendencies Moreover, a stricter version of the hierarchy roerty also allows us to distinguish our three models, which we demonstrate with an examle transformation in each case These examle transformations are taken from the literature, but instead of resenting the full roof from the literature, we simly add natural deendencies to the transformation and then show that no deendency comutable by a certain model is comatible with these deendencies This aroach highlights the essential and illustrative art of the formal roof and avoids the technical art of the roof, which is still needed to justify the initial deendencies The interested reader can find these technical arts in the cited literature or can rove them via a case analysis Thus, the aroach ursued here does not offer full roofs, but it will be obvious from the examles that those deendencies should be resent The survey is structured as follows: Section 2 recalls basic notions and notation In each of the next three sections (Sections 3 5) we recall one of our three tree transformation models in order of increasing exressive ower (ie, the order in which they are mentioned above) Section 3 also contains the definitions of the hierarchy roerties of deendencies that we will investigate We conclude with a short summary, which is resented as a table showing the identified roerties of deendencies (including those for synchronous tree-adjoining grammars) 2 Notation The set of all nonnegative integers is IN A relation ρ from a set S to a set T is a subset ρ S T The set of all finite words over S is S, where ε is the

3 emty word The concatenation of the words v, w S is vw or simly vw The length of a word w S is denoted by w An alhabet Σ is a nonemty and finite set, of which the elements are called symbols A ranked alhabet is a air (Σ, rk) consisting of an alhabet Σ and a rank maing rk: Σ IN For every k IN, let Σ k = rk 1 (k) We tyically write (k) to indicate that rk() = k A doubly ranked alhabet simly has a rank maing rk: Σ IN 2 We use the same notations for ranked and doubly ranked alhabets Moreover, we tyically assume that the maing rk is clear from the context The set T Σ (S) of Σ-trees with leaf labels S is the smallest set T such that S T and (t 1,, t k ) T for every Σ k and t 1,, t k T We generally assume that Σ S =, and thus we write () simly as for every Σ 0 Moreover, we write k (t) for ( (t) ) containing k occurrences of in the abbreviated list We write T Σ for T Σ ( ) The set os(t) IN of ositions of t T Σ (S) is inductively defined by os(s) = {ε} for every s S and os((t 1,, t k )) = {ε} k {iw w os(t i )} for every Σ k and t 1,, t k T Σ (S) The ositions os(t) are totally ordered by the lexicograhic order on IN and the refix order on IN Let t, t T Σ (S) and w os(t) The label of t at w is t(w), and the w-rooted subtree of t is t w Formally, s(ε) = s ε = s for every s S and t(w) = i=1 { if w = ε t i (v) if w = iv and i IN and t w = { t t i v if w = ε if w = iv and i IN where t = (t 1,, t k ) for every Σ k and t 1,, t k T Σ (S) For every L S, we let os L (t) = {w os(t) t(w) L} and os s (t) = os {s} (t) for every s S The tree t is linear in L if os l (t) 1 for every l L Moreover, var(t) = {s S os s (t) } The exression t[u] w denotes the tree that is obtained from t T Σ (S) by relacing the subtree t w at w by u T Σ (S) We extend this notation to seuences u = u 1,, u n of trees and ositions w = w 1,, w n of t that are airwise incomarable with resect to the refix order Thus, t[u] w denotes the tree obtained from t by relacing the subtree t wi at w i by u i for all 1 i n 3 Extended to-down tree transducer Our first model is the (linear and nondeleting) extended to-down tree transducer [13, 2, 29, 26] (xto), which is based on the classical to-down tree transducer [41, 46] A to-down tree transducer is a secial xto, in which all lefthand sides of rules contain exactly one inut symbol In general, the left-hand side of an xto can contain any number of inut symbols [34, 37] We resent a syntactic version here that is closer to synchronized grammars [10], but eually exressive as the classical version [41, 46, 29, 26] with term

4 rewrite rules In general, eual states in the left and right-hand side of a rule are linked In a derivation, they will be relaced at the same time In a rule of an xto, these links are bijective Definition 1 (see [37, Sect 22]) A (linear and nondeleting) extended tree transducer ( xto) is a tule (Q, Σ,, I, R), where Q is a finite set of states, Σ and are ranked alhabets of inut and outut symbols, I Q is a set of initial states, and R T Σ (Q) Q T (Q) is a finite set of rules such that l and r are linear in Q and var(l) = var(r) for every (l,, r) R In the following, let M = (Q, Σ,, I, R) be an xto As already mentioned, M is a to-down tree transducer if for every (l,, r) R there exist Σ k and 1,, k Q such that l = ( 1,, k ) To simlify the notation, we often write rules as l r instead of (l,, r) Examle 2 (see [3, Sect 34]) Let M bin = (Q, Σ, Σ, {}, R) and M debin = (Q, Σ, Σ, {}, R ) be the xto with Q = {,,, r}, Σ = { (3), (2), (1), (0) }, R, which contains the following rules for all x {,, r}: (, ) (, ) (,, ) (, (, )) (,, r) (, (, r)) (x) x (x) x, and R, which contains the following rules for all x {,, r}: (, ) (, ) (, (, )) (,, ) (, ()) (, ()) (x) x (x) x Clearly, M bin is even a to-down tree transducer, whereas M debin is not a todown tree transducer The rules of M bin are illustrated in Fig 1 Next, we move to the semantics of an xto M, which is given by synchronous substitution While the links in an xto rule are imlicit and established due to occurrences of eual states, we need an exlicit linking structure for our sentential forms In addition, these links will form the deendencies that we are interested in To this end, we store a relation between ositions of the inut and outut tree, which encodes the links Let L = P(IN IN ) = {S S IN IN } be the set of all link structures First, we define general sentential forms Roughly seaking, we have an inut tree and an outut tree, in which ositions are linked

5 r r x x x x Fig 1 Examle rules of the xto M bin of Ex 2 Definition 3 (see [23, Sect 3]) An element ξ, D, ζ T Σ (Q) L T (Q) is a sentential form if v os(ξ) and w os(ζ) for every (v, w) D Now we lift the imlicit link structure in an xto rule into an exlicit link relation This link relation will then be used in the derivation rocess once the rule is alied to determine the next links in the obtained sentential form Definition 4 Let l r R be a rule, and let v, w IN The rule s link structure links v,w (l r) L is links v,w (l r) = {(vv, ww ) v os (l), w os (r)} Q Note that links v,w (l r) is a bijective relation on the state occurrences The derivation rocess is started with a simle sentential form, {(ε, ε)}, consisting of the inut tree and the outut tree for some initial state I and the trivial link relating both states This is clearly a link structure that is bijective between state occurrences Next, we (nondeterministically) aly a rule l r to a air of linked occurrences of the state Such an alication relaces the linked occurrences of by the left and right-hand side of the rule The imlicit links in the rule are added to the (exlicit) link structure to obtain a new sentential form This yields another link structure that is bijective between state occurrences Since we are interested in the deendencies created during derivation, we reserve all links and never remove a link from the linking structure Note that this reservation causes that the link structure need not be functional on all ositions because we kee the links that were used in the relacement rocess This relacement rocess is reeated until no linked occurrences of states remain Definition 5 (see [23, Sect 3]) Given two sentential forms ξ, D, ζ and ξ, D, ζ such that D and D are bijective on state occurrences, we write ξ, D, ζ ξ, D, ζ

6 if there exists a rule l r R and an inut osition v os (ξ) such that ξ = ξ[l] v and ζ = ζ[r] w, where w os (ζ) is the uniue -labelled osition such that (v, w) D, and D = D links v,w (l r) As usual M is the reflexive and transitive closure of The xto M comutes the deendencies de(m) T Σ L T, which are given by de(m) = { t, D, u T Σ L T I :, {(ε, ε)}, M t, D, u } Moreover, the xto M comutes the tree transformation M T Σ T, which is given by M = {(t, u) (t, D, u) de(m)} r r r r 3 M Fig 2 Examle derivation where M = M bin (see Ex 6)

7 Examle 6 Let M = M bin be the xto of Ex 2, and let (, ((),, (,, ))) be the inut tree Selecting the only initial state, we can obtain the derivation that is dislayed in Fig 2 Overall (, ((), (, (, (, ))))) is a translation of the inut tree The translations of M bin and M debin of Ex 2 are illustrated in Fig 3 t 1 t 2 t 3 t n 4 t n 3 t n 2 t n 1 t n M bin t 1 t 2 t 3 t 4 t n 3 t n 2 M debin t 2 t 1 t 4 t 3 t n 2 t n 3 t n 1 t n t n 1 t n Fig 3 Translations of [3] that are individually comuted by the xto M bin and M debin Since every translation (t, u) M is ultimately created by (at least) one successful derivation, we can insect the links in the derivation rocess to exhibit the deendencies Roughly seaking, the links establish which arts of the outut tree were generated due to a articular art of the inut tree This corresondence is called contribution in [17] Examle 7 Recall the xto M bin of Ex 2 and the derivation in Ex 6, which is dislayed in Fig 2 Looking at the last sentential form, which is dislayed in Fig 4 for easier reference, its linking structure is D = {(ε, ε), (1, 1), (2, 2), (21, 21), (211, 211), (22, 221), (23, 222), (231, 2221), (232, 22221), (233, 22222)}, which reresents the deendencies introduced by the rule alications Next, let us observe some imortant roerties of the comuted deendencies To this end, we disregard the actual inut and outut trees and say that a linking structure D L is comuted by M if there exist an initial state I and trees t T Σ and u T such that, {(ε, ε)}, M t, D, u The set of all linking structures comuted by M is links(m) Definition 8 A linking structure D L is inut hierarchical if for every (v 1, w 1 ), (v 2, w 2 ) D with v 1 < v 2 we have w 2 w 1 and there exists (v 1, w 1) D such that w 1 w 2 It is strictly inut hierarchical if additionally

8 Fig 4 Deendencies comuted during the derivation of Fig 2 w w or w w for all (v, w), (v, w ) D and v 1 v 2 for all (v 1, w 1 ), (v 2, w 2 ) D with w 1 w 2 Roughly seaking, inut hierarchical linking structures have no crossing links (or deendencies) More formally, let (v, w), (v, w ) D be such that v < v and w < w Then (v, w) and (v, w ) are crossing links (or deendencies) Clearly, such links cannot exist in an inut hierarchical linking structure The same notions can be defined for the outut side by reuiring the corresonding roerties for the linking structure D 1 For examle, D is strictly outut hierarchical if D 1 is strictly inut hierarchical Moreover, it is strictly hierarchical if it is both strictly inut hierarchical and strictly outut hierarchical Finally, a set D L of linking structures has a certain hierarchical roerty if each element has it Examle 9 The linking structure D of Ex 7 is strictly hierarchical In addition, we also need a roerty that guarantees that there are enough links Roughly seaking, there should be an integer that limits the distance between links Definition 10 A set D L of link structures has bounded distance if there exists an integer k IN such that for every D D we have that for all (v, w), (vv, w ) D with v > k there exist v 1, v 2 v and w 1, w 2 such that v 1 k v v 2 and (vv 1, w 1 ), (vv 2, w 2 ) D, and for all (v, w), (v, ww ) D with w > k there exist v 1, v 2 and w 1, w 2 w such that w 1 k w w 2 and (v 1, ww 1 ), (v 2, ww 2 ) D In other words, between any two source- or target-nested links of large distance, there should exist links whose distance to the original links is small This yields that the distance to the next nested link (if such a link does exist) can be at most k Note however, that the above roerty does not reuire a link every k symbols This roerty would also be true for all xto, but it would no longer be true for all mbot, which are discussed in the next section To kee the resentation simle, we only discuss bounded distance as introduced

9 Examle 11 The set links(m bin ), where M bin is the xto of Ex 2, has bounded distance For the inut side, the distance is bounded by 1, and for the outut side, it is bounded by 2 Lemma 12 The set links(m) comuted by an xto M is strictly hierarchical with bounded distance Proof This lemma follows trivially from Definition 5 t 1 t 2 t 1 t 2 t 3 t n 4 t n 3 t 4 t 3 t n 2 t n 3 t n 2 t n 1 t n t n 1 t n Fig 5 Examle translation of [3] with deendencies, where the inverse arrow heads indicate that the deendencies oint to any node (not necessarily the root) inside the subtrees In addition, for a (linear and nondeleting) to-down tree transducer every inut osition has exactly one link (ie, the linking structures encountered with to-down tree transducers are functional) Next, we define the notion of comatibility of linking structures (or deendencies) This notion will allow us to rescribe a semantic deendency and then analyze whether a certain class of tree transformation devices can handle such linking structures Naturally, the imlementation in a tree transformation device can add more deendencies, which are created by the articular choice of rules Conseuently, comatibility only reuires that the given deendencies are a subset of the realized links in the linking structure Moreover, given a set of deendencies for a given inut and outut tree, it is sufficient to be comatible to at least one deendency because already one comatible deendency would render the translation lausible Definition 13 Let ξ, D, ζ and ξ, D, ζ be sentential forms with the same inut and outut trees Then ξ, D, ζ is comatible with ξ, D, ζ if D D Given sets L and L of sentential forms, L is comatible with L if for every ξ, D, ζ L there exist ξ, D, ζ L and ξ, D, ζ L such that ξ, D, ζ is comatible with ξ, D, ζ

10 Figure 5 shows the comosition of the tree transformations that are comuted by the xto M bin and M debin of Ex 2 This examle was used in [3] to show that the class of transformations comuted by xto is not closed under comosition In fact, assuming the deendencies indicated in Fig 5, we can rove this statement by observing that this set of deendencies is not comatible to a strictly hierarchical deendence with bounded distance Conseuently, these deendencies cannot be comuted by an xto Lemma 14 The deendencies deicted in Fig 5 are not comatible with the deendencies comuted by any xto Proof Suose that there is an xto that comutes deendencies that are comatible with the deendencies deicted in Fig 5 Then there exits a bound n such that all inut and outut tree airs whose -sine is longer than n must have a link on this -sine However, such a link together with the existing deendencies makes it incomatible to any strictly hierarchical deendency The revious lemma also yields that the tree transformation of Fig 5 cannot be comuted by an xto Actually, the difficult, but not very illustrative art of the full roof establishes that the deendencies deicted in Fig 5 are really necessary This art remains and is roved in [3] Theorem 15 (see [3, Sect 34]) The tree transformation illustrated in Fig 5 cannot be comuted by any xto 4 Extended multi bottom-u tree transducer In this section, we recall the (linear and nondeleting) extended multi bottom-u tree transducer (mbot), which was introduced in [33, 3] in the shae of a articular bimorhism The name multi bottom-u tree transducer seems to originate from [21, 22], where the deterministic variant of the model was rediscovered A more detailed resentation of various multi bottom-u tree transducers can be found in [15], and [35] reorts some results for the weighted model Definition 16 (see [35, Def 2]) A (linear and nondeleting) extended multi bottom-u tree transducer ( mbot) is a system (Q, Σ,, I, R) where Q, Σ, and are ranked alhabets of states, inut symbols, and outut symbols, resectively, I Q 1 is a subset of initial states, all of which are unary, and R T Σ (Q) Q T (Q) is a finite set of rules such that l is linear in Q, rk() = n, and n i=1 var(r i) var(l) for every (l,, r 1 r n ) R For all the remaining discussions, let M = (Q, Σ,, I, R) be an mbot Clearly, any xto is an mbot In addition, two items deserve exlicit mention First, the set Q of states is a ranked alhabet in contrast to xto or traditional to-down or bottom-u tree transducers [46, 41, 47] Roughly seaking, the rank rk() of a state Q coincides with the number r of trees in

11 f x x x x r r Fig 6 mbot rules of the mbot M com of Ex 17 the right-hand side of all rules (l,, r) R For examle, a nullary state has no outut trees at all and can be understood as a ure look-ahead [15] in the inut tree Second, all initial states are unary (ie, have exactly one outut tree) In this way, we obtain exactly one outut tree and ultimately a relation between inut and outut trees To simlify the discussion, we call l and r of a rule (l,, r) R the left- and right-hand side, resectively In accordance, we sometimes write l r instead of (l,, r) Examle 17 (see [3, Sect 34]) Let M com = (Q, Σ, Σ, {f}, R) be the mbot with Q = { (2), (1), (1), r (1), f (1) }, Σ = { (3), (2), (1), (0) }, and R, which contains the following rules for every x {,, r}: (, ) f (,, ) (,, ) (,, ) (,, r) (, r) (x) x (x) x, where we searate trees in a seuence by full stos The rules of M com are illustrated in Fig 6 Since our rules now have a more general structure, we again need to lift the imlicit link structure in an mbot rule into an exlicit link relation This time we rovide an inut osition and additionally as many outut ositions as reuired The reuired number is the rank of the state in a rule l r Definition 18 Let l r R be a rule Moreover, let v, w 1,, w n IN and w = w 1 w n where n = rk() The rule s link structure links v,w (l r) L is links v,w (l r) = Q i=1 n {(vv, w i w i) v os (l), w i os (r i )} Note that the inverse relation links v,w (l r) 1 is functional on the state occurrences However, in general, it is not bijective, and in articular, a state occurrence in the inut might be without any link

12 The semantics is again resented using synchronous substitution However, this time several states in the outut side of a sentential form can be linked to the state that is relaced in the inut side of the sentential side As before, the derivation rocess is started with a simle sentential form, {(ε, ε)}, Next, we (nondeterministically) aly a rule l r to an occurrence of a state in the inut side and all its linked occurrences on the outut side Those occurrences are relaced by the left and right-hand side of the rule, where the otentially several trees in the right-hand side relace the linked occurrences in lexicograhic order The final ste adds the imlicit links in the rule to the (exlicit) link structure to obtain a new sentential form Note that the functionality of the inverse imlicit linking structure of a rule is lost in the sentential forms due to the reservation of old links Definition 19 (see [36, Sect 3]) Given two sentential forms ξ, D, ζ and ξ, D, ζ, we write ξ, D, ζ ξ, D, ζ if there exists a rule l r R and an inut osition v os (ξ) such that rk() = n, ξ = ξ[l] v and ζ = ζ[r] w, where w = w 1 w n with (i) w 1,, w n os (ζ), (ii) w 1 w n, and (iii) {w 1,, w n } = {w (v, w) D}, and D = D links v,w (l r) As usual M is the reflexive and transitive closure of The mbot M comutes the deendencies de(m) T Σ L T, which are given by de(m) = { t, D, u T Σ L T I :, {(ε, ε)}, M t, D, u } Moreover, the mbot M comutes the relation M T Σ T, which is given by M = {(t, u) (t, D, u) de(m)}, and links(m) = {D (t, D, u) de(m)} Examle 20 It can easily be verified that M com of Ex 17 comutes the tree transformation deicted in Fig 5 An examle derivation using M com is shown in Fig 7 Conseuently, the tree transformation used in the revious section (see Fig 5) can be comuted by an mbot Figure 9 roughly sketches the deendencies created during the comutation of this transformation with the mbot M com of Ex 17 Next, let us look at the roerties of the deendencies reresented in Figs 8 and 9 Examle 21 Figure 8 reresents the sentential form t, D, u, where t = (, ((),, (,, ))) u = ((),, (,, (, ))) D = {(ε, ε), (1, 2), (2, 1), (2, 3), (21, 1), (211, 11), (22, 32), (23, 31), (23, 33), (231, 31), (232, 331), (233, 332)} The linking structure D is inut hierarchical and strictly outut hierarchical The same roerties also hold for the deendencies indicated in Fig 9

13 3 M r r Fig 7 Examle derivation where M = M com (see Ex 17) Fig 8 Examle deendency comuted by M com (see Ex 17)

14 t 1 t 2 t 1 t 2 t 3 t n 4 t n 3 t 4 t 3 t n 2 t n 3 t n 2 t n 1 t n t n 1 t n Fig 9 Examle translation of [3] with deendencies suitable for an mbot The roerties of the deendencies exhibited in Ex 21 are indicative for all deendencies comuted by mbot This is observed in the next lemma, which follows straightforwardly from Def 19 As usual, the finite size of the rules yields bounded distance Note that there can be unboundedly large arts of the inut tree without any link For examle, inut subtrees created by nullary states can have this roerty because the mbot does only check a regular roerty [24, 25] and does not roduce any corresonding outut Lemma 22 The set links(m) comuted by an mbot M is inut hierarchical and strictly outut hierarchical with bounded distance Next, we again use our knowledge about the tye of deendencies that are comutable by an mbot to illustrate a tree transformation that cannot be comuted by any mbot Examle 23 (see [39, Ex 45] and [40]) The mbot M sort = (Q, Σ,, {f}, R) and M sort2 = (Q, Σ,, {f}, R ) are given by Q = { (3), (3), r (3), f (1) }, Σ = { (0), (1), β (1), γ (1) } and = Σ { (3) }, the following rules in R: () () β() β() r r r r f (,, ) γ(r) r r r γ(r) r, and the following rules in R : () () β() β() r f (r, r, r) γ(r) r r r γ(r) r

15 Figure 10 dislays the rules R of the examle mbot M sort Since the rules R are very similar, we omitted a grahical reresentation The mbot M sort and M sort2 comute the tree transformations M sort = {( l (β m (γ n ())), ( l (), β m (), γ n ())) l, m, n IN} M sort2 = {(γ n (β m ( l ())), ( l (), β m (), γ n ())) l, m, n IN}, resectively In other words, M sort sorts all -symbols into the first outut subtree (below ), the β-symbols into the second subtree, and the γ-symbols into the third subtree f β β γ r γ r r r r r r r r r Fig 10 Examle rules of the mbot M sort of Ex 23 From the definition of the tree transformations of Ex 23 we can evidently conclude some deendencies, which we deict in Fig 11 Clearly, the shown deendencies are not strictly inut hierarchical However, they are inut hierarchical and strictly outut hierarchical Conseuently, the inverse deendencies are strictly inut hierarchical and outut hierarchical, but not strictly outut hierarchical In the same manner as for xto, we can conclude the following statement Lemma 24 The inverse deendencies deicted in Fig 11 are not comatible with the deendencies comuted by any mbot Theorem 25 (see [39, Ex 45]) The inverse of the tree transformation illustrated in Fig 11 cannot be comuted by any mbot 5 Synchronous tree-seuence substitution grammar In this final section before the summary, we recall the synchronous tree-seuence substitution grammar (stssg), which was introduced in [40, 51, 52, 45] We kee the resentation terse because most mechanisms have been exlained on the revious models

16 β β γ β γ β γ γ Fig 11 Some deendencies of the tree transformation of Ex 23 Definition 26 (see [45, Sect 2]) A synchronous tree-seuence substitution grammar ( stssg) is a system (Q, Σ,, I, R) where Q is a doubly ranked alhabet, Σ and are ranked alhabets of inut and outut symbols, resectively, I Q 1,1 is a subset of initial states, all of which are doubly unary, and R T Σ (Q) Q T (Q) is a finite set of rules such that rk() = (m, n) for every (l 1 l m,, r 1 r n ) R For the rest of this section, let M = (Q, Σ,, I, R) be an stssg Clearly, any mbot is an stssg, and moreover, any inverse transformation comuted by an mbot can be imlemented by an stssg The ranks rk() of a state Q coincide with numbers l and r of trees in the left- and right-hand side of all rules (l,, r) R As before all initial states are doubly unary (ie, have exactly one inut and exactly one outut tree) In this way, we again obtain a relation between inut and outut trees As before, we call l and r of a rule (l,, r) R the left- and right-hand side, resectively In accordance, we sometimes write l r instead of (l,, r) Definition 27 Let l r R be a rule, and let v 1,, v m, w 1,, w n IN, v = v 1 v m, and w = w 1 w n where rk() = (m, n) The rule s link structure links v,w (l r) L is links v,w (l r) = m n {(v j v j, w i w i) v j os (l j ), w i os (r i )} Q j=1 i=1 As before, the semantics is resented using synchronous substitution This time several states can be relaced in both the inut and the outut side of a sentential form

17 Definition 28 (see [45, Sect 2]) Given two sentential forms ξ, D, ζ and ξ, D, ζ, we write ξ, D, ζ ξ, D, ζ if there exists a rule l r R, inut ositions v 1,, v m os (ξ), and outut ositions w 1,, w n os (ζ) such that rk() = (m, n), ξ = ξ[l] v and ζ = ζ[r] w, where v = v 1 v m with v 1 v m and w = w 1 w n with w 1 w n, the ositions are linked; ie, m {w 1,, w n } = {w (v j, w) D} D = D links v,w (l r) {v 1,, v m } = j=1 n {v (v, w i ) D}, i=1 As usual M is the reflexive and transitive closure of The stssg M comutes the deendencies de(m) T Σ L T, which are given by de(m) = { t, D, u T Σ L T I :, {(ε, ε)}, M t, D, u } Moreover, the stssg M comutes the relation M T Σ T, which is given by M = {(t, u) (t, D, u) de(m)}, and links(m) = {D (t, D, u) de(m)} Examle 29 It can easily be verified that M 1 sort2 (ie, the inverse of M sort2 in which left- and right-hand side are exchanged) of Ex 23 is an stssg Lemma 30 The set links(m) comuted by an stssg M is hierarchical with bounded distance A final examle will use this knowledge about the tye of deendencies that are comutable by an stssg to show a tree transformation that cannot be comuted by any stssg Examle 31 (see [39, Ex 45] and [40]) The comosition of the tree transformations M sort and M 1 sort2 is which is shown in Fig 12 {( l (β m (γ n ())), γ n (β m ( l ()))) l, m, n IN}, Figure 12 already shows some evident deendencies, and we easily notice that they are crossing Conseuently, no stssg deendency is comatible with this deendence because they are all hierarchical by Lemma 30 Lemma 32 The deendencies deicted in Fig 12 are not comatible with the deendencies comuted by any stssg Theorem 33 (see [39, Ex 45]) The tree transformation illustrated in Fig 12 cannot be comuted by any stssg

18 γ β β γ γ γ β β Fig 12 Some deendencies of the tree transformation of Ex 31 6 Summary We resent the essential findings in the table below It additionally contains the synchronous tree-adjoining grammar (stag) [44, 42, 43], which has none of our hierarchy roerties References inut side outut side xto strictly hierarchical strictly hierarchical mbot hierarchical strictly hierarchical stssg hierarchical hierarchical stag 1 Alshawi, H, Bangalore, S, Douglas, S: Learning deendency translation models as collections of finite state head transducers Comut Linguist 26(1), (2000) 2 Arnold, A, Dauchet, M: Bi-transductions de forêts In: Michaelson, S, Milner, R (eds) Proc ICALP Edinburgh University Press (1976) 3 Arnold, A, Dauchet, M: Morhismes et bimorhismes d arbres Theoret Comut Sci 20(1), (1982) 4 Baader, F, Nikow, T: Term rewriting and all that Cambridge University Press (1998) 5 Berstel, J, Reutenauer, C: Recognizable formal ower series on trees Theoret Comut Sci 18(2), (1982) 6 Bikel, DM: On the Parameter Sace of Generative Lexicalized Statistical Parsing Models PhD thesis, University of Pennsylvania (2004) 7 Bloem, R, Engelfriet, J: A comarison of tree transductions defined by monadic second order logic and by attribute grammars J Comut System Sci 61(1), 1 50 (2000)

19 8 Borchardt, B: The Theory of Recognizable Tree Series PhD thesis, Technische Universität Dresden (2005) 9 Charniak, E, Knight, K, Yamada, K: Syntax-based language models for statistical machine translation In: Proc MT Summit IX (2003) 10 Chiang, D: An introduction to synchronous grammars In: Proc ACL Association for Comutational Linguistics (2006), art of a tutorial given with Kevin Knight 11 Collins, M: Head-Driven Statistical Models for Natural Language Parsing PhD thesis, University of Pennsylvania (1999) 12 Courcelle, B, Franchi-Zannettacci, P: Attribute grammars and recursive rogram schemes Theoret Comut Sci 17(2 3), , (1982) 13 Dauchet, M: Transductions inversibles de forêts Thèse 3ème cycle, Université de Lille (1975) 14 Engelfriet, J, Fülö, Z, Vogler, H: Bottom-u and to-down tree series transformations J Autom Lang Combin 7(1), (2002) 15 Engelfriet, J, Lilin, E, Maletti, A: Comosition and decomosition of extended multi bottom-u tree transducers Acta Inf 46(8), (2009) 16 Engelfriet, J, Maneth, S: Macro tree transducers, attribute grammars, and MSO definable tree translations Inform and Comut 154(1), (1999) 17 Engelfriet, J, Maneth, S: Macro tree translations of linear size increase are MSO definable SIAM J Comut 32(4), (2003) 18 Engelfriet, J, Vogler, H: Macro tree transducers J Comut System Sci 31(1), (1985) 19 Engelfriet, J, Vogler, H: Modular tree transducers Theoret Comut Sci 78(2), (1991) 20 Fülö, Z: On attributed tree transducers Acta Cybernet 5(3), (1981) 21 Fülö, Z, Kühnemann, A, Vogler, H: A bottom-u characterization of deterministic to-down tree transducers with regular look-ahead Inf Process Lett 91(2), (2004) 22 Fülö, Z, Kühnemann, A, Vogler, H: Linear deterministic multi bottom-u tree transducers Theoret Comut Sci 347(1 2), (2005) 23 Fülö, Z, Maletti, A, Vogler, H: Preservation of recognizability for synchronous tree substitution grammars In: Drewes, F, Kuhlmann, M (eds) Proc ATANLP 1 9 Association for Comutational Linguistics (2010) 24 Gécseg, F, Steinby, M: Tree Automata Akadémiai Kiadó, Budaest (1984) 25 Gécseg, F, Steinby, M: Tree languages In: Rozenberg, G, Salomaa, A (eds) Handbook of Formal Languages, vol 3, cha 1, 1 68 Sringer (1997) 26 Graehl, J, Knight, K, May, J: Training tree transducers Comut Linguist 34(3), (2008) 27 Klein, D, Manning, CD: Accurate unlexicalized arsing In: Proc ACL Association for Comutational Linguistics (2003) 28 Klein, D, Manning, CD: Fast exact inference with a factored model for natural language arsing In: Proc NIPS 3 10 MIT Press (2003) 29 Knight, K, Graehl, J: An overview of robabilistic tree transducers for natural language rocessing In: CICLing LNCS, vol 3406, 1 24 Sringer (2005) 30 Knuth, DE: Semantics of context-free languages Math Systems Theory 2(2), (1968) 31 Koehn, P: Statistical Machine Translation Cambridge University Press (2010) 32 Kuich, W: Tree transducers and formal tree series Acta Cybernet 14(1), (1999) 33 Lilin, E: Proriétés de clôture d une extension de transducteurs d arbres déterministes In: Proc CAAP LNCS, vol 112, Sringer (1981)

20 34 Maletti, A: Comositions of extended to-down tree transducers Inform and Comut 206(9 10), (2008) 35 Maletti, A: An alternative to synchronous tree substitution grammars J Natur Lang Engrg 17(2), (2011) 36 Maletti, A: How to train your multi bottom-u tree transducer In: Proc ACL Association for Comutational Linguistics (2011) 37 Maletti, A, Graehl, J, Hokins, M, Knight, K: The ower of extended to-down tree transducers SIAM J Comut 39(2), (2009) 38 Milo, T, Suciu, D, Vianu, V: Tyechecking for XML transformers J Comut System Sci 66(1), (2003) 39 Radmacher, FG: An automata theoretic aroach to the theory of rational tree relations Tech Re AIB , RWTH Aachen (2008) 40 Raoult, JC: Rational tree relations Bull Belg Math Soc 4, (1997) 41 Rounds, WC: Maings and grammars on trees Math Systems Theory 4(3), (1970) 42 Shieber, SM: Synchronous grammars as tree transducers In: Proc TAG (2004) 43 Shieber, SM: Probabilistic synchronous tree-adjoining grammars for machine translation: The argument from bilingual dictionaries In: Proc SSST Association for Comutational Linguistics (2007) 44 Shieber, SM, Schabes, Y: Synchronous tree-adjoining grammars In: Proc CoLing vol 3, (1990) 45 Sun, J, Zhang, M, Tan, CL: A non-contiguous tree seuence alignment-based model for statistical machine translation In: Proc ACL Association for Comutational Linguistics (2009) 46 Thatcher, JW: Generalized 2 seuential machine mas J Comut System Sci 4(4), (1970) 47 Thatcher, JW: Tree automata: An informal survey In: Aho, AV (ed) Currents in the Theory of Comuting, Prentice Hall (1973) 48 Wu, D: Stochastic inversion transduction grammars and bilingual arsing of arallel corora Comut Linguist 23(3), (1997) 49 Yamada, K, Knight, K: A decoder for syntax-based statistical MT In: Proc ACL Association for Comutational Linguistics (2002) 50 Yu, S: Regular languages In: Rozenberg, G, Salomaa, A (eds) Handbook of Formal Languages, vol 1, cha 2, Sringer (1997) 51 Zhang, M, Jiang, H, Aw, A, Li, H, Tan, CL, Li, S: A tree seuence alignmentbased tree-to-tree translation model In: Proc ACL Association for Comutational Linguistics (2008) 52 Zhang, M, Jiang, H, Li, H, Aw, A, Li, S: Grammar comarison study for translational euivalence modeling and statistical machine translation In: Proc CoLing Association for Comutational Linguistics (2008) 53 Zollmann, A, Venugoal, A, Och, F, Ponte, J: A systematic comarison of hrase-based, hierarchical and syntax-augmented statistical MT In: Proc CoLing Association for Comutational Linguistics (2008)