Efficient Parsing with the Product-Free Lambek Calculus

Transcription

1 Efficient Parsing wit te Prouct-Free Lambek Calculus Timoty A. D. Fowler Department of Computer Science University of Toronto 10 King s College Roa, Toronto, ON, M5S 3G4, Canaa tfowler@cs.toronto.eu Abstract Tis paper provies a parsing algoritm for te Lambek calculus wic is polynomial time for a more general fragment of te Lambek calculus tan any previously known algoritm. Te algoritm runs in worst-case time O(n 5 ) wen restricte to a certain fragment of te Lambek calculus wic is motivate by empirical analysis. In aition, a set of parameterize inputs are given, sowing wy te algoritm as exponential worst-case running time for te Lambek calculus in general. 1 Introuction A wie variety of grammar formalisms ave been explore in te past for parsing natural language sentences. Te most prominent of tese formalisms as been context free grammars (CFGs) but a collection of formalisms known as categorial grammar (CG) (Ajukiewicz, 1935; Dowty et al., 1981; Steeman, 2000) as receive interest because of some significant avantages over CFGs. First, CG is inerently lexicalize ue to te fact tat all of te variation between grammars is capture by te lexicon. Tis is a result of te ric categories wic CG uses in its lexicon to specify te functor-argument relationsips between lexical items. A istinct avantage of tis lexicalization is tat te processing of sentences epens upon only tose categories containe in te string an not some global set of rules. Secon, CG as te avantage tat it centrally aopts te principle of compositionality, as outline in Montague c License uner te Creative Commons Attribution-Noncommercial-Sare Alike 3.0 Unporte license (ttp://creativecommons.org/licenses/by-nc-sa/3.0/). Some rigts reserve. grammar (Montague, 1974), allowing te semantic erivation to exactly parallel te syntactic erivation. Tis leas to a semantical form wic is easily extractable from te syntactic parse. A large number of CG formalisms ave been introuce incluing, among oters, te Lambek calculus (Lambek, 1958) an Combinatory Categorial Grammar (CCG) (Steeman, 2000). Of tese, CCG as receive te most zealous computational attention. Impressive results ave been acieve culminating in te state-of-te-art parser of Clark an Curran (2004) wic as been use as te parser for te PASCAL Recognizing Textual Entailment Callenge entry of Bos an Markert (2005). Te appeal of CCG can be attribute to te existence of efficient parsing algoritms for it an te fact tat it recognizes a milly contextsensitive language class (Josi et al., 1989), a language class more powerful tan te context free languages (CFLs) tat as been argue to be necessary for natural language syntax. Te Lambek calculus provies an ieal contrast between CCG an CFGs by being a CG formalism like CCG but by recognizing te CFLs like CFGs (Pentus, 1997). Te primary goal of tis paper is to provie an algoritm for parsing wit te Lambek calculus an to sketc its correctness. Furtermore, a time boun of O(n 5 ) will be sown for tis algoritm wen restricte to prouct-free categories of boune orer (see section 2 for a efinition). Te restriction to boune orer is not a significant restriction, ue to te fact tat categories in CCGbank 1 (Hockenmaier, 2003), a CCG corpus, ave a maximum orer of 5 an an average orer of 0.78 by token. In aition to te presentation of te algoritm, we will provie a parameterize set of in- 1 Altoug CCGbank was built for CCG, we believe tat transforming it into a Lambek calculus bank is feasible.

2 puts (of unboune orer) on wic te algoritm as exponential running time. Te variant of te Lambek calculus consiere ere is te prouct-free Lambek calculus cosen for tree reasons. First, it is te founation of all oter non-associative variants of te Lambek calculus incluing te original Lambek calculus (Lambek, 1958) an te multi-moal Lambek calculus (Moortgat, 1996). Secon, te calculus wit prouct is NP-complete (Pentus, 2006), wile te sequent erivability in te prouct-free fragment is still unknown. Finally, te only connectives inclue are / an \, wic are te same connectives as in CCG, proviing a corpus for future work suc as builing a probabilistic Lambek calculus parser. 2 Problem specification Parsing wit te Lambek calculus is treate as a logical erivation problem. First, te wors of a sentence are assigne categories wic are built from basic categories (e.g. NP an S) an te connectives \ an /. For example, te category for transitive verbs is (NP\S)/NP an te category for averbs is (S/NP)\(S/NP) 2. Intuitively, te \ an / operators specify te arguments of a wor an te irection in wic tose arguments nee to be foun. Next, te sequent is built by combining te sequence of te categories for te wors wit te symbol an te sentence category (e.g. S). Strictly speaking, tis paper only consiers te parsing of categories witout consiering multiple lexical entries per wor. However, using tecniques suc as supertagging, te results presente ere yiel an efficient meto for te broaer problem of parsing sentences. Terefore, we can take te size of te input n to be te number of basic categories in te sequent. A parse tree for te sentence correspons to a proof of its sequent an is restricte to rules following te templates in figure 1. In figure 1, lowercase Greek letters represent categories an uppercase Greek letters represent sequences of categories. A proof for te sentence Wo loves im? is given in figure 2. Te version of te Lambek calculus presente above is known as te prouct-free Lambek calculus allowing empty premises an will be enote by L. In aition, we will consier te fragment L k, obtaine by restricting L to categories of orer boune by k. Te orer of a category, wic can 2 We use Ajukiewicz notation, not Steeman notation. α α Γ α βθ γ Γα\βΘ γ Γ α βθ γ β/αγθ γ αγ β Γ α\β Γα β Γ β/α Figure 1: Te sequent presentation of L. NP NP S S NP NP S S NP NP \S S NP \S NP \S S/(NP \S) NP \S S S/(NP \S) (NP \S)/NP NP S Wo loves im Figure 2: A erivation for Wo loves im?. be viewe as te ept of te nesting of argument implications, is efine as: o(α) = 0 for α a basic category o(α/β) = o(β\α) = max(o(α),o(β) + 1) For example, o((np\s)/np) = 1 an o((s/ NP)\(S/NP)) = 2. 3 Relate work Two oter papers ave provie algoritms similar to te one presente ere. Carpenter an Morrill (2005) provie a grap representation an a ynamic programming algoritm for parsing in te Lambek calculus wit prouct. However, ue to tere use of te Lambek calculus wit prouct an to teir coice of correctness conitions, tey i not obtain a polynomial time algoritm for any significant fragment of te calculus. Aarts (1994) provie an algoritm for L 2 wic is not correct for L. Ours is polynomial time for L k, for any constant k, an is correct for L, albeit in exponential running time. A number of autors ave provie polynomial time algoritms for parsing wit CCG wic gives some insigt into ow goo our boun of O(n 5 ) is. In particular, Vijay-Sanker an Weir (1994) provie a cart parsing algoritm for CCG wit a time boun of O(n 6 ). 4 An algoritm for parsing wit L Tis section presents a cart parsing algoritm similar to CYK were entries in te cart are arcs annotate wit graps. Te graps will be referre

3 to as abstract term graps (ATGs) since tey are grap representations of abstractions over semantic terms. ATGs will be presente in tis section by construction. See section 5 for teir connection to te proof structures of Roora (1991). Te algoritm consists of two steps. First, te base case is compute by builing te base ATG B an etermining te set of surface variables by using te proof frames of Roora (1991). Secon, te cart is fille in iteratively accoring to te algoritms specifie in te appenix. Te etails for tese two steps can be foun in sections 4.1 an 4.2, respectively. Section 4.3 introuces a proceure for culling extraneous ATGs wic is necessary for te polynomial time proof an section 4.4 iscusses recovery of proofs from te packe cart. An example of te algoritm is given in figure 3. For parsing wit L, te input is a sequent an for parsing wit L k, te input is a sequent wit categories wose orer is boune by k. Upon completion, te algoritm outputs YES if tere is an arc from 0 to n 1 an NO oterwise. 4.1 Computing te base case Computing te base case consists of builing te proof frame an ten translating it into a grap, te base ATG B Builing te proof frame Proof frames are te part of te teory of proof nets wic we nee to buil te base ATG. Te proof frame for a sequent is a structure built on top of te categories of te sentence. To buil te proof frame, all categories in te sequent are assigne a polarity an labelle by a fres variable. Categories to te left of are assigne negative polarity an te category to te rigt of is assigne positive polarity. Ten, te four ecomposition rules sown in table 1 are use to buil a tree-like structure (see figure 3). Te ecomposition rules are rea from bottom to top an sow ow to ecompose a category base on its main connective an polarity. In table 1, is te label of te category being ecompose, f, g an are fres variables an orer of premises is important. Te bottom of te proof frame consists of te original sequent s categories wit labels an polarities. Tese are calle terminal formulae. Te top of te proof frame consists of basic categories wit labels an polarities. Tese are calle te axiomatic formulae. In aition, we will istinguis α + : f β : f α\β : α : f β + : f α/β : β + : α : g α\β + : β : g α + : α/β + : Table 1: Te proof frame ecomposition rules. te leftmost variable in te label of eac axiomatic formula as its surface variable. See figure 3 for an example Builing te Base ATG Te base ATG B is built from te proof frame in te following way. Te vertices of te base ATG are te surface variables plus a new special vertex τ. Te eges of ATGs come in two forms: Labelle an unlabele, specifie as s,, l an s,, respectively, were s is te source, is te estination an l, were present, is te label. To efine te ege set of B, we nee te following: Definition. For a variable u tat labels a positive category in a proof frame, te axiomatic reflection, ρ(u), is te unique surface variable v suc tat on te upwar pat from u an v in te proof frame, tere is no formula of negative polarity. For example, in figure 3, ρ(b) = c. Te egeset E of te base ATG is as follows: 1. m,ρ(p i ) E for 1 i k were mp 1...p k appears as te label of some negative axiomatic formula 2. τ,ρ(t) E were t is te label of te positive terminal formula 3. For eac rule wit a positive conclusion, negative premise labelle by g an positive premise labelle by, ρ(), g, g E A labele ege in an ATG specifies tat its source must eventually connect to its estination to complete a pat corresponing to its label. For example, G 1 contains te ege c,e, wic inicates tat to complete te pat from c to, we must connect c to e. In contrast, an unlabele ege in an ATG specifies tat its source is alreay connecte to its estination. For example, in figure 3, G 3 contains te ege a,f wic inicates tat tere is some pat, over previously elete noes, wic connects a to f.

4 B = a c e f g G 6 = G 5 = τ a G 3 = a f G 4 = τ g a c G 1 = a c e f G 2 = a c e g B B B B B B B Cart 0 a 1 c 2 3 g 4 e 5 f 6 7 i S : ab S + : c NP : NP + : g S : efg NP + : f NP : S + : i NP \S + : b NP \S : ef S/(NP \S) : a (NP \S)/NP : e Wo loves im? Figure 3: Te algoritm s final state on te sequent S/(NP \S) (NP \S)/NP NP S. Surface Variables Proof Frame Sentence Note tat all noes in an ATG ave unlabele in-egree of eiter 0 or 1 an tat te vertices of an ATG are te surface variables foun outsie its arc. 4.2 Filling in te cart Once te base ATG an te sequence of surface variables is etermine, we can begin filling in te cart. Te term entry refers to te collection of arcs beginning an ening at te same noes of te cart. An arc s lengt is te ifference between its beginning an en points, wic is always o. Note tat eac entry in te example in figure 3 contains only one arc. We will iterate across te entries of te cart an at eac entry, we will attempt a Bracketing an a number of Ajoinings. If an attempt results in a violation, no new ATG is inserte into te cart. Oterwise, a new ATG is compute an inserte at an appropriate entry. Bracketing is an operation on a single ATG were we attempt to exten its arc by connecting two noes wit te same basic category an opposite polarity. For example, G 3 is te result of bracketing G 1. Ajoining, on te oter an, is an operation on two ajacent ATGs were we attempt to unify teir ATGs into one larger ATG. For example, G 5 is te result of ajoining G 3 an G 2. Te cart filling process is escribe by algoritm 1 in te appenix. Te cart in figure 3 is fille by te graps G 1,...,G 6, in tat orer. A walk troug of te example is given in te remainer of tis section. Arcs of lengt 1 are treate specially, since tey are erive irectly from te base ATG. To sow tis, te base ATG is sown at pseuo-noes, labele by Bs Inserting arcs of lengt 1 Tis section correspons to lines 1-2 of algoritm 1 in te appenix. For eac arc from i to i+1, we will attempt to bracket te base ATG from axiomatic formula i to axiomatic formula i + 1. To follow our example, te first step is to consier inserting an arc from 0 to 1 by bracketing B. Bracketing causes a positive surface variable to be connecte to a negative surface variable an in tis case, a cycle from a to c an back to a is forme resulting in te violation on line 12 of algoritm 2. Terefore, no arc is inserte. Ten, te secon step consiers inserting an arc from 1 to 2. However, axiomatic formula 1 as category S an axiomatic formula 2 as category NP wic results in te violation on line 3 of algoritm 2 since tey are not te same. Next, we attempt to insert an arc from 2 to 3. In tis case, no violations occur meaning tat we can insert te arc. Te intuition is tat te ATG for tis arc is obtaine by connecting g to in te base ATG. Since c must eventually connect to (c ), an now g connects to, te inegree constraint on ATG noes requires tat te pat connecting c to pass troug g. Furter-

5 a c e f a c e f a f Figure 4: Te intuition for bracketing from c to e. more, te only way to connect c to g is troug e. So c e. Ten, we elete an g. Tis proceure continues until we ave consiere all possible arcs of lengt Inserting arcs of lengt 3 an greater Next, we iterate across graps in te cart an for eac, consier weter its ATG can be brackete wit te axiomatic formulae on eiter sie of it an weter it can be ajoine wit any of te oter graps in te cart. Tis process closely resembles CYK parsing as escribe on lines 3-10 of algoritm 1. Te coice of sortest to longest is important because part of te invariant of our ynamic program is tat all erivable ATGs on sorter arcs ave alreay been ae. Following our example, te first grap to be consiere is G 1. First, we attempt to bracket it from axiomatic formulae 1 to 4. As before, tis intuitively involves connecting c to e in te ATG for tis arc. Tis is allowe because no cycles are forme an no labelle eges are proibite from eventually being connecte. Ten, as before, we elete te vertices c an e an as a result connect a to f, resulting in G 3. Te bracketing process is illustrate in figure 4. Next, we consier all graps to wic G 1 coul ajoin an tere are none, since suc graps woul nee to annotate arcs wic eiter en at 1 or begin at 4. After processing G 1, we process G 2, wic as a successful bracketing resulting in G 4 an no successful ajoinings. Next, we process G 3. Bracketing it is proibite, as it woul result in a cycle from a to f an back to a. However, it is possible to ajoin G 3 wit G 2, since tey are ajacent. Te ajoining of two graps can be viewe as a kin of intersection of te two ATGs, in te sense tat we are combining te information in bot graps to yiel a single more concise grap. Attempting an ajoining involves traversing te two graps being ajoine an te base ATG in bot a forwar an a backwar irection as specifie in algoritms 4 an 5 in te appenix. Te intuition bein tese traversals is to generate a picture of wat te combination of te two a f a c e g a c e f g a Figure 5: Te intuition for ajoining two ATGs. graps must look like as illustrate in figure 5. In general, we can only reconstruct tose parts of te grap wic are necessary for etermining te resultant ATG an no more. Te otte eges inicate uncertainty about te eges present at tis stage of te algoritm. Ajoining G 2 an G 3 oes not fail an te resultant grap is G 5. Note tat tis example oes not contain any instances of two ientical ATGs being inserte multiple times into te cart wic occurs often in large examples yieling significant savings of computation. 4.3 Culling of extraneous ATGs It often appens tat an entry in te cart contains two ATGs suc tat if one of tem is extenable to a complete proof ten te oter necessarily is as well. In tis case, te former can be iscare. We will outline suc a meto ere tat is important for te polynomial time proof. Definition. ATGs G 1 an G 2 are equivalent if some surjection of ege labels to ege labels applie to te tose of G 1 yiels tose of G 2. Ten, if two ATGs in a cart are equivalent, one can be iscare. 4.4 Recovering proofs from a packe cart Te algoritm as escribe above is a meto for answering te ecision problem for sequent erivability in te Lambek calculus. However, we can annotate te ATGs wit te ATGs tey are erive from so tat a complete set of Roora-style proof nets, an tus te proofs temselves, can be recovere. 5 Correctness Correctness of te algoritm is obtaine by using structural inuction to prove te equivalence of te constructive efinition of ATGs outline in section 4 an a efinition base on semantic terms given in tis section:

6 S + : c NP : S : ab NP \S + : b S/(NP \S) : a NP + : g S : efg NP \S : ef NP + : f (NP \S)/NP : e NP : S + : i Figure 6: A proof structure for Wo loves im?. Definition. A partial proof structure is a proof frame togeter wit a matcing of te axiomatic formulae. A proof structure is a partial proof structure wose matcing is complete. An example is given in figure 6. Proof structures correspon to proofs uner certain conitions an our conitions will be base on te semantic term of te proof given to us by te Curry-Howar isomorpism for te Lambek calculus (Roora, 1991). To o tis, we interpret left rules as functional application an rigt rules as functional abstraction of lamba terms. Uner tis interpretation, te semantic term obtaine from te proof structure in figure 6 is aλ.e. As in Roora (1991), proof structures correspon to a proof if te semantic term assigne to te sentence category is a well forme lamba term wic inclues all te terms assigne to te wors of te sentence. Ten, ATGs are grap representations of abstractions of te unetermine portion of semantic terms of partial proof structures. Unlabele eges correspon to functional applications wose arguments must still be etermine an labelle eges correspon to functional abstractions wose boy oes not yet contain an instance of te abstracte variable. Te violations wic occur uring te execution of te algoritm correspon to te various ways in wic a lamba term can be ill forme. 6 Asymptotic Running Time Complexity In tis section we provie proof sketces for te runtime of te algoritm. Let f(n) be a boun on te number of arcs occurring in an entry in te cart were n is te number of axiomatic formulae. Ten, observe tat te number of eges witin an ATG is O(n 2 ) an te number of eges ajacent to a vertex is O(n), ue to basic properties of ATGs. Ten, it is not ar to prove tat te worst case running time of Bracketing is O(n 2 ), wic is ominate by te for loops of lines of algoritm 2. Next, wit some effort, we can see tat te worst case running time of Ajoining is ominate by te execution of te proceures Fore an Back. But, since tere are at most a linear number of labels l an for eac label l we nee to visit eac vertex in G 1 an G 2 at most a constant number of times, te worst case running time is O(n 2 ). Ten, for eac ATG, we attempt at most one bracketing an ajoinings wit at most 2n+1 oter entries for wic tere can be (2n+1)f(n) ATGs. Terefore, eac entry can be processe in worst case time O(n 3 f(n) 2 ). Finally, tere are O(n 2 ) entries in te cart, wic means tat te entire algoritm takes time O(n 5 f(n) 2 ) in te worst case. Sections 6.1 an 6.2 iscuss te function f(n). 6.1 Runtime for L k By structural inuction on te proof frame ecomposition rules an te base ATG builing algoritm, it can be proven tat in L k te lengt of te longest pat in te base ATG is boune by k. Next, consier a partition of te surface variables into a pair of sets suc tat te axiomatic formulae corresponing to te surface variables witin eac set are contiguous. For te example in figure 3, one suc pair of sets is S 1 = {a,c,,g} an S 2 = {e,f,,i}. Ten, given suc a partition, it can be proven tat tere is at least one maximal pat P in te base ATG suc tat all vertices in one set tat are ajacent to a vertex in te oter set are eiter in P or ajacent to some vertex in P. For example, a maximal pat for S 1 an S 2 is P = e g. An entry in te cart inuces two suc partitions, one at te left ege of te entry an one at te rigt ege. Terefore, we obtain two suc maximal pats an for any ATG G in tis entry an any vertex v not in or ajacent to one of tese pats, eiter v is not in G or v as te same neigbouroo in G as it as in te base ATG. Ten, te number of vertices ajacent to vertices in tese pats can be as many as n. However, if we put tese vertices into sets suc tat vertices in a set ave ientical

7 neigbouroos, te number of sets is epenant only on k. In te worst case, te out-neigbouroo of one of tese sets can be any set of tese sets. So, we get a boun for f(n) to be O(k 2 4 k ). Terefore, because k is constant in L k, f(n) is constant an te running time of te algoritm for L k is O(n 5 ) in te worst case. 6.2 Runtime for L Despite te results of section 6.1, tis algoritm as an exponential running time for L. We emonstrate tis wit te following set of parameterize sequents: F(1) = ((A/A)\A)\A F(i) = ((A/(A/F i 1 ))\A)\A for i > 1 U(n) = F n F n A\A Teorem. Tere are (2n 1)! n!(n 1)! Θ(4 n ) istinct arcs in te entry from n to 3n 1 in te cart for U(n). Proof. By inuction an a mapping from te possible matcings to te possible permutations of a sequence of lengt 2n 1 suc tat two subsequences of lengt n an n 1 are in orer. 7 Conclusions an Future Work We ave presente a novel algoritm for parsing in te Lambek calculus, sketce its correctness an sown tat it is polynomial time in te bouneorer case. Furtermore, we presente a set of parameterize sequents proving tat te algoritm is exponential time in te general case, wic ais future researc in fining eiter a polynomial time algoritm or an NP-completeness proof for L. In aition, tis algoritm provies anoter step towar evaluating te Lambek calculus against bot CFGs (to evaluate te importance of Categorial Grammar) an CCG (to evaluate te importance of te milly context-sensitive languages). In te future, we plan on etermining te running time of tis algoritm on an actual corpus, suc as a moifie version of CCGbank, an ten to empirically evaluate te Lambek calculus for natural language processing. In aition, we woul like to investigate extening tis algoritm to more complex variants of te Lambek calculus suc as te multi-moal calculus using te proof nets of Moot an Puite (2002). Acknowlegments Many tanks to Geral Penn, for is insigtful comments an for guiing tis researc. References Aarts, Erik Proving Teorems of te Secon Orer Lambek Calculus in Polynomial Time. Stuia Logica, 53: Ajukiewicz, Kazimierz Die syntaktisce Konnexitat. Stuia Pilosopica, 1(1-27). Bos, Joan an Katja Markert Recognising textual entailment wit logical inference. Proceeings of HLT an EMNLP, pages Carpenter, Bob an Glyn Morrill Switc Graps for Parsing Type Logical Grammars. Proceeings of IWPT 05, Vancouver. Clark, Steven an James R. Curran Parsing te WSJ using CCG an log-linear moels. Proceeings of ACL 04, pages Dowty, Davi R., Robert E. Wall, an Stanley Peters Introuction to Montague Semantics. Reiel. Hockenmaier, Julia Data an Moels for Statistical Parsing wit Combinatory Categorial Grammar. P.D. tesis, University of Einburg. Josi, Aravin K., K. Vijay-Sanker, an Davi J. Weir Te Convergence of Milly Context-sensitive Grammar Formalisms. University of Pennsylvania. Lambek, Joacim Te matematics of sentence structure. American Matematical Montly, 65: Montague, Ricar Formal pilosopy: selecte papers of Ricar Montague. Yale University Press New Haven. Moortgat, Micael Multimoal linguistic inference. Journal of Logic, Language an Information, 5(3): Moot, Ricar an Quintijn Puite Proof Nets for te Multimoal Lambek Calculus. Stuia Logica, 71(3): Pentus, Mati Prouct-Free Lambek Calculus an Context-Free Grammars. Te Journal of Symbolic Logic, 62(2): Pentus, Mati Lambek calculus is NP-complete. Teoretical Computer Science, 357(1-3): Roora, Dirk Resource Logics: Proofteoretical Investigations. P.D. tesis, Universiteit van Amsteram. Steeman, Mark Te Syntactic Process. Brafor Books.

8 Vijay-Sanker, K. an Davi J. Weir Parsing Some Constraine Grammar Formalisms. Computational Linguistics, 19(4): Appenix. Algoritm Pseuocoe Te term source set refers to te outneigbouroo of τ. Te term minus variable refers to surface variables obtaine from negative axiomatic formulae plus τ. X i refers to te i t axiomatic formula. Algoritm 1 Cart Iteration 1: for i = 0 to n 1 o 2: Bracketing(B, X i, X i+1) 3: for l = 1, 3,5,... to n 1 o 4: for e = 0 to n l 1 o 5: for eac arc from e to e + l wit ATG G o 6: Bracketing(G, X e 1 to X e+l+1 ) 7: Ajoin G to ATGs from e l 1 to e 1 8: for al = 1, 3,..., l 2 o 9: Ajoin G to ATGs from e al 1 to e 1 10: Ajoin G to ATGs from e+l+1 to e+l+al+1 Algoritm 2 Bracketing(G, X i, X j ) p i p j 1: C i : l i = X i an C j : l j = X j 2: if C i C j ten 3: V iolation : Mismatce Basic Categories 4: if p i = p j ten 5: V iolation : Mismatce Polarities 6: Let m,p {i, j} suc tat p m is negative an p p is positive 7: if G is not from 1 to n 1 an te source set of G is te singleton l p an l m as out-egree 0 in G ten 8: V iolation : Empty Source Set 9: if te ege l m, l p G ten 10: V iolation : Cycle Exists 11: if l p is in te source set of G an tere exists an in-ege of m wit label l suc tat no ege from p to m as label l an no ege from a vertex oter tan p to a vertex oter tan m as label l ten 12: V iolation : Pat Completion Impossible 13: if m as out-egree 0 an an tere exists an out-ege of p wit label l suc tat no ege from p to m as label l an no ege from a vertex oter tan p to a vertex oter tan m as label l ten 14: V iolation : Pat Completion Impossible 15: Copy G to yiel H 16: for eac ege l p, l m, l G o 17: Delete all eges from H wit label l 18: Delete l m, l p an all teir incient eges from H 19: Let in p be te in-neigbour of l p in G 20: for eac q in te out-neigbouroo of l m in G o 21: Insert in p, q into H 22: for eac ege p,, l in G o 23: Insert q,, l into H 24: for eac ege q, m, l in G o 25: Insert q, in p, l into H 26: if H contains a cycle ten 27: V iolation : Future Cycle Require 28: return H Algoritm 3 Ajoining(G 1, G 2 ) 1: Let V H be te intersection of te vertices in G 1 an G 2 2: if V H τ an Fore(τ, G 1, G 2) V H = ten 3: V iolation : Empty Source Set 4: for eac l suc tat l labels an ege in G 1 an G 2 o 5: Let p, m, l be te unique ege labelle l in B 6: if Fore(p,G 1, G 2, l) Back(m,G 1, G 2) = ten 7: if Fore(p) V H = ten 8: V iolation : Pat Completion Impossible 9: if Back(m) V H = τ ten 10: V iolation : Pat Completion Impossible 11: Let H be te grap wit vertex set V H an no eges 12: for eac minus variable m V H o 13: for eac p Fore(m,G 1, G 2, ) o 14: Insert m,p into H 15: for eac l suc tat l labels an ege in G 1 an G 2 o 16: Let p, m, l be te unique ege labelle l in B 17: if Fore(p,G 1, G 2, l) Back(m,G 1, G 2) = ten 18: for eac q Fore(p,G 1, G 2, l) V H o 19: Insert q, Back(m, G 1, G 2) V H, l into H 20: return H Algoritm 4 Fore(v, G 1, G 2, l) 1: if v G 1 an v G 2 ten 2: return {v} 3: else 4: if v is a minus vertex ten 5: S = i {1,2} Out-neigbouroo Gi v 6: else if v is a plus vertex ten 7: Let j be suc tat v G j 8: S = e Eges labelle byl Source of e F = S 9: wile S is not empty o 10: Remove any element u from S 11: Let m be te in-neigbour of u in B 12: if u oes not appear in one of G 1, G 2 an m oes not appear in te oter ten 13: Let i be suc tat m G i 14: Let O be te out-neigbouroo of m in G i 15: S = S O 16: F = F O 17: F = F {m} 18: return F Algoritm 5 Back(m, G 1, G 2 ) 1: if m G 1 an m G 2 ten 2: return {m} 3: else 4: Let i, j {1, 2} be suc tat m G i an m / G j 5: Let m be te estination of te eges labelle by m in G j 6: M = {m, m } 7: wile m / G 1 an m / G 2 o 8: Let i, j {1, 2} be suc tat m G i an m / G j 9: Let p G j be an out-neigbour of m in B 10: Let m be te in-neigbour of p in G j 11: m = m 12: M = M {m } 13: return M