arxiv: v2 [cs.fl] 17 Feb 2016

Transcription

1 Symmetry Breking Predites for SAT-sed DFA Identifition Vldimir Ulyntsev, Ily Zkirzynov, Antoly Shlyto ITMO University, Kronverkskiy venue 49, Sint-Petersurg, Russi rxiv: v [s.fl] 17 Fe 016 Astrt It ws shown efore tht the NP-hrd prolem of deterministi finite utomt (DFA) identifition n e effetively trnslted to Boolen stisfiility (SAT). Modern SAT-solvers n tkle hrd DFA identifition instnes effiiently. We present tehnique to redue the prolem serh spe y enforing n enumertion of DFA sttes in depth-first serh(dfs) or redth-first serh(bfs) order. We propose symmetry reking predites, whih n e dded to Boolen formule representing vrious DFA identifition prolems. We show how to pply this tehnique to DFA identifition from oth noiseless nd noisy dt. Also we propose method to identify ll utomt of the desired size. The proposed pproh outperforms the urrent stte-of-the-rt DFASAT method for DFA identifition from noiseless dt. A ig dvntge of the proposed pproh is tht it llows to determine extly the existene or non-existene of solution of the noisy DFA identifition prolem unlike metheuristi pprohes suh s geneti lgorithms. Keywords: Grmmtil inferene, Boolen stisfiility, utomt identifition, symmetry reking tehniques 1. Introdution Deterministi finite utomt (DFA) re models tht reognize regulr lnguges [1], therefore the prolem of DFA identifition (indution, lerning) is one of the est studied [] in grmmtil inferene. The identifition prolem onsists of finding DFA with the miniml numer of sttes tht is onsistent with given set of strings with lnguge ttriution lels. This mens tht suh DFA rejets the negtive exmple strings nd epts the positive exmple strings. It ws shown in [] tht finding DFA with given upper ound on its size (numer of sttes) is n NP-omplete prolem. Besides, in [4] it ws shown tht this prolem nnot e pproximted within ny polynomil. Despite this theoretil diffiulty, severl effiient DFA identifition lgorithms exist []. The most ommon pproh is the evidene driven stte-merging(edsm) lgorithm[5]. The key ide of this lgorithm is to first onstrut n ugmented prefix tree eptor (APTA), tree-shped utomton, from the given leled strings, nd then to pply itertively stte-merging proedure until no vlid merges re left. Thus EDSM is polynomil-time greedy method tht tries to find good lol optimum. EDSM prtiipted in the Adingo DFA lerning ompetition [5] nd won it (in tie). To improve the EDSM lgorithm severl speilized serh proedures were proposed, see, e.g., [6, 7]. One of the most suessful pprohes is the EDSM lgorithm in the red-lue frmework [5], lso lled the Blue-fringe lgorithm. The seond pproh for DFA lerning is sed on evolutionry omputtion; erly work inludes [8, 9]. Lter the uthors of [10] presented n effetive sheme for evolving DFA with multi-strt rndom hill limer, whih ws used to optimize the trnsition mtrix of the identified DFA. A so-lled smrt stte leling sheme ws pplied to hoose stte lels optimlly, given the trnsition mtrix nd the trining Prinipl orresponding uthor Corresponding uthor Emil ddresses: ulyntsev@rin.ifmo.ru (Vldimir Ulyntsev), zkirzynov@rin.ifmo.ru (Ily Zkirzynov), shlyto@mil.ifmo.ru (Antoly Shlyto) Preprint sumitted to Elsevier Ferury 5, 016

2 set. The uthors emphsized tht smrt seletion of stte lels gve the evolutionry method signifint oost whih llowed it to ompete with EDSM. Authors find tht the proposed evolutionry lgorithm (EA) outperforms the EDSM lgorithm on smll trget DFA when the trining set is sprse. For lrger utomt with sttes, the hill limer fils nd EDSM lerly outperforms it. The hllenge of the GECCO 004 Noisy DFA ompetition [11] ws to lern the trget DFA when 10 perent of the given trining string lels hd een rndomly flipped. In [1] Lus nd Reynolds show tht within limited time the EA with smrt stte leling is le to identify the trget DFA even t suh high noise level. The uthors ompred their lgorithm with the results of the GECCO ompetition nd found tht the EA lerly outperformed ll the entries. Therey it is the stte-of-the-rt tehnique for lerning DFA from noisy trining dt. In severl ses the est solution for noiseless DFA identifition is the trnsltion-to-sat tehnique [1], whih ws ltered to suit the StMInA(Stte Mhine Inferene Approhes) ompetition[14] nd ultimtely won. The min ide of tht lgorithm is to trnslte the DFA identifition prolem to Boolen stisfiility (SAT). Thus it is possile to use highly optimized modern DPLL-style SAT solving tehniques [15]. The trnsltion-to-sat pproh ws lso used to effiiently tkle prolems suh s ounded model heking [16], solving SQL onstrints y inrementl trnsltion [17], nlysis of JML-nnotted Jv sequentil progrms [18], extended finite-stte mhine indution [19]. Mny optimiztion prolems exhiit symmetries groups of solutions whih n e otined from eh other vi some simple trnsformtions. To speed up the solution serh proess we n redue the prolem serh spe y performing symmetry reking. In DFA identifition prolems the most strightforwrd symmetries re groups of isomorphi utomt. The ide of voiding isomorphi DFAs y fixing stte numers in redth-first serh(bfs) order ws used in the stte-merging pproh[0](funtion NtOrder) nd in the geneti lgorithm from [1] (Move To Front reorgniztion). Besides, in [1] symmetry reking ws performed y fixing some olors of the APTA verties from lique provided y greedy mx-lique lgorithm pplied in preproessing step of trnsltion-to-sat tehnique. In this pper we propose new symmetry reking predites [15] whih n e dded to Boolen formule representing vrious DFA identifition prolems. These predites enfore DFA sttes to e enumerted in the DFS (depth-first serh) or BFS order. This pproh lerly outperforms urrent stte-of-the-rt DFASAT from [1]. The proposed predites nnot e pplied with the mx-lique tehnique [1] t the smetime, ut our pprohis more flexile. To show the flexiility ofthe pproh,we py ourttention to the se of noisy DFA identfition. Therefore we propose modifition of the noiseless trnsltion-to-sat for the noisy se (Setion ). We show tht the previously proposed mx-lique tehnique is not pplile in this se while our BFS-sed pproh is. The ig dvntge of our pproh is tht we n determine the existene or non-existene of solution in this se unlike geneti lgorithms. Experiments showed tht using BFS-sed symmetry reking predites n signifintly redue the time of lgorithm exeution. We lso show tht our strtegy outperforms the urrent stte-of-the-rt EA from [1] if the numer of the trget DFA sttes, the noise level nd the numer of strings re smll. We lso propose modifition of this method to solve the prolem of finding ll utomt with the miniml numer of sttes whih re onsistent with given set of strings.. Enoding DFA Identifition into SAT The gol of DFA identifition is to find smllest DFA A suh tht every string from S +, set of positive exmples, is epted y A, nd every string from S, set of negtive exmples, is rejeted. The size of A is defined s the numer of sttes C it hs. The lphet Σ = {l 1,...,l L } of the sought DFA A is the set of ll symols from S + nd S where L is the lphet size. The exmple of the smllest DFA for S + = {,,,} nd S = {,} is shown in Fig. 1. In this work we ssume tht DFA sttes re numered from 1 to C nd the strt stte hs numer 1. In [1] Heule nd Verwer proposed ompt trnsltion of the DFA identifition prolem into SAT. Here we riefly review the proposed tehnique, sine our symmetry reking predites supplement it. The first step of oth the stte-merging nd the trnsltion-to-sat tehniques is the ugmented prefix tree

3 1 Figure 1: An exmple of DFA eptor (APTA) onstrution from the given exmples S + nd S. An APTA is tree-shped utomton suh tht pths orresponding to two strings reh the sme stte v if nd only if these strings shre the sme prefix in whih the lst symol orresponds to v. We denote y V the set of ll APTA sttes; y v r the APTA root; y V + the set of epting sttes; nd y V the set of rejeting sttes. Moreover, for stte v (exept v r ) we denote its inoming symol y l(v) nd its prent y p(v). The APTA for S + nd S mentioned ove is shown in Fig () An exmple of n APTA for S + = {,,,} nd S = {,} () The onsisteny grph for the APTA from Fig. Figure : An exmple of n APTA nd its onsisteny grph The seond step of the tehnique proposed in [1] is the onstrution of the onsisteny grph (CG) for the otined APTA. The set of nodes of the CG is identil to the set of APTA sttes. Two CG nodes v nd w re onneted with n edge (nd lled inonsistent) if merging v nd w in the APTA results in n inonsisteny: n epting stte is merged with rejeting stte. Let E denote the set of CG edges. The CG for APTA of Fig. is shown in Fig.. The key prt of the lgorithm is trnslting the DFA identifition prolem into Boolen formul in onjuntive norml form (CNF) nd using SAT solver to find stisfying ssignment. For given set of exmples nd fixed DFA size C the solver returns stisfying ssignment (tht defines DFA with C sttes tht is omplint with S + nd S ) or messge tht it does not exist. The min ide of this trnsltion is to use distint olor for every stte of the identified DFA nd to find onsistent mpping of APTA sttes to olors. Three types of vriles were used in the proposed ompt trnsltion: 1. olor vriles x v,i 1 (v V; 1 i C) if nd only if APTA stte v hs olor i (APTA stte v will e merged into DFA stte i);. prent reltion vriles y l,i,j 1 (l Σ; 1 i,j C) if nd only if DFA trnsition with symol l from stte i ends in stte j;

4 . epting olor vriles z i 1 (1 i C) if nd only if DFA stte i is epting. Diret enoding, desried in [1], uses only vriles x v,i ; vriles y l,i,j nd z i re uxiliry nd used in ompt enoding predites, whih re desried elow. The ompt trnsltion proposed in [1] uses nine types of luses: 1. x v,i z i (v V + ; 1 i C) definitions of z i vlues for epting sttes ( x v,i z i );. x v,i z i (v V ; 1 i C) definitions of z i vlues for rejeting sttes ( x v,i z i );. x v,1 x v,... x v,c (v V) eh stte v hs t lest one olor; 4. x p(v),i x v,j y l(v),i,j (v V \ {v r }; 1 i,j C) DFA trnsition is set when stte nd its prent re olored (y l(v),i,j x p(v),i x v,j ); 5. y l,i,j y l,i,k (l Σ; 1 i,j,k C; j < k) eh DFA trnsition n trget t most one stte ( y l,i,j y l,i,k ); 6. x v,i x v,j (v V; 1 i < j C) eh stte hs t most one olor; 7. y l,i,1 y l,i,... y l,i,c (l Σ; 1 i C) eh DFA trnsition must trget t lest one stte; 8. y l(v),i,j x p(v),i x v,j (v V \{v r }; 1 i,j C) stte olor is set when DFA trnsition nd prent olor re set ( y l(v),i,j x p(v),i x v,j ); 9. x v,i x w,i ((v,w) E; 1 i C) the olors of two sttes onneted with n edge in the onsisteny grph must e different ( x v,i x w,i ). Thus, the onstruted formul onsists of O(C V ) luses nd, if the SAT solver finds solution, we n identify the DFA. To find miniml DFA, the uthors use inrementl SAT solving. The initil DFA size C is equl to the size of lrge lique found in the CG. To find suh lique, greedy lgorithm proposed in [1] n e pplied. Then the miniml DFA is found y iterting over the DFA size C until the formul is stisfied. Algorithm 1 illustrtes this pproh. The found lique ws lso used to perform symmetry reking: in ny vlid oloring of grph, ll sttes in lique must hve different olor. Thus, we n fix the stte olors in the lique during the preproessing step. Lter we will see tht the mx-lique symmetry reking is not omptile with the one proposed in this pper. To redue the SAT serh spe signifintly, the uthors pplied severl EDSM steps efore trnslting the prolem to SAT. Sine EDSM nnot gurntee the minimlity of solution, we will omit the onsidertion of this step in our pper.. Lerning DFA from Noisy Smples The trnsltion desried in the previous setion dels with ext DFA identifition. In this setion we show how to modify the trnsltion in order to pply it to noisy exmples. We ssume tht not more thn K ttriution lels of the given trining strings were rndomly flipped. Solving this prolem ws the gol of the GECCO 004 Noisy DFA ompetition [11] (with K equl to 10 perent of the numer of the given trining strings). The EA with smrt stte leling ws lter proposed in [1], nd sine tht time it hs een, to the est of our knowledge, the stte-of-the-rt tehnique for lerning DFA from noisy trining dt. In the noisy se we nnot use the APTA node onsisteny: we nnot determine whether n epting stte is merged with rejeting stte sine orret string lels re unknown. Thus we nnot use CG nd the mx-lique symmetry reking. The ide of our modifition is rther simple: for eh leled stte of the APTA we define vrile whih sttes whether the lel n e flipped. The numer of flips is limited y K. Formlly, for eh 4

5 Dt: set of strings SS, minimum size of the trget DFA minsize, mximum size of the trget DFA mxsize, symmetry reking strtegy SBStrtegy, externl SAT-solver solver, time limit for SAT-solver TL APTA uildapta(ss) CG uildcg(apta) lique findclique(apta, CG) minsize mx(minsize, lique.size) for olors minsize to mxsize do dimsfile genertefileindimsformt(apta, CG, olors, SBStrtegy) result solver.solve(dimsfile, TL) if result is SAT then solution solver.getsolution() DFA uildsolution(solution) return DFA end end return null Algorithm 1: Sheme of the SAT-sed lgorithm v V ± = V + V we define f v whih is true if nd only if the lel of stte v n (ut does not hve to) e inorret (f lipped). Using these vriles, we n modify the trnsltion proposed in [1] to tke into ount mistkes in string lels. To do this, we hnge the z i definition luses (items 1 nd from list in Setion ): euse of mistke possiility they hold in se f v is flse. Thus, new z i vlue definitions re expressed in the following wy: f v (x v,i z i ) for v V + ; f v (x v,i z i ) for v V. To limit the numer of orretions to K we use n uxiliry rry of K integer vriles. This rry stores the numers of the APTA sttes for whih lels n e flipped. Thus, f v is true if nd only if the rry ontins v. To void the onsidertion of isomorphi permuttions we enfore the rry to e sorted in the inresing order. To represent the uxiliry rry s Boolen formul we define vriles r i,v for 1 i K nd v V ± = {v 1,...,v W }. r i,v is true if nd only if v is stored in the i-th position of the rry. To onnet vriles f v with r i,v we dd so-lled hnneling onstrins: f v (r 1,v... r K,v ) for eh v V ±. We hve to stte tht extly one r i,v is true for eh position i in the uxiliry rry. To hieve tht we use the order enoding method []. We dd uxiliry order vriles o i,v for 1 i K nd v V ± = {v 1,...,v W }. We ssume tht o i,v for v {v 1,...,v j } nd o i,v for v {v j+1,...,v W } for some j. This n e expressed y the following onstrint: o i,vj+1 o i,vj for 1 j < W. Now we define tht r i,vj o i,vj o i,vj+1. We lso dd luses o i,vj o i+1,vj+1 (for 1 i < K nd 1 j < W) to store orretions in the inresing order. The proposed onstrints in CNF re listed in Tle 1; there re O(C V ± +K V ± ) luses. Thus, to modify the trnsltion for the noiseless se to del with noise we n reple the z i vlue definition nd inonsisteny luses (items 1, nd 9 from list in Setion ) with the ones listed in Tle DFS-sed Symmetry Breking Predites In this setion we propose wy to fix utomt stte enumertion to void onsidertion of isomorphi utomt during SAT solving. The min ide of our symmetry reking is to enfore DFA sttes to e enumerted in the depth-first serh (DFS) order. Thus only one representtive of eh equivlene lss with respet to the isomorphi reltion will e proessed. It is neessry to find ll djent unvisited sttes for eh unvisited stte of the DFA during DFS proessing. First the lgorithm hndles the strt DFA stte. Then the lgorithm proesses the hildren of this stte nd reursively runs for eh of them. We will use n uxiliry struture, n rry with trnsitions whih onnet its elements. These elements re the numers of sttes ordered ording to the 5

6 Tle 1: Cluses for noisy DFA identifition Cluses CNF representtion Rnge f v (x v,i z i ) x v,j z j f v 1 j C; v V + f v (x v,i z i ) x v,j z j f v 1 j C; v V f v (r 1,v... r K,v ) f v r 1,v... r K,v v V ± r i,v f v r i,v f v 1 i K; v V ± r i,vj o i,vj r i,vj o i,vj 1 i K; 1 j W r i,vj o i,vj+1 r i,vj o i,vj+1 1 i K; 1 j < W o i,vj o i,vj+1 r i,vj o i,vj o i,vj+1 r i,vj 1 i K; 1 j < W o K,vW r K,vW o K,vW r K,vW o i,vj+1 o i,vj o i,vj+1 o i,vj 1 i K; 1 j < W o i,vj o i+1,vj+1 o i,vj o i+1,vj+1 1 i < K; 1 j < W DFS enumertion. Eh trnsition onneting the elements of the rry will e opy of DFA trnsition whih ws used y DFS. Sine our trnsitions re leled with symols from Σ, we proess hild sttes in the lphetil order of symols l on trnsitions i l j. We ll DFA DFS-enumerted if its uxiliry rry is filled y onseutive numers in the inresing order strting from 1. An exmple of DFS-enumerted DFA with six sttes is shown in Fig. (DFS tree trnsitions tht were used to dd sttes into the rry re mrked old); the DFS uxiliry rry for this DFA is shown in Fig. 6. The DFA shown in Fig. 1 is not DFS-enumerted DFS first hndles stte rther thn stte (we onsider 1 efore 1 ).,, 1, 4,, 6, () DFS-enumerted DFA with olded DFS tree edges () DFS uxiliry rry. The elements orrespond to DFA sttes, nd the trnsitions re the ones used in the DFS trversl Figure : An exmple of DFS-numerted DFA nd its DFS uxiliry rry We propose the onstrints tht enfore DFA to e DFS-enumerted. We ssume tht trnsltion of given DFA identifition prolem to SAT dels with Boolen vriles y l,i,j (l Σ; 1 i,j C) to set the DFA trnsition funtion: y l,i,j 1 if nd only if the trnsition with symol l from stte i ends in stte j. The min ide is to determine eh stte s prent in the DFS-tree nd set onstrins etween sttes prents. We store prents in vlues p j,i (for eh 1 i < j C). p j,i is true if nd only if stte i is the prent of j in the DFS-tree. Eh stte exept the initil one must hve prent with smller numer, thus (p j,1 p j,... p j,j 1 ). j C We set prent vriles p j,i through y l,i,j using uxiliry vriles t i,j. In the DFS-enumertion stte 6

7 i k j q Figure 4: Prt of the rry illustrting the prent ordering predites. The trnsitions show prent reltions. The dotted trnsition is not llowed due to the DFS-enumertion j ws dded into the rry while proessing the stte with mximl numer i mong sttes tht hve trnsition to j: (p j,i t i,j t i+1,j... t j 1,j ), 1 i<j C where t i,j 1 if nd only if there is trnsition etween i nd j. We define these uxiliry vriles using y l,i,j : (t i,j y l1,i,j... y ll,i,j). 1 i<j C Moreover, in the DFS-enumertion sttes prents must e ordered. If i is the prent of the stte j nd k is stte etween i nd j (i < k < j) then there is no trnsition from stte k to stte q whih is igger then j (see Fig. 4): (p j,i t k,q ). 1 i<k<j<q<c Now to enfore the DFA to e DFS-enumerted we hve to order hildren in the lphetil order of symols on trnsitions. We onsider two ses: lphet Σ onsists of two symols {,} nd more thn two symols {l 1,...,l L }. In the se of two symols only two sttes j nd k n hve the sme prent i (where without loss of generlity j < k). In this se we fore the trnsition tht strts in stte i leled with symol to end in stte j insted of k: (p j,i p k,i y,i,j ). 1 i<j<k<c In the seond se we hve to introdue the third type of vriles in our symmetry reking predites. We store the lphetilly miniml symol on trnsitions etween sttes: m l,i,j is true if nd only if there l is trnsition i j nd there is no suh trnsition with n lphetilly smller symol. We onnet these vriles with DFA trnsitions y dding the following hnneling predites: (m ln,i,j y ln,i,j y ln 1,i,j... y l1,i,j). 1 i<j C 1 n L Now it remins to rrnge sttes j nd k with the sme prent i in the lphetil order of miniml symols on trnsitions etween them nd i (see Fig. 5): (p j,i p k,i m ln,i,j m lm,i,k). 1 i<j<k C 1 m<n L Thus we propose symmetry reking predites tht re omposed of the listed onstrints. Predites (for the se of three or more symols) trnslted into O(C 4 +C L ) CNF luses re listed in Tle. 5. BFS-sed Symmetry Breking Predites In this setion we onsider modifition of the DFS-sed pproh whih enfore DFA sttes to e enumerted in the redth-first serh (BFS) order. This ide ws lso used in funtion NtOrder in 7

8 l i j k l Figure 5: Illustrtion of lphetil ordering predites. If i is the prent of j nd k, l n (l m) is the lphetilly miniml symol on trnsitions etween i nd j (i nd k) then l m nnot e lphetilly smller thn l n the stte-merging pproh from [0] nd the Move To Front reorgniztion lgorithm used in the geneti lgorithm [1]. BFS uses the queue dt struture to store intermedite results s it trverses the grph. First we enqueue the initil DFA stte. While the queue is not empty we deque stte i nd enqueue ny diret hild sttes j tht hve not yet een disovered (enqueued efore). We enqueue hild sttes in lphetil order of symols l on trnsitions i l j the sme s in the DFS-sed pproh. We ll DFA BFS-enumerted if its sttes re enumerted in dequeuing (equls to enqueuing) order. An exmple of BFS-enumerted DFA with six sttes is shown in Fig. 6 (BFS tree trnsitions tht were used to enqueue sttes re mrked old); BFS enqueues re shown in Fig. 6. The DFA shown in Fig. 1 is not BFS-enumerted BFS first dequeues stte rther thn stte (we onsider 1 efore 1 ).,, 4,, 1 6, () BFS-enumerted DFA with olded BFStree edges () BFS queue. Cells orrespond to DFA sttes, trnsitions orrespond to enqueues Figure 6: An exmple of BFS-enumerted DFA nd its BFS queue All vriles whih were used for the DFS enumertion re lso used for the BFS enumertion, ut we hve to onsider the onstrints whih must e hnged. In the BFS-enumertion p j,i vrile definition is hnged: stte j should e enqueued while proessing the stte with the miniml numer i mong the sttes tht hve trnsition to j: (p j,i t i,j t i 1,j... t 1,j ). 1 i<j C In the BFS-enumertion sttes prents ordering onstrints re lso hnged: stte j must e enqueued efore the next stte j +1, thus the next stte s prent k nnot e less thn the urrent stte s prent i (see Fig. 7): (p j,i p j+1,k ). 1 k<i<j<c 8

9 k i j j+1 Figure 7: Prt of the queue illustrting the prent ordering predites. Trnsitions show prent reltions. The dotted trnsition is not llowed due to BFS-enumertion l i j j+1 l Figure 8: Illustrtion of lphetil ordering predites. If i is the prent of j nd j+1, l n (l k ) is the lphetilly miniml symol on trnsitions etween i nd j (i nd j +1) then l k nnot e lphetilly smller thn l n It is enough to onsider only two onseutive sttes in the onstrints where two sttes with the sme prent were onsidered (see Fig. 8 for the seond type of the onstrints): (p j,i p j+1,i y,i,j ), 1 i<j<c 1 i<j<c 1 m<n L (p j,i p j+1,i m ln,i,j m lm,i,j+1). Predites (for the se of three or more symols) trnslted into O(C +C L ) CNF luses re listed in Tle. Our implementtion of the proposed predites nd ll lgorithms n e found on the githu repository of our lortory The find-ll prolem In this setion we py our ttention to the prolem of finding ll non-isomorphi DFA with the miniml numer of sttes whih re onsistent with given set of strings. We propose the wy to modify the SATsed method in order to pply it to this prolem. We onsider two wys of using SAT-solvers: restrting non-inrementl solver fter finding eh utomton nd using n inrementl solver: if suh solver finds solution, it retins its stte nd is redy to ept new luses. The most ommon interfe nd tehnique for inrementl SAT-solving ws proposed in []. We lso onsider the heuristi Bktrking method s seline for ompring it with SAT-sed ones SAT-sed methods The min ide of SAT-sed methods is to n stisfying interprettions whih hve lredy een found. It is ovious tht if the proposed symmetry reking predites re not used then this pproh finds lot of isomorphi utomt (extly n! for eh equivlene lss where n is the DFA size). Sine mx-lique predites fix k olors only (where k is the lique size), the proposed lgorithm finds (n k)! isomorphi utomtons whih is lso d. The BFS-sed nd DFS-sed symmetry reking predites llow to n isomorphi DFA from one equlity lss y nning ordingly enumerted representtive. It is esy to implement yddingndditionlluseinto theboolenformul. Sine weknowthty l,i,j vrilesdefine the trget DFA entirely, it is enough to forid only vlues of these vriles from found interprettion: y 1 y... y n Σ, 1 9

10 Both DFS BFS Tle : DFS-sed nd BFS-sed symmetry reking luses Cluses CNF representtion Rnge t i,j (y l1,i,j... y ll,i,j) t i,j y l1,i,j... y ll,i,j 1 i < j C y i,j,l t i,j y l,i,j t i,j 1 i < j C; l Σ p j,i t i,j p j,i t i,j 1 i < j C p j,1 p j,... p j,j 1 p j,1 p j,... p j,j 1 j C m l,i,j y l,i,j m l,i,j y l,i,j 1 i < j C; l Σ m ln,i,j y lk,i,j m ln,i,j y lk,i,j 1 i < j C; 1 k < n L (y ln,i,j y ln 1,i,j... y ln,i,j y ln 1,i,j... 1 i < j C; 1 n L y l1,i,j) m ln,i,j y l1,i,j m ln,i,j p j,i t k,j p j,i t k,j 1 i < k < j C (t i,j t i+1,j... t j 1,j ) p j,i t i,j t i+1,j... t j 1,j p j,i 1 i < j C p j,i t k,q p j,i t k,q 1 i < k < j < q C (p j,i p k,i m ln,i,j) m lm,i,k p j,i p k,i m ln,i,j m lm,i,k 1 i < j < k C; 1 m < n L p j,i t k,j p j,i t k,j 1 k < i < j C (t i,j t i 1,j... t 1,j ) p j,i t i,j t i 1,j... t 1,j p j,i 1 i < j C p j,i p j+1,k p j,i p j+1,k 1 k < i < j < C (p j,i p j+1,i m ln,i,j) m lm,i,j+1 p j,i p j+1,i m ln,i,j m lm,i,j+1 1 i < j < C; 1 m < n L where y k is some true y l,i,j from the found interprettion for 1 < k < n Σ. There re two different wys of using SAT-solvers s it ws stted ove. First, we n restrt non-inrementl SAT-solver with the new Boolen formul with the dditionl luse fter finding eh utomton. The seond pproh is sed on n inrementl SAT-solver: fter eh found utomton we dd the dditionl luse to the solver nd ontinue its exeution. It is neessry to mention the se when some trnsitions of the found DFA re not overed y the APTA. It mens tht there re some free trnsitions whih re not used during proessing ny given word nd eh suh trnsition n end in ny stte, sine this does not influene the onsisteny of the DFA with given set of strings. But in the se of the find-ll prolem we do not wish to find ll these utomtons differed only y suh trnsitions. Thus we propose the wy to fore ll free trnsitions to finish in the sme stte s they strt. In other words we fore them to e loop. To hieve tht we dd uxiliry used vriles: u l,i is true if nd only if there is l-leled APTA edge from the i-olored stte: l Σ1 i C u l,i x 1,i... x Vl,i, where V l is the set of ll the APTA sttes whih hve n outoming edge lelled with l. To fore unused trnsitions to e loop we dd the following onstrints: u l,i y l,i,i. l Σ1 i C These dditionl onstrints re trnslted into O(C V ) luses. See Fig. 9 for the exmple of n APTA for S + = {,,,}nd S = {} (it is the sme exmple s on the Fig. 1 nd Fig. ut without the string in the S ) nd its onsistent DFA with unused trnsition. If we dd the proposed onstrints then this trnsition will e fored to e loop s shown y dshed line t the Fig Bktrking method The solution sed on ktrking does not use ny externl tools like SAT-solvers. This lgorithm works s follows. Initilly there is n empty DFA with n sttes. Also there is frontier the set of edges from the APTA whih re not represented yet in the DFA. Initilly the frontier ontins ll the outoming edges of the APTA root. The reursive funtion Bktrking mintins the frontier in the proper stte. If the frontier is not empty, then the funtion tries to ugment the DFA with one of its edges. Eh found DFA is heked to e onsistent with the APTA nd if the DFA omplies with it then n updted frontier 10

11 () An exmple of n APTA for S + = {,,,} nd S = {} () The DFA is uilt y the APTA from Fig. 9 with unused -leled trnsition from the stte Figure 9: An exmple of n APTA nd its onsistent DFA is found. If the frontier is empty then the DFA is heked for ompleteness ( DFA is omplete if there re trnsitions from eh stte leled with ll lphet symols). If it is not omplete nd there re nodes whih hve the numer of outoming edges less then the lphet size then we dd missing edges s loops with funtion MkeComplete. Algorithm illustrtes the solution. The funtion FindNewFrontier returns the new frontier for the ugmented DFA or null if the DFA is inonsistent with the APTA. Dt: ugmented prefix tree eptor APTA, urrent DFA (initilly empty), frontier (initilly ontins ll APTA root outoming edges) DFAset new Set<DFA> edge ny edge from frontier foreh destintion 1.. S do soure the stte of DFA from whih edge should e dded DFA DFA trnsition(soure, destintion, edge.lel) frontier FindNewFrontier(APTA, DFA, frontier) if frontier null then if frontier = then DFAset.dd(MkeComplete(DFA )) else DFAset.dd(Bktrking(APTA, DFA, frontier )) end end end return DFAset Algorithm : Bktrking solution 7. Experiments All experiments were performed using mhine with n AMD Opteron GHz proessor running Uuntu All lgorithms were implemented in Jv, the lingeling SAT-solver ws used. Our own lgorithm ws used for generting prolem instnes for ll the experiments sed on rndomly generted dtsets. Thislgorithmuildssetofstringswiththefollowingprmeters: sizen ofdfatoegenerted, lphet size A, the numer S of strings to e generted, noise level K (perentge of ttriution lels of generted strings whih hve to e rndomly flipped). For ext DFA identifition we used rndomly generted instnes. We used the following prmeters: N [10;0],A =,S = 50N. WeompredtheSAT-sedpprohwiththreetypesofsymmetryreking predites: the mx-lique lgorithm from [1] nd the proposed DFS-sed nd BFS-sed methods. Eh 11

12 experiment ws repeted 100 times. The time limit ws set to 600 seonds. Vlues in itlis men tht not ll 100 instnes were solved within the time limit. If less thn 50 instnes were solved then TL is shown insted of vlue. The results re listed in Tle. It n e seen from the tle tht oth DFS-sed nd BFS-sed strtegies lerly outperform the mx-lique pproh whih is the urrent stte-of-the-rt. BFS-sed strtegy in its turn outperforms DFS-sed one when trget utomton size is lrger thn 14. Tle : Men exeution times in seonds of solving ext DFA identifition N DFS BFS mx-lique TL TL TL TL TL TL 0 TL 90.1 TL For noisy DFA identifition we lso used rndomly generted instnes. First we onsidered the se when the trget DFA exists nd the Boolen formul is stisfile. We used the following prmeters: N [5;10], A =, S {10N,5N,50N}. We ompred three methods: the SAT-sed pproh without ny symmetry reking predites, our solution using BFS-sed symmetry reking predites, nd the urrent stte-of-the-rt EA from [1]. Eh experiment ws repeted 100 times. The time limit ws set to 1800 seonds. The initil experiments showed tht the EA outperforms our method lerly when K > 4%. Therefore we set this prmeter to 1% 4%. We left only instnes whih were solved within the time limit. These results indite tht the BFS-sed strtegy finds the solution slightly fster thn the urrent stte-of-the-rt EA only when N is smll (< 7), noise level is smll (1% 4%) nd the numer of strings is lso smll (< 50N). But BFS-sed strtegy finds the solution extremely fster thn SAT pproh without the symmetry reking strtegy. The third experiment onsidered the se when the trget DFA does not exist nd the Boolen formul is unstisfile. Rndom dtset ws lso used here. We tried to find the trget DFA using the following prmeters: N [5;7], A =, S = 50N, K [1%;%]. The input set of strings ws generted from n (N + 1)-sized DFA. It should e noted tht the EA from [1] nnot determine tht the utomton onsistent with given set of strings does not exist. On the other hnd, ll SAT-sed methods re ple of tht. Therefore we ompred our implementtion of ompt SAT enoding without using symmetry reking predites (WO) nd the sme with the BFS-sed predites. Eh experiment ws repeted 100 times nd the time limit ws set to 1800 seonds gin. The results re listed in Tle 4. It n e seen from the tle tht the BFS-sed strtegy redues signifintly the men time of determintion tht n utomton does not exist. Tle 4: Men exeution times in seonds of solving noisy DFA identifition when the trget DFA does not exist N K BFS WO TL TL TL 7 TL TL 1

13 Tle 5: Men exeution times in seonds of SAT-sed restrt method, SAT-sed inrementl method nd ktrking method N S = 5 N S = 10 N S = 5 N >1 REST INC BTR >1 REST INC BTR >1 REST INC BTR TL TL TL TL TL TL TL The lst experiment onerned the find-ll prolem. A rndom dtset ws lso used here. We used the following prmeters: N [5;11], A =, S {5N,10N,5N}. We ompred the SAT-sed method with the restrting strtegy (REST), the SAT-sed method with the inrementl strtegy (INC) nd the ktrking method (BTR). Eh experiment ws repeted 100 times s well. The time limit ws set to 600 seonds. The first olumn in eh sutle ontins the numer of instnes whih hs more thn one DFA in the solution (> 1). Vlues in itlis men tht not ll 100 instnes were solved within the time limit. If less thn 50 instnes were solved then TL is shown insted of vlue. The results re listed in Tle 5. It n e seen from the tle SAT-sed methods work signifintly fster then ktrking one. As we see inrementl strtegy in its turn lerly outperforms restrt strtegy. It n e explined s inrementl SAT-solver sves its stte ut non-inrementl solver does the sme tions eh exeution. 8. Conlusions nd Future Work We hve proposed symmetry reking predites whih n e dded to the Boolen formul representing vrious DFA identifition prolems. By dding the predites we n redue the prolem serh spe through enforing DFA sttes to e enumerted in the depth-first serh or the redth-first serh order. In the se of noiseless DFA identifition we hve ompred trnsltion-to-sat method from [1] to the sme one with proposed symmetry reking predites insted of originl mx-lique predites. The proposed pproh lerly improved trnsltion-to-sat tehnique whih ws shown with the experiments on the rndomly generted input dt. The BFS-sed pproh hve shown etter result thn the DFS-sed one if the trget DFA size is lrge enough. We lso hve drwn our ttention to the se of noisy DFA identifition. We hve proposed modifition of the noiseless pproh for the noisy se. To hieve ompt enoding for tht se we hve used the order enoding method. We hve shown tht the previously proposed mx-lique tehnique for symmetry reking is not pplile in the noisy se while our BFS-sed pproh is. We hve shown tht the BFS-sed strtegy n e pplied in the noisy se when n utomton whih is onsistent with given set of strings does not exists. The urrent stte-of-the-rt EA from [1] nnot determine tht. In experimentl results, we hve shown tht our pproh with BFS-sed symmetry reking predites lerly outperforms the lgorithm without ny predites. We hve proposed solution for the find-ll DFA prolem. The proposed pproh n solve the prolem tht the previously developed methods nnot e pplied for effiiently. We hd performed the experiments whih hve shown tht our pproh with the inrementl SAT-solver lerly outperfoms the ktrking lgorithm. We pln to trnslte the prolem of noisy DFA identifition to Mx-SAT in order to limit the numer of orretions without using n uxiliry rry of integer vriles. We lso pln to experiment with lterntive integer enoding methods. 1

14 Aknowledgements The uthors would like to thnk Igor Buzhinsky, Dniil Chivilikhin nd Mxim Buzdlov for useful omments. This work ws finnilly supported y the Government of Russin Federtion, Grnt 074-U01, nd lso prtilly supported y RFBR, reserh projet No mol. Referenes [1] J. Hoproft, R. Motwni, J. Ullmn, Introdution to Automt Theory, Lnguges, nd Computtion, Addison-Wesley, 006. [] C. De L Higuer, A iliogrphil study of grmmtil inferene, Pttern reognition 8 (9) (005) [] E. M. Gold, Complexity of utomton identifition from given dt, Informtion nd Control 7 () (1978) 0 0. [4] L. Pitt, M. K. Wrmuth, The Minimum Consistent DFA Prolem Cnnot e Approximted within Any Polynomil, Journl of the ACM 40 (1) (199) [5] K. J. Lng, B. A. Perlmutter, R. A. Prie, Results of the Adingo One DFA Lerning Competition nd New Evidene-driven Stte Merging Algorithm, in: Grmmtil Inferene, Springer, 1998, pp [6] K. J. Lng, Fster Algorithms for Finding Miniml Consistent DFAs, Teh. rep. (1999). [7] M. Buglho, A. L. Oliveir, Inferene of regulr lnguges using stte merging lgorithms with serh, Pttern Reognition 8 (9) (005) [8] P. Dupont, Regulr Grmmtil Inferene from Positive nd Negtive Smples y Geneti Serh: the GIG Method, in: Grmmtil Inferene nd Applitions, Springer, 1994, pp [9] S. Luke, S. Hmhshi, H. Kitno, geneti progrmming, in: Proeedings of the Geneti nd Evolutionry Computtion Conferene, Vol., 1999, pp [10] S. M. Lus, T. J. Reynolds, Lerning DFA: Evolution Versus Evidene Driven Stte Merging, in: Evolutionry Computtion, 00. CEC 0. The 00 Congress on, Vol. 1, IEEE, 00, pp [11] S. Lus, GECCO 004 Noisy DFA Results, in: GECCO Pro., 004. URL [1] S. M. Lus, T. J. Reynolds, Lerning deterministi finite utomt with smrt stte leling evolutionry lgorithm, Pttern Anlysis nd Mhine Intelligene, IEEE Trnstions on 7 (7) (005) [1] M. J. Heule, S. Verwer, Ext DFA Identifition Using SAT Solvers, in: Grmmtil Inferene: Theoretil Results nd Applitions, Springer, 010, pp [14] N. Wlkinshw, B. Lmeu, C. Dms, K. Bogdnov, P. Dupont, STAMINA: Competition to Enourge the Development nd Assessment of Softwre Model Inferene Tehniques, Empiril Softwre Engineering 18 (4) (01) [15] A. Biere, M. Heule, H. vn Mren, Hndook of Stisfiility, Vol. 185, IOS Press, 009. [16] N. Aml, X. Du, A. Kuehlmnn, R. P. Kurshn, K. L. MMilln, An Anlysis of SAT-sed Model Cheking Tehniques in n Industril Environment, in: Corret Hrdwre Design nd Verifition Methods, Springer, 005, pp [17] R. Lohfert, J. Lu, D. Zho, Solving SQL Constrints y Inrementl Trnsltion to SAT, in: N. Nguyen, L. Borzemski, A. Grzeh, M. Ali (Eds.), New Frontiers in Applied Artifiil Intelligene, Vol. 507 of LNCS, 008, pp [18] J. P. Gleotti, N. Rosner, C. G. Lopez Pomo, M. F. Fris, TACO: Effiient SAT-Bsed Bounded Verifition Using Symmetry Breking nd Tight Bounds, IEEE Trnstions on Softwre Engineering 9 (9) (01) [19] V. Ulyntsev, F. Tsrev, Extended Finite-stte Mhine Indution Using SAT-solver, in: Pro. of ICMLA 011, Vol., IEEE, 011, pp [0] B. Lmeu, C. Dms, P. Dupont, Stte-merging DFA Indution Algorithms with Mndtory Merge Constrints, in: Grmmtil Inferene: Algorithms nd Applitions, Springer, 008, pp [1] L. D. Chmers, Prtil Hndook of Geneti Algorithms: Complex Coding Systems, Vol., CRC press, 010. [] P. Brhon, S. Hölldoler, V. Nguyen, Effiient SAT-Enoding of Liner CSP Constrints, in: 1rd Interntionl Symposium on Artifiil Intelligene nd Mthemtis-ISAIM, Fort Luderdle, Florid, USA, 014. [] N. Eén, N. Sörensson, An extensile SAT-solver, in: Theory nd pplitions of stisfiility testing, Springer, 004, pp