Inference of regular expressions/grammars for given data entities

Transcription

1 Inference of regulr expressions/grmmrs for given dt entities Lis Busser University of Kiserslutern, Embedded Systems Group l busser13@cs.uni-kl.de Abstrct We consider three different lgorithmic ides to build regulr lnguge out of set of positive exmples of the lnguge nd their dvntges nd disdvntges. We contrst the tree lgorithms SL in f ere, DeLeTe2 nd n lgorithm for the regulr lnguge R1 + 1 Introduction In this pper, we discuss some lgorithms for lerning regulr expressions for given dt entities. Over the lst forty yers there hve been plenty of reserch on this topic nd mny different strtegies nd lgorithms where published. All these lgorithms join the trget to be ble, fter fixed number of given smples of the lnguge, to built regulr expression or utomton, which describes the questioned lnguge s detiled s possible. All the lgorithms, which solves tht kind of problem re more heuristic thn proper lgorithm becuse there does not exist ny specifiction, which describes the clss of lnguges in wy, tht could be lerned by lerner. Lerning regulr lnguge often mens to infer with regulr expressions becuse for humns, regulr expression re best comprehensible. But even when the output is regulr expression the lgorithms often del with utomtons to clculte the regulr expression. Only few lgorithms del directly with the regulr expression, like the one presented in chpter 3. You could lso find lgorithms in literture which uses form of nture inspires progrmming especilly something like n evolutionry lgorithm but this isn t considered in this pper. So bsiclly there re two different wys to write such n lgorithm. You could either use form of nture inspired progrmming or you cn solve the problem on the bse of utomton theory. In this pper we tke look t the second option nd which clsses of lnguge cn be lerned of the different lgorithms. We focus on three different ides to pproch n lgorithm, where everyone is using different method to solve the problem. The first solution, tht is presented, is from Efim Kinber[6] who bsed his ide on Angluin s model. The lerning process in this model is n interction between lerner nd techer. Actul the lerner is sking questions of certin type nd the techer gives correct nswers. Out of this questions nd fter finite number of them the lerner must be ble to give regulr expression, which is in certin clss. This clss must be known by the lerner nd is set by the techer. D. Angluin herself presented polynomil-time lgorithm which uses membership n equivlence query for building (miniml) deterministic finite utomt. In the lgorithm presented her, the membership query is chnged bit. If the queried string is not in the tsked lnguge the techer does not only response no, wht he does in

2 2 membership query, he returns n exmple, which is in the trget lnguge, not mentioned before nd hs the smllest edit distnce from the queried string. Out of these questions the regulr expression is built directly nd the utomton is just implicit in the bckground. The reson why regulr expressions re better for describing lnguge is, tht mostly they re better delble for humns thn describing lnguge by n utomton. This lgorithm will be explined in section 3. The next lgorithmic strtegy is from Henning Fernu[2]. It is bsed on the usge of deterministic finite utomtons to find the regulr expression for the given lnguge. Most lgorithms for infering regulr expressions re bsed on utomton so this strtegy is most common. The disdvntge of using deterministic finite utomtons is tht building regulr expressions out of them will often result in clumsy nd lengthy expression. This expression is not very comprehensible for humns. This blow up of the expression hs two resons. Even when the utomton hs no loops the regulr expression will experience blow up of qudrtic size due to repeted sub-structures. If the utomton hs loops the blow-up is ctully bout c n, to build the regulr expression out of n n-stte deterministic finite utomton. A humn who wnts to check such n expression for correctness might be esily detested of the result. But opertions on DFA s re very fst which might outblnce the blow up in size for specil pplictions. The third solution is to use non-deterministic finite utomtons. The lgorithm, presented in section 5 DeLeTe2 from Frncios Denis, Aurelien Lemy nd Alin Terlutte[4] belongs to this clss. It s bsed on Residul Finite Stte Automton(RFSA) which re specil form of non-deterministic finite utomtons. Every regulr lnguge cn be represented by n RFSA. Cnonicl RFSA s re much smller thn their relted miniml deterministic finite utomton nd cn be used for rndomly generted lnguges. If the regulr lnguge is built by rndom generted deterministic finite utomton the dvntge in size is not so big, but if the lnguge is built by rndom generted non-deterministic finite utomton or regulr expression the cnonicl RFSA is much smller thn the miniml deterministic finite utomton. The dvntge of this lgorithm is tht it cn del with ny regulr expression nd hs not such strict limittions on the lnguge tht cn be build. The disdvntge is, tht opertions on RFSA s re more expensive thn on DFA s. 2 Nottion nd Preliminries In this Section, we wnt to fix some nottions nd preliminries tht re used in every lgorithm of this pper. Some specil definitions re done in the explicit cption of the lgorithms. For more detiled informtions nd the missing proofs plese hve look into [1],[2],[3],[4]nd[6]. Let Σ be finite lphbet nd Σ set of words on Σ. A lnguge cn be ny subset of Σ. The length of word u is u. uv is the connection of the two words u nd v. The Levenshtein edit distnce d(u,v) is defined s minimum number of opertions, which re needed to convert the one string in the other. An opertion cn be n insertion, deletion or substitution opertion. 3 Lerning R1 + Vi Membership nd Correction Queries With the lgorithm presented now, you cn lern so clled indexble clsses of recursive lnguges over Σ. An indexble recursive clss of lnguges L is clss where you cn find n effective numbering L i,i =,1,2,... of ll the lnguges L L so tht for ny word w Σ nd n index i L i exists function which outputs 1 if w L i nd if not.

3 3 The lerning process is bsed on n lgorithmic device which is the lerner nd techer, who hs n orcle function tht truthfully nswers the questions of the lerner. After sking the techer question the lerner cn either generte new question to the orcle, or he cn chnge the index of lnguge in L. The lerner lerned the lnguge if L L nd if the index-shift ws right. The lerner lerned the complete lnguge L if he hs lerned every lnguge L L. membership nd correction queries : If the lerner is sking membership queries he cn only become the nswers yes or no. If he is sking correction query nd the nswer is negtive, he is not just receiving no, the techer is lso correcting the queried string w nd responses v such tht: v = w d(v,w) =min{d(v,w) u L,u / A} where A is the set of words tht hve been mentioned before in the querying process. If string v doesn t exists, the techer responses the shortest string v such tht d(v, w) = min{d(v, w) u L, u / A}. He never mentions string twice nd he lwys returns the smllest string in lexicogrphic order. If even no such string exists the techer returns ε / Σ. Le ft ligned regulr expressions : A regulr expression over n lphbet Σ: u 1 (v 1 ) +...u n (v n ) +, where (v i ) + indictes loop. u i,v i,i = 1,2,..n re strings over Σ. The regulr expression is clled left ligned when ll loops re s left s possible. The regulr expression: belongs to the unique left-ligned expression: () + bccd(cd) + ddbb + b + (bb) + () + bc(cd) + cdddb + bb(bb) +. The lest preliminries we hve to mke is tht if we hve sub-expression-loop ((u) i ) + there must not be string like ((u) j) + where i, j N in the direct neighbourhood of the first string. All regulr expressions following these rules re in the clss R1 +. lerning R1 + The following lerning lgorithm is lerning ny lnguge in R1 + with one correction query nd membership queries. The work of the lgorithm will be explined by n exmple. Let s tke the R1 + expression () + b + b(b) + bb + b + s our trget lnguge L. The first step of the lgorithm is to sk the one specil correction query. The techer will nswer the shortest possible nswer: bbbbbb. Now the rel tsk strts. As first step we re serching for one-letter-loops. So the lerner is seprting the input string whenever new token is red: r 1 =,bb,,b,,bb,,b. You cn seprte the word like this, becuse of the left-lignment of

4 4 the lnguge R1 + nd becuse it is not llowed to hve loops of the sme letter in directly neighbourhood. Out of r the lerner is sking for every built string, if there could be loop. For testing if in is one-letter-loop the lerner is sking bbbbbb nd the nswer is no, becuse includes twoletter-loop nd single, so you lwys hve n odd number of. If would involve one-letter-loop you would lso be ble to build n even number of. The lerner repets his question for every string out of r. So the next question will be: bbbbbbb nd the nswer of the orcle is: yes. Now the lerner knows tht there must be one-letter-loop t the bb prt nd becuse of the left-lignment the lerner knows now tht our lnguge looks like b + bbbbb. After testing every possible one-letter-loop the lerner knows tht the expression looks like p 1 = b + bbbb + b +. In the next step, the lgorithm will serch for every two-letter-loops. The proceed is the sme like serching for the one-letter-loops. At first you seprte the word into every possible two-letter-loop string r 2 =,b,b,b,b,bb. The lgorithm will test every possibility with membership query. For testing the prt, whether there is two-letter-loop the lerner sks: bbbbbb nd the nswer is yes. Tht cn only men tht the loop is in the two leftmost s. So now the lerner knows tht the expression looks like: () + b + bbbb + b +. After testing every two-letter-loop the lerner knows, tht the expression looks like p 2 = () + b + b(b) + bb + b + nd of course this is the lnguge we re serching for but our lerner doesn t know this. So he will try to find the next loop, three-letter one. For this he proceed like before seprting the word into every possible three-letter-loop. This is only possible t r 3 = bb becuse this is the only three letters together with none of them included into loop before. The lerner will sk the techer if bbbbbbbb is correct word nd the nswer is no. But now the techer hs no more correct word tht hsn t been mentioned in the querying process before so he cn t correct the question nd will return ε which is not in Σ. This results in the determintion of our lgorithmic lerner becuse he knows, tht he hs finished nd returns the lerned expression: () + b + b(b) + bb + b + This is only the ide of the lgorithm. The pseudo-code implementtion of Efim Kimber [6] fits in the time complexity clss of O(n 5 ). The totl numbers of queries is in O(n 3 ). For every query the lerner sks, the lgorithm tkes time of O(n 2 ). It might be interesting if some modifictions of the clss R1 + re lso lernble in polynomil-time. The clss R1 might be such modifiction, where loops (v) re rther used thn the (v) + loops in R1 +. Probbly this clss is too hrd to lern in polynomil time but the modifiction R2 which only contins loops like (v) is possible under the constrint tht v doesn t contin ny repetitions e.g. v = b would not be llowed. So only one-letter-loops re llowed. A regulr expression of this type could look like b b b b. You see know tht the lgorithm doesn t only find the regulr expression for lnguges which re in R1 +. If you chnge the preliminries just little bit you cn lso use the lgorithm for modifictions of R1.

5 5 4 Lerning SL vi SL in f ere In this section we discuss how to lern the lnguge S L. The bsic concept is the sme like lerning R1 + but this time we use deterministic finite utomtons to lern the lnguge. These so clled exct lnguge S L is lso indexble like R1 +. But it lso hs some different specifictions: 1. If the lerners new hypothesis only depends on the previous hypothesis nd the lst input word, the lerner is clled itertive. 2. A conservtive lerner sserts his hypothesis s long s he hsn t seen dt dissenting his hypothesis. 3. A consistent lerner reflects ll the dt he hs immeditely in correct hypotheses. 4. A strong monotonic lerner produces lwys strem of hypothesis describing n ugmenting chin of lnguges, i.e., the new hypothesis lnguge is lwys superset of the previous one. [pper] 5. The lerner is clled rerrngement-independent or order-independent, if he concludes to the sme hypothesis if he hs seen either the sequence E()1,E(2),...E(n) or E(π n (1)),...E(π n ()) where π n could be ny permuttion of {1,...n}. 6. The lerner is set-driven, if his hypothesis h doesn t depend on if he hs either seen E(1),...,E(n) or F(1),...,F(n), where {E(1),...,E(n)} = {F(1),...,F(n)}. Ech of this rules is restriction of the lernble lnguge clss themselves. This lgorithm uses the bsic techniques of left-lignment nd blockwise grouping. Blockwisegrouping is very fmilir to wht we hve done in 3. In 3. we hve split our word whenever we hve red different chrcter. Let s tke: bc, b, b, bbcc s n exmple. Then the blockwise grouping would give us: [][b][c][],[][b],[][b][],[][bb][c][][c]. The squre brckets indictes the block letters. Any [x n ] n 1 is clled block letter whenever x Σ. x is clled the bsic letter of [x n ]. In the next step we lign the blocks: [] [b] [c] [] [] [b] [] [b] [] [] [bb] [c] [] [c] The exmple bove shows left-ligned lignment. This is wht we, s humns, would utomticlly do. If we should describe how the exmple regulr expression is build we would do this s following: the

6 6 expression strts with one or two s followed by one or two b s possibly followed by nothing or n or c... nd so one. You could lso choose other lignments for this expression. Mybe you would prefer to describe the exmple s follows: The expression mybe strts with one or two s followed mybe by one or two b s or n. Then perhps cme c or b followed mybe by on or two s. And finishes propbly by b or c. [] [b] [c] [] [] [b] [] [b] [] [] [bb] [c] [] [c] Finding the best lignment is NP-hrd. Due to this nd becuse of the psychologicl point of the left-lignment we restrict our ttention to leftmost lignments. So for our lnguge we would find the following regulr expression: ( )(b bb)(λ c( )(λ c) ) This regulr expression is still finite lnguge, but very specil one becuse we sy only one or two repetitions nd if we would hve up to five repetitions we would hve to write ( ) This is very clumsy so we generlize to one or more repetitions with +. Moreover is shorter explntion (regulr expression) of the given input better to rech the gol of Minimum Description-Length. + b + (λ c + (λ c) ) We use the x + nottion nd not the x nottion to not destroy the blockwise redings we provide t the beginning. Now we see the lerning lgorithm for SL in f er. The lgorithm cn be split up into two min prts. Algorithm-prt 1 crete tree(i) At first we hve to mke the definition how to split up the tree. For ech symbol Σ nd I Σ pplies: I() = w I w strts with. I() mens tht there exists t lest one word w witch strts with t lest one. I = v bσ b n ( n v I()) I elimintes ll leding s of ll words w in the set of given words. The I + is stnding for rooted lbelled directed tree. We first compute I(),I for ech Σ: I + =I = I + =I +(b) = {bc,b,b,bbcc} {bc,b,b,bbcc} I+ b =I+ b () = I+ b (c) = {c,,cc} I+ b = {} I bc + =I bc + () = {, c} I+ bc =I+ bc (c) = {c} I+ bcc = {}

7 7 q I + = {bc,b,b,bbcc} q b I + = {bc,b,b,bbcc} b q c I b + = {c,,cc} c I b q d + = {} q e I+ bc = {,c} q f I bc + = {c} c q g I bcc + = {} Figure 1: A strt tree Expressions like or bb doesn t leve ny trce in I +. Both terms of bc re eliminted in one step when I+ or I+ bc is clled. Becuse I+ b = {} we continue to lign I+ bc = {,c}. So the tree of our exmple is lbeled s follwos: I +,I+,I + b,i+ b,i+ bc,i+ bc,i+ bcc. In the tree leves re lwys lbelled with the empty set. This is shown in Fig. 1. The lterntive node lbeling convention is shown in Fig. 2. In fig. 2 the tree from fig. 1 is trnsformed into generlized NFA. The root becme n initil stte. If node is lbeld by the stte I x (), every word, which beginns t this stte of the utomton with n is written behind I x (). Between the prent node nd the child node is written, those letters which cn be red next. So we hve t first written on the edge {,} nd then the child node is written I + () nd () is eliminted when the utomton leves the node of I + (). This leds to some generlistion becuse the word bb will now be ccepted lthough it is not in our set of lerned words (bb / I + ). [...]According to the following principle, which is common chrcteristics of lignment-bsed lerning lgorithms: ligned pieces of words re considered to be interchngeble.[...] (Henning Fernu, 29). So we hve to do some generlistion nd this form we did so fr is not very ggressive, since it cretes only finite lnguges given finite number of exmples. For building infinite lnguges out of finite exmples we need to introduce loops. Now we turn the generlize NFA into generlize deterministic finite utomton(dfa). Algorithm-prt 2 generlizesimple

8 8 r {,} I + () = {bc,b,b,bbcc} {b,bb} I +(b) = {bc,b,b,bbcc} c I b + () = {} I b + (c) = {c,cc} {,} I bc + () = {,c} c I bc + (c) = {c} Figure 2: The resulting generlized NFA

9 9 b b c c Figure 3: The resulting DFA In our DFA every trnsition crries exctly one lbel. So if we tke look t fig. 2 the first trnsition is lbelled with {, }. This lbelling will not be possible nymore. Whenever trnsition α is lbelled with m>1 words, then we tke only the shortest word of the set s lbel of the trnsition α. In our exmple we would tke s lbelling for α. To mke it possible, tht the longer repetitions of, in our cse cn be reched by the utomton we use self-loop, lbelled with t the node, the trnsition α is showed in fig. 3. Also see fig. 3 for the resulting DFA. The lnguge SL includes ll lnguges tht re left-ligned nd simple looping. You hve lredy seen wht left-lignment mens. Simple looping mens, tht the expression is [...] finite union of pirwise left-lighned expressions α such tht either α = λ or the cn be written in the following norml f orm: where α = α k 1 1 αx 1 1 αk 2 2 αx αk k k αx k k. ll i re single symbols from the bsic lphbet Σ for ll 1 i < K, i i+1 ech k i is some positive integer, nd

10 1 x i equls either or * (if x i =, the prt i of the expression will not be written down, since it denotes the empty word). [...] Henning Fernu,29). So SL in f ere is getting finite set of exmples nd computes out of them n infinite lnguge. This lgorithm needs set of words before he strts his computtion becuse the lgorithm prt one is preprocessing of the prt two. Algorithms like this re clled exct lerning lgorithms. In distinction to the first lgorithm discussed in this pper, which gives us directly regulr expression s result, this one gives deterministic finite utomton. 5 Lerning rndom lnguges Vi DeLeTe2 The lgorithm DeLeTe2 is using residul finite utomtons (RFSA). To understnd the lgorithm nd this specil form of non-deterministic utomtons we need some more preliminries: Residul Lnguge: The residul lnguge of the lnguge L regrding to u u Σ is defined by u 1 L = v Σ uv L. For every stte q in DFA exists unique residul lnguge u 1 L such tht u 1 L = L q nd lso reverse.[3] Cnonicl RFSA nd prime residul lnguges: A miniml cnonicl RFSA cn represent every regulr lnguge L. The sttes of the miniml cnonicl RFSA corresponds to the prime residul lnguge. A lnguge u 1 is prime if u 1 cn not be build s union of residul lnguges it strictly contins. u 1 is prime if nd only if: {v 1 L v 1 L u 1 L} u 1 L If the lnguge is not prime, it is composed. We tke L = ε Σ + s n exmple for rndom generted regulr expression. The miniml DFA ccepting L is shown in fig.4. The residul lnguge of L A is: L q = Σ = L q1 =1 + 1 Σ = L q2 =Σ + = L q3 =Σ = ε () 1 L q1 L q 2 = prime sttes. L q3 = L q L q1 L q2 so it is not prime, it is composite. The residul lnguge is the prt behind the second eqution. For L q ε is the shortest word in lexicogrphic order, tht cn be tken to unmbiguous describe how to rech stte L q from the strt stte. It s the sme for the other sttes.

11 11 1 q 1 1 strt q,1,1 q 2 q 3 Figure 4: The miniml DFA (A) 1,1 strt ε,1 ε,,1 Figure 5: The cnonicl RFSA

12 12 Short Prefix SP(L) nd Kernl K(L) The Short Prefix nd the Kernl re defined s following: SP(L) = {u q q p} K(L) = {u q x q p,x Σ,δ(q,x) /} where q is the mximum prime stte of the utomton A nd x ny symbol in Σ. In the exmple the mximum prime stte is q 2. So SP(L) = {ε,,1} K(L) = {,1,,1,1,11} complete smple (for short) A complete smple S for inclusion reltions mens, tht this is the minimum smple which describes the inclusion reltions between the residul lnguges without loosing informtion. In this smple SP(L) must not be longer thn q becuse the prime sttes re in lexicogrphic order nd the biggest prime stte is the one, reched t lest. So if the length of the smple is possible to rech the lst prime stte, it is lso possible to rech ll other. K(L) is SP(L) x x Σndx ε. So smple S is complete for inclusion over regulr lnguge L if u SP(L) K(L),u Pre f (S + ) SP(L) L s + u SP(L), v SP(L) K(L),u 1 L v 1 L w such tht uw S + nd vw S. Pre f is ordered in length-lexicogrphic order. Where S + is the set of minimum positive smples nd S the minimum set of negtive smples. S + S = S. The size of complete smple is in O(n 5 ) which includes O(n 2 ) words whose length is t most O(n 3 ) nd n is the size of the minimum DFT. If w is for instnce word of u 1 Lv 1 L, so w n 2 nd u n thn uw thn S contins words smller thn n 3. For every regulr lnguge L there exists complete smple like tht. Let u,v S + nd vw S for smple S we define: u v if w uw S + vw S u v if u v nd v u for ny S, nd ny u,v Pre f (S + ) it obeys tht u 1 L = v 1 L u v nd u 1 L v 1 L u v. Refer to [Pper 3]. So in the exmple S + = {ε,1,,1,1} nd S = {,1,111}. The following tble is showing the reltion between SP(L) nd SP(L) K(L). If there is word w in the tble, where u is the lbel of the row nd v the lbel of the column, it mens tht uw S + nd vw S. An utomton A = Σ,Q,Q,F,δ, where Σ is the lphbet, the utomton uses, Q is the set of sttes, Q is the initil stte, F re the finl sttes nd δ is the trnsfer function. Now we present the lgorithm DeLeTe2. From smple S of the trget lnguge L, which is complete for inclusion reltions of L, DeLeTe2 builds the utomton A up. So the lgorithm is getting Pre f = {u,u 1,...u n } s set of prefixes of S + s input. Strting with n empty utomton Q = Q = F = δ = / nd u = ε the lgorithm checks the prefixes of positive exmples if they considers s new stte in A up.

13 13 Tble 1: reltion between SP(L) nd SP(L) K(L) ε ε ε ε ε We check first, if u is not rel new informtion. If there lredy exits n u Q such tht u u the informtion in u is not new one. u is equivlent to stte in Q nd cn lredy be reched by the utomton. In this cse we simply delete uσ from Pre f. If we hven t hd true t the first condition, u is new stte in Q so we dd u to Q (Q = Q {u}). Then the lgorithm checks, if u ε becuse then u is strt stte nd we would hve to dd it there: Q = Q {u}. u is F in detil u is n element of the finl sttes if u S +. If u S + u hs to be dded in F = F {u}. Also we hve to dd the corresponding trnsitions δ. There we check in to steps if there is trnsition from u to u which hs to dd or trnsition u to u, where u Q δ = δ {(u,x,u) u Q,u x Pre f,u u x} {(u,x,u ) u Q,ux Pre f,u ux} The lst two prts re working equivlent for trnsitions from u to u nd from u to u. It is checking for ech u Q including the just before dded u (becuse of self loops) if there is trnsition from one stte to the other. Exemplrily of the first prt, the next check is if there is u x Pre f where x Σ. This checks, if there exists n u tht is not finished lredy. An u where there re still shortest prefixes tht hve not been eliminted from pre f strting with u. u u x checks, tht there is no trnsition dded, tht is impossible becuse of L. If ll conditions re true, the trnsition u,x,u, in detil the trnsition from u to u vi trnsition condition x. We set u = the next element of Pre f. If u ws the lst word of Pre f or if the utomton A up is consistent with the given smple S the lgorithm ends, returning A up. In our exmple Pre f = {ε,,1,,1,1,11}. In the first step, stte ε is dded becuse t the beginning of the lgorithm before strting the loop, we set u = ε. ε / Q becuse Q is empty. So we dd ε to Q. ε ε so ε is lso Q. ε S + so dditionlly ε F. The trnsitions, tht hve to be dd cn be seen directly out of tble 1: ε ε1. Where every entry in tble 1 is used, where u = ε nd v kσ k Q. This limittion is becuse n entry ε 1 mens, tht there is trnsition from 1 to ε with the trnsition condition. We cn t dd n entry with k = 1 becuse 1 / Q, so we hve no stte Q lerned until know nd of cuse cn t dd trnsitions to stte we doesn t know so fr. Only the trnsition ε ε1 is ok, becuse we use there the implicit ε stnding before the 1. The result fter the first step is shown in fig. 6. Pre f = {ε,,1,,1,1,11} Q = {ε} Q = {ε} F = {ε} δ = {ε ε1}

14 14 1 strt ε Figure 6: FSA fter the first step,1 1 strt ε,1 Figure 7: FSA fter the second step Now the lgorithm tke look t becuse it is the next element of Pre f. ε so is dd to the sttes Q. ε so is no initil stte. / S + so is lso no finl stte. The trnsitions, tht hve to be dded re: ε,, 1,ε,ε 1. The result fter step 2 is shown in fig. 7. Pre f = {ε,,1,,1,1,11} Q = {ε,} Q = {ε} F = {ε} δ = {ε ε1, ε,, 1,ε,ε 1} In the third step, the lgorithm is tking u = 1 becuse 1 is the next element in pre f. 1 ε becuse 1 ε nd ε 1. So 1Σ is deleted form Pe f. Pre f = {ε,,,1} Q = {ε,} Q = {ε} F = {ε} δ = {ε ε1, ε,, 1,ε,ε 1} In the fourth step the lgorithm tkes the next element in Pre f =. ε nd, so is new stte. ε so is not new strt stte but S + so is finl stte. The trnsitions, tht hve to be dd re: ε, ε, 1,, 1. Pre f = {ε,,,1} Q = {ε,,}

15 15 1,1 strt ε,1 ε,,1 Figure 8: FSA fter fourth step Q = {ε} F = {ε,} δ = { ε,, 1,ε,ε 1, ε, ε, 1,, 1} Now, the lgorithm termintes, becuse the utomton is consistent with S. The output is sturted subutomton A up of L. This lgorithm is very flexible over the input lnguge. There re no strict limittions except, tht L + must be complete for inclusion reltions. If the given smple S + is complete, this lgorithm cn build cnonicl FSA. The miniml cnonicl FSA is lwys shorter thn n miniml DFA, so the resulting regulr expression will be shorter. [...]The time complexity of DeLeTe2 is bounded by O(n 4 ).[...](Topics in grmmticl inference, 216)[9] 6 Conclusions, Results, Discussion In the three sections bove we hve seen three different lgorithms to solve the problem of lerning infinite regulr lnguge just by finite set of smples. The three lgorithms re joint by the property, tht there is lerner (the lgorithm) nd techer or orcle, who knows which smples re correct n which not. This techer must know how the lnguge should look like, but the formultion itself, s regulr lnguge, is done by the lgorithm. The first lgorithm, we presented, ws lerning R1 + vi membership nd correction queries. This one gives directly regulr expression s solution. By contrst to the other two lgorithms, the lerning process is continuously interction from lerner nd techer. Becuse the lgorithm is lwys sking if the next possibility, he computes, is in the lnguge or not, nd if not he directly gets correction to true positive smple of the lnguge. The dvntge of this procedure is, tht the techer hs not to think bout complete smple, describing the lnguge. This is done by the lgorithm by sking the next correction query. The disdvntge is, tht the lnguge R1 + is very strict, nd you must be sure tht your trget lnguge is in R1 + or it wouldn t work. The lgorithm is in the complexity clss of O(n 5 ). The second lgorithm SL in f ere is lerning the lnguge vi DFT. Here the interction between the lerner nd the techer is just t the beginning, where the lerner gets set of smples. Out of this set he clcultes strt tree, which is prefix tree. From this strt tree he clcultes NFA which looks like the strt tree, becuse there ws just relbelling of nodes nd trnsitions. From this NFA, the lgorithm is clculting the DFA by introducing loops, but the structure of the tree is still intct. The dvntge of

16 16 using DFA s is, tht clcultions on DFA s re very fst, but the lnguge is lso limited s the lnguge in the first lgorithm. And the second negtive point is, tht building regulr expression out of DFA, is clumsy nd not good redble or comprehensible. The third lgorithm ws DeLeTe2. This lgorithm hs no restrictions to the lernble lnguges, it just needs n input set, tht is complete for inclusion reltions, becuse it hst no structure or limittion of the lnguge, where it cn bse on his suggestions. In the exmple we showed the input on DFA nd regulr expression. We hve done some pre-clcultions, to show, wht DeLeTe2 needs in ddition to its input smple. In this lgorithm, it is only described wht the lgorithm does, if it hs this pre-clcultions, but of cuse this is n implementtion decision. After the pre-clcultions re done, DeLeTe2 checks for every symbol, if it is new stte or not. If it is new stte, he clcultes, whether it is strt or finl stte nd the new trnsitions tht hve to be fit in the utomton. The resulting NFA is smller thn the DFA from the lgorithm bove, but clcultions on NFA s re not s fst s clcultions on DFT s. There hsn t been mny reserch bout DeLeTe2 t ll so there cn be lot of chnge in the next yers. The use of one of the lgorithms depends on the lnguge-clss the lerned lnguge is in. If the lnguge is for sure in one of the first two clsses, one of these two lgorithms might be better. Not only from performnce perspective but lso for pprecition. At lest we wnt to give smll outlook wht cn be done with this kind of lgorithms in embedded systems. In embedded systems there re lwys hrd limittions on resources like computtion power nd storge plce. These lgorithms help to mke expressions more compct, so tht they doesn t wst so much power or plce. References [1] Colin de l Higuer. Grmmticl Inference: Lerning Automt nd Grmmrs. 21. [2] Henning Fernu. Algorithms for lerning regulr expressions from positive dt. 29. [3] Alin Terlutte Frncois Denis, Aurelien Lemy. Residul finite stte utomt. [4] Alin Terlutte Frncois Denis, Aurelien Lemy. Lerning regulr lguges using rfss. 24. [5] José M. Sempere Jeffrey Heinz. Topics in Grmmticl Inference [6] Efim Kimber. On lerning regulr expressions nd ptterns vi membership nd correction queries. 28. [7] R.J. vn Glbbeek. The eptcs bibliogrphy style, 21.