a b [^ab] ^a [^ab] [^ab]

Transcription

1 Genertion of Pttern-Mtcing Algoritm y Extended Regulr Expreion Ikuo NAKATA 3 Summry. It i dicult to expre te denition of te comment of C lnguge in regulr expreion. However, te denition cn e expreed y imple regulr expreion y introducing pecil ymol, clled te ny-ymol, tt repreent ny ingle crcter, or y introducing kind of negtion ymol into regulr expreion. In generl, te prolem of tring pttern mtcing cn e expreed uc n extended regulr expreion, nd te correponding nite tte utomton generted from te expreion i equivlent to te Knut-Morri- Prtt pttern-mtcing lgoritm [4]. In prticulr, if we ue te ny-ymol, te pttern i not retricted to tring of crcter. It cn e ny regulr expreion. Our metod cn lo e pplied to te prolem of repeted pttern mtcing. Te Ao-Corick lgoritm [3] cn e derived mecniclly from n extended regulr expreion tt contin ny-ymol. 1 Introduction Te denition of comment of C lnguge cn e given tring compoed of tree tring in te following order: /*, tring tt doe not contin */ (my e n empty tring), */. Expreing te ove denition y regulr expreion i complicted ecue it i dicult to expre \ tring tt doe not contin */." Here, */ men tring of lengt two compoed of * followed y /. Altoug we cn ue te negtion ymol in lex, well-known cnner genertor, te expreion for te comment i till complex{for exmple, in [6], or /*/*([^*/]j[^*]/j*[^/])****/ (1) /*([^*]j***[^*/])****/ (2) in [5]. Here, (,),j,*,[,],^ re met ymol, * denote te repetition of te immedite predeceor zero or more time, nd [^*/] expree one crcter tt i neiter * nor /. Ti prolem cn e imply expreed if we cn deignte \ tring tt doe not contin */" directly. If te expreion of te ove tring i ^(*/) 3 Intitute of Informtion Science nd Electronic, Univerity of Tuku. 1 Copyrigt c 1996y Acdemic Pre, Inc. nd Iwnmi Soten, Pulier. All rigt of reproduction in ny form reerved. ISBN {12{3713{

2 2 Advnce in Softwre Science nd Tecnology 5, 1993 * ^* {^/} Fig. 1 Te utomton to recognize ^ (*/) ^* * / {^ } / Fig. 2 DFA for (^ (*/))**/ ten te denition of te comment i /*(^(*/))**/. Ti i te direct expreion of te denition. Here, we dene te mening of ^(*/) to e \te ed crcter of tring tt doe not contin */." In Figure 1, ti denition i own n utomton tt recognize ^(*/). In Figure 1, ^* denote te trnition y ny crcter except *, nd f^/g indicte te look-ed trnition y ^/ (tt i, te trnition y looking t ny crcter except / witout reding te crcter). Te utomton for te extended regulr expreion (^(*/))**/ i own in Figure 2. It i otined y pplying te well-known lgoritm tt generte rt n NFA (Nondeterminitic Finite Automton) from regulr expreion nd ten trnform it into DFA (Determinitic Finite Automton) [1, 5]. Ti i te utomton tt recognize te tring tt follow te ed tring /* of comment. It i lo te pttern-mtcing utomton tt recognize \te rt tring */" in given text. Anoter regulr expreion for \te rt tring */" cn e *<*/> were i dened, in te context of eing followed y <*/>, ny crcter tt i not te rt crcter of */. Nmely, in ti ce i te indirect expreion for ^(*/). y itelf men ny ingle crcter. We cn contruct te DFA for *<*/> follow: contruct te DFA for *(*/) rt, nd ten eliminte ll te trnition from te tte tt red */. Te reultnt utomton correpond to te previou regulr expreion (2) (ee pge 59 of [5]). In ti pper, we generlize te ove metod nd decrie metod tt mecniclly generte te pttern-mtcing lgoritm for ny regulr expreion (in te following, we identify n utomton wit te correponding lgoritm). If repreent xed crcter tring, ten te generted lgoritm i equivlent to te ptternmtcing lgoritm of Knut-Morri-Prtt (KMP) [4]. We furter ow metod

3 Genertion of Pttern Mtcing Algoritm y Extended Regulr Expreion 3 for generting te lgoritm of Ao-Corick (A-C) [3], well known n ecient lgoritm tt erce everl pttern repetedly. 2 Te Knut-Morri-Prtt Pttern-Mtcing Algoritm 2.1 Uing ny-ymol Let 6 e n lpet, e regulr expreion over 6, nd [] e te lnguge tt denote. We dd to 6 te pecil ymol. Te mening of i dened []=6 wen we ignore it context, nd te context-dependent mening i dened in te following. We introduce form of regulr expreion, *< >, wit te purpoe of expreing te ed tring u of given text x uc tt te til of u i te rt tring tt mtce. Tt i, *< > mtce u = vw, were u i te ortet tring uc tt u i te ed of x, nd < > mtce w. Here we tke w te rt tring if mtce ot w nd wz. We exclude te ce " 2 [], ecue in ti ce it mtce every " t ny plce in given text. We dene te ove regulr expreion follow: Denition 1: Let " 62 []. [*< >] = f x 2 6* j Tere exit y nd z uc tt x=yz; z 2 [], nd tere exit no xi uc tt x=x 1 x 2 x 3 ; x 2 2 []; x g. For exmple, if =* nd x=c, ten *< > mtce c ecue * in mtce ". In te following, we conider te tring pttern-mtcing prolem. In given text, te prolem of ercing te rt tring tt i mtced y ecome te prolem of recognizing te regulr expreion *< >. We cn contruct te utomton for ti expreion wit te following tep. Firt contruct te NFA y ppending te utomton for to te one for *. Next trnform te NFA to te DFA y te well-known metod. Ten eliminte ll te trnition from te tte tt contin te nl tte of te NFA for. We will illutrte ti y uing te exmple expreion *. Te utomton for * ppended y te one for * i own in Figure 3. In trnforming ti NFA to DFA, we follow te well-known metod except for te -trnition. For te -trnition from te tte to te tte t in NFA, we dd t to ll te trget tte of oter trnition from in DFA, nd we cnge te -trnition to te trnition y ny crcter tt i not contined in oter trnition from, own in Figure 4. For exmple, te trget tte of te - trnition in DFA of Figure 4 contin NFA Stte 1 nd 2, ecue tee re te trget tte of te -trnition nd -trnition in te NFA. Ten, to mke te rt mtced tte into te nl tte, we eliminte ll te trnition from te mtced tte{tt i, te tte tt contin nl tte in NFA. By pplying tee metod to Figure 3, we otin Figure 5. Here, ll te trnition from te tte compoed of NFA Stte 1 nd 4 ve een eliminted. Figure 5 correpond to te pttern-mtcing lgoritm of Knut-Morri-Prtt (KMP). However, KMP del only wit pttern of te form of = n; i 2 6. Our metod cn del wit generl regulr expreion pttern. For exmple, te ove * i regulr expreion tt contin te repeting ymol *. Te

4 4 Advnce in Softwre Science nd Tecnology 5, Fig. 3 NFA for ** [^] Fig. 4 Trnltion from NFA tt contin into DFA correponding utomton, te pttern-mtcing lgoritm, cn e generted y te uul metod modied y te metod own in Figure Uing negtion ymol of *< > in te previou ection cn e interpreted \te rt crcter of tring tt i dierent from." To expre ti more directly, we introduce te nottion ^ nd dene it follow. Te mening of ^ i imilr to te negtion of, ut it depend on it context nd i dierent from negtion dened [^]=6*[] in [2]. Denition 2: [(^)] = fx 2 6* j Tere exit nd z uc tt x=z; 2 6, z 2[], nd tere exit no xi uc tt x=x 1 x 2 ; x 1 2[]g. By uing ti nottion, te pttern-mtcing prolem for cn e expreed te prolem of recognizing te regulr expreion (^)*. Here, y Denition 2, (^) i interpreted under it rigt-nd context. Terefore (^)(^) i interpreted (^)((^)). Furtermore, we exclude te ce " 2 [], in te previou ection. Now, n exmple, we preent te pttern-mtcing prolem for c; tt i, we will contruct DFA for (^(c))*c. ^ [^] [^] Fig. 5 DFA tt recognize te rt *

5 Genertion of Pttern Mtcing Algoritm y Extended Regulr Expreion 5 { ^ { ^ } ^ } ^ } ^c } Fig. 6 DFA for ^ (c) { c ^ {^ } ^ } ^ } ^c } Fig. 7 DFA for (^ (c))*c DFA for ^(c) i own in Figure 6. In Figure 6, te tring compoed of crcter etween f nd g denote looked tring; tt i, tte trnition occur y tee crcter in uul utomton, ut, fter recognizing ^(), tee crcter re treted if tey ve not een red yet. In term of n utomton, te input tpe i mrked t te eginning of f-trnition, nd te input tpe i plced t te mrk t te end of g-trnition. DFA for (^(c))*c cn e otined y rt pplying te uul metod for NFA contruction to Figure 6 nd c, ten y pplying te uul lgoritm for trnltion from NFA to DFA. In ti proce, te crcter ccompnied y f or g re treted ordinl crcter. Te reult own in Figure 7 i te nve pttern mtcing lgoritm. For exmple, if te text cd i red y te utomton of Figure 7, te equence of tte trnition i 123c1f1c1g2345d1f123d1gf1d1g, nd te recognition fil. Here, te encloing rce men \reding gin." In ti lgoritm, fter ^ (c in ti exmple) een red, ^ i red gin, own in f^g. Ti i equivlent to reding two ^ t te initil tte. Te reultnt tte i te initil tte own in 1f1c1g. Terefore, in ti ce, it i equivlent to returning to te initil tte witout eing red gin. In oter word, f nd g cn e eliminted from te utomton. By pplying imilr optimiztion to oter trnition, DFA in Figure 8 i otined. Wen ^c i red t Stte 5 of Figure 7, te trnition proceed trictly 1f123g, ten te ^c tt i lt red ould e red gin to ee weter it i or not. Ti trnition i own in te trnition from Stte 5 to Stte 3 in Figure 8. Te lgoritm of Figure 8 i equivlent to te pttern-mtcing lgoritm of KMP (tt i own in te rt prt of [4] nd i referred in mny text ook of lgoritm KMP' pttern-mtcing lgoritm).

6 6 Advnce in Softwre Science nd Tecnology 5, 1993 { ^c } ^ c {^ } ^ {^ } Fig. 8 DFA for (^ (c))*c(fter optimiztion) ^ c [^ ] ^ [^ ] [^c ] Fig. 9 DFA for *< c > Note tt ti optimiztion cnnot e pplied, in generl, to te utomton tt contin loop, ecue te tring tt ould e red gin cnnot e determined uniquely. * in te previou ection i uc n exmple. Figure 9 i otined y pplying te metod in te previou ection tt ue ny-ymol to te ove exmple c. Ti i te optimized verion of Figure 8, ecue ti doe not contin ny red-gin trnition. Ti i equivlent to te lgoritm written in te lt prt of [4]. 3 Te Ao-Corick Algoritm Te Ao-Corick lgoritm i well known n ecient lgoritm for ercing everl pttern imultneouly nd repetedly. Ti lgoritm cn e derived y metod imilr to te one in te previou ection. 3.1 Uing ny-ymol In given text x, in order to erc ll te tring tt re mtced y te pttern of, it i ucient to erc for ll te tring in te ed prt of x tt re mtced y rt, ten to erc for in te ed of x, nd ten to erc for in te ed of x, nd o on. Terefore ti prolem i equivlent to te prolem of pttern mtcing wit *. Te lgoritm cn e derived DFA for * in wic te tte tt contin te nl tte of record te fct tt it recognized. For exmple, let = (ejejijer). Ti i te exmple in [3]. NFA for * i own in Figure 1.

7 Genertion of Pttern Mtcing Algoritm y Extended Regulr Expreion 7 e r i e Fig. 1 NFA for *(ejejijer) oter e r r 7 6 i 3 7 i 2 5 r e Fig. 11 DFA for repeted recognition of (ejejijer) DFA own in Figure 11 cn e otined from ti NFA y te well-known metod modied y -trnition, own in Figure 4. In Figure 11, every tte contin one or more tte in te originl NFA, nd numer on te rigt ide of tte re te numer of te nl tte in NFA. Te numer 2, 5, 7, nd 9 indicte tt tey re te tte tt record te recognition of e, e, i, nd er, repectively. An rrow to te numer in tringle indicte te trnition to te Stte in te ce of \oter crcter." For exmple, tte trnition from Stte ( 1) to Stte occur in ce of crcter oter tn,, e, i. Altoug Figure 11 look complex, it i not dicult to undertnd. For exmple, te trnition from Stte ( 2 5) re te me te trnition from Stte ( 2) ecue tere i no trnition from Stte 5 in NFA. Similrly, te trnition from Stte ( 3 7) nd from Stte ( 3 9) re te me te one from Stte ( 3). Every tte of ti DFA contin Stte of NFA nd te correponding trnition. Te ret of DFA tt i own in old rrow in Figure 11 look imilr to Figure 1. Te lgoritm in Figure 11 i te me te nl optimized lgoritm in [3]. Here it i derived mecniclly y ligtly expnding te well-known metod. 3.2 Uing negtion ymol ^ w introduced to indicte te ed crcter of tring tt i not mtced y. Here, we introduce one more ymol to indicte te ed crcter of tring

8 8 Advnce in Softwre Science nd Tecnology 5, 1993 tt i mtced y :. By uing ti ymol, te lgoritm for ercing ll tring tt re mtced y in given text cn e derived from te following regulr expreion: (^j )*. We need to dd to te derived utomton te ction of recording te recognition of tt i found wen i recognized. By pplying imilr metod to te one in Section 2.2 to ti (^j )*, we cn otin n lgoritm tt i equivlent to te one in [3]. However, we do not explin te metod ere, ecue it i not imple te previou metod. 4 Concluion We introduced te ymol for repreenting ny ingle crcter in regulr expreion, nd y giving context dependent interprettion to * we owed tt te denition of te comment in C lnguge, wic i dicult to dene y regulr expreion, cn e expreed nturlly y imple expreion. By pplying ti metod to generl regulr expreion, te pttern-mtcing lgoritm of Knut-Morri-Prtt cn e derived mecniclly not only for te pttern of crcter tring ut for te pttern expreed y generl regulr expreion. We lo owed tt imilr proce i poile y introducing ymol tt indicte kind of negtion expreion. Furtermore, we owed tt te prolem of repeted pttern mtcing for everl pttern cn e expreed in imple regulr expreion y uing *, nd te Ao-Corick lgoritm cn e derived mecniclly from te expreion. Altoug we did not invent ny eentilly new pttern-mtcing lgoritm y tee metod, we ucceeded in generlizing te pttern expreion. Becue te prolem of pttern mtcing cn e expreed in regulr expreion, our metod cn e implemented in lexicl nlyzer genertor uc lex. Acknowledgment. We would like to expre our tnk to Prof. Mtk S, Tokyo Intitute of Tecnology, for i vlule comment. Reference [1] Ao, A.V., Seti, R. nd Ullmn, J.D.: Compiler: Principle, Tecnique, nd Tool, Addion-Weley, [2] Ao, A.V., Seti, R. nd Ullmn, J.D.: Deign nd Anlyi of Computer Algoritm, Addion-Weley, [3] Ao, A.V. nd Corick, M.J.: Ecient String Mtcing: An Aid to Biliogrpic Serc, Comm. ACM, Vol. 18, No. 6 (1975), pp [4] Knut, D.E., Morri, J.H. Jr. nd Prtt, V.R.: Ft Pttern Mtcing in String, SIAM J. Comput., Vol. 6, No. 2 (1977), pp [5] Nkt, I.: Compiler, Sngyo-Too, Tokyo, 1981 (in Jpnee). [6] Screiner, A.T. nd Friedmn, H.G. Jr.: Introduction to Compiler Contruction wit UNIX, Prentice-Hll, 1985.

9 Genertion of Pttern Mtcing Algoritm y Extended Regulr Expreion 9 Ikuo Nkt Intitute of Informtion Science nd Electronic Univerity of Tuku Tuku, Irki 35 Jpn