FLAC Closure Properties

Transcription

1 FLAC Closure Properties 1 Nondeterministic Mchines Klus Sutner Crnegie Mellon Universlity 2 Determiniztion Fll Closure Properties Where Are We? 3 The Logic Link 4 We hve definition of regulr lnguges in terms of deterministic finite utomt (DFAs). There re two killer pps for regulr lnguges: pttern mtching nd decision lgorithms. In order to get etter understnding of regulr lnguges, it turns out tht other chrcteriztions cn e very useful, oth from the theory perspective s well s for the construction of lgorithms. As lredy mentioned, we re primrily interested in using utomt to solve decision prolems in logic. To this end we hve to construct mchines for vrious lnguges, quite often over lrge lphets of the form Γ = Σ Σ... Σ This turns out to e the right environment for checking vlidity of logicl formule over certin structures. Note tht these lphets cn get quite lrge. Lnguge Opertions 5 More Opertions 6 The key to mny of these lgorithms is the fct tht regulr lnguges re closed under gret mny more opertions. To egin with, there is closure under Boolen opertions: union intersection complement This is useful for pttern mtching ut lso for decision lgorithms (one cn del with propositionl logic; in while, we will see how to hndle quntifiers). Here re some more opertions on lnguges tht do not ffect regulrity: reversl conctention Kleene str homomorphisms inverse homomorphisms Als, it is difficult to estlish these properties within the frmework of DFAs: the constructions of the corresponding mchines ecome rther complicted. One elegnt wy to void these prolems is to generlize our mchine model to llow for nondeterminism, nd show tht the generl mchines still only ccept regulr lnguges.

2 Nondeterministic FSMs 7 Trces nd Runs 8 Here is strightforwrd generliztion of DFAs tht llows for nondeterministic ehvior. Recll tht trnsition systems my well e nondeterministic. Definition A nondeterministic finite utomton (NFA) is structure A = Q, Σ, τ; I, F where Q, Σ, τ is trnsition system nd the cceptnce condition is given y I, F Q, the initil nd finl sttes, respectively. So in generl there is no unique next stte in n NFA: there my e no next stte, or there my e mny. Of course, we cn think of DFA s specil type of NFA. It is strightforwrd to lift the definition of cceptnce from DFAs to NFAs (it ll comes down to pth existence, nywy). Recll tht in ny trnsition system Q, Σ, τ run is n lternting sequence π = p 0, 1, p 1,..., r, p r where p i Q, i Σ nd τ(p i 1, i, p i) for ll i = 1,..., r. p 0 is the source of the run nd p r its trget. The length of π is r. The corresponding trce or lel is the word r. Some uthors insist tht I = {q 0}. This mkes no sense. The Ftl Definition 9 Sources of Nondeterminism 10 The cceptnce condition is essentilly the sme s for DFAs, except tht initil sttes re no longer unique (nd even if they were, there could e multiple trces). Definition An NFA A = Q, Σ, τ; I, F ccepts word w Σ if there is run of A with lel w, source in I nd trget in F. We write L(A) for the cceptnce lnguge of A. But note tht now there my e exponentilly mny runs with the sme lel. In prticulr, some of the runs strting in I my end up in F, others my not. There is hidden existentil quntifier here. Agin: ll tht is needed for cceptnce is one ccepting run. Note tht nondeterminism cn rise from two different sources: Trnsition nondeterminism: there re different trnsitions p q nd p q. Initil stte nondeterminism: there re multiple initil sttes. In other words, even if the trnsition reltion is deterministic we otin nondeterministic mchine y llowing multiple initil sttes. Intuitively, this second type of nondeterminism is less wild. Autonomous Trnsitions Epsilon Moves 11 Σ ε 12 While we re t it: there is yet nother nturl generliztion eyond just nondeterminism: utonomous trnsitions, k epsilon moves. These re trnsitions where no symol is red, only the stte chnges. This is perfectly fine considering our Turing mchines ncestors. We will encounter severl occsions where it is convenient to enlrge the lphet Σ y dding the empty word ε: Σ ε = Σ {ε} Definition A nondeterministic finite utomton with ε-moves (NFAE) is defined like n NFA, except tht the trnsition reltion hs the formt τ Q (Σ {ε}) Q. Thus, n NFAE my perform severl trnsitions without scnning symol. Hence trce my now e longer thn the corresponding input word. Other thn tht, the cceptnce condition is the sme s for NFAs: there hs to e run from n initil stte to finl stte. Of course, ε is not new lphet symol. Wht s relly going on? Σ freely genertes the monoid Σ, nd ε is the unit element of this monoid. We cn dd the unit element to the genertors without chnging the monoid. We could even dd ritrry words nd llow super-trnsitions like p q Explin why this mkes no difference s fr s lnguges re concerned.

3 Reversl Closure 13 Exmple: Third Symol 14 Here is perfect exmple of n opertion tht preserves regulrity, ut is difficult to cpture within the confines of DFAs. Let L op = { x op x L } It is very esy to uild DFA for L,3 = { x x 3 = }. We omit the sink to keep the digrm simple. e the reversl of lnguge, (x 1x 2... x n 1x n) op = x nx n 1... x 2x 1. The direction in which we red string should e of supreme irrelevnce, so for regulr lnguges to form resonle clss they should e closed under reversl.,,, Suppose L is regulr. How would we go out constructing mchine for L op? But L op,3 = { x x 3 = } = L, 3 is hrd for DFAs: we don t know how fr from the end we re. By flipping trnsitions nd interchnging initil nd finl sttes we otin mchine tht looks like so: Nondeterministic Mchines,,, Determiniztion Closure Properties It is cler tht the new mchine ccepts L, 3. Of course, it s no longer DFA. Conversion to DFA 17 Terminology 18 Our first order of usiness is to show tht NFAs nd NFAEs re no more powerful thn DFAs in the sense tht they only ccept regulr lnguges. Note, though, tht the size of the mchines my chnge in the conversion process, so one needs to e it creful. The trnsformtion is effective: the key lgorithms re For the time eing, we will refer to DFAs, NFAs nd NFAEs simply s finite utomt. Strictly speking, ll three types re distinct, ut there re nturl inclusions DFA NFA NFAE Epsilon Elimintion Convert n NFAE into n equivlent NFA. Determiniztion Convert n NFA into n equivlent DFA. The hert of the OO fntic now ets fster...

4 NFAE to NFA 19 ε-closure 20 A trnsitive closure prolem: we hve to replce chins of trnsitions Epsilon elimintion is quite strightforwrd nd cn esily e hndled in polynomil time: ε ε ε introduce new ordinry trnsitions tht hve the sme effect s chins of ε trnsitions, nd remove ll ε-trnsitions. Since there my e chins of ε-trnsitions this is in essence trnsitive closure prolem. Hence prt I of the lgorithm cn e hndled with the usul grph techniques. y new trnsitions Epsilon Elimintion 21 Determiniztion 22 Theorem For every NFAE there is n equivlent NFA. Proof. This requires no new sttes, only chnge in trnsitions. Suppose A = Q, Σ, τ; I, F is n NFAE for L. Let A = Q, Σ, τ ; I, F where τ is otined from τ s on the lst slide. I is the ε-closure of I: ll sttes rechle from I using only ε-trnsitions. Conversion of nondeterministic mchine to deterministic one ppered first in seminl pper y Rin nd Scott titled Finite Automt nd Their Decision Prolem. In fct, nondeterministic mchines were introduced there. Theorem (Rin, Scott 1959) For every NFA there is n equivlent DFA. Agin, there my e qudrtic low-up in the numer of trnsitions nd it my well e worth the effort to try to construct the NFAE in such wy tht this low-up does not occur. The ide is to keep trck of the set of possile sttes the NFA could e in. This produces DFA whose sttes re sets of sttes of the originl mchine. Generl Astrct Nonsense to the Rescue 23 Proof of Rin-Scott 24 The trnsition reltion in n NFA hs the form τ Q Σ Q Suppose A = Q, Σ, τ; I, F is n NFA. Let A = P(Q), Σ, δ; I, F By GAN we cn think of it s function: nd this function nturlly extends to τ : Q Σ P(Q) τ : P(Q) Σ P(Q) The ltter function cn e interpreted s the trnsition function of DFA on P(Q). Done. ;-) where δ(p, ) = { q Q p P τ(p,, q) } F = { P Q P F } It is strightforwrd to check y induction tht A nd A re equivlent. The mchine from the proof is the full power utomton of A, written pow f (A), mchine of size 2 n. Of course, for equivlence only the ccessile prt pow(a), the power utomton of A, is required.

5 Accessile Prt 25 Coccessile/Trim Prt 26 This is s good plce s ny to tlk out useless sttes: sttes tht cnnot pper in ny ccepting computtion nd tht cn therefore e eliminted. Definition A stte p in finite utomton A is ccessile if there is run with source n initil stte nd trget p. The utomton is ccessile if ll its sttes re. Now suppose we remove ll the inccessile sttes from utomton A (mening: djust the trnsition system nd the set of finl sttes). We otin new utomton A, the so-clled ccessile prt of A. Lemm The mchines A nd A re equivlent. There is dul notion of coccessiility: stte p is coccessile if there is t lest one run from p to finl stte. Likewise, n utomton is coccessile if ll its sttes re. An utomton is trim if it is ccessile nd coccessile. It is esy to see tht the trim prt of n utomton is equivlent to the whole mchine. Moreover, we cn construct the coccessile nd trim prt in liner time using stndrd grph lgorithms. Wrning: Note tht the coccessile prt of DFA my not e DFA: the mchine my ecome incomplete nd we wind up with prtil DFA. The ccessile prt of DFA lwys is DFA, though. Keeping Trim 27 Smrt Power Automt 28 Of course, in implementtions we would like to keep mchines smll, so mking them ccessile or trim is good ide. There re relly two seprte issues here. First, we my need to clen up mchines y running n ccessile or trim prt lgorithm whenever necessry this is esy. Much more interesting is to void the construction of inccessile sttes of mchine in the first plce: idelly ny lgorithm should only produce ccessile mchines. While ccessiility is esy to gurntee, coccessiility is not: while constructing mchine we do not usully know the set of finl sttes hed of time. So, there my y need to eliminte non-coccessile sttes. The right wy to construct the Rin-Scott utomton for A = Q, Σ, τ; I, F is to tke closure in the mient set P(Q): clos( I, (τ ) Σ) Here τ is the function P(Q) Σ P(Q) defined y τ (P ) = { q Q p P (p q) } This produces the ccessile prt only, nd, with luck, is much smller thn the full power utomton. WTF? 29 Virtul Grphs 30 Given collection f = (f 1,..., f k), of endofunctions f i : A A on set A nd suset B A, the closure of B under f is defined y clos( B, f ) := { X A B X, f i(x) X, i [k] } In other words, the closure is the lest suset of A tht contins B nd ll the imges of B under the f i. As written, the definition is impredictive, ut we cn esily turn this into n efficient lgorithm (t lest in the finite cse). Think of G = A; f 1, f 2,..., f k i s leled digrph with edges p q for f i(p) = q, the virtul grph (or mient grph) where we live. We need to compute the rechle prt of B in this grph G. This cn e done using stndrd lgorithms such s Depth-First-Serch or Bredth-First-Serch. The only difference is tht we re not given n djcency list representtion of G: we compute edges on the fly. No prolem t ll. This is very importnt when the mient grph is huge: we my only need to touch smll prt.

6 Power Automton Algorithm 31 Exmple: L, 3 32 Recll Here is slightly more hcky version of this construction. ctive = QQ = {I}; while( ctive!= empty ) P = ctive.extrct(); forech in Sigm do compute R = tu(p,) keep trck of trnsition P to R if( R notin QQ ) then dd R to QQ nd ctive L,k = { x {, } x k = }. For negtive k this mens: kth symol from the end. It is trivil to construct n NFA for L, 3:,,, Upon completion, QQ P(Q) is the stte set of the ccessile prt of the full power utomton. We write pow(a) for this mchine. Rin-Scott 33 The Digrm 34 Here is the corresponding digrm, rendered in prticulrly rillint wy. This is so-clled de Bruijn grph (inry, rnk 3). Applying the Rin-Scott construction we otin mchine with 8 sttes {1}, {1, 2}, {1, 2, 3}, {1, 3}, {1, 2, 3, 4}, {1, 3, 4}, {1, 2, 4}, {1, 4} where 1 is initil nd 5, 6, 7, nd 8 re finl. The trnsitions re given y Note tht the full power set hs size 16, our construction only uilds the ccessile prt (which hppens to hve size 8). Explin this picture in terms of the Rin-Scott construction. Acceptnce Testing 35 Memership Testing 36 Here is nturl modifiction of the DFA cceptnce testing progrm. Recll one of the key pplictions of FSMs: cceptnce testing is very fst nd cn e used to del with pttern mtching prolems. How much of computtionl hit do we tke when we switch to nondeterministic mchines? We cn use the sme pproch s in determiniztion: insted of computing ll possile sets of sttes rechle from I, we only compute the ones tht ctully occur long prticulr trce given y some input word. P = I; while( = x.next() ) // next input symol P = tu_(p); return ( P intersect F!= empty ); The updte step uses the sme mps τ : P(Q) P(Q) s in the Rin-Scott construction. Think of dirty tricks like hshing to speed things up.

7 Running Time 37 A Better Mousetrp? 38 The loop executes x times, just s with DFAs. Unfortuntely, the loop ody is no longer constnt time: we hve to updte set of sttes P Q. This cn certinly e done in O( Q 2 ) steps though smrt dt structures my sometimes give etter performnce. Actully, it seems tht in prctice (i.e. in NFAs tht pper nturlly in some ppliction such s pttern mtching) one often dels with overhed tht is liner in Q rther thn qudrtic. At ny rte, we cn check cceptnce in n NFA in O( x Q 2 ) steps. For fixed mchines this is still liner in x, ut the hidden constnt my e significnt. Acceptnce testing is slower, nondeterministic mchines re not simply ll-round superior to DFAs. Advntges: Esier to construct nd mnipulte. Sometimes exponentilly smller. Sometimes lgorithms much esier. Drwcks: Acceptnce testing slower. Sometimes lgorithms more complicted. Which type of mchine to choose in prticulr ppliction cn e hrd question, there is no esy generl nswer. Products 40 Nondeterministic Mchines Suppose we hve two trnsition systems T 1 = Q 1, Σ, τ 1 nd T 2 = Q 2, Σ, τ 2 over the sme lphet Σ. Determiniztion Construct new trnsition system T = T 1 T 2, the so-clled (Crtesin) product s follows. Q = Q 1 Q 2 τ((p, q),, (p, q )) τ 1(p,, p ) τ 2(q,, q ) 3 Closure Properties It is often helpful to think of the new trnsition system s running T 1 nd T 2 in prllel. Als, the size of T is qudrtic in the sizes of T 1 nd T 2. This cuses prolems if product mchine construction is used repetedly. Product Mchine 41 It s DFA 42 To get mchine we need to define n cceptnce condition. The new initil stte set is I 1 I 2. By selecting finl sttes in the product, we cn get union nd intersection L(A 1) nd L(A 2): union F = F 1 Q 2 Q 1 F 2 intersection F = F 1 F 2 In the cse where T 1 nd T 2 come from DFAs A 1 nd A 2 the product is gin DFA: the new trnsition function δ = δ 1 δ 2 looks like Q = Q 1 Q 2 δ((p, q), ) = (δ 1(p, ), δ 2(q, )) More importntly, we cn lso construct mchine for the complement L(A 1) L(A 2). difference F = F 1 (Q 2 F 2)

8 Asolute Complements 43 And ck to closures In the specil cse where A 1 is universl we get the plin complement Σ L(A 2). Of course, we do not need product construction here, ll we need to do is switch finl nd non-finl sttes in the given DFA. F = Q F. For the umpteenth time: rel lgorithm for product mchines should not construct the full Crtesin product. Insted, one should compute closures of the pproprite initil sttes under the trnsition function/reltion of the product mchine. For exmple, for product DFA construction we need Dire Wrning: Determinism is essentil here, we will see shortly tht complementtion for nondeterministic mchines is much hrder. clos( (q 01, q 02), δ 1 δ 2) Construct counterexmple tht shows tht the switch-sttes construction in generl fils to produce complements in NFAs. Figure out in detil how to do these constructions producing the ccessile prt only. Exmple 45 Full Product Automton 46 Consider the product utomton for DFAs A nd A, ccepting nd, respectively. A :, 0 1 2,,, , , The Accessile Prt 47 Closure Properties of Reg Σ 48 0, 1 2, At ny rte, we hve estlished criticl closure property for regulr lnguges:,, Lemm Regulr lnguges form Boolen lger: they re closed under union, intersection nd complement. Moreover, the closure is effective nd even polynomil time for DFAs. Effective here mens tht, given two mchines A 1 nd A 2, we cn compute new mchine for L(A 1) L(A 2), L(A 1) L(A 2) nd L(A 1) L(A 2). If the mchines re DFAs ll opertions re polynomil time (ut complement my low up for NFAs) We will hve more to sy out the complexity of the corresponding lgorithms; this is criticl for pplictions.

9 Déjà Vu All Over Agin 49 Deciding Equivlence 50 The lst lemm should sound very fmilir y now: there re nlogous results for semi-decidle sets, decidle set nd primitive recursive sets. Of course, for semi-decidle ones we hve to omit complements. We cn now del with the Equivlence prolem from lst time. Prolem: Equivlence Instnce: Two DFAs A 1 nd A 2. Question: Are the two mchines equivlent? In ll cses, we cn effectively construct mchines/progrms tht recognize the sets otined y Boolen opertions. The rel chllenge is stte complexity: in mny pplictions one needs to del with very lrge mchines. The corresponding lgorithm my require lot of ingenuity. Lemm A 1 nd A 2 re equivlent iff L(A 1) L(A 2) = nd L(A 2) L(A 1) =. Note tht the lemm yields qudrtic time lgorithm. We will see etter method lter. Deciding Inclusion 51 Conctention nd Kleene Str 52 Oserve tht we ctully re solving two instnces of closely relted prolem here: Prolem: Inclusion Instnce: Two DFAs A 1 nd A 2. Question: Is L(A 1) L(A 2)? which prolem cn e hndled y Lemm L(A 1) L(A 2) iff L(A 1) L(A 2) =. Definition Given two lnguges L 1, L 2 Σ their conctention (or product) is defined y L 1 L 2 = { xy x L 1, y L 2 }. Let L e lnguge. The powers of L re the lnguges otined y repeted conctention: L 0 = {ε} L k+1 = L k L Note tht for ny clss of lnguges Equivlence is decidle when Inclusion is so decidle. However, the converse my e flse ut it s not so esy to come up with n exmple. The Kleene str of L is the lnguge L = L 0 L 1 L 2... L n... Kleene str corresponds roughly to while-loop or itertion. Str Exmples 53 Conctention is Esy 54 Exmple {, } : ll words over {, } Exmple {, } {}{, } {}{, } : ll words over {, } contining t lest two s Exmple {ε,, }{,, } : ll words over {, } not contining suword Exmple {0, 1}{0, 1} : ll numers in inry, with leding 0 s {1}{0, 1} {0}: ll numers in inry, no leding 0 s Suppose we hve two NFAs A 1 nd A 2 for L 1 nd L 2. To uild mchine for L 1 L 2 it is esiest to go with n NFAE A: Let Q = Q 1 Q 2, keep ll the old trnsitions nd dd ε-trnsitions from F 1 to I 2. I 1 re the initil sttes nd F 2 the finl sttes in A. It is cler tht L(A) = L 1 L 2. But note tht this construction my introduce qudrticlly mny trnsitions. Find wy to keep the numer of new trnsitions liner.

10 Conctention is Hrd 55 Kleene Str nd More Nondeterminism 56 Now suppose we hve two DFAs A 1 nd A 2 for L 1 nd L 2. We cn get DFA from the previous mchine, ut cn we uild DFA for L 1 L 2 directly? The prolem is tht given word w we need to split it s w = xy nd then feed x to A 1 nd y to A 2. But there re w + 1 mny wys to do the split, nd we hve priori no ide where the rek should e. One cn lso think of this s guess nd verify prolem: guess x nd y, nd then check tht indeed A 1 ccepts x, nd A 2 ccepts y. Of course, there is slight prolem: DFAs don t know how to guess. The sitution gets worse if we try to construct DFA for the Kleene str of lnguge: L = L 0 L 1 L 2... L n... Not only do we not know where to split the string, we lso don t know how mny locks there re. Moreover, the numer of locks is unounded (t lest in generl), nd it is fr from cler how this type of processing cn e hndled y DFA. Of course, we cn simply strt with n NFA or even nd NFAE, nd then convert ck to DFA. NFAEs 57 Peles 58 If we re willing to del with ε-trnsitions closure under conctention nd Kleene str is not hrd t ll. Given two NFAs, construct n NFAE for the conctention of the two mchines. Given NFA, construct n NFAE for the Kleene str of the mchine. Another wy of thinking out this is to plce peles on the sttes. Initilly, ech stte in I hs pele. Under input, pele on p multiplies nd moves to ll q such tht p q. Multiple peles on stte re condensed into single one. We ccept whenever pele ppers in F. Peles re very helpful in prticulr in the direct construction of DFAs: the movement of the set of ll peles is perfectly deterministic. Pele Automton for Conctention 59 Quoi? 60 We strt with one copy of DFA A 1, the mster, nd one copy of DFA A 2, the slve. Plce one pele on the initil stte of the mster mchine. Move this nd ll other peles long trnsitions ccording to stndrd rules. Whenever the mster pele reches finl stte, plce new pele on the initil stte of the slve utomton. The composite mchine ccepts if pele sits on finl stte in the slve mchine. First, the mchine just descried is deterministic: we plce nd remove collection of peles ccording to n entirely deterministic rule. The sttes re of the form (p, P ) where p Q 1 nd P Q 2, corresponding to complete record of the positions of ll the peles. Now let n i e the stte complexity of A i. Then the numer of sttes is t most n 1 2 n2 Of course, the ccessile prt my well e smller. So the numer of sttes is ounded y A 1 2 A2 : the A 1 prt is deterministic ut the A 2 prt is not.

11 s 61 Aside: Complicted Intersections 62 There re severl gps nd inccurcies in the outline ove, fix them ll. Crry out this construction for the lnguges E = even numer of s nd E = even numer of s nd run some exmples. Product constructions re importnt even for reltively simple lnguges, it cn e quite difficult to uild utomt for, sy, the intersection of two regulr lnguges directly y hnd. Here is n exmple: uild DFA for the lnguge of ll words tht contin the scttered suword (not fctor) 3 times, nd multiple-of-3 numer of s. Building the two component mchines nd tking their product we get Explin why the peling construction relly defines DFA Crry out peling construction for Kleene str Sizes of Product Mchines 63 Bd News: DFA Intersection 64 More generlly, suppose we hve DFAs A i of size n i, respectively. Then the full product mchine hs n = n 1n 2... n s sttes. A = A 1 A 2... A s 1 A s The full product mchine grows exponentilly, ut its ccessile prt my e much smller. Als, there re cses where exponentil low-up cnnot e voided. Here is the Emptiness Prolem for list of DFAs rther thn just single mchine: Prolem: Instnce: Question: DFA Intersection A list A 1,..., A n of DFAs Is L(A i) empty? This is esily decidle: we cn check Emptiness on the product mchine A = A i. The Emptiness lgorithm is liner, ut it is liner in the size of A, which is itself exponentil. And, there is no universl fix for this: Theorem The DFA Intersection Prolem is PSPACE-hrd.