DFA minimistion using the Myhill-Nerode theorem Johnn Högerg Lrs Lrsson Astrct The Myhill-Nerode theorem is n importnt chrcteristion of regulr lnguges, nd it lso hs mny prcticl implictions. In this chpter, we introduce the theorem n present its proof. One consequence of the theorem is n lgorithm for minimising DFAs tht is outlined in the ltter prt of this pper. To clrify how the lgorithm works, we conclude with n exmple of its ppliction. 1 Introduction This chpter gives n introduction to DFA minimistion using the Myhill- Nerode theorem. DFA minimistion is n importnt topic ecuse it cn e pplied oth theoreticlly nd prcticlly, in for instnce compilers. Minimising DFA increses its efficiency y reducing its mount of sttes nd it lso enles us to determine if two DFAs re equivlent. The chpter is structured s follows. Following this introduction in Section 2, some relevnt ckground informtion nd definitions re stted, which will e used in the susequent sections. In Section 3, the Myhill-Nerode theorem is stted nd its proof presented. Section 4 contins the lgorithm for DFA minimistion tht uses the Myhill-Nerode theorem. An exmple showing how such minimistion might e conducted in specific cse is found in Exmple 4.2. 2 Bckground informtion nd definitions The reder is dvised to skim through this section to recll, t lest t n intuitive level, the terms tht will e used in this chpter. Following these intuitive descriptions, more forml definitions re provided. A deterministic finite utomton, or DFA for short, my e considered kind of very sic computtionl device. It processes strings of symols of certin lphet nd yields result tht is sid to e either ccepting or rejecting. Normlly, n ccepting result mens tht the string is prt of recognised lnguge. The lnguge is then defined s ll strings tht ring the DFA into n ccepting stte. 1
q 1 q 2 q 3 q 4 Figure 1: DFA ccepting the lnguge n,n > 2 Definition 2.1 (DFA) A deterministic finite utomton (DFA) A is quintuple A = (S,Σ,δ,s,F) consisting of: finite set of sttes S, finite set of symols Σ, clled the lphet, trnsition function δ : S Σ S, strting stte s S, nd set of ccepting sttes F S. We extend the definition of δ to the domin of Σ y letting 1. δ(q,λ) = q, where λ denotes the empty string, nd 2. δ(q,w) = δ(δ(q,),w), where Σ nd w Σ. Finlly, A is sid to ccept the string w Σ if δ(q 0,w) F. DFA re commonly represented y stte digrms, such s in Figure 1 nd Figure 2. The sttes tht re ccepting hve doule order, the initil stte is the stte with the ngle on the left nd the rrows or trnsitions signify wht stte should e entered fter reding the symol on the rrow. There re some limittions s to wht kind of lnguge the DFA cn recognise. The numer of symols in the lphet s well s the numer of sttes in the DFA hs to e finite. This mens tht while it is entirely possile to construct DFA tht ccepts strings tht for instnce look like n,n > 2 (let the strting stte of the DFA e followed y two rejecting sttes nd finlly y stte tht ccepts ny numer of s, see Figure 1) it is not possile to recognise lnguge such s n n,n N (the symol repeted n times, followed y the exct sme numer of s) ecuse tht would require infinitely mny sttes to keep trck of the numer of s to ensure tht the sme numer of s re red. See Figure 2 for n exmple of how DFA ccepting the lnguge n n,n 2 might e constructed. Note the ovious difficulty tht rises if n my e ritrrily lrge s well s the difference etween this lnguge nd n. The clss of lnguges tht cn e recognised y DFA re clled regulr lnguges or Type-3 lnguges in the Chomsky hierrchy. Every regulr lnguge cn e recognised y red-only Turing mchine, descried y regulr expression, generted y regulr grmmr, nd defined in so-clled mondic second-order logic. As fr s this chpter is concerned, the forml definition including syntx for regulr lnguges hs no useful mening. Insted, it is sufficient to define regulr lnguges s follows. 2
q 00 q 10 q 20 q 11 q 21, q in, q 22, Figure 2: DFA ccepting the lnguge n n,n 2 Definition 2.2 (regulr lnguge) The clss of lnguges tht cn e recognised y deterministic finite utomton. The Myhill-Nerode theorem tht will provide the theoreticl ckground tht llows us to minimise DFA requires the following definitions tht re relted to equivlence reltions. Definition 2.3 (equivlence reltion) An equivlence reltion,, is inry reltion on set if it is Reflexive ( ) Symmetric (if then ) Trnsitive (if nd c then c) Definition 2.4 (equivlence clsses, index) An equivlence reltion on set S imposes prtition S/ on S, such tht elements e,e S re in the sme prt of S/ if e e. The prts of S/ re clled equivlence clsses, nd we write [e] to denote the unique equivlence clss of S/ to which e elongs. The index of equivlence reltion is S/. Definition 2.5 (right invrint) An equivlence reltion on strings of symols from some lphet Σ is sid to e right invrint if x,y Σ with x y nd ll w Σ we hve tht xw yw. This lst definition sttes tht n equivlence reltion hs the right invrint property if two equivlent strings (x nd y) tht re in the lnguge still re equivlent if third string (w) is ppended to the right of oth of them. 3
In the rest of the chpter, the following two equivlence reltions will e of prticulr importnce. Definition 2.6 (equivlence reltion for DFA:s) Let A e DFA. For strings x,y Σ, let x A y iff the sme stte in the DFA is reched from the initil stte y reding the strings x nd y. Definition 2.7 (equivlence reltion for lnguges) Let L e ny lnguge over Σ. For strings x,y Σ, let x L y iff w Σ we hve tht xw L yw L. Definition 2.8 (distinguishing string) Let A = (Q,Σ,δ,q 0,F) e DFA. The string w Σ is distinguishing string for sttes p,q Q if exctly one of δ(p,w) nd δ(q,w) is in F. Intuitively, distinguishing string s for sttes p nd q in n utomton A is string which is mpped to n ccepting stte from exctly one of p nd q. In other words, the existence of distinguishing string for pir of sttes p,q Q proves tht p nd q re not equivlent. 3 Myhill-Nerode theorem In this section, the Myhill-Nerode theorem [Ner58] nd its proof re presented. Theorem 3.1 Let L Σ. Then the following sttements re equivlent: 1. There is DFA tht ccepts L (L is regulr). 2. There is right-invrint equivlence reltion of finite index such tht L is the union of some of the equivlence clsses of. 3. L is of finite index. Proof The proof tht we give here is n dption of tht in [Vis05], which uses the impliction order (1) (2) (3) (1). (1) (2). Since L is regulr lnguge, there exits DFA A = (Q,Σ,δ,q 0,F) tht recognises L. Recll tht A is right-invrint reltion of finite index. It remins to oserve tht L = [x], where x Σ. x:δ(q 0,x) F (2) (3). Assume tht L is the union of some equivlence clsses of rightinvrint reltion of finite index. Now, if (the prtition imposed y) is refinement of (the prtition imposed y) L, then L must e of finite index since it hs fewer equivlence clsses thn, which y ssumption is of finite index. As we shll see, is indeed refinement of L. Since is right-invrint, we hve tht for every pir x,y Σ such tht x y, it must hold tht (z) xz yz. Moreover, we hve tht L is the union of some 4
equivlence clsses of, so if x y, then x L y L. Comining these implictions gives us x y (z)xz yz (z)xz L yz L x L y. We now know tht x y implies x L y, nd drw the conclusion tht is refinement of L. (3) (1). To prove tht if L is of finite index, then L is regulr, it suffices to construct, for n ritrry L, DFA A which recognises L. The ide tht underlies the construction is to use the equivlence clsses of L s sttes in A. First, we choose x 1,...,x k s representtives for the k equivlence clsses of L, nd then ssemle the DFA A = (Q,Σ,δ,q 0,F), where Q = {[x 1 ],...,[x k ]}, δ([x],) = [x], q 0 = [λ], where λ denotes the empty string, nd F = {[x] x L}. It should e ovious tht δ is well defined nd tht δ(q 0,x) F iff x L. Plese note tht the Myhill-Nerode theorem lets us decide whether lnguge L is regulr, y determining if either of Condition 2 or Condition 3 is met. 4 DFA minimistion Not only does the Myhill-Nerode Theorem provide n informtive chrcteristion of the recognisle lnguges, ut it lso suggests n lgorithm for minimising DFAs. Before we ttempt to descrie this lgorithm, we wish to mke few definitions tht will shorten nd simplify the lter discussion. Definition 4.1 (prtition, lock) Let S e set. A (finite) prtition of S is set P of mutully disjoint non-empty sets {S 1,...,S k }, such tht S i S, where 1 i k, nd the union of the sets in the prtition equls the whole of S. Formlly, k S i = S. i=1 The components S 1,...,S k re clled the locks or prts of P. If s S, then we denote y [s] P the unique lock of P to which s elongs. When P is not importnt, or follows from the context, we omit the suscript of [s] P. In the proof of Theorem 3.1, it is estlished tht if L is regulr lnguge, nd the index of L is i L N, then it is oth necessry nd sufficient for DFA to hve i L sttes in order to recognise L correctly. Now, let A = (Q,Σ,δ,q 0,F) e DFA recognising the regulr lnguge L(A). Furthermore, let [q], where q Q, e the set of strings {x x Σ δ(q 0,x) = q}. We note tht {[q] q Q} 5
prtitions Σ. We oserve tht for every q Q, nd x,y [q], the equivlence x L(A) y holds nd drw the conclusion tht the prtition imposed y Q is refinement of the prtition imposed y L(A). This leds us to elieve tht if the the numer of sttes of n utomton A exceeds the numer of equivlence clsses of L(A), then the utomton cn e reduced in size without ltering the lnguge tht it recognises. A first ttempt to minimise A could e to find, for ech equivlence clss C in the prtition of Σ tht follows from L(A), the suset Q of Q such tht q Q [q ] = C nd merge the sttes of Q into single stte. The prolem with this pproch is tht to e solutely sure tht two sttes p nd q cn e merged, we must exmine δ(p,w) nd δ(q,w) for every string in {w w Q } Σ. A more efficient method is to strt y ssuming tht ll ccepting sttes cn e merged into one, nd ll rejecting into nother, nd then refine this initil prtition until it is stle. By stle, we men tht if sttes p nd q oth elong to lock B, then for ll σ Σ, the stte p goes to lock B on σ, if nd only if q goes to B on σ. If the refinement process is llowed to continue long enough, then the prtition will eventully ecome set of singletons, in which cse it is stle nd termintion gurnteed. The minimistion lgorithm tht stems from the ltter method is listed in Algorithm 1 on pge 10. Although it is ovious tht miniml utomton my not contin unrechle sttes, the prtitioning procedure lone will not purge such sttes from the input utomton A. Insted, the removl of unrechle sttes is done in RemoveUnrechleSttes, while Collpse merges the sttes of A in ccordnce with the prtitioning produced y RefinePrtition. q 1 q 2 q 5 q 6 q 3 q 4 Figure 3: The DFA tht is given s input to Algorithm 1 in Exmple 4.2. Exmple 4.2 We shll now illustrte how RefinePrtition of Algorithm 1 finds the corsest stle prtition with running exmple. The input DFA with lphet Σ = {,} is given in Figure 3, nd the reder should note tht ll of its sttes re rechle from the initil stte q 1. The lgorithm egins with the trivil prtition, P 0 = {I, II}, where I = {q 1,q 3,q 4,q 5 }, nd II = {q 2,q 6 }, nd determines, for ech q Q nd σ Σ, the lock [δ(q,σ)] P0. 6
I: [δ(q 1,)] = [q 2 ] = II [δ(q 1,)] = [q 5 ] = I [δ(q 3,)] = [q 4 ] = I [δ(q 3,)] = [q 3 ] = I [δ(q 4,)] = [q 4 ] = I [δ(q 4,)] = [q 3 ] = I [δ(q 5,)] = [q 6 ] = II [δ(q 5,)] = [q 1 ] = I II: [δ(q 2,)] = [q 3 ] = I [δ(q 2,)] = [q 6 ] = II [δ(q 6,)] = [q 5 ] = I [δ(q 6,)] = [q 2 ] = II The prtition P 0 is not stle, s q 1 nd q 5 goes to rejecting stte on, while q 3 nd q 4 goes to n ccepting stte on the sme input symol. As the sttes in II ehve identiclly on oth input symols, only I hs to e refined, yielding the next prtition in sequence P 1 = {I, II, III}, where I = {q 1,q 5 }, II = {q 2,q 6 }, nd III = {q 3,q 4 }. P 1 is lso unstle, which the lgorithm determines y exmining the ehviour of ll q Q for ll input symols. I: [δ(q 1,)] = [q 2 ] = II [δ(q 1,)] = [q 5 ] = I [δ(q 5,)] = [q 6 ] = II [δ(q 5,)] = [q 1 ] = I II: [δ(q 2,)] = [q 3 ] = III [δ(q 2,)] = [q 6 ] = II [δ(q 6,)] = [q 5 ] = I [δ(q 6,)] = [q 2 ] = II III: [δ(q 3,)] = [q 4 ] = III [δ(q 3,)] = [q 3 ] = III [δ(q 4,)] = [q 4 ] = III [δ(q 4,)] = [q 3 ] = III In this itertion, the only sttes tht elong to the sme lock nd disgrees re q 2 nd q 6 of lock II. After II is divided into {q 2 } nd {q 6 }, prtition P 2 = {I, II, III, IV}, where I = {q 1,q 5 }, II = {q 2 }, III = {q 3,q 4 } nd IV = {q 6 } remins. When deciding whether P 2 is stle, the lgorithm could in theory tke shortcut nd only exmine lock I, s it contins ll sttes tht were mpped to lock III P1 in the previous itertion, nd is hence the only lock tht could hve een ffected y the refinement of III P1. Indeed, it turns out tht I hs indeed ecome inconsistent. I: [δ(q 1,)] = [q 2 ] = II [δ(q 1,)] = [q 5 ] = I [δ(q 5,)] = [q 6 ] = IV [δ(q 5,)] = [q 1 ] = I Block I is consequently split, yielding P 3 = {I, II, III, IV, V }, where I = {q 1 }, II = {q 2 }, III = {q 3, q 4 }, IV = {q 6 } nd V = {q 5 }. The lgorithm would need nother itertion to notice tht the prtition does not chnge nd tht it is done, ut we cn estlish this y simple oservtion. 7
The only locks tht could hve een ffected y the lst refinement re I, II nd IV, ut these re ll singletons nd cnnot e refined further in ny cse. It ppers tht the originl DFA ws close to miniml, s the minimised version contins only one stte less. If we nlyse the utomton, we see tht it recognises the lnguge L, given y L = L 1 L 2, where L 1 = {w Σ w is even}, nd L 2 = {w Σ w = uv, where u is even, nd u is odd.}, where w denotes the numer of occurences of in w. In the light of this, sttes q 1,q 2,q 5 nd q 6 ll rememer unique spect of the the input string w red so fr. Stte q 1 rememers tht w hs n even numer of s nd s, q 2 tht it hs nd odd numer of s nd s, while q 5 nd q 6 covers the remining two comintions. As these four sttes re ll oth useful nd distinct, no two of the cn e merged. Sttes q 3 nd q 4 re oth ccepting nd rememer tht w cn e divided in the mnner required y the definition of L 2. Since q 3 nd q 4 re equivlent in ll spects tht mtter, they cn e merged. 4.1 Distinguishing strings One could esily modify Algorithm 1 to generte, for ech (rechle) stte q of the input utomton A, string which distinguishes it from every stte q [q] L(A) : With every stte q, we ssocite string s q Σ. When the lgorithm first enters RefinePrtition, ll ssocited strings re empty. Every time lock B is found to e unstle with respect to some input symol nd is consequently prtitioned into smller locks, the symol which forced the split is ppended to s q for ll q tht elonged to B. Eventully, the lgorithm termintes y outputting the DFA A = (Q,Σ,δ,q 0,F ), the sttes of which re the equivlence clsses of L(A). It then holds for every pir of sttes [q 1 ],[q 2 ] Q, tht s q, where q [q 1 ] [q 2 ], is distinguishing string for [q 1 ],[q 2 ]. As finl oservtion, we note tht the set {q s q Q} is prefixed-closed nd thus forms tree over Σ. 5 Further reding In 1971, Hopcroft presented minimistion lgorithm for DFAs tht runs in time O(n log n), where n is the numer of sttes of the input lgorithm. Hopcrofts lgorithm is similr to the one presented in this chpter, ut uses the so-clled process the smller hlf -strtegy to void unnecessry comprisons [Hop71]. During the fll semester of 2005, course covering this topic nd the ckground topics were given t the deprtment of Computer Science t the University of Illinois y Prof. Viswnthn. The lecture notes from tht course re quite extensive nd my e suitle s further reding [Vis05]. 8
References [Hop71] J. E. Hopcroft. An n log n lgorithm for minimizing sttes in finite utomton. Theory of Mchines nd Computtions, 1971. [Ner58] A. Nerode. Liner utomton trnsformtions. In In Proc. of the Americn Mthemticl Society 9, pges 541 544, 1958. [Vis05] Professor Mhesh Viswnthn. Lecture notes. CS475: Automt, Forml Lnguges, nd Computtionl Complexity http://www.cs.uiuc.edu/clss/f05/cs475/lectures/, 2005. 9
Input: deterministic finite utomton A Output: the miniml DFA recognising L(A) 01: procedure Minimise (DFA A) 02: A := RemoveUnrechleSttes (A); 03: (Q, Σ, δ, q 0, F) := A ; 04: P := RefinePrtition (A, {Q \ F, F }); 05: return Collpse (A, P); 01: procedure RemoveUnrechleSttes (DFA (Q, Σ, δ, q 0, F)) 02: Q := q 0 ; 03: while q Q such tht q is not mrked 04: choose q Q such tht q is not mrked; 05: mrk q; 06: for ech Σ 07: if δ(q, ) Q 08: dd δ(q, ) to Q ; 09: δ (q, ) := δ(q, ); 10: 11: return (Q, Σ, δ, q 0, F Q ); 01: procedure RefinePrtition (DFA A = (Q, Σ, δ, q 0, F), Prtition P) 02: P := ; 03: for ech q in Q 04: os q := ; 05: for ech symol in Σ 06: dd (, [δ(q, )] P ) to os q ; 07: dd q to [os q ] P ; 08: 09: if P is equl to P 10: return P ; 11: else 12: return RefinePrtition (A, P ); 01: procedure Collpse (DFA A = (Q, Σ, δ, q 0, F), Prtition P) 02: for ech q Q 03: dd [q] P to Q ; 04: for ech Σ 05: δ ([q] P, ) := [δ(q, )] P ; 06: 07: for ech q F 08: dd [q] P to F ; 09: 10: return (Q, Σ, δ, [q 0 ] P, F ); Algorithm 1: A minimistion lgorithm for DFAs. 10