3. Finite automata and regular languages: theory

Transcription

1 series-prllel-loop construction (3.5, 3.6) -free DFA RE -hull (3.3) power set (3.4) dynmic progrmming (3.6) 3. Finite utomt nd regulr lnguges: theory jn Among the vrious forml lnguges tht rise nturlly in the theory of utomt nd phrse structure grmmrs, regulr lnguges re the simplest nd rguly the most useful. They re universlly used in text serching pplictions, nd they pper s importnt components of prcticlly ll the forml nottions developed y computer science, such s those prts of progrmming lnguges tht cn e processed y lexicl nlysis. Regulr lnguges cn e defined in three distinct wys: s the lnguges ccepted y deterministic finite utomt (DFA), y non-deterministic finite utomt (), nd y regulr expressions (RE). Ech of these formlisms sheds light on different properties of regulr lnguges. The min gol of this chpter is to show the equivlence of DFAs, s, nd regulr expressions, using the cyclic reduction shown elow, where X Y mens whtever cn e expressed y X, cn lso e expressed y Y. Since s re generliztion of DFAs, denoted y s et inclusion, it is cler tht they re t lest s powerful s DFAs - the interesting comprisons re those indicted y. From the equivlence of these different models of computtion, ll the mjor properties of regulr lnguges follow redily. series-prllel-loop construction (3.5, 3.6) -free DFA RE -hull (3.3) power set (3.4) dynmic progrmming (3.6) 3. Vrieties of utomt: concepts, definitions, terminology Nottion: Alphet A, e.g. A = {,,..}. Kleene str A* = { w w =... m, m, i A}. A + = AA*. Nullstring. Empty set {} or Ø. Lnguge L A*. Set S = {s, s,.., s, s,..}. Crdinlity S. Power set S. not or complement. Deterministic Finite Automton (FA, DFA) M = (S, A, f, s, F) Set of sttes S, lphet A, trnsition function f: S x A -> S, initil stte s, ccepting or finl sttes F S Extend f from S x A -> S to f: S x A* -> S s follows: f(s, ) = s, f(s, w) = f( f(s, w), ) for w Α Df: M ccepts w A* iff f(s, w) F. Set L A* ccepted y M: L(M) = { w f(s, w) F}. Non-deterministic Finite Automton () with -trnsitions: f: S x (A {}) -> S. Specil cse: without -trnsitions: f: S x A -> S. Extend f: S x A* -> S : f(s, ) = -hull of s = ll sttes rechle from s vi -trnsitions (including s); f(s, w) = f(s, ) for s f(s, w). Extend f further f: S x A* -> S s follows: f(s,.., sk, ) = f(si, ) for i =,.., k. Df: M ccepts w A* iff f(s, w) F {}. Notice: w is ccepted iff some w-pth from s fo F. Set L A* ccepted y M: L(M) = { w f(s, w) F {}}. The opertion of non-deterministic mchine cn e interpreted in equivlent wys: - Orcle: For ny given word w, if there is ny sequence of trnsitions tht leds to n ccepting stte, the mchine will mgiclly find it, like sleep wlker, voiding ll the pths tht led to rejecting stte. - Concurrent computtion: The mchine spwns multiple copies of itself, ech one trcing its own root-to-lef

2 pth of the tree of ll possile choices. If ny copy reches n ccepting stte, it rodcsts success. Non-determinism yields n exponentil increse in computing power! Df: Two FAs (of ny type) re equivlent iff they ccept the sme lnguge. HW 3.: Design ) n nd ) DFA tht ccepts the lnguge L = ( )* of ll strings tht terminte in. Given n ritrry string w over {, }, descrie generl rule for constructing ) n N(w) nd ) DFA M(w) tht ccepts the lnguge L(w) = ( )*w. 3. Exmples: the cse for non-determinism Ex : FAs cn t count, or more precisely, DFA M cn count up to constnt which is t most equl to the numer M of its sttes. The fixed, finite memory cpcity is the most importnt property of FAs, nd it severely limits their computing power. As n exmple, we show tht no FA cn recognize L = { k k k > }. By wy of contrdiction, ssume FA M tht ccepts L, nd denote M s numer of sttes y M = n. In the course of ccepting w = n n, s M reds the prefix n, it goes thru n+ sttes s, s,.., sn. By the pigeon hole principle, some sttes si, sj in this sequence must e equl, si = sj, i < j. Thus, M cnnot distinguish the prefixes i nd j, nd hence lso ccepts, incorrectly, w = n -(j-i) n, nd mny other ill-formed strings. Contrdiction, QED. Ex : Creful out cn t count! Let L = {w { }* () = () }. In ny w, ups = nd downs = lternte, so () = () ±. Solution: L = {w first chrcter = lst chrcter } "" Exercise: We sw -stte fsm seril dder. Show tht there is no fsm multiplier for numers of ritrry size. DFAs re conceptully strightforwrd: when in stte s you red input, go to stte s (nd perhps produce some output. s cn e considerly trickier, s the numer of ction sequences they might produce explodes. Three min resons rgue for introducing non-determinism despite the dded complictions. The most generl reson is tht non-deterministic lgorithms re interesting nd importnt, nd the theory should e le to model them. A second reson ecomes pprent when developing the complexity theory towrds the end of this course, where the P vs. NP question reltes the two most importnt complexity clsses. The third reson ecomes pprent in the course of this chpter: s re very convenient technique to desing FAs nd prove their properties. Ex 3: s clirvoynce yields n exponentil reduction of the size of the stte spce s compred to DFAs. Consider L k = ( )* ( ) k- i.e. ll strings whose k-th lst it is. A ccepts L k using only k+ sttes (s shown for k = nd k = 3) y guessing where the til-end k its strt., L s s,, L3 s s,, A DFA tht ccepts L k must contin shift register k its long, hence hs t lest k sttes. This shows tht, in generl, simulting y DFA requires n exponentil increse in the size of the stte spce. The following DFA M(L ) hs stte for ech of the -it suffixes,,,. Ech stte s corresponds to lnguge, i.e. set of words tht led M from its initil stte to s. The short strings,, cn e ssocited with some set of long strings with the following semntics:,, *: no hs een seen tht might e useful s the next-to-lst it, *: the current it is, if this turns out to e the next-to-lst it, we must ccept *: ccept if this is the end of the input; if not, go to stte * or * depending on the next it red *: ccept if this is the end of the input; if not, go to stte * or * depending on the next it red

3 *, * * * This DFA M(L ) is suset of of DFA M(L ) with 8 sttes tht is otined from the generl construction of section 3.4 to simulte N y some DFA M. 3.3 Spontneous trnsitions: convenient, ut not essentil Lemm (-trnsitions): Any N with -trnsitions cn e converted to n equivlent M without -trnsitions. The sic ide is simple, s illustrted elow. An -trnsitions from r to s implies tht nything tht N cn do strting in s, such s n -trnsition from s to t, N cn lredy do strting in r. After dding pproprite new trnsitions, such s the -trnsition from r to t, the -trnsition from r to s cn e deleted. r s t r s t Df: the -closure E(s) of stte s is the set of sttes tht cn e reched from s following -trnsitions. In the exmple ove: E(r) = {r, s}, E(s) = {s}, E(t) = {t}. The significnce of this concept is due to the fct tht whenever N reches s, it might lso rech ny stte in E(s) without reding ny input. The following exmple Ex4 illustrtes the generl construction tht trnsforms N with -trnsitions (t left) into n equivlent M without -trnsitions (t right). Ex 4: L = { i j k i, j, k } = ***. This lnguge is typicl of the structure of communiction protocols. A messge consists of prefix, ody, nd suffix, in this order. If ny prt my e of ritrry length, including zero length, the lnguge of legl messges hs the structure of L. N s s M E(s) E(s) E() Replce ech stte s of N y stte E(s) of M: E(s) = {s, s, }, E(s) = {s, }, E() = {}. This reflects the fct tht whenever N got to s, it might spontneously hve proceeded to s or to. E(s) is the strting stte of M. Any stte E(s) of M tht contins n ccepting stte of N must e mde n ccepting stte of M - in our exmple, ll the sttes of M re ccepting! Finlly, we djust the trnsitions: since E(s) = {s, s, }, M s trnsition function f ssigns to f( E(s), ) the union of ll of N s trnsitions outgoing from s, s, : f( s, ), f( s, ), f(, ). Despite the fct tht ll of M s sttes re ccepting, M only ccepts the strings in L. The first symol of the string, for exmple, leds M from E(s) to E(), ut E() cnnot process the secons symol, hence is not ccepted. Ex 5: N converted to n -free M. E(s) = {s, }. L = ( * ( ) )* N s, M E(s),

4 3.4 DFA simulting : the power set construction. Thm (equivlence -DFA): Any N cn e converted into n equivlent DFA M. Pf: Thnks to Lemm on -trnsitions, ssume without loss of generlity tht N = (S, A, f, s, F) hs no - trnsitions. Define M = ( S, A, f, {s}, F ) s follows. S is the power set of S, {s}the initil stte. F consists of ll those susets R S tht contin some finl stte of N i.e. R F {}. f : S x A -> S is defined s: for R S nd A, f (R, ) = { s S s f(r, ) for some r R}. N nd M re equivlent due the following invrint: for ll x A*, f ({s}, x) = f(s, x). QED Ex (modified from Sipser p57-58). Convert the of Ex 5 (t right) in section 3.3 to n equivlent DFA., { }, 3, 3, 3,, 3 The power set construction tends to introduce unrechle sttes. These cn e eliminted using trnsitive closure lgorithm. Alterntively, we generte sttes of M only s the corresponding susets of S re eing reched, thus comining trnsitive closure with the construction of the stte spce. For ese of comprison, let s redrw the DFA ove without unrechle sttes, with lyout similr to Ex5. The non-determinism in exmple Ex5: f(, ) = {, } is resolved y introducing sttes {, } nd { s,, }: M 3,3,,3 { } 3.5 Regulr expressions Df: Given n lphet A, the clss R(A) of regulr expressions over A is otined s follows: Primitive expressions: for ever A, (nullstring), Ø (empty set). Compound expressions: if R, R re regulr expressions, (R R ), (R R ), (R*) re regulr expressions. Convention on opertor priority: * > >. Use prentheses s needed to define structure. A regulr expression denotes regulr set y ssociting the expression opertors *,, with the set opertions Kleene str, ctention, nd union, respectively. Thm: A set L is regulr iff L is descried y some regulr expression. Pf <=: Convert given regulr expression R into n N. Use trivil s for the primitive expressions, the constructions used in the Closure Thm for the compound expressions. QED

5 Pf =>( McNughton, Ymd 96. Compre: Wrshll s trnsitive closure lgorithm, 96): Let DFA M = (S, A, f, s, F) ccept L. S = { s,.., sn}. Define R k ij = the set of ll strings w tht led M from stte si to stte sj without pssing thru ny stte with lel > k. Initilize: R ij = { f(si, ) = sj } for i j. R i i = { f(si, ) = si { }. Induction step: R k ij = R k- ij R k- ik (R k- kk )* R k- kj Termintion: R n ij = the set of ll strings w tht led M from stte si to stte sj without ny restriction. L(M) = R n j for ll sj F. The right hnd side is regulr expression tht denotes L(M). QED Intuitive verifiction. Rememer Wrshll s trnsitive closure nd Floyd s ll distnces lgorithms. Wrshll Floyd B i j = Ai j djcency mtrix, B i i = true B i j = Ai j edge length mtrix, B i i = B k i j = B k- i j or ( B k- ik nd B k- kj ) B k i j = min ( B k- i j, B k- ik + B k- kj ) B n i j = Ci j connectivity mtrix B n i j = Di j distnce mtrix In Wrshll s nd Floyd s lgorithms, cycles re irrelevnt for the issue of connectedness nd hrmful for computing distnces. Regulr expressions, on the other hnd, descrie ll pths in grph (stte spce), in prticulr the infinitely mny cyclic pths. Thus, we dd loop R k- kk in the Fig. t right, nd insert the regulr expression (R k- kk )* etween R k- ik nd R k- kj. B k- ij { },,.., k- R k- ij { } R k- kk i B k- ik k B k- kj j i R k- ik k R k- kj j 3.6 Closure of the clss of regulr sets under union, ctention, nd Kleene str Df: A lnguge (or set) L A* is clled regulr iff L is ccepted y some FA. It turns out tht ll FAs (DFA or, with or without -trnsitions) re equivlent w.r.t. ccepting power. Given L, L A*, define union L L, ctention L L = { v = ww w L, w L }. Define L = { }, L k = L L k- for k >. Kleene str: L* = (k = to ) L k. Thm (closure under the regulr opertions): If L, L A* re regulr sets, L L, L L nd L* re regulr sets. Pf: Given FAs tht ccept L, L respectively, construct s to ccept L L, L L nd L* s shown. The given FAs re represented s oxes with strting stte t left (smll) nd one ccepting stte (representtive of ll others) t right (lrge). In ech cse we dd new strting stte nd some -trnsitions s shown. In ddition to closure under the regulr opertions union, ctention, nd Kleene str, we hve: Thm: if L is regulr, the complement L is lso regulr. Pf: Tke DFA M = (S, A, f, s, F) tht ccepts L. M = (S, A, f, s, S-F) ccepts L. QED

6 Thm: If L, L A* re regulr, the intersection L L is lso regulr. Pf: L L = ( L L ). QED Thus, the clss of regulr lnguges over n lphet A forms Boolen lger. 3.7 DFAs nd right invrint equivlence reltions. Stte minimiztion. Ex rel constnts : A = {,,, }. L = ( ( ( ) + ( )* ( )* ( ) + ) )* Interpret word in L s sequence of rel constnts with mndtory inry point, e.g,,. A constnt must hve t lest one it or, the inry point lone is excluded. Constnts re seprted y. To get DFA, imgine tht the trnsitions not shown in the figure ll led to non-ccepting trp stte s5.,,, s4 s s,, Stte identifiction, equivlent sttes: Given the stte digrm of DFA M, devise n experiment to ) determine the current stte of M, or ) to distinguish two given sttes r, s. Ex: In order to identify s, feed into M - no other stte is ccepting. uniquely identifies s. distinguishes etween nd s4. No experiment distinguishes from s4: nd s4 re equivlent. Equivlent sttes cn e merged to otin smller FA M equivlent to M. Df: Sttes r nd s of M re equivlent (indistinguishle) iff for ll w A*, f(r, w) F f(s, w) F. It turns out tht in order to prove sttes equivlent, it suffices to test ll words w of length w n = S. Before proving this result, consider dynmic progrmming lgorithm to identify non-equivlent stte pirs. We strt with the oservtion tht ll sttes might e equivlent. As pirs of non-equivlent sttes re grdully eing identified, we record for ech such pir s, r shortest witness tht distinguishes s nd r. We illustrte this lgorithm using the exmple of the FA rel constnts ove. At left in the figure elow, ll stte pirs si sj re mrked tht cn e distinguished y some word of length. This distinguishes ccepting sttes from non-ccepting sttes, nd is shortest witness. Unmrked slots identify pirs tht hve not yet een proven distinguishle. For ech of these unmrked pirs r, s, nd ll A, check whether the pir f(r, ), f(s, ) hs een mrked distinguishle. If so, mrk r, s distinguishle with shortest witness w = w, where w is inherited from f(r, ), f(s, ). When computing the entry for s, t right, for exmple, notice tht f(s, B) = s, f(, B) = s4. Since s, s4 hve lredy een proven distinguishle y w =, s, re distinguishle y w = B. Checking the lst unmrked pir, s4 t right yields no new distinguishle pir: f(, ) = f(s4, ) = s; f(, ) = f(s4, ) = trp stte s5; f(,b) =, f(s4,b) = s4, ut, s4 hve not yet een proven distinguishle. This termintes the process with the informtion tht, s4 re equivlent nd cn e merged. Distinguishle sttes oviously cnnot e merged -> this is stte minimiztion lgorithm. s s s s4 w = o s s s s4 w = s s s s4 w = Hw 3..: Invent nother interesting exmple of DFA M with equivlent sttes nd pply this dynmic progrmming lgorithm to otin n equivlent M with the minimum numer of sttes. Hw 3.3: Anlyze the complexity of this dynmic progrmming lgorithm in terms of S = n nd A. Hw 3.4: Prove the following Thm: If sttes r, s re indistinguishle y words w of length w n = S, r nd s re equivlent. Hint: use the concepts nd nottions elow, nd prove the lemms.

7 Df r, s re indistinguishle y words of length k : r ~ k s for k iff for ll w A*, w k: f(r, w) F f(s, w) F Importnt properties of of the equivlence reltions ~ k : Lemm (inductive contruction): r ~ k s iff r ~ k- s nd for ll : f( r, ) ~ k- f( s, ) Lemm (termintion): If ~ k = ~ k-, ~ k = ~ m for ll m > k. Thm: If r, s re indistinguishle y words w of length w n = S, r nd s re equivlent. An lgeric pproch to stte minimiztion Given ny L A*, define the equivlence reltion (reflexive, symmetric, trnsitive) R L over A*: x R L y iff All z A*, xz L yz L. I.e., either xz nd yz oth in L, or xz nd yz oth in L Notice: R L is right invrint : x R L y All z A*, xz R L yz. Intuition: x R L y iff the prefixes x nd y cuse ll pirs xz, yz to shre [non-]memership sttus w.r.t. L. Given DFA M, define equivlence reltion R M over A*: x R M y iff f(s, x) = f(s, y). R M is right invrint. Df: index of equivlence reltion R = of equivlence clsses of R. Thm (regulr sets nd equivlence reltions of finite index). The following 3 sttements re equivlent: ) L A* is ccepted y some DFA M ) L is the union of some of the equivlence clsses of right invrint equivlence reltion of finite index 3) R(L) is of finite index. Thm: The minimum stte DFA ccepting L is unique up to isomorphism (renming sttes). In contrst, minimum stte s re not necessrily unique. Ex A ={, }, L = + : non-deterministic deterministic, 3.8 Odds nd ends out regulr lnguges nd FAs The pumping lemm (itertion lemm): For ny regulr L A* there is n integer n > (the pumping length ) with the following property: ny w L of length w n cn e sliced into 3 prts w = xyz stisfying the following conditions: ) xy n, ) y >, 3) for ll i, x y i z L. Pf: Consider ny DFA M tht ccepts L, e.g. the minimum stte DFA M(L). Choose n = S s the pumping length. Feed ny w L of length w n into M. On its wy from s to some ccepting stte, M goes through w + n + sttes. Among the first n+ of these sttes, s, s,.., sn,.., there must e duplicte stte si = sj for some i < j, with loop leled y leding from si ck to si. Thus, xz, xyz, xyyz,... re ll in L. QED The pumping lemm is used to prove, y contrdiction, tht some lnguge is not regulr. Ex: L = { i j i < j} is not regulr. Assume L is regulr. Let n e L s pumping length. Consider w = n n+, w L. Even if we don t know how to slice w, we know xy n nd hence y = k for some k >. But then n n+, n+k n+, n+k n+,.. re ll L, contrdicting the definition L = { i j i < j}. L is not regulr, QED. End of Ch3