8 Automata and formal languages. 8.1 Formal languages

Transcription

1 8 Automt nd forml lnguges This exposition ws developed y Clemens Gröpl nd Knut Reinert. It is sed on the following references, ll of which re recommended reding: 1. Uwe Schöning: Theoretische Informtik - kurz gefsst. 3. Auflge. Spektrum Akdemischer Verlg, Heidelerg, ISBN PROSITE user mnul 3. Sigrist C.J., Cerutti L., Hulo N., Gttiker A., Flquet L., Pgni M., Biroch A., Bucher P.. PROSITE: documented dtse using ptterns nd profiles s motif descriptors. Brief Bioinform. 3: (2002). We will present sic fcts out: forml lnguges, regulr nd contex-free grmmrs, deterministic finite utomt, nondeterministic finite utomt, pushdown utomt. 8.1 Forml lnguges An lphet Σ is nonempty set of symols (lso clled letters). In the following, Σ will lwys denote finite lphet. A word over n lphet Σ is sequence of elements of Σ. This includes the empty word, which contins no letters nd is denoted y ε. For ny lphet Σ, the set Σ is defined to e the set of ll words over the lphet Σ. The set Σ + := Σ \ {ε} contins ll nonempty words Σ. nd E. g., if Σ = {, }, then Σ = {ε,,,,,,,,,...} Σ + = {,,,,,,,,...}. The length of word x is denoted y x. For words x, y Σ we denote their conctention y xy: If x = x 1,... x m nd y = y 1,..., y n (where x i, y j Σ) then xy = x 1,..., x m, y 1,..., y n. For x Σ let x n := xx... x. (Tht is, n conctented copies of x). } {{ } n

2 Finite utomt nd regulr grmmrs, y Clemens Gröpl, Jnury 9, 2014, 12: Thus x n = n x, xy = x + y, nd ε = 0. A (forml) lnguge A over n lphet Σ is simply set of words over Σ, i. e., suset of Σ. The empty lnguge := {} contins no words. (Note: The empty lnguge must not to e confused with the lnguge {ε}, which contins only the empty word.) The complement of lnguge A (over n lphet Σ) is the lnguge Ā := Σ \ A. For lnguges A, B we define their product s using the conctention opertion defined ove. AB := {xy x A, y B}, E. g., if A = {, h}, B = {ε, t, ttu} then AB = {, h, t, ht, ttu, httu}. The powers of lnguge L re defined y L 0 := {ε} L n := L n 1 L, for n 1. The Kleene str (or Kleene hull) of lnguge L is L := L n. i=0 Funnily, n = for n 1, ut = 0 = {ε}, ccording to these definitions. But (defined this wy) lnguge exponentition works s expected: For every lnguge A nd m, n 0, we hve A m A n = A m+n. Most forml lnguges re infinite ojects. In order to del with them lgorithmiclly, we need finite descriptions for them. There re two pproches to this: Grmmrs descrie rules how to produce words from given lnguge. We cn clssify lnguges ccording to the kinds of rules which re llowed. Automt descrie how to test whether word elongs to given lnguge. We cn clssify lnguges ccording to the computtionl power of the utomt which re llowed. Fortuntely, the two pproches cn e shown to e equivlent in mny cses. 8.2 Grmmrs A grmmr is 4-tuple G = (V, Σ, P, S) stisfying the following conditions. V is finite set of vrile symols. For revity, the vrile symols re often simply clled vriles, or nonterminls.

3 8002 Finite utomt nd regulr grmmrs, y Clemens Gröpl, Jnury 9, 2014, 12:48 Σ is finite set of terminl symols, lso clled the terminl lphet. This is the lphet of the lnguge we wnt to descrie. We require tht vrile symols nd terminl symols cn e distinguished, i. e., V Σ =. P is finite set of rules or productions. A rule hs the form where lhs (V Σ) + nd rhs (V Σ). S V is the strt vrile. (left hnd side) (right hnd side), We cn think of the productions of grmmr s wys to trnsform words over the lphet V Σ into other words over V Σ. We cn derive v from u in G in one step if there is production y y in P nd x, z (V Σ) such tht u = xyz nd v = xy z. This is denoted y u G v. If the grmmr is cler from the context, we write just u v. The reflexive nd trnsitive closure G of G is defined s follows. We hve u v if nd only if u = v or v cn G e derived from u in series u w 1 w 2... w n v of steps using the grmmr G. Then L(G) := {w Σ S G w} is the lnguge generted y G. Note: The crucil point is tht we wrote w Σ, not w (V Σ). Vriles re not llowed in generted words which re output. (Note 2: This rises nother question: Productions cn lengthen nd shorten the words. How cn we tell how long it will tke until we hve removed ll vrile symols? Well, tht s nother story.) Exmple 1. The following grmmr genertes ll words over Σ = {, } with eqully mny s nd s. G = ({S}, Σ, P, S), where P := { } S ε, S S, (Cn you prove this?) Exmple 2. The following grmmr genertes well-formed rithmetic expressions over the lphet Σ = {(, ),,, c, +, }.

4 Finite utomt nd regulr grmmrs, y Clemens Gröpl, Jnury 9, 2014, 12: G = ({A, M, K}, Σ, P, A), where P := { } A M, A A + M M K, M M K, K, K, K c, K (A), Chomsky descried four sorts of restrictions on grmmr s rewriting rules. The resulting four clsses of grmmrs form hierrchy known s the Chomsky hierchy. In wht follows we use cpitl letters A, B, W, S,... to denote nonterminl symols, smll letters,, c,... to denote terminl symols nd greek letters α, β, γ,... to represent string of terminl nd non-terminl letters. 1. Regulr grmmrs. Only production rules of the form W W or W re llowed. 2. Context-free grmmrs. Any production of the form W α is llowed. 3. Context-sensitive grmmrs. Productions of the form α 1 Wα 2 α 1 βα 2 re llowed. 4. Unrestricted grmmrs. Any production rule of the form α 1 Wα 2 γ is llowed. 8.3 Regulr grmmrs For Bioinformtics we will e interested in the regulr nd context-free grmmrs. definition: Hence more detiled A grmmr is clled regulr if ll productions hve the form l r, where l V nd r Σ ΣV. Tht is, we cn only replce vrile with: terminl (r Σ), or terminl followed y vrile (r ΣV). Exmple. The following regulr grmmr genertes vlid identifier nmes in mny progrmming lnguges. G = ({[lph], [lnum]}, {A,... Z,,..., z,, 0, 1,..., 9}, P, [lph]}, where P := { [lph] A,...,, [lph] A[lnum],..., [lnum], [lnum] A,...,, 0,..., 9, [lnum] A[lnum],..., [lnum], 0[lnum],..., 9[lnum] } Here we used comms to write severl productions shring the sme left hnd side in one line.

5 8004 Finite utomt nd regulr grmmrs, y Clemens Gröpl, Jnury 9, 2014, 12: Context-free grmmrs Here the definition of context-free grmmr: Definition 1. A context free grmmr G is 4-tuple G = (V, Σ, P, S) with V nd Σ eing lphets with V Σ =. V is the nonterminl lphet. Σ is the terminl lphet. S N is the strt symol. P V (V Σ) is the finite set of ll productions. Consider the context-free grmmr G = ( {S}, {, }, {S S S }, S ). This CFG produces the lnguge of ll plindromes of the form αα R. For exmple the string cn e generted using the following derivtion: S S S S. The plindrome grmmr cn e redily extended to hndle RNA hirpin loops. For exmple, we could model hirpin loops with three se pirs nd gc or g loop using the following productions. S W 1 u cw 1 g gw 1 c uw 1, W 1 W 2 u cw 2 g gw 2 c uw 2, W 2 W 3 u cw 3 g gw 3 c uw 3, W 3 g gc. (We don t mention the lphets V nd Σ explicitly if they re cler from the context.) Grmmrs generte lnguges. They re mens to quickly specify ll (possily n infinite numer) words in lnguge. Now we will turn the ttention to the utomt tht cn decide whether word is in the lnguge or not. If the word is in the lnguge the utomton ccepts the word. We strt with finite utomt nd proove tht they re le to ccept exctly the words generted y regulr grmmr. 8.5 Deterministic finite utomt A deterministic finite utomton (DFA) is 5-tuple M = (Z, Σ, δ, z 0, E) stisfying the following conditions. Z is finite set of sttes the utomton cn e in. Σ is the lphet. The utomton moves long the input from left to right. In ech step, it reds single chrcter from the input. δ : Z Σ Z is the trnsition function. When the chrcter hs een red, M chnges its stte depending on the chrcter nd its current stte. Then it proceeds to the next input position.

6 Finite utomt nd regulr grmmrs, y Clemens Gröpl, Jnury 9, 2014, 12: z 0 is the initil stte of M efore the first chrcter is red. E is the set of end sttes or ccepting sttes. If M is in stte contined in E fter the lst letter hs een red, the input is ccepted. DFAs cn e drwn s digrphs very intuitively. Sttes correspond to vertices. They re drwn s single circles; ccepting sttes re indicted y doule circles. Edges correspond to trnsitions nd re leled with letters from Σ. There is n rc from u to v leled if nd only if there is trnsition δ(u, ) = v. The initil stte is mrked y n ingoing rrow. Exmple. The following utomton ccepts the lnguge (where % denotes modulus, i. e. reminder of division) L = {x {, } (# (x) # (x)) % 3 = 1}. z 0 z 1 z 2 Using the definition of DFA, we hve M = ({z 0, z 1, z 2 }, {, }, δ, z 0, {z 1 }), where δ(z 0, ) = z 1 δ(z 1, ) = z 2 δ(z 2, ) = z 0 δ(z 0, ) = z 2 δ(z 1, ) = z 0 δ(z 2, ) = z 1 A lnguge L is clled regulr if there is regulr grmmr tht produces L \ {ε}. Lengthy remrk: The issue with ε is relly just technicl compliction. We cn lwys modify grmmr G tht genertes lnguge L into grmmr G tht genertes the lnguge L {ε} y the following trick: Let S e the strt vrile of G. Let S e new vrile symol not used y G. Then G is otined y replcing the strt vrile y S nd dding the following productions: S S ε. Whether the resulting grmmr G is lso clled regulr (if G ws regulr) depends on the literture. Schöning uses the following ε-sonderregelung : If ε L(G) is desired, then the production S ε is dmitted, where S is the strt vrile. However, in this cse S must not pper on the right hnd side of production. 8.6 From DFAs to regulr grmmrs Agin, let M = (Z, Σ, δ, z 0, E) e deterministic finite utomton. It is useful to extend the trnsition function δ : Z Σ Z to mpping δ : Z Σ Z, clled the extended trnsition function. We define δ (z, ε) := z for every stte z Z nd inductively, δ (z, x) := δ(δ (z, x), ) for x Σ, Σ.

7 8006 Finite utomt nd regulr grmmrs, y Clemens Gröpl, Jnury 9, 2014, 12:48 Oserve tht if x = x[1.. n] is n input string, then δ (z 0, x[1.. 0]), δ (z 0, x[1.. 1]),..., δ (z 0, x[1.. n]) is the pth of sttes followed y the DFA. Hence, the lnguge ccepted y M is L(M) := {x Σ δ (z 0, x) E}. We re now redy to prove: Theorem 2. Every lnguge which is ccepted y deterministic finite utomton is regulr. Proof: Let M = (Z, Σ, δ, z 0, E) e DFA nd A := L(M). We will construct regulr grmmr G = (V, Σ, P, S) tht genertes A. We let V := Z nd S := z 0. Every rc δ(u, ) = v ecomes production u v P, nd if v E we lso include production u. Tht is, P := {u v δ(u, ) = v} {u δ(u, ) = v E}. Now we hve x[1.. n] L(M) there re sttes z 1,..., z n Z such tht δ(z i 1, x[i]) = z i for i = 1,..., n, where z 0 is the strt stte nd z n E is n ccepting stte there re vriles z 1,..., z n V such tht z i 1 x[i]z i is production in P, where z 0 is the strt vrile, nd z n 1 x[n] is lso production in P we cn derive S = z 0 x[1]z 1 x[1]x[2]z 2... x[1.. n 1]z n 1 x[1.. n] in G, i. e., S G x x[1.. n] L(G). If ε A, i. e., z 0 E, then we need to pply the ε-sonderregelung nd modify G ccordingly. 8.7 Nondeterministic finite utomt In DFAs, the pth followed upon given input ws completely determined. Next we will introduce nondeterministic finite utomt (NFAs). These re defined similr to DFAs, ut ech stte cn hve more thn one successor stte for ny given letter, or none t ll. A nondeterministic finite utomton (NFA) is 5-tuple M = (Z, Σ, δ, U 0, E) stisfying the following conditions. Z is finite set of sttes. Σ is the lphet. δ : Z Σ P(Z) is the trnsition function. Here P(Z) is the power set of Z, i. e. the set of ll susets of Z.

8 Finite utomt nd regulr grmmrs, y Clemens Gröpl, Jnury 9, 2014, 12: U 0 is the set of initil sttes. E is the set of ccepting sttes. When the utomton reds Σ nd is in stte z, it is free to choose one of severl sucessor sttes in δ(z, ), or its gets stuck if δ(z, ) =. A nondeterministic finite utomton ccepts nd input if there is t lest one ccepting pth. Agin, we cn define n extended trnsition function δ : P(Z) Σ P(Z). We let δ (U, ε) := U for ll susets of sttes U Z nd inductively, δ (U, x) := δ(v, ) for x Σ, Σ. v δ (U,x) Then the lnguge ccepted y M is L(M) := {x Σ δ (U 0, x) E }. The following illustrtes the definition of δ. The lrge ule on the left side is δ (U, x), the lrge ule on the right side is δ (U, x), where is some letter. The stte spce (ft dots) is shown twice for clrity. Time goes from left to right. The smller cones indicte δ(v, ) for ech v δ (U, x). NFAs cn e drwn s digrphs, similr to DFAs. The resulting digrphs re more generl: We cn hve severl rrows pointing to strt sttes. The numer of rcs with given lel leving vertex is no longer required to e exctly 1, it cn e ny numer (including 0). Exmple. The following NFA ccepts ll words over the lphet Σ = {, } which do not strt or end with the letter.

9 8008 Finite utomt nd regulr grmmrs, y Clemens Gröpl, Jnury 9, 2014, 12:48 z 0 z 1 z 2 z DFAs nd NFAs re equivlent Although NFAs re generliztion of DFAs, they ccept the sme lnguges: Theorem 3 (Rin, Scott). For every nondeterministic finite utomton M there is deterministic finite utomton M such tht L(M) = L(M ). Proof. Let M = (Z, Σ, δ, U 0, E) e n NFA. The sic ide of the proof is to view the susets of sttes of Z s single sttes of n DFA M whose stte spce is P(Z). Then the rest of the definition of M is strightforwrd. The power set utomton is defined s M := (P(Z), Σ, δ, U 0, E ), where P(Z) is the stte spce The trnsition function δ : P(Z) Σ P(Z) is defined y δ (U, ) := δ(v, ) = δ (U, ) for U P(Z). U 0 P(Z) is the new strt stte (note tht in M it ws the set of strt sttes). E := {U Z U E } is the new set of end sttes. v U Using the definitions of M nd M, we hve: x[1.. n] L(M) there re ccepting pths in M, δ (U 0, x[1.. n]) E there re susets U 1, U 2,..., U n Z such tht δ (U i 1, x[i]) = U i nd U n E there is n ccepting pth in M, δ (U 0, x[1.. n]) = U n E x[1.. n] L(M ). Remrks:

10 Finite utomt nd regulr grmmrs, y Clemens Gröpl, Jnury 9, 2014, 12: In the power set construction, we cn sfely leve out sttes which cnnot e reched from the strt stte U 0. Tht is, we cn generte the reched stte sets on the fly. 2. The exponentil low up of the numer of sttes (from Z to 2 Z ) cnnot e voided in generl. For exmple, the lnguge L := {x {, } x k nd the k-lst letter of x is n } hs n NFA with k + 1 sttes, ut it is not hrd to show tht no DFA for L cn hve less tht 2 k sttes. 8.9 From regulr grmmrs to NFAs We hve just seen how to trnsform nondeterministic finite utomton into deterministic finite utomton. We hve seen efore how to trnsform deterministic finite utomton into regulr grmmr. Next we will see how to trnsform regulr grmmr into nondeterministic finite utomton. This concludes the proof tht the regulr lnguges re precisely those which re ccepted y finite utomt (of oth kinds). Theorem 4. Every regulr lnguge is ccepted y nondeterministic finite utomton. Proof. Let G = (V, Σ, P, S) e regulr grmmr nd A := L(G). We will construct n NFA M = (Z, Σ, δ, {z 0 }, E) such tht L(M) = A. Note tht in every derivtion in regulr grmmr, the intermedite words contin exctly one vrile, nd the vrile must e t the end. This vrile will ecome stte of the NFA. We need one more extr stte, which M enters when the vrile is eliminted in the lst step. Thus we let the stte set e Z := V {X}. The only possile initil stte is z 0 := S, the strt vrile of G. The set of end sttes is E := {X} if ε A. If ε A, then we let E := {X, S}. Next we trnslte productions into trnsitions. We define δ : Z Σ P(Z) y Tht is, δ(u, ) v δ(u, ) X iff u v P iff u P δ(u, ) = {v u v P} {X u P}. Note tht the end stte X hs no successor sttes. And if S is n end stte, then y the ε-sonderregelung there is no wy to get ck to S, s it does not pper on the right side of production. Now we hve for n 1: x[1.. n] L(G) there re vriles z 1,..., z n V such tht we cn derive z 0 = S G x[1]z 1 G x[1]x[2]z 2 G... G x[1.. n 1]z n 1 G x[1.. n] in G, tht is, z i 1 x[i]z i is production in P, where z 0 = S is the strt vrile, nd z n 1 x[n] is lso production in P there re sttes z 1,..., z n Z {X} such tht δ(z i 1, x[i]) z i for i = 1,..., n, where z 0 is the strt stte, nd z n = X is only end stte which is fesile for word of length 1 x[1.. n] L(M) Moreover, y construction we hve ε L(M) ε A.

11 8010 Finite utomt nd regulr grmmrs, y Clemens Gröpl, Jnury 9, 2014, 12: Regulr expressions Regulr expressions re perhps the most populr wy to descrie regulr lnguges in forml terms. Introductory Exmples. At UNIX shell, you might type: > ls s*[re]?t*x script.ux script.tex slides.tex Here we hve used the wildcrd chrcters * nd?. The PROSITE dtse contins common motifs of protein fmilies, some of which re descried y ( restricted form of) regulr expressions. The following two lines re from the PROSITE entry with ccession numer PS00518, which contins pttern for zinc finger C3HC4 domin: ID PA ZINC_FINGER_C3HC4; PATTERN. C-x-H-x-[LIVMFY]-C-x(2)-C-[LIVMYA]. The precise syntx used for regulr expressions vries. Here is one. following rules. A regulr expression γ nd the lnguge L(γ) it represents re defined inductively y the 1. is regulr expression. It denotes the empty lnguge. L( ) =. 2. ε is regulr expression. Its lnguge consists of the empty word. L(ε) = {ε}. 3. For ech single letter, Σ, is regulr expression. L() = {}. 4. Let α nd β e regulr expressions. Then the following re lso regulr expressions: (Closure properties) (i) αβ with L(αβ) := L(α)L(β) product (ii) (α β) with L((α β)) := L(α) L(β) union (iii) (α) with L((α) ) := (L(α)) str hull Some oservtions. 1. For every regulr expression α, it holds αε = εα = α nd α = α =. 2. Let x = x[1.. n] Σ. Then x is regulr expression nd L(x) = {x}, y rules 2., 3., nd 4.(i). 3. Let A = {x 1, x 2,..., x k } Σ e finite lnguge. Then (... ((x 1 x 2 ) x 3 )... x k ) is regulr expression nd L((... ((x 1 x 2 ) x 3 )... x k )) = A, y rule 4.(ii) nd the previous oservtion. We shll rther write this expression s (x 1 x 2 x 3... x k ). Exmple 1. The lnguge L = {x {, } x contins s sustring } cn e represented y the following regulr expression: ( ) ( ).

12 Finite utomt nd regulr grmmrs, y Clemens Gröpl, Jnury 9, 2014, 12: Exmple 2. The lnguge L = {x {, } x does not contin s sustring } cn e represented y the following regulr expression: This cn e seen s follows. A DFA for L is ( ) ( ε) z 1 z 2 z 3 We hve δ(z 1, x) = z 1 iff x L(( ) ). After tht we cn mke one more trnsition from z 1 to z 2 reding n Finite utomt nd regulr expressions re equivlent Theorem 5 (Kleene). A lnguge is regulr if nd only if it is descried y regulr expression. (the following proof is not needed for the exmintion. You cn red it t your convenience). Proof. We show oth inclusions in turn. ( ) Let γ e regulr expression. We show tht L(γ) is ccepted y nondeterministic finite utomton. We follow the cses in the inductive definition of regulr expressions. Cses 1., 2., nd 3. re trivil: Clerly there re NFAs for L( ) =, for L(ε) = {ε}, nd for L() = {} (where Σ). Cse 4.(ii): γ hs the form γ = (α β). We cn ssume y induction tht we lredy hve two NFAs M 1 = (Z 1, Σ, δ 1, U 1, E 1 ) nd M 2 = (Z 2, Σ, δ 2, U 2, E 2 ) stisfying L(α) = L(M 1 ), L(β) = L(M 2 ), nd Z 1 Z 2 =. Then we cn simply tke the union of M 1 nd M 2. Tht is, we uild the NFA M := (Z 1 Z 2, Σ, δ 1 δ 2, U 1 U 2, E 1 E 2 ). Here δ 1 δ 2 susumes ll the trnsitions present in M 1 nd M 2. There re no trnsitions etween the M 1 nd M 2 prts in M. Formlly, (δ 1 δ 2 )(U) = δ 1 (U Z 1 ) δ 2 (U Z 2 ). Then L(M) = L(α) L(β) = L(γ). Cse 4.(i): γ hs the form γ = αβ. Here we plug two NFAs M 1 nd M 2 for L(α) nd L(β) in series to otin M. (We cn ssume tht the stte sets re disjoint, Z 1 Z 2 =.) M hs the strt stte set U 1, if ε L(α). If ε L(α), then M hs the strt stte set U 1 U 2. In oth cses, the end stte set of M is E 2.

13 8012 Finite utomt nd regulr grmmrs, y Clemens Gröpl, Jnury 9, 2014, 12:48 Moreover we dd rcs from M 1 to M 2 s follows: For every rc δ 1 (u, ) v in M 1 tht enters stte in E 1, we dd couple of rcs δ(u, ) s 2, for ll s 2 U 2, in M. Hence in M we cn rech stte in U 2 if nd only if the chrcters which re red long the wy form word in L(M 1 ). After tht the rest of the input is processed using M 2. This implies L(M) = L(M 1 )L(M 2 ) = L(α)L(β) = L(αβ) = L(γ). Cse 4.(iii): γ hs the form γ = (α). Agin let M 1 s ove e n NFA for L(α). Then we construct n NFA M which ccepts L((α) ) s follows. M hs the sme sttes, initil sttes, nd end sttes s M 1. The difference is tht we dd trnsitions similr to Cse 4.(ii). Nmely, for ech trnsition δ 1 (u, ) v, where v E 1, we dd couple of rcs δ(u, ) s 1 for ll s 1 U 1. This mens tht whenever we would rech n end stte of M 1, we cn s well strt reding nother word of L(M 1 ) from one of its initil sttes. Using the union construction, we cn dd ε to the ccepted lnguge using Cse 4.(i), if necessry. Note tht L((M) ) = L(ε L(M 1 ) + ), s needed. ( ) Now let M = (Z, Σ, δ, z 1, E) e deterministic finite utomton. We show tht L(M) is descried y regulr expression. We cn ssume w.l.o.g. tht the vertex set of M is numered Z = {z 1, z 2,..., z n } nd z 1 is the initil stte. The key to the proof is the next definition. R k i,j := {x Σ δ (z i, x) = z j nd δ (z i, x[1.. l]) {z 1,..., z k } for ll 1 l < x }. A word x is in R k i,j iff when we strt reding x eginning in stte z i, then we end up in stte x j, nd moreover, ll intermedite sttes (i. e. those sttes where we hve een fter reding proper, non-empty prefix x[1.. l] of x), hve indices t most k. R k is so importnt ecuse it llows use to solve the prolem using 3-prmeter recursion nd dynmic i,j progrmming. The initil cses k = 0 re esy: No intermedite sttes re llowed t ll, thus we hve R 0 i,j = { Σ δ(z i, ) = z j } for i j { Σ δ(z i, ) = z i } {ɛ} for i = j. In oth cses, R 0 is finite lnguge nd we cn write down regulr expression for it. i,j Now for the recursive cse k 1. The difference etween R k+1 nd R k i,j i,j is tht z k+1 ecomes llowed s n intermedite stte. Thus we cn prtition ny word x R k+1 \ R k i,j i,j etween its visits of the stte z k+1. We strt t z i. Either we go to z j without visiting z k+1. Or we visit z k+1 t lest once. The sustring etween two consecutive visits of z k+i is word from the lnguge R k k+1,k+1. Then we go from z k+1 to z j without nother visit t z k+1. Thus it holds: R k+1 i,j = R k i,j Rk i,k+1 (Rk k+1,k+1 ) R k k+1,j. This trnsltes lmost literlly into regulr expression. expressions γ k i,j such tht Rk i,j = L(γk ). Then i,j Assume y induction tht we hve regulr is regulr expression such tht L(γ k+1 i,j ) = R k+1 i,j. γ k+1 i,j := (γ k i,j γk i,k+1 (γk k+1,k+1 ) γ k k+1,j )

14 Finite utomt nd regulr grmmrs, y Clemens Gröpl, Jnury 9, 2014, 12: where To summrize, R k i,j := {x Σ δ (z i, x) = z j nd δ (z i, x[1.. l]) {z 1,..., z k } for ll 1 l < x }, R 0 i,j = { Σ δ(z i, ) = z j } for i j { Σ δ(z i, ) = z i } {ɛ} for i = j. R k+1 i,j = R k i,j Rk i,k+1 (Rk k+1,k+1 ) R k k+1,j. When we hve reched k = n, then the restriction which is imposed y the upper index in R k i,j ecomes void. Note tht L(M) = {R n 1,j z j E}. Therefore we let γ := (γ n 1,i 1... γ n 1,i l ), where E = {i 1,..., i l } is n enumertion of the end sttes. Then L(γ) = L(M) Exmple for Kleene s trnsformtion We hve seen DFA for the lnguge efore. L = {x {, } (# (x) # (x)) 1 mod 3} The recurrences from the theorem yield γ 0 i,j γ 1 i,j ε 0 ε 1 ε 1 (ε ) ( ) 2 ε 2 ( ) (ε ) z 0 z 1 z 2 γ 2 i,j (ε () ) () ( () ( )) 1 () () () ( ) 2 ( ( )() ) ( )() ((ε ) ( )() ( )) Since z 2 is the only end stte, regulr expression for L is γ 3 0,1 = γ2 0,2 (γ2 2,2 ) γ 2 2,1 = ( () ( ))(((ε ) ( )() ( ))) ( )().

15 8014 Finite utomt nd regulr grmmrs, y Clemens Gröpl, Jnury 9, 2014, 12:48 Finlly we return to context-free grmmrs nd introduce the utomton tht ccepts context-free lnguge, the pushdown utomton (PDA). We will first define it nd then show t n exmple how we cn decide whether string is in the lnguge generted y given CFG. Recll our smll hirpin generting CFG: S W 1 u cw 1 g gw 1 c uw 1, W 1 W 2 u cw 2 g gw 2 c uw 2, W 2 W 3 u cw 3 g gw 3 c uw 3, W 3 g gc. There is n elegnt representtion for derivtions of sequence in CFG clled the prse tree. The root of the tree is the nonterminl S. The leves re the terminl symols, nd the inner nodes re nonterminls. For exmple if we extend the ove productions with S SS we cn get the following: S S W 1 S W 1 W 2 W 2 W 3 W 3 c g g c u g g g u g c c c Using pushdown utomton we cn prse sequence left to right ccording. A (nondeterministic) PDA is formlly defined s 6-tuple: M = (Z, Σ, Γ, δ, z 0, S) where Z is finite set of sttes, Σ is finite input lphet, Γ is finite stck lphet δ : Z Σ {ɛ} Γ P(Z Γ ) is the trnsition function, where P(S) denotes the power set of S. z 0 is the strt stte, S is the initil stck symol S Γ is the lowest stck symol. There is of course lso deterministic version, however the nondeterministic PDA llows for simple construction when CFG is given. If M is in stte z nd reds the input nd if A is the top stck symol, then M cn go to stte z nd replce A y other stck symols. This implies, tht A cn e deleted, replced, or ugmented.

16 Finite utomt nd regulr grmmrs, y Clemens Gröpl, Jnury 9, 2014, 12: After reding the input, the utomton ccepts the word if the stck is empty. Given CFG G = (V, Σ, P, S) we define the corresponding PDA s M = ({z}, Σ, V Σ, δ, z, S). Using the production set P we cn define δ. For ech rule A α P we define δ such tht (z, α) δ(z, ɛ, A) nd (z, ɛ) δ(z,, ). Lets look t our exmple: S W 1 u cw 1 g gw 1 c uw 1, W 1 W 2 u cw 2 g gw 2 c uw 2, W 2 W 3 u cw 3 g gw 3 c uw 3, W 3 g gc. Hence M = ({z}, {, c, g, u}, {, c, g, u, W 1, W 2, W 3, S}, δ, z, S) with δ s explined (lckord). Lets see how the utomton prses word in our hirpin lnguge. Given our CFG, the utomton s stck is initilized with the strt symol S. Then the following steps re iterted until no symols remin. If the stck is empty when no input symols remin, then the sequence hs een successfully prsed. 1. Pop symol off the stck. 2. If the popped symol is non-terminl: Peek hed in the input nd choose vlid production for the symol. (For deterministic PDAs, there is t most one choice. For non-deterministic PDAs, ll possile choices need to e evluted individully.) If there is no vlid trnsision, terminte nd reject. Push the right side of the production on the stck, rightmost symols first. 3. If the popped symol is terminl: Compre it to the current symol of the input. If it mtches, move the utomton to the right on the input. If not, reject nd terminte. Lets try this with the string gccgcggc. Exmple. (The current symol is written using cpitl letter): Input string Stck Opertion Gccgcggc S Pop S. Produce S->gW1c Gccgcggc gw1c Pop g. Accept g. Move right on input. gccgcggc W1c Pop W1. Produce W1->cW2g gccgcggc cw2gc Pop c. Accept c. Move right on input. gccgcggc W2gc Pop W2. Produce W2->cW3g gccgcggc cw3ggc Pop c. Accept c. Move right on input. gccgcggc W3ggc Pop W3. Produce W3->gc gccgcggc gcggc Pop g. Accept g. Move right on input (severl cceptnces) gccgcggc c Pop c. Accept c. Move right on input. gccgcggc$ - Stck empty, input string empty. Accept!

17 8016 Finite utomt nd regulr grmmrs, y Clemens Gröpl, Jnury 9, 2014, 12: Summry Forml lnguges re sic mens in computer science to formlly descrie ojects tht follow certin rules, tht is tht cn e generted using grmmr. Two fundmentl views on forml lnguges re i) to view them s generted y grmmr, or ii) to view them s ccepted y n utomton. The Chomsky hierrchy plces different restrictions on the grmmrs. This limits the possiilities you hve, ut mkes the decision whether word is in such lnguge esier. The nondeterministic utomt (DFA nd PDA) re s powerful s te deterministic counterprts in deciding whether word is in lnguge or not. However, it is esier to define nondeterministic utomton. The importnce for ioinfomtics lies in the stochstic versions of the grmmrs which re used to trin cceptors for iologicl sequence ojects (i.e. Genes using Hidden Mrkov models, or RNA using stochstic CFGs).