References Theory of Computtion Alexndre Duret-Lutz dl@lrde.epit.fr Septemer 10, 2010 Introduction to the Theory of Computtion (Michel Sipser, 2005). Lecture notes from Pierre Wolper's course t http://www.montefiore.ulg.c.e/~pw/cours/clc.html (The pge is in French, ut the lecture notes lelled chpitre 1 to chpitre 8 re in English). Elements of Automt Theory (Jcques Skrovitch, 2009). Compilers: Principles, Techniques, nd Tools (A. Aho, R. Sethi, J. Ullmn, 2006). ADL Theory of Computtion 1 / 121 ADL Theory of Computtion 2 / 121 Introduction Wht would e your rection if someone cme t you to explin he hs invented perpetul motion mchine (i.e. device tht cn sustin continuous motion without losing energy or mtter)? You would proly lugh. Without looking t the mchine, you know outright tht such the device cnnot sustin perpetul motion. Indeed the lws of thermodynmics demonstrte tht perpetul motion devices cnnot e creted. We know eforehnd, from scientic knowledge, tht uilding such mchine is impossile. The ultimte gol of this course is to develop similr knowledge for computer progrms. Theory of Computtion Theory of computtion studies whether nd how eciently prolems cn e solved using progrm on model of computtion (strctions of computer). Computility theory dels with the whether, i.e., is prolem solvle for given model. For instnce strong result we will lern is tht the hlting prolem is not solvle y Turing mchine. Complexity theory dels with the how eciently. It cn e seen s continution of the Θ/O nottions you lerned lst yer. Here prolems re grouped into clsses ccording to their complexity for given model of computtion. For exmple P is the clss of ll prolems solvle y deterministic Turing mchine in polynomil time. NP is the clss of ll prolems solvle y nondeterministic Turing mchine in polynomil time. An open question is whether P=NP. ADL Theory of Computtion 3 / 121 ADL Theory of Computtion 4 / 121
Pln for the course Side Gols The rst hlf of the semester will del with models tht re simpler thn Turing mchine, ut still hve importnt pplictions for progrmmers. Week 1 Introduction, Bsic nottions, Regulr lnguges Week 2 Regulr expressions nd introduction of utomt Weeks 34 Opertions on utomt Week 5 Stility of Regulr lnguges, Regulr Grmmrs, Push-down utomt Week 6 Context-Free Grmmrs Weeks 78 Prsing Context-Free Grmmrs The second hlf of the semester will ddress Turing mchines nd complexity theory. Besides studying models of computtion nd complexity clsses we will hve two importnt side gols for the rst hlf of the semester: 1 Understnd how Finite Automt cn e used to mtch regulr expressions. This is importnt to write tools such s grep. 2 Understnd how Context-Free grmmrs cn e recognized using Push-Down Automt. An ppliction is writing the prser of lnguge. For instnce we will write prser for the lnguge used in CS350. ADL Theory of Computtion 5 / 121 ADL Theory of Computtion 6 / 121 Prolems nd progrms Wht is prolem? Exmple prolem 1: Recll our gol: study whether (nd how eciently) prolem cn e solved y progrm executed on computer We need to formlize these two notions: prolem progrm executed on computer Find out whether nturl numer is odd or even. A prolem is generic question tht pplies to set of elements (here nturl numers). Ech instnce of prolem, i.e. the question sked for given element (e.g. is 42 odd? ), hs n nswer. The notions of prolem nd progrm re independent: we cn write progrm tht solves prolem, ut the progrm does not dene the prolem. Severl progrms my exists tht solve the sme prolem. ADL Theory of Computtion 7 / 121 ADL Theory of Computtion 8 / 121
Odd/Even prolem exmple continued Other prolem exmples The instnces of Prolem 1, the nturl numers, cn e represented in se 2. A progrm tht solves Prolem 1 will just hve to look t the lst digit of the representtion of the numer: the nswer is Odd if tht digit is 1, it is Even if the digit is 2. The sme prolem could e solved y nother progrm tht converts the inry representtion into se 10, nd then check whether the lst digit is in {0, 2, 4, 6, 8} or not. 1 Find the medin of n rry of numers 2 Determine whether progrm will stop for ny input vlue (this is the hlting prolem) 3 Determine whether given polynomil with integer coecients hs n integer solution (Hilert's 10th prolem) The rst prolem (medin) is solvle using progrm executed on computer: you might even know its complexity (liner!). The other two prolems cnnot e solved y computer. ADL Theory of Computtion 9 / 121 ADL Theory of Computtion 10 / 121 Hlting Prolem in Pseudo-Code Assume we hve function willhlt(f, rgs) tht cn tells whether cll to f(rgs) will terminte. foo(rgs): = willhlt(foo, rgs) if == true: loop forever else: return Wht do you think is the result of clling foo(0)? If willhlt thinks foo(0) will terminte, then =true nd foo does not terminte. This is contrdiction. If willhlt thinks foo(0) will not terminte, then =flse nd foo does terminte. This is contrdiction. The only solution is tht willhlt() cnnot exist. ADL Theory of Computtion 11 / 121 Progrms s eective procedures We wnt to distinguish two kinds of solutions to prolem: Solutions tht cn e written s progrms nd executed on computer (= eective procedures) Other solutions. Exmples: A progrm written in C++ is n eective procedure, ecuse it cn e compiled into mchine code tht is executle nd does not require ingenuity from the computer. The instruction check tht the progrm hs no innite loops or recursive cll sequence is not n eective procedure for the hlting prolem. It does not explin how to check these properties. ADL Theory of Computtion 12 / 121
Binry Prolems Representing the Inputs of Prolems In the sequel will shll study only prolems with inry nswers (yes/no, 0/1). the hlting prolem is inry prolem Hilert's 10th prolem is inry prolem is nturl numer odd? is inry prolem determining squre root is not inry prolem sorting n rry is not inry prolem It does not relly mtters: more complex nswer could e sked for it fter it. Binry prolems dene prtition of their instnces: the set of positive instnces for which the nswer is yes, nd the set of negtive instnces for which the nswer is no. A prolem cn thus e seen s testing set memership (on set tht might e complex to dene). An eective procedure (e.g. C++ progrm) hs to receive representtion of its input (the instnce of the prolem). In C++ progrm this representtion might e string, n int, n rry of flots or more complex structure. At lower level, we cn see ll these types s sequences of its. So we could formlize eective procedures s functions tht tkes sequence of its nd return it. Becuse we cn, nd ecuse it will e esier to illustrte some prolems, we will generlize this to functions tht tke sequence of symols, nd we will keep the result inry. ADL Theory of Computtion 13 / 121 ADL Theory of Computtion 14 / 121 Alphets nd Words Size of words An lphet is nite nd non-empty set of symols (clled letters). Alphets re often denoted Σ. A word over n lphet is nite sequence of letters from tht lphet. Exmples: 01010100 is word over Σ = {0, 1} jodhpur nd qzpqsd re words over Σ = {,..., z} is word over the Morse lphet Sigm = {,, } (1 + 2) 3 = 9 is word over Σ = {0,..., 9, +,,, /, (, ), =} The empty word (sequence of no letters) is represented y (you my lso encounter λ). The length of word w is denoted y w. Exmples: = 0 01001 = 5 A word w over the lphet Σ cn e seen s function w : {1,..., w } Σ. Exmple: w = jodhpur w(1) = j, w(2) = o,..., w(7) = r. ADL Theory of Computtion 15 / 121 ADL Theory of Computtion 16 / 121
Lnguges Why studying lnguges? A lnguge is (possily innite) set of words over the sme lphet. Exmples: {,,, }, {}, {}, nd re nite lnguges over Σ = {, }. {0, 00, 10, 000, 010, 100, 110, 0000,...} is n innite lnguge over Σ = {0, 1}. It represents ll even numers. The prolem of testing evenness mounts to testing memership to this set. the set of words (over the ASCII lphet) encoding n entire progrm tht lwys stop is n innite lnguge. Two points of view: The linguistic/pplictive point of view: For computers: compilers, interpreters Biotechs (the 4 ses of DNA: ACGT, or the 20 mino cids used s uilding locks for proteins) Nturl Lnguge Processing The computtionl point of view: set memership s ideliztion of computing prolems distinguish lnguges y the computtionl power required to recognize them (complexity clsses) ADL Theory of Computtion 17 / 121 ADL Theory of Computtion 18 / 121 The Conctention Opertion Let w 1 nd w 2 e two words on the sme lphet. The conctention of w 1 nd w 2 is the word w 3 denoted w 3 = w 1 w 2 of size w 1 + w 2 nd such tht { w 1 (i) if i w 1 w 3 (i) = w 2 (i w 1 ) if w 1 < i w 1 + w 2 Exmples: = 0 1 0 = 010 xzw = xzw Conctention is ssocitive, ut it is not commuttive if the lphet hs 2 letters or more. ADL Theory of Computtion 19 / 121 Power For word w, let us denote w n the conctention of n copies of w. With the specil cse w 0 =. w n = ((w w) w) }{{} n times Alterntively, recursive denition of w n cn e given s: { w n if n = 0 = w n 1 w if n > 0 Exmples: (01) 3 = 010101, () 0 =, 4 =. Power is n opertion tht cn e dened using the internl opertion of ny Monoïd. ADL Theory of Computtion 20 / 121
Monoïd A monoïd M,, 1 M is set M, equipped with n ssocitive inry opertion (often denoted using multiplictive symol), nd neutrl element for this opertion. It does not need to hve inverse elements s in group. The power cn e recursively dened for ny m M, n N s { m n 1 M if n = 0 = m n 1 m if n > 0 For instnce: Z,, 1 is monoïd. The powers of the elements of this monoïd correspond to the usul powers of integers. Z, +, 0 is monoïd (nd even group). The power opertion mounts to multipliction. If we denote Σ the set of ll words over Σ, then Σ,, is monoïd. Its power opertion repets the words s just shown. ADL Theory of Computtion 21 / 121 Prexes, Suxes, Fctors, nd Suwords Let v, w Σ e words. prex v is prex of w if there exist word h Σ such tht v = w h. It is proper prex if h. sux v is sux of w if there exist word h Σ such tht v = h w. It is proper sux if h. fctor v is fctor of w if there exist two words h 1, h 2 Σ such tht v = h 1 w h 2. It is proper fctor if (h 1, h 2 ) (, ). suword v is suword of w if you cn trnsform w in v y removing some letters. Free Monoïd For suset S of monoïd M,, 1 M, let us denote S the smllest sumonoïd of M tht contins S. It cn e dened s S = {x M n N, (s 1,..., s n ) S n, x = s 1 s n }. We sy tht the memers of S re the genertors of S. A monoïd M is free if there exists suset S such tht S = M, nd such tht ech element cn e decomposed s product of elements of S in unique wy: x M,!n N,!(s 1,..., s n ) S n, x = s 1 s n If it exists, S is unique. We sy tht M is the free monoïd on S. Exmples: N, +, 0 is free monoïd with single genertor: 1. Z, +, 0 is not free monoïd. For ny lphet Σ, Σ,, is oviously the free monoïd on Σ. ADL Theory of Computtion 22 / 121 Left nd Right Quotients Let v, w Σ e words. right quotient The right quotient of v y w, noted v /w or v w 1 is the prex h of v such tht v = hw. left quotient The left quotient of v y w, noted \w v or w 1 v is the sux h of v such tht v = hw. Exmple: () 1 =. Note: w 1 is just convenient nottion, it is not word. ADL Theory of Computtion 23 / 121 ADL Theory of Computtion 24 / 121
Order on Words If < is totl order on Σ, then the following re totl orders on Σ : lexicogrphic order: v l w if either v is prex or w or v = u v, w = u w with v, w, nd v (1) < w (1). rdix order (.k.. genelogicl order): v r w if v < w or v = w nd v l w Exercise: prove tht the reltions l nd r re eectively totl orders (i.e. tht the reltions re ntisymmetric, trnsitive, nd totl). ADL Theory of Computtion 25 / 121 Some Opertions on Lnguges Let L 1 Σ nd L 2 Σ e two lnguges over the sme lphet. Here re severl opertion we could wnt to pply to these lnguges. L 1 L 2, L 1 L 2 re nturlly dened L 1 = {w Σ w L 1 } L 1 L 2 = {w 1 w 2 w 1 L 1, w 2 L 2 } L k 1 = (L 1 L 1 ) L }{{} 1, with L 0 1 = {}. k times L 1 = {w Σ k 0, w L k 1} This opertor is clled the Kleene str. L + = 1 {w Σ k 1, w L k 1} w\l 1 = w 1 L 1 = {v Σ w v L 1 } This is the left quotient. L 1/w = L 1 w 1 = {v Σ v w L 1 } This is the right quotient. ADL Theory of Computtion 27 / 121 Distnce etween Words Let lcp(v, w) denote the longest common prex of v nd w. Dene similrly the longest common sux lcs, fctor lcf, nd suword lcw. The following re distnce functions (or metrics): d p (v, w) = v + w 2 lcp(v, w) d s (v, w) = v + w 2 lcs(v, w) d f (v, w) = v + w 2 lcf (v, w) d w (v, w) = v + w 2 lcw(v, w) d w is lso known s the Levenshtein distnce, or string edit distnce, ecuse it counts the numer of letters to remove nd insert to trnsform v in w. Exercises: Prove tht these re distnce functions indeed. Find dynmic progrmming implementtion for d w. ADL Theory of Computtion 26 / 121 Regulr Lnguges The set R of regulr lnguges over n lphet Σ is the smllest set of lnguges such tht R, {} R, {} R for ll Σ, if L 1 R nd L 2 R, then L 1 L 2 R, L 1 L 2 R, nd L 1 R. In other words, lnguge is regulr if it cn e uilt using only the elementry lnguges nd the union, conctention, nd Kleene str opertions. Exmple: The innite lnguge {0, 00, 10, 000, 010, 100, 110, 0000,...} tht represents ll even inry numers, is regulr ecuse it cn e constructed s ({0} {1}) {0}. ADL Theory of Computtion 28 / 121
Regulr Lnguges Questions Some questions rise: If L is regulr, is L regulr too? (i.e., cn we lwys descrie L using only,, nd opertions.) Similrly re L /w, w\ L, nd L 1 L 2 regulr? More generlly, re ll lnguges regulr? Exercises For two words x, y on given lphet Σ, prove the if x y = y x then there exists word u nd two numers i nd j such tht x = u i nd y = u j. Dene the lnguge of rithmetic expressions on {0,..., 9, +}. E.g. 1 + 1 + 2 is vlid ut 0 + +2+ is not. For Σ, nd three lnguges A, L, M on Σ, nd n > 1: prove tht {} L = {} M = L = M prove tht A L = A M = / L = M prove tht L = M = / L = M prove tht L n {w n w L} prove tht L n = M n = / L = M Which of the following regulr lnguges re equl? (L M) (L M) L L (L M) (L M) (M L) (L M ) (M L ) (L M ) ADL Theory of Computtion 29 / 121 ADL Theory of Computtion 30 / 121 A Tste of Clculility Recursive vs. Recursively Enumerle A lnguge or set L is recursively enumerle (.k.. semidecidle) if there exists n lgorithm tht, when given n input word w, eventully hlts if nd only if w L. Equivlently: there is n lgorithm tht enumertes the memers of L. Its output is simply 1 list of the words of L. If necessry, this lgorithm my run forever. recursive (.k.. decidle) if there exists n lgorithm tht, when given n input word w, will determine in nite mount of time if w L or not. A recursive lnguge is oviously recursively enumerle. 1 Bewre: N 2 is r.e., ut nive lgorithm with two nested innite loops over N will only enumerte {1} N. A suitle enumertion lgorithm is less trivil. ADL Theory of Computtion 31 / 121 Some exmples: ny nite lnguge given extensively is recursive, the set of ll even numer is recursive lnguge, the set of prime numers is recursive lnguge, the set of input-less progrms tht terminte is recursively enumerle, the set of input-less progrms tht terminte within 10s is recursive, the set of progrms tht lwys terminte on ny input is recursively enumerle, the set of progrms tht do not terminte on some input is not recursively enumerle. ADL Theory of Computtion 32 / 121
Regulr Expressions Regulr expressions re convenient nottion to descrie lnguges. Regulr expressions over Σ re formed using the following rules:, re regulr expressions ech element of Σ is regulr expressions if α nd β re two regulr expressions, then (α + β), (αβ), nd α re regulr expressions. A regulr expression e denotes the lnguge L (e) dened s follows: L ( ) =, L () = {} Σ, L () = {} L ((α + β)) = L (α) + L (β) L ((αβ)) = L (α) L (β) L (α ) = L (α) In prctice, we will omit useless prentheses. ADL Theory of Computtion 33 / 121 Exmples of Regulr Expressions (0 + 1) 0 is regulr expression denoting the even inry numers. The set of ll words dened on the lphet Σ = {,,..., z} is denoted y the regulr expression ( + + + z). This regulr expression Σ : using Σ like this in regulr expression just syntctic sugr. The set of ll nonempty words dened on the lphet Σ = {,,..., z} is denoted y the regulr expression ( + + + z)( + + + z) or ΣΣ which is even revited s Σ +. (Generlly α + is syntctic sugr for αα.) (0 + 1) 0000(0 + 1) denotes the set of ll inry numers whose representtion contins t lest 4 consecutive 0. ((0 + 1) 1) + )0000((1(0 + 1) ) + ) denotes inry numers with group of exctly 4 consecutive 0 (there might e other groups with more or less 0s). ADL Theory of Computtion 34 / 121 Some Regulr Expressions re Equivlent Let us show tht L (( ) + ( ) ) = L (( + ) ). It is ovious tht L (( ) + ( ) ) L (( + ) ) since ( + ) denotes ll the words on {, }. For the other wy, let w L (( + ) ) nd consider four cses: if w = n then w L (() ) L (( ) ), if w = n then w L (() ) L (( ) ), if w contins s nd s nd ends on, we cn split w s }.{{.. } }.{{.. }}.{{.. } }.{{.. } showing tht it indeed elongs to ( ) ( ) L (( ) + ( ) ). if w contins s nd s nd ends on, similr decomposition is possile. Question: Cn you think of n lgorithm to decide whether two regulr expressions denote the sme lnguge? In other words: is the equivlence of two regulr expressions decidle? ADL Theory of Computtion 35 / 121 Exercises (1/2) Write regulr expression tht denotes the set of nturl numers in se 10, with no leding 0 (except to represent 0). Modify the ove expression to cover ll integers (i.e., including negtive numers). An identier in Jv/C/C++ is word uilt using letters, digits, or underscore, ut tht my no strt with digit. Write regulr expression denoting the set of ll vlid identiers. Reding C++ source le line y line, nd we consider ech line s word on the ASCII lphet. We wnt to detect lines tht perform two ssignments (like = = c; or = ; c = d + ; ut not == ). Write regulr expression tht denotes the set of lines contining two ssignments. Let L 1 nd L 2 e the two lnguges over Σ = {,, c} respectively denoted y + c + nd c. Cn you uild regulr expression denoting the lngge L 1 L 2 L 2 L 1? ADL Theory of Computtion 36 / 121
Exercises (2/2) For ech of the following pirs of regulr expressions, tell whether L (ϕ) L (ϕ)ψ or L (ϕ) L (ϕ)ψ or L (ϕ) = L (ϕ)ψ or if they re incomprle. ϕ () () ( + ) ψ () ( + ) c + c ( + c)(c + ) c + c (c + c) (c + c) ((c) (c) ) (c + c) + ((c) (c) ) + (c + c) (c(c) ) (c + c) ((c) (c) ) Regulr expressions over Σ, cn e seen s words over the lphet Σ {(, ), +, }. Cn you write regulr expression tht denotes the set of regulr expressions? ADL Theory of Computtion 37 / 121 Non Regulr Lnguges Oviously ll regulr lnguges re lnguges. Let us show tht not ll lnguge re regulr lnguges using counting rgument: there re not enough regulr expressions to descrie ll lnguges. Such n rgument would e esy with nite sets: we would just compre the crdinls of oth sets. One wy to estlish tht two innite sets hve similr size is to estlish ijection etween the two sets. A rst clss of innite set re the countle sets: An innite set A is countle if you cn nd ijection etween A nd N. Our pln is to show tht the set of regulr lnguges is countle while the set of lnguges is not (it's igger). ADL Theory of Computtion 38 / 121 Exmple of Countle Innite sets Even numers re countle. Bijection is ovious. N 2 is countle: you cn use Cntor's piring function to enumerte the pirs such tht the sum of the two elements is incresing: (0, 0), (1, 0), (0, 1), (2, 0), (1, 1), (0, 2),... Generliztion: the Crtesin product of countle sets is countle. Σ is countle: use the rdix order (i.e., order words y size nd then lexogrphiclly). Any suset of countle set is countle. You cn use the sme order, skipping the missing items. ADL Theory of Computtion 39 / 121 Cntor's Digonl Argument Let A = { 1, 2,...} e countle set nd S the set of susets (.k.. powerset) of A. 1 2 3 Assume, y wy of contrdiction, tht S is s 1 1 0 1 countle: S = {s 1, s 2,...}. We cn s 2 1 1 0 represent S s n innite rry showing with s 3 0 1 0 0/1 whether i elongs to s i.. Now consider the set D = { i i s i }. This is suset of A, so it elongs to S. Cll it s j. Ws is the jth vlue on s j 's line? If it is 0, then j does not elong to s j nd y denition of D it must elong to D = s j... If it is 1, then j elongs to s j nd y denition of D it must not elong to D = s j. These contrdictions prove tht S is not countle. The powerset of ny innite countle set is not countle. ADL Theory of Computtion 40 / 121
Regulr Expressions re Not Enough Finite Stte Mchines (1/2) Let L e lnguge. Regulr expressions re words over Σ = Σ {(, ), +, }. The set of regulr expressions, Σ, is thus countle. Lnguges re susets of Σ. The set of lnguges, i.e. the powerset of Σ, is not countle (y Cntor's rgument). Consequently, there re mny more lnguges thn regulr expressions. There must e some lnguges tht re not denoted y regulr expressions. Consider very simple progrm tht reds word letter y letter, nd nlly returns whether the word elong to L. Ech time the progrm reds letter, its internl stte chnge: the progrm counter my hve progressed, the vlue of some vrile hs chnged, etc. The internl stte of the progrm is uniquely dened y the sequence of letters it hs red so fr. In its lst stte, the progrm should e le to tell whether the word elong to lnguge. Any execution could represented y such sequence of sttes. If the computer hs m its of memory, the numer of dierent possile sttes is nite nd cnnot exceed 2 m. ADL Theory of Computtion 41 / 121 ADL Theory of Computtion 42 / 121 Finite Stte Mchines (2/2) Deterministic Finite Automt We cn therefore mke n strction of such simple progrm s set of sttes some function tht sy how to chnge sttes when letter is red initil stte, from which the computtion should strt some we do distinguish whether the output should e yes or no We cn do the ltter using set of nl sttes: sttes from which ll the letter red so fr form word of the lnguge. A Deterministic Finite Automton (or DFA for short) is tuple Σ, Q, δ, q 0, F where: Σ is n lphet Q is nonempty nite set of sttes δ : Q Σ Q is (totl) trnsition function q 0 is the initil stte F Q is the set of nl sttes ADL Theory of Computtion 43 / 121 ADL Theory of Computtion 44 / 121
DFA Representtion Here is grphicl representtion of the utomton A 1 dened with Σ = {, }, Q = {0, 1, 2}, q 0 = 0, F = {2}, nd δ given y: δ 0 1 0 1 2 1 2 0 2 0 1 2 The initil stte is represented using n input rrow, nd nl sttes re represented y doule circles. ADL Theory of Computtion 45 / 121 Acceptnce of Word: Exmple 0 1 2 Let's try to evlute the word. (0, ) A1 (1, ) A1 (1, ) A1 (1, ) A1 (2, ) A1 (0, ) A1 (1, ) A1 (1, ) A1 (2, ) A1 (2, ). Becuse this execution ends on stte 2 F this word is ccepted. Acceptnce of Word The determine whether word w is ccepted y n utomton A = Σ, Q, δ, q 0, F, we hve to feed the word to the utomton nd wtch it progress step y step s it reds the letters. We will represent these steps using congurtions. A congurtion is pir (q, s) Q Σ : q is the stte reched y the utomton, nd s is the sux of the word tht hs yet to e red. If s is not empty, we cn write s = s(0) s, nd the utomton cn mke step y reding s(0) nd going to stte q = δ(q, s(0)). We sy tht (q, s ) is derivle in one step from (q, s) nd write (q, s) A (q, s ) Once ll letters hve een red, we will rech congurtion (q f, ). The word is ccepted y the utomton i q f F. ADL Theory of Computtion 46 / 121 Lnguge of n utomton Let (q, s) A (q, s ) denote the fct tht (q, s ) is derivle from (q, s) in mny steps. In other words, (q, s) A (q, s ) if nd only if there exist (q 1, s 1 ),..., (q k, s k ) such tht (q, s) = (q 1, s 1 ), (q, s ) = (q k, s k ), nd for ll 1 i < k, (q i, s i ) A (q i+1, s i+1 ). A word w Σ is ccepted y the utomton A = Σ, Q, δ, q 0, F i q f F such tht (q 0, w) A (q f, ). The lnguge L (A) of n utomton A is the set of words it recognizes: On the other hnd, the word is not ccepted: (0, ) A1 (1, ) A1 (1, ) A1 (1, ), nd 1 F. L (A) = {w Σ q f F, (q 0, w) A (q f, )} ADL Theory of Computtion 47 / 121 ADL Theory of Computtion 48 / 121
Exercises Nondeterministic Finite Automt Let D 3 e the following utomton on Σ = {0, 1}: 0 1 0 0 1 2 1 Execute D 3 on the words 101010, nd 11111. 1 2 Prove tht D 3 recognizes the inry representtions of ll the nturl numers tht re divisile y 3. (Hint: interpret stte numers.) 3 Construct n utomton tht recognizes the inry representtions of even numers. 4 Cn you give concise English description of L (A 1 ) (shown on pge 45). ADL Theory of Computtion 49 / 121 0 1 Let's generlize DFA y llowing severl trnsitions for the sme letter in ech stte spontneous trnsitions (chnging of stte without reding ny letter) llowing trnsitions leled y words This generliztion will llow more thn one execution of the sme word (this is the nondeterminism). We will consider tht word is ccepted i one of this executions ends in nl stte. ADL Theory of Computtion 50 / 121 Denition of NFA A Deterministic Finite Automton (or DFA for short) is tuple A = Σ, Q,, q 0, F where: Σ is n lphet Q is nonempty nite set of sttes Q Σ Q is trnsition reltion q 0 is the initil stte F Q is the set of nl sttes Exmple NFA (1/2) Here is grphicl representtion of the NFA A 2 dened with Σ = {, }, Q = {0, 1, 2}, q 0 = 0, F = {2}, nd = {(0,, 0), (0,, 1), (1,, 2), (0,, 1), (0,, 0), (1,, 1), (2,, 2), (2,, 0)}. 0 1 2 An element (q 1, l, q 2 ) denotes trnsition of source q 1, lel l, nd destintion q 2. We hve (q, w) A (q, w ) i l such tht w = lw nd (q, l, q ). ADL Theory of Computtion 51 / 121 ADL Theory of Computtion 52 / 121
Exmple NFA (2/2) From NFA to DFA Exmple of indeterminism: From (0, ) you cn continue with A2 (1, ) A2 (1 vrepsilon) which is not ccepting, to with A2 (0, ) A2 (0 vrepsilon) which is ccepting. Since n ccepting execution exists, is recognized y A 2. Derivtions cn get stuck, consider (0, ) A2 (1, ) nd we cnnot progress. (Fortuntely, (0, ) A2 (0, ) A2 (1, ) A2 (2, ) A2 (0, ) is n ccepting derivtion.) It should e ovious tht ny DFA cn e seen s NFA (with = {(q,, q ) Q Σ Q q = δ(q, )}). There fore NFAs cn do s much s DFAs. Cn they do more? Cn NFA recognize lnguge tht no DFA cn recognize? We will show tht NFA re s powerful s DFA y trnslting NFA to DFA in three steps: eliminting trnsition leled y words of length > 1 eliminting spontneous trnsitions (i.e. leled y words of length < 1) eliminting nondeterminisms (cses with multiple outgoing trnsitions with the sme letter). ADL Theory of Computtion 53 / 121 ADL Theory of Computtion 54 / 121 Eliminting Word Trnsitions (1/2) Eliminting Word Trnsitions (2/2) Simply rewrite Our exmple utomton A 2 is therefore rewritten s follows 3 4 5 s 0 1 2 ADL Theory of Computtion 55 / 121 ADL Theory of Computtion 56 / 121
Eliminting Spontneous Trnsitions (1/2) Eliminting Spontneous Trnsition (2/2) Let E(q) the list of sttes tht cn e reched from q following only -trnsitions. E(q) is the -closure of q. To remove spontneous trnsition (q 1,, q 2 ) from do the following: 1 replce it y the following set of trnsition: {(q 1, l, q 3 ) q E(q 2 ), (q, l, q 3 ) 2 dd q 1 to F if E(q 3 ) F }. Bsiclly we re mking sure tht if (q 1, w) (q 2, w) (q 3, w ) for some words w w, then (q 1, w) (q 3, w ) is still possile in the updted utomton. Our exmple utomton A 2 is therefore rewritten s follows 3 4 5 0 1 2 Such NFA with ll lels of size 1 is clled proper NFA. ADL Theory of Computtion 57 / 121 ADL Theory of Computtion 58 / 121 Eliminting Nondeterminism (1/3) Eliminting Nondeterminism (2/3) The sic ide is to keep trck of ll possile execution in prllel. In other words: keep trck of ll dierent the sttes we cn rech while reding word. We do tht y creting new utomton the sttes of which represent sets of sttes of the originl utomton. 0 1 is trnsformed into {0} {0, 1} More formlly let A = Σ, Q,, q 0, F e proper NFA nd let D = Σ, 2 Q, δ, {q 0 }, F e DFA such tht δ(q, ) = {d Q (q,, d) } F = {q 2 Q q F } Then D nd A re equivlent (they recognize the sme lnguges). Note: 2 Q designtes to powerset of Q. This construction is clled determiniztion or powerset construction. 2 {2} ADL Theory of Computtion 59 / 121 ADL Theory of Computtion 60 / 121
Eliminting Nondeterminism (3/3) Exercises Exmple:, 0 1 2 gets determinized into: {0} {1, 2} {1} {2} Determinize the utomton A 2 (strting from the proper version given on pge 58). Let's further generlize NFAs y llowing multiple initil sttes. A word is ccepted if there is n ccepting execution from one of the initil stte. Show tht these generlized NFAs re s powerful s DFAs., (Here the trnsition leled, use syntctic sugr to represent two trnsitions nd.) ADL Theory of Computtion 61 / 121 ADL Theory of Computtion 62 / 121 Useless Sttes ccessile sttes re sttes tht cn e reched from the initil stte. co-ccessile sttes re sttes from which it is possile to rech nl stte. Oviously executions cnnot rech sttes tht re not ccessile: such sttes cn e removed from the utomton without chnging the lnguge. When n execution reches stte tht is not co-ccessile, we cn immeditely sy tht the word is not ccessile, without reding the end of the word. If we relx our denition of DFA to llow δ to e prtil function, we cn lso remove these useless sttes. (E.g. the stte of the DFA of pge 61 is not co-ccessile.) A trimmed utomton is utomton whose sttes re ll ccessile nd co-ccessile. ADL Theory of Computtion 63 / 121 Thompson's Algorithm: Bsic Cses Thompson's Algorithm uilds NFA tht recognizes given regulr expression. Do you rememer how regulr expression re dened using,, ll Σ, nd the union, conctention, nd Kleene str opertions? Thompson proceeds similrly y providing trnsltion for these se symols nd opertions. This llows to construct the utomton recursively on the denition of the regulr expression. The utomt constructed for ech suexpression ll hve exctly one initil stte, nd one nl stte. Automton for : 0 1 Automton for : 0 1 Automton for : 0 1 ADL Theory of Computtion 64 / 121
Thompson: Union Thompson: Conctention Automton for e 1 + e 2 : q 1 0 A 1 q 1 f Automton for e 1 e 2 : q 0 q f q 1 0 A 1 1 f q 2 0 A 2 q 2 f q 2 0 A 2 q 2 f Here q i, 0 Ai, nd q i f, represents the utomton tht hs een recursively constructed for the regulr expression e i. q i nd 0 qi f re the designted initil nd nl sttes, while A i denotes the rest of the utomt. ADL Theory of Computtion 65 / 121 Thompson: Kleene str ADL Theory of Computtion 66 / 121 Thompson: Exmple Automton for e 1 : Here is Thompson utomton for ( + (cc) )( + c): 1 2 11 12 0 9 10 c c c 3 4 5 6 7 8 13 14 15 q 0 q 1 0 A 1 q 1 f q f You cn see in the construction rules tht we lwys dd two sttes (new initil nd nl sttes) ech time we process letter,,, or the opertions + nd. The only cse we do not dd sttes is in the conctention opertion. Here our expression uses 5 letters, 2 unions, nd one Kleene str: we cn verify tht the Thompson utomton hs 8 2 = 16 sttes. ADL Theory of Computtion 67 / 121 ADL Theory of Computtion 68 / 121
Thompson: Conclusion Exercise Thompson's lgorithm is simple to progrm nd to prove correct (ecuse it is so close to the recursive denition of rtionl expressions). However the utomt it produces re rther ig, nd usully full of nondeterminism. They should e trimmed, simplied using -closure, which require dditionl time. There exist severl other lgorithms tht cn trnslte regulr expressions to (proper) NFA or DFA. The min point here is tht we hve shown tht utomt cn recognize regulr lnguges. Cn they recognize lnguges tht re not regulr? For ech of the following regulr expressions, construct the Thompson utomton, trim it (if needed), uild its -closure, nd determinize the result. 1 c( + c) 2 (( + ) c) 3 ( + + c) 4 ( ( + )) ADL Theory of Computtion 69 / 121 ADL Theory of Computtion 70 / 121 Brzozowski nd McCluskey's Algorithm (1/3) The BMC lgorithm trnsforms n NFA into regulr expression. It uses generliztion of NFA, clled generlized utomt, in which lels re regulr expressions. To trnslte NFA into regulr expression, the generl ide is the enumerte ll the pths etween the initil stte nd nl stte, nd sum the words recognized y ll these pths. The only diculty is tht loops in the utomt cn generte innite pths. Brzozowski nd McCluskey's Algorithm (2/3) Strting from the NFA to trnslte, the BMC lgorithm, lso clled sttes elimintion lgorithm proceeds s follows: 1 dd new initil stte I, nd connect it with trnsition to the originl initil stte 2 dd new nl stte F, nd connect it with trnsitions to ll originl nl sttes 3 let I nd F e the only initil nd nl sttes 4 pick ny stte of the utomton (except I nd F ), remove it nd recrete ll the pths tht were going through tht stte, using trnsitions lelled with equivlent regulr expression 5 repet previous step until the only two sttes left re I nd F. 6 the sum of ll trnsitions etween I nd F is regulr expression denoting the regulr lnguge recognized y the utomton. ADL Theory of Computtion 71 / 121 ADL Theory of Computtion 72 / 121
Brzozowski nd McCluskey's Algorithm (3/3) BMC Illustrtion How to eliminte stte: Let q i denote the stte to eliminte. Let e ii e the lel of the trnsition going from q i to itself (if there re mny trnsitions sum them, nd if there re none, use e ii = ). For ech pir of sttes (q j, q k ) with j i, k i, such tht there exists trnsition q j q i lelled e ji (gin, sum ll the lels if there re mny trnsitions) nd trnsition q i q k lelled e ik, dd new trnsition q j q k, with lel e ji e ii e ik. If trnsition q j q k did lredy exists with lel e jk, you my simply updte its lel with e jk = e jk + e ji e ii e ik. (This should e done for ech pir of stte, including when q j = q k.) Then, delete q i nd its incident trnsitions. Eliminting stte q i : q j q i q k e ji e jk e ii e ik e jk e ii q e ik + e jk j q k ADL Theory of Computtion 73 / 121 ADL Theory of Computtion 74 / 121 BMC Exmple (1/2) Let's compute regulr expression of this utomton: 0 1 2 First we dd the new initil nd nl sttes. I 0 1 2 F BMC Exmple (2/2) We decide to delete sttes 2, 1, nd 0 in tht order. I 0 1 + F I 0 + F ( + ) ( + ) I F ADL Theory of Computtion 75 / 121 ADL Theory of Computtion 76 / 121
Review of Equivlences Regulr Opertions So fr, we hve estlished tht the following formlisms re equivlent: Regulr lnguges. Regulr expressions. NFA. DFA. We could sy tht nite utomt (deterministic or not) re le to solve prolems whose positive instnces form regulr lnguge. Conctention, Union of two utomt, nd Kleene str of one utomton cn e implemented s in Thompson's construction (if t some point we hve too much nl sttes, it is esy to dd new unique nl stte, connected to ll the other with -trnsitions). Wht out: Complementtion? Intersection? Left nd Right Quotient? Trnsposition? Do these opertions preserve the regulr property of lnguge? ADL Theory of Computtion 77 / 121 ADL Theory of Computtion 78 / 121 Complementtion Intersection Let A = Σ, Q, δ, q 0, F e complete (i.e. δ is totl) deterministic utomton. The utomton A = Σ, Q, δ, q 0, Q \ F is the complement of A. We hve L (CA) = L (A). Exercise: Let L e the lnguge denoted y (( + )). Compute regulr expression tht denotes L. (Hint: Trnslte the expression into n NFA, determinize this utomton, complement it, nd then trnslte it ck into regulr expression.) Using De Morgn's lw: L 1 L 2 = L 1 L 2. It is quite complex since it involves three complementtions (hence tree determiniztions). Using synchronous product is fster: Let A = Σ, Q, δ, q 0, F nd A = Σ, Q, δ, q 0, F e two DFAs. The synchronous product of A nd A, denoted A A is the utomton (Σ, Q, δ, q 0, F ) where Q = Q Q, δ = {((s, s ), l, (d, d )) Q Σ Q (s, l, d) δ nd (s, l, d ) δ }, q 0 = (q 0, q 0 ), F = F F. ADL Theory of Computtion 79 / 121 ADL Theory of Computtion 80 / 121
Trnsposition The trnsposition of word is the word printed in the opposite direction: w t (i) = w( w i 1). E.g. () t =. L t = {w t w L} This opertion is esily done on n utomton y exchnging the nl nd initil sttes (if there re mny nl sttes, just connect them ll with spontneous trnsition to new nl stte efore doing the exchnge) nd reversing ll trnsitions. Left nd Right Quotients If L is recognized y DFA A = Σ, Q, δ, q 0, F. We cn recognize \w L with the DFA \w A = Σ, Q, δ, q 0, F where q 0 is the only stte such tht (q 0, w) A (q 0, ). We my lso write A[q 0] to denote the utomton A in which the initil stte hs een replced y q. 0 0 1 2, 3 4, ADL Theory of Computtion 81 / 121 \L (A) = Σ + is denoted y the utomton A[3]. Wht out right quotients? ADL Theory of Computtion 82 / 121 Decidle Prolems on Regulr Expressions Stte Equivlence memership w L emptiness L = universlity L = Σ inclusion L 1 L 2 equivlence L 1 = L 2 For NFA A = Σ, Q,, q 0, F, nd stte x Q, let A[x] designte the utomton Σ, Q,, x, F in which the strting stte hs een replced y x. We sy tht two sttes x, y Q of A re equivlent, written x A y, i L (A[x]) = L (A[y]). Intuitively, if two sttes re equivlent we cn remove one of the two nd direct ll its incoming trnsition to the other. ADL Theory of Computtion 83 / 121 ADL Theory of Computtion 84 / 121
Quotient Automton Computing A y Rening For NFA A = Σ, Q,, q 0, F, the quotient utomton A / = Σ, Q,, q 0, F is dened s follows: Q = Q/ is the set of A -equivlence clsses (S,, D) i there exist two sttes s S nd d D such tht (s,, d). q 0 = [q 0 ] A the A -equivlence clss of q 0 S F i there exists stte s S F If A is deterministic, then A / will e deterministic. In other words, if x A y, then δ(x, ) A δ(y, ). Proof: consider word w L (A[δ(x, )]). Then w L (A[x]). Since x A y, we hve w L (A[y]). Becuse A is deterministic, w L (A[δ(y, )]). Let L i (A) designte the words of L (A) with t most i letters. We sy tht x i A y i L i (A[x]) = L i (A[y]). i+1 A x 0 A y i either x, y F or x, y F. x i+1 A y i x i A y nd Σ, δ(x, ) i A is true only for DFAs.) δ(y, ) (Note: this is therefore renement of i. Becuse the numer of possile prtition is nite, t some point we will hve ( j+1 ) = A ( j A ), nd then it follows tht ( j ) = ( A A) ADL Theory of Computtion 85 / 121 ADL Theory of Computtion 86 / 121 The minimiztion Algorithm Strt with n utomton A. Prtition the sttes ccording to 0 A, i.e., seprte nl sttes from non-nl sttes. Rene the prtition to otin 1 A y nding the letters such tht δ(x, ) 0 A δ(x, ). Rene the prtition to otin 2 A y nding the letters such tht δ(x, ) 1 A δ(x, ). Repet until i+1 = i. The nl prtition dene the stte tht cn e merged. Word Equivlence Let L e regulr lnguge over Σ. We sy tht to words x, y of Σ re L-equivlent, written x L y i z Σ, xz L yz L. This equivlence reltion is right congruence: x L y We note [x] L = {y Σ x L y} the equivlence clss of x. For instnce on Σ = {, } the lnguge L = Σ Σ hs four equivlence clsses: Σ Σ Σ + Σ + + = x L y. ADL Theory of Computtion 87 / 121 The numer of equivlence clsses of L is the index of L. ADL Theory of Computtion 88 / 121
Myhill-Nerode Theorem (1/2) The reltion L chrcterizes exctly wht n utomton tht recognize L should rememer. When it hs red prex w of the input, it should e in the sme stte s fter reding ny other word of [w] L. So the stte of the utomton just hve correspond to equivlence clsses. If the index of L nite nd equl to n, there exists n-sttes DFA M L = Σ, Q, δ, q 0, F tht recognizes L: Q = {[w] L w Σ } δ(q, ) = [w] L for some word w q. q 0 = [] L F = {q Q q L} Determinism follows from the fct tht L is right congruence. It cn e proven tht for ny DFA A, A / = M L (A) up to some renming of sttes. ADL Theory of Computtion 89 / 121 Introduction to Grmmrs An Automton gives rules to recognize the words of some lnguge. It is n ccepting device. A grmmr give rules to generte/produce the words of some lnguges. It is genertive device. The grmmr rules re rewriting rules. For instnce: A sentence hs the form suject ver A suject cn e he or she A ver cn e ets or sleeps With these rules sentence cn e rewritten s he ets, he sleeps, she ets, or she sleeps. ADL Theory of Computtion 91 / 121 Myhill-Nerode Theorem (2/2) If DFA A hs k sttes, then the index of L (A) is t most k. (Inded, if two words w 1 nd w 2 move A to the sme stte, then L w2 so the numer of equivlence clsses cnnot exceed the w 1 numer of sttes of A.) It follows tht lnguge is regulr i it hs nite index. Exmple: let L e regulr lnguge nd let L 2 = {ww w L}. Question: Is L 2 regulr? Consider the L 2 -equivlence on words. Oviously two dierent words x, y L re not L 2 equivlent, ecuse they re distinguished y the suxes x nd y. So the index of L 2 is t lest L. If L is n innite lnguge, then L 2 is not regulr. On the other hnd if L is nite, then L 2 is nite, nd we know tht nite lnguge is regulr. ADL Theory of Computtion 90 / 121 Grmmr Denition A grmmr is tuple G = V, Σ, R, S where V is n lphet Σ V is the set of terminl symols (these re the symols used in the lnguge generted y the grmmr). R V + V is nite set of rewriting rules (the rst element, in V +, cn e rewritten s the second element of the rule), lso clled production rules S V \ Σ is the strt symol. The symols V \ Σ re clled the non-terminl symols. They re only used during the genertion. Exmple: V = {SENTENCE, SUBJECT, VERB, he, she, ets, sleeps}, Σ = {he, she, ets, sleeps}, R = {(SENTENCE, SUBJECT VERB), (suject, he), (suject, she), (ver, ets), (ver, sleeps)}, ADL Theory of Computtion 92 / 121 S = SENTENCE.
Grmmr Conventions Here re some conventions when descriing grmmrs or lgorithms on grmmrs: Nonterminl symols (V \ Σ) re denoted y uppercse letters: A, B,... Terminl symols (Σ) re denoted using lowercse letters:,,... Rewriting Rules (α, β) R re denoted α β, or even α G β (if we need to specify the grmmr). The strting symol is usully denoted S The empty word is denoted s we did so fr. Grmmr Exmple Consider the following grmmr G = V, Σ, R, S : V = {S, A, B,, }, Σ = {, }, R = {S A, S B, B B, A A, A, B }, S is the strting symol. Let's show tht elongs to the lnguge L (G) generted y G : the strt symol S cn e rewritten s A A A A A y rule S A A A A A A A A A A ADL Theory of Computtion 93 / 121 ADL Theory of Computtion 94 / 121 Derivtion Between Words Let G = V, Σ, R, S, v V + nd w v. We sy tht G derives in one step w from v, written v w, i x, y, y, z such tht v = xyz, G w = xy z nd y G y. We lso write v tht v G v 1 G G v 2 G w is there exists mny words x 1, x 2,..., x n such G v n G w. Finlly the lnguge of G = V, Σ, R, S is L (G) = {w Σ S G w} The Chomsky Hierrchy Chomsky hs clssied grmmrs in four ctegories: Type 0 No restriction on rules. Type 1 Context-sensitive grmmrs. For ny rule α β, we require tht α β. One exception (to enle grmmrs to generte the empty word), we llow S s long s S does not pper on the right side of ny rule. Type 2 Context-free grmmrs (CFG). Any rule should hve the form A β where A V \ Σ is nonterminl symol. Type 3 Regulr grmmrs. Rules cn only hve the following two forms: A wb or A w, with A, B V \ Σ, nd w Σ. It cn e shown tht type 3 type 2 type 1 type 0. The only diculty is tht type 2 grmmrs cn hve rules of the form A tht re not llowed y type 1 grmmr. ADL Theory of Computtion 95 / 121 ADL Theory of Computtion 96 / 121
Eliminting A Rules Regulr Grmmrs (1/2) Let G = V, Σ, R, S e type 2 grmmr with some rules of the form A tht we wnt to remove. 1 If L (G) crete new strting symol S nd dd two rules: S nd S S. 2 Repet the following step until there re no more A rules: Pick rule of the form A (other thn S ) nd remove it from R For ech rule α β such tht A ppers in β, dd rule α β where β is otined y replcing A y in β. Clim: A lnguge is regulr i it is generted y regulr grmmr. Proof (1/2). Let us show tht ny regulr lnguge cn e generted y grmmr. Consider NFA M = Σ, Q,, q 0, F recognizing the lnguge. Then the following Grmmr G = V, Σ, R, S genertes the sme lnguge: V = Q Σ (the sttes corresponds to nonterminl symols) S = q 0 R = {A wb (A, w, B) F} {A A F} It should e firly ovious tht (q, w) M (p, v), with w = uv i q up. So in prticulr G (q 0, w) M (p, ) with p F i S G w ADL Theory of Computtion 97 / 121 ADL Theory of Computtion 98 / 121 Regulr Grmmrs (2/2) Proving tht Lnguge is Regulr Proof (2/2). Let us show tht regulr grmmr genertes regulr lnguge. Given regulr lnguge G = V, Σ, R, S, let's construct the NFA M = Σ, Q,, q 0, F where Q = (V \ Σ) {f }: sttes re nonterminl symols plus new stte f, q 0 = S, F = {f }, = {(A, w, B) (A wb) R} (A, w, f ) (A w) R} Then L (M) = L (G). We hve seen dierent wys to prove tht lnguge is regulr: Descrie the lnguge using only sic regulr opertions (conctention, union, Kleene str) Descrie the lnguge using other opertions tht preserve regulrity (intersection, set dierence, complementtion, trnsposition, left nd right quotients) Descrie the lnguge using nite utomton (DFA or NFA) Descrie the lnguge using regulr grmmr (.k.. right liner grmmr). It cn e proved tht ny regulr lnguge cn lso e represented using left liner grmmr (i.e. with rules of the form A Bw or A w). ADL Theory of Computtion 99 / 121 ADL Theory of Computtion 100 / 121
Proving tht Lnguge is Not Regulr Some fcts: Any non-regulr lnguge must hve n innite numer of words (ecuse every nite lnguge is regulr). An innite lnguge does not hve upper ound for the length of its words (if it did, it would hve nite numer of words). Any regulr lnguge is ccepted y nite utomton with nite numer of sttes (cll it m). Consider regulr lnguge ccepted y m-stte nite utomton. Then we the utomton evlutes words of size m it must visit some stte t lest twice, forming loop. We hve seen one use of the Myhill-Nerode Theorem to prove tht L 2 = {ww w L} is not regulr when L is innite (ecuse L 2 's index would e innite). Another useful tool is the pumping lemm, sed on the ove oservtions. ADL Theory of Computtion 101 / 121 Tools for Proving Non-Regulrity Pumping Lemm Two versions of the pumping lemm cn e used: 1 Let L e n innite regulr lnguge. Then there exist x, u, y Σ with u such tht x u n y L for ll n 0. 2 Let L e n innite regulr lnguge nd w L such tht w Q (ssuming Q denotes the sttes of DFA recognizing L). Then x, u, y with u nd xy Q such tht xuy = w nd n N, x u n y L. Exmples: Use the pumping lemm to show tht { n n n N} is not regulr lnguge. (The rst version of the lemm is enough.) Show tht { n2 n N} is not regulr (use the second version of the lemm). ADL Theory of Computtion 102 / 121 Intuition For Non-Regulrity 1 Pumping Lemm 2 Myhill-Nerode Theorem 3 Show tht the lnguge (the one tht you wnt to prove is nonregulr) cn e comined with regulr lnguge nd using opertions tht preserve regulrity in order to uild lnguge tht is known to e nonregulr. Exmple for the third cse: prove tht L = {w {, } w s the sme numers of s nd s} is not regulr. We hve L L ( ) = { n n n N}, so if L ws regulr, then { n n } would lso e regulr, which we know is wrong. Therefore L is not regulr. Finite utomt model mchines with nite mount of memory (the numer of sttes). We cn sy tht the memership to regulr lnguge cn e decided in constnt spce. Or sid otherwise, REGULAR the set of ll regulr lnguges, is equl to DSPACE (O(1)), the set of decision prolem tht cn e solved in constnt spce using deterministic Turing mchine. n n is not regulr ecuse it require counting the numer of s nd s. Here counting just does not require n integer, ecuse the size of the word my e too long to t 32 or 64 its. Counting letters in words of n letters requires Θ(log n) its, so the memory is not ounded. ADL Theory of Computtion 103 / 121 ADL Theory of Computtion 104 / 121
Exercises Write regulr grmmr for ( + )() Write regulr grmmr for utomton A 2 on pge 58 Show tht suset of regulr set is not lwys regulr. Write Context-Free Grmmr for { n n n N}. Explin why { n c m n n N, m N} is not regulr. Explin why the set of regulr expressions is not regulr lnguge. Write Context-Free Grmmr generting ll regulr expressions. ADL Theory of Computtion 105 / 121 Congurtion of PDA The congurtion of PDA is tripled (x, w, α) Q Σ Γ where x is stte w is the prt of the input tht hs not een red yet α is the contents of the stck. A congurtion (x, w, α ) is derivle from (x, w, α) in one step, denoted (x, w, α) P (x, w, α ) if w = uw α = βδ α = γδ ((x, u, β), (x, γ)) The lnguge of P is ll words tht cn move the PDA into nl stte: L (P) = {w Σ q F, γ Γ, (q 0, w, Z) P (q,, γ)} ADL Theory of Computtion 107 / 121 Pushdown Automt A pushdown utomton is tuple P = Q, Σ, Γ,, Z, q 0, F where: Q is set of sttes Σ is n input lphet Γ is stck lphet Z Γ is n initil stck symol q 0 Q is the initil stte F Q is the set of nl sttes ((Q Σ Γ ) (Q Γ ) is the trnsition reltion. These utomt hve stck. When they red symol from the input, nd chnge stte, they cn lsot the sme timereplce word t the top of the stck y nother word. A trnsition ((x, w, α), (y, β)) mens tht the utomton cn go from stte x to stte y if ADL Theory of Computtion 106 / 121 w is prex of the input word α is t the top of the stck Exmple PDA (1/2) If these conditions re mtched nd the utomton chnges stte, it should The PDA replce P = α Q, y βσ, onγ, the, Z, stck. q 0, F with Q = {0, 1, 2} Σ = {, } Γ = {A, Z} = {((0,, ), (0, A)), ((0,, ), (1, )), ((1,, A), (1, )), ((1,, Z), (2, Z))} q 0 = 0 F = {2} ccepts the lnguge { n n n N}. ; /A ; A/ ; / ; Z/Z 0 1 2 ADL Theory of Computtion 108 / 121
Exmple PDA (2/2) The PDA P = Q, Σ, Γ,, Z, q 0, F with Q = {0, 1, 2}, Σ = {, }, Γ = {A, B, Z} = {((0,, ), (0, A)), ((0,, ), (0, B)), ((0,, ), (1, )), ((1,, A), (1, )), ((1,, B), (1, )), ((1,, Z), (2, Z))} q 0 = 0 F = {2} ccepts the plindromes on {, }, i.e. {ww t w {, } }. ; /A ; A/ ; / ; Z/Z 0 1 2 ; /B ; B/ ADL Theory of Computtion 109 / 121 Context-Free Grmmrs A Grmmr G = V, Σ, R, S is Context-Free Grmmr (CFG) if ny rule of R should hs the form A β where A V \ Σ is nonterminl symol (no constrint on β). The following Context-Free Grmmr genertes { n n n N}: S S S The following Context-Free Grmmr genertes plindromes on {, }: S S S S S ADL Theory of Computtion 110 / 121 Grmmr for Regulr Expressions (1/3) The following grmmrs genertes ll regulr expressions over {,, c} with prentheses round opertors, nd ssuming 1 is the regulr expression for the empty word, nd 0 for the empty lnguge. S S S c S 0 S 1 S (SS) S (S + S) S S How cn we modify it to ccept expressions like ( + + c) + insted of (((( + ) + c)(( ))) + )? I.e., without the unneeded prentheses? ADL Theory of Computtion 111 / 121 Grmmr for Regulr Expressions (2/3) Let's introduce A α β γ s syntctic sugr for A α, A β, A γ. S c 0 1 (S) SS S + S S This grmmr cn generte + c in dierent wys: S SS Sc S + Sc + Sc + c S S + S + S + SS + S + c Other derivtions exist, ecuse you cn sustitute,, c in dierent orders. The ove two derivtions should e quite chocking if you look t them from mthemticl stndpoint: one correspond to the interprettion of + c s sum of products, nd the other s product of sums. We sy tht the grmmr is miguous. How cn we x the miguity, ssuming tht hs priority over conctention, nd tht conctention hs priority over +. ADL Theory of Computtion 112 / 121