Context-Free Grmmrs nd Lnguges (Bsed on Hopcroft, Motwni nd Ullmn (2007) & Cohen (1997)) Introduction Consider n exmple sentence: A smll ct ets the fish English grmmr hs rules for constructing sentences; e.g., 1. A sentence cn e suject followed y predicte (Suject: A smll ct) (Predicte: ets the fish) 2. A suject cn e noun-phrse (Noun-Phrse: A smll ct) 3. A noun-phrse cn e n rticle, followed y n djective followed y noun (Article: A) (Adjective: smll) (Noun: ct) 4. A predicte cn e ver-phrse (Ver-Phrse: ets the fish) 5. A ver-phrse cn e ver followed y noun-phrse (Ver: ets) (Noun-Phrse: the fish) 7. A noun-phrse cn e n rticle, followed y noun (Article: the) (Noun: fish) 8. A ver cn e: uries, touches, grs, ets 9. An djective cn e: ig, smll 10. An rticle cn e: the,, n Definition: The things tht cnnot e replced y nything re clled terminls. Definition: The things tht must e replced y other things re clled nonterminls or vriles In the ove exmple, smll nd ets re terminls, nd noun-phrse nd ver-phrse re nonterminls. Definition: The sequence of pplictions of the rules tht produces the finished string of terminls from the strting symol is clled derivtion. For derivtion of smll ct ets the fish using Context-Free Grmmr (CFG), visit this pge: http://www.sethome.org/mthfoundtions/english_prse_tree.pdf Context-Free Grmmr (CFG) Exmple 1: terminls: = { } Nonterminls: N = { S } productions: S S S ^ The ove CFG defines * = { ^,,,,,,... } It cn generte or derive s follows: S => S => S => S => S => 1
The sme derivtion cn lso e written s: S => S => S => S => S => Exmple 2: Terminl: = { } Nonterminl: N = { S } Productions: S SS S S ^ This CFG cn e written in more compct nottion: S SS ^ which is lso clled the Bckus Norml Form or Bckus-Nur Form (BNF). The ove CFG defines the sme CFL: * = { ^,,,,,... } This CFG cn generte the string s follows: S => SS => SSS => SS => SSS => SS => S => In this grmmr, string in the CFL hs infinitely mny derivtions. In other exmples, string is usully generted in limited numer of wys. Definition: A context-free grmmr (CFG) is composed of: 1. : A finite non-empty set of symols clled terminls from which strings of the lnguge re mde. 2. N: A finite set of symols clled nonterminls. nd N re disjoint. 3. S: A specil strt symol in N. 4. P: A finite set of productions of the form: N i string from ( + N)* where N i is in N. Thus, ech production is of the form: one nonterminl -> finite string of terminls nd/or nonterminls, where the string of terminls nd/or nonterminls cn consist of only terminls or only nonterminls, or ny mixture of terminls nd nonterminls or even the empty string. We require tht t lest one production hs the nonterminl S s its left side. On the right side of the rrow ny string from ( + N)* is llowed. Some ooks represent terminls y T; in tht cse, the productions of the form N i (T + N)* re llowed in CFG. Convention: Terminls will typiclly e lowercse letters. Nonterminls will typiclly e uppercse letters. Exmple 3: A CFG for * = {,,,,,.... } 1. Terminls: = {, }, 2. Nonterminl: N = {S, B} 3. Productions: P = { S B B B ^ } DERIVATION of using the CFG of exmple 3: S => B => B => B => B => 2
Most of the time the set of productions re explicitly given for CFG nd the terminls nd the non-terminls re understood from the context s shown in exmples 4-6 : Exmple 4: A CFG for * = {,,,,.... } S A A A ^ DERIVATION of using the CFG of exmple 4: S => A => A => A => A => Exmple 5: A CFG for * = {,,,,.... } S B B B ^ DERIVATION of using the CFG of exmple 5: S => B => B => B => B => B => Exmple 6: A CFG for { n c n :n > 0} = { c, c, c,.... } S S c DERIVATION of c using the CFG of exmple 6: S => S => S => c Definition: The lnguge generted (defined, derived, produced) y CFG, G, is the set of ll strings of terminls tht cn e produced from the strt symol S using the productions s sustitutions. A lnguge generted y CFG, G, is clled context-free lnguge (CFL) nd is denoted y L(G). Exmple: Terminls: = {}, Nonterminl: N = {S} Productions: P = S S S ^ Let L1 the lnguge generted y this CFG, nd let L2 e the lnguge generted y regulr expression *. Clim: L1 = L2. Proof: We first show tht L2 L1. Consider ^n L2 for n >= 1. We cn generte n y using first production n times, nd then second production. Cn generte ^ L2 y using second production only. 3
Hence L2 L1. We now show tht L1 L2. Since is the only terminl, CFG cn only produce strings hving only s. Thus, L1 L2. Note tht Two types of rrows: -> used in sttement of productions => used in derivtion of string in the ove derivtion of 4, there were mny unfinished stges tht consisted of oth terminls nd nonterminls. These re clled working strings. ^ is neither nonterminl (since it cnnot e replced with something else) nor terminl (since it disppers from the string). Exmple: terminls: = {, } Nonterminls: { S } Productions: S -> S S -> S S -> S -> More compct nottion: S -> S S This CFG cn derive the string s follows: S => S => S => S => S => Let L1 e the CFL, nd let L2 e the lnguge generted y the regulr expression ( + )+. Clim: L1 = L2. Proof: First we show tht L2 L1. Consider ny string w L2. Red letters of w from left to right. For ech letter red in, if it is not the lst, then use the production S -> S if the letter is or use the production S -> S if the letter is For the lst letter of the string, use the production S -> if the letter is or use the production S -> if the letter is In ech stge of the derivtion, the working string hs the form (string of terminls)s. Hence, we hve shown how to generte w using the CFG, which mens tht w L1. Hence, L2 L1. Now we show tht L1 L2. To show this, we need to show tht if w L1, then w L2. This is equivlent to showing tht if w L2, then w L1. 4
Note tht the only string w L2 is w = A. But note tht A cnnot e generted y the CFG, so A L1. Hence, we hve proven tht L1 L2. Trees cn e used to illustrte how string is derived from CFG. Definition: A tree is n cyclic grph. Trees sed on grmmr (or syntx definition) re clled syntx trees, prse trees, genertion trees, production trees, or derivtion trees. Exmple: The following CFG defines L 6 = { { n c*} n } Terminls: { {, }, c } Nonterminls: { S, X } Productions: S {S} {X} X cx ^ Every string generted y this CFG hs lnced open nd closed rces like progrmming lnguges such s Jv. Consider the following derivtion: S => {S} => {{S}} => {{{X}}} => {{{cx}}} => {{{ccx}}} => {{{cc}}} The ove derivtion corresponds to the following derivtion tree or prse tree: 5
Exmple: CFG for simplified rithmetic expressions. Terminls : +, _, 0, 1, 2,..., 9 Nonterminls : S Productions : S S + S S - S 0 1 2 9 Consider the expression 2-3 + 4. Amiguous how to evlute this: Does this men (2-3) + 4 = 10 or 2 - (3 + 4) = 14? Cn eliminte miguity y exmining the two possile derivtion trees S S / \ / \ / \ / \ / \ / \ / \ / \ S + S S * S / \ / \ / \ / \ 2 * 3 4 2 3 + 4 Eliminte the S s s follows: + * / \ / \ / \ / \ / \ / \ / \ / \ * 4 2 + / \ / \ / \ / \ 2 3 3 4 Note tht we cn construct new nottion for mthemticl expressions: strt t top of tree wlk round tree keeping left hnd touching tree first time hit ech terminl, print it out. This gives us string which is in opertor prefix nottion or Polish nottion. In ove exmples, first tree yields + _ 2 3 4 second tree yields _ 2 + 3 4 To evlute the string: 1. scn string from left to right. 6
2. the first time we red sustring of the form opertor-opernd-opernd (o-o-o), replce the three symols with the one result of the indicted rithmetic clcultion. 3. go ck to step 1 Exmple: (from ove) first tree yields: string first o-o-o sustring + _ 2 3 4 _ 2 3 + 6 4 + 6 4 10 second tree yields: string first o-o-o sustring _ 2 + 3 4 + 3 4 _ 2 7 _ 2 7 14 Definition: A CFG is miguous if for t lest one string in its CFL there re two possile derivtions of the string tht correspond to two different syntx trees. Exmple: PALINDROME Terminls :, Nonterminls : S Productions : S S S ^ Cn generte the string s follows: S => S => S => S => which hs derivtion tree: S / \ S / \ S / \ S ^ Cn show tht this CFG is unmiguous. 7
Definition: For given CFG, the totl lnguge tree is the tree with root S, whose children re ll the productions of S, whose second descendents re ll the working strings tht cn e constructed y pplying one production to the leftmost nonterminl in ech of the children, nd so on. Exmple: Terminls :, Nonterminls : S, X Productions : S -> X X XX X -> This CFG hs totl lnguge tree s follows: S / \ / \ / \ / \ / \ X X XX / / / \ / / / \ X X / \ / \ The CFL is finite. Other References: http://we.njit.edu/~mrvin/cis341/chp12.pdf http://www.cs.ppstte.edu/~dp/clsses/2490/chp12.html 8
Chpter 13 Grmmticl Formt Regulr Grmmrs We previously sw tht CFG s cn generte ll regulr lnguges. CFG s cn generte some non-regulr lnguges (tht re Context-Free). All regulr lnguges cn e generted y CFG s, ecuse ll regulr lnguges re CF. some non-regulr lnguges cnnot e generted y CFG s, ecuse they re non-cf. One cn turn n FA into CFG s follows: Exmple: L = ll strings ending in. FA: A+ S- B Definition: The pth development of string processed on mchine: Strt in strting stte S. For ech stte visited, print out the input letters used thus fr nd the current stte. 9
The string hs following pth development on the FA: S A B A B B A Now we define the following productions: S -> A B A -> A B _ B > A B Note tht: The CFG hs production X -> cy if nd only if in the FA, there is n rc from stte X to stte Y leled with c. The CFG hs production X -> A if nd only if stte X in the FA is finl stte. Derivtion of using the CFG: S => A => B => A => B => B => A => There is one-to-one correspondence etween pth developments on the FA nd derivtions in the CFG; i.e., we cn use the pigeonhole principle. The derivtion of the string using the CFG is exctly the sme s the pth development given ove. Theorem 21 All regulr lnguges re CFL s. 10
Exmple: FA: A+ S- C+ B productions: S => S A A => C B A B => B C C => A B A Consider CFG G = (E,, R, S), where E is the set of terminls is the set of nonterminls, nd S is the strting nonterminl R (E+ )* is the set of productions, where production (N, U) R with N nd U (E + )* is written s N -> U Definition: For given CFG G = (E,,R, S), W is semiword if W E * ; i.e., W is string of terminls (mye none) cnctented with exctly one nonterminl (on the right). Exmple: N is semiword if N is nonterminl nd nd re terminls. Definition: A Regulr grmmr, RG, is composed of 1. : A finite non-empty set of symols clled terminls 11
2. N: A finite set of symols, clled Non-terminls. nd N re disjoint 3. S: A specil strt symol in N. 4. P: A set of productions in one of the following two forms (not oth): 4. Right RG: N i *N j * 4. Left RG: N i N j * * A regulr grmmr cn e either Right Regulr or Left Regulr not oth. Ech of N i nd N j is single non-terminl nd they could e the sme. Tht is, N i = N j is llowed. Theorem 22 If CFG is regulr grmmr, then the lnguge generted y this CFG is regulr. Proof. We will prove theorem y showing tht there is TG tht ccepts the lnguge enerted y the CFG. Suppose CFG is s follows: N1 -> w1m1 N2 -> w2m2... Nn -> wnmn Nn+1 -> wn+1 Nn+2 -> wn+2... Nn+m! wn+m where Ni nd Mi re nonterminls (not necessrily distinct) nd wi E* re strings of terminls. Thus, wimi is semiword. At lest one of the Ni = S. Assume tht N1 = S. Crete stte of the TG for ech nonterminl Ni nd for ech nonterminl Mj. Also crete stte +. Mke the stte for nonterminl S the initil stte of the trnsition grph. Drw n rc leled with wi from stte Ni to stte Mi if nd only if there is production Ni -> wimi. Drw n rc leled with wi from stte Ni to stte + if nd only if there is production Ni -> wi. Thus, we hve creted TG. By considering the pth developments of words ccepted y the TG, we cn show tht there is one-to-one correspondence etween words ccepted y TG nd words in CFL. Thus, these re the sme lnguge. Kleene s Theorem implies tht the lnguge hs regulr expression. Thus, lnguge is regulr. Remrks: ll regulr lnguges cn e generted y some regulr grmmrs (Theorem 21) 12
ll regulr grmmrs generte some regulr lnguge. regulr lnguge my hve mny CFG s tht generte it, where some of the CFG s my not e regulr grmmrs. Exmple: CFG productions: S -> B A A B A -> A B -> A Corresponding TG (note tht CFG does not generte A): A S-, + B ^ Definition: A production (N, U) R is A-production if U = A, i.e., the production is N -> A. If CFG does not contin A -production, then A CFL. However, CFG my hve A -production nd A CFL. Exmple: productions: S -> X X -> A Chomsky Norml Form 13
A Productions nd Nullle Nonterminls Recll we previously defined A -production: N-> A where N is some nonterminl. Note tht If some CFL contins the word A, then the CFG must hve A -production. However, if CFG hs A -production, then the CFL does not necessrily contin A; e.g., S -> X X -> A which defines the CFL {}. Definition: For given CFG with s its set of nonterminls E nd A s its set of terminls, working string W 2 (E + )* is ny string of nonterminls nd/or terminls tht cn e generted from the CFG strting from ny nonterminl. Definition: For given CFG hving nonterminl X nd W possile working string, we use the nottion X => W if there is some derivtion in the CFG strting from X tht cn result in the working string W. Exmple: CFG: S X Y X Y A Y X Since we hve the following derivtion S => Y => X => we cn write S *=> Y nd S *=> X nd S *=>. Definition: In given CFG, nonterminl X is nullle if 1. There is production X -> A, or 2. X *=> A; i.e., there is derivtion tht strts t X nd leds to A 3. X =>. => A Chomsky Norml Form Definition: A CFG G = (E,,R, S) is in Chomsky Norml Form (CNF) if (N, U) R implies U () + E; i.e., ech of its productions hs one of the two forms: 14
1. Nonterminl -> string of exctly two Nonterminls 2. Nonterminl -> one terminl Theorem 26 For ny CFL L, the non- A words of L cn e generted y CFG in CNF. Leftmost Nonterminls nd Derivtions Definition: The leftmost nonterminl (LMN) in working string is the first nonterminl tht we encounter when we scn the string from left to right. Exmple: In the string XY SXY, the LMN is X. Definition: If word w is generted y CFG y certin derivtion nd t ech step in the derivtion, rule of production is pplied to the leftmost nonterminl in the working string, then this derivtion is clled leftmost derivtion (LMD). Exmple: CFG: S XS X X The following is Left-Most Derivtion (LMD): S => XS => XS => XS => S => Exmple: CFG: S XY X Y X Y Y Y XX The word hs the following derivtion tree: S / \ / \ 15
/ \ / \ / \ X _ Y _ / \ / / \ \ Y / \ / \ X X / \ / \ X / \ / \ Y Y / \ / \ Note tht if we wlk round the tree strting down the left rnch of the root with our left hnd lwys touching the tree, then the order in which we first visit ech nonterminl corresponds to the order in which the nonterminls re replced in LMD. This is true for ny derivtion in ny CFG Theorem 27 Any word tht cn e generted y given CFG y some derivtion lso hs LMD. Other References: http://we.njit.edu/~mrvin/cis341/chp13.pdf 16
Chpter 14 Pushdown Automt Introduction Previously, we sw connection etween Regulr lnguges nd Finite utomt We sw tht certin lnguges generted y CFG s could not e ccepted y FA s. Pushdown Automt Now we will introduce new kind of mchine: pushdown utomton (PDA). Will see connection etween context-free lnguges nd pushdown utomt Pushdown utomt nd FA s shre some fetures, ut PDA cn hve one extr key feture: STACK. (infinitely long INPUT TAPE on which input is written.) INPUT TAPE is divided into cells, nd ech cell holds one input letter or lnk Δ. A Once lnk Δ is encountered on INPUT TAPE, ll of the following cells lso contin Δ. Red TAPE one cell t time, from left to right. Cnnot go ck. START, ACCEPT, nd REJECT sttes. 17
Once enter either ACCEPT or REJECT stte, cnnot ever leve. READ stte to red input letter from INPUT TAPE. Also, hve n infinitely tll PUSHDOWN STACK, which hs Lst-In-First-Out (LIFO) discipline. Alwys strt with STACK empty. STACK cn hold letters of STACK lphet (which cn e sme s input lphet) nd lnks. PUSH nd POP sttes lter contents of STACK. PUSH dds something to the top of the STACK. POP tkes off the thing on the top of the STACK. A B B.. Exmple: Convert FA to PDA - + Figure 2: An FA which is converted to PDA in Figure 3 18
Strt Red Red Red Accept Reject Reject Figure 3: A PDA 19
A pushdown utomton for { n n : where n >= 0 } is given elow in Figure 4. Plese note tht X is used stck symol; tht is, when n is red from input X is pushed into the stck nd when is red from the input X is poped out of the stck. On the other hnd, could lso hve een used s stck symol. Figure 4. A Pushdown Automton 20
Determinism nd Nondeterminism Definition: A PDA is deterministic if ech input string cn only e processed y the mchine in one wy. Definition: A PDA is nondeterministic if there is some string tht cn e processed y the mchine in more thn one wy. A nondeterministic PDA my hve more thn one edge with the sme lel leding out of certin READ stte or POP stte. my hve more thn one rc leving the START stte. Both deterministic nd nondeterministic PDAs my hve no edge with certin lel leding out of certin READ stte or POP stte. if we re in READ or POP stte nd encounter letter for which there is no outedge from this stte, the PDA crshes. Remrks: For FA s, nondeterminism does not increse power of mchines. For PDA s, nondeterminism does increse power of mchines. Exmple: Lnguge PALINDROMEX, which consists of ll words of the form sxreverse(s) where s is ny string generted y ( + )_. PALINDROMEX = {X, X, X, X, X, X, X,...} Ech word in PALINDROMEX hs odd length nd X in middle. When processing word on PDA, first red letters from TAPE nd PUSH letters onto STACK until red in X. Then POP letters off STACK, nd check if they re the sme s rest of input string on TAPE. PDA: Input lphet E = {,,X} Stck lphet L = {, } 21
Strt Push Red 1 X Red 2 POP1 Push POP2 Accept POP3 Forml Definition of PDA Definition: A pushdown utomton (PDA) is collection of eight things: An lphet of input letters. An input TAPE (infinite in one direction), which initilly contins the input string to e processed followed y n infinite numer of lnks An lphet E of STACK chrcters. A pushdown STACK (infinite in one direction), which initilly contins ll lnks. One START stte tht hs only out-edges, no in-edges. Cn hve more thn one rc leving the START stte. There re no lels on rcs leving the START stte. Hlt sttes of two kinds: 1. zero or more ACCEPT sttes 2. zero or more REJECT sttes Ech of the Hlt sttes hve in-edges ut no out-edges. Finitely mny nonrnching PUSH sttes tht introduce chrcters from E onto the top of the STACK. Finitely mny rnching sttes of two kinds: 1. READ sttes, which red the next unused letter from TAPE nd my hve out-edges leled with letters from or lnk. (There is no 22
restriction on dupliction of lels nd no requirement tht there e lel for ech letter of, or.) 2. POP sttes, which red the top chrcter of STACK nd my hve outedges leled with letters of E nd the lnk chrcter, with no restrictions. Remrks: The definition for PDA llows for nondeterminism. If we wnt to consider PDA tht does not hve nondeterminism, then we will cll it deterministic PDA. Deterministic PDA (DPDA) ccept only suset of CF lnguges. Tht is, PDA nd DPDA re not equivlent. Other References: http://www.cs.odu.edu/~toid/nerzic/390teched/cfl/cfg.html http://www.sethome.org/dey/relting_acm_p168-dey.pdf http://www.cs.duke.edu/csed/jflpods/mrtin/redings/summries/deyetal.html 23