Prse trees, miguity, nd Chomsky norml form In this lecture we will discuss few importnt notions connected with contextfree grmmrs, including prse trees, miguity, nd specil form for context-free grmmrs known s the Chomsky norml form. 8.1 Left-most derivtions nd prse trees In the previous lecture we covered the definition of context-free grmmrs s well s derivtions of strings y context-free grmmrs. Let us consider one of the contextfree grmmrs from the previous lecture: 0 1 1 0. (8.1) Agin we ll cll this CFG G, nd s we proved lst time we hve L(G) = { w Σ : w 0 = w 1 }, (8.2) where Σ = {0, 1} is the inry lphet nd w 0 nd w 1 denote the numer of times the symols 0 nd 1 pper in w, respectively. Left-most derivtions Here is n exmple of derivtion of the string 0101: 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1. (8.3) This is n exmple of left-most derivtion, which mens tht it is lwys the leftmost vrile tht gets replced t ech step. For the first step there is only one 1
C 360 Introduction to the Theory of Computing vrile tht cn possily e replced this is true oth in this exmple nd in generl. For the second step, however, one could choose to replce either of the occurrences of the vrile, nd in the derivtion ove it is the left-most occurrence tht gets replced. Tht is, if we underline the vrile tht gets replced nd the symols nd vriles tht replce it, we see tht this step replces the left-most occurrence of the vrile : 0 1 0 1 0 1. (8.4) The sme is true for every other step lwys we choose the left-most vrile occurrence to replce, nd tht is why we cll this left-most derivtion. The sme terminology is used in generl, for ny context-free grmmr. If you think out it for moment, you will quickly relize tht every string tht cn e generted y prticulr context-free grmmr cn lso e generted y tht sme grmmr using left-most derivtion. This is ecuse there is no interction mong multiple vriles nd/or symols in ny context-free grmmr derivtion; if we know which rule is used to sustitute ech vrile, then it doesn t mtter wht order the vrile occurrences re sustituted, so you might s well lwys tke cre of the left-most vrile during ech step. We could lso define the notion of right-most derivtion, in which the rightmost vrile occurrence is lwys evluted first ut there isn t relly nything importnt out right-most derivtions tht isn t lredy represented y the notion of left-most derivtion, t lest from the viewpoint of this course. For this reson, we won t hve ny reson to discuss right-most derivtions further. Prse trees With ny derivtion of string y context-free grmmr we my ssocite tree, clled prse tree, ccording to the following rules: We hve one node of the tree for ech new occurrence of either vrile, symol, or n in the derivtion, with the root node of the tree corresponds to the strt vrile. (We only hve nodes lelled when rules of the form V re pplied.) Ech node corresponding to symol or n is lef node (hving no children), while ech node corresponding to vrile hs one child for ech symol or vrile with which it is replced. The children of ech (vrile) node re ordered in the sme wy s the symols nd vriles in the rule used to replce tht vrile. For exmple, the derivtion (8.3) yields the prse tree illustrted in Figure 8.1. 2
0 1 1 0 Figure 8.1: The prse tree corresponding to the derivtion (8.3) of the string 0101. 0 1 0 1 Figure 8.2: A prse tree corresponding to the derivtion (8.5) of the string 0101. There is one-to-one nd onto correspondence etween prse trees nd leftmost derivtions, mening tht every prse tree uniquely determines left-most derivtion nd ech left-most derivtion uniquely determines prse tree. 8.2 Amiguity ometimes context-free grmmr will llow multiple prse trees (or, equivlently, multiple left-most derivtions) for some strings in the lnguge tht it genertes. For exmple, different left-most derivtion of the string 0101 y the CFG (8.1) from the derivtion (8.3) is given y 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1. (8.5) The prse tree corresponding to this derivtion is illustrted in Figure 8.2. When it is the cse, for given context-free grmmr G, tht there exists t lest one string w L(G) hving t lest two different prse trees, then the CFG G is 3
C 360 Introduction to the Theory of Computing sid to e miguous. Note tht this is so even if there is just single string hving multiple prse trees in order to e unmiguous, CFG must hve just single, unique prse tree for ech string it genertes. Being unmiguous is generlly considered to e positive ttriute of CFG, nd indeed it is requirement for some pplictions of context-free grmmrs. Designing unmiguous CFGs In some cses it is possile to come up with n unmiguous context-free grmmr tht genertes the sme lnguge s n miguous context-free grmmr. For exmple, we cn come up with different context-free grmmr for the lnguge { w {0, 1} : w 0 = w 1 } (8.6) tht, unlike the CFG (8.1), is unmiguous. Here is such CFG: 0 X 1 1 Y 0 X 0 X 1 X Y 1 Y 0 Y (8.7) We won t tke the time to go through proof tht this CFG is unmiguous ut if you think out it for few moments, it shouldn t e too hrd to convince yourself tht it is unmiguous. The vrile X genertes strings hving the sme numer of 0s nd 1s, where the numer of 1s never exceeds the numer of 0s, nd the vrile Y is similr except the role of the 0s nd 1s is reversed. If try to generte prticulr string, you ll never hve more thn one option s to which rule to pply, ssuming you restrict your ttention to left-most derivtions. Here is nother exmple of how n miguous CFG cn e modified to mke it unmiguous. Let us define n lphet long with CFG Σ = {,, +,, (, ) } (8.8) + ( ) (8.9) This grmmr genertes strings tht look like rithmetic expressions in vriles nd, where we llow the opertions nd +, long with prentheses. For instnce, the ( + ) + (8.10) is such n expression, nd we cn generte it (for instnce) s follows: () ( + ) ( + ) ( + ) ( + ) + ( + ) + ( + ) +. (8.11) 4
( ) + + Figure 8.3: Prse tree for ( + ) + corresponding to the derivtion (8.11). This hppens to e left-most derivtion, s it is lwys the left-most vrile tht is sustituted. The prse tree corresponding to this derivtion is shown in Figure 8.3. You cn imgine more complex version of this grmmr llowing for other rithmetic opertions, vriles, nd so on, ut we will stick to the grmmr in (8.9) for the ske of simplicity. Now, the CFG (8.9) is certinly miguous. For instnce, different (left-most) derivtion for the sme string ( + ) + s efore is + + () + ( + ) + ( + ) + ( + ) + ( + ) + ( + ) +, (8.12) nd the prse tree for this derivtion is shown in Figure 8.4. Notice tht there is something ppeling out the prse tree illustrted in Figure 8.4, which is tht it ctully crries the mening of the expression ( + ) +, in the sense tht the tree structure properly cptures the order in which the opertions should e pplied. In contrst, the first prse tree seems to represent wht the expression ( + ) + would evlute to if we lived in society where ddition ws given higher precedence thn multipliction. The miguity of the grmmr (8.9), long with the fct tht prse trees my not represent the mening of n rithmetic expression in the sense just descried, is prolem in some settings. For exmple, if we were designing compiler nd wnted prt of it to represent rithmetic expressions (presumly llowing much more complicted ones thn our grmmr from ove llows), CFG long the lines of (8.9) would e completely indequte. 5
C 360 Introduction to the Theory of Computing + ( ) + Figure 8.4: Prse tree for ( + ) + corresponding to the derivtion (8.12). We cn, however, come up with new CFG for the sme lnguge tht is much etter it is unmiguous nd it properly cptures the mening of rithmetic expressions. Here it is: T + T T F T F F I ( ) I (8.13) For exmple, the unique prse tree corresponding to the string ( + ) + is s shown in Figure 8.5. In order to etter understnd the CFG (8.13), it my help to ssocite menings with the different vriles. In this CFG, the vrile T genertes terms, the vrile F genertes fctors, nd the vrile I genertes identifiers. An expression is either term or sum of terms, term is either fctor or product of fctors, nd fctor is either n identifier or n entire expression inside of prentheses. Inherently miguous lnguges While we hve seen tht it is sometime possile to come up with n unmiguous CFG tht genertes the sme lnguge s n miguous CFG, it is not lwys possile. There re some context-free lnguges tht cn only e generted y miguous CFGs. uch lnguges re clled inherently miguous context-free lnguges. 6
+ T T F T F I F I ( ) T F I + T F I Figure 8.5: Unique prse tree for ( + ) + for the CFG (8.13). An exmple of n inherently miguous context-free lnguge is this one: { 0 n 1 m 2 k : n = m or m = k }. (8.14) We will not discuss proof tht this lnguge is inherently miguous, ut the intuition is tht the string 0 n 1 n 2 n will lwys hve multiple prse trees for some sufficiently lrge nturl numer n. 8.3 Chomsky norml form ome context-free grmmrs re strnge. For exmple, the CFG (8.15) 7
C 360 Introduction to the Theory of Computing simply genertes the lnguge {}; ut it is oviously miguous, nd even worse it hs infinitely mny prse trees (which of course cn e ritrrily lrge) for the only string it genertes. While we know we cnnot lwys eliminte miguity from CFGs s some context-free lnguges re inherently miguous we cn t lest eliminte the possiility to hve infinitely mny prse trees for given string. Perhps more importntly, for ny given CFG G, we cn lwys come up with new CFG H for which it holds tht L(H) = L(G), nd for which we re gurnteed tht every prse tree for given string w L(H) hs the sme size nd very simple, inry-tree-like structure. To e more precise out the specific sort of CFGs nd prse trees we re tlking out, it is pproprite t this point to define wht is clled the Chomsky norml form for context-free grmmrs. Definition 8.1. A context-free grmmr G is in Chomsky norml form if every rule of G hs one of the following three forms: 1. X YZ, for vriles X, Y, nd Z, nd where neither Y nor Z is the strt vrile, 2. X σ, for vrile X nd symol σ, or 3., for the strt vrile. Now, the reson why CFG in Chomsky norml form is nice is tht every prse tree for such grmmr hs simple form: the vrile nodes form inry tree, nd for ech vrile node tht doesn t hve ny vrile node children, single symol node hngs off. A hypotheticl exmple ment to illustrte the structure we re tlking out is given in Figure 8.6. Notice tht the strt vrile lwys ppers exctly once t the root of the tree ecuse it is never llowed on the righthnd side of ny rule. If the rule is present in CFG in Chomsky norml form, then we hve specil cse tht doesn t fit exctly into the structure descried ove. In this cse we cn hve the very simple prse tree shown in Figure 8.7 for, nd this is the only possile prse tree for this string. Becuse of the very specil form tht prse tree must tke for CFG G in Chomsky norml form, we hve tht every prse tree for given string w L(G) must hve exctly 2 w 1 vrile nodes nd w lef nodes (except for the specil cse w =, in which we hve one vrile node nd 1 lef node). An equivlent sttement is tht every derivtion of (nonempty) string w y CFG in Chomsky norml form requires exctly 2 w 1 sustitutions. The following theorem estlishes tht every context-free lnguge is generted y CFG in Chomsky norml form. 8
X Y X Z 0 0 Y X 1 1 Figure 8.6: A hypotheticl exmple of prse tree for CFG in Chomsky norml form. Figure 8.7: The unique prse tree for for CFG in Chomsky norml form, ssuming it includes the rule. Theorem 8.2. Let Σ e n lphet nd let A Σ e context-free lnguge. There exists CFG G in Chomsky norml form such tht A = L(G). The usul wy to prove this theorem is through construction tht converts n ritrry CFG G into CFG H in Chomsky norml form for which it holds tht L(H) = L(G). The conversion is, in fct, quite strightforwrd summry of the steps one my perform to do this conversion for n ritrry CFG G = (V, Σ, R, ) is s follows: 1. Add new strt vrile 0 long with the rule 0. Doing this will ensure tht the strt vrile 0 never ppers on the right-hnd side of ny rule. 2. Eliminte -rules of the form X nd repir the dmge. Aside from the specil cse 0, there is never ny need for rules of the form X ; you cn get the sme effect y simply duplicting rules in which X ppers on the right-hnd side, nd directly replcing or not replcing X 9
C 360 Introduction to the Theory of Computing with in ll possile comintions. For exmple, if we hve the CFG we cn esily eliminte the -rule in this wy: 0 ( ) (8.16) 0 ( ) ( ) ( ) ( ) (8.17) It cn get quite messy for lrger grmmrs, nd when -rules for multiple vriles re involved one needs to tke cre in mking the process terminte correctly, ut it is lwys possile to remove ll -rules (side from 0 ) in this wy. 3. Eliminte unit rules, which re rules of the form X Y. Rules like this re never necessry, nd they cn e eliminted y simply hrd coding the sustitution X Y into every other (non-unit) rule where X ppers on the right-hnd side. For instnce, crrying on the exmple from ove, we cn eliminte the unit rule 0 like this: 0 ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) (8.18) 4. Introduce new vrile X σ for ech symol σ. Include the rule X σ σ, nd replce every instnce of σ ppering on the righthnd side of rule y X σ. (Of course you don t hve to do this in cses where σ ppers ll y itself on the right-hnd side of rule.) For the exmple ove, we my use L nd R (for left-prenthesis nd right-prenthesis) to otin 0 L R L R L R L R L R L R L R L R L ( R ) (8.19) 5. Finlly, we cn split up rules of the form X Y 1 Y m using uxiliry vriles in strightforwrd wy. For instnce, X Y 1 Y m cn e roken up s X Y 1 X 2 X 2 Y 2 X 3. X m 1 Y m 1 Y m. (8.20) 10
We need to e sure to use seprte uxiliry vriles for ech rule so tht there is no cross tlk etween seprte rules. Let s not write this out explicitly for our exmple ecuse it will e lengthy nd hopefully the ide is cler. The description ove is only ment to give you the sic ide of how the construction works nd does not constitute proof of Theorem 8.2. It is possile, however, to e more forml nd precise in descriing this construction in order to otin proper proof of Theorem 8.2. As you my y now suspect, this conversion of CFG to Chomsky norml form often produces very lrge CFGs. The steps re routine ut things get messy nd it is esy to mke mistke when doing it y hnd. I will never sk you to perform this conversion on n ctul CFG, ut we will mke use of the theorem from time to time when we re proving things out context-free lnguges it is sometimes extremely helpful to know tht we cn lwys ssume tht given context-free lnguge is generted y CFG in Chomsky norml form. Finlly, it must e stressed tht the Chomsky norml form sys nothing out miguity in generl CFG in Chomsky norml form my or my not e miguous, just like we hve for ritrry CFGs. On the other hnd, if you strt with n unmiguous CFG nd perform the conversion descried ove, the resulting CFG in Chomsky norml form will still e unmiguous. 11