C 314 Principles of Progrmming Lnguges Lecture 6: LL(1) Prsing Zheng (Eddy) Zhng Rutgers University Ferury 5, 2018
Clss Informtion Homework 2 due tomorrow. Homework 3 will e posted erly next week. 2
Top - Down Prsing - LL(1) <> <A> α β Bsic Ide: x The prse tree is constructed from the root, expnding non-terminl nodes on the tree s frontier following leftmost derivtion. The input progrm is red from left to right, nd input tokens re red (consumed) s the progrm is prsed. The next non-terminl symol is replced using one of its rules. The prticulr choice hs to e unique nd uses prts of the input (prtilly prsed progrm), for instnce the first token of the remining input. y 3
Top - Down Prsing - LL(1) (cont.) Exmple: ::= ε How cn we prse (utomticlly construct leftmost derivtion) the input string using PDA (push-down utomton) nd only the first symol of the remining input? INPUT: eof 4
LL(1) Prsing Exmple ::= ε Remining Input: ententil Form: 5
LL(1) Prsing Exmple ::= ε Remining Input: ententil Form: ::= 6
LL(1) Prsing Exmple ::= ε Remining Input: Mtch! ententil Form: 7
LL(1) Prsing Exmple ::= ε Remining Input: ententil Form: 8
LL(1) Prsing Exmple ::= ε Remining Input: ententil Form: ::= 9
LL(1) Prsing Exmple ::= ε Remining Input: ententil Form: Mtch! 10
LL(1) Prsing Exmple ::= ε Remining Input: ententil Form: 11
LL(1) Prsing Exmple ::= ε Remining Input: ententil Form: ::= 12
LL(1) Prsing Exmple ::= ε Remining Input: ententil Form: Mtch! 13
LL(1) Prsing Exmple ::= ε Remining Input: ententil Form: 14
LL(1) Prsing Exmple ::= ε Remining Input: ententil Form: ::= ε ε 15
LL(1) Prsing Exmple ::= ε Remining Input: ententil Form: Mtch! ε 16
LL(1) Prsing Exmple ::= ε Remining Input: ententil Form: ε 17
LL(1) Prsing Exmple ::= ε Remining Input: ententil Form: Mtch! ε 18
LL(1) Prsing Exmple ::= ε Remining Input: ententil Form: ε 19
LL(1) Prsing Exmple ::= ε Remining Input: Mtch! ententil Form: ε 20
LL(1) Prsing Exmple ::= ε Remining Input: ententil Form: ε 21
Another LL(1) Prsing Exmple Consider this exmple grmmr: id_list ::= id id_list_til id_list_til ::=, id id_list_til id_list_til ::= ; How to prse the following input string? A, B, C; 22
Another LL(1) Prsing Exmple id_list ::= id id_list_til id_list_til ::=, id id_list_til id_list_til ::= ; Remining Input: A, B, C; id_list ententil Form: id_list 23
Another LL(1) Prsing Exmple id_list ::= id id_list_til id_list_til ::=, id id_list_til id_list_til ::= ; Remining Input: A, B, C; id_list id(a) id_list_til ententil Form: id(a) id_list_til id_list ::= id id_list_til 24
Another LL(1) Prsing Exmple id_list ::= id id_list_til id_list_til ::=, id id_list_til id_list_til ::= ; Remining Input: A, B, C ; id_list id(a) Mtch! id_list_til ententil Form: id(a) id_list_til 25
Another LL(1) Prsing Exmple id_list ::= id id_list_til id_list_til ::=, id id_list_til id_list_til ::= ; Remining Input:, B, C ; id_list id(a) id_list_til ententil Form: id(a) id_list_til 26
Another LL(1) Prsing Exmple id_list ::= id id_list_til id_list_til ::=, id id_list_til id_list_til ::= ; Remining Input:, B, C ; id(a) id_list id_list_til, id(b) id_list_til ententil Form: id(a), id(b) id_list_til id_list_til ::=, id id_list_til 27
Another LL(1) Prsing Exmple id_list ::= id id_list_til id_list_til ::=, id id_list_til id_list_til ::= ; Remining Input:, B, C ; id_list id(a) id_list_til, id(b) id_list_til Mtch! ententil Form: id(a), id(b) id_list_til 28
Another LL(1) Prsing Exmple id_list ::= id id_list_til id_list_til ::=, id id_list_til id_list_til ::= ; Remining Input: B, C ; id(a) id_list id_list_til, id(b) id_list_til ententil Form: id(a), id(b) id_list_til 29
Another LL(1) Prsing Exmple id_list ::= id id_list_til id_list_til ::=, id id_list_til id_list_til ::= ; Remining Input: B, C ; id(a) id_list id_list_til, id(b) id_list_til Mtch! ententil Form: id(a), id(b) id_list_til 30
Another LL(1) Prsing Exmple id_list ::= id id_list_til id_list_til ::=, id id_list_til id_list_til ::= ; Remining Input:, C ; id(a) id_list id_list_til, id(b) id_list_til ententil Form: id(a), id(b) id_list_til 31
Another LL(1) Prsing Exmple id_list ::= id id_list_til id_list_til ::=, id id_list_til id_list_til ::= ; Remining Input:, C ; id(a) id_list id_list_til, id(b) id_list_til ententil Form: id(a), id(b), id(c) id_list_til, id(c) id_list_til id_list_til ::=, id id_list_til 32
Another LL(1) Prsing Exmple id_list ::= id id_list_til id_list_til ::=, id id_list_til id_list_til ::= ; Remining Input:, C ; id(a) id_list id_list_til, id(b) id_list_til ententil Form: id(a), id(b), id(c) id_list_til, id(c) id_list_til Mtch! 33
Another LL(1) Prsing Exmple id_list ::= id id_list_til id_list_til ::=, id id_list_til id_list_til ::= ; Remining Input: C ; id(a) id_list id_list_til, id(b) id_list_til ententil Form: id(a), id(b), id(c) id_list_til, id(c) id_list_til 34
Another LL(1) Prsing Exmple id_list ::= id id_list_til id_list_til ::=, id id_list_til id_list_til ::= ; Remining Input: C ; id(a) id_list id_list_til, id(b) id_list_til ententil Form: id(a), id(b), id(c) id_list_til, id(c) id_list_til Mtch! 35
Another LL(1) Prsing Exmple id_list ::= id id_list_til id_list_til ::=, id id_list_til id_list_til ::= ; Remining Input: ; id(a) id_list id_list_til, id(b) id_list_til ententil Form: id(a), id(b), id(c) id_list_til, id(c) id_list_til 36
Another LL(1) Prsing Exmple id_list ::= id id_list_til id_list_til ::=, id id_list_til id_list_til ::= ; Remining Input: ; id(a) id_list id_list_til, id(b) id_list_til ententil Form: id(a), id(b), id(c) ;, id(c) id_list_til id_list_til ::= ; ; 37
Another LL(1) Prsing Exmple id_list ::= id id_list_til id_list_til ::=, id id_list_til id_list_til ::= ; Remining Input: ; id(a) id_list id_list_til, id(b) id_list_til ententil Form: id(a), id(b), id(c) ;, id(c) id_list_til ; Mtch! 38
Another LL(1) Prsing Exmple id_list ::= id id_list_til id_list_til ::=, id id_list_til id_list_til ::= ; Remining Input: id(a) id_list id_list_til, id(b) id_list_til ententil Form: id(a), id(b), id(c) ;, id(c) id_list_til ; 39
Predictive Prsing Bsic ide: For ny two productions A ::= α β, we would like distinct wy of choosing the correct production to expnd. For some rhs α G, define FIRT(α) s the set of tokens tht pper s the first symol in some string derived from α. Tht is x FIRT(α) iff α xγ for some γ, nd 40
Revisiting the id_list Exmple id_list ::= id id_list_til id_list_til ::=, id id_list_til id_list_til ::= ; Remining Input:, B, C ; id_list id(a) id_list_til 41
Revisiting the id_list Exmple id_list ::= id id_list_til id_list_til ::=, id id_list_til id_list_til ::= ; Remining Input:, B, C ; id_list id(a) id_list_til, id(b) id_list_til id_list_til ::=, id id_list_til 42
Revisiting the id_list Exmple id_list ::= id id_list_til id_list_til ::=, id id_list_til id_list_til ::= ; Remining Input:, B, C ; id_list id(a) id_list_til, id(b) id_list_til Mtch! id_list_til ::=, id id_list_til 43
Revisiting the id_list Exmple id_list ::= id id_list_til id_list_til ::=, id id_list_til id_list_til ::= ; Remining Input:, B, C ; id_list id(a) id_list_til 44
Revisiting the id_list Exmple id_list ::= id id_list_til id_list_til ::=, id id_list_til id_list_til ::= ; Remining Input:, B, C ; id_list id(a) id_list_til ; Mismtch! id_list_til ::= ; 45
Revisiting the id_list Exmple id_list ::= id id_list_til id_list_til ::=, id id_list_til id_list_til ::= ; Remining Input:, B, C ; id_list id(a) id_list_til FIRT(, id id_list_til ) = {, } FIRT( ; ) = { ; } Given id_list_til s the first non-terminl to expnd in the tree: If the first token of remining input is, we choose the rue id_list_til ::=, id id_list_til If the first token of remining input is ; we choose the rule id_list_til ::= ; 46
Predictive Prsing Key Property: Whenever two productions A ::= α nd A ::= β oth pper in the grmmr, we would like FIRT (α) FIRT (β) =, nd if α * ε, then FIRT (β) FOLLOW (A) = Anlogue cse for β * ε. Note: due to first condition, t most one of α nd β cn derive ε. 47
Revisiting the id_list Exmple id_list ::= id id_list_til id_list_til ::=, id id_list_til id_list_til ::= ; Remining Input:, B, C ; id_list id(a) id_list_til FIRT(, id id_list_til ) = {, } FIRT( ; ) = { ; } FIRT (, id id_list_til FIRT ( ; ) = Given id_list_til s the first non-terminl to expnd in the tree: If the first token of remining input is, we choose the rue id_list_til ::=, id id_list_til If the first token of remining input is ; we choose the rule id_list_til ::= ; 48
Predictive Prsing Key Property: Whenever two productions A ::= α nd A ::= β oth pper in the grmmr, we would like FIRT (α) FIRT (β) =, nd if α * ε, then FIRT (β) FOLLOW (A) = Anlogue cse for β * ε. Note: due to first condition, t most This one rule of is α intuitive. nd β cn However, derive ε. it is not correct, ecuse it doesn t hndle ε rules. How to hndle ε rules? 49
Revisiting the LL(1) Prsing Exmple ::= ε Remining Input: 50
Revisiting the LL(1) Prsing Exmple ::= ε Remining Input: ::= Mismtch! It only mens ::= is not the right production rule to use! 51
Revisiting the LL(1) Prsing Exmple ::= ε Remining Input: 52
Revisiting the LL(1) Prsing Exmple ::= ε Remining Input: ::= ε ε ::= ε turns out to e the right rule lter. However, t this point: ε does not mtch either! 53
Predictive Prsing For non-terminl A, define FOLLOW(A) s the set of terminls tht cn pper immeditely to the right of A in some sententil form. Thus, non-terminl s FOLLOW set specifies the tokens tht cn leglly pper fter it. A terminl symol hs no FOLLOW set. FIRT nd FOLLOW sets cn e constructed utomticlly 54
Predictive Prsing Key Property: Whenever two productions A ::= α nd A ::= β oth pper in the grmmr, we would like FIRT (α) FIRT (β) =, nd if α * ε, then FIRT (β) FOLLOW (A) = Anlogue cse for β * ε. Note: due to first condition, t most one of α nd β cn derive ε. This would llow the prser to mke correct choice with lookhed of only one symol! 55
Predictive Prsing Key Property: Whenever two productions A ::= α nd A ::= β oth pper in the grmmr, we would like FIRT (α) FIRT (β) =, nd if α * ε, then FIRT (β) FOLLOW (A) = Anlogue cse for β * ε. Note: due to first condition, t most one of α nd β cn derive ε. This would llow the prser to mke correct choice with lookhed of only one symol! 56
LL(1) Grmmr Define FIRT+(A ::= δ) for rule A ::= δ FIRT (δ) - { ε } U Follow (A), if ε FIRT(δ) FIRT (δ) otherwise A Grmmr is LL(1) iff (A ::= α nd A ::= β) implies FIRT + ( A ::= α) FIRT + ( A ::= β) = 57
Bck to Our Exmple trt ::= eof ::= ε FIRT() = {} FIRT(ε) = {ε} Is the grmmr LL(1)? FOLLOW() = {eof, } FIRT + ( ::= ) = {} FIRT + ( ::= ε) = ( FIRT(ε) - {ε} ) FOLLOW() ={eof, } Define FIRT+(δ) for rule A ::= δ FIRT (δ) - { ε } U Follow (A), if ε FIRT(δ) FIRT (δ) otherwise 58
Tle Driven LL(1) Prsing Exmple: ::= ε LL(1) prse tle eof other ε ε error How to prse input? 59
Tle Driven LL(1) Prsing Input: string w nd prsing tle M for G push eof push trt ymol token next_token() X top-of-stck repet if X is terminl then if X == token then pop X token next_token() else error() else /* X is non-terminl */ if M[X, token] == X Y1Y2... Yk then pop X push Yk, Yk 1,..., Y1 else error() X top-of-stck until X = eof if token!= eof then error() 60
Predictive Prsing Now, predictive prser looks like: stck source code scnner prser intermedite representtion prsing tles Rther thn writing code, we uild tles. Building tles cn e utomted! 61
Predictive Prsing Now, predictive prser looks like: stck source code scnner prser intermedite representtion grmmr prser genertor prsing tles Rther thn writing code, we uild tles. Building tles cn e utomted! 62
Recursive Descent Prsing Now, we cn produce recursive descent prser for our LL(1) grmmr. Recursive descent is one of the simplest prsing techniques used in prcticl compilers: Ech non-terminl hs n ssocited prsing procedure tht cn recognize ny sequence of tokens generted y tht non-terminl There is min routine to initilize ll glols (e.g: tokens) nd cll the strt symol. On return, check whether token==eof, nd whether errors occurred Within prsing procedure, oth non-terminls nd terminls cn e mtched: non-terminl A: cll procedure for A token t: compre t with current input token; if mtched, consume input, otherwise, ERROR Prsing procedure my contin code tht performs some useful computtions (syntx directed trnsltion) 63
Recursive Descent Prsing (pseudo code) eof other ε ε error min: { token := next_token( ); if (( ) nd token == eof) print ccept else print error ; } 64
Recursive Descent Prsing (pseudo code) eof other ε ε error ool : { switch token { cse : token := next_token( ); cll ( ); if (token == ) { token := next_token( ); return true; } else return flse; rek; cse : cse eof: return true; rek; defult: return flse; } } 65
Next Lecture Next Time: Review of LL(1) prsing nd syntx directed trnsltion Red cott, Chpter 2.3.1-2.3.2 66