Syntax Analysis Part I

Transcription

1 1 Syntax Analysis Part I Chapter 4 COP5621 Compiler Construction Copyright Robert van Engelen, Florida State University,

2 2 Position of a Parser in the Compiler Model Source Program Lexical Analyzer Token, tokenval Get next token Parser and rest of front-end Intermediate representation Lexical error Syntax error Semantic error Symbol Table

3 3 The Parser A parser implements a C-F grammar as a recognizer of strings The role of the parser in a compiler is twofold: 1. To check syntax (= string recognizer) And to report syntax errors accurately 2. To invoke semantic actions For static semantics checking, e.g. type checking of expressions, functions, etc. For syntax-directed translation of the source code to an intermediate representation

4 4 Syntax-Directed Translation One of the major roles of the parser is to produce an intermediate representation (IR) of the source program using syntax-directed translation methods Possible IR output: Abstract syntax trees (ASTs) Control-flow graphs (CFGs) with triples, three-address code, or register transfer list notation WHIRL (SGI Pro64 compiler) has 5 IR levels!

5 5 Error Handling A good compiler should assist in identifying and locating errors Lexical errors: important, compiler can easily recover and continue Syntax errors: most important for compiler, can almost always recover Static semantic errors: important, can sometimes recover Dynamic semantic errors: hard or impossible to detect at compile time, runtime checks are required Logical errors: hard or impossible to detect

6 6 Viable-Prefix Property Prefix The viable-prefix property of parsers allows early detection of syntax errors Goal: detection of an error as soon as possible without further consuming unnecessary input How: detect an error as soon as the prefix of the input does not match a prefix of any string in the language for (;) Error is detected here Prefix Error is detected here DO 10 I = 1;0

7 7 Error Recovery Strategies Panic mode Discard input until a token in a set of designated synchronizing tokens is found Phrase-level recovery Perform local correction on the input to repair the error Error productions Augment grammar with productions for erroneous constructs Global correction Choose a minimal sequence of changes to obtain a global least-cost correction

8 8 Grammars (Recap) Context-free grammar is a 4-tuple G = (N, T, P, S) where T is a finite set of tokens (terminal symbols) N is a finite set of nonterminals P is a finite set of productions of the form a b where a Î (NÈT)* N (NÈT)* and b Î (NÈT)* S Î N is a designated start symbol

9 9 Notational Conventions Used Terminals a,b,c, Î T specific terminals: 0, 1, id, + Nonterminals A,B,C, Î N specific nonterminals: expr, term, stmt Grammar symbols X,Y,Z Î (NÈT) Strings of terminals u,v,w,x,y,z Î T* Strings of grammar symbols a,b,g Î (NÈT)*

10 10 Derivations (Recap) The one-step derivation is defined by a A b Þ a g b where A g is a production in the grammar In addition, we define Þ is leftmost Þ lm if a does not contain a nonterminal Þ is rightmost Þ rm if b does not contain a nonterminal Transitive closure Þ * (zero or more steps) Positive closure Þ + (one or more steps) The language generated by G is defined by L(G) = {w Î T* S Þ + w}

11 11 Derivation (Example) Grammar G = ({E}, {+,*,(,),-,id}, P, E) with productions P = E E + E E E * E E ( E ) E - E E id Example derivations: E Þ - E Þ - id E Þ rm E + E Þ rm E + id Þ rm id + id E Þ * E E Þ * id + id E Þ + id * id + id

12 12 Chomsky Hierarchy: Language Classification A grammar G is said to be Regular if it is right linear where each production is of the form A w B or A w or left linear where each production is of the form A B w or A w Context free if each production is of the form A a where A Î N and a Î (NÈT)* Context sensitive if each production is of the form a A b a g b where A Î N, a,g,b Î (NÈT)*, g > 0 Unrestricted

13 13 Chomsky Hierarchy L(regular) Ì L(context free) Ì L(context sensitive) Ì L(unrestricted) Where L(T) = { L(G) G is of type T } That is: the set of all languages generated by grammars G of type T Examples: Every finite language is regular! (construct a FSA for strings in L(G)) L 1 = { a n b n n ³ 1 } is context free L 2 = { a n b n c n n ³ 1 } is context sensitive

14 14 Parsing Universal (any C-F grammar) Cocke-Younger-Kasimi Earley Top-down (C-F grammar with restrictions) Recursive descent (predictive parsing) LL (Left-to-right, Leftmost derivation) methods Bottom-up (C-F grammar with restrictions) Operator precedence parsing LR (Left-to-right, Rightmost derivation) methods SLR, canonical LR, LALR

15 15 Top-Down Parsing LL methods (Left-to-right, Leftmost derivation) and recursive-descent parsing E Grammar: E T + T T ( E ) T - E T id E E Leftmost derivation: E Þ lm T + T Þ lm id + T Þ lm id + id E T T T T T T + id + id + id

16 16 Left Recursion (Recap) Productions of the form A A a b g are left recursive When one of the productions in a grammar is left recursive then a predictive parser loops forever on certain inputs

17 17 A General Systematic Left Recursion Elimination Method Input: Grammar G with no cycles or e-productions Arrange the nonterminals in some order A 1, A 2,, A n for i = 1,, n do for j = 1,, i-1 do replace each A i A j g with A i d 1 g d 2 g d k g where A j d 1 d 2 d k enddo eliminate the immediate left recursion in A i enddo

18 18 Immediate Left-Recursion Elimination Rewrite every left-recursive production A A a b g A d into a right-recursive production: A b A R g A R A R a A R d A R e

19 Example Left Recursion Elim. 19 A B C a B C A A b C A B C C a Choose arrangement: A, B, C i = 1: i = 2, j = 1: i = 3, j = 1: i = 3, j = 2: nothing to do B C A A b Þ B C A B C b a b Þ (imm) B C A B R a b B R B R C b B R e C A B C C a Þ C B C B a B C C a C B C B a B C C a Þ C C A B R C B a b B R C B a B C C a Þ (imm) C a b B R C B C R a B C R a C R C R A B R C B C R C C R e

20 20 Left Factoring When a nonterminal has two or more productions whose right-hand sides start with the same grammar symbols, the grammar is not LL(1) and cannot be used for predictive parsing Replace productions A a b 1 a b 2 a b n g with A a A R g A R b 1 b 2 b n

21 21 Predictive Parsing Eliminate left recursion from grammar Left factor the grammar Compute FIRST and FOLLOW Two variants: Recursive (recursive-descent parsing) Non-recursive (table-driven parsing)

22 22 FIRST (Revisited) FIRST(a) = { the set of terminals that begin all strings derived from a } FIRST(a) = {a} if a Î T FIRST(e) = {e} FIRST(A) = È A a FIRST(a) for A a Î P FIRST(X 1 X 2 X k ) = if for all j = 1,, i-1 : e Î FIRST(X j ) then add non-e in FIRST(X i ) to FIRST(X 1 X 2 X k ) if for all j = 1,, k : e Î FIRST(X j ) then add e to FIRST(X 1 X 2 X k )

23 23 FOLLOW FOLLOW(A) = { the set of terminals that can immediately follow nonterminal A } FOLLOW(A) = for all (B a A b) Î P do add FIRST(b)\{e} to FOLLOW(A) for all (B a A b) Î P and e Î FIRST(b) do add FOLLOW(B) to FOLLOW(A) for all (B a A) Î P do add FOLLOW(B) to FOLLOW(A) if A is the start symbol S then add $ to FOLLOW(A)

24 24 LL(1) Grammar A grammar G is LL(1) if it is not left recursive and for each collection of productions A a 1 a 2 a n for nonterminal A the following holds: 1. FIRST(a i ) Ç FIRST(a j ) = Æ for all i ¹ j 2. if a i Þ * e then 2.a. a j Þ * e for all i ¹ j 2.b. FIRST(a j ) Ç FOLLOW(A) = Æ for all i ¹ j

25 25 Non-LL(1) Examples Grammar S S a a S a S a S a R e R S e S a R a R S e Not LL(1) because: Left recursive FIRST(a S) Ç FIRST(a) ¹ Æ For R: S Þ * e and e Þ * e For R: FIRST(S) Ç FOLLOW(R) ¹ Æ

26 26 Recursive-Descent Parsing (Recap) Grammar must be LL(1) Every nonterminal has one (recursive) procedure responsible for parsing the nonterminal s syntactic category of input tokens When a nonterminal has multiple productions, each production is implemented in a branch of a selection statement based on input look-ahead information

27 27 Using FIRST and FOLLOW in a Recursive-Descent Parser expr term rest rest + term rest - term rest e term id procedure rest(); begin if lookahead in FIRST(+ term rest) then match( + ); term(); rest() else if lookahead in FIRST(- term rest) then match( - ); term(); rest() else if lookahead in FOLLOW(rest) then return else error() end; where FIRST(+ term rest) = { + } FIRST(- term rest) = { - } FOLLOW(rest) = { $ }

28 28 Non-Recursive Predictive Parsing: Table-Driven Parsing Given an LL(1) grammar G = (N, T, P, S) construct a table M[A,a] for A Î N, a Î T and use a driver program with a stack input a + b $ stack X Y Z $ Predictive parsing program (driver) Parsing table M output

29 29 Constructing an LL(1) Predictive Parsing Table for each production A a do for each a Î FIRST(a) do add A a to M[A,a] enddo if e Î FIRST(a) then for each b Î FOLLOW(A) do add A a to M[A,b] enddo endif enddo Mark each undefined entry in M error

30 Example Table E T E R E R + T E R e T F T R T R * F T R e F ( E ) id A a FIRST(a) FOLLOW(A) E T E R ( id $ ) E R + T E R + E R e e $ ) T F T R ( id + $ ) T R * F T R * T R e e + $ ) F ( E ) ( * + $ ) F id id * + $ ) 30 id + * ( ) $ E E T E R E T E R E R E R + T E R E R e E R e T T F T R T F T R T R T R e T R * F T R T R e T R e F F id F ( E )

31 31 LL(1) Grammars are Unambiguous Ambiguous grammar S i E t S S R a S R e S e E b Error: duplicate table entry A a FIRST(a) FOLLOW(A) S i E t S S R i S a a e $ S R e S e S R e e e $ E b b t a b e i t $ S S a S i E t S S R S R E E b S R e S R e S S R e

32 32 Predictive Parsing Program (Driver) push($) push(s) a := lookahead repeat X := pop() if X is a terminal or X = $ then match(x) // moves to next token and a := lookahead else if M[X,a] = X Y 1 Y 2 Y k then push(y k, Y k-1,, Y 2, Y 1 ) // such that Y 1 is on top invoke actions and/or produce IR output else error() endif until X = $

33 Example Table-Driven Parsing 33 Stack $E $E R T $E R T R F $E R T R id $E R T R $E R $E R T+ $E R T $E R T R F $E R T R id $E R T R $E R T R F* $E R T R F $E R T R id $E R T R $E R $ Input id+id*id$ id+id*id$ id+id*id$ id+id*id$ +id*id$ +id*id$ +id*id$ id*id$ id*id$ id*id$ *id$ *id$ id$ id$ $ $ $ Production applied E T E R T F T R F id T R e E R + T E R T F T R F id T R * F T R F id T R e E R e

34 34 Panic Mode Recovery Add synchronizing actions to undefined entries based on FOLLOW Pro: Can be automated Cons: Error messages are needed FOLLOW(E) = { ) $ } FOLLOW(E R ) = { ) $ } FOLLOW(T) = { + ) $ } FOLLOW(T R ) = { + ) $ } FOLLOW(F) = { + * ) $ } id + * ( ) $ E E T E R E T E R synch synch E R E R + T E R E R e E R e T T F T R synch T F T R synch synch T R T R e T R * F T R T R e T R e F F id synch synch F ( E ) synch synch synch: the driver pops current nonterminal A and skips input till synch token or skips input until one of FIRST(A) is found

35 35 Phrase-Level Recovery Change input stream by inserting missing tokens For example: id id is changed into id * id Pro: Can be fully automated Cons: Recovery not always intuitive Can then continue here id + * ( ) $ E E T E R E T E R synch synch E R E R + T E R E R e E R e T T F T R synch T F T R synch synch T R insert * T R e T R * F T R T R e T R e F F id synch synch F ( E ) synch synch insert *: driver inserts missing * and retries the production

36 36 Error Productions E T E R E R + T E R e T F T R T R * F T R e F ( E ) id Add error production : T R F T R to ignore missing *, e.g.: id id Pro: Powerful recovery method Cons: Manual addition of productions id + * ( ) $ E E T E R E T E R synch synch E R E R + T E R E R e E R e T T F T R synch T F T R synch synch T R T R F T R T R e T R * F T R T R e T R e F F id synch synch F ( E ) synch synch