The Parser. CISC 5920: Compiler Construction Chapter 3 Syntactic Analysis (I) Grammars (cont d) Grammars
|
|
- Deirdre Walters
- 5 years ago
- Views:
Transcription
1 The Parser CISC 5920: Compiler Construction Chapter 3 Syntactic Analysis (I) Arthur. Werschulz Fordham University Department of Computer and Information Sciences Copyright c Arthur. Werschulz, All rights reserved. Spring, 2017 Part of compiler s front end Tasks: Analyzes program structure Syntax error checking Supervise intermediate code generation Controls the lexer Parser design Based on formal language theory Uses context-free grammar, and so doesn t cover all issues 1 / 1 2 / 1 rammars rammars (cont d) enerative/descriptive, rather than proscriptive (Chomsky) Example: The tree has green leaves. Parse tree shows structure Need to have rules, comprising a grammar, to build parse tree: sentence noun phrase verb phrase noun phrase article noun noun phrase adjective noun verb phrase verb noun phrase noun tree, leaves, dog,... verb has, ate,... adjective green, old,... Example: a + b c. rammar: expression expression expression expression expression + expression expression a, b, c,... 3 / 1 4 / 1
2 rammars: Syntax vs. Semantics What s the difference? Syntactically correct semantically correct For natural languages, these grammars don t cover all grammatical sentences. rammars: Formal Definition A grammar has the form = (T, N, S, R), where T : finite set of terminal symbols N: finite set of nonterminal symbols Must have N T =. S N: unique start symbol R: set of productions α β, where α, beta (N T ) (sentential forms). Context-free grammar: Productions x β, where x N. Notation: α β if rule in R that transforms α into β. Omit where possible. If α, β (N T ) and finite sequence α α 1, α 1 α 2, α 2 α 3,... α n 1 β, then we write α β. 5 / 1 6 / 1 rammars: Formal Definition (cont d) Language generated by : { } L() = w T : S w Notational conventions: A, B, C... : non-terminals a, b, c,... : terminals w, x,... : strings of terminals α, β, γ... : sentential forms Boldface: programming language terminals Backus-Naur form (BNF) uses ::= instead of for alternatives Extended Backus-Naur form (EBNF) uses {... }: indefinite repetition (Kleene closure) [... ]: optional item rammars: Parse Trees and Derivations Simple expression grammar E = (T, N, S, R) where T = {i, +,,, /} N = {E} S = E R = {E E + E, E E E, E E E, E E/E, E (E), E i } Analyze (a b) (c + d) (i i) (i + i) E Draw parse tree Shorter E E E E (E) E (E + E) E (E + i) E (i + i) (E) (i + i) (E E) (i + i) (E i) (i + i) (i i) (i + i) 7 / 1 8 / 1
3 rammars: Rightmost, Leftmost Derivations Previous example: rightmost derivation Leftmost derivation of (a b) (c + d) (i i) (i + i)? E E E E E (E) (E E) E (i E) E (i i) E (i i) (E) (i i) (i + E) (i i) (i + i) Left vs. right sentential forms Different kinds of parsers yield the two derivations rammars: Ambiguous rammars Consider i i + i for our grammar E. Possible parsings? E E E i Ei i E + E i i + E i i + i and E E + E E + i E E + i E i + i i i + i Our grammar is ambiguous: both are correct! How to disambiguate? Look outside of grammar (operator hierarchy) Revise the grammar E E T E T T T T F T /F F F (E) i 9 / 1 10 / 1 rammars: The Chomsky Hierarchy rammars: The Chomsky Hierarchy (cont d) rammars of types 0, 1, 2, 3. Let L k = { L() : is of type k } Then L k L k 1. Type 0 (unrestricted, phrase-structure, semi-thue) grammars: Productions α β, where α, β (N T ). Typical form: γaδfebruary28, 2017γβδ for A N. γ, δ: left and right contexts of A Recognizable by Turing machine. Type 1 (context-sensitive) grammars: Productions: as in Type 0, but β ɛ. Recognizable by linear bounded automaton (special TM). Type 2 (context-free) grammars: Productions: A β where A N and β ɛ. Recognizable by non-deterministic stack automaton. NB: NDSA is not equivalent to DSA. Recognize deterministic context-free languages. This includes practical programming languages. Type 3 (regular) grammars: Productions: A a or A Bc, where A, B N, a, c T. Recognizable by FSA. 11 / 1 12 / 1
4 rammars: Some Examples rammars: Some Examples (cont d) L abc = { a n b n c n : n 0 }. Not a context-free language (why not?). rammar for L abc : L P = { legal (balanced) sets of parentheses }. Context-free grammar for L P : S (S) () L P L 2, but L P L 3. Hence L 3 L 2. S asbc abc CB BC bb bb bc bc cc cc 13 / 1 Example: Check aabbcc S asbc aabcbc aabbcc aabbcc aabbcc aabbcc L abc L 1, but L abc L 2. Hence L 2 L / 1 Top-Down Parsers Top-Down Parsers: Left Recursion Consider the grammar: Top-down parsers Start at the root of the parse tree. Leftmost derivation. Bottom-up parsers Start at leaves of the parse tree. Rightmost derivation. Must scan tokens from left to right. Entails some difficulties. Can be done with a set of recursive functions. Eventual goal: Table-driven top-down parser. Parse i i + i Top-down parse: E E + T T T T F F F i (E) E E + T E + T + T E + T + T... Problem? Left recursion, in this case E E + T (also, T T F ). Solution? Remove left-recursive productions from grammar, since top-down parser gets stuck. 15 / 1 16 / 1
5 Top-Down Parsers: Left Recursion (cont d) Top Down Parsers: Left Recursion (cont d) Immediate left-recursion: A Aα Non-immediate left recursion: A Bα, B Aβ et A Bα Aβα Bαβα Aαβα... Removing immediate left recursions? for each A N do begin separate A-left recursions A Aα 1 Aα 2 Aα 3... from non-left recursion A δ 1 δ 2 δ 3... introduce new nonterminal A replace non-left-recursive productions with A δ 1 A δ 2 A δ 3 A... replace left-recursive productions with A ɛ α 1 A α 2 A α 3 A... end Example: Suppose grammar consists of A Ab Ac d Note that L() = d(bc), Separate the productions: A Ab Ac A d Transformed: A da A ɛ ba ca 17 / 1 18 / 1 Top Down Parsers: Left Recursion (cont d) Removing non-immediate left recursion? for each production B whatever... do begin while productions B Aβ where A appears in an earlier-appearing production A γ 1 γ 2 γ 3... do replace B Aβ with B γ 1 β γ 2 β γ 3 β... end Top-Down Parsers: Left Recursion (cont d) Example: S A Sa; A Bb c; B Sd S: first on list. No non-immediate let recursion. But we have S Sa, fix by adding new non-terminal P: S A; S Sa becomes S AP; P ɛ ap A production: ok. B production: S appears earlier in list, so fix: Since we have B Sd with earlier S AP, the former becomes S APd. rammar is now S AP; P ɛ ap; A Bb C; B APd Change B APd to B BbPd; B cpd rammar is now S AP; P ɛ ap; A Bb C; B cpd; B BbPd 19 / 1 20 / 1
6 Top-Down Parsers: Left Recursion (cont d) Example (cont d): S A Sa; A Bb c; B Sd rammar was S AP; P ɛ ap; A Bb C; B APd We changed B APd to B BbPd; B cpd rammar is now S AP; P ɛ ap; A Bb C; B cpd; B BbPd Fix B BbPd: Introduce new non-terminal Q and so B cpd; B BbPd becomes B cpdq; q ɛ bpdq. Final grammar: S AP; P ɛ ap; A Bb c; B cpdq; Q ɛ bpdq Top-Down Parsers: Left Recursion (cont d) Example: Expression grammar E E + T T ; T T F F ; E i (E) E-productions: new non-terminal Q: E TQ; Q ɛ + TQ T -production: new non-terminal R F -production: no change Final grammar: T FR; R ɛ FR E TQ; Q ɛ + TQ; T FR R ɛ FR; F i (E) 21 / 1 22 / 1 Top-Down Parsers: Backtracking Carry out top-down parse by having parser try all possible derivations (brute-force method) This may lead to the wrong tree! Example: Suppose grammar is Token string: bcdc S ee bac baca Exhaustive search (brute force) first tries S bac bcac bcdc We wanted the derivation S bcde. d ca Need to backtrack. How? This could be painful! Need to put back input, deconstruct (part of) tree. Top-Down Parsing: Backtracking (cont d) Better idea: Use a better language design, keeping factor out of the common prefix (left factorization): S ee baq Q c e A d ca We now get proper derivation S baq bcaq bcdq bcde. 23 / 1 24 / 1
7 Recursive-Descent Parsing Try to avoid backtracking. Idea: Devote one function to recognize each nonterminal Example: rammar S aa b A csd e bool S() { if (token_is( a )&& A())) { cout << "S->aA\n"; return true; } if (token_is( b )) { cout << "S->b\n"; return true; } error( S ); return false; } Recursive-Descent Parsing (cont d) Example (cont d): rammar S aa b A csd e bool A() { if (token_is( c ) && S() && token_is( d )) { cout << "A->cSd\n"; return true; } if (tokan_is( e )) { cout << "A->e\n"; return true; } error( A ); return false; } 25 / 1 26 / 1 Recursive-Descent Parsing (cont d) Predictive Parsers Example (cont d): rammar Main program calls S(). S aa b A csd e Parse succeeds iff S() returns true and all input is used up, Do sample parse of token stream acbd (call treee and parse tree). Reamrks: This is a CF. Recognizing CFLs requires a stack. Where is it? In the recursion! Compare grammars S aa b A csd e and S Aa Bb A c da B e fb Second grammar production for S: neither RHS begins with nonterminal. 27 / 1 28 / 1
8 rammar is Suppose input string is feb. S Aa Bb A c da B e fb Proper parse: S Bb fbb feb. If recursive-descent parse starts with S Aa, then parser must... determine that this is a bad decsision, backtrack, start over from S Bb. How to avoid all this? Allow parser to look ahead in input stream. Decide which terminals are (leftmost-)derivaable from each nonterminal on RHS of production. Example (cont d): rammar is Parse tree: S Aa Bb S Aa Bb A c da B e fb daa ca fbb eb Input starts with c or d: use S Aa... FIRST(Aa) = {c, d}. Input starts with e or f : use S Bb... FIRST(Bb) = {e, f }. 29 / 1 30 / 1 Function S tries S Aa if input token is FIRST(Aa) = {c, d}. Function S tries S Bb if input token is FIRST(Bb) = {e, f }. In more detail, let = (T, N, S, R) be a grammar. For any sentential form α generated by : { FIRST(α) = a T : γ (N T ) such that α } aγ. For small grammars, can find FIRST sets by hand. Example: Find FIRST(E) for the grammar E TQ Q ɛ + TQ T FR R ɛ FR F i (E) We find E TQ FRQ (E)RQ... E TQ FRQ irq... and so FIRST(E) = {(, i}. But in general, we d like an algorithm to compute FIRST(α). 31 / 1 32 / 1
9 Predictiive Parsers (cont d) To help derive an algorithm, note the following: 1. If α = aβ, then FIRST(α) = {a}. 2. We say that α is nullable if α ɛ. If α is nullable, then ɛ FIRST(α). 3. FIRST(ɛ) = {ɛ}. 4. If α = Aβ for some A N, then FIRST(A) {ɛ} FIRST(α). Hidden trap in (4): Suppose α = ABδ where A is nullable. THen must follow up withg possibilities from B. Moreover, if B is nullable, must follow possiblities from δ. Example: Let S ABa A b c ɛ B d ɛ Here, A and B are nullable. We have Also, note that FIRST(S) = FIRST(ABa) = {a, b, c, d, e} ɛ FIRST(A) and ɛ FIRST(B), but ɛ FIRST(ABa), since ABa ɛ 33 / 1 34 / 1 Holub s formulation of the general rule: Let α = βx δ, where β N, X T or X is first non-nullable terminal. Then FIRST(α) = (FIRST(β) {ɛ}) FIRST(X ). If α = β (i.e., α is nullable), then FIRST(α) = FIRST(β). Predictive Parser (cont d) Algorithm for computing FIRST(α): if α T N {ɛ} then if α T then FIRST(α) = {α} else if α = ɛ then FIRST(α) = {ɛ} else // α N has form α β 1 β 2 β 3... FIRST(α) = k FIRST(β k) else // α = X 1 X 2... X n FIRST(α) = for (int j = 1; j n X j ɛ; j++) FIRST(α) = FIRST(X j ) Example (cont d): rammar is S ABa, A b c ɛ, B d ɛ. Both A and B are nullable, so FIRST(ABa) = {b, c} {d, e} {a} = {a, b, c, d, e} 35 / 1 36 / 1
10 Predictive Parser (cont d)... uses FIRST sets. Doesn t always work! Algorithm: Suppose A α β, where FIRST(α) FIRST(β). Do we choose α or β? Supppose grammar acquires ɛ-production A ɛ when removing left recursions. When do we choose A ɛ? if A = S then FOLLOW(A) $ for all productions Q xay do if y = qz for some q T then FOLLOW(A) = {q} else FOLLOW(A) = FIRST(y) {ɛ} // typo in book if y = ɛ y ɛ then FOLLOW(A) = FOLLOW(Q) Example: rammar E TQ; Q +TQ ɛ; T FR; R FR ɛ; F (E) i We have Of course FOLLOW sets? FIRST(E) = FIRST(T ) = FIRST(F ) = {(, i} FIRST(Q) = {+, ɛ} FIRST(R) = {, ɛ} FIRST(+TQ) = {+} FIRST((E)) = {(} FIRST( FR) = { } FIRST(i) = {i} FOLLOW(E) = {$, )} FOLLOW(Q) = {$, )} FOLLOW(T ) = {=, $, )} FOLLOW(R) = {+, $, )} FOLLOW(F ) = {+,, $, )} 37 / 1 38 / 1 How to use in a parsing function? Suppose production A rhs 1 rhs 2... rhs n Then bool A() { if next token() FIRST(rhs 1 ) then try rule A rhs 1 else if next token() FIRST(rhs 2 ) then try rule A rhs 2 }. else if next token() FIRST(rhs n ) then try rule A rhs n else if next token() FOLLOW(A) then error( A ); return false return true; 39 / 1 This is an example of an LL(1) grammar: L R token scan, Leftmost derivation, 1-char lookahead. A grammar is LL(1) if for all productions A α β: 1. ( FIRST(α) {ɛ} ) ( FIRST(β) {ɛ} ) =. 2. α ɛ = FIRST(β) FOLLOW(α) =. 40 / 1
11 A Predicitve Recursive-Descent Parser Consider the expression grammar Remove left recursion: E E + T T ; T T F F ; E i (E) Table-Driven Predictive Parsers Problem: Need one function per production. Brittle. Idea: Use general control procedure, specify grammar via table. Example: rammar E E + T T T T F F F (E) i E TQ; Q ɛ + TQ; T FR Remove left recursions: R ɛ FR; F i (E) E TQ; Q +TQ ɛ; T FQ; R FR ɛ; F (E) i See Pascal program on pp A few points: 1. Assume one-letter identifiers, to simplify lexing. 2. BACKUP() is needed if current token is in a FOLLOW set. 3. Functions Q, T (etc.) to recognize nonterminals Q, T, etc. Note the use of forward declarations. 41 / 1 The table for this grammar: Blank: error condition i + ( ) $ E TQ TQ Q +TQ ɛ ɛ T FR FR R ɛ FR ɛ ɛ F i (E) 42 / 1 Table-Driven Predicitve Parser (cont d) Control algorithm uses a stack named symstack: Constructing the Table symstack.push($) str = str + $ (start symbol, stack); while!symstack.empty() x = sysmstack.top(); a = incoming token; if x T then if x == a then sysmstack.pop(); a = next token() else error(); else if table[x,a] blank then symstack.pop() symstack.push(reverse(table[x,a])) else error() Trace execution on the strings i (i + i) and (i+) i. Draw parse tree... compare with Canonical parse tree. 43 / 1 Incoming token must tell us what to ddo next (avoid backtracking) Suppose sysmatck.top() = X and input token is a. Either RHS must begin with a, or RHS must lead to sentential form beginning with a. Example: Suppose input string is (... Need production E (, but no such production exists! However, have derivation E TQ FRQ (E)RQ Only one such path from E to ( Table takes this path Need to choose RHS α if token FIRST(α). But if ɛ FIRST(alpha), this won t work (no column headed ɛ), so use FOLLOW sets. 44 / 1
12 Constructing the Table (con td) Constructing the Table (cont d) Example: Our grammar is E TQ; Q +TQ ɛ; T FQ; R ɛfr ɛ; F (E) i Algorithm: for all productions X β do for all a FIRST() {ɛ} do table[x, a] = ɛ if β = ɛ) (ɛ FIRST(β)) do for all a FOLLOW(X ) do table[x, a] = ɛ. FIRST and FOLLOW sets? FIRST(E) = FIRST(T ) = FIRST(F ) = {i, (} FIRST(Q) = {+, ɛ} FIRST(R) = {, ɛ} FIRST(+TQ) = {+} FIRST( FR) = { } FOLLOW(E) = {$, )} FOLLOW(Q) = FOLLOW(E) = {$, )} FOLLOW(T ) = {+, ), $} FOLLOW(R) = FOLLOW(T ) = {+, ), $}Q FOLLOW(F ) = {+,, ), $} 45 / 1 46 / 1 Constructing the Table (cont d) Conflicts E TQ: FIRST(TQ) = FIRST(T ) = {i, (} = table[e, i] = table[e, (] = TQ Q +TQ ɛ = table[q, +] = +TQ For ɛ: FOLLOW(Q) = {$, )}impliestable[q, $] = table[q, )] = ɛ T FR: FIRST(FR) = FIRST(F ) = {i, (} = table[t, i] = table[t, (] = FR R FR ɛ = table[r, ] = FR For ɛ: FOLLOW(R) = {+, ), $} = table[r, +] = table[r, $] = table[r, )] = ɛ F (E) i = FIRST(F ) = {(, i} = table[r, +] = (E) table[f, i] = i What if table[x, a] has multiple entries? Trouble! Which one to pick? Maybe factorization helps. Example: A abe abf becomes A aq, Q be cf. Maybe not / 1 48 / 1
13 Conflicts (cont d) Example: The if-then-else grammar: S if E then S if E then S else S a b E x y Factorization yields Consider S if E then SQ a b E x y Q else S ɛ if x then if y then a else b Two parse trees, depending on where dangling else is attached: 1. Match else with first if. 2. Match else with most recently unmatched then (standard choice in most programming languages) 49 / 1 Conflicts (cont d) Example (cont d): FIRST, FOLLOW sets reflect this ambiguity Table? FIRST(Q) = {else, ɛ} FOLLOW(S) = {$, else} FOLLOW(Q) = {$, else} if x y then a b else $ S if E then S else S a b E x y Q else S ɛ ɛ 50 / 1
Top-Down Parsing and Intro to Bottom-Up Parsing
Predictive Parsers op-down Parsing and Intro to Bottom-Up Parsing Lecture 7 Like recursive-descent but parser can predict which production to use By looking at the next few tokens No backtracking Predictive
More informationSyntactic Analysis. Top-Down Parsing
Syntactic Analysis Top-Down Parsing Copyright 2015, Pedro C. Diniz, all rights reserved. Students enrolled in Compilers class at University of Southern California (USC) have explicit permission to make
More informationCONTEXT FREE GRAMMAR AND
CONTEXT FREE GRAMMAR AND PARSING STATIC ANALYSIS - PARSING Source language Scanner (lexical analysis) tokens Parser (syntax analysis) Syntatic structure Semantic Analysis (IC generator) Syntatic/sema ntic
More informationAmbiguity, Precedence, Associativity & Top-Down Parsing. Lecture 9-10
Ambiguity, Precedence, Associativity & Top-Down Parsing Lecture 9-10 (From slides by G. Necula & R. Bodik) 2/13/2008 Prof. Hilfinger CS164 Lecture 9 1 Administrivia Team assignments this evening for all
More informationCompiling Techniques
Lecture 5: Top-Down Parsing 26 September 2017 The Parser Context-Free Grammar (CFG) Lexer Source code Scanner char Tokeniser token Parser AST Semantic Analyser AST IR Generator IR Errors Checks the stream
More informationCMSC 330: Organization of Programming Languages. Pushdown Automata Parsing
CMSC 330: Organization of Programming Languages Pushdown Automata Parsing Chomsky Hierarchy Categorization of various languages and grammars Each is strictly more restrictive than the previous First described
More informationSyntax Analysis Part I
1 Syntax Analysis Part I Chapter 4 COP5621 Compiler Construction Copyright Robert van Engelen, Florida State University, 2007-2013 2 Position of a Parser in the Compiler Model Source Program Lexical Analyzer
More informationSyntax Analysis Part III
Syntax Analysis Part III Chapter 4: Top-Down Parsing Slides adapted from : Robert van Engelen, Florida State University Eliminating Ambiguity stmt if expr then stmt if expr then stmt else stmt other The
More informationParsing -3. A View During TD Parsing
Parsing -3 Deterministic table-driven parsing techniques Pictorial view of TD and BU parsing BU (shift-reduce) Parsing Handle, viable prefix, items, closures, goto s LR(k): SLR(1), LR(1) Problems with
More informationIntroduction to Bottom-Up Parsing
Introduction to Bottom-Up Parsing Outline Review LL parsing Shift-reduce parsing The LR parsing algorithm Constructing LR parsing tables 2 Top-Down Parsing: Review Top-down parsing expands a parse tree
More informationCompiling Techniques
Lecture 5: Top-Down Parsing 6 October 2015 The Parser Context-Free Grammar (CFG) Lexer Source code Scanner char Tokeniser token Parser AST Semantic Analyser AST IR Generator IR Errors Checks the stream
More informationIntroduction to Bottom-Up Parsing
Introduction to Bottom-Up Parsing Outline Review LL parsing Shift-reduce parsing The LR parsing algorithm Constructing LR parsing tables Compiler Design 1 (2011) 2 Top-Down Parsing: Review Top-down parsing
More informationSyntax Analysis Part I
1 Syntax Analysis Part I Chapter 4 COP5621 Compiler Construction Copyright Robert van Engelen, Florida State University, 2007-2013 2 Position of a Parser in the Compiler Model Source Program Lexical Analyzer
More informationCompiler Principles, PS4
Top-Down Parsing Compiler Principles, PS4 Parsing problem definition: The general parsing problem is - given set of rules and input stream (in our case scheme token input stream), how to find the parse
More informationIntroduction to Bottom-Up Parsing
Outline Introduction to Bottom-Up Parsing Review LL parsing Shift-reduce parsing he LR parsing algorithm Constructing LR parsing tables Compiler Design 1 (2011) 2 op-down Parsing: Review op-down parsing
More informationCSE302: Compiler Design
CSE302: Compiler Design Instructor: Dr. Liang Cheng Department of Computer Science and Engineering P.C. Rossin College of Engineering & Applied Science Lehigh University February 27, 2007 Outline Recap
More informationIntroduction to Bottom-Up Parsing
Outline Introduction to Bottom-Up Parsing Review LL parsing Shift-reduce parsing he LR parsing algorithm Constructing LR parsing tables 2 op-down Parsing: Review op-down parsing expands a parse tree from
More informationSyntax Analysis: Context-free Grammars, Pushdown Automata and Parsing Part - 3. Y.N. Srikant
Syntax Analysis: Context-free Grammars, Pushdown Automata and Part - 3 Department of Computer Science and Automation Indian Institute of Science Bangalore 560 012 NPTEL Course on Principles of Compiler
More informationEXAM. Please read all instructions, including these, carefully NAME : Problem Max points Points 1 10 TOTAL 100
EXAM Please read all instructions, including these, carefully There are 7 questions on the exam, with multiple parts. You have 3 hours to work on the exam. The exam is open book, open notes. Please write
More informationLanguages. Languages. An Example Grammar. Grammars. Suppose we have an alphabet V. Then we can write:
Languages A language is a set (usually infinite) of strings, also known as sentences Each string consists of a sequence of symbols taken from some alphabet An alphabet, V, is a finite set of symbols, e.g.
More informationCS415 Compilers Syntax Analysis Top-down Parsing
CS415 Compilers Syntax Analysis Top-down Parsing These slides are based on slides copyrighted by Keith Cooper, Ken Kennedy & Linda Torczon at Rice University Announcements Midterm on Thursday, March 13
More informationParsing Algorithms. CS 4447/CS Stephen Watt University of Western Ontario
Parsing Algorithms CS 4447/CS 9545 -- Stephen Watt University of Western Ontario The Big Picture Develop parsers based on grammars Figure out properties of the grammars Make tables that drive parsing engines
More informationPredictive parsing as a specific subclass of recursive descent parsing complexity comparisons with general parsing
Plan for Today Recall Predictive Parsing when it works and when it doesn t necessary to remove left-recursion might have to left-factor Error recovery for predictive parsers Predictive parsing as a specific
More informationSyntax Analysis Part I. Position of a Parser in the Compiler Model. The Parser. Chapter 4
1 Syntax Analysis Part I Chapter 4 COP5621 Compiler Construction Copyright Robert van ngelen, Flora State University, 2007 Position of a Parser in the Compiler Model 2 Source Program Lexical Analyzer Lexical
More informationContext free languages
Context free languages Syntatic parsers and parse trees E! E! *! E! (! E! )! E + E! id! id! id! 2 Context Free Grammars The CF grammar production rules have the following structure X α being X N and α
More informationCSC 4181Compiler Construction. Context-Free Grammars Using grammars in parsers. Parsing Process. Context-Free Grammar
CSC 4181Compiler Construction Context-ree Grammars Using grammars in parsers CG 1 Parsing Process Call the scanner to get tokens Build a parse tree from the stream of tokens A parse tree shows the syntactic
More informationn Top-down parsing vs. bottom-up parsing n Top-down parsing n Introduction n A top-down depth-first parser (with backtracking)
Announcements n Quiz 1 n Hold on to paper, bring over at the end n HW1 due today n HW2 will be posted tonight n Due Tue, Sep 18 at 2pm in Submitty! n Team assignment. Form teams in Submitty! n Top-down
More informationGrammars and Context-free Languages; Chomsky Hierarchy
Regular and Context-free Languages; Chomsky Hierarchy H. Geuvers Institute for Computing and Information Sciences Version: fall 2015 H. Geuvers Version: fall 2015 Huygens College 1 / 23 Outline Regular
More informationCS 314 Principles of Programming Languages
CS 314 Principles of Programming Languages Lecture 7: LL(1) Parsing Zheng (Eddy) Zhang Rutgers University February 7, 2018 Class Information Homework 3 will be posted this coming Monday. 2 Review: Top-Down
More informationComputer Science 160 Translation of Programming Languages
Computer Science 160 Translation of Programming Languages Instructor: Christopher Kruegel Building a Handle Recognizing Machine: [now, with a look-ahead token, which is LR(1) ] LR(k) items An LR(k) item
More informationCompiler Principles, PS7
Top-Down Parsing Compiler Principles, PS7 Parsing problem definition: The general parsing problem is - given set of rules and input stream (in our case scheme token input stream), how to find the parse
More informationSyntactical analysis. Syntactical analysis. Syntactical analysis. Syntactical analysis
Context-free grammars Derivations Parse Trees Left-recursive grammars Top-down parsing non-recursive predictive parsers construction of parse tables Bottom-up parsing shift/reduce parsers LR parsers GLR
More informationCS153: Compilers Lecture 5: LL Parsing
CS153: Compilers Lecture 5: LL Parsing Stephen Chong https://www.seas.harvard.edu/courses/cs153 Announcements Proj 1 out Due Thursday Sept 20 (2 days away) Proj 2 out Due Thursday Oct 4 (16 days away)
More informationEXAM. CS331 Compiler Design Spring Please read all instructions, including these, carefully
EXAM Please read all instructions, including these, carefully There are 7 questions on the exam, with multiple parts. You have 3 hours to work on the exam. The exam is open book, open notes. Please write
More informationCreating a Recursive Descent Parse Table
Creating a Recursive Descent Parse Table Recursive descent parsing is sometimes called LL parsing (Left to right examination of input, Left derivation) Consider the following grammar E TE' E' +TE' T FT'
More informationCISC4090: Theory of Computation
CISC4090: Theory of Computation Chapter 2 Context-Free Languages Courtesy of Prof. Arthur G. Werschulz Fordham University Department of Computer and Information Sciences Spring, 2014 Overview In Chapter
More informationSyntax Analysis - Part 1. Syntax Analysis
Syntax Analysis Outline Context-Free Grammars (CFGs) Parsing Top-Down Recursive Descent Table-Driven Bottom-Up LR Parsing Algorithm How to Build LR Tables Parser Generators Grammar Issues for Programming
More informationMA/CSSE 474 Theory of Computation
MA/CSSE 474 Theory of Computation CFL Hierarchy CFL Decision Problems Your Questions? Previous class days' material Reading Assignments HW 12 or 13 problems Anything else I have included some slides online
More information(NB. Pages are intended for those who need repeated study in formal languages) Length of a string. Formal languages. Substrings: Prefix, suffix.
(NB. Pages 22-40 are intended for those who need repeated study in formal languages) Length of a string Number of symbols in the string. Formal languages Basic concepts for symbols, strings and languages:
More informationContext Free Grammars
Automata and Formal Languages Context Free Grammars Sipser pages 101-111 Lecture 11 Tim Sheard 1 Formal Languages 1. Context free languages provide a convenient notation for recursive description of languages.
More informationTasks of lexer. CISC 5920: Compiler Construction Chapter 2 Lexical Analysis. Tokens and lexemes. Buffering
Tasks of lexer CISC 5920: Compiler Construction Chapter 2 Lexical Analysis Arthur G. Werschulz Fordham University Department of Computer and Information Sciences Copyright Arthur G. Werschulz, 2017. All
More informationLL(1) Grammar and parser
Crafting a Compiler with C (XI) 資科系 林偉川 LL(1) Grammar and parser LL(1) grammar is the class of CFG and is suitable for RDP. Define LL(1) parsers which use an LL(1) parse table rather than recursive procedures
More informationCompiling Techniques
Lecture 6: 9 October 2015 Announcement New tutorial session: Friday 2pm check ct course webpage to find your allocated group Table of contents 1 2 Ambiguity s Bottom-Up Parser A bottom-up parser builds
More informationComputing if a token can follow
Computing if a token can follow first(b 1... B p ) = {a B 1...B p... aw } follow(x) = {a S......Xa... } There exists a derivation from the start symbol that produces a sequence of terminals and nonterminals
More informationTalen en Compilers. Johan Jeuring , period 2. October 29, Department of Information and Computing Sciences Utrecht University
Talen en Compilers 2015-2016, period 2 Johan Jeuring Department of Information and Computing Sciences Utrecht University October 29, 2015 12. LL parsing 12-1 This lecture LL parsing Motivation Stack-based
More informationINF5110 Compiler Construction
INF5110 Compiler Construction Parsing Spring 2016 1 / 84 Overview First and Follow set: general concepts for grammars textbook looks at one parsing technique (top-down) [Louden, 1997, Chap. 4] before studying
More informationCompiler Construction Lent Term 2015 Lectures (of 16)
Compiler Construction Lent Term 2015 Lectures 13 --- 16 (of 16) 1. Return to lexical analysis : application of Theory of Regular Languages and Finite Automata 2. Generating Recursive descent parsers 3.
More informationCompiler Construction Lent Term 2015 Lectures (of 16)
Compiler Construction Lent Term 2015 Lectures 13 --- 16 (of 16) 1. Return to lexical analysis : application of Theory of Regular Languages and Finite Automata 2. Generating Recursive descent parsers 3.
More informationCompiler Construction Lectures 13 16
Compiler Construction Lectures 13 16 Lent Term, 2013 Lecturer: Timothy G. Griffin 1 Generating Lexical Analyzers Source Program Lexical Analyzer tokens Parser Lexical specification Scanner Generator LEX
More informationNotes for Comp 497 (454) Week 10
Notes for Comp 497 (454) Week 10 Today we look at the last two chapters in Part II. Cohen presents some results concerning the two categories of language we have seen so far: Regular languages (RL). Context-free
More informationComputer Science 160 Translation of Programming Languages
Computer Science 160 Translation of Programming Languages Instructor: Christopher Kruegel Top-Down Parsing Top-down Parsing Algorithm Construct the root node of the parse tree, label it with the start
More informationContext-Free Grammars and Languages. Reading: Chapter 5
Context-Free Grammars and Languages Reading: Chapter 5 1 Context-Free Languages The class of context-free languages generalizes the class of regular languages, i.e., every regular language is a context-free
More informationFoundations of Informatics: a Bridging Course
Foundations of Informatics: a Bridging Course Week 3: Formal Languages and Semantics Thomas Noll Lehrstuhl für Informatik 2 RWTH Aachen University noll@cs.rwth-aachen.de http://www.b-it-center.de/wob/en/view/class211_id948.html
More informationTHEORY OF COMPILATION
Lecture 04 Syntax analysis: top-down and bottom-up parsing THEORY OF COMPILATION EranYahav 1 You are here Compiler txt Source Lexical Analysis Syntax Analysis Parsing Semantic Analysis Inter. Rep. (IR)
More informationFollow sets. LL(1) Parsing Table
Follow sets. LL(1) Parsing Table Exercise Introducing Follow Sets Compute nullable, first for this grammar: stmtlist ::= ε stmt stmtlist stmt ::= assign block assign ::= ID = ID ; block ::= beginof ID
More informationParsing. Context-Free Grammars (CFG) Laura Kallmeyer. Winter 2017/18. Heinrich-Heine-Universität Düsseldorf 1 / 26
Parsing Context-Free Grammars (CFG) Laura Kallmeyer Heinrich-Heine-Universität Düsseldorf Winter 2017/18 1 / 26 Table of contents 1 Context-Free Grammars 2 Simplifying CFGs Removing useless symbols Eliminating
More informationCompilerconstructie. najaar Rudy van Vliet kamer 124 Snellius, tel rvvliet(at)liacs.
Compilerconstructie najaar 2012 http://www.liacs.nl/home/rvvliet/coco/ Rudy van Vliet kamer 124 Snellius, tel. 071-527 5777 rvvliet(at)liacs.nl werkcollege 9, dinsdag 27 november 2012 SLR Parsing / Backpatching
More informationComputational Models - Lecture 4
Computational Models - Lecture 4 Regular languages: The Myhill-Nerode Theorem Context-free Grammars Chomsky Normal Form Pumping Lemma for context free languages Non context-free languages: Examples Push
More informationSection 1 (closed-book) Total points 30
CS 454 Theory of Computation Fall 2011 Section 1 (closed-book) Total points 30 1. Which of the following are true? (a) a PDA can always be converted to an equivalent PDA that at each step pops or pushes
More informationContext Sensitive Grammar
Context Sensitive Grammar Aparna S Vijayan Department of Computer Science and Automation December 2, 2011 Aparna S Vijayan (CSA) CSG December 2, 2011 1 / 12 Contents Aparna S Vijayan (CSA) CSG December
More informationAdministrivia. Test I during class on 10 March. Bottom-Up Parsing. Lecture An Introductory Example
Administrivia Test I during class on 10 March. Bottom-Up Parsing Lecture 11-12 From slides by G. Necula & R. Bodik) 2/20/08 Prof. Hilfinger CS14 Lecture 11 1 2/20/08 Prof. Hilfinger CS14 Lecture 11 2 Bottom-Up
More informationLR(1) Parsers Part III Last Parsing Lecture. Copyright 2010, Keith D. Cooper & Linda Torczon, all rights reserved.
LR(1) Parsers Part III Last Parsing Lecture Copyright 2010, Keith D. Cooper & Linda Torczon, all rights reserved. LR(1) Parsers A table-driven LR(1) parser looks like source code Scanner Table-driven Parser
More informationI 1 : {S S } I 2 : {S X ay, Y X } I 3 : {S Y } I 4 : {X b Y, Y X, X by, X c} I 5 : {X c } I 6 : {S Xa Y, Y X, X by, X c} I 7 : {X by } I 8 : {Y X }
Let s try building an SLR parsing table for another simple grammar: S XaY Y X by c Y X S XaY Y X by c Y X Canonical collection: I 0 : {S S, S XaY, S Y, X by, X c, Y X} I 1 : {S S } I 2 : {S X ay, Y X }
More informationSyntax Analysis (Part 2)
Syntax Analysis (Part 2) Martin Sulzmann Martin Sulzmann Syntax Analysis (Part 2) 1 / 42 Bottom-Up Parsing Idea Build right-most derivation. Scan input and seek for matching right hand sides. Terminology
More informationThis lecture covers Chapter 5 of HMU: Context-free Grammars
This lecture covers Chapter 5 of HMU: Context-free rammars (Context-free) rammars (Leftmost and Rightmost) Derivations Parse Trees An quivalence between Derivations and Parse Trees Ambiguity in rammars
More informationFORMAL LANGUAGES, AUTOMATA AND COMPUTABILITY
15-453 FORMAL LANGUAGES, AUTOMATA AND COMPUTABILITY Chomsky Normal Form and TURING MACHINES TUESDAY Feb 4 CHOMSKY NORMAL FORM A context-free grammar is in Chomsky normal form if every rule is of the form:
More informationParsing VI LR(1) Parsers
Parsing VI LR(1) Parsers N.B.: This lecture uses a left-recursive version of the SheepNoise grammar. The book uses a rightrecursive version. The derivations (& the tables) are different. Copyright 2005,
More informationNotes for Comp 497 (Comp 454) Week 10 4/5/05
Notes for Comp 497 (Comp 454) Week 10 4/5/05 Today look at the last two chapters in Part II. Cohen presents some results concerning context-free languages (CFL) and regular languages (RL) also some decidability
More informationParsing with Context-Free Grammars
Parsing with Context-Free Grammars Berlin Chen 2005 References: 1. Natural Language Understanding, chapter 3 (3.1~3.4, 3.6) 2. Speech and Language Processing, chapters 9, 10 NLP-Berlin Chen 1 Grammars
More informationLecture VII Part 2: Syntactic Analysis Bottom-up Parsing: LR Parsing. Prof. Bodik CS Berkley University 1
Lecture VII Part 2: Syntactic Analysis Bottom-up Parsing: LR Parsing. Prof. Bodik CS 164 -- Berkley University 1 Bottom-Up Parsing Bottom-up parsing is more general than topdown parsing And just as efficient
More informationCourse Script INF 5110: Compiler construction
Course Script INF 5110: Compiler construction INF5110, spring 2018 Martin Steffen ii Contents Contents 4 Parsing 1 4.1 Introduction to parsing........................ 1 4.2 Top-down parsing...........................
More informationCFLs and Regular Languages. CFLs and Regular Languages. CFLs and Regular Languages. Will show that all Regular Languages are CFLs. Union.
We can show that every RL is also a CFL Since a regular grammar is certainly context free. We can also show by only using Regular Expressions and Context Free Grammars That is what we will do in this half.
More informationLecture 12 Simplification of Context-Free Grammars and Normal Forms
Lecture 12 Simplification of Context-Free Grammars and Normal Forms COT 4420 Theory of Computation Chapter 6 Normal Forms for CFGs 1. Chomsky Normal Form CNF Productions of form A BC A, B, C V A a a T
More informationSyntax Analysis, VI Examples from LR Parsing. Comp 412
COMP 412 FALL 2017 Syntax Analysis, VI Examples from LR Parsing Comp 412 source code IR IR target code Front End Optimizer Back End Copyright 2017, Keith D. Cooper & Linda Torczon, all rights reserved.
More informationCS20a: summary (Oct 24, 2002)
CS20a: summary (Oct 24, 2002) Context-free languages Grammars G = (V, T, P, S) Pushdown automata N-PDA = CFG D-PDA < CFG Today What languages are context-free? Pumping lemma (similar to pumping lemma for
More informationFrom EBNF to PEG. Roman R. Redziejowski. Concurrency, Specification and Programming Berlin 2012
Concurrency, Specification and Programming Berlin 2012 EBNF: Extended Backus-Naur Form A way to define grammar. EBNF: Extended Backus-Naur Form A way to define grammar. Literal = Decimal Binary Decimal
More informationCMPT-825 Natural Language Processing. Why are parsing algorithms important?
CMPT-825 Natural Language Processing Anoop Sarkar http://www.cs.sfu.ca/ anoop October 26, 2010 1/34 Why are parsing algorithms important? A linguistic theory is implemented in a formal system to generate
More informationType 3 languages. Type 2 languages. Regular grammars Finite automata. Regular expressions. Type 2 grammars. Deterministic Nondeterministic.
Course 7 1 Type 3 languages Regular grammars Finite automata Deterministic Nondeterministic Regular expressions a, a, ε, E 1.E 2, E 1 E 2, E 1*, (E 1 ) Type 2 languages Type 2 grammars 2 Brief history
More informationFORMAL LANGUAGES, AUTOMATA AND COMPUTABILITY
15-453 FORMAL LANGUAGES, AUTOMATA AND COMPUTABILITY REVIEW for MIDTERM 1 THURSDAY Feb 6 Midterm 1 will cover everything we have seen so far The PROBLEMS will be from Sipser, Chapters 1, 2, 3 It will be
More informationChomsky Normal Form and TURING MACHINES. TUESDAY Feb 4
Chomsky Normal Form and TURING MACHINES TUESDAY Feb 4 CHOMSKY NORMAL FORM A context-free grammar is in Chomsky normal form if every rule is of the form: A BC A a S ε B and C aren t start variables a is
More informationLR2: LR(0) Parsing. LR Parsing. CMPT 379: Compilers Instructor: Anoop Sarkar. anoopsarkar.github.io/compilers-class
LR2: LR(0) Parsing LR Parsing CMPT 379: Compilers Instructor: Anoop Sarkar anoopsarkar.github.io/compilers-class Parsing - Roadmap Parser: decision procedure: builds a parse tree Top-down vs. bottom-up
More informationCA Compiler Construction
CA4003 - Compiler Construction Bottom Up Parsing David Sinclair Bottom Up Parsing LL(1) parsers have enjoyed a bit of a revival thanks to JavaCC. LL(k) parsers must predict which production rule to use
More informationCompiler Design 1. LR Parsing. Goutam Biswas. Lect 7
Compiler Design 1 LR Parsing Compiler Design 2 LR(0) Parsing An LR(0) parser can take shift-reduce decisions entirely on the basis of the states of LR(0) automaton a of the grammar. Consider the following
More informationCompiler Design Spring 2017
Compiler Design Spring 2017 3.4 Bottom-up parsing Dr. Zoltán Majó Compiler Group Java HotSpot Virtual Machine Oracle Corporation 1 Bottom up parsing Goal: Obtain rightmost derivation in reverse w S Reduce
More informationEinführung in die Computerlinguistik
Einführung in die Computerlinguistik Context-Free Grammars (CFG) Laura Kallmeyer Heinrich-Heine-Universität Düsseldorf Summer 2016 1 / 22 CFG (1) Example: Grammar G telescope : Productions: S NP VP NP
More informationIntroduction to Theory of Computing
CSCI 2670, Fall 2012 Introduction to Theory of Computing Department of Computer Science University of Georgia Athens, GA 30602 Instructor: Liming Cai www.cs.uga.edu/ cai 0 Lecture Note 3 Context-Free Languages
More informationChapter 4: Context-Free Grammars
Chapter 4: Context-Free Grammars 4.1 Basics of Context-Free Grammars Definition A context-free grammars, or CFG, G is specified by a quadruple (N, Σ, P, S), where N is the nonterminal or variable alphabet;
More informationTHEORY OF COMPUTATION (AUBER) EXAM CRIB SHEET
THEORY OF COMPUTATION (AUBER) EXAM CRIB SHEET Regular Languages and FA A language is a set of strings over a finite alphabet Σ. All languages are finite or countably infinite. The set of all languages
More informationParsing with CFGs L445 / L545 / B659. Dept. of Linguistics, Indiana University Spring Parsing with CFGs. Direction of processing
L445 / L545 / B659 Dept. of Linguistics, Indiana University Spring 2016 1 / 46 : Overview Input: a string Output: a (single) parse tree A useful step in the process of obtaining meaning We can view the
More informationParsing with CFGs. Direction of processing. Top-down. Bottom-up. Left-corner parsing. Chart parsing CYK. Earley 1 / 46.
: Overview L545 Dept. of Linguistics, Indiana University Spring 2013 Input: a string Output: a (single) parse tree A useful step in the process of obtaining meaning We can view the problem as searching
More informationShift-Reduce parser E + (E + (E) E [a-z] In each stage, we shift a symbol from the input to the stack, or reduce according to one of the rules.
Bottom-up Parsing Bottom-up Parsing Until now we started with the starting nonterminal S and tried to derive the input from it. In a way, this isn t the natural thing to do. It s much more logical to start
More informationAn Efficient Context-Free Parsing Algorithm. Speakers: Morad Ankri Yaniv Elia
An Efficient Context-Free Parsing Algorithm Speakers: Morad Ankri Yaniv Elia Yaniv: Introduction Terminology Informal Explanation The Recognizer Morad: Example Time and Space Bounds Empirical results Practical
More informationBottom-up syntactic parsing. LR(k) grammars. LR(0) grammars. Bottom-up parser. Definition Properties
Course 9-10 1 Bottom-up syntactic parsing Bottom-up parser LR(k) grammars Definition Properties LR(0) grammars Characterization theorem for LR(0) LR(0) automaton 2 a 1... a i... a n # X 1 X 1 Control Parsing
More informationUNRESTRICTED GRAMMARS
136 UNRESTRICTED GRAMMARS Context-free grammar allows to substitute only variables with strings In an unrestricted grammar (or a rewriting system) one may substitute any non-empty string (containing variables
More informationCompiler construction
Compiler construction Martin Steffen February 20, 2017 Contents 1 Abstract 1 1.1 Parsing.................................................. 1 1.1.1 First and follow sets.....................................
More informationINF5110 Compiler Construction
INF5110 Compiler Construction Spring 2017 1 / 330 Outline 1. Parsing First and follow sets Top-down parsing Bottom-up parsing References 2 / 330 INF5110 Compiler Construction Parsing Spring 2017 3 / 330
More informationPushdown Automata. Reading: Chapter 6
Pushdown Automata Reading: Chapter 6 1 Pushdown Automata (PDA) Informally: A PDA is an NFA-ε with a infinite stack. Transitions are modified to accommodate stack operations. Questions: What is a stack?
More informationCS 406: Bottom-Up Parsing
CS 406: Bottom-Up Parsing Stefan D. Bruda Winter 2016 BOTTOM-UP PUSH-DOWN AUTOMATA A different way to construct a push-down automaton equivalent to a given grammar = shift-reduce parser: Given G = (N,
More informationBottom-up syntactic parsing. LR(k) grammars. LR(0) grammars. Bottom-up parser. Definition Properties
Course 8 1 Bottom-up syntactic parsing Bottom-up parser LR(k) grammars Definition Properties LR(0) grammars Characterization theorem for LR(0) LR(0) automaton 2 a 1... a i... a n # X 1 X 1 Control Parsing
More informationBottom-Up Parsing. Ÿ rm E + F *idÿ rm E +id*idÿ rm T +id*id. Ÿ rm F +id*id Ÿ rm id + id * id
Bottom-Up Parsing Attempts to traverse a parse tree bottom up (post-order traversal) Reduces a sequence of tokens to the start symbol At each reduction step, the RHS of a production is replaced with LHS
More information