Talen en Compilers. Johan Jeuring , period 2. October 29, Department of Information and Computing Sciences Utrecht University

Transcription

1 Talen en Compilers , period 2 Johan Jeuring Department of Information and Computing Sciences Utrecht University October 29, 2015

2 12. LL parsing 12-1

3 This lecture LL parsing Motivation Stack-based parsing Good expansion choices LL(1) grammars Computing empty, first, and follow 12-2

4 12.1 Motivation 12-3

5 Time complexity of parsing Earley s algorithm can parse any context-free language. It runs in O (n 3 ) where n is the length of the input. (It behaves better on many important special cases.) Our parser combinators can parse a large class of context-free languages, but no left-recursive grammars. They are fast for many important classes of grammars, but slow for ambiguous grammars or grammars that are not left-factored. A DFA can parse any regular language in linear time. Are there more languages we can parse in linear time? 12-4

6 12.2 Stack-based parsing 12-5

7 Parsing with a stack machine The LL parsing algorithm works with a state consisting of a stack containing a sequence of terminals and nonterminals, the current input. We often write (α, x) to denote the state consisting of the stack α and input x. 12-6

8 Parsing with a stack machine contd. We start with state (S, x) where S is the start symbol and x is the input string. In each step, we do one of the following: Expand If the top symbol on the stack is a nonterminal, we replace it with a corresponding right hand side of a production. Match If the top symbol on the stack is a terminal and the first symbol of the input matches, we remove the symbol from both stack and input. If both stack and input are empty, we succeed. In any other case, we signal an error. 12-7

9 Example S as cs b Let us parse aab: stack input S aab initial state expand, but which production? 12-8 cs aab one option match fails, thus error as aab other option match succeeds S ab expand again as ab match succeeds S b expand b b match ε ε parse successful

10 The name LL The machine traverses the input starting from the left (first L ) The machine produces a leftmost derivation (second L ). 12-9

11 Acceptance by an LL stack machine The LL machine is nondeterministic: in every expand step, we have to choose one of the productions for the given nonterminal, the choice can have an effect on whether the parse succeeds or not. An LL machine accepts a word if any sequence of choices leads to success (similar to NFA acceptance)

12 12.3 Good expansion choices 12-11

13 Idea By grammar analysis, let us try to determine which expansions make sense and which do not. For example, recall: S as cs b We started at: stack S input aab By looking at the first symbol in the input (a), we know that only the production as can lead to potential success. Can we do something similar for other grammars? 12-12

14 Example 1 S ca b A cbc bsa a B cc Cb C as ba Let us parse ccccba. The first set of C consists of a and b. S ccccba initial state, expand ca ccccba match A cccba expand cbc cccba match BC ccba expand, but how? ccc ccba match cc cba match C ba expand ba ba match the rest a a ε ε parse successful 12-13

15 Example 2 S aba aa A bb bs Let us parse abbb. S abbb expand, but how? aba abbb match twice ba bbb A bb expand, but how? bb bb match the rest b b ε ε parse successful The first symbol of both productions for S is a, we need the first two symbols to resolve the choice. We again need the first set of S to resolve the choice for A

16 Example 3 S AaS B A cs ε B b Let us parse acbab. S acbab expand, but how? AaS acbab expand, but how? as acbab match S cbab expand AaS cbab expand csas cbab match SaS bab rest is easy Because A can derive the empty string, the production S AaS can start with a. Because A can derive ε, we have to know which symbols can follow on an A in a derivation

17 Quiz 1,2,

18 12.4 LL(1) grammars 12-17

19 Definition of LL(1) grammars Definition The lookahead set of a production N α is defined as lookahead (N α) = {x S γnδ γαδ γxβ } Definition A grammar is called LL(1) if all pairs of productions of the same nonterminal have disjoint lookahead sets, i.e., if lookahead (N α) lookahead (N β) = for all pairs of different productions N α, N β

20 Example 1 revisited Before we learn how to compute lookahead systematically, some simple examples: S ca b A cbc bsa a B cc Cb C as ba This grammar is LL(1). lookahead (S ca) = {c} lookahead (S b) = {b} lookahead (A cbc) = {c} lookahead (A bsa) = {b} lookahead (A a) = {a} lookahead (B cc) = {c} lookahead (B Cb) = {a, b} lookahead (C as) = {a} lookahead (C ba) = {b}

21 Example 2 revisited S aba aa A bb bs lookahead (S aba) = {a} lookahead (S aa) = {a} lookahead (A bb) = {b} lookahead (A bs) = {b} This grammar is not LL(1) (but it is LL(2))

22 The need for grammar analysis In order to determine lookahead sets systematically, we need the following information: whether or not a nonterminal can derive the empty string, which terminals can appear as the first symbol in a string derived from a nonterminal, which terminals can follow a nonterminal in any derivation

23 Derivation of the empty string Definition A nonterminal N can derive the empty string if N ε. We thus define empty N = N ε Example S AaS B A cs ε B b Here, empty A holds, but empty S and empty B do not

24 First sets Definition The first set of a nonterminal N is the set of terminals that can appear as the first symbol of a string derived from N: first N = {x N xβ } Example S AaS B A cs ε B b first S = {a, b, c} first A = {c} first B = {b} 12-23

25 Follow set Definition The follow set of a nonterminal N is the set of terminals that can follow N in any derivation starting with the start symbol S: follow N = {x S αnxβ } Example S AaS B A cs ε B b follow S = {a} follow A = {a} follow B = {a} 12-24

26 12.5 Computing empty, first, and follow 12-25

27 Computing empty The following equations hold: empty n A = A β (emptyrhs n β) emptyrhs n ε = True emptyrhs n (Bβ) = empty n 1 B emptyrhs n β emptyrhs n (xβ) = False This recursive definition cannot be used directly as an implementation why not? However, we can read it as an iterative algorithm. To start, we define empty 0 A = False 12-26

28 Computing empty continued If we look at the sequence of sets {A empty 0 A} {A empty 1 A} {A empty 2 A}... then: The first set is empty. Each set is at least as large as the previous. We reach a fixed point sooner or later. (Why?) The fixed point contains the information we are interested in

29 Computing first We use the same approach: first 0 A = first n A = A β (firstrhs n β) firstrhs n ε = ε firstrhs n (Bβ) = first n 1 B if empty B then firstrhs n β else firstrhs n (xβ) = {x} The sequence {(A, first n A) A nonterminal} will reach a fixed point

30 Computing follow Once again the same approach: follow 0 A = follow n A = B αaβ (followrhs n B β) followrhs n B β = firstrhs β if emptyrhs β then follow n 1 B else The sequence {(A, follow n A) A nonterminal} will reach a fixed point

31 Computing lookahead Using first and follow, we can finally define lookahead: lookahead (A β) = followrhs A β 12-30

32 Quiz 4,