Type 3 languages. Type 2 languages. Regular grammars Finite automata. Regular expressions. Type 2 grammars. Deterministic Nondeterministic.

Transcription

1 Course 7 1

2 Type 3 languages Regular grammars Finite automata Deterministic Nondeterministic Regular expressions a, a, ε, E 1.E 2, E 1 E 2, E 1*, (E 1 ) Type 2 languages Type 2 grammars 2

3 Brief history of programming Stages of compilation Lexical analysis Lexical description Interpretation Right oriented interpretation Well formed lexical description Bottom up parsing General bottom-up parser LR analysis LR(0) 3

4 Programs written using processor instructions Few computers Few programmers 4

5 Fortran (1957): First compiler (artihmetic expressions, instructions, procedures) Still used for computationally complex applications or for benchmarking systems Algol (1958): BNF grammars (Backus-Naur Normal Form), instruction blocks, recursion, etc. Precursor of modern syntax Lisp (1958) Functional programming Tree data structures, automatic storage management, dynamic typing, higher-order functions, recursion COBOL (1959) Similar to English language Business oriented Emphasis on inputs and outputs 5

6 Simula (1965) Based on ALGOL 60 First object oriented language Objects, classes, inheritance, virtual functions, etc. Structured programming (1968) Edsger Dijkstra GOTO Considered Harmful Pascal (1970) C (1973) IRQ, dynamic variables, multitasking 6

7 ADA (1980) First standardized language Objective C (1984) Inspired by Smalltalk Object oriented C++ (1985) C with Classes; Object oriented, exceptions, templates Inspired by Simula Java (1995) just-in-time compilation C# (2000 ).NET framework 7

8 Source code Characters Lexical analyzer Lexemes Syntactic analyzer Syntactic tree Semantic analyzer Processor code Decorated syntactic tree Code generator Intermediary code Assembler Interpreter 8

9 Def. 1 Let Σ be an alphabet (of a programming language). A lexical description over is a regular expression E = (E 1 E 2 E n ) +, whre n is the number of lexical units, and E i describes a lexical unit, 1 i n. Def. 2 Let E be a lexical description over Σ which contains n lexical units and w +. The word w isccorrect relative to the description E if w L(E). An interpretation of the word w L(E) is a sequence of pairs (u 1, k 1 ), (u 2, k 2 ),, (u m, k m ), where w = u 1 u 2 u m, u i L(E ki ) 1 i m, 1 ki n. 9

10 w = alpha := beta = 542 Interpretations for the word w: (alpha, Id), (:=, Assign), (beta, Id), (=, Equal), (542, Integer) (alp, Id), (ha, Id), (:=, Assign), (beta, Id), (=, Equal), (542, Integer) (alpha, Id), (:, Colon), ( =, Equal), (beta, Id), (=, Equal), (542, Integer) 10

11 Def. 3 Let E be a lexical description over and w L(E). An interpretation of the word w, (u 1, k 1 )(u 2, k 2 ), (u m, k m ), is a right-oriented interpretation if ( i) 1 i m: u i = max{ v, v L(E 1 E 2 E n ) Pref(u i u i+1 u m )}. (where Pref(w) is the set of prefixes for the word w ). There are some lexical descriptions E in which not every word of L(E) allows for a right-oriented interpretation. E = (a ab bc) + şi w = abc. 11

12 Def. 4 A lexical description E is well-formed if every word w L(E) has exactly one rightoriented interpretation. Theorem Given a lexical description E, it can be determined if it is well-formed. Def. 5 Let E be a well-formed lexical description over. A lexical analyzer (scanner) for E is a program that recognizes the language L(E) and determines, for each w L(E), its right-oriented interpretation. 12

13 Let E be a lexical description over Σ. To produce a lexical analyzer for E means to: 1. Build the finite automaton A, equivalent to E 2. From A, obtain the deterministic automaton equivalent to E, A. 3. (Optional) Obtain the minimal automaton equivalent to A. 4. Implement the automaton A. 13

14 Lexical description: letter a b z digit identifier letter (letter digit)* sign + - number (sign ε) digit+ operator + - * / < > <= >= < > assign := Colon : reserved_words if then else parenthesis ) ( 14

15 A i A n A o q 0 A a A : A r A p 15

16 letter, digit letter 1 # i or # r digit 2 digit # n +,- 3 digit # o operator {+,-} 4 # o : ),( = # a # : # p 0 16

17 Source code Characters Lexical analyzer Lexemes Syntactic analyzer Syntactic tree Semantic analyzer Processor code Decorated syntactic tree Code generator Intermediary code Assembler Interpreter 17

18 a 1... a i... a n # X 1 X 1 Control Parsing table... # p 1 p

19 A configuration ( #γ, u#, π) is described as follows: #γ is the stack content, with the # symbol at the bottom. u# is the input. π is the output. C 0 = {(#, w#, ε) w ε T*} is the set of initial configurations. 19

20 The bottom-up parser attached to a grammar G is the pair (C 0, ) where C 0 is the set of initial configurations and is the trasition relation, defined below: (# γ, au#, π) (# γa, u#, π) (shift) for any γ ε Σ*, a ε T, u ε T*, π ε P*. (#αβ, u#, π) (#αa, u#, πr) if r = A β ε P(reduce). The configuration (#S, #, π), where π ε, is called acceptance configuration. Any configuration which is not an acceptance configuration and which is not in a relation to another configuration is an error configuration. Shift /reduce parsers. 20

21 S asb ε. Possible transitions are: (#γ, u#, π) (#γs, u#, π2) (#γasb, u#, π) (#γs, u#, π1) (#γ, au#, π) (#γa, u#, π) (#γ, bu#, π) (#γb, u#, π) A sequence of transitions, such as the one below, is called a calculation (#, #, ε) (#S, #, 2) (#, aabb#, ε) (#a, abb#, ε) (#aa, bb#, ε) (#aas, bb#, 2) (#aasb, b#, 2) (#as, b#, 21) (#asb, #, 21) (#S, #, 211) 21

22 The parser is non-deterministic: For any configuration (#αβ, au#, π), S β, there exist two possibilities (shift/reduce conflict): (#αβ, au#, π) (#αs, au#, πr) (reduce with S β) (#αβ, au#, π) (# αβa, u#, π) (shift) For any configuration (#γ, u#, π) where γ=α 1 β 1 =α 2 β 2 and A β 1, B β 2 ε P (reduce/reduce conflict) (#α 1 β 1, u#, π) (# α 1 A, u#, πr 1 ) (#α 2 β 2, u#, π) (# α 2 B, u#, πr 2 ) 22

23 A word wεt* is accepted by a bottom-up parser if there exits at least one calculation (#, w#, ε) + (#S, #, π) The described parser is correct if it will accept all the words from the set L(G), and only those words. Theorem The bottom-up parser attached to a grammar G is correct: ( ) wεt*, wεl(g) iff the calculation (#, w#, ε) + (#S, #, π) is correct. 23

24 LR(k) grammars: Left to right scanning of the input, constructing a Rightmost derivation in reverse, using k symbols lookahead Definition A grammar G is LR(k), k 0, if, for any two derivations : S S dr * αau dr αβu = δu S S dr * α A u dr α β u = αβv = δv where k:u = k:v, then α=α, β=β, A=A 24

25 Theorem 1 If the grammar G is LR(k), k 0, then G is not ambiguos. A language is in the class of LR(k) if there exists a LR(k) grammar that can generate it Theorem 2 Any LR(k) language is a type 2 deterministic language. Theorem 3 Any type 2 deterministic language is in the class LR(1). Theorem 4 For any language in LR(k), k 1, there exists a LR(1) grammar that can generate that language, i.e. LR(0) LR(1) = LR(k), k 1. 25

26 Definition Let G = (V, T, S, P) be a type 2 reduced grammar. Let the symbol Σ (T N). An article for the grammar G is a production rule A γ to which the symbol has been added on some position in γ. An article is denoted A α β if γ = αβ. An article with the symbol to the rightmost position is called a complete article. Definition A viable prefix for the grammar G is any prefix of a word αβ if S r * αau r αβu. If β= β 1 β 2 and φ= αβ 1 the article A β 1 β 2 is valid for the viable prefix φ. 26

27 S A, A aaa bab c ε. Articles: S A, S A, A aaa, A a Aa, A aa a, A aaa, A bab, A b Ab, A ba b, A bab, A c, A c, A. Valid articles for viable prefixes: Viable prefix Valid article Derivation ab ε A b Ab A aaa A bab S A A bab A c S A aaa ababa S A aaa ababa abaaaba S A aaa ababa abbabba S A S A bab S A c 27

28 Lemma Let G be a grammar and A β 1 Bβ 2 a valid article for the viable prefix γ. Then, B β ε P, the article B β is valid for γ. Theorem (characterization of LR(0)) The grammar G is LR(0) iff, γ a viable prefix, the following are true: 1. two complete articles valid for γ. 2. if the article A β is valid for γ, B β 1 aβ 2, aεt, valid for γ. 28

29 Theorem Let G = (V, T, S, P) be a type 2 grammar. The set of viable prefixes for the grammar G is a type 3 language. Proof G is G augmented with the S S rule. M = (Q, Σ, δ, q 0, Q), where: Q is the set of articles of G, Σ = V T, q 0 = S S δ:qx(σ {ε}) 2 Q defined below: δ(a α Bβ, ε) = {B α B γ εp}. δ(a α Xβ, X) = { A αx β}, X ε Σ. δ(a α aβ, ε) =, a ε T. δ(a α Xβ, Y) =, X,Y ε Σ cu X Y. We show that: (A α β ε δ ^(q 0, γ ) γ is a viable prefix and A α β is valid for γ. 29

30 S S, S asa bsb c 30

31 Algorithm 1(procedure for closure) Input: G = (V, T, S, P); The set of articles of G; Output: t =closure( t)={qεq pεt, qε δ(p,ε) = δ(t,ε)} 31

32 t = t ; flag = true; while( flag ) { flag = false; for (A α Bβ ε t ) { for (B γ ε P ) if (B γ t ) { t = t {B γ}; flag = true; }//endif }//endforb }//endfora }//endwhile return t ; 32

33 Algorithm 2 - LR(0) automaton Input: G = (N, T, S, P) augmented with the rule S S; Output: M = (T, Σ, g, t 0, T), deterministic automaton equivalent to M. 33

34 t 0 =closure(s S); T={t 0 }; marked(t 0 )=false; while( t ε T &&!marked(t)) { // marked(t) = false for( X ε Σ) {// Σ = N T t = ; for(a α Xβ εt) t = t {B αx β A α Xβ εt}; if( t ){ t = closure( t ); if( t T ) { T = T { t }; marked( t ) = false; }//endif g(t, X) = t ; }//endif }//endfor }//endfor marked( t ) = true; }// endwhile 34

35 S S, S asa bsb c 35

36 Definition: Let G be a grammar and M the LR(0) automaton attached to G. A state of M has a reduce/reduce conflict if it contains two complete and distinct articles A α, B β. A state of M has a shift/reduce conflict if it contains a complete article A α and an incomplete article with a terminal to the right of the symbol B β aγ. A state is consistent if it does not have any conflicts and is inconsistent otherwise. Theorem Let G be a grammar and M its LR(0) automaton. The grammar G is LR(0) iff the automaton M does not contain any inconsistent states. 36

37 S aad bab, A ca c, B d 37

38 The parsing table is the LR(0) automaton, M. Configuration: (σ, u#, π) where σεt 0 T*, uεt*, πεp*. Initial configuration (t 0, w#, ε), Transitions: Shift: (σt, au#, π) (σtt, u#, π) if g(t, a) = t. Reduce: (σtσ t, u#, π) ( σtt, u#, πr) if A β ε t, r = A β, σ t = β şi t = g (t, A). Accept: (t 0 t 1, #, π) is the acceptance configuration if S S ε t1, π is the parsing of the word. Error: a configuration from which no transitions are possible 38

39 char ps[]= w# ; //ps is the input string w i = 0; // position in the input string STACK.push(t0); // the stack is initialized with t0 while(true) { // repeat until accept or error t = STACK.top(); a = ps[i] // a is the current symbol at the input if( g(t, a) ){ //shift STACK.push(g(t, a)); i++; //move forward in the input string } else { if(a X 1 X 2 X m t){ if(a == S ) if(a == # )exit( accept ); else exit( error ); else // reduce for( i = 1; i <= m; i++) STACK.pop(); STACK.push(g(top(STACK), A)); } //endif else exit( error ); }//endelse }//endwhile 39

40 S S S E$ E E+T T (E) E T T a 40

41 S S S E$ E E+T T (E) E T T a Stack Input Action Output 0 a+(a+a)$# shift 05 +(a+a)$# reduce T a 03 +(a+a)$# reduce E T 02 +(a+a)$# shift 027 (a+a)$# shift 0274 a+a)$# shift a)$# reduce T a a)$# reduce E T a)$# shift a)$# shift )$# reduce T a )$# reduce E E+T )$# shift 02748'10' $# reduce T (E) 0279 $# reduce E E+T 02 $# shift 026 # reduce S E$ 01 # accept 41

42 Lemma 1, 2 Let G = (N, T, S, P) be an LR(0), t 0 σ, t 0 τ paths in the LR(0) automaton, labeled with φ and γ respectively and u, v ε T*. It follows that, if in the LR(0) parser the calculation (t 0 σ, uv#, ε) + (t 0 τ, v#, π) holds, then the derivation φ dr π u holds for the grammar G, and reciprocally. Theorem If G is an LR(0) grammar, then for any word w ε T*, the LR(0) parser will reach an acceptance configuration for w, i.e. (t 0 σ, uv#, ε) + (t 0 τ, v#, π) iff φ dr π u. 42

43 Grigoraş Gh., Construcţia compilatoarelor. Algoritmi fundamentali, Editura Universităţii Alexandru Ioan Cuza, Iaşi,