Announcements. H6 posted 2 days ago (due on Tue) Midterm went well: Very nice curve. Average 65% High score 74/75

Transcription

1 Announcements H6 posted 2 days ago (due on Tue) Mterm went well: Average 65% High score 74/75 Very nice curve

2 LR BoHom- up Parsing 2

3 Roadmap Last class Name analysis Previous- ish last class LL(1) Today s class LR Parsing SLR(1) 3

4 Lecture Outline BoHom- Up parsing Talk about the language class / theory Describe the state that it keeps / intuiton Show how it works Show how it is built 4

5 LL(1) Not Powerful Enough for all PL LeW- recursion Not lew factored Doesn t mean LL(1) is bad Right tool for simple parsing jobs stmtlist ::= stmtlist stmt /* epsilon */ ; 5

6 We Need a Li#le More Power Could increase the lookahead Up untl the m 90s, this was consered impractcal Could increase the runtme complexity CYK has us covered there Could increase the memory complexity i.e. more elaborate parse table 6

7 LR Parsers LeW- to- right scan of the input file Reverse rightmost derivaton Advantages Can recognize almost any programming language Time and space O(n) in the input size More powerful than the corresponding LL parser i.e. LL(1) < LR(1) Disadvantages More complex parser generaton Larger parse tables 7

8 LR Parser Power Let S α 1 α 2 w be a rightmost derivaton, where ω is a terminal string Let αaγ αβγ be a step in the derivaton So A β must have been a producton in the grammar αβγ must be some α i or w A grammar is LR(k) if for every derivaton step, A β can be inferred using only a scan of αβ and at most k symbols of γ Much like LL(1), you generally just have to go ahead and try it 8

9 LR Parser types LR(1) Can recognize any DCFG Can experience blowup in parse table size LALR(1) SLR(1) Both proposed at the same Tme to limit parse table size Recognizable by a determinis3c PDA LR LALR SLR 9

10 Which parser should we use? Different variants mostly differ in how they build the parse table, we can stll talk about all the family in general terms Today we ll cover SLR PreHy easy to learn LALR from there LALR(1) Generally consered a good compromise between parse table size and expressiveness Class for Java CUP, yacc, and bison 10

11 How does BoHom- up Parsing work? Already seen 1 such parser: CYK Simultaneously tracked every possible parse tree LR parsers work in a similar same way Contrast to top- down parser We know exactly where we are in the parse Make predictons about what s next 11

12 Parser State Top- down parser state Current token Stack of symbols Represented what we expect in the rest of our descent to the leaves Worked down and to the lew through tree BoHom- up parser state Also maintains a stack and token Represents summary of input we ve seen Works upward and to the right through the tree Also has an auxiliary state machine to help disambiguate rules Grammar S ::= ε ( S ) [ S ] Stack [ S ] ) EOF Current [ 12

13 LR DerivaTon Order Let s remember derivaton orders again Reverse Rightmost derivajon E E + T E + T * F E + T * E + F * E + * E E + T T 6 1 T * F T + * F + * F 7 F *

14 Parser OperaTons Top- down parser Scan the next input token Push a bunch of RHS symbols Pop a single symbol BoHom- up parser Shi2 an input token into a stack item Reduce a bunch of stack items into a new parent item (on the stack) 14

15 Parser AcTons: Simplified view Stack Input Ac3on F T + * EOF + * EOF + * EOF + * EOF shim() reduce by F reduce by T F reduce by E T E E + + * EOF * EOF shim + shim E E + E + F * EOF * EOF reduce by F reduce by T F E + T E + T E + T * * EOF EOF shim * shim T T * F E + T * E + T * F EOF EOF reduce by F reduce by T T * F F F E + T E EOF EOF reduce by E E + T accept 15

16 Stack Items Note that the previous sle was called simplified Stack elements are representatve of symbols Actually known as items Indicate a producton and a positon within the producton X α. B β Means we are in a producton of X We believe we ve parsed (arbitrary) symbol string α We could handle a producton of B AWer that we ll have β 16

17 Stack Item Examples Example 1 PList (. IDList ) Example 2 PList ( IDList. ) Example 3 PList ( IDList ). Example 4 PList. ( IDList ) 17

18 Stack Item State You may not know exactly which item you are parsing LR Parsers actually track the set of states that you could have been in Grammar snippet S A A B C B D C E D E {S. A, A. B, A. C, } 18

19 LR Parser FSM I 0 PList S. PList S PList. PList. ( IDList ) I 1 Grammar G S' PList PList ( IDList ) IDList IDList IDList I 2 PList (. IDList ) IDList. IDList. IDList ( I 3 I 5 PList ( IDList ). PList ( IDList. ) IDList IDList. ) I 4 IDList. I 6 IDList IDList Id. 19

20 ( PList I 0 I 1 I 2 I 5 ) I I 3 4 Automaton as a table Shi4 corresponds to taking a terminal edge Reduce corresponds to taking a nonterminal edge I ( ) eof S 2 Ac3on table S 5 S 4 S 6 PList 1 ShiM and go to state 6 GoTo table IDList 3 20

21 How do we know when to reduce? ( ) eof S 2 Ac3on table S 5 S 4 S 6 R ❸ R ❸ R ❹ R ❹ R ❷ GoTo table PList IDList 1 3 Only see terminals in the input Actually do reduce steps in 2 phases AcTon table will tell us when to reduce (and how much) Grammar G ❶ S' PList ❷ PList ( IDList ) ❸ IDList ❹ IDList IDList GoTo will tell us where to go to 21

22 How do we know we re done? Ac3on table ( ) eof GoTo table PList IDList Add an accept token S 2 S 5 S 4 S 6 R ❸ R ❸ J 1 3 Any other cell is an error 5 R ❷ 6 R ❹ R ❹ Grammar G ❶ S' PList ❷ PList ( IDList ) ❸ IDList ❹ IDList IDList 22

23 Full Parse Table OperaTon Initialize stack a = scan() do forever t = top-of-stack (state) symbol switch action[t, a] { case shift s: push(s) a = scan() case reduce by A alpha: for i = 1 to length(alpha) do pop() end t = top-of-stack symbol push(goto[t, A]) case accept: return( SUCCESS ) case error: call the error handler return( FAILURE ) } end do 23

24 Example Time 24

25 I 2 I 0 S. PList PList S PList. PList. ( IDList ) PList (. IDList ) IDList. IDList. IDList ( PList ( IDList. ) IDList IDList. I 4 IDList. I 6 I 5 I 1 I 3 PList ( IDList ). IDList IDList Id. ) current ( ) eof Grammar G ❶ S' PList ❷ PList ( IDList ) ❸ IDList ❹ IDList IDList [I 65 ] [I 4 3 ] [I 21 ] [I 0 ] ( ) eof PList IDList 0 S S 4 J 3 3 S 5 S 6 4 R ❸ R ❸ 5 R❷ 6 R ❹ R ❹ 25

26 Seems that LR Parser works great What could possible go wrong? 26

27 LR Parser State Explosion Tracking sets of states can cause the size of the FSM to blow up The SLR and LALR variants exist to combat this explosion Slight modificaton to item and table form 27

28 Building the SLR Automaton Uses 2 sets Closure(I) What is the set of items we could be in? Given I: what is the set of items that could be mistaken for I (reflexive) Goto(s,X) If we are in state I, where might we be awer parsing X? Vaguely reminiscent of FIRST and FOLLOW 28

29 Closure Sets Put I itself into Closure(I) While there exists an item in Closure(I) of form X α. B β such that there is a producton B γ and B. γ is not in Closure(I) add B. γ to Closure(I) 29

30 GoTo Sets Goto(I, X) = Closure({ A α X. Β A α. X β is in I }) 30

31 I 2 I 0 PList S. PList S PList. PList. ( IDList ) PList (. IDList ) IDList. IDList. IDList I 4 ( IDList. Parse Table Construc3on 1: Add new start S and S S 2: Build State I 0 for Closure( {S. S } ) 3: Saturate FSM: for each symbol X s.t. there is a item in state j containing. X add transijon from state j to state for GoTo(j, X) I 1 I 3 I 6 I 5 Grammar G S' PList PList ( IDList ) IDList IDList IDList PList ( IDList ). Closure(I) GoTo(I,X) ) Put Closure I in Closure(I) of all items Repeat A for αx.β s.t. A α.xβ ϵ I PList ( IDList. ) X α.bβ ϵ Closure(I) s.t. IDList IDList. B γ, add B.γ to Closure(I) GoTo(I GoTo(I unjl 0, ( ) saturajon 0, PList) all GoTo(I Items (all 2 Items A, IDList α(.β A ) αplist.β) all Items A αidlist.β [1] PList [1] S (. PList IDList. ) IDList IDList Id. those [1] (those PList where where A ( IDList α.(β A.) ϵ α.plist I β ϵ I Closure{ S. PList} = { 0 ) [2] IDList IDList. 0 for [1] for PList [1] S. (. IDList PList is ) is in in I S. PList 0 I 0 set those to (take closure where closure is A of α.idlistβ the following): ϵ I 2 for [1] PList (rules (. PList IDList ) is. γ) in I { PList { S (. PList IDList.}) } 2 for [2] PList PList. IDList. ( IDList also ) in I Items (adds IDList nothing). γ where IDList γ 2 ϵ G } { IDList set = to { S closure. PList is.} { PList ( IDList. ), IDList ( IDList. ) } IDList. IDList } Only terminals aser. so closure done Done with closure, and GoTo 31

32 From FSM to parse table(s) I 0 PList S. PList S PList. PList. ( IDList ) I 1 Need to connect the FSM back to the grammar I 2 PList (. IDList ) IDList. IDList. IDList I 4 ( IDList. I 3 I 6 I 5 PList ( IDList ). PList ( IDList. ) IDList IDList. IDList IDList Id. ) Grammar G ❶ S' PList ❷ PList ( IDList ) ❸ IDList ❹ IDList IDList 32

33 Can Now Build AcTon and GoTo Tables I 0 PList S. PList S PList. PList. ( IDList ) I 1 I 2 PList (. IDList ) IDList. IDList. IDList ( I 3 I 5 PList ( IDList ). PList ( IDList. ) IDList IDList. ) I 4 IDList. I 6 IDList IDList Id. 33

34 Building the GoTo Table I 2 I 0 PList S. PList S PList. PList. ( IDList ) PList (. IDList ) IDList. IDList. IDList I 4 ( IDList. I 1 I 3 I 6 I 5 PList ( IDList ). PList ( IDList. ) IDList IDList. IDList IDList Id. ) For every nonterminal X if there is an (i,j) edge on X set GoTo[i,X] = j PList IDList

35 Building the AcTon Table If state i includes item A α. t β - where t is a terminal - and there is an (i,j) transiton on t - set AcTon[i,t] = shiw j If state i includes item A α. - where A is not S - for each t in FOLLOW(A): - set AcTon[i,t] = reduce by A α If state i includes item S S. - set AcTon[i, eof] = accept All other entries are error actons 35

36 AcTon Table: ShiW I 2 I 0 PList S. PList S PList. PList. ( IDList ) PList (. IDList ) IDList. IDList. IDList I 4 ( IDList. I 1 I 3 I 6 I 5 PList ( IDList ). PList ( IDList. ) IDList IDList. IDList IDList Id. ) if state i includes item A α. t β where t is a terminal and there is an (i,j) transijon on t set AcJon[i,t] = shim j ( ) eof S 2 S 5 S 4 S 6 36

37 AcTon Table: Reduce I 2 I 0 PList S. PList S PList. PList. ( IDList ) PList (. IDList ) IDList. IDList. IDList I 4 ( IDList. I 1 I 3 I 6 I 5 PList ( IDList ). PList ( IDList. ) IDList IDList. IDList IDList Id. Grammar G ❶ S' PList ❷ PList ( IDList ) ❸ IDList ❹ IDList IDList ) if state i includes item A α. where A is not S for each t in FOLLOW(A): set AcJon[i,t] = reduce by A α FOLLOW(IDList) = { ), } FOLLOW(PList) = { eof } ( ) eof S 2 S 5 S 4 S 6 R ❸ R ❸ R ❹ R ❹ R ❷ 37

38 AcTon Table: Accept I 0 PList S. PList S PList. PList. ( IDList ) I 1 if state i includes item S S. set AcJon[i,eof] = accept I 2 ( I 5 PList ( IDList ). PList (. IDList ) IDList. IDList. IDList I 3 PList ( IDList. ) IDList IDList. ) 0 1 ( ) eof S 2 J I 4 IDList. I 6 IDList IDList Id S 5 S 4 S 6 R ❸ R ❸ Grammar G ❶ S' PList ❷ PList ( IDList ) ❸ IDList ❹ IDList IDList 5 6 R ❹ R ❹ R ❷ 38

39 Some Final Thoughts on LR Parsing A bit complicated to build the parse table Fortunately, algorithms exist STll not as powerful as CYK ShiW/reduce: acton table cell includes S and R Reduce/reduce: cell include > 1 R rule SDT similar to LL(1) Embed SDT acton numbers in acton table Fire off on reduce rules 39