The Course covers: Lexical Analysis Syntax Analysis Semantic Analysis Runtime environments Code Generation Code Optimization. CS 540 Spring 2013 GMU 2

Transcription

1 CS 540 Spring 2013

2 Th Cours covrs: Lxicl Anlysis Syntx Anlysis Smntic Anlysis Runtim nvironmnts Cod Gnrtion Cod Optimiztion CS 540 Spring 2013 GMU 2

3 Pr-rquisit courss Strong progrmming ckground in C, C++ or Jv CS 310 Forml Lngug (NFAs, DFAs, CFG) CS 330 Assmly Lngug Progrmming nd Mchin Architctur CS 367 CS 540 Spring 2013 GMU 3

4 Oprtionl Informtion Offic: Enginring Building, Rm E-mil: Clss W Pg: Blckord Discussion ord: Piss Computr Accounts on zus.vs.gmu.du (link on Usful Links ) CS 540 Spring 2013 GMU 4

5 CS 540 Cours Grding Progrmming Assignmnts (45%) 5% + 10% + 10% + 20% Exms midtrm nd finl (25%, 30%) CS 540 Spring 2013 GMU 5

6 Rsourcs Txtooks: Compilrs: Principls, Tchniqus nd Tools, Aho, Lm, Sthi & Ullmn, 2007 (rquird) lx & ycc, Lvin t. l. Slids Smpl cod for Lx/YACC (C, C++, Jv) CS 540 Spring 2013 GMU 6

7 Distnc Eduction CS 540 Spring 13 sssion is dlivrd to th Intrnt sction (Sction 540-DL) onlin y NEW Studnts in distnc sction will ccss to onlin lcturs nd cn ply ck th lcturs nd downlod th PDF slid fils Th distnc duction studnts will givn th midtrm nd finl xm on cmpus, on th sm dy/tim s in clss studnts. Exm loctions will nnouncd closr to th xm dts. CS 540 Spring 2013 GMU 7

8 Lctur 1: Introduction to Lngug Procssing & Lxicl Anlysis CS 540

9 Wht is compilr? A progrm tht rds progrm writtn in on lngug nd trnslts it into nothr lngug. Sourc lngug Trgt lngug Trditionlly, compilrs go from high-lvl lngugs to low-lvl lngugs. CS 540 Spring 2013 GMU 9

10 Compilr Architctur In mor dtil: Intrmdit Lngug Sourc Lngug Front End lngug spcific Bck End mchin spcific Trgt Lngug Sprtion of Concrns Rtrgting CS 540 Spring 2013 GMU 10

11 Compilr Architctur Intrmdit Lngug Intrmdit Lngug Sourc lngug Scnnr (lxicl nlysis) tokns Prsr (syntx nlysis) Syntctic structur Smntic Anlysis (IC gnrtor) Cod Optimizr Cod Gnrtor Trgt lngug Symol Tl CS 540 Spring 2013 GMU 11

12 Lxicl Anlysis - Scnning Sourc lngug Scnnr (lxicl nlysis) tokns Prsr (syntx nlysis) Smntic Anlysis (IC gnrtor) Cod Gnrtor Cod Optimizr Tokns dscrid formlly Brks input into tokns Rmov whit spc Symol Tl CS 540 Spring 2013 GMU 12

13 Input: rsult = + * c / d Tokns: rsult, =,, +,, *, c, /, d idntifirs oprtors CS 540 Spring 2013 GMU 13

14 Sttic Anlysis - Prsing Sourc lngug Scnnr (lxicl nlysis) tokns Prsr (syntx nlysis) Syntctic structur Smntic Anlysis (IC gnrtor) Cod Gnrtor Trgt lngug Cod Optimizr Syntx dscrid formlly Tokns orgnizd into syntx tr tht dscris structur Error chcking (syntx) Symol Tl CS 540 Spring 2013 GMU 14

15 Input: rsult = + * c / d Exp ::= Exp + Exp Exp - Exp Exp * Exp Exp / Exp ID Assign ::= ID = Exp Assign ID = Exp Exp + Exp ID Exp * Exp ID Exp / Exp ID ID CS 540 Spring 2013 GMU 15

16 Smntic Anlysis Sourc lngug Scnnr (lxicl nlysis) Prsr (syntx nlysis) Syntctic structur Smntic Anlysis (IC gnrtor) Syntctic/smntic structur Cod Gnrtor Trgt lngug Syntctic/smntic structur Cod Optimizr Mning Typ/Error Chcking Intrmdit Cod Gnrtion strct mchin Symol Tl CS 540 Spring 2013 GMU 16

17 Optimiztion Sourc lngug Scnnr (lxicl nlysis) Prsr (syntx nlysis) Syntctic/smntic structur Smntic Anlysis (IC gnrtor) Cod Optimizr Cod Gnrtor Syntctic/smntic structur Trgt lngug Improving fficincy (mchin indpndnt) Finding optiml cod is NP Symol Tl CS 540 Spring 2013 GMU 17

18 Cod Gnrtion Sourc lngug Scnnr (lxicl nlysis) Prsr (syntx nlysis) Syntctic/smntic structur Smntic Anlysis (IC gnrtor) Cod Gnrtor Trgt lngug Cod Optimizr Syntctic/smntic structur IC to rl mchin cod Mmory mngmnt, rgistr lloction, instruction slction, instruction schduling, Symol Tl CS 540 Spring 2013 GMU 18

19 Issus Driving Compilr Dsign Corrctnss Spd (runtim nd compil tim) Dgrs of optimiztion Multipl psss Spc Fdck to usr Dugging CS 540 Spring 2013 GMU 19

20 Rltd to Compilrs Intrprtrs (dirct xcution) Assmlrs Prprocssors Txt formttrs (non-wysiwyg) Anlysis tools CS 540 Spring 2013 GMU 20

21 Why study compilrs? Bring togthr: Dt structurs & Algorithms Forml Lngugs Computr Architctur Influnc: Lngug Dsign Architctur (influnc is i-dirctionl) Tchniqus usd influnc othr rs (progrm nlysis, tsting, ) CS 540 Spring 2013 GMU 21

22 Rviw of Forml Lngugs Rgulr xprssions, NFA, DFA Trnslting twn formlisms Using ths formlisms CS 540 Spring 2013 GMU 22

23 Wht is lngug? Alpht finit chrctr st (S) String finit squnc of chrctrs cn, th mpty string (Som txts us l s th mpty string) Lngug possily infinit st of strings ovr som lpht cn { }, th mpty lngug. CS 540 Spring 2013 GMU 23

24 Suppos S = {,,c}. Som lngugs ovr S could : {,,c,,c,cc} {,c,cc,ccc,...} { } { } {,,c,} CS 540 Spring 2013 GMU 24

25 Why do w cr out Rgulr Lngugs? Formlly dscri tokns in th lngug Rgulr Exprssions NFA DFA Rgulr Exprssions finit utomt Tools ssist in th procss CS 540 Spring 2013 GMU 25

26 Rgulr Exprssions Th rgulr xprssions ovr finit S r th strings ovr th lpht S + { ), (,, * } such tht: 1. { } (mpty st) is rgulr xprssion for th mpty st 2. is rgulr xprssion dnoting { } 3. is rgulr xprssion dnoting st { } for ny in S CS 540 Spring 2013 GMU 26

27 Rgulr Exprssions 4. If P nd Q r rgulr xprssions ovr S, thn so r: P Q (union) If P dnots th st {,,}, Q dnots th st {0,,9} thn P Q dnots th st {,,,0,,9} PQ (conctntion) If P dnots th st {,,}, Q dnots th st {0,,9} thn PQ dnots th st {0,,0,1,,9} Q* (closur) If Q dnots th st {0,,9} thn Q* dnots th st {,0,,9,00, 99, } CS 540 Spring 2013 GMU 27

28 Exmpls If S = {,} ( )( ) ( )* *** * (lso known s +) (*) (*) CS 540 Spring 2013 GMU 28

29 Nondtrministic Finit Automt A nondtrministic finit utomton (NFA) is mthmticl modl tht consists of 1. A st of stts S 2. A st of input symols S 3. A trnsition function tht mps stt/symol pirs to st of stts: S x {S + } st of S 4. A spcil stt s 0 clld th strt stt 5. A st of stts F (sust of S) of finl stts INPUT: string OUTPUT: ys or no CS 540 Spring 2013 GMU 29

30 Exmpl NFA , S = {0,1,2,3} S 0 = 0 S = {,} F = {3} Trnsition Tl: STATE 0 0, CS 540 Spring 2013 GMU 30

31 NFA Excution An NFA sys ys for n input string if thr is som pth from th strt stt to som finl stt whr ll input hs n procssd. NFA(int s0, int input) { if (ll input procssd && s 0 is finl stt) rturn Ys; if (ll input procssd && s 0 not finl stt) rturn No; for ll stts s 1 whr trnsition(s 0,tl[input]) = s 1 if (NFA(s 1,input_lmnt+1) == Ys) rturn Ys; } for ll stts s 1 whr trnsition(s 0,) = s 1 if (NFA(s 1,input_lmnt) == Ys) rturn Ys; rturn No; Uss cktrcking to srch ll possil pths CS 540 Spring 2013 GMU 31

32 Dtrministic Finit Automt A dtrministic finit utomton (DFA) is mthmticl modl tht consists of 1. A st of stts S 2. A st of input symols S 3. A trnsition function tht mps stt/symol pirs to stt: S x S S 4. A spcil stt s 0 clld th strt stt 5. A st of stts F (sust of S) of finl stts INPUT: string OUTPUT: ys or no CS 540 Spring 2013 GMU 32

33 DFA Excution DFA(int strt_stt) { stt currnt = strt_stt; input_lmnt = nxt_tokn(); whil (input to procssd) { currnt = trnsition(currnt,tl[input_lmnt]) if currnt is n rror stt rturn No; input_lmnt = nxt_tokn(); } if currnt is finl stt rturn Ys; ls rturn No; } CS 540 Spring 2013 GMU 33

34 Rgulr Lngugs 1. Thr is n lgorithm for convrting ny RE into n NFA. 2. Thr is n lgorithm for convrting ny NFA to DFA. 3. Thr is n lgorithm for convrting ny DFA to RE. Ths fcts tll us tht REs, NFAs nd DFAs hv quivlnt xprssiv powr. All thr dscri th clss of rgulr lngugs. CS 540 Spring 2013 GMU 34

35 Convrting Rgulr Exprssions to NFAs Th rgulr xprssions ovr finit S r th strings ovr th lpht S + { ), (,, *} such tht: { } (mpty st) is rgulr xprssion for th mpty st Empty string is rgulr xprssion dnoting { } is rgulr xprssion dnoting { } for ny in S CS 540 Spring 2013 GMU 35

36 Convrting Rgulr Exprssions to NFAs If P nd Q r rgulr xprssions with NFAs N p, N q : P Q (union) N p N q PQ (conctntion) N p N q CS 540 Spring 2013 GMU 36

37 Convrting Rgulr Exprssions to NFAs If Q is rgulr xprssion with NFA N q : Q* (closur) N q CS 540 Spring 2013 GMU 37

38 Strting with: Exmpl (* *)* * 1 2 * 3 4 * * CS 540 Spring 2013 GMU 38

39 Exmpl (* *)* * * (* *)* CS 540 Spring 2013 GMU 39

40 Convrting NFAs to DFAs Id: Ech stt in th nw DFA will corrspond to som st of stts from th NFA. Th DFA will in stt {s 0,s 1, } ftr input if th NFA could in ny of ths stts for th sm input. Input: NFA N with stt st S N, lpht S, strt stt s N, finl stts F N, trnsition function T N : S N x S + {} st of S N Output: DFA D with stt st S D, lpht S, strt stt s D = -closur(s N ), finl stts F D, trnsition function T D : S D x S S D CS 540 Spring 2013 GMU 40

41 -closur() Dfn: -closur(t) = T + ll NFA stts rchl from ny stt in T using only trnsitions closur({1,2,5}) = {1,2,5} -closur({4}) = {1,4} -closur({3}) = {1,3,4} -closur({3,5}) = {1,3,4,5} CS 540 Spring 2013 GMU 41

42 Algorithm: Sust Construction s D = -closur(s N ) S D = {s D } (unmrkd) -- crt strt stt for DFA whil thr is som unmrkd stt R in S D mrk stt R for ll in S do s = -closur(t N (R,)); if s not lrdy in S D thn dd it (unmrkd) T D (R,) = s; nd for nd whil F D = ny lmnt of S D tht contins stt in F N CS 540 Spring 2013 GMU 42

43 Exmpl 1: Sust Construction NFA 1 2, 5 3 4, CS 540 Spring 2013 GMU 43

44 Exmpl 1: Sust Construction NFA 1,2 1 2, 5 3 4, {1,2} CS 540 Spring 2013 GMU 44

45 Exmpl 1: Sust Construction NFA 1,2 4,5 1 2, 5 3,5 3 4, {1,2} {3,5} {4,5} {3,5} {4,5} CS 540 Spring 2013 GMU 45

46 Exmpl 1: Sust Construction NFA 1,2 4,5 1 2, 5 3, , {1,2} {3,5} {4,5} {3,5} - {4} {4,5} {4} CS 540 Spring 2013 GMU 46

47 Exmpl 1: Sust Construction NFA, 1,2 4, , 5 3, , {1,2} {3,5} {4,5} {3,5} - {4} {4,5} {5} {5} {4} {5} CS 540 Spring 2013 GMU 47

48 Exmpl 1: Sust Construction NFA, 1,2 4, , 5 3,5 4, 3 4, All finl stts sinc th NFA finl stt is includd {1,2} {3,5} {4,5} {3,5} - {4} {4,5} {5} {5} {4} {5} {5} {5} - - CS 540 Spring 2013 GMU 48

49 Exmpl 2: Sust Construction NFA CS 540 Spring 2013 GMU 49

50 Exmpl 2: Sust Construction NFA DFA ,3,4 1,3,4,5 1,4,5 CS 540 Spring 2013 GMU 50

51 Exmpl 3: Sust Construction NFA DFA ,2, ,4 3,5 3 4 CS 540 Spring 2013 GMU 51

52 Convrting DFAs to REs 1. Comin sril links y conctntion 2. Comin prlll links y ltrntion 3. Rmov slf-loops y Kln closur 4. Slct nod (othr thn initil or finl) for rmovl. Rplc it with st of quivlnt links whos pth xprssions corrspond to th in nd out links 5. Rpt stps 1-4 until th grph consists of singl link twn th ntry nd xit nods. CS 540 Spring 2013 GMU 52

53 Exmpl 0 d 1 d d c 5 d 6 c 7 prlll dgs com ltrntion 0 d c d d d c 6 7 CS 540 Spring 2013 GMU 53

54 Exmpl 0 d c d d d c 6 7 sril dgs com conctntion 0 d ( c) d 3 4 d 5 ( c) d CS 540 Spring 2013 GMU 54

55 Exmpl 0 d ( c) d 3 4 d 5 ( c) d Find pths tht cn shortnd 0 d ( c) d d ( c)d CS 540 Spring 2013 GMU 55

56 Exmpl 0 d ( c) d d ( c)d limint slf-loops 0 d ( c) d (( c)d)*d sril dgs com conctntion 0 d ( c) d (( c)d)*d 5 CS 540 Spring 2013 GMU 56

57 Dscriing Rgulr Lngugs Gnrt ll strings in th lngug Gnrt only strings in th lngug Try th following: Strings of {,} tht nd with Strings of {,} tht don t nd with Strings of {,} whr vry is followd y t lst on CS 540 Spring 2013 GMU 57

58 Strings of ( )* tht nd in r: ( )* , NFA DFA CS 540 Spring 2013 GMU 58

59 Strings of ( )* tht don t nd in r:?? DFA/NFA CS 540 Spring 2013 GMU 59

60 Strings of ( )* tht don t nd in * * * * * * * * * CS 540 Spring 2013 GMU 60

61 Suggstions for writing NFA/DFA/RE Typiclly, on of ths formlisms is mor nturl for th prolm. Strt with tht nd convrt if ncssry. In NFA/DFAs, ch stt typiclly cpturs som prtil solution B sur tht you includ ll rlvnt dgs (sk dos vry stt hv n outgoing trnsition for ll lpht symols?) CS 540 Spring 2013 GMU 61

62 Non-Rgulr Lngugs Not ll lngugs r rgulr Th lngug ww whr w=( )* Non-rgulr lngugs cnnot dscrid using REs, NFAs nd DFAs. CS 540 Spring 2013 GMU 62