First Class Traversal Idioms with Nested Attribute Grammars

Transcription

1 First Class Traversal Idioms with Nested Attribute Grammars Arie Middelkoop Dept. of Information and Computing Sciences, Utrecht University P.O. Box , 3508 TB Utrecht, The Netherlands Web pages: 15 April 2010 & Software Technology Colloquium

2 Introduction 2

3 Main Question Software component for name analysis? 3

4 Main Question Software component for name analysis? Why? Pattern that occurs often in compilers Software engineering practice (e.g. elimination of code duplication) 3

5 Main Question Software component for name analysis? Why? Pattern that occurs often in compilers Software engineering practice (e.g. elimination of code duplication) Actual question: mechanism for any traversal idiom? More intriguing idiom: damas-milner inference 3

6 4

7 Warning! Work in progress! Controversial/not a silver bullet. I hope this talk leads to: 5 References to related material Improvements to the concepts Improvements to understanding and explanation Ideas for a nicer implementation

8 Related Work This talk is related to attribute grammars: Higher-order Attribute Grammars Remote Attributes / Reference Attributes... and probably many other formalisms/technologies 6

9 Background 7

10 Name Analysis Concrete syntax: let x = 1 f = λx.x in f x Abstract syntax: Let [Decl "x" (Val 1), Decl "f" (Lam "x" (Var "x"))] (App (Var "f") (Var "x"))) Questions: 8 All identifiers defined? No duplicate definitions? What is the type of an identifier?

11 Name Analysis applied - gather environment let app x = f x val f = nil λx x 9

12 Name Analysis applied - gather environment let app x = (x, τ 1 ) f x val f = nil λx x 9

13 Name Analysis applied - gather environment let app x = (x, τ 1 ) f x val f = nil λx x 9

14 Name Analysis applied - gather environment let app x = (x, τ 1 ) f x val f = nil λx (x, τ 2 ) x 9

15 Name Analysis applied - gather environment let app x = (x, τ 1 ) f x val f = (f, τ 3 ) nil λx (x, τ 2 ) x 9

16 Name Analysis applied - gather environment let app x = (x, τ 1 ) (f, τ 3 ) f x val f = (f, τ 3 ) nil λx (x, τ 2 ) x 9

17 Name Analysis applied - gather environment let (x, τ 1 ) (f, τ 3 ) app x = (x, τ 1 ) (f, τ 3 ) f x val f = (f, τ 3 ) nil λx (x, τ 2 ) x 9

18 Name Analysis applied - gather environment let (x, τ 1) (f, τ 3 ) (x, τ 1 ) (f, τ 3 ) app x = (x, τ 1 ) (f, τ 3 ) f x val f = (f, τ 3 ) nil λx (x, τ 2 ) x 9

19 Name Analysis applied - distribute environment let (x, τ 1) (f, τ 3 ) app x = f x val f = nil λx (x, τ 2 ) x 10

20 Name Analysis applied - distribute environment let (x, τ 1) (f, τ 3 ) (x, τ 1 ) (f, τ 3 ) app x = f x val f = nil λx (x, τ 2 ) x 10

21 Name Analysis applied - distribute environment let (x, τ 1) (f, τ 3 ) (x, τ 1 ) (f, τ 3 ) app x = f x val f = nil (x, τ 1 ) (f, τ 3 ) λx (x, τ 2 ) (x, τ 2 ) (f, τ 3 ) x 10

22 Name Analysis applied - distribute environment let (x, τ 1) (f, τ 3 ) (x, τ 1 ) (f, τ 3 ) (x, τ 1 ) (f, τ 3 ) app x = (x, τ 1 ) (f, τ 3 ) f x val f = nil (x, τ 1 ) (f, τ 3 ) λx (x, τ 2 ) (x, τ 2 ) (f, τ 3 ) x 10

23 Name Analysis applied - distribute environment let (x, τ 1) (f, τ 3 ) (x, τ 1 ) (f, τ 3 ) (x, τ 1 ) (f, τ 3 ) app x = (x, τ 1 ) (f, τ 3 ) f x val f = nil (x, τ 1 ) (f, τ 3 ) λx (x, τ 2 ) (x, τ 2 ) (f, τ 3 ) x 10

24 Name Analysis applied - distribute environment let (x, τ 1) (f, τ 3 ) (x, τ 1 ) (f, τ 3 ) (x, τ 1 ) (f, τ 3 ) app x = f = (x, τ 1 ) (f, τ 3 ) f x val f = nil (x, τ 1 ) (f, τ 3 ) λx (x, τ 2 ) (x, τ 2 ) (f, τ 3 ) x 10

25 Implementation - parser pexpr :: Parser Token T Expr pexpr = sem Expr Let $ pkey "let" pdecls pkey "in" pexpr sem Expr Lam $ pkey "\\" pident pkey "." pexpr pexprapps pexprapps = pchainl sem Expr App pexprbase pexprbase = sem Expr Var $ pident sem Expr Val $ pvalue pparens pexpr pdecls = pfoldr (sem Decls Cons, sem Decls Nil) pdecl pdecl = sem Decl Decl $ pident pkey "=" pexpr 11

26 Implementation - tructors (1) sem Expr Var nm = sem :: Expr... sem Expr Val val = sem :: Expr... sem Expr App l r = sem :: Expr... sem Expr Lam nm b = sem :: Expr... sem Expr Let ds b = sem :: Expr... sem Decls Cons d ds = sem :: Decls... sem Decls Nil = sem :: Decls... sem Decl Decl nm e = sem :: Decl... 12

27 Implementation - tructors (1) sem Expr Var nm = sem :: Expr... sem Expr Val val = sem :: Expr... sem Expr App l r = sem :: Expr... sem Expr Lam nm b = sem :: Expr... sem Expr Let ds b = sem :: Expr... sem Decls Cons d ds = sem :: Decls... sem Decls Nil = sem :: Decls... sem Decl Decl nm e = sem :: Decl... sem Expr Lam :: String T Expr T Expr sem Expr Let :: T Decls T Expr T Expr 12

28 Implementation - interfaces itf Expr Decls Decl visit gather syn gathenv :: [(String, Ty)] visit distribute inh finenv :: [(String, Ty)] syn errors :: [String ] order gather < distribute 13

29 Implementation - interfaces itf Expr Decls Decl visit gather syn gathenv :: [(String, Ty)] visit distribute inh finenv :: [(String, Ty)] syn errors :: [String ] order gather < distribute newtype T Expr st =... data Inh Expr = Inh Expr {finenv Inh Expr :: [(String, Ty)]} data Syn Expr = Syn Expr {gathenv Syn Expr :: [(String, Ty)], errors Syn Expr :: [String ] } eval Expr :: (forall st.t Expr st) Inh Expr Syn Expr 13

30 Implementation - Constructors (2) sem Expr Var nm = sem :: Expr loc.mbval = lookup nm lhs.finenv lhs.errors = maybe ["missing: " + nm ] (t [ ]) loc.mbval lhs.gathenv = 14

31 Implementation - Constructors (2) sem Expr Var nm = sem :: Expr loc.mbval = lookup nm lhs.finenv lhs.errors = maybe ["missing: " + nm ] (t [ ]) loc.mbval lhs.gathenv = sem Expr Lam nm b = sem :: Expr child body :: Expr = b loc.(envwith, envwithout) = partition (( nm).fst) lhs.finenv loc.duperrs = if length loc.envwith > 1 then ["dup: " + nm ] lhs.errors = loc.duperrs + body.errors loc.binding = [(nm, loc.argtype)] body.finenv = loc.binding + loc.envwithout lhs.gathenv = filter (( nm).fst) body.gathenv 14

32 Implementation - Constructors (2) sem Expr App l r = sem :: Expr child left :: Expr = l child right :: Expr = r left.finenv = lhs.finenv right.finenv = lhs.finenv lhs.gathenv = left.gathenv + right.gathenv lhs.errors = left.errors + right.errors 15

33 Implementation - Constructors (2) sem Expr App l r = sem :: Expr child left :: Expr = l child right :: Expr = r left.finenv = lhs.finenv right.finenv = lhs.finenv lhs.gathenv = left.gathenv + right.gathenv lhs.errors = left.errors + right.errors Other cases: similar... 15

34 Name Analysis in UHC EH AST has 39 tructors that deal with names Core AST about 16 tructors Lost the count in the HS AST Many more ASTs in UHC... think of all other compilers Code duplication! 16

35 Components 17

36 Towards Attributed Trees as Components (1) visit gather syn gathenv let visit distribute inh finenv syn errs Expr decl Decls val 18

37 Towards Attributed Trees as Components (1) node visit gather syn gathenv let visit distribute inh finenv syn errs Expr decl Decls val 18

38 Towards Attributed Trees as Components (1) node sem :: RealTree loc.c =... child comp :: Comp = loc.c comp.finenv = [ ] visit gather syn gathenv let visit distribute inh finenv syn errs Expr decl Decls val 18

39 Show stoppers 1. Verbose AST Expressions, declarations: too fine grained Essence of name analysis: binding sites, use sites, scoping 2. Interfacing only with root insufficient Access to the values found for identifiers Access to the error messages at one particular node 3. Extend a component? Mismatch between offered and needed functionality Opening up the component, without affecting the original Overriding attributes, adding new attributes 4. Intuitiveness and predictability of code? Delegation of work: finished when results are needed? Functional code (side effect free/order independent) 19

40 More abstract AST 20

41 Coarser AST let app x = f x val f = nil λx 21 x

42 Coarser AST sc let split split app split x = def x split use f f use x val f = def f nil sc λx split 21 x def x use x

43 Coarser AST sc split def x use x 22

44 More Abstract Code 23 itf Name α visit gather syn gathenv use { +} {[ ]} :: [(String, α)] visit distribute inh finenv :: [(String, α)] syn errors use { +} {[ ]} :: [String ] sem Name Scope b = sem :: Name α child body :: Name α = b body.finenv = (lhs.finenv \\ (map fst body.gathenv)) + body.gathenv sem Name Split l r = sem :: Name α child left :: Name α = l child right :: Name α = r sem Name Def x v = sem :: Name α lhs.gathenv = [(x, v)] lhs.errors = if length (filter (( x).fst) lhs.finenv) > 1 then ["dup: " + x ] else [ ] sem Name Use x = sem :: Name α loc.mbval = lookup x lhs.finenv lhs.errors = if isnothing loc.mbval then ["missing: " + x ] else [ ] loc.val = maybe id loc.mbval -- how to access this??

45 Instantiation of the component itf Expr syn nmtree :: T Name st Ty sem Expr Lam x b = sem :: Expr child body :: Expr = b loc.τ =... lhs.nmtree = sem Name Scope $ sem Name Split (sem Name Def x loc.τ) $ body.nmtree sem Expr Var x = sem :: Expr lhs.nmtree = sem Name Use x sem Root Root b = sem :: Root child body :: Expr = b child name :: Name Ty = body.nmtree lhs.errors = name.errors 24

46 Interface to Subtrees 25

47 Towards Attributed Trees as Components (2) itf Name visit gather syn gathenv visit distribute inh finenv sc split itf NameDef visit supply syn nm syn value visit outcome inh errors def x use x itf NameUse visit supply syn nm visit outcome inh value inh errors 26

48 Instantiation and Exchange root app sc split var use 27

49 Instantiation and Exchange root app sc var value, errors nm use split 27

50 Nested AG Code itf NameUse α visit supply syn nm :: String visit outcome inh value :: α inh errors :: [String ] sem Name Use exttree = sem :: Name α child ext :: NameUse α = exttree loc.mbval = lookup ext.nm lhs.finenv ext.value = maybe id loc.mbval ext.errors = if isnothing loc.mbval then ["missing: " + ext.nm ] sem Expr Var nm = sem :: Expr lhs.nmtree = sem Name Use $ sem :: NameUse Type -- nested sem lhs.nm = nm loc.errors = lhs.errors loc.value = lhs.value... = loc.errors -- local attributes... = loc.value -- shared with nested sem 28

51 Referentially transparent? Unable to guarantee that nested sem is executed exactly once We can guarantee referential transparency Requirement: code is orderable Then: an expression for an attribute can be replaced with its value, without affecting the program Referential transparency is important! 29

52 Component Extensions Extend via extra filter nodes 30

53 Example: let-bound polymorphism sc split body decls 31

54 Example: let-bound polymorphism let sc split body decls 31

55 Example: let-bound polymorphism let sc split body decls 31

56 Example: let-bound polymorphism - code 32 sem Expr Let ds b = sem :: Expr lhs.nmtree = loc.semtop $ sem Name Scope $ sem Name Split decls.nmtree $ loc.sembody body.nmtree loc.semtop t = sem :: Name Ty child body :: Name Ty = t loc.parentenv = lhs.finenv loc.sembody t = sem :: Name Ty child body :: Name Ty = t body.finenv = generalize lhs.finenv loc.parentenv

57 AG convenience Override one or two attributes with filter nodes Copy rules take care of the rest 33

58 Limitations 34

59 Order Analysis 35

60 Interfaces and Dependencies finenv :: Env Name: gather distribute gathenv :: Env nmtree.nameuse Main: name supply outcome type nmtree :: T Name st Ty τ :: Ty 36

61 Conclusions 37

62 Type Check Idiom name gen gen main gen type + sig gen kind extends gen type type+kind extends 38

63 Discussion performance views idioms generics proofs functional backend examples UUAG tradeoffs validation usability 39

64 Conclusion Traversal components: Produce an attributed tree Interface with nodes of this attributed tree Extend functionality with filter nodes 40