Typed metaprogramming with effects

Size: px

Start display at page:

Download "Typed metaprogramming with effects"

Miranda Washington
5 years ago
Views:

1 Typed metaprogramming with effects Chung-chieh Shan (Rutgers University) with Chris Barker, Yukiyoshi Kameyama, Oleg Kiselyov LFMTP 14 July 2010

2 2/32 Accidental language design Documents Fonts T E X Pages laid-out

3 2/32 Accidental language design Documents Fonts T E X Pages laid-out

4 2/32 Accidental language design Formulas Models Evaluation, inference Truth, entailment

Accidental language design Formulas Models Evaluation, inference Truth, entailment It is probably more perspicuous to proceed indirectly, by 1.

5 Accidental language design Formulas Models Evaluation, inference Truth, entailment It is probably more perspicuous to proceed indirectly, by 1. setting up a certain simple artificial language, that of tensed intensional logic, 2. giving the semantics of that language, and 3. interpreting English indirectly by showing in a rigorous way how to translate it into the artificial language. This is the procedure we shall adopt... Richard Montague 2/32

6 2/32 Accidental language design General programs Static data Code generator Specialized programs

7 Accidental language design General programs Static data Code generator Specialized programs Optimizations specific to... Gaussian elimination Fast Fourier Transform Linear signal processing Embedded devices Generate code using... Binding-time annotations Extensible compilers Side effects Custom generators 2/32

8 Accidental language design Code generators Static data 2-level language Specialized programs Optimizations specific to... Gaussian elimination Fast Fourier Transform Linear signal processing Embedded devices Generate code using... Binding-time annotations Extensible compilers Side effects Custom generators 2/32

9 Accidental language design ATLAS generates optimized code for matrix multiplication: for (j=0; j < nu; j++) { for (i=0; i < mu; i++) { if (Asg1stC &&!k) fprintf(fpout, "%s %s%d_%d = %s%d * %s%d;\n", spc, rc, i, j, ra, i, rb, j); else fprintf(fpout, "%s %s%d_%d += %s%d * %s%d;\n", spc, rc, i, j, ra, i, rb, j); opfetch(fpout, spc, nfetch, ra, rb, pa, pb, mu, nu, offa, offb, lda, ldb, mula, mulb, rowa, rowb, &ia, &ib); } } 3/32

10 4/32 Accidental language design Code generators Static data 2-level language Specialized programs Want safety: generate well-formed programs only track object variable bindings Want clarity: generators resemble textbook algorithms provide delimited control operators

11 5/32 Outline Delimited control for program generation Example Formalization Natural-language semantics Delimited control Quotation Variable binding Breaking the fourth wall Contextual modalities Environment classifiers

12 Gibonacci example Like Fibonacci, but not always starting with 1 and 1. let gib x y = let rec loop n = if n = 0 then x else if n = 1 then y else loop (n-1) + loop (n-2) in loop gib Other domains: Gaussian elimination Fast Fourier Transform Linear signal processing Embedded devices... 6/32

13 7/32 Gibonacci example, specialized Familiar from quasiquotation, macros, PE, or just printf. let gib x y = let rec loop n = if n = 0 then x else if n = 1 then y else.<.~(loop (n-1)) +.~(loop (n-2))>. in loop.<fun x y ->.~(gib.<x>..<y>. 5)>..<fun x_0 -> fun y_1 -> ((((y_1 + x_0) + y_1) + (y_1 + x_0)) + ((y_1 + x_0) + y_1))>. Code values can be open when evaluating under generated, but the generated code is always well-scoped. Binding context follows evaluation context, implicitly!

14 Gibonacci example, memoized Keep a memo table as mutable state. let gib x y = let memo = new_memo () in let rec loop n = if n = 0 then x else if n = 1 then y else memo loop (n-1) + memo loop (n-2) in loop gib Other domain-specific optimizations: Dynamic programming Pivoting matrices Simplifying arithmetic on complex roots of unity... 8/32

15 9/32 Gibonacci example, specialized, memoized? A naive combination duplicates code, as when unfolding in PE. let gib x y = let memo = new_memo () in let rec loop n = if n = 0 then x else if n = 1 then y else.<.~(memo loop (n-1)) +.~(memo loop (n-2))>. in loop.<fun x y ->.~(gib.<x>..<y>. 5)>..<fun x_0 -> fun y_1 -> ((((y_1 + x_0) + y_1) + (y_1 + x_0)) + ((y_1 + x_0) + y_1))>. Generating code fast is not generating fast code!

16 Two problems 1. Code in state voids safety, due to scope extrusion. let r = ref.<1>. in.<fun y ->.~(r :=.<y>.;.<()>.)>.;!r.<y_1>. 2. Need to insert let at top, not to duplicate specialized code..<fun x y ->.~(gib.<x>..<y>. 4)>..<fun x_0 -> fun y_1 -> let t_2 = y_1 + x_0 in let t_3 = t_2 + y_1 in t_3 + t_2>. 10/32

17 10/32 Two problems 1. Code in state voids safety, due to scope extrusion. let r = ref.<1>. in.<fun y ->.~(r :=.<y>.;.<()>.)>.;!r.<y_1>. 2. Need to insert let at top, not to duplicate specialized code..<fun x y ->.~(gib.<x>..<y>. 4)>. loop 2.<fun x_0 -> fun y_1 -> let t_2 = y_1 + x_0 in let t_3 = t_2 + y_1 in t_3 + t_2>. loop 3 (Similar: need to insert if/assert.)

18 Two solutions 1. Use CPS or monadic style to write the generator. (Match compiler, CPS translator (Danvy & Filinski), PE (Bondorf)) let gib x y = let rec loop n k = if n = 0 then k x else if n = 1 then k y else memo loop (n-1) (fun r1 -> memo loop (n-2) (fun r2 -> k.<.~r1 +.~r2>.)) in loop 11/32

19 Two solutions 1. Use CPS or monadic style to write the generator. (Match compiler, CPS translator (Danvy & Filinski), PE (Bondorf)) let gib x y = let rec loop n k = if n = 0 then k x else if n = 1 then k y else memo loop (n-1) (fun r1 -> memo loop (n-2) (fun r2 -> k.<.~r1 +.~r2>.)) in loop loop 2 k table loop 3 k table 0.<let t_2 = y_1 + x_0 in.~(k.<t_2>. table 0 )>..<let t_3 = t_2 + y_1 in.~(k.<t_3>. table 00 )>. Importing k under let is ok because code is opaque! 11/32

20 Two solutions 1. Use CPS or monadic style to write the generator. (Match compiler, CPS translator (Danvy & Filinski), PE (Bondorf)) let gib x y = let rec loop n k = if n = 0 then k x else if n = 1 then k y else memo loop (n-1) (fun r1 -> memo loop (n-2) (fun r2 -> k.<.~r1 +.~r2>.)) in loop.<fun x y ->.~(top_fn (gib.<x>..<y>. 5))>..<fun x_0 -> fun y_1 -> let t_1 = y_1 in let t_0 = x_0 in let t_2 = t_1 + t_0 in let t_3 = t_2 + t_1 in let t_4 = t_3 + t_2 in t_4 + t_3>. 11/32

else if n = 1 then k y else memo loop (n-1) (fun r1 -> memo loop (n-2) (fun r2 -> k.<.~r1 +.~r2>.)) in loop.

21 Two solutions 1. Use CPS or monadic style to write the generator. (Match compiler, CPS translator (Danvy & Filinski), PE (Bondorf)) let gib x y = let rec loop n k = if n = 0 then k x else if n = 1 then k y else memo loop (n-1) (fun r1 -> memo loop (n-2) (fun r2 -> k.<.~r1 +.~r2>.)) in loop.<fun x y ->.~(top_fn (gib.<x>..<y>. 5))>..<fun x_0 -> fun y_1 -> let t_1 = y_1 in let t_0 = x_0 in let t_2 = t_1 + t_0 in let t_3 = t_2 + t_1 in let t_4 = t_3 + t_2 in t_4 + t_3>. 11/32

gib x y = let rec loop n = if n = 0 then x else if n = 1 then

22 Two solutions 2. Use delimited control operators to hide CPS. (CPS translator (Danvy & Filinski), PE (Lawall & Danvy)) let gib x y = let rec loop n = if n = 0 then x else if n = 1 then y else.<.~(memo loop (n-1)) +.~(memo loop (n-2))>. in loop 12/32

23 Two solutions 2. Use delimited control operators to hide CPS. (CPS translator (Danvy & Filinski), PE (Lawall & Danvy)) let gib x y = let rec loop n = if n = 0 then x else if n = 1 then y else.<.~(memo loop (n-1)) +.~(memo loop (n-2))>. in loop D[loop 2] table D[loop 3] table 0.<let t_2 = y_1 + x_0 in.~( D[.<t_2>.] table 0 )>..<let t_3 = t_2 + y_1 in.~( D[.<t_3>.] table 00 )>. 12/32

24 Two solutions 2. Use delimited control operators to hide CPS. (CPS translator (Danvy & Filinski), PE (Lawall & Danvy)) let gib x y = let rec loop n = if n = 0 then x else if n = 1 then y else.<.~(memo loop (n-1)) +.~(memo loop (n-2))>. in loop.<fun x y ->.~(top_fn (fun () -> gib.<x>..<y>. 5))>..<fun x_0 -> fun y_1 -> let t_1 = y_1 in let t_0 = x_0 in let t_2 = t_1 + t_0 in let t_3 = t_2 + t_1 in let t_4 = t_3 + t_2 in t_4 + t_3>. 12/32

if n = 1 then y else.<.~(memo loop (n-1)) +.~(memo loop (n-2))>. in loop top_fn (fun () ->.<fun x y ->.~(gib.

25 Two solutions 2. Use delimited control operators to hide CPS. (CPS translator (Danvy & Filinski), PE (Lawall & Danvy)) let gib x y = let rec loop n = if n = 0 then x else if n = 1 then y else.<.~(memo loop (n-1)) +.~(memo loop (n-2))>. in loop top_fn (fun () ->.<fun x y ->.~(gib.<x>..<y>. 5)>.).<let t_1 = y_1 in let t_0 = x_0 in let t_2 = t_1 + t_0 in let t_3 = t_2 + t_1 in let t_4 = t_3 + t_2 in fun x_0 -> fun y_1 -> t_4 + t_3>. 12/32

26 13/32 Preventing scope extrusion Custom generators 6= Fixed generator Low-hanging fruit: For safety, simply treat later binders as earlier delimiters in the operational semantics and type system. (Existing practice; Thiemann & Dussart s constraint on state)

27 Our source language 1 Expressions e ::= x j i j e + e j x: e j fix j ee j (e; e) j fst j snd j ifz e then e else e j 出 j feg j e j e C (x: e) v C e[x := v] ( v ). 14/32

28 Our source language 1 Expressions e ::= x j i j e + e j x: e j fix j ee j (e; e) j fst j snd j ifz e then e else e j 出 j feg j e j e {z } {z } Delimited control Code generation (Felleisen,..., Danvy & Filinski) (Davies & Pfenning,..., Taha) C (x: e) v C e[x := v] ( v ). 14/32

29 Staging Two levels: present 0, future 1. Expressions e ::= x j i j e + e j x: e j fix j ee j (e; e) j fst j snd j ifz e then e else e j 出 j feg j e j e {z } {z } Delimited control Code generation (Felleisen,..., Danvy & Filinski) (Davies & Pfenning,..., Taha) C (x: e) v C e[x := v] ( v ). 14/32

30 Staging Two levels: present 0, future 1. Values v 0 ::= x j x: e j v 1 j v 1 ::= x j x: v 1 j v 1 v 1 j Contexts C 0 ::= C 0 [ e] j C 0 [v 0 ] j C 1 [ ] j j C 1 ::= C 1 [ e] j C 1 [v 1 ] j C 0 [ ] j C 0 (x: e) v 0 C 0 e[x := v 0 ] ( v ) C 1 v 1 C 1 v 1 ( ). 14/32

31 Staging Two levels: present 0, future 1. let f = x: x in t: (f t ) v t: ((x: x) t ) v t: t t: t C 0 (x: e) v 0 C 0 e[x := v 0 ] ( v ) C 1 v 1 C 1 v 1 ( ). 14/32

32 Control Two operators: shift 出, reset f g. Expressions e ::= x j i j e + e j x: e j fix j ee j (e; e) j fst j snd j ifz e then e else e j 出 j feg j e j e {z } {z } Delimited control Code generation (Felleisen,..., Danvy & Filinski) (Davies & Pfenning,..., Taha) C 0 (x: e) v 0 C 0 e[x := v 0 ] ( v ) C 1 v 1 C 1 v 1 ( ) C 0 fv 0 g C 0 v 0 (fg) C 0 fd[ 出 v 0 ]g C 0 fv 0 (x: fd[x]g)g ( 出 0 ). 14/32

33 Control Two operators: shift 出, reset f g. f1 + 1g f2g + 1 fg C 0 (x: e) v 0 C 0 e[x := v 0 ] ( v ) C 1 v 1 C 1 v 1 ( ) C 0 fv 0 g C 0 v 0 (fg) C 0 fd[ 出 v 0 ]g C 0 fv 0 (x: fd[x]g)g ( 出 0 ). 14/32

34 Control Two operators: shift 出, reset f g. Emulate state (Filinski). const = y:z:y get = 出 (k:z:kzz) put = z 0 : 出 (k:z:kz 0 z 0 ) fconst (get + 40)g 2 出 0 f(k: z: kzz)(x: fconst(x + 40)g)g 2 v fz: (x: fconst(x + 40)g)zzg 2 fg (z: (x: fconst(x + 40)g)zz) 2 v (x: fconst(x + 40)g) 2 2 v fconst(2 + 40)g 2 v fz: 42g C 0 (x: e) v 0 C 0 e[x := v 0 ] ( v ) C 1 v 1 C 1 v 1 ( ) C 0 fv 0 g C 0 v 0 (fg) C 0 fd[ 出 v 0 ]g C 0 fv 0 (x: fd[x]g)g ( 出 0 ). 14/32

35 Control Two operators: shift 出, reset f g. Emulate state (Filinski). const = y:z:y get = 出 (k:z:kzz) put = z 0 : 出 (k:z:kz 0 z 0 ) fconst (put(get + 1) + get)g 2 + fconst (put(2 + 1) + get)g 2 + fconst (put 3 + get)g 2 + (x: fconst(x + get)g) 3 3 v fconst(3 + get)g 3 + fconst(3 + 3)g C 0 (x: e) v 0 C 0 e[x := v 0 ] ( v ) C 1 v 1 C 1 v 1 ( ) C 0 fv 0 g C 0 v 0 (fg) C 0 fd[ 出 v 0 ]g C 0 fv 0 (x: fd[x]g)g ( 出 0 ). 14/32

36 Staging + Control Is scope extrusion possible? fconst (let x = y: (put y ) in get)g 0 + fconst (let x = y: ( y ) in get)g y + fconst getg y + fconst y g y + y 15/32

37 Staging + Control Is scope extrusion possible? No. Level-1 delimits control. fconst (let x = y: (put y ) in get)g 0 + fconst (let x = y: f(k: z: k y y )(x: f x g)g in get)g 0 15/32

38 Staging + Control Is scope extrusion possible? No. Level-1 delimits control. fconst (let x = y: (put y ) in get)g 0 + fconst (let x = y: f(k: z: k y y )(x: f x g)g in get)g 0 Can write: memoizing fixpoint, CPS translation, partial evaluation dynamic programming Gaussian elimination Markov models with symbolic matrix multiplications Cannot write: loop-invariant code motion inserting let/if/assert at outermost possible scope f i: ( 出 k: let x = in (k i + x ) ) g 15/32

39 16/32 Outline Delimited control for program generation Example Formalization Natural-language semantics Delimited control Quotation Variable binding Breaking the fourth wall Contextual modalities Environment classifiers

40 17/32 Formal linguistics Goal: relate forms to meanings in a concise specification. Science, rather than engineering.

41 17/32 Formal linguistics Goal: relate forms to meanings in a concise specification. Science, rather than engineering. Abstract syntax Concrete syntax Operational semantics Denotational semantics

42 17/32 Formal linguistics Goal: relate forms to meanings in a concise specification. Science, rather than engineering. Abstract syntax Concrete syntax Operational semantics Denotational semantics Formulas Metalanguage Cf. introductory logic.

43 18/32 Truth and entailment Alice left and Bob left. left a ^ left b

44 18/32 Truth and entailment Alice left and Bob left. Alice left. left a ^ left b left a

45 18/32 Truth and entailment Alice left and Bob left. Alice left. left a ^ left b left a Alice and Bob left. Alice left. Alice and Bob met. 6 Alice met.

46 19/32 Delimited control for quantifiers Alice saw everyone. Alice saw Bob. 8x: saw a x saw a b

47 19/32 Delimited control for quantifiers Alice saw everyone. Alice saw Bob. 8x: saw a x saw a b fsaw a ( 出 k: 8x: kx)g fsaw a bg

48 19/32 Delimited control for quantifiers Alice saw everyone. Alice saw Bob. 8x: saw a x saw a b fsaw a ( 出 k: 8x: kx)g fsaw a bg Everyone saw someone. 8x: 9y: saw x y fsaw ( 出 k: 8x: kx) ( 出 k: 9y: ky)g

49 Delimited control for quantifiers Alice saw everyone. Alice saw Bob. 8x: saw a x saw a b fsaw a ( 出 k: 8x: kx)g fsaw a bg Everyone saw someone. 8x: 9y: saw x y fsaw ( 出 k: 8x: kx) ( 出 k: 9y: ky)g Simulate other linguistic side effects: pronouns, questions,... 19/32

50 Evaluation order Surface scope is preferred over inverse scope: Everyone saw someone. Anaphora is preferred over cataphora: Everyone s father saw her mother. Her father saw everyone s mother. Gap tends to precede wh-phrase: Who do you think saw what? What do you think who saw? Reuse the same default of left-to-right evaluation for a more concise explanation. 20/32

51 21/32 Outline Delimited control for program generation Example Formalization Natural-language semantics Delimited control Quotation Variable binding Breaking the fourth wall Contextual modalities Environment classifiers

52 22/32 Varieties of quotation Bachelor has eight letters. pure has-8-letters `bachelor' Quine says quotation has a certain anomalous feature. direct say q quotation has a certain anomalous feature

53 22/32 Varieties of quotation Bachelor has eight letters. pure has-8-letters `bachelor' Quine says quotation has a certain anomalous feature. direct say q quotation has a certain anomalous feature Quine says quotation has a certain anomalous feature. indirect say q (has-a-certain-anomalous-feature quotation)

54 22/32 Varieties of quotation Bachelor has eight letters. pure has-8-letters `bachelor' Quine says quotation has a certain anomalous feature. direct say q quotation has a certain anomalous feature Quine says quotation has a certain anomalous feature. indirect say q (has-a-certain-anomalous-feature quotation) Quine says quotation has a certain anomalous feature. mixed say q ( has a certain anomalous feature quotation)???

55 Mixing mention and use Quine says quotation has a certain anomalous feature. mixed : : : (eval q has a certain anomalous feature ) : : : Bush is proud of his eckullectic reading list. mixed : : : (eval b eckullectic ) : : : 23/32

56 Mixing mention and use Quine says quotation has a certain anomalous feature. mixed : : : (eval q has a certain anomalous feature ) : : : Bush is proud of his eckullectic reading list. mixed : : : (eval b eckullectic ) : : : Yet Cheney s reading list is far more eckullectic, not to mention longer. mixed : : : (eval b eckullectic ) : : : 23/32

57 Program generation in natural language Bush boasted of my [Cheney s favorite adjective] reading list. syntactic unquotation : : : (favorite adjective c) : : : Bush boasted of my [eclectic] reading list. semantic unquotation : : : ( 出 k: 9x: eval b x = eclectic ^ kx) : : : 24/32

58 Program generation in natural language Bush boasted of my [Cheney s favorite adjective] reading list. syntactic unquotation : : : (favorite adjective c) : : : Bush boasted of my [eclectic] reading list. semantic unquotation : : : ( 出 k: 9x: eval b x = eclectic ^ kx) : : : Bush complained about the utterly [inaudible] loudspeakers in the room. : : : ( 出 k: 9x: inaudible x ^ kx) : : : : : : ( 出 k: 9x: eval b x = inaudible ^ kx) : : : 24/32

59 25/32 Variable binding in natural language The teacher praised every boy who did his homework. : : : ( 出 k: 8x: (boy x ^ did x (homework x))! kx) : : : The teacher praised every boy who did [his homework]. : : : eval t 出 k: 8x: (boy x ^ did x : : :)! kx : : :

60 26/32 Inverse quantifier scope It is an attractive scientific hypothesis that evaluation order is always from left to right. Everyone saw someone. fsaw ( 出 k: 8x: kx) ( 出 k: 9y: ky)g 8x: 9y: saw x y f fsaw ( 出 k: 8x: kx) ( 出 k: 9y: (ky) )g g 9y: saw ( 出 k: 8x: kx) y However, reducing generated shift statically is hard. If only... f 8x: saw x ( 出 k: 9y: (ky) ) g

61 27/32 Outline Delimited control for program generation Example Formalization Natural-language semantics Delimited control Quotation Variable binding Breaking the fourth wall Contextual modalities Environment classifiers

62 Loop-invariant code motion We want: i: let y = i + (: : : : : : ) in y + y let x = in i: let y = i + x in y + y Or in a two-level calculus: i: (y: y + y)(i + (: : : : : : )) Or in CPS: (i: k: (y: k 0 : k 0 (y + y))(l: (: : : : : : )(x: k(l(i + x))))) (z: z) 28/32

63 Contextual modalities In a two-level calculus: i: (y: y + y) (i + (: : : : : : )) Manage environment explicitly using de Bruijn indices: (((zero + zero)) (zero + 出 k: ((throw (import k) zero))(40 + 2))) The continuation k : [i : int] int! [] int import k : [i : int ; x : int] int! [x : int] int (Nanevski, Pfenning & Pientka 2008) MetaOCaml today! 29/32

64 30/32 Environment classifiers Judgments: e : = 0 = 0 e 1 : int e 2 : int n : int e 1 + e 2 : int e 1 : 1! e 1 e 2 : e 2 : 1 [x : 1 ] e : (! 0 )! 0 = 0 x: e : ( 1!! 0 )! 0 0=String [ = 0 ] e : (! 0 )! 0 e : region e : (! 0 )! 0 e :

65 30/32 Environment classifiers Judgments: e : = 0 = 0 e 1 : int e 2 : int n : int e 1 + e 2 : int e 1 : 1! e 1 e 2 : e 2 : 1 [x : 1 ] e : (! 0 )! 0 = 0 x: e : ( 1!! 0 )! 0 0=String [ = 0 ] e : (! 0 )! 0 e : region e : (! 0 )! 0 e :

66 30/32 Environment classifiers Judgments: e : = 0 = 0 e 1 : int e 2 : int n : int e 1 + e 2 : int e 1 : 1! e 1 e 2 : e 2 : 1 [x : 1 ] e : (! 0 )! 0 = 0 x: e : ( 1!! 0 )! 0 0=String [ = 0 ] e : (! 0 )! 0 e : region e : (! 0 )! 0 e :

67 30/32 Environment classifiers Judgments: e : = 0 = 0 e 1 : int e 2 : int n : int e 1 + e 2 : int e 1 : 1! e 1 e 2 : e 2 : 1 [x : 1 ] e : (! 0 )! 0 = 0 x: e : ( 1!! 0 )! 0 0=String [ = 0 ] e : (! 0 )! 0 e : region e : (! 0 )! 0 e :

68 30/32 Environment classifiers Judgments: e : = 0 = 0 e 1 : int e 2 : int n : int e 1 + e 2 : int e 1 : 1! e 1 e 2 : e 2 : 1 [x : 1 ] e : (! 0 )! 0 = 0 x: e : ( 1!! 0 )! 0 0=String [ = 0 ] e : (! 0 )! 0 e : region e : (! 0 )! 0 e :

69 31/32 Using environment classifiers In CPS: (i: k: (y: k 0 : k 0 (y + y))(l: (: : : : : : )(x: k(l(i + x))))) (z: z) Create a region for i: region (i: k: (y: k 0 : k 0 (y + y))(l: (k: m: n: (x: kxm)(l: n(l(40 + 2))))(x: k(l(i + x))))) (z: k: kz)(z: k: kz) Continuation hierarchy: k up to i: m up to x: n beyond OCaml today!

70 The ends Metalanguages for high-performance/embedded computing natural-language semantics of scope and quotation need safety track object variable bindings clarity provide delimited control operators Help! 32/32

Embedded probabilistic programming

Embedded probabilistic programming Oleg Kiselyov FNMOC oleg@pobox.com Chung-chieh Shan Rutgers University ccshan@cs.rutgers.edu 17 July 2009 Probabilistic inference Model (what) Pr(Reality) Pr(Obs j Reality)