THE STRONG INVARIANCE THESIS

Transcription

1 THE STRONG INVARIANCE THESIS FOR A λ-calculus LOLA WORKSHOP 2017 Yannick Forster 1 Fabian Kunze 1,2 Marc Roth 1,3 1 SAARLAND UNIVERSITY 2 MAX PLANCK INSTITUTE FOR INFORMATICS 3 CLUSTER OF EXCELLENCE (MMCI)

2 TURING MACHINES 2

3 TURING MACHINES (PROS) easy to imagine easy to explain de-facto the standard model of computation for computation theory and complexity theory 3

4 TURING MACHINES (CONS) Notoriously hard to reason about (in a formally precise way): not compositional tedious encodings no nice abstractions for verification (e.g. no separation logic) Formalisation of Computability Theory is out of reach Formalisation of Complexity Theory is even further away 4

5 EXEMPLARY RELATED WORK Ugo Dal Lago and Simone Martini The Weak Lambda Calculus as a Reasonable Machine Theoretical Computer Science, 2008 Beniamino Accattoli and Ugo Dal Lago (Leftmost-Outermost) Beta Reduction is Invariant, Indeed Logical Methods in Computer Science,

6 EXEMPLARY RELATED WORK Ugo Dal Lago and Simone Martini The Weak Lambda Calculus as a Reasonable Machine Theoretical Computer Science, 2008 Beniamino Accattoli and Ugo Dal Lago (Leftmost-Outermost) Beta Reduction is Invariant, Indeed Logical Methods in Computer Science, 2016 Andrea Asperti and Wilmer Ricciotti A formalization of multi-tape Turing machines Theoretical Computer Science,

7 ANOTHER MODEL OF COMPUTATION A certain flavour of λ-calculus called L compositional straightforward encodings of data types equational reasoning for verification Formalisation for Computability theory Yannick Forster and Gert Smolka Weak Call-by-Value Lambda Calculus as a Model of Computation in Coq ITP 2017 Reasonable with respect to time [Dal Lago, Martini (2008)] Reasonable with respect to space? 6

8 THE INVARIANCE THESIS (Strong) Invariance Thesis Reasonable machines can simulate each other within a polynomially bounded overhead in time and a constant-factor overhead in space. [Slot, van Emde Boas (1998)] 7

9 THE INVARIANCE THESIS (Strong) Invariance Thesis Reasonable machines can simulate each other within a polynomially bounded overhead in time and a constant-factor overhead in space. Ensures consistency w.r.t classes closed under poly-time/constant-space reductions. [Slot, van Emde Boas (1998)] 7

10 CONTRIBUTION Simple time and space measures for L substitution-based interpreter with constant-factor overhead in space heap-based interpreter with polynomially bounded overhead in time hybrid interpreter fulfilling the strong invariance thesis 8

11 CONTRIBUTION Theorem (Strong Invariance Thesis for L) L and Turing Machines can simulate each other within a polynomially bounded overhead in time and a constant-factor overhead in space for decision functions with non-sublinear running time. 9

12 L: WEAK CALL-BY-VALUE λ-calculus s, t ::= x λx.s s t (λx.s)(λy.t) s[x := λy.t] s s st s t t t st st uniformly confluent (reductions to normal forms have the same length) data represented by abstractions (Scott encoding) recursion using fixed-point combinator [Dal Lago, Martini (2008)] 10

13 TIME MEASURE If s = s 0 s 1 s k then Time(s) := k i.e. the number of β-reduction steps 11

14 SPACE MEASURE Space(s) := max s i {s i s s i } i.e. size of the largest intermediate term of the reduction for x = de Bruijn index of x st = 1 + s + t λx.s = 1 + s 12

15 DEFINITION OF TURING MACHINE a finite type of states Q a transition function δ : Q Σ n+1 Q Σ n+1 {L, N, R} a start state s : Q a halting function Q B Semantics: Loop δ until a halting state is reached. [Asperti, Ricciotti (2015)], [Dal Lago, Martini (2008)] 13

16 DEFINITION OF TURING MACHINE a finite type of states Q a transition function δ : Q Σ n+1 Q Σ n+1 {L, N, R} a start state s : Q a halting function Q B Semantics: Loop δ until a halting state is reached. Encode δ and halting function using Scott encodings (linear size, polynomial operations) and loop. [Asperti, Ricciotti (2015)], [Dal Lago, Martini (2008)] 13

17 DEFINITION OF TURING MACHINE a finite type of states Q a transition function δ : Q Σ n+1 Q Σ n+1 {L, N, R} a start state s : Q a halting function Q B Semantics: Loop δ until a halting state is reached. Encode δ and halting function using Scott encodings (linear size, polynomial operations) and loop. In Coq: Generation and verification of L-code from functional specification is automatic with our framework. Time-complexity of the extract is semi-automatic. Space-complexity has to be done by hand. [Asperti, Ricciotti (2015)], [Dal Lago, Martini (2008)] 13

18 Theorem (Invariance thesis part I) L can simulate Turing machines with a polynomially bounded overhead in time and a constant-factor overhead in space. 14

19 EXAMPLE: EVALUATING BY SUBSTITUTION Let I := λx.x: (λxy.x x) I ((λxy.x x)ii) 15

20 EXAMPLE: EVALUATING BY SUBSTITUTION Let I := λx.x: (λxy.x x) I ((λxy.x x)ii) (λy.i I) ((λxy.x x)ii) 15

21 EXAMPLE: EVALUATING BY SUBSTITUTION Let I := λx.x: (λxy.x x) I ((λxy.x x)ii) (λy.i I) ((λxy.x x)ii) (λy.i I) ((λy.ii)i) 15

22 EXAMPLE: EVALUATING BY SUBSTITUTION Let I := λx.x: (λxy.x x) I ((λxy.x x)ii) (λy.i I) ((λxy.x x)ii) (λy.i I) ((λy.ii)i) (λy.i I) (I I) 15

23 EXAMPLE: EVALUATING BY SUBSTITUTION Let I := λx.x: (λxy.x x) I ((λxy.x x)ii) (λy.i I) ((λxy.x x)ii) (λy.i I) ((λy.ii)i) (λy.i I) (I I) (λy.i I)I 15

24 EXAMPLE: EVALUATING BY SUBSTITUTION Let I := λx.x: (λxy.x x) I ((λxy.x x)ii) (λy.i I) ((λxy.x x)ii) (λy.i I) ((λy.ii)i) (λy.i I) (I I) (λy.i I)I I I 15

25 EXAMPLE: EVALUATING BY SUBSTITUTION Let I := λx.x: (λxy.x x) I ((λxy.x x)ii) (λy.i I) ((λxy.x x)ii) (λy.i I) ((λy.ii)i) (λy.i I) (I I) (λy.i I)I I I I 15

26 ENCODING TERMS terms: prefix notation with λ, and. Positions: strings with R, λ Example (λxy.x y) (λx.x) (λλ10) (λ0) is encoded by λ. In this term, 1 occurs at L λλ@ L 16

27 SUBSTITUTION-BASED INTERPRETER To compute s s, use tapes pre, funct, arg, post, position: λ }{{} pre arg }{{} funct }{{} post 1. Find the first β-redex, copy to pre is read copy next complete term to funct (and remember its position on the position tape) if the next token is λ, copy the next term to arg and remaining tokens to post otherwise, move funct onto pre and start from beginning 2. copy funct to pre, replacing variable with arg 3. copy post to pre [Dal Lago, Martini (2008)] 17

28 COMPLEXITY ANALYSIS O( s 2 ) time Per step for s s : O( s + s ) space 18

29 COMPLEXITY ANALYSIS O( s 2 ) time Per step for s s : O( s + s ) space In total for s = s 0 s 1 s k : O( i s i 2 ) time O(max i s i ) = O(Space(s)) space 18

30 Theorem (Invariance thesis part II for space) Turing machines can simulate L with a constant-factor overhead in space. 19

31 EXPLOSIVE TERMS 2 := λxy.x (x y) can double the size of a term in one step: So, with I := λx.x: 2 t λy.t (t y) 2 (2 ( (2 I)...) }{{} k times normalizes in k L-steps, but needs Ω(2 k ) interpretation time 20

32 EXAMPLE: EVALUATING WITH A HEAP 2 := λxy.x (x y) 2(2I)I 21

33 EXAMPLE: EVALUATING WITH A HEAP 2 := λxy.x (x y) 2(2I)I 2(λy.h 1 (h 1 y))i h 1 := I 21

34 EXAMPLE: EVALUATING WITH A HEAP 2 := λxy.x (x y) 2(2I)I 2(λy.h 1 (h 1 y))i (λy.h 2 (h 2 y))i h 1 := I h 1 := I, h 2 := (λy.h 1 (h 1 y)) 21

35 EXAMPLE: EVALUATING WITH A HEAP 2 := λxy.x (x y) 2(2I)I 2(λy.h 1 (h 1 y))i (λy.h 2 (h 2 y))i h 2 (h 2 h 3 ) h 1 := I h 1 := I, h 2 := (λy.h 1 (h 1 y)) h 1 := I, h 2 := λy.h 1 (h 1 y), h 3 := I 21

36 EXAMPLE: EVALUATING WITH A HEAP 2 := λxy.x (x y) 2(2I)I 2(λy.h 1 (h 1 y))i (λy.h 2 (h 2 y))i h 2 (h 2 h 3 ) 2 h 2 ((λy.h 1 (h 1 y))i) h 1 := I h 1 := I, h 2 := (λy.h 1 (h 1 y)) h 1 := I, h 2 := λy.h 1 (h 1 y), h 3 := I 21

37 EXAMPLE: EVALUATING WITH A HEAP 2 := λxy.x (x y) 2(2I)I 2(λy.h 1 (h 1 y))i (λy.h 2 (h 2 y))i h 2 (h 2 h 3 ) 2 h 2 ((λy.h 1 (h 1 y))i) h 2 (h 1 (h 1 h 4 )) h 1 := I h 1 := I, h 2 := (λy.h 1 (h 1 y)) h 1 := I, h 2 := λy.h 1 (h 1 y), h 3 := I..., h 4 := I 21

38 EXAMPLE: EVALUATING WITH A HEAP 2 := λxy.x (x y) 2(2I)I 2(λy.h 1 (h 1 y))i (λy.h 2 (h 2 y))i h 2 (h 2 h 3 ) 2 h 2 ((λy.h 1 (h 1 y))i) h 2 (h 1 (h 1 h 4 )) 2 h 2 (h 1 (II)) h 1 := I h 1 := I, h 2 := (λy.h 1 (h 1 y)) h 1 := I, h 2 := λy.h 1 (h 1 y), h 3 := I..., h 4 := I 21

39 EXAMPLE: EVALUATING WITH A HEAP 2 := λxy.x (x y) 2(2I)I 2(λy.h 1 (h 1 y))i (λy.h 2 (h 2 y))i h 2 (h 2 h 3 ) 2 h 2 ((λy.h 1 (h 1 y))i) h 2 (h 1 (h 1 h 4 )) 2 h 2 (h 1 (II)) 5 h 2 I h 1 := I h 1 := I, h 2 := (λy.h 1 (h 1 y)) h 1 := I, h 2 := λy.h 1 (h 1 y), h 3 := I..., h 4 := I..., h 5 := I, h 6 := I 21

40 EXAMPLE: EVALUATING WITH A HEAP 2 := λxy.x (x y) 2(2I)I 2(λy.h 1 (h 1 y))i (λy.h 2 (h 2 y))i h 2 (h 2 h 3 ) 2 h 2 ((λy.h 1 (h 1 y))i) h 2 (h 1 (h 1 h 4 )) 2 h 2 (h 1 (II)) 5 h 2 I h 1 (h 1 h 7 ) h 1 := I h 1 := I, h 2 := (λy.h 1 (h 1 y)) h 1 := I, h 2 := λy.h 1 (h 1 y), h 3 := I..., h 4 := I..., h 5 := I, h 6 := I 21

41 EXAMPLE: EVALUATING WITH A HEAP 2 := λxy.x (x y) 2(2I)I 2(λy.h 1 (h 1 y))i (λy.h 2 (h 2 y))i h 2 (h 2 h 3 ) 2 h 2 ((λy.h 1 (h 1 y))i) h 2 (h 1 (h 1 h 4 )) 2 h 2 (h 1 (II)) 5 h 2 I h 1 (h 1 h 7 ) I h 1 := I h 1 := I, h 2 := (λy.h 1 (h 1 y)) h 1 := I, h 2 := λy.h 1 (h 1 y), h 3 := I..., h 4 := I..., h 5 := I, h 6 := I 21

42 HEAP-BASED INTERPRETER s = s 0 s i s k Tapes: main (contains s i ), heap, hc (heap counter) Invariant: size of heap is always polynomial in k and s. 22

43 HEAP-BASED INTERPRETER s = s 0 s i s k Tapes: main (contains s i ), heap, hc (heap counter) Invariant: size of heap is always polynomial in k and s. For beta-reduction of (λx.t 1 )t 2 : Copy t 2 to heap, replace x in t 1 with address 22

44 HEAP-BASED INTERPRETER s = s 0 s i s k Tapes: main (contains s i ), heap, hc (heap counter) Invariant: size of heap is always polynomial in k and s. For beta-reduction of (λx.t 1 )t 2 : Copy t 2 to heap, replace x in t 1 with address (linear in the heap, O( t 1 ) many copies of an address linear in the heap) 22

45 HEAP-BASED INTERPRETER s = s 0 s i s k Tapes: main (contains s i ), heap, hc (heap counter) Invariant: size of heap is always polynomial in k and s. For beta-reduction of (λx.t 1 )t 2 : Copy t 2 to heap, replace x in t 1 with address (linear in the heap, O( t 1 ) many copies of an address linear in the heap) For variable-unfolding of x: Find the element associated to x in the heap 22

46 HEAP-BASED INTERPRETER s = s 0 s i s k Tapes: main (contains s i ), heap, hc (heap counter) Invariant: size of heap is always polynomial in k and s. For beta-reduction of (λx.t 1 )t 2 : Copy t 2 to heap, replace x in t 1 with address (linear in the heap, O( t 1 ) many copies of an address linear in the heap) For variable-unfolding of x: Find the element associated to x in the heap (linear in the heap) 22

47 HEAP-BASED INTERPRETER s = s 0 s i s k Tapes: main (contains s i ), heap, hc (heap counter) Invariant: size of heap is always polynomial in k and s. For beta-reduction of (λx.t 1 )t 2 : Copy t 2 to heap, replace x in t 1 with address (linear in the heap, O( t 1 ) many copies of an address linear in the heap) For variable-unfolding of x: Find the element associated to x in the heap (linear in the heap) A bit more complicated for de-bruijn, but doable. 22

48 COMPLEXITY ANALYSIS Theorem There is a constant c such that any reduction s = s 0 s k in L can be simulated by the heap-based Turing machine in time and space O( s k c ). 23

49 Theorem (Invariance thesis part II for time) Turing machines can simulate L with a polynomially bounded overhead in time. 24

50 SUB-LINEAR-LOGARITHMICLY SMALL TERMS Let N := (λxy.x x) I, then N ( (N I)...) }{{} k times k (λy.i I) ( ((λy.i I) I)...) }{{} k times 2k I Needs 3k entries (with addresses of size O(k)) on heap, but definition permits only O(k) space 25

51 COMPLEXITY OVERVIEW s = s 0 s 1 s k for s k with constant size: substitution-based heap-based time O( i s i 2 ) O(poly(Time(s))) space O(Space(s)) O( s k c ) 26

52 PROBLEM ANALYSIS s = s 0 s k Heap-based interpreter needs O( s k c ) space on sublinar-logarithmically reducing terms (in k steps). 27

53 PROBLEM ANALYSIS s = s 0 s k Heap-based interpreter needs O( s k c ) space on sublinar-logarithmically reducing terms (in k steps). Substitution-based interpreter needs more than polynomial time on explosive terms where s i is asymptotically non-polynomial. 27

54 PROBLEM ANALYSIS s = s 0 s k Heap-based interpreter needs O( s k c ) space on sublinar-logarithmically reducing terms (in k steps). Substitution-based interpreter needs more than polynomial time on explosive terms where s i is asymptotically non-polynomial. But: Heap-based interpreter works on explosive terms! 27

55 HYBRID INTERPRETER Input: A term s. Set k = 0. Execute the substitution-based interpreter on s for k steps: If a normal form is reached, output it. If the space consumption is larger than s k c, abort and use the heap-based interpreter for k steps. If no normal form is reached, delete everything except s, set k := k + 1 and repeat. 28

56 TIME ANALYSIS Running time for fixed s and k: In total: O( ) 29

57 TIME ANALYSIS Running time for fixed s and k: heap-based interpreter: s k c In total: O( ) 29

58 TIME ANALYSIS Running time for fixed s and k: heap-based interpreter: s k c In total: O( Time(s) k=0 s k c }{{} heap-based ) 29

59 TIME ANALYSIS Running time for fixed s and k: unfolding the normal form: poly( s, Time(s))) In total: O( Time(s) k=0 s k c }{{} heap-based ) 29

60 TIME ANALYSIS Running time for fixed s and k: unfolding the normal form: poly( s, Time(s))) In total: O( poly( t, Time(s)) + }{{} unfolding the normal form t Time(s) k=0 s k c }{{} heap-based ) 29

61 TIME ANALYSIS Running time for fixed s and k: substitution-based interpreter: O( k i=1 s i 2 ) In total: O( poly( t, Time(s)) + }{{} unfolding the normal form t Time(s) k=0 s k c }{{} heap-based ) 29

62 TIME ANALYSIS Running time for fixed s and k: substitution-based interpreter: O( k i=1 s i 2 ) O( poly( t, Time(s)) + }{{} unfolding the normal form t In total: Time(s) k=0 s k c }{{} heap-based + k s i 2 i=1 }{{} substitution-based ) 29

63 TIME ANALYSIS Running time for fixed s and k: because of the space bound: s i s k c O( poly( t, Time(s)) + }{{} unfolding the normal form t In total: Time(s) k=0 s k c }{{} heap-based + k s i 2 i=1 }{{} substitution-based ) 29

64 TIME ANALYSIS Running time for fixed s and k: because of the space bound: s i s k c thus k i=1 s i 2 k s 2 k 2c O( poly( t, Time(s)) + }{{} unfolding the normal form t In total: Time(s) k=0 s k c }{{} heap-based + k s i 2 i=1 }{{} substitution-based ) 29

65 TIME ANALYSIS Running time for fixed s and k: because of the space bound: s i s k c thus k i=1 s i 2 k s 2 k 2c O( poly( t, Time(s)) + }{{} unfolding the normal form t In total: Time(s) k=0 s k c + k s k 2c ) }{{}}{{} heap-based substitution-based 29

66 TIME ANALYSIS Simplified: O( poly( t, Time(s)) + }{{} unfolding the normal form t Time(s) k=0 s k c + k s k 2c ) }{{}}{{} heap-based substitution-based O(poly( s, Time(s)) + Time(s) 2c+2 s 2 ) 29

67 SPACE ANALYSIS Space consumption for fixed s and k: In total: O( max k Space(s) ) 30

68 SPACE ANALYSIS Space consumption for fixed s and k: substitution-based interpreter: max i {0,...,k} s i In total: O( max k Space(s) ) 30

69 SPACE ANALYSIS Space consumption for fixed s and k: substitution-based interpreter: max i {0,...,k} s i = O(Space k (s)) In total: O( max k Space(s) ) 30

70 SPACE ANALYSIS Space consumption for fixed s and k: substitution-based interpreter: max i {0,...,k} s i = O(Space k (s)) In total: O( max k Space(s) Space k (s) ) 30

71 SPACE ANALYSIS Space consumption for fixed s and k: heap-based interpreter: O( s k c ) In total: O( max k Space(s) Space k (s) ) 30

72 SPACE ANALYSIS Space consumption for fixed s and k: heap-based interpreter: O( s k c ) In total: O( max k Space(s) Space k (s)+ s kc ) 30

73 SPACE ANALYSIS Space consumption for fixed s and k: because of the space bound: Space k (s) s k c In total: O( max k Space(s) Space k (s)+ s kc ) 30

74 SPACE ANALYSIS Space consumption for fixed s and k: because of the space bound: Space k (s) s k c In total: O( max k Space(s) Space k (s)+space k (s)) 30

75 SPACE ANALYSIS Simplified: O( max k Space(s) Space k (s)+space k (s)) O(Space(s)) 30

76 Theorem (Strong Invariance Thesis for L) L and Turing Machines can simulate each other within a polynomially bounded overhead in time and a constant-factor overhead in space for decision functions with non-sublinear running time. 31

77 WORK IN PROGRESS: FORMALISATION spec proof Functional correctness of L-interpreters L-extraction framework TM-interpreter (no verified complexity analysis)

78 WORK IN PROGRESS: FORMALISATION spec proof Functional correctness of L-interpreters L-extraction framework TM-interpreter (no verified complexity analysis) Missing: TM implementation and verification of L-interpreters Time and space analysis of L-interpreters Time and space analysis of TM-interpreter 32

79 SUMMARY The weak call-by-value λ-calculus L is as reasonable for complexity theory as Turing machines. 33

80 SUMMARY The weak call-by-value λ-calculus L is as reasonable for complexity theory as Turing machines. Future work: Formalise the complexity analysis Complexity theory using L: NP, many-one-reductions, hierarchy theorems,... 33

81 SUMMARY The weak call-by-value λ-calculus L is as reasonable for complexity theory as Turing machines. Future work: Formalise the complexity analysis Complexity theory using L: NP, many-one-reductions, hierarchy theorems,... Thanks! 33

82 THE HEAP-BASED INTERPRETER Use environments on a heap to delay substitutions: call (thunk) c = s E : pair of encoded L-term s and heap-address E heap H: list of entries ( or c#e ), addressed by position. call stack CS: list of tuples (@ L, c) or (@ R, c) R, c fully reduced) interpreter state: current call CC, CS and H. initial state: CC = s 0, CS = [] and H = [ ]) Example The result of (λx.x) ((λxy.x y) (λx.x)) (λx.x) (λx.x y) y λx.x is represented by CC =(λ@ ) 1 CS =[(@ R, (λ ) 0 )] H =[, (λ ) 0 )#0] 34

83 THE HEAP-BASED INTERPRETER (2) Each step of the interpreter depends on the current call CC = s E : if s = s L s R : push (@ L, s R [E]) on CS and set CC to s R E if s = x: get new CC by lookup of x in E if s = λs : if CS is empty: the term is fully evaluated if CS = (@ L, c R ) :: CS : set CC := c R and put (@ R, CC) on stack instead. if CS = (@R, λt E ) :: CS : store s R E #E on heap as Ê and set CC := t Ê 35

84 THE HEAP-BASED INTERPRETER (2) Each step of the interpreter depends on the current call CC = s E : if s = s L s R : push (@ L, s R [E]) on CS and set CC to s R E if s = x: get new CC by lookup of x in E if s = λs : if CS is empty: the term is fully evaluated if CS = (@ L, c R ) :: CS : set CC := c R and put (@ R, CC) on stack instead. if CS = (@R, λt E ) :: CS : store s R E #E on heap as Ê and set CC := t Ê Observations for evaluation s 0 s 1 s k : all calls contain subterms of s Heap contains #H = k + 1 elements, each of size s + 2 log(#h) CS & CC representing s i have size O( s i ) space consumption: O((max i s i ) + k ( s + log(k))) 35

85 THE HEAP-BASED INTERPRETER (2) Each step of the interpreter depends on the current call CC = s E : if s = s L s R : push (@ L, s R [E]) on CS and set CC to s R E if s = x: get new CC by lookup of x in E if s = λs : if CS is empty: the term is fully evaluated if CS = (@ L, c R ) :: CS : set CC := c R and put (@ R, CC) on stack instead. if CS = (@R, λt E ) :: CS : store s R E #E on heap as Ê and set CC := t Ê Observations for evaluation s 0 s 1 s k : all calls contain subterms of s Heap contains #H = k + 1 elements, each of size s + 2 log(#h) CS & CC representing s i have size O( s i ) space consumption: O((max i s i ) + k ( s + log(k))) time per interpreter step: O( s i #H + CC + CS) amortized, poly( s 0 ) interpreter-steps per β-reduction. time consumption: O(poly(k, s )) 35

86 LC: L WITH CLOSURES 36