Automatic Resource Bound Analysis and Linear Optimization. Jan Hoffmann

Transcription

1 Automatic Resource Bound Analysis and Linear Optimization Jan Hoffmann

2 Beyond worst-case analysis

3 Beyond worst-case analysis of algorithms

4 Beyond worst-case analysis of algorithms Resource bound analysis

5 Beyond worst-case analysis of algorithms Resource bound analysis of programs / software

6 Motivation: Why analyze resource usage of programs?

7 Resource Usage in Safety-Critical Systems Memory Usage Timing

8 Resource Usage in Safety-Critical Systems Memory Usage Timing Unintended acceleration in Toyota cars in the US

9 Resource Usage in Safety-Critical Systems Memory Usage Timing Unintended acceleration in Toyota cars in the US ICE 3 Velaro D delivery delayed by one year because of software performance issues in 2013.

10 Performance Bugs are Common and Expensive HealthCare.gov debacle has been mainly caused by performance issues. Google Apps Developer: Cannot test for performance bugs with regression test.

11 Software Security Algorithmic Complexity Attacks Side-Channel Attacks

12 Software Security Algorithmic Complexity Attacks Side-Channel Attacks Space/Time Analysis for Cybersecurity (STAC) October 2014 $ 53M program Our team: Computed Bound Actual Behavior Performance Quick Sort (Integers) 12n n + 3 O(n 2 ) 0.1 s Split and Sort 16n n + 9 O(n 2 ) 2.1 s

13 Static Resource Bound Analysis Given: Question: A program P What is the worst-case resource consumption of P as a function of the size of its inputs?

14 Static Resource Bound Analysis Given: A program P Clock cycles, heap space, power,... Question: What is the worst-case resource consumption of P as a function of the size of its inputs?

15 Static Resource Bound Analysis Given: A program P Clock cycles, heap space, power,... Question: What is the worst-case resource consumption of P as a function of the size of its inputs? Not only asymptotic bounds but concrete constant factors.

16 Automatic Static Resource Bound Analysis Given: A program P Clock cycles, heap space, power,... Question: What is the worst-case resource consumption of P as a function of the size of its inputs? Not only asymptotic bounds but concrete constant factors.

17 Automatic Static Resource Bound Analysis Undecidable problem Given: A program P Clock cycles, heap space, power,... Question: What is the worst-case resource consumption of P as a function of the size of its inputs? Not only asymptotic bounds but concrete constant factors.

18 Research Challenges Model and predict resource usage at development time. Source code

19 Research Challenges Model and predict resource usage at development time. Compiler Libraries Source code Run-time system Hardware

20 Research Challenges Help developers to reason about quantitative properties. Computer support: Modeling, specification, verification, automation and user interaction Compositionally: Track size changes and specify resource-usage of library code Language features: Concurrency, higher-order, data structures,

21 Why is this Related? 1. Beyond worst-case analysis of algorithms Automation Concrete (non-asymptotic) bounds for specific hardware 2. We face similar challenges in resource-bound analysis The worst-case behavior doesn t happen in practice. 3. We reduce bound inference to linear optimization Linear programs we get are solvable in linear time in practice Theoretical worst-case of the algorithm is exponential (simplex)

22 Outline Motivation How does automatic resource bound analysis work? How well does automatic resource bound analysis work? (implementation and experiments) What are the properties of the LP instances that we get?

23 Source Code 12n n +3 Resource Bound Bird s Eye View Type-Based Resource Analysis

24 Source Code 12n n +3 Resource Bound applies to Machine Code Bird s Eye View Type-Based Resource Analysis

25 Source Code Compiler 12n n +3 Resource Bound applies to Machine Code Bird s Eye View Type-Based Resource Analysis

26 Source Code Compiler Run-time system Hardware 12n n +3 Resource Bound applies to Machine Code Bird s Eye View Type-Based Resource Analysis

27 Source Code Formal Cost Semantics Compiler Run-time system Hardware 12n n +3 Resource Bound applies to Machine Code Bird s Eye View Type-Based Resource Analysis

28 Source Code Formal Cost Semantics Compiler ait WCET Analyzers Run-time system Hardware 12n n +3 Resource Bound applies to Machine Code Bird s Eye View Type-Based Resource Analysis

29 Source Code CPLEX CLP Type Inference Formal Cost Semantics Compiler ait WCET Analyzers acc:int q v q 0 v acc:int,l:l p (int) (A:VAR) acc : int q q 0 ß(last) = (int,l p ß q (int))! ß /q 0 ß int acc:int, x:int,xs:l p q a (A:APP) (int) last(x,xs):int match l with nil! acc cons(x,xs)! last(x,xs):int (A:MATL) Type Derivation q 0 a Run-time system Hardware 12n n +3 Resource Bound applies to Machine Code Bird s Eye View Type-Based Resource Analysis

30 Source Code CPLEX CLP Type Inference Formal Cost Semantics Compiler ait WCET Analyzers acc:int q v q 0 v acc:int,l:l p (int) (A:VAR) acc : int q q 0 ß(last) = (int,l p ß q (int))! ß /q 0 ß int acc:int, x:int,xs:l p q a (A:APP) (int) last(x,xs):int match l with nil! acc cons(x,xs)! last(x,xs):int (A:MATL) Type Derivation q 0 a Run-time system Hardware 12n n +3 Resource Bound applies to Machine Code Bird s Eye View Type-Based Resource Analysis

31 Source Code CPLEX Type Inference CLP acc:int q v q 0 v acc:int,l:l p (int) (A:VAR) acc : int q q 0 ß(last) = (int,l p ß q (int))! ß /q 0 ß int acc:int, x:int,xs:l p q a (A:APP) (int) last(x,xs):int match l with nil! acc cons(x,xs)! last(x,xs):int (A:MATL) Type Derivation q 0 a 12n n +3 Resource Bound Bird s Eye View Type-Based Resource Analysis

32 Source Code CPLEX Type Inference CLP Clear soundness theorem. acc:int q v q 0 v acc:int,l:l p (int) (A:VAR) acc : int q q 0 ß(last) = (int,l p ß q (int))! ß /q 0 ß int acc:int, x:int,xs:l p q a (A:APP) (int) last(x,xs):int match l with nil! acc cons(x,xs)! last(x,xs):int (A:MATL) Type Derivation q 0 a Naturally compositional. Efficient inference via LP solving. 12n n +3 Resource Bound Bird s Eye View Type-Based Resource Analysis

33 Idea: Automate Amortized Analysis Assign potential functions to data structures States are mapped to non-negative numbers (state) 0 Potential pays the resource consumption and the potential at the following program point (before) (after)+cost telescoping Initial potential is an upper bound (initial state) P cost

34 Idea: Automate Amortized Analysis Assign potential functions to data structures States are mapped to non-negative numbers (state) 0 Potential pays the resource consumption and the potential at the following program point (before) (after)+cost telescoping Initial potential is an upper bound (initial state) P cost Type systems for automatic analysis Fix a format of potential functions Develop type rules that manipulate potential functions

35 Idea: Automate Amortized Analysis Assign potential functions to data structures States are mapped to non-negative numbers (state) 0 Potential pays the resource consumption and the potential at the following program point (before) (after)+cost telescoping Initial potential is an upper bound (initial state) P cost Type systems for automatic analysis Fix a format of potential functions Potential is given by types. Develop type rules that manipulate potential functions

36 Example: Append for Persistent Lists append(x,y) Heap-space usage is 2n if n is the length of list x One list element requires two heap cells (data and pointer)

37 Example: Append for Persistent Lists append(x,y) Heap-space usage is 2n if n is the length of list x One list element requires two heap cells (data and pointer) Example evaluation:

38 Example: Append for Persistent Lists append(x,y) Heap-space usage is 2n if n is the length of list x One list element requires two heap cells (data and pointer) Example evaluation: x a y d b e c

39 Example: Append for Persistent Lists append(x,y) Heap-space usage is 2n if n is the length of list x One list element requires two heap cells (data and pointer) Example evaluation: x a y d c b e c

40 Example: Append for Persistent Lists append(x,y) Heap-space usage is 2n if n is the length of list x One list element requires two heap cells (data and pointer) Example evaluation: x a y d c b b e c

41 Example: Append for Persistent Lists append(x,y) Heap-space usage is 2n if n is the length of list x One list element requires two heap cells (data and pointer) Example evaluation: x a y d c b a b e c

42 Example: Append for Persistent Lists append(x,y) Heap-space usage is 2n if n is the length of list x One list element requires two heap cells (data and pointer) Example evaluation: x a y d c b a append(x,y) b e c

43 Example: Append for Persistent Lists append(x,y) Heap-space usage is 2n if n is the length of list x One list element requires two heap cells (data and pointer) Example evaluation: x a y d c b a append(x,y) b e c Heap usage: 2*n = 2*3 = 6

44 Example: Composing Calls of Append f(x,y,z) = { t = append(x,y); append(t,z) } Heap usage of f(x,y,z) is 2n + 2(n+m) if n is the length of list x m is the length of list y

45 Example: Composing Calls of Append f(x,y,z) = { t = append(x,y); append(t,z) } Heap usage of f(x,y,z) is 2n + 2(n+m) if n is the length of list x m is the length of list y x a 4 y d b 4 e 2 2 c 4 z f Initial potential: 4*n + 2*m = 4*3 + 2*2 = 16

46 Example: Composing Calls of Append f(x,y,z) = { t = append(x,y); append(t,z) } Heap usage of f(x,y,z) is 2n + 2(n+m) if n is the length of list x m is the length of list y x a 4 y d b 4 e 2 2 append(x,y) c 4 z f Initial potential: 4*n + 2*m = 4*3 + 2*2 = 16

47 Example: Composing Calls of Append f(x,y,z) = { t = append(x,y); append(t,z) } Heap usage of f(x,y,z) is 2n + 2(n+m) if n is the length of list x m is the length of list y x a 4 y d b 4 e 2 2 c 2 append(x,y) c z f Initial potential: 4*n + 2*m = 4*3 + 2*2 = 16

48 Example: Composing Calls of Append f(x,y,z) = { t = append(x,y); append(t,z) } Heap usage of f(x,y,z) is 2n + 2(n+m) if n is the length of list x m is the length of list y x a 4 y d b e 2 2 c 2 b 2 append(x,y) c z f Initial potential: 4*n + 2*m = 4*3 + 2*2 = 16

49 Example: Composing Calls of Append f(x,y,z) = { t = append(x,y); append(t,z) } Heap usage of f(x,y,z) is 2n + 2(n+m) if n is the length of list x m is the length of list y x a y d b e 2 2 c 2 b 2 a 2 append(x,y) c z f Initial potential: 4*n + 2*m = 4*3 + 2*2 = 16

50 Example: Composing Calls of Append f(x,y,z) = { t = append(x,y); append(t,z) } Heap usage of f(x,y,z) is 2n + 2(n+m) if n is the length of list x m is the length of list y x a y d b e 2 2 c 2 b 2 a 2 t c z f Initial potential: 4*n + 2*m = 4*3 + 2*2 = 16

51 Example: Composing Calls of Append f(x,y,z) = { t = append(x,y); append(t,z) } Heap usage of f(x,y,z) is 2n + 2(n+m) if n is the length of list x m is the length of list y x a y d b e 2 2 c 2 b 2 a 2 t append(t,z) c z f Initial potential: 4*n + 2*m = 4*3 + 2*2 = 16

52 Example: Composing Calls of Append f(x,y,z) = { t = append(x,y); append(t,z) } Heap usage of f(x,y,z) is 2n + 2(n+m) if n is the length of list x m is the length of list y x a y d 2 c 2 b 2 a 2 t append(t,z) b e c z f e Initial potential: 4*n + 2*m = 4*3 + 2*2 = 16

53 Example: Composing Calls of Append f(x,y,z) = { t = append(x,y); append(t,z) } Heap usage of f(x,y,z) is 2n + 2(n+m) if n is the length of list x m is the length of list y x a y d c 2 b 2 a 2 t append(t,z) b e c z f e d Initial potential: 4*n + 2*m = 4*3 + 2*2 = 16

54 Example: Composing Calls of Append f(x,y,z) = { t = append(x,y); append(t,z) } Heap usage of f(x,y,z) is 2n + 2(n+m) if n is the length of list x m is the length of list y x a y d c b 2 a 2 t append(t,z) b e c z f e d c Initial potential: 4*n + 2*m = 4*3 + 2*2 = 16

55 Example: Composing Calls of Append f(x,y,z) = { t = append(x,y); append(t,z) } Heap usage of f(x,y,z) is 2n + 2(n+m) if n is the length of list x m is the length of list y x a y d c b a 2 t append(t,z) b e c z f e d c b Initial potential: 4*n + 2*m = 4*3 + 2*2 = 16

56 Example: Composing Calls of Append f(x,y,z) = { t = append(x,y); append(t,z) } Heap usage of f(x,y,z) is 2n + 2(n+m) if n is the length of list x m is the length of list y x a y d c b a t append(t,z) b e c z f e d c b a Initial potential: 4*n + 2*m = 4*3 + 2*2 = 16

57 Example: Composing Calls of Append f(x,y,z) = { t = append(x,y); append(t,z) } Heap usage of f(x,y,z) is 2n + 2(n+m) if n is the length of list x m is the length of list y x a y d c b a t b e c z f e d c b a append(t,z) Initial potential: 4*n + 2*m = 4*3 + 2*2 = 16

58 Example: Composing Calls of Append f(x,y,z) = { t = append(x,y); append(t,z) } Heap usage of f(x,y,z) is 2n + 2(n+m) if n is the length of list x m is the length of list y x a y d b e c b a t Implicit reasoning about size-changes. c z f e d c b a append(t,z) Initial potential: 4*n + 2*m = 4*3 + 2*2 = 16

59 Example: Composing Calls of Append f(x,y,z) = { 4 2 0/0 2 append: (L (int),l (int)) ----> L (int) t = append(x,y); append(t,z) 2 0 } append: (L (int),l (int)) ----> 0/0 0 L (int) The most general type of append is specialized at call-sites: q p append: (L (int),l (int)) ----> s/t r L (int) Φ Linear constraints.

60 Polynomial Potential Functions

61 Linear Potential Functions User-defined resource metrics (i.e., by tick(q) in the code) Naturally compositional: tracks size changes, types are specifications Bound inference by reduction to efficient LP solving Type derivations prove bounds with respect to the cost semantics Phd Thesis: Polynomial Potential Functions

62 User-defined resource metrics (i.e., by tick(q) in the code) Naturally compositional: tracks size changes, types are specifications Bound inference by reduction to efficient LP solving Type derivations prove bounds with respect to the cost semantics Linear Potential Functions Multivariate Polynomial Potential Functions Phd Thesis: Polynomial Potential Functions

63 For example m*n 2. User-defined resource metrics (i.e., by tick(q) in the code) Naturally compositional: tracks size changes, types are specifications Bound inference by reduction to efficient LP solving Type derivations prove bounds with respect to the cost semantics Linear Potential Functions Multivariate Polynomial Potential Functions Phd Thesis: Polynomial Potential Functions

64 For example m*n 2. User-defined resource metrics (i.e., by tick(q) in the code) Naturally compositional: tracks size changes, types are specifications Bound inference by reduction to efficient LP solving Type derivations prove bounds with respect to the cost semantics Linear Potential Functions Multivariate Polynomial Potential Functions Phd Thesis: Polynomial Potential Functions First automatic, type-based resource analysis for polynomial bounds.

65 Example: Polynomial Potential Functions t = append(xs,ys); quicksort(t) Computed time bound: 12m mn + 12n m + 22n + 11

66 Example: Polynomial Potential Functions t = append(xs,ys); quicksort(t) Computed time bound: 12m mn + 12n m + 22n + 11 append :((L(int), L(int)), ! )! (L(int), (3, 26, 24)) quicksort :(L(int), (3, 26, 24))! (L(int), (0, 0, 0))

67 Example: Polynomial Potential Functions t = append(xs,ys); quicksort(t) Computed time bound: 12m mn + 12n m + 22n + 11 append :((L(int), L(int)),! n m n + 26m nm )! (L(int), (3, 26, 24)) quicksort :(L(int), (3, 26, 24))! (L(int), (0, 0, 0))

68 Example: Polynomial Potential Functions t = append(xs,ys); quicksort(t) Computed time bound: 12m mn + 12n m + 22n + 11 Cost of append 8n +3 append :((L(int), L(int)),! n m n + 26m nm )! (L(int), (3, 26, 24)) quicksort :(L(int), (3, 26, 24))! (L(int), (0, 0, 0))

69 Example: Polynomial Potential Functions t = append(xs,ys); quicksort(t) Computed time bound: 12m mn + 12n m + 22n + 11 Cost of append append :((L(int), L(int)), 8n +3 n+m (n+m) + 24! )! (L(int), (3, 26, 24)) 24 n m n + 26m nm quicksort :(L(int), (3, 26, 24))! (L(int), (0, 0, 0))

70 Multivariate Resource Polynomials [POPL 11] Map data structures to non-negative rational numbers p : JAK! Q + 0 Are non-negative linear combinations of the following base polynomials: P(Int) ={a 1} P(A 1, A 2 )={(a 1, a 2 ) p 1 (a 1 ) p 2 (a 2 ) p i P(A i )} P(L(A)) = {[a 1,...,a n ] X 1 j 1 < <j k n ky i=1 p i (a ji ) k N, p i P(A)}

71 Multivariate Resource Polynomials [POPL 11] Map data structures to non-negative rational numbers p : JAK! Q + 0 Are non-negative linear combinations of the following base polynomials: P(Int) ={a 1} Important innovation: sigma-pi formula for data structures P(A 1, A 2 )={(a 1, a 2 ) p 1 (a 1 ) p 2 (a 2 ) p i P(A i )} P(L(A)) = {[a 1,...,a n ] X 1 j 1 < <j k n ky i=1 p i (a ji ) k N, p i P(A)}

72 Resource Polynomials: Examples P(L(A)) = {[a 1,...,a n ] X 1 j 1 < <j k n ky p i (a ji ) k N, p i P(A)} i=1

73 Resource Polynomials: Examples [a 1,...,a n ] 7! 8n +3 P(L(A)) = {[a 1,...,a n ] X 1 j 1 < <j k n ky i=1 p i (a ji ) k N, p i P(A)}

74 Resource Polynomials: Examples [a 1,...,a n ] 7! 8n +3 n [a 1,...,a n ] 7! 36 3 n n +3 P(L(A)) = {[a 1,...,a n ] X 1 j 1 < <j k n ky i=1 p i (a ji ) k N, p i P(A)}

75 Resource Polynomials: Examples [a 1,...,a n ] 7! 8n +3 n [a 1,...,a n ] 7! 36 3 n n +3 ([a 1,...,a n ], [b 1,...,b m ]) 7! 39mn +6m + 21n + 19 P(L(A)) = {[a 1,...,a n ] X 1 j 1 < <j k n ky i=1 p i (a ji ) k N, p i P(A)}

76 Resource Polynomials: Examples [a 1,...,a n ] 7! 8n +3 n [a 1,...,a n ] 7! 36 3 n n +3 ([a 1,...,a n ], [b 1,...,b m ]) 7! 39mn +6m + 21n + 19 n [[a1,...,a 1 m 1 1 ],..., [a1,...a n m n n ]] 7! n +3+ X 1applei<japplen 12m i P(L(A)) = {[a 1,...,a n ] X 1 j 1 < <j k n ky i=1 p i (a ji ) k N, p i P(A)}

77 Automatic Computation of the Bounds 1. Fix a maximal degree of resource polynomials 2. Annotate each type with (yet unknown) coefficients for resource polynomials Example for degree 2: ((L(int), L(int)), q 0,0, q 1,0, q 2,0, q 1,1, q 0,1, q 0,2 ) General case: index system that enumerates resource polynomials 3. Extract linear constraints for the coefficients during type inference 4. Solve the constraints with an LP solver

78 Automatic Computation of the Bounds 1. Fix a maximal degree of resource polynomials 2. Annotate each type with (yet unknown) coefficients for resource polynomials q 0,0 + q 1,1 nm + q 1,0 n + q 2,0 n 2 + q 0,1 m + q 0,2 m 2 Example for degree 2: ((L(int), L(int)), q 0,0, q 1,0, q 2,0, q 1,1, q 0,1, q 0,2 ) General case: index system that enumerates resource polynomials 3. Extract linear constraints for the coefficients during type inference 4. Solve the constraints with an LP solver

79 Outline Motivation How does automatic resource bound analysis work? How well does automatic resource bound analysis work? (Implementation and experiments) What are the properties of the LP instances that we get?

80 Resource Aware ML Based on Inria s OCaml compiler ~12,000 lines of code (+ ~29,000 loc form the OCaml compiler) Currently we use Coin-Or s CLP C interface Features: Higher-order functions and polymorphism User defined inductive types Parallel evaluation Side effects User defined (non-monotone) resource metrics Upper and lower bounds

81 Resource Aware ML Web interface at Based on Inria s OCaml compiler ~12,000 lines of code (+ ~29,000 loc form the OCaml compiler) Currently we use Coin-Or s CLP C interface Features: Higher-order functions and polymorphism User defined inductive types Parallel evaluation Side effects User defined (non-monotone) resource metrics Upper and lower bounds

82 Experimental Evaluation

83 Computed Bound Actual Behavior Performance Quick Sort (Integers) 12n n + 3 O(n 2 ) 0.1 s Split and Sort 16n n + 9 O(n 2 ) 2.1 s Insertion Sort (Strings) 8n 2 m + 8n 2-8nm + 4n + 3 O(n 2 m) 0.91 s Duplicate Elimination 6n 2 m + 9n 2-6nm + 3n + 3 O(n 2 m) 0.97 s Longest Common Subsequence 39nm + 6m + 21n + 19 O(nm) 0.36 s Matrix Multiplication 28xmn + 32xm + 2x + 14n + 21 O(xmn) 1.96 s Breadth-First Matrix Multiplication Dijkstra s Shortest-Path Algorithm In-Place Quick Sort for Arrays 2yz + 15ynmx + 14ynm + 15yn + 104n + 51 O(ynmx) 4.98 s 79.5n n + 38 O(n 2 ) 2.50 s 12.25x x + 3 O(x 2 ) 0.64 s Micro Benchmarks Evaluation-Step Bounds

84 Quick Sort for Integers Evaluation-step bound vs. measured behavior

85 First tight bound for quick sort. Quick Sort for Integers Evaluation-step bound vs. measured behavior

86 Longest Common Subsequence Evaluation-step bound vs. measured behavior

87 First automatically derived bound for LCS. Longest Common Subsequence Evaluation-step bound vs. measured behavior

88 Insertion Sort for Strings Evaluation-step bound vs. measured behavior

89 First automatic bound for a sorting algorithm for strings. Insertion Sort for Strings Evaluation-step bound vs. measured behavior

90 Macro Benchmarks 1) OCaml s standard list library list.ml Evaluation-step bounds for 47 of 51 top-level functions 428 lines of code; 3.2 seconds on a Macbook Pro 2) CompCert C Compiler OCaml code extracted from the Coq specification Evaluation-step bounds for 13 topmost modules in the dependency graph 138 of 164 functions bounded; 2740 lines of code; 21min

91 Macro Benchmarks 1) OCaml s standard list library list.ml Evaluation-step bounds for 47 of 51 top-level functions 428 lines of code; 3.2 seconds on a Macbook Pro 2) CompCert C Compiler OCaml code extracted from the Coq specification Evaluation-step bounds for 13 topmost modules in the dependency graph 138 of 164 functions bounded; 2740 lines of code; 21min Problems: Modules and untyped code.

92 Metric #Funs LOC Time #Const #Lin #Quad #Cubic #Poly #Fail Asym. Tight steps s heap s tick s CompCert: tick s steps s Macro Benchmarks

93 How can we make predictions about compiled code?

94 Machine Learning Cost Models How to obtain realistic cost metrics for high-level analysis? Treat hardware, compiler, and runtime systems as black box Select training programs that cover relevant operations Use linear regression to obtain average time and memory costs of operations Combine time and memory predictions to get a time model for execution with garbage collection

95 Bound for List Append on x GHz Intel Core i5-5250u processor Error Actual Time Expected Time 35 Execution Time (ms)

96 Outline Motivation How does automatic resource bound analysis work? How well does automatic resource bound analysis work? (Implementation and experiments) What are the properties of the LP instances that we get?

97 Network-Flow Problems Constraints we derive are almost network-flow problems

98 Network-Flow Problems Constraints we derive are almost network-flow problems Network-flow constraints: X x i X x j = b i j

99 Network-Flow Problems Constraints we derive are almost network-flow problems Network-flow constraints: X x i X x j = b Inflow. i j

100 Network-Flow Problems Constraints we derive are almost network-flow problems Network-flow constraints: X x i X x j = b Inflow. i j Outflow.

101 Network-Flow Problems Constraints we derive are almost network-flow problems Network-flow constraints: X x i X x j = b Constant sink/ source. Inflow. i j Outflow.

102 Network-Flow Problems Constraints we derive are almost network-flow problems Network-flow constraints: X x i X x j = b Constant sink/ source. Inflow. i j Outflow. l i apple x i apple u i

103 Network-Flow Problems Constraints we derive are almost network-flow problems Network-flow constraints: X x i X x j = b Constant sink/ source. Inflow. i j Outflow. l i apple x i apple u i Flow capacity.

104 Linear Constraints have a Simple Form Pay for constant cost q = q 0 + c Account for size changes q 0 i = q i + q i+1 Recursive call q = p Conditional branches q p 1 q p 2

105 Linear Constraints have a Simple Form Pay for constant cost q = q 0 + c Network constraints. Account for size changes q 0 i = q i + q i+1 Recursive call q = p Conditional branches q p 1 q p 2

106 Linear Constraints have a Simple Form Pay for constant cost q = q 0 + c Network constraints. Account for size changes q 0 i = q i + q i+1 Flow through an edge is used twice. Recursive call q = p Conditional branches q p 1 q p 2

107 Linear Constraints have a Simple Form Pay for constant cost q = q 0 + c Network constraints. Account for size changes q 0 i = q i + q i+1 Flow through an edge is used twice. Recursive call q = p Account for cost recursively. Conditional branches q p 1 q p 2

108 Linear Constraints have a Simple Form Pay for constant cost q = q 0 + c Network constraints. Account for size changes q 0 i = q i + q i+1 Flow through an edge is used twice. Recursive call q = p Account for cost recursively. Conditional branches q p 1 q p 2 Cover cost in both branches (possible waste).

109 Accounting for Size Change List types: Potential: L (q 1,...,q k ) (A) (` : L (q 1,...,q k ) )= P i=1,...,k q i ` i Additive shift: C(q 1,...,q k )=(q 1 + q 2,...,q k 1 + q k, q k ) X i=1,...,k n +1 q i i = q 1 + X i=1,...,k 1 q i+1 n i + X i=1,...,k q i n i

110 Accounting for Size Change List types: Potential: L (q 1,...,q k ) (A) (` : L (q 1,...,q k ) )= P i=1,...,k q i ` i Additive shift: C(q 1,...,q k )=(q 1 + q 2,...,q k 1 + q k, q k ) potential of a list X i=1,...,k n +1 q i i = q 1 + X i=1,...,k 1 q i+1 n i + X i=1,...,k q i n i

111 Accounting for Size Change List types: Potential: L (q 1,...,q k ) (A) (` : L (q 1,...,q k ) )= P i=1,...,k q i ` i Additive shift: C(q 1,...,q k )=(q 1 + q 2,...,q k 1 + q k, q k ) potential of a list constant potential of the tail using the shift X i=1,...,k n +1 q i i = q 1 + X i=1,...,k 1 q i+1 n i + X i=1,...,k q i n i

112 Generation of Linear Constraints 1. It is easy to pass potential to list tails without loss ((x::xs) :L ~q )+c = (xs : L C(~q) )+(q 1 + c) 2. Pattern: one recursive call and polynomial spill ((x::xs):l ~q ) (xs:l ~q )= (xs:l (q 2,...,q k,0) )+q 1 3. It is easy to share potential when aliasing data (` : L ~q+~p )= (` : L ~q )+ (` : L ~p )

113 Generation of Linear Constraints 1. It is easy to pass potential to list tails without loss C(q 1,...,q k )=(q 1 + q 2,...,q k 1 + q k, q k ) ((x::xs) :L ~q )+c = (xs : L C(~q) )+(q 1 + c) 2. Pattern: one recursive call and polynomial spill ((x::xs):l ~q ) (xs:l ~q )= (xs:l (q 2,...,q k,0) )+q 1 3. It is easy to share potential when aliasing data (` : L ~q+~p )= (` : L ~q )+ (` : L ~p )

114 Most Complex Constraints: Sharing f(x) = g(x,x) Assume cost of g(x,y) is 10 x y Bound for f is given as X q i x i i

115 Most Complex Constraints: Sharing f(x) = g(x,x) Assume cost of g(x,y) is 10 x y Bound for f is given as Need to covert q x 2 X x q i i to i X x q i i i

116 Most Complex Constraints: Sharing f(x) = g(x,x) Assume cost of g(x,y) is 10 x y Bound for f is given as Need to covert q x 2 X x q i i to i X x q i i i Constraints: q 1 =1 q q 2 =2 q q k =0 q for k > 2

117 Most Complex Constraints: Sharing f(x) = g(x,x) Assume cost of g(x,y) is 10 x y Bound for f is given as Need to covert q x 2 Constraints: X to i q i x i X i q i x i q 1 =1 q q 2 =2 q q k =0 q for k > 2 Coefficients for change of basis.

118 Constraint Solving in Practice LP solving of our constraints is linear in practice CLP and CPLEX are similar; lp_solve is slow (non-linear) Large programs (with high degree search space) have around 1 million constraints Solving 1 million constraints takes about 1 minute with CLP Generating the constraints takes about as much time as solving them

119 Automatic Amortized Resource Analysis Precise: bounds are multivariate resource polynomials Efficient: inference via linear programming Reliable: formal soundness proof of the bounds Verifiable: type derivation is a certificate Current and future research: Non-polynomial bounds Garbage collection Concurrency Better hardware models

120 Automatic Amortized Resource Analysis Precise: bounds are multivariate resource polynomials Efficient: inference via linear programming Reliable: formal soundness proof of the bounds Verifiable: type derivation is a certificate Current and future research: Non-polynomial bounds Garbage collection Web interface at Concurrency Better hardware models