Synthesizing from Components: Building from Blocks

Transcription

1 Synthesizing from Components: Building from Blocks Ashish Tiwari SRI International 333 Ravenswood Ave Menlo Park, CA Joint work with Sumit Gulwani (MSR), Vijay Anand Korthikanti (UIUC), Susmit Jha (UC Berkeley), Sanjit Seshia (UC Berkeley), Thomas Sturm (Munich), Ankur Taly (Stanford), Ramarathnam Venkatesan (MSR) MSR, Redmond Component-based Synthesis: 1

2 Component-Based Synthesis... Problem: How to wire the components to synthesize a desired system? MSR, Redmond Component-based Synthesis: 2

3 Concrete Examples Desired SystemF spec Components f i s sort an array comparators compute x+y 2 modulo arithmetic ops find rightmost one bitwise ops, arithmetic ops compute x 243 accept ω-regular language safe hybrid system geometry construction deobfuscated code verification proof multiplication Buchi automata multiple operating modes ruler-compass steps parts of obfuscated code verification inference rules Question: C : x : F spec (x) = C(f 1,f 2,...)(x) MSR, Redmond Component-based Synthesis: 3

4 Synthesis Problem Classes This is difficult This is ill posed This is too general to be solvable C : x : F spec (x) = C(f 1,f 2,...)(x) Parameters that define the synthesis problem: composition operator C class of specifications F spec class of component specifications f i Fixing the synthesis problem: fix these parameters, fix representation off spec,f i MSR, Redmond Component-based Synthesis: 4

5 Bounded Synthesis The synthesis problem is still hard We make it feasible by replacing the unbounded quantifier, C, by a bounded quantifier C : x : F spec (x) = C(f 1,f 2,...)(x) c : x : F spec (x) = c(f 1,f 2,f 3 )(x),c in some finite set This bounded synthesis problem is solved by deciding the formula MSR, Redmond Component-based Synthesis: 5

6 Straight-Line Program Synthesis composition operator components system function composition primitive functions complex function Bounded synthesis version: fix length of program fix upper bound on number of each component P : x : F spec (x) = P(x), P a straight-line program composingf i s π : x : F spec (x) = f π(1) (f π(2) (f π(3) (x))) MSR, Redmond Component-based Synthesis: 6

7 Example: Straight-Line Program Synthesis Specification: Evaluate polynomial a h 2 +b h+c Budget: two multiplication and two addition operators Finite search space Synthesized Program: 1. o 1 := a h; 2. o 2 := o 1 +b; 3. o 3 := o 2 h; 4. return o 3 +c; Correctness: (a h+b) h+c = a h 2 +b h+c MSR, Redmond Component-based Synthesis: 7

8 Example: Straight-Line Program Synthesis Specification: Turn-off rightmost contiguous 1 bits Example: Budget: two addition and at most four bitwise Boolean operators Finite search space: Also need some constants Synthesized Program: 1. o 1 := x+( 1); 2. o 2 := o 1 x; 3. o 3 := o 2 +1; 4. return o 3 &x; Correctness on sample input: MSR, Redmond Component-based Synthesis: 8

9 Loop-free Program Synthesis composition operator components system function composition primitive functions, if-then-else complex function Bounded synthesis version: fix length of program fix upper bound on number of each component including if-then-else P : x : F spec (x) = P(x), P a straight-line program composingf i s π : x : F spec (x) = f π(ǫ) (f π(1) (f π(11) (x 1 ),f π(12) (x 2,x 1 ))) MSR, Redmond Component-based Synthesis: 9

10 Example: Loop-free Program Synthesis Specification: Obfuscated code Example: We are given if (h(x)) if (x*(x+1)% 2 == 1) y := f(x) else y := g(x) else y := f(g(x)) Components Budget: f, g, h, if-then-else Synthesized Program: o := g(x); if (h(x)) y := o; else y := f(o); Correctness: Equivalence of two loop-free programs MSR, Redmond Component-based Synthesis: 10

11 Loop-free Program Synthesis π : x : F spec (x) = f π(ǫ) (f π(1) (f π(11) (x 1 ),f π(12) (x 2,x 1 ))) Enumerate all possible programs and check Enumerate all permutations π and check Checking if a synthesized program is the desired program is a verification problem Bounded Synthesis := iteratively perform verification But we can learn from failures... MSR, Redmond Component-based Synthesis: 11

12 φ Solvers Bounded Synthesis solving How to solve u : x : φ formulas? A1 Counter-example guided iterative solver A2 Distinguishing input solver Applies even when φ not fully known A3 Numerical solver MSR, Redmond Component-based Synthesis: 12

13 A1: Solving φ Counter-example guided iterative procedure for solving u : x : φ( u, x) 1. Guess u 0 for u 2. (Verification) Check if x : φ( u 0, x) 3. If true, then return u 0 4. Get counterexample x 0, add it tox 5. (Finite Synthesis) Find new u 0 such that u 0 : φ( u 0, x 0 ) x 0 X 6. Go to Step 2 MSR, Redmond Component-based Synthesis: 13

14 A1: Counter-example Guided Iterative Solving Needs a backend quantifier-free solver That can return counterexamples We use an SMT solver The structure of φ, and additional knowledge about what φ encodes, is used optimize the above procedure to expedite convergence Related Work: Sketch, Aha Reference: Synthesis of loop-free programs, PLDI 2011 MSR, Redmond Component-based Synthesis: 14

15 A2: Distinguishing Input Solver Solving u : x : φ( u, x) 1. X := some finite set of choices for x 2. Find two programs that work for X, but differ on some x 0 u 1, u 2, x 0 : ( (φ( u 1, x) φ( u 2, x))) (φ( u 1, x 0 ) φ( u 2, x 0 )) x X 3. If satisfiable, we add x 0 tox and go to (2) 4. If unsatisfiable, then find one program that works for X u 1 : φ( u 1, x) 5. If satisfiable, return u 1 x X 6. Otherwise, return not synthesizable MSR, Redmond Component-based Synthesis: 15

16 A2: Properties of the A2 Solver The second algorithm for solving u : x : φ( u, x) Does not need the full specification of the desired program We only need the knowledge of the specification on the set X Does not perform the verification step An interative implementation of A2: 1. Tool asks user for the expected output on input x 0 2. Tool synthesizes internally two programs that work correctly for X := { x 0 }, but differ on input x 1 3. Tool asks user for the expected output on input x 1 4. Add x 1 tox and repeat MSR, Redmond Component-based Synthesis: 16

17 A3: Nonsymbolic Solver A third algorithm for solving u : x : φ( u, x) 1. Find finite set X of input-output pairs of the specification 2. Synthesize program that works for finite set X 3. Verify the synthesized program on randomly sampled inputs We solved Step (2) using an SMT solver previously We can avoid the SMT solver and instead 1. hierarchical program synthesis: first synthesize high-level components 2. enumerate composition of high-level components guided by goal MSR, Redmond Component-based Synthesis: 17

18 Example: Synthesis Without Symbolic Reasoning Specification: Construct a triangle, given its base, a base angle and sum of the other two sides. Components: Ruler compass constructions Formal specification: Given pointsp 1,p 2 and numbers a,r, find point p φ pre := r > length(p 1,p 2 ) φ post := Angle(p,p 1,p 2 ) = a length(p,p 1 )+length(p,p 2 ) = r Construction: L1 := ConstructLineGivenAngleLine(L,a); C1 := ConstructCircleGivenPointLength(p1,r); (p3,p4) := LineCircleIntersection(L1,C1); L2 := PerpendicularBisector2Points(p2,p3); p5 := LineLineIntersection(L1,L2); MSR, Redmond Component-based Synthesis: 18

19 Example: Geometry Construction Synthesis Step 1 find concrete input-output pair consistent with specification L = Line( 81.62, 99.62, 99.62, ) r = a = 0.81 radians Compute output for this input: p := , Step 2 Start enumerating partial programs built using an extended library Step 3 Evaluate if intermediate objects generated by the partial program are good and try other choices in Step (2) otherwise MSR, Redmond Component-based Synthesis: 19

20 Geometry Construction Synthesis Evaluting effect of making search goal directed Points generated by goal-directed search Points generated by brute-force search depth 0 (Input) depth 1 depth 2 depth 3 depth 4 depth 5 (Output) 100 depth 0 (Input) depth 1 depth 2 depth 3 depth 4 depth 5 depth 6 (Output) y 100 y x x Points visited in a goal-directed search (left) and a brute-force search (right). MSR, Redmond Component-based Synthesis: 20

21 Geometry Construction Synthesis Extended library is forward search Encodes knowledge / concept taught in class Goal directness is backward search Corresponds to reasoning student expected to do Sample input-output points generated using numerical techniques MSR, Redmond Component-based Synthesis: 21

22 Switching Logic Synthesis Given a multimodal dynamical system Synthesize conditions for switching between modes such that some requirements are met MSR, Redmond Component-based Synthesis: 22

23 Example: Driving a Robot 10 The goal is to drive the robot starting from Init to Reach while remaining inside Safe: 3 0 Using the 2 modes: 3 Init := (x [ 1,1], y = 0, v x = 0, v y = 0) Reach := (y 10) Safe := ( x 3) Mode 1: Force applied in(1,1)-direction dx dt = v x, dv x dt = 1 v x, Mode 2: Force applied in( 1,1)-direction dx dt = v x, dv x dt = 1 v x, dy dt = v y, dy dt = v y, dv y dt = 1 v y dv y dt = 1 v y MSR, Redmond Component-based Synthesis: 23

24 Example: Driving a Robot We synthesize a non-deterministic controller: a set of different possible switchings that each satisfy the requirement SafeUReach. Two possible trajectories: Position x= 3 x=3 Alt Position 8 Position y Position x How to discover the correct switching logic? MSR, Redmond Component-based Synthesis: 24

25 Switching Logic Synthesis switching conditions : state variables : correctness We can again bound the search for switching conditions But that is a bad solution Need to go back to verification MSR, Redmond Component-based Synthesis: 25

26 Verification Techniques 1. Reachability-Based Verification 2. Abstraction-Based Verification 3. Certificate-Based Verification Key Observation: Verification = searching for right certificate Property Stability Safety Liveness Witness/Certificate Lyapunov function Inductive Invariant Ranking function MSR, Redmond Component-based Synthesis: 26

27 Certificate-Based Verification Verifying property P in system S := C : C is a certificate forp ins Can do a bounded search for C Also known as the constraint-based approach Certificates for Synthesis Problem: Property Safety Stability Witness/Certificate Controlled Inductive Invariant Controlled Lyapunov function MSR, Redmond Component-based Synthesis: 27

28 Bounded Synthesis of Switching Logic Given multimodal dynamical system, and property Safe: Guess templates for the certificate for controlled-safety Generate the a,b,... : x,y,... : φ Solve the formula to get values for a,b,... MSR, Redmond Component-based Synthesis: 28

29 Solvers Need u : x : φ solvers for the reals We can use the same ideas as before Symbolic Numeric Approach: Symbolic: A combination of QEPCAD, redlog, slfq to eliminate inner Numeric: Gradient descent to findufrom resulting formula Iterative learning: Iteratively prune out u values based on simulations MSR, Redmond Component-based Synthesis: 29

30 Conclusion Synthesis: solving Bounded synthesis: Make problem tractable by making afinite quantification Component-based Synthesis Various approaches to solve depending on application Switching logic synthesis : search for controlled certificates MSR, Redmond Component-based Synthesis: 30