Integrating Induction, Deduction and Structure for Synthesis

Transcription

1 Integrating Induction, Deduction and Structure for Synthesis Sanjit A. Seshia Associate Professor EECS Department UC Berkeley Students & Postdocs: S. Jha, W.Li, A. Donze, L. Dworkin, B. Brady, D. Holcomb, J. Kotker, D. Sadigh Collaborators: A. Tiwari, S. Gulwani, J. Deshmukh, X. Jin NSF ExCAPE Summer School June 12, 2013

2 Connections in this Talk Software Synthesis Formal Verification Machine Learning (Theory) 2

3 Two Messages 1. Synthesis Everywhere Many problems can be solved effectively when viewed as synthesis 2. Effective Approach: Induction + Deduction + Structure Induction: Learning from examples Deduction: Logical inference and constraint solving Structure: Hypothesis on syntactic form of artifact to be synthesized 3

4 Many Problems map to Synthesis Formal Verification Synthesis of verification artifacts E.g.: Inductive invariants, abstraction functions Formal Specification E.g., synthesis of temporal logic formulas Reverse Engineering E.g., malware analysis System Modeling E.g., timing models for embedded platforms 4

5 Formal Verification as Synthesis Inductive Invariants Abstraction Functions 5

6 Example Verification Problem Transition System Init: I x = 1 y = 1 Transition Relation: x = x+y y = y+x Property: = G (y 1) Attempted Proof by Induction: y 1 x = x+y y = y+x y 1 Fails. Need to Strengthen Invariant: Find s.t. x = 1 y = 1 y 1 x = x+y y = y+x y 1 6

7 Example Verification Problem Transition System Init: I x = 1 y = 1 Transition Relation: x = x+y y = y+x Property: = G (y 1) Attempted Proof by Induction: y 1 x = x+y y = y+x y 1 Fails. Need to Strengthen Invariant: Find s.t. x 1 y 1 x = x+y y = y+x x 1 y 1 Safety Verification Invariant Synthesis 7

8 One Reduction from Verification to Synthesis NOTATION Transition system M = (I, ) Safety property = G( ) VERIFICATION PROBLEM Does M satisfy? SYNTHESIS PROBLEM Synthesize s.t. I 8

9 Two Reductions from Verification to Synthesis NOTATION Transition system M = (I, ), S = set of states Safety property = G( ) VERIFICATION PROBLEM Does M satisfy? SYNTHESIS PROBLEM #1 Synthesize s.t. I SYNTHESIS PROBLEM #2 Synthesize : S Ŝ where (M) = (I, ˆ ) ˆ s.t. (M) satisfies iff M satisfies 9

10 Common Approach for both: Inductive Synthesis Synthesis of:- Inductive Invariants Choose templates for invariants Infer likely invariants from tests (examples) Check if any are true inductive invariants, possibly iterate Abstraction Functions Choose an abstract domain Use Counter-Example Guided Abstraction Refinement (CEGAR) 10

11 Counterexample-Guided Abstraction Refinement is Inductive Synthesis System +Property Initial Abstraction Function SYNTHESIS Abstract Domain [Anubhav Gupta, 06] VERIFICATION Generate Abstraction Abstract Model + Property Invoke Model Checker Valid Done New Abstraction Function Counterexample Fail Refine Abstraction Function Spurious Counterexample YES Check Counterexample: Spurious? NO Done 11

12 CEGAR = Inductive Synthesis INITIALIZE Structure Hypothesis, Initial Examples Candidate Artifact SYNTHESIZE VERIFY Counterexample Synthesis Fails Verification Succeeds 12

13 CEGAR = Inductive Synthesis = Learning from (Counter)Examples INITIALIZE Concept Class, Initial Examples LEARNING ALGORITHM Candidate Concept Counterexample VERIFICATION ORACLE Learning Fails Learning Succeeds 13

14 Active Learning: Key Elements ACTIVE LEARNING ALGORITHM Search Strategy Selection Strategy Examples 1. Search Strategy: How to search the space of candidate concepts? 2. Example Selection: Which examples to learn from? 14

15 Counterexample-Guidance: A Successful Paradigm for Synthesis and Learning Active Learning from Queries and Counterexamples [Angluin 87a, 87b] Counterexample-Guided Inductive Synthesis (CEGIS) [Solar-Lezama et al., 06] See upcoming lectures Both rely heavily on Verification Oracle Other effective active learning methods? Especially when Verification Oracle is expensive 15

16 Induction + Deduction + Structure Structure Hypotheses (on artifacts to be synthesized) SKETCH, TEMPLATE, Etc. Deductive Procedure Lightweight : solves lower complexity problem or special case of original decision problem Inductive Procedure Active Learning: selects examples to learn from S. A. Seshia, Sciduction: Combining Induction, Deduction, and Structure for Verification and Synthesis, DAC

17 Reverse Engineering Malware Obfuscated code: Input: y Output: modified value of y What it does: { a=1; b=0; z=1; c=0; while(1) { if (a == 0) { if (b == 0) { y=z+y; a =~a; b=~b; c=~c; if (~c) break; } else { z=z+y; a=~a; b=~b; c=~c; if (~c) break; } } else if (b == 0) {z=y << 2; a=~a;} else { z=y << 3; a=~a; b=~b;} } } y = y * 45 Paper: S. Jha et al., Oracle-Guided Component-Based Program Synthesis, ICSE We solve this using program synthesis. FROM CONFICKER WORM 17

18 Reverse Engineering Malware Obfuscated code: Input: y Output: modified value of y What it does: { a=1; b=0; z=1; c=0; y = y * 45 while(1) Loop-free { compositions of components if (a == 0) {(+, -, <<, >>, *, if-then-else, ) We solve this using if (b == 0) { y=z+y; a =~a; + program synthesis. b=~b; Learning c=~c; if from (~c) break; Distinguishing } I/O Examples else { + z=z+y; a=~a; b=~b; c=~c; SMT Solving (bit-vector arithmetic) if (~c) break; } } else if (b == 0) {z=y << 2; a=~a;} else { z=y << 3; a=~a; b=~b;} FROM } } CONFICKER WORM Paper: S. Jha et al., Oracle-Guided Component-Based Program Synthesis, ICSE

19 Synthesizing Switching Logic for Hybrid Systems???? SAFETY: Room Temperature x must lie between 20 and 22 C. OPTIMALITY: Minimize switching between modes to save energy Papers: S. Jha et al., ICCPS 2010 and EMSOFT

20 Synthesizing Switching Logic for Hybrid Systems? Guards are Hyperboxes? +? Hyperbox Learning from +/- Examples (safe/unsafe switching? states) + Numerical Simulation (constraint solving) SAFETY: Room Temperature x must lie between 20 and 22 C. OPTIMALITY: Minimize switching between modes to save energy Papers: S. Jha et al., ICCPS 2010 and EMSOFT

21 Reactive Synthesis from LTL Environment Assumptions System Requirements e s Synthesis Tool Realizable e s FSM Unrealizable Often due to missing environment assumptions! Paper: W. Li et al., Mining Assumptions for Synthesis, MEMOCODE

22 Reactive Synthesis from LTL Environment Assumptions System Requirements e Synthesis Tool Realizable Env Assumptions are Restricted GR(1) + Version Space Learning (from Counterstrategies) s + (finite-state) Unrealizable Model Checking Often due to missing environment assumptions! e s FSM Paper: W. Li et al., Mining Assumptions for Synthesis, MEMOCODE

23 Outline Program Synthesis from Input-Output Examples Synthesizing Environment Assumptions for Reactive Synthesis from LTL Conclusions and Future Directions 23

24 Reverse Engineering Malware by Program Synthesis Obfuscated code: Input: y Output: modified value of y What it does: { a=1; b=0; z=1; c=0; while(1) { if (a == 0) { if (b == 0) { y=z+y; a =~a; b=~b; c=~c; if (~c) break; } else { z=z+y; a=~a; b=~b; c=~c; if (~c) break; } } else if (b == 0) {z=y << 2; a=~a;} else { z=y << 3; a=~a; b=~b;} } } y = y * 45 Paper: S. Jha et al., Oracle-Guided Component-Based Program Synthesis, ICSE We solve this using program synthesis. FROM CONFICKER WORM 24

25 Class of Programs: Loop-Free Programs implementing functions: I O P(I): O 1 = f 1 (V 1 ) O 2 = f 2 (V 2 ) O n = f n (V n ) where f 1,f 2,,f n are functions from a given component library Functions could be if-then-else definitions and hence, the above represents any loop-free code. 25

26 Problem Specification Oracle Program Space I/O Oracle I/O Examples that identify the correct program? Space of all possible programs Each dot represents semantically unique program 26

27 Program Learning as Set Cover [Goldman & Kearns, 1995] Space of all possible programs Each dot represents semantically unique program 27

28 Program Learning as Set Cover Space of all possible programs Each dot represents semantically unique program 28

29 Program Learning as Set Cover (i 1, o 1 ) Example Set of programs ruled out by that example Space of all possible programs Each dot represents semantically unique program 29

30 Program Learning as Set Cover (i 1, o 1 ) - E 1 (i 2, o 2 ) - E 2 (i n, o n ) - E n Theorem: [Goldman & Kearns, 95] Smallest set of I/O examples to learn correct program Space of all possible programs Each dot represents semantically unique program IS Minimum size subset of {E 1, E 2,, E n } that covers all the incorrect programs 30

31 Program Learning as Set Cover (i 1, o 1 ) - E 1 (i 2, o 2 ) - E 2 Practical challenge: can t enumerate all inputs and find set E i for each (i n, o n ) - E n Smallest set of I/O examples to learn correct design Space of all possible programs Each dot represents semantically unique program IS Minimum size subset of {E 1, E 2,, E n } that cover all the incorrect programs 31

32 Program Learning as Set Cover (i 1, o 1 ) - E 1 (i 2, o 2 ) - E 2 (i n, o n ) - E n ONLINE set-cover: Space of all possible programs Each dot represents semantically unique program In each step, choose some (i j,o j ) pair eliminated incorrect programs E j disclosed 32

33 Program Learning as Set Cover (i 1, o 1 ) - E 1 (i 2, o 2 ) - E 2 Our heuristic: E j 1: atleast one incorrect program identified (i n, o n ) - E n ONLINE set-cover: Space of all possible programs Each dot represents semantically unique program In each step, choose some (i j,o j ) pair eliminated incorrect programs E j disclosed Efficient model counting could yield better algorithm 33

34 Our Approach: Learning based on Distinguishing Inputs [Jha et al., ICSE 10] Space of all possible programs Each dot represents semantically unique program 34

35 Our Approach Example I/O set E := {(i 1,o 1 )} Space of all possible programs 35

36 Our Approach Example I/O set E := {(i 1,o 1 )} SMT P1 P2 SMT Space of all possible programs 36

37 Our Approach Example I/O set E := {(i 1,o 1 )} P1 SMT P2 i 2 Space of all possible programs 37

38 Our Approach Example I/O set E := {(i 1,o 1 )} P1 P2 i 2 (i 2, o 2 ) Input/output Oracle Space of all possible programs 38

39 Our Approach Example I/O set E := E {(i 2,o 2 )} Space of all possible programs 39

40 Our Approach Example I/O set E := E {(i j,o j )} Space of all possible programs 40

41 Our Approach Example I/O set E := E {(i k,o k )} Space of all possible programs 41

42 Our Approach Example I/O set E := E {(i n,o n )} Semantically Unique Program Correct Program? Space of all possible programs 42

43 Soundness Conditional on Validity of Structure Hypothesis YES Library of components is sufficient? NO YES I/O pairs show infeasibility? NO Correct design Infeasibility reported Incorrect design Can invoke a Verification Oracle at the end and iterate 43

44 Other Important Details [Jha et al., ICSE 10, PLDI 11] Novel encoding of the space of possible programs as an SMT formula Obtain a feasible program for given set of input/output pairs using SMT solving Obtain second feasible program and a distinguishing input using SMT solving 44

45 Result Highlights Malware Deobfuscation Conficker worm MyDoom and survey paper on obfuscations by Collberg et al* Synthesized over 35 bit-manipulation programs from Hacker s delight (the Bible of bit-manipulation ). Program length: 3-15 Number of input/output examples: 2 to 13. Total runtime: < 1 second to 5 minutes. *C. Collberg, C. Thomborson, and D. Low. A taxonomy of obfuscating transformations. Technical Report 148, Dept. Comp. Sci., The Univ. of Auckland, July

46 Outline Program Synthesis from Input-Output Examples Synthesizing Environment Assumptions for Reactive Synthesis from LTL Conclusions and Future Directions 46

47 Synthesis from LTL Environment Assumptions System Requirements e s Synthesis Tool Unrealizable Often due to incomplete environment assumptions! Realizable e s FSM 47

48 Satisfiability and Realizability A LTL formula is satisfiable if there exists an infinite word (i.e. sequence of inputs and outputs) that satisfies. A LTL specification is realizable if there exists a finitestate transducer M (e.g. a Moore machine) which, for any input sequence, generates computations that satisfies. 48

49 Problem Description Goal: Generate additional assumptions to enable synthesis Context: Original specification is satisfiable but unrealizable Assume: - Given a few interesting user scenarios (satisfying traces) - Specifications are in the GR(1) class Challenge: - Space of possible additional assumptions is huge -Want assumptions that can be understood and analyzed by a human user 49

50 Example Inputs: request r and cancel c Outputs: grant g System specification s : - G (r X F g) - G(c g X g) Environment assumption e : - True No user scenarios. Not realizable because the environment can force c to be high all the time 50

51 Our Contribution [Wenchao Li et al., MEMOCODE 2011] Counterstrategy-guided synthesis of environment assumptions Demonstrated to generate useful/intuitive environment assumptions for digital circuits and robotic controllers 51

52 Approach for Synthesizing Environment Assumptions Structure Hypothesis: Environment Assumptions are Restricted GR(1) properties + Inductive Inference: Version Space Learning from Counterstrategies + Deductive Engine: (Finite-state) Model Checking 52

53 GR(1) Synthesis [Piterman, Pnueli, Saar] Formulas in the form: e s o Input and output partitions I and O. o i : initial state formulas. {e, s} o t : transition formulas, in the form of G B, where B is a Boolean combination of variables in I O and expressions X u, u I if = e and u I O if = s. o f : fairness formulas, in the form of G F B, where B is a Boolean formula over I O. o Synthesis as a turn-based two-player game between the system and the environment o Realizable if the system has a winning strategy, otherwise env wins; Strategy representable as finite-state transducer 53

54 Counter-strategy and counter-trace Counter-strategy is a strategy for the environment to force violation of the specification. Counter-trace is a fixed input sequence such that the specification is violated regardless of the outputs generated by the system. 54

55 Example System s : - G (r X F g) - G(c g X g) A counter-trace: r: 1 1 (1) c: 1 1 (1) 55

56 CounterStrategy-Guided Environment Assumption Synthesis Start Formal Specification Synthesis Tool Realizable Done Specification Templates User Scenarios Add Mine Assumptions Unrealizable Compute Counterstrategy 56

57 Assumption Synthesis Algorithm Specification Templates User Scenarios e.g. GF? Generate Candidate φ e.g. GFp Eliminate φ 57 Counterstrategy Eliminate or Accept Candidate Accept φ Done (check consistency with existing assumptions)

58 Assumption Synthesis Algorithm Specification Templates User Scenarios e.g. GF? Generate Candidate φ e.g. GFp VERSION SPACES: Retain candidates consistent with examples Traverse weakest to strongest NO: Eliminate φ 58 Counterstrategy E Invoke Model Checker: Does E satisfy φ? YES: Accept φ Done (check consistency with existing assumptions)

59 Assumption Templates follow GR(1) 1 : G F?b, where b I 2 : G (?b 1 X?b 2 ), where b 1 I O and b 2 I 3 : G (?b 1?b 2 ), where b 1, b 2 I The specification remains in GR(1) after the addition of new assumptions. Occam s Razor: Simple propositional structure generalizable Selecting candidate assumptions: (next slide) Weakest to Strongest, with heuristics G F b weaker than G X b weaker than G b 59

60 Version Space Learning Originally due to [Mitchell, 1978] G p 1 false G p 2... STRONGEST (most specific) G p n G p 1 p G p n-1 p n G X p 1 G X p 2... G X p n G (p 1 X p 2 )... G F p 1 G F p 2 true... G F p n WEAKEST (most general)

61 Theoretical Results Theorem 1: [Completeness] If there exist environment assumptions under our structure hypothesis that make the spec realizable, then the procedure finds them (terminates successfully). conditional completeness guarantee Theorem 2: [Soundness] The procedure never adds inconsistent environment assumptions. 61

62 Example System s : - G (r X F g) - G(c g X g) A counter-trace: r: 1 1 (1) c: 1 1 (1) Test assumption candidates by checking its negation: o G (F c) o G (F c) System s : - G (r X F g) - G(c g X g) Environment e : - G (F c) 62

63 Experimental Results Summary Experiment Setup: - Remove assumptions from an originally realizable specification - Use a few (often a single) satisfying traces of the original specification as representative user scenarios - Mine additional assumptions until the specification is realizable Use Cadence SMV to generate the satisfying traces and model check the counter-strategies Use RATSY [Bloem et al. 2010] to check realizability of the specifications and compute the counter-strategies and counter-traces in case of unrealizability 63

64 Experimental Results Summary Result Summary: - Case studies in existing literature: AMBA AHB, IBM Gen Buffer, robotic vehicle controller, etc. - Recovered the missing assumption in most cases - AMBA AHB Example: Original assumption: G (HLOCK[0] = 1 HBUSREQ[0] = 1) Mined assumption: G (F HLOCK[0] = 0) Master 0 requests locked access to the bus Master 0 requests access to the bus 64

65 Outline Switching Logic Synthesis for Hybrid Systems Reflections on the Approach Synthesizing Environment Assumptions for Reactive Synthesis from LTL Conclusions and Future Directions 65

66 Induction + Deduction + Structure Structure Hypothesis encodes human insight about form of artifact to be synthesized Synthesis procedure combines inductive inference with deductive reasoning Many instances of this approach exist already Key Points of Difference between instances: 1. Search Strategy: How to search the space of candidate concepts? 2. Example Selection: Which examples to learn from? 3. Deductive Oracles 66

67 Comparison of Three Approaches Traditional Counterexample- Guided Synthesis Program Synthesis from I/O Examples Env Assumption Generation for LTL Synthesis Search Strategy for Candidate Programs Determined by Constraint Encoding & Solver Heuristics Determined by Constraint Encoding & Solver Heuristics Version Spaces Example Selection Strategy Counter-Example guided Distinguishing Input guided Counter-Strategy (counter-example) guided 67

68 Ongoing and Future Work Many Other Applications Switching logic synthesis for safety and optimality [Jha et al 10, 11] Fixed-point code from floating-point [Jha & S, 11, 13] Synthesizing Abstractions for Hardware Verification [Brady et al., 11] Synthesizing Requirements for Industrial Control Models [Jin et al, 13] Need better understanding of Inductive Synthesis Explore range of learning models Role of structure hypothesis Control over deductive oracles (e.g., via problem encoding / search strategy in SAT/SMT) 68