Quantifier Instantiation Techniques for Finite Model Finding in SMT

Transcription

1 Quantifier Instantiation Techniques for Finite Model Finding in SMT Andrew Reynolds, Cesare Tinelli Amit Goel, Sava Krstic Morgan Deters, Clark Barrett

2 Satisfiability Modulo Theories (SMT) SMT solvers are powerful tools Used in many formal methods applications Support many background theories Arithmetic, bitvectors, arrays, datatypes, May generate: Proofs Theorem proving, software/hardware verification Models Failing instances of aforementioned applications Invariant synthesis, scheduling, test case generation

3 SMT: Limitations SMT solvers effective handling ground formulas Fast decision procedures for UF, arithmetic, Ongoing challenge: quantified formulas Heuristic methods for answering unsat Limited capability of answering sat Often will return unknown after some effort

4 Contributions Finite Model Finding in SMT Different from ATP finite model finders: Native support for background (ground) theories Different from SMT solvers: Increased ability to answer satisfiable New techniques for: Constructing good candidate models Efficiently checking candidate models

5 DPLL(T) Architecture Formula F F is sat Satisfying assignment A for F UNSAT, proof F is unsat SAT Solver Clauses to add to F Theory Solvers A is T-Inconsistent A is T-Consistent SAT, model

6 DPLL(T) Architecture : Challenge Formula F F is sat Satisfying assignment A for F UNSAT, proof F is unsat SAT Solver Theory Solvers A is T-Consistent SAT, model Clauses to add to F A is T-Inconsistent Challenge: What if determining the consistency of A is difficult? For quantified formulas, determining consistency is undecidable

7 Heuristic Instantiation for Q Formula F F is sat Satisfying assignment A for F (containing Q) UNSAT, proof F is unsat SAT Solver Instances of Q to add to F Theory Solvers Consistency of A is unknown SAT, model If sat assignment contains quantified formula Q, Heuristically add instances of Q to F Typically based on pattern matching May discover refutation, if right instances are added

8 Why Models are Important Verification Condition for P (with quantifiers) UNSAT Property P is verified SMT solver Unknown Candidate Model Manual Inspection

9 Why Models are Important Verification Condition for P (with quantifiers) UNSAT Property P is verified SMT solver Unknown Candidate Model Manual Inspection SAT Concrete counterexample for Property P

10 Model-Based Approach for Quantifiers Given: Set of ground formulas F Set of universally quantified formulas Q To determine the satisfiability of F Q, Construct candidate models for Q, based on satisfying assignments for F Model-Based Quantifier Instantiation (MBQI) [Ge/deMoura 2009]

11 DPLL(T) Architecture (Extended) Ground Formulas F F is sat Satisfying assignment A for F A is T-Consistent Candidate model M Quantified Formulas Q UNSAT, proof else SAT Solver Theory Solvers Model Verifier Clauses to add to F else else SAT, model M M is a model for Q

12 When can we represent/check models for Q? Focus of talk: Finite Model Finding Limited to quantifiers over: Uninterpreted sorts Can represent memory addresses, values, sets, etc. Other finite sorts Fixed width bitvectors, datatypes, Useful in applications: Software/hardware verification

13 Constructing/Checking Candidate Finite Models F Sat assignment A for F (Finite) Candidate model M Q UNSAT, proof Ground Solver Model Verifier Instances of Q to add to F SAT, model M 1. How do we construct good candidate models M? 2. How do we efficiently check if M is a model for Q? If we fail, which instances do we add to F?

14 Constructing Good Candidate Models

15 Constructing Good Candidate Models Naïvely, to determine whether M is model for Q: Check if M satisfies all instances S of Q Challenge: S can be very large For Q with n variables, domain size d, S can be O( d n ) Solutions: Find models with small domain sizes Use theory of finite cardinality constraints [CAV 2013] Only consider instances of Q that are false in M Construct M such that most instances of Q are true

16 Constructing Models : Example F distinct(newyork, Boston, Seattle) travels(person 1, Boston) salesman(person 2 ) salesman(person 3 ) person 1, person 2, person 3 : Person NewYork, Boston, Seattle : City salesman: Person Bool travels : Person City Bool x : Person, y : City. salesman(x) travels(x,y) Q

17 Constructing Models : Example F distinct(newyork, Boston, Seattle) travels(person 1, Boston) salesman(person 2 ) salesman(person 3 ) person 1, person 2, person 3 : Person NewYork, Boston, Seattle : City salesman: Person Bool travels : Person City Bool x : Person, y : City. salesman(x) travels(x,y) Q Interpretation of salesman : salesman(person 2 ), salesman(person 3 ) T, salesman( x 1 ) Interpretation of travels : travels(person 1, Boston) T, travels( x 1, x 2 )

18 Model Representation Represent functions/predicates using defining maps Given sort S with domain V, A defining map for f : S S S is: Set of equations D f of the form f( t 1,, t n ) v, where v V Each t i is either a unique variable or in V If t 1 v 1 and t 2 v 2 D f, unifiable with mgu s, then: s is non-empty t 1 s v D f for some v f( x 1,, x n ) v D f for some v

19 Interpretation in Model Interpretation f( t 1,, t n ) is v, where: t v D f t is most specific generalization of f( t 1,, t n ) among LHS in D f Guaranteed to exist and be unique

20 Constructing Models Defining map D f is a union of: Entailed ground equalities Non-ground equalities for defining default values How to chose default values? Guided by sat assignment for distinguished elements See how one instance is satisfied, generalize this for all

21 Constructing Models : Example F distinct(newyork, Boston, Seattle) travels(person 1, Boston) salesman(person 2 ) salesman(person 3 ) person 1, person 2, person 3 : Person NewYork, Boston, Seattle : City salesman: Person Bool travels : Person City Bool salesman(person 1 ) travels(person 1, NewYork) x : Person, y: City. salesman(x) travels(x,y) Q Instantiate Q with distinguished elements

22 Constructing Models : Example F distinct(newyork, Boston, Seattle) travels(person 1, Boston) salesman(person 2 ) salesman(person 3 ) person 1, person 2, person 3 : Person NewYork, Boston, Seattle : City salesman: Person Bool travels : Person City Bool salesman(person 1 ) travels(person 1, NewYork) x : Person, y: City. salesman(x) travels(x,y) Choose defaults based on distinguished instance Analagous to Inst Gen Calculus [Korovin 2008] T Q D salesman : { salesman(person 2 ), salesman(person 3 ) T, salesman( x 1 ) T } D travels : { travels(person 1, NewYork) T travels(person 1, Boston), travels( x 1, x 2 ) T }

23 Efficiently Checking Candidate Models

24 Checking Candidate Models F Instances of Q to add to F Ground Solver (Finite) Candidate model M Model Verifier SAT, model M Q M is a model for Q To check if M is a model for Q: Naively, add every instance of Q to F Alternatively, only add instances that are false in M Identify sets of instances of Q that are equisatisfiable

25 Checking Candidate Models F distinct(newyork, Boston, Seattle) travels(person 1, Boston) salesman(person 2 ) salesman(person 3 ) person 1, person 2, person 3 : Person NewYork, Boston, Seattle : City salesman: Person Bool travels : Person City Bool x : Person, y: City. salesman(x) travels(x,y) Q D salesman : salesman(person 1 ) travels(person 1, NewYork) { salesman(person 2 ), salesman(person 3 ) T, salesman( x 1 ) T } D travels : { travels(person 1, NewYork) T travels(person 1, Boston), travels( x 1, x 2 ) T } Q[person 1, NewYork] Q[person 1, Boston] Q[person 1, Seattle] Q[person 2, NewYork] Q[person 2, Boston] Q[person 2, Seattle] Q[person 3, NewYork] Q[person 3, Boston] Q[person 3, Seattle]

26 Checking Candidate Models F distinct(newyork, Boston, Seattle) travels(person 1, Boston) salesman(person 2 ) salesman(person 3 ) person 1, person 2, person 3 : Person NewYork, Boston, Seattle : City salesman: Person Bool travels : Person City Bool x : Person, y: City. salesman(x) travels(x,y) Q D salesman : salesman(person 1 ) travels(person 1, NewYork) { salesman(person 2 ), salesman(person 3 ) T, salesman( x 1 ) T } D travels : { travels(person 1, NewYork) T travels(person 1, Boston), travels( x 1, x 2 ) T } Q[person 1, NewYork] Q[person 1, Boston] Q[person 1, Seattle] Q[person 2, NewYork] Q[person 2, Boston] Q[person 2, Seattle] Q[person 3, NewYork] Q[person 3, Boston] Q[person 3, Seattle] true false true true true

27 Enhancement: Heuristic Instantiation Idea: First see if instantiations based on heuristics exist If not, resort to model-based instantiation May lead to: Discovering easy conflicts, if they exist Arriving at model faster Instantiations rule out spurious models

28 DVF Benchmarks Experiments Taken from verification tool DVF used by Intel Both SAT/UNSAT benchmarks SAT benchmarks generated by removing necessary pf assumptions Many theories: UF, arithmetic, arrays, datatypes Quantifiers only over free sorts Memory addresses, Values, Sets, TPTP Benchmarks Isabelle Benchmarks Provable and unprovable goals, contains some arithmetic

29 Results: DVF SAT german refcount agree apg bmk Total Time # z cvc4+i cvc4+f cvc4+fi cvc4+fm cvc4+fmi UNSAT german refcount agree apg bmk Total Time # z cvc4+i cvc4+f cvc4+fi cvc4+fm cvc4+fmi cvc4 : cvc4 with finite model finding (cvc4+f) Effective for answering sat Using heuristic instantiation, solves 4 unsat that cvc4 cannot f : finite model i : heuristic m : model-based

30 Results: TPTP Using techniques described in this work: Of 1995 satisfiable benchmarks: Paradox solves 1305 iprover solves 1231 cvc4 solves 1109 (with model-based instantiation) Includes 2 problems with rating 1.0 Of unsatisfiable benchmarks: z3 solves 5934 cvc4 solves 3028 (with model-based+heuristic instantiation) Orthogonal, 282 cannot be solved by z3 Placed 3 rd in FNT (non-theorem) division of CASC 24

31 Results : TPTP Model-Based Instantiation is often essential Solves where exh. instantiation require >1 billion instances

32 Results: Isabelle SAT Arrow FFT FTA Hoare NS QEP SNorm TwoSq TypeSafe Total z cvc4+i cvc4+f cvc4+fm cvc4+fmi UNSAT Arrow FFT FTA Hoare NS QEP SNorm TwoSq TypeSafe Total z cvc4+i cvc4+f cvc4+fm cvc4+fmi cvc4+fmi solves 244 unsat that z3 cannot, 164 that cvc4 cannot cvc4 : f : finite model i : heuristic m : model-based

33 Summary CVC4 with finite model finding: Constructs good candidate models Incorporates various instantiation strategies Model-based quantifier instantiation Heuristic instantiation (E-matching) Increased ability to answer satisfiable Publicly available:

34 Further Work Further work: Would like to show: Finite model completeness Refutational completeness for certain fragments Improved algorithm for checking candidate models Apply similar techniques to: Bounded integer quantification Datatypes, strings with bounded length

35 Questions? Thank you