Course An Introduction to SAT and SMT. Cap. 2: Satisfiability Modulo Theories

Transcription

1 Course An Introduction to SAT and SMT Chapter 2: Satisfiability Modulo Theories Roberto Sebastiani DISI, Università di Trento, Italy URL: Int. Graduate School on ICT, University of Trento, Academic year last update: Wednesday 12 th December, 2018 Copyright notice: some material contained in these slides is courtesy of Alessandro Cimatti, Alberto Griggio and Marco Roveri, who detain its copyright. All the other material is copyrighted by Roberto Sebastiani. Any commercial use of this material is strictly forbidden by the copyright laws without the authorization of the authors. No copy of these slides can be displayed in public without containing this copyright notice. Wednesday 12 th December, /

2 Outline 1 Motivations and goals 2 Efficient SMT solving Combining SAT with Theory Solvers Theory Solvers for theories of interest SMT for combinations of theories 3 Beyond Solving: advanced SMT functionalities Proofs and unsatisfiable cores Interpolants All-SMT & Predicate Abstraction SMT with cost optimization (Optimization Modulo Theories) 4 Conclusions & current research directions Wednesday 12 th December, /

3 Outline Motivations and goals 1 Motivations and goals 2 Efficient SMT solving Combining SAT with Theory Solvers Theory Solvers for theories of interest SMT for combinations of theories 3 Beyond Solving: advanced SMT functionalities Proofs and unsatisfiable cores Interpolants All-SMT & Predicate Abstraction SMT with cost optimization (Optimization Modulo Theories) 4 Conclusions & current research directions Wednesday 12 th December, /

4 Motivations and goals Satisfiability Modulo Theories (SMT(T )) Satisfiability Modulo Theories (SMT(T )) The problem of deciding the satisfiability of (typically quantifier-free) formulas in some decidable first-order theory T T can also be a combination of theories i T i. Wednesday 12 th December, /

5 Motivations and goals SMT(T ): theories of interest Some theories of interest (e.g., for formal verification) Equality and Uninterpreted Functions (EU F): ((x = y) (y = f (z))) (g(x) = g(f (z))) Difference logic (DL): ((x = y) (y z 4)) (x z 6) UTVPI (UT VPI): ((x = y) (y z 4)) (x + z 6) Linear arithmetic over the rationals (LRA): (T δ (s 1 = s t 3.4 t 0 )) ( T δ (s 1 = s 0 )) Linear arithmetic over the integers (LIA): (x := x l x h ) (x 0) (x ) Arrays (AR): (i = j) read(write(a, i, e), j) = read(a, j) Bit vectors (BV): x [16] [15 : 0] = (y [16] [15 : 8] :: z [16] [7 : 0]) << w [8] [3 : 0] Non-Linear arithmetic over the reals (N LA(R)) : ((c = a b) (a 1 = a 1) (b 1 = b + 1)) (c = a 1 b 1 + 1)... Wednesday 12 th December, /

6 Motivations and goals Satisfiability Modulo Theories (SMT(T )): Example Example: SMT(LIA EUF AR) ϕ = def (d 0) (d < 1) ((f (d) = f (0)) (read(write(v, i, x), i + d) = x + 1)) involves arithmetical, arrays, and uninterpreted function/predicate symbols, plus Boolean operators Is it consistent? No: ϕ = LIA (d = 0) = EUF (f (d) = f (0)) = Bool (read(write(v, i, x), i + d) = x + 1) = LIA (read(write(v, i, x), i) = x + 1) = LIA (read(write(v, i, x), i) = x) = AR Wednesday 12 th December, /

7 Motivations and goals Satisfiability Modulo Theories (SMT(T )): Example Example: SMT(LIA EUF AR) ϕ = def (d 0) (d < 1) ((f (d) = f (0)) (read(write(v, i, x), i + d) = x + 1)) involves arithmetical, arrays, and uninterpreted function/predicate symbols, plus Boolean operators Is it consistent? No: ϕ = LIA (d = 0) = EUF (f (d) = f (0)) = Bool (read(write(v, i, x), i + d) = x + 1) = LIA (read(write(v, i, x), i) = x + 1) = LIA (read(write(v, i, x), i) = x) = AR Wednesday 12 th December, /

8 Motivations and goals Satisfiability Modulo Theories (SMT(T )): Example Example: SMT(LIA EUF AR) ϕ = def (d 0) (d < 1) ((f (d) = f (0)) (read(write(v, i, x), i + d) = x + 1)) involves arithmetical, arrays, and uninterpreted function/predicate symbols, plus Boolean operators Is it consistent? No: ϕ = LIA (d = 0) = EUF (f (d) = f (0)) = Bool (read(write(v, i, x), i + d) = x + 1) = LIA (read(write(v, i, x), i) = x + 1) = LIA (read(write(v, i, x), i) = x) = AR Wednesday 12 th December, /

9 Motivations and goals Some Motivating Applications Interest in SMT triggered by some real-word applications Verification of Hybrid & Timed Systems Verification of RTL Circuit Designs & of Microcode SW Verification Planning with Resources Temporal reasoning Scheduling Compiler optimization... Wednesday 12 th December, /

10 Motivations and goals Verification of Timed Systems T12 x>=1 a l1 l2 x <= 2 x:=0 b T21 x<=3 Bounded/inductive model checking of Timed Systems [6, 36, 58],... Timed Automata encoded into T -formulas: discrete information (locations, transitions, events) with Boolean vars. timed information (clocks, elapsed time) with differences (t 3 x 3 2), equalities (x 4 = x 3 ) and linear constraints (t 8 x 8 = t 2 x 2 ) on Q = SMT on DL(Q) or LRA required Wednesday 12 th December, /

11 Motivations and goals Verification of Hybrid Systems... T12 Grow Steady w >=L cruise Stable w. = 2 L <= w<= U Bounded model checking of Hybrid Systems [5],... Hybrid Automata encoded into L-formulas: discrete information (locs, trans., events) with Boolean vars. timed information (clocks, elapsed time) with differences (t 3 x 3 2), equalities (x 4 = x 3 ) and linear constraints (t 8 x 8 = t 2 x 2 ) on Q Evolution of Physical Variables (e.g., speed, pressure) with linear (ω 4 = 2ω 3 ) and non-linear constraints (P 1 V 1 = 4T 1 ) on Q Undecidable under simple hypotheses! = SMT on DL(Q), LRA or N LA(R) required Wednesday 12 th December, /

12 Motivations and goals Verification of HW circuit designs & microcode CK B1 B2 CK CK a itea g = 2*f b f f = itea+2*e B3 0 Adder L Shifter Adder itea = ITE(B1;a;0) c 0 L Shifter 1 e 1 e = ITE(B2;b;0)+2*ite(B3;c;0) L Shifter control 0 1 data g SAT/SMT-based Model Checking & Equiv. Checking of RTL designs, symbolic simulation of µ-code [25, 22, 42] Control paths handled by Boolean reasoning Data paths information abstracted into theory-specific terms words (bit-vectors, integers, EUF vars,... ): a[31 : 0], a word operations: (BV, EUF, AR, LIA, N LA(Z) operators) x [16] [15 : 0] = (y [16] [15 : 8] :: z [16] [7 : 0]) << w [8] [3 : 0], (a = a L a H ), (m 1 = store(m 0, l 0, v 0 )),... Trades heavy Boolean reasoning ( 2 64 factors) with T -solving = SMT on BV, EUF, AR, modulo-lia [N LA(Z) ] required Wednesday 12 th December, /

13 Motivations and goals Verification of SW systems... i = 0; acc = 0.0; while (i<dim) { acc += V[i]; i++; } (i 0 = 0) (acc 0 = 0.0) ((i 0 < dim) ( ( (i 0 < dim) ( ((i 1 < dim) ( ( (i 1 < dim) (... (acc 1 = acc 0 + read(v, i 0 )) (i 1 = i 0 + 1))) (acc 1 = acc 0 ) (i 1 = i 0 ))) (acc 2 = acc 1 + read(v, i 1 )) (i 2 = i 1 + 1))) (acc 2 = acc 1 ) (i 2 = i 1 ))) Verification of SW code BMC, K-induction, Check of proof obligations, interpolation-based model checking, symbolic simulation, concolic testing,... = SMT on BV, EUF, AR, (modulo-)lia [N LA(Z) ] required Wednesday 12 th December, /

14 Motivations and goals Planning with Resources [82] SAT-bases planning augmented with numerical constraints Straightforward to encode into into SMT(LRA) Example (sketch) [82] (Deliver) // goal (MaxLoad) // load constraint (MaxFuel) // fuel constraint (Move MinFuel) // move requires fuel (Move Deliver) // move implies delivery (GoodTrip Deliver) // a good trip requires (GoodTrip AllLoaded) // a full delivery (MaxLoad (load 30)) // load limit (MaxFuel (fuel 15)) // fuel limit (MinFuel (fuel load)) // fuel constraint (AllLoaded (load = 45)) // Wednesday 12 th December, /

15 Motivations and goals (Disjunctive) Temporal Reasoning [79, 2] Temporal reasoning problems encoded as disjunctions of difference constraints ((x 1 x 2 6) (x 3 x 4 2)) ((x 2 x 3 2) (x 4 x 5 5)) ((x 2 x 1 4) (x 3 x 7 6))... Straightforward to encode into into SMT(DL) Wednesday 12 th December, /

16 Motivations and goals SMT and SMT solvers Common fact about SMT problems from various applications SMT requires capabilities for heavy Boolean reasoning combined with capabilities for reasoning in expressive decidable F.O. theories SAT alone not expressive enough standard automated theorem proving inadequate (e.g., arithmetic) may involve also numerical computation (e.g., simplex) Modern SMT solvers combine SAT solvers with decision procedures (theory solvers) contributions from SAT, Automated Theorem Proving (ATP), formal verification (FV) and operational research (OR) Wednesday 12 th December, /

17 Motivations and goals SMT and SMT solvers Common fact about SMT problems from various applications SMT requires capabilities for heavy Boolean reasoning combined with capabilities for reasoning in expressive decidable F.O. theories SAT alone not expressive enough standard automated theorem proving inadequate (e.g., arithmetic) may involve also numerical computation (e.g., simplex) Modern SMT solvers combine SAT solvers with decision procedures (theory solvers) contributions from SAT, Automated Theorem Proving (ATP), formal verification (FV) and operational research (OR) Wednesday 12 th December, /

18 Motivations and goals SMT and SMT solvers Common fact about SMT problems from various applications SMT requires capabilities for heavy Boolean reasoning combined with capabilities for reasoning in expressive decidable F.O. theories SAT alone not expressive enough standard automated theorem proving inadequate (e.g., arithmetic) may involve also numerical computation (e.g., simplex) Modern SMT solvers combine SAT solvers with decision procedures (theory solvers) contributions from SAT, Automated Theorem Proving (ATP), formal verification (FV) and operational research (OR) Wednesday 12 th December, /

19 Motivations and goals SMT and SMT solvers Common fact about SMT problems from various applications SMT requires capabilities for heavy Boolean reasoning combined with capabilities for reasoning in expressive decidable F.O. theories SAT alone not expressive enough standard automated theorem proving inadequate (e.g., arithmetic) may involve also numerical computation (e.g., simplex) Modern SMT solvers combine SAT solvers with decision procedures (theory solvers) contributions from SAT, Automated Theorem Proving (ATP), formal verification (FV) and operational research (OR) Wednesday 12 th December, /

20 Motivations and goals SMT and SMT solvers Common fact about SMT problems from various applications SMT requires capabilities for heavy Boolean reasoning combined with capabilities for reasoning in expressive decidable F.O. theories SAT alone not expressive enough standard automated theorem proving inadequate (e.g., arithmetic) may involve also numerical computation (e.g., simplex) Modern SMT solvers combine SAT solvers with decision procedures (theory solvers) contributions from SAT, Automated Theorem Proving (ATP), formal verification (FV) and operational research (OR) Wednesday 12 th December, /

21 Motivations and goals Goal Provide an overview of standard lazy SMT: foundations SMT-solving techniques beyond solving: advanced SMT functionalities ongoing research We do not cover related approaches like: Eager SAT encodings Rewrite-based approaches We refer to [71, 10] for an overview and references. Wednesday 12 th December, /

22 Motivations and goals Goal Provide an overview of standard lazy SMT: foundations SMT-solving techniques beyond solving: advanced SMT functionalities ongoing research We do not cover related approaches like: Eager SAT encodings Rewrite-based approaches We refer to [71, 10] for an overview and references. Wednesday 12 th December, /

23 Motivations and goals Notational remark (1): most/all examples in LRA For better readability, in most/all the examples of this presentation we will use the theory of linear arithmetic on rational numbers (LRA) because of its intuitive semantics. E.g.: ( A 1 (3x 1 2x 2 3 5)) (A 2 ( 2x 1 + 4x = 3)) Nevertheless, analogous examples can be built with all other theories of interest. Wednesday 12 th December, /

24 Motivations and goals Notational remark (2): constants vs. variables Consider, e.g., the formula: ( A 1 (3x 1 2x 2 3 5)) (A 2 ( 2x 1 + 4x = 3)) How do we call A 1, A 2?: (a) Boolean/propositional variables? (b) uninterpreted 0-ary predicates? How do we call x 1, x 2, x 3?: (a) domain variables? (b) uninterpreted Skolem constants/0-ary uninterpreted functions? Hint: (a) typically used in SAT, CSP and OR communities (b) typically used in logic & ATP communities Hereafter we call A 1, A 2 Boolean/propositional variables and x 1, x 2, x 3 "domain variables (logic purists, please forgive me!) Wednesday 12 th December, /

25 Motivations and goals Notational remark (2): constants vs. variables Consider, e.g., the formula: ( A 1 (3x 1 2x 2 3 5)) (A 2 ( 2x 1 + 4x = 3)) How do we call A 1, A 2?: (a) Boolean/propositional variables? (b) uninterpreted 0-ary predicates? How do we call x 1, x 2, x 3?: (a) domain variables? (b) uninterpreted Skolem constants/0-ary uninterpreted functions? Hint: (a) typically used in SAT, CSP and OR communities (b) typically used in logic & ATP communities Hereafter we call A 1, A 2 Boolean/propositional variables and x 1, x 2, x 3 "domain variables (logic purists, please forgive me!) Wednesday 12 th December, /

26 Motivations and goals Notational remark (2): constants vs. variables Consider, e.g., the formula: ( A 1 (3x 1 2x 2 3 5)) (A 2 ( 2x 1 + 4x = 3)) How do we call A 1, A 2?: (a) Boolean/propositional variables? (b) uninterpreted 0-ary predicates? How do we call x 1, x 2, x 3?: (a) domain variables? (b) uninterpreted Skolem constants/0-ary uninterpreted functions? Hint: (a) typically used in SAT, CSP and OR communities (b) typically used in logic & ATP communities Hereafter we call A 1, A 2 Boolean/propositional variables and x 1, x 2, x 3 "domain variables (logic purists, please forgive me!) Wednesday 12 th December, /

27 Outline Efficient SMT solving 1 Motivations and goals 2 Efficient SMT solving Combining SAT with Theory Solvers Theory Solvers for theories of interest SMT for combinations of theories 3 Beyond Solving: advanced SMT functionalities Proofs and unsatisfiable cores Interpolants All-SMT & Predicate Abstraction SMT with cost optimization (Optimization Modulo Theories) 4 Conclusions & current research directions Wednesday 12 th December, /

28 Outline Efficient SMT solving Combining SAT with Theory Solvers 1 Motivations and goals 2 Efficient SMT solving Combining SAT with Theory Solvers Theory Solvers for theories of interest SMT for combinations of theories 3 Beyond Solving: advanced SMT functionalities Proofs and unsatisfiable cores Interpolants All-SMT & Predicate Abstraction SMT with cost optimization (Optimization Modulo Theories) 4 Conclusions & current research directions Wednesday 12 th December, /

29 Combining SAT with Theory Solvers Modern lazy SMT(T ) solvers A prominent lazy approach [45, 2, 82, 3, 8, 36] (aka DPLL(T ) ) a CDCL SAT solver is used to enumerate truth assignments µ i for (the Boolean abstraction of) the input formula ϕ a theory-specific solver T -solver checks the T -consistency of the set of T -literals corresponding to each assignment Many techniques to maximize the benefits of integration [71, 10] Many lazy SMT tools available ( Barcelogic, CVC4, MathSAT, OpenSMT, Yices, Z3,... ) Wednesday 12 th December, /

30 Basic schema: example Combining SAT with Theory Solvers ϕ = ϕ p = c 1 : (2v 2 v 3 > 2) A 1 B 1 A 1 c 2 : A 2 (v 1 v 5 1) A 2 B 2 c 3 : (3v 1 2v 2 3) A 2 B 3 A 2 c 4 : (2v 3 + v 4 5) (3v 1 v 3 6) A 1 B 4 B 5 A 1 c 5 : A 1 (3v 1 2v 2 3) A 1 B 3 c 6 : (v 2 v 4 6) (v 5 = 5 3v 4 ) A 1 B 6 B 7 A 1 c 7 : A 1 (v 3 = 3v 5 + 4) A 2 A 1 B 8 A 2 true, false µ p = { B 5, B 8, B 6, B 1, B 3, A 1, A 2, B 2 } µ = { (3v 1 v 3 6), (v 3 = 3v 5 + 4), (v 2 v 4 6), (2v 2 v 3 > 2), (3v 1 2v 2 3), (v 1 v 5 1)} = inconsistent in LRA = backtrack Wednesday 12 th December, /

31 Basic schema: example Combining SAT with Theory Solvers ϕ = ϕ p = c 1 : (2v 2 v 3 > 2) A 1 B 1 A 1 c 2 : A 2 (v 1 v 5 1) A 2 B 2 c 3 : (3v 1 2v 2 3) A 2 B 3 A 2 c 4 : (2v 3 + v 4 5) (3v 1 v 3 6) A 1 B 4 B 5 A 1 c 5 : A 1 (3v 1 2v 2 3) A 1 B 3 c 6 : (v 2 v 4 6) (v 5 = 5 3v 4 ) A 1 B 6 B 7 A 1 c 7 : A 1 (v 3 = 3v 5 + 4) A 2 A 1 B 8 A 2 true, false µ p = { B 5, B 8, B 6, B 1, B 3, A 1, A 2, B 2 } µ = { (3v 1 v 3 6), (v 3 = 3v 5 + 4), (v 2 v 4 6), (2v 2 v 3 > 2), (3v 1 2v 2 3), (v 1 v 5 1)} = inconsistent in LRA = backtrack Wednesday 12 th December, /

32 Basic schema: example Combining SAT with Theory Solvers ϕ = ϕ p = c 1 : (2v 2 v 3 > 2) A 1 B 1 A 1 c 2 : A 2 (v 1 v 5 1) A 2 B 2 c 3 : (3v 1 2v 2 3) A 2 B 3 A 2 c 4 : (2v 3 + v 4 5) (3v 1 v 3 6) A 1 B 4 B 5 A 1 B 1 c 5 : A 1 (3v 1 2v 2 3) A 1 B 3 B 3 T c 6 : (v 2 v 4 6) (v 5 = 5 3v 4 ) A 1 B 6 B 7 A 1 c 7 : A 1 (v 3 = 3v 5 + 4) A 2 A 1 B 8 A 2 A 1 A 2 true, false B 2 B 6 B 8 B 5 µ p = { B 5, B 8, B 6, B 1, B 3, A 1, A 2, B 2 } µ = { (3v 1 v 3 6), (v 3 = 3v 5 + 4), (v 2 v 4 6), (2v 2 v 3 > 2), (3v 1 2v 2 3), (v 1 v 5 1)} = inconsistent in LRA = backtrack Wednesday 12 th December, /

33 Basic schema: example Combining SAT with Theory Solvers ϕ = ϕ p = c 1 : (2v 2 v 3 > 2) A 1 B 1 A 1 c 2 : A 2 (v 1 v 5 1) A 2 B 2 c 3 : (3v 1 2v 2 3) A 2 B 3 A 2 c 4 : (2v 3 + v 4 5) (3v 1 v 3 6) A 1 B 4 B 5 A 1 B 1 c 5 : A 1 (3v 1 2v 2 3) A 1 B 3 B 3 T c 6 : (v 2 v 4 6) (v 5 = 5 3v 4 ) A 1 B 6 B 7 A 1 c 7 : A 1 (v 3 = 3v 5 + 4) A 2 A 1 B 8 A 2 A 1 A 2 true, false B 2 B 6 B 8 B 5 µ p = { B 5, B 8, B 6, B 1, B 3, A 1, A 2, B 2 } µ = { (3v 1 v 3 6), (v 3 = 3v 5 + 4), (v 2 v 4 6), (2v 2 v 3 > 2), (3v 1 2v 2 3), (v 1 v 5 1)} = inconsistent in LRA = backtrack Wednesday 12 th December, /

34 Combining SAT with Theory Solvers T -Backjumping & T -learning [50, 82, 3, 8, 36] l 1 Similar to Boolean backjumping & learning important property of T -solver: extraction of T -conflict sets: if µ is T -unsatisfiable, then T -solver (µ) returns the subset η of µ causing the T -inconsistency of µ (T -conflict set) If so, the T -conflict clause C := η is used to drive the backjumping & learning mechanism of the SAT solver = lots of search saved the less redundant is η, the more search is saved l 5 l 4 l 3 l 2 l 1 l 2 l 3 l 4 l 5 Wednesday 12 th December, /

35 Combining SAT with Theory Solvers T -Backjumping & T -learning: example ϕ = ϕ p = c 1 : (2v 2 v 3 > 2) A 1 B 1 A 1 c 2 : A 2 (v 1 v 5 1) A 2 B 2 c 3 : (3v 1 2v 2 3) A 2 B 3 A 2 c 4 : (2v 3 + v 4 5) (3v 1 v 3 6) A 1 B 4 B 5 A 1 c 5 : A 1 (3v 1 2v 2 3) A 1 B 3 c 6 : (v 2 v 4 6) (v 5 = 5 3v 4 ) A 1 B 6 B 7 A 1 c 7 : A 1 (v 3 = 3v 5 + 4) A 2 A 1 B 8 A 2 c 8 : (3v 1 v 3 6) (v 3 = 3v 5 + 4)... B 5 B 8 B 2 true, false B 3 A 1 A 2 B 2 B 1 T B 6 B 8 B 5 µ p = { B 5, B 8, B 6, B 1, B 3, A 1, A 2, B 2 } µ = { (3v 1 v 3 6), (v 3 = 3v 5 + 4), (v 2 v 4 6), (2v 2 v 3 > 2), (3v 1 2v 2 3), (v 1 v 5 1)} η = { (3v 1 v 3 6), (v 3 = 3v 5 + 4), (v 1 v 5 1)} η p = { B 5, B 8, B 2 } Wednesday 12 th December, /

36 Combining SAT with Theory Solvers T -Backjumping & T -learning: example ϕ = ϕ p = c 1 : (2v 2 v 3 > 2) A 1 B 1 A 1 c 2 : A 2 (v 1 v 5 1) A 2 B 2 c 3 : (3v 1 2v 2 3) A 2 B 3 A 2 c 4 : (2v 3 + v 4 5) (3v 1 v 3 6) A 1 B 4 B 5 A 1 c 5 : A 1 (3v 1 2v 2 3) A 1 B 3 c 6 : (v 2 v 4 6) (v 5 = 5 3v 4 ) A 1 B 6 B 7 A 1 c 7 : A 1 (v 3 = 3v 5 + 4) A 2 A 1 B 8 A 2 c 8 : (3v 1 v 3 6) (v 3 = 3v 5 + 4)... B 5 B 8 B 2 true, false B 3 A 1 A 2 B 2 B 1 T B 6 B 8 B 5 c 8 : B 5 B 8 B 2 µ p = { B 5, B 8, B 6, B 1, B 3, A 1, A 2, B 2 } µ = { (3v 1 v 3 6), (v 3 = 3v 5 + 4), (v 2 v 4 6), (2v 2 v 3 > 2), (3v 1 2v 2 3), (v 1 v 5 1)} η = { (3v 1 v 3 6), (v 3 = 3v 5 + 4), (v 1 v 5 1)} η p = { B 5, B 8, B 2 } Wednesday 12 th December, /

37 Combining SAT with Theory Solvers T -Backjumping & T -learning: example ϕ = ϕ p = c 1 : (2v 2 v 3 > 2) A 1 B 1 A 1 c 2 : A 2 (v 1 v 5 1) A 2 B 2 c 3 : (3v 1 2v 2 3) A 2 B 3 A 2 c 4 : (2v 3 + v 4 5) (3v 1 v 3 6) A 1 B 4 B 5 A 1 c 5 : A 1 (3v 1 2v 2 3) A 1 B 3 c 6 : (v 2 v 4 6) (v 5 = 5 3v 4 ) A 1 B 6 B 7 A 1 c 7 : A 1 (v 3 = 3v 5 + 4) A 2 A 1 B 8 A 2 c 8 : (3v 1 v 3 6) (v 3 = 3v 5 + 4)... B 5 B 8 B 2 true, false B 3 A 1 A 2 B 2 B 1 T B 6 B 8 B 5 B 2 A 2 B 3 c 8 : B 5 B 8 B 2 µ p = { B 5, B 8, B 6, B 1, B 3, A 1, A 2, B 2 } µ = { (3v 1 v 3 6), (v 3 = 3v 5 + 4), (v 2 v 4 6), (2v 2 v 3 > 2), (3v 1 2v 2 3), (v 1 v 5 1)} η = { (3v 1 v 3 6), (v 3 = 3v 5 + 4), (v 1 v 5 1)} η p = { B 5, B 8, B 2 } Wednesday 12 th December, /

38 Combining SAT with Theory Solvers T -Backjumping & T -learning: example (2) ϕ = ϕ p = c 1 : (2v 2 v 3 > 2) A 1 B 1 A 1 c 2 : A 2 (v 1 v 5 1) A 2 B 2 c 3 : (3v 1 2v 2 3) A 2 B 3 A 2 c 4 : (2v 3 + v 4 5) (3v 1 v 3 6) A 1 B 4 B 5 A 1 c 5 : A 1 (3v 1 2v 2 3) A 1 B 3 c 6 : (v 2 v 4 6) (v 5 = 5 3v 4 ) A 1 B 6 B 7 A 1 c 7 : A 1 (v 3 = 3v 5 + 4) A 2 A 1 B 8 A 2 c 8 : (3v 1 v 3 6) (v 3 = 3v 5 + 4)... B 5 B 8 B 1 c 8 : (3v 1 v 3 6) (v 3 = 3v 5 + 4)... true, false B 5 B 8 B 2 c 8 : theory conflicting clause c 2 {}}{{}}{ c B 5 B 8 B 2 A 2 B 3 2 (B 2 ) {}}{ B 5 B 8 A 2 B 3 A 2 ( A 2 ) B 5 B 8 B 3 B 5 B 8 B }{{} 1 c 8 : mixed Boolean+theory conflict clause B 3 A 1 A 2 B 2 B 1 T B 6 B 8 B 5 c 8 : B 5 B 8 B 2 c T {}}{ B 5 B 1 B 3 (B 3 ) Wednesday 12 th December, /

39 Combining SAT with Theory Solvers T -Backjumping & T -learning: example (2) ϕ = ϕ p = c 1 : (2v 2 v 3 > 2) A 1 B 1 A 1 c 2 : A 2 (v 1 v 5 1) A 2 B 2 c 3 : (3v 1 2v 2 3) A 2 B 3 A 2 c 4 : (2v 3 + v 4 5) (3v 1 v 3 6) A 1 B 4 B 5 A 1 c 5 : A 1 (3v 1 2v 2 3) A 1 B 3 c 6 : (v 2 v 4 6) (v 5 = 5 3v 4 ) A 1 B 6 B 7 A 1 c 7 : A 1 (v 3 = 3v 5 + 4) A 2 A 1 B 8 A 2 c 8 : (3v 1 v 3 6) (v 3 = 3v 5 + 4)... B 5 B 8 B 1 c 8 : (3v 1 v 3 6) (v 3 = 3v 5 + 4)... true, false B 5 B 8 B 2 B 3 A 1 A 2 B 2 B 1 T B 6 B 8 B 5 B 1 A 1 c 8 : B 5 B 8 B 1 c 8 : theory conflicting clause c 2 {}}{{}}{ c B 5 B 8 B 2 A 2 B 3 2 (B 2 ) {}}{ B 5 B 8 A 2 B 3 A 2 ( A 2 ) B 5 B 8 B 3 B 5 B 8 B }{{} 1 c 8 : mixed Boolean+theory conflict clause c T {}}{ B 5 B 1 B 3 (B 3 ) Wednesday 12 th December, /

40 Combining SAT with Theory Solvers T -Backjumping & T -learning: example (2) ϕ = ϕ p = c 1 : (2v 2 v 3 > 2) A 1 c 2 : A 2 (v 1 v 5 1) c 3 : (3v 1 2v 2 3) A 2 c 4 : (2v 3 + v 4 5) (3v 1 v 3 6) A 1 c 5 : A 1 (3v 1 2v 2 3) c 6 : (v 2 v 4 6) (v 5 = 5 3v 4 ) A 1 c 7 : A 1 (v 3 = 3v 5 + 4) A 2 c 8 : (3v 1 v 3 6) (v 3 = 3v 5 + 4)... c 8 : (3v 1 v 3 6) (v 3 = 3v 5 + 4)... true, false B 1 A 1 A 2 B 2 B 3 A 2 B 4 B 5 A 1 A 1 B 3 B 6 B 7 A 1 A 1 B 8 A 2 B 5 B 8 B 1 B 5 B 8 B 2 c 8 : theory conflicting clause c 2 {}}{{}}{ c B 5 B 8 B 2 A 2 B 3 2 (B 2 ) {}}{ B 5 B 8 A 2 B 3 A 2 ( A 2 ) B 5 B 8 B 3 B 5 B 8 B }{{} 1 c 8 : mixed Boolean+theory conflict clause B 3 A 1 A 2 B 2 T B 8 B 5 B 6 B 1 B 1 A 1 B 2 A 2 B 3 c 8 : B 5 B 8 B 1 c 8 : B 5 B 8 B 2 c T {}}{ B 5 B 1 B 3 (B 3 ) Wednesday 12 th December, /

41 Early Pruning [45, 2, 82] I Combining SAT with Theory Solvers Introduce a T -satisfiability test on intermediate assignments: if T -solver returns UNSAT, the procedure backtracks. benefit: prunes drastically the Boolean search Drawback: possibly many useless calls to T -solver Calls to T Solve() SAT WITHOUT EARLY PRUNING WITH EARLY PRUNING SAT SAT SAT UNSAT SAT SAT SAT UNSAT SAT SAT UNSAT SAT UNSAT SAT UNSAT SAT SAT UNSAT UNSAT SAT UNSAT UNSAT UNSAT UNSAT UNSAT UNSAT UNSAT UNSAT UNSAT Wednesday 12 th December, /

42 Early Pruning [45, 2, 82] II Combining SAT with Theory Solvers Different strategies for interleaving Boolean search steps and T -solver calls Eager E.P. [82, 11, 80, 44]): invoke T -solver every time a new T -atom is added to the assignment (unit propagations included) Selective E.P.: Do not call T -solver if the have been added only literals which hardly cause any T -conflict with the previous assignment (e.g., Boolean literals, disequalities (x y 3), T -literals introducing new variables (x z = 3) ) Weakened E.P.: for intermediate checks only, use weaker but faster versions of T -solver (e.g., check µ on R rather than on Z): {(x y 4), (z x 6), (z = y), (3x + 2y 3z = 4)} Wednesday 12 th December, /

43 Early pruning: example Combining SAT with Theory Solvers ϕ = { (2v 2 v 3 > 2) A 1 } { A 2 (2v 1 4v 5 > 3)} {(3v 1 2v 2 3) A 2 } { (2v 3 + v 4 5) (3v 1 v 3 6) A 1 } {A 1 (3v 1 2v 2 3)} {(v 1 v 5 1) (v 5 = 5 3v 4 ) A 1 } {A 1 (v 3 = 3v 5 + 4) A 2 }. ϕ p = { B 1 A 1 } { A 2 B 2 } {B 3 A 2 } { B 4 B 5 A 1 } {A 1 B 3 } {B 6 B 7 A 1 } {A 1 B 8 A 2 }. Suppose it is built the intermediate assignment: µ p = B 1 A 2 B 3 B 5. corresponding to the following set of T -literals µ = (2v 2 v 3 > 2) A 2 (3v 1 2v 2 3) (3v 1 v 3 6). If T -solver is invoked on µ, then it returns UNSAT, and DPLL backtracks without exploring any extension of µ. Wednesday 12 th December, /

44 Combining SAT with Theory Solvers Early pruning: remark Incrementality & Backtrackability of T -solvers With early pruning, lots of incremental calls to T -solver: T -solver (µ 1 ) Sat Undo µ 4, µ 3, µ 2 T -solver (µ 1 µ 2 ) Sat T -solver (µ 1 µ 2 ) Sat T -solver (µ 1 µ 2 µ 3 ) Sat T -solver (µ 1 µ 2 µ 3 ) Sat T -solver (µ 1 µ 2 µ 3 µ 4 ) Unsat... = Desirable features of T -solvers: incrementality: T -solver(µ 1 µ 2 ) reuses computation of T -solver(µ 1 ) without restarting from scratch backtrackability (resettability): T -solver can efficiently undo steps and return to a previous status on the stack = T -solver requires a stack-based interface Wednesday 12 th December, /

45 Combining SAT with Theory Solvers Early pruning: remark Incrementality & Backtrackability of T -solvers With early pruning, lots of incremental calls to T -solver: T -solver (µ 1 ) Sat Undo µ 4, µ 3, µ 2 T -solver (µ 1 µ 2 ) Sat T -solver (µ 1 µ 2 ) Sat T -solver (µ 1 µ 2 µ 3 ) Sat T -solver (µ 1 µ 2 µ 3 ) Sat T -solver (µ 1 µ 2 µ 3 µ 4 ) Unsat... = Desirable features of T -solvers: incrementality: T -solver(µ 1 µ 2 ) reuses computation of T -solver(µ 1 ) without restarting from scratch backtrackability (resettability): T -solver can efficiently undo steps and return to a previous status on the stack = T -solver requires a stack-based interface Wednesday 12 th December, /

46 Combining SAT with Theory Solvers Early pruning: remark Incrementality & Backtrackability of T -solvers With early pruning, lots of incremental calls to T -solver: T -solver (µ 1 ) Sat Undo µ 4, µ 3, µ 2 T -solver (µ 1 µ 2 ) Sat T -solver (µ 1 µ 2 ) Sat T -solver (µ 1 µ 2 µ 3 ) Sat T -solver (µ 1 µ 2 µ 3 ) Sat T -solver (µ 1 µ 2 µ 3 µ 4 ) Unsat... = Desirable features of T -solvers: incrementality: T -solver(µ 1 µ 2 ) reuses computation of T -solver(µ 1 ) without restarting from scratch backtrackability (resettability): T -solver can efficiently undo steps and return to a previous status on the stack = T -solver requires a stack-based interface Wednesday 12 th December, /

47 Combining SAT with Theory Solvers T -Propagation [2, 3, 44] strictly related to early pruning important property of T -solver: T -deduction: when a partial assignment µ is T -satisfiable, T -solver may be able to return also an assignment η to some unassigned atom occurring in ϕ s.t. µ = T η. If so: the literal η is then unit-propagated; optionally, a T -deduction clause C := µ η can be learned, µ being the subset of µ which caused the deduction (µ = T η) lazy explanation: compute C only if needed for conflict analysis = may prune drastically the search Both T -deduction clauses and T -conflict clauses are called T -lemmas since they are valid in T Wednesday 12 th December, /

48 Combining SAT with Theory Solvers T -Propagation [2, 3, 44] strictly related to early pruning important property of T -solver: T -deduction: when a partial assignment µ is T -satisfiable, T -solver may be able to return also an assignment η to some unassigned atom occurring in ϕ s.t. µ = T η. If so: the literal η is then unit-propagated; optionally, a T -deduction clause C := µ η can be learned, µ being the subset of µ which caused the deduction (µ = T η) lazy explanation: compute C only if needed for conflict analysis = may prune drastically the search Both T -deduction clauses and T -conflict clauses are called T -lemmas since they are valid in T Wednesday 12 th December, /

49 T -propagation: example Combining SAT with Theory Solvers ϕ = ϕ p = c 1 : (2v 2 v 3 > 2) A 1 B 1 A 1 c 2 : A 2 (v 1 v 5 1) A 2 B 2 c 3 : (3v 1 2v 2 3) A 2 B 3 A 2 c 4 : (2v 3 + v 4 5) (3v 1 v 3 6) A 1 B 4 B 5 A 1 c 5 : A 1 (3v 1 2v 2 3) A 1 B 3 c 6 : (v 2 v 4 6) (v 5 = 5 3v 4 ) A 1 B 6 B 7 A 1 c 7 : A 1 (v 3 = 3v 5 + 4) A 2 A 1 B 8 A 2 true, false B 1 B 6 B 8 B 5 µ p = { B 5, B 8, B 6, B 1 } µ = { (3v 1 v 3 6), (v 3 = 3v 5 + 4), (v 2 v 4 6), (2v 2 v 3 > 2)} = LRA (3v 1 2v 2 3) }{{} B 3 = propagate B 3 [and learn the deduction clause B 5 B 1 B 3 ] Wednesday 12 th December, /

50 T -propagation: example Combining SAT with Theory Solvers ϕ = ϕ p = c 1 : (2v 2 v 3 > 2) A 1 B 1 A 1 c 2 : A 2 (v 1 v 5 1) A 2 B 2 c 3 : (3v 1 2v 2 3) A 2 B 3 A 2 c 4 : (2v 3 + v 4 5) (3v 1 v 3 6) A 1 B 4 B 5 A 1 c 5 : A 1 (3v 1 2v 2 3) A 1 B 3 c 6 : (v 2 v 4 6) (v 5 = 5 3v 4 ) A 1 B 6 B 7 A 1 c 7 : A 1 (v 3 = 3v 5 + 4) A 2 A 1 B 8 A 2 true, false B 1 B 6 B 8 B 5 µ p = { B 5, B 8, B 6, B 1 } µ = { (3v 1 v 3 6), (v 3 = 3v 5 + 4), (v 2 v 4 6), (2v 2 v 3 > 2)} = LRA (3v 1 2v 2 3) }{{} B 3 = propagate B 3 [and learn the deduction clause B 5 B 1 B 3 ] Wednesday 12 th December, /

51 T -propagation: example Combining SAT with Theory Solvers ϕ = ϕ p = B 5 c 1 : (2v 2 v 3 > 2) A 1 B 1 A 1 B c 2 : A 2 (v 1 v 5 1) A 2 B 8 2 c 3 : (3v 1 2v 2 3) A 2 B 3 A B 6 2 c 4 : (2v 3 + v 4 5) (3v 1 v 3 6) A 1 B 4 B 5 A 1 B 1 c 5 : A 1 (3v 1 2v 2 3) A 1 B 3 B 3 T -propagate c 6 : (v 2 v 4 6) (v 5 = 5 3v 4 ) A 1 B 6 B 7 A 1 c 7 : A 1 (v 3 = 3v 5 + 4) A 2 A 1 B 8 A 2 true, false µ p = { B 5, B 8, B 6, B 1 } µ = { (3v 1 v 3 6), (v 3 = 3v 5 + 4), (v 2 v 4 6), (2v 2 v 3 > 2)} = LRA (3v 1 2v 2 3) }{{} B 3 = propagate B 3 [and learn the deduction clause B 5 B 1 B 3 ] Wednesday 12 th December, /

52 Combining SAT with Theory Solvers Pure-literal filtering [82, 3, 17] Property If we have non-boolean T -atoms occurring only positively [negatively] in the original formula ϕ (learned clauses are not considered), we can drop every negative [positive] occurrence of them from the assignment to be checked by T -solver (and from the T -deducible ones). increases the chances of finding a model reduces the effort for the T -solver eliminates unnecessary nasty negated literals (e.g. negative equalities like (3v 1 9v 2 = 3) in LIA force splitting: (3v 1 9v 2 > 3) (3v 1 9v 2 < 3)). may weaken the effect of early pruning. Wednesday 12 th December, /

53 Pure literal filtering: example Combining SAT with Theory Solvers ϕ = { (2v 2 v 3 > 2) A 1 } { A 2 (2v 1 4v 5 > 3)} {(3v 1 2v 2 3) A 2 } { (2v 3 + v 4 5) (3v 1 v 3 6) A 1 } {A 1 (3v 1 2v 2 3)} {(v 1 v 5 1) (v 5 = 5 3v 4 ) A 1 } {A 1 (v 3 = 3v 5 + 4) A 2 } {(2v 2 v 3 > 2) (3v 1 2v 2 3) (3v 1 v 3 6)} learned µ = { (2v 2 v 3 > 2), A 2, (3v 1 2v 2 3), A 1, (v 3 = 3v 5 + 4), (3v 1 v 3 6)}. = Sat: v 1 = v 2 = v 3 = 0, v 5 = 4/3 is a solution N.B. (3v 1 v 3 6) filtered out from µ because it occurs only negatively in the original formula ϕ Wednesday 12 th December, /

54 Combining SAT with Theory Solvers Preprocessing atoms [45, 50, 4] Source of inefficiency: semantically equivalent but syntactically different atoms are not recognized to be identical [resp. one the negation of the other] = they may be assigned different [resp. identical] truth values. = lots of redundant unsatisfiable assignment generated Solution Rewrite a priori trivially-equivalent atoms/literals into the same atom/literal. Wednesday 12 th December, /

55 Combining SAT with Theory Solvers Preprocessing atoms [45, 50, 4] Source of inefficiency: semantically equivalent but syntactically different atoms are not recognized to be identical [resp. one the negation of the other] = they may be assigned different [resp. identical] truth values. = lots of redundant unsatisfiable assignment generated Solution Rewrite a priori trivially-equivalent atoms/literals into the same atom/literal. Wednesday 12 th December, /

56 Combining SAT with Theory Solvers Preprocessing atoms [45, 50, 4] Source of inefficiency: semantically equivalent but syntactically different atoms are not recognized to be identical [resp. one the negation of the other] = they may be assigned different [resp. identical] truth values. = lots of redundant unsatisfiable assignment generated Solution Rewrite a priori trivially-equivalent atoms/literals into the same atom/literal. Wednesday 12 th December, /

57 Combining SAT with Theory Solvers Preprocessing atoms [45, 50, 4] Source of inefficiency: semantically equivalent but syntactically different atoms are not recognized to be identical [resp. one the negation of the other] = they may be assigned different [resp. identical] truth values. = lots of redundant unsatisfiable assignment generated Solution Rewrite a priori trivially-equivalent atoms/literals into the same atom/literal. Wednesday 12 th December, /

58 Combining SAT with Theory Solvers Preprocessing atoms (cont.) Sorting: (v 1 + v 2 v 3 + 1), (v 2 + v 1 v 3 + 1), (v 1 + v 2 1 v 3 ) = (v 1 + v 2 v 3 1)); Rewriting dual operators: (v 1 < v 2 ), (v 1 v 2 ) = (v 1 < v 2 ), (v 1 < v 2 ) Exploiting associativity: (v 1 + (v 2 + v 3 ) = 1), ((v 1 + v 2 ) + v 3 ) = 1) = (v 1 + v 2 + v 3 = 1); Factoring (v v 2 4.0), ( 2.0v 1 4.0v 2 8.0), = (0.25v v 2 1.0); Exploiting properties of T : (v 1 3), (v 1 < 4) = (v 1 3) if v 1 Z;... Wednesday 12 th December, /

59 Combining SAT with Theory Solvers Preprocessing atoms (cont.) Sorting: (v 1 + v 2 v 3 + 1), (v 2 + v 1 v 3 + 1), (v 1 + v 2 1 v 3 ) = (v 1 + v 2 v 3 1)); Rewriting dual operators: (v 1 < v 2 ), (v 1 v 2 ) = (v 1 < v 2 ), (v 1 < v 2 ) Exploiting associativity: (v 1 + (v 2 + v 3 ) = 1), ((v 1 + v 2 ) + v 3 ) = 1) = (v 1 + v 2 + v 3 = 1); Factoring (v v 2 4.0), ( 2.0v 1 4.0v 2 8.0), = (0.25v v 2 1.0); Exploiting properties of T : (v 1 3), (v 1 < 4) = (v 1 3) if v 1 Z;... Wednesday 12 th December, /

60 Combining SAT with Theory Solvers Preprocessing atoms (cont.) Sorting: (v 1 + v 2 v 3 + 1), (v 2 + v 1 v 3 + 1), (v 1 + v 2 1 v 3 ) = (v 1 + v 2 v 3 1)); Rewriting dual operators: (v 1 < v 2 ), (v 1 v 2 ) = (v 1 < v 2 ), (v 1 < v 2 ) Exploiting associativity: (v 1 + (v 2 + v 3 ) = 1), ((v 1 + v 2 ) + v 3 ) = 1) = (v 1 + v 2 + v 3 = 1); Factoring (v v 2 4.0), ( 2.0v 1 4.0v 2 8.0), = (0.25v v 2 1.0); Exploiting properties of T : (v 1 3), (v 1 < 4) = (v 1 3) if v 1 Z;... Wednesday 12 th December, /

61 Combining SAT with Theory Solvers Preprocessing atoms (cont.) Sorting: (v 1 + v 2 v 3 + 1), (v 2 + v 1 v 3 + 1), (v 1 + v 2 1 v 3 ) = (v 1 + v 2 v 3 1)); Rewriting dual operators: (v 1 < v 2 ), (v 1 v 2 ) = (v 1 < v 2 ), (v 1 < v 2 ) Exploiting associativity: (v 1 + (v 2 + v 3 ) = 1), ((v 1 + v 2 ) + v 3 ) = 1) = (v 1 + v 2 + v 3 = 1); Factoring (v v 2 4.0), ( 2.0v 1 4.0v 2 8.0), = (0.25v v 2 1.0); Exploiting properties of T : (v 1 3), (v 1 < 4) = (v 1 3) if v 1 Z;... Wednesday 12 th December, /

62 Combining SAT with Theory Solvers Preprocessing atoms (cont.) Sorting: (v 1 + v 2 v 3 + 1), (v 2 + v 1 v 3 + 1), (v 1 + v 2 1 v 3 ) = (v 1 + v 2 v 3 1)); Rewriting dual operators: (v 1 < v 2 ), (v 1 v 2 ) = (v 1 < v 2 ), (v 1 < v 2 ) Exploiting associativity: (v 1 + (v 2 + v 3 ) = 1), ((v 1 + v 2 ) + v 3 ) = 1) = (v 1 + v 2 + v 3 = 1); Factoring (v v 2 4.0), ( 2.0v 1 4.0v 2 8.0), = (0.25v v 2 1.0); Exploiting properties of T : (v 1 3), (v 1 < 4) = (v 1 3) if v 1 Z;... Wednesday 12 th December, /

63 Combining SAT with Theory Solvers Static Learning [2, 4] Often possible to quickly detect a priori short and obviously inconsistent pairs or triplets of literals occurring in ϕ. mutual exclusion {x = 0, x = 1}, congruence {(x 1 = y 1 ), (x 2 = y 2 ), (f (x 1, x 2 ) = f (y 1, y 2 ))}, transitivity {(x y = 2), (y z 4), (x z 7)}, substitution {(x = y), (2x 3z 3), (2y 3z 3)}... Preprocessing step: detect these literals and add blocking clauses to the input formula: (e.g., (x = 0) (x = 1)) = No assignment including one such group of literals is ever generated: as soon as all but one literals are assigned, the remaining one is immediately assigned false by unit-propagation. Wednesday 12 th December, /

64 Combining SAT with Theory Solvers Static Learning [2, 4] Often possible to quickly detect a priori short and obviously inconsistent pairs or triplets of literals occurring in ϕ. mutual exclusion {x = 0, x = 1}, congruence {(x 1 = y 1 ), (x 2 = y 2 ), (f (x 1, x 2 ) = f (y 1, y 2 ))}, transitivity {(x y = 2), (y z 4), (x z 7)}, substitution {(x = y), (2x 3z 3), (2y 3z 3)}... Preprocessing step: detect these literals and add blocking clauses to the input formula: (e.g., (x = 0) (x = 1)) = No assignment including one such group of literals is ever generated: as soon as all but one literals are assigned, the remaining one is immediately assigned false by unit-propagation. Wednesday 12 th December, /

65 Combining SAT with Theory Solvers Static Learning [2, 4] Often possible to quickly detect a priori short and obviously inconsistent pairs or triplets of literals occurring in ϕ. mutual exclusion {x = 0, x = 1}, congruence {(x 1 = y 1 ), (x 2 = y 2 ), (f (x 1, x 2 ) = f (y 1, y 2 ))}, transitivity {(x y = 2), (y z 4), (x z 7)}, substitution {(x = y), (2x 3z 3), (2y 3z 3)}... Preprocessing step: detect these literals and add blocking clauses to the input formula: (e.g., (x = 0) (x = 1)) = No assignment including one such group of literals is ever generated: as soon as all but one literals are assigned, the remaining one is immediately assigned false by unit-propagation. Wednesday 12 th December, /

66 Combining SAT with Theory Solvers Other optimization techniques T -deduced-literal filtering Ghost-literal filtering T -solver layering T -solver clustering... (see [71, 10] for an overview) Wednesday 12 th December, /

67 Combining SAT with Theory Solvers Other SAT-solving techniques for SMT? Frequently-asked question: Are CDCL SAT solvers the only suitable Boolean Engines for SMT? Some previous attempts: Ordered Binary Decision Diagrams (OBDDs) [83, 60, 1] Stochastic Local Search [49] CDCL based currently much more efficient. Wednesday 12 th December, /

68 Combining SAT with Theory Solvers SMT formulas = partially-invisible SAT formulas An SMT problem ϕ from the perspective of a SAT solver: a partially-invisible Boolean CNF formula ϕ p τ p : ϕ p : the Boolean abstraction of the input formula ϕ τ p : (the B. abst. of) the set τ of all T -lemmas on atoms in ϕ. ϕ T -satisfiable iff ϕ p τ p satisfiable. the SAT solver: sees only ϕ p finds µ p s.t. µ p = ϕ p cannot state if µ p = τ p invokes T -solver on µ p the T -solver: sees τ p checks if µ p = τ p : if yes, returns SAT if no, returns UNSAT and some falsified clauses c p 1,..., cp k τ p Wednesday 12 th December, /

69 Combining SAT with Theory Solvers SMT formulas = partially-invisible SAT formulas An SMT problem ϕ from the perspective of a SAT solver: a partially-invisible Boolean CNF formula ϕ p τ p : ϕ p : the Boolean abstraction of the input formula ϕ τ p : (the B. abst. of) the set τ of all T -lemmas on atoms in ϕ. ϕ T -satisfiable iff ϕ p τ p satisfiable. the SAT solver: sees only ϕ p finds µ p s.t. µ p = ϕ p cannot state if µ p = τ p invokes T -solver on µ p the T -solver: sees τ p checks if µ p = τ p : if yes, returns SAT if no, returns UNSAT and some falsified clauses c p 1,..., cp k τ p SAT solver ϕ p c τ p Wednesday 12 th December, /

70 Combining SAT with Theory Solvers SMT formulas = partially-invisible SAT formulas An SMT problem ϕ from the perspective of a SAT solver: a partially-invisible Boolean CNF formula ϕ p τ p : ϕ p : the Boolean abstraction of the input formula ϕ τ p : (the B. abst. of) the set τ of all T -lemmas on atoms in ϕ. ϕ T -satisfiable iff ϕ p τ p satisfiable. the SAT solver: sees only ϕ p finds µ p s.t. µ p = ϕ p cannot state if µ p = τ p invokes T -solver on µ p the T -solver: sees τ p checks if µ p = τ p : if yes, returns SAT if no, returns UNSAT and some falsified clauses c p 1,..., cp k τ p µ p SAT solver ϕ p c τ p Wednesday 12 th December, /

71 Combining SAT with Theory Solvers SMT formulas = partially-invisible SAT formulas An SMT problem ϕ from the perspective of a SAT solver: a partially-invisible Boolean CNF formula ϕ p τ p : ϕ p : the Boolean abstraction of the input formula ϕ τ p : (the B. abst. of) the set τ of all T -lemmas on atoms in ϕ. ϕ T -satisfiable iff ϕ p τ p satisfiable. the SAT solver: sees only ϕ p finds µ p s.t. µ p = ϕ p cannot state if µ p = τ p invokes T -solver on µ p the T -solver: sees τ p checks if µ p = τ p : if yes, returns SAT if no, returns UNSAT and some falsified clauses c p 1,..., cp k τ p µ p = τ p? µ p SAT solver ϕ p c τ p Wednesday 12 th December, /

72 Combining SAT with Theory Solvers SMT formulas = partially-invisible SAT formulas An SMT problem ϕ from the perspective of a SAT solver: a partially-invisible Boolean CNF formula ϕ p τ p : ϕ p : the Boolean abstraction of the input formula ϕ τ p : (the B. abst. of) the set τ of all T -lemmas on atoms in ϕ. ϕ T -satisfiable iff ϕ p τ p satisfiable. the SAT solver: sees only ϕ p finds µ p s.t. µ p = ϕ p cannot state if µ p = τ p invokes T -solver on µ p the T -solver: sees τ p checks if µ p = τ p : if yes, returns SAT if no, returns UNSAT and some falsified clauses c p 1,..., cp k τ p µ p SAT solver ϕ p c T -solver τ p Wednesday 12 th December, /

73 Combining SAT with Theory Solvers SMT formulas = partially-invisible SAT formulas An SMT problem ϕ from the perspective of a SAT solver: a partially-invisible Boolean CNF formula ϕ p τ p : ϕ p : the Boolean abstraction of the input formula ϕ τ p : (the B. abst. of) the set τ of all T -lemmas on atoms in ϕ. ϕ T -satisfiable iff ϕ p τ p satisfiable. the SAT solver: sees only ϕ p finds µ p s.t. µ p = ϕ p cannot state if µ p = τ p invokes T -solver on µ p the T -solver: sees τ p checks if µ p = τ p : if yes, returns SAT if no, returns UNSAT and some falsified clauses c p 1,..., cp k τ p Hey T -solver! µ p = τ p? µ p SAT solver ϕ p c T -solver τ p Wednesday 12 th December, /

74 Combining SAT with Theory Solvers SMT formulas = partially-invisible SAT formulas An SMT problem ϕ from the perspective of a SAT solver: a partially-invisible Boolean CNF formula ϕ p τ p : ϕ p : the Boolean abstraction of the input formula ϕ τ p : (the B. abst. of) the set τ of all T -lemmas on atoms in ϕ. ϕ T -satisfiable iff ϕ p τ p satisfiable. the SAT solver: sees only ϕ p finds µ p s.t. µ p = ϕ p cannot state if µ p = τ p invokes T -solver on µ p the T -solver: sees τ p checks if µ p = τ p : if yes, returns SAT if no, returns UNSAT and some falsified clauses c p 1,..., cp k τ p µ p SAT solver ϕ p c µ p = τ p? T -solver τ p Wednesday 12 th December, /

75 Combining SAT with Theory Solvers SMT formulas = partially-invisible SAT formulas An SMT problem ϕ from the perspective of a SAT solver: a partially-invisible Boolean CNF formula ϕ p τ p : ϕ p : the Boolean abstraction of the input formula ϕ τ p : (the B. abst. of) the set τ of all T -lemmas on atoms in ϕ. ϕ T -satisfiable iff ϕ p τ p satisfiable. the SAT solver: sees only ϕ p finds µ p s.t. µ p = ϕ p cannot state if µ p = τ p invokes T -solver on µ p the T -solver: sees τ p checks if µ p = τ p : if yes, returns SAT if no, returns UNSAT and some falsified clauses c p 1,..., cp k τ p µ p SAT solver Yes, µ p = τ p! ϕ p c T -solver τ p Wednesday 12 th December, /

76 Combining SAT with Theory Solvers SMT formulas = partially-invisible SAT formulas An SMT problem ϕ from the perspective of a SAT solver: a partially-invisible Boolean CNF formula ϕ p τ p : ϕ p : the Boolean abstraction of the input formula ϕ τ p : (the B. abst. of) the set τ of all T -lemmas on atoms in ϕ. ϕ T -satisfiable iff ϕ p τ p satisfiable. the SAT solver: sees only ϕ p finds µ p s.t. µ p = ϕ p cannot state if µ p = τ p invokes T -solver on µ p the T -solver: sees τ p checks if µ p = τ p : if yes, returns SAT if no, returns UNSAT and some falsified clauses c p 1,..., cp k τ p No! c p 1,..., cp k are falsified! µ p SAT solver ϕ p c T -solver τ p Wednesday 12 th December, /

77 Combining SAT with Theory Solvers Example ϕ : c 1 : {A 1 } c 2 : { A 1 (x z > 4)} c 3 : { A 3 A 1 (y 1)} c 4 : { A 2 (x z > 4) A 1 } c 5 : {(x y 3) A 4 A 5} c 6 : { (y z 1) (x + y = 1) A 5} c 7 : {A 3 (x + y = 0) A 2 } c 8 : { A 3 (z + y = 2)} τ : (all possible T -lemmas on the T -atoms of ϕ) c 9 : { (x + y = 0) (x + y = 1)} c 10 : { (x z > 4) (x y 3) (y z 1)} c 11 : {(x z > 4) (x y 3) (y z 1)} c 12 : { (x z > 4) (x + y = 1) (z + y = 2)} c 13 : { (x z > 4) (x + y = 0) (z + y = 2)} ϕ p : c 1 : {A 1 } c 2 : { A 1 B 1 } c 3 : { A 3 A 1 B 2 } c 4 : { A 2 B 1 A 1 } c 5 : {B 3 A 4 A 5} c 6 : { B 4 B 5 A 5} c 7 : {A 3 B 6 A 2 } c 8 : { A 3 B 7} τ p : c 9 : { B 6 B 5} c 10 : { B 1 B 3 B 4 } c 11 : {B 1 B 3 B 4 } c 12 : { B 1 B 5 B 7} c 13 : { B 1 B 6 B 7} µ p 1 : {A 1, B 1, A 2, A 3, A 4, A 5, B 6, B 5, B 3, B 4, B 7, B 2 } µ 1 : {(x z > 4), (x + y = 0), (x + y = 1), (x y 3), (y z 1), (z + y = 2), (y 1 satisfies ϕ p, but violates both c 10 and c 12 in τ p. Wednesday 12 th December, /

78 Combining SAT with Theory Solvers Example ϕ : c 1 : {A 1 } c 2 : { A 1 (x z > 4)} c 3 : { A 3 A 1 (y 1)} c 4 : { A 2 (x z > 4) A 1 } c 5 : {(x y 3) A 4 A 5} c 6 : { (y z 1) (x + y = 1) A 5} c 7 : {A 3 (x + y = 0) A 2 } c 8 : { A 3 (z + y = 2)} τ : (all possible T -lemmas on the T -atoms of ϕ) c 9 : { (x + y = 0) (x + y = 1)} c 10 : { (x z > 4) (x y 3) (y z 1)} c 11 : {(x z > 4) (x y 3) (y z 1)} c 12 : { (x z > 4) (x + y = 1) (z + y = 2)} c 13 : { (x z > 4) (x + y = 0) (z + y = 2)} ϕ p : c 1 : {A 1 } c 2 : { A 1 B 1 } c 3 : { A 3 A 1 B 2 } c 4 : { A 2 B 1 A 1 } c 5 : {B 3 A 4 A 5} c 6 : { B 4 B 5 A 5} c 7 : {A 3 B 6 A 2 } c 8 : { A 3 B 7} τ p : c 9 : { B 6 B 5} c 10 : { B 1 B 3 B 4 } c 11 : {B 1 B 3 B 4 } c 12 : { B 1 B 5 B 7} c 13 : { B 1 B 6 B 7} µ p 1 : {A 1, B 1, A 2, A 3, A 4, A 5, B 6, B 5, B 3, B 4, B 7, B 2 } µ 1 : {(x z > 4), (x + y = 0), (x + y = 1), (x y 3), (y z 1), (z + y = 2), (y 1 satisfies ϕ p, but violates both c 10 and c 12 in τ p. Wednesday 12 th December, /

79 Outline Efficient SMT solving Theory Solvers for theories of interest 1 Motivations and goals 2 Efficient SMT solving Combining SAT with Theory Solvers Theory Solvers for theories of interest SMT for combinations of theories 3 Beyond Solving: advanced SMT functionalities Proofs and unsatisfiable cores Interpolants All-SMT & Predicate Abstraction SMT with cost optimization (Optimization Modulo Theories) 4 Conclusions & current research directions Wednesday 12 th December, /

80 Theory Solvers for theories of interest T -solvers for Equality and Uninterpreted Functions (EUF) Typically used as a core T -solver EU F polynomial: O(n log(n)) Fully incremental and backtrackable (stack-based) use a congruence closure data structures (E-Graphs) [40, 64, 35], based on the Union-Find data-structure for equivalence classes Supports efficient T -propagation Exhaustive for positive equalities Incomplete for disequalities Supports Lazy explanations and conflict generation However, minimality not guaranteed Supports efficient extensions (e.g., Integer offsets, Bit-vector slicing and concatenation) Wednesday 12 th December, /

81 Theory Solvers for theories of interest T -solvers for EUF: Example Idea (sketch): given the set of terms occurring in the formula represented as nodes in a DAG (aka term bank), if (t = s), then merge the eq. classes of t and s if i 1...k, t i and s i pairwise belong to the same eq. classes, then merge the the eq. classes of f (t 1,..., t k ) and f (s 1,..., s k ) if (t s) and t and s belong to the same eq. class, then conflict g f f g f (a, b) = a f (f (a, b), b) = c g(a) g(c) a b c Example borrowed from [40]. Wednesday 12 th December, /

82 Theory Solvers for theories of interest T -solvers for EUF: Example Idea (sketch): given the set of terms occurring in the formula represented as nodes in a DAG (aka term bank), if (t = s), then merge the eq. classes of t and s if i 1...k, t i and s i pairwise belong to the same eq. classes, then merge the the eq. classes of f (t 1,..., t k ) and f (s 1,..., s k ) if (t s) and t and s belong to the same eq. class, then conflict g f f g f (a, b) = a f (f (a, b), b) = c g(a) g(c) a b c Example borrowed from [40]. Wednesday 12 th December, /

83 Theory Solvers for theories of interest T -solvers for EUF: Example Idea (sketch): given the set of terms occurring in the formula represented as nodes in a DAG (aka term bank), if (t = s), then merge the eq. classes of t and s if i 1...k, t i and s i pairwise belong to the same eq. classes, then merge the the eq. classes of f (t 1,..., t k ) and f (s 1,..., s k ) if (t s) and t and s belong to the same eq. class, then conflict g f f g f (a, b) = a f (f (a, b), b) = c g(a) g(c) a b c Example borrowed from [40]. Wednesday 12 th December, /

84 Theory Solvers for theories of interest T -solvers for EUF: Example Idea (sketch): given the set of terms occurring in the formula represented as nodes in a DAG (aka term bank), if (t = s), then merge the eq. classes of t and s if i 1...k, t i and s i pairwise belong to the same eq. classes, then merge the the eq. classes of f (t 1,..., t k ) and f (s 1,..., s k ) if (t s) and t and s belong to the same eq. class, then conflict g f f g f (a, b) = a f (f (a, b), b) = c g(a) g(c) a b c Example borrowed from [40]. Wednesday 12 th December, /

85 Theory Solvers for theories of interest T -solvers for EUF: Example Idea (sketch): given the set of terms occurring in the formula represented as nodes in a DAG (aka term bank), if (t = s), then merge the eq. classes of t and s if i 1...k, t i and s i pairwise belong to the same eq. classes, then merge the the eq. classes of f (t 1,..., t k ) and f (s 1,..., s k ) if (t s) and t and s belong to the same eq. class, then conflict g f f g f (a, b) = a f (f (a, b), b) = c g(a) g(c) a b c = conflict Example borrowed from [40]. Wednesday 12 th December, /

86 Theory Solvers for theories of interest T -solvers for Difference logic (DL) DL polynomial: O(#vars #constraints) variants of the Bellman-Ford shortest-path algorithm: a negative cycle reveals a conflict [65, 34] Ex: {(x 1 x 2 1), (x 1 x 4 1), (x 1 x 3 2), (x 3 x 4 2), (x 3 x 2 1), (x 4 x 2 3), (x 4 x 3 6)} 0 [ 4, x 3] x 1 1 [0, x 0] x 2 [0, x 0] x x 3 6 x 4 [ 2, x 4] 2 [0, x 0] = Sat Wednesday 12 th December, /

87 Theory Solvers for theories of interest T -solvers for Difference logic (DL) DL polynomial: O(#vars #constraints) variants of the Bellman-Ford shortest-path algorithm: a negative cycle reveals a conflict [65, 34] Ex: {(x 1 x 2 1), (x 1 x 4 1), (x 1 x 3 2), (x 2 x 1 2), (x 3 x 4 2), (x 3 x 2 1), (x 4 x 2 3), (x 4 x 3 6)} [0, x 0] x [ 4, x 3] x 1 2 x [0, x 0] x 2 3 x 0 [0,x 0] x [-6,x 3] 1 [-4,x 1] x 1 x x 3 6 x 4 [ 2, x 4] 2 [0, x 0] [-5,x 2] 2 [0,x 0] = Sat = Unsat Wednesday 12 th December, /

88 Theory Solvers for theories of interest T -solvers for Linear arithmetic over the rationals (LRA) EX: {(s 1 s 2 5.2), (s 1 = s t 3.4 t 0 ), (s 1 = s 0 )} LRA polynomial variants of the simplex LP algorithm [41] [41] allows for detecting conflict sets & performing T -propagation strict inequalities t < 0 rewritten as t + ɛ 0, ɛ treated symbolically B x 1. x i. x N = Invariant: β(x j ) [l j, u j ] x j N... A 1j.... A i1... A ij... A im.... A Nj... N x N+1. x j. x N+M ; Wednesday 12 th December, /

89 Remark: infinite precision arithmetic Theory Solvers for theories of interest In order to avoid incorrect results due to numerical errors and to overflows, all T -solvers for LRA, LIA and their subtheories which are based on numerical algorithms must be implemented on top of infinite-precision-arithmetic software packages. Wednesday 12 th December, /

90 Theory Solvers for theories of interest T -solvers for Linear arithmetic over the integers (LIA) EX: {(x := x l x h ), (x 0), (x )} LIA NP-complete combination of many techniques: simplex, branch&bound, cutting planes,... [41, 47] DPLL LIA-conflict 1 3 conflict LRA-solver no conflict 3 trail simplifications no conflict 1 2 no conflict equality elimination 4 conflict timeout 5 LIA-solver conflict 2 Diophantine equations handler LIA model 1 Internal Branch and Bound 4 LIA model 5 Branch and Bound-lemma Branch and Bound lemmas generator SAT Figure courtesy of A. Griggio [47] Wednesday 12 th December, /

91 T -solvers for Arrays (AR) Theory Solvers for theories of interest EX: (write(a, i, v) = write(b, i, w)) (v = w) NP-complete congruence closure (EU F) plus on-the-fly instantiation of array s axioms: a. i. e. (read(write(a, i, e), i) = e), (1) a. i. j. e. ((i j) read(write(a, i, e), j) = read(a, j)),(2) a. b. ( i.(read(a, i) = read(b, i)) (a = b)). (3) EX: Input : (write(a, i, v) = write(b, i, w)) (v = w) inst. (1) : (read(write(a, i, v), i) = v) (read(write(b, i, w), i) = w) = EUF (v = w) = Bool Wednesday 12 th December, / many Sebastiani strategies () discussed Cap. 2: Satisfiability in themodulo literature Theories (e.g., [40, 46, 20, 39])

92 Theory Solvers for theories of interest T -solvers for Bit vectors (BV) Bit vectors (BV) EX: {(x [16] [15 : 0] = (y [16] [15 : 8] :: z [16] [7 : 0]) << w [16] [3 : 0]),...} NP-hard involve complex word-level operations: word partition/concat, modulo-2 N arithmetic, shifts, bitwise-operations, multiplexers,... T -solving: combination of rewriting & simplification techniques with either: final encoding into LIA [19, 22] final encoding into SAT (lazy bit-blasting) [25, 43, 21, 42] Eager approach Most solvers use an eager approach for BV (e.g., [21]): Heavy preprocessing, based on rewriting rules bit-blasting Wednesday 12 th December, /

93 Theory Solvers for theories of interest T -solvers for Bit vectors (BV) [cont.] PREPROCESSOR Bool/word1 encoding SOLVER Concat. Elimination (match) Control paths extraction Variable elimination Unconstrained variables Concat. elimination (no match) Frontier propagation Deduction rules ITE expansion Formula Normalizer LITERAL NORMALIZATION TERM BANK Evaluation of Ground Terms Bit mask Elimination Example borrowed from [22] Selection Propagation Wednesday 12 th December, /

94 T -solvers for Bit vectors (BV) [cont.] Theory Solvers for theories of interest Lazy bit-blasting Two nested SAT solvers bit-blast each BV atom ψ i = Φ = def i (A i BB(ψ i )), A i fresh variables labeling atoms in ϕ p = ϕ BV-satisfiable iff ϕ p Φ satisfiable Exploit SAT under assumptions let µ p an assignment for ϕ p, s.t. µ p def = {[ ]A 1,..., [ ]A n } T -solver for BV: SAT assumption (Φ, µ p ) If UNSAT, generate the unsat core η p µ p = η p used as blocking clause Wednesday 12 th December, /

95 Outline Efficient SMT solving SMT for combinations of theories 1 Motivations and goals 2 Efficient SMT solving Combining SAT with Theory Solvers Theory Solvers for theories of interest SMT for combinations of theories 3 Beyond Solving: advanced SMT functionalities Proofs and unsatisfiable cores Interpolants All-SMT & Predicate Abstraction SMT with cost optimization (Optimization Modulo Theories) 4 Conclusions & current research directions Wednesday 12 th December, /

96 SMT for combinations of theories SMT for combined theories: SMT ( i T i) Problem: Many problems can be expressed as SMT problems only in combination of theories i T i SMT ( i T i) v 0 v 1 v 3 v 2 v 6 GE 01 LE 01 h h Sub 0 f f = EQ 67 v 4 v 8 RESET 5 v 5 v 7 LIA : (GE 01 (v 0 v 1 )) (LE 01 (v 0 v 1 )) EUF : (v 3 = h(v 0 )) (v 4 = h(v 1 )) LIA : (v 2 = v 3 v 4 ) (RESET 5 (v 5 = 0)) EUF or LIA : ( RESET 5 (v 5 = v 8 )) EUF : (v 6 = f (v 2 )) (v 7 = f (v 5)) EUF or LIA : (EQ 67 (v 6 = v 7))... Wednesday 12 th December, /

97 SMT for combinations of theories SMT for combined theories: SMT(T 1 T 2 ) Standard approach for combining T i -solver s: (deterministic) Nelson-Oppen/Shostak (N.O.) [61, 63, 77] based on deduction and exchange of equalities on shared variables combined T i -solver s integrated with a SAT tool More-recent alternative approaches: Delayed Theory Combination [15, 14] and Model-Based Theory Combination [37] based on Boolean search on equalities on shared variables T i -solver s integrated directly with a SAT tool Problem: N.O. approaches have some drawbacks and limitations when used within a SMT framework Wednesday 12 th December, /

98 Background: Pure Formulas SMT for combinations of theories Consider two theories T 1, T 2 with equality and disjoint signatures Σ 1, Σ 2 W.l.o.g. we assume all input formulas φ T 1 T 2 are pure. A formula φ is pure iff every atom in φ is i-pure for some i {1, 2}. An atom/literal in φ is i-pure if only =, variables and symbols from Σ i can occur in φ Purification: maps a formula into an equisatisfiable pure formula by labeling terms with fresh variables (f (x + 3y) = g(2x y)) [not pure] }{{}}{{} w t (w = x + 3y) (t = 2x y) (f (w) = g(t)) [pure] Wednesday 12 th December, /

99 Background: Interface equalities Interface variables & equalities SMT for combinations of theories A variable v occurring in a pure formula φ is an interface variable iff it occurs in both 1-pure and 2-pure atoms of φ. An equality (v i = v j ) is an interface equality for φ iff v i, v j are interface variables for φ. We denote the interface equality v i = v j by e ij Example: LIA : (GE 01 (v 0 v 1 )) (LE 01 (v 0 v 1 )) EUF : (v 3 = h(v 0 )) (v 4 = h(v 1 )) LIA : (v 2 = v 3 v 4 ) (RESET 5 (v 5 = 0)) EUF or LIA : ( RESET 5 (v 5 = v 8 )) EUF : (v 6 = f (v 2 )) (v 7 = f (v 5)) EUF or LIA : (EQ 67 (v 6 = v 7))... v 0, v 1, v 2, v 3, v 4, v 5 are interface variables, v 6, v 7, v 8 are not = (v 0 = v 1 ) is an interface equality, (v 0 = v 6 ) is not. Wednesday 12 th December, /

100 SMT for combinations of theories Background: Stably-infinite & Convex Theories Stably-infinite Theories A theory T is stably-infinite iff every quantifier-free T -satisfiable formula is satisfiable in an infinite model of T. EU F, DL, LRA, LIA are stably-infinite bit-vector theories typically are not stably-infinite Convex Theories A theory T is convex iff, for every collection l 1,..., l k, l, l of literals in T s.t. l, l are in the form (x = y), x, y being variables, we have that: {l 1,..., l k } = T (l l ) {l 1,..., l k } = T l or {l 1,..., l k } = T l EUF, DL, LRA are convex LIA is not convex: {(v 0 = 0), (v 1 = 1), (v v 0 ), (v v 1 )} = ((v = v 0 ) (v = v 1 )), {(v 0 = 0), (v 1 = 1), (v v 0 ), (v v 1 )} = (v = v 0 ) {(v 0 = 0), (v 1 = 1), (v 0), (v v 1 )} = (v = v 1 ) Wednesday 12 th December, /

101 SMT for combinations of theories Background: Stably-infinite & Convex Theories Stably-infinite Theories A theory T is stably-infinite iff every quantifier-free T -satisfiable formula is satisfiable in an infinite model of T. EU F, DL, LRA, LIA are stably-infinite bit-vector theories typically are not stably-infinite Convex Theories A theory T is convex iff, for every collection l 1,..., l k, l, l of literals in T s.t. l, l are in the form (x = y), x, y being variables, we have that: {l 1,..., l k } = T (l l ) {l 1,..., l k } = T l or {l 1,..., l k } = T l EUF, DL, LRA are convex LIA is not convex: {(v 0 = 0), (v 1 = 1), (v v 0 ), (v v 1 )} = ((v = v 0 ) (v = v 1 )), {(v 0 = 0), (v 1 = 1), (v v 0 ), (v v 1 )} = (v = v 0 ) {(v 0 = 0), (v 1 = 1), (v 0), (v v 1 )} = (v = v 1 ) Wednesday 12 th December, /

102 SMT for combinations of theories SMT ( i T i) via classic Nelson-Oppen Main idea Combine two or more T i -solvers into one ( i T i)-solver via Nelson-Oppen/Shostak (N.O.) combination procedure [62, 78] based on the deduction and exchange of equalities between shared variables/terms (interface equalities, e ij s) important improvements and evolutions [69, 7, 40] Wednesday 12 th December, /

103 SMT for combinations of theories Schema of N.O. combination of T-solvers: no(t 1, T 2 ) For i {1, 2}, let T i be a stably infinite theory admitting a satisfiability T i -solver, and µ i a set of i-pure literals. We want to to decide the T 1 T 2 -satisfiability of µ 1 µ 2 each T i -solver, in turn checks the T i -satisfiability of µ i, deduces all the (disjunctions of) interface equalities which derive from µ i passes them to T j -solve, j i, which adds them to µ j until either: one T i -solver detects inconsistency (µ 1 µ 2 is T 1 T 2 -unsat) no more deductions are possible (µ 1 µ 2 is T 1 T 2 -sat) disjunctions of literals (due to non-convexity) force case-splitting Wednesday 12 th December, /

104 SMT for combinations of theories Schema of N.O. combination of T-solvers: no(t 1, T 2 ) For i {1, 2}, let T i be a stably infinite theory admitting a satisfiability T i -solver, and µ i a set of i-pure literals. We want to to decide the T 1 T 2 -satisfiability of µ 1 µ 2 each T i -solver, in turn checks the T i -satisfiability of µ i, deduces all the (disjunctions of) interface equalities which derive from µ i passes them to T j -solve, j i, which adds them to µ j until either: one T i -solver detects inconsistency (µ 1 µ 2 is T 1 T 2 -unsat) no more deductions are possible (µ 1 µ 2 is T 1 T 2 -sat) disjunctions of literals (due to non-convexity) force case-splitting Wednesday 12 th December, /

105 SMT for combinations of theories Schema of N.O. combination of T-solvers: no(t 1, T 2 ) For i {1, 2}, let T i be a stably infinite theory admitting a satisfiability T i -solver, and µ i a set of i-pure literals. We want to to decide the T 1 T 2 -satisfiability of µ 1 µ 2 each T i -solver, in turn checks the T i -satisfiability of µ i, deduces all the (disjunctions of) interface equalities which derive from µ i passes them to T j -solve, j i, which adds them to µ j until either: one T i -solver detects inconsistency (µ 1 µ 2 is T 1 T 2 -unsat) no more deductions are possible (µ 1 µ 2 is T 1 T 2 -sat) disjunctions of literals (due to non-convexity) force case-splitting Wednesday 12 th December, /

106 SMT for combinations of theories Schema of N.O. combination of T-solvers: no(t 1, T 2 ) For i {1, 2}, let T i be a stably infinite theory admitting a satisfiability T i -solver, and µ i a set of i-pure literals. We want to to decide the T 1 T 2 -satisfiability of µ 1 µ 2 each T i -solver, in turn checks the T i -satisfiability of µ i, deduces all the (disjunctions of) interface equalities which derive from µ i passes them to T j -solve, j i, which adds them to µ j until either: one T i -solver detects inconsistency (µ 1 µ 2 is T 1 T 2 -unsat) no more deductions are possible (µ 1 µ 2 is T 1 T 2 -sat) disjunctions of literals (due to non-convexity) force case-splitting Wednesday 12 th December, /

107 SMT for combinations of theories Schema of N.O. combination of T-solvers: no(t 1, T 2 ) For i {1, 2}, let T i be a stably infinite theory admitting a satisfiability T i -solver, and µ i a set of i-pure literals. We want to to decide the T 1 T 2 -satisfiability of µ 1 µ 2 each T i -solver, in turn checks the T i -satisfiability of µ i, deduces all the (disjunctions of) interface equalities which derive from µ i passes them to T j -solve, j i, which adds them to µ j until either: one T i -solver detects inconsistency (µ 1 µ 2 is T 1 T 2 -unsat) no more deductions are possible (µ 1 µ 2 is T 1 T 2 -sat) disjunctions of literals (due to non-convexity) force case-splitting Wednesday 12 th December, /

108 SMT for combinations of theories Schema of N.O. combination of T-solvers: no(t 1, T 2 ) For i {1, 2}, let T i be a stably infinite theory admitting a satisfiability T i -solver, and µ i a set of i-pure literals. We want to to decide the T 1 T 2 -satisfiability of µ 1 µ 2 each T i -solver, in turn checks the T i -satisfiability of µ i, deduces all the (disjunctions of) interface equalities which derive from µ i passes them to T j -solve, j i, which adds them to µ j until either: one T i -solver detects inconsistency (µ 1 µ 2 is T 1 T 2 -unsat) no more deductions are possible (µ 1 µ 2 is T 1 T 2 -sat) disjunctions of literals (due to non-convexity) force case-splitting Wednesday 12 th December, /

109 SMT for combinations of theories Schema of N.O. combination of T-solvers: no(t 1, T 2 ) For i {1, 2}, let T i be a stably infinite theory admitting a satisfiability T i -solver, and µ i a set of i-pure literals. We want to to decide the T 1 T 2 -satisfiability of µ 1 µ 2 each T i -solver, in turn checks the T i -satisfiability of µ i, deduces all the (disjunctions of) interface equalities which derive from µ i passes them to T j -solve, j i, which adds them to µ j until either: one T i -solver detects inconsistency (µ 1 µ 2 is T 1 T 2 -unsat) no more deductions are possible (µ 1 µ 2 is T 1 T 2 -sat) disjunctions of literals (due to non-convexity) force case-splitting Wednesday 12 th December, /

110 SMT for combinations of theories Schema of N.O. combination of T-solvers: no(t 1, T 2 ) For i {1, 2}, let T i be a stably infinite theory admitting a satisfiability T i -solver, and µ i a set of i-pure literals. We want to to decide the T 1 T 2 -satisfiability of µ 1 µ 2 each T i -solver, in turn checks the T i -satisfiability of µ i, deduces all the (disjunctions of) interface equalities which derive from µ i passes them to T j -solve, j i, which adds them to µ j until either: one T i -solver detects inconsistency (µ 1 µ 2 is T 1 T 2 -unsat) no more deductions are possible (µ 1 µ 2 is T 1 T 2 -sat) disjunctions of literals (due to non-convexity) force case-splitting Wednesday 12 th December, /

111 SMT for combinations of theories Schema of N.O. combination of T-solvers: no(t 1, T 2 ) For i {1, 2}, let T i be a stably infinite theory admitting a satisfiability T i -solver, and µ i a set of i-pure literals. We want to to decide the T 1 T 2 -satisfiability of µ 1 µ 2 each T i -solver, in turn checks the T i -satisfiability of µ i, deduces all the (disjunctions of) interface equalities which derive from µ i passes them to T j -solve, j i, which adds them to µ j until either: one T i -solver detects inconsistency (µ 1 µ 2 is T 1 T 2 -unsat) no more deductions are possible (µ 1 µ 2 is T 1 T 2 -sat) disjunctions of literals (due to non-convexity) force case-splitting Wednesday 12 th December, /

112 SMT for combinations of theories Schema of N.O. combination of T-solvers: no(t 1, T 2 ) no(t 1, T 2 ) T 1 -solver T 1 -deduce T 2 -solver T 2 -satisfiable (v i = v j ) T 1 -satisfiable T2 -deduce Wednesday 12 th December, /

113 N.O.: example (convex theory) SMT for combinations of theories EUF : (v 3 = h(v 0 )) (v 4 = h(v 1 )) (v 6 = f (v 2 )) (v 7 = f (v 5)) LRA : (v 0 v 1 ) (v 0 v 1 ) (v 2 = v 3 v 4 ) (RESET 5 (v 5 = 0)) Both : ( RESET 5 (v 5 = v 8 )) (v 6 = v 7). v 3 = h(v 0 ) v 4 = h(v 1 ) v 6 = f (v 2 ) v 7 = f (v 5 ) (v 6 = v 7 ) RESET 5 Branch 1 Branch 2 RESET 5 v 0 v 1 v 3 = h(v 0 ) v 0 v 1 v 0 v 1 v 4 = h(v 1 ) v 0 v 1 v 2 = v 3 v 4 v 6 = f (v 2 ) v 2 = v 3 v 4 v 5 = 0 v 7 = f (v 5 ) v 5 = v 8 (v 6 = v 7 ) EUF LRA EUF LRA e ij-deduction e ij-deduction v 0 = v 1 v 3 = v 4 v 2 = v 5 v 2 = v 5 v 0 = v 1 v 0 = v 1 e ij-deduction e ij-deduction v 3 = v 4 e ij-deduction v 3 = v 4 v 0 = v 1 v 3 = v 4 EUF LRA-Satisfiable! Wednesday 12 th December, /

114 SMT for combinations of theories N.O.: example (convex theory) [cont.] v 3 = h(v 0 ) v 4 = h(v 1 ) v 6 = f (v 2 ) v 7 = f (v 5 ) (v 6 = v 7 ) RESET 5 Branch 1 Branch 2 RESET 5 v 0 v 1 v 3 = h(v 0 ) v 0 v 1 v 0 v 1 v 4 = h(v 1 ) v 0 v 1 v 2 = v 3 v 4 v 6 = f (v 2 ) v 2 = v 3 v 4 v 5 = 0 v 7 = f (v 5 ) v 5 = v 8 (v 6 = v 7 ) EUF LRA EUF LRA e ij-deduction e ij-deduction v 0 = v 1 v 3 = v 4 v 2 = v 5 v 2 = v 5 v 0 = v 1 v 0 = v 1 e ij-deduction e ij-deduction v 3 = v 4 e ij-deduction v 3 = v 4 v 0 = v 1 v 3 = v 4 EUF LRA-Satisfiable! EUF-conflict : ((v 6 = f (v 2 )) (v 7 = f (v 5)) (v 6 = v 7) (v 2 = v 5)) LRA-deduction : ((v 2 = v 3 v 4 ) (v 5 = 0) (v 3 = v 4 )) (v 2 = v 5) EUF-deduction : ((v 3 = h(v 0 )) (v 4 = h(v 1 )) (v 0 = v 1 )) (v 3 = v 4 ) LRA-deduction : ((v 0 v 1 ) (v 0 v 1 )) (v 0 = v 1 ) = EUF LRA-conflict : ((v 6 = f (v 2 )) (v 7 = f (v 5)) (v 6 = v 7) (v 2 = v 3 v 4 ) (v 5 = 0) (v 3 = h(v 0 )) (v 4 = h(v 1 )) (v 0 v 1 )). Wednesday 12 th December, /

115 N.O.: example (non-convex theory) SMT for combinations of theories µ LIA v 1 0 v 5 = v 4 1 v 1 1 v 3 = 0 v 2 v 6 v 2 v v 4 = 1 µ EUF (f (v1 ) = f (v 2 )) (f (v 2 ) = f (v 4 )) f (v 3 ) = v 5 f (v 1 ) = v 6 Wednesday 12 th December, /

116 SMT for combinations of theories N.O.: example (non-convex theory) µ LIA v 1 0 v 5 = v 4 1 v 1 1 v 3 = 0 v 2 v 6 v 2 v v 4 = 1 e ij -deduction µ EUF (f (v1 ) = f (v 2 )) (f (v 2 ) = f (v 4 )) f (v 3 ) = v 5 f (v 1 ) = v 6 v 1 = v 3 v 1 = v 4 Wednesday 12 th December, /

117 SMT for combinations of theories N.O.: example (non-convex theory) µ LIA v 1 0 v 5 = v 4 1 v 1 1 v 3 = 0 v 2 v 6 v 2 v v 4 = 1 e ij -deduction v 1 = v 3 v 1 = v 4 µ EUF (f (v1 ) = f (v 2 )) (f (v 2 ) = f (v 4 )) f (v 3 ) = v 5 f (v 1 ) = v 6 v 1 = v 3 Wednesday 12 th December, /

118 SMT for combinations of theories N.O.: example (non-convex theory) µ LIA v 1 0 v 5 = v 4 1 v 1 1 v 3 = 0 v 2 v 6 v 2 v v 4 = 1 e ij -deduction v 1 = v 3 v 1 = v 4 µ EUF (f (v1 ) = f (v 2 )) (f (v 2 ) = f (v 4 )) f (v 3 ) = v 5 f (v 1 ) = v 6 v 1 = v 3 e ij -deduction v 5 = v 6 Wednesday 12 th December, /

119 SMT for combinations of theories N.O.: example (non-convex theory) µ LIA v 1 0 v 5 = v 4 1 v 1 1 v 3 = 0 v 2 v 6 v 2 v v 4 = 1 e ij -deduction v 1 = v 3 v 1 = v 4 v 5 = v 6 e ij -deduction v 2 = v 3 v 2 = v 4 µ EUF (f (v1 ) = f (v 2 )) (f (v 2 ) = f (v 4 )) f (v 3 ) = v 5 f (v 1 ) = v 6 v 1 = v 3 e ij -deduction v 5 = v 6 Wednesday 12 th December, /

120 SMT for combinations of theories N.O.: example (non-convex theory) µ LIA v 1 0 v 5 = v 4 1 v 1 1 v 3 = 0 v 2 v 6 v 2 v v 4 = 1 e ij -deduction v 1 = v 3 v 1 = v 4 v 5 = v 6 e ij -deduction v 2 = v 3 v 2 = v 4 µ EUF (f (v1 ) = f (v 2 )) (f (v 2 ) = f (v 4 )) f (v 3 ) = v 5 f (v 1 ) = v 6 v 1 = v 3 e ij -deduction v 5 = v 6 v 2 = v 3 Wednesday 12 th December, /

121 SMT for combinations of theories N.O.: example (non-convex theory) µ LIA v 1 0 v 5 = v 4 1 v 1 1 v 3 = 0 v 2 v 6 v 2 v v 4 = 1 e ij -deduction v 1 = v 3 v 1 = v 4 v 5 = v 6 e ij -deduction v 2 = v 3 v 2 = v 4 µ EUF (f (v1 ) = f (v 2 )) (f (v 2 ) = f (v 4 )) f (v 3 ) = v 5 f (v 1 ) = v 6 v 1 = v 3 e ij -deduction v 5 = v 6 v 2 = v 3 v 2 = v 4 Wednesday 12 th December, /

122 SMT for combinations of theories N.O.: example (non-convex theory) µ LIA v 1 0 v 5 = v 4 1 v 1 1 v 3 = 0 v 2 v 6 v 2 v v 4 = 1 e ij -deduction v 1 = v 3 v 1 = v 4 v 5 = v 6 e ij -deduction v 2 = v 3 v 2 = v 4 µ EUF (f (v1 ) = f (v 2 )) (f (v 2 ) = f (v 4 )) f (v 3 ) = v 5 f (v 1 ) = v 6 v 1 = v 3 e ij -deduction v 5 = v 6 v 2 = v 3 v 2 = v 4 v 1 = v 4 SAT! Wednesday 12 th December, /

123 SMT for combinations of theories N.O.: example (non-convex theory) µ LIA v 1 0 v 5 = v 4 1 v 1 1 v 3 = 0 v 2 v 6 v 2 v v 4 = 1 e ij -deduction v 1 = v 3 v 1 = v 4 v 5 = v 6 e ij -deduction v 2 = v 3 v 2 = v 4 µ EUF (f (v1 ) = f (v 2 )) (f (v 2 ) = f (v 4 )) f (v 3 ) = v 5 f (v 1 ) = v 6 v 1 = v 3 e ij -deduction v 5 = v 6 v 2 = v 3 v 2 = v 4 v 1 = v 4 SAT! 3 e ij -deductions, 3 branches Wednesday 12 th December, /

124 SMT for combinations of theories SMT ( i T i) via classic Nelson-Oppen Main idea Combine two or more T i -solvers into one ( i T i)-solver via Nelson-Oppen/Shostak (N.O.) combination procedure [62, 78] based on the deduction and exchange of equalities between shared variables/terms (interface equalities, e ij s) important improvements and evolutions [69, 7, 40] drawbacks [23, 24]: require (possibly expensive) deduction capabilities from T i -solvers [ with non-convex theories ] case-splits forced by the deduction of disjunctions of e ij s generate (typically long) ( i T i)-lemmas, without interface equalities = no backjumping & learning from e ij -reasoning Wednesday 12 th December, /

125 SMT for combinations of theories SMT ( i T i) via classic Nelson-Oppen Main idea Combine two or more T i -solvers into one ( i T i)-solver via Nelson-Oppen/Shostak (N.O.) combination procedure [62, 78] based on the deduction and exchange of equalities between shared variables/terms (interface equalities, e ij s) important improvements and evolutions [69, 7, 40] drawbacks [23, 24]: require (possibly expensive) deduction capabilities from T i -solvers [ with non-convex theories ] case-splits forced by the deduction of disjunctions of e ij s generate (typically long) ( i T i)-lemmas, without interface equalities = no backjumping & learning from e ij -reasoning Wednesday 12 th December, /

126 SMT for combinations of theories SMT ( i T i) via Delayed Theory Combination (DTC) Main idea Delegate to the CDCL SAT solver part/most of the (possibly very expensive) reasoning effort on interface equalities previously due to the T i -solvers (e ij -deduction, case-split). [15, 16, 24] based on Boolean reasoning on interface equalities via CDCL (plus T -propagation) important improvements and evolutions [37, 9] feature wrt N.O. [23, 24] do not require (possibly expensive) deduction capabilities from T i -solvers with non-convex theories, case-splits on e ij s handled by SAT generate T i -lemmas with interface equalities = backjumping & learning from e ij -reasoning Wednesday 12 th December, /

127 DTC: Basic schema Efficient SMT solving SMT for combinations of theories BOOLEAN MODEL ENUMERATION Atoms {e ij } ij Sat/Unsat µ µ e µ Sat/Unsat 1 2 T 1 -satisfiable T 2 -satisfiable Wednesday 12 th December, /

128 DTC: Basic schema Efficient SMT solving SMT for combinations of theories BOOLEAN MODEL ENUMERATION Atoms {e ij } ij Sat/Unsat µ µ e µ Sat/Unsat 1 2 T 1 -satisfiable T 2 -satisfiable The boolean solver assigns values not only to atoms in Atoms(φ), but also to interface equalities {(v i = v j )} ij : µ = µ 1 µ 2 µ e, µ e := {[ ](v i = v j ) v i, v j µ 1 µ 2 } Wednesday 12 th December, /

129 DTC: Basic schema Efficient SMT solving SMT for combinations of theories BOOLEAN MODEL ENUMERATION Atoms {e ij } ij Sat/Unsat µ µ e µ Sat/Unsat 1 2 T 1 -satisfiable T 2 -satisfiable Each T i -solver interacts only with the boolean solver receives µ i := µ i µ e from Bool checks the T i -satisfiability of µ i Wednesday 12 th December, /

130 DTC: Basic schema Efficient SMT solving SMT for combinations of theories BOOLEAN MODEL ENUMERATION Atoms {e ij } ij Sat/Unsat µ µ e µ Sat/Unsat 1 2 T 1 -satisfiable T 2 -satisfiable...until either: some µ propositionally satisfies φ and both µ i := µ i µ e are T i -consistent = (φ is T 1 T 2 -sat) no more assignment µ are available = (φ is T 1 T 2 -unsat) Wednesday 12 th December, /

131 DTC: enhanced schema SMT for combinations of theories DPLL-based assignment enumeration on Atoms(φ) {e ij } ij, = benefits of state-of-the-art SAT techniques Early pruning: invoke the T i -solver s before every Boolean decision = total assignments generated only when strictly necessary Branching: branching on e ij s postponed = Boolean search on e ij s performed only when strictly necessary Theory-Backjumping & Learning: e ij s are involved in conflicts = e ij s can be assigned by unit propagation [ Theory-deduction & learning: T i -solver deduces unassigned literals l on Atoms(φ) {e ij } ij l is passed back to the Boolean solver, which unit-propagates it the deduction µ = l is learned as a clause µ l (deduction clause) ]... Wednesday 12 th December, /

132 SMT for combinations of theories DTC: example w.out T -prop. (non-convex theory) µ EUF : (f (v 1 ) = f (v 2 )) (f (v 2 ) = f (v 4 )) f (v 3 ) = v 5 f (v 1 ) = v 6 µ LIA : v 1 0 v 5 = v 4 1 v 1 1 v 3 = 0 v 2 v 6 v 4 = 1 v 2 v Wednesday 12 th December, /

133 SMT for combinations of theories DTC: example w.out T -prop. (non-convex theory) µ EUF : (f (v 1 ) = f (v 2 )) (f (v 2 ) = f (v 4 )) f (v 3 ) = v 5 f (v 1 ) = v 6 µ LIA : v 1 0 v 5 = v 4 1 v 1 1 v 3 = 0 v 2 v 6 v 4 = 1 v 2 v (v 1 = v 4 ) (v 1 = v 3 ) LIA-unsat, C 13 C 13 : (µ LIA ) ((v 1 = v 3 ) (v 1 = v 4 )) Wednesday 12 th December, /

134 SMT for combinations of theories DTC: example w.out T -prop. (non-convex theory) µ EUF : (f (v 1 ) = f (v 2 )) (f (v 2 ) = f (v 4 )) f (v 3 ) = v 5 f (v 1 ) = v 6 µ LIA : v 1 0 v 5 = v 4 1 v 1 1 v 3 = 0 v 2 v 6 v 4 = 1 v 2 v (v 1 = v 4 ) (v 1 = v 3 ) v 1 = v 3 C 13 : (µ LIA ) ((v 1 = v 3 ) (v 1 = v 4 )) Wednesday 12 th December, /

135 SMT for combinations of theories DTC: example w.out T -prop. (non-convex theory) µ EUF : (f (v 1 ) = f (v 2 )) (f (v 2 ) = f (v 4 )) f (v 3 ) = v 5 f (v 1 ) = v 6 µ LIA : v 1 0 v 5 = v 4 1 v 1 1 v 3 = 0 v 2 v 6 v 4 = 1 v 2 v (v 1 = v 4 ) (v 1 = v 3 ) v 1 = v 3 (v 5 = v 6 ) EUF-unsat, C 56 C 13 : (µ LIA ) ((v 1 = v 3 ) (v 1 = v 4 )) C 56 : (µ EUF (v 1 = v 3 )) (v 5 = v 6 ) Wednesday 12 th December, /

136 SMT for combinations of theories DTC: example w.out T -prop. (non-convex theory) µ EUF : (f (v 1 ) = f (v 2 )) (f (v 2 ) = f (v 4 )) f (v 3 ) = v 5 f (v 1 ) = v 6 µ LIA : v 1 0 v 5 = v 4 1 v 1 1 v 3 = 0 v 2 v 6 v 4 = 1 v 2 v (v 1 = v 4 ) (v 1 = v 3 ) v 1 = v 3 v 5 = v 6 (v 5 = v 6 ) C 13 : (µ LIA ) ((v 1 = v 3 ) (v 1 = v 4 )) C 56 : (µ EUF (v 1 = v 3 )) (v 5 = v 6 ) Wednesday 12 th December, /

137 SMT for combinations of theories DTC: example w.out T -prop. (non-convex theory) µ EUF : (f (v 1 ) = f (v 2 )) (f (v 2 ) = f (v 4 )) f (v 3 ) = v 5 f (v 1 ) = v 6 µ LIA : v 1 0 v 5 = v 4 1 v 1 1 v 3 = 0 v 2 v 6 v 4 = 1 v 2 v (v 1 = v 4 ) (v 1 = v 3 ) v 1 = v 3 v 5 = v 6 (v 5 = v 6 ) (v 2 = v 3 ) (v 2 = v 4 ) C 13 : (µ LIA ) ((v 1 = v 3 ) (v 1 = v 4 )) C 56 : (µ EUF (v 1 = v 3 )) (v 5 = v 6 ) C 23 : (µ LIA (v 5 = v 6 )) ((v 2 = v 3 ) (v 2 = v 4 )) LIA-unsat, C 23 Wednesday 12 th December, /

138 SMT for combinations of theories DTC: example w.out T -prop. (non-convex theory) µ EUF : (f (v 1 ) = f (v 2 )) (f (v 2 ) = f (v 4 )) f (v 3 ) = v 5 f (v 1 ) = v 6 µ LIA : v 1 0 v 5 = v 4 1 v 1 1 v 3 = 0 v 2 v 6 v 4 = 1 v 2 v (v 1 = v 4 ) (v 1 = v 3 ) v 1 = v 3 v 5 = v 6 (v 5 = v 6 ) C 13 : (µ LIA (v 2 = v 4 ) ) ((v 1 = v 3 ) (v 1 = v 4 )) C 56 : (µ v 2 = v 3 C 23 : (µ EUF (v 1 = v 3 )) (v 5 = v 6 ) LIA EUF-unsat, C (v 5 = v 6 )) ((v 2 = v 3 ) (v 2 = v 4 )) (v 2 = v 3 ) 24 C 24 : (µ EUF (v 1 = v 3 ) (v 2 = v 3 )) Wednesday 12 th December, /

139 SMT for combinations of theories DTC: example w.out T -prop. (non-convex theory) µ EUF : (f (v 1 ) = f (v 2 )) (f (v 2 ) = f (v 4 )) f (v 3 ) = v 5 f (v 1 ) = v 6 µ LIA : v 1 0 v 5 = v 4 1 v 1 1 v 3 = 0 v 2 v 6 v 4 = 1 v 2 v (v 1 = v 4 ) (v 1 = v 3 ) v 1 = v 3 v 5 = v 6 (v 5 = v 6 ) v 2 = v 4 EUF-unsat, C 14 C 13 : (µ LIA (v 2 = v 4 ) ) ((v 1 = v 3 ) (v 1 = v 4 )) C 56 : (µ EUF v 2 = v (v 1 = v 3 )) (v 5 = v 6 ) 3 C 23 : (µ LIA (v 5 = v 6 )) ((v 2 = v 3 ) (v 2 = v 4 )) (v 2 = v 3 ) C 24 : (µ EUF (v 1 = v 3 ) (v 2 = v 3 )) C 14 : (µ EUF (v 1 = v 3 ) (v 2 = v 4 )) Wednesday 12 th December, /

140 SMT for combinations of theories DTC: example w.out T -prop. (non-convex theory) µ EUF : (f (v 1 ) = f (v 2 )) (f (v 2 ) = f (v 4 )) f (v 3 ) = v 5 f (v 1 ) = v 6 (v 1 = v 4 ) µ LIA : v 1 0 v 5 = v 4 1 v 1 1 v 3 = 0 v 2 v 6 v 4 = 1 v 2 v v 1 = v 4 (v 1 = v 3 ) v 1 = v 3 v 5 = v 6 (v 5 = v 6 ) v 2 = v 4 C 13 : (µ LIA (v 2 = v 4 ) ) ((v 1 = v 3 ) (v 1 = v 4 )) C 56 : (µ EUF v 2 = v (v 1 = v 3 )) (v 5 = v 6 ) 3 C 23 : (µ LIA (v 5 = v 6 )) ((v 2 = v 3 ) (v 2 = v 4 )) (v 2 = v 3 ) C 24 : (µ EUF (v 1 = v 3 ) (v 2 = v 3 )) C 14 : (µ EUF (v 1 = v 3 ) (v 2 = v 4 )) Wednesday 12 th December, /

141 SMT for combinations of theories DTC: example w.out T -prop. (non-convex theory) µ EUF : (f (v 1 ) = f (v 2 )) (f (v 2 ) = f (v 4 )) f (v 3 ) = v 5 f (v 1 ) = v 6 µ LIA : v 1 0 v 5 = v 4 1 v 1 1 v 3 = 0 v 2 v 6 v 4 = 1 v 2 v (v 1 = v 3 ) (v 5 = v 6 ) (v 1 = v 4 ) SAT! v 1 = v 3 v 5 = v 6 v 2 = v 4 v 1 = v 4 C 13 : (µ LIA (v 2 = v 4 ) ) ((v 1 = v 3 ) (v 1 = v 4 )) C 56 : (µ v 2 = v 3 C 23 : (µ EUF (v 1 = v 3 )) (v 5 = v 6 ) LIA (v 5 = v 6 )) ((v 2 = v 3 ) (v 2 = v 4 )) (v 2 = v 3 ) C 24 : (µ EUF (v 1 = v 3 ) (v 2 = v 3 )) C 14 : (µ EUF (v 1 = v 3 ) (v 2 = v 4 )) Wednesday 12 th December, /

142 SMT for combinations of theories DTC: example w.out T -prop. (non-convex theory) µ EUF : (f (v 1 ) = f (v 2 )) (f (v 2 ) = f (v 4 )) f (v 3 ) = v 5 f (v 1 ) = v 6 µ LIA : v 1 0 v 5 = v 4 1 v 1 1 v 3 = 0 v 2 v 6 v 4 = 1 v 2 v (v 1 = v 3 ) (v 5 = v 6 ) (v 1 = v 4 ) SAT! v 1 = v 3 v 5 = v 6 v 2 = v 4 v 1 = v 4 6 branches C 13 : (µ LIA (v 2 = v 4 ) ) ((v 1 = v 3 ) (v 1 = v 4 )) C 56 : (µ EUF v 2 = v (v 1 = v 3 )) (v 5 = v 6 ) 3 C 23 : (µ LIA (v 5 = v 6 )) ((v 2 = v 3 ) (v 2 = v 4 )) (v 2 = v 3 ) C 24 : (µ EUF (v 1 = v 3 ) (v 2 = v 3 )) C 14 : (µ EUF (v 1 = v 3 ) (v 2 = v 4 )) Wednesday 12 th December, /

143 SMT for combinations of theories DTC: example w.out T -prop. (non-convex theory) µ EUF : (f (v 1 ) = f (v 2 )) (f (v 2 ) = f (v 4 )) f (v 3 ) = v 5 f (v 1 ) = v 6 µ LIA : v 1 0 v 5 = v 4 1 v 1 1 v 3 = 0 v 2 v 6 v 4 = 1 v 2 v (v 1 = v 4 ) SAT! v 1 = v 3 (v 1 = v 3 ) v 5 = v 6 v 1 = v 4 Mimics the e ij -deduction µ LIA = LIA ((v 1 = v 3 ) (v 1 = v 4 )) and the two branches (v 1 = v 3 ), (v 1 = v 4 ) (v 5 = v 6 ) v 2 = v 4 C 13 : (µ LIA (v 2 = v 4 ) ) ((v 1 = v 3 ) (v 1 = v 4 )) C 56 : (µ EUF v 2 = v (v 1 = v 3 )) (v 5 = v 6 ) 3 C 23 : (µ LIA (v 5 = v 6 )) ((v 2 = v 3 ) (v 2 = v 4 )) (v 2 = v 3 ) C 24 : (µ EUF (v 1 = v 3 ) (v 2 = v 3 )) C 14 : (µ EUF (v 1 = v 3 ) (v 2 = v 4 )) Wednesday 12 th December, /

144 SMT for combinations of theories DTC: example with T -prop. (non-convex theory) µ EUF : (f (v 1 ) = f (v 2 )) (f (v 2 ) = f (v 4 )) f (v 3 ) = v 5 f (v 1 ) = v 6 µ LIA : v 1 0 v 5 = v 4 1 v 1 1 v 3 = 0 v 2 v 6 v 4 = 1 v 2 v Wednesday 12 th December, /

145 SMT for combinations of theories DTC: example with T -prop. (non-convex theory) µ EUF : (f (v 1 ) = f (v 2 )) (f (v 2 ) = f (v 4 )) f (v 3 ) = v 5 f (v 1 ) = v 6 µ LIA : v 1 0 v 5 = v 4 1 v 1 1 v 3 = 0 v 2 v 6 v 4 = 1 v 2 v LIA-deduce (v 1 = v 4 ) (v 1 = v 3 ), C 13 C 13 : (µ LIA ) ((v 1 = v 3 ) (v 1 = v 4 )) Wednesday 12 th December, /

146 SMT for combinations of theories DTC: example with T -prop. (non-convex theory) µ EUF : (f (v 1 ) = f (v 2 )) (f (v 2 ) = f (v 4 )) f (v 3 ) = v 5 f (v 1 ) = v 6 µ LIA : v 1 0 v 5 = v 4 1 v 1 1 v 3 = 0 v 2 v 6 v 4 = 1 v 2 v (v 1 = v 4 ) v 1 = v 3 C 13 : (µ LIA ) ((v 1 = v 3 ) (v 1 = v 4 )) Wednesday 12 th December, /

147 SMT for combinations of theories DTC: example with T -prop. (non-convex theory) µ EUF : (f (v 1 ) = f (v 2 )) (f (v 2 ) = f (v 4 )) f (v 3 ) = v 5 f (v 1 ) = v 6 µ LIA : v 1 0 v 5 = v 4 1 v 1 1 v 3 = 0 v 2 v 6 v 4 = 1 v 2 v (v 1 = v 4 ) v 1 = v 3 v 5 = v 6 EUF-deduce (v 5 = v 6 ), C 56 C 13 : (µ C 56 : (µ LIA ) ((v 1 = v 3 ) (v 1 = v 4 )) EUF (v 1 = v 3 )) (v 5 = v 6 ) Wednesday 12 th December, /

148 SMT for combinations of theories DTC: example with T -prop. (non-convex theory) µ EUF : (f (v 1 ) = f (v 2 )) (f (v 2 ) = f (v 4 )) f (v 3 ) = v 5 f (v 1 ) = v 6 µ LIA : v 1 0 v 5 = v 4 1 v 1 1 v 3 = 0 v 2 v 6 v 4 = 1 v 2 v (v 1 = v 4 ) v 1 = v 3 v 5 = v 6 LIA-deduce (v 2 = v 4 ) (v 2 = v 3 ), C 23 C 13 : (µ LIA ) ((v 1 = v 3 ) (v 1 = v 4 )) C 56 : (µ C 23 : (µ EUF (v LIA (v 1 = v 3 )) (v 5 = v 6 ) 5 = v 6 )) ((v 2 = v 3 ) (v 2 = v 4 )) Wednesday 12 th December, /

149 SMT for combinations of theories DTC: example with T -prop. (non-convex theory) µ EUF : (f (v 1 ) = f (v 2 )) (f (v 2 ) = f (v 4 )) f (v 3 ) = v 5 f (v 1 ) = v 6 µ LIA : v 1 0 v 5 = v 4 1 v 1 1 v 3 = 0 v 2 v 6 v 4 = 1 v 2 v (v 1 = v 4 ) v 1 = v 3 v 5 = v 6 (v 2 = v 4 ) v 2 = v 3 EUF-unsat, C 24 C 13 : (µ C 56 : (µ LIA ) ((v 1 = v 3 ) (v 1 = v 4 )) EUF (v C 23 : (µ C 24 : (µ LIA (v 1 = v 3 )) (v 5 = v 6 ) 5 = v 6 )) ((v 2 = v 3 ) (v 2 = v 4 )) EUF (v 1 = v 3 ) (v 2 = v 3 )) Wednesday 12 th December, /

150 SMT for combinations of theories DTC: example with T -prop. (non-convex theory) µ EUF : (f (v 1 ) = f (v 2 )) (f (v 2 ) = f (v 4 )) f (v 3 ) = v 5 f (v 1 ) = v 6 µ LIA : v 1 0 v 5 = v 4 1 v 1 1 v 3 = 0 v 2 v 6 v 4 = 1 v 2 v (v 1 = v 4 ) v 1 = v 3 v 5 = v 6 (v 2 = v 4 ) v 2 = v 4 v 2 = v 3 EUF-unsat, C 14 C 13 : (µ C 56 : (µ LIA ) ((v 1 = v 3 ) (v 1 = v 4 )) C 23 : (µ EUF (v LIA (v 1 = v 3 )) (v 5 = v 6 ) 5 = v 6 )) ((v 2 = v 3 ) (v 2 = v 4 )) C 24 : (µ EUF (v 1 = v 3 ) (v 2 = v 3 )) C 14 : (µ EUF (v 1 = v 3 ) (v 2 = v 4 )) Wednesday 12 th December, /

151 SMT for combinations of theories DTC: example with T -prop. (non-convex theory) µ EUF : (f (v 1 ) = f (v 2 )) (f (v 2 ) = f (v 4 )) f (v 3 ) = v 5 f (v 1 ) = v 6 µ LIA : v 1 0 v 5 = v 4 1 v 1 1 v 3 = 0 v 2 v 6 v 4 = 1 v 2 v (v 1 = v 4 ) v 1 = v 3 v 5 = v 6 (v 2 = v 4 ) v 2 = v 3 v 1 = v 4 SAT! v 2 = v 4 C 13 : (µ C 56 : (µ LIA ) ((v 1 = v 3 ) (v 1 = v 4 )) C 23 : (µ EUF (v C 24 : (µ LIA (v 1 = v 3 )) (v 5 = v 6 ) 5 = v 6 )) ((v 2 = v 3 ) (v 2 = v 4 )) EUF (v 1 = v 3 ) (v 2 = v 3 )) C 14 : (µ EUF (v 1 = v 3 ) (v 2 = v 4 )) Wednesday 12 th December, /

152 SMT for combinations of theories DTC: example with T -prop. (non-convex theory) µ EUF : (f (v 1 ) = f (v 2 )) (f (v 2 ) = f (v 4 )) f (v 3 ) = v 5 f (v 1 ) = v 6 µ LIA : v 1 0 v 5 = v 4 1 v 1 1 v 3 = 0 v 2 v 6 v 4 = 1 v 2 v (v 1 = v 4 ) v 1 = v 3 v 5 = v 6 (v 2 = v 4 ) v 2 = v 3 v 1 = v 4 SAT! v 2 = v 4 3 e ij -deductions 3 branches C 13 : (µ C 56 : (µ LIA ) ((v 1 = v 3 ) (v 1 = v 4 )) EUF (v C 23 : (µ LIA (v 1 = v 3 )) (v 5 = v 6 ) 5 = v 6 )) ((v 2 = v 3 ) (v 2 = v 4 )) C 24 : (µ EUF (v 1 = v 3 ) (v 2 = v 3 )) C 14 : (µ EUF (v 1 = v 3 ) (v 2 = v 4 )) Wednesday 12 th December, /

153 SMT for combinations of theories DTC: example without T -propagation (convex theory) EUF : (v 3 = h(v 0 )) (v 4 = h(v 1 )) (v 6 = f (v 2 )) (v 7 = f (v 5)) LRA : (v 0 v 1 ) (v 0 v 1 ) (v 2 = v 3 v 4 ) (RESET 5 (v 5 = 0)) Both : ( RESET 5 (v 5 = v 8 )) (v 6 = v 7). µ EUF : {(v 3 = h(v 0)), (v 4 = h(v 1)), (v 6 = v 7), (v 6 = f (v 2)), (v 7 = f (v 5))} µ LRA : {(v 0 v 1), (v 0 v 1), (v 2 = v 3 v 4)} Search for an assignment µ propositionally satisfying ϕ Search on e ij s: check the T 1 T 2- satisfiability of µ (v 2 = v 5) e ij RESET 5 (v 5 = 0) e ij C 67 (v 0 = v 1) (v 3 = v 4) (v 2 = v 5) EUF-unsat (v 3 = v 4) LRA-unsat C 25 (v 0 = v 1) EUF-unsat C 01 : (µ LRA ) (v0 = v1) C 34 C LRA-unsat 34 : (µ EUF (v0 = v1)) (v3 = v4) C 01 C 25 : (µ LRA (v5 = 0) (v3 = v4)) (v2 = v5) C 67 : (µ EUF (v2 = v5)) (v6 = v7)... RESET 5 (v 5 = v 8) Wednesday 12 th December, /

154 SMT for combinations of theories DTC: example with T -propagation (convex theory) EUF : (v 3 = h(v 0 )) (v 4 = h(v 1 )) (v 6 = f (v 2 )) (v 7 = f (v 5)) LRA : (v 0 v 1 ) (v 0 v 1 ) (v 2 = v 3 v 4 ) (RESET 5 (v 5 = 0)) Both : ( RESET 5 (v 5 = v 8 )) (v 6 = v 7). µ LRA : {(v 0 v 1 ), (v 0 v 1 ), (v 2 = v 3 v 4 )} µ EUF : {(v 3 = h(v 0 )), (v 4 = h(v 1 )), (v 6 = v 7 ), (v 6 = f (v 2 )), (v 7 = f (v 5 ))} RESET 5 (v 5 = 0) LRA-deduce (v 0 = v 1 ) learn C (v 0 = v 1 ) 01 EUF-deduce (v 3 = v 4 ) (v 3 = v 4 ) learn C 34 LRA-deduce (v 2 = v 5 ) (v 2 = v 5 ) learn C 25 EUF-unsat C 67 RESET 5 (v 5 = v 8 ) (v 0 = v 1 ) (v 3 = v 4 ) LRA-deduce (v 0 = v 1 ) learn C 01 SAT C 01 : (µ C 34 : (µ LRA ) (v 0 = v 1 ) EUF (v 0 = v 1 )) (v 3 = v 4 ) C 25 : (µ LRA (v 5 = 0) (v 3 = v 4 )) (v 2 = v 5 ) C 67 : (µ EUF (v 2 = v 5 )) (v 6 = v 7 ) Wednesday 12 th December, /

155 SMT for combinations of theories DTC + Model-based heuristic (aka Model-Based Theory Combination) [37] Initially, no interface equalities generated When a model is found, check against all the possible interface equalities If T 1 and T 2 agree on the implied equalities, then return SAT Otherwise, branch on equalities implied by T 1 -model but not by T 2 -model Optimistic approach, similar to axiom instantiation Wednesday 12 th December, /

156 Outline Beyond Solving: advanced SMT functionalities 1 Motivations and goals 2 Efficient SMT solving Combining SAT with Theory Solvers Theory Solvers for theories of interest SMT for combinations of theories 3 Beyond Solving: advanced SMT functionalities Proofs and unsatisfiable cores Interpolants All-SMT & Predicate Abstraction SMT with cost optimization (Optimization Modulo Theories) 4 Conclusions & current research directions Wednesday 12 th December, /

157 Beyond Solving: advanced SMT functionalities Beyond Solving: advanced SAT & SMT functionalities Advanced SMT functionalities (very important in FV): Building proofs of T -unsatisfiability Extracting T -unsatisfiable Cores Computing Craig interpolants Performing All-SMT and Predicate Abstraction Deciding/optimizing SMT problems with costs Wednesday 12 th December, /

158 Outline Beyond Solving: advanced SMT functionalities Proofs and unsatisfiable cores 1 Motivations and goals 2 Efficient SMT solving Combining SAT with Theory Solvers Theory Solvers for theories of interest SMT for combinations of theories 3 Beyond Solving: advanced SMT functionalities Proofs and unsatisfiable cores Interpolants All-SMT & Predicate Abstraction SMT with cost optimization (Optimization Modulo Theories) 4 Conclusions & current research directions Wednesday 12 th December, /

159 Beyond Solving: advanced SMT functionalities Proofs and unsatisfiable cores Building (Resolution) Proofs of T -Unsatisfiability Resolution proof of T -unsatisfiability Very similar to building proofs with plain SAT: resolution proofs whose leaves are original clauses and T -lemmas returned by the T -solver (i.e., T -conflict and T -deduction clauses) built by backward traversal of implication graphs, as in CDCL SAT Sub-proofs of T -lemmas can be built in some T -specific deduction framework if requested Important for: certifying T -unsatisfiability results computing unsatisfiable cores computing interpolants Wednesday 12 th December, /

160 Beyond Solving: advanced SMT functionalities Proofs and unsatisfiable cores Building Proofs of T -Unsatisfiability: example (x = 0 (x = 1) A 1 ) (x = 0 x = 1 A 2 ) ( (x = 0) x = 1 A 2 ) ( A 2 y = 1) ( A 1 x+y > 3) (y < 0) (A 2 x y = 4) (y = 2 A 1 ) (x 0), ( (x = 0) (x = 1))LIA (x = 1 (x = 0) A2) (x = 0 (x = 1) A1) (x = 1 x = 0 A2) ( (x = 0) A2) (x = 0 A1 A2) ( A1 y = 2) (A1 A2) (y = 2 A2) ( (y = 2) (y < 0))LIA (A2 (y < 0)) ( A2 y = 1) ( (y = 1) (y < 0))LIA ( (y < 0) y = 1) (y < 0) ( (y < 0)) relevant original clauses, irrelevant original clauses, T -lemmas Wednesday 12 th December, /

161 Beyond Solving: advanced SMT functionalities Proofs and unsatisfiable cores Example: proof on non-strict LRA inequalities A proof of unsatisfiability for a set of non-strict LRA inequalities can be obtained by building a linear combination of such inequalities, each time eliminating one or more variables, until you get a contradictory inequality on constant values. Example: ϕ def = (0 x 1 3x 2 + 1), (0 x 1 + x 2 ),(0 x 3 2x 1 3), (0 1 2x 3 ). A proof of unsatisfiability P for ϕ is the following: (0 x 1 3x 2 + 1) (0 x 1 + x 2 ) COMB (0 4x 1 + 1) with coeffs 1 and 3 COMB (0 4) with coeffs 1 and 1 (0 x 3 2x 1 3) (0 1 2x 3 ) COMB (0 4x 1 5) with coeffs 2 and 1 It is possible to produce such proof from an inconsistent tableau in Simplex procedure for LRA [30, 32] It is straightforward to produce such proof from a negative cycle in the graph-based procedure for DL [30, 32] Wednesday 12 th December, /

162 Beyond Solving: advanced SMT functionalities Proofs and unsatisfiable cores Example: proof on non-strict LRA inequalities A proof of unsatisfiability for a set of non-strict LRA inequalities can be obtained by building a linear combination of such inequalities, each time eliminating one or more variables, until you get a contradictory inequality on constant values. Example: ϕ def = (0 x 1 3x 2 + 1), (0 x 1 + x 2 ),(0 x 3 2x 1 3), (0 1 2x 3 ). A proof of unsatisfiability P for ϕ is the following: (0 x 1 3x 2 + 1) (0 x 1 + x 2 ) COMB (0 4x 1 + 1) with coeffs 1 and 3 COMB (0 4) with coeffs 1 and 1 (0 x 3 2x 1 3) (0 1 2x 3 ) COMB (0 4x 1 5) with coeffs 2 and 1 It is possible to produce such proof from an inconsistent tableau in Simplex procedure for LRA [30, 32] It is straightforward to produce such proof from a negative cycle in the graph-based procedure for DL [30, 32] Wednesday 12 th December, /

163 Beyond Solving: advanced SMT functionalities Proofs and unsatisfiable cores Example: proof on non-strict LRA inequalities A proof of unsatisfiability for a set of non-strict LRA inequalities can be obtained by building a linear combination of such inequalities, each time eliminating one or more variables, until you get a contradictory inequality on constant values. Example: ϕ def = (0 x 1 3x 2 + 1), (0 x 1 + x 2 ),(0 x 3 2x 1 3), (0 1 2x 3 ). A proof of unsatisfiability P for ϕ is the following: (0 x 1 3x 2 + 1) (0 x 1 + x 2 ) COMB (0 4x 1 + 1) with coeffs 1 and 3 COMB (0 4) with coeffs 1 and 1 (0 x 3 2x 1 3) (0 1 2x 3 ) COMB (0 4x 1 5) with coeffs 2 and 1 It is possible to produce such proof from an inconsistent tableau in Simplex procedure for LRA [30, 32] It is straightforward to produce such proof from a negative cycle in the graph-based procedure for DL [30, 32] Wednesday 12 th December, /

164 Beyond Solving: advanced SMT functionalities Proofs and unsatisfiable cores Example: proof on non-strict LRA inequalities A proof of unsatisfiability for a set of non-strict LRA inequalities can be obtained by building a linear combination of such inequalities, each time eliminating one or more variables, until you get a contradictory inequality on constant values. Example: ϕ def = (0 x 1 3x 2 + 1), (0 x 1 + x 2 ),(0 x 3 2x 1 3), (0 1 2x 3 ). A proof of unsatisfiability P for ϕ is the following: (0 x 1 3x 2 + 1) (0 x 1 + x 2 ) COMB (0 4x 1 + 1) with coeffs 1 and 3 COMB (0 4) with coeffs 1 and 1 (0 x 3 2x 1 3) (0 1 2x 3 ) COMB (0 4x 1 5) with coeffs 2 and 1 It is possible to produce such proof from an inconsistent tableau in Simplex procedure for LRA [30, 32] It is straightforward to produce such proof from a negative cycle in the graph-based procedure for DL [30, 32] Wednesday 12 th December, /

165 Beyond Solving: advanced SMT functionalities Proofs and unsatisfiable cores Extraction of T -unsatisfiable cores The problem Given a T -unsatisfiable set of clauses, extract from it a (possibly small/minimal/minimum) T -unsatisfiable subset (T -unsatisfiable core) wide literature in SAT Some implementations, very few literature for SMT [29, 56] We recognize three approaches: Proof-based approach (CVClite, MathSAT): byproduct of finding a resolution proof Assumption-based approach (Yices): use extra variables labeling clauses, as in the plain Boolean case Lemma-Lifting approach [29] : use an external (possibly-optimized) Boolean unsat-core extractor Wednesday 12 th December, /

166 Beyond Solving: advanced SMT functionalities Proofs and unsatisfiable cores The proof-based approach to T -unsat cores Idea (adapted from [84]) Unsatisfiable core of ϕ: in SAT: the set of leaf clauses of a resolution proof of unsatisfiability of ϕ in SMT(T ): the set of leaf clauses of a resolution proof of T -unsatisfiability of ϕ, minus the T -lemmas Wednesday 12 th December, /

167 Beyond Solving: advanced SMT functionalities Proofs and unsatisfiable cores The proof-based approach to T -unsat cores: example (x = 0 (x = 1) A 1 ) (x = 0 x = 1 A 2 ) ( (x = 0) x = 1 A 2 ) ( A 2 y = 1) ( A 1 x+y > 3) (y < 0) (A 2 x y = 4) (y = 2 A 1 ) (x 0), ( (x = 0) (x = 1)) LIA (x = 1 (x = 0) A 2) (x = 0 (x = 1) A 1) (x = 1 x = 0 A 2) ( (x = 0) A 2) (x = 0 A 1 A 2) ( A 1 y = 2) (A 1 A 2) (y = 2 A 2) ( (y = 2) (y < 0)) LIA (A 2 (y < 0)) ( A 2 y = 1) ( (y = 1) (y < 0)) LIA ( (y < 0) y = 1) (y < 0) ( (y < 0)) Wednesday 12 th December, /

168 Beyond Solving: advanced SMT functionalities Proofs and unsatisfiable cores The assumption-based approach to T -unsat cores Let ϕ be n i=1 C i s.t. ϕ inconsistent. Idea (adapted from [57]) 1 each clause C i in ϕ is substituted by S i C i, s.t. S i fresh selector variable 2 the resulting formula is checked for satisfiability under the assumption of all S i s 3 final conflict clause at dec. level 0: j S j = {C j } j is the unsat core extends straightforwardly to SMT(T ). Wednesday 12 th December, /

169 Beyond Solving: advanced SMT functionalities Proofs and unsatisfiable cores The assumption-based approach to T -unsat cores: Example (S 1 (x = 0 (x = 1) A 1 )) (S 2 (x = 0 x = 1 A 2 )) (S 3 ( (x = 0) x = 1 A 2 )) (S 4 ( A 2 y = 1)) (S 5 ( A 1 x + y > 3)) (S 6 y < 0) (S 7 (A 2 x y = 4)) (S 8 (y = 2 A 1 )) (S 9 x 0) Conflict analysis (Yices 1.0.6) returns: S 1 S 2 S 3 S 4 S 6 S 7 S 8, corresponding to the unsat core in red. Wednesday 12 th December, /

170 Beyond Solving: advanced SMT functionalities Proofs and unsatisfiable cores The lemma-lifting approach to T -unsat cores Idea [29, 33] (i) The T -lemmas D i are valid in T (ii) The conjunction of ϕ with all the T -lemmas D 1,..., D k is propositionally unsatisfiable: T 2B(ϕ n i=1 D i) =. Input clauses: {C 1,..., C n} Result: SAT/UNSAT T -unsat core: {C 1,..., C m} T 2B({C 1,..., C n, D 1,..., D k }) Boolean abstraction: T 2B Lazy_SMT_Solver Stored T -Lemmas: {D 1,..., D k } Boolean_Unsat_Core_Extractor Refinement: B2T T -valid clauses: {D 1,..., D j } Boolean unsat core: T 2B({C 1,..., C m, D 1,..., D j }) interfaces with an external Boolean Unsat-core Extractor = benefits for free of all state-of-the-art size-reduction techniques Wednesday 12 th December, /

171 Beyond Solving: advanced SMT functionalities Proofs and unsatisfiable cores The lemma-lifting approach to T -unsat cores: example (x = 0 (x = 1) A 1 ) (x = 0 x = 1 A 2 ) ( (x = 0) x = 1 A 2 ) ( A 2 y = 1) ( A 1 x+y > 3) (y < 0) (A 2 x y = 4) (y = 2 A 1 ) (x 0), 1 The SMT solver generates the following set of LIA-lemmas: {( (x = 1) (x = 0)), ( (y = 2) (y < 0)), ( (y = 1) (y < 0))}. 2 The following formula is passed to the external Boolean core extractor (B 0 B 1 A 1 ) (B 0 B 1 A 2 ) ( B 0 B 1 A 2 ) ( A 2 B 2 ) ( A 1 B 3 ) B 4 (A 2 B 5) (B 6 A 1 ) B 7 ( B 1 B 0 ) ( B 6 B 4 ) ( B 2 B 4 ) which returns the unsat core in red. 3 The unsat-core is mapped back, the three T -lemmas are removed = the final T -unsat core (in red above). Wednesday 12 th December, /

172 Outline Beyond Solving: advanced SMT functionalities Interpolants 1 Motivations and goals 2 Efficient SMT solving Combining SAT with Theory Solvers Theory Solvers for theories of interest SMT for combinations of theories 3 Beyond Solving: advanced SMT functionalities Proofs and unsatisfiable cores Interpolants All-SMT & Predicate Abstraction SMT with cost optimization (Optimization Modulo Theories) 4 Conclusions & current research directions Wednesday 12 th December, /

173 Beyond Solving: advanced SMT functionalities Interpolants Computing (Craig) Interpolants in SMT Craig Interpolant Given an ordered pair (A, B) of formulas such that A B = T, a Craig interpolant is a formula I s.t.: a) A = T I, b) I B = T, c) I A and I B. I A meaning that all uninterpreted (in T ) symbols in I occur in A. Very important in many FV applications A few works presented for various theories: EUF [59, 70], DL [30, 32], UT VPI [31, 32], LRA [59, 70, 30, 32], LIA [51, 18, 48], BV [52],... Wednesday 12 th December, /

174 Beyond Solving: advanced SMT functionalities A General Algorithm Interpolants Algorithm: Interpolant generation for SMT(T ) [68, 59] (i) Generate a resolution proof of T -unsatisfiability P for A B. (ii)... def def (iii) For every original leaf clause C in P, set I C = C B if C A, and I C = if C B. def (iv) For every inner node C of P obtained by resolution from C 1 = p φ 1 and def def def C 2 = p φ 2, set I C = I C1 I C2 if p does not occur in B, and I C = I C1 I C2 otherwise. (v) Output I as an interpolant for (A, B). η \ B [resp. η B ] is the set of literals in η whose atoms do not [resp. do] occur in B. row 2. only takes place where T comes in to play = Reduced to the problem of finding an interpolant for two sets of T -literals (Boolean and T -specific component decoupled) Wednesday 12 th December, /

175 Beyond Solving: advanced SMT functionalities A General Algorithm Interpolants Algorithm: Interpolant generation for SMT(T ) [68, 59] (i) Generate a resolution proof of T -unsatisfiability P for A B. (ii) Foreach T -lemma η in P, generate an interpolant I η for (η \ B, η B). def def (iii) For every original leaf clause C in P, set I C = C B if C A, and I C = if C B. def (iv) For every inner node C of P obtained by resolution from C 1 = p φ 1 and def def def C 2 = p φ 2, set I C = I C1 I C2 if p does not occur in B, and I C = I C1 I C2 otherwise. (v) Output I as an interpolant for (A, B). η \ B [resp. η B ] is the set of literals in η whose atoms do not [resp. do] occur in B. row 2. only takes place where T comes in to play = Reduced to the problem of finding an interpolant for two sets of T -literals (Boolean and T -specific component decoupled) Wednesday 12 th December, /

176 Beyond Solving: advanced SMT functionalities Interpolants Computing Craig Interpolants in SMT: example A def = (B 1 (0 x 1 3x 2 + 1)) (0 x 1 + x 2 ) ( B 2 (0 x 1 + x 2 )) B def = ( (0 x 3 2x 1 3) (0 1 2x 3 )) ( B 1 B 2 ) (B 1 (0 x 3 2x 1 3)) (0 x1 3x2 + 1) (0 x1 + x2) (0 x3 2x1 3) (0 1 2x3) (0 x3 2x1 3) (0 1 2x3) (0 4x1 + 1) (0 x1 3x2 + 1) (0 x1 + x2) (0 x3 2x1 3) B1 (0 x3 2x1 3) (0 4x1 + 1) B1 (0 x1 3x2 + 1) (0 x1 3x2 + 1) (0 x1 + x2) B1 B1 (0 4x1 + 1) (0 x1 + x2) B1 B1 B2 B1 (0 4x1 + 1) (0 x1 + x2) B2 B2 (0 x1 + x2) B1 (0 4x1 + 1) B2 (0 x1 + x2) (0 x1 + x2) (B1 (0 4x1 + 1)) B2 original proof (B1 (0 4x1 + 1)) B2 interpolant proof Wednesday 12 th December, /

177 Beyond Solving: advanced SMT functionalities Interpolants McMillan s algorithm for non-strict LRA inequalities A B def = {(0 x 1 3x 2 + 1), (0 x 1 + x 2 } def = {(0 x 3 2x 1 3), (0 1 2x 3 )}. A proof of unsatisfiability P for A B is the following: (0 x 1 3x 2 + 1) (0 x 1 + x 2 ) COMB (0 4x 1 + 1) with coeffs 1 and 3 COMB (0 4) with coeffs 1 and 1 (0 x 3 2x 1 3) (0 1 2x 3 ) COMB (0 4x 1 5) with coeffs 2 and 1 By replacing inequalities in B with (0 0), we obtain the proof P : (0 x 1 3x 2 + 1) (0 x 1 + x 2 ) (0 0) (0 0) COMB (0 4x 1 + 1) COMB (0 0) COMB (0 4x 1 + 1) Thus, the interpolant obtained is (0 4x 1 + 1). Wednesday 12 th December, /

178 Beyond Solving: advanced SMT functionalities Interpolants Example: interpolation algorithms for difference logic An inference-based algorithm [59] (0 x 1 x 2 + 1) (0 x 2 x 3 ) COMB (0 x 1 x 3 + 1) (0 x 4 x 5 1) COMB (0 x 1 x 3 + x 4 x 5) (0 0) COMB (0 x 1 x 3 + x 4 x 5) (0 0) COMB (0 x 1 x 3 + x 4 x 5) = Interpolant: (0 x 1 x 3 + x 4 x 5 ) (not in DL, and weaker). A graph-based algorithm [30, 32] A { Chord: (0 x 1 x 3 +1) }} { B = {(0 x 1 x 2 + 1), (0 x 2 x 3 ), (0 x 4 x 5 1)} A def B def x 1 x = {(0 x 5 x 1 ), (0 x 3 x 4 1)}. 1 = Interpolant: (0 x 1 x 3 + 1) (0 x 4 x 5 1) (still in DL) x x 3 x 4 Wednesday 12 th December, /

179 Outline Beyond Solving: advanced SMT functionalities All-SMT & Predicate Abstraction 1 Motivations and goals 2 Efficient SMT solving Combining SAT with Theory Solvers Theory Solvers for theories of interest SMT for combinations of theories 3 Beyond Solving: advanced SMT functionalities Proofs and unsatisfiable cores Interpolants All-SMT & Predicate Abstraction SMT with cost optimization (Optimization Modulo Theories) 4 Conclusions & current research directions Wednesday 12 th December, /

180 Beyond Solving: advanced SMT functionalities All-SMT & Predicate Abstraction All-SAT/All-SMT All-SAT: enumerate all truth assignments satisfying ϕ All-SMT: enumerate all T -satisfiable truth assignments propositionally satisfying ϕ All-SMT over an important subset of atoms P = def {P i } i : enumerate all assignments over P which can be extended to T -satisfiable truth assignments propositionally satisfying ϕ = can compute predicate abstraction Algorithms: BCLT [53] each time a T -satisfiable assignment {l 1,..., l n } is found, perform conflict-driven backjumping as if the restricted clause ( i l i) P belonged to the clause set MathSAT/NuSMV [26] As above, plus the Boolean search of the SMT solver is driven by an OBDD. Wednesday 12 th December, /

181 Beyond Solving: advanced SMT functionalities Predicate Abstraction All-SMT & Predicate Abstraction Predicate abstraction if ϕ(v) is a SMT formula over the domain variables v = def {v j } j, {γ i } i is a set of relevant predicates over v, and P = def {P i } i a set of Boolean labels, then: PredAbs P (ϕ) def = v.( ϕ(v) i P i γ i (v) ) = { µ µ truth assignment on P s.t. µ ϕ i (P i γ i ) is T -satisfiable } projection of ϕ over (the Boolean abstraction of) the set {γ i } i. essential step in FV: extracts finite-state abstractions from a infinite state space Wednesday 12 th December, /

182 Beyond Solving: advanced SMT functionalities All-SMT & Predicate Abstraction Predicate Abstraction: example ϕ γ 1 γ 2 def = (v 1 + v 2 > 12) def = (v 1 + v 2 = 2) def = (v 1 v 2 < 10) PreAbs(ϕ) {P1,P 2 } def = v 1 v 2. (v 1 + v 2 > 12) (P 1 (v 1 + v 2 = 2)) (P 2 (v 1 v 2 < 10)) = ( P 1 P 2 ) ( P 1 P 2 ) = P 1. Wednesday 12 th December, /

183 Outline Beyond Solving: advanced SMT functionalities SMT with cost optimization (Optimization Modulo Theories) 1 Motivations and goals 2 Efficient SMT solving Combining SAT with Theory Solvers Theory Solvers for theories of interest SMT for combinations of theories 3 Beyond Solving: advanced SMT functionalities Proofs and unsatisfiable cores Interpolants All-SMT & Predicate Abstraction SMT with cost optimization (Optimization Modulo Theories) 4 Conclusions & current research directions Wednesday 12 th December, /

184 Beyond Solving: advanced SMT functionalities SMT with cost optimization (Optimization Modulo Theories) SMT with Pseudo-Boolean (PB) cost-minimization The problem SMT(T ) problem ϕ for some T, augmented with cost functions: cost i = N i j=1 ite(pij, c ij 1, cij 2 ), s.t. costi (l i, u i ], c ij {1,2} > 0 Decision problem: is there a model complying with cost ranges? Optimization problem: find model minimizing some cost i. allows for encoding MaxSAT/MaxSMT and PseudoBoolean Proposed solution: [66, 27] SMT(T C), C is an ad-hoc theory of costs a specialized very-fast theory-solver for C added to MathSAT very fast & aggressive search pruning and theory-propagation cost minimization handled by linear or binary search Wednesday 12 th December, /

185 Beyond Solving: advanced SMT functionalities SMT with cost optimization (Optimization Modulo Theories) SMT with Pseudo-Boolean (PB) cost-minimization The problem SMT(T ) problem ϕ for some T, augmented with cost functions: cost i = N i j=1 ite(pij, c ij 1, cij 2 ), s.t. costi (l i, u i ], c ij {1,2} > 0 Decision problem: is there a model complying with cost ranges? Optimization problem: find model minimizing some cost i. allows for encoding MaxSAT/MaxSMT and PseudoBoolean Proposed solution: [66, 27] SMT(T C), C is an ad-hoc theory of costs a specialized very-fast theory-solver for C added to MathSAT very fast & aggressive search pruning and theory-propagation cost minimization handled by linear or binary search Wednesday 12 th December, /

186 Beyond Solving: advanced SMT functionalities SMT(T C): main ideas SMT with cost optimization (Optimization Modulo Theories) A theory of costs C: Cost variables cost i bound cost BC(cost i, k): cost i k" incur cost IC(cost i, j, kj i): the jth addend of cost i := kj i cost i = N i j=1 ite(pi j, k j i i, 0), s.t. cost (l i, u i ] encoded as BC(cost i, l i ) BC(cost i, u i ) N i j=1 (Pi j IC(cost i, j, kj i)) very-fast theory solver: C-solver 1. IC(cost i, j, kj i) = = costi = cost i + kj i 2. cost i > ub i = conflict 3. cost i + {total cost of all unassigned IC s} lb i = conflict 4. IC(cost i, j, kj i) = causes 2. = C-propagate IC(costi, j, kj i) 5. IC(cost i, j, kj i) = causes 3. = C-propagate IC(costi, j, kj i) no symbol shared with T = independent theory solvers for T and C Wednesday 12 th December, /

187 Beyond Solving: advanced SMT functionalities SMT with cost optimization (Optimization Modulo Theories) Optimization Modulo Theories with LA costs I Ingredients an SMT formula ϕ on LA T LA can be LRA, LIA or a combination of both T def = i T i, possibly empty LA and T i disjoint Nelson-Oppen theories a LA variable [term] cost occurring in ϕ (optionally) two constant numbers lb (lower bound) and ub (upper bound) s.t. lb cost < ub (lb, ub may be ) Optimization Modulo Theories with LA costs (OMT(LA T ) ) Find a model for ϕ whose value of cost is minimum. maximization dual Wednesday 12 th December, /

188 Beyond Solving: advanced SMT functionalities SMT with cost optimization (Optimization Modulo Theories) Optimization Modulo Theories with LA costs II We restrict to the case LA = LRA and i T i = {} (OMT(LRA) ). Basic idea [72]: SMT(LRA) augmented with a LP optimization routine: once each assignment µ is found LRA-satisfiable, an LP optimization is invoked, finding the minimum min (cost < min) is learned the search proceeds, until UNSAT = the latest value of min is returned Wednesday 12 th December, /

189 Beyond Solving: advanced SMT functionalities SMT with cost optimization (Optimization Modulo Theories) Optimization Modulo Theories with LA costs III Extensions both linear and binary search, and combination [72, 73] cost minimization embedded inside the CDCL search [72, 73] combination with other theories: OMT(LRA T ) via DTC [73] extension to integers via ILP techniques: OMT(LIA T ) [13, 76, 54] extension to multiple independent objectives [55, 13, 76] incremental OMT [13, 76] other combinations of objectives (min-max, lexicograpohic) [13, 76] OMT with Pareto fronts [13]. Wednesday 12 th December, /

190 Beyond Solving: advanced SMT functionalities A toy example (linear search) [w. pure-literal filt. = partial assignments] OMT(LRA) problem: ϕ def = ( A 1 (2x + y 2)) ( A 1 (x + y 3)) ( A 2 (4x y 4)) ( A 2 (2x y 6)) (cost < 0.2) (cost < 1.0) (cost < 2.0) cost def = x A 1, A 1, A 2, A 2, (4x y 4), (x + y 3), (2x + y 2), µ = (2x y 6) (cost < 0.2) (cost < 1.0) (cost < 2.0) SMT with cost optimization (Optimization Modulo Theories) (2x + y 2) (2x y 6) (x + y 3) (cost < 0.2) (cost < 1.0) (cost < 2.0) y (4x y 4) x Wednesday 12 th December, /

191 Beyond Solving: advanced SMT functionalities A toy example (linear search) [w. pure-literal filt. = partial assignments] OMT(LRA) problem: ϕ def = ( A 1 (2x + y 2)) ( A 1 (x + y 3)) ( A 2 (4x y 4)) ( A 2 (2x y 6)) (cost < 0.2) (cost < 1.0) (cost < 2.0) cost def = x A 1, A 1, A 2, A 2, (4x y 4), (x + y 3), (2x + y 2), µ = (2x y 6) (cost < 0.2) (cost < 1.0) (cost < 2.0) = SAT, min = 0.2 SMT with cost optimization (Optimization Modulo Theories) (2x + y 2) (2x y 6) (x + y 3) (cost < 0.2) (cost < 1.0) (cost < 2.0) y (4x y 4) Wednesday 12 th December, / x

192 Beyond Solving: advanced SMT functionalities A toy example (linear search) [w. pure-literal filt. = partial assignments] OMT(LRA) problem: ϕ def = ( A 1 (2x + y 2)) ( A 1 (x + y 3)) ( A 2 (4x y 4)) ( A 2 (2x y 6)) (cost < 0.2) (cost < 1.0) (cost < 2.0) cost def = x A 1, A 1, A 2, A 2, (4x y 4), (x + y 3), (2x + y 2), µ = (2x y 6) (cost < 0.2) (cost < 1.0) (cost < 2.0) = SAT, min = 1.0 SMT with cost optimization (Optimization Modulo Theories) (2x + y 2) (2x y 6) (cost < 2.0) (x + y 3) (cost (cost < 1.0) < 0.2) y (4x y 4) Wednesday 12 th December, / x

193 Beyond Solving: advanced SMT functionalities A toy example (linear search) [w. pure-literal filt. = partial assignments] OMT(LRA) problem: ϕ def = ( A 1 (2x + y 2)) ( A 1 (x + y 3)) ( A 2 (4x y 4)) ( A 2 (2x y 6)) (cost < 0.2) (cost < 1.0) (cost < 2.0) cost def = x A 1, A 1, A 2, A 2, (4x y 4), (x + y 3), (2x + y 2), µ = (2x y 6) (cost < 0.2) (cost < 1.0) (cost < 2.0) = SAT, min = 2.0 SMT with cost optimization (Optimization Modulo Theories) (2x + y 2) (2x y 6) (x + y 3) (cost < 0.2) (cost < 1.0) (cost < 2.0) y (4x y 4) Wednesday 12 th December, / x

194 Beyond Solving: advanced SMT functionalities A toy example (linear search) [w. pure-literal filt. = partial assignments] OMT(LRA) problem: ϕ def = ( A 1 (2x + y 2)) ( A 1 (x + y 3)) ( A 2 (4x y 4)) ( A 2 (2x y 6)) (cost < 0.2) (cost < 1.0) (cost < 2.0) cost def = x A 1, A 1, A 2, A 2, (4x y 4), (x + y 3), (2x + y 2), µ = (2x y 6) (cost < 0.2) (cost < 1.0) (cost < 2.0) = UNSAT, min = 2.0 SMT with cost optimization (Optimization Modulo Theories) (2x + y 2) (2x y 6) (x + y 3) (cost < 0.2) (cost < 1.0) (cost < 2.0) y (4x y 4) Wednesday 12 th December, / x

195 Beyond Solving: advanced SMT functionalities SMT with cost optimization (Optimization Modulo Theories) OMT with Independent Objectives (aka Boxed OMT) [55, 76] The problem: ϕ, {cost 1,..., cost k } [55] Given ϕ, C s.t.: ϕ is the input formula C = def {cost 1,..., cost k } is a set of LA-terms on variables in ϕ, ϕ, C is the problem of finding a set of independent LA-models M 1,..., M k s.t. s.t. each M i makes cost i minimum. Notes derives from SW verification problems [55] equivalent to k independent problems ϕ, cost 1,..., ϕ, cost k intuition: share search effort for the different objectives generalizes to OMT(LA T ) straightforwardly Wednesday 12 th December, /

196 Beyond Solving: advanced SMT functionalities SMT with cost optimization (Optimization Modulo Theories) OMT with Multiple Objectives [55, 13, 76] Solution Intuition: when a T -consistent satisfying assignment µ is found, foreach cost i min i := min{min i, T solver.minimize(µ, cost i )}; learn i (cost i < min i ); // (cost i < ) proceed until UNSAT; Notice: for each µ, guaranteed improvement of at least one min i in practice, for each µ, multiple cost i minima are improved Implemented improvements: (a) drop previous clauses i (cost i < min i ) (b) (cost i < min i ) pushed in µ first: if T -inconsistent, skip minimization (c) learn (cost i < min i ) (cost i < mini old ), s.t. mini old previous min i = reuse previously-learned clauses like (cost i < mini old ) C Wednesday 12 th December, /

197 Beyond Solving: advanced SMT functionalities Boxed OMT: Example [55, 76] SMT with cost optimization (Optimization Modulo Theories) cost µ 1 µ 2 cost ϕ = (1 y) (y 3) (((1 x) (x 3)) (x 4)) (cost 1 = y) (cost 2 = x y) µ 1 = {(1 y), (y 3), (1 x), (x 3)} = SAT = [ 3, 6] = learn {(cost 1 < 3) (cost 2 < 6)} µ 2 = {(1 y), (y 3), (x 4)} = SAT = [ 3, ] = learn {(cost 1 < 3)} = UNSAT Wednesday 12 th December, /

198 Beyond Solving: advanced SMT functionalities SMT with cost optimization (Optimization Modulo Theories) OMT with Lexicographic Combination of Objectives [13] The problem Find one optimal model M minimizing c = def cost 1, cost 2,..., cost k lexicographically. Solution Intuition: {minimize cost 1 } when UNSAT {substitute unit clause (cost 1 < min 1 ) with (cost 1 = min 1 )} {minimize cost 2 }... improvement: each time UNSAT is found, add i (cost i M i (cost i )) to ϕ Wednesday 12 th December, /

199 Beyond Solving: advanced SMT functionalities SMT with cost optimization (Optimization Modulo Theories) Optimization problems encoded into OMT(LA T ) I SMT with Pseudo-Boolean Constraints & Weighted MaxSMT OMT + PB : j w j A j, w i > 0 //( j ite(a j, w j, 0)) j x j, x j fresh s.t.... j (A j (x j = w j )) ( A j (x j = 0)) (x j 0) (x j w j ) MaxSMT : ϕ h, j ψ j s.t. ψ j soft, w j = weight(ψ j ), w i > 0 minimize j x j, x j, A j fresh ϕ h j (A j ψ j ) j ( A j (x j = w j )) (A j (x j = 0) (x j 0) (x j w j ) Wednesday 12 th December, /

200 Beyond Solving: advanced SMT functionalities SMT with cost optimization (Optimization Modulo Theories) Remark: range constraints (x j 0) (x j w j ) OMT + PB : j w j A j, w i > 0 //( j ite(a j, w j, 0)) j x j, x j fresh s.t.... j (A j (x j = w j )) ( A j (x j = 0)) (x j 0) (x j w j ) Range constraints (x j 0) (x j w j ) logically redundant, but essential for efficiency: Without range constraints, the SMT solver can detect the violation of a bound only after all A i s are assigned : Ex: w 1 = 4, w 2 = 7, i=1 x i < 10, A 1 = A 2 =, A i = i > 2. With range constraints, the SMT solver detects the violation as soon as the assigned A i s violate a bound = drastic pruning of the search same for weighted MaxSMT Wednesday 12 th December, /

201 Beyond Solving: advanced SMT functionalities SMT with cost optimization (Optimization Modulo Theories) Remark: range constraints (x j 0) (x j w j ) OMT + PB : j w j A j, w i > 0 //( j ite(a j, w j, 0)) j x j, x j fresh s.t.... j (A j (x j = w j )) ( A j (x j = 0)) (x j 0) (x j w j ) Range constraints (x j 0) (x j w j ) logically redundant, but essential for efficiency: Without range constraints, the SMT solver can detect the violation of a bound only after all A i s are assigned : Ex: w 1 = 4, w 2 = 7, i=1 x i < 10, A 1 = A 2 =, A i = i > 2. With range constraints, the SMT solver detects the violation as soon as the assigned A i s violate a bound = drastic pruning of the search same for weighted MaxSMT Wednesday 12 th December, /

202 Beyond Solving: advanced SMT functionalities SMT with cost optimization (Optimization Modulo Theories) Remark: range constraints (x j 0) (x j w j ) OMT + PB : j w j A j, w i > 0 //( j ite(a j, w j, 0)) j x j, x j fresh s.t.... j (A j (x j = w j )) ( A j (x j = 0)) (x j 0) (x j w j ) Range constraints (x j 0) (x j w j ) logically redundant, but essential for efficiency: Without range constraints, the SMT solver can detect the violation of a bound only after all A i s are assigned : Ex: w 1 = 4, w 2 = 7, i=1 x i < 10, A 1 = A 2 =, A i = i > 2. With range constraints, the SMT solver detects the violation as soon as the assigned A i s violate a bound = drastic pruning of the search same for weighted MaxSMT Wednesday 12 th December, /

203 Beyond Solving: advanced SMT functionalities SMT with cost optimization (Optimization Modulo Theories) Remark: range constraints (x j 0) (x j w j ) OMT + PB : j w j A j, w i > 0 //( j ite(a j, w j, 0)) j x j, x j fresh s.t.... j (A j (x j = w j )) ( A j (x j = 0)) (x j 0) (x j w j ) Range constraints (x j 0) (x j w j ) logically redundant, but essential for efficiency: Without range constraints, the SMT solver can detect the violation of a bound only after all A i s are assigned : Ex: w 1 = 4, w 2 = 7, i=1 x i < 10, A 1 = A 2 =, A i = i > 2. With range constraints, the SMT solver detects the violation as soon as the assigned A i s violate a bound = drastic pruning of the search same for weighted MaxSMT Wednesday 12 th December, /

204 Beyond Solving: advanced SMT functionalities SMT with cost optimization (Optimization Modulo Theories) Optimization problems encoded into OMT(LA T ) II OMT with Min-Max [Max-Min] optimization Given ϕ, {cost 1,..., cost k }, find a solution which minimizes the maximum value among {cost 1,..., cost k }. (Max-Min dual.) Frequent in some applications (e.g. [74, 81]) = encode into OMT(LA T ) problem {ϕ i (cost i cost), cost} s.t. cost fresh. OMT with linear combinations of costs Given ϕ, {cost 1,..., cost k } and a set of weights {w 1,..., w k }, find a solution which minimizes i w i cost i. = encode into OMT(LA T ) problem {ϕ (cost = i w i cost i ), cost} s.t. cost fresh. These objectives can be composed with other OMT(LA) objectives. Wednesday 12 th December, /

205 Beyond Solving: advanced SMT functionalities SMT with cost optimization (Optimization Modulo Theories) Other OMT Functionalities [hints] Incremental interface [13, 76] Allows for pushing/popping sub-formulas into a stack, and then run OMT incrementally over them, reusing previous search. useful in some applications (e.g., BMC with parametric systems) straightforward variant of incremental SAT and SMT solvers Pareto Fronts [13, 12] Given cost 1, cost 2, compute M 1,..., M i,..., M j,... s.t.: either M i (cost 1 ) > M j (cost 1 ) or M i (cost 2 ) > M j (cost 2 ) and M i (cost 1 ) < M j (cost 1 ) or M i (cost 2 ) < M j (cost 2 ) for each M i, no M dominates M i no objective can be improved without degrading some other one Wednesday 12 th December, /

206 Beyond Solving: advanced SMT functionalities SMT with cost optimization (Optimization Modulo Theories) Some OMT tools BCLT [66, 54] OPTIMATHSAT [72, 74, 76, 75], on top of MATHSAT [28] SYMBA [55], on top of Z3 [38] νz [13, 12], on top of Z3 [38] Wednesday 12 th December, /

207 Outline Conclusions & current research directions 1 Motivations and goals 2 Efficient SMT solving Combining SAT with Theory Solvers Theory Solvers for theories of interest SMT for combinations of theories 3 Beyond Solving: advanced SMT functionalities Proofs and unsatisfiable cores Interpolants All-SMT & Predicate Abstraction SMT with cost optimization (Optimization Modulo Theories) 4 Conclusions & current research directions Wednesday 12 th December, /

208 Conclusions & current research directions Conclusions SMT very popular, due to successful application in many domains Combines techniques from SAT, ATP and operational research Not only satisfiability, but also advanced functionalities Wednesday 12 th December, /

209 Conclusions & current research directions Open/ongoing research directions Solving: improve efficiency (e.g. BV, AR, LIA & their combinations) a never-ending fight against the search-space explosion problem [E. Clarke, Turing-award winner 2007] develop efficient solvers for other theories (N LA(R), N LA(Z)) develop new theories & solvers (e.g., floating-point arithmetic)... Functionalities Interpolation in some theories (LIA, BV) still very challenging Predicate abstraction (AllSMT) still a bottleneck in SMT-based FV SMT with costs/optimization still in very early stage... Combination of SMT solvers and ATP (SMT with quantifiers) Integration & customization of SMT solvers with (FV) tools See also [67] Wednesday 12 th December, /

210 Links I Links & References survey papers: Roberto Sebastiani: "Lazy Satisfiability Modulo Theories". Journal on Satisfiability, Boolean Modeling and Computation, JSAT. Vol. 3, Pag , c IOS Press. Clark Barrett, Roberto Sebastiani, Sanjit Seshia, Cesare Tinelli "Satisfiability Modulo Theories". Part II, Chapter 26, The Handbook of Satisfiability c IOS press. Leonardo de Moura and Nikolaj Bjørner. Satisfiability modulo theories: introduction and applications. Communications of the ACM, 54 (9), c ACM press. web links: The SMT library SMT-LIB: The SMT Competition SMT-COMP: The SAT/SMT Schools Wednesday 12 th December, /

211 References I Links & References [1] A. Armando. Simplifying OBDDs in Decidable Theories. In Proc. PDPAR 03., [2] A. Armando, C. Castellini, and E. Giunchiglia. SAT-based procedures for temporal reasoning. In Proc. European Conference on Planning, CP-99, [3] G. Audemard, P. Bertoli, A. Cimatti, A. Korniłowicz, and R. Sebastiani. A SAT Based Approach for Solving Formulas over Boolean and Linear Mathematical Propositions. In Proc. CADE 2002., volume 2392 of LNAI. Springer, July [4] G. Audemard, P. Bertoli, A. Cimatti, A. Korniłowicz, and R. Sebastiani. Integrating Boolean and Mathematical Solving: Foundations, Basic Algorithms and Requirements. In Proc. AIARSC 2002, volume 2385 of LNAI. Springer, [5] G. Audemard, M. Bozzano, A. Cimatti, and R. Sebastiani. Verifying Industrial Hybrid Systems with MathSAT. In Proc. PDPAR 03, [6] G. Audemard, A. Cimatti, A. Korniłowicz, and R. Sebastiani. SAT-Based Bounded Model Checking for Timed Systems. In Proc. FORTE 02., volume 2529 of LNCS. Springer, November [7] C. Barret, D. Dill, and A. Stump. A Generalization of Shostak s Method for Combining Decision Procedures. In Proc. FROCOS 02, [8] C. Barrett, D. Dill, and A. Stump. Checking Satisfiability of First-Order Formulas by Incremental Translation to SAT. In 14th International Conference on Computer-Aided Verification, Wednesday 12 th December, /

212 References II Links & References [9] C. Barrett, R. Nieuwenhuis, A. Oliveras, and C. Tinelli. Splitting on Demand in SAT Modulo Theories. In Proc. LPAR 06, volume 4246 of LNAI. Springer, [10] C. W. Barrett, R. Sebastiani, S. A. Seshia, and C. Tinelli. Satisfiability Modulo Theories. In Handbook of Satisfiability, chapter 26, pages IOS Press, [11] P. Baumgartner. FDPLL - A First Order Davis-Putnam-Longeman-Loveland Procedure. In Proceedings of CADE-17, pages Springer-Verlag, [12] N. Bjørner, A. Phan, and L. Fleckenstein. νz - an optimizing SMT solver. In Tools and Algorithms for the Construction and Analysis of Systems - 21st International Conference, TACAS 2015, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2015, London, UK, April 11-18, Proceedings, pages , [13] N. Bjorner and A.-D. Phan. νz - Maximal Satisfaction with Z3. In Proc International Symposium on Symbolic Computation in Software Science, Gammart, Tunisia, December EasyChair Proceedings in Computing (EPiC). [14] M. Bozzano, R. Bruttomesso, A. Cimatti, T. Junttila, P. van Rossum, S. Ranise, and R. Sebastiani. Efficient Satisfiability Modulo Theories via Boolean Search. Information and Computation, [15] M. Bozzano, R. Bruttomesso, A. Cimatti, T. Junttila, P. van Rossum, S. Ranise, and R. Sebastiani. Efficient Satisfiability Modulo Theories via Delayed Theory Combination. In Proc. CAV 2005, volume 3576 of LNCS. Springer, Wednesday 12 th December, /

213 References III Links & References [16] M. Bozzano, R. Bruttomesso, A. Cimatti, T. A. Junttila, S. Ranise, P. van Rossum, and R. Sebastiani. Efficient Theory Combination via Boolean Search. Information and Computation, 204(10): , [17] M. Bozzano, R. Bruttomesso, A. Cimatti, T. A. Junttila, P. van Rossum, S. Schulz, and R. Sebastiani. Mathsat: Tight integration of sat and mathematical decision procedures. Journal of Automated Reasoning, 35(1-3): , [18] A. Brillout, D. Kroening, P. Rümmer, and T. Wahl. An Interpolating Sequent Calculus for Quantifier-Free Presburger Arithmetic. In Proc. IJCAR, volume 6173 of LNCS. Springer, [19] R. Brinkmann and R. Drechsler. RTL-datapath verification using integer linear programming. In Proc. ASP-DAC 2002, pages IEEE, [20] R. Brummaryer and A. Biere. Lemmas on Demand for the Extensional Theory of Arrays. Journal on Satisfiability, Boolean Modeling and Computation (JSAT), 6, [21] R. Brummayer and A. Biere. Boolector: An efficient smt solver for bit-vectors and arrays. In TACAS, volume 5505 of LNCS, pages Springer, [22] R. Bruttomesso, A. Cimatti, A. Franzén, A. Griggio, Z. Hanna, A. Nadel, A. Palti, and R. Sebastiani. A Lazy and Layered SMT(BV) Solver for Hard Industrial Verification Problems. In CAV, volume 4590 of LNCS, pages Springer, [23] R. Bruttomesso, A. Cimatti, A. Franzén, A. Griggio, and R. Sebastiani. Delayed Theory Combination vs. Nelson-Oppen for Satisfiability Modulo Theories: A Comparative Analysis. In Proc. LPAR, volume 4246 of LNCS. Springer, Wednesday 12 th December, /

214 References IV Links & References [24] R. Bruttomesso, A. Cimatti, A. Franzén, A. Griggio, and R. Sebastiani. Delayed Theory Combination vs. Nelson-Oppen for Satisfiability Modulo Theories: A Comparative Analysis.. Annals of Mathematics and Artificial Intelligence., 55(1-2), [25] J. R. Burch and D. L. Dill. Automatic Verification of Pipelined Microprocessor Control. In Proc. CAV 94, volume 818 of LNCS. Springer, [26] R. Cavada, A. Cimatti, A. Franzén, K. Kalyanasundaram, M. Roveri, and R. K. Shyamasundar. Computing Predicate Abstractions by Integrating BDDs and SMT Solvers. In FMCAD, pages IEEE Computer Society, [27] A. Cimatti, A. Franzén, A. Griggio, R. Sebastiani, and C. Stenico. Satisfiability modulo the theory of costs: Foundations and applications. In TACAS, volume 6015 of LNCS, pages Springer, [28] A. Cimatti, A. Griggio, B. J. Schaafsma, and R. Sebastiani. The MathSAT 5 SMT Solver. In Tools and Algorithms for the Construction and Analysis of Systems, TACAS 13., volume 7795 of LNCS, pages Springer, [29] A. Cimatti, A. Griggio, and R. Sebastiani. A Simple and Flexible Way of Computing Small Unsatisfiable Cores in SAT Modulo Theories. In SAT, volume 4501 of LNCS, pages Springer, [30] A. Cimatti, A. Griggio, and R. Sebastiani. Efficient Interpolant Generation in Satisfiability Modulo Theories. In Tools and Algorithms for the Construction and Analysis of Systems, TACAS 08., volume 4963 of LNCS. Springer, Wednesday 12 th December, /

215 References V Links & References [31] A. Cimatti, A. Griggio, and R. Sebastiani. Interpolant Generation for UTVPI. In CADE, volume 5663 of LNCS, pages , [32] A. Cimatti, A. Griggio, and R. Sebastiani. Efficient Generation of Craig Interpolants in Satisfiability Modulo Theories. ACM Transaction on Computational Logics TOCL, 12(1), October [33] A. Cimatti, A. Griggio, and R. Sebastiani. Computing Small Unsatisfiable Cores in SAT Modulo Theories. Journal of Artificial Intelligence Research, JAIR, 40: , April [34] S. Cotton and O. Maler. Fast and Flexible Difference Logic Propagation for DPLL(T). In Proc. SAT 06, volume 4121 of LNCS. Springer, [35] L. de Moura and N. Bjørner. Efficient E-matching for SMT solvers. In Proc. CADE-21, 21st International Conference on Automated Deduction, volume 4603 of LNCS. Springer, [36] L. de Moura, H. Ruess, and M. Sorea. Lazy Theorem Proving for Bounded Model Checking over Infinite Domains. In Proc. CADE 2002., volume 2392 of LNAI. Springer, July [37] L. M. de Moura and N. Bjørner. Model-based theory combination. Electr. Notes Theor. Comput. Sci., 198(2):37 49, Wednesday 12 th December, /

216 References VI Links & References [38] L. M. de Moura and N. Bjørner. Z3: an efficient SMT solver. In Tools and Algorithms for the Construction and Analysis of Systems, 14th International Conference, TACAS 2008, Held as Part of the Joint European Conferences on Theory and Practice of Software, ETAPS 2008, Budapest, Hungary, March 29-April 6, Proceedings, pages , [39] L. M. de Moura and N. Bjørner. Generalized, efficient array decision procedures. In FMCAD, pages IEEE, [40] D. Detlefs, G. Nelson, and J. Saxe. Simplify: a theorem prover for program checking. Journal of the ACM, 52(3): , [41] B. Dutertre and L. de Moura. System Description: Yices 1.0. In Proc. on 2nd SMT competition, SMT-COMP 06, Available at yices.csl.sri.com/yices-smtcomp06.pdf. [42] A. Franzen, A. Cimatti, A. Nadel, R. Sebastiani, and J. Shalev. Applying SMT in Symbolic Execution of Microcode. In Proc. Int. Conference on Formal Methods in Computer Aided Design (FMCAD 10). IEEE, [43] V. Ganesh and D. L. Dill. A Decision Procedure for Bit-Vectors and Arrays. In CAV, [44] H. Ganzinger, G. Hagen, R. Nieuwenhuis, A. Oliveras, and C. Tinelli. DPLL(T): Fast decision procedures. In R. Alur and D. Peled, editors, Proceedings of the 16th International Conference on Computer Aided Verification, CAV 04 (Boston, Massachusetts), LNCS. Springer, Wednesday 12 th December, /

217 References VII Links & References [45] F. Giunchiglia and R. Sebastiani. Building decision procedures for modal logics from propositional decision procedures - the case study of modal K. In Proc. CADE 13, LNAI, New Brunswick, NJ, USA, August Springer. [46] A. Goel, S. Krstić, and A. Fuchs. Deciding array formulas with frugal axiom instantiation. In Proceedings of SMT 08/BPR 08, pages 12 17, New York, NY, USA, ACM. [47] A. Griggio. A Practical Approach to SMT(LA(Z)). In Proc. SMT 2010, [48] A. Griggio, T. T. H. Le, and R. Sebastiani. Efficient Interpolant Generation in Satisfiability Modulo Linear Integer Arithmetic. In Proc. Tools and Algorithms for the Construction and Analysis of Systems, TACAS 11, LNCS. Springer, [49] A. Griggio, Q. S. Phan, R. Sebastiani, and S. Tomasi. Stochastic Local Search for SMT: Combining Theory Solvers with WalkSAT. In Frontiers of Combining Systems, FroCoS 11, volume 6989 of LNAI. Springer, [50] I. Horrocks and P. F. Patel-Schneider. FaCT and DLP. In Proc. Tableaux 98, pages 27 30, [51] H. Jain, E. M. Clarke, and O. Grumberg. Efficient Craig Interpolation for Linear Diophantine (Dis)Equations and Linear Modular Equations. In A. Gupta and S. Malik, editors, CAV, volume 5123 of Lecture Notes in Computer Science, pages Springer, [52] D. Kroening and G. Weissenbacher. Lifting Propositional Interpolants to the Word-Level. In FMCAD, pages 85 89, Los Alamitos, CA, USA, IEEE Computer Society. Wednesday 12 th December, /

218 References VIII Links & References [53] S. K. Lahiri, R. Nieuwenhuis, and A. Oliveras. SMT techniques for fast predicate abstraction. In Proc. CAV, LNCS Springer, [54] D. Larraz, A. Oliveras, E. Rodríguez-Carbonell, and A. Rubio. Minimal-Model-Guided Approaches to Solving Polynomial Constraints and Extensions. In C. Sinz and U. Egly, editors, SAT, volume 8561 of Lecture Notes in Computer Science, pages Springer, [55] Y. Li, A. Albarghouthi, Z. Kincaid, A. Gurfinkel, and M. Chechik. Symbolic optimization with smt solvers. In POPL, pages , [56] M. Liffiton and K. Sakallah. Algortithms for Computing Minimal Unsatisfiable Subsets of Constraints. Journal of Automated Reasoning, 40(1), [57] I. Lynce and J. P. Marques-Silva. On computing minimum unsatisfiable cores. In SAT, [58] M. Mahfoudh, P. Niebert, E. Asarin, and O. Maler. A Satisfibaility Checker for Difference Logic. In Proceedings of SAT-02, pages , [59] K. L. McMillan. An interpolating theorem prover. Theor. Comput. Sci., 345(1): , [60] J. Moeller, J. Lichtenberg, H. R. Andersen, and H. Hulgaard. Fully symbolic model checking of timed systems using difference decision diagrams. In Proc. Workshop on Symbolic Model Checking (SMC), FLoC 99, Trento, Italy, July Wednesday 12 th December, /

219 References IX Links & References [61] C. G. Nelson and D. C. Oppen. Simplification by cooperating decision procedures. TOPLAS, 1(2): , [62] G. Nelson and D. Oppen. Simplification by Cooperating Decision Procedures. ACM Trans. on Programming Languages and Systems, 1(2): , [63] G. Nelson and D. Oppen. Fast Decision Procedures Based on Congruence Closure. Journal of the ACM, 27(2): , [64] R. Nieuwenhuis and A. Oliveras. Congruence closure with integer offsets. In In 10th Int. Conf. Logic for Programming, Artif. Intell. and Reasoning (LPAR), volume 2850 of LNAI, pages Springer, [65] R. Nieuwenhuis and A. Oliveras. DPLL(T) with Exhaustive Theory Propagation and its Application to Difference Logic. In Proc. CAV 05, volume 3576 of LNCS. Springer, [66] R. Nieuwenhuis and A. Oliveras. On SAT Modulo Theories and Optimization Problems. In Proc. Theory and Applications of Satisfiability Testing - SAT 2006, volume 4121 of LNCS. Springer, [67] R. Nieuwenhuis, A. Oliveras, E. Rodríguez-Carbonell, and A. Rubio. Challenges in Satisfiability Modulo Theories. In Proc. RTA 07, volume 4533 of LNCS. Springer, Wednesday 12 th December, /

220 References X Links & References [68] P. Pudlák. Lower bounds for resolution and cutting planes proofs and monotone computations. J. of Symb. Logic, 62(3), [69] H. Rueßand N. Shankar. Deconstructing Shostak. In Proc. LICS 01. IEEE Computer Society, [70] A. Rybalchenko and V. Sofronie-Stokkermans. Constraint Solving for Interpolation. In Proc. VMCAI, volume 4349 of LNCS. Springer, [71] R. Sebastiani. Lazy Satisfiability Modulo Theories. Journal on Satisfiability, Boolean Modeling and Computation, JSAT, 3(3-4): , [72] R. Sebastiani and S. Tomasi. Optimization in SMT with LA(Q) Cost Functions. In IJCAR, volume 7364 of LNAI, pages Springer, July [73] R. Sebastiani and S. Tomasi. Optimization Modulo Theories with Linear Rational Costs. ACM Transactions on Computational Logics, 16(2), March [74] R. Sebastiani and S. Tomasi. Optimization Modulo Theories with Linear Rational Costs. ACM Transactions on Computational Logics, 16(2), March [75] R. Sebastiani and P. Trentin. OptiMathSAT: A Tool for Optimization Modulo Theories. In Proc. International Conference on Computer-Aided Verification, CAV 2015, volume 9206 of LNCS. Springer, Wednesday 12 th December, /

221 References XI Links & References [76] R. Sebastiani and P. Trentin. Pushing the Envelope of Optimization Modulo Theories with Linear-Arithmetic Cost Functions. In Proc. Int. Conference on Tools and Algorithms for the Construction and Analysis of Systems, TACAS 15, volume 9035 of LNCS. Springer, [77] R. Shostak. A Pratical Decision Procedure for Arithmetic with Function Symbols. Journal of the ACM, 26(2): , [78] R. Shostak. Deciding Combinations of Theories. Journal of the ACM, 31:1 12, [79] K. Stergiou and M. Koubarakis. Backtracking algorithms for disjunctions of temporal constraints. In Proc. AAAI, pages , [80] A. Stump, C. W. Barrett, and D. L. Dill. CVC: A Cooperating Validity Checker. In Proc. CAV 02, number 2404 in LNCS. Springer Verlag, [81] S. Teso, R. Sebastiani, and A. Passerini. Structured learning modulo theories. Artificial Intelligence, 244: , [82] S. Wolfman and D. Weld. The LPSAT Engine & its Application to Resource Planning. In Proc. IJCAI, [83] T. Yavuz-Kahveci, M. Tuncer, and T. Bultan. A Library for Composite Symbolic Representation. In Proc. TACAS2001, volume 2031 of LNCS. Springer Verlag, Wednesday 12 th December, /

222 Links & References References XII [84] L. Zhang and S. Malik. Extracting small unsatisfiable cores from unsatisfiable boolean formula. In Proc. of SAT, Disclaimer The list of references above is by no means intended to be all-inclusive. I apologize both with the authors and with the readers for all the relevant works which are not cited here. Wednesday 12 th December, /

223 Links & References c Warner Bros. Inc. Wednesday 12 th December, 2018 /