Decision Procedures for Verification

Transcription

1 Decision Procedures for Verification Zohar Manna with Aaron R. Bradley Computer Science Department Stanford University 1

2 Motivation int[] BubbleSort(int[] a) { int i, j, t; for (i := a 1; i > 0; i := i 1) { for (j := 0; j < i; j := j + 1) { if (a[j] > a[j + 1]) { t := a[j]; a[j] := a[j + 1]; a[j + 1] := t; } } } return a; } Does BubbleSort return a sorted array? j i j i j i j, i j i 2

3 Motivation l 0 a 0 input specification l f sorted(rv, 0, a 1) output specification int[] BubbleSort(int[] a) { int i, j, t; [ loop assertions ] 1 i < a a = a 0 for l 1 partitioned(a, 0, i, i + 1, a 1) sorted(a, i, a 1) (i := a 1; i > 0; i := i 1) for l 2 1 i < a 0 j i a = a 0 partitioned(a, 0, i, i + 1, a 1) partitioned(a, 0, j 1, j, j) sorted(a, i, a 1) (j := 0; j < i; j := j + 1) if (a[j] > a[j + 1]) { t := a[j]; a[j] := a[j + 1]; a[j + 1] := t; } return a; } Does BubbleSort return a sorted array? Yes! 3

4 Motivation Predicates: sorted(a, l, u): array a is sorted in range [l, u] ( i, j)[l i j u a[i] a[j]] partitioned(a, l 1, u 1, l 2, u 2 ) ( i, j)[l 1 i u 1 < l 2 j u 2 a[i] a[j]] At the top of the inner loop (l 2 ): partitioned(a, 0, j 1, j, j) partitioned(a, 0, i, i + 1, a 1) sorted(a, i, a 1) j i 4

5 Motivation Verification process: Generate verification conditions. Prove that each verification condition is valid. Example: verification condition (from l 2 to l 2, with swapping) 1 i < a 0 j i a = a 0 partitioned(a, 0, i, i + 1, a 1) partitioned(a, 0, j 1, j, j) sorted(a, i, a 1) j < i a[j] > a[j + 1] ( ) 1 i < a 0 j + 1 i a = a 0 partitioned(a{j a[j + 1]}{j + 1 a[j]}, 0, i, i + 1, a 1) partitioned(a{j a[j + 1]}{j + 1 a[j]}, 0, j, j + 1, j + 1) sorted(a{j a[j + 1]}{j + 1 a[j]}, i, a 1) How do we prove that verification conditions are valid? Decision Procedures! 5

6 Outline 0. Introduction 1. Theories 2. Quantifier Elimination (QE) 3. of Integer Linear (Presburger) Arithmetic, T Z 4. of Real Linear Arithmetic, T R 5. Quantifier-free Real Linear Arithmetic, T R 6. Integer Linear (Presburger) Arithmetic, T Z 7. Theory of Equality, T E 8. Theory of Recursive Data Structures, T D 9. Nelson-Oppen Combination 10. Shostak Theories & Combination 11. Theory of Arrays, T A 12. Incorporating DPs in Deductive Systems 6

7 What is a Decision Procedure? Theory T : collection of formulas (axioms) Decision procedure for T : algorithm for deciding whether or not a formula ϕ is satisfiable in T satisfiable ϕ decision procedure unsatisfiable always terminates with the right answer T is said to be a decidable theory 7

8 What is a Decision Procedure? Same algorithm can be used for deciding whether or not a formula ϕ is valid in T : ϕ satisfiable ϕ invalid ϕ decision procedure ϕ unsatisfiable ϕ valid 8

9 Advantages of Decision Procedures Efficiency Using decision procedures is more effective than encoding the axioms of the theory and employing first-order logic reasoning. Availability Decision procedures are available for many useful theories. Applications Decision procedures have been used in theorem proving model checking verification synthesis 9

10 Outline 0. Introduction 1. Theories 2. Quantifier Elimination (QE) 3. of Integer Linear (Presburger) Arithmetic, T Z 4. of Real Linear Arithmetic, T R 5. Quantifier-free Real Linear Arithmetic, T R 6. Integer Linear (Presburger) Arithmetic, T Z 7. Theory of Equality, T E 8. Theory of Recursive Data Structures, T D 9. Nelson-Oppen Combination 10. Shostak Theories & Combination 11. Theory of Arrays, T A 12. Incorporating DPs in Deductive Systems 10

11 Theory of Equality T E (with variables, quantifiers, and logical connectives) Σ = {a, b, c,..., f, g, h,..., p, q, r,..., =} Uninterpreted symbols: signature Constants: a, b, c,... Functions: f, g, h,... Predicates: p, q, r,... Examples: f(f(f(a))) = a f(f(f(f(f(a))))) = a f(a) a T E -unsatisfiable x = y f(x) f(y) T E -unsatisfiable ( x)( y)[x = f(y)] T E -satisfiable 11

12 Theory of Equality T E f(f(f(a))) = a f(f(f(f(f(a))))) = a f(a) a Decision Procedure unsatisfiable ( x)( y)[x = f(y)] Decision Procedure satisfiable 12

13 Theory of Equality T E Axiom schema: ( x)[x = x] (reflexivity) ( x, y)[x = y y = x] (symmetry) ( x, y, z)[x = y y = z x = z] [ ] ( x, y) x i = y i f(x) = f(y) i (transitivity) (congruence) 13

14 Theory of Equality T E Full: undecidable [Church, 36] [Turing, 36] Quantifier-free: decidable [Ackerman, 54] Efficient algorithms based on Congruence Closure (++) [Shostak, 78] [Downey, Sethi and Tarjan, 80] [Nelson and Oppen, 80] 14

15 Theory of Integers T Z (Presburger Arithmetic) Σ = {0, 1, +,, =, <} Domain: Z = {..., 2, 1, 0, 1, 2,...} Constants: n for each n Z Functions: + (addition), (subtraction) Predicates: = (equality), < (comparison) Examples: ( x)( y)[y = 2x] T Z -valid ( x)( y)[x = 2y] T Z -invalid x > 0 y > 0 x + y = 1 T Z -unsatisfiable 15

16 Theory of Integers T Z (Presburger Arithmetic) Full: decidable Quantifier-elimination [Presburger, 29] [Cooper, 72] [Fisher & Rabin, 74] ( ) Quantifier-free: decidable [Papadimitriou, 81] ( ) Omega test [Pugh, 94] (+) Introduce (multiplications): ( x, y, z)[x 3 + y 3 = z 3 ] T Z -unsatisfiable undecidable [Gödel, 31] [Church, 36] even for single quantifier-free equation [Matiyasevich, 70] 16

17 Theory of Reals T R Σ = {0, 1, +,, =, <} Domain: R = {..., 0,..., 3 2,...} Constants: n for each n Z Functions: + (addition), (subtraction) Predicates: = (equality), < (comparison) Examples: ( x)( y)[x = 2y] T R -valid x > 0 y > 0 x + y = 1 T R -satisfiable 17

18 Theory of Reals T R Why are constants n Z? Idea: 3 2 x < 4 y 9x < 8y 3 Why theory of reals? Linear case: cannot express irrational numbers. If there is an irrational solution, then there is a rational solution. Polynomial case: x x = 2 expresses x = 2, which is irrational. 18

19 Theory of Reals T R Full: decidable Quantifier-elimination [Tarski, 51] ( ) Cylindrical algebraic decomposition [Collins, 75] ( ) Quantifier-free: decidable (many methods) Fourier-Motzkin [Lassez & Mahler, 92] (+) Simplex [Dantzig, 61] (++) [Kachiyan, 79] (++), [Karmarker, 84] (++) 19

20 Theory of Reals T R (with Multiplication) Introduce (multiplication): Full: decidable [Tarski, 51] ( ) Cylindrical algebraic decomposition [Collins, 75] ( ) Inherently doubly-exponential [Davenport & Heintz, 88] Remark: If we add ceiling x = min{y : y Z y x} or floor x = max{y : y Z y x} then we are able to encode integers, and we lose decidability. 20

21 Theory of Recursive Data Structures (RDS) T D Parametric theory. Each RDS has n C -ary constructor C(x) n C projection functions πi C one atom predicate atom C Axiom schema: axioms of T E + ( x)[π i (C(x)) = x i ] (projection) ( x)[ atom C (x) C(..., π i (x),...) = x] (construction) ( x)[ atom C (C(x))] (atom) 21

22 Example: List RDS constructor cons projection functions car, cdr atom predicate atom Axiom schema: axioms of T E + ( x, y)[car(cons(x, y)) = x] ( x, y)[cdr(cons(x, y)) = y] ( x)[ atom(x) cons(car(x), cdr(x)) = x] ( x, y)[ atom(cons(x, y))] 22

23 Theory of Recursive Data Structures (RDS) T D Full: decidable ( ) Quantifier-elimination over term algebras [Mal cev, 71] [Hodges, 93] Via pairing functions [Tenney, 72] [Oppen, 80] Not elementary recursive [Tenny, 72] Quantifier-free: decidable [Oppen, 80] (++) if values of projection functions on atoms are defined: NP-complete [Oppen, 80] 23

24 Decidable Domains There is a decision procedure for deciding the validity of sentences over this domain. Examples: Presburger arithmetic real numbers abelian groups dense linear order term algebras boolean algebras. 24

25 Abelian Groups G 1 : ( x)[x e = x] G 2 : ( x)[x x 1 = e] (right identity) (right inverse) G 3 : ( x, y, z) [(x y) z = x (y z)] (associativity) G 4 : ( x, y) [x y = y x] (commutativity) The theory G 1 + G 2 + G 3 + G 4 is decidable. Remark: The theory G 1 + G 2 + G 3 is not decidable. 25

26 Dense Linear Order without Endpoints ( x)[ (x x)] (irreflexivity) ( x, y, z)[x y y z x z] (transitivity) ( x, y)[x y y x x = y] (trichotomy) ( x, y)[x y ( z)[x z z y]] (density) ( x)( y)( z)[y x x z] (without endpoints) The theory of dense linear order without endpoints is decidable. 26

27 Combining Decision Procedures Σ 1 -theory T 1 Σ 2 -theory T 2 P 1 for T 1 -satisfiability P 2 for T 2 -satisfiability P? for (T 1 T 2 )-satisfiability Problem: Decision procedures are domain specific. How do we combine them? Example: 1 x x 2 f(x) f(1) f(x) f(2) (T E T Z )-unsat 27

28 Nelson-Oppen Combination Method Σ 1 Σ 2 = Σ 1 -theory T 1 Σ 2 -theory T 2 stably infinite stably infinite P 1 for T 1 -satisfiability P 2 for T 2 -satisfiability of quantifier-free Σ 1 -formulae of quantifier-free Σ 2 -formulae P for (T 1 T 2 )-satisfiability of quantifier-free (Σ 1 Σ 2 )-formulae 28

29 Theory of Arrays T A Σ = Σ Z Σ elem { [ ], { }} Parameter theory: Element theory T elem Domain: A = {Z elem} Constants: constants of T Z and T elem Functions: functions of T Z and T elem and [ ] (read), { } (write) E.g., read a[5]: value of a[5] write a{0 7}: array with value 7 at 0; otherwise equal to a Predicates: predicates of T Z and T elem 29

30 Example: Theory of Arrays T A sorted(0, 5, a{0 7}{5 9}) sorted(0, 5, a{0 11}{5 13}) where sorted(l, u, a) def = ( i, j)[l i j u a[i] a[j]] T A -unsatisfiable: Impossible! 7 a[1] 9 and 11 a[1] 13 30

31 Theory of Arrays T A Axiom: ( arrays a)( elem e)( i, j Z) i = j a{i e}[j] = e i j a{i e}[j] = a[j] (read-over-write) 31

32 Theory of Arrays T A Satisfiability: Full: undecidable Quantifier-free: decidable read-over-write ([McCarthy, 62]) with combination of theories ([Nelson & Oppen, 79]) Quantifier-free with = (extensional theory): decidable ([Stump, Barrett, Dill & Levitt, 01]) Array property fragment: decidable One alternation of quantifiers, with syntactic constraints ([Bradley, Manna & Sipma 2005]) Assumption: Quantifier-free combination of T E T Z T elem is decidable 32

33 Outline 0. Introduction 1. Theories 2. Quantifier Elimination (QE) 3. of Integer Linear (Presburger) Arithmetic, T Z 4. of Real Linear Arithmetic, T R 5. Quantifier-free Real Linear Arithmetic, T R 6. Integer Linear (Presburger) Arithmetic, T Z 7. Theory of Equality, T E 8. Theory of Recursive Data Structures, T D 9. Nelson-Oppen Combination 10. Shostak Theories & Combination 11. Theory of Arrays, T A 12. Incorporating DPs in Deductive Systems 33

34 Quantifier Elimination (QE) A theory T admits quantifier elimination if there is an algorithm that given arbitrary T -formula ϕ, produces T -formula ψ s.t.: ψ is quantifier-free ϕ ψ (ϕ is equivalent to ψ) So if T admits quantifier elimination, and satisfiability problem of quantifier-free theory of T is decidable, then T is decidable. Example: T : Z + ϕ : ( x)[y = 2x] ψ : 2 y 34

35 QE: Simplification Only consider formula of form ( x)f, for quantifier-free F. Why? Given arbitrary ϕ: Since ( x)f ( x) F, replace ( x)f with ( x) F. Repeat: Choose innermost ( x)f. Apply quantifier elimination to get quantifier-free G s.t. ( x)f G. Replace ( x)f with G. 35

36 QE: Simplification Example: G 1 : ( x)( y) ( z)f 1 (x, y, z) }{{} QE-algorithm G 2 : ( x)( y)f 2 (x, y) G 3 : ( x)( ( y) F 2 (x, y) ) }{{} QE-algorithm G 4 : ( x) F 3 (x) }{{} QE-algorithm true G 5 : false The quantifier-free formula G 5 is equivalent to G 1. 36

37 Outline 0. Introduction 1. Theories 2. Quantifier Elimination (QE) 3. of Integer Linear (Presburger) Arithmetic, T Z 4. of Real Linear Arithmetic, T R 5. Quantifier-free Real Linear Arithmetic, T R 6. Integer Linear (Presburger) Arithmetic, T Z 7. Theory of Equality, T E 8. Theory of Recursive Data Structures, T D 9. Nelson-Oppen Combination 10. Shostak Theories & Combination 11. Theory of Arrays, T A 12. Incorporating DPs in Deductive Systems 37

38 Theory of Integers T Z (Presburger Arithmetic) Σ = {0, 1, +,, =, <} Domain: Z = {..., 2, 1, 0, 1, 2,...} Constants: n for each n Z Functions: + (addition), (subtraction) Predicates: = (equality), < (comparison) Examples: ( x)(y = 2x) T Z -invalid x > 0 y > 0 x + y = 1 T Z -unsatisfiable 38

39 Theory of Integers: T Z Consider ( x)(y = 2x). Quantifier-free equivalent formula? Lemma Given quantifier-free formula ϕ from T Z s.t. free(ϕ) = {y}. ϕ represents some set of integers S Z. Either S Z + or Z + \ S is finite. Proof by structural induction. Example: S = {y : ( x)(y = 2x)} S Z + is set of nonnegative even integers. Z + \ S is set of positive odd integers. Both have infinite cardinality. So no quantifier-free T Z -formula equivalent to ( x)(y = 2x); i.e., T Z does not admit QE. 39

40 Augmented Theory of Integers: T Z Introduce infinite number of divisibility predicates k for k > 0 k x holds iff k divides x without any remainder Augmented theory T Z : theory T Z plus infinite number of axioms: ( x)(k x ( y)(x = ky)) for k > 0 T Z admits QE. 40

41 Algorithm: Quantifier Elimination for Integers, T Z [Cooper, 72] Given ( x)f (x), where F is quantifier free. Step 1 Push negations in F (x) all the way in to atoms. Step 2 Replace Remark All atoms now of form s = t s < t + 1 t < s + 1 (s = t) s < t t < s (s < t) t < s + 1 s < t k t (k t) 41

42 Example: Steps 1 & 2 (x < y) (x = y + 3) y < x + 1 (x < y + 3 y + 3 < x) 42

43 Algorithm: Quantifier Elimination for Integers Step 3 Collect terms containing x so that atoms have form hx < t t < hx k hx + t (k hx + t) where k and h are positive integers, and t is an integer expression that does not contain x. Example: Step 3 x + x + y < z + 3z + 2y 4x 6x < 4z + y 43

44 Algorithm: Quantifier Elimination for Integers Step 4 Let δ = lcm{h : h is coefficient of x in F (x)} Multiply atoms in F (x) by constants so that δ is coefficient of x everywhere: hx < t δ x < h t where h h = δ t < hx h t < δ x where h h = δ k hx + t h k δ x + h t where h h = δ (k hx + t) (h k δ x + h t) where h h = δ Replace ( x)f (δ x) with (δ x x ) ( x )[F (x ) δ x ] 44

45 Example: Step 4 ( x)[3x + 1 > y 2x 6 < z 4 5x + 1] ( x)[2x < z + 6 y 1 < 3x 4 5x + 1] lcm{2, 3, 5} = 30 ( x)[30x < 15z y 10 < 30x 24 30x + 6] δ = 30 ( x )[x < 15z y 10 < x 24 x x ] 45

46 Algorithm: Quantifier Elimination for Integers Remark Atoms now have form (A) x < a i (B) b i < x (C) h i x + c i (D) (k i x + d i ) where a i, b i, c i, d i are integer expressions that do not contain x, and h i, k i are positive integers. 46

47 Algorithm: Quantifier Elimination for Integers Step 5 Obtain F (x) from F (x) by replacing all (A) atoms x < a i by true (B) atoms b i < x by false Let h i of (C) atoms h i x + c i δ = lcm k i of (D) atoms (k i x + d i ) Return F : δ F (j) j=1 }{{} δ F (b i + j) j=1 b i }{{} no least n Z s.t. F (n); (only constraints) there is a least n Z s.t. F (n), and all b i < n are satisfied ( x)f (x) F 47

48 Intuition Property (periodicity) if k δ, then k n iff k n + λδ for λ Z cannot distinguish between k n and k n + λδ Step 5 says that two cases are possible: 1. F (j) There is no least n satisfying F (n): just satisfy constraints. Suppose n Z is s.t. F (n). Then also F (n + λδ), for λ Z. }{{} δ a 2 a 1 a 3 All (infinitely many) arrows point to solutions. 48

49 Intuition 2. F (b i + j) There is a least n satisfying F (n). Larger than all b i. Let b be the largest b i in interpretation. If n Z is s.t. F (n), then ( j [1..δ])[b + j n F (b + j)] b 2 b 1 b 3 }{{} a 3 a 1 δ If there is a solution, then one must appear in δ interval to the right of b. b 49

50 Example 1 Step 4 δ = lcm{2, 3} = 6, so ( x)[3x < 2 1 < 2x] ( x)[ 3 x < 2 1 < 2 x] ( x)[6x < 4 3 < 6x] ( x)[x < 4 3 < x }{{} F (x) 6 x] }{{} δ x 50

51 Example 1 ( x)[x < 4 3 < x 6 x] Step 5 F (x): true }{{} (A): x<4 false }{{} (B): 3<x 6 x false δ = lcm{6} = 6 51

52 Example 1 ( x)[x < 4 3 < x 6 x] Return 6 j=1 false 6 j=1 F (j) 6 j=1 b i {3} F (b i + j) 6 (3 + j < 4 3 < 3 + j 6 (3 + j)) j=1 } {{ } false ( x)[3x < 2 2x > 1] false 52

53 Example 2: Even Integers ( x)[y = 2x] Step 2 Replace y = 2x by y < 2x + 1 2x < y + 1: ( x)[y < 2x + 1 2x < y + 1] Step 3 Isolate x-terms: ( x)[y 1 < 2 x 2 x < y + 1] Step 4 δ = lcm{2, 2} = 2, so ( x)[y 1 < x x < y + 1 }{{} F (x) 2 x] }{{} δ x 53

54 Example 2: Even Integers Step 5 F (x): ( x)[x < y + 1 }{{} (A) y 1 < x }{{} (B) 2 x] true }{{} (A): x<y+1 false }{{} (B): y 1<x 2 x false δ = lcm{2} = 2 54

55 Example 2: Even Integers ( x)[x < y + 1 y 1 < x 2 x] Return false }{{} F (1) 2 j=1 false }{{} F (2) F (j) 2 j=1 b i {y 1} (y < y + 1 y 1 < y 2 y) }{{} b i +j=(y 1)+1=y F (b i + j) (y + 1 < y + 1 y 1 < y y + 1) }{{} b i +j=(y 1)+2=y+1 ( x)[y = 2x] 2 y 2 y 55

56 Example 3 Step 3 Isolate x-terms: ( x)[3x + 1 < 10 7x 6 > 7 2 x] Step 4 δ = lcm{3, 7} = 21, so ( x)[ 3 x < 9 13 < 7 x 2 x] ( x)[x < < x 42 x 21 x] Given 42 x, 21 x is redundant. 56

57 Example 3 Step 5 F (x): ( x)[x < 63 }{{} (A) 39 < x }{{} (B) 42 x] δ = lcm{42} = 42 true }{{} false }{{} 42 x false (A): x<63 (B): 39<x 57

58 Example 3 ( x)[x < < x 42 x] Return false 42 j=1 42 j=1 F (j) 42 j=1 b i {39} F (b i + j) [39 + j < < 39 + j 42 (39 + j)] In particular, choose j = 3, so that 39 + j = 42. So ( x)[3x + 1 < 10 7x 6 > 7 2 x] true 58

59 Optimization: Block of ( x k,..., x n )F (x 1,..., x n ) 1 k n Each round of elimination produces a disjunction. Push remaining quantifiers over disjunctions. Eliminate each subblock. 59

60 Optimization: Block of ( x k,..., x n )F (x 1,..., x n ) δ F (x 1,..., x n 1, j) j=1 ( x k,..., x n 1 ) δ F (x 1,..., x n 1, b i + j) j=1 b i δ ( x k,..., x n 1 )F (x 1,..., x n 1, j) j=1 δ j=1 ( x k,..., x n 1 )F (x 1,..., x n 1, b i + j) b i 60

61 Example: Optimization ( y)( x)[1 5y < x 1 + y < 13x x < 2] lcm{1, 13} = 13 ( y)( x)[13x < y < 13x 1 + y < 13x] ( y)( x)[x < y < x 1 + y < x 13 x] ( y) 13 j=1 e {13 65y, 1+y} e + j < y < e + j 1 + y < e + j 13 e + j 61

62 Example: Optimization 13 j=1 ( y) 13 j=1 e {13 65y, 1+y} ( y) ( y) e + j < y < e + j 1 + y < e + j 13 e + j 13 65y + j < y < 13 65y + j 1 + y < 13 65y + j y + j 1 + y + j < y < 1 + y + j 1 + y < 1 + y + j y + j 62

63 Outline 0. Introduction 1. Theories 2. Quantifier Elimination (QE) 3. of Integer Linear (Presburger) Arithmetic, T Z 4. of Real Linear Arithmetic, T R 5. Quantifier-free Real Linear Arithmetic, T R 6. Integer Linear (Presburger) Arithmetic, T Z 7. Theory of Equality, T E 8. Theory of Recursive Data Structures, T D 9. Nelson-Oppen Combination 10. Shostak Theories & Combination 11. Theory of Arrays, T A 12. Incorporating DPs in Deductive Systems 63

64 Theory of Reals T R Σ = {0, 1, +,, =, <} Domain: R = {..., 0,..., 3 2,...} Constants: n for each n Z Functions: + (addition), (subtraction) Predicates: = (equality), < (comparison) Examples: ( x)[y = 2x] T R -valid 3x + 1 < 10 7x 6 > 8 T R -satisfiable 64

65 Theory of Reals T R Why are constants n Z? Idea: 3 2 x < 4 y 9x < 8y 3 Why theory of reals? Linear case: cannot express irrational numbers. If there is an irrational solution, then there is a rational solution. Polynomial case: x x = 2 expresses x = 2, which is irrational. 65

66 Algorithm: Quantifier Elimination for Reals [Ferrante & Rackoff, 75] Given ( x)f (x), where F is quantifier free. Step 0 Push negations all the way in: (α < β) β < α β = α (α > β) β > α β = α (α = β) β < α β > α 66

67 Algorithm: Quantifier Elimination for Reals Step 1 Solve for x in each atom of F (x): Collect x-terms and divide by coefficient of x. Atoms now have the form (A) x < a i (B) b i < x (C) x = c i where a i, b i, c i are real expressions that do not contain x. 67

68 Algorithm: Quantifier Elimination for Reals Step 2 Obtain F from F (x) by replacing all (A) atoms x < a i by true (B) atoms b i < x by false (C) atoms x = c i by false (A) x < a i }{{} true (B) b i < x }{{} false Obtain F + from F (x) by replacing all (A) atoms x < a i by false (B) atoms b i < x by true (C) atoms x = c i by false (C) x = c i }{{} false (A) x < a i }{{} false (B) b i < x }{{} true (C) x = c i }{{} false 68

69 Algorithm: Quantifier Elimination for Reals Let terms be set of a i, b i, c i terms. For every (symbolic) pair of terms s, t (s t), introduce (symbolic) term s + t 2 as representative of interval between s and t. Return F : F F + s terms F (s) s,t terms s t F ( ) s + t 2 ( x)f (x) F 69

70 Algorithm: Quantifier Elimination for Reals Return F }{{} F + }{{} no least n R s.t. F (n) no greatest n R s.t. F (n) s terms F (s) }{{} some term s satisfies F (s) s,t terms s t F ( ) s + t }{{} for some term pair s, t, all n (s, t) satisfy F (n) 2 70

71 Intuition Step 2 says that four cases are possible: 1. There is a left open interval s.t. all elements satisfy F (x). ) 2. There is a right open interval s.t. all elements satisfy F (x). ( 3. Some a i, b i, or c i satisfies F (x). b 2 c 1 a 2 4. There is an open interval between two a i, b i, or c i terms s.t. every element satisfies F (x). ( ) b 2 b 1 a 2 b 1 +a

72 Example 1: Reals ( x)[y = 2x] Step 1 Solve for x in atom y = 2x: ( x) [ x = y ] 2 72

73 Example 1: Reals Step 2 terms = { } y 2 Return ( x)[ x = y 2 ] }{{} (C) atom F F + F ( y 2 ) ( y false false 2 = y ) }{{ 2 } true ( x)[y = 2x] true 73

74 Example 2: Reals ( x)[3x + 1 < 10 7x 6 > 7] Step 1 Solve for x: ( x)[3x < 9 7x > 13] ( x)[x < 3 x > 13 7 ] 74

75 Example 2: Reals ( x)[ x < 3 }{{} (A) atom x > 13 7 ] }{{} (B) atom Step 2 F true false false (i.e., small n R satisfy x < 3, but not x > 13 7 ) F + false true false (i.e., large n R satisfy x > 13 7, but not x < 3) terms = { 3, 13 7 } 75

76 Example 2: Reals ( x) [ x < 3 x > 13 7 ] Return F F + s terms F (s) s,t terms s t false F false F + ( 13 7 < > ) 13 7 s 13 7 ( ) 3 < 3 3 > 13 7 s 3 ( 13 ) < > 13 7 s 13 7, t 3 ( x)[3x + 1 < 10 7x 6 > 7] true F ( ) s + t 2 false } true 76

77 Outline 0. Introduction 1. Theories 2. Quantifier Elimination (QE) 3. of Integer Linear (Presburger) Arithmetic, T Z 4. of Real Linear Arithmetic, T R 5. Quantifier-free Real Linear Arithmetic, T R 6. Integer Linear (Presburger) Arithmetic, T Z 7. Theory of Equality, T E 8. Theory of Recursive Data Structures, T D 9. Nelson-Oppen Combination 10. Shostak Theories & Combination 11. Theory of Arrays, T A 12. Incorporating DPs in Deductive Systems 77

78 Quantifier-free Theories Goal: Show quantifier-free F is satisfiable. Schema: Transform F into the disjunctive normal form (DNF): F F 1 F n where F i are conjuntions of atomic sentences or negations of atomic sentences. Check if any conjunctive F i is satisfiable. If there exists i such that F i is satisfiable, then F is satisfiable. Otherwise, F is unsatisfiable. 78

79 Fourier-Motzkin Algorithm: For Quantifier-free T R [Lassez & Mahler, 92] Given quantifier-free conjunctive F 0 (x 1,..., x n ) over R. Determine if F 0 (x 1,..., x n ) is satisfiable. Motivation: Do it faster than quantifier-elimination. Outline: 1. Eliminate equations (Gaussian elimination). The resulting F (x 1,..., x n ) contains only inequalities. 2. Repeat: (a) Choose x i to eliminate. (b) Eliminate x i, forming F (x 1,..., x i 1, x i+1,..., x n ) s.t. F (...) is satisfiable iff F (...) is satisfiable. 3. Trivial problem when no variables. E.g.,... 4 < 3... F 0 is unsatisfiable. 79

80 Algorithm: Fourier-Motzkin (Step 2a) Choose to eliminate x 1. F (x 1,..., x n ): a 1 x 1 + a 1,2 x a 1,n x n b 1 {}}{ α 1 < 0. a m x 1 + α m < 0 Rearrange: where a i, a j > 0 a i x 1 < α i for 1 i m α j < a j x 1 for m < j m α k < 0 for m < k m 80

81 Algorithm: Fourier-Motzkin (Step 2b) Combine each pair as a i x 1 < α i for 1 i m α j < a j x 1 for m < j m which is satisfiable iff a i α j < a i a j x 1 < a j α i a i α j < a j α i is satisfiable. F (x 2,..., x n ) : a i α j < a j α i α k < 0 1 i m m <j m m <k m F (x 1,..., x n ) is satisfiable iff F (x 2,..., x n ) is satisfiable. 81

82 Algorithm: Fourier-Motzkin (Step 2b) Consider case (no loer bounds): Then a i x 1 < α i for 1 i m α k < 0 for m < k m F (x 2,..., x n ) : m <k m α k < 0 and F (x 1,..., x n ) is satisfiable iff F (x 2,..., x n ) is satisfiable, because x 1 can be chosen arbitrarily small. No upper bound case is similar. 82

83 Example 1: Fourier-Motzkin F (x, y, z) : y < 1 z > 1 3x + y > 2 x = z y Step 1 Eliminate equations: x = z y z = x + y substitute x + y for z in F (x, y, z): F (x, y) : y < 1 x + y > 1 3x + y > 2 F (x, y) satisfiable iff F (x, y, z) satisfiable 83

84 Example 1: Fourier-Motzkin F (x, y) : y < 1 x + y > 1 3x + y > 2 Step 2a Choose to eliminate x: Step 2b 3x < y 2 y + 1 < x y 1 < 0 3y + 3 < 3x < y 2 satisfiable iff iff 3y + 3 < y 2 satisfiable 4y < 5 satisfiable F (y) : 4y < 5 y < 1 F (y) satisfiable iff F (x, y) satisfiable 84

85 Example 1: Fourier-Motzkin Step 2a Choose to eliminate y: Step 2b F (y) : 4y < 5 y < 1 5 < 4y y < 1 F : 5 < 4 5 < 4y < 4 satisfiable iff 5 < 4 satisfiable F satisfiable iff F (y) sat. Contradiction F (x, y, z) is unsatisfiable. 85

86 Example 2: Fourier-Motzkin F (x, y) : 2y < 3 x + y > 1 3x + y > 2 Step 2a Choose to eliminate x: Step 2b 3x < y 2 y + 1 < x 2y 3 < 0 3y + 3 < 3x < y 2 satisfiableiff iff 3y + 3 < y 2 satisfiable 4y < 5 satisfiable F (y) : 4y < 5 2y < 3 F (y) satisfiable iff F (x, y) satisfiable 86

87 Example 2: Fourier-Motzkin Step 2a Choose to eliminate y: Step 2b F (y) : 4y < 5 2y < 3 5 < 4y 2y < 3 5 < 4y < 6 satisfiable iff 5 }{{ < 6 } true satisfiable F : 5 < 6 F satisfiable iff F (y) satisfiable F (x, y) is satisfiable. 87

88 Time Complexity For n = length of formula, Theory Lower bound Upper bound Full T Z 2 2Θ(n) 2 22Θ(n) VERY expensive Full T R 2 Θ(n) 2 2Θ(n) } Q-free, conj. T R Θ(n c ) VERY efficient QE algorithm for T Z by Cooper (1972), upper bound by Oppen (1978). QE algorithm for T R by Ferrante and Rackoff (1975). Upper bound by Ferrante and Rackoff (1975). Lower bound by Fischer and Rabin (1974). Polynomial-time algorithm for quantifier-free, conjuntive T R -formulae by Kachiyan (1979) (see also Karmarkar (1984)). Fourier-Motzkin is 2 2Θ(n). 88

89 Outline 0. Introduction 1. Theories 2. Quantifier Elimination (QE) 3. of Integer Linear (Presburger) Arithmetic, T Z 4. of Real Linear Arithmetic, T R 5. Quantifier-free Real Linear Arithmetic, T R 6. Integer Linear (Presburger) Arithmetic, T Z 7. Theory of Equality, T E 8. Theory of Recursive Data Structures, T D 9. Nelson-Oppen Combination 10. Shostak Theories & Combination 11. Theory of Arrays, T A 12. Incorporating DPs in Deductive Systems 89

90 Omega Test: For Quantifier-free T Z [Pugh, 94] Given quantifier-free conjunctive F 0 (x 1,..., x n ) over Z. Determine if F 0 (x 1,..., x n ) is satisfiable. Motivation: Do it faster than quantifier-elimination. Outline: 1. Eliminate equations, forming F (x 1,..., x n ). The resulting F (x 1,..., x n ) contains only inequalities. 2. Repeat: (a) Choose x i to eliminate. (b) Apply Omega test to eliminate x i. 3. Trivial problem when no variables. E.g.,... 4 < 3... F 0 is unsatisfiable. 90

91 Normal Form Maintain constraints in normal form. a i x i = b 1 i n Let g = gcd{a i }. Set a i = a i g. If g b, then return unsatisfiable; otherwise, set b = b g. Example: 2x + 4y = 9 gcd{2, 4} = x + 4y = 9 is T Z -unsatisfiable. 91

92 Normal Form Maintain constraints in normal form. 1 i n Let g = gcd{a i }. Set a i = a i Set b =. Example: gcd{2, 4} = 2 b g a i x i g. b 2x + 4y 9 x + 2y x + 2y

93 Notation Let x 0 = 1 and a 0 = b: a i x i b 1 i n a i x i = b 1 i n a i x i 0 0 i n a i x i = 0 0 i n 93

94 Algorithm: Eliminate Equations (Unit Coefficients) Consider (for k 1) a k x k + where a k a i for 1 i n. Case a k = 1: and substitute everywhere. Case a k = 1: and substitute everywhere. 0 i n i k x k = x k = 0 i n i k 0 i n i k a i x i = 0 a i x i a i x i 94

95 Algorithm: Eliminate Equations Case a k > 1: a k x k + a i x i = 0 Define 0 i n i k a mod b = a b a b Let m = a k + 1. Create new constraint for fresh variable σ: mσ = (a i mod m)xi Solve for x k : 0 i n x k = sign(a k )mσ + sign(a k ) 0 i n i k (a i mod m)xi since a k mod m = sign(ak ). Substitute everywhere. 95

96 Algorithm: Eliminate Equations Substitute in original constraint: a k mσ + ) (a i + a k (a i mod m) x i = 0 0 i n i k But a k = m 1, so a k mσ + 0 i n i k a i (a i mod m) }{{} m a i m m(a i mod m) x i = 0 Normalize: a k σ + 0 i n i k ( ai m + 1 ) + a i mod m x i = 0 2 }{{} a i 96

97 Algorithm: Eliminate Equations a k σ + ( ai m + 1 ) + a i mod m x i = i n }{{} i k a i Observe: Absolute value of coefficient of σ is same as that of original coefficient of x k. For all other a i x i, a i 2 3 a i. Repeated application eventually forces unit coefficient, and therefore must terminate. 97

98 Algorithm: Eliminate Equations (Why 2 3?) a i = ai m + 1 ( ai + a i m 2 m 2 ) + 1 = a ai i + (1 m) m = a i (m 1) ai m }{{} 1 3 a i since a i m a i Why? Four cases (since a i m 1 = a k > 1): a i = ±(m 1) ( a i = m 1) a i = ±(nm + k) for n 1, 0 k m 1 ( a i > m 1) 98

99 Algorithm: Eliminate Equations (Case a i = nm + k) We have to show (m 1) ai m + 1 a i 2 3 3(m 1) ai m a i 3(m 1) nm + k m nm + k 3(m 1) n + k m nm + k 99

100 Algorithm: Eliminate Equations (Case a i = nm + k) But we have and 3(m 1) n + k m n(m 1) nm + m nm + k since 0 k m 1. Finally, 3n(m 1) nm + m when m 3, n 1, which is enough (m = a k + 1 3). 100

101 Example: Eliminate Equations Consider (1) 7x + 12y + 31z = 17 (2) 3x + 5y + 14z = 7 Choose x in (1). Then m = = 8: 8α = ( ( 2 ) 1 x ) 1 y + ( ) ( 1 2 z ) = x 4y z 1 Substitute everywhere x = 8α 4y z 1: (1 ) 7α 2y + 3z = 3 (2 ) 24α 7y + 11z =

102 Example: Eliminate Equations (1 ) 7α 2y + 3z = 3 (2 ) 24α 7y + 11z = 10 Choose y in (1 ). Then m = = 3: 3β =... = α + y Substitute everywhere y = α + 3β: (1 ) 3α 2β + z = 1 (2 ) 31α 21β + 11z = 10 Unit coefficient for z in (1 ); solve for z = 3α + 2β + 1, substitute: (1 ) 2α + β = 1 Solve for β = 2α 1 in (1 ). 102

103 Omega Test One step of Omega test on F (x 1,..., x n ): 1. Choose variable to eliminate, x Compute real shadow F (x 2,..., x n ) and dark shadow F (x 2,..., x n ) of F (x 1,..., x n ). real shadow F unsatisfiable F unsatisfiable dark shadow F satisfiable F satisfiable 3. If F is identical to F, then F is satisfiable iff F is satisfiable. 4. Otherwise, If F is unsatisfiable, return unsatisfiable. If F is satisfiable, return satisfiable. Otherwise, special case. (real shadow satisfiable, dark shadow unsatisfiable) 103

104 Omega Test High level rules: If a α and α b, for a > b, are constraints, then return unsatisfiable. If α a and α b, for a < b, are constraints, then delete α b (redundant). If α a and a α are constraints, replace with α = a and eliminate equation. Note that equations may be generated during the omega test. 104

105 Omega Test: Real Shadow Given F (x 1,..., x n ), form F (x 2,..., x n ) such that If F (x 2,..., x n ) is unsatisfiable, then F (x 1,..., x n ) is unsatisfiable. Use Fourier-Motzkin trick: F (x 1,..., x n ): a i x 1 α i for 1 i m α j a j x 1 for m < j m α k 0 for m < k m where a i, a j >

106 Omega Test: Real Shadow Combine each pair as a i x 1 α i for 1 i m α j a j x 1 for m < j m a i α j a i a j x 1 a j α i which is unsatisfiable over Z if a i α j a j α i is unsatisfiable over R. F (x 2,..., x n ) : 1 i m m <j m F unsatisfiable F unsatisfiable a i α j a j α i m <k m α k 0 106

107 Example: Real Shadow Consider F (x, y) : 27 11x + 13y x 9y 5 To eliminate x, rewrite: 11x 30 13y 7x 5 + 9y 27 13y 11x y 7x 107

108 Example: Real Shadow Combine pairs: 27 13y 11x 30 13y y 7x 5 + 9y 7(27 13y) 77x 11( 5 + 9y) 11( y) 77x 7(30 13y) F (x, y) is unsatisfiable if F (y) : 27 13y 30 13y y 5 + 9y y y y y is unsatisfiable. 108

109 Example: Real Shadow Combine terms: y y 320 That is, F (y) is unsatisfiable if is unsatisfiable. F (y) : y 320 But F (y) is satisfiable over R (e.g., y = 3 2 ), so cannot conclude anything. But is there an integer solution? 109

110 Omega Test: Dark Shadow Given F (x 1,..., x n ), form F (x 2,..., x n ) such that If F (x 2,..., x n ) is satisfiable, then F (x 1,..., x n ) is satisfiable. Use Fourier-Motzkin trick again: F (x 1,..., x n ): a i x 1 α i for 1 i m α j a j x 1 for m < j m α k 0 for m < k m where a i, a j >

111 Omega Test: Dark Shadow Combine each pair as a i x 1 α i for 1 i m α j a j x 1 for m < j m which is satisfiable over Z if a i α j a i a j x 1 a j α i a j α i a i α j (a i 1)(a j 1) is satisfiable over R. F (x 2,..., x n ): a j α i a i α j (a i 1)(a j 1) 1 i m m <j m m <k m α k 0 F satisfiable F satisfiable 111

112 Omega Test: Dark Shadow (Motivation) Combine each pair as ax 1 α β bx 1 for a, b > 0 aβ abx 1 bα Assume aβ bα is satisfiable (otherwise, return unsatisfiable). Suppose aβ abx 1 bα is unsatisfiable over Z. Define i = because in normal form). Then β b (< β b abi = ab ( ) β b < aβ bα? < ab(i + 1) Why? If not, then bα ab(i + 1), so could choose x 1 = i + 1: ) aβ ab + 1 = ab(i + 1) bα ( β b 112

113 Omega Test: Dark Shadow (Motivation) abi < aβ bα < ab(i + 1) Then and similarly Thus so ab(i + 1) > bα a(i + 1) > α a(i + 1) α 1 ab(i + 1) bα b abi + aβ a bα ab(i + 1) b aβ abi a bα aβ ab a b 113

114 Omega Test: Dark Shadow (Motivation) In short, aβ abx 1 bα unsatisfiable over Z bα aβ ab a b valid over R bα aβ > ab a b unsatisfiable over R bα aβ (a 1)(b 1) unsatisfiable over R Contrapositive: If bα aβ (a 1)(b 1) is satisfiable over R, then aβ abx 1 bα is satisfiable over Z. 114

115 Example: Dark Shadow Consider F (x, y) : 27 11x + 13y x 9y 5 To eliminate x, rewrite: 11x 30 13y 7x 5 + 9y 27 13y 11x y 7x 115

116 Example: Dark Shadow Combine pairs: 27 13y 11x 30 13y y 7x 5 + 9y 7(27 13y) 77x 11( 5 + 9y) 11( y) 77x 7(30 13y) F (x, y) is satisfiable if F (y) : 11(30 13y) 11(27 13y) (11 1)(11 1) 7( 5 + 9y) 7( y) (7 1)(7 1) 11( 5 + 9y) 7(27 13y) (11 1)(7 1) 7(30 13y) 11( y) (7 1)(11 1) is satisfiable. 116

117 Example: Dark Shadow Expand and simplify: F (y) : y y Unsatisfiable, so cannot conclude anything. 117

118 Omega Test: Special Case Situation: There is a pair of constraints ax 1 α, β bx 1 such that real shadow is satisfiable (aβ bα) dark shadow is unsatisfiable (bα aβ ab a b) So if there is a solution, then ab a b bα aβ bα abx 1 aβ ab a b + aβ bα abx 1 aβ ab a b + aβ abx 1 aβ ab a b a + β bx 1 β 118

119 Omega Test: Special Case Form constraint systems F (x 1,..., x n ) bx 1 = β + i for each 0 i ab a b a and recurse. Satisfiable iff at least one new constraint system is satisfiable. 119

120 Example: Special Case Consider F (x, y) : 27 11x + 13y x 9y 5 Real shadow is satisfiable. Dark shadow is unsatisfiable. Check if original constraints augmented with any of 11x = 27 13y + j for 0 j 9 7x = y + j for 0 j 5 is satisfiable. None is satisfiable F (x, y) is unsatisfiable. 9 = max { , } 11 5 = max { , }

121 Outline 0. Introduction 1. Theories 2. Quantifier Elimination (QE) 3. of Integer Linear (Presburger) Arithmetic, T Z 4. of Real Linear Arithmetic, T R 5. Quantifier-free Real Linear Arithmetic, T R 6. Integer Linear (Presburger) Arithmetic, T Z 7. Theory of Equality, T E 8. Theory of Recursive Data Structures, T D 9. Nelson-Oppen Combination 10. Shostak Theories & Combination 11. Theory of Arrays, T A 12. Incorporating DPs in Deductive Systems 121

122 Theory of Equality T E Σ = {a, b, c,..., f, g, h,..., p, q, r,..., =} Uninterpreted symbols: Constants: a, b, c,... Functions: f, g, h,... Predicates: p, q, r,... Example: f(f(f(a))) = a f(f(f(f(f(a))))) = a f(a) a x = y f(x) f(y) f(x) = f(y) x y T E -unsatisfiable T E -unsatisfiable T E -satisfiable 122

123 Theory of Equality T E Axiom schema: ( x)[x = x] (reflexivity) ( x, y)[x = y y = x] (symmetry) ( x, y, z)[x = y y = z x = z] [ ] ( x, y) (x i = y i ) f(x) = f(y) i (transitivity) (congruence) 123

124 Relations is an equivalence relation on a set S if it is a binary relation reflexive: ( s S)[s s] symmetric: ( s 1, s 2 S)[s 1 s 2 s 2 s 1 ] transitive: ( s 1, s 2, s 3 S)[s 1 s 2 s 2 s 3 s 1 s 3 ] is a congruence relation on S if it is an equivalence relation it obeys congruence: for every n-ary function f [ n ] ( s, t) (s i t i ) f(s) f(t) i=1 Equality is equivalence and congruence relation. 124

125 Classes Given set S with equivalence relation. The equivalence class of s S under is the set [s] = {s S : s s } If is a congruence relation over S, then [s] is the congruence class over s. Example: S : Z with equivalence relation 2 s.t. m 2 n iff (m mod 2) = (n mod 2) i.e., either both m, n are even or both are odd. The equivalence class of 3 under 2 is the set [3] 2 = {n : n is odd} 125

126 Partitions A partition P of S is a set of subsets of S that is total: S = S S P disjoint: ( S 1, S 2 P )[S 1 S 2 = ] Given set S and equivalence (congruence) relation. The set of equivalence (congruence) classes is a partition of S. S/ = {[s] : s S} Example: The partition Z/ 2 of Z by the equivalence relation 2 is the set of equivalence classes {{n : n is odd}, {n : n is even}} 126

127 Back to Relations We saw that an equivalence relation over S induces a partition of S into equivalence classes. Conversely, a partition of S, s.t. each class is an equivalence class, induces an equivalence relation over S. Same for congruence classes, partitions, and relations. Duality between relations and classes. 127

128 Refinement Given binary relations R 1 and R 2 over set S. R 1 is a refinement of R 2 (R 1 R 2 ) if ( s 1, s 2 S)[s 1 R 1 s 2 s 1 R 2 s 2 ] Say: R 1 refines R 2, R 1 is finer than R 2, R 2 is coarser than R 1. Examples: R 1 : {ar 1 b} R 2 : {ar 2 b, br 2 b} For any set S, the relation R 1 induced by the partition P 1 : {{s} : s S} refines the relation R 2 induced by the partition P 2 : {S}; i.e., R 1 R 2. S : Z R 1 : {xr 1 y : 2 x 2 y} R 2 : {xr 2 y : 4 x 4 y} R 2 R 1 128

129 Closures Given binary relation R over S. The equivalence closure of R is the finest equivalence relation R E that is coarser than R, i.e., smallest equivalence relation R E s.t. R R E (R E covers R). Example: If R = {arb, brc, drd}, then R E = {arb, bra, ara, brb, brc, crb, crc, arc, cra, drd}. The congruence closure of R is the finest congruence relation R C that is coarser than R, i.e., smallest congruence relation R C s.t. R R C (R E covers R). 129

130 Subterm Set Given formula F : s 1 = t 1 s m = t m s m+1 t m+1 s n t n its subterm set S contains every subterm of F. Example: The subterm set of F : f(a, b) = a f(f(a, b), b) a is {a, b, f(a, b), f(f(a, b), b)} 130

131 Congruence Relation as Model Given formula F : s 1 = t 1 s m = t m s m+1 t m+1 s n t n with subterm set S. If there exists congruence relation s.t. for each i {1,..., m} and for each i {m + 1,..., n} s i t i s i t i then = F ( is a model of F ) and F is satisfiable. Otherwise, F is unsatisfiable. 131

132 Algorithm: Congruence Closure Problem instance: F : s 1 = t 1 s m = t }{{ m s } m+1 t m+1 s n t n }{{} generate congruence closure search for contradiction Algorithm: 1. Construct the congruence closure of over F s subterm set. Then {s 1 = t 1,..., s m = t m } = s 1 = t 1 s m = t m 2. If for any i {m + 1,..., n}, s i t i, return unsatisfiable. 3. Else = F, so return satisfiable. 132

133 Algorithm: Constructing the Congruence Closure Given s 1 = t 1 s m = t m how do we construct the congruence closure? Idea: 1. Start with finest congruence relation 0 : ( s S) [[s] 0 = {s}] 2. For each i {1,..., m}, merge the congruence classes [s i ] i 1 and [t i ] i 1 to form a new congruence relation i : union [s i ] i 1 and [t i ] i 1 ; the union is already an equivalence class (Why?); propagate new congruences. Then at least [s i ] i = [t i ] i. 133

134 Example 1: Congruence Closure F : f(a, b) = a f(f(a, b), b) a 1. Initial: {{a}, {b}, {f(a, b)}, {f(f(a, b), b)}} 2. f(a, b) = a f(a, b) a (a) Merge {a} and {f(a, b)}: {{a, f(a, b)}, {b}, {f(f(a, b), b)}} (b) Propagate congruences: f(a, b) a, b b f(f(a, b), b) f(a, b) {{a, f(a, b), f(f(a, b), b)}, {b}} 3. {{a, f(a, b), f(f(a, b), b)}, {b}} = F? No: f(f(a, b), b) a but f(f(a, b), b) a F is unsatisfiable. 134

135 Example 2: Congruence Closure F : f(f(f(a))) = a f(f(f(f(f(a))))) = a f(a) a 1. Initial: {{a}, {f(a)}, {f 2 (a)}, {f 3 (a)}, {f 4 (a)}, {f 5 (a)}} 2. f 3 (a) = a f 3 (a) a (a) Merge {a} and {f 3 (a)}: {{a, f 3 (a)}, {f(a)}, {f 2 (a)}, {f 4 (a)}, {f 5 (a)}} (b) Propagate congruences: f 3 (a) a f(f 3 (a)) f(a), i.e. f 4 (a) f(a) f 4 (a) f(a) f(f 4 (a)) f(f(a)), i.e. f 5 (a) f 2 (a) {{a, f 3 (a)}, {f(a), f 4 (a)}, {f 2 (a), f 5 (a)}} 135

136 Example 2: Congruence Closure F : f(f(f(a))) = a f(f(f(f(f(a))))) = a f(a) a 1. Initial: {{a}, {f(a)}, {f 2 (a)}, {f 3 (a)}, {f 4 (a)}, {f 5 (a)}} 2. {{a, f 3 (a)}, {f(a), f 4 (a)}, {f 2 (a), f 5 (a)}} 3. f 5 (a) = a f 5 (a) a (a) Merge {a, f 3 (a)} and {f 2 (a), f 5 (a)}: {{a, f 2 (a), f 3 (a), f 5 (a)}, {f(a), f 4 (a)}} (b) Propagate congruences: f 3 (a) f 2 (a) f(f 3 (a)) f(f 2 (a)), i.e. f 4 (a) f 3 (a) {{a, f(a), f 2 (a), f 3 (a), f 4 (a), f 5 (a)}} 4. {{a, f(a), f 2 (a), f 3 (a), f 4 (a), f 5 (a)}} = F? No: f(a) a but f(a) a F is unsatisfiable. 136

137 Example 3: Congruence Closure F : f(x) = f(y) x y 1. Initial: {{x}, {y}, {f(x)}, {f(y)}} 2. f(x) = f(y) f(x) f(y) (a) Merge {f(x)} and {f(y)}: {{x}, {y}, {f(x), f(y)}} (b) Propagate congruences: None 3. {{x}, {y}, {f(x), f(y)}} = F? Yes. F is satisfiable. 137

138 Directed Acyclic Graph (DAG) Problem: Need to represent terms efficiently. Solution: Directed Acyclic Graph (DAG) representation. Data structure to represent terms and equations. Uniqueness A term corresponds to exactly 1 node in DAG. 1 : f f(f(a, b), b) 2 : f f(a, b) 3 : a 4 : b a b 138

139 Example 1: T E -Satisfiability f(a, b) = a f(f(a, b), b) a 1 : f 1 : f 1 : f 2 : f 2 : f 2 : f 3 : a 4 : b Initial DAG 3 : a 4 : b f(a, b) = a merge f(a, b) a explicit } equation find f(f(a, b), b) = a = find a Unsatisfiable f(f(a, b), b) a 3 : a 4 : b f(a, b) a, b b f(f(a, b), b) f(a, b) merge f(f(a, b), b) f(a, b) by congruence 139

140 Implementation: Summary of Functions Union/Find: For manipulating congruence classes (CCs) find: find the representative of a node s CC union: form the union of two congruence classes CC Parents: ccpar: return the parents (by subterm relation) of nodes in CC (these are the functions that are applied to members of CC) Congruent/Merge: congruent: return whether two terms are congruent merge: merge two CCs by union + propagating new congruences 140

141 DAG Representation type node = { id : id fn : string args : id list mutable find : id mutable ccpar : id set } f... [3, 4] node t returns the node for term t: 1 : f 2 : f node f(a, b) 3 : a 4 : b 141

142 DAG Representation type node = { id : id fn : string args : id list mutable find : id mutable ccpar : id set } a... [] {1, 2} 1 : f 2 : f 3 : a 4 : b node a 142

143 Implementation: Find find id: Return representative of id s CC. let n be the node corresponding to id if n.find is itself, return id (it s the representative) otherwise, call recursively on n.find Example: 1 : f 2 : f Suppose n = node f(a, b). Then find 2 = find n.find = 3 3 : a 4 : b 143

144 Implementation: Union union id 1 id 2 : Union CCs of id 1 and id 2. let id 1 and id 2 be representatives of CC of id 1 and CC of id 2 let n 1, n 2 be the nodes corresponding to id 1, id 2 set n 1 s representative to n 2 s representative (which is n 2 ) join CC parents Example: 2 : f 1 : f 3 : a 4 : b union 1 2 : n 1 = node (find 1) = node f(f(a, b), b) n 2 = node (find 2) = node a n 1.find n 2.find; n 2.ccpar n 1.ccpar n 2.ccpar; n 1.ccpar 144

145 Implementation: CC Parents ccpar id: Return id s CC parents. Example: 1 : f 2 : f ccpar 2 : n = node (find 2) = node a n.ccpar = {1, 2} 3 : a 4 : b 145

146 Implementation: Congruent congruent id 1 id 2 : Are id 1, id 2 congruent? let n 1, n 2 be the nodes corresponding to id 1, id 2 if the names of n 1 and n 2 are different, return false if the number of arguments is different, return false if any argument of n 1 is not in the CC of the corresponding argument of n 2, return false; otherwise, return true Example: 1 : f 2 : f 3 : a 4 : b congruent 1 2 : n 1 = node f(f(a, b), b) n 2 = node f(a, b) n 1.fn = n 2.fn = f n 1.args = [2; 4] n 2.args = [3; 4] find 2 = find 3 = 3 true 146

147 Implementation: Merge merge id 1 id 2 : Merge the CCs of id 1, id 2. let n 1, n 2 be the nodes corresponding to id 1, id 2 if find n 1 = find n 2, return let P 1, P 2 be the CC parents of n 1, n 2 P i is the set of functions that are applied to members of n i s CC union id 1 id 2 for each (p 1, p 2 ) P 1 P 2, if find p 1 find p 2 but congruent p 1 p 2, then merge p 1 p 2 147

148 Example: Merge 1 : f 1 : f 1 : f 2 : f 2 : f 2 : f 3 : a 4 : b 3 : a 4 : b 3 : a 4 : b merge 2 3 : P 2 = {1} P 3 = {2} union 2 3 congruent 1 2 merge

149 Implementation: Find, Union, Parents let rec find id = let n = node id in if n.find = id then id else find n.find let union id 1 id 2 = let n 1 = node (find id 1 ) in let n 2 = node (find id 2 ) in n 1.find n 2.find; n 2.ccpar n 1.ccpar n 2.ccpar; n 1.ccpar let ccpar id = (node (find id)).ccpar 149

150 Implementation: Congruent, Merge let congruent id 1 id 2 = let n 1 = node id 1 in let n 2 = node id 2 in n 1.fn = n 2.fn n 1.args = n 2.args ( i {1,..., n 1.args } find n 1.args[i] = find n 2.args[i] let rec merge id 1 id 2 = if find id 1 find id 2 then begin let P 1 = ccpar id 1 in let P 2 = ccpar id 2 in union id 1 id 2 ; foreach p 1, p 2 P 1 P 2 do if find p 1 find p 2 congruent p 1 p 2 then merge p 1 p 2 done end 150

151 Time Complexity Suppose e is the number of edges in the DAG. Suppose n is the number of nodes in the DAG. Presentation based on [Nelson & Oppen 1980]: Worst-case time for O(n) merges: O(e 2 ). [Downey, Sethi & Tarjan 1980]: O(e log e) average time. 151

152 Decision Procedure: T E -Satisfiability Problem instance: s 1 = t 1 s m = t }{{ m s } m+1 t m+1 s n t n }{{} generate congruence closure search for contradiction Algorithm: 1. for 1 i m, merge s i t i 2. for m + 1 i n, if find s i = find t i, return unsatisfiable 3. return satisfiable 152

153 Example 1: T E -Satisfiability f(a, b) = a f(f(a, b), b) a 1 : f 1 : f 1 : f 2 : f 2 : f 2 : f 3 : a 4 : b 3 : a 4 : b Initial DAG merge 2 3 P 2 = {1} P 3 = {2} union 2 3 congruent : a 4 : b merge 1 2 P 1 = {} P 2 = {1, 2} union 1 2 find f(f(a, b), b) = a = find a Unsatisfiable 153

154 Example 2: T E -Satisfiability f(f(f(a))) = a f(f(f(f(f(a))))) = a f(a) a 5 : f 4 : f 3 : f 2 : f 1 : f 0 : a Initial DAG 5 : f 4 : f 3 : f 2 : f 1 : f 0 : a f(f(f(a))) = a merge 3 0 P 3 = {4} P 0 = {1} merge 4 1 P 4 = {5} P 1 = {2} merge 5 2 P 5 = {} P 2 = {3} 154

155 Example 2: T E -Satisfiability f(f(f(a))) = a f(f(f(f(f(a))))) = a f(a) a 5 : f 4 : f 3 : f 2 : f 1 : f 0 : a 5 : f 4 : f 3 : f 2 : f 1 : f 0 : a f(f(f(f(f(a))))) = a merge 5 0 P 5 = {3} P 0 = {1, 4} merge 3 1 find f(a) = f(a) = find a Unsatisfiable 155

156 Postscript: What about Uninterpreted Predicates? Treat predicates as functions. Example: p(x, y) p(x, z) y = z p(x, y) = p(x, z) y = z Now use the equality decidable procedure. 156

157 Outline 0. Introduction 1. Theories 2. Quantifier Elimination (QE) 3. of Integer Linear (Presburger) Arithmetic, T Z 4. of Real Linear Arithmetic, T R 5. Quantifier-free Real Linear Arithmetic, T R 6. Integer Linear (Presburger) Arithmetic, T Z 7. Theory of Equality, T E 8. Theory of Recursive Data Structures, T D 9. Nelson-Oppen Combination 10. Shostak Theories & Combination 11. Theory of Arrays, T A 12. Incorporating DPs in Deductive Systems 157

158 Theory of Recursive Data Structures (RDS) T D Parametric theory. Each RDS has constructor: n C -ary function C(x) projection: unary functions πi C atom: unary predicate atom C Axiom schema: axioms of T E + ( x)[π C i (C(x)) = x i] (projection) ( x)[ atom C (x) C(π 1 (x),..., π nc (x)) = x] (construction) ( x)[ atom C (C(x))] (atom) 158

159 Example: List RDS constructor: cons projection: unary functions car, cdr atom: unary predicate atom binary predicate = 159

160 Example: Axioms of T cons reflexivity, symmetry, transitivity congruence axioms: ( x 1, x 2, y 1, y 2 ) x 1 = x 2 y 1 = y 2 cons(x 1, y 1 ) = cons(x 2, y 2 ) ( x, y) x = y car(x) = car(y) ( x, y) x = y cdr(x) = cdr(y) equivalence axiom: ( x, y) x = y (atom(x) atom(y)) (A1)( x, y)[car(cons(x, y)) = x] (A2)( x, y)[cdr(cons(x, y)) = y] (A3)( x)[ atom(x) cons(car(x), cdr(x)) = x] (A4)( x, y)[ atom(cons(x, y))] (left projection) (right projection) (construction) (atom) 160

161 Problem: T D -Satisfiability (Lists) Convert Problem instance: (A3 ) atom(u i ) u i = cons(u 1 i, u 2 i ) s 1 = t 1 s m = t }{{ m } generate congruence closure s m+1 t m+1 s n t n }{{} search for contradiction atom(u 1 ) atom(u l ) }{{} search for contradiction where DAG is closed w.r.t. the projection axioms: (A1) ( x, y)[car(cons(x, y)) = x] (A2) ( x, y)[cdr(cons(x, y)) = y] 161

162 Algorithm: T D -Satisfiability (Lists) Algorithm: 1. for each node n with n.fn = cons add car(n) and merge car(n) n.args[1] add cdr(n) and merge cdr(n) n.args[2] by axioms (A1), (A2) car cons cdr 2. for 1 i m, merge s i t i 3. for m + 1 i n, if find s i = find t i, return unsatisfiable 4. for 1 i l, if ( v)[find v = find u i v.fn = cons], return unsatisfiable 5. return satisfiable x y 162

163 Example: T D -Satisfiability car(x) = car(y) cdr(x) = cdr(y) atom(x) atom(y) f(x) f(y) (1) car(x) = car(y) (2) cdr(x) = cdr(y) (3) x = cons(u 1, v 1 ) (4) y = cons(u 2, v 2 ) (5) f(x) f(y) 163

164 Example: T D -Satisfiability (Initial DAG) car f cdr car f cdr x y car cdr car cdr axioms (A1), (A2) cons cons u 1 v 1 u 2 v 2 164

165 Example: T D -Satisfiability (merge) (1) (2) car f cdr car f cdr x y explicit equation by congruence car (3) cdr car cdr 1 : merge car(x) car(y) 2 : merge cdr(x) cdr(y) cons cons 3 : merge x cons(u 1, v 1 ) u 1 v 1 u 2 v 2 165

166 Example: T D -Satisfiability (Propagation) car f cdr car f cdr x y car cdr car cdr Congruent: car(x) car(cons(u 1, v 1 )) find car(x) = car(y) find car(cons(...)) = u 1 cons cons Congruent: cdr(x) cdr(cons(u 1, v 1 )) find cdr(x) = cdr(y) find cdr(cons(...)) = v 1 u 1 v 1 u 2 v 2 166

167 Example: T D -Satisfiability (merge) car f cdr car f cdr x y 4 : merge y cons(u 2, v 2 ) Congruent: car(y) car(cons(u 2, v 2 )) find car(y) = u 1 car cdr car (4) cdr find car(cons(...)) = u 2 cons cons Congruent: cdr(y) cdr(cons(u 2, v 2 )) find cdr(y) = v 1 u 1 v 1 u 2 v 2 find cdr(cons(...)) = v 2 167

168 Example: T D -Satisfiability (congruence) car f cdr car f cdr x y car cdr car cdr Congruent: cons(u 1, v 1 ) cons(u 2, v 2 ) Congruent: f(x) f(y) cons cons find f(x) = f(y) 5 : find f(y) = f(y) Unsatisfiable u 1 v 1 u 2 v 2 168