Algorithms and Complexity of Constraint Satisfaction Problems (course number 3) Nicolas (Miki) Hermann LIX, École Polytechnique hermann@lix.polytechnique.fr Miki Hermann Algorithms and Complexity of csp (3) 1 / 48
General method for satisfiability testing Davis-Putnam-Logemann-Loveland (DPLL) algorithm Complete backtracking-based search algorithm for deciding the satisfiability of propositional formulas Pure literal A variable x occurring in ϕ only as a positive (resp. a negative) literal is called a pure literal. Miki Hermann Algorithms and Complexity of csp (3) 2 / 48
General method for satisfiability testing... DPLL algorithm Input: Propositional formula ϕ(x 1,...,x n ) in cnf. Output: 1 if ϕ is satisfiable, 0 otherwise. if ϕ = 1 then return 1 end if if ϕ = 0 then return 0 end if for all unit clauses l in ϕ do if l = x then ϕ ϕ[x 1] else ϕ ϕ[x 0] end if end for unit propagation Miki Hermann Algorithms and Complexity of csp (3) 3 / 48
General method for satisfiability testing... DPLL algorithm (cont) for all pure literals l in ϕ do pure literal if l = x then ϕ ϕ[x 1] else ϕ ϕ[x 0] end if end for while ϕ not simplified do simplification (optional) apply to ϕ the rules 0 x x, 1 x 1, 0 x 0, 1 x x end while x choose variable from ϕ different possible strategies return dpll(ϕ[x 0]) dpll(ϕ[x 1]) Miki Hermann Algorithms and Complexity of csp (3) 4 / 48
General method for satisfiability testing... dpll(ϕ(x 1,x 2,x 3 )) x 1 = 0 ϕ x 1 = 1 x 2 = 0 x 2 = 1 x 3 = 0 x 3 = 1 0/1 0/1 Miki Hermann Algorithms and Complexity of csp (3) 5 / 48
General method for satisfiability testing... dpll algorithm analysis (n = number of variables) Time complexity: O(2 n ) Space complexity: O(n) Other strategies to enhance sat solvers clause learning backjumping restarts Exercise 5 1 Change dpll to produce a satisfying assignment m for ϕ. 2 Change dpll to produce all satisfying assignments for ϕ. Miki Hermann Algorithms and Complexity of csp (3) 6 / 48
Properties of Horn relations Let us study the following Horn relation: [x y z] = {000,001,010,100,101,110,111} = R Choose two vectors m = 101 and m = 110, and compute the conjunction m m = (m[1] m [1], m[2] m [2], m[3] m [3]) coordinate by coordinate. m = 101 m = 110 m m = 100 We easily see that m m R. This is true for each couple of vectors m,m R. Miki Hermann Algorithms and Complexity of csp (3) 7 / 48
Properties of Horn relations The elements (vectors) of the relation [x y z] can be organized in an acyclic graph as follows, where the conjunction m m is the greatest common ancestor of m and m. 111 101 110 001 100 010 000 Miki Hermann Algorithms and Complexity of csp (3) 8 / 48
Conjunction of vectors Definition Let m,m {0,1} k be two boolean vectors of arity k. The conjunction m m is the vector (m[1] m [1],..., m[k] m [k]) constructed coordinate-wise by the boolean function. A boolean relation R {0,1} k is closed with respect to conjunction if for each couple of vectors m,m R we have m m R. Formally: m m (m,m R (m m ) R) Miki Hermann Algorithms and Complexity of csp (3) 9 / 48
Characterization of Horn formulas Theorem If ϕ is a Horn formula then the set of models [ϕ] is closed by conjunction. Proof. We will show for each clause c of ϕ the following implication: if m = c and m = c then (m m ) = c. Case analysis. 1 The clause c contains a positive literal x satisfied by m and m. Then we have m(x) = m (x) = 1, what implies (m m )(x) = 1. Hence the conjunction m m satisfies the clause c. 2 The positive literal of c is not satisfied by m and m or c does not have a positive literal. Then at least one of two models, say m, must satisfy a negative literal x of c, what implies m(x) = 0. Then we have (m m )(x) = m(x) = 0, hence the conjunction m m satisfies c. Miki Hermann Algorithms and Complexity of csp (3) 10 / 48
Characterization of Horn relations Question Can we prove the inverse of the previous theorem? Answer The answer is not so easy, since the general algorithm, which constructs a cnf formula for a relation, does not necessarily produce a Horn formula from a Horn relation. We will show how to transform a formula ϕ R, corresponding to a Horn relation R, into a Horn formula h(ϕ R ). Miki Hermann Algorithms and Complexity of csp (3) 11 / 48
Construction of Horn formulas Let ϕ R be a cnf formula, corresponding to a Horn relation R {0,1} k, produced by the general algorithm (cf. Course 2) and let c be a clause of ϕ R. Denote by c the sub-clause containing all negative literals of c. The vectors m R satisfaing a negative literal of c, also satisfy the restricted clause c. Hence, we only need to take care of the vectors of R satisfaying the positive literal but no negative literal of c. These vectors constitute the following set: We proceed by case analysis. R c = {m R m = c }. Miki Hermann Algorithms and Complexity of csp (3) 12 / 48
Construction of Horn formulas 1 If R c is empy, we can replace the clause c by the Horn clause h(c) = c. This operation does not change the identity [ϕ R ] = R. 2 Otherwise, note that the set R c is closed under conjunction, since the set R is already closed under the same operation. Indeed, if the vectors m and m falsify each negative literal x of c, then the conjunction m m falsifies the same literals. Therefore the set R c contains a minimal unique model m = R c. Each positive literal in c, satisfied by m, is also satisfied by all vectors in R c. Let l be a positive literal of c satisfied by m. There must be at least one, since otherwise m would falsify c and also each positive literal in c, which would imply m / R c. Hence we can replace the clause c by the Horn clause h(c) = l c and keep the identity. [ϕ R ] = R. Miki Hermann Algorithms and Complexity of csp (3) 13 / 48
Example The relation M = {000, 111} is Horn, since m m = m and 000 111 = 000. The general algorithm constructs the complementary relation M = {0,1} 3 M = {001,010,100,011,101,110} which implies the formula ϕ M = (x y z) (x y z) ( x y z) (x y z) ( x y z) ( x y z) The first three clauses of ϕ M are not Horn. For c = (x y z) we construct c = z and M c = {111}. Hence h(c) = (x z). We obtain this way the Horn formula h(ϕ M ) = (x z) (x y) ( x y) (x y z) ( x y z) ( x y z) Miki Hermann Algorithms and Complexity of csp (3) 14 / 48
Construction of Horn formulas Observation By the aforementioned method we obtain a Horn formula h(ϕ R ) for each relation Horn R by replacing in ϕ R each clause c by its restriction h(c). Question Why that method does not work for every formula? Where is the hidden error? Analysis The previous algorithm transforms ϕ R into h(ϕ R ) in time O( R 2 k 2 ). There exists a more sophisticated algorithm with time complexity O( R k 2 ). Research problem Are you able to prove the lower bound Ω( R k 2 )? Miki Hermann Algorithms and Complexity of csp (3) 15 / 48
Construction of Horn formulas Theorem For each relation R, closed under conjunction, there exists a Horn formula ϕ R such that [ϕ R ] = R. Corollary A boolean relation R is closed under conjunction if and only if it is Horn, i.e., if there exists a Horn formula ϕ R satisfying the identity [ϕ R ] = R. Miki Hermann Algorithms and Complexity of csp (3) 16 / 48
Properties of dual Horn relations Let us study the following dual Horn relation: [x y z] = {000,010,011,100,101,110,111} = S Choose two vectors m = 010 et m = 101 and compute the disjunction m m = (m[1] m [1], m[2] m [2], m[3] m [3]) coordinate by coordinate m = 010 m = 101 m m = 111 It is easy to see that m m S. This is true for each couple of vectors m,m S. Miki Hermann Algorithms and Complexity of csp (3) 17 / 48
Properties of dual Horn relations The models of the relation [x y z] can be organized in the following acyclic graph, where the disjunction m m is the smallest common descendant of m and m. 111 011 110 101 010 100 000 Note that this graph is dual to the graph on the slide for Horn relations Miki Hermann Algorithms and Complexity of csp (3) 18 / 48
Disjunction of vectors Definition Let m,m {0,1} k be two boolean vectors of arity k. The disjunction m m is the vector (m[1] m [1],..., m[k] m [k]) constructed coordinatewise by the boolean function. A boolean relation R {0,1} k is closed with respect to disjonction if for each couple of vectors m,m R we have m m R. Formally: m m (m,m R (m m ) R) Miki Hermann Algorithms and Complexity of csp (3) 19 / 48
Characterization of dual Horn formulas Theorem If ϕ is a dual Horn formula then the set of models [ϕ] is closed under disjonction. Proof. By duality with Horn formulas. Miki Hermann Algorithms and Complexity of csp (3) 20 / 48
Characterization of dual Horn relations Question Can we prove the inverse of the previous theorem? Answer The answer in this case is simple because we can make use of duality. Theorem For each relation R closed under disjonction there exists a dual Horn formula ϕ R such that [ϕ R ] = R. Corollary A boolean relation R is closed under disjonction if and only if it is dual Horn, i.e., if there exists a dual Horn formula ϕ R satisfying the identity [ϕ R ] = R. Miki Hermann Algorithms and Complexity of csp (3) 21 / 48
Properties of bijunctive relations Let us study the following bijunctive relation: [(x y) (x z)] = {010,100,101,110,111} = S Choose three vectors m = 010, m = 101, m = 110 and compute the majority function maj(m,m,m ) = ( maj(m[1],m [1],m [1]),...,maj(m[3],m [3],m [3]) ) defined by the identities maj(0,0,0) = maj(0,0,1) = maj(0,1,0) = maj(1,0,0) = 0 maj(0,1,1) = maj(1,0,1) = maj(1,1,0) = maj(1,1,1) = 1 We see that maj(m,m,m ) S. This is true for each triplet of vectors m,m,m S. Miki Hermann Algorithms and Complexity of csp (3) 22 / 48
Majority of vecteurs Definition Let m,m,m {0,1} k be three boolean vectors of arity k. The majority maj(m,m,m ) is the vector ( maj(m[1],m [1],m [1]),..., maj(m[k],m [k],m [k]) ) constructed coordinatewise by the boolean function maj. A boolean relation R {0,1} k is closed under majority if for each triplet of vectors m,m,m R we have that maj(m,m,m ) R. Formally: m m m (m,m,m R maj(m,m,m ) R) Miki Hermann Algorithms and Complexity of csp (3) 23 / 48
Exercise 6 Prove that for each triplet of vectors m, m, m the majority function, sometimes also called median, satisfies the following identities: maj(m,m,m ) = (m m ) (m m ) (m m) = (m m ) (m m ) (m m) Attention! Do this exercise, otherwise you will have difficulties to understand the proof of the following lemma! Miki Hermann Algorithms and Complexity of csp (3) 24 / 48
Characterization of bijunctive formulas The closure property observed in the previous example is not accidental. To prove it, we need the following lemma. Lemma The vector maj(m,m,m ) satisfies a literal l if and only if at least two of the three vectors m, m, and m satisfy l. Proof. If the literal l is satisfied by at least two vectors, say m and m, then it is satisfied by the disjunctions (m m ), (m m ), and (m m). Hence l is satisfied by maj(m,m,m ) = (m m ) (m m ) (m m). If l is satisfied by one vector only, say m, or if l is not satisfied by any of the three vectors, then it is falsified by the disjunction (m m ). Therefore, it cannot be satisfied by maj(m,m,m ). Miki Hermann Algorithms and Complexity of csp (3) 25 / 48
Characterization of bijunctive formulas Theorem If ϕ is a bijunctive formula then the set of models [ϕ] is closed under majority. Proof. Given that ϕ is a conjunction of clauses, it is sufficient to prouve the closure property for one clause c = l l. Let m, m and m be three models of c. Following the Pigeonhole Principle, at least one of the two literals l and l is satisfied by at least two of the aforementioned models. Hence, according to the previous lemma, this literal as well as the clause c are satisfied by the model maj(m,m,m ). Miki Hermann Algorithms and Complexity of csp (3) 26 / 48
Characterization of bijunctives relations Question Can ve prove the inverse of the previous theorem? Answer The answer is constructive, as in the previous cases. The general algorithm does not necessarily construct a bijunctive formula ϕ R for a bijunctive relation R. We will show how to transform a formula ϕ R, corresponding to a bijunctive relation, into a bijunctive formula b(ϕ R ). Miki Hermann Algorithms and Complexity of csp (3) 27 / 48
Construction of bijunctive formulas Definition Let ϕ R be a formula in cnf, corresponding to a bijunctive relation R {0,1} k, produced by the general algorithm (cf. Course 2) and let c be a clause of ϕ R. We say that a literal l is essential is the clause c if there exists a model m R which satisfies l, but no other literal in c; we also say that m is the justification for l. Remark Of course, we can eliminate the non-essentiel literals of c without changing the identity [ϕ R ] = R. It is sufficient to show that no clause of ϕ R contains more than two essential literals. Miki Hermann Algorithms and Complexity of csp (3) 28 / 48
Construction of the algorithm Construction To produce a contradiction, suppose that there exists a clause c in ϕ R with at least three essential literals, say l 1, l 2, and l 3. Let m 1, m 2, and m 3 be their respective justifications, i.e., for each i we have m i = l i and m i satisfies no other literal in c. Following the Lemma, in this case maj(m 1,m 2,m 3 ) falsifies the clause c and therefore also the formula ϕ R. This is a contradiction with the hypothesis following which R = [ϕ R ] is a bijunctive relation. Miki Hermann Algorithms and Complexity of csp (3) 29 / 48
Algorithm The following algorithm transforms a formula ϕ R, corresponding to a bijunctive relation R, into a bijunctive formula b(ϕ R ). Algorithm for bijunctive formulas 1 Let R = {m 1,...,m p } be a bijunctive relation. For each clause c = l 1 l q of ϕ R we will fill in the table T of size R q in the following way: 1 if m i = l j, T[i,j] = 0 otherwise. 2 Using a greedy algorithm, we search for two essential literals l, l and set b(c) = l l. 3 The bijunctive formula b(ϕ R ) is the conjunction c ϕ R b(c). Miki Hermann Algorithms and Complexity of csp (3) 30 / 48
Analysis of the algorithm Analysis The number of literals q is bounded by the arity k of the relation R. The size of table T is therefore O( R k) and the algorithm works in time O( R nk), where n is the number of clauses of the formula ϕ R. Remark This algorithm, slightly adapted (search for all essential literals), also works for Horn relations. Exercise 7 Perform the correstness proof of this algorithm for Horn relations. Miki Hermann Algorithms and Complexity of csp (3) 31 / 48
Example The relation M = {000,111} is bijunctive since we have maj(000, 000, 111) = 000, maj(000, 111, 111) = 111, and maj(m,m,m) = m. The general algorithm constructs the formula ϕ M = (x y z) (x y z) ( x y z) (x y z) ( x y z) ( x y z) The tables T for the clauses c = (x y z) and c = (x y z) are x y z x y z 000 0 0 1 000 0 1 1 111 1 1 0 111 1 0 0 Hence b(c) = (y z) and b(c ) = (x y). This way we get the formula b(ϕ M ) = (x y) (y z) (z x) = (y x) (z y) (x z) Miki Hermann Algorithms and Complexity of csp (3) 32 / 48
Bijunctive characterization The proof on the slide..., as well as the previous algorithm, imply the following result. Theorem For each relation R closed under majority there exists a bijunctive formula ϕ R such that [ϕ R ] = R. Corollary A boolean relation R is closed under majority if and only if it is bijunctive, i.e., if there exists a bijunctive formula ϕ R such that [ϕ R ] = R. Miki Hermann Algorithms and Complexity of csp (3) 33 / 48
Properties of affine relations Let us study the following affine relation: [x + y + z = 1] = {001,010,100,111} = S Choose three vectors m = 010, m = 100, m = 111 and compute the function affinity, also called minority or Mal tsev function aff(m,m,m ) = ( aff(m[1],m [1],m [1]),...,aff(m[3],m [3],m [3]) ) defined by aff(m,m,m ) = m + m + m mod 2 = m m + m We see that aff(m,m,m ) S. This is true for each triplet of vectors m,m,m S. Miki Hermann Algorithms and Complexity of csp (3) 34 / 48
Affinity or minority of vectors Definition Let m,m,m {0,1} k be three boolean vectors of arity k. The affinity or minority aff(m,m,m ) is the vector ( aff(m[1],m [1],m [1]),..., aff(m[k],m [k],m [k]) ) constructed coordinatewise by the boolean function aff. A boolean relation R {0,1} k is closed under affinity (or minority) if for each triplet of vectors m,m,m R we have aff(m,m,m ) R. Formally: m m m (m,m,m R aff(m,m,m ) R) Remark In the affine case we always work in the ring (field) (Z 2,+, ). Miki Hermann Algorithms and Complexity of csp (3) 35 / 48
Characterization of affine formulas The property found in the previous example is not random. Theorem If ϕ is an affine formula then the set of models [ϕ] is closed under affinity. Proof. Each affine formula corresponds to a system of linear equations A x = b over Z 2. The solutions of the system A x = b constitute an affine space A, which is a vector space V produced by the solutions of the corresponding homogeneous system A x = 0, translated by a solution a of the system A x = b. Formally: A = V + a. Each vector space is closed under addition of two vectors, i.e., a 1, a 2 V implies a 1 + a 2 V. Therefore, each affine space is closed under addition of three vectors: a,( a 1 + a),( a 2 + a) A implies ( a 1 + a) + ( a 2 + a) + a = ( a 1 + a 2 ) + a A. Miki Hermann Algorithms and Complexity of csp (3) 36 / 48
Characterization of affine relations Question Can we prove the inverse of the previous theorem? Answer YES, but the usual answer is not constructive. Indeed, we have the following theorem in linear algebra. Miki Hermann Algorithms and Complexity of csp (3) 37 / 48
Characterization of affine relations Theorem If a set of vecteurs M Z k 2 of arity k is closed under addition then M is a vector space. Corollary If a set of vectors M Z k 2 of arity k is closed under affinity then M is an affine space. Proof. It is sufficient to choose a vector m M and construct the set of vectors M = {m m m M} = {m + m m M}. The set M is an affine space if and only if M is a vector space. Miki Hermann Algorithms and Complexity of csp (3) 38 / 48
Characterization of affine relations Remark The proof of Corollary is not constructive and therefore it does not produce an algorithm. The constructive answer corresponds to a search for a basis of a vector space. We do not use the general algorithm followed by a transformation, as in the previous cases, but we construct directly the linear affine system A x = b. However, it is better to construct a system (I B) x = b, where I is the identity matrix. The matrix B is composed of rows B i and the element of the matrix B located on row i and column j is denoted by B j i. Miki Hermann Algorithms and Complexity of csp (3) 39 / 48
Characterization of affine relations Construction Let M {0,1} k be a set of boolean vectors of arity k. We must first verify if the cardinality M is equal to an power of 2, otherwise M cannot be an affine space over Z 2. In the positive case, the system (I B) x = b will have l = k log 2 M rows. Hence, I is the identity matrix of size l l and B is a matrix of size l (k l) over Z 2. The ith row of the system (I B) x = b is x i + B 1 i x l+1 + + B k l i x k = b i Miki Hermann Algorithms and Complexity of csp (3) 40 / 48
Algorithm Algorithm for affine formulas For each row x i + Bi 1x l+1 + + B k l i x k = b i, where i = 1,...,l, of the system (I B) x = b we proceed in the following way: 1 For each vector m M, substitute m for variables x, i.e., substitute m[j] for x j, j = 1,...,k. We get the following system of equations m[i] + B 1 i m[l + 1] + + B k l i m[k] = b i (1) for each m M, i.e., with 2 k l rows, where B i = (B 1 i,...,bk l i ) and b i are variables. 2 Solve the system (1) to determine the values of the row B i and of b i. Miki Hermann Algorithms and Complexity of csp (3) 41 / 48
Example The relation M = {000,111} is affine, because 000 + 000 + 111 = 111 + 111 + 111 = 111 and 000 + 000 + 000 = 000 + 111 + 111 = 000. The algorithm constructs the linear system (I B)(xyz) T = b with 2 rows and 3 columns: x + B 1 1z = b 1 y + B 1 2z = b 2 We substitute the models of M for variables of the previous system. The first equation produces 0 = b 1 et 1 + B 1 1 = b 1, what implies B 1 1 = 1. The second equation produces 0 = b 2 and 1 + B 1 2 = b 2, what implies B 1 2 = 1. Therefore the sought system is x + z = 0 y + z = 0 Miki Hermann Algorithms and Complexity of csp (3) 42 / 48
Properties of other relations Question What are the (closure) properties satisfied by the other relations which we saw during Course 2? Miki Hermann Algorithms and Complexity of csp (3) 43 / 48
Properties of other relations nae and negation There are two relations to consider: nae = {001, 010, 011, 100, 101, 110} 1-in-3 = {001, 010, 100} For the relation nae, choose one vector m = 010 and compute its negation m = ( m[1], m[2], m[3]) coordinatewise. We get m = 101 nae. This is true for each vector m nae. Resistance of 1-in-3 On the contrary, the relation 1-in-3 resists to any characterization tentative. Miki Hermann Algorithms and Complexity of csp (3) 44 / 48
Negation of vectors Definition Let m {0,1} k be a boolean vector of arity k. The negation m is the vector ( m[1],..., m[k]) constructed coordinatewise by the boolean function. A boolean relation R {0,1} k is closed under negation if for each vector m R we have m R. Formally: m(m R m R) Miki Hermann Algorithms and Complexity of csp (3) 45 / 48
Complementive relations Definition A relation R {0,1} k is complementive if it is closed under negation. Example The relation nae = {001, 010, 100, 110, 101, 011} is complementive. However, it is not Horn, since 001 010 = 000 / nae; it is not dual Horn, since 010 101 = 111 / nae; it is not bijunctive, since maj(001, 010, 100) = 000 / nae; it is not affine, since 001 + 010 + 100 = 111 / nae; it is neither 0-valid, nor 1-valid, since 000 / nae and 111 / nae. Miki Hermann Algorithms and Complexity of csp (3) 46 / 48
Exercise 8 1 Under which previously listed boolean functions is the relation R 1 = {001,010,101,110} closed? 2 Show that the relation R 2 = {011,100,101,110,111} is closed under implication and majority. Miki Hermann Algorithms and Complexity of csp (3) 47 / 48
End It s all for today. Do you have questions? Miki Hermann Algorithms and Complexity of csp (3) 48 / 48