An empirical complexity study for a 2CPA solver

Transcription

1 Modern Information Processing: From Theory to Applications 1 B. Bouchon-Meunier, G. Coletti and R.R. Yager, Editors c 2005 Elsevier B.V. All rights reserved An empirical complexity study for a 2CPA solver M. Baioletti and A. Capotorti and S. Tulipani Dipartimento di Matematica ed Informatica, Università degli Studi di Perugia Perugia, Italy Abstract The computational decision problem CPA [1 4] is a variant of the probabilitistic satisfiability problem PSAT [5,11,15]. In this paper we investigate the behavior of an algorithm which decides CPA applied to the still NP-complete subproblem 2CPA, whose instances have at most two literals per clause. We locate, as it is done for some satisfiability problems [10,12 14,17], a critical value for the ratio α = m/n, where m is the number of binary clauses present in the instance and n is the number of events. This point divides almost all coherent instances from almost all not coherent ; moreover the most difficult instances lies near this point. Keywords: Probability assessments, Coherence decision, NP-complete problems, Simplification rules. We had the pleasure to work with our friend and colleague Sauro to this last job. He was always inspired by an interest of pure research and he has been our guide and promoter. This contribution is dedicated to him. Andrea & Marco 1. Introduction Probabilistic models based on partial assessments play a central role in the treatment of partial knowledge and they have a wide relevance in both unconditional and conditional frameworks. For an exhaustive overview on this subject refer to [8]

2 2 M. Baioletti et al. where there is a complete description of such methodology and where all the most recent developments have been summarized. One of the major obstacles to the use of general probabilistic models in applications has been the high computational cost of the most common adopted methods. Anyway, in the last period several techniques have been developed to bypass such drawback, both in conditional assessments ([8], chapt. 14) and in unconditional assessments (see [16]). We recently proposed a new approach [1 4] to the problem of the coherence of unconditional assessments by formulating an algorithm based on a variable elimination procedure in the style of Davis Putnam procedure for satisfiability. This approach, in line with de Finetti thought [9], deals with probability assessments on events which are represented implicitly by their explicit logical relations. An instance of the problem, called CPA for Coherent Probability Assessment, is an assessment on events, say E 1,..., E n, together with a set of dual clauses 1 on Boolean variables X 1,..., X n. Each dual clause determines a relation on E 1,..., E n by substituting each variable X i for the corresponding E i and by letting the clause to be equate to 0, which represents false. Problem CPA is adequate to treat any partial probabilistic framework as well as the problem PSAT [5,11,15], to which it is naturally reducible [1]. Hence, CPA is a NP-complete problem. Actually, it is NP-complete also its restriction 2CPA to clauses with at most two literals. In this paper we will investigate the behavior of our algorithm for CPA in the case of instances of 2CPA. We have already proved in [3] that a suitable elimination procedure, based on our rewriting rules introduced in [1,2], solves instances of 2CPA in polynomial average time. Here, we perform an experimental analysis, as announced in [4], on random instances. In Section 2, we briefly present the more general problem CPA and the procedure based on simplification rules of instances. In Section 3, we present our procedure to generate random 2CPA instances which has the peculiarity to avoid logically unsatisfiable or trivially incoherent instances. Finally, in Section 4, we report the experimental results of our procedure applied to such random instances. We are able to observe that the behavior of our algorithm is related to the index α = m/n, where m is the number of binary clauses present in the instance and n is the number of Boolean variables involved. Like most of the satisfiability problems [10,12 14,17], our procedure presents an almost unimodal path depending on the index α, with a peak in instances for which the α is quite concentrated around a critical value. This particular value of α is also a crossover where the instances of the problem pass from almost all coherent to almost all not coherent. 1 Dual means in disjunctive normal form.

3 An empirical complexity study for a 2CPA solver 3 2. CPA and simplification rules In this section we formalize the CPA problem in its general form and we describe the rules of simulations used in our decision procedure. Let us give first the basic definitions. Definition 1. An instance of CPA is given by a pair (C, p) where p = (p 1,..., p n ) is a vector of real numbers with 0 p i 1, i = 1,..., n, and C = {D 1,..., D m } is a set of dual clauses in Boolean variables X 1,..., X n. Each dual clause D j is a conjunction of literals D j = X ε 1 j 1 X ε 2 j 2... X ε r j r j k {1,..., n}, ε k {0, 1} where, as usual, X 0 j k = X j k (the negation of X jk ) and X 1 j k = X jk. It is understood that, under the substitution of each variable X i for an event E i, the instance (C, p) represents a probabilistic assessment E i p i, i = 1,..., n, and each clause D j, for j = 1,..., m, when equated to 0 (i.e. false) represents a Boolean relation among the events. Events E i s generate a Boolean algebra B modulo the given relations, see [1], and it is not trivial if and only if the set of dual clauses C is zero satisfiable. Definition 2. An instance (C, p) is called coherent, or satisfiable, if there is a probability distribution P : B [0, 1] on the Boolean algebra described above such that P (E i ) = p i for i = 1,..., n. Actually, inside our procedure we will need to generalize the notion of CPA instance, first by considering an assessment that takes value into a generic interval [0, p 0 ], instead of [0, 1]. This is required since, in the elimination process, the use of Splitting Rule 2 will divide an assessment of total mass q in two assessments of total masses q 1 and q 2, respectively, such that q 1 + q 2 = q. Moreover, since the new assessments generated by the Splitting Rule are parametric, we need the elements of assessment p i and the total mass p 0 to be linear polynomials of some real variables. Such variables could also be constrained by the a set of inequalities. Therefore an assessment can be generalized to a symbolic one as follows: Definition 3. A symbolic assessment S is a pair (C, p) where C is a set of dual clauses, as in Definition 1, and p = (p 0, p 1,..., p n ) is an array of linear polynomials with real constant terms and integer coefficients for the real variables, say z = (z 1,..., z t ). If U is a set of linear inequalities in the real variables z, we say that (S; U) is a symbolic assessment with constraints U if, for every list a of real numbers 2 This rule will be defined forward

4 4 M. Baioletti et al. which satisfy all the inequalities in U with the substitution of z for a, the following condition holds 0 p i ( a) p 0 ( a) for i = 1,..., n. The notion of satisfiability can be easily extended to a symbolic assessment with constraints, see [1]. Now we present some rules for rewriting an instance of CPA into simpler instances. Each rule can produce, as a side effect, a set of constraints on the assessment values. So we work in general with a sequence S 1, S 2,..., S r ; U (1) of general assessments on the same set of constraints U. A sequence of symbolic constrained assessments on real variables z is satisfiable if there is a list of real numbers a which satisfies every constraint in U and such that each ground assessment S i ( a) is coherent. The rules described below can be applied to any S = (C, p) belonging to the list S 1, S 2,..., S r. The new sequence is satisfiable if and only if the old sequence was such (for a proof see [1]). To simplify the notation, if p i is any polynomial in the list (p 0, p 1,..., p n ) and ε {0, 1}, we will denote by p ε i the polynomial p i, if ε = 1, and the polynomial p 0 p i, if ε = 0. For the sake of space limitation, we report here only the three most relevant rules. Anyway, each rule is numbered according to the list presented in [2] which we refer to for a complete description of all the rules. R1: Unitary clause rule. If Xi ε appears as a clause in C then Delete all clauses containing Xi ε Delete the literal X 1 ε i in each clause where it appears Add the constraint p ε i = 0. R5: Splitting Rule. Fix a variable that, without loss of generality, can be X 1. Then replace S by the two general assessments S 1 = (C 1, p 1 ) and S 0 = (C 0, p 0 ) in the Boolean variables X 2,..., X n such that p 1 = (p 1, z 2,..., z n ) and p 0 = (p 0 p 1, p 2 z 2,..., p n z n ) where z 2,..., z n are new real variables not already being used C ε = C[X 1 ε], for ε = 0, 1 where [X 1 ε] means the substitution of variable X 1 for the Boolean value ε Add the constraints 0 z i p 1, 0 p i z i p 0 p 1 for i = 2,..., n. R8: Inclusion rule. If in S = (C, p) all the clauses of C are binary clauses and the literal literal graph G(C) is bipartite, then Delete S from the sequence Add the constraints p ɛ i + p η j p 0 for all Xi ɛ X η j in C.

5 An empirical complexity study for a 2CPA solver 5 These rules are correct with respect to the notion of satisfiability. Hence they can be used in an algorithm which checks the coherence of a given probability assessment (C, p). The algorithm starts with the symbolic assessment S; as input, where S = (C, (1, p 1,..., p n )) and halts when a contradiction in U is found or when there are no more clauses. In this case the coherence of (C, p) is equivalent to the solvability of U, the constraint system returned by the procedure, which can be checked by using one of several methods based on linear programming. At each step the algorithm keeps the current sequence S 1, S 2,..., S r in a stack and applies one of the simplification rules to the topmost assessment, trying to use as late as possible the Splitting Rule. 3. Algorithm behavior for 2CPA: random instances generation As already stated in the introduction, the main goal of this paper is to produce a qualitative analysis of the performance of the algorithm described in the previous section when it is applied to 2CPA instances, i.e. when the clauses D j s, j = 1,..., m, have at most two literals. Since, due to rule R1, unitary clauses would be immediately simplified, without loss of generality, we will focus on instances containing clauses with exactly two literals. The practical relevance of such instances and the good performance (polynomial average time complexity) of the algorithm on them were already discussed in [3], but the present empirical study will show interesting features hardly attainable by pure theoretical tools. The first issue we had to face to perform good simulations was to find a sound random generator of 2CPA instances. In fact, a fully unconstrained generator is not suitable for our purposes because trivial not coherent instances can easily appear. In this section we discuss three generators of random instances for the 2CPA problems; anyway, in the next section we will present only the empirical results obtained by the second and the third. The first two generators share as a first common step the random generation of the probabilities p i, associated to each event X i, i = 1,..., n, with a uniform distribution in [0, 1]. The second step is the random generation of the clauses. The two generators differ in this step. A first common principle used in both generators is that clauses must be zero satisfiable, i.e. there must exists a truth assignment on the variables which falsifies all the clauses. In fact it is easy to show that a 2CPA instance whose clauses are not zero satisfiable, is incoherent. Fortunately, the problem of checking if a 2CPA has zero satisfiable clauses is equivalent to 2SAT and then it is solvable in a time linear with respect the number of clauses. A second principle is to avoid to generate instances whose clauses violate in an

6 6 M. Baioletti et al. apparent way some rules of probability theory. In particular, for each binary clause Xi ɛ Xη j, where ɛ, η {0, 1}, the constraint p ɛ i + pη j 1 must hold. Otherwise the instance is not coherent and any procedure, based on our rules, would be able to detect the incoherence in a few steps. In fact applying the splitting rule (R5) on one of the two variables present in the clause, and then applying the unit clause rule (R1) on the other variable, two numerical contradictory constraints would be added to the system. This inconsistency would be immediately detected by the procedure. When performing an empirical analysis on the percentage of coherent instances or on the difficulty of solution, the presence of instances with clauses violating p ɛ i + pη j 1 can cause a bias on the results of an empirical analysis because this kind of clauses are very easy to solve and always incoherent. Moreover, a totally random generator of instances, i.e. which generates the p i s independently of the clauses, has a very high probability (of order 1 2 n ) to create at least one clause violating the former constraint. Therefore a 2CPA instances generator must take into account the clauses D j s when it generates the probabilities p i s. On the other hand, it is computationally easier to reverse the process by taking into account the probabilities p i s meanwhile the clauses D j s are generated. Now, we observe that the constraint reduces to p i + p j 1 if ɛ = 1 and η = 1, to p i + p j 1 if ɛ = 0 and η = 0, to p i p j if ɛ = 1 and η = 0 and to p i p j if ɛ = 0 and η = 1. So, given p i and p j the polarities can be chosen according to the following table p i + p j 1 p i + p j 1 ɛ = 1, η = 0 ɛ = 1, η = 0 p i p j or or ɛ = 1, η = 1 ɛ = 0, η = 0 ɛ = 0, η = 1 ɛ = 0, η = 1 p i p j or or ɛ = 1, η = 1 ɛ = 0, η = 0 (2) Except in cases where p i = p j or p i + p j = 1, only two clauses, among the four possible, are admissible; moreover, one of them has equal polarities while the other has different polarities. Hence a generator can first decide about equal or different polarities and later choose the right ones according to table (2), where, to be definite, in case p i = p j it uses only the first line and in case p i + p j = 1 the first column. Finally this way of selecting clauses also fulfills the first principle because the truth assignment

7 An empirical complexity study for a 2CPA solver 7 0 if p i 1/2 v(x i ) = 1 otherwise is able to zero satisfy all the selected clauses. The first generator 3 takes as input a positive integer n and a probability value p. Then, after randomly choosing the assessment vector (p 1,..., p n ), the generator decides, on the base of the fixed probability p, for every 1 i j n if to generate a clause on the variables X i, X j or not. If yes, it chooses, with probability 1/2, one of the two admissible clauses according to the above table (2). This operational procedure generates a random instance of 2CPA where the expected value ᾱ of M/n, where M is the random number of clauses, is ( ) n 1 p(n 1) ᾱ = p = 2 n 2 The analysis of the experiments can be done by varying the probability p or, equivalently, by varying ᾱ and taking p = (2ᾱ)/(n 1). On the contrary, the analysis with the second generator of random clauses is done with respect to the fixed index α = m/n. This generator, whose inputs are now m and n, after the first step draws randomly m couples of variables without replacement and it selects the polarities of the variables in each clause according to table (2). An apparent advantage of the second generator is that the instances produced have a fixed number of clauses, while the first generator is only able to control the average number of clauses. The third generator produces instances which are generally much harder than those produced by the first two. It exploits the linear time reduction of the problem 3 COLORABILITY to 2CPA, which was already used in [1] to prove that CPA is NP complete. This generator has as input parameters v and p and generates a random graph G with v vertices and a random number E of edges, where each possible edge has probability p to be present in G. Denoting by e the realization of E, G is then converted into a 2CPA instance with n = 3v variables and m = 3(v + e) clauses, and with all the event s probabilities set to 1/3. 4. Algorithm behavior for 2CPA: empirical results We have performed a series of experiments to determine the computational hardness to solve random instances of 2CPA and to estimate the percentage of those that are coherent. 3 This generator was already used in a simplified form to prove that a procedure based on our rules is polynomial average-time on 2CPA [3]

8 8 M. Baioletti et al. Here, for the sake of synthesis, we report the results obtained by using only the second and the third random generator described in the previous section. With the second generator, the parameter n varies among the integers from 20 to 45 by a step of length 5, while the parameter α varies in [0, 3] with a fineness of For each joint combination of n and α, the algorithm solved random instances, storing the following data cp = percentage of coherent instances t = execution time nn = number of visited nodes in the search tree ps = percentage of instances solved by satisfiability of U In the following figure we report the graphs of cp against α. Note the path with a threshold, typical of satisfiability problems. In fact, also in 2CPA we find that cp Fig. 1. Percentages of coherent instances, with n varying from 20 to 45 by a step of length 5 the percentage of coherent (the analogous of satisfiable for k SAT) instances passes from values close to 100% to values almost null. This quick decrease happens in the neighborhood of a critical value of α, usually named threshold or crossover and denoted by α c. Such behavior has been already observed in other combinatorial and physical problems and it has been named phase transition (see [10]). In figure 2 the average hardness of the procedure measured by nn against α are plotted. Obviously, for any fixed value of α, nn increases according to the number n of variables. All the paths share a common behavior. They are unimodal and strongly asymmetric, i.e. they are quite fast increasing till a value of α around 1.7, while they are slightly decreasing for higher values of α. A similar behavior can be observed in figure 3, in which the average execution times in seconds are reported.

9 An empirical complexity study for a 2CPA solver 9 nn Fig. 2. Number of visited nodes in the search tree, with n varying from 20 to 45 by a step of length t Fig. 3. Procedure execution times, with n varying from 20 to 45 by a step of length 5 On the contrary, in figure 4 the graphs of ps against α show more symmetrical paths. This is in accordance with the behavior of cp. In fact, varying the number of variables n, the maximum values accordingly locate close to the crossovers α c. This phenomenon could suggest that far from the transition area 2CPA is essentially an almost purely combinatorial problem, while close to the crossover point it has also a numerical counterpart. Anyhow this aspect must be deepened in some future investigation.

10 10 M. Baioletti et al. ps Fig. 4. Percentage of instances solved by satisfiability of U, with n varying from 20 to 45 by a step of length 5 For the sake of comparison, we report also the results of runs performed with the third random generator. Due to hardness of instances generated in such a way, we restricted our attention only to instances generated by random graphs with the number or vertices in between 3 and 8. Note that in this case the potential value of 3 colorability instances 3 colorability instances % coherent var 6 var 8 var number of nodes var 6 var 8 var Fig. 5. Percentages of coherence and computational hardness for instances generated with random graphs with 4 (solid), 6 (dashed) and 8 (dotted) vertices the crossover for cp should be between 2 and 3 and it is significantly influenced by the number of vertices of the graph (and hence by the number of variables of the instances). Moreover, we can underline that the hardness in solving such instances has an increasing path, which differs from that one almost-unimodal observed with the second generator. This was, in a way, an expected behavior since instances generated by the third generator are harder and of special type with respect to the other which are almost generic.

11 4.1. Crossover location An empirical complexity study for a 2CPA solver 11 In figure 1, it appears that the crossover value should be close to 1. This, by the way, is the same results for the transition phase point for 2SAT [12,7,6]. Recall that the crossover is defined as the value α c such that, in mean, the 50% of the instances are coherent. Hence, to better locate its value, we have used the following strategy. First of all, for each run (of instances) with a fixed value of the parameter n, we have estimated the critical value α c (n). Its computation has been done by a linear regression among the values of cp against α with the restriction to the central range, i.e. cp [0.3, 0.7], where the paths are all almost linear. The particular value α c (n) has been computed as α c (n) = 0.5 a n b n where a n and b n are the least squares estimates of the linear regression. % coherent _c Number of vars Fig. 6. Linear regressions of cp against α and of α c against n Varying n, we obtained a mean value of with a standard deviation of Anyway, the value of α c (n) slightly decreases as n increases. Hence we have computed also the linear regression of α c (n) against n, obtaining the following linear relation, with R 2 = , α c (n) = n. These results are summarized in figure 4, where both linear regression and the locations of cp are plotted. 5. Conclusion We can conclude saying that the empirical results confirm our theoretical believes about the behavior of the procedure. What we plan to do in the next future is, from the theoretical point of view, to try to prove that the real crossover point is 1, while from the operational side, to search for a more efficient algorithm which allows to solve instances with a higher number of variables.

12 12 M. Baioletti et al. References 1. M. Baioletti, A. Capotorti, S. Tulipani, B. Vantaggi (2000) Elimination of Boolean variables for probabilistic coherence. Soft Computing vol. 4 N. 2, M. Baioletti, A. Capotorti, S. Tulipani, B. Vantaggi (2002) Simplification Rules for the Coherent Probability Assessment Problem. Ann. of Math. and Artif. Intell., Vol. 35, M. Baioletti, A. Capotorti, S. Tulipani (2003) Polynomial average time complexity of an algorithm for 2CPA. submitted to Information and Computation. 4. M. Baioletti, A. Capotorti, S. Tulipani (2003) Procedures for the CPA problem based on the elimination of Boolean constraints, in Proceedings of Sixth International Conference on Theory and Applications of Satisfiability Testing SAT 03, S.Margherita Ligure - Portofino (Italy), May G.Boole (1854) An investigation of the laws of thought. London: Walton and Maberley, (reprint New York: Dover 1958). 6. V. Chvatal, B. Read (1992) Mick gets some (the odds arc on his side), in Proocedings of 33 rd Ann. Symp. on Foundat. of Computer Science, J.M. Crawford, L.D. Auton (1993) Experimental results on the crossover point in SAT problem, in Proceedings of 8 th AAAI, G. Coletti, R. Scozzafava (2002) Probabilistic Logic in a Coherent Setting, Dordrecht: Kluwer, Series Trends in Logic. 9. B. de Finetti (1970) Teoria della probabilità vol.i,ii. Torino: Einaudi (Engl. Transl. (1974) Theory of probability vol.i,ii, London: Wiley & Sons). 10. J. Franco and M. Paull (1983) Probabilistic analysis of the Davis Putnam procedure for solving the satisfiability problem. Discrete Applied Math. 5, 1983, G.Georgakopoulos, D.Kavvadias and C.H.Papadimitriou (1988) Probabilistic Satisfiability. Journal of Complexity, 4, A. Goerdt (1992) A treshold for unsatisfiability, Proceedings of 17 t h Int. Symp. Mathematical Foundations of Computer Science, A. Goerdt (1999) A remark on random 2-SAT, Discrete Applied Mathematics,96-97, A. Goldberg, Jr. P.W. Purdom and C.A. Brown (1982) Average time analysis of simplified Davis-Putnam procedures. Information Process. Lett., vol. 15, 72 75; see also Errata, vol. 16, 1983, p T.Hailperin (1986) Boole s logic and probability, studies in logic and the foundations of mathematics 85. New York: North Holland. 16. P.Hansen, B.Jaumard, M.Poggi de Aragão (1995) Boole s Conditions of Possible Experience and Reasoning Under Uncertainty. Discrete Applied Mathematics, 60, D. Mitchell, B. Selman and H. Levesque (1992) Hard and easy distributions of SAT problems. In Proceedings of the Tenth National Conference on Artificial Intelligence (AAAI-92), San Jose, CA, July 1992,