Artificial Intelligence and Mathematics

Transcription

1 Eighth International Symposium on Artificial Intelligence and Mathematics January 4-6, 2004 Fort Lauderdale, Florida

2 Organizing Committee General Chair Martin Golumbic University of Haifa, Israel Program Co-Chairs Fahiem Bacchus University of Toronto, Canada Peter van Beek University of Waterloo, Canada Conference Chair Frederick Hoffman Florida Atlantic University, USA Publicity Chair George Katsirelos University of Toronto, Canada Program Committee Franz Baader (TU Dresden, Germany) Peter Bartlett (UC Berkeley, USA) Endre Boros (Rutgers, USA) Adnan Darwiche (UCLA, USA) Rina Dechter (UC Irvine, USA) Boi Faltings (EPFL, Switzerland) Ronen Feldman (Bar-Ilan University, Israel) John Franco (U. of Cincinnati, USA) Hector Geffner (UPF, Spain) Ian Gent (U. of St. Andrews, UK) Enrico Giunchiglia (U. of Genova, Italy) Robert Givan (Purdue, USA) Joerg Hoffmann (Albert-Ludwigs, Germany) Holger Hoos (UBC, Canada) Peter Jonsson (U. of Linkoeping, Sweden) Henry Kautz (U. of Washington, USA) Sven Koenig (Georgia Tech, USA) Richard Korf (UCLA, USA) Gerhard Lakemeyer (RWTH Aachen, Germany) Omid Madani (U. of Alberta, Canada) Heikki Mannila (U. of Helsinki, Finland) Anil Nerode (Cornell, USA) Ronald Parr (Duke, USA) Pascal Poupart (U. of Toronto, Canada) Jeff Rosenschein (Hebrew University, Israel) Sam Roweis (U. of Toronto, Canada) Dale Schuurmans (U. of Alberta, Canada) Allen Van Gelder (UCSC, USA) Toby Walsh (4C, Ireland) Sponsors The Symposium is partially supported by the Annals of Math and AI and Florida Atlantic University. Additional support is provided by the Caesarea Edmond Benjamin de Rothschild Institute at the University of Haifa and by the Florida-Israel Institute.

3 Contents Production Inference, Nonmonotonicity and Abduction Alexander Bochman Spines of Random Constraint Satisfaction Problems: Definition and Impact on Computational Complexity Stefan Boettcher, Gabriel Istrate, and Allon G. Percus Interval-Based Multicriteria Decision Making Martine Ceberio and François Modave Using Logic Programs to Reason about Infinite Sets Douglas Cenzer, V. Wiktor Marek, and Jeffrey B. Remmel The Expressive Rate of Constraints Hubie Chen Using the Central Limit Theorem for Belief Network Learning Ian Davidson and Minoo Aminian Approximate Probabilistic Constraints and Risk-Sensitive Optimization Criteria in Markov Decision Processes Dmitri A. Dolgov and Edmund H. Durfee Generalized Opinion Pooling Ashutosh Garg, T.S. Jayram, Shivakumar Vaithyanathan, and Huaiyu Zhu A Framework for Sequential Planning in Multi-Agent Settings Piotr J. Gmytrasiewicz and Prashant Doshi Heuristics for a Brokering Set Packing Problem Y. Guo, A. Lim, B. Rodrigues, and Y. Zhu Combining Symmetry Breaking with Other Constraints: Lexicographic Ordering with Sums Brahim Hnich, Zeynep Kiziltan, and Toby Walsh A Simple Yet Effective Heuristic Framework for Optimization Problems Gaofeng Huang and Andrew Lim Biased Minimax Probability Machine for Medical Diagnosis Kaizhu Huang, Haiqin Yang, Irwin King, Michael R. Lyu, and Laiwan Chan Combining Cardinal Direction Relations and Relative Orientation Relations in QSR Amar Isli Unrestricted vs Restricted Cut in a Tableau Method for Boolean Circuits Matti Järvisalo, Tommi A. Junttila, and Ilkka Niemelä Parameter Reusing in Learning Latent Class Models Gytis Karčiauskas, Finn Jensen, and Tomás Kočka New Look-Ahead Schemes for Constraint Satisfaction Kalev Kask, Rina Dechter, and Vibhav Gogate Learning via Finitely Many Queries Andrew C. Lee

4 Modeling and Reasoning with Star Calculus Debasis Mitra Analysis of Greedy Robot-Navigation Methods Apurva Mugdal, Craig Tovey, and Sven Koenig Symmetry Breaking in Constraint Satisfaction with Graph-Isomorphism: Comma-Free Codes Justin Pearson Deductive Algorithmic Knowledge Riccardo Pucella Inferring Implicit Preferences from Negotiation Actions Angelo Restificar and Peter Haddawy Improving Exact Algorithms for MAX-2-SAT Haiou Shen and Hantao Zhang Explicit Manifold Representations for Value-Function Approximation in Reinforcement Learning William D. Smart Warped Landscapes and Random Acts of SAT Solving Dave A. D. Tompkins and Holger H. Hoos Using Automatic Case Splits and Efficient CNF Translation to Guide a SAT-Solver When Formally Verifying Out-of-Order Processors Miroslav N. Velev Multi-Agent Dialogue Protocols Christopher D. Walton Bayesian Model Averaging Across Model Spaces via Compact Encoding Ke Yin and Ian Davidson Crane Scheduling with Spatial Constraints: Mathematical Model and Solving Approaches Yi Zhu and Andrew Lim Papers from the Special Session on Intelligent Text Processing Organizer: Shlomo Argamon, Illinois Institute of Technology, USA Efffective Use of Phrases in Language Modeling to Improve Information Retrieval Maojin Jiang, Eric Jenson, Steve Beitzel, and Shlomo Argamon Text Categorization for Authorship Verification Moshe Koppel, Jonathan Schler, and Dror Mughaz Mapping Dependencies Trees: An Application to Question Answering Vasin Punyakanok, Dan Roth, and Wen-tau Yih A Linear Programming Formulation for Global Inference in Natural Language Tasks Dan Roth and Wen-tau Yih

5 Production Inference, Nonmonotonicity and Abduction Alexander Bochman Computer Science Department, Holon Academic Institute of Technology, Israel Abstract We introduce a general formalism of production inference relations that posses both a standard monotonic semantics and a natural nonmonotonic semantics. The resulting nonmonotonic system is shown to provide a syntax-independent representation of abductive reasoning. Abduction is a reasoning from facts to their possible explanations that is widely used now in many areas of AI, including diagnosis, truth maintenance, knowledge assimilation, database updates and logic programming. In this study we are going to show that this kind of reasoning can be given a formal, syntax-independent representation in terms of production inference relations that constitute a particular formalization of input-output logics [MdT00]. Among other things, such a representation will clarify the relation between abduction and nonmonotonic reasoning, as well as show the expressive capabilities of production inference as a general-purpose nonmonotonic formalism. We will assume that our basic language is a classical propositional language with the usual connectives and constants {,,,, t, f}. will denote the classical entailment, and Th the associated provability operator. 1 Production Inference Relations We begin with the following general notion of production inference. 1

6 Definition 1.1. A production inference relation is a binary relation on the set of classical propositions satisfying the following conditions: (Strengthening) If A B and B C, then A C; (Weakening) If A B and B C, then A C; (And) If A B and A C, then A B C; (Truth) t t; (Falsity) f f. A distinctive feature of production inference relations is that reflexivity A A does not hold. It is this omission, however, that determines their representation capabilities in describing nonmonotonicity and abduction. In what follows, conditionals A B will be called production rules. We extend such rules to rules having arbitrary sets of propositions in premises as follows: for a set u of propositions, we define u A as holding when a A, for some finite a u. C(u) will denote the set of propositions produced by u, that is, C(u) = {A u A}. The production operator C will play much the same role as the derivability operator for consequence relations. 1.1 Kinds of production inference A classification of the main kinds of production inference relevant for our study is based on the validity of the following additional postulates: (Cut) If A B and A B C, then A C. (Or) If A C and B C, then A B C. (Weak Deduction) If A B, then t (A B). A production inference relation will be called regular, if it satisfies Cut; basic, if it satisfies Or 1 ; causal if it is both basic and regular; and quasiclassical, if it is causal and satisfies Weak Deduction. The rule Cut allows for a reuse of produced propositions as premises in further productions. Any regular production relation will be transitive. 1 Basic production relations correspond to basic input-output logics from [MdT00], while regular productions correspond to input-output logics with reusable output. 2

7 The rule Or allows for reasoning by cases, and hence basic production relations can already be seen as systems of objective production inference, namely as systems of reasoning about complete worlds (see below). Causal production relations have been introduced in [Boc03]; they have been shown to provide a complete characterization for the reasoning with causal theories from [MT97]. Finally, quasi-classical production relations will be shown below to characterize classical abductive reasoning. 1.2 The Monotonic Semantics A semantic interpretation of production relations is based on pairs of deductively closed theories called bimodels. By the input-output understanding of productions, a bimodel represents an initial state (input) and a possible final state (output) of a production process. Definition 1.2. A pair of classically consistent deductively closed sets of propositions will be called a bimodel. A set of bimodels will be called a production semantics. Note that a production semantics can also be viewed as a binary relation on the set of deductive theories. Definition 1.3. A production rule A B will be said to be valid in a production semantics B if, for any bimodel (u, v) from B, A u only if B v. Then the following completeness result can be shown: Theorem 1.1. A relation on the set of propositions is a production inference relation if and only if it is determined by a production semantics. This representation result serves as a basis for semantic characterizations of different kinds of production relations, described above. The semantics of regular production relations can be obtained by considering only bimodels (u, v) such that v u. We will call such bimodels (and corresponding semantics) inclusive ones. Theorem 1.2. is a regular production relation iff it is generated by an inclusive production semantics. 3

8 The semantics for the three other kinds of production is obtained by restricting the set of bimodels to bimodels of the form (α, β), where α, β are worlds. The corresponding production semantics can be seen as a relational possible worlds model W = (W, B), where W is a set of worlds with an accessibility relation B. Validity of productions can now be defined as follows: Definition 1.4. A rule A B is valid in a possible worlds model (W, B) if, for any α, β W such that αbβ, if A holds in α, then B holds in β. A possible worlds model (W, B) will be called reflexive, if αbα, for any world α, and quasi-reflexive, if αbβ implies αbα, for any α, β W. Theorem 1.3. A production inference relation is basic if and only if it has a possible worlds model. causal iff it has a quasi-reflexive possible worlds model. quasi-classical iff it has a reflexive possible worlds model. 1.3 The Nonmonotonic Semantics The fact that the production operator C is not reflexive creates an important distinction between theories of a production inference relation. Definition 1.5. A set u of propositions is a theory of a production relation, if it is deductively closed, and C(u) u. A theory is exact, if u = C(u). A theory of a production relation is closed with respect to its production rules, while an exact theory describes an informational state in which every proposition is also produced, or explained, by other propositions accepted in this state. Accordingly, restricting our universe of discourse to exact theories amounts to imposing a kind of an explanatory closure assumption on admissible states. This suggests the following notion: Definition 1.6. A (general) nonmonotonic semantics of a production inference relation is the set of all its exact theories. The above nonmonotonic semantics is indeed nonmonotonic, since adding new rules to the production relation may lead to a nonmonotonic change of the associated semantics, and thereby to a nonmonotonic change in the 4

9 derived information. This happens even though production rules themselves are monotonic, since they satisfy Strengthening the Antecedent. Exact theories are precisely the fixed points of the production operator C. Since the latter operator is monotonic and continuous, exact theories (and hence the nonmonotonic semantics) always exist. Since a basic production inference is already world-based, it naturally sanctions the following strengthening of the general nonmonotonic semantics. Definition 1.7. An objective nonmonotonic semantics of a (basic) production inference relation is the set of all its exact worlds. As has been shown already in [MT97], the above semantics is representable as a set of all worlds (interpretations) that satisfy a certain classical completion of the set of production rules. 2 Abduction versus Production An abductive framework can be defined as a pair A = (Cn, A), where Cn is a consequence relation that subsumes classical entailment 2, while A is a distinguished set of propositions that play the role of abducibles, or explanations, for other propositions. A proposition B is explainable in an abductive framework A if there exists a consistent set of abducibles a A such that B Cn(a). It turns out that explanatory relations in an abductive framework can be captured by considering only theories that are generated by the abducibles. Definition 2.1. The abductive semantics AS of an abductive framework A is the set of theories {Cn(a) a A}. The information embodied in the abductive semantics can be made explicit by considering the following generated Scott consequence relation: b A c ( u AS)(b u c u ) b A c holds if and only if any set of abducibles that explains b explains also at least one proposition from c. 3 This consequence relation is an extension of Cn that describes not only forward explanatory relations, but also 2 Such consequence relations are called supraclassical. 3 A Tarski consequence relation of this kind has been used for the same purposes in [LU97]. 5

10 abductive inferences from propositions to their explanations. For example, if C and D are the only abducibles that imply A in an abductive framework, then we will have A A C, D. Speaking more generally, the above abductive consequence relation describes the explanatory closure, or completion, of an abductive framework, and allow thereby to capture the abductive process by deductive means (see [CDT91, Kon92]). The following definition arises from viewing explanation as a kind of production inference. Definition 2.2. A production inference relation associated with an abductive framework A is a production relation A determined by all bimodels of the form (u, Cn(u A)), where u is a consistent theory of Cn. Since the above production semantics is inclusive, the associated production relation will always be regular. Moreover, the following result shows how it is related to the source abductive framework. Theorem 2.1. The abductive semantics of an abductive framework coincides with the nonmonotonic semantics of its associated production relation. We assume below that the set A is closed with respect to conjunctions, that is, if A and B are abducibles, so is A B. To deal with limit cases, we assume also that t and f are abducibles. Then it turns out that the above production relation admits a very simple syntactic characterization, namely B A C iff ( A A)(A Cn(B) & C Cn(A)) Note that A A A holds if and only if A is Cn-equivalent to an abducible from A. Accordingly, we will say that A is an abducible of a production inference relation, if A A. The set of such abducibles is closed with respect to conjunctions. Now, production relations associated with abductive frameworks satisfy the following characteristic property: (Abduction) If B C, then B A C, for some abducible A. Regular production relations satisfying Abduction will by called abductive production relations. For such relations, the production process always goes through abducibles. The next theorem shows that they are precisely production inference relations that are generated by abductive frameworks. Theorem 2.2. A production relation is abductive if and only if it is generated by an abductive framework. 6

11 Due to the above results, abductive production relations can be seen as a faithful logical representation of abductive reasoning. Notice that abductive production relations provide in this sense a syntax-independent description of abduction: the set of abducibles is determined as a set of propositions having a certain logical property, namely reflexivity. The abductive subrelation. Any regular production relation includes an important abductive subrelation defined as follows: A a B ( C)(A C C B) a is the greatest abductive relation included in, and in many natural cases it produces the same nonmonotonic semantics (see [Boc03]). If A is the set of abducibles of, and Cn the least supraclassical consequence relation including, then the following result can be shown: Lemma 2.3. If is a regular production relation, then its abductive subrelation a is generated by the abductive framework (Cn, A ). 2.1 Causal and classical abductive inference Abductive frameworks corresponding to causal production relations are described in the next definition. Definition 2.3. An abductive framework A = (Cn, A) will be called A- disjunctive if A is closed with respect to disjunctions, and Cn satisfies the following conditions, for any abducibles A, A 1 A, and arbitrary B, C: If A Cn(B) and A Cn(C), then A Cn(B C); If B Cn(A) and B Cn(A 1 ), then B Cn(A A 1 ). 4 Theorem 2.4. An abductive production relation is causal if and only if it is generated by an A-disjunctive abductive framework. As we already mentioned, the objective nonmonotonic semantics of such production relations is obtainable by forming a classical completion of the set of production rules (cf. [CDT91]). 4 This rule corresponds to the rule Ab-Or in [LU97]. 7

12 An abductive framework will be called classical if Cn is a classical consequence relation (that is, it is supraclassical and satisfies the deduction theorem). Such a framework is reducible to a pair (Σ, A), where Σ is a domain theory (with the implicit assumption that the background logic is classical). Our last result provides a production counterpart of such frameworks. Theorem 2.5. An abductive production relation is quasi-classical if and only if it is generated by a classical abductive framework. An interesting negative consequence from the above result is that classical abductive frameworks are already inadequate for reasoning with causal theories of McCain and Turner; the latter is captured, however, by a broader class of A-disjunctive abductive frameworks. References [Boc03] A. Bochman. A logic for causal reasoning. In G. Gottlob and T. Walsh, editors, Proceedings Int. Joint Conference on Artificial Intelligence, IJCAI 03, pages , Acapulco, Morgan Kaufmann. [CDT91] L. Console, D. Theseider Dupre, and P. Torasso. On the relationship between abduction and deduction. Journal of Logic and Computation, 1: , [Kon92] [LU97] K. Konolige. Abduction versus closure in causal theories. Artificial Intelligence, 53: , J. Lobo and C. Uzcátegui. Abductive consequence relations. Artificial Intelligence, 89: , [MdT00] D. Makinson and L. Van der Torre. Input/Output logics. Journal of Philosophical Logic, 29: , [MT97] N. McCain and H. Turner. Causal theories of action and change. In Proceedings AAAI-97, pages ,

13 Spines of random Constraint Satisfaction Problems: definition and impact on Computational Complexity Stefan Boettcher, Gabriel Istrate and Allon G. Percus 1 Introduction The major promise of phase transitions in combinatorial problems was to shed light on the practical algorithmic complexity of combinatorial problems. A possible connection has been highlighted by the results (based on experimental evidence and nonrigorous arguments from statistical mechanics) of Monasson et al. [1, 2]. Studying a version of random satisfiability that interpolates between 2-SAT and 3-SAT, they concluded that the order of the phase transition, combinatorially expressed by continuity of an order parameter called the backbone, might have algorithmic implications for the complexity of the important class of Davis-Putnam-Longman-Loveland (DPLL) algorithms [3]. A discontinuous or first-order transition was symptomatic of exponential complexity, whereas a continuous or second-order transition correlated with polynomial complexity. It is well understood by now that this connection is limited. For instance, k-xor-sat is a problem believed, based on arguments from statistical mechanics [4], to have a first-order phase transition. But it is easily solved by a polynomial algorithm, Gaussian elimination. One way to clarify the connection between phase transitions and computational complexity is to formalize the underlying intuition in a purely combinatorial way, devoid of any physics considerations. Firstorder phase transitions amount to a discontinuity in the (normalized) size of the backbone. For random k-sat [5], and more specifically for the optimization problem MAX-k-SAT, the backbone has a combinatorial interpretation: it is the the set of literals that are frozen (assume the same value) in all optimal assignments. Intuitively, a large backbone size has implications for the complexity of finding such assignments: all literals in the backbone require well-defined values in order to satisfy the formula, but an algorithm assigning variables in an iterative fashion has very few ways to know what the right values to assign are. In the case in a first-order phase transition, the backbone of formulas just above the transition contains with high probability a fraction of the literals that is bounded away from zero. DPLL algorithms would then misassign a variable having Ω(n) height in the tree representing the behavior of the algorithm, forcing it to backtrack on the given variable. Assuming the algorithm cannot significantly reduce the size of the explored portion of this tree, a first-order phase transition would then w.h.p imply a 2 Ω(n) lower bound for the running time of DPLL on random instances located slightly above the transition. There exists, however, a significant flaw in the heuristic argument above: the backbone is defined with respect to optimal solutions, and would seem to imply that it is difficult to find solutions to, e.g., MAX- K-SAT using algorithms that assign variables iteratively. But why should the continuity/discontinuity of the backbone be the relevant predictor for the complexity of the (often easier) decision problem, which is what DPLL algorithms try to solve anyway? stb@physics.emory.edu Emory University, Atlanta, GA {istrate,percus}@lanl.gov Los Alamos National Laboratory, Los Alamos, NM Correspondence to: Gabriel Istrate.

14 Fortunately, it turns out that the intuition of the previous argument also holds for a different order parameter, a weaker version of the backbone called the spine, introduced in [6] in order to prove that random 2-SAT has a second-order phase transition. Unlike the backbone, the spine is defined in terms of the decision problem, hence it could conceivably have a larger impact on the complexity of these problems. Of course, the same caveat applies as for the backbone: we are referring to complexity with respect to classes of algorithms weaker than polynomial time computations, and that in particular are not strong enough to capture the polynomial time Gaussian elimination algorithm for k-xor-sat. We aim in this paper to provide evidence that the behavior of the spine, rather than the backbone, impacts the complexity of the underlying decision problem. To accomplish this: 1. We discuss the proper definition of the backbone/spine for random CSP. 2. We formally establish a simple connection between a discontinuity in the (relative size of the) spine at the threshold and the resolution complexity of random satisfiability problems. In a nutshell, a necessary and sufficient condition for the existence of a discontinuity in the spine is the existence of a Ω(n) lower bound (w.h.p.) on the size of minimally unsatisfiable subformulas of a random (unsatisfiable) subformula. But standard methods from proof complexity [7] imply that (in conjunction with the expansion of the formula hypergraph, independent of the precise definition of the problem at hand) in all cases where we can prove the existence of a first-order phase transition, such problems have a 2 Ω(n) lower bound on their resolution complexity (and hence the complexity of DPLL algorithms as well [3]). In contrast, we show (Theorem 1) that, for any generalized satisfiability problem, a second-order phase transition implies, for every α>0, ao(2 α n ) upper bound on the resolution complexity of their random instances (in the region where most formulas are unsatisfiable) 3. We give a sufficient condition (Theorem 2) for the existence of a discontinuous jump in the size of the spine. We then show (Theorem 3) that all problems whose constraints have no implicates of size at most two satisfy this condition. Qualitatively, our results suggest that all satisfiability problems with a second-order phase transition in the spine are like 2-SAT. 4. Finally, we present some experimental results that attempt to clarify the issue whether the backbone and the spine can behave differently at the phase transition. The Graph bipartition problem is one case where this seems to happen. In contrast, the backbone and spine of random 3-coloring seem to have similar behavior. A note on the significance of our results: a discontinuity in the spine is weaker than a first-order phase transition (i.e., a discontinuity in the size of the backbone). Also, unlike the backbone, the spine has no physical interpretation. But this is not our intention: we have seen that the argument connecting the backbone size and the complexity of decision problems is problematic. What we rigorously show (with no physics considerations in mind) is that the intuitive argument holds for the spine order parameter. Moreover, the last section of the paper presents experimental work suggesting that the backbone and the spine can behave differently. 2 Preliminaries Throughout the paper we will assume familiarity with the general concepts of phase transitions in combinatorial problems (see e.g., [8]), random structures [9], proof complexity [10]. Some papers whose concepts and methods we use in detail (and we assume greater familiarity with) include [11], [12], [7]. We will use the model of random constraint satisfaction from Molloy [13]:

15 Definition 1 Let D = {0, 1,...,t 1}, t 2 be a fixed set. Consider all 2 tk 1 possible nonempty sets of constraints (relations) on k variables X 1,...,X k with values taken from D. Let C be such a nonempty set of constraints. A random formula from CSP n,m (C) is specified by the following procedure: n is the number of variables. m is the number of clauses, chosen by the following procedure: first select, uniformly at random and with replacement, m hyperedges of the complete k-uniform hypergraph on n variables. for each hyperedge, choose a random ordering of the variables involved. Choose a random constraint from C and apply it on the list of (ordered) variables. We use the notation SAT(C) (instead of CSP(C)) when t=2. Also, for Φ an instance of CSP(C) we denote by opt C (Φ) the smallest number of constraints left unsatisfied by some assignment. Just as in random graphs [9], under fairly liberal conditions one can use the constant probability model instead of the counting model from the previous definition. The interesting range of the parameter m is when the ratio m/n is a constant, c, the constraint density (details are left for the final version of the paper). The original investigation of the order of the phase transition in k-sat used an order parameter called the backbone, B(Φ) = {x Lit λ {0, 1} : Ξ MAXSAT(Φ), Ξ(x) =λ}, (1) or more precisely the backbone fraction f, the fraction of the n variables that belong to B(Φ). Bollobás et al. [6] have investigated the order of the phase transition in k-sat (for k =2) under a different order parameter, a monotonic version of the backbone called the spine. S(Φ) = {x Lit Ξ Φ:Ξ k SAT, Ξ {x} k SAT}. (2) They showed that random 2-SAT has a continuous (second-order) phase transition: the size of the spine, normalized by dividing it by the number of variables, approaches zero (as n ) for c<c 2 SAT = 1, and is continuous at c = c 2 SAT. By contrast, nonrigorous arguments from statistical mechanics [5] imply that for 3-SAT the backbone jumps discontinuously from zero to positive values at the transition point c = c 3 SAT (a first-order phase transition). 3 How to define the backbone/spine for random CSP (and beyond) We would like to extend the concepts of backbone and spine to general constraint satisfaction problems. Certain differences between the case of random k-sat and the general case force us to employ an alternative definition of the backbone/spine. The most obvious is that formula (2) involves negations of variables, unlike Molloy s model. Also, these definitions are inadequate for problems whose solution space presents a relabelling symmetry, such as the case of graph coloring where the set of (optimal) colorings is closed under permutations of the colors. Due to this symmetry, no variable can be frozen in this way. The new definitions have to retain as many of the properties of the backbone/spine as possible. In particular, the new definitions must give rise to order parameters, i.e., quantities that are zero up to the critical value of the control parameter (in our case constraint density c) and positive above it. The formal statement that we wish to extend to CSP(C) is presented next for the spine:

16 Lemma 3.1 Let c be an arbitrary constant value for the constraint density function. 1. If c<lim n c k SAT (n) then lim n S(Φ) n =0. 2. If for some c there exists δ>0such that w.h.p. (as n ) S(Φ) n SAT]=0, that is c>lim n c k SAT (n). >δthen lim n Prob[Φ Our solution is to define the backbone/spine of a random instance of CSP(C) slightly differently. Definition 2 B(Φ) = {x Var C C: x C, opt C (Φ C) >opt C (Φ)}, S(Φ) = {x Var C Cand Ξ Φ:x C, Ξ CSP, Ξ C CSP}. For k-cnf formulas whose (original) backbone/spine contains at least three literals, a variable x is in the (new version of the) backbone/spine if and only if either x or x were present in the old version. In particular the new definition does not change the order of the phase transition of random k-sat. Alternatively, in studying 3-colorability of random graphs G =(V,E), Culberson and Gent [14] define S(G) ={(x, y) V 2 H G : H 3-COL,H E(x, y) 3-COL}, so one may define the relative backbone and spine sizes in terms of constraints rather than variables. Definition 3 B(Φ) = {C C opt C (Φ C) >opt C (Φ)}, S(Φ) = {C C Ξ Φ:Ξ CSP, Ξ C CSP}. Since we are looking at a combinatorial definition, with no physics considerations in mind, the only principled way to choose between the two types of order parameters (one based on variables, the other based on constraints) is to look at the class of algorithms we are concerned with. In the case of random constraint satisfaction problems (and DPLL algorithms) it is variables that get assigned values, so Definition 2 is preferred. On the other hand, we will see an example in a later section (the case of graph bipartitioning) where it makes more sense to use Definition 3. Finally, note that if the backbone/spine of an instance of CSP(C) (in the sense of definition 2) has size u, then the backbone/spine (in the sense of definition 3) has size O(u k ). It follows readily that the discontinuity of the second version of backbone/spine implies the discontinuity of the corresponding first version. This will notably be the case in our experimental study of 3-COL. 4 Spine discontinuity and resolution complexity of random CSP Definition 4 Let C be such that SAT(C) has a sharp threshold. Problem SAT(C) has a discontinuous spine if there exists η>0 such that for every sequence m = m(n) we have lim Prob [Φ SAT]=0 lim Prob [ S(Φ) η] =1. (3) n m=m(n) n m=m(n) n If, on the other hand, for every ɛ>0there exists m ɛ (n) with lim Prob n m=m ɛ (n) we say that SAT(C) has a continuous spine. [Φ SAT] = 0 and lim Prob [ S(Φ) ɛ] =0 (4) n n m=m ɛ (n)

17 Claim 1 Let Φ be minimally unsatisfiable, and let x be a literal that appears in Φ. Then x S(Φ). Corollary 1 k-sat, k 3 has a discontinuous spine. The resolution complexity of an instance Φ of SAT(C) is defined as the resolution complexity of the formula obtained by converting each constraint of Φ to CNF-form. A simple observation is that a continuous spine has implications for resolution complexity: Theorem 1 Let C be a set of constraints such that SAT(C) has a continuous spine. Then for every value of the constraint density c>lim n c SAT(C) (n), and every α>0, random formulas of constraint density c have w.h.p. resolution complexity O(2 k α n ). Definition 5 For a formula F define c (F )=max{ Constraints(G) : G F }. Var(G) The next result gives a sufficient condition for a generalized satisfiability problem to have a discontinuous spine. Interestingly, it is one condition studied in [13]. Theorem 2 Let C be such that SAT(C) has a sharp threshold. If there exists ɛ>0such that for every minimally unsatisfiable formulas F it holds that c (F ) > 1+ɛ, then SAT(C) has a discontinuous spine. k 1 One can give an explicitly defined class of satisfiability problems for which the previous result applies: Theorem 3 Let C be such that SAT(C) has a sharp threshold. If no clause C Chas an implicate of length at most 2 then 1. for every minimally unsatisfiable formula F, c (F ) 2. Therefore SAT(C) satisfies the conditions of the previous theorem, i.e., it has a discontinuous 2k 3 spine. 2. Moreover SAT(C) also has 2 Ω(n) resolution complexity 1. The condition in the theorem is violated (as expected) by random 2-SAT, as well as by the random version of the NP-complete problem 1-in-k SAT. This problem can be represented as CSP(C), for C a set of 2 k constraints (corresponding to all ways to negate some of the variables) and has a rigorously determined 2-SAT like location of the transition point [17]. But the formula C(x 1,x 2,...,x k 1,x k ) C(x k,x k+1,...,x 2k 2,x 1 ) C(x 1,x 2k 1,...,x 3k 3, x k ) C(x k,x 3k 2,...,x 4k 4,x 1 ), where C is the constraint 1-in-k, is minimally unsatisfiable but has clause/variable ratio 1/(k 1) and implicates x 1 x k and x 1 x k. 4.1 Threshold location and discontinuous spines Molloy [13] has shown (Theorem 3 in his paper) that the condition that turned out to be sufficient for the existence of a phase transition has implications for the location of the threshold. A natural question therefore arises: is it possible to read the order of the phase transition from the location of the threshold? We cannot answer this question in full. However, we have already seen two problems that do not satisfy our sufficient condition for a discontinuous spine: random 2-SAT, for which the transition has been proven to be of second order [6], and random 1-in-k SAT, for which we believe a similar result holds [17]. Molloy s result does not provide the correct location of the threshold for these two problems. It is, however, striking, that for both problems the actual location of the threshold is twice the value given by Theorem 3 in [13], at clause/variable ratio 2/k(k 1). We give, therefore the following observation, a variant of Molloy s result (proved in the journal version of the paper). 1 this result subsumes some of the results in [15]. Related results have been given independently in [16]

18 Proposition 1 Modify the random model from Definition 1 to: allow application of constraints to both variables and negated variables. only allow constraints that cannot be made equal via negation of some of their variables. Denote by CSP neg (C) the new random model. Let C be such that for every minimally unsatisfiable subformula F whose constraints are drawn from F the ratio of constraints to variables of F is at least 1+ɛ, for some ɛ>0. Then there is a constant δ>0 k 1 such that CSP neg (C) is a.s satisfiable for m 2 (1 + δ). k(k 1) 5 Beyond random satisfiability: comparing the behavior of the backbone and spine In this section we investigate empirically the continuity of the backbone for two graph problems, random three coloring (3-COL) and graph bipartition (GBP). We consider a large number of instances of random graphs, of sizes up to n = 1024 and over a range of mean degree values near the threshold. for each instance we determine the backbone fraction f. Culberson and Gent [14] have shown experimentally that the 3-COL spine (as defined in Definition 3) exhibits a discontinuous transition. To be consistent with this study (and since it leads to a stronger result anyhow) we use the backbone from the same definition. We employ a rapid heuristic called extremal optimization [18] that, based on testbed comparisons with an exact algorithm, yields an excellent approximation of f around the critical region. Figure 1 shows f as a function of mean degree. Above the threshold, for 3-COL (Fig. 1a) f does not appear to vanish, suggesting a discontinuous large n backbone. Culberson and Gent have speculated that at the 3-COL threshold, although their spine is discontinuous, the backbone might be continuous. Our numerical results suggest otherwise. We next study graph bipartitioning (GBP). This problem cannot, strictly speaking, be cast in the setup of random constraint satisfaction problems from Definition 1, since not every partition of vertices of G is allowed. It can be cast to a variant of this model (with variables associated to nodes, values associated to each partition and constraint x = y associated to the edge between the corresponding vertices) but we must add the additional requirement that all satisfying assignments contain an equal number of ones and zeros. Thus the complexity-theoretic observations of Section 4 do not automatically apply to it. We can, however, give a DPLL-like class of algorithms for GBP, that assigns vertices (variables) in pairs, one to each partition. This class of algorithms motivates investigating the backbone/spine under the model in Definition 3. It is easy to see that the spine of a GBP instance contains all edges belonging to a connected component of size larger than n/2. Since the phase transition in GBP takes place where the giant component becomes larger than n/2, GBP has a discontinuous spine. The backbone (Fig. 1b), on the other hand, appears to remains continuous, vanishing at large n on both sides of the threshold. One obvious question raised by the previous result is whether the discontinuity of the spine in GBP really has computational implications for the complexity of deciding whether a perfect bipartition exists. After all, unlike 3-COL, GBP can easily be solved in polynomial time by dynamic programming. This situation, however, is similar to that of XOR-SAT, where a polynomial algorithm exists but the complexity of resolution proofs/dpll algorithms is exponential. The class of DPLL-like algorithms outlined for GBP can no longer be simulated in the straightforward manner by resolution proofs, however it can be simulated using proof systems Res(k) that are extensions of resolution [19]. Some of the hardness results for resolution extend to these more powerful

19 3 COL (a) GBP (b) Backbone Fraction Mean degree n=32 n=64 n=128 n=256 n=512 Backbone Fraction 0.1 n=32 n=64 n=128 n=256 n=512 n= Mean degree Figure 1: Plot of the estimated backbone fraction (a) for 3-COL and (b) for the GBP, on random graphs, as a function of mean degree α. For 3-COL, the systematic error based on benchmark comparisons with random graphs is negligible compared to the statistical error bars; for the GBP, f is found by exact enumeration. The thresholds α 4.73 for 3-COL and α =2ln2for the GBP are shown by dashed lines. proof systems, and in [20] we investigate the extent to which the results of this paper apply to this class of proof systems. These preliminary results imply that, indeed, the discontinuity of the spine does have computational implications for GBP. 6 Discussion We have shown that the existence of a discontinuous spine in a random satisfiability problem is often correlated with a 2 Ω(n) peak in the complexity of resolution/dpll algorithms at the transition point. The underlying reason is that the two phenomena (the jump in the order parameter and the resolution complexity lower bound) have common causes. The example of random k-xor-sat shows that a general connection between first-order phase transition and the complexity of the decision problems is hopeless: Ricci-Tersenghi et al. [4] have presented a non-rigorous argument using the replica method that shows that this problem has a first-order phase transition, and we can formally show (as a direct consequence of Theorem 3) the following weaker result: Proposition 2 Random k-xor-sat, k 3, has a discontinuous spine. However, experimental evidence in previous section suggests that the backbone and the spine do not always behave in the same way. Therefore our results (and the work in progress mentioned above) suggest that the continuity/discontinuity of the spine, rather than the backbone, is a predictor for the complexity of the restricted classes of decision algorithms that can be simulated by resolution-like proof systems. References [1] R. Monasson, R. Zecchina, S. Kirkpatrick, B. Selman, and L. Troyansky. Determining computational complexity from characteristic phase transitions. Nature, 400(8): , 1999.

20 [2] R. Monasson, R. Zecchina, S. Kirkpatrick, B. Selman, and L. Troyansky. (2+p)-SAT: Relation of typical-case complexity to the nature of the phase transition. Random Structures and Algorithms, 15(3 4): , [3] P. Beame, R. Karp, T. Pitassi, and M. Saks. The efficiency of resolution and Davis-Putnam procedures. SIAM Journal of Computing, 31(4): , [4] F. Ricci-Tersenghi, M. Weight, and R. Zecchina. Simplest random k-satisfiability problem. Physical Reviews E, 63:026702, [5] R. Monasson and R. Zecchina. Statistical mechanics of the random k-sat model. Physical Review E, 56:1357, [6] B. Bollobás, C. Borgs, J.T. Chayes, J. H. Kim, and D. B. Wilson. The scaling window of the 2-SAT transition. Random Structures and Algorithms, 18(3): , [7] E. Ben-Sasson and A. Wigderson. Short Proofs are Narrow:Resolution made Simple. Journal of the ACM, 48(2), [8] O. Martin, R. Monasson, and R. Zecchina. Statistical mechanics methods and phase transitions in combinatorial optimization problems. Theoretical Computer Science, 265(1-2):3 67, [9] B. Bollobás. Random Graphs. Academic Press, [10] P. Beame and T. Pitassi. Propositional proof complexity: Past present and future. In Current Trends in Theoretical Computer Science, pages [11] E. Friedgut. Necessary and sufficient conditions for sharp thresholds of graph properties, and the k-sat problem. with an appendix by J. Bourgain. Journal of the A.M.S., 12: , [12] V. Chvátal and E. Szemerédi. Many hard examples for resolution. Journal of the ACM, 35(4): , [13] M. Molloy. Models for random constraint satisfaction problems. In Proceedings of the 32nd ACM Symposium on Theory of Computing, [14] J. Culberson and I. Gent. Frozen development in graph coloring. Theoretical Computer Science, 265(1-2): , [15] D. Mitchell. Resolution complexity of Random Constraints. In Eigth International Conference on Principles and Practice of Constraint Programming, [16] M. Molloy and M. Salavatipour. The resolution complexity of random constraint satisfaction problems. In Proceedings of the 44th Symposium on Foundations in Computer Science, [17] D. Achlioptas, A. Chtcherba, G. Istrate, and C. Moore. The phase transition in random 1-in-k SAT and NAE 3-SAT. In Proceedings of the 13th ACM-SIAM Symposium on Discrete Algorithms, [18] S. Boettcher and A. Percus. Nature s way of optimizing. Artificial Intelligence, 119: , [19] J. Krajicek. On the weak pigeonhole principle. Fundamenta Matematicae, 170(1 3): , [20] G. Istrate. Descriptive complexity and first-order phase transitions. (in progress).

21 Appendix 6.1 Proof of Claim 1 Let C be a clause that contains x. By the minimal unsatisfiability of Φ, Φ \{C} SAT. On the other hand Φ \{C} {x} SAT, otherwise Φ would also be satisfiable. Thus x S(Φ \{C}). 6.2 Proof of Corollary 1 To show that 3-SAT has a discontinuous spine it is enough to show that a random unsatisfiable formula contains w.h.p. a minimally unsatisfiable subformula containing a linear number of literals. A way to accomplish this is by using the two ingredients of the Chvátal-Szemerédi proof [12] that random 3-SAT has exponential resolution size w.h.p. 6.3 Proof of Theorem 1 Proof. By the analog of Claim 1 for the general case, if SAT(C) has a continuous spine) then for every c > c SAT(C) and for every α > 0, minimally unsatisfiable subformulas of a random formula Φ with constraint density c have w.h.p. size at most α n. Consider the backtrack tree of the natural DPLL algorithm (that tries to satisfies clauses one at a time) on such a minimally unsatisfiable subformula F. By the usual correspondence between DPLL trees and resolution complexity (e.g., [3], pp. 1) it yields a resolution proof of the unsatisfiability of Φ having size at most 2 k α n Proof of Theorem 2 Proof. We first recall the following concept from [12]: Definition 6 Let x, y > 0. Ak-uniform hypergraph with n vertices is (x,y)-sparse if every set of s xn vertices contains at most ys edges. We also recall Lemma 1 from the same paper. Lemma 6.1 Let k, c > 0 and y be such that (k 1)y >1. Then w.h.p. the random k-uniform hypergraph with n vertices and cn edges is (x, y)-sparse, where ɛ = y 1/(k 1), x =( 1 2e ( y ce )y ) 1/ɛ. The critical observation is that the existence of a minimally unsatisfiable formula of size xn and with c (F ) > 1+ɛ implies that the k-uniform hypergraph associated to the given formula is not (x, y)-sparse, k 1 for y = ɛ. But, according to Lemma 6.1, w.h.p. a random k-uniform hypergraph with cn edges is (x k 1 0,y) sparse, for x 0 =( 1 ( y 2e ce )y ) 1/ɛ (a direct application of Lemma 1 in their paper). We infer that any formula with less than x 0 n/k constraints is satisfiable, therefore the same is true for any formula with x 0 n/k clauses picked up from the clausal representation of constraints in Φ. The second condition (expansion of the formula hypergraph) can be proved similarly.

22 6.5 Proof of Theorem 3 Proof. 1. For any real r 1, formula F and set of clauses G F,define the r-deficiency of G, δ r (G) = r Clauses(G) Vars(G). Also define δ r(f )=max{δ r (G) : G F } (5) We claim that for any minimally unsatisfiable F, δ2k 3 (F ) 0. Indeed, assume this was not true. Then there exists such F such that: δ 2k 3 (G) 1 for all G F. (6) Proposition 3 Let F be a formula for which condition 6 holds. Then there exists an ordering C 1,...,C F of constraints in F such that each constraint C i contains at least k 2 variables that appear in no C j, j<i. Proof. Denote by v i the number of variables that appear in exactly i constraints of F. We have i 1 i v i = k Constraints(F ), therefore 2 Var(F ) v 1 k Constraints(F ). This can be rewritten as v 1 2 Var(F ) k Constraints(F ) > Constraints(F ) (2k 3 k) = (k 3) Constraints(F ) (we have used the upper bound on c (F ). Therefore there exists at least one constraint in F with at least k 2 variables that are free in F. We set C F = C and apply this argument recursively to F \ C. Call the k 2 new variables of C i free in C i. Call the other two variables bound in C i. Let us show now that F cannot be minimally unsatisfiable. Construct a satisfying assignment for F incrementally: Consider constraint C j. At most two of the variables in C j are bound for C j. Since C has no implicates of size at most two, one can set the remaining variables in a way that satisfies C j. This yields a satisfying assignment for F, a contradiction with our assumption that F was minimally unsatisfiable. Therefore δ2k 3 (F ) 0, a statement equivalent to our conclusion. 2. To prove the resolution complexity lower bound we use the size-width connection for resolution complexity obtained in [7]: we prove that there exists η>0 such that w.h.p. random instances of SAT(C) having constraint density c have resolution width at least η n. We use the same strategy as in [7] (a) (prove that) w.h.p. minimally unsatisfiable subformulas are large, and (b) any clause implied by a satisfiable formula of intermediate size contains w.h.p. many literals. Indeed, define for a unsatisfiable formula Φ and (possibly empty) clause C µ(c) =min{ Ξ :Ξ Φ, Ξ = C}. Claim 2 There exists η 1 > 0 such that for any c>0, w.h.p. µ( ) η 1 n (where Φ is a random instance of SAT(C) having constraint density c).

23 Proof. In the proof of Theorem 2 we have shown that there exists η 0 > 0 such that w.h.p. any unsatisfiable subformula of a given formula has at least η 0 n constraints. Therefore any formula made of clauses in the CNF-representation of constraints in Φ, and which has less than η 0 n clauses is satisfiable, and the claim follows, by taking η 1 = η 0. The only (slightly) nontrivial step of the proof, which critically uses the fact that constraints in C do not have implicates of length at most two, is to prove that clause implicates of subformulas of medium size have many variables. Formally (and proved in the Appendix) Claim 3 There exists d>0 and η 2 > 0 such that w.h.p., for every clause C such that d/2 n< µ(c) <= dn, C η 2 n. Proof. Take 0 <ɛ. It is easy to see that if c (F ) < 2 then w.h.p. for every subformula G of 2k 3+ɛ F, at least ɛ Constraints(G) have at least k 2 private variables: Indeed, since 3 c (G) < 2, 2k 3+ɛ by a reasoning similar to the one we made previously v 1 (G) (k 3+ɛ) Constraints(G). Since constraints in G have arity k, at least ɛ/3 Constraints(G) have at least k 2 private variables. Choose y = 2 in Lemma 6.1 for ɛ>0asmall enough constant. Since the problem has a sharp 2k 3+ɛ threshold in the region where the number of clauses is linear, d =inf{x(y, c) :c>= c SAT(C) } > 0. W.h.p. all subformulas of Φ having size less than d/k n have a formula hypergraph that is (x, y)- sparse, therefore fall under the scope of the previous argument. Let Ξ be a subformula of Φ, having minimal size, such that Ξ = C. We claim: Claim 4 For every clause P of Ξ with k 2 private variables, (i.e., one that does not appear in any other clause), at least one of these private variables appears in C. Indeed, suppose there exists a clause D of Ξ such that none of its private variables appears in C. Because of the minimality of Ξ there exists an assignment F that satisfies Ξ \{D} but does not satisfy D or C. Since D has no implicates of size two, there exists an assignment G, that differs from F only on the private variables of D, that satisfies Ξ. But since C does not contain any of the private variables of D, F coincides with G on variables in C. The conclusion is that G does not satisfy C, which contradicts the fact that Ξ = C. The proof of Claim 3 (and of item 2. of Theorem 3) follows: since for any clause K of one of the original constraints µ(k) =1, since µ( ) >η 1 n and since w.l.o.g. 0 <d<η 1 (otherwise replace d with the smaller value) there exists a clause C such that µ(c) [d/2k n, d/k n]. (7) Indeed, let C be a clause in the resolution refutation of Φ minimal with the property that µ(c ) > dn. Then at least one clause C, of the two involved in deriving C satisfies equation 7. By the previous claim it C contains at least one private variable from each clause of Ξ. Therefore C η 2 n, with η 2 = d/2k ɛ.