Statistical physics analysis of the computational complexity of solving random satisfiability problems using backtrack algorithms

Transcription

1 Eur. Phys. J. B, ) THE EUROPEAN PHYSICAL JOURNAL B EDP Sienes Soietà Italiana di Fisia Springer-Verlag Statistial physis analysis of the omputational omplexity of solving random satisfiability problems using baktrak algorithms S. Coo, and R. Monasson,3,a CNRS-Laboratoire de Physique Théorique de l ENS, 4 rue Lhomond, 755 Paris, Frane Department of Physis, The University of Illinois at Chiago, 845 W. Taylor St., Chiago IL 667, USA 3 The James Frank Institute, The University of Chiago, 564 S. Ellis Av., Chiago IL 6637, USA Reeived Marh Abstrat. The omputational omplexity of solving random 3-Satisfiability 3-SAT) problems is investigated using statistial physis onepts and tehniques related to phase transitions, growth proesses and real spae) renormalization flows. 3-SAT is a representative example of hard omputational tasks; it onsists in knowing whether a set of αn randomly drawn logial onstraints involving N Boolean variables an be satisfied altogether or not. Widely used solving proedures, as the Davis-Putnam-Loveland-Logemann DPLL) algorithm, perform a systemati searh for a solution, through a sequene of trials and errors represented by a searh tree. The size of the searh tree aounts for the omputational omplexity, i.e. the amount of omputational efforts, required to ahieve resolution. In the present study, we identify, using theory and numerial experiments, easy size of the searh tree saling polynomially with N) and hard exponential saling) regimes as a funtion of the ratio α of onstraints per variable. The typial omplexity is expliitly alulated in the different regimes, in very good agreement with numerial simulations. Our theoretial approah is based on the analysis of the growth of the branhes in the searh tree under the operation of DPLL. On eah branh, the initial 3-SAT problem is dynamially turned into a more generi +p-sat problem, where p and p are the frations of onstraints involving three and two variables respetively. The growth of eah branh is monitored by the dynamial evolution of α and p and is represented by a trajetory in the stati phase diagram of the random +p-sat problem. Depending on whether or not the trajetories ross the boundary between satisfiable and unsatisfiable phases, single branhes or full trees are generated by DPLL, resulting in easy or hard resolutions. Our piture for the origin of omplexity an be applied to other omputational problems solved by branh and bound algorithms. PACS. 5..-a Computational methods in statistial physis and nonlinear dynamis 5.7.-a Thermodynamis 89..Ff Computer siene and tehnology Introdution Out-of-equilibrium dynamial properties of physial systems form the subjet of intense studies in modern statistial physis []. Over the past deades, muh progress has been made in fields as various as glassy dynamis, growth proesses, persistene phenomena, vortex depinning..., where dynamial aspets play a entral role. Among all the questions related to these issues, the existene and haraterization of stationary states reahed in some asymptoti limit of large times is of entral importane. In turn, the notion of asymptoti regime raises the question of relaxation, or transient, behavior: what time do we need to wait for in order to let the system relax? How does this time grow with the size of the system? Suh interrogations are not limited to out-of-equilibrium dynamis but a monasson@physique.ens.fr also arise in the study of ritial slowing down phenomena aompanying seond order phase transitions. Computer siene is another sientifi disipline where dynamial issues are of entral importane. There, the main question is to know the time or, more preisely, the amount of omputational resoures required to solve some given omputational problem, and how this time inreases with the size of the problem to be solved []. Consider for instane the sorting problem [3]. One is given a list L of N integer numbers to be sorted in inreasing order. What is the omputational omplexity of this task, that is, the minimal number of operations essentially omparisons) neessary to sort any list L of length N? Knuth answered this question in the early seventies: omplexity sales at least as N log N and there exists a sorting algorithm, alled Mergesort, ahieving this lower bound [3]. To alulate omputational omplexity, one has to study how the onfiguration of data representing the

2 56 The European Physial Journal B omputational problem dynamially evolves under the presriptions enoded in the algorithm. Let us onsider the sorting problem again and think of the initial list L as a random) permutation of I = {,,...,N}. Starting from L) = L, at eah time step operation) T,the sorting algorithm transforms the list LT ) into another list LT +), reahing finally the identity permutation, i.e. the ordered list I. Obviously, the dynamial rules imposed by a solving algorithm are of somewhat unusual nature from a physiist s point of view. They might be highly non-loal and non-markovian. Yet, the operation of algorithms gives rise to well posed dynamial problems, to whih methods and tehniques of statistial physis may be applied as we argue in this paper. Unfortunately, not all problems enountered in omputer siene are as simple as sorting. Many omputational problems originating from industrial appliations, e.g. sheduling, planning and more generally optimization tasks, demand omputing efforts growing enormously with their size N. For suh problems, alled NP omplete, all known solving algorithms have exeution times inreasing a priori exponentially with N and it is a fundamental onjeture of omputer siene that no polynomial solving proedure exists. To be more onrete, let us fous on the 3-satisfiability 3-SAT) problem, a paradigm of the lass of NP omplete omputational problems []. A pedagogial introdution to the 3-SAT problem and some of the urrent open issues in theoretial omputer siene may be found in [4]. 3-SAT is defined as follows. Consider a set of N Boolean variables and a set of M = αn onstraints alled lauses), eah of whih being the logial OR of three variables or of their negation. Then, try to figure out whether there exists or not an assignment of variables satisfying all lauses. If suh a solution exists, the set of lauses alled instane of the 3-SAT problem) is said satisfiable sat); otherwise the instane is unsatisfiable unsat). To solve a 3-SAT instane, i.e. to know whether it is sat or unsat, one usually resorts to searh algorithms, as the ubiquitous Davis Putnam Loveland Logemann DPLL) proedure [4 6]. DPLL operates by trials and errors, the sequene of whih an be graphially represented as a searh tree. Computational omplexity is the amount of operations performed by the solving algorithm and is onventionally measured by the size of the searh tree. Complexity may, in pratie, vary enormously with the instane of the 3-SAT problem under onsideration. To understand why instanes are easy or hard to solve, omputer sientists have foused on model lasses of 3-SAT instanes. Probabilisti models, that define distributions of random instanes ontrolled by few parameters, are partiularly useful. An example, that has attrated a lot of attention over the past years, is random 3-SAT: all lauses are drawn randomly and eah variable negated or left unhanged with equal probabilities. Experiments [6 ] and theory [,] indiate that instanes are almost surely always sat respetively unsat) if α is smaller resp. larger) than a ritial threshold α C 4.3 assoonasm,n go to median omputational omplexity α C number of lauses per variable α var. 75 var. 5 var. Fig.. Solving omplexity of random 3-SAT as a funtion of the ratio α of lauses per variables, and for three inreasing problem sizes N. Data are averaged over, randomly drawn samples. Complexity is maximal at the threshold α C 4.3. infinity at fixed ratio α. This phase transition [,3] is aompanied by a drasti peak of omputational hardness at threshold [6 8], see Figure. Random 3-SAT generates simplified and idealized versions of real-world instanes. Yet, it reprodues essential features sat vs. unsat, easy vs. hard) and an shed light on the onset of omplexity, in the same way as models of ondensed matter physis help to understand global properties of real materials. Phases in random 3-SAT, or in physial systems, haraterize the overall stati behavior of a sample in the large size limit a large instane with ratio e.g. α = 3 will be almost surely sat existene proof) but do not onvey diret information of dynamial aspets how long it will take to atually find a solution onstrutive proof). This situation is reminisent of the learning problem in neural networks equilibrium statistial mehanis allows to ompute the maximal storage apaity, irrespetive of the memorization proedure and of the learning time) [4], or liquids at low enough temperatures that should rystallize from a thermodynamial point of view but undergo some kinetial glassy arrest) [5]. The analogy between relaxation in physial systems and omputational omplexity in ombinatorial problems is even learer when the latter are solved using loal searh algorithms, e.g. simulated annealing or other solving strategies making loal moves based on some energy funtion. Consider a random 3-SAT instane below threshold. We define the energy funtion depending on the onfiguration of Boolean variables) as the number of unsatisfied lauses [6]. The goal of the algorithm is to find a solution, i.e. to minimize the energy. The onfiguration of Boolean variables evolve from some initial value to some solution under the ation of the algorithm. During this evolution, the energy of the system relaxes from an initial large) value to zero. Computational omplexity is, in this ase, equal to the relaxation time of the dynamis before reahing zero temperature) equilibrium.

3 S. Coo and R. Monasson: Analysis of the dynamis of baktrak algorithms 57 A B solving random SAT models, and explain the relationship with stati studies of the phase transition [3]. Last of all, we show in Setion 8 how our study suggests some possible ways to improve existing algorithms. The reader is kindly reommended to go through [7] to get an overview of our approah and results, before pursuing and reading the present artile. S Fig.. Examples of searh trees. A) simple branh: the algorithm finds easily a solution without ever or with a negligible amount of) baktraking. B) dense tree: in the absene of solution, the algorithm builds a bushy tree, with many branhes of various lengths, before stopping. C) mixed ase, branh + tree: if many ontraditions arise before reahing a solution, the resulting searh tree an be deomposed in a single branh followed by a dense tree. The juntion G is the highest baktraking node reahed bak by DPLL. This paper is an extended version of a previous work [7], where we showed how the dynamis indued by the DPLL searh algorithm ould be analyzed using off-equilibrium statistial mehanis and ombined to the stati phase diagram of random K-SAT with K =, 3) to alulate omputational omplexity. We start by exposing in a detailed way the definition of the random K-SAT problem and the DPLL proedure in Setion. We then expose the experimental measures of omplexity in Setion 3. Our analytial approah is based on the fat that, under DPLL ation, the initial instane is modified and follows some trajetory in the phase diagram. The struture of the searh tree generated by DPLL proedure is losely related to the nature of the region visited by the instane trajetory. Searh trees redue to essentially one branh sat instanes at low ratio α, Setion4 orare dense, inluding an exponential number of branhes unsat instanes, Setion 5. Mixed strutures SAT instanes with ratios slightly below threshold, Setion 6 are made of a branh followed by a dense tree and reflet trajetories rossing the phase boundary between sat and unsat regimes Fig. ). While branh trajetories ould be obtained straightforwardly from previous works by Chao and Frano [8], we develop in Setion 5 a formalism to study the onstrution of dense trees by DPLL. We show that the latter an be reformulated in terms of a bidimensional) growth proess desribed by a non-linear partial differential equation. The resolution of this growth equation allows an analytial predition of the omplexity that ompares very well to extensive numerial experiments. We present in Setion 7 the full omplexity diagram of G C S Davis-Putnam-Loveland-Logemann algorithm and random 3-SAT In this setion, the reader will be briefly realled the main features of the random 3-Satisfiability model. We then present the Davis-Putnam-Loveland-Logemann DPLL) solving proedure, a paradigm of branh and bound algorithm, and the notion of searh tree. Finally, we introdue the idea of dynamial trajetory, followed by an instane under the ation of DPLL.. A reminder on random satisfiability Random K-SAT is defined as follows [8]. Let us onsider N Boolean variables x i that an be either true T) or false F) i =,...,N). We hoose randomly K among the N possible indies i and then, for eah of them, a literal, that is, the orresponding x i or its negation x i with equal probabilities one half. A lause C is the logial OR of the K previously hosen literals, that is C will be true or satisfied) if and only if at least one literal is true. Next, we repeat this proess to obtain M independently hosen lauses {C l } l=,...,m and ask for all of them to be true at the same time, i.e. we take the logial AND of the M lauses. The resulting logial formula is alled an instane of the K-SAT problem. A logial assignment of the x i s satisfying all lauses, if any, is alled a solution of the instane. For large instanes M,N ), K-SAT exhibits a striking threshold phenomenon as a funtion of the ratio α = M/N of the number of lauses per variable. Numerial simulations indiate that the probability of finding a solution falls abruptly from one down to zero when α rosses a ritial value α C K) [6 8]. Above α C K), all lauses annot be satisfied any longer. This senario is rigorously established in the K =ase,whereα C = [9]. For K 3, muh less is known; K 3)-SAT belongs to the lass of hard, NP-omplete omputational problems []. Studies have mainly onentrated on the K =3 ase, whose instanes are simpler to generate than for larger values of K. Some lower [] and upper [] bounds on α C 3) have been derived, and numerial simulations have reently allowed to find preise estimates of α C, e.g. α C 3) 4.3 [8,]. The phase transition taking plae in random 3-SAT has attrated a large deal of interest over the past years due to its lose relationship with the emergene of omputational omplexity. Roughly speaking, instanes are muh harder to solve at threshold than far from ritiality [6 9].

4 58 The European Physial Journal B We now expose the solving proedure used to takle the 3-SAT problem.. The Davis-Putnam-Loveland-Logemann solving proedure step lauses searh tree w x y w x z w x y w x y x y z.. Main operations of the solving proedure and searh trees split : w = T 3-SAT is among the most diffiult problems to solve as its size N beomes large. In pratie, one resorts to methods that need, a priori, exponentially large omputational resoures. One of these algorithms, the Davis-Putnam- Loveland-Logemann DPLL) solving proedure [4,5], is illustrated in Figure 3. DPLL operates by trials and errors, the sequene of whih an be graphially represented as a searh tree made of nodes onneted through edges as follows:. A node orresponds to the hoie of a variable. Depending on the value of the latter, DPLL takes one of the two possible edges.. Along an edge, all logial impliations of the last hoie made are extrated. 3. DPLL goes bak to step unless a solution is found or a ontradition arises; in the latter ase, DPLL baktraks to the losest inomplete node with a single desendent edge), inverts the attahed variable and goes to step ; if all nodes arry two desendent edges, unsatisfiability is proven. Examples of searh trees for satisfiable sat) or unsatisfiable unsat) instanes are shown Figure. Computational omplexity is the amount of operations performed by DPLL, and is measured by the size of the searh tree, i.e. the number of nodes... Heuristis of hoie In the above proedure, step requires to hoose one literal among the variables not assigned yet. The hoie of the variable and of its value obeys some empirial rule alled splitting heuristi. The key idea is to hoose variables that will lead to the maximum number of logial impliations []. Here are some simple heuristis: Truth table rule: fix unknown variables in lexiographi order, from x up to x N and assign them to e.g. true. This is an ineffiient rule that does not follow the key priniple exposed above. Generalized Unit-Clause GUC) rule: hoose randomly one literal among the shortest lauses [8]. This is an extension of unit-propagation that fixes literal in unitary lauses. GUC is based on the fat that a lause of length K needs at most K splittings to produe a logial impliation. So variables are hosen preferentially among short lauses x z x y x y x y z split : : y x = T y y propagation : ontradition z z y = F, y = T baktraking to stage 3 : x = F propagation : z = T, y = T solution : w = T, x = F, y = T, z = T Fig. 3. Example of 3-SAT instane and Davis-Putnam- Loveland-Logemann resolution. Step ) The instane onsists of M = 5 lauses involving N = 4 variables that an be assigned to true T) or false F). w means NOT w) and denotes the logial OR. The searh tree is empty. ) DPLL randomly selets a variable among the shortest lauses and assigns it to satisfy the lause it belongs to, e.g. w = T splitting with the Generalized Unit Clause GUC heuristi). A node and an edge symbolizing respetively the variable hosen w) and its value T) are added to the tree. ) The logial impliations of the last hoie are extrated: lauses ontaining w are satisfied and eliminated, lauses inluding w are simplified, and the remaining lauses are left unhanged. If no unitary lause i.e. with a single variable) is present, a new hoie of variable has to be made. 3) Splitting takes over. Another node and another edge are added to the tree. 4) Same as step but now unitary lauses are present. The variables they ontain have to be fixed aordingly propagation). 5) The propagation of the unitary lauses results in a ontradition. The urrent branh dies out and gets marked with C. 6) DPLL baktraks to the last split variable x), inverts it F) and reates a new edge. 7) Same as step 4. 8) Propagation of the unitary lauses eliminates all the lauses. A solution S is found. This example show how DPLL find a solution for a satisfiable instane. For an unsatisfiable instane, unsatisfiability is proven when baktraking see step 6) is not possible anymore sine all split variables have already been inverted. In this ase, all the nodes in the final searh tree have two desendent edges and all branhes terminate by a ontradition C. C C C S

5 S. Coo and R. Monasson: Analysis of the dynamis of baktrak algorithms 59 Maximal ourrene in minimum size lauses MOMS) rule: pik up the literal appearing most often in shortest lauses. This rule is a refinement of GUC. Global performanes of DPLL depend quantitatively on the splitting rule. From a qualitative point of view, however, the easy-hard-easy piture emerging from experiments is very robust [7,8,]. Hardest instanes seem to be loated at threshold. Solving them demand an exponentially large omputational effort saling as NωC.Thevaluesofω C found in literature roughly range from.5 to., depending on the splitting rule used by DPLL [6,]. In this paper, we shall fous on the GUC heuristi whih is simple enough to allow analytial studies and, yet, is already quite effiient. 3 Numerial experiments 3. Desription of the numerial implementation of the DPLL algorithm We have implemented DPLL with the GUC rule, see Figure 3 and Setion.., to have a fast unit propagation and an inexpensive baktraking [6]. The program is divided in three parts. The first routine draws the lauses and represents the data in a onvenient struture. The seond, main routine updates and saves the state of the searh, i.e. the indies and values of assigned variables, to allow an easy baktraking. Then, it heks if a solution is found; if not, a new variable is assigned. The third routine extrats the impliation of the hoie propagation). If unit lauses have been generated, the orresponding literals are fixed, or a ontradition is deteted. 3.. Data representation.3 +p-sat and instane trajetory We shall present in Setion 3 the experimental results on solving 3-SAT instanes using DPLL proedure in a detailed way. The main sope of this paper is to ompute in an analytial way the omputational omplexity ω in the easy and hard regimes. To do so, we have made use of the preious notion of dynamial trajetory, that we now expose. As shown in Figure 3, the ation of DPLL on an instane of 3-SAT auses the redution of 3-lauses to -lauses. Thus, a mixed +p-sat distribution [3], where p is the fration of 3-lauses, may be used to model what remains of the input instane at a node of the searh tree. A+p-SAT formula of parameters p, α is the logial AND of two unorrelated random -SAT and 3-SAT instanes inluding α p) N and α pn lauses respetively. Using experiments [3] and statistial mehanis alulations [], the threshold line α C p) may be obtained with the results shown in Figure 4 full line). Replia alulations suggest that the sat/unsat transition taking plae at α C p) is ontinuous if p<p S and disontinuous if p>p S, where p S.4 [3]. The triritial point T S is shown in Figure 4. Below p S, the threshold α C p) oinides with the upper bound / p), obtained when requiring that the -SAT subformula only be satisfied. Rigorous studies have shown that p S /5, leaving open the possibility it is atually equal to /5 [4], or to some slightly larger value [5]. The phase diagram of +p-sat is the natural spae in whih DPLL dynami takes plae [7]. An input 3-SAT instane with ratio α shows up on the right vertial boundary of Figure 4 as a point of oordinates p =,α ). Under the ation of DPLL, the representative point moves aside from the 3-SAT axis and follows a trajetory. The loation of this trajetory in the phase diagram allows a preise understanding of the searh tree struture and of omplexity. Three arrays are used to enode the data: the two first arrays are labelled by the lause number m =,...,M and the number b =,...,K of the omponents in the lause with K =, 3). The entries of these arrays, initially drawn at random, are the indies lausnum m, b) and the values lausval m, b), true or false, of the variables. The indies of the third array are integers i =,...,N and j =,...,o i where o i is the number of ourrenes of x i in the lauses from zero to M). The entries of the matrix, a i, j), are the numbers of the orresponding lauses between and M). 3.. Updating of the searh state If the third routine has found a ontradition, the seond routine goes bak along the branh, inverts the last assigned variable and alls again the third routine. If not, the desent along the branh is pursued. A Boolean-valued vetor points to the assigned variables, while the values are stored in another unidimensional array. For eah lause, we hek if the variables are already assigned and, if so, whether they are in agreement with the lauses. When splitting ours, a new variable is fixed to satisfy a -lause, i.e. a lause with one false literal and two unknown variables, and the third subroutine is alled. If there are only 3-lauses, a new variable is fixed to satisfy any 3-lause and the third subroutine is alled. The variable hosen and its value are stored in a vetor with index the length of the branh, i.e. the number of nodes it ontains, to allow future baktraking. If there are neither - nor 3-lauses left, a solution is found Consequenes of a hoie and unit propagation All lauses ontaining the last fixed variable are analyzed by taking into aount all possibilities:. the lause is satisfied;. the lause is redued to a - or -lause; 3. the

6 5 The European Physial Journal B 8 ratio of lauses per variable α 6 4 T,T S G 7 α C =4.3 S SAT) fration of 3 lauses p 3 SAT) Fig. 4. Phase diagram of the +p-sat model and trajetories generated by DPLL. The threshold line α Cp) bold full line) separates sat lower part of the plane) from unsat upper part) phases. Extremities lie on the vertial -SAT left) and 3-SAT right) axis at oordinates p =,α C =)andp =,α C 4.3) respetively. The threshold line oinides with the α =/ p) urve dotted line) when p p S.4, that is, up to the triritial point T S. Departure points for DPLL trajetories are loated on the 3-SAT vertial axis and the orresponding values of α are expliitly given. Dashed urves represent tree trajetories in the unsat region thik lines, blak arrows) and branh trajetories in the sat phase thin lines, empty arrows). Arrows indiate the diretion of motion along trajetories parametrized by the fration t of variables set by DPLL. For small ratios, e.g. α =<α L 3.3, trajetories remain onfined in the sat phase. At α L, the single branh trajetory hits tangentially the threshold line in T of oordinates /5, 5/3), very lose to T S. In the intermediate range α L <α <α C, the branh trajetory intersets the threshold line at some point G that depends on α. A dense tree then grows in the unsat phase, as happens when 3-SAT departure ratios are above threshold α>α C 4.3. The tree trajetory halts on the dot-dashed urve α.59/ p) where the tree growth proess stops. At this point, DPLL has reahed bak the highest baktraking node in the searh tree, that is, the first node when α >α C, or node G for α L <α <α C. In the latter ase, a solution an be reahed from a new desending branh rightmost path in Figure C) while, in the former ase, unsatisfiability is proven. Small squares show the trajetory orresponding to this suessful branh for α = 3.5, as obtained from simulations for N = 3. The trajetory oinides perfetly with the theoretial branh trajetory up to point G not shown), and then reahes the α = axis when a solution is found..6 lause is violated ontradition). In the seond ase, the -lause is stored to be analyzed by unit-propagation one all lauses ontaining the variable have been reviewed. 3. Charateristi running times We have implemented the DPLL searh algorithm in Fortran 77; the algorithm runs on a Pentium II PC with a 433M Hz frequeny lok. The number of nodes added per minute ranges from 3, typially obtained for α =3.5) to, α = ) sine unit propagation is more and more frequent as α inreases. The order of magnitude of the omputational time needed to solve an instane are listed in Table for ratios orresponding to hard instanes. These times limit the maximal size N of the instanes we have experimentally studied and the number of samples over whih we have averaged. Some rare instanes may be muh harder than the typial times indiated in Table. For instane, for α =3. andn = 5, instanes are usually solved in about 4 minutes but some samples required more than hours of omputation. This phenomenon will be disussed in Setion Table. Typial omputational time required to solve some hard random 3-SAT instanes for different ratios α and sizes N. Slightly below threshold, some rare instanes may require muh longer omputational time Set. 3.4.). Ratio α lause/var. Nb. of var. Resolution time 9 6 seonds 7 hour 7 4 minutes days minutes 3.3 Overview of experiments 3.3. Number of nodes of the searh tree We have first investigated omplexity by a diret ount of the number of splittings, that is the number of nodes blak points) in Figure, for sat Figs. A, C) and unsat Fig. B) trees.

7 S. Coo and R. Monasson: Analysis of the dynamis of baktrak algorithms Histogram of branh length We have also experimentally investigated the struture of searh trees for unsat instanes Fig. B). A branh is defined as a path joining the root first node at the top of the searh tree) to a leaf marked with a ontradition C or a solution S for sat instane) in Figure. The length of a branh is the number of nodes it ontains. For an unsat instane, the omplete searh tree depends on the variables hosen at eah split, but not on the values they are assigned. Indeed, to prove that there is no solution, all nodes in the searh tree have to arry two outgoing branhes, orresponding to the two hoies of the attahed variables. What hoie is made first does not matter. This simple remark will be of ruial importane in the theoretial analysis of Setion 5... We have derived the histogram of the branh lengths by ounting the number Bl) of branhes having length ln one the tree is entirely built up. The histogram is very useful to dedue the omplexity in the unsat phase, sine in a omplete tree the total number of branhes B is related to the number of nodes Q through the identity, B = Bl) =Q + ) l= N,..., that an be inferred from Figure B Highest baktraking point Another key property we have foused upon is the highest baktraking point in the searh tree. In the unsat phase, DPLL baktraks all the nodes of the tree sine no solution an be present. The highest baktraking point in the tree simply oinides with the top root) node. In the sat phase, the situation is more involved. A solution generally requires some baktraking and the highest baktraking node may be defined as the losest node to the origin through whih two branhes pass, node G in Figure B. We experimentally keep trae of the highest baktraking point by measuring the numbers C G), C 3 G) of - and 3-lauses, the number of not-yet-assigned variables NG), and omputing the oordinates p G = C 3 G)/C G) + C 3 G)),α G = C G) + C 3 G))/N G) of G in the phase diagram of Figure Experimental results 3.4. Flutuations of omplexity The size of the searh tree built by DPLL is a random variable, due to the quenhed) randomness of the 3-SAT instane and the hoies made by the splitting rule thermal noise ). We show on Figure 5 the distribution of the logarithms in base, and divided by N) ofthenumberof nodes for different values of N. The distributions are more and more peaked around their mean values ω N α) asthe size N inreases. However, flutuations are dramatially different at low and large ratios. For α =, and more generally in the unsat phase, the distributions are roughly symmetri Fig. 5A). Tails are small and omplexity does not flutuate too muh from sample to sample [6,7]. In the viinity of α L, e.g. α =3., muh bigger flutuations are present. There are large tails on the right flanks of the distributions in Figure 5B, due to the presene of rare and very hard samples [7]. As a onsequene, while the logarithm of the omplexity seems to be a self-averaging quantity in the thermodynami limit, omplexity itself is not e.g. at α =3.). We will ome bak to this point in Setion The easy-hard-easy pattern We have averaged the logarithm of the number of nodes over, randomly drawn instanes to obtain ω N α). The typial size Q of the searh tree is simply Q = NωN. Results are shown in Figure. An easy-hard-easy pattern of omplexity appears learly as the ratio α varies. At small ratios, omplexity inreases as γα)n, that is, only linearly with the size of the instane. DPLL easily finds a solution and the searh tree essentially redues to a single branh shown in Figure A. For the GUC heuristi, the linear regime extends up to α L 3.3 [8,]. Above threshold, omplexity grows exponentially with N [8]. The logarithm ωα), limit of ω N α)asn, is maximal at ritiality and dereases at large ratios as /α [9]. The easy region on the right hand side of Figure is still exponential but with small oeffiients ω. Of partiular interest is the intermediate region α L <α< α C. We shall show that omplexity is exponential in this range of ratios, and that the searh tree is a mixture of the searh trees taking plae in the other ranges α<α L and α>α C. Let us mention that, while this paper is devoted to typial-ase happening with probability one) omplexity, rigorous results have been obtained that apply to any instane. So far, using a refined version of DPLL, any instane is guaranteed to be solved in less than.54 N steps, i.e. ω<.588. The reader is referred to referene [3] for this worst-ase analysis Lower sat phase α <α L ) The omplexity data of Figure obtained for different sizes N =5, 75, are plotted again in Figure 6 after division by N. Data ollapse on a single urve, proving that omplexity is linear in the regime α<α L.Inthe viinity of the ross over ratio α = α L finite-size effets beame important in this region. We have thus performed extra simulations for larger sizes N = 5, in the range.5 α 3. that onfirm the linear saling of the omplexity.

8 5 The European Physial Journal B 8 8 Frequeny 6 Frequeny Logarithm of omplexity divided by size N) Logarithm of omplexity divided by size N) A B Fig. 5. Distribution of the logarithms in base, and divided by N) of the omplexity for four different sizes N =3dotdashed), 5 dashed), dotted) and full line). The histograms are normalized to unity and represent 5, instanes. The distribution gets more and more onentrated as the size grows. A) Ratio α =. Curves are roughly symmetrial around their mean with small tails on their flanks; large flutuations from sample to sample are absent. B) Ratio α =3.. Curves have large tails on the right, refleting the presene of rare, very hard samples. fixed N, branhes are shorter and shorter and less and less numerous, making omplexity derease Fig. ). As N gets large at fixed α, the histogram bl) beomes a smooth funtion of l and we an replae the disrete sum in ) with a ontinuous integral on the length, Q += l= N,..., Nbl) N dl Nbl). ) Fig. 6. Complexity of solving in the sat region for α<α L 3.3, divided by the size N of the instanes. Numerial data are for sizes N =5 ross), 75 square), diamond), 5 triangle) and irle). For the two biggest sizes, simulations have been arried out for ratios larger than.5 only. Data for different N ollapse onto the same urve, proving that omplexity sales linearly with N. The bold ontinuous urve is the analytial predition γα) from Setion Note the perfet agreement with numeris exept at large ratios where finite size effets are important, due to the ross-over to the exponential regime above α L Unsat phase α >α C ) Results for the shape of the searh trees are shown in Figure 7. We represent the logarithm bl), in base and divided by N,ofthenumberBl) of branhes as a funtion of the branh length ln, averaged over many samples and for different sizes N and ratios α. When α inreases at The integral is exponentially dominated by the maximal value b max of bl). ω, the limit of the logarithm of the omplexity divided by N, is therefore equal to b max. Niely indeed, the height b max of the histogram does not depend on N within the statistial errors) and gives a straightforward and preise estimate of ω, not affeted by finite-size effets. The values of b max as a funtion of α are listed in the third olumn of Table. The above disussion is also very useful to interpret the data on the size Q of the searh trees. From the quadrati orretion around the saddle-point, bl) b max βl l max ) /, the expeted finite size orretion to ω N read ω N ω + N log N + [ ] ) π N log + o β ln N 3) We have heked the validity of this equation by fitting ω N log N/N) as a polynomial funtion of /N.The onstant at the origin gives values of ω in very good agreement with b max seond olumn in Tab. ) while the linear term gives aess to the urvature β. We ompare in Table 3 this urvature with the diret measurements of β from the histogram. The agreement is fair, showing that equation 3) is an aurate way of extrapolating data on Q to the infinite size limit.

9 S. Coo and R. Monasson: Analysis of the dynamis of baktrak algorithms 53.9 α= α= α= Number of branhes ) /N α= α=7 α= α= Log. α= Length of branhes /N Fig. 7. Logarithm of the number of branhes in unsat searh trees as a funtion of the branh length. Main figure: the size of the searh tree is a dereasing funtion of α at fixed N. Histograms are presented here for ratios equal to α =4.3 solid line, N = ), α = 7 dotted line, N = 4), α = dashed line, N = 6) and α = dotted-dashed line, N = 9) and have been averaged over hundreds of samples. Inset: the heights of the tops of the histograms show a very weak dependene upon N. Numerial extrapolations of ω N to N and statistial errors are reported Table. Sat instanes whih may be present for small sizes at α =4.3) have not be onsidered in the averaging proedure. For eah inset, we give below the sizes N followed by the number of instanes in parenthesis used for averaging. Ratio α =4.3: solid line: 5), dotted line: 5 5), dashed line: 4); α = 7: solid line: ), dotted line: 3 ), dashed line: 4 ). α = : solid line: 4 5), dotted line: 5 4), dashed line: 6 ); α = : solid line: 7 6), dotted line: 8 ), dashed line: 9 ). Table. Logarithm of the omplexity ω from experiments and theory in the unsat phase. Values of ω from measures of searh tree sizes nodes), histograms of branh lengths histogram) and theory within linear lin.) and quadrati quad.) approximations. Note the exellent agreement between theory and experiments for large α. Ratio α Experiments Theory of lause/var. nodes histogram lin. quad..53 ±..5 ± ±..6 ± ±.5.37 ± ± ± ±..895 ± Table 3. Details on the fits of searh tree sizes from equation 3). For different ratios α, the slope γ of the fit of ω N log N/N vs. /N isshownaswellastheorresponding urvature β dedued from 3) olumn nodes ). The urvatures measured diretly at the top of the branh lengths histograms are listed in the histogram olumn. Ratio α Slope Curvature β of lause/var. γ nodes histogram Upper sat phase α L <α<α C ) To investigate the sat region slightly below threshold α L <α<α C, we have arried out simulations with a starting ratio α =3.5. Results are shown in Figure 8A. As instanes are sat with a high probability, no simple identity relates the number of nodes Q to the number of branhes B, see searh tree in Figure C and we measure the omplexity through Q only. Complexity learly sales exponentially with the size of the instane and exhibits large flutuations from sample to sample. The annealed omplexity logarithm of the average omplexity), ω3 ann, is larger than the typial solving hardness average of the logarithm of the omplexity), ω typ 3, see Table 4. To reah a better understanding of the struture of the searh tree, we have foused on the highest baktraking point G defined in Figure C and Setion The oordinates p G,α G of point G, averaged over instanes are shown for inreasing sizes N in Figure 9. The oordinates of G exhibit strong sample flutuations whih make the large N extrapolation, p G =.78 ±., α G =3. ±. rather impreise.

10 54 The European Physial Journal B Complexity.5 Complexity log N / N log N / N A B Fig. 8. Solving omplexity in the upper sat phase for sizes N ranging from N = to N = 4. The size of the symbols aounts for the largest statistial error bar. A) 3-SAT problem with ratio α = 3.5: typial average of the logarithm, full irles) and annealed logarithm of the average, full triangles) size of the searh tree. Dotted lines are the quadrati and linear fits of the typial and annealed omplexities, giving ω typ 3 =.35 ±.3 and ω3 ann =.43 ±. in the infinite size limit. B) Related +p-sat problem with parameters p G =.78,α G =3.. The typial omplexity is measured from the size of the searh tree irles) and the top of the branh length distribution diamonds), with the same large N extrapolation: ω typ +p =.4 ±.. This value is slightly smaller than the annealed omplexity ω+p ann =.44 ±. triangles). All fits are linear. Table 4. Logarithm of the omplexity ω from experiments and theory in the upper sat phase. Experiments determine ω from measures of the annealed omplexity ann.), of the typial searh tree sizes nod.) and of histograms of branh lengths his.). Data are presented for 3-SAT instanes with ratio α = 3.5, and +p-sat instanes with parameters p =.78,α = 3.. Theoretial preditions within linear lin.) and quadrati quad.) approximations are reported for the +p-sat model, then for 3-SAT using equation 6). Parameters Experiments Theory p α ω ann ω typ nod. ω typ his. lin. quad ±..35 ± ±..4 ±..4 ± P G.77 α G log N / N log N / N Fig. 9. Coordinates of the highest baktraking point G in the searh tree with a starting ratio α =3.5 the upper sat phase) for different sizes N =,...,5 and averaged over, small sizes) to 8 N = 5) instanes. The fits shown are quadrati funtions of the plotting oordinate log N/N, and give the extrapolated loation of G in the large size limit: p G =.78 ±., α G = 3. ±.. Note the unertainty on these values due to the small number of instanes available at large instane sizes.

11 S. Coo and R. Monasson: Analysis of the dynamis of baktrak algorithms 55 In Setion 6, we shall show how the solving omplexity in the upper sat phase is related to the solving omplexity of orresponding +p-sat problems with parameters p G,α G. e 3 w w e e 4 Branh trajetories and the linear regime lower sat phase) In this setion, we investigate the dynamis of DPLL in the low ratio regime, where a solution is rapidly found in a linear time) and the searh tree essentially redues to a single branh shown Figure A. We start with some general omments on the dynamis indued by DPLL Set. 4.), useful to understand how the trajetory followed by the instane an be omputed in the p, α plane Set. 4.). These two first setions merely expose some previous works by Chao and Frano, and the reader is asked to onsult [8] for more details. In the last Setion 4.3, our numerial and analytial results for the solving omplexity are presented. In this setion, as well as in Setions 5 and 6, the ratio of the 3-SAT instane to be solved will be denoted by α. 4. Remarks on the dynamis of lauses 4.. Dynamial flows of populations of lauses As pointed out in Setion.3, under the ation of DPLL, some lauses are eliminated while other ones are redued. Let us all C j T ) the number of lauses of length j inluding j variables), one T variables have been assigned by the solving proedure. T will be alled hereafter time, not to be onfused with the omputational time neessary to solve a given instane. At time T =, we obviously have C 3 ) = α N, C ) = C ) =. As Boolean variables are assigned, T inreases and lauses of length one or two are produed. A skethy piture of DPLL dynamis at some instant T is proposed in Figure. We all e,e,e 3 and w,w the flows of lauses represented in Figure when time inreases from T to T +, that is, when one more variable is hosen by DPLL after T have already been assigned. The evolution equations for the three populations of lauses read, C 3 T +)=C 3 T ) e 3 T ) w T ) C T +)=C T ) e T )+w T ) w T ) C T +)=C T ) e T )+w T ). 4) The flows e j and w j are of ourse random variables that depend on the instane under onsideration at time T, and on the hoie of the variable label and value) done by DPLL. For a single desent, i.e. in the absene of baktraking, and for the GUC heuristi, the evolution proess 4) is Markovian and unbiased. The distribution of instanes generated by DPLL at time T is uniform over the set of all the instanes having C j T ) lauses of length j =,, 3 and drawn from a set of N T variables [8]. C3 C C 3-lauses -lauses -lauses Fig.. Shemati view of the dynamis of lauses. Clauses are sorted into three reipients aording to their lengths, i.e. the number of variables they inlude. Eah time a variable is assigned by DPLL, lauses are modified, resulting in a dynamis of the reipients populations lines with arrows). Dashed lines indiate the elimination of satisfied) lauses of lengths, or 3. Bold lines represent the redution of 3-lauses into -lauses, or -lauses into -lauses. The flows of lauses are denoted by e,e,e 3 and w,w respetively. A solution is found when all reipients are empty. The level of the rightmost reipient oinides with the number of unitary lauses. If this level is lowi.e. O)), the probability that two ontraditory lauses x and x are present in the reipient is vanishingly small. When the level is high i.e. O N)), ontraditions will our with high probability. 4.. Conentration of distributions of populations As a result of the additivity of 4), some onentration phenomenon takes plae in the large size limit. The numbers of lauses of lengths and 3, a priori extensive in N, do not flutuate too muh, ) T C j T )=N j + on) j =, 3). 5) N where the j s are the densities of lauses of length j averaged over the instane quenhed disorder) and the hoies of variables thermal disorder). In other words, the populations of - and 3-lauses are self-averaging quantities and we shall attempt at alulating their mean densities only. Note that, in order to prevent the ourrene of ontraditions, the number of unitary lauses must remain small and the density of -lauses has to vanish Time sales separation and deterministi vs. stohasti evolutions Formula 5) also illustrates another essential feature of the dynamis of lause populations. Two time sales are at play. The short time sale, of the order of the unity, orresponds to the fast variations of the numbers of lauses C j T ) j =,, 3). When time inreases from T to T + O) with respet to the size N), all C j s vary by O) amounts. Consequently, the densities of lauses j, that is, their numbers divide by N, are not modified. The densities j s evolve on a long time sale of the order of N and depend on the redued time t = T/N only.

12 56 The European Physial Journal B Due to the onentration phenomenon underlined above, the densities j t) will evolve deterministially with the redued time t. We shall see below how Chao and Frano alulated their values. On the short time sale, the relative numbers of lauses D j T )=C j T ) N j T/N) flutuate with amplitude N) and are stohasti variables. As said above the evolution proess for these relative numbers of lauses is Markovian and the probability rates master equation) are funtions of slow variables only, i.e. of the redued time t and of the densities and 3.As a onsequene, on intermediary time sales, muh larger than unity and muh smaller than N, thed j s may reah some stationary distribution that depend upon the slow variables. This situation is best exemplified in the ase j = where t) = as long as no ontradition ours and D T ) = C T ). Consider for instane a time delay T N, e.g. T = N. For times T lying in between T = tn and T = T + T = tn + N,the numbers of - and 3-lauses flutuate but their densities are left unhanged and equal to t) and 3 t). The average number of -lauses flutuates and follows some master equation whose transition rates from C = C T )to C = C T +)) define a matrix MC,C ) and depend on t,, 3 only. M has generally a single eigenvetor µc ) with eigenvalue unity, alled equilibrium distribution, and other eigenvetors with smaller eigenvalues in modulus). Therefore, at time T, C has forgotten the initial ondition C T ) and is distributed aording to the equilibrium distribution µc ) of the master equation. To sum up, the dynamial evolution of the lause populations may be seen as a slow and deterministi evolution of the lause densities to whih are superimposed fast, small flutuations. The equilibrium distribution of the latter adiabatially follows the slow trajetory. 4. Mathematial analysis In this setion, we expose Chao and Frano s alulation of the densities of - and 3-lauses. 4.. Differential equations for the densities of lauses Consider first the evolution equation 4) for the number of 3-lauses. This an be rewritten in terms of the average density 3 of 3-lauses and of the redued time t, d 3 t) = z 3 t), 6) dt where z 3 = e 3 + w denotes the averaged total outflow of 3-lauses Set. 4..). At some time step T T +, 3-lauses are eliminated or redued if and only if they ontain the variable hosen by DPLL. Let us first suppose that the variable is hosen in some - or -lauses. A 3-lause will inlude this variable or its negation with probability 3/N T ) and disappear with the same probability. Due to the unorrelation of lauses, we obtain z 3 t) = 3 3 t)/ t). If the literal assigned by DPLL is hosen among some 3-lause, this result has to be inreased by one sine this lause will neessarily be eliminated) in the large N limit. Let us all ρ j t) the probability that a literal is hosen by DPLL in a lause of length j =,, 3). The normalization of probability imposes that ρ t)+ρ t)+ρ 3 t) =, 7) at any time t. Extending the above disussion to -lauses, we obtain d 3 t) = 3 dt t 3t) ρ 3 t) d t) 3 = dt t) 3t) t t) ρ t), 8) In order to solve the above set of oupled differential equations, we need to know the probabilities ρ j.aswe shall see, the values of the ρ j an diretly be dedued from the heuristi of hoie, the so-alled generalized unitlause GUC) rule exposed in Setion... The solutions of the differential equations 8) will be expressed in terms of the fration p of 3-lauses and the ratio α of lauses per variable using the identities pt) = 3 t) t)+ 3 t), 4.. Solution for α /3 αt) = t)+ 3 t) t 9) When DPLL is launhed, -lauses are reated with an initial flow w ) =3α /. Let us suppose that α /3, i.e. w ). In other words, less than one -lause is reated eah time a variable is assigned. Sine the GUC rule ompels DPLL to look for literals in the smallest available lauses, -lauses are immediately removed just after reation and do not aumulate in their reipient. Unitary lauses are almost absent and we have ρ t) =; ρ t) = 3 3t) t) ; ρ 3t) = ρ t) α < /3). ) The solutions of 8) with the initial ondition p) =,α) = α read pt) =, αt) =α + ) t) 3/ t). ) Solution ) onfirms that the instane never ontains an extensive number of -lauses. At some final time t end, depending on the initial ratio, αt end ) vanishes: no lause is left and a solution is found.

13 S. Coo and R. Monasson: Analysis of the dynamis of baktrak algorithms Solution for /3 <α <α L We now assume that α > /3, i.e. w ) >. In other words, more than one -lause is reated eah time a variable is assigned. -lauses now aumulate, and give rise to unitary lauses. Due to the GUC presription, in presene of - or -lauses, a literal is never hosen in a 3-lause. Thus, ρ t) = t) t ; ρ t) = ρ t); ρ 3 t) = α > /3), ) as soon as t>. The solutions of 8) now read 4α t) pt) = α t) +3α +4ln t), αt) = α 4 t) + 3α +ln t). 3) 4 Solution 3) requires that the instane ontains an extensive number of -lauses. This is true at small times sine p ) = /α 3/ <. At some time t >, depending on the initial ratio, pt ) reahes bak unity: no - lause are left and hypothesis ) breaks down. DPLL has therefore redued the initial formula to a smaller 3-SAT instane with a ratio α = αt ). It an be shown that α < /3. Thus, as the dynamial proess is Markovian, the further evolution of the instane an be alulated from Setion Trajetories in the p,α plane We show in Figure 4 the trajetories obtained for initial ratios α =.6, α =andα =.8. When α > /3, the trajetory first heads to the left Set. 4..3) and then reverses to the right until reahing a point on the 3-SAT axis at small ratio α < /3) without ever leaving the sat region. Further ation of DPLL leads to a rapid elimination of the remaining lauses and the trajetory ends up at the right lower orner S, where a solution is ahieved Set. 4..). As α inreases up to α L, the trajetory gets loser and loser to the threshold line α C p). Finally, at α L 3.3, the trajetory touhes the threshold urve tangentially at point T with oordinates p T =/5,α T =5/3). Note the identity α T =/ p T ). 4.3 Complexity In this setion, we ompute the omputational omplexity in the range α α L from the previous results Absene of baktraking The trajetories obtained in Setion 4..4 represent the deterministi evolution of the densities of - and 3-lauses when more and more variables are assigned. Equilibrium flutuations of number of -lauses have been alulated by Frieze and Suen []. The stationary distribution µ t C ) of the population of -lauses an be exatly omputed at any time t. The most important result is the probability that C vanishes, µ t ) = αt) pt) ). 4) µ t ) respetively µ t )) may be interpreted as the probability that a variable assigned by DPLL at time t is hosen through splitting resp. unit propagation). When DPLL starts solving a 3-SAT instane, µ t= ) = and many splits are neessary. If the initial ratio α is smaller than /3, this statement remains true till the final time t end and the absene of -lauses prevents the onset of ontradition. Conversely, if /3 <α <α L,ast grows, µ t ) dereases are more and more variables are fixed through unit-propagation. The population of -lauses remains finite and the probability that a ontradition ours when a new variable is assigned is O/N ) only. However small is this probability, ON) variables are fixed along a omplete trajetory. The resulting probability that a ontradition never ours is stritly smaller than unity [], Probano ontradition) = exp 4 tend dt t µ t )) µ t ) ) 5) Frieze and Suen have shown that ontraditions have no dramati onsequenes. The number of baktrakings neessary to find a solution is bounded from above by a power of log N. The final trajetory in the p, α plane is idential to the one shown in Setion 4..4 and the inrease of omplexity is negligible with respet to ON). When α reahes α L, the trajetory intersets the α = / p) line in T. At this point, µ) vanishes and baktraking enters massively into play, signaling the ross-over to the exponential regime Length of trajetories From the above disussion, it appears that a solution is found by DPLL essentially at the end of a single desent Fig. A). Complexity thus sales linearly with N with a proportionality oeffiient γα ) smaller than unity. For α < /3, lauses of length unity are never reated by DPLL Set. 4..). Thus, DPLL assigns the overwhelming majority of variables by splittings. γα )simply equals the total fration t end of variables hosen by DPLL. From ), we obtain γα )= 4 α +), α /3. 6) For larger ratios, i.e. α > /3, the trajetory must be deomposed into two suessive portions Set. 4..3). During the first portion, for times < t < t, -lauses are present with a non vanishing density t). Some of

14 58 The European Physial Journal B these -lauses are redued to -lauses that have to be eliminated next. Consequently, when DPLL assigns an infinitesimal fration dt of variables, a fration ρ t) = αt) pt))dt are fixed by unit-propagation only. The number of nodes divided by N) along the first part of the branh thus reads, depth T : C 3 3-lauses one variable assigned t γ = t dt αt) pt)). 7) At time t, the trajetory touhes bak the 3-SAT axis p = at ratio α αt ) < /3. The initial instane is then redues to a smaller and smaller 3-SAT formula, with a ratio αt) vanishing at t end. Aording to the above disussion, the length of this seond part of the trajetory equals γ = t end t. 8) It results onvenient to plot the total omplexity γ = γ + γ in a parametri way. To do so, we express the initial ratio α and the omplexity γ in terms of the end time t of the first part of the branh. A simple alulation from equations 3) leads to α t )= 4ln t ) 3t t ) γt 4 t ) )= + t ) α t )) + t + t )ln t ) 4 α t )t ) 3 t ). 9) As t grows from zero to t L.89, the initial ratio α spans the range [/3; α L ]. The omplexity oeffiient γα ) an be omputed from equations 6,9) with the results shown Figure 6. The agreement with the numerial data of Setion is exellent. 5 Tree trajetories and the exponential regime UNSAT phase) To present our analytial study of the exponentially large searh trees generated when solving hard instanes, we onsider first a simplified growth tree dynamis in whih variables, on eah branh, are hosen independently of the - or -lauses and all branhes split at eah depth T.This toy model is too simple a growth proess to reprodue a searh tree analogous to the ones generated by DPLL on unsat instanes. In partiular, it laks two essential ingredients of the DPLL proedure: the generalized unit lause rule literals are hosen from the shortest lauses), and the possible emergene of ontraditions halting a branh growth. Yet, the study of the toy model allows us to expose and test the main analytial ideas, before turning to the full analysis of DPLL in Setion 5.. depth T+ : C ^ 3 and C 3 3-lauses Fig.. Shemati representation of the searh tree buildingupinthetoydynamialproess.abranharryingaformula with C 3 3-lauses splits one a variable is assigned, and gives birth to two branhes having C 3 and Ĉ3 3-lauses respetively. The variable is assigned randomly, independently of the 3-lauses. Clauses of lengths one or two are not onsidered. 5. Analytial approah for exponentially large searh trees: a toy ase 5.. The toy growth dynamis In the toy model of searh tree onsidered hereafter, only 3-lauses matter. Initially, the searh tree is empty and the 3-SAT instane is a olletion of C 3 3-lauses drawn from N variables. Next, a variable is randomly piked up and fixed to true or false, leading to the reation of one node and two outgoing branhes arrying formulae with C 3 and Ĉ3 3-lauses respetively Fig. ). This elementary proess is then repeated for eah branh. At depth or time T, that is when T variables have been assigned along eah branh, there are T branhes in the tree. Thequantitywefousonisthenumberofbranhes having a given number of 3-lauses at depth T. On eah branh, after a variable has been assigned, C 3 dereases by lause redution and elimination) or remains unhanged when the hosen variable does not appear in the lauses). So, the reipients of 3-lauses attahed to eah branh leak out with time T Fig. ). We now assume that eah branh in the tree evolves independently of the other ones and obeys a Markovian stohasti proess. This amounts to neglet orrelations between the branhes, whih ould arise from the seletion of the same variable on different branhes at different times. As a onsequene of this deorrelation approximation, the value of C 3 at time T + is a random variable whose distribution depends only on C 3 and time T ). The deorrelation approximation allows to write a master equation for the average number BC 3 ; T ) of branhes arrying C 3 3-lauses at depth T in the tree, BC 3 ; T +)= C 3 = K toy C 3,C 3; T ) BC 3; T ). ) K toy is the branhing matrix; the entry K toy C 3,C 3 )isthe averaged number of branhes with C 3 lauses issued from a branh arrying C 3 lauses after one variable is assigned. This number lies between zero if C 3 >C 3 ) and two the maximum number of branhes produed by a split), and

15 S. Coo and R. Monasson: Analysis of the dynamis of baktrak algorithms 59 is easily dedued from the evolution of the reipient of 3-lauses in Figure, ) K toy C 3,C 3; T )=χc 3 C C 3 ) 3 C 3 C 3 ) C 3 3 C3 3 ) C3 ) N T N T The fator omes from the two branhes reated at eah split; χc 3 C 3) equals unity if C 3 C 3, and zero otherwise. The binomial distribution in ) expresses the probability that the variable fixed at time T appears in C 3 C 3 3-lauses. 5.. Partial differential equation for the distribution of branhes For large instanes N,M, this binomial distribution simplifies to a Poisson distribution, with parameter m 3 T )=3C 3 /N T ). The branhing matrix ) thus reads, K toy C 3,C 3 ; T ) K toy C 3 C 3,m 3 T )) = χ C 3 C 3 )e m3t ) m 3T ) C 3 C3 C 3 C 3)! ) Consider now the variations of the entries of K ) over a time interval T = tn T T =t+ɛ) N. Here, ɛ is a small parameter but of the order of one with respet to N. In other words, we shall first send N to infinity, and then ɛ to zero. m 3 T ) weakly varies between T and T : m 3 t) = 3 3 / t) +Oɛ) where 3 = C 3 /N is the intensive variable for the number of 3-lauses. The branhing matrix ) an thus be rewritten, for all times T ranging from T to T,as K toy C 3 C 3,m 3 t)) = χ C 3 C 3 ) e m3t) m 3t) C 3 C3 C 3 C + Oɛ). 3) 3)! We may now iterate equation ) over the whole time interval of length T = T T = ɛn, BC 3 ; tn+ T )= C 3 = [K toy ] T C 3 C 3 ; t) BC 3; tn), 4) where. T denotes the T th power of the branhing matrix. As K depends only on the differene C 3 C 3,itanbe diagonalized by plane waves vq 3,C 3 )=e i q3 C3 / π with wave numbers q 3 < π beause C 3 is an integervalued number). The orresponding eigenvalues read [ λ q3 t) = exp m 3 t) e i q3 ) ]. 5) Reexpressing matrix 3) using its eigenvetors and eigenvalues, equation 4) reads BC 3 ; tn + T )= C 3 = π dq 3 π λq3 t) ) T e ic3 C 3 ) q3 BC 3; tn). 6) Branhes dupliate at eah time step and proliferate exponentially with T = tn. A sensible saling hypothesis for their number is, at fixed fration 3, lim N N ln B 3 N; tn)=ω 3,t). 7) Note that ω has to be divided by ln to obtain the logarithm of the number of branhes in base to math the definition of Setion Similarly, the sum over C 3 in 4, 6) will be replaed in the large N limit with an integral over the real-valued variable r 3 defined through C 3 C 3 = r 3 T. 8) r 3 is of the order of unity when T gets very large, sine the number of 3-lauses that disappear after having fixed T variables is typially of the order of T. We finally obtain from equation 6), ) 3/ɛ π dq 3 exp Nω 3,t+ ɛ) = T dr 3 π [ exp N ɛ ln λ q3 t) i ɛr 3 q 3 + ω 3 + ɛr 3,t) )]. 9) In the N limit, the integrals in 9) may be evaluated using the saddle point method, [ ] ω 3,t+ ɛ) =max ɛ ln λ q3 t) i ɛr 3 q 3 + ω 3 + ɛr 3,t) r 3,q 3 + Oɛ ), 3) due to the terms negleted in 3). Assuming that ω 3,t) is a partially differentiable funtions of its arguments, and expanding 3) at the first order in ɛ, we obtain the partial differential equation, ω t 3,t)=max r 3,q 3 ln λ q3 t) i r 3 q 3 + r 3 ω 3 3,t) ). 3) The saddle point lies in q 3 = i ω 3, leading to, ω t 3,t)=ln 3 3 t + 3 [ ] 3 ω t exp 3,t). 3) 3 It is partiularly interesting to note that a partial differential equation emerges in 3). In ontradistintion with the evolution of a single branh desribed by a set of ordinary differential equations Set. 4), the analysis of a full tree requires to handle information about the whole distribution ω of branhes and, thus, to a partial differential equation.

16 5 The European Physial Journal B 5..3 Legendre transform and solution of the partial differential equation.35.3 t=.3 To solve this equation is onvenient to define the Legendre transformation of the funtion ω 3,t), ϕy 3,t)=max 3 [ω 3,t)+y 3 3 ]. 33) From a statistial physis point of view this is equivalent to pass, at fixed time t, from a miroanonial entropy ω defined as a funtion of the internal energy 3, to a free energy ϕ defined as a funtion of the temperature y 3. More preisely, ϕy 3,t) is the logarithm divided by N of the generating funtion of the number of nodes BC 3 ; tn). Equation 33) defines the Legendre relations between ω and ϕ, ω ϕ 3 = y 3 and 3y 3) y 3 = 3. 34) y3 3) In terms of ϕ and y 3, the partial differential equation 3) reads ϕ t y 3,t)=ln 3 t ) ϕ y e y3 3,t), 35) y 3 and is linear in the partial derivatives. This is a onsequene of the Poissonian nature of the distribution entering K toy ). The initial ondition for the funtion ϕy 3,t) is smoother than for ω 3,t). At time t =,the searh tree is empty: ω 3,t= ) equals zero for 3 = α, and for 3 α. The Legendre transform is thus ϕy 3, ) = α y 3, a linear funtion of y 3. The solution of equation 35) reads [ ϕy 3,t)=t ln + α ln e y3 ) t) 3] 36) and, through Legendre inversion, ω 3,t)=t ln ln t)+α ln α [ ] α 3 3 ln 3 α 3 )ln t) 3, 37) for < 3 <α,andω = outside this range. We show in Figure the behavior of ω 3,t) for inreasing times t. The urve has a smooth and bell-like shape, with a maximum loated in ĉ 3 t) =α t) 3, ˆωt) =t ln. The number of branhes at depth t equals Bt) =e N ˆωt) = Nt, and almost all branhes arry ĉ 3 t) N 3-lauses, in agreement with the expression for 3 t) found for the simple branh trajetories in the ase /3 < α < 3.3 Set. 4..3). The top of the urve ω 3,t), at fixed t, provides diret information about the dominant, most numerous branhes. For the real DPLL dynamis studied in next setion, the partial differential equation for the growth proess is muh more ompliated, and exat analytial solutions are not available. If we fous on exponentially dominant log number of branhes ϖ t=. t=.5 t=. t=.5 t= number of 3 lauses 3 Fig.. Logarithm ω of the number of branhes base, and divided by N) as a funtion of the number 3 of 3-lauses at different times t for the simplified growth proess of Setion 5.. At time t =, the searh tree is empty and the ratio of lauses per variable equals α = 5. Later on, the tree grows and a whole distribution of branhes appears. Dominant branhes orrespond to the top of the distributions of oordinates ĉ 3t), ˆωt). Branhes beome exponentially more numerous with time, while they arry less and less 3-lauses. branhes, we may as a first approximation follow the dynamial evolution of the points of the urve ω 3,t) around the top. To do so, we linearize the partial differential equation 35) for the Legendre transform ϕ around the origin y 3 =, see 34), ϕ t y 3,t) ln 3 t y ϕ 3 y 3,t). 38) y 3 The solution of the linearized equation, ϕy 3,t)=t ln + α t) 3 y 3, is itself a linear funtion of y 3. Through Legendre inversion, the slope gives us the oordinate ĉ 3 t) of the maximum of ω, and the onstant term, the height ˆωt) of the top. 5. Analysis of the full DPLL dynamis in the unsat phase 5.. Parallel versus sequential growth of the searh tree A generi refutation searh tree in the unsat phase is shown in Figure B. It is the output of a sequential building proess: nodes and edges are added by DPLL through suessive desents and baktrakings. We have imagined a different building up, that results in the same omplete tree but an be mathematially analyzed: the tree grows in parallel, layer after layer Fig. 3). A new layer is added by assigning, aording to DPLL heuristi, one more variable along eah living branh. As a result, some branhes split, others keep growing and the remaining ones arry ontraditions and die out. As in Setion 5.., when going from depth T to T +, we neglet all the orrelations between branhes that are not immediately issued from a splitting node Fig. ). As a result, the evolution of the searh tree is Markovian and an be desribed by a master equation for the average

17 KC,C,C 3; C,C,C 3; T )= 8< : δ C ) X C z = C X z = C z! N T S. Coo and R. Monasson: Analysis of the dynamis of baktrak algorithms 5! C 3 3 C 3 C 3 C 3 3 C3 C3 N T N T C z z X! C z N T z N T z N T C z z X w = w = z z X C 3 C 3 C 3 C 3 C 3 C 3! w w = z z! δ w C C w +z δ C C w + + δ C! 9= ; δ w C C w +z + [δ C w + δ C w ], 4) instane to be solved searh tree at depth T- depth T+ Fig. 3. Imaginary, parallel growth proess of an unsat searh tree used in the theoretial analysis of the omputational omplexity. Variables are fixed through unit-propagation, or the splitting heuristi as in the DPLL proedure, but branhes evolve in parallel. T denotes the depth in the tree, that is the number of variables assigned by DPLL along eah living) branh. At depth T, one literal is hosen on eah branh among -lauses unit-propagation, grey irles not represented in Fig. ), or, 3-lauses split, blak irles as in Fig. ). If a ontradition ours as a result of unit-propagation, the branh gets marked with C and dies out. The growth of the tree proeeds until all branhes arry C leaves. The resulting tree is idential to the one built through the usual, sequential operation of DPLL. number of branhes B arrying instanes with C j j-lauses j =,, 3), BC,C,C 3 ; T +)= C,C,C 3 = K C,C,C 3,C,C,C 3 ; T ) BC,C,C 3 ; T ). 5.. Branhing matrix for DPLL T 39) The branhing matrix K for DPLL with GUC heuristi equals See equation 4) above where δ C denotes the Kroneker delta funtion: δ C =if C =,δ C = otherwise. To understand formula 4), the piture of reipients in Figure proves to be useful. K expresses the average flow of lauses into the sink elimination), or to the reipient to the right redution), when passing from depth T to depth T + by fixing one variable. Among the C 3 C 3 lauses that flow out from the leftmost 3-lauses reipient, w lauses are redued and go into the -lauses ontainer, while the remaining C 3 C 3 w are eliminated. w is a random variable in the range w C 3 C 3 and drawn from a binomial distribution of parameter /, whih represents the probability that the hosen literal is the negation of the one in the lause. We have assumed that the algorithm never hooses the variable among 3-lauses. This hypothesis is justified a posteriori beause in the unsat region, there is always exept at the initial time t = ) an extensive number of -lauses. Variable are hosen among -lauses or, in the absene of the latters, among -lauses. The term on the r.h.s. of equation 4) beginning with δ C respetively δ C ) orresponds to the latter resp. former) ase. z is the number of lauses other than the one from whih the variable is hosen) flowing out from the seond reipient; it obeys a binomial distribution with parameter /N T ), equal to the probability that the hosen variable appears in a -lause. The -lauses ontainer is, at the same time, poured with w lauses. In an analogous way, the unitary lauses reipient welomes w new lauses if it was empty at the previous step. If not, a -lause is eliminated by fixing the orresponding literal. The branh keeps growing as long as the level C of the unit lauses reipient remains low, i.e. C remains of the order of unity and the probability to have two, or more, -lauses with opposite literals an be negleted. For this reason, we do not take into aount the extremely rare) event of -lauses inluding equal or opposite literals and z is always equal to zero. We shall onsider later a halt riterion for the tree growth proess dot-dashed line in the phase diagram of Fig. 4), where the ondition C = O) breaks down due to an avalanhe of -lauses. Finally, we sum over all possible flow values w,z,w that satisfy the onservation laws C C = w z, C C = w whenc or, when C =,C C = w z, C = w if the literal is the same as the one in the lause or C = w + if the literal is the negation of the one in the lause. The presene of two δ is responsible for the growth of the number of branhes. In the real sequential DPLL dynamis, the inversion of a literal at a node requires baktraking; here, the two edges grow in parallel at eah node aording to Setion 5...

18 5 The European Physial Journal B In the large N limit, the matrix 4) an be written in terms of Poisson distributions through the introdution of the parameters m 3 =3C 3 /N T )andm =C /N T ), see equation ) The ground state of the branhing matrix, and its transition of deloalization Due to the translational invarianes of K in C 3 C 3 and C C, the vetors v q,q,q 3 C,C,C 3)=e iqc +q3c 3 ) ṽ q C ) 4) are eigenvetors of the matrix K with eigenvalues { [ e iq 3 ]} λ q,q,q 3 = exp m 3 +e iq ) λ q,q, 4) if and only if ṽ q C ) is an eigenvetor, with eigenvalue λ q,q, of the redued matrix K q C,C )= e m m e iq ) z z ) z z z! w = w z = } {δ C C w+ δ C )+ e i q δ C w +δ C w ) δ C. 43) Note that, while q,q 3 are wave numbers, q is a formal index used to label eigenvetors. The matrix K q 43) has been obtained by applying K onto the vetor v 4), and summing over C 3,C,w and z 3. The diagonalization of the non hermitian matrix K q is exposed in Appendix A, and relies upon the introdution of the generating funtions of the eigenvetors ṽ q, Ṽ q x) = ṽ q C ) x C. 44) C = The eigenvalue equation for K q translates into a selfonsistent equation for Ṽqx), the singularities of whih an be analyzed in the x plane, and permit to alulate the largest eigenvalue of K q, λ,q = + +4e { iq e i q exp m + m + ) } +4e 4 iq. 45) The properties of the orresponding, maximal eigenvetor ṽ are important. Depending on parameters q and m, ṽ C ) is either loalized around small integer values of C the average number of -lauses is of the order of the unity), or extended the average value of C is of the order of N). As ontraditions inevitably arise when C = ON), the deloalization transition undergone by the maximal eigenvetor provides a halt riterion for the growth of the tree. We show in next setion that q is purely imaginary at the saddle-point, and therefore the eigenvalue in equation 45) is real-valued Partial differential equation for the searh tree growth Following Setion 5.. and equation 6), we write the evolution of the number of branhes between times tn and tn + T using the spetral deomposition of K, BC,C,C 3 ; tn + T )= C,C,C 3 q π dq π dq 3 π e i[q3 C3 C 3 )+q C C )] ṽ q C )ṽ q + C ) λ q,q,q 3 ) T BC,C,C 3,tN). 46) q denotes the disrete or ontinuous) sum on all eigenvetors and ṽ q + the left eigenvetor of K. Wemakethe adiabati hypothesis that the probability to have C unit lauses at time tn + T beomes stationary on the time sale T N and is independent of the number C of -lauses at time tn Set. 4). As T gets large, and at fixed q,q 3,thesumoverq is more and more dominated by the largest eigenvalue q =, due to the gap between the first eigenvalue assoiated to a loalized eigenvetor) and the ontinuous spetrum of deloalized eigenvetors Appendix A). Let us all Λq,q 3 ) λ,q,q 3 this largest eigenvalue, obtained from equations 45, 4). Defining the average of the number of branhes over the equilibrium distribution of -lauses, BC,C 3 ; T )= equation 46) leads to BC,C 3 ; tn + T )= BC,C,C 3 )ṽ C ), 47) C = C,C 3 = π dq π dq 3 π e i[q3 C3 C 3 )+q C C )] Λq,q 3 ) ) T BC,C 3; tn). 48) The alulation now follows losely the lines of Setion 5... We all ω, 3,t) the limit value of the logarithm of the number of branhes arrying an instane with N -lauses and 3 N 3-lauses at depth tn as N. Similarly, we rewrite the sums on C,C 3 on the r.h.s. of equation 48) as integrals over the redued variable r =C C )/T, r 3 =C 3 C 3)/T, see equations 6, 8, 9). A saddle-point alulation of the four integrals over q,q 3,r,r 3 an be arried out, resulting in a partial differential equation for ω, 3,t), ω t, 3,t)=lnΛ i ω, i ω ), 49) 3

19 S. Coo and R. Monasson: Analysis of the dynamis of baktrak algorithms 53 or, equivalently, ω t = ω +ln + 4e ω t + t [e ω ω e ) [e ω 4 ] ) ] + 4e ω +. 5) 5..5 Approximate solution of the partial differential equation with ĉ t) =α p 9 ) 5+75 [ t) t) 3] 6 ) + α p )++ 5 t) ) t), ĉ 3 t) =α p t) 3, 5 4 ) 5 ˆωt) =α p t) 3 ) 58 + ) ) 5 +t ln α ) 5+75 p + α p ) ) ) 5 t) ) As in Setion 5..3, we introdue the Legendre transformation of ω, 3,t), ϕy,y 3,t)=max ω, 3,t)+y + y 3 3 ). 5), 3 The resulting partial differential equation on ϕ is given in Appendix B, and annot be solved analytially. We therefore limit ourselves to the neighborhood of the top of the surfae ω, 3,t) through a linearization around y = y 3 =, ϕ t ln + ) 5 5+ ) 5 y 3 y 3 y ) ϕ t y 3 3 ) ϕ y 5) t y The solution of equation 5) is given by the ombination of a partiular solution with ϕ y 3 = and the solution of the homogeneous ounterpart of equation 5). We write below the general solution for any +p-sat unsat problem with parameters p,α >α C p ), the 3-SAT ase orresponding to p =. The initial ondition at time t = reads ϕy,y 3, ) = α p y 3 + α p ) y.weobtain ϕy,y 3,t)=ĉ t) y +ĉ 3 t) y 3 +ˆωt) 53) Within the linearized approximation, the distribution ω, 3,t) has its maximum loated in ĉ t), ĉ 3 t) with a height ˆωt), and is equal to minus infinity elsewhere. The oordinates of the maximum as funtions of the depth t defines the tree trajetory, i.e. the evolution, driven by the ation of DPLL, of the dominant branhes in the phase diagram of Figure 4. We obtain straightforwardly this trajetory from equations 54) and the transformation rules 9). The numerial resolution of PDE 5), through the use of the Legendre transform 5), is presented in [3] Interpretation of the tree trajetories and results for the omplexity In Figure 4, the tree trajetories orresponding to solving 3-SAT instanes with ratios α =4.3, 7 and are shown. The trajetories start on the right vertial axis p =and head to the left until they hit the halt line dot-dashed urve), α = 3+ 5 ln + ) 5 p.59 p, 55) at some time t h >, whih depends on α. On the halt line, a deloalization transition for the largest eigenvetor takes plae and auses an avalanhe of unitary lauses with the emergene of ontraditions, preventing branhes from further growing. The deloalization transition taking plae on the halt line means that the stationary probability µ t ) of having no unit-lause µ t ) = ṽ ) C ṽc = ) = Ṽ) Ṽ ), 56) vanishes at t = t h. From equations 4, 45, 56, A., A.3), the largest eigenvalue for dominant branhes, Λ, )

20 54 The European Physial Journal B reahes its lowest value, Λ, ) =, on the halt line 55) for parameters y = y 3 =, see Appendix A and Fig. 7). As expeted, the emergene of ontraditions on dominant branhes oinides with the halt of the tree growth, see equation 49). The logarithm ˆωt) of the number of dominant branhes inreases along the tree trajetory, from zero at t = up to some value ˆωt h ) > on the halt line. This final value, divided by ln, is our analytial predition within the linear approximation of Set. 5..5) for the omplexity ω. We desribe in Appendix B, a refined, quadrati expression for the Legendre transform ϕ 5) that provides another estimate of ω. The theoretial values of ω, within linear and quadrati approximations, are shown in Table for α =, 5,, 7, 4.3, and ompare very well with numerial results. Our alulation, whih is fat an annealed estimate of ω Set. 3.4.), is very aurate. The deorrelation approximation Set. 5.) beomes more and more preise with larger and larger ratios α. Indeed, the repetition of variables in the searh tree dereases with the size of the tree. For large values of α,weobtain ωα ) 3 + [ 5) ln + )] ) 6lnα α The /α saling of ω has been previously proven by Beam, Karp, and Pitassi [9], independently of the partiular heuristis used. Showing that there is no solution for random 3-SAT instanes with large ratios α is relatively easy, sine assumptions on few variables generate a large number of logial onsequenes, and ontraditions emerge quikly Fig. ). This result an be inferred from Figure 4. As α inreases, the distane between the vertial 3-SAT axis and the halt line dereases; onsequently, the trajetory beomes shorter, and the size of the searh tree, smaller Length of the dominant branhes To end with, we alulate the length of dominant branhes. The probability that a splitting ours at time t is µ t ) defined in 56). Let us define nl, T ) as the number of branhes having L nodes at depth T in the tree. The evolution equation for n is nl, T +) = µ t )) nl, T )+µ t ) nl,t). The average branh length at time T, LT ) = L= LnL, T )/ L= nl, T ),obeysthesimpleevolution relation LT +) LT ) =µ t )/ + µ t )). Therefore, the average number of nodes divided by N), l = LtN) /N, present along dominant branhes one the tree is omplete, is equal to l = th dt µ t ) +µ t ) 58) where t h is the halt time. For large ratios α, the average length of dominant branhes sales as ) ln ) l α ).48 59) 3 α α The good agreement between this predition and the numerial results an be heked on the insets of Figure 7 for different values of α. 6 Mixed trajetories and the intermediate exponential regime upper sat phase) In this setion, we show how the omplexity of solving 3-SAT instanes with ratios in the intermediate range α L <α <α C an be understood by ombining the previous results on branh and tree trajetories. 6. Branh trajetories and the ritial line of +p-sat In the upper sat phase, the single branh trajetory intersets the ritial line α C p) at some point G, whose oordinates p G,α G depend on the initial ratio α.thepointg orresponding to α =3.5 is shown in Figure 4. Beyond G, the branh trajetory whih desribes the first desending branh in the tree) enters the unsat phase, meaning that the solving proedure has turned a satisfiable 3-SAT instane into an unsatisfiable +p-sat instane. This situation is shematized in Figure 4. G denotes the highest node in the tree arrying an unsat instane. G is its immediate anestor. We also denote by N and N the average numbers of variables not assigned by DPLL at points G and G respetively: N <N <N. The rossing of the ritial line orresponds to the move from G to G through the assignment of one variable, say x to true. Note that, while the parameters p, α of the +p-sat instanes orresponding to G and G are not equal for finite size instanes, they both tend to p G,α G in the N limit. Similarly, N /N and N /N onverge to t G,wheret G is the fration of variable assigned by the algorithm when reahing point G. One DPLL has reahed node G, a refutation subtree must be built by DPLL below G in the tree. The orresponding sub)tree trajetory, penetrating the unsat phase from G up to the arrest line, is shown in Figure 4. Let us all Q Nω the size of this subtree. One the subtree has been entirely baktraked, DPLL is bak in G, assigns x to false, starts desending on the right side of Fig. 4). Further baktraking will be neessary but a solution will eventually be found. The branh ontaining the solution is the rightmost path in Figure C. Its trajetory is highly non typial among the branhes in the tree whih almost surely follow the sub)tree trajetory desribedabove),andisplottedinfigure4. Q obviously provides a lower bound to the total omplexity. It is a reasonable assumption that the amount of

21 S. Coo and R. Monasson: Analysis of the dynamis of baktrak algorithms 55 satisfiable 3-SAT instane satisfiable +p-sat instanes unsatisfiable +p-sat instane G UNSAT subtree G solution total height of the searh tree = ON) Fig. 4. Detailed struture of the searh tree in the upper sat phase α L <α<α C). DPLL starts with a satisfiable 3-SAT instane and transforms it into a sequene of +p-sat instanes. The leftmost branh in the tree symbolizes the first desent made by DPLL. Above node G, instanes are satisfiable while below G, instanes have no solutions. A grey triangle aounts for the exponentially) large refutation subtree that DPLL has to go through before baktraking above G and reahing G. By definition, the highest node reahed bak by DPLL is G. Further baktraking, below G, will be neessary but a solution will be eventually found right subtree), see Figure C. baktraking in the right subtree ontaining the solution) will be, to the exponentially dominant order, equal to Q. Therefore, the logarithm of the total size of the searh tree divided by N) is simply equal to ω. 6. Analytial alulation of the size of the refutation subtree The oordinates p G,α G = α C p G ) of the rossing point G depend on the initial 3-SAT ratio α and may be omputed from the knowledge of the +p-sat ritial line α C p) and the branh trajetory equations 3). For α =3.5, we obtain p G =.78 and α G =3.. Point G is reahed by the branh trajetory one a fration t G.9 of variables have been assigned by DPLL. One G is known, we onsider the unsatisfiable +p- SAT instanes with parameters p G,α G as a starting point for DPLL. The alulation exposed in Setion 5 an be used with initial onditions p G,α G.WeshowinTable4 the results of the analytial alulation of ω G, within linear and quadrati approximations for a starting ratio α =3.5. Note that the disrepany between both preditions is larger than for higher values of α. The logarithm ω of the total omplexity is defined through the identity Nω = N tg) ωg, or equivalently, ω = ω G t G ). 6) The resulting value for α =3.5 is shown in Table Comparison with numeris for α = 3.5 We have heked numerially the senario of Setion 6. in two ways. First, we have omputed during the ation of DPLL, the oordinates in the p, α plane of the highest baktraking point in the searh tree. The agreement with the oordinates of G omputed in the previous paragraph is very good Set. 3). However, the numerial data show large flutuation and the experimental fits are not very aurate, leading to unertainties on p G and α G of the order of. and. respetively. In addition, note that the analytial values of the oordinates of G are not exat sine the ritial line α C p) is not rigorously known Set..3). Seondly, we ompare in Table 4 the experimental measures and theoretial preditions of the omplexity of solving unsatisfiable +p-sat instanes with parameters p G,α G. The agreement between all values is quite good and leads to ω G =.4 ±.. Numeris indiate that the annealed value of the omplexity is equal or slightly larger) than the typial value. Therefore the annealed alulation developed in Setion 5 agrees well the data obtained for +p-sat instanes. One ω G and t G are known, equation 6) gives aess to the theoretial value of ω. The agreement between theory and experiment is very satisfatory Tab. 4). Nevertheless, let us stress the existene of some unertainty regarding the values of the highest baktraking point oordinates p G,α G.Numerial simulations on +p-sat instanes and analytial heks show that ω depends strongly on the initial fration p. Variations of the initial parameter p G around.78 by p =. hange the final result for the omplexity by ω =.3.4, twie as large as the statistial

22 56 The European Physial Journal B unertainty at fixed p = p G =.78. Improving the auray of the data would require a preise determination of the oordinates of G. We show in Figure 4 the trajetory of the atypial, rightmost branh ending with a solution) in the tree, obtained from simulations for N = 3. It omes as no surprise that this trajetory, whih arries a satisfiable and highly biased +p-sat instane, may enter the unsat region defined for the unbiased +p-sat distribution. The trajetory eventually reahes the α = axis when all lauses are eliminated. Notie that the end point is not S, but the lower left orner of the phase diagram. As a onlusion, our work shows that, in the α L <α< α C range, the omplexity of solving 3-SAT is related to the existene of a ritial point of +p-sat. The right part of the +p-sat ritial line, omprised between T and the threshold point of 3-SAT, an be determined experimentally as the lous of the highest baktraking points in 3-SAT solving searh trees, when the starting ratio α spans the interval α L α α C. 7 Complexity of +p-sat solving and relationship with stati 7. Complexity diagram We have analyzed in the previous setions the omputational omplexity of 3-SAT solving. The analysis may extended to any +p-sat instane, with the results shown in Figure 5. In addition to the three regimes unveiled in the study of 3-SAT 3, a new omplexity region appears, on the left side of the line α =/ p), referred to as weak deloalization line. To the right respetively left) of the weak deloalization line, the seond largest eigenvetor of the branhing matrix K is loalized resp. deloalized), see Appendix A. When solving a 3-SAT instane, or more generally a +p-sat instane with parameters p,α / p ), the size of the searh tree is exponential in N when the weak deloalization line is rossed by the tree trajetory Fig. 4). Thus, the ontribution to the average flow of unit-lauses oming from the seond largest eigenvetor is exponentially damped by the largest eigenvetor ontribution, and no ontradition arises until the halt, strong deloalization line is hit. If one now desires to solve a +p-sat instane whose representative point p,α lies to the left of the weak deloalization urve, the deloalization of the seond largest eigenvetor stops immediately the growth of the tree, before the distribution of -lauses ould reah equilibrium, see disussion of Setion Therefore, in the range of parameters p,α / p ), proving unsatisfiability 3 Though differential equations 8) depend on t when written in terms of, 3, they are Markovian if rewritten in term of the variables p, α. Therefore, the lous of the +p-sat instanes points, p,α, giving rise to trajetories touhing the threshold line in T, simply oinides with the 3-SAT trajetory starting from α = α L. Fig. 5. Four different regions oexist in the phase diagram of the +p-sat model, aording to whether omplexity is polynomial or exponential, and formulae are sat or unsat. Borderlines are from top to down): α = / p) dotted line), α Cp) full line), and the branh trajetory dashed line), starting in,α L) and ending at point T tangentially to the threshold line. The triritial point T S, with oordinates p S.4,α S =/ p S), separates seond from first order ritial point on the threshold line, and lies very lose to T. Inset: shemati blow-up of the T, T S region same symbols as in the main piture). does not require an exponentially large omputational effort. 7. Polynomial/exponential rossover and the triritial point The inset of Figure 5 show a shemati blow up of the neighborhood of T and T S, where all omplexity regions meet. From the above disussion, the omplexity of solving ritially onstrained +p-sat instanes is polynomial up to p S, and exponential above, in agreement with previous laims based on numerial investigations [3]. In the range /5 <p <p S, omputational omplexity exhibits a somewhat unusual behavior as a funtion of α. The peak of hardness is indeed not loated at ritiality where the saling of omplexity is only polynomial), but slightly below threshold, where omplexity is exponential. Unfortunately, the narrowness of the region shown in the inset of Figure 5 seems to forbid to hek this statement through experiments. Let us reall that, though replia alulations provide p S.4, no rigorous proof of the inequality p S > /5 is available, leaving open the possibility that T and T S ould atually oinide. To end with, let us stress that T, onversely to T S,depends a priori on the splitting heuristi. Nevertheless, the loation of T seems to be more insensitive to the hoie of the heuristis than branh trajetories. For instane, the UC and GUC heuristis both lead to the same tangential hit point T, while the starting ratios of the orresponding