Searching Ground States in Ising Spin Glass Systems Steven Homer Computer Science Department Boston University Boston, MA 02215 Marcus Peinado German National Research Center for Information Technology (GMD) Scientic Computation and Algorithms Institute (SCAI) D-53754 Sankt Augustin, Germany Abstract We present an application of the MaxCut problem in statistical physics. We design a branch-and-bound algorithm for MaxCut and use it to nd all ground states in three dimensional 5 5 5 Ising spin glass systems. Keywords: Ising spin glass systems, brand-and-bound, MaxCut, ground state magnetization, statistical physics 1 Introduction Given a weighted undirected graph G = (V; E), a set S V of vertices denes the cut (S; V n S). The size of the cut is the sum of P the weights w ij of all edges which connect a vertex in S with a vertex in V n S, i.e. cut(s) = i2s;j2v ns w ij. The MaxCut problem is the problem of nding a cut of maximum size in the input graph G. The MaxCut problem is one of the classical NP complete problems. As such, it has been thoroughly studied. The ground state structure of Ising spin glass systems has attracted much attention in statistical physics. Analytical as well as numerical attempts to solve large Ising spin glass systems have proved to be quite dicult. An explanation for those diculties was provided by Barahona [1] who proved for some of the most actively studied Ising systems that nding their ground states is NP-hard. In this paper, we describe a branch-and-bound algorithm for the MaxCut problem. By means of a well known reduction between MaxCut and the problem of nding ground states in Ising spin glass systems, we use it to solve a class of small Ising systems. In particular, we enumerate all ground states of 5 5 5 systems. Although this size is smaller than the systems usually studied by means of Monte Carlo simulations, the approach described here has the advantage not only of guaranteeing that the result is indeed a ground state but even of nding all ground states of the system. Similar approaches have been reported by Barahona [2] who studies two dimensional systems with a somewhat dierent edge distribution and Klotz and Kobe [7], [6] who study the properties of 4 4 4 systems. 1
2 The MaxCut Algorithm Branch and bound is a more sophisticated version of exhaustive tree search. With the help of a known feasible solution (lower bound LB) and an upper bound UB on the solution sizes in a subtree, the search of entire subtrees can be omitted if UB < LB. The following is a generic version of the branch and bound algorithm used in this paper. Branch&Bound(cut; i): IF i = 0 AND cutsize(cut) LB THEN output(cut) LB=cutsize(cut) IF i > 0 AND upperbound(cut; i) > LB THEN Branch&Bound(cut [ fig; i? 1) Branch&Bound(cut; i? 1) For simplicity of notation, it is assumed that V = f1; : : : ; ng. The initial call to the algorithm is Branch&Bound(fng; n? 1). If the comparison of the upper and lower bounds is omitted, the algorithm performs an exhaustive search. It remains to specify how the bounds are computed. Any reasonable method which provides a suboptimal solution (e.g. any heuristic) can be used to obtain the (hopefully large) initial lower bound. The method of computing the upper bound is more critical as a dierent { hopefully tight { upper bound is needed in each recursive call. This section describes an upper bound for MaxCut based on a positive semidenite (PSD) relaxation. Given an undirected weighted graph (G; w), where G = (V; E) and w = (w ij ) i;j2v is the matrix of edge weights, dene the matrix A as A ij =( wij if i 6= j v i if i = j where v i 2 IR + 0 (i 2 V ) are non-negative real variables. Thus, except for the diagonal, A is the weight matrix w of G. As G is undirected (w ij = w ji ), A is symmetric. It is easy to see that the (v i ) i2v can always be chosen such that A is positive semidenite. If A is positive semidenite, there exists a lower triangular n n matrix M such that MM T = A. Let m i be the i-th row of M. Then A ij = hm i ; m j i. With this, MaxCut can be formulated and relaxed as follows: XX * max w ij = max X X + m i ; m j SV SV i2s i2s j2s min A is P SD j2 S Pi2V m i ; 2 Pi2V m i 2 = 1 2 X i<j w ij + 1 4 min X A is P SD i2v kv i k 2 For the relaxation step note that for any y 2 IR n the function f y : IR n! IR; f y (x) = hx; y? xi is maximized at x = y=2. Thus, the upper bound on MaxCut can be found by solving the positive semidenite programming problem min X i2v kv i k 2 A is P SD (1) As strong duality can be shown to hold between (1) and the relaxation used by Goemans and Williamson in [5], this upper bound is guaranteed to be within a factor of 1=0:878 of the maximum cut if all w ij are non-negative. 2
Branch&Bound(cut; i; d): IF i = 0 AND cutsize(cut) LB THEN LB= cutsize(cut) output(cut) IF i > 0 THEN m i?1 := m i?1 + m i IF M 2 =4 < d THEN Branch&Bound(cut [ fig; i? 1; d? M 2 =4) i?1 i?1 i?1 i?1 m i?1 := m i?1? m i m i?1 := m i?1? m i IF M 2 i?1 i?1 =4 < d THEN Branch&Bound(cut; i? 1; d? M 2 i?1 i?1 =4) m i?1 := m i?1 + m i Figure 1: The branch and bound algorithm 2.1 Implementation Issues The upper bound derived so far applies in itself only to the rst call to Branch&Bound. Each recursive call of Branch&Bound corresponds to a set of constraints on the cut. The two kinds of constraints are that two vertices i, j have to be on the same side of the cut (denoted by i j) or on dierent sides of the cut (denoted by i 6 j). The upper bound for each recursive call should be tight enough to reect these constraints. This means that in any recursive call, the upper bound should not be much larger than the largest solution permitted by the corresponding set of constraints. We use a method of eciently deriving the upper bound in each recursive call from previously computed upper bounds, similar to the one used in [9]. The resulting algorithm is shown in Figure 1. The details are omitted due to space limitations. Although the PSD upper bound is fairly tight and eliminates a very large fraction of the search space, it still leads to unreasonably long running times for 5 5 5 Ising graphs. However, it is possible to decrease the running time by several orders of magnitude by introducing an additional local bound. Let l (left) and r (right) be the two sets into which a given cut partitions the vertex set V. In any recursive call of Branch&Bound, each vertex can be classied as belonging to either l (left), r (right), or u (undecided). A call to Branch&Bound(S; i; d) can make two recursive calls: one assigning vertex i to l and one assigning i to r. Given i, let w l = P j2l w ij, w r = P j2r w ij, and w u = P j2u jw ijj. Clearly, a cut with i 2 l cannot be optimal if w l > w r + w u. Similarly, the possibility i 2 u can be ignored if w r > w l + w u. It is only slightly more complicated to extend these statements to the case of equality, i.e. w l = w r + w u. If w u > 0, there is only one possibility how i 2 l could lead to an optimal cut: all j 2 u with w ij > 0 have to be in r and all j 2 u with w ij < 0 have to be in l. However, under this condition, the side of i is irrelevant as it does not change the cut size. Therefore, the recursive call for i 2 l does not have to be made as any optimal solution it might nd can be obtained from an optimal solution in which i 2 r, by changing the side of i. By symmetry, the same argument holds for i 2 r given w r = w l + w u and w u > 0. If w l = w r + w u and w u = 0, i.e. w l = w r, the side of i is irrelevant and making either one of the two recursive calls is sucient. 3
Summarizing, whenever a vertex is to be assigned to a side of the cut, we only explore this possibility if w l < w r + w u or w l = w r for assignments to l w r < w l + w u for assignments to r hold for the vertex itself and for all its neighbors. Any optimal cuts which are lost, are later recovered by changing the sides of single vertices in the optimal cuts. The running time of the algorithm depends heavily on the input graph. For the 5 5 5 inputs, we have observed running times between one and 24 hours on an SGI Indy workstation. We were able to solve within three weeks the 300 Ising systems on which the following results are based by using the idle CPU cycles on a network of ten such workstations. 3 Results Our goal was to examine the ground state structure of three dimensional 555 EA Ising systems with periodic boundary conditions. In order to facilitate comparisons with previous work (e.g. [4], [7]), we chose the bond distribution as follows. The bonds had values 1. The systems were generated uniformly at random such that the number of +1 and?1 bonds diers by at most one. Normally these numbers are chosen to be equal but as the number of bonds in our systems is odd, they have to dier by one. These systems correspond to graphs as described above. We began by generating 300 systems from this distribution and determining all their ground states using our algorithm and the well known reduction between MaxCut and the problem of nding ground states (e.g. [3]). We have obtained the following average ground state energy per spin: E 0 =n =?1:759; where E0 is the ground state energy of the system and n is the number of spins. Note, that we are averaging over exact quantities and that the only uncertainty comes from the fact that we are averaging over a small sample. This number appears to be consistent with those obtained in [7], [4]. Indeed, it appears that Monte Carlo methods can estimate the energy reasonably well, for systems sizes of up to 8 8 8. However, our exact method provides much more detailed information about the ground states. Extending the observations of Klotz and Kobe [7] to 5 5 5 systems, we note that the con- gurations of minimum energy can be partitioned into clusters. A cluster is dened as follows: Consider a graph whose vertices are the minimum energy congurations. Let there be an edge between two congurations if they dier in only one spin. Then the clusters are the connected components of this graph. The existence of large clusters is an artifact of the discrete 1 bond distribution. Therefore, we treat each cluster as one ground state. Figure 2 plots the number of minimum energy congurations of each system (degeneracy) vs. the ground state energy. The wide variation in the degeneracy among dierent systems which had already been observed in [7] for the 4 4 4 case has become even more pronounced. It is not totally clear how justied it is to identify clusters with ground states as we are trying to extrapolate the results from our small nite systems to the thermodynamic limit where the ground states (if more than one exists) would be separated by innitely high `energy walls' [8]. While we did not try to measure the height of the energy walls between the clusters we found, we 4
100000 ground state degeneracy 10000 1000 100 10 1-240 -235-230 -225-220 -215-210 -205-200 ground state energy Figure 2: The ground state energies and degeneracies (log scale) of the 100 random systems we solved. did investigate the robustness of the clusters by displaying measures for the coherence within each cluster and for the distance between dierent clusters. Consider the correlation C(; ) = h; i=n (2) between two congurations and. We have measured the average correlation between congurations of the same cluster and between congurations of dierent clusters A and B (C AB ). The results are displayed in Figure 3 (left). Assuming that the clusters can indeed be interpreted as ground states, these quantities are related to the order parameter of Parisi [8] as P 2A P2B C(; ) 1 C AB nx 0 = = i X 1 i jaj jbj jaj jbj A =n = nx m A i mb i =n i=1 @ X 2A where m A i is the average magnetization of spin i in cluster A. The last expression is the same as expression (7) in [8] which is the denition of the Parisi order parameter q. Figure 3 (left) shows the autocorrelation distribution highly concentrated close to 1 whereas the correlations between pairs of clusters are fairly atly distributed falling o toward zero between 0.8 and 1. This indicates that each cluster is fairly coherent and that dierent clusters are well separated in the sense that the average correlation between congurations of the same cluster is much higher than the average correlation between congurations which belong to dierent clusters. Figure 3 (right) displays the distribution of the number of clusters (ground states) per system. As the systems are symmetric by a global spin ip, the clusters appear in pairs of two. In the gure, these pairs are counted as one cluster. It has been observed by Klotz and Kobe [7] that even 4 4 4 systems have more than one pair of ground states. We observe that with the step from 4 4 4 to 5 5 5 systems, the average number of ground states increases. About two thirds 5 2B i=1
0.2 0.18 0.16 0.14 0.12 0.1 0.08 0.06 0.04 0.02 correlation autocorrelation 0-1 -0.8-0.6-0.4-0.2 0 0.2 0.4 0.6 0.8 1 ground state degeneracy 0.4 0.35 0.3 0.25 0.2 0.15 0.1 0.05 0 0 2 4 6 8 10 12 14 16 18 ground state energy Figure 3: Left: The distributions of the autocorrelations of clusters and of the correlations between pairs of clusters. Right: Distribution of the number of clusters per system. of the systems have more than one pair of ground states. The average number of pairs of clusters in a system is 4:1. While this number in itself could not possibly be judged as evidence that the number of ground states will become innite in the thermodynamic limit, it indicates that, at least in the the small systems studied here, there is usually not a unique ground state. References [1] F. Barahona. On the computational complexity of Ising spin glass models. Journal of Physics A, 15:3241{3255, 1982. [2] F. Barahona. Ground-state magnetization of Ising spin glasses. Physical Review B, 49(18):12864{12867, 1994. [3] F. Barahona, M. Grotschel, M. Junger, and G. Reinelt. An application of combinatorial optimization to statistical optimization and circuit layout design. Operations Research, 36(3):493{ 513, 1988. [4] B. Berg, U. Hansmann, and T. Celik. Groundstate properties of the 3d Ising spin glass. Los Alamos National Laboratory Preprint Server: cond-mat/9312029. [5] M. X. Goemans and D. P. Williamson. Improved approximation algorithms for maximum cut and satisfyability problems using semidenite programming. Submitted to J. ACM. Preliminary version in Proc. 26th ACM Symposium on the Theory of Computing. 1994. [6] T. Klotz and S. Kobe. Exact low-energy landscape and relaxation phenomena in ISING spin glasses. Los Alamos National Laboratory Preprint Server: cond-mat/9406023. [7] T. Klotz and S. Kobe. Valley structures in the spase space of a nite 3d Ising spin glass with i interactions. Journal of Physics A, 27:L95{L100, 1994. [8] G. Parisi. Order parameter for spin glasses. Physical Review Letters, 50(24):1946{1948, 1983. [9] S. Poljak and F. Rendl. Solving the max-cut problem using eigenvalues. Technical Report 91735-OR, Universitat Bonn, 1991. 6