Faculteit der Technische Wiskunde en Informatica Faculty of Technical Mathematics and Informatics

Potential reduction algorithms for structured combinatorial optimization problems Report 95-88 J.P. Warners T. Terlaky C. Roos B. Jansen Technische Universiteit Delft Delft University of Technology Faculteit der Technische Wiskunde en Informatica Faculty of Technical Mathematics and Informatics

J.P. Warners 1, T. Terlaky, C. Roos, B. Jansen 2 Department of Statistics, Probability and Operations Research Faculty of Technical Mathematics and Informatics Delft University of Technology P.O. Box 5031, 2600 GA Delft, The Netherlands. Tel.: (015)2787271, FA: (015)2787255 e{mail: j.p.warners@twi.tudelft.nl This research was partially supported by the EUCLID program, SEPA 6 (Articial Intelligence), RTP 6.4 (Combinatorial Algorithms for Military Applications). 1 This author was partially supported by the Dutch Organization for Scientic Research (NWO) under grant 612{33{001. 2 This author was partially supported by the Dutch Organization for Scientic Research (NWO) under grant 611{304{028. ISSN 0922-5641 Copyright c 1995 by the Faculty of Technical Mathematics and Informatics, Delft, The Netherlands. No part of this Journal may be reproduced in any form, by print, photoprint, microlm, or any other means without permission from the Faculty of Technical Mathematics and Informatics, Delft University of Technology, The Netherlands. Copies of these reports may be obtained from the bureau of the Faculty of Technical Mathematics and Informatics, Julianalaan 132, 2628 BL Delft, phone +31152784568. A selection of these reports is available in PostScript form at the Faculty's anonymous ftp-site. They are located in the directory /pub/publications/tech-reports at ftp.twi.tudelft.nl

Abstract Recently Karmarkar proposed a potential reduction algorithm for binary feasibility problems. In this paper we point out a practical drawback of his potential function and we propose a modied potential function that is computationally more attractive. As the main result of the paper, we will consider a special class of binary feasibility problems, and show how problems of this class can be reformulated as nonconvex quadratic optimization problems. The reformulation is very compact and a further interesting property is, that (instead of just one) multiple solutions may be found by optimizing it. We introduce a potential function to optimize the new model. Finally, we report on computational results on several instances of the graph coloring problem, comparing the three potential functions. Key words: interior point methods, potential reduction methods, binary programming, combinatorial optimization, graph coloring.

1 Introduction In 1984 Karmarkar showed that linear programming problems can be solved by an interior point approach in polynomial time [6]. An interior point method traverses the interior of the feasible region in search of an optimum of the linear program, rather than moving along the boundary of the feasible region as the simplex method does. Karmarkar uses a logarithmic potential function to measure the progress of the algorithm; solving a linear program is equivalent to sequentially minimizing this convex potential function under ane constraints. Karmarkar's work initiated an extensive research concerning the development of interior point methods for linear and, more general, convex programming (see, e.g., bibliography [9]). In practice, it has become clear that interior point methods compete favorably with the simplex method, especially for large scale problems. More recently, research has been done by Karmarkar et al. [7, 5, 8] to extend the potential reduction idea to solve dicult combinatorial optimization problems. Karmarkar [7] describes an approximate interior point algorithm to solve f0; 1g feasibility problems. He claims that ecient algorithms for many dicult combinatorial problems can be based on this approach. Results are reported of the application of the algorithm to two combinatorial problems; the satisability problem [5] (see also Shi et al. [10]) and the set covering problem [8]. The obtained results are encouraging. However, the potential function that Karmarkar et al. propose [5, 8] is not suitable to solve large scale combinatorial problems, since it involves solving linear systems with completely dense matrices. Therefore, when using this potential function there is no way to utilize sparsity properties of the specic combinatorial optimization problem under consideration. In this paper we propose two improvements on Karmarkar's original algorithm. First, we propose an alternative potential function that yields sparse linear systems. This potential function is valid for any optimization problem to which Karmarkar's original algorithm is applicable. Second, as the main result of the paper, we consider a special class of binary feasibility problems and reformulate problems of this class as nonconvex quadratic optimization problems. The quadratic reformulation is much more compact than the linear formulation, since all inequality constraints are incorporated in the objective function. A further property of the quadratic model is, that instead of just one, multiple feasible solutions may be found by solving the model. We introduce a potential function to optimize the quadratic model. We solve several instances of the graph coloring problem, using all three potential functions. It will become clear that the modied Karmarkar potential function is much more ecient than the original. Furthermore, the new quadratic formulation and its potential function will prove to be a major improvement on both other potential functions. This paper is organized as follows. Karmarkar's original method is briey summarized in Section 2. In Section 3 we will give the improved Karmarkar type potential function. We will consider the special class of combinatorial problems in Section 4, and introduce the potential function to optimize the reformulation in Section 5. In Section 6 we will report on computational results on the graph coloring problem. Finally, concluding remarks will be made in Section 7. 2 Karmarkar's algorithm for binary programming Karmarkar et al. [7, 5, 8] consider the following binary feasibility problem: where A 2 IR tm and b 2 IR t. (IP ) nd x 2 f0; 1g m such that Ax b; 1

2.1 The basic algorithm The interior point algorithm proposed by Karmarkar et al. generates a sequence of iterates in the interior of the feasible region. It consists of the following steps: Transform the f0; 1g feasibility problem to a f?1; 1g feasibility problem, using the substitution resulting in the problem x i := 2x i? 1; (IP ) nd x 2 f?1; 1g m such that ~ Ax ~ b. Relax the integrality constraints x 2 f?1; 1g m to linear constraints?1 x i 1; i = 1; : : :; m. Introduce a nonconvex potential function, whose minimizers are feasible solutions of (IP ). Starting from a feasible interior point, minimize the potential function in the following way: 1. Minimize a quadratic approximation of the potential function over an inscribed ellipsoid in the feasible region around the current feasible interior point, to obtain a descent direction for the potential function. 2. Use a line search to nd the minimum of the potential function along the descent direction (this is an extension of the algorithm proposed by Shi et al. [10]). Thus the new iterate is obtained. 3. Round the new iterate to a f?1; 1g integer solution. If this solution is feasible the problem is solved. If it is infeasible, the algorithm proceeds. 4. If the potential value of the new iterate is lower than the previous potential value, go to step 1; otherwise, a (non{integral) local minimum is found. Modify the potential function in some way to avoid running in this local minimum again, and restart the process. In the next subsections we explain the main elements of the algorithm, with the purpose to point out a practical drawback of the algorithm proposed in [5, 8]. 2.2 Karmarkar's potential function Let us consider (IP ). We relax the integrality constraints and incorporate them in the set of constraints, denoting this by Ax b, where A is an n m matrix with full rank m (note that n = t + 2m). Karmarkar et al. introduce the following quadratic optimization problem: (P ) max m i=1 s.t. Ax b: x 2 i = xt x As problem (P ) is a concave maximization problem, it is NP{complete. Further we have that x T x m, with equality if and only if x is integral, hence the optima of this quadratic problem automatically provide integer solutions of (IP ). To solve (P ) Karmarkar et al. [5, 8] propose the potential function p '(x) = log m? x T x? 1 log s i ; (1) n i=1 where s = b? Ax is the slack vector. n 2

2.3 Minimizing the potential function Instead of (P ) consider the nonconvex optimization problem: (P ' ) min '(x) s.t. Ax b: To solve (P ' ) the algorithm starts with a given initial interior point x 0, i.e. Ax 0 < b, and generates a sequence of points fx k g in the interior of the feasible region. Denote the k th iterate by x k. Let S = diag (s k 1; : : :; s k n) and f 0 = m? x kt x k. Then the gradient h ' and the Hessian H ' of ' in x k are: h ' = r'(x k ) =? 1 f 0 x k + 1 n AT S?1 e; (2) H ' = r 2 '(x k ) =? 1 I? 2 f 0 f0 2 x k x kt + 1 n AT S?2 A; (3) where e is an all{one vector of appropriate length. The quadratic approximation of ' around x k is given by Q(x) = 1 2 (x? xk ) T H ' (x? x k ) + h T ' (x? xk ) + '(x k ): (4) In general, minimizing (4) subject to Ax b is NP{complete. However, when the polytope Ax b is substituted by an inscribed ellipsoid, the so{called Dikin ellipsoid [2], we obtain a problem which can be solved in polynomial time (see Ye [13]). The Dikin ellipsoid around x k is given by where 0 < r < 1. E(r) = fx 2 IR m j (x? x k ) T A T S?2 A(x? x k ) r 2 g; Substituting the polytope by the appropriate Dikin ellipsoid and letting x x? x k we nd the following optimization problem: min 1 (P E ) 2 (x)t H ' (x) + h T ' (x) s.t. (x) T A T S?2 A(x) r 2 : This problem has been studied by, among others, Sorensen [11] and Flippo and Jansen [4]. The optimal solution x to (P E ) is a descent direction of Q(x) from x k. We formulate the optimality conditions of (P E ) (see [11, 8, 4]). The vector x is an optimal solution of (P E ) if and only if there exists a 0, such that: (H ' + A T S?2 A)x =?h ' (5) ((x ) T A T S?2 A(x )? r 2 ) = 0 (6) H ' + A T S?2 A is positive semidenite: (7) Without going into further detail, we observe that to nd a solution x that both satises the optimality conditions and lies on an appropriate ellipsoid, the linear system (5) needs to be solved at least once. If the matrix on the left hand side is sparse, solving this system can be done more eciently by sparse matrix techniques, than when the matrix is dense. The density of system (5) is determined by the Hessian of the potential function used and/or by the matrices A and A T A. We note that the Hessian H ' of Karmarkar's potential function is completely dense due to its second term. Therefore, solving large optimization problems requires unacceptable computational eort. This motivates the need for a potential function that yields a sparse Hessian. 3

3 An improved potential function Instead of (1) we propose to use the following potential function: (x) = m? x T x? where > 0 is some parameter. The gradient and Hessian of (8) are: n i=1 log s i ; (8) h =?2x + A T S?1 e; (9) H =?2I + A T S?2 A: (10) The density of H is determined by the density of A T A. So if A and A T A are sparse matrices, the left hand side of (5) will be sparse, and the linear system can eciently be solved [3]. Note that is built from two parts. The rst term, m? x T x, represents the objective function of (P ). The second term is the logarithmic barrier function. Karmarkar et al. [5, 8] use a xed weight = 1 n of the barrier term. An important dierence concerning (1) and (8) is, that the rst term of (1) approaches?1 when x k approaches a feasible f?1; 1g solution, whereas the rst term of (8) is equal to zero for any f?1; 1g solution. Therefore, to ascertain that the value of (8) approaches zero when the iterates x k approach an integer solution, the weight must subsequentially be decreased during the minimization process. To make this process more exible, we introduce the weighted logarithmic barrier function: n (x; w) = m? x T x? w i log s i ; (11) i=1 where w 2 IR n is a positive weight vector. For the sake of completeness we give expressions of the gradient and Hessian of, using the notation W = diag(w 1 ; : : :; w n ). h =?2x + A T S?1 w; (12) H =?2I + A T S?1 W S?1 A: (13) The potential function allows us to change weights of dierent constraints independently of each other. This may be helpful, for example, after nding a local minimum, to avoid running into the same local minimum again. 4 A special class of binary feasibility problems The idea behind the previous potential functions is to introduce an objective function that forces the variables to integer values. In this section we will consider a special class of binary feasibility problems, to which a large number of combinatorial problems belong, such as the graph coloring and the maximum independent set problem. It will be shown how problems of this class can be reformulated as nonconvex quadratic programming problems with known optimal value, by making use of the special property that for any feasible solution also the slacks are binary. Instead of explicitly forcing the variables to binary values, we do this implicitly by using an objective function that forces the slacks to binary values. This reformulation yields a reduction of the number of constraints which can be quite signicant, as all inequality constraints are incorporated in the objective function. We consider the binary feasibility problem: (BP ) nd x 2 f0; 1g m such that Ax e; Bx = d: Here A 2 IR nm ; B 2 IR pm ; e = (1; : : :; 1) T 2 IR n ; d 2 IR p. We make the following assumptions: 4

Assumption 1 All elements of A are binary. Assumption 2 All elements of B are binary. Assumption 3 Each column of B contains at most one nonzero element. Note that Assumption 2 implies that d is integral (provided (BP ) is feasible). Assumption 3 implies that each variable occurs in at most one equality constraint. So, the equality constraints are of the following form: x i = d k ; k = 1; : : :; p; where without loss of generality we may assume that [ p k=1 E k = f1; : : :; mg, and the sets E k ; k = 1; : : :; p are disjunct: E j \ E k = ;; 8 j; k = 1; : : :; p; j 6= k: If we are given a nonnegative integer matrix A which contains elements larger than one, the columns in which these elements occur can be removed from the problem, as the corresponding variable must be zero. Also, if a variable does not occur in an equality constraint, we may remove it from the problem. We introduce the (symmetric) matrix Q: Q = sgn A T A? diag(a T A) ; (14) where diag(a T A) denotes the diagonal matrix containing the diagonal entries of the matrix A T A and the function sgn is 1, 0 or {1 if the corresponding entry is positive, zero or negative respectively. Due to Assumption 1, the matrix Q is binary and it has the same nonzero structure as A T A, except for the diagonal: all diagonal entries are set to zero. This implies that Q is indenite. Now let us consider the following optimization problem: (QP ) min s.t. x T Qx Bx = d x i 2 f0; 1g; i = 1; : : :; m: By replacing the constraints x i 2 f0; 1g by 0 x i 1 we obtain the relaxation (QR) of (QP ). Since Q is indenite, the programs (QP ) and (QR) are nonconvex. Furthermore, the construction of Q implies the following result. Proposition 1 x T Qx 0 for any feasible solution x of (QP ) and (QR). Now we are ready to prove the following important relation between (BP ) and (QP ). Lemma 1 Under Assumption 1 it holds that the vector x is a feasible solution of (BP ) if and only if x is an optimal solution of (QP ) with x T Qx = 0. Proof: We observe that a (binary) solution x of (BP ) is feasible if and only if the slack vector s = e? Ax is (also) binary and the equality constraints are satised. So we have that x is a feasible solution if and only if Bx = d and 0 =?s T (e? s) = (Ax? e) T Ax = x T A T Ax? e T Ax: (15) 5

Since A is binary, for every column a j ; j = 1; : : :; m, of A we have e T a j = a T j a j: Note that the right hand side of this expression is the j th diagonal element of the matrix A T A. Using that x is binary, we nd that e T Ax = e T diag(a T A)x = x T diag(a T A)x: Substituting this in (15) we have that x is a feasible solution of (BP ) if and only if Bx = d and 0 = x T A T Ax? e T Ax = x T? A T A? diag(a T A) x = x T sgn A T A? diag(a T A) x = x T Qx; where we use that A T A and Q are nonnegative matrices. 2 The next lemma gives a relation between (QP ) and (QR). Lemma 2 Consider (QP ) and its relaxation (QR). The following statements hold: 1. Assume the optimal value of (QP ) is zero. If x is an optimal solution of (QP ) then x is also an optimal solution of (QR). 2. Assume that Assumptions 2 and 3 hold. If x is an optimal solution of (QR) with x T Qx = 0, then x is either a (binary) solution of (QP ), or we can trivially construct multiple optimal solutions of (QP ) from x. Proof: The rst statement is trivial since the objectives of both problems are nonnegative (Proposition 1), hence a feasible solution with zero value is optimal. Now we consider the second statement. Let x be an optimal solution of (QR) such that x T Qx = 0. If x is binary, x is a solution of (QP ) and the lemma is proven. Suppose x is not binary. Since we have 0 = x T Qx = 2 m m n a li a lj x i x j ; i=1 j=i+1 l=1 a li a lj x i x j = 0; 8 l = 1; : : :; n; 8 i; j = 1; : : :; m; i 6= j: We conclude that if a li ; a lj > 0 then x i x j = 0. So if we let x = sgn(x), i.e. x is a binary vector, then we still have x T Qx = 0: For x we may have that Bx d. However, as B contains only binary values (Assumption 2) and each variable occurs in at most one equality constraint (Assumption 3), we can set variables x i from one to zero until the left hand sides of the equality constraints are decreased to the desired d value, thus constructing a feasible solution ~x of (QP ). Thus, it is possible to construct multiple solutions, since any binary vector ~x that satises ~x x and B~x = d is a feasible solution of (QP ): 2 Combining Lemmas 1 and 2 we can state the following theorem. Theorem 1 If the Assumptions 1, 2 and 3 hold, then we have 1. If x is a feasible solution of (BP ), then x is an optimal solution of (QR). 2. If x is an optimal solution of (QR) with x T Qx = 0, then x is either a (binary) solution of (BP ), or we can trivially construct multiple solutions of (BP ) from x. 6

Observe that, given a (partly) fractional solution x of (QR) with x T Qx = 0, the number of solutions of (BP ) that we can construct from x can explicitly be computed. Let f k be the number of positive variables in the set E k ; k = 1; : : :; p. Then the number of solutions is given by! py f k : (16) d k k=1 In the case that (BP ) is infeasible, the global minima of (QP ) and (QR) will not be equal to zero. Also, we are not certain that a global minimum of (QR) corresponds to one or more binary vectors. However, if the following assumption is satised, all minima (local and global) of (QR) yield one ore more binary solutions. Assumption 4 For each equality constraint k 2 f1; : : :; pg at least one of the following statements holds: 1. d k = 1. 2. Two variables that occur simultaneously in equality constraint k do not occur simultaneously in any inequality constraint. We can state the second part of Assumption 4 more formally. Given k 2 f1; : : :; pg with d k > 1, then for all i; j 2 E k ; i 6= j, we have that a li a lj = 0, for all l = 1; : : :; n. Theorem 2 Let the Assumptions 1,2,3 and 4 hold. Given a feasible non{integral solution x of (QR), we can construct a feasible integral solution ~x of (QP ), such that ~x T Q~x x T Qx. Proof: Without loss of generality we assume that x is a binary solution, except for at least one of the variables x i ; i 2 E k ; for a given k 2 f1; : : :; pg. (Note that this implies that at least two variables are fractional, since the vector d is integral.) First we rewrite the product x T Qx to a form that is more convenient for our purpose, using the symmetry of Q: x T Qx = q ij x i x j + 2 q ij x i x j + q ij x i x j : (17) j2e k j =2E k j =2E k Denote the rst term of (17) by K 1 0, the third by K 2 0, and let the cost coecients be given by c i = 2 q ij x j 0, then we can rewrite (17) as: j =2E k x T Qx = K 1 + i =2E k c i x i + K 2 : (18) Note that the value of K 1 depends on the values of x i ; i 2 E k, but that K 2 and c i are independent of them. Now let us assume that the rst part of Assumption 4 holds. Let and set i = arg min c i ; ~x i := 1; ~x i := 0; 8 i 2 E k nfi g; ~x i := x i ; 8 i =2 E k : Using (18), K 1 0, x i 0; 8 i, and d k = 1, it holds x T Qx? K 2 c i x i min c i x i = 7 min c i d k = c i : (19)

We observe that c i ~x i = c i and q ij ~x i ~x j = q i i = 0. Substituting this in (19) and using (18), j2e k we nd that x T Qx? K 2 c i = c i ~x i = ~x T Q~x? K 2 : Thus the variables ~x i ; i 2 E k, are set to binary values, without increasing the objective value. Under the second part of Assumption 4 the analysis becomes a little more complex. In this case, we have that K 1 = 0, since n q ij = a li a lj = 0; 8 i; j 2 E k ; i 6= j: l=1 Now we can follow a procedure similar to the one described above. We pick the d k lowest cost coecients c i and set the corresponding variables ~x i to one, while setting the rest to zero. Let I k = fi 1 ; : : :; i dk g be the set of indices corresponding to the d k lowest cost coecients and let Now let Using (18) and K 1 = 0, it follows x T Qx? K 2 = # = max i2i k c i c j ; 8j 2 E k ni k : ~x i := 1; 8 i 2 I k ; ~x i := 0; 8 i 2 E k ni k ; ~x i := x i ; 8 i =2 E k : c i x i = i2i k c i x i + ni k c i x i = i2i k c i ~x i + i2i k c i (x i? ~x i ) + From (i) x i? ~x i 0; 8 i 2 I k ; (ii) x i 0; 8 i 2 E k ; and (iii) the denition of #, we nd that i2i k c i (x i? ~x i ) # i2i k (x i? ~x i ) and Furthermore, by the construction of ~x it is obvious that c i ~x i = i2i k Substituting (21) and (22) in (20) we obtain x T Qx? K 2 c i ~x i + # 2 4 i2ik ni k c i x i # c i ~x i and i2i k ~x i = (x i? ~x i ) + ni k x i 3 5 = ni k c i x i : (20) ni k x i : (21) x i = d k : (22) c i ~x i + #(d k? d k ) = c i ~x i = ~x T Q~x? K 2 ; Again, we have set the variables ~x i ; i 2 E k ; to binary values, without increasing the objective value. So given an arbitrary fractional solution x, we can repeat this procedure for each k = 1; : : :; p, until there are no fractional variables left, thereby constructing a binary solution ~x with ~x T Q~x x T Qx: 2 Due to Theorem 2, the optimal values of (QP ) and (QR) are equal, also when (BP ) is infeasible. Furthermore, all minima (local and global) of (QR) have integral values, and yield one ore more binary solutions. Using formula (16), we can again explicitly compute the number of binary solutions with the same objective value that can be constructed from a (partly) fractional solution which yields a minimum of (QR). From Theorem 2 we can give the following interpretation to the (integral) objective value of a local minimizer x of (QR). Let x T Qx = 2. Any solution ~x of (QP ) that we can construct from x has the same objective value 2. For such a solution ~x there are pairs i; j for which there exists an l 2 f1; : : :; ng such that a li = a lj = x i = x j = 1. This implies that the number of constraint violations of the solution ~x in (BP ) is. Finally, we observe that Theorem 2 holds for any nonnegative matrix ~ Q with the same nonzero structure as Q. This allows us to weigh the constraints; if the constraint concerning the variables i 1 and j 1 is considered to be more important than the constraint concerning a pair of variables i 2 and j 2 we can set ~q i1j 1 to a large value L >> 1, while setting ~q i2j 2 to one. 8

5 A potential function to solve the new model To solve problem (QR) we can use the weighted logarithmic barrier function. 2m (x; w) = 1 2 xt Qx? w i log s i ; (23) i=1 where w 2 IR 2m is a positive weight vector, and the values s i are the slacks of the constraints 0 x e. The gradient and Hessian of are: h = Qx? (I?I) S?1 w; (24) H = Q? (I?I) S?1 W S?1 (I?I) T : (25) The nonzero structures (and so the densities) of the Hessian of and the Hessian of (11) are identical. This can be seen immediately, as the nonzero structure of the rst is determined by Q, and that of the second by A T A. Note that the equality constraints Bx = d do not occur in (23). As in other potential reduction methods, there are several ways to deal with the equality constraints: The equality constraints may be replaced by inequality constraints: Bx d, and subsequently be added to the polytope 0 x e. A projection onto the null space of B may be used. 6 Computational results The three dierent potential functions and corresponding models have been used to solve a number of instances of the Graph Coloring Problem (GCP). The feasibility version of the GCP can be formulated as follows: Given an undirected graph G = (V; E), with V the set of vertices and E the set of edges, and a set of colors C, nd a coloring of the vertices of the graph such that any two connected vertices have dierent colors. We can model the GCP as follows. Dened are the binary decision variables: x vc = ( 1 color c is assigned to vertex v, 0 otherwise, 8v 2 V; 8c 2 C: We construct a set of linear constraints modeling the GCP and show that it satises the assumptions made in Section 4. First, we have to assign exactly one color to each vertex: c2c Second, two connected vertices may not get the same color: x vc = 1; 8v 2 V: (26) x uc + x vc 1; 8(u; v) 2 E; 8c 2 C: (27) Now we can write the GCP in the form (BP ), where A and B are binary matrices, given by (27) and (26) respectively. Each variable occurs in exactly one equality constraint and two variables occurring in the same 9

equality constraint do not occur simultaneously in an inequality constraint. So Theorems 1 and 2 apply. The density of the matrix Q can be computed with the following formula: density(q) = 2 jejjcj (jv jjcj) 2 : This shows that Q (hence A T A) is sparse, even if G is a dense graph. Therefore, computation times will reduce considerably when using a potential function that benets from this sparsity. Example We illustrate the construction of the matrices A; B and Q with a small example. Let G be the graph shown in Figure 1. The number of variables required to model this GCP is m = jv jjcj. The incidence matrix M 1 2 3 4 5 Figure 1: Example graph and the matrices A and B are given by: M = 0 B @ 0 1 0 0 1 1 0 1 1 1 0 1 0 1 1 0 1 1 0 0 1 1 1 0 0 1 C A ; A = 0 B @ I I 0 0 0 I 0 0 0 I 0 I I 0 0 0 I 0 I 0 0 I 0 0 I 0 0 I I 0 0 0 I 0 I 1 C A ; B = 0 B @ e T 0 0 0 0 0 e T 0 0 0 0 0 e T 0 0 0 0 0 e T 0 0 0 0 0 e T 1 C A ; where I denotes the jcj jcj identity matrix and e denotes the all{one jcj{vector. The number of rows of the matrix A is n = jejjcj and the number of nonzeros is 2n. We can now readily compute the matrix Q: Q = 0 B @ 0 I 0 0 I I 0 I I I 0 I 0 I I 0 I I 0 0 I I I 0 0 Note the similarity of the incidence matrix M and Q. The element (uc 1; vc 2) of the matrix Q is equal to one if (and only if) [u; v] 2 E (so m uv = 1) and c 1 = c 2: 2 In the following we shall discuss a number of implementational issues. 1 C A : Starting points We deal with the equality constraints by relaxing them to inequality constraints. Therefore our initial point needs to satisfy x vc > 1; 8v 2 V: c2c 10

We could simply take x 0 vc = 1 jcj? 1 ; 8v 2 V; 8c 2 C as our starting point. When using potential function (1), experimentation shows that this starting point is close to a maximum of the potential function as the gradient is close to zero. Therefore, the initial steps that are made are very short. However, if we slightly perturb the starting point no such problem is encountered. We perturb the starting point (in f0; 1g formulation) by multiplying each component by a factor that is chosen from a normal distribution with mean = 1 and variance 2 = jcj?1 1. In the results reported in this section, the same perturbed starting point was used for all three potential functions. Weights The weights of the barrier functions are chosen as follows. When using potential function ' the (constant) weight of the barrier is set to 1 10n. Karmarkar et al. [5, 8] propose to use 1 n, but this resulted in slow progress for this particular problem class. The weights w i of the potential functions and are initially set to 1 and 100, for all i, respectively. The weights are decreased by a factor two in each iteration. n n Rounding scheme We use the following rounding scheme. First, the largest fractional variable is found and this is set to one. All variables corresponding to both the same color and connected vertices are set to zero and subsequently again the largest fractional variable is determined. This rounding scheme terminates either when a feasible coloring has been found, or when a partial coloring has been found that cannot be extended without violating constraints. Given a fractional solution x: while any x vc is not rounded (v ; c ) := argmax fx vc j v 2 V; c 2 C; x vc fractionalg; x v c := 1; x v c := 0; 8c 2 Cnfc g; x vc := 0; 8v 2 V : [v; v ] 2 E; endwhile Stopping criteria As stopping criteria of the algorithm we take: m? x T x < or x T Qx < ; where = 10?3, or the algorithm stops when the potential value does not improve during two subsequent iterations. Local minima If the algorithm runs into a local minimum, the weights of the near{active constraints in the nal interior solution x K are increased. A constraint i is called near{active when s K i is close to zero. Subsequently, the process is restarted from a new starting point x 0 new = x K + (x 0? x K ) where > 1 is such that Ax 0 new < b. Implementation The algorithm was implemented in MATLAB TM and uses some FORTRAN routines provided by the linear programming interior point solver LIPSOL (Zhang [14]). These FORTRAN routines use sparse matrix techniques to do the minimum degree ordering, symbolic factorization, Cholesky factorization and back substitution to solve the linear system (5) provided this system is sparse. The tests were run on a HP9000/720 workstation, 144 Mb, 50 mhz. 11

Test problem generation The GCP test problems were generated using the test problem generator GRAP H (Van Benthem [1]). This generator was originally intended to generate frequency assignment problems, but it was adapted to generate GCP s. The GCP s it generates have known optimal value. In the computational tests, we set the number of available colors jcj equal to the optimal value of the instance of the GCP under consideration. Results Table 1 shows for each of the potential functions the time and number of iterations (i.e. runs through steps 1; : : :; 4 on page 2) required to generate the rst admissible coloring. Using potential function ' problems up to a size of 1350 variables were solved; for larger problems the memory requirements and solution times appeared to be too high using our implementation. ' (1) (11) (23) GjV j:jcj jej time iter. time iter. time iter. G50.7 161 68 14 23 17.9 1 G50.10 229 274 24 58 23 1.9 1 G50.12 237 407 22 46 13 2.7 1 G50.15 224 733 16 73 16 6.0 2 G50.18 288 1113 20 129 20 4.9 1 G100.5 251 318 25 14 7 7.9 5 G100.8 379 641 17 88 21 5.5 2 G100.12 484 2260 20 174 20 7.9 1 G100.16 449 332 27 12 1 G150.6 438 1083 22 87 20 41 12 G150.9 612 3224 22 204 24 12 2 G150.16 663 772 30 30 1 G200.5 504 274 48 39 8 G200.10 954 67 4 G200.14 952 3416 41 46 1 Table 1: Solution times and number of iterations for the three potential functions. ' ' means no solution was found; '*' means that the solution was found after initially running into a local minimum; ' ' means that the problem was not tried. To see which potential function is easiest to minimize, we let the algorithm run until a minimum was found. Table 2 shows for all three potential functions the time and number of iterations required to converge to a minimum. If the minimum found was global, the numbers are printed in bold face. The last column of the table shows the number of feasible solutions that could be constructed from the solution found by minimizing potential function. This number is always larger than one, and in some cases quite substantial. It gives an indication of the minimal number of feasible solutions of the given GCP. An examination of the tables leads us to the conclusion that potential function leads to signicant improvement when compared to '. Furthermore, the new quadratic model gives the best results, both with respect to solution time and required number of iterations. The average number of linear systems that had to be solved per iteration for the three potential functions was 1.48, 1.56 and 1.34 linear systems per iteration respectively. Also in this respect seems to be the most stable. 12

' (1) (11) (23) GjV j:jcj time iter. time iter. time iter. # sol. G50.7 91 21 27 21 19 22 84 G50.10 281 25 81 29 56 35 4 G50.12 555 33 80 23 50 23 192 G50.15 1094 30 132 30 74 24 1024 G50.18 1596 33 201 33 127 31 32 G100.5 426 34 43 22 30 19 73728 G100.8 898 27 159 36 60 21 8 G100.12 3150 32 282 32 153 23 1536 G100.16 527 39 293 32 16384 G150.6 1801 33 147 33 70 21 32 G150.9 4614 34 424 45 151 24 320 G150.16 1200 42 632 29 1536 G200.5 1670 23 384 65 103 22 786432 G200.10 895 48 372 27 32 G200.14 4186 99 903 26 49152 Table 2: Times and number of iterations required to converge to rst minimum (local or global). Also the number of solutions if larger than one is given. '*' : Second minimum; ' ' means that the problem was not tried. 7 Concluding remarks In this paper a number of potential reduction methods for binary feasibility problems have been investigated. It has been shown that the potential function that Karmarkar et al. propose [7, 8, 5] has a major practical drawback when solving large scale combinatorial optimization problems, due to its dense Hessian. An improved potential function has been proposed. As this potential function makes use of the sparsity of the problem, problems of larger sizes can be solved. A nonconvex quadratic formulation for a special class of binary feasibility problems has been developed, which results in a much smaller and computationally more attractive problem, as all inequality constraints are incorporated in the objective function. Furthermore, optimizing this model may result in nding multiple feasible solutions. All three potential functions (Karmarkar's potential function, the modied Karmarkar type potential function and the potential function to optimize the new model) have been applied to several randomly generated instances of the graph coloring problem. { It appears that the modied Karmarkar potential function yields much shorter computation times than the original potential function, especially for larger problems. The number of iterations required for those potential functions are comparable. Also the number of times the algorithm converges to a global minimum are almost the same. { Using the new quadratic model leads to the best results. In all cases it nds a solution in the least time, usually after only one or a few iterations. Furthermore, it nds a global minimum for all instances, and requires, at least for the larger problems, fewer iterations to converge to a global minimum. { The number of solutions that can be constructed from a minimum of the new quadratic model can be quite substantial. In some cases hundreds or thousands of solutions are found simultaneously, whereas using the other potential functions results in nding just one solution. 13

Other combinatorial problems can also be tackled using one of the above mentioned potential functions. Any binary feasibility problem can be solved by using Karmarkar's or the modied Karmarkar potential function, where the latter is preferable due to its sparse Hessian. Some combinatorial problems can be solved by using the new quadratic model. Examples of such problems are, apart from the GCP : { The maximum independent set problem. { Frequency assignment problems (see Warners [12]). The results described in this paper were obtained using a MATLAB TM /FORTRAN implementation. When using a more ecient low level implementation computation times can considerably be improved. References [1] H.P. van Benthem (1995), "GRAP H: Generating Radiolink frequency Assignment Problems Heuristically", Master's Thesis, Faculty of Technical Mathematics and Informatics, Delft University of Technology, Delft, The Netherlands. [2] I.I. Dikin (1967), "Iterative solution of problems of linear and quadratic programming", Doklady Akademiia Nauk SSSR 174, 747{748. Translated into English in Soviet Mathematics Doklady 8, 674{675. [3] I.S. Du, A.M. Erisman and J.K. Reid (1989), Direct methods for sparse matrices, Oxford University Press, New York. [4] O.E. Flippo and B. Jansen (1992), "Duality and sensitivity in nonconvex quadratic optimization over an ellipsoid", Technical Report 92-65, Faculty of Technical Mathematics and Informatics, Delft University of Technology, Delft, The Netherlands. To appear in European Journal of Operational Research. [5] A.P. Kamath, N.K. Karmarkar, K.G. Ramakrishnan and M.G.C. Resende (1990), "Computational experience with an interior point algorithm on the Satisability problem", Annals of Operations Research 25, 43{58. [6] N. Karmarkar (1984), "A new polynomial-time algorithm for linear programming", Combinatorica 4, 373{395. [7] N. Karmarkar (1990), "An interior{point approach to NP{complete problems part I", Contemporary Mathematics 114, 297{308. [8] N. Karmarkar, M.G.C. Resende and K.G. Ramakrishnan (1991), "An interior point algorithm to solve computationally dicult set covering problems", Mathematical Programming 52, 597{618. [9] E. Kranich (1991), "Interior point methods for mathematical programming: a bibliography", Discussion paper 171, Institute of Economy and Operations Research, Fern Universitat Hagen, P.O. Box 940, D{5800 Hagen 1, Germany. [10] C.-J. Shi, A. Vannelli and J. Vlach (1992), "An improvement on Karmarkar's algorithm for integer programming", COAL Bulletin, Mathematical Programming Society, vol. 21, 23{28. [11] D.C. Sorensen (1982), "Newton's method with a model trust region modication", SIAM Journal on Numerical Analysis 19: 409{426. [12] J.P. Warners (1995), "A potential reduction approach to the Radio Link Frequency Assignment Problem", Master's Thesis, Faculty of Technical Mathematics and Informatics, Delft University of Technology, Delft, The Netherlands. [13] Y. Ye (1992), "On ane scaling algorithms for nonconvex quadratic programming", Mathematical Programming 56, 285{300. [14] Y. Zhang (1994), "LIPSOL - a MATLAB TM toolkit for linear programming interior-point solvers", Department of Mathematics and Statistics, University of Maryland Baltimore Country, Baltimore, Maryland. FORTRAN routines written by E.G. Ng and B.W. Peyton (ORNL), J.W.H. Liu (Waterloo), Y. and D. Zhang (UMBC). 14