Eigenvalue techniques for convex objective, nonconvex optimization problems Daniel Bienstock, Columbia University, New York.

Transcription

1 Eigenvalue techniques for convex objective, nonconvex optimization problems Daniel Bienstock, Columbia University, New York November, 2009 Abstract Consider a minimization problem given by a nonlinear, convex objective function over a nonconvex feasible region. Traditional optimization approaches will frequently encounter a fundamental difficulty when dealing with such problems: even if we can efficiently optimize over the convex hull of the feasible region, the optimum will likely lie in the interior of a high dimensional face, far away from any feasible point. As a result (and in particular, because of the nonconvex objective) the lower bound provided by a convex relaxation will typically be extremely poor. Furthermore, we will tend to see very large branch-and-bound (or -cut) trees with little or no improvement over the lower bound. In this work we present theory and implementation for an approach that relies on three ingredients: (a) the S-lemma, a major tool in convex analysis (b) efficient projection of quadratics to lower dimensional hyperplanes, and (c) efficient computation of combinatorial bounds for the minimum distance from a given point to the feasible set, in the case of several signficant optimization problems. Altogether, our approach strongly improves lower bounds at a small computational cost, even in very large examples. 1

2 1 Introduction We consider problems with the general form Here, (F) : F := min F (x), (1) s.t. x P, (2) x K. (3) F (x) is a convex function; in this abstract, a convex quadratic, i.e. F (x) = x T Mx + v T x (with M 0 and v R n ). P R n is a convex set over which we can efficiently optimize F, K R n is a non-convex set with special structure. In the applications we consider, n can be quite large. We assume that a given convex relaxation of the set described by (2), (3) is under consideration. For example, when dealing with a particularly complex set K, this may be the only sensible relaxation that we are able to produce. Or, it may be the only computationally practicable relaxation we have, especially in the case of large n. Or, as it is often in the case of practice, the relaxation is one from which it is easy to quickly produce (perhaps, good) heuristic solutions to problem F. Regardless of the case, one common fundamental difficulty is likely to be encountered: because of the convexity of F, the optimum solution to a convex relaxation will frequently be attained in the interior of a high-dimentional face of the relaxation, and far from the set K. Thus, the lower bound proved by the relaxation will be weak (often, very weak) compared to F. What is more, if one were to rely on branch-and-cut (the major tool of modern mixed-integer programming) the proved lower bound may improve little if at all when n is large, even after massive amounts of branching and extremely long computational time. This stalling of the lower bounding procedure is commonly encountered in practice and constitutes a significant challenge, the primary subject of our study. We present a body of techniques that are designed to alleviate this difficulty. After obtaining the solution x to the given relaxation for problem F, our methods will use techniques of convex analysis, of eigenvalue optimization, and combinatorial estimations, in order to quickly obtain a valid lower on F which is strictly larger (often, significantly so) than F (x ). We will describe an important class of problems where our method, applied to a cheap but weak formulation, produces bounds comparable to or better than those produced by much more sophisticated formulations, and at a small fraction of the computational cost. To motivate our discussion, we introduce two significant examples. Cardinality constrained optimization problems. Here, for some integer 0 < K n, K = { x R n : x 0 K }, where the zero-norm v 0 of a vector v is used to denote the number of nonzero entries of v. A classical example of such a constraint arises in portfolio optimization (see e.g. [2]) but modern applications involving this constraint arise in statistics, machine learning [12], 2

3 and, especially, in engineering and biology [18]. Problems related to compressive sensing have an explicit cardinality constraint (see for material). Also see [7]. The simplest canonical example of problem F is as follows: s.t. F = min F (x), (4) x j = 1, x 0, (5) j x 0 K. (6) This example is significant because (a) one can show that this problem is strongly NP-hard, and (b) it does arise in practice, exactly as stated. Note that modulo a simple change in variables, the linear constraint in (5) is simply a stand-in for a general non-homogeneous linear equation with nonzero coefficients. In spite of its difficulty, { this example already incorporates the fundamental difficulty alluded to above: clearly, conv x R n + : } { j x j = 1, x 0 K = x R n + : } j x j = 1. In other words, from a convexity standpoint the cardinality constraint disappears. Moreover, if the quadratic in F is positive definite and dominates the linear term, then the minimizer of F over the unit simplex will be an interior point, i.e., a point with all coordinates positive (and in real-life examples, of similar orders of magnitude); whereas K n in practice. Multi-term disjunctive inequalites. Consider vectors vectors c i R n and reals β i (1 i m). We are interested in a non-convex set of the form m K i, where K i = {x R n : c it x β i or c it x β i + 1}, for 1 i m. (7) i=1 More generally, we are interested in (and our methodology applies to) cases where each K i is a disjunction among multiple polyhedral sets. The classical split cuts [6] provide an example of disjunctions based on inequalities as in (7), with integral c i and β i. However, disjunctive inequalities naturally arise in numerous settings as valid strong statements on a combinatorial set (see e.g. [3], [4]). Note that in general, given a polyhedral set P, the convex hull of the intersection of P and a set (7) will have exponentially many facets (even the case m = 1 is not trivial). Thus, most likely, one would work with a relaxation of P m i=1 K i. Moreover, in the typical application of disjunctions (or split cuts) the disjunctions are found sequentially as a result of a cutting procedure. For simplicity, consider the case m = 1 the very first disjunction is added because it is violated by the solution to an initial relaxation. But thinking about the resulting geometry in the case of a (strictly) convex F (x), it is clear that, quite possibly, argmin{f (x) : x conv(p K 1 )} will still violate the first disjunction. For m > 1 the likelihood of such a stall in the cutting procedure increases. To the extent that disjunctive sets are a general-purpose technique for formulating combinatorial constraints, the methods in this paper apply to a wide variety of optimization problems, and should prove effective when the objective is strictly convex. 1.1 Techniques Our methods embody two primary techniques: 3

4 (a) The S-lemma (see [19], also [1], [5], [14]). Let f, g : R n R be quadratic functions and suppose there exists x R N such that g( x) > 0. Then f(x) 0 whenever g(x) 0 if and only if there exists µ 0 such that (f µg)(x) 0 for all x. Remark: here, a quadratic may contain a linear as well as a constant term. The S-lemma can be used as an algorithmic framework for minimizing a quadratic subject to a quadratic constraint. Let p, q be quadratic functions and let α, β be reals. Then min{p(x) : q(x) β} α, iff µ 0 s.t. p(x) α µq(x) + µβ 0 x. (8) In other words, the minimization problem in (8) can be approached as a simultaneous search for two reals α and µ 0, with α largest possible such that the last inequality in (8) holds. The S-lemma is significant in that it provides a good characterization (i.e. polynomial-time) for a a usually non-convex optimization problem. See [13], [15], [16], [17], [20] and the references therein, in particular regarding the connection to the trust-region subproblem. (b) Consider a given nonconvex set K. We will assume, as a primitive, that (possibly after an appropriate change of coordinates), given a point ˆx R n, we can efficiently compute a strong (combinatorial) lower bound for the Euclidean distance between ˆx and the nearest point in P K. We will make this assumption more precise in Section 1.2, but we will show that this is indeed the case for the cardinality constrained case (see Section 1.4; in the full paper we will show how to do so for the multi-term disjunctive case). Roughly speaking, it is the structure of a set K of interest that makes the assumption possible. In the rest of this section we will denote by D(ˆx) our lower bound on the minimum distance from ˆx to P K. We can put together (a) and (b) into a simple template for proving lower bounds for F : S.1 Compute an optimal solution x to the given relaxation to problem F. S.2 Next we obtain the quantity D(x ). S.3 Finally, we apply the S-lemma as in (8), using F (x) for p(x), and (the exterior of) the ball centered at x with radius D(x ) for q(x) β. x* y 0 01 Figure 1: A simple case. For a simple application of this template, consider Figure 1. This shows an instance of problem (4)-(6), with n = 3 and K = 2 where all coordinates of x are positive. The figure also assumes that D(x ) is exact it equals the minimum distance from x to the feasible region. If we minimize F (x), subject to being on the exterior of this ball (in other words, if we apply the S-lemma using F (x) as the objective and the ball as the constraint) the optimum will be attained at y. Thus, F (y) is a valid lower bound on F ; we have F (y) = F (x ) + λ 1 R 2, where R is the radius of the ball and λ 1 is the minimum eigenvalue of the restriction of F (x) to the unit simplex. Note: λ 1 is lower bounded by the smallest eigenvalue of F (x). 4

5 Now consider the example in Figure 2, corresponding to the case of a single disjunctive inequality. Here, x F is the optimizer of F (x) over the affine hull of the set P. A straightforward application of the S-Lemma will yield as a lower bound (on F ) the value F (y), which is weak weaker, in fact, than F (x ). The problem is caused by the fact that x F is not in the relative interior of the convex hull of the feasible region. In summary, a direct use of our template will not work. 1.2 Adapting the template In order to correct the general form of the difficulty depicted by Figure 2 we would need to solve a problem of the form: { V := min F (x) : x x C, (x x ) T (x x ) δ 2} (9) where δ > 0, and C is the cone of feasible directions (for P) at x. We can view this as a cone constrained version of the problem addressed by the S-Lemma. Clearly, F (x ) V F with the first inequality in general strict. If we are dealing with polyhedral sets, (9) becomes (after some renaming): { min F (ω) : Cω 0, ω T ω δ 2} (10) where C is an appropriate matrix. However, we have (proof in full paper): Theorem 1.1 Problem (10) is strongly NP-hard. We stress that the NP-hardness result is not simply a consequence of the nonconvex constraint in (10) without the linear constraints, the problem becomes polynomially solvable (i.e., it is handled by the S-lemma, see the references). To bypass this negative result, we will adopt a different approach. We assume that there is a positive-definite quadratic function q(x) such that for any y R n, in polynomial time we can produce a (strong, combinatorial) lower bound Dmin 2 (y, q) on the quantity x* y x F min{q(y x) : x P K}. In Section 1.4 we will address how to produce the Figure 2: The simple template fails. quadratic q(x) and the value D 2 (y, q) when K is defined by a cardinality constraint (and a similar construct exists in the disjunctive inequalities case). Let c = F (x ) (other choices for c discussed in full paper). Note that for any x P K, c T (x x ) 0. For α 0, let p α = x + αc, and let H α be the hyperplane through p α orthogonal to c. Finally, define V (α) := min{f (x) : q(x p α ) D 2 (p α, q), x H α }, (11) and let y α attain the minimum. Note: computing V (α) entails an application of the S-lemma, restricted to H α. See Figure 3. Clearly, V (α) F. Then Suppose α = 0, i.e. p α = x. Then x is a minimizer of F (x) subject to x H 0. Thus V (0) > F (x ) when F is positive-definite. 5

6 Suppose α > 0. Since c T (y α x ) > 0, by convexity V (α) = F (y) > F (x ). Thus, F (x ) inf α 0 V (α) F ; the first inequality being strict in the positive-definite case. [It can be shown that the inf is a min ]. Each value V (α) incorporates combinatorial information (through the quantity D 2 (p α, q)) and thus the computation of min α 0 V (α) cannot be obtained through direct convex optimization techniques. As a counterpoint to Theorem 1.1, we have (proof in full paper): Theorem 1.2 In (10), if C has one row and q(x) = j x2 j then V inf α 0 V (α). In order to develop a computationally practicable approach that uses these observations, let 0 = α (0) < α (1) <... < α (J), such that for any x P K, c T x α (J) c 2 2. Then: Updated Template 1. For 0 i < J, compute a value Ṽ (i) min{ V (α) : α(i) α α (i+1) }. 2. Output min 0 i<j Ṽ (i). The idea here is that if (for all i) α (i+1) α (i) is small then V (α (i) ) V (α (i+1) ). Thus the quantity output in (2) will closely approximate min α 0 V (α). In our implementation, we compute Ṽ (i) by appropriately interpolating between V (α (i) ) and V (α (i+1) ) (details, full paper). Thus our approach reduces to computing quantities of the form V (α). We need a fast procedure for this task (since J may be large). Considering eq. (11) we see that this H α Bounding ellipsoid in H α c p α x* yα minimizer H α of F(x) in Figure 3: A better paradigm. involves an application of the S-lemma, restricted to the hyperplane H α. An efficient realization of this idea, which allows for additional leveraging of combinatorial information, is obtained by computing the projection of the quadratic F (x) to H α. This is the subject of the next section. 1.3 Projecting a quadratic Let M = QΛQ T be a n n matrix. Here the columns of Q are the eigenvectors of M and Λ = diag{λ 1,.{.., λ n } where the λ} i are the eigenvalues of M. We assume λ 1... λ n. Let c 0, denote H = x R n : c T x = 0, and let P be the projection matrix onto H. In this section we describe an efficient algorithm for computing an eigenvalue-eigenvector decomposition of the projected quadratic P MP. Note that if x H, x T P MP x = x T Mx. The vector c could be dense (is dense in important cases) and Q could also be dense. Our approach reverse engineers, and extends, results from [8] (also see Section 12.6 of [9] and references therein). Clearly, c is an eigenvector of P M P (corresponding to eigenvalue 0). The remaining eigenvalues λ 1,..., λ n 1 are known to satisfy λ 1 λ 1 λ 2 λ 2... λ n 1 λ n 1 λ n. Definition 1.3 An eigenvector q of M is called acute if q T c 0. An eigenvalue λ of M is called acute if at least one eigenvector corresponding to λ is acute. In (e.2) below we will use the convention 0/0 = 0. 6

7 Lemma 1.4 Let α 1 < α 2 <... < α q be the acute eigenvalues of M. Write d = Q T c. Then, for 1 i q 1, (e.1) The equation n d 2 j λ j λ = 0 has a unique solution ˆλ i in (α i, α i+1 ). (e.2) Let w i = Q(λ ˆλ i I) 1 d. Then c T w i = 0 and P MP w i = ˆλ i w i. Altogether, Lemma 1.4 produces q 1 eigenvalue/eigenvector pairs of P MP. The vector in (e.2) should not be explicitly computed; rather the factorized form in (e.2) will suffice. The root to the equation in (e.1) can be quickly obtained using numerical methods (such as golden section search) since the expression in (e.1) is monotonely increasing in (α i, α i+1 ). A different construction, which handles the non-acute eigenvectors and the eigenvalues of M with multiplicity greater than one, produces n q + 1 additional distinct eigenvalues/eigenvector pairs for P MP orthogonal to c which are also distinct from those obtained through Lemma 1.4 (details in full paper). To conclude this section, we note that it is straightforward to iterate the procedure in this section, so as to project a quadratic to hyperplanes of dimension less than n 1. More details will be provided in the talk and in the full paper. 1.4 Combinatorial bounds on distance functions Here we take up the problem of computing strong lower bounds on the Euclidean distance from a point to the set P K. In this abstract we will focus on the cardinality constrained problem, but results of a similar flavor hold for the disjunctive inequalities case. Let a R n, b R, K < n be a positive integer, and ω R n. Consider the problem n Dmin(ω, 2 a) := min (x j ω j ) 2, : a T x = b and x 0 K. (12) Clearly, the sum of smallest n K values ωj 2 constitutes a ( naive ) lower bound for problem (12). But it is straightforward to show that an exact solution to (12) is obtained by choosing S {1,..., n} with S K, so as to minimize (b j S a jω j ) 2 j S a2 j + j / S ω 2 j. (13) [We use the convention that 0/0 = 0.] Empirically, the naive bound mentioned above is very weak since the first term in (13) is typically at least an order of magnitude larger than the second; and it is the bound, rather than the set S itself, that matters. Suppose a j = 1 for all j. It can be shown, using (13), that the optimal set S has the following structure: S = P N, where P + N K, and P consists of the indices of the P smallest nonnegative ω j (resp., N consists of the indices of the N smallest ω j with ω j < 0). The optimal S can be computed in O(K) time, after sorting the ω j. When ω 0 or ω 0 we recover the naive procedure mentioned above (though again we stress that the first term in (13) dominates). In general, however, we have: Theorem 1.5 (a) It is NP-hard to compute Dmin 2 (ω, a). (b) Let 0 < ɛ < 1. We can compute a vector ˆx with j a j ˆx j = b and ˆx 0 K, and such that n (ˆx j ω j ) 2 (1 + ɛ)dmin(ω, 2 a), 7

8 in time polynomial in n, ɛ 1, and the number of bits needed to represent ω and a. In our current implementation we have not used the algorithm in part (b) of the Lemma, though we certainly plan to evaluate this option. Instead, we proceed as follows. Assume a j 0 for all j. Rather than solving problem (12), instead we consider n min a 2 j(x j ω j ) 2 : a T x = b and x 0 K. { n Writing ω j = a j ω j (for all j), this becomes min (x j ω j ) 2 : which as noted above can be efficiently solved. 1.5 Application of the S-Lemma } j x j = b and x 0 K, Let M = QΛQ T 0 be a matrix given by its eigenvector factorization. Let H be a hyperplane through the origin, ˆx H, v R n, δ j > 0 for 1 j n, β > 0, and v R n. Here we solve the problem min x T Mx + v T x, subject to n δ i (x i ˆx i ) 2 β, and x H. (14) i=1 By rescaling, translating, and appropriately changing notation, the problem becomes: min x T Mx + v T x, subject to n x 2 i β, and x H. (15) i=1 Let P be the n n matrix corresponding to projection onto H. Using Section 1.3 we can produce a representation of P MP as Q Λ Q T, where the the n th eigenvector q n is orthogonal to H, and λ 1 = min i<n { λ i }. Thus, problem (15) becomes, for appropriately defined ṽ, Γ := min n 1 λ j y 2 j + 2ṽ T y, subject to n 1 y 2 j β. (16) Using the S-lemma, we have that Γ γ, iff there exists µ 0 s.t. n 1 n 1 λ j yj 2 + 2ṽ T y γ µ yj 2 β 0 y R n 1. (17) Using some linear algebra, this is equivalent to { Γ = max µβ n 1 i=1 ṽ 2 i λ i µ : 0 µ < λ 1 This is a simple numerical task, since in [0, λ 1 ) the objective in (18) is concave in µ. }. (18) Remarks: (1) Our updated template in Section 1.2 requires the solution of multiple problems of the form 18 (for different β and ṽ) but just one computation of Q and Λ. (2) Consider any integer 1 p < n 1. When µ < λ 1, the expression maximized in (18) is lower bounded by µβ n 1 p ṽi 2 i=1 λ i=p+1 ṽ2 i i µ λ p+1 µ. This, and related facts, yield an approximate version of our approach which only asks for the first p elements of the eigenspace of P MP (and M). 8

9 1.5.1 Capturing the second eigenvalue We see that Γ < λ 1 β (and frequently this bound is close). In experiments, the solution y to (15) often cheats in that y1 is close to zero. We can then improve on the bound if the second projected eigenvalue, λ 2, is significantly larger than λ 1. Assuming that is the case, pick a value θ with y1 2 /β < θ < 1. (a) If we assert that y 2 1 θβ then we can strengthen the constraint in (14) to n i=1 δ i (x i ˆx i ) 2 γ, where γ = γ(θ) > β. This is certainly the case for the cardinality constraint and for the disjunctive inequalities case (details, full paper). So the assertion amounts to applying the S-lemma, but using γ in place of β. (b) Otherwise, we have that n 1 i=2 y2 i (1 θ)β. In this case, instead of the right-hand side of (18), we will have max { µ(1 θ)β n 1 i=2 ṽ 2 i λ i µ : 0 µ λ 2 }. (19) The minimum of the quantities obtained in (a) and (b) yields a valid lower bound on Γ; we can evaluate several candidates for θ and choose the strongest bound. When λ 2 is significantly larger than λ 1 we often obtain an improvement over the basic approach as in Section 1.5. Note: the approach in this section constitutes a form of branching and in our testing has proved very useful when λ 2 > λ 1. It is, intrinsically, a combinatorial approach, and quite distinct from the S-lemma and related problems (e.g. the trust region subproblem). It is thus not easily reproducible using convexity arguments alone. Also see remark (2) of the previous section. 2 Computational experiments For the sake of brevity, we describe a partial set of experiments (more in the talk and in the full paper). The purpose of our experiments is to (a) investigate the speed of our numerical algebra routines, in particular the projection of quadratics, for large n and large number of nonzeroes in the quadratic, and (b) to study the strength of the bound our method produces. We study problems of the form min{ x T Mx + v T x : j x j = 1, x 0, x 0 K }. The matrix M is given in its eigenvector/eigenvalue factorization QΛQ T. To stress-test our linear algebar routines, we construct Q as the product of random rotations: as the number of rotations increases, so does the number of nonzeroes in Q, and the overall complexity of M. In our experiments, we ran our procedure after computing the solution to the (diagonalized) weak formulation min{ y T Λy + v T x : Q T x = y, x j = 1, x 0}. j We compare our bounds to those obtained by running the (again, diagonalized) perspective formulation [10], [11], a strong conic formulation (here, λ min is the minumum λ i ): 9

10 min λ min j w j s.t. Q T x = y, + (λ j λ min )yj 2 j x j = 1 j x 2 j w j z j 0, 0 z j 1 j, (20) z j k, x j z j j, j x, w R n +. For our experiments, we used Cplex 12.1 on a single-core 2.66GHz Xeon machine with 16 Gb of physical memory, which was never exceeded, even in the largest examples. For the set of tests in Table 1, we used n = 2443 and a vector or real-world eigenvalues from a finance application. Q is the product of 5000 random rotations, resulting in nonzeros in Q (and thus, not particularly large). K rqmip PRSP SLE rqmip PRSP SLE LB LB LB sec sec sec Table 1: Examples with few nonzeroes Here, rqmip refers to the weak formulation, PRSP to the perspective formulation, and SLE to the approach in this paper. LB is the lower bound produced by a given approach, and sec is the CPU time in seconds. We see that that rqmip is quite weak especially for smaller K. Both PRSP and SLE substantially improve on rqmip, though PRSP is significantly more expensive. In Table 2 we consider examples with n = and random Λ. In the table, Nonz indicates the number of nonzeroes in Q; as this number increases the quadratic becomes less diagonal dominant. Nonz rqmip PRSP SLE rqmip PRSP SLE in Q LB LB LB sec sec sec 5.3e e e e e e e e e e e e e e e e e e e e e e e e e e Table 2: Larger examples 10

11 As in Table 1, SLE and PRSP provide similar improvements over rqmip (which is clearly extremely weak). Moreover, SLE proves uniformly fast essentially, free compared to rqmip. In the examples in Table 2, the smallest ten (or so) eigenvalues are approximately equal, with larger values after that. The techniques in Section should extend to this situation in order to obtain an even stronger bound we hope to present results on this in the talk. Also note that the perspective formulation quickly proves impractical. A cutting-plane procedure that replaces the conic constraints in (20) with (outer approximating) linear inequalities is outlined in [10], [11] and tested on random problems with n 400 (which we will also test in the full paper). Such a procedure begins by solving rqmip and then iteratively adds the inequalities; or it could simply solve a formulation consisting of rqmip, augmented with a set of pre-computed inequalities. In either case the running time will be slower than that for rqmip. In our initial experiments with this linearized approximation, we found that (a) it can provide a very good lower bound to the conic perspective formulation, (b) it can run significantly faster than the full conic formulation, but, (c) it proves significantly slower than rqmip, and, in particular, still significantly slower than the combination of rqmip and SLE. The discussion regarding the perspective formulation is of interest because it is precisely an example of the paradigm that we consider in this paper: a convex formulation for a nonconvex problem with a convex objective. In our tests involving large cases, and using branch-and-cut, the lower bound proved by the perspective formulation exhibited the expected stalling behavior. Figure 3 concerns the K = 70 case of Table 1 where we ran the mixed-integer programming version of several formulations using Cplex 12.1 on a faster machine (on which rqmip requires 4.35 seconds and our method, 3.54 seconds, to prove the lower bound of ). The runs in the table used four parallel threads of execution. formulation nodes time LB UB QPMIP (2.75 sec/node) PRSP-MIP (56.04 sec/node) LPRSP-MIP (11.21 sec/node) root Table 3: Detailed analysis of K = 70 case of Table 1 In Table 3, QPMIP is the weak formulation, PRSP-MIP is the perspective formulation [Comment: on conic MIPs, Cplex appears to keep the lower bound fixed at 0 for a very long time]. LPRSP-MIP it the linearized perspective version. The figures in parentheses indicate CPU seconds per branch-and-cut node. time indicates the observed time (i.e. total CPU time will be, roughly, four times larger). Note the stalling of the lower bound in LPRSP-MIP at a value strictly smaller than the bound SLE proved with no branching. QPMIP with all Cplex cuts suppressed (not shown in the table) yielded an upper bound of in 200 seconds this underlines the point concerning fast heuristics that we made in the introduction. Our thechniques apply to the solution computed by the perspective formulation (after projecting out the auxiliary variables). This approach can only be desirable if there is a fast and tight approximation to the perspective formulation. However, Table 2 suggests that for large problems rqmip is already too expensive. It might simply better to approximate its solution, quickly, perhaps using a first-order method (and apply our techniques to the resulting solution vector). We plan to investigate these issues in the full paper. A more significant focus of our upcoming work concerns problems in complex applications where auxiliary binary variables cannot be naturally employed. A major issue that we plan to investigate is how to incorporate our techniques within branch-and-cut, so that the bound proved at each node strictly improves on the bound at its parent node no stalling. In this context, the single most important idea that we will investigate and report on concerns branching based on eigenvector structure, along the lines of Section

12 References [1] A. Ben-Tal and A. Nemirovsky, em Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications (2001) MPS-SIAM Series on Optimization, SIAM, Philadelphia, PA. [2] D. Bienstock, Computational study of a family of mixed-integer quadratic programming problems, Math. Programming 74 (1996), [3] D. Bienstock and M. Zuckerberg, Subset algebra lift algorithms for 0-1 integer programming, SIAM J. Optimization 105 (2006), [4] D. Bienstock and B. McClosky, Tightening simple mixed-integer sets with guaranteed bounds (2008). Submitted. [5] S. Boyd, L. El Ghaoui, E. Feron, and V. Balakrishnan, Linear matrix inequalities in system and control theory (1994). SIAM, Philadelphia, PA. [6] W. Cook, R. Kannan and A. Schrijver, Chv atal closures for mixed ineger programs, Math. Programming 47 (1990), [7] I. De Farias, E. Johnson and G. Nemhauser, A polyhedral study of the cardinality constrained knapsack problem, Math. Programming 95 (2003), [8] G.H. Golub, Some modified matrix eigenvalue problems, SIAM Review 15 (1973), [9] G.H. Golub and C. van Loan, Matrix Computations. Johns Hopkins University Press (1996). [10] A. Frangioni and C. Gentile, Perspective cuts for a class of convex 0-1 mixed integer programs, Mathematical Programming 106, (2006). [11] O. Günlük and J. Linderoth, Perspective Reformulations of Mixed Integer Nonlinear Programs with Indicator Variables, Optimization Tech. Report, U. of Wisconsin-Madison (2008). [12] B. Moghaddam, Y. Weiss, S. Avidan, Generalized spectral bounds for sparse LDA, Proc. 23rd Int. Conf. on Machine Learning (2006), [13] J.J. Moré and D.C. Sorensen, Computing a trust region step, SIAM J. Sci. Statisr Comput 4 (1983), [14] I. Pólik and T. Terlaky, A survey of the S-lemma, SIAM Review 49 (2007), [15] F. Rendl, H. Wolkowicz, A semidefinite framework for trust region subproblems with applications to large scale minimization, Math. Program 77 (1997) [16] R. J. Stern and H. Wolkowicz, Indefinite trust region subproblems and nonsymmetric eigenvalue perturbations, SIAM J. Optim. 5 (1995) [17] J. Sturm and S. Zhang, On cones of nonnegative quadratic functions, Mathematics of Operations Research 28 (2003), [18] W. Miller, S. Wright, Y. Zhang, S. Schuster and V. Hayes, Optimization methods for selecting founder individuals for captive breeding or reintroduction of endangered species, manuscript (2009). [19] V. A. Yakubovich, S-procedure in nonlinear control theory, Vestnik Leningrad University, 1 (1971) [20] Y. Ye and S. Zhang, New results on quadratic minimization, SIAM J. Optim. 14 (2003),