MA 796S: Convex Optimization and Interior Point Methods October 8, 2007 Lecture 1 Lecturer: Kartik Sivaramakrishnan Scribe: Kartik Sivaramakrishnan 1 Conic programming Consider the conic program min s.t. r c T i x i r A i x i = b x i K i, i = 1,..., r (1) and its dual max b T y s.t A T i y + s i = c i, i = 1,..., r s i Ki (2) where b, y IR m ; c i, x i, s i IR ni, A i IR m ni, i = 1,..., r. For each i = 1,..., r, x i and s i are the primal and dual slack variables associated with the ith cone and K i = { s i IR ni : x T i s i 0, x i K i } (3) is the dual cone to K i. We assume that K i, Ki, i = 1,..., r are pointed closed convex cones with nonempty interiors. Let K = K 1 K 2... K r be the r overall cone in (1) and let n = n i denote its size. The overall dual cone is K = K 1 K 2... K r. We are interested in self-dual cones K where K = K. Let int(k) denote the interior of the closed convex cone K. The important self-dual cones include the following: 1. Linear cone: We have int(ir n +) = {x IR n : x > 0}. IR n + = {x IR n : x 0}. (4) 1-1
2. Second order cone: Q n + = x n IRn : x 1 i=2 x 2 i. (5) The notation x Q 0 indicates that x lies in a second order cone. We also have int(q n +) = x IRn : x 1 > n x 2 i. 3. Semidefinite cone: i=2 S n + = {X S n : X is symmetric and positive semidefinite}. (6) The notation X 0 indicates that the matrix X lies in a semidefinite cone. We also have int(s n +) = {X S n : X is symmetric and positive definite}. Exercise: Show that IR n +, Q n +, and S n + are closed convex cones. Every convex optimization problem can be written as a conic program. To illustrate the idea consider the strictly convex quadratic program min x T Qx s.t. Ax = b x 0 (7) where x IR n, Q is a symmetric positive definite matrix of size n, A IR m n, and b IR m. One can reformulate (7) as min t,x s.t. x T Qx t Ax = b x 0. t (8) Consider the Cholesky factorization of Q, i.e., Q = RR T where R IR n n. We can rewrite the quadratic inequality constraint in (8) as a second order cone constraint as follows: x T Qx t x T (RR T )x t Rx 2 t t + 1 2Rx t 1 Q 0. This allows us to rewrite (8) as the following conic program min t,x s.t. Ax = b x 0 t + 1 2Rx Q 0. t 1 t (9) 1-2
Setting t = t1 t2 where t1, t2 0 and u = t + 1 2Rx we can rewrite (9) as t 1 the following second order cone program in standard form min c T x x s.t. Ā x = b x K (10) where x = ( x 1 x 2 ) T with x 1 = (t1 t2 x) T and x 2 = u. We have x 1 0 and x 2 Q 0. Therefore, in this problem K = K 1 K 2 where K 1, K 2 IR n+2 are linear and second order cones, respectively. The parameters c, Ā, and b in (10) can be easily obtained. Exercise: What are c, Ā, and b in (10)? The purpose of this lecture is to provide a short introduction to duality theory, strict complementarity, optimality conditions, and introduce the notions of extreme point solutions and nondegeneracy in conic programs. 2 Conic duality Consider the conic program min c T x s.t. Ax = b x K (11) with dual (12) max b T y s.t. A T y + s = c s K where c, x IR n, b IR m, and A IR m n with m < n. Let x and (y, s) be a feasible solutions in (11) and (12), respectively. We have c T x = (A T y + s) T x = y T Ax + s T x = b T y + s T x 0 (13) where the last inequality in (13) follows from x K, s K and the definition (3) of the dual cone. Therefore, we have c T x b T y and so the quantity b T y is a lower bound on the optimal value of (11). The difference c T x b T y is also referred to as the duality gap. The weak duality theorem suggests that the objective value of any feasible solution to the primal (minimization) problem is greater than the objective value of any feasible solution to the dual (maximization) problem. The best lower bound we can obtain on the optimal objective value of (11) is precisely the dual problem (12). We will henceforth refer to (12) as the dual problem to (11). The conic problems (11) and (12) minimize a linear functional over the intersection of an affine subspace and a convex cone. To see this, we will make the following assumption. 1-3
Assumption 1 The matrix A is surjective, i.e. b IR m, x IR n, such that Ax = b. This implies that the adjoint operator A T is injective (one-to-one), i.e., w = z A T w = A T z, i.e. A T has only the trivial nullspace 0. From linear algebra, this implies that A has full row rank. If assumption 1 is not met, i.e., there is a b IR m such that Ax b for all x IR n. Then there is a nonzero vector e IR m satisfying A T e = 0 and b T e 0. Without loss of generality, we assume that b T e > 0 (else choose e = e). Now if the dual problem (12) is feasible then (ȳ, s) satisfying A T ȳ + s = c and s K. Now consider y = ȳ + µe with µ > 0. Since A T e = 0, we have A T y + s = A T (ȳ + µe) + s = A T ȳ + s = c indicating that (y, s) is feasible in (12) too. Moreover b T y = b T (ȳ + µe) (as µ ) indicating that (12) is unbounded. So if assumption (1) is not met, then the dual problem is unbounded whenever it is feasible. Using assumption 1, we can write (11) in the following fashion. Let L = {d IR n : Ad = 0} be the null space of A. Assumption 1 ensures that L is non-empty. Consider x satisfying Ax = b and (y, s ) satisfying A T y + s = c. We have and Ax = b x x + L c T x = b T y + (s ) T x for all feasible x in (11). So we can rewrite (11) as Moreover, we have min (s ) T x s.t. x (x + L) K A T y + s = c s = s A T (y y ) s s + L where we have used the fact that the row space of a matrix A is orthogonal to its null space. Also, we have b T y = c T x (x ) T s. for all feasible (y, s) in (12). So we can rewrite (12) as min (x ) T s s.t. s (s + L ) K. Therefore, any conic program can be expressed as the problem of minimizing a linear function over a convex set that is described as the intersection of an affine subspace and a closed convex cone. 1-4
Unlike linear programs, we have to impose some conditions to achieve strong duality. We will first introduce the notions of strictly feasible solutions (Slater points) for the primal and dual conic programs. Consider the following regions F (P ) = {x IR n : Ax = b, x K} F 0 (P ) = {x IR n : Ax = b, x int(k)} F (D) = {y IR m, s IR n : A T y + s = c, s K} F 0 (D) = {y IR m, s IR n : A T y + s = c, s int(k)}. (14) The sets F (P ) and F (D) denote the feasible regions for the conic programs (11) and (12), respectively. Moreover, F 0 (P ) and F 0 (D) represent the interiors of F (P ) and F (D), respectively. Note that F 0 (P ) and F 0 (D) are open sets (they have no boundary). We will refer to any x F 0 (P ) ((y, s) F 0 (D)) as a strictly feasible primal (dual) Slater point. We now discuss the strong duality theorem for optimal solutions to the primal and dual conic programs. First, we will describe the strong duality theorem for linear programs whose proof can be found in Chvátal [4]. Theorem 2 Consider the primal linear program (11) along with its dual (12). If F (P ) and F (D) are nonempty, then both problems (11) and (12) attain their optimal solutions and the optimal objective values of (11) and (12) are the same. We will now state the strong duality theorem for general conic programs whose proof can be found in Section 2.4 of Ben Tal and Nemirovskii [3], Chapter 3 of Renegar [6], or Section 4 of Todd [7]. Theorem 3 Consider the primal conic problem (11) along with its dual (12). 1. If F 0 (P ) and F (D) are nonempty, then the dual problem (12) attains its optimal solution and the optimal objective values of (11) and (12) are the same. 2. If F (P ) and F 0 (D) are nonempty, then the primal problem (11) attains its optimal solution and the optimal objective values of (11) and (12) are the same. 3. If F 0 (P ) and F 0 (D) are nonempty, then both problems (11) and (12) attain their optimal solutions and the optimal objective values of (11) and (12) are the same. Case 3 in theorem 3 is similar to theorem 2 in the LP case. Note that we need stronger conditions, i.e., strictly feasible primal and dual solutions (unlike feasible primal and dual solutions in the LP case) in the general conic case. We shall now consider some examples involving semidefinite cones to illustrate some of the pathologies that can occur in the conic case. Our primal (11) semidefinite program is min C X s.t. A i X = b i, i = 1,..., m X 0 and the dual (12) semidefinite program is max b T y s.t. S = C S 0. m y i A i (15) (16) 1-5
The matrices X, S, C, and A i, i = 1,..., m are symmetric matrices of size n. Moreover, X, S are positive semidefinite matrices. The notation C X = trace(cx) n n = C ij X (17) ij j=1 represents the Frobenius inner product for symmetric matrices. Moreover, assumption 1 requires that the matrices A i, i = 1,..., m in (15) are linearly independent. Since S n is isomorphic to IR (n+1 2 ) (, this implies that m n+1 ) 2. Consider min 0 0 X 1 s.t. 1 0 X = 0 0 0 1 0 1 X = 2 2 X 0 with dual max 2y 2 s.t. S = y 1 y 2 0 y 2 1 2y 2 S 0 The primal constraints suggest that X 11 = 0, and X 12 + X 33 = 1. Thus any 0 ξ 1 ξ 2 feasible X is of the form ξ 1 ξ 3 ξ 4. Also X must be positive semidefinite, and this forces ξ 1, ξ 2 = 0 (since 0 ξ 1 ξ 1 ξ 3 ξ 2 ξ 4 1 ξ 1 0 and 0 ξ 2 ξ 2 1 ξ 1 0). 0 Thus any feasible X = 0 ξ 3 ξ 4. The optimal objective value for the 0 ξ 4 1 primal is 1, and an optimal solution is 0 0. 1 In the dual we require S = y 1 y 2 0 y 2 0. So y 2 = 0 (since 1 2y 2 y 1 y 2 y 2 0 0). Moreover, y 1 has to be nonpositive. Thus y = () T is an optimal solution to the dual problem and the dual optimal value is 0. Note that both problems attain their optimal solutions, but there is a duality gap. We note that neither the primal nor the dual semidefinite program has a Slater point and this explains the duality gap. Moreover, we can also construct examples where there is an infinite duality gap between the primal and dual 1-6
objective values. Now consider ( ) 0 1 min X ( 1 0 ) 1 0 s.t. X = 1, ( ) X = 0, 0 1 X 0 with dual max y 1 ( ) y1 1 s.t. S = 1 y 2 S 0 ( ) 1 0 It can be easily seen that the primal has only one feasible point X = which is not a strictly feasible solution. So, the primal problem ( has an ) optimal y1 1 objective value of 0. Now consider the dual. We require 0. So 1 y 2 the feasible region is {(y 1, y 2 ) : y 1 > 0, y 2 > 0, y 1 y 2 1}. So the dual optimal objective value is 0, but it is not attained. We can get arbitrarily close with solutions (ɛ, 1 ɛ ) for some arbitrarily small ɛ > 0. We also want to relate the existence of primal-dual Slater points to the notion of robust solvability of these problems. Here is an example, which shows that tweaking one of the data parameters by a small amount can lead to something drastically different. Consider min 0 0 X 1 1 s.t. 0 X = 0 0 0 1 0 1 X = 2 2 X 0 Any feasible X is of the form feasible X must have the form X = It follows that an optimal X is 0 ξ 1 ξ 2 ξ 1 ξ 3 ξ 4 ξ 2 ξ 4 1 ξ 1 0 0 ξ 3 ξ 4 0 ξ 4 1 0 0 1., and since X 0, any with an optimal value 1. The 1-7
dual problem is max 2y 2 s.t. S = y 1 y 2 0 y 2 1 2y 2 S 0 and so y 2 = 0, and y 1 should be nonpositive. Thus y = () T is optimal, with optimal value 0. Here both problems attain their optimal values, but there is a gap between them. Note that the dual problem does not have a Slater point either in this case. Now let us see what happens if we perturb b 1 from its present value 0 to ɛ > 0. In this case X 11 = ɛ, and so X 12 is no longer constrained to be 0. In fact any feasible X has the form X = ɛ ξ 1 ξ 2 ξ 1 ξ 3 ξ 4 ξ 2 ξ 4 1 ξ 1 and in fact an optimal X is given by ɛ 1 0 X = 1 1 ɛ 0 0 for an optimal value of 0. It can be easily verified (check this!) that the dual optimal solution is attained at the same point as before the perturbation, and its optimal value is once again zero. So after the perturbation, we see that both points attain their optimal solution, and the duality gap is zero. This brings up the issue of robustness. We see that the properties of this semidefinite program are not robust with respect to small perturbations in the data, i.e. for a small change in parameters, something entirely different can happen. 3 Complementary slackness, strict complementarity, and optimality conditions Consider the conic programs (1) and (2). The strong duality theorem for this r pair of conic programs suggests that x T s = x T i s i = 0. Since x T i s i 0, i = 1,..., r we must have x T i s i = 0, i = 1,..., r. Let K i be the ith cone in (1). We will use the condition x T i s i = 0 to derive the complementary slackness conditions for K i as follows: 1. Linear cone: Suppose K i = IR ni. Consider x i, s i K i. In this case, we have x T i s n i i = (x i ) j (s i ) j = 0 with x i, s i 0 j=1 which gives the following complementary slackness conditions (x i ) j (s i ) j = 0, j = 1,..., n i (18) for a linear cone. See chapter 5 in Chvátal [4] for more details. 1-8
2. Second order cone: Suppose K i = Q ni +. Consider x i = (x i0 x i ) T and s i = (s i0 s i ) T where x i0, s i0 IR and x i, s i IR ni 1. In this case, we have x T i s i = x i0 s i0 + x it s i = 0 with x i, s i Q ni + which gives the following complementary slackness conditions x i0 s i + s i0 x i = 0 (19) for a second order cone. See section 5 in Alizadeh and Goldfarb [1] for more details. 3. Semidefinite cone: Suppose K i = S ni +. Consider X i, S i K i, i.e., X i, S i 0. In this case, we have X i S i = 0 with X i, S i 0 which gives the following complementary slackness conditions X i S i = 0 (20) for a semidefinite cone. Note that the right hand side in (20) is the zero matrix of size n. See section 1 in Alizadeh et al. [2] for more details. The complementary slackness conditions (20) indicate that X i and S i commute and so they are simultaneously diagonalizable (see Horn and Johnson [5]), i.e., X i = P i Diag(λ i1,..., λ ini )Pi T and S = P i Diag(ω i1,..., ω ini )Pi T where P i IR ni ni is an orthogonal matrix containing the common eigenspace of X i and S i. Moreover, λ i1,..., λ ini and ω i1,..., ω ini are the eigenvalues of X i and S i, respectively. Therefore, we can also rewrite the conditions (20) as λ ij ω ij = 0, j = 1,..., n i. (21) We say that a pair of optimal solutions x and (y, s ) to (11) and (12) satisfy strict complementarity if x + s int(k). The Goldman-Tucker theorem (see theorem 2.4 in Wright [8]) shows that there is always a primal-dual pair of optimal solutions that satisfy strict complementarity in the linear case. However, there are semidefinite and second order cone programs that do not have any primal-dual pair of optimal solutions satisfying strict complementarity (see Alizadeh et al. [2] and Alizadeh and Goldfarb [1]). The optimality conditions for conic programs include the primal feasibility, dual feasibility, and complementary slackness conditions. Theorem 4 The vectors x IR n and y IR m, s IR n are optimal in (1) and (2) if and only if the following conditions hold: 1. Primal feasibility: Let x = (x 1 x 2... x r). We have and x i K i, i = 1,..., r. r A i x i = b 2. Dual feasibility: Let s = (s 1 s 2... s r). We have A T i y + s i = c i, s i K i, i = 1,..., r. 3. Complementary slackness conditions: For i = 1,..., r, x i and s i satisfy the complementary slackness conditions for the ith cone. 1-9
4 Extreme point solutions and nondegeneracy We noted in section 2 that a conic program involves the minimization of a linear function over a convex set that is the intersection of an affine subspace and a closed convex cone. Therefore, we know that an optimal solution will be attained at an extreme point of the feasible region. In this section, we review the notions of extreme points and nondegeneracy for linear and semidefinite programs. We will first introduce some definitions: For any point x IR n, the distance from x to the convex cone K is defined as the distance from x to the unique closest point in K and is denoted by dist(x, K). Consider the primal conic program. (11) and its feasible region F (P ). A point x F (P ) is said to be an extreme point of F (P ) if and only if there is no d IR n such that x±λd F (P ) for some λ > 0. Let Definition 5 Bx K = {d IR n : x ± λd K for some λ > 0} Tx K = {d IR n : dist(x ± λd, K) = O(λ 2 ) for some λ > 0} N = {d IR n : Ad = 0} N = {d IR n : d = A T y for some y IR m }. (22) Note that N is the orthogonal complement of N. From linear algebra, we know that N is the row space of A. Exercise: Show that the row space of A is the orthogonal complement of the null space of A. Theorem 6 Let x be a feasible solution in (11). x is an extreme point of (11) B K x N =. Proof: Consider a x F (P ). If x is not an extreme point of (11) then there is a nonzero d IR n such that x ± λd F (P ) for some λ > 0. Therefore, A(x ± λd) = Ax = b which implies that d N. Moreover, x ± λd 0 for some λ > 0 which implies that d Bx K. Therefore, d Bx K N which implies that Bx K N =. Similarly, if there is a nonzero d Bx K N then x ± λd F (P ) for some λ > 0 which implies that x is not an extreme point of (11). We will now describe B K x for linear and semidefinite cones: 1. Let F (P ) = {x IR n : Ax = b, x 0} be the feasible set of a linear program and let x F (P ). We have B LP x = {d IR n : d i = 0 if x i = 0, i = 1,..., n}. (23) 2. Let F (P ) = {X S n : A i X = b i, i = 1,..., m, [ X 0} be ] the feasible Λ 0 set of a semidefinite program and let X = [P Q] [P Q] T be a feasible solution where P IR n r with P T P = I is an orthonormal matrix containing the positive eigenspace of X and Λ 0 is a diagonal matrix of size r containing the positive eigenvalues of X. We have { [ ] } U 0 BX SDP = [P Q] [P Q] T : U S r. (24) 1-10
For the linear cone Tx LP. In fact, we have T SDP X T SDP X = { [ U V [P Q] V T 0 = Bx LP. However, for the semidefinite cone BX SDP ] [P Q] T : U S r V IR r (n r) }. (25) To see this, consider sufficiently small perturbations X ± λ X, with X TX SDP. These matrices are typically indefinite, but a [ psd matrix ] can be recovered by adding a matrix W of the form W = [P Q] [P Q] 0 Σ T to them where Σ S+ n r. Thus, the distance from X ± λ X to S+ n is the norm of the matrix Σ. We utilize the Schur complement idea to obtain a bound on Σ. Notice that [ Λ + λu λv X + λ X + W = [P Q] λv T Σ ] [P Q] T so we obtain the following condition on Σ: X + λ X + W 0 [ ] Λ + λu λv λv T 0 Σ Σ λ 2 V T (Λ + λu) 1 V 0 We have utilized the fact that Λ (since it is positive definite) dominates λu, so the matrix Λ + λu is invertible in the Schur complement. Thus loosely speaking, we can choose Σ with Σ = O(λ 2 ), so X is in TX SDP. For a sufficiently small perturbation λ we can say that whenever X TX SDP, then the matrix X ±λ X is sufficiently close to being a positive semidefinite matrix. Moreover, we have { [ ] } TX SDP = [P Q] [P Q] 0 W T : W S n r. (26) We will now introduce the notion of primal nondegeneracy. For more details, see Alizadeh et al. [2]. Definition 7 Let x IR n be a feasible solution in (11) and then X is primal nondegenerate. Note that the definition (27) can also be stated as T K x + N = IR n (27) Tx K N = (28) where Tx K and N denote the orthogonal complements of TX K and N, respectively. It is easy to see that the definitions (27) and (28) are equivalent: For instance, if x Tx K N then x is neither in Tx K Similarly, we can define dual nondegeneracy as follows: nor N and so T K x +N IR n. Definition 8 Let y IR m and s IR n be a feasible solution in (12) and T K z + N = IR n (29) 1-11
We will review the notion of extreme points and primal nondegeneracy for linear and semidefinite programs. 1. Let x be a feasible solution to a linear program. Without loss of generality, we will assume that x i > 0, i = 1,..., r and x i = 0, i = r + 1,..., n. (a) x is an extreme point if and only if the first r columns of A corresponding to the positive components of x are linearly independent in IR m. This suggests that r m. (b) x is primal nondegenerate if and only if the m rows of A are linearly independent in IR r. This suggests that r m. Note that if x is nondegenerate extreme point solution to a linear program then we have Bx LP N = IR n and that r = m. Exercise: Show all the above statements. [ ] Λ 0 2. Let X = [P Q] [P Q] T be a feasible solution to a semidefinite program where P IR n r and Q IR n (n r) are orthonormal matrices containing the positive eigenspace and null space of X, respectively and Λ 0 is a diagonal matrix containing the positive eigenvalues of X. Theorem 9 X is an extreme point if and only if the matrices P T A i P, i = 1,..., m span S r. Proof: Let X be an extreme point solution. Consider X BX SDP. We have X = P UP T where U S r. Since X is an extreme point, X / N. Therefore, the equations A i (P UP T ) = (P T A i P ) U = 0, i = 1,..., m have only the trivial solution U = 0. This implies that the matrices P T A i P, i = 1,..., m span S r. By reversing the arguments, we can show that if P T A i P span S r then X = P ΛP T is an extreme point solution. r(r + 1) Moreover, theorem 9 implies that m, i.e., r = 2m. This 2 suggests that the rank of an optimal solution to the primal semidefinite program is O( m). Theorem 10 X is primal nondegenerate if and only if the matrices [ ] P B k = T A k P P T A k Q Q T, k = 1,..., m A k P 0 are linearly independent in S n. Proof: Let X be a feasible solution that is primal nondegenerate. Suppose, the matrices B k, k = 1,..., m are not linearly independent. m Then, we have y k B k = 0 where y 0, i.e., some of the components y k k=1 of y are nonzero. Using the definition of the B k matrices, we have ( m ) [ ] [P Q] T y k A k [P Q] =. 0 W k=1 1-12
Therefore, Since, N = m k=1 m y k A k k=1 [ y k A k = [P Q] 0 W we have TX SDP N ] [P Q] T. which contradicts the nondegeneracy assumption on X. Moreover, if the matrices B k, k = 1,..., m are linearly independent in S n, then we have TX SDP N = which implies that X is nondegenerate. For linear programs, if the primal (dual) optimal solution is nondegenerate then the dual (primal) optimal solution is unique (see Chvátal [4]). For semidefinite programs, if the primal optimal solution is nondegenerate then the dual optimal solution is unique as we shall show below: [ ] Λ 0 Theorem 11 Let X = [P Q] [P Q] T with Λ 0 be a primal nondegenerate optimal solution to the primal semidefinite program. Then there exists a unique optimal dual solution (y, S). Proof: We assume that our semidefinite program satisfies assumption 1, which in turn implies that the dual slack variable S is uniquely determined by the y variable. Therefore, we have only to show that there exists a unique optimal solution y to the dual semidefinite program. The complementary slackness conditions XS = 0 [ for semidefinite ] programs suggests that an optimal S is of the form S = [P Q] [P Q] 0 Σ T. Consider [P Q] T S[P Q] = 0. This gives [ ] [ ] P = T CP P T CQ m [ ] P Q T y T A i P P T A i Q CP 0 i Q T A i P 0 m Q T SQ = Q T CQ y i (Q T A i Q). and (30) Suppose, there are two optimal solutions ȳ and ỹ to the dual semidefinite program, then the first equation in (30) implies that ȳ = ỹ (using the linear independence of the B k matrices). This indicates that the dual problem has a unique optimal solution (y, S). The converse of theorem 11 also holds under the addition of strict complementarity (see Alizadeh et al. [2]). Thus, the notions of nondegeneracy and extreme point solutions in conic programs (under the additional assumption of strict complementarity) are complementary, i.e., if the primal (dual) optimal is nondegenerate then the dual (primal) optimal solution is an extreme point. Since, strictly complementary optimal solutions always exist for linear programs (Goldman-Tucker theorem), this is true for all linear programs. This is not always true for second order and semidefinite programs and we refer the reader to examples in Alizadeh et al. [2, 1]. Similarly, we can show that if the dual optimal solution to a semidefinite program is nondegenerate then the primal semidefinite program has a unique optimal solution. Moreover, the converse is also true under the assumption of strict complementarity (see Alizadeh et al. [2]). 1-13
We have not discussed the notion of extreme point solutions and primal and dual nondegeneracy in second order cone programs. For more details, we refer the reader to Alizadeh and Goldfarb [1]. References [1] F. Alizadeh and D. Goldfarb, Second order cone programming, Mathematical Programming, 95(2003), pp. 3-51. [2] F. Alizadeh, J.A. Haeberly, and M.L. Overton, Complementarity and nondegeneracy in semidefinite programming, Mathematical Programming, 77(1997), pp. 129-162. [3] A. Ben-Tal and A. Nemirovskii, Lectures on Modern Convex Optimization, MPS-SIAM Series on Optimization, 2001. [4] V. Chvátal, Linear Programming, W.H. Freeman and Company, New York, 1983. [5] R. Horn and C. Johnson, Matrix Analysis, Cambridge University Press, 1990. [6] J. Renegar, A Mathematical View of Interior-Point Methods in Convex Optimization, MPS-SIAM Series on Optimization, 2001. [7] M.J. Todd, Semidefinite Optimization, Acta Numerica 10(2001), pp. 515-560. [8] S.J. Wright, Primal-Dual Interior-Point Methods, SIAM, Philadelphia, 1996. 1-14