Semidefinite Programming

Chapter 2 Semidefinite Programming 2.0.1 Semi-definite programming (SDP) Given C M n, A i M n, i = 1, 2,..., m, and b R m, the semi-definite programming problem is to find a matrix X M n for the optimization problem: (SDP ) inf C X subject to A i X = b i, i = 1, 2,..., m, X 0. Recall that the operation is the matrix inner product A B := tra T B. The notation X 0 means that X is a positive semi-definite matrix, and X 0 means that X is a positive definite matrix. If a point X 0 and satisfies all equations in (SDP), it is called a (primal) strictly or interior feasible solution.. The dual problem to (SDP) can be written as: (SDD) sup b T y m subject to i y i A i + S = C, S 0, which is analogous to the dual (LD) of LP. Here y R m and S M n. If a point (y, S 0) satisfies all equations in (SDD), it is called a dual interior feasible solution. 31

32 CHAPTER 2. SEMIDEFINITE PROGRAMMING Example 2.1 Let P (y R m ) = C + m i y i A i, where C and A i, i = 1,..., m, are given symmetric matrices. The problem of minimizing the max-eigenvalue of P (y) can be cast as a (SDD) problem. In semi-definite programming, we minimize a linear function of a matrix in the positive semi-definite matrix cone subject to affine constraints. In contrast to the positive orthant cone of linear programming, the positive semi-definite matrix cone is non-polyhedral (or non-linear ), but convex. So positive semi-definite programs are convex optimization problems. Semi-definite programming unifies several standard problems, such as linear programming, quadratic programming, and convex quadratic minimization with convex quadratic constraints, and finds many applications in engineering, control, and combinatorial optimization. We have several theorems analogous to Farkas lemma. Theorem 2.1 (Farkas lemma in SDP) Let A i M n, i = 1,..., m, have rank m (i.e., m i y i A i = 0 implies y = 0) and b R m. Then, there exists a symmetric matrix X 0 with A i X = b i, i = 1,..., m, if and only if m i y i A i 0 and m i y i A i 0 implies b T y < 0. Note the difference between the LP and SDP. Theorem 2.2 (Weak duality theorem in SDP) Let F p and F d, the feasible sets for the primal and dual, be non-empty. Then, C X b T y where X F p, (y, S) F d. The weak duality theorem is identical to that of (LP) and (LD). Corollary 2.3 (Strong duality theorem in SDP) Let F p and F d be nonempty and have an interior. Then, X is optimal for (PS) if and only if the following conditions hold: i) X F p ; ii) there is (y, S) F d ; iii) C X = b T y or X S = 0. Again note the difference between the above theorem and the strong duality theorem for LP.

2.1. ANALYTIC CENTER 33 Example 2.2 Let C = 0 1 0 1 0 0, A 1 == 0 0 0 0 1 0, A 1 == 0 1 0 1 0 0 0 0 0 0 0 0 0 0 2 and ( 0 b = 10 ). Two positive semi-definite matrices are complementary to each other, X S = 0, if and only if XS = 0 (Exercise 1.22). From the optimality conditions, the solution set for certain (SDP) and (SDD) is or S p = {X F p, (y, S) F d : C X b T y = 0}, S p = {X F p, (y, S) F d : XS = 0}, which is a system of linear matrix inequalities and equations. In general, we have Theorem 2.4 (SDP duality theorem) If one of (SDP) or (SDD) has a strictly or interior feasible solution and its optimal value is finite, then the other is feasible and has the same optimal value. If one of (SDP) or (SDD) is unbounded then the other has no feasible solution. Note that a duality gap may exists if neither (SDP) nor (SDD) has a strictly feasible point. This is in contrast to (LP) and (LD) where no duality gap exists if both are feasible. Although semi-definite programs are much more general than linear programs, they are not much harder to solve. It has turned out that most interior-point methods for LP have been generalized to semi-definite programs. As in LP, these algorithms possess polynomial worst-case complexity under certain computation models. They also perform well in practice. We will describe such extensions later in this book. 2.1 Analytic Center 2.1.1 AC for polytope Let Ω be a bounded polytope in R m represented by n (> m) linear inequalities, i.e., Ω = {y R m : c A T y 0},

34 CHAPTER 2. SEMIDEFINITE PROGRAMMING where A R m n and c R n are given and A has rank m. Denote the interior of Ω by Ω= {y R m : c A T y > 0}. Define d(y) = n (c j a T j y), y Ω, where a.j is the jth column of A. Traditionally, we let s := c A T y and call it a slack vector. Thus, the function is the product of all slack variables. Its logarithm is called the (dual) potential function, B(y) := log d(y) = log(c j a Ṭ jy) = log s j, (2.1) and B(y) is the classical logarithmic barrier function. For convenience, in what follows we may write B(s) to replace B(y) where s is always equal to c A T y. Example 2.3 Let A = (1, 1) and c = (1; 1). Then the set of Ω is the interval [ 1, 1]. Let A = (1, 1, 1) and c = (1; 1; 1). Then the set of Ω is also the interval [ 1, 1]. Note that and d( 1/2) = (3/2)(1/2) = 3/4 and B( 1/2) = log(3/4), d ( 1/2) = (3/)(1/2)(1/2) = 3/8 and B ( 1/2) = log(3/8). The interior point, denoted by y a and s a = c A T y a, in Ω that maximizes the potential function is called the analytic center of Ω, i.e., B(Ω) := B(y a, Ω) = max log d(y, Ω). y Ω (y a, s a ) is uniquely defined, since the potential function is strictly concave in a bounded convex Ω. Setting B(y, Ω) = 0 and letting x a = (S a ) 1 e, the analytic center (y a, s a ) together with x a satisfy the following optimality conditions: Xs = e Ax = 0 (2.2) A T y s = c. Note that adding or deleting a redundant inequality changes the location of the analytic center.

2.1. ANALYTIC CENTER 35 Example 2.4 Consider Ω = {y R : y 0, y 1}, which is interval [0, 1]. The analytic center is y a = 1/2 with x a = (2, 2) T. Consider Ω = {y R : n times {}}{ y 0,, y 0, y 1}, which is, again, interval [0, 1] but y 0 is copied n times. The analytic center for this system is y a = n/(n + 1) with x a = ((n + 1)/n,, (n + 1)/n, (n + 1)) T. The analytic center can be defined when the interior is empty or equalities are presented, such as Ω = {y R m : c A T y 0, By = b}. Then the analytic center is chosen on the hyperplane {y : By = b} to maximize the product of the slack variables s = c A T y. Thus, the interior of Ω is not used in the sense that the topological interior for a set is used. Rather, it refers to the interior of the positive orthant of slack variables: R n + := {s : s 0}. When say Ω has an interior, we mean that Again R n + {s : s = c A T y for some y where By = b} =. R n +:= {s R n + : s > 0}, i.e., the interior of the orthant R n +. Thus, if Ω has only a single point y with s = c A T y > 0, we still say Ω is not empty. Example 2.5 Consider the system Ω = {x : Ax = 0, e T x = n, x 0}, which is called Karmarkar s canonical set. If x = e is in Ω then e is the analytic center of Ω, the intersection of the simplex {x : e T x = n, x 0} and the hyperplane {x : Ax = 0} (Figure 2.1). 2.1.2 AC for SDP Let Ω be a bounded convex set in R m represented by n (> m) a matrix inequality, i.e., m Ω = {y R m : C y i A i 0, }. Let S = C m i y i A i and B(y) := log det(s)) = log det(c i m y i A i ). (2.3) i

36 CHAPTER 2. SEMIDEFINITE PROGRAMMING x 3 (0,0,3) Ax=0. (1,1,1) (3,0,0) x 1 (0,3,0) x 2 Figure 2.1: Illustration of the Karmarkar (simplex) polytope and its analytic center. The interior point, denoted by y a and S a = C m i yi aa i, in Ω that maximizes the potential function is called the analytic center of Ω, i.e., max B(y). y Ω (y a, S a ) is uniquely defined, since the potential function is strictly concave in a bounded convex Ω. Setting B(y, Ω) = 0 and letting X a = (S a ) 1, the analytic center (y a, S a ) together with X a satisfy the following optimality conditions: XS = I AX = 0 (2.4) A T y S = C. 2.2 Potential Functions for LP and SDP We show how potential functions can be defined to solve linear programming problems and semi-definite programming. We assume that for a given LP data set (A, b, c), both the primal and dual have interior feasible point. We also let z be the optimal value of the standard form (LP) and (LD). Denote the feasible sets of (LP) and (LD) by F p and F d, respectively. Denote by F = F p F d, and the interior of F by F.

2.2. POTENTIAL FUNCTIONS FOR LP AND SDP 37 2.2.1 Primal potential function for LP Consider the level set Ω(z) = {y R m : c A T y 0, z + b T y 0}, (2.5) where z < z. Since both (LP) and (LD) have interior feasible point for given (A, b, c), Ω(z) is bounded and has an interior for any finite z, even Ω := F d is unbounded (Exercise 1.23). Clearly, Ω(z) Ω, and if z 2 z 1, Ω(z 2 ) Ω(z 1 ) and the inequality z + b T y is translated from z = z 1 to z = z 2. From the duality theorem again, finding a point in Ω(z) has a homogeneous primal problem For (x, x 0) satisfying let x := x /x 0 F p, i.e., minimize c T x zx 0 s.t. Ax bx 0 = 0, (x, x 0) 0. Ax bx 0 = 0, (x, x 0) > 0, Ax = b, x > 0. Then, the primal potential function for Ω(z) (Figure 2.2), as described in the preceding section, is P(x, Ω(z)) = (n + 1) log(c T x zx 0) = (n + 1) log(c T x z) log x j j=0 log x j =: P n+1 (x, z). The latter, P n+1 (x, z), is the Karmarkar potential function in the standard LP form with a lower bound z for z. One algorithm for solving (LD) is suggested in Figure 2.2. If the objective hyperplane is repeatedly translated to the analytic center, the sequence of new analytic centers will converge to an optimal solution and the potentials of the new polytopes will decrease to. As we illustrated before, one can represent Ω(z) differently: ρ times {}}{ Ω(z) = {y : c A T y 0, z + b T y 0,, z + b T y 0}, (2.6)

38 CHAPTER 2. SEMIDEFINITE PROGRAMMING y a b T y = b T y a y a b T y = z The objective hyperplane The updated objective hyperplane Figure 2.2: Intersections of a dual feasible region and the objective hyperplane; b T y z on the left and b T y b T y a on the right. i.e., z + b T y 0 is copied ρ times. Geometrically, this representation does not change Ω(z), but it changes the location of its analytic center. Since the last ρ inequalities in Ω(z) are identical, they must share the same slack value and the same corresponding primal variable. Let (x, x 0) be the primal variables. Then the primal problem can be written as ρ times {}}{ minimize c T x zx 0 zx 0 ρ times {}}{ s.t. Ax bx 0 bx 0 = 0, (x, x 0) 0. Let x = x /(ρx 0) F p. Then, the primal potential function for the new Ω(z) given by (2.6) is P(x, Ω(z)) = (n + ρ) log(c T x z(ρx 0)) = (n + ρ) log(c T x z) =: P n+ρ (x, z) + ρ log ρ. log x j ρ log x 0 log x j + ρ log ρ

2.2. POTENTIAL FUNCTIONS FOR LP AND SDP 39 The function P n+ρ (x, z) = (n + ρ) log(c T x z) log x j (2.7) is an extension of the Karmarkar potential function in the standard LP form with a lower bound z for z. It represents the volume of a coordinate-aligned ellipsoid whose intersection with A Ω(z) contains S Ω(z), where z + b T y 0 is duplicated ρ times. 2.2.2 Dual potential function for LP We can also develop a dual potential function, symmetric to the primal, for (y, s) F d B n+ρ (y, s, z) = (n + ρ) log(z b T y) log s j, (2.8) where z is a upper bound of z. One can show that it represents the volume of a coordinate-aligned ellipsoid whose intersection with the affine set {x : Ax = b} contains the primal level set {x F p : ρ times {}}{ c T x z 0,, c T x z 0}, where c T x z 0 is copied ρ times (Exercise 2.7). For symmetry, we may write B n+ρ (y, s, z) simply by B n+ρ (s, z), since we can always recover y from s using equation A T y = c s. 2.2.3 Primal-dual potential function for LP A primal-dual potential function for linear programming will be used later. For x F p and (y, s) F d it is defined by where ρ 0. We have the relation: ψ n+ρ (x, s) := (n + ρ) log(x T s) ψ n+ρ (x, s) = (n + ρ) log(c T x b T y) log(x j s j ), (2.9) log x j log s j

40 CHAPTER 2. SEMIDEFINITE PROGRAMMING Since = P n+ρ (x, b T y) = B n+ρ (s, c T x) log s j log x j. ψ n+ρ (x, s) = ρ log(x T s) + ψ n (x, s) ρ log(x T s) + n log n, then, for ρ > 0, ψ n+ρ (x, s) implies that x T s 0. More precisely, we have x T s exp( ψ n+ρ(x, s) n log n ). ρ We have the following theorem: Theorem 2.5 Define the level set i) Ψ(δ) := {(x, y, s) F: ψ n+ρ (x, s) δ}. Ψ(δ 1 ) Ψ(δ 2 ) if δ 1 δ 2. ii) Ψ (δ) = {(x, y, s) F : ψ n+ρ (x, s) < δ}. iii) For every δ, Ψ(δ) is bounded and its closure ˆΨ(δ) has non-empty intersection with the solution set. Later we will show that a potential reduction algorithm generates sequences {x k, y k, s k } F such that ψ n+ n (x k+1, y k+1, s k+1 ) ψ n+ n (x k, y k, s k ).05 for k = 0, 1, 2,... This indicates that the level sets shrink at least a constant rate independently of m or n.

2.2. POTENTIAL FUNCTIONS FOR LP AND SDP 41 2.2.4 Potential function for SDP The potential functions for SDP of Section 2.0.1 are analogous to those for LP. For given data, we assume that both (SDP) and (SDD) have interior feasible points. Then, for any X F p and (y, S) F d, the primal potential function is defined by P n+ρ (X, z) := (n + ρ) log(c X z) log det(x), z z ; the dual potential function is defined by B n+ρ (y, S, z) := (n + ρ) log(z b T y) log det(s), z z, where ρ 0 and z designates the optimal objective value. For X F p and (y, S) F d the primal-dual potential function for SDP is defined by ψ n+ρ (X, S) := (n + ρ) log(x S) log(det(x) det(s)) = (n + ρ) log(c X b T y) log det(x) log det(s) = P n+ρ (X, b T y) log det(s) = B n+ρ (S, C X) log det(x), where ρ 0. Note that if X and S are diagonal matrices, these definitions reduce to those for LP. Note that we still have (Exercise 2.8) ψ n+ρ (X, S) = ρ log(x S) + ψ n (X, S) ρ log(x S) + n log n. Then, for ρ > 0, ψ n+ρ (X, S) implies that X S 0. More precisely, we have X S exp( ψ n+ρ(x, S) n log n ). ρ We also have the following corollary: Corollary 2.6 Let (SDP) and (SDD) have non-empty interior and define the level set Ψ(δ) := {(X, y, S) F: ψ n+ρ (X, S) δ}. i) ii) Ψ(δ 1 ) Ψ(δ 2 ) if δ 1 δ 2. Ψ (δ) = {(X, y, S) F : ψ n+ρ (X, S) < δ}. iii) For every δ, Ψ(δ) is bounded and its closure ˆΨ(δ) has non-empty intersection with the solution set.

42 CHAPTER 2. SEMIDEFINITE PROGRAMMING 2.3 Central Paths of LP and SDP Many interior-point algorithms find a sequence of feasible points along a central path that connects the analytic center and the solution set. We now present this one of the most important foundations for the development of interior-point algorithms. 2.3.1 Central path for LP Consider a linear program in the standard form (LP) and (LD). Assume that F, i.e., both F p and F d, and denote z the optimal objective value. The central path can be expressed as { } C = (x, y, s) F: Xs = xt s n e in the primal-dual form. We also see { C = (x, y, s) F: } ψ n (x, s) = n log n. For any µ > 0 one can derive the central path simply by minimizing the primal LP with a logarithmic barrier function: (P ) minimize c T x µ n log x j s.t. Ax = b, x 0. Let x(µ) F p be the (unique) minimizer of (P). Then, for some y R m it satisfies the optimality conditions Xs = µe Ax = b A T y s = c. (2.10) Consider minimizing the dual LP with the barrier function: (D) maximize b T y + µ n log s j s.t. A T y + s = c, s 0. Let (y(µ), s(µ)) F d be the (unique) minimizer of (D). Then, for some x R n it satisfies the optimality conditions (2.10) as well. Thus, both minimizers x(µ) and (y(µ), s(µ)) are on the central path with x(µ) T s(µ) = nµ.

2.3. CENTRAL PATHS OF LP AND SDP 43 Another way to derive the central path is to consider again the dual level set Ω(z) of (2.5) for any z < z (Figure 2.3). Then, the analytic center (y(z), s(z)) of Ω(z) and a unique point (x (z), x 0(z)) satisfies Ax (z) bx 0(z) = 0, X (z)s = e, s = c A T y, and x 0(z)(b T y z) = 1. Let x(z) = x (z)/x 0(z), then we have Ax(z) = b, X(z)s(z) = e/x 0(z) = (b T y(z) z)e. Thus, the point (x(z), y(z), s(z)) is on the central path with µ = b T y(z) z and c T x(z) b T y(z) = x(z) T s(z) = n(b T y(z) z) = nµ. As we proved earlier in Section 2.2, (x(z), y(z), s(z)) exists and is uniquely defined, which imply the following theorem: Theorem 2.7 Let both (LP) and (LD) have interior feasible points for the given data set (A, b, c). Then for any 0 < µ <, the central path point (x(µ), y(µ), s(µ)) exists and is unique. y a The objective hyperplanes Figure 2.3: The central path of y(z) in a dual feasible region. The following theorem further characterizes the central path and utilizes it to solving linear programs.

44 CHAPTER 2. SEMIDEFINITE PROGRAMMING Theorem 2.8 Let (x(µ), y(µ), s(µ)) be on the central path. i) The central path point (x(µ), s(µ)) is bounded for 0 < µ µ 0 and any given 0 < µ 0 <. ii) For 0 < µ < µ, c T x(µ ) < c T x(µ) and b T y(µ ) > b T y(µ). iii) (x(µ), s(µ)) converges to an optimal solution pair for (LP) and (LD). Moreover, the limit point x(0) P is the analytic center on the primal optimal face, and the limit point s(0) Z is the analytic center on the dual optimal face, where (P, Z ) is the strict complementarity partition of the index set {1, 2,..., n}. Proof. Note that (x(µ 0 ) x(µ)) T (s(µ 0 ) s(µ)) = 0, since (x(µ 0 ) x(µ)) N (A) and (s(µ 0 ) s(µ)) R(A T ). This can be rewritten as or ( s(µ 0 ) j x(µ) j + x(µ 0 ) ) j s(µ) j = n(µ 0 + µ) 2nµ 0, j j ( x(µ)j x(µ 0 + s(µ) ) j ) j s(µ 0 2n. ) j Thus, x(µ) and s(µ) are bounded, which proves (i). We leave the proof of (ii) as an exercise. Since x(µ) and s(µ) are both bounded, they have at least one limit point which we denote by x(0) and s(0). Let x P (x Z = 0) and s Z (s P = 0), respectively, be the unique analytic centers on the primal and dual optimal faces: {x P : A P x P = b, x P 0} and {s Z : s Z = c Z A T Z y 0, c P A T P y = 0}. Again, we have ( s j x(µ) j + x ) j s(µ) j = nµ, j or ( x j x(µ) j j P ) + j Z ( s ) j = n. s(µ) j

2.3. CENTRAL PATHS OF LP AND SDP 45 Thus, we have and This implies that and Furthermore, or j P x j x(µ) j x j /n > 0, j P s(µ) j s j /n > 0, j Z. j P x(µ) j 0, j Z s(µ) j 0, j P. x j s j 1 x(µ) j s(µ) j j Z s j j Z x(µ) j s(µ) j. j P j Z However, ( j P x j )( j Z s j ) is the maximal value of the potential function over all interior point pairs on the optimal faces, and x(0) P and s(0) Z is one interior point pair on the optimal face. Thus, we must have = x(0) j s(0) j. j P j Z Therefore, j P x j j Z s j x(0) P = x P and s(0) Z = s Z, since x P and s Z are the unique maximizer pair of the potential function. This also implies that the limit point of the central path is unique. We usually define a neighborhood of the central path as { } N (η) = (x, y, s) F: Xs µe ηµ and µ = xt s, n where. can be any norm, or even a one-sided norm as We have the following theorem: x = min(0, min(x)).

46 CHAPTER 2. SEMIDEFINITE PROGRAMMING Theorem 2.9 Let (x, y, s) N (η) for constant 0 < η < 1. i) The N (η) {(x, y, s) : x T s nµ 0 } is bounded for any given µ 0 <. ii) Any limit point of N (η) as µ 0 is an optimal solution pair for (LP) and (LD). Moreover, for any j P x j (1 η)x j n where x is any optimal primal solution; for any j Z, s j (1 η)s j n where s is any optimal dual solution., 2.3.2 Central path for SDP Consider a SDP problem in Section 2.0.1 and Assume that F, i.e., both F p and F d. The central path can be expressed as { C = (X, y, S) F: } XS = µi, 0 < µ <, or a symmetric form { C = (X, y, S) F: } X.5 SX.5 = µi, 0 < µ <, where X.5 M n + is the square root matrix of X M n +, i.e., X.5 X.5 = X. We also see { C = (X, y, S) F: } ψ n (X, S) = n log n. When X and S are diagonal matrices, this definition is identical to LP. We also have the following corollary: Corollary 2.10 Let both (SDP) and (SDD) have interior feasible points. Then for any 0 < µ <, the central path point (X(µ), y(µ), S(µ)) exists and is unique. Moreover, i) the central path point (X(µ), S(µ)) is bounded where 0 < µ µ 0 for any given 0 < µ 0 <. ii) For 0 < µ < µ, C X(µ ) < C X(µ) and b T y(µ ) > b T y(µ).

2.4. NOTES 47 iii) (X(µ), S(µ)) converges to an optimal solution pair for (SDP) and (SDD), and the rank of the limit of X(µ) is maximal among all optimal solutions of (SDP) and the rank of the limit S(µ) is maximal among all optimal solutions of (SDD). 2.4 Notes General convex problems, such as membership, separation, validity, and optimization, can be solved by the central-section method; see Grötschel, Lovász and Schrijver [170]. Levin [244] and Newman [318] considered the center of gravity of a convex body; Elzinga and Moore [110] considered the center of the max-volume sphere contained in a convex body. A number of Russian mathematicians (for example, Tarasov, Khachiyan and Érlikh [403]) considered the center of the max-volume ellipsoid inscribing the body; Huard and Liêu [190, 191] considered a generic center in the body that maximizes a distance function; and Vaidya [438] considered the volumetric center, the maximizer of the determinant of the Hessian matrix of the logarithmic barrier function. See Kaiser, Morin and Trafalis [210] for a complete survey. Dyer and Frieze [104] proved that computing the volume of a convex polytope, either given as a list of facets or vertices, is itself #P -Hard. Furthermore, Elekes [109] has shown that no polynomial time algorithm can compute the volume of a convex body with less than exponential relative error. Bárány and Fürendi [42] further showed that for Ω R m, any polynomial time algorithm that gives an upper and lower bound on the volume of Ω, represented as V (Ω) and V (Ω), respectively, necessarily has an exponential gap between them. They showed V (Ω)/V (Ω) (cm/ log m) m, where c is a constant independent of m. In other words, there is no polynomial time algorithm that would compute V (Ω) and V (Ω) such that V (Ω)/V (Ω) < (cm/ log m) m. Recently, Dyer, Frieze and Kannan [105] developed a random polynomial time algorithm that can, with high probability, find a good estimate for the volume of Ω. Apparently, the result that every convex body contains a unique ellipsoid of maximal volume and is contained in a unique ellipsoid of minimal volume, was discovered independently by several mathematicians see, for example, Danzer, Grünbaum and Klee [94]. These authors attributed the first proof to K. Löwner. John [208] later proved the Theorem.

48 CHAPTER 2. SEMIDEFINITE PROGRAMMING Khachiyan and Todd [222] established a polynomial complexity bound for computing an approximate point of the center of the maximal inscribing ellipsoid if the convex body is represented by linear inequalities, The analytic center for a convex polyhedron given by linear inequalities was introduced by Huard [190], and later by Sonnevend [383]. The function d(y, Ω) is very similar to Huard s generic distance function, with one exception, where property (3) there was stated as If Ω Ω, then d(y, Ω ) d(y, Ω). The reason for the difference is that the distance function may return different values even if we have the same polytope but two different representations. The negative logarithmic function d(y, Ω), called the barrier function, was introduced by Frisch [126]. Todd [405] and Ye [465] showed that Karmarkar s potential function represents the logarithmic volume of a coordinate-aligned ellipsoid who contains the feasible region. The Karmarkar potential function in the standard form (LP) with a lower bound z for z was seen in Todd and Burrell [413], Anstreicher [24], Gay [133], and Ye and Kojima [477]. The primal potential function with ρ > 1 was proposed by Gonzaga [160], Freund [123], and Ye [466, 468]. The primal-dual potential function was proposed by Tanabe [400], and Todd and Ye [415]. Potential functions for LCP and SDP were studied by Kojima et al. [230, 228], Alizadeh [9], and Nesterov and Nemirovskii [327]. McLinden [267] earlier, then Bayer and Lagarias [45, 46], Megiddo [271], and Sonnevend [383], analyzed the central path for linear programming and convex optimization. Megiddo [271] derived the central path simply minimizing the primal with a logarithmic barrier function as in Fiacco and McCormick [116]. The central path for LCP, with more general matrix M, was given by Kojima et al. [227] and Güler [174]; the central path theory for SDP was first developed by Nesterov and Nemirovskii [327]. McLinden [267] proved Theorem 2.8 for the monotone LCP, which includes LP. 2.5 Exercises 2.1 Find the min-volume ellipsoid containing a half of the unit ball {x R n : x 1}. 2.2 Verify Examples 2.3, 2.4, and 2.5. 2.3 Compare and contrast the center of gravity of a polytope and its analytic center.

2.5. EXERCISES 49 2.4 Consider the maximum problem maximize f(x) = x e n x j s.t. e T x = n, x > 0 R n. Prove that its maximizer is achieved at x 1 = β and x 2 =... = x n = (n β)/(n 1) > 0 for some 1 < β < n. 2.5 Let Ω= {y R m : c A T y > 0} =, Ω = {y R m : c A T y > 0} =, and c c. Prove B(Ω ) B(Ω). 2.6 If Ω = {y : c A T y 0} is nonempty, prove the minimal value of its primal problem is 0; if Ω is bounded and has an interior, prove the interior of X Ω := {x R n : Ax = 0, x 0} is nonempty and x = 0 is the unique primal solution. 2.7 Let (LP) and (LD) have interior. Prove the dual potential function B n+1 (y, s, z), where z is a upper bound of z, represents the volume of a coordinate-aligned ellipsoid whose intersection with the affine set {x : Ax = b} contains the primal level set {x F p : c T x z}. 2.8 Let X, S M n be both positive definite. Then prove ψ n (X, S) = n log(x S) log(det(x) det(s)) n log n. 2.9 Consider linear programming and the level set Prove that Ψ(δ) := {(x, y, s) F: ψ n+ρ (x, s) δ}. Ψ(δ 1 ) Ψ(δ 2 ) if δ 1 δ 2, and for every δ Ψ(δ) is bounded and its closure ˆΨ(δ) has non-empty intersection with the solution set. 2.10 Consider the linear program max b T y s.t. 0 y 1 1, 0 y 2 1. Draw the feasible region the the (dual) potential level sets respectively, for {y : B 5 (y, s, 2) 0} and {y : B 5 (y, s, 2) 10},

50 CHAPTER 2. SEMIDEFINITE PROGRAMMING 1. b = (1; 0); 2. b = (1; 1)/2; 3. b = (2; 1)/3. 2.11 Consider the polytope {y R 2 : 0 y 1, 0 y 2 1, y 1 + y 2 z}. Describe how the analytic center changes as z decreases from 10 to 1. 2.12 Consider the linear program max b T y s.t. 0 y 1 1, 0 y 2 1. Draw the feasible region, central path, and solution set for 1. b = (1; 0); 2. b = (1; 1)/2; 3. b = (2; 1)/3. Finally, sketch a neighborhood for the third central path. 2.13 Prove (ii) of Theorem 2.8. 2.14 Prove Theorem 2.9.