Real Symmetric Matrices and Semidefinite Programming Tatsiana Maskalevich Abstract Symmetric real matrices attain an important property stating that all their eigenvalues are real. This gives rise to many important applications in areas across mathematical and earth sciences. In semidefinite programming, we try to optimize a linear function subject to the constraint that a combination of symmetric matrices is positive semidefinite. Semidefinite programming is used to model problems that arise in the areas of operational research and optimization. This paper gives a brief introduction to symmetric matrices, as well as theory and applications of semidefinite programming. 1 Introduction The beautiful properties of symmetric positive semidefinite matrices and their associated convex quadratic forms have fascinated mathematicians since the discovery of conic sections. Many properties of nonlinear objects have therefore been related to the behavior of convex quadratic functions that are used in control theory or in combinatorial optimization [6]. The development of interior point methods for semidefinite programming in the late eighties made it possible to optimize over this set. That aroused much interest and lead to heavy activity in this field. In fact, semidefinite programming has become one of the basic modeling and optimization tools along with linear and quadratic programming. The article is organized as follows. In the Background section, we review some basic notions of symmetric matrices and fundamental properties of the cone of positive semidefinite matrices. Semidefinite programs and their characteristics are introduced in Section 3. The applications is one of the semidefinite programming will be discussed in the Section 4. 2 Background Let M be a square matrix with entries in a eld R. Define a transpose matrix of n m matrix M is the m n matrix M T defined as m T ij := m ji 1 i m 1 j n From this definition it is easy to observe that (M T ) T = M. When M is a square matrix, M and M T have the same size and we can compare them 1
Proposition 2.1 A square matrixm with real entries is symmetric if M T = M skew-symmetric if M T = M orthogonal if M T = M 1 A matrix M is diagonal if all its off-diagonal entries are zero. Moreover, M is diagonalisable if there is an invertible matrix B such that BMB 1 is diagonal. We say, matrix is unitary if Mx = x. Unitary matrices satisfy det(m) = ±1 since det(m T M) = (det(m)) 2 for every matrix M and M T M = MM T = I when M is unitary. Thus we have the following Lemma: Lemma 2.1 A matrix with real entries is orthogonal if and only if it is unitary. An eigenvector of an n n matrix M is a nonzero vector x such that Mx = λx. We call λ an eigenvalue of M [3]. Proposition 2.2 The eigenvalues of real symmetric matrices are real [1]. Sketch of the proof: If M is the symmetric matrix and λ is an eigenvalue of M. Take transpose of the Mx = λx to obtain x T M = λx T, thus λx T x = x T (Mx) = (xm)x = λx T x Obtain λ λ = 0 implies λ is real. Thus we are guarantee to have only real eigenvalues. Lemma 2.2 A is a real symmetric matrix if and only if A is orthogonally similar to a real-diagonal matrix D, i.e D = P T AP for some orthogonal P [1]. Since the symmetric structure of a matrix forces its eigenvalues to be real, what additional property will force all eigenvalues to be positive (or non- negative)? If A R n n is symmetric, then, as observed above, there is an orthogonal matrix P such λ 1 0... 0 that A = P DP T 0 λ 2... 0, where D =.... is real. If λ i 0 for each i, then D 0 0... λ n exists, so A = P DP T = P D DP T = B T B for B = DP T 2
and λ i > 0 for each i if and only if B is nonsingular. Conversely, if A can be factored as A = B T B, then all eigenvalues of A are nonnegative because for any eigenvector x, λ = xt Ax x T x = xt B T Bx x T x = Bx 2 2 bx 2 2 0 Moreover,if B is nonsingular,then λ > 0. In other words, a real-symmetric matrix A has nonnegative eigenvalues if and only if A can be factored as A = B T B, and all eigenvalues are positive if and only if B is nonsingular. Definition 1 A symmetric matrix A whose eigenvalues are positive is called positive definite, and when the eigenvalues are just nonnegative, A is said to be positive semidefinite. This leads us to the following proposition [5]: Proposition 2.3 For A in R n n the following statements are equivalent: 1. A is positive semidefinite 2. λ i 0, i = 1,.., n 3. There exists matrix B so that A = B T B and rank(b) = rank(a). 4. Let S n denote the space of real, symmetric n n matrices. The usual inner product on this space, denoted by,, is defined by A, B = Tr(A T B) = j a ijb ij. Then A, B 0 for all positive semidefinite B. The set of positive semidefinite matrices (denoted S n + is a full dimensional, closed pointed cone in R (n+1 2 ) Since the eigenvalues are the roots of the characteristic polynomial, they depend continuously on the matrix elements. To characterize these matrices we need the following definition [4] Definition 2 (Convex Cone) A convex cone is a subset of a vector space that is closed under linear combinations with positive coefficients. We call a set K a convex cone if and only if any nonnegative combination of elements from K remains in K. The set of all convex cones is a proper subset of all cones. Therefore, the set of positive definite matrices forms the interior of the cone. The boundary of this cone consists of the positive semidefinite matrices having at least one zero eigenvalue. 3
3 Semidefinite Programming Symmetric matrices play an important role in many areas of algebra and optimization. Let S n denote the space of real, symmetric n n matrices. If X S n we can consider a linear function C(X) defined by the inner product on this space,,, so that C, X = Tr(C T X) = ij C ijx ij. We assume that C is a symmetric matrix. We consider semidefinite programs (SDP) (optimization problem) in the following standard form: min C, X such that (3.1) A k, X = b k, k = 1,..., m; X 0 where C, A k for (k = 1, 2,..., m) and X are symmetric matrices, with b = {b 1,..., b m } is the m-vector that forms m linear equations. We say X 0 meaning X lies in the closed, convex cone of positive semidefinite matrices S+ n.[2]. In order to derive the dual of this program let A = Since AX, y = y i A i, X = y i A i, X for all X S n and y R n we get where AX = A 1, X. A n, X The dual of (3.1) is [2]: A T y = m y i A i i=1 A 1. A n. max b T y such that (3.2) m y k A k + Z = C; k=1 Z 0 where Z S n is a positive semidefinite dual variable. In other words for dual problem given y 1,..., y m we want to maximize linear function b T y. According to constraints of this semidefinite dual matrix Z, defined as Z = C m k=1 y ka k, has to be positive semidefinite. 4
4 Application of Semidefinite Programming A fundamental problem in real algebraic geometry is the whether representation of a multivariate polynomial as a sum of squares (SOS) exists and can be computed[7]. While algebraic techniques have been proposed to decide the problem, recent results suggest that this problem can be solved much more efficiently using numerical optimization techniques such as semidefinite programming. Let p(x) be multivariate polynomial. Then p(x) can be written as p(x) = α p α x α 1 1 xαn n = α p α xα Then the degree of p(x) is max deg(x α ) = n i=1 α i such that p α 0. p(x) is a SOS is m p(x) = (h j (x)) 2 for some polynomials h 1, h 2,..., h m j=1 Then degree of p(x) is even and deg(h j ) deg(p(x). 2 Example 1 x 2 + y 2 + 2xy + z 6 = (x + y) 2 + (z 3 ) is SOS. To solve whether representation of a multivariate polynomial as a sum of squares (SOS) exists we can consider the following semidefinite program f(x) = { X 0 β, γ d β+γ=α X β,γ = p α ( α 2d) where X symmetrix matrix of order ( ) ( n+d d n+d ) ( d with n+d2 ) 2d equations. Example 2 Let p(x, y) = 2x 4 + 2x 3 y x 2 y 2 + 5y 4. Is it SOS? If so that there exists decomposition of the following form: p(x, y) = [ x 2 y 2 xy ] a b c b d e c e f } {{ } X 0 Multiply out and compute coefficients of the monomial degree x 4 = x 2 x 2 = a = 2 x 3 y = x 2 xy = 2c = 2 x 2 y 2 = (xy)(xy) = 2b + f = 1 y 3 = y 2 x = 2e = 0 y 4 = y 2 y 2 = d = 5 5 x 2 y 2 xy
Hence X = 2 b 1 b 5 0 1 0 1 2b for b = 3, X 0, X = 1 2 Thus SOS decomposition is 2 0 3 1 1 3 [ 2 3 1 0 1 3 p(x) = 1 2 (2x2 3y 2 + xy) 2 + 1 2 (y2 + 3xy) 2 ] 6
References [1] Leslie Hogben Handbook of Linear Algebra, Chapman & Hall/CRC, Taylor and Francis Group, 2007. [2] Etienne de Klerk Aspects of Semidefinite Programming, Kluwer Academic Publishers, 2002. [3] Denis Serre Matrices: Theory and Applications Springer Science Business Media, 20104 [4] Dattorro, Convex Optimization & Euclidean Distance Geometry, M??oo, 2005, v2010.10.26. [5] Thomas S. Shores Applied Linear Algebra and Matrix Analysis, Springer Science Business Media, 2007 [6] L. Vandenberghe and S. Boyd Semidefinite Programming, SIAM Review, 38(1): 49-95, March 1996. [7] Karin Gatermann and Pablo A. Parrilo Symmetry groups, semidefinite programs, and sums of squares, arxiv:math/0211450v1, 2002. 7