L. Vandenberghe EE236C (Spring 2016) 18. Symmetric cones definition spectral decomposition quadratic representation log-det barrier 18-1
Introduction This lecture: theoretical properties of the following cones nonnegative orthant R p + = {x R p x k 0, k = 1,..., p} second-order cone Q p = {(x 0, x 1 ) R R p 1 x 1 2 x 0 } positive semidefinite cone S p = {x R p(p+1)/2 mat(x) 0} these cones are not only self-dual, but symmetric (also known as self-scaled) Symmetric cones 18-2
Outline definition spectral decomposition quadratic representation log-det barrier
Cones of squares the three basic cones can be expressed as cones of squares x 2 = x x for appropriately defined vector products x y nonnegative orthant: componentwise product x y = diag(x)y second-order cone: the product of x = (x 0, x 1 ) and y = (y 0, y 1 ) is x y = 1 2 [ x T y x 0 y 1 + y 0 x 1 ] positive semidefinite cone: symmetrized matrix product x y = 1 2 vec(xy + Y X) with X = mat(x), Y = mat(y ) Symmetric cones 18-3
Symmetric cones the vector products satisfy the following properties 1. x y is bilinear (linear in x for fixed y and vice-versa) 2. x y = y x 3. x 2 (y x) = (x 2 y) x 4. x T (y z) = (x y) T z except for the componentwise product, the products are not associative: x (y z) (x y) z in general Definition: a cone is symmetric if it is the cone of squares {x 2 = x x x R n } for a product x y that satisfies these four properties Symmetric cones 18-4
Classification symmetric cones are studied in the theory of Euclidean Jordan algebras all possible symmetric cones have been characterized List of symmetric cones the second-order cone the p.s.d. cone of Hermitian matrices with real, complex, or quaternion entries 3 3 positive semidefinite matrices with octonion entries Cartesian products of these primitive symmetric cones (such as R p + ) Practical implication can focus on Q p, S p and study these cones using elementary linear algebra Symmetric cones 18-5
Outline definition spectral decomposition quadratic representation log-det barrier
Vector product with each symmetric cone K we associate a bilinear vector product for R p +, Qp, S p we use the products on page 18-3 for a cone K = K 1 K N, with K i of one of the three basic types, (x 1,..., x N ) (y 1,..., y N ) = (x 1 y 1,..., x N y N ) we refer to the product associated with the cone K as the product for K Symmetric cones 18-6
Identity element Identity element: the element e that satisfies e x = x e = x for all x product for R p + : e = 1 = (1, 1,..., 1) product for Q p : e = ( 2, 0,..., 0) product for S p : e = vec(i) product for K 1 K N : the product of the N identity elements note we use the same symbol e for the identity element for each product Rank of the cone: θ = e T e is called the rank of K θ = p (K = R p +), θ = 2 (K = Q p ), θ = p (K = S p ) and θ = N θ i if K = K 1 K N and θ i is the rank of K i Symmetric cones 18-7
Spectral decomposition with each symmetric cone/product we associate a spectral decomposition x = θ λ i q i λ i are the eigenvalues of x; the eigenvectors q i satisfy q 2 i = q i, q i q j = 0 (i j), θ q i = e theory can be developed from properties of the vector product on page 18-4 we will define the decomposition by enumerating the symmetric cones Symmetric cones 18-8
Spectral decomposition for primitive cones Positive semidefinite cone (K = S p ) spectral decomposition of x follows from eigendecomposition of mat(x): mat(x) = p λ i v i vi T, q i = vec(v i vi T ) Second-order cone (K = Q p ) spectral decomposition of x = (x 0, x 1 ) R R p 1 is λ i = x 0 ± x 1 2 2, q i = 1 2 [ 1 ±y ], i = 1, 2 y = x 1 / x 1 2 if x 1 0, and y is an arbitrary unit-norm vector otherwise Symmetric cones 18-9
Spectral decomposition for composite cones Product cone (K = K 1 K N ) spectral decomposition follows from decomposition of different blocks example (K = K 1 K 2 ): decomposition of x = (x 1, x 2 ) is where x j = θ j [ x1 x 2 ] = θ 1 λ 1i [ q1i 0 ] + θ 2 λ 2i [ 0 q 2i ] λ ji q ji is the spectral decomposition of x j, j = 1, 2 Nonnegative orthant (K = R p + ) λ i = x i, q i = e i (ith unit vector), i = 1,..., n Symmetric cones 18-10
Some properties the eigenvectors are normalized ( q i 2 = 1) and nonnegative (q i K) eigenvectors are orthogonal: q T i q j = 0 for i j can be verified from the definitions, or from the properties on page 18-4 q T i q j = q T i (q j q j ) = (q i q j ) T q j = 0 e T q i = ( j q j) T q i = q T i q i = 1 e T x = θ λ i x K if and only λ i 0 for i = 1,..., θ x int K if and only λ i > 0 for i = 1,..., θ Symmetric cones 18-11
Trace, determinant, and norm tr x = θ λ i, det x = θ θ λ i, x F = ( λ 2 i ) 1/2 positive semidefinite cone (K = S p ) tr x = tr(mat(x)), det x = det(mat(x)), x F = mat(x) F second-order cone (K = Q p ) tr x = 2x 0, det x = 1 2 (x2 0 x T 1 x 1 ), x F = x 2 nonnegative orthant (K = R p + ) tr x = p x i, det x = p x i, x F = x 2 Symmetric cones 18-12
Powers and inverse powers of x can be defined in terms of the spectral decomposition x α = i λ α i q i (exists if λ α i is defined for all i) x α x β = x α+β x α x β = ( θ ) λ α i q i ( θ ) λ β i q i = θ λ α+β i q i = x α+β x is invertible if all λ i 0 (i.e., det x 0) inverse x 1 = θ λ 1 i q i satisfies x x 1 = x 1 x = e Symmetric cones 18-13
Expressions for inverse for invertible x (i.e., λ i 0 for i = 1,..., θ) nonnegative orthant (K = R p + ) x 1 = ( 1 x 1, 1 x 2,..., 1 x p ) second-order cone (K = Q p ) x 1 = 2 x T Jx Jx, J = [ 1 0 0 I ] semidefinite cone (K = S p ) x 1 = vec ( mat(x) 1) Symmetric cones 18-14
Expressions for square root for nonnegative x (i.e., λ i 0 for i = 1,..., θ) nonnegative orthant (K = R p + ) x 1/2 = ( x1, x 2,..., x p ) second-order cone (K = Q p ) [ x 1/2 1 x0 = 2 (x 1/4 0 + + x T Jx ) 1/2 x T x 1 Jx ] semidefinite cone (K = S p ) x 1/2 = vec (mat(x) 1/2) Symmetric cones 18-15
Outline definition spectral decomposition quadratic representation log-det barrier
Matrix representation of product since the product is bilinear, it can be expressed as x y = L(x)y L(x) is a symmetric matrix, linearly dependent on x nonnegative orthant: L(x) = diag(x) second-order cone L(x) = 1 [ x0 x T 1 2 x 0 I x 1 ] semidefinite cone: the matrix defined by L(x)y = 1 (XY + Y X), X = mat(x), Y = mat(y) 2 Symmetric cones 18-16
Quadratic representation the quadratic representation of x is the matrix P (x) = 2L(x) 2 L(x 2 ) (terminology is motivated by the property P (x)e = x 2 ) nonnegative orthant P (x) = diag(x) 2 second-order cone P (x) = xx T xt Jx J, J = 2 [ 1 0 0 I ] positive semidefinite cone P (x)y = vec(xy X) where X = mat(x), Y = mat(y) Symmetric cones 18-17
Some useful properties Powers P (x) α = P (x α ) if x α exists P (x)x 1 = x if x is invertible Derivative of x 1 : for x invertible, P (x) 1 is the derivative of x 1, i.e., d (x + αy) 1 dα = P (x) 1 y α=0 Symmetric cones 18-18
Scaling with P (x) affine transformations y P (x)y have important properties if x is invertible, then P (x)k = K, P (x) int K = int K multiplication with P (x) preserves the conic inequalities if x and y are invertible, then P (x)y is invertible with inverse (P (x)y) 1 = P (x 1 )y 1 = P (x) 1 y 1 quadratic representation of P (x)y P (P (x)y) = P (x)p (y)p (x) hence also det (P (x)y) = (det x) 2 det y Symmetric cones 18-19
Distance to cone boundary for x 0 define σ x (y) = 0 if y 0 and ) σ x (y) = λ min (P (x 1/2 )y otherwise σ x (y) characterizes distance of x to the boundary of K in the direction y: x + αy 0 ασ x (y) 1 Proof: from the definition of P on page 18-17: P (x 1/2 )e = x therefore, with v = P (x 1/2 )y x + αy 0 P (x 1/2 )(e + αv) 0 e + αv 0 1 + αλ i (v) 0 Symmetric cones 18-20
Scaling point for a pair s, z 0, the point w = P (z 1/2 ) ( ) 1/2 P (z 1/2 )s satisfies w 0 and s = P (w)z the linear transformation P (w) preserves the cone and maps z to s equivalently, v = w 1/2 defines a scaling P (v) = P (w) 1/2 that satisfies P (v) 1 s = P (v)z Symmetric cones 18-21
Proof: we use the properties P (x)e = x 2, P (P (x)y) = P (x)p (y)p (x) if we define u = P (z 1/2 )s, we can write P (w) as P (w) = P (z 1/2 )P (u 1/2 )P (z 1/2 ) therefore P (w)z = P (z 1/2 )P (u 1/2 )e = P (z 1/2 )u = s Symmetric cones 18-22
Scaling point for nonnegative orthant Scaling point w = ( s1 /z 1, s 2 /z 2,..., s p /z p ) Scaling transformation: a positive diagonal scaling P (w) = s 1 /z 1 0 0 0 s 2 /z 2 0...... 0 0 s p /z p Symmetric cones 18-23
Scaling point for positive semidefinite cone Scaling point: w = vec(rr T ) R simultaneously diagonalizes mat(z) and mat(s) 1 : R T mat(z)r = R 1 mat(s)r T = Σ can be computed from two Cholesky factorizations and an SVD: if mat(s) = L 1 L T 1, mat(z) = L 2 L T 2, L T 2 L 1 = UΣV T then R = L 1 V Σ 1/2 = L 2 UΣ 1/2 Scaling transformation: a congruence transformation P (w)y = RR T mat(y)rr T Symmetric cones 18-24
Outline definition spectral decomposition quadratic representation log-det barrier
Log-det barrier Definition: the log-det barrier of a symmetric cone K is φ(x) = log det x = θ log λ i, dom φ = int K φ is logarithmically homogeneous with degree θ (i.e., the rank of K) nonnegative orthant (K = R p + ): φ(x) = p log x i second-order cone (K = Q p ): φ(x) = log(x 2 0 x T 1 x 1 ) + log 2 semidefinite cone (K = S p ): φ(x) = log det(mat x) composition K = K 1 K N : sum of the log-det barriers Symmetric cones 18-25
Convexity φ is a convex function Proof: consider arbitrary y and let λ i be the eigenvalues of v = P (x 1/2 )y the restriction g(α) = φ(x + αy) of φ to the line x + αy is g(α) = log det(x + αy) ( ) = log det P (x 1/2 )(e + αv) = log det x log det(e + αv) θ = log det x log(1 + αλ i ) (line 3 follows from page 18-18 and page 18-21) hence restriction of φ to arbitrary line is convex Symmetric cones 18-26
Gradient and Hessian the gradient and Hessian of φ at a point x 0 are φ(x) = x 1, 2 φ(x) = P (x) 1 = P (x 1 ) Proof: continues from last page φ(x) T y = g (0) = θ λ i = tr(p (x 1/2 )y) = e T P (x 1/2 )y since this holds for all y, φ(x) = P (x 1/2 )e = x 1 the expression for the Hessian follows from page 18-18 Symmetric cones 18-27
Dikin ellipsoid theorem for symmetric cones recall the definition of the Dikin ellipsoid at x 0: E x = {x + y y T 2 φ(x)y 1} = {x + y y T P (x) 1 y 1} x + y E x if and only if the eigenvalues λ i of P (x 1/2 )y satisfy θ λ 2 i 1 for symmetric cones the Dikin ellipsoid theorem E x K follows from θ λ 2 i 1 = min i λ i 1 therefore x + y E x implies σ x (y) 1 and x + y 0 Symmetric cones 18-28
Generalized convexity of inverse for all w 0, the function f(x) = w T x 1, dom f = int K is convex Proof: restriction g(α) = f(x + αy) of f to a line is ( ) 1 g(α) = w T (x + αy) 1 = w T P (x 1/2 )(e + αv) = w T P (x 1/2 )(e + αv) 1 = θ i 1 w T P (x 1/2 )q i 1 + αλ i λ i, q i are eigenvalues and eigenvectors of v = P (x 1/2 )y g(α) is convex because P (x 1/2 )K = K; therefore w T P (x 1/2 )q i 0 Symmetric cones 18-29
Self-concordance for all x 0 and all y, d dα 2 φ(x + αy) 2σ x (y) 2 φ(x) α=0 from page 18-20, with v = P (x 1/2 )y and λ i = λ i (v) θ σ x (y) = min{0, λ min } ( λ 2 i ) 1/2 = y x hence, inequality implies self-concordance inequality on page 16-2 this shows that the log-det barrier φ is a θ-normal barrier Symmetric cones 18-30
Proof: choose any σ > σ x (y); by definition of σ x we have σx + y 0 d dα 2 φ(x + αy) = d dα 2 φ(x ασx) + d dα 2 φ(x + α(σx + y)) from page 16-24, the first term on the r.h.s. (evaluated at α = 0) is σ d dt 2 φ(tx) = 2σ 2 φ(x) t=1 2nd term is 2 g(x) where g(u) = w T φ(u) and w = σx + y: g(u) = d φ(u + αw) dα, α=0 2 g(u) = d dα 2 φ(u + αw) α=0 2 g(u) 0 because from page 18-29, g(u) = w T u 1 is concave Symmetric cones 18-31
References J. Faraut and A. Korányi, Analysis on Symmetric Cones (1994). Yu. E. Nesterov and M. J. Todd, Self-scaled barriers and interior-point methods for convex programming, Mathematics of Operations Research (1997). L. Faybusovich, Euclidean Jordan algebras and interior-point algorithms, Positivity (1997). J. F. Sturm, Similarity and other spectral relations for symmetric cones, Linear Algebra and Its Applications (2000). S. H. Schmieta and F. Alizadeh, Extension of primal-dual interior point algorithms to symmetric cones, Math. Prog. (2003). F. Alizadeh and D. Goldfarb, Second-order cone programming, Math. Prog. (2003). Symmetric cones 18-32