COURSE ON LMI PART I.2 GEOMETRY OF LMI SETS Didier HENRION www.laas.fr/ henrion October 2006
Geometry of LMI sets Given symmetric matrices F i we want to characterize the shape in R n of the LMI set F = {x R n : F (x) = F 0 + n i=1 x i F i 0} Matrix F (x) is PSD iff its diagonal minors f i (x) are nonnegative Diagonal minors are multivariate polynomials of indeterminates x i So the LMI set can be described as F = {x R n : f i (x) 0, i = 1, 2,...} which is a basic semialgebraic set (basic = intersection of polynomial level-sets) Moreover, it is a convex set
Semialgebraic characterisation of positive semidefiniteness For an m-by-m matrix F (x) we have 2 m 1 diagonal minors A simpler and equivalent criterion follows from the fact that a polynomial t f(t) = k f m k t k = k (t t k ) which has only real roots satisfies t k 0 iff f k 0 Apply to characteristic polynomial t f(t, x) = det(ti m + F (x)) = which is monic, i.e. f 0 (x) = 1 m k=0 Only m polynomial inequalities f k (x) 0 to be checked Polynomials f k (x) are (signed) sums of principal minors of F (x) of order k f m k (x)t k
Example of 2D LMI feasible set F (x) = 1 x 1 x 1 + x 2 x 1 x 1 + x 2 2 x 2 0 x 1 0 1 + x 2 0 System of 3 polynomial inequalities f i (x) 0 1st order minors: f 1 (x) = 4 x 1 0
2nd order minors: f 2 (x) = 5 3x 1 + x 2 2x 2 1 2x 1x 2 2x 2 2 0 4 3 2 1 x 2 0 1 2 3 4 4 3 2 1 0 1 2 3 4 x 1
3rd order minors: f 3 (x) = 2 2x 1 + x 2 3x 2 1 3x 1x 2 2x 2 2 x 1x 2 2 x3 2 0 4 3 2 1 x 2 0 1 2 3 4 4 3 2 1 0 1 2 3 4 x 1
LMI feasible set = intersection of semialgebraic sets f k (x) 0, k = 1, 2, 3 4 3 2 1 x 2 0 1 2 3 4 4 3 2 1 0 1 2 3 4 x 1 Boundary of LMI region = determinant other polynomials only isolate component
Example of 3D LMI feasible set LMI set F = {x R 3 : 1 x 1 x 2 x 1 1 x 3 x 2 x 3 1 0} arising in relaxation of MAXCUT Semialgebraic set F = {x R 3 : 1 + 2x 1 x 2 x 3 (x 2 1 + x2 2 + x2 3 ) 0, 3 x 2 1 x2 2 x2 3 0}
Intersection of LMI sets Intersection of LMI feasible sets F (x) 0 x 1 2 2x 1 + x 2 0 is also an LMI F (x) 0 0 0 x 1 + 2 0 0 0 2x 1 x 2 0
LMI representability and liftings LMI sets are convex basic semialgebraic sets.. but are all convex basic semialgebraic sets representable by LMIs? To address this question we make a fundamental distinction between LMI representable sets lifted-lmi representable sets We say that a convex set X R n is LMI representable if there exists an affine mapping F (x) such that x X F (x) 0 We say that a convex set X R n is lifted-lmi representable if there exists an affine mapping F (x, u) such that x X u R m : F (x, u) 0
LMI and lifted-lmi representability A set X is lifted-lmi representable when x X u : F (x, u) 0 i.e. when it is the projection of the solution set of the LMI F (x, u) 0 onto the x-space and u are additional, or lifting variables In other words, lifting variables u are not allowed in LMI representations Similarly, a convex function f : R n R is LMI (or lifted-lmi) representable if its epigraph {x, t : f(x) t} is an LMI (or lifted-lmi) representable set
Conic quadratic forms The Lorentz, or ice-cream cone {x, t R n R : x 2 t} is LMI representable as { [ tin x ] } x, t R n R : x T t 0 As a result, all second-order conic sets are LMI representable In the sequel we give a list of LMI and lifted-lmi representable sets (following Ben-Tal, Nemirovski, Nesterov)
Quadratic forms The Euclidean norm {x, t R n R : x 2 t} is LMI representable (see previous slide) The squared Euclidean norm {x, t R n R : x T x t} is also LMI representable as [ t x T x I n ] 0
Quadratic forms (2) More generally, the convex quadratic set {x R n : x T Ax + b T x + c 0} with A = A T 0 is LMI representable as [ b T x c x T D T Dx I n ] 0 where D is the Cholesky factor of A = D T D
Who is Cholesky? André Louis Cholesky (1875-1918) was a French military officer (graduated from Ecole Polytechnique) involved in geodesy He proposed a new procedure for solving least-squares triangulation problems He fell for his country during World War I Work posthumously published in Commandant Benoît. Procédé du Commandant Cholesky. Bulletin Géodésique, No. 2, pp. 67-77, Toulouse, Privat, 1924. Nice biography in C. Brezinski. André Louis Cholesky. Bulletin of the Belgian Mathematical Society, Vol. 3, pp. 45-50, 1996.
Hyperbola The branch of hyperbola {x, y R 2 : x 0, xy 1} is LMI representable as [ x 1 1 y ] 0
Geometric mean of two variables The hypograph of the geometric mean of 2 variables {x 1, x 2, t R 3 : x 1, x 2 0, x 1 x 2 t} is LMI representable as [ x1 t t x 2 ] 0
Geometric mean of several variables The hypograph of the geometric mean of 2 k variables {x 1,..., x 2 k, t R 2k +1 : x i 0, (x 1 x 2 k) 1/2k t} is lifted-lmi representable Proof: iterate the previous construction by introducing lifting variables Example with k = 3: x1 x 2 x 11 x3 x 4 x 12 x5 x 6 x 13 x7 x 8 x 14 (x 1 x 2 x 8 ) 1/8 t } x11 x 12 x 21 x21 x x13 x 14 x 22 t 22 Useful idea in other LMI representability problems
Rational power functions Using similar ideas, we can show that the increasing rational power functions f(x) = x p/q, x 0 with rational p/q 1, as well as the decreasing g(x) = x p/q, x 0 with rational p/q 0, are both lifted-lmi representable
Rational power functions Example: {x, t : x 0, x 7/3 t} Start from lifted-lmi representable and replace to get ˆt (ˆx 1 ˆx 8 ) 1/8 ˆt = ˆx 1 = x 0 ˆx 2 = ˆx 3 = ˆx 4 = t 0 ˆx 5 = ˆx 6 = ˆx 7 = ˆx 8 = 1 x x 1/8 t 3/8 x 7/8 t 3/8 x 7/3 t Same idea works for any rational p/q 1 lift = use additional variables, and project in the space of original variables
Even power monomial The epigraph of even power monomial F = {x, t : x 2p t} where p is a positive integer is lifted-lmi representable Note that {x, t : x 2p t} {x, y, t : x 2 y} {x, y, t : y 0, y p t} both lifted-lmi representable Use lifting y and project back onto x, t Similarly, even power polynomials are lifted- LMI representable (several monomials)
Quartic level set Model quartic level set as F = {x, t : x 4 t} F = {x, t : y : y x 2, t y 2, y 0} LMI in x, t and y [ 1 x x y ] 0 [ 1 y y t ] 0 It can be shown that it is impossible to remove the lifting variable y while keeping a (finitedimensional) LMI formulation
Quartic level set (2) F = {x, t : x 4 t}
Largest eigenvalue The epigraph of the function largest eigenvalue of a symmetric matrix {X = X T R n n, t R : λ max (X) t} is LMI representable as X ti n Eigenvalues of matrix [ ] 1 x1 x 1 x 2
Sums of largest eigenvalues Let S k (X) = k i=1 λ i (X), k = 1,..., n denote the sum of the k largest eigenvalues of an n-by-n symmetric matrix X The epigraph {X = X T R n n, t R : S k (X) t} is lifted-lmi representable as t ks trace Z 0 Z 0 Z X + si n 0 where Z and s are liftings
The determinant Determinant of a PSD matrix det(x) = n λ i (X) i=1 is not a convex function of X, but the function f q (X) = det q (X), X = X T 0 is convex when q [0, 1/n] is rational The epigraph {X = X T R n n, t R : f q (X) t} is lifted-lmi representable as [ ] X T 0 diag t (δ 1 δ n ) q since we know that the latter constraint (hypograph of a concave monomial) is lifted-lmi representable Here is a lower triangular matrix of liftings with diagonal entries δ i
Application: extremal ellipsoids A little excursion in the world of ellipsoids and polytopes.. Crystal structure Various representations of an ellipsoid in R n E = {x R n : x T P x + 2x T q + r 0} = {x R n : (x x c ) T P (x x c ) 1} = {x = Qy + x c R n : y T y 1} = {x R n : Rx x c 1} where Q = R 1 = P 1/2 0
Ellipsoid volume Volume of ellipsoid E = {Qy + x c : y T y 1} vol E = k n det Q where k n is volume of n-dimensional unit ball k n = 2 (n+1)/2 π (n 1)/2 n(n 2)!! 2π n/2 n(n/2 1)! for n odd for n even n 1 2 3 4 5 6 7 8 k n 2.00 3.14 4.19 4.93 5.26 5.17 4.72 4.06 Unit ball has maximum volume for n = 5
Outer and inner ellipsoidal approximations Let S R n be a solid = a closed bounded convex set with nonempty interior the largest volume ellipsoid E in contained in S is unique and satisfies E in S ne in the smallest volume ellipsoid E out containing S is unique and satisfies E out /n S E out These are Löwner-John ellipsoids Factor n reduces to n if S is symmetric How can these ellipsoids be computed?
Ellipsoid in polytope Let the intersection of hyperplanes S = {x R n : a T i x b i, i = 1,..., m} describe a polytope = bounded nonempty polyhedron The largest volume ellipsoid contained in S is E = {Qy + x c : y T y 1} where Q, x c are optimal solutions of the LMI problem max det 1/n Q Q 0 Qa i 2 b i a T i x c, i = 1,..., m
Polytope in ellipsoid Let the convex hull of vertices describe a polytope S = conv {x 1,..., x m } The smallest volume ellipsoid containing S is E = {x : (x x c ) T P (x x c ) 1} where P, x c = P 1 q are optimal solutions of the LMI problem max t t[ det 1/n ] P P q q T 0 r x T i P x i + 2x T i q + r 1, i = 1,..., m
Sums of largest singular values Let Σ k (X) = k i=1 σ i (X), k = 1,..., n denote the sum of the k largest singular values of an n-by-m matrix X Then the epigraph {X R n m, t R : Σ k (X) t} is lifted-lmi representable since X T ([ 0 σ i (X) = λ i X 0 ]) and the sum of largest eigenvalues of a symmetric matrix is lifted-lmi representable
Positive polynomials The set of univariate polynomials that are positive on the real axis is lifted-lmi representable in the coefficient space Can be proved with cone duality (Nesterov) or with theory of moments (Lasserre) - more on that later The even polynomial p(s) = p 0 + p 1 s + + p 2n s 2n satisfies p(s) 0 for all s R if and only if p k = i+j=k X ij, k = 0, 1,..., 2n = trace H k X for some lifting matrix X = X T 0
Sum-of-squares decomposition The expression of p k with Hankel matrices H k comes from p(s) = [1 s s n ]X[1 s s n ] hence X 0 naturally implies p(s) 0 Conversely, existence of X for any polynomial p(s) 0 follows from the existence of a sumof-squares (SOS) decomposition (with at most two elements) of p(s) = k q 2 k (s) 0 Matrix X has entries X ij = k q ki q kj Seeking the lifting matrix amounts to seeking an SOS decomposition
Primal and dual formulations Global minimization of polynomial p(s) = n k=0 p k s k Global optimum p : maximum value of ˆp such that p(s) ˆp stays globally nonnegative Primal LMI problem max ˆp = p 0 trace H 0 X s.t. trace H k X = p k, k = 1,..., n X 0 Dual LMI problem min p 0 + n k=1 p k y k s.t. H 0 + n k=1 H k y k 0 with Hankel structure (moment matrix)
Positive polynomials and LMIs Example: Global minimization of the polynomial p(s) = 48 92s + 56s 2 13s 3 + s 4 We just have to solve the dual LMI problem min 48 92y 1 + 56y 2 13y 3 + y 4 s.t. 1 y 1 y 2 y 1 y 2 y 3 0 y 2 y 3 y 4 to obtain p = p(5.25) = 12.89
Complex LMIs The complex valued LMI F (x) = A(x) + jb(x) 0 is equivalent to the real valued LMI [ A(x) B(x) B(x) A(x) ] 0 If there is a complex solution to the LMI then there is a real solution to the same LMI Note that matrix A(x) = A T (x) is symmetric whereas B(x) = B T (x) is skew-symmetric