Continuous Optimisation, Chpt 9: Semidefinite Problems Peter J.C. Dickinson DMMP, University of Twente p.j.c.dickinson@utwente.nl http://dickinson.website/teaching/2016co.html version: 21/11/16 Monday 21st November 2016
Book Semidefinite Optimization M.J. Todd http://people.orie.cornell.edu/miketodd/soa5.ps Primal Problem: Chpt 2: Problems Dual problems: Chpt 4: Duality Examples: Chpt 3: Examples
Table of Contents 1 Introduction 2 Symmetric Matrices Inner Product Nonnegative symmetric matrices Primal and Dual Problem 3 Positive Semidefinite Cone
Inner Product For a vector space V, we say that, : V V R is an inner product if the following properties hold: 1 Symmetry: x, y = y, x for all x, y V, 2 Linearity 1: λx, y = λ x, y for all x, y V, λ R, 3 Linearity 2: x + y, z = x, z + y, z for all x, y, z V, 4 Positive definiteness: x, x > 0 for all x V \ {0}. Example For vector space R n and a matrix A S n, A O have an (induced) inner product x, y A = x T Ay. Usually consider the standard inner product x, y = x T y = x, y I. Example For B = {x R n : x 2 1}, we can consider the space of continuous functions from B to R, which has an inner product f, g = B f (x)g(x)dx.
Primal and Dual Problem Consider convex cone K V and c, a 1,..., a m V and b R m : min x c, x s. t. a i, x = b i for all i = 1,..., m (P) x K, max y b T y s. t. c m y i a i K i=1 y R m. (D) K := {z V x, z 0 for all x K}. Ex. 9.1 Show that x feas(p), y feas(d) c, x b T y.
Symmetric Matrices Definition 9.1 For space of symmetric matrices, S n, define inner product A, B := trace(ab) = n i,j=1 a ijb ij. The definitions/results from the previous two lectures can be naturally extended for this, noting that S n is a space of dimension n(n + 1). 1 2 Lemma 9.2 For A R n m and B R m n have trace(ab) = trace(ba). Lemma 9.3 Have ab T + ba T, X = 2a T Xb for all a, b R n, X S n.
Nonnegative symmetric matrices Cone of Nonnegative symmetric matrices: N n := {X S n x ij 0 for all i, j} = conic{e i e T j + e j e T i i, j}. (N n ) = {e i e T j + e j e T i i, j} = {Y S n e i e T j + e j e T i, Y 0 for all i, j} = {Y S n y ji + y ij 0 for all i, j} = N n. Closed convex cone as intersection of closed convex cones. Pointed as ±X N n ±x ij 0 for all i, j X = O. Full-dimensional as dual to a pointed convex cone. Denote A B iff A B N n.
Primal and Dual Problem Proper cone K S n and C, A 1,..., A m S n and b R m : min X C, X s. t. A i, X = b i for all i = 1,..., m (P) X K, max b T y y s. t. C m i=1 y ia i K (D) y R m. K := {Z S n X, Z 0 for all X K}. (P) and (D) are dual problems to each other. Slater s condition Strong duality.
Table of Contents 1 Introduction 2 Symmetric Matrices 3 Positive Semidefinite Cone Definition Correlation matrices Eigenvalue problems Approximating Quadratic Problems
Positive semidefinite cone Positive Semidefinite cone, PSD n := {X S n v T Xv 0 v R n } = conic{bb T b R n }. Ex. 9.2 Prove that (PSD n ) = PSD n. Ex. 9.3 Prove that PSD n is a proper cone. Denote A B iff A B PSD n. Solvers Commercial: MOSEK. Free: SDPT3, SEDUMI
Correlation matrices Definition 9.4 For two random variables [( X)( 1, X 2, define their correlation to be corr(x 1, X 2 ) := E X1 µ 1 X2 µ 2 σ 1 σ 2 )]. For vector of random variables X = ( ) T X 1 X n define correlation matrix corr(x ) S n s.t. [corr(x )] ij = corr(x i, X j ). Theorem 9.5 For A R n n, there exists a distribution such that A = corr(x ) iff A PSD n and a ii = 1 for all i.
Example What is the maximum possible corr(x 1, X 2 ) given corr(x 1, X 3 ) = 0.6 and corr(x 2, X 3 ) = 0? max y 1 1 y 1 0.6 s. t. y 1 1 0 0 0.6 0 1 Ex. 9.4 What is the dual problem to the example above? Ex. 9.5 Formulate as a semidefinite problem, the problem of finding the minimum possible corr(x 1, X 2 ) given 0.5 corr(x 1, X 3 ) 0.6 and 0.1 corr(x 2, X 3 ) 0.
Eigenvalue problems Lemma 9.6 For A S n and s, t R {± } with s t, all the eigenvalues of A are between s and t if and only if si A ti. (Where I S n is the identity matrix.) Example Find x R m such that the absolute values of the eigenvalues of C n i=1 A ix i are as small as possible: min x,t t s. t. ti C n A i x i ti. i=1
Approximating quadratic problems min x R n x T Qx + q T x s. t. x T A i x + a T i x = α i for all i = 1,..., m, min x R n s. t. Q, xx T + q T x A i, xx T + a T i x = α i for all i = 1,..., m ( ) ( ) ( ) 1 x T T 1 1 x xx T = PSD n+1, x x (1) (2) min x R n, X S n Q, X + q T x s. t. A i, X + a T i x = α i for all i = 1,..., m ( ) (3) 1 x T PSD n+1 x X val(1) = val(2) val(3).
Examples Example min x 2 1 + 2x 2 2 4x 1 x 2 + 2x 2 s. t. x 2 1 + x 2 2 x 1 x 2 4x 1 = 1, x R 2, min s. t. ( 1 2 2 2 ( 1 1/2 1/2 1 ), X + 2x 2 ), X 4x 1 = 1, ( ) 1 x T PSD 3. x X Ex. 9.6 For the following problem, give a finite upper bound and formulate a PSD problem which would give a lower bound. min 2x1 2 + 2x2 2 6x 1 x 2 + 2x 2 4x 3 s. t. 2x1 2 + x2 2 2x 1 x 2 4x 1 = 1 2x 2 x3 2 = 0, x R 3.