Semidefinite Programming Basics and SOS Fernando Mário de Oliveira Filho Campos do Jordão, 2 November 23 Available at: www.ime.usp.br/~fmario under talks
Conic programming V is a real vector space h, i is an inner product Convex cone: set C V such that x + y 2 C 8 x, y 2 C and, Given: c, a,...,a m 2 V and b,...,b m 2 R max hc, xi ha i, xi = b i x 2 C for i =,..., m Complexity: depends on the cone!
Examples V = R n and C = R n +: linear programming V = S n and C = cone of PSD matrices: SDP A, B 2 S n ha, Bi =tr(a T B)= nx i,j= A ij B ij = trace inner product A is PSD (also A ): x T Ax 8 x 2 R n () all eigenvalues are nonnegative () A = Q T Q for some Q 2 R n n () A = P n i= iu i u T i,with i
Semidefinite programming Given: C, A,...,A m 2 S n and b,...,b m 2 R max hc, Xi ha i, Xi = b i X for i =,..., m x z z y x, y and xy z 2 () Generalizes LP: make all matrices diagonal! Duality theory: comes from conic programming Solvable in poly-time: under some conditions (later) Numerical methods: efficient interior-point methods
Application: polynomial optimization Is the polynomial p 2 R[x,...,x n ] nonnegative? Sure, if it is a sum-of-squares: p = q 2 + + qk 2 Exercise: A univariate poly is nonnegative iff it is SOS P n,d = nonnegative polys, n variables, degree d n,d = SOS polys, n variables, degree d Theorem. n,d P n,d with equality i bivariate polys (n =2) quadratics (d =2) n =3, d =4 David Hilbert 888
SOS is SDP p 2 R[x,...,x n ] degree d v(x) = vector with all monomials degree apple d/2 Theorem. p is SOS i p(x) =hq, v(x)v(x) T i for some Q. n = 2, d =6 v(x) =, x,x 2,x 3,y,xy,x 2 y, y 2,xy 2,y 3 B @ x x 2 x 3 y xy x 2 y y 2 xy 2 y 3 x x 2 x 3 x 4 xy x 2 y x 3 y xy 2 x 2 y 2 xy 3 x 2 x 3 x 4 x 5 x 2 y x 3 y x 4 y x 2 y 2 x 3 y 2 x 2 y 3 x 3 x 4 x 5 x 6 x 3 y x 4 y x 5 y x 3 y 2 x 4 y 2 x 3 y 3 y xy x 2 y x 3 y y 2 xy 2 x 2 y 2 y 3 xy 3 y 4 xy x 2 y x 3 y x 4 y xy 2 x 2 y 2 x 3 y 2 xy 3 x 2 y 3 xy 4 x 2 y x 3 y x 4 y x 5 y x 2 y 2 x 3 y 2 x 4 y 2 x 2 y 3 x 3 y 3 x 2 y 4 y 2 xy 2 x 2 y 2 x 3 y 2 y 3 xy 3 x 2 y 3 y 4 xy 4 y 5 xy 2 x 2 y 2 x 3 y 2 x 4 y 2 xy 3 x 2 y 3 x 3 y 3 xy 4 x 2 y 4 xy 5 y 3 xy 3 x 2 y 3 x 3 y 3 y 4 xy 4 x 2 y 4 y 5 xy 5 y 6 C A
A small example p(x, y) =x 4 +2x 3 y + x 2 y 2 6x 2 y 6xy 2 + x 2 + xy +9y 2 3y + 25 v(x) =(,x,x 2,y,y 2,xy) B @ Q x x 2 y y 2 xy q q 2 q 3 q 4 q 5 q 6 x q 2 q 22 q 23 q 24 q 25 q 26 x 2 q 3 q 32 q 33 q 34 q 35 q 36 y q 4 q 42 q 43 q 44 q 45 q 46 y 2 q 5 q 52 q 53 q 54 q 55 q 56 xy q 6 q 62 q 63 q 64 q 65 q 66 C A B @ v(x)v(x) T x x 2 y y 2 xy x x 2 y y 2 xy x x x 2 x 3 xy xy 2 x 2 y x 2 x 2 x 3 x 4 x 2 y x 2 y 2 x 3 y y y xy x 2 y y 2 y 3 xy 2 y 2 y 2 xy 2 x 2 y 2 y 3 y 4 xy 3 xy xy x 2 y x 3 y xy 2 xy 3 x 2 y 2 C A q = 25 q 23 + q 32 = q 25 + q 52 + q 46 + q 64 = 6 Q = X u i u T i =) p = X (u T i v(x)) 2
Sparse SDPA format: Setup maximize hf,yi hf i,yi = c i Y. for i =,..., m 2 Matrices have a block structure 3 Block 4, a diagonal block
Sparse SDPA format maximize Text file problem.sdpa : Number of constraint matrices: Number of blocks: hf,yi hf i,yi = c i Y. m N for i =,..., m Block sizes: Right-hand side: Matrix entries: s s 2 s N c c 2 c m h matrix number i h block number i h i i h j i h entry i Remarks: Indices always start at Write the size of a block as a negative number to indicate a diagonal block Only upper-diagonal entries need to be given!
A feasibility problem Is the polynomial a sum of squares? p(x) =x 4 +6x 3 + 5x 2 x + 7 v(x) =(,x,x 2 ) F = F 3 = @ A F = @ A F 4 = @ A F 2 = @ A F 5 = @ A @ A c = 7, c 2 =, c 3 = 5, c 4 = 6, c 5 = maximize hf,yi hf i,yi = c i for i =,..., 5 Y.
The SDPA file F = F 3 = @ A F = @ A F 4 = @ A F 2 = @ A F 5 = @ A @ A c = 7, c 2 =, c 3 = 5, c 4 = 6, c 5 = File sos.sdpa : 5 Number of constraint matrices! We have only one block! 3 Which is 3x3! 7-5 6 The right-hand side. Matrix entries! 2 2.! 3 3.! 3 2 2.! 4 2 3.! 5 3 3.
CSDP output..... Dual solution, only numbers! 2.8867534594829e- Dual solution, matrix! 2 2 2.8867534594829e-! 3 3 2.8867534594829e-! 2.7e+ Primal solution, matrix! 2 2-5.e+! 2 3-2.9653763427438397e+! 2 2 2 2.937526843487679e+! 2 2 3 3.e+! 2 3 3.e+ Y = @ 7. 5. 2.9653763427438 5. 2.937526843488 3. 2.9653763427438 3.. A p (x) =.7929446728x 2.2267825866x +4.2356256766 p 2 (x) =.4823565635256x 2 +4.43676392992x p 3 (x) =.5747634394x 2 p 2 + p 2 2 + p 2 3 = x 4 +6x 3 + 5x 2 x + 7
Other uses of SOS Finding the minimum of a polynomial minimum of p is () p is nonnegative () p is SOS Formulate as SDP: max{ : p is SOS } Nonnegativity in a domain p 2 R[x] is nonnegative in [a, b] () 9 q, q 2 SOS s.t. p =(x a)(b x)q + q 2
Duality in conic programming C V a cone Dual cone: C = { x 2 V : hx, yi for all y 2 C } Primal standard form: max hc, xi ha i, xi = b i for i =,..., m x 2 C strictly feasible: x in interior of C Dual standard form: min y b + + y m b m y a + + y m a m strictly feasible: y a + + y m a m c 2 C c in interior of C
Duality theorems Weak duality: x is primal feasible, y is dual feasible =) hc, xi appley T b Strong duality: If: Then: If: Then: dual is bounded and strictly feasible primal attains optimal and values coincide primal is bounded and strictly feasible dual attains optimal and values coincide
Applied to SDP Exercise: PSD cone is self-dual Primal standard form: max hc, Xi ha i, Xi = b i X for i =,..., m Dual standard form: strictly feasible: X is positive definite min y b + + y m b m y A + + y m A m C strictly feasible: y A + + y m A m C is positive definite
SDP and duality gone wrong Optimal not attained: min max X 2 X 2 X =,X 22 = Positive duality gap: max X X 22 X = 2X 3 + X 22 = X min y 2 y + y 2 @ y 2 + A y 2 Optimal: - Optimal:
Summary and coming soon Semidefinite programming Generalizes LP Fits framework of conic programming Solvable in poly time (under some mild assumptions) Has a duality theory (inherits from conic programming) One useful application: SOS polynomials Tomorrow: Approximation algorithms Maximum-cut problem Grothendieck inequalities and applications