9. Interpretations, Lifting, SOS and Moments

Transcription

1 9-1 Interpretations, Lifting, SOS and Moments P. Parrilo and S. Lall, CDC Interpretations, Lifting, SOS and Moments Polynomial nonnegativity Sum of squares (SOS) decomposition Eample of SOS decomposition Computing SOS using semidefinite programming Conveity Positivity in one variable Background Global optimization Optimizing in parameter space Lyapunov functions

2 9-2 Interpretations, Lifting, SOS and Moments P. Parrilo and S. Lall, CDC Interpretations So far, we have seen how to compute certificates of polynomial nonnegativity As we will see, these are dual SDP relaations We can also interpret the corresponding primal SDPs These arise through liftings

3 9-3 Interpretations, Lifting, SOS and Moments P. Parrilo and S. Lall, CDC A General Method: Liftings Consider this polytope in R 3 (a zonotope). It has 56 facets, and 58 vertices. Optimizing a linear function over this set, requires a linear program with 56 constraints (one per face). However, this polyhedron is a threedimensional projection of the 8-dimensional hypercube { R 8, 1 i 1}. Therefore, by using additional variables, we can solve the same problem, by using an LP with only 16 constraints.

4 9-4 Interpretations, Lifting, SOS and Moments P. Parrilo and S. Lall, CDC Liftings By going to higher dimensional representations, things may become easier: Complicated sets can be the projection of much simpler ones. A polyhedron in R n with a small number of faces can project to a lower dimensional space with eponentially many faces. Basic semialgebraic sets can project into non-basic semialgebraic sets. Feasible sets of SDPs may project to non-spectrahedral sets. An essential technique in integer programming. Advantages: compact representations, avoiding case distinctions, etc.

5 9-5 Interpretations, Lifting, SOS and Moments P. Parrilo and S. Lall, CDC Eample minimize ( 3) 2 subject to ( 4) 0 The feasible set is [, 0] [4, ]. Not conve, or even connected. Consider the lifting L : R R 2, with L() = (, 2 ) =: (, y). Rewrite the problem in terms of the lifted variables. For every lifted point, [ 1 y Constraint becomes: y 4 0 Objective is now: y ] 0. y We get around nonconveity: interior points are now on the boundary.

6 9-6 Interpretations, Lifting, SOS and Moments P. Parrilo and S. Lall, CDC Quadratically Constrained Quadratic Programming A general QCQP is minimize subject to [ ] T 1 Q [ ] 1 [ ] T [ ] 1 1 A i = 0 for all i = 1,..., m The Lagrangian is L(, λ) = [ ] T ( 1 Q m i=1 λ i A i ) [ 1 ] T so the dual feasible set is defined by semidefinite constraints

7 9-7 Interpretations, Lifting, SOS and Moments P. Parrilo and S. Lall, CDC QCQP Dual The dual is the SDP maimize subject to t Q m i=1 λ i A i t [ ] and the dual of the dual is minimize trace QY subject to trace A i Y = 0 for all i = 1,..., m Y 0 Y 11 = 1

8 9-8 Interpretations, Lifting, SOS and Moments P. Parrilo and S. Lall, CDC Lifting Lifting is a general approach for constructing primal relaations; the idea is Introduce new variables Y which are polynomial in This embeds the problem in a higher dimensional space Write valid inequalities in the new variables The feasible set of the original problem is the projection of the lifted feasible set

9 9-9 Interpretations, Lifting, SOS and Moments P. Parrilo and S. Lall, CDC Lifting QCQP We have the QCQP minimize subject to [ ] T 1 Q [ ] 1 [ ] T [ ] 1 1 A i = 0 for all i = 1,..., m Use lifted variables Y S n, defined by Y = [ ] [ 1 1 ] T We have valid constraints Y 0, Y 11 = 1, rank Y = 1 Every such Y corresponds to a unique

10 9-10 Interpretations, Lifting, SOS and Moments P. Parrilo and S. Lall, CDC Lifted QCQP The lifted problem is minimize trace QY subject to trace A i Y = 0 for all i = 1,..., m Y 0 Y 11 = 1 rank Y = 1 Again, we can drop the non-conve constraint to obtain a relaation This (happens to) give the same as the dual of the dual

11 9-11 Interpretations, Lifting, SOS and Moments P. Parrilo and S. Lall, CDC QCQP Interpretation of Polynomial Programs We can also lift polynomial programs; consider the eample minimize 6 k=0 a k k We ll choose lifted variables then the cost function is y = 2 3 f = a 0 + a 1 y 1 + a 2 y 2 + a 3 y 3 + a 4 y 1 y 3 + a 5 y 2 y 3 + a 6 y 2 3 a quadratic function of y (many other choices possible)

12 9-12 Interpretations, Lifting, SOS and Moments P. Parrilo and S. Lall, CDC QCQP Interpretation of Polynomial Programs We have the equivalent QCQP minimize 1 y 1 y 2 y 3 T a a 1 a 2 a a a 5 2 a 6 1 y 1 y 2 y 3 subject to y 2 y 2 1 = 0 y 3 y 1 y 2 = 0 to make the Lagrange dual tighter, we can add the valid constraint y 2 2 y 1y 3 = 0 Every polynomial program can be epressed as an equivalent QCQP

13 9-13 Interpretations, Lifting, SOS and Moments P. Parrilo and S. Lall, CDC Quadratic Constraints The above quadratic constraints are T y y y T y y y T y y y y 1 y 2 y 3 y 1 y 2 y 3 = 0 1 = 0 1 y 1 y 2 = 0 y 3

14 9-14 Interpretations, Lifting, SOS and Moments P. Parrilo and S. Lall, CDC Relaations We can now construct the SDP primal and dual relaations of this QCQP Eample Suppose f = , then the SDP dual relaation is maimize subject to t 1 t λ 2 λ 3 0 2λ 2 λ 3 λ λ 2 λ 3 2λ 1 0 λ 3 λ this is eactly the condition that f t be sum of squares

15 9-15 Interpretations, Lifting, SOS and Moments P. Parrilo and S. Lall, CDC The Primal Relaation of a Polynomial Program Since we have a QCQP, there is also an SDP primal relaation, constructed via the lifting [ ] [ ] T 1 1 Y = y y It is the SDP minimize trace subject to Y 0 a a 1 a 2 a a a 5 2 Y a 6 Y 11 = 1 Y 24 = Y 33 Y 22 = Y 13 Y 14 = Y 23

16 9-16 Interpretations, Lifting, SOS and Moments P. Parrilo and S. Lall, CDC The Primal Relaation of a Polynomial Program This is constructed by Y = [ ] [ ] T 1 1 = y y T = One may construct this directly from the polynomial program Direct etensions to the multivariable case The feasible set of Y may be projected to give a feasible set of If the optimal Y has rank Y = 1 then the relaation is eact

17 9-17 Interpretations, Lifting, SOS and Moments P. Parrilo and S. Lall, CDC Lifting Higher dimensional representations have several possible advantages One may find simpler representations, e.g., polytopes Basic semialgebraic sets may project to non-basic ones Adding new variables via lifting allows new valid inequalities, which tightens the dual Using polynomial lifting allows more constraints to be represented in LP or SDP form Lifting wraps the feasible set onto a higher dimensional variety; this tends to map interior points to boundary points

18 9-18 Interpretations, Lifting, SOS and Moments P. Parrilo and S. Lall, CDC Outer Approimation of Semialgebraic Sets The primal SDP relaation allows us to construct outer approimation of a semialgebraic set For eample, one can compute an outer approimation of the epigraph { S = ( 1, 2 ) } 2 f( 1 ) In one variable, the SDP relaation gives eactly the conve hull, since S is contained in a halfspace { R 2 a T b } if and only if the following polynomial inequality holds a 1 + a 2 f() b for all

19 9-19 Interpretations, Lifting, SOS and Moments P. Parrilo and S. Lall, CDC Eample: Outer Approimation of the Epigraph Let s look at the univariate eample f = 1 ( 1)( 2)( 3)( 5) If y f() then the following SDP is feasible y trace X X 0 X 22 = 2X 12 X 11 = 1 X 12 =

20 9-20 Interpretations, Lifting, SOS and Moments P. Parrilo and S. Lall, CDC Moments Interpretation of the Primal Relaation Instead of trying to minimize directly f, we can solve minimize E f = f()p() d Rn subject to p is a probability distribution on R n This is a dual problem to minimizing f If f has a unique minimum at 0, then the optimal will be a point measure at 0 Essentially due to Lasserre

21 9-21 Interpretations, Lifting, SOS and Moments P. Parrilo and S. Lall, CDC Moments Interpretation of the Primal Relaation suppose y = [ 1 y y 2... ] T, then f = c T y and E f = c T E y E y is the vector of moments of the distribution so we have the equivalent problem minimize subject to c T z z is a vector of moments of y

22 9-22 Interpretations, Lifting, SOS and Moments P. Parrilo and S. Lall, CDC Eample Since E yy T 0 for any distribution, we have valid inequalities T y E = E 2 y 0 y y y y y 2 so to find a lower bound 2 + 2y + 3y 2 we solve the SDP [ ] minimize z subject to M 0 z 1 = M 22, z 2 = M 12, z 3 = M 22 This is eactly the primal SDP relaation; the dual of SOS Similar to MAXCUT, where the SDP relaation may be viewed as a covariance matri

23 9-23 Interpretations, Lifting, SOS and Moments P. Parrilo and S. Lall, CDC A General Scheme Polynomial Program Lifting and Valid Constraints y = Lifting and Relaation QCQP Y = ô 1 õ ô 1 y y õ T Lagrangian Relaation Primal SDP Relaation Moments SDP Duality Dual SDP Relaation Sum of Squares Primal: the solution to the lifted problem may suggest candidate points where the polynomial is negative. Dual: the sum of squares certifies or proves polynomial nonnegativity.