Global polynomial optimization with Moments Matrices and Border Basis

Global polynomial optimization with Moments Matrices and Border Basis Marta Abril Bucero, Bernard Mourrain, Philippe Trebuchet Inria Méditerranée, Galaad team Juillet 15-19, 2013 M. Abril-Bucero, B. Mourrain, P. Trebuchet Constrained (Inria Méditerranée, polynomial Galaad optimization team ) Juillet 15-19, 2013 1 / 21

Our Problem inf x Rnf (x) s.t. g 1 (x) = = g n1 (x) = 0 h 1 (x) 0,..., h n2 (x) 0 (1) where S = {x R n g 1 (x) = = g n1 (x) = 0, h 1 (x) 0,..., h n2 (x) 0} and H = {h 1,..., h n2 }. M. Abril-Bucero, B. Mourrain, P. Trebuchet Constrained (Inria Méditerranée, polynomial Galaad optimization team ) Juillet 15-19, 2013 2 / 21

Our Objectives 1 Show that the hierarchy of relaxation problems is exact.. Abril-Bucero, B. Mourrain, P. Trebuchet Constrained (Inria Méditerranée, polynomial Galaad optimization team ) Juillet 15-19, 2013 3 / 21

Our Objectives 1 Show that the hierarchy of relaxation problems is exact. 2 Find a criterion to detect when the optimum is reached and show that this criterion is eventually satised.. Abril-Bucero, B. Mourrain, P. Trebuchet Constrained (Inria Méditerranée, polynomial Galaad optimization team ) Juillet 15-19, 2013 3 / 21

Our Objectives 1 Show that the hierarchy of relaxation problems is exact. 2 Find a criterion to detect when the optimum is reached and show that this criterion is eventually satised. 3 When this criterion is satised, recover all the points where this optimum is achieved.. Abril-Bucero, B. Mourrain, P. Trebuchet Constrained (Inria Méditerranée, polynomial Galaad optimization team ) Juillet 15-19, 2013 3 / 21

Related works First work Global optimization with polynomials and the problem of the moments - J.B. Lasserre. Abril-Bucero, B. Mourrain, P. Trebuchet Constrained (Inria Méditerranée, polynomial Galaad optimization team ) Juillet 15-19, 2013 4 / 21

Related works First work Global optimization with polynomials and the problem of the moments - J.B. Lasserre Finite convergence SemiDenite Representation for nite varieties - M. Laurent - V(G) nite Minimizing polynomials via sum of squares over gradient ideal - Nie, Demmel, Sturmfels - Gradient ideal is radical Representations of non-negatives polynomials via critical, degree bounds and applications to optimization - M. Marshall - BHC Representation of positive polynomials on non-compact semialgebraic sets via KKT ideals - Demmel, Nie, Powers - KKT ideal is radical An exact Jacobian Sdp Relaxation - Nie - regularity conditions. Abril-Bucero, B. Mourrain, P. Trebuchet Constrained (Inria Méditerranée, polynomial Galaad optimization team ) Juillet 15-19, 2013 4 / 21

Related works First work Global optimization with polynomials and the problem of the moments - J.B. Lasserre Finite convergence SemiDenite Representation for nite varieties - M. Laurent - V(G) nite Minimizing polynomials via sum of squares over gradient ideal - Nie, Demmel, Sturmfels - Gradient ideal is radical Representations of non-negatives polynomials via critical, degree bounds and applications to optimization - M. Marshall - BHC Representation of positive polynomials on non-compact semialgebraic sets via KKT ideals - Demmel, Nie, Powers - KKT ideal is radical An exact Jacobian Sdp Relaxation - Nie - regularity conditions Convergence certicate Certifying convergence via Flat truncation - Nie. Abril-Bucero, B. Mourrain, P. Trebuchet Constrained (Inria Méditerranée, polynomial Galaad optimization team ) Juillet 15-19, 2013 4 / 21

Related works First work Global optimization with polynomials and the problem of the moments - J.B. Lasserre Finite convergence SemiDenite Representation for nite varieties - M. Laurent - V(G) nite Minimizing polynomials via sum of squares over gradient ideal - Nie, Demmel, Sturmfels - Gradient ideal is radical Representations of non-negatives polynomials via critical, degree bounds and applications to optimization - M. Marshall - BHC Representation of positive polynomials on non-compact semialgebraic sets via KKT ideals - Demmel, Nie, Powers - KKT ideal is radical An exact Jacobian Sdp Relaxation - Nie - regularity conditions Convergence certicate Certifying convergence via Flat truncation - Nie How to recover the minimizer ideal Real Radical Computation - Lasserre, Laurent, Mourrain, Trebuchet M. Abril-Bucero, B. Mourrain, P. Trebuchet Constrained (Inria Méditerranée, polynomial Galaad optimization team ) Juillet 15-19, 2013 4 / 21

Gradient Variety With Lagrange Multipliers: inf (x,u,v,s)=z R n+n 1 +2 n 2 f (x) where F (x, u, v, s) = f (x) P n1 i=1 u i g i (x) P n2 j=1 v j (h j (x) s 2 j ), u = (u 1,..., u n 1 ), v = (v 1,..., v n 2 ) and s = (s 1,..., s n 2 ). Denition s.t. F (x, u, v, s) = 0 (2) I grad = ( F (z)) = (F 1,..., F n, g 1,..., g n 1, h 1 s 2 1,..., hn 2 s2 n2, v 1s 1,..., v n 2 sn 2 ) R[z] V grad = V(I grad ) C n+n 1+2n2 M. Abril-Bucero, B. Mourrain, P. Trebuchet Constrained (Inria Méditerranée, polynomial Galaad optimization team ) Juillet 15-19, 2013 5 / 21

KKT Variety inf (x,u,v)=y R n+n 1 +n 2 f (x) s.t. F 1 = = F n = 0 g 1 = = g n 1 = 0 v 1 h 1 = = v n 2 hn 2 = 0 h 1 0,..., h n 2 0 (3) where F i = f P n1 x i j=1 u g j j x i P n2 j=1 v h j j. x i Denition x is a KKT point if there exists u 1,..., u n 1, v 1,..., v n 2 R s.t. n1x n2x f (x ) u i g i (x ) v i h i (x ) = 0, g i (x ) = 0, v i h i (x ) = 0 i=1 S KKT = set of KKT point of S. j=0 A KKT-minimizer of f on S is a point x S KKT such that f (x ) = min x S KKT f (x). I KKT = (F 1,..., F n, g 1,..., g n 1, v 1h 1,..., v n 2 hn 2 ) R[y]. V KKT = V(I KKT ) C n+n 1+n2 M. Abril-Bucero, B. Mourrain, P. Trebuchet Constrained (Inria Méditerranée, polynomial Galaad optimization team ) Juillet 15-19, 2013 6 / 21

Minimizer Variety Denition V min = {z R n+n 1+2 n 2 I min = I(V min ) s.t F (z ) = 0 and f (x ) is minimum}. Abril-Bucero, B. Mourrain, P. Trebuchet Constrained (Inria Méditerranée, polynomial Galaad optimization team ) Juillet 15-19, 2013 7 / 21

Minimizer Variety Denition V min = {z R n+n 1+2 n 2 I min = I(V min ) s.t F (z ) = 0 and f (x ) is minimum} Proposition V R KKT W +(H) = π y (V R grad ), W +(H) = {y R n h(x) 0 h H}. I KKT = I grad R[y] = I y grad. M. Abril-Bucero, B. Mourrain, P. Trebuchet Constrained (Inria Méditerranée, polynomial Galaad optimization team ) Juillet 15-19, 2013 7 / 21

Our contributions When f is reached in a KKT-minimizer of f on S. Abril-Bucero, B. Mourrain, P. Trebuchet Constrained (Inria Méditerranée, polynomial Galaad optimization team ) Juillet 15-19, 2013 8 / 21

Our contributions When f is reached in a KKT-minimizer of f on S Hierarchy of relaxations problem asociated to problem 3 or to its projection is exact.. Abril-Bucero, B. Mourrain, P. Trebuchet Constrained (Inria Méditerranée, polynomial Galaad optimization team ) Juillet 15-19, 2013 8 / 21

Our contributions When f is reached in a KKT-minimizer of f on S Hierarchy of relaxations problem asociated to problem 3 or to its projection is exact. When f is attained we nd the generators of I min.. Abril-Bucero, B. Mourrain, P. Trebuchet Constrained (Inria Méditerranée, polynomial Galaad optimization team ) Juillet 15-19, 2013 8 / 21

Our contributions When f is reached in a KKT-minimizer of f on S Hierarchy of relaxations problem asociated to problem 3 or to its projection is exact. When f is attained we nd the generators of I min. New representation of positive polynomials on S.. Abril-Bucero, B. Mourrain, P. Trebuchet Constrained (Inria Méditerranée, polynomial Galaad optimization team ) Juillet 15-19, 2013 8 / 21

Our contributions When f is reached in a KKT-minimizer of f on S Hierarchy of relaxations problem asociated to problem 3 or to its projection is exact. When f is attained we nd the generators of I min. New representation of positive polynomials on S. And if V min is nite we propose: New criterion for deciding when the minimum is reached.. Abril-Bucero, B. Mourrain, P. Trebuchet Constrained (Inria Méditerranée, polynomial Galaad optimization team ) Juillet 15-19, 2013 8 / 21

Our contributions When f is reached in a KKT-minimizer of f on S Hierarchy of relaxations problem asociated to problem 3 or to its projection is exact. When f is attained we nd the generators of I min. New representation of positive polynomials on S. And if V min is nite we propose: New criterion for deciding when the minimum is reached. Satised Certicate for high degree. Abril-Bucero, B. Mourrain, P. Trebuchet Constrained (Inria Méditerranée, polynomial Galaad optimization team ) Juillet 15-19, 2013 8 / 21

Our contributions When f is reached in a KKT-minimizer of f on S Hierarchy of relaxations problem asociated to problem 3 or to its projection is exact. When f is attained we nd the generators of I min. New representation of positive polynomials on S. And if V min is nite we propose: New criterion for deciding when the minimum is reached. Satised Certicate for high degree New algorithm to nd the minimizer ideal of a real polynomial function on S.. Abril-Bucero, B. Mourrain, P. Trebuchet Constrained (Inria Méditerranée, polynomial Galaad optimization team ) Juillet 15-19, 2013 8 / 21

Our contributions When f is reached in a KKT-minimizer of f on S Hierarchy of relaxations problem asociated to problem 3 or to its projection is exact. When f is attained we nd the generators of I min. New representation of positive polynomials on S. And if V min is nite we propose: New criterion for deciding when the minimum is reached. Satised Certicate for high degree New algorithm to nd the minimizer ideal of a real polynomial function on S. Reduction of the size of SDP problems for using border basis.. Abril-Bucero, B. Mourrain, P. Trebuchet Constrained (Inria Méditerranée, polynomial Galaad optimization team ) Juillet 15-19, 2013 8 / 21

Hierarchy of relaxation problems Denition Given a vector space E R[y] with 1 E, and nite subsets G, H R[y], we dene P E,H = { P ɛ {0,1} n 2 sɛ hɛ 1 1 hɛn 2 n2 s ɛ P E 2, s ɛ h ɛ 1 1 hɛn 2 n2 E E }. P E,G,H := {g + q g G E E, q P E,H } L E,G,H := {Λ E E Λ(p) 0, p P E,G,H } f µ E,G,H = inf {Λ(f ) s.t. Λ L E,G,H, Λ(1) = 1} f sos E,G,H = max {γ s.t. f γ P E,G,H} f = inf x S KKT f (x). Hierarchy For t N, consider the prolongation G 2t of G to degree 2t and E = R[y] t L t+1,g,h L t,g,h and P t,g,h P t+1,g,h Properties If V y min 0 and V y min V R (G) W +(H), f µ t,g,h f µ t+1,g,h f and f sos t,g,h f sos t+1,g,h f and f sos t,g,h f µ t,g,h f M. Abril-Bucero, B. Mourrain, P. Trebuchet Constrained (Inria Méditerranée, polynomial Galaad optimization team ) Juillet 15-19, 2013 9 / 21

Truncated Hankel operators and Generic linear form Denition Let E R[x] be a vector space and let Λ E E be a linear form, the map MΛ E : E E by MΛ E (p) = p Λ for p E is a truncated Hankel operator. ker MΛ E = {p E p Λ = 0, i.e, Λ(pq) = 0 q E} The matrix MΛ E is the Moment Matrix: [ME Λ ] = (Λ(xα+β )) α E,β E Denition Λ L E,G,H is generic for f if rankm E Λ = max Λ LE,G,H,Λ(f )=f µ E,G,H rankm E Λ. How to nd Λ L E,G,H generic for f? Solve the SDP program with interior point methods. Existing tools: SDPA, CSDP,... M. Abril-Bucero, B. Mourrain, P. Trebuchet Constrained (Inria Méditerranée, polynomial Galaad optimization team ) Juillet 15-19, 2013 10 / 21

Representation of positive polynomials Theorem Let f R[x]. 1 If f 0 on S KKT, then f P x H + q I x grad = P x H + p I x KKT. 2 If f > 0 on S KKT, then f P x H + I x grad = P x H + I x KKT.. Abril-Bucero, B. Mourrain, P. Trebuchet Constrained (Inria Méditerranée, polynomial Galaad optimization team ) Juillet 15-19, 2013 11 / 21

Representation of positive polynomials Theorem Let f R[x]. 1 If f 0 on S KKT, then f P x H + q I x grad = P x H + p I x KKT. 2 If f > 0 on S KKT, then f P x H + I x grad = P x H + I x KKT. Proposition - Demmel, Nie and Powers 2007 Let f R[x]. 1 If f 0 on S KKT, then f P H + I KKT. 2 If f > 0 on S KKT, then f P H + I KKT.. Abril-Bucero, B. Mourrain, P. Trebuchet Constrained (Inria Méditerranée, polynomial Galaad optimization team ) Juillet 15-19, 2013 11 / 21

Representation of positive polynomials Theorem Let f R[x]. 1 If f 0 on S KKT, then f P x H + q I x grad = P x H + p I x KKT. 2 If f > 0 on S KKT, then f P x H + I x grad = P x H + I x KKT. Proposition - Demmel, Nie and Powers 2007 Let f R[x]. 1 If f 0 on S KKT, then f P H + I KKT. 2 If f > 0 on S KKT, then f P H + I KKT. Corollary - Nie, Demmel and Sturmfels 2005 If n 1 = n 2 = 0 then I grad = ( f ) = ( f,..., f ). x 1 xn 1 If f 0 on V R ( f ), then f P x + p ( f ). 2 If f > 0 on V R ( f ), then f P x + ( f ). M. Abril-Bucero, B. Mourrain, P. Trebuchet Constrained (Inria Méditerranée, polynomial Galaad optimization team ) Juillet 15-19, 2013 11 / 21

Finite convergence Theorem - Finite Convergence and minimizer ideal For G R[x] with hyp{g, V min } : [V x min V R (G) W + (H) π x (V R KKT ) and V min ], t 2 N s.t. t t 2, for Λ L t,g,h generic for f, we have Λ (f ) = f µ t,g,h = f. (ker M t Λ ) = I x min. If V(G) π x (V KKT ) then f sos t,g,h = f µ t,g,h = f. Similar result for G R[y] with V y min V R (G) W + (H) V R KKT M. Abril-Bucero, B. Mourrain, P. Trebuchet Constrained (Inria Méditerranée, polynomial Galaad optimization team ) Juillet 15-19, 2013 12 / 21

Convergence certication and satisfaction Theorem - Convergence Certication Let B be a monomial set connected to 1, E R[x] with (1) f E E, (2) B + E, G B E E, H B B E E and hyp{g, V min } + V min is nite and let Λ L E,G,H generic for f with: (3) rank M E Λ = rank M B Λ = B. Then f = f µ E,G,H = f sos E,G,H (ker M E Λ ) = I x min. B + = B x 1 B x nb Theorem - Satised Certication Let G, H R[x] with hyp{g, V min } + V min is nite. Then, t 3 N s.t. t t 3, and Λ L t,g,h generic for f, a vector space E R[x] t and B connected to 1 such that the properties (1), (2) and (3) of the above theorem are satised. M. Abril-Bucero, B. Mourrain, P. Trebuchet Constrained (Inria Méditerranée, polynomial Galaad optimization team ) Juillet 15-19, 2013 13 / 21

Consequences Generic case: Generalization of Nie 2011 A J := [ f, g 1,..., g n 1, h j1,..., h j k ], J = {j 1,..., j k }. V( J 1,..., J ) = {x C n rank(a l J ) n 1 + J }. J G = (g 1,, g n 1, g Q J,i ) with g J,i := J i j J h j for i = 1,..., l J, J [0, n 2 ]. Conclusion: If (g 1,... g n 1 ; h 1,..., h n 2 ) is C-regular ( points x Cn with J(x) = {j 1,..., j k }, g 1 (x),..., g n 1 (x), h j1 (x),..., h j k (x) s. t. h jl (x) = 0 are l. i.) +f reaches its minimum at x S Finite Convergence and minimizer ideal Theorem. Regular case: Generalization of Ha and Pham 2010 A J := [ f, g 1,..., g n 1, h j1,..., h j k ], J = {j 1,..., j k }. Over R n, J i = J = det(a J A T J ) G = (g 1,, g n 1, g J) with g J := J Qj J h j for i = 1,..., l J, J [0, n 2 ]. Conclusion: If S is regular ( x S, g 1 (x),..., g n 1 (x), h j1 (x),..., h j k (x) s. t. h jl (x) = 0 are l. i.) +f reaches its minimum at x S Finite Convergence and minimizer ideal Theorem. If (g 1,... g n 1 ; h 1,..., h n 2 ) is C-regular we have not duality gap. M. Abril-Bucero, B. Mourrain, P. Trebuchet Constrained (Inria Méditerranée, polynomial Galaad optimization team ) Juillet 15-19, 2013 14 / 21

Algorithm We suppose V min nite. Input:f R[x], G R[x] s.t. V x min VR (G) W + (H) π x (V R KKT ), H = {h 1,..., h n2 }. 1 t := 1 2 max{deg(f ), deg(g i), deg(h j )} ; 2 Compute Λ L t,g,h generic for f (solving a nite dimensional SDP problem by an interior point method); 3 Check the convergence certicate for M t Λ ; 4 If it is not satised, t := t + 1 and repeat from step (2); Otherwise compute K = ker M t Λ and Λ (f ) = f. Output: f = Λ (f ), K generators of I x min. we combine this algorithm with the border basis algorithm in order to reduce the size of the moments matrices. M. Abril-Bucero, B. Mourrain, P. Trebuchet Constrained (Inria Méditerranée, polynomial Galaad optimization team ) Juillet 15-19, 2013 15 / 21

Example - Robinson polynomial f (x, y) = 1 + x 6 x 4 x 2 + y 6 y 4 y 2 x 4 y 2 x 2 y 4 + 3x 2 y 2 ; Iteration Order Size min Λ(f ) Gap dim kerh Λ 1 3 10-0.93 N - 2 4 15 0 N 7 3 5 19 0 N 11 4 6 22 0 N - In the second iteration we obtain B = {1, x, y, x 2, xy, y 2, x 2 y, xy 2, x 2 y 2 } there exits an element with degree 4. In the third iteration we obtain B = {1, x, y, x 2, xy, y 2, x 2 y, xy 2 } After fourth iteration the algoritm stops because kerh B 5 Λ = 0 and we obtain 1 I min = (x 3 x, y 3 y, x 2 y 2 x 2 y 2 + 1). 2 The basis B = {1, x, y, x 2, xy, y 2, x 2 y, xy 2 }. 3 The real roots {(x = 1, y = 1), (x = 1, y = 1), (x = 1, y = 1), (x = 1, y = 1), (x = 1, y = 0), (x = 1, y = 0), (x = 0, y = 1), (x = 0, y = 1)}. M. Abril-Bucero, B. Mourrain, P. Trebuchet Constrained (Inria Méditerranée, polynomial Galaad optimization team ) Juillet 15-19, 2013 16 / 21

Example with constraints f (x, y) = 12x 7y + y 2 ; s.t 2x 4 + 2 y = 0; 0 x 2; 0 y 3; Iteration Order Size min Λ(f ) Gap dim kerh Λ 1 1 7(3) -36 N - 2 2 10(6) -16.7 N 9(5) 3 3 14(10) -16.7 N - In the second iteration we obtain B = {1}. After third iteration the algoritm stops because kerh B 2 Λ = 0 and we obtain 1 I min = (x 0,7175, y 1,4698). 2 The basis B = {1}. 3 The real roots {(x = 0,7175, y = 1,4698)}. M. Abril-Bucero, B. Mourrain, P. Trebuchet Constrained (Inria Méditerranée, polynomial Galaad optimization team ) Juillet 15-19, 2013 17 / 21

Implementation and comparasion with Gloptipoly Implementation This algorithm is implemented in the package BORDERBASIX of MATHEMAGIX software. SDP problems are computed using SDPA or CSDP software.. Abril-Bucero, B. Mourrain, P. Trebuchet Constrained (Inria Méditerranée, polynomial Galaad optimization team ) Juillet 15-19, 2013 18 / 21

Implementation and comparasion with Gloptipoly Implementation This algorithm is implemented in the package BORDERBASIX of MATHEMAGIX software. SDP problems are computed using SDPA or CSDP software. Comparasions with Gloptipoly The size of the moments matrices in our method is smaller due to the use of border basis We obtain the mininum with only one execution instead of many executions in the case of Gloptipoly Example variables degree Gloptipoly Our method Motzkin 2 6 order 9 - size 55 order 5 - size 19 Robinson 2 6 order 7 - size 36 order 6 - size 22 Nie 3 4 order 4 - size 34 order 3 - size 9 Simplices 14 2 - order 1 M. Abril-Bucero, B. Mourrain, P. Trebuchet Constrained (Inria Méditerranée, polynomial Galaad optimization team ) Juillet 15-19, 2013 18 / 21

Ongoing and Future works Applications: Tensor decomposition of rank 1 or 2 What happens when I min is not zero-dimensional? What happens when f is not reached? Solver numerical problem of the SDPA M. Abril-Bucero, B. Mourrain, P. Trebuchet Constrained (Inria Méditerranée, polynomial Galaad optimization team ) Juillet 15-19, 2013 19 / 21

Numerical problem f (x, y) = 16 + x 2 y 4 + 2x 2 y 3 4x 3 y 3 + 4xy 2 + 20x 2 y 2 + 8x 3 y 2 + 6x 4 y 2 + 8xy 16x 2 y f = 0,60902 and (x, y ) = ( 3,3884, 0,1434) We nd the minimum in order 4. Behavior in gloptipoly (nd local minimum not global minimum) msdp(min(f)) status =-1 (infeasible) msdp(min(f),di(f)==0) status=0 (not global optimality), f = 0,601 msdp(min(f),di(f)==0,10) status=1,f = 15,8283 and (x, y ) = (0,1156, 0,4839) Possible solution: use orthogonal basis instead of monomial basis M. Abril-Bucero, B. Mourrain, P. Trebuchet Constrained (Inria Méditerranée, polynomial Galaad optimization team ) Juillet 15-19, 2013 20 / 21

THANK YOU FOR YOUR ATTENTION!!! M. Abril-Bucero, B. Mourrain, P. Trebuchet Constrained (Inria Méditerranée, polynomial Galaad optimization team ) Juillet 15-19, 2013 21 / 21