A new look at nonnegativity on closed sets

Transcription

1 A new look at nonnegativity on closed sets LAAS-CNRS and Institute of Mathematics, Toulouse, France IPAM, UCLA September 2010

2 Positivstellensatze for semi-algebraic sets K R n from the knowledge of defining polynomials inner approximations of the cone of polynomials nonnegative on K Optimization: Semidefinite relaxations yield lower bounds Another look at nonnegativity from knowledge of a measure supported on K. outer approximations of the cone of polynomials nonnegative on K Optimization: Semidefinite approximations yield upper bounds

3 Positivstellensatze for semi-algebraic sets K R n from the knowledge of defining polynomials inner approximations of the cone of polynomials nonnegative on K Optimization: Semidefinite relaxations yield lower bounds Another look at nonnegativity from knowledge of a measure supported on K. outer approximations of the cone of polynomials nonnegative on K Optimization: Semidefinite approximations yield upper bounds

4 Positivstellensatze for semi-algebraic sets K R n from the knowledge of defining polynomials inner approximations of the cone of polynomials nonnegative on K Optimization: Semidefinite relaxations yield lower bounds Another look at nonnegativity from knowledge of a measure supported on K. outer approximations of the cone of polynomials nonnegative on K Optimization: Semidefinite approximations yield upper bounds

5 Positivstellensatze for semi-algebraic sets K R n from the knowledge of defining polynomials inner approximations of the cone of polynomials nonnegative on K Optimization: Semidefinite relaxations yield lower bounds Another look at nonnegativity from knowledge of a measure supported on K. outer approximations of the cone of polynomials nonnegative on K Optimization: Semidefinite approximations yield upper bounds

6 Positivstellensatze for semi-algebraic sets K R n from the knowledge of defining polynomials inner approximations of the cone of polynomials nonnegative on K Optimization: Semidefinite relaxations yield lower bounds Another look at nonnegativity from knowledge of a measure supported on K. outer approximations of the cone of polynomials nonnegative on K Optimization: Semidefinite approximations yield upper bounds

7 Positivstellensatze for semi-algebraic sets K R n from the knowledge of defining polynomials inner approximations of the cone of polynomials nonnegative on K Optimization: Semidefinite relaxations yield lower bounds Another look at nonnegativity from knowledge of a measure supported on K. outer approximations of the cone of polynomials nonnegative on K Optimization: Semidefinite approximations yield upper bounds

8 Positivstellensatze for semi-algebraic sets K R n from the knowledge of defining polynomials inner approximations of the cone of polynomials nonnegative on K Optimization: Semidefinite relaxations yield lower bounds Another look at nonnegativity from knowledge of a measure supported on K. outer approximations of the cone of polynomials nonnegative on K Optimization: Semidefinite approximations yield upper bounds

9 Let K R n be closed A basic question is: Characterize the continuous functions f : R n R that are nonnegative on K

10 and if possible... a characterization amenable to practical computation! Because then...

11 Positivstellensatze for basic semi-algebraic sets Let K := {x R n : g j (x) 0, j = 1,..., m}, for some polynomials (g j ) R[x]. Here, knowledge on K is through its defining polynomials (g j ) R[x]. Let C(K) d be the CONVEX cone of polynomials of degree at most d, nonnegative on K, and C d the CONVEX cone of polynomials of degree at most d, nonnegative on R n. Define x g J (x) := k J g k (x), J {1,..., m}.

12 The preordering associated with (g j ) is the set P(g) := σ J g J : σ J Σ[x] J {1,...,m} The quadratic module associated with (g j ) is the set m Q(g) := σ j g j : σ j Σ[x] j=1 Of course every element of P(g) or Q(g) is nonnegative on K, and the σ J (or the σ j ) provide certificates of nonnegativity on K.

13 The preordering associated with (g j ) is the set P(g) := σ J g J : σ J Σ[x] J {1,...,m} The quadratic module associated with (g j ) is the set m Q(g) := σ j g j : σ j Σ[x] j=1 Of course every element of P(g) or Q(g) is nonnegative on K, and the σ J (or the σ j ) provide certificates of nonnegativity on K.

14 Truncated versions The k-truncated preordering associated with (g j ) is the set P k (g) := J {1,...,m} σ J g J : σ J Σ[x], deg σ J g J 2k The k-truncated quadratic module associated with (g j ) is the set m Q k (g) := σ j g j : σ j Σ[x], deg σ J g J 2k j=1

15 Truncated versions The k-truncated preordering associated with (g j ) is the set P k (g) := J {1,...,m} σ J g J : σ J Σ[x], deg σ J g J 2k The k-truncated quadratic module associated with (g j ) is the set m Q k (g) := σ j g j : σ j Σ[x], deg σ J g J 2k j=1

16 d-truncated versions One may also define the convex cones P d k (g) := P k(g) R[x] d Q d k (g) := Q k(g) R[x] d

17 Observe that Q d k (g) Pd k (g) C(K) d, and so, the convex cones Qk d(g) and Pd k (g) provide inner approximations of C(K) d.... and... TESTING whether f P d k (g), or f Qd k (g) IS SOLVING an SDP! Provides the basis of moment-sos relaxations for polynomial programming!

18 Observe that Q d k (g) Pd k (g) C(K) d, and so, the convex cones Qk d(g) and Pd k (g) provide inner approximations of C(K) d.... and... TESTING whether f P d k (g), or f Qd k (g) IS SOLVING an SDP! Provides the basis of moment-sos relaxations for polynomial programming!

19 Observe that Q d k (g) Pd k (g) C(K) d, and so, the convex cones Qk d(g) and Pd k (g) provide inner approximations of C(K) d.... and... TESTING whether f P d k (g), or f Qd k (g) IS SOLVING an SDP! Provides the basis of moment-sos relaxations for polynomial programming!

20 Stengle s NichtNegativstellensatz f 0 on K h f = f 2s + p for some integer s, and polynomials h, p P(g). Moreover, bounds for s and degrees of h, p exist! Hence, GIVEN f R[x] d, cheking whether f 0 on K... reduces to solve a SINGLE SDP!...BUT.. its size is out of reach...!!! (hence try small degree certificates) it does not provide a NICE characterization of C(K) d, and not very practical for optimization purpose

21 Stengle s NichtNegativstellensatz f 0 on K h f = f 2s + p for some integer s, and polynomials h, p P(g). Moreover, bounds for s and degrees of h, p exist! Hence, GIVEN f R[x] d, cheking whether f 0 on K... reduces to solve a SINGLE SDP!...BUT.. its size is out of reach...!!! (hence try small degree certificates) it does not provide a NICE characterization of C(K) d, and not very practical for optimization purpose

22 Stengle s NichtNegativstellensatz f 0 on K h f = f 2s + p for some integer s, and polynomials h, p P(g). Moreover, bounds for s and degrees of h, p exist! Hence, GIVEN f R[x] d, cheking whether f 0 on K... reduces to solve a SINGLE SDP!...BUT.. its size is out of reach...!!! (hence try small degree certificates) it does not provide a NICE characterization of C(K) d, and not very practical for optimization purpose

23 Stengle s NichtNegativstellensatz f 0 on K h f = f 2s + p for some integer s, and polynomials h, p P(g). Moreover, bounds for s and degrees of h, p exist! Hence, GIVEN f R[x] d, cheking whether f 0 on K... reduces to solve a SINGLE SDP!...BUT.. its size is out of reach...!!! (hence try small degree certificates) it does not provide a NICE characterization of C(K) d, and not very practical for optimization purpose

24 Stengle s NichtNegativstellensatz f 0 on K h f = f 2s + p for some integer s, and polynomials h, p P(g). Moreover, bounds for s and degrees of h, p exist! Hence, GIVEN f R[x] d, cheking whether f 0 on K... reduces to solve a SINGLE SDP!...BUT.. its size is out of reach...!!! (hence try small degree certificates) it does not provide a NICE characterization of C(K) d, and not very practical for optimization purpose

25 Note in passing that f 0 on R n (i.e., f C d ) h f = p for some integer s, and polynomials h, p P(g). But again, it does not provide a nice characterization of the convex cone C d

26 Note in passing that f 0 on R n (i.e., f C d ) h f = p for some integer s, and polynomials h, p P(g). But again, it does not provide a nice characterization of the convex cone C d

27 Schmüdgen s Positivstellensatz [ K compact and f > 0 on K ] f P k (g) for some integer k. Putinar Positivstellensatz Assume that for some M > 0, the quadratic polynomial x M x 2 is in Q(g). Then: [ K compact and f > 0 on K ] f Q k (g) for some integer k.

28 Schmüdgen s Positivstellensatz [ K compact and f > 0 on K ] f P k (g) for some integer k. Putinar Positivstellensatz Assume that for some M > 0, the quadratic polynomial x M x 2 is in Q(g). Then: [ K compact and f > 0 on K ] f Q k (g) for some integer k.

29 Schmüdgen s Positivstellensatz [ K compact and f > 0 on K ] f P k (g) for some integer k. Putinar Positivstellensatz Assume that for some M > 0, the quadratic polynomial x M x 2 is in Q(g). Then: [ K compact and f > 0 on K ] f Q k (g) for some integer k.

30 Observe that if f 0 on K then for every ɛ > 0, there exists k such that f + ɛ Qk d(g) (or f + ɛ Qd k (g)) for some k... And so, the previous Positivstellensatze state that ( ) Pk d (g) = C(K) d k=0 and if x M x 2 is in Q(g) ( ) Qk d (g) k=0 = C(K) d

31 Duality Given a sequence y = (y α ), α N n, define the linear functional L y : R[x] R by: f (= α f α x α ) L y (f ) := α f α y α, f R[x] A sequence y has a representing Borel measure on K if there exists a finite Borel measure µ supported on K, such that y α = x α dµ(x), α N n. Theorem (Dual version of Putinar s theorem) K Let K be compact and assume that the polynomial M x 2 belongs to Q(g). Then y has a representing mesure supported on K if L y (h 2 ) 0, L y (h 2 g j ) 0, h R[x]

32 Moment matrix M k (y) with rows and columns indexed in N n k = {α Nn : i α i k}. M k (y)(α, β) := L y (x α+β ) = y α+β, α, β N n k For instance in R 2 : M 1 (y) = y 00 y 10 y 01 y 10 y 20 y 11 y 01 y 11 y 02 Then [ L y (f 2 ) 0, ] f, deg(f ) k M k (y) 0

33 Localizing matrix M r (θy) with respect to θ R[x] With x θ(x) = γ θ γ x γ M r (θ y)(α, β) = L y (θ x α+β ) = θ γ y α+β+γ, γ N n α, β N n k For instance, in R 2, and with X θ(x) := 1 x 2 1 x 2 2, M 1 (θ y) = y 00 y 20 y 02, y 10 y 30 y 12, y 01 y 21 y 03 y 10 y 30 y 12, y 20 y 40 y 22, y 11 y 21 y 12 y 01 y 21 y 03, y 11 y 21 y 12, y 02 y 22 y 04. Then [ L y (f 2 θ) 0, ] f, deg(f ) k M k (θ y) 0

34 Optimization: Hierarchy of semidefinite relaxations Consider the global optimization problem f = min{f (x) : x K} For every j, let v j := deg(g j )/2. Theorem Let K be compact and assume that the polynomial M x 2 belongs to Q(g). Consider the semidefinite programs: ρ k := max { λ : f λ Q k(g) } ρ k := min y L y (f ) s.t. L y (1) = 1 M k (y), M k vj (g j y) 0, j = 1,..., m Then ρ k ρ k for all k, and ρ k, ρ k f := min {f (x) : x K }

35 Optimization: Hierarchy of semidefinite relaxations Consider the global optimization problem f = min{f (x) : x K} For every j, let v j := deg(g j )/2. Theorem Let K be compact and assume that the polynomial M x 2 belongs to Q(g). Consider the semidefinite programs: ρ k := max { λ : f λ Q k(g) } ρ k := min y L y (f ) s.t. L y (1) = 1 M k (y), M k vj (g j y) 0, j = 1,..., m Then ρ k ρ k for all k, and ρ k, ρ k f := min {f (x) : x K }

36 Notice that the primal semidefinite program ρ k := min y L y (f ) s.t. L y (1) = 1 M k (y), M k vj (g j y) 0, j = 1,..., m is a relaxation of f = { } min f dµ : µ(k) = 1 µ M(K) K where M(K) is the space of finite Borel measures on K. Let y k be an optimal solution of the primal SDP If there is a unique global minimizer x K then µ = δ x and for every i = 1,..., n, L y k (x i ) x i as k.

37 Notice that the primal semidefinite program ρ k := min y L y (f ) s.t. L y (1) = 1 M k (y), M k vj (g j y) 0, j = 1,..., m is a relaxation of f = { } min f dµ : µ(k) = 1 µ M(K) K where M(K) is the space of finite Borel measures on K. Let y k be an optimal solution of the primal SDP If there is a unique global minimizer x K then µ = δ x and for every i = 1,..., n, L y k (x i ) x i as k.

38 Another look at of nonnegativity

39 Let K R n be an arbitrary closed set, and let f : R n R be a continuous function. Support of a measure On a separable metric space X, the support supp µ of a Borel measure µ is the (unique) smallest closed set such that µ(x \ K) = 0. Here the knowledge on K is through a measure µ with supp µ = K, and is independent of the representation of K. Lemma (Let µ be such that supp µ = K) A continuous function f : X R is nonnegative on K if and only if the signed Borel measure ν(b) = K B f dµ, B B, is a positive measure.

40 Let K R n be an arbitrary closed set, and let f : R n R be a continuous function. Support of a measure On a separable metric space X, the support supp µ of a Borel measure µ is the (unique) smallest closed set such that µ(x \ K) = 0. Here the knowledge on K is through a measure µ with supp µ = K, and is independent of the representation of K. Lemma (Let µ be such that supp µ = K) A continuous function f : X R is nonnegative on K if and only if the signed Borel measure ν(b) = K B f dµ, B B, is a positive measure.

41 proof The only if part is straightforward. For the if part, if ν is a positive measure then f (x) 0 for µ-almost all x K. That is, there is a Borel set G K such that µ(g) = 0 and f (x) 0 on K \ G. Next, observe that K \ G K and µ(k \ G) = µ(k). Therefore K \ G = K by minimality of K. Hence, let x K be fixed, arbitrary. As K = K \ G, there is a sequence (x k ) K \ G, k N, with x k x as k. But since f is continuous and f (x k ) 0 for every k N, we obtain the desired result f (x) 0.

42 proof The only if part is straightforward. For the if part, if ν is a positive measure then f (x) 0 for µ-almost all x K. That is, there is a Borel set G K such that µ(g) = 0 and f (x) 0 on K \ G. Next, observe that K \ G K and µ(k \ G) = µ(k). Therefore K \ G = K by minimality of K. Hence, let x K be fixed, arbitrary. As K = K \ G, there is a sequence (x k ) K \ G, k N, with x k x as k. But since f is continuous and f (x k ) 0 for every k N, we obtain the desired result f (x) 0.

43 proof The only if part is straightforward. For the if part, if ν is a positive measure then f (x) 0 for µ-almost all x K. That is, there is a Borel set G K such that µ(g) = 0 and f (x) 0 on K \ G. Next, observe that K \ G K and µ(k \ G) = µ(k). Therefore K \ G = K by minimality of K. Hence, let x K be fixed, arbitrary. As K = K \ G, there is a sequence (x k ) K \ G, k N, with x k x as k. But since f is continuous and f (x k ) 0 for every k N, we obtain the desired result f (x) 0.

44 Theorem Let K [ 1, 1] n be compact and let µ be an arbitrary, fixed, finite Borel measure on K with supp µ = K. Let f be a continuous function on R n and let z = (z α ), α N n, with z α = x α f (x)dµ(x), α N n. (a) f 0 on K if and only if K M k (z) 0, k = 0, 1,..., and if f R[x] then f 0 on K if and only if M k (f y) 0, k = 0, 1,... (b) If in addition to be continuous, f is also concave on K, then one may replace K with co (K).

45 Theorem Let K [ 1, 1] n be compact and let µ be an arbitrary, fixed, finite Borel measure on K with supp µ = K. Let f be a continuous function on R n and let z = (z α ), α N n, with z α = x α f (x)dµ(x), α N n. (a) f 0 on K if and only if K M k (z) 0, k = 0, 1,..., and if f R[x] then f 0 on K if and only if M k (f y) 0, k = 0, 1,... (b) If in addition to be continuous, f is also concave on K, then one may replace K with co (K).

46 Theorem Let K [ 1, 1] n be compact and let µ be an arbitrary, fixed, finite Borel measure on K with supp µ = K. Let f be a continuous function on R n and let z = (z α ), α N n, with z α = x α f (x)dµ(x), α N n. (a) f 0 on K if and only if K M k (z) 0, k = 0, 1,..., and if f R[x] then f 0 on K if and only if M k (f y) 0, k = 0, 1,... (b) If in addition to be continuous, f is also concave on K, then one may replace K with co (K).

47 Sketch of proof Consider the signed measure dν = f dµ. As K [ 1, 1] n, z α = x α f dµ f dµ = f 1, α N n. K K and so z is the moment sequence of a finite (positive) Borel measure ψ on [ 1, 1] n. As K is compact this implies ν = ψ, and so, ν is a positive Borel measure, and with support equal to K. By the Lemma that we have seen, f 0 on K.

48 Sketch of proof Consider the signed measure dν = f dµ. As K [ 1, 1] n, z α = x α f dµ f dµ = f 1, α N n. K K and so z is the moment sequence of a finite (positive) Borel measure ψ on [ 1, 1] n. As K is compact this implies ν = ψ, and so, ν is a positive Borel measure, and with support equal to K. By the Lemma that we have seen, f 0 on K.

49 Sketch of proof Consider the signed measure dν = f dµ. As K [ 1, 1] n, z α = x α f dµ f dµ = f 1, α N n. K K and so z is the moment sequence of a finite (positive) Borel measure ψ on [ 1, 1] n. As K is compact this implies ν = ψ, and so, ν is a positive Borel measure, and with support equal to K. By the Lemma that we have seen, f 0 on K.

50 Let identify f R[x] d with its vector of coefficient f R s(d), with s(d) = ( ) n+d n. Observe that, for every k = 1,... k := {f R s(d) : M k (f y) 0} is a spectrahedron in R s(d), that is,... one obtains a nested hierarchy of spectrahedra 0 1 k C(K) d, with no lifting, that provide tighter and tighter outer approximations of C(K) d.

51 Let identify f R[x] d with its vector of coefficient f R s(d), with s(d) = ( ) n+d n. Observe that, for every k = 1,... k := {f R s(d) : M k (f y) 0} is a spectrahedron in R s(d), that is,... one obtains a nested hierarchy of spectrahedra 0 1 k C(K) d, with no lifting, that provide tighter and tighter outer approximations of C(K) d.

52 So we get the sandwich Pk d(g) C(K) d k for all k, and ( ) ( ) Pk d (g) = C(K) d = k k=0 Inner approximations Outer approximations representation dependent independent of representation k=0

53 Application to optimization Theorem (A hierarchy of upper bounds) Let f R[x] d be fixed and K R n be closed. Let µ be such that supp µ = K and with moment sequence y = (y α ), α N n. Consider the hierarchy of semidefinite programs: u k = min σ K f σ dµ }{{} dν : K σdµ = 1; σ Σ[x] }{{} d, dν u k = max λ { λ : M k(f λ, y) 0 } = max λ { λ : λ M k(y) M k (f, y) } Then u k, u k f = min x {f (x) : x K}.

54 Interpetation of u k recall that f = min{ ψ K whereas u k = min ν { K f dψ : ψ(k) = 1, ψ(r n \ K) = 0} f σdµ }{{} dν : ν(k) = 1, ν(r n \ K) = 0; σ Σ[x] k } that is, one optimizes over the subspace of Borel probability measures absolutely continuous with respect to µ, and with density σ Σ[x] k. Ideally, when k is large, σ(x) > 0 in a neighborhood of a global minimizer x K.

55 Interpetation of u k recall that f = min{ ψ K whereas u k = min ν { K f dψ : ψ(k) = 1, ψ(r n \ K) = 0} f σdµ }{{} dν : ν(k) = 1, ν(r n \ K) = 0; σ Σ[x] k } that is, one optimizes over the subspace of Borel probability measures absolutely continuous with respect to µ, and with density σ Σ[x] k. Ideally, when k is large, σ(x) > 0 in a neighborhood of a global minimizer x K.

56 Interpetation of u k recall that f = min{ ψ K whereas u k = min ν { K f dψ : ψ(k) = 1, ψ(r n \ K) = 0} f σdµ }{{} dν : ν(k) = 1, ν(r n \ K) = 0; σ Σ[x] k } that is, one optimizes over the subspace of Borel probability measures absolutely continuous with respect to µ, and with density σ Σ[x] k. Ideally, when k is large, σ(x) > 0 in a neighborhood of a global minimizer x K.

57 Interpetation of u k recall that f = min{ ψ K whereas u k = min ν { K f dψ : ψ(k) = 1, ψ(r n \ K) = 0} f σdµ }{{} dν : ν(k) = 1, ν(r n \ K) = 0; σ Σ[x] k } that is, one optimizes over the subspace of Borel probability measures absolutely continuous with respect to µ, and with density σ Σ[x] k. Ideally, when k is large, σ(x) > 0 in a neighborhood of a global minimizer x K.

58 Also works for non-compact closed sets but then µ has to satisfy Carleman-type sufficient condition which limits the growth of the moments. For example, take dµ = e x 2 /2 dx The sequences of upper bounds (u k, uk ) complement the sequences of lower bounds (ρ k, ρ k ) obtained from SDP-relaxations. Of course, for practical computation, the previous semidefinite relaxations require knowledge of the moment sequence y = (y α ), α N n. This is possible for relatively simple sets K like a box, a simplex, the discrete set { 1, 1} n, an ellipsoid, etc., where one can take µ to be uniformly distributed, or K = R n (or K = R n +) with dµ = e x 2 /2 dx, K = R n dµ = e P i x i dx, K = R n +

59 Also works for non-compact closed sets but then µ has to satisfy Carleman-type sufficient condition which limits the growth of the moments. For example, take dµ = e x 2 /2 dx The sequences of upper bounds (u k, uk ) complement the sequences of lower bounds (ρ k, ρ k ) obtained from SDP-relaxations. Of course, for practical computation, the previous semidefinite relaxations require knowledge of the moment sequence y = (y α ), α N n. This is possible for relatively simple sets K like a box, a simplex, the discrete set { 1, 1} n, an ellipsoid, etc., where one can take µ to be uniformly distributed, or K = R n (or K = R n +) with dµ = e x 2 /2 dx, K = R n dµ = e P i x i dx, K = R n +

60 Also works for non-compact closed sets but then µ has to satisfy Carleman-type sufficient condition which limits the growth of the moments. For example, take dµ = e x 2 /2 dx The sequences of upper bounds (u k, uk ) complement the sequences of lower bounds (ρ k, ρ k ) obtained from SDP-relaxations. Of course, for practical computation, the previous semidefinite relaxations require knowledge of the moment sequence y = (y α ), α N n. This is possible for relatively simple sets K like a box, a simplex, the discrete set { 1, 1} n, an ellipsoid, etc., where one can take µ to be uniformly distributed, or K = R n (or K = R n +) with dµ = e x 2 /2 dx, K = R n dµ = e P i x i dx, K = R n +

61 Some experiments K = R 2 + with dµ = e P i x i dx so that y ij = i! j!, i, j = 0, 1,... x f (x) := x 2 1 x 2 2 (x x 2 2 1) with f = 1/ u k K = R 2 + and x f (x) = x 1 + (1 x 1 x 2 ) 2 with f = 0, not attained. u k

62 Some experiments K = R 2 + with dµ = e P i x i dx so that y ij = i! j!, i, j = 0, 1,... x f (x) := x 2 1 x 2 2 (x x 2 2 1) with f = 1/ u k K = R 2 + and x f (x) = x 1 + (1 x 1 x 2 ) 2 with f = 0, not attained. u k

63 Preliminary conclusions Rapid decrease in first steps, but poor convergence Numerical stability problems to be expected. Use bases different from the monomial basis. Rather see this technique as a complement to lower bounds obtained from semidefinite relaxations

64 Preliminary conclusions Rapid decrease in first steps, but poor convergence Numerical stability problems to be expected. Use bases different from the monomial basis. Rather see this technique as a complement to lower bounds obtained from semidefinite relaxations

65 Preliminary conclusions Rapid decrease in first steps, but poor convergence Numerical stability problems to be expected. Use bases different from the monomial basis. Rather see this technique as a complement to lower bounds obtained from semidefinite relaxations

66 Preliminary conclusions Rapid decrease in first steps, but poor convergence Numerical stability problems to be expected. Use bases different from the monomial basis. Rather see this technique as a complement to lower bounds obtained from semidefinite relaxations

67 THANK YOU!