Representations of Positive Polynomials: Theory, Practice, and

Representations of Positive Polynomials: Theory, Practice, and Applications Dept. of Mathematics and Computer Science Emory University, Atlanta, GA Currently: National Science Foundation Temple University November 17, 2014

In theory, theory and practice are the same. But in practice, they are different. Albert Einstein

SOS and Positive Polynomials Let R[X ] := R[X 1,..., X n ]. f R[X ] is a sum of squares, or sos if f = g 2 1 + + g 2 k for g i R[X ]. Let R[X ] 2 denote the sos polynomials. For S R n, we write f 0 (f > 0) on S to mean f takes only nonnegative (strictly positive) values on S. If f 0 on R n, we say f is positive semidefinite, or psd. Clearly f sos implies f is psd the square of a real number is positive. This simple fact and generalizations of it underlie a large body of theoretical and computational work on representations of positive polynomials.

Suppose S R n, f R[X ], and f 0 on S. By a (positivity) representation of f, or a certificate of positivity for f, on S we mean an algebraic expression for f, usually involving elements in R[X ] 2, from which one can deduce the positivity condition immediately. A theorem about the existence of such certificates is a representation theorem. For example, I claim f is psd, i.e. f 0 on R 2 : f (x, y) := x 4 + 4y 4 4xy 2 + 2x 2 y x 2 + y 2 2y + 1 The following representation for f proves the claim: f = (x 2 + y 1) 2 + (2y 2 x) 2

This talk is about the theory and practice of such representations. Theory: Results concerning the existence of a representation. Practice: Results on computational and algorithmic issues, e.g., finding and counting representations. We will give some history and background and take a random walk through results and mention an application or two.

Our story begins in 1888, when the 26-year-old David Hilbert published his seminal paper on sums of squares. It was well-known at the time that psd implies sos in two cases: In R[x] (one variable), by the Fundamental Theorem of Algebra, and f R[X ] of degree 2, by Sylvester s work. Hilbert showed (the homogenized version of) the following: every psd f of degree 4 in R[x, y] is sos, in all other cases there exist psd polynomials that are not sos.

In 1893, Hilbert proved that for n = 2 every psd polynomial in R[X ] can be written as a sum of squares of rational functions, equivalently, for such f there exists nonzero h R[X ] such that h 2 f R[X ] 2. Unable to prove that this holds for all psd polynomials, it became the 17th problem on Hilbert s list of 23 problems he gave in his address to the International Congress of Mathematicians in 1900.

E. Artin solve Hilbert s 17th problem: Theorem (Artin 1927) Suppose f R[X ] is psd. Then there is a nonzero h R[X ] such that h 2 f is sos. Note that an identity h 2 f = g1 2 + + g k 2 positivity for f on R n. is a certificate of Artin (along with Schreier) developed the theory of ordered fields in order to solve the 17th problem. And thus the subject of Real Algebra was born (more or less).

PSD ternary quartics The first result in Hilbert s 1888 paper was that every psd ternary quartic homogenous polynomial in 3 variables of degree 4 is a sum of squares of three quadratic forms. Hilbert s proof is not easy to understand and lacks detail in a number of key points. In 1977, Choi and Lam gave a short elementary proof that every psd ternary quartic is a sum of squares of five quadratic forms. In 2004, A. Pfister gave an elementary proof with five replaced by four and a constructive argument in the case that the t. q. has a non-trival real zero. In 2012, Pfister and C. Scheiderer gave a complete proof of Hilberts Theorem, using only elementary tools such as the theorems on implicit functions and symmetric functions.

Hilbert s paper did not address constructive questions: Given a psd ternary quartic p, how can one find q. forms f, g, h so that p = f 2 + g 2 + h 2? In how many fundamentally different ways can this be done? In 1929, A. Coble showed that every ternary quartic over C can be written as a sum of three squares of quadratic forms in 63 inequivalent ways. Work on quartic curves due to D. Plaumann, B. Sturmfels, and C. Vinzant from 2011 gives a new proof of the Coble result, yielding an algorithm for computing all representations in the smooth case.

What about the computational questions above? In 2000, in joint work with B. Reznick, we described methods for counting representations of a psd ternary quartic as a sum of 3 squares of q. forms. Experimentally, the Coble result (63 representations over C) occured, with 8 representations as a sum of 3 squares of q. forms over R. In 2004, in joint work with Reznick, F. Sottile, and C. Scheiderer, we showed that if p = 0 is smooth, a psd ternary quartic p has exactly 8 inequivalent representations as a sum of 3 squares of quadratic forms. In 2010, Scheiderer extended this analysis to the singular case.

Hilbert did not give an explicit example of a psd, not sos, polynomial. The first published examples did not appear until nearly 80 years later; the most famous example is due to Motzkin: M(x, y, z) = x 4 y 2 + x 2 y 4 + z 6 3x 2 y 2 z 2 M(x, y, z) is psd by the arithmetic-geometric inequality. Suppose M is sos, then we have M = i g 2 i ; g i = A i x 3 +B i x 2 y +C i x 2 z +D i xy 2 +E i xyz +...+J i z 3 Since M has no x 6 term A i = 0 for all i. Similarly, there are no y 3 terms in g i. Then since M has no x 4 z 2 and no y 4 z 2 terms, g i has no x 2 z and no y 2 z terms. Similarly, the g i have no xz 2, yz 2 terms. Thus we have M = i (B ix 2 y + D i xy 2 + E i xyz + J i z 3 ) 2, and the coefficient of x 2 y 2 z 2 is i E i 2 0, which is impossible.

Since then, many other examples and families of psd, not sos, polynomials have appeared, due to Robinson; Choi, Lam, and Reznick; and others. More recently, there have been attempts to understand Hilbert s proof and make it accessible. In a recent preprint, Reznick has isolated the underlying mechanism of Hilbert s proof and shown that it applies to more general situations. He produced new families of psd forms that are not sos.

The Cayley-Bacharach Theorem says that if two cubics in the projective plane meet in nine different points, then every cubic that passes through any eight of the points passes through the ninth point. Hilbert s proof of the existence of a psd, not sos, form uses this theorem. His construction starts from two cubics in general position; Reznick s work simplifies Hilbert s proof by starting with specific cubics. In 2012, G. Blekherman showed that the Cayley-Bacharach relations are in some sense the fundamental reason that there are psd polynomials that are not sos. In small cases, he gives a characterization of the difference between psd and sos forms.

In a recent preprint, using ideas and results from convex algebraic geometry, Blekherman, G. Smith, and M. Velasco have proven a striking theorem related to Hilbert s 1888 paper. They show that if X is a real irreducible nondegenerate projective subvariety where the real points are Zariski dense, then every nonnegative real quadratic form on X is a sum of squares of linear forms iff X is a variety of minimal degree. From this result they are able to deduce Hilbert s result on psd, not sos, forms. Further, they show that the exceptional Veronese surface corresponds to the exceptional case of ternary quartics thus giving a geometric explanation for the algebraic fact that a psd ternary quartic is a sum of squares of quadratic forms.

Pólya s Theorem for forms positive on a simplex In 1928, Pólya s proved a representation theorem for forms positive on the standard simplex n := {(x 1,..., x n ) R n x i 0, i x i = 1}. Theorem (Pólya s Theorem) Suppose f R[X ] is homogeneous and is strictly positive on n, then for sufficiently large N, all of the coefficients of (X 1 + + X n ) N f are strictly positive. Notice that ( X i ) N f = a α X α with all a α > 0 is a certificate of positivity for f on n.

Bounds for Pólya s Theorem and generalizations In 2000, in joint work with Reznick, we gave an explicit bound on the exponent N in terms of the degree, the coefficients and the minimum of the form on the simplex. It is not too hard to show that there cannot exist a bound on N which depends only on the degree. In 2012, in joint work with M. Castle and Reznick, we gave a complete characterization of forms p such that p is Pólya semi-positive, i.e., for large enough N N, (X 1 + + X n ) N p has nonnegative coefficients, along with a bound on the N needed. If p has zeros on n, then (X 1 + + X n ) N p will always have some zero coefficients.

Application: Copositive programming Copositive programming is linear programming over the cone of copositive matrices: C n = {n n symmetric matrices M x M x T 0, for all x R n +}. In 2001, De Klerk & Pasechnik showed that the stability number of a graph can be written as the solution to a copositive programming problem. They approximated the copositive programming problem using a sequence of cones which converge to C n and using the degree bounds for Pólya s Theorem, gave results for approximating the stability number of a graph.

Introduction and History psd ternary quartics PSD, not SOS polynomials Po lya s Theorem Finding SOS decompositions Rat Finding SOS decompositions For a given sos f R[X ], thanks to the magic of linear algebra, there are effective algorithms for writing f as a sum of squares. Suppose deg f = 2d and f = k X gi (x)2 (1) i=1 The linear algebra version of (1) is f (X ) = V BB T V T, V = monomials of deg d, and B is the matrix with i-th column the coefficients of gi. A = B B T is psd symmetric matrix, called a Gram matrix for f corresponding to (1).

For example, p = x 4 + y 4 + 1 = (x 2 ) 2 + (y 2 ) 2 + 1 2. With V = [ x 2 y 2 xy x y 1 ], 1 0 0 0 0 0 0 1 0 0 0 0 p = V 0 0 0 0 0 0 0 0 0 0 0 0 V T 0 0 0 0 0 0 0 0 0 0 0 1 But also p = (x 2 y 2 ) 2 + 2(xy) 2 + (1) 2 = 1 1 0 0 0 0 1 1 0 0 0 0 V 0 0 2 0 0 0 0 0 0 0 0 0 V T 0 0 0 0 0 0 0 0 0 0 0 1

f is sos is equivalent to the existence of a Gram matrix for f, i.e., a psd symmetric matrix A such that f (X ) = V A V T, (2) This was proven by Choi, Lam, and Reznick who also noted that: Inequivalent sos representations of f correspond to different Gram matrices If A is a Gram matrix for f then rank A is the number of squares in the sos representation of f corresponding to A. Further, there is an easy algorithm for producing an explicit decomposition f = g 2 i from a Gram matrix for f.

Suppose f R[X ] is sos with deg f = 2d, and A is a Gram matrix for f. Then A is an N N matrix, where N = ( ) n+d d. The set of all symmetric N N matrices A such that f (X ) = V A V T (3) is an affine subset L of the space of N N symmetric matrices. f is sos iff L P N, where P N is the convex cone of psd symmetric N N matrices over R. Finding this intersection is a semidefinite program (SDP). There are good numerical algorithms and software! for solving semidefinite programs. There is even software for deciding if f is sos: SOSTools (P. Parrilo et. al.)

Application: Global Optimization Problem: Find inf{f (α) α R n } (if it exists). Equivalent to finding the inf of {λ R f λ is psd }. We can look for the smallest λ so that f λ is sos: f sos := min{λ f λ R 2 }. (4) Then f sos is a lower bound for the minimum of f. Of course, f sos = is possible: psd sos! If f sos, then there is an a priori bound on the degree of the squares that occur and finding f sos can be implemented as an SDP.

Representations on semialgebraic sets In 1974, Stengle proved his celebrated Positivstellensatz a representation theorem for polynomials on semialgebraic sets. Later it was discovered that in the 1964, Krivine proved a similar theorem that went largely unnoticed. A semialgebraic set in R n is a subset defined by finitely many polynomial inequalities. For G = {g 1,..., g r } R[X ], S(G) denotes the basic closed semialgebraic set generated by G: S(G) = {α R n g 1 (α) 0,..., g r (α) 0}. Question: Suppose f 0 or f > 0 on S(G), does there exist a certificate of positivity for f on S(G)?

Fix S = S(G), there are two algebraic objects associated to G: The quadratic module generated by G, Q(G) := {s 0 + s 1 g 1 + + s r g r each s i R[X ] 2 }. The preorder generated by G, P(G) := { s ɛ g ɛ1 1... g r ɛr s ɛ R[X ] 2 }. ɛ {0,1} n A representation of f in either Q(G) or P(G) yields a certificate of positivity for f on S. Classic Positivstellensatz. (Stengle, Krivine) f > 0 on S iff there exist h 1, h 2 P(G) such that f h 1 = 1 + h 2.

In the 1990 s, connections with moment problems were discovered, which led to denominator free representation theorems due to Schmüdgen and Putinar. Fix a basic closed semialgebraic set S = S(G), the corresponding quadratic module Q = Q(G), and the corresponding preorder P = P(G). P (resp. Q) is archimedean if there is N N such that N Xi 2 P (resp. Q). Notice that N Xi 2 P = N Xi 2 0 on S. Hence S must be contained in the compact set {N Xi 2 0} and therefore is compact. And similarly for Q. We can think of an explicit representation of N Xi 2 P or Q as a certificate of compactness for S.

Two classic theorems in real algebraic geometry We continue to assume S is our basic closed semialgebraic set and P the corresponding preorder. Theorem S if and only if 1 P. Theorem S is compact if and only if P is archimedean. This is algebraic geometry at its most beautiful very natural algebraic properties correspond to very natural geometric properties. Quadratic modules are simpler computationally than preorders and are very useful. However, it is not true that if S is compact then the corresponding quadratic module is always archimedian.

Representations on compact semialgebraic sets Theorem (Schmüdgen, 1991) If S is compact, then f > 0 on S implies f P. Theorem (Putinar, 1993) If Q is archimedean and f > 0 on S, then f Q. This means that if S is compact, then there exists a certificate of positivity for every f > 0 on S. Notice that these hold *regardless of the polynomials we choose to describe S*. Schuüdgen s result was a corollary to his solution to the multi-dimensional Moment Problem for compact semialgebraic sets. The proof uses the classic Positivstellensatz to non-contructively produce a certificate of compactness for S.

Optimizing on compact semialgebraic sets Suppose S = S(g 1,..., g k ) is a compact semialgebraic set and Q the quadratic module. Assume Q is archimedean, so Putinar s Theorem applies. For f R[X ], let f denote the minimum of f on S (which always exists, since S is compact). Let f sos := min{λ f λ Q}. As in the sos case, the problem of finding a representation of f λ in Q with a fixed upper bound on the degree of the sums of squares that occur can be implemented as an SDP. Unfortunately, there cannot be any such bound in terms of deg f only any bound must depend on some measure of how close f is to having a zero on Q.

The Lasserre Method Unlike the global optimization case, f sos always exists. But we cannot implement finding f sos as a SDP directly. Let Q a := {g Q deg g a}. J.-P. Lasserre developed a method for approximating f by implementing a series of SDP s corresponding to finding f a sos := min{λ f λ Q a }. By Putinar s Theorem, the values of f a sos converge to f. We can ensure that Q is archimedean by adding a certificate of compactness N Xi 2 to the set of generators. Question: Given G = {g 1,..., g k R[X ], is there an algorithm to decide if S(G) is compact and/or compute N so that N Xi 2 > 0 on S?

In 2002, M. Schweighofer gave a proof of Schmüdgen s theorem that is algorithmic modulo one application of the classic Positivstellensatz. His proof reduces the theorem to Pólya s Theorem. Using the bounds for Pólya s Theorem, Schweighofer obtains degree bounds for the sums of squares needed in a represenation, in terms of the coefficients and degrees of the input data and the degree of the sums of squares needed in a certificate of compactness. In 2007, Schweighofer and J. Nie gave a similar constructive proof of Putinar s Theorem using Pólya s Theorem with bound. Their results give information about the convergence rate of Lasserre s method for optimization.

Question: Can the Pólya-with-zeros theorem allow the constructive proof of Schmüdgen s Theorem and the bound to be generalized to include polynomials in the preorder which have zeros on S? (Or Putinar s Theorem) One problem: The algorithm in the Schweighofer s proof yields representations in a particularly nice form. It is not hard to show that not all elements in P have this nice form there is something special about f which are strictly positive on S. A corollary to a (deep!) result of C. Scheiderer says that any f which is nonnegative on the square {1 x 0, 1 + x 0, 1 y 0, 1 + y 0} is in the quadratic module generated by {1 ± x, 1 ± y}. Open question: Does there exist an algorithmic/constructive proof of this?

Rational sums of squares Consider the following example, due to C. Hillar: Is f = 3 12x 6x 3 + 18y 2 + 3x 6 + 12x 3 y 6xy 3 + 6x 2 y 4 sos? Our SDP program might say yes and return a certificate: f = (x 3 + 3.53y +.347xy 2 1) 2 + (x 3 +.12y + 1.53xy 2 1) 2 + (x 3 + 2.35y 1.88xy 2 1) 2 Note that f RHS has terms such as.006xy 2 which are nonzero. Is f really a sum of squares? It turns out that our SDP program has approximated an sos decomposition for f of the form (x 3 +a 2 y +bxy 2 1) 2 +(x 3 +b 2 y +cxy 2 1) 2 +(x 3 +c 2 y +axy 2 1) 2, where a, b, c are real roots of the cubic x 3 3x + 1.

For many applications of certificates of positivity, we need exact certificates, but SDP s are numerical. In theory, SDP problems can be solved purely algebraically, e.g., using quantifier elimination. In practice, this is impossible for all but trivial problems. Work by J. Nie, K. Ranestand, and B. Sturmfels in 2010 shows that optimal solutions of relatively small SDP s can have minimum defining polynomials of huge degree. Much more promising are hybrid symbolic-numeric approaches.

Problem: Given a numerical (approximate) certificate f = g 2 i (via SDP software, say) find an exact certificate. Peyrl and Parrilo gave an algorithm for solving this problem, in some cases. The idea: We want to find a symmetric psd matrix A with rational entries so that f = V A V T (5) The SDP software will produce a psd matrix A which only approximately satisfies (5). The idea is to project A onto the affine space of solutions to (5) in such a way that the projection remains in the cone of psd symmetric matrices.

The Peyrl-Parrilo method is (in theory!) guaranteed to work IF there exists a rational solution and the underlying SDP is strictly feasible, i.e., there is a solution with full rank. In 2009, E. Kaltofen, B. Li, Z. Yang, and L. Zhi generalized the technique of Peyrl and Parrilo and found sos certificates certifying rational lower bounds for some approximate polynomial g.c.d. and irreducibility problems.

Rational sums of squares decompositions The above algorithms assume that there is a rational sos certificate for f. But maybe one doesn t exist, even if f has rational coefficients. Suppose f Q[X ] is in R[X ] 2. Sturmfels asked: Is f Q[X ] 2? A trivial, but illustrative example: 2x 2 = ( 2x) 2 R[x] 2 2x 2 = x 2 + x 2 Q[x] 2

Recall the Hillar example f = 3 12x 6x 3 + 18y 2 + 3x 6 + 12x 3 y 6xy 3 + 6x 2 y 4, which is in R[X ] 2. It turns out that f Q[X ] 2 : f = (x 3 + xy 2 + 3 2 y 1)2 + (x 3 + 2y 1) 2 + (x 3 xy 2 + 5 2 y 1)2 + (2y xy 2 ) 2 + 3 2 y 2 + 3x 2 y 4 In both of these examples, the sos decompositions in Q[X ] 2 use more squares than the ones in R[X ] 2.

Sturmfels asked: If f Q[X ] and f R[X ] 2, is f Q[X ] 2? In 2009, Hillar showed that the answer to Sturmfel s question is yes if f K 2, where K is a totally real extension of Q. He gave bounds for the number of squares needed. There is a simple proof of a slightly more general result with a better bound given (independently) by Kaltofen, Scheiderer, and Quarez. In 2012, Scheiderer showed that the answer is no, in general.

Back to the compact semialgebraic sets What can we say about rational certificates of positivity in the compact semialgebraic set case? Fix G = {g 1,..., g k } R[X ] and let S = S(G), the basic closed semialgebraic set generated by G, Q = Q(G), the quadratic module generated by G, and P = P(G), the preorder generated by G. Question: Suppose G Q[X ], f Q[X ], and f > 0 on S. Does there exist a rational certificate of positivity for f on S?

Theorem Suppose S = S(G) is compact. Then for any f Q[X ] such that f > 0 on S, there is a representation of f in the preordering P(G), with all σ e Q[X ] 2. f = σ e g e1 1... g es e {0,1} s The proof follows from an algebraic proof of Schmüdgen s Theorem, due to T. Wörmann. s,

Theorem (2011) Suppose G = {g 1,..., g s } Q[X ] and N Xi 2 Q for some N N. Then given any f Q[X ] such that f > 0 on S, there exist σ 0... σ s, σ Q[X ] 2 so that f = σ 0 + σ 1 g 1 + + σ s g s + σ(n X 2 i ). This follows from the algorithmic proof of Putinar s Theorem due to Schweighofer. We follow the proof, making sure each step preserves rationality. Open question: If G Q[X ] and N Xi 2 Q, can we find a certificate of compactness N Xi 2 in Q with the sos s in Q[X ] 2?

Other Topics There are so many interesting topics that we haven t mentioned! Here are some topics we missed: Representations on noncompact semialgebraic sets Invariant sums of squares Convex real algebraic geometry Positivity in a non-commutative setting, free positivity. Applications to graph theoretic problems max cut, max clique and others. Applications to control theory, e.g., Lapynov functions Applications to operator algebras and functional analysis, e.g., moment problems. Linear matrix inequalities, hyperbolic polynomials, spectral cones

Thanks for listening!