Moments Tensors, Hilbert s Identity, and k-wise Uncorrelated Random Variables

Transcription

1 Moments Tensors, Hilbert s Ientity, an k-wise Uncorrelate Ranom Variables Bo JIANG Simai HE Zhening LI Shuzhong ZHANG First version January 011; final version September 013 Abstract In this paper we introuce a notion to be calle k-wise uncorrelate ranom variables, which is similar but not ientical to the so-calle k-wise inepenent ranom variables in the literature. We show how to construct k-wise uncorrelate ranom variables by a simple proceure. The constructe ranom variables can be applie, e.g. to express the quartic polynomial (x T Qx), where Q is an n n positive semiefinite matrix, by a sum of fourth powere polynomial terms, known as Hilbert s ientity. By virtue of the propose construction, the number of require terms is no more than n 4 + n. This implies that it is possible to fin a (n 4 + n)-point istribution whose fourth moments tensor is exactly the symmetrization of Q Q. Moreover, we prove that the number of require fourth powere polynomial terms to express (x T Qx) is at least n(n + 1)/. The result is applie to prove that computing the matrix 4 norm is NP-har. Extensions of the results to complex ranom variables are iscusse as well. Keywors: cone of moments, uncorrelate ranom variables, Hilbert s ientity, matrix norm. Mathematics Subect Classification: 78M05, 6H0, 15A60. Department of Inustrial an Systems Engineering, University of Minnesota, Minneapolis, MN iang373@umn.eu Department of Management Sciences, City University of Hong Kong, Hong Kong. simaihe@cityu.eu.hk. Research of this author was supporte in part by Hong Kong GRF Grant uner Grant Number CityU Department of Mathematics, Shanghai University, Shanghai 00444, China. zheningli@gmail.com. Research of this author was supporte in part by Natural Science Founation of China #113714, Natural Science Founation of Shanghai #1ZR , an Ph.D. Programs Founation of Chinese Ministry of Eucation # Current aress: Department of Mathematics, University of Portsmouth, Portsmouth PO1 3HF, Unite Kingom. Department of Inustrial an Systems Engineering, University of Minnesota, Minneapolis, MN zhangs@umn.eu. Number CMMI Research of this author was supporte in part by the National Science Founation uner Grant 1

2 1 Introuction Given an n-imensional ranom vector ξ (ξ 1, ξ,..., ξ n ) T with oint ensity function p( ), let us enote the n-imensional -th orer tensor F to be the -th orer moments tensor associate with ξ as follows: [ ] F i1 i...i E ξ ik k1 R n k1 u ik p(u)u 1 i 1, i,..., i n; or equivalently, F u u u p(u)u. R n Since tensor F is in a finite imensional space, by Carathéoory s theorem [6], it can be further rewritten as a sum of finite rank one terms, i.e., there exist t vectors b 1, b,..., b t such that F t b i b i b i. (1) An immeiate consequence of the above construction is that F is super-symmetric, meaning that its component is invariant uner permutation of the inices. For instance, the secon orer moments tensor can be easily erive from its covariance matrix, which is naturally symmetric an positive semiefinite. Inee, thanks to the formulation (1), any -th orer moments tensor is always positive semiefinite, in other wors, the homogeneous polynomial function inuce by this tensor is always nonnegative, i.e., f(x) F(x, x,..., x) : F i1 i...i 1 i 1,i,...,i n k1 x ik t ( (b i ) T x ) 0. However, the term nonnegativity can be ambiguous in the case of higher orer tensors. In our recent paper [11], this issue was particularly aresse. We shall only note here that the -th moments tensors form a specific nonnegative convex cone, whose membership query is a har problem in general (see [11]). It is therefore interesting to know what kin of tensors are containe in this cone. For instance, one may woner if the super-symmetric tensor associate with the polynomial (x T x), which is clearly nonnegative, is a fourth orer moments tensor or not. Interestingly, the answer is yes, ue to a result of Hilbert [10], who showe that it is possible to express (x T x) as t (xt a i ). As a consequence, the polynomial (x T x) (the case ) can be viewe as E[ξ T x] 4 where ξ is a ranom vector, taking value t 1/4 a i with probability 1/t. Therefore, sym (I I) with I being the ientity matrix is a fourth moments tensor, where the symmetrization mapping sym turns a given tensor into a super-symmetric one by making the entries with the same set of inices all the same (taking the value of the average). Apart from the above example, there are several other representations for general -th moments tensor other than (1). For example, with the help of Hilbert s ientity [4], we can easily verify that

3 sym (A A A ) with A 0 also belongs to -th moments cone. Specifically, one can fin vectors a 1, a,..., a t such that sym (A A A ) t a i a i a i. () On the other han, by letting the orer of the tensor be an A i b i b i b i (b i ) T in (1), we have F t b i b i b i t sym (A i A i A i ), with A i 0 an rank(a i ) 1. (3) This implies that the rank-one constraint is reunant in terms of requiring F to be a -th moments tensor in (3). In general, such ecomposition of () is not unique. For example, one may verify that (x 1 + x + x 3) x 4 i (x i + β x ) i< 3 β ±1 3 x 4 i β ±1 β 3 ±1 (x 1 + β x + β 3 x 3 ) 4, which leas to two ifferent representations of the tensor sym (I 3 I 3 ). An interesting question is to fin a succinct (preferably the shortest) representation among all the ifferent representations, incluing the one from Hilbert s ecomposition. However, from the original Hilbert s construction, the ( n+ 1 representation on the right han sie of () is exponential in n. By Carathéoory s ) theorem, there exists a ecomposition such that the value of t in () is no more than + 1. Unfortunately, Carathéoory s theorem is non-constructive. This motivates us to construct a polynomial-size representation, i.e., t O(n k ) for some constant k in (). One contribution of this paper is to give a short (polynomial-size) representation for Hilbert s ientity when. In fact, we also prove the number of terms for any representation can never be less than n(n + 1)/. An application of this polynomial-size representation will be iscusse. Towar this en, let us first introuce the new notion of k-wise uncorrelate ranom variables, which may appear to be completely unrelate to the iscussion of Hilbert s ientity at first glance. Definition 1.1 (k-wise uncorrelation) A set of ranom variables {ξ 1, ξ,..., ξ n is calle k-wise uncorrelate if n E ξ p n [ E ξ p ] p 1, p,..., p n Z + with p i k. For instance, if ξ 1, ξ,..., ξ n are i.i.. ranom variables with finite supporting set q, then they are k-wise uncorrelate. However the size of its corresponing sample space is q n, which is 3

4 exponential in n. It turns out that reucing the sample space while keeping the k-wise uncorrelation structure can be of great importance in many applications. For example, our result shows that the polynomial-size representation () can be obtaine by fining k-wise uncorrelate ranom variables with polynomial-size sample space. Before aressing the issue of fining such ranom variables, below we shall first iscuss a relate notion known as the k-wise inepenence. Definition 1. (k-wise inepenence) A set of ranom variables Ξ {ξ 1, ξ,..., ξ n with each taking values on the set {δ 1, δ,..., δ q is calle k-wise inepenent, if any k ifferent ranom variables ξ i1, ξ i,..., ξ ik of Ξ are inepenent, i.e., k Prob {ξ i1 δ i1, ξ i δ i,..., ξ ik δ ik Prob { ξ i δ i δ i, 1,,..., k. Note that when k, k-wise inepenence is usually calle pair-wise inepenence. Since 1980 s, k-wise inepenence has been a popular topic in theoretical computer science. Essentially, working with k-wise inepenence (instea of the full inepenence) means that one can reuce the size of the sample space in question. In many cases, this feature is crucial. For instance, when {0, 1 an Prob {ξ 1 0 Prob {ξ 1 1 1, Alon, Babai, an Itai [1] constructe a sample space of size being approximately n k. For the same, when ξ 1, ξ,..., ξ n are inepenent but not ientical, Karloff an Mansour [13] prove that the size of sample space can be upper boune by O(n k ). In the case of {0, 1,..., q 1 with q being a prime number, the total number of ranom variables being k-wise inepenent are quite restricte. For given k < q, Joffe [1] showe that there are up to q + 1 ranom variables form a k-wise inepenent set an the size of the sample space is q k. Clearly, k-wise inepenence implies k-wise uncorrelation. Therefore, we may apply the existing results of k-wise inepenence to get k-wise uncorrelate ranom variables. However, the aforementione constructions of k-wise inepenent ranom variables heavily epen on the structure of (e.g. it requires that or k < ). Moreover, the construction of k-wise inepenent ranom variables is typically complicate an technically involve (see [13]). In fact, for certain problems (e.g. polynomial-size representation of Hilbert s ientity in this case), we only nee the ranom variables to be k-wise uncorrelate. Therefore, in this paper we propose a tailor-mae simple construction which suits the structure of k-wise uncorrelate ranom variables. As we shall see later, our approach can hanle the more general support set: q : {1, ω q,..., ω q 1 q, with ω q e i π q cos π q + i sin π q an q is prime, (4) an k can be any parameter. Conceptually, our approach is rather generic: the k-wise uncorrelate ranom variables are constructe base only on the prouct of a small set of i.i.. ranom variables with their powers; the sample space woul be polynomial-size if the number of such i.i.. ranom variables is O(log n). Consequently, we not only fin polynomial-size representation for the fourth 4

5 moments tensor in form of sym (A A), but also for complex q-th moments tensor. As an application, this construction can be use to prove that the matrix 4 norm problem [5], whose complexity was previously unknown 1, is actually NP-har. The rest of this paper is organize as follows. In Section we introuce Hilbert s ientity an its connections to -th moments tensor. Then, in Section 3 we present a ranomize algorithm, as well as a eterministic one, to construct k-wise uncorrelate ranom variables. As a result, we fin polynomial-size representation of fourth moments tensor an complex q-th moments tensor in Section 4. In Section 5, we iscuss the shortest representation of Hilbert s ientity an its relate tensor rank problem, in particular proviing a lower boun for the number of terms in the ientity. Finally, we conclue this paper with an application of etermining the complexity of matrix 4 norm problem, to illustrate the usefulness of our approach. Notation. Throughout we aopt the notation of the lower-case letters to enote vectors (e.g. x R n ), the capital letters to enote matrices (e.g. A R n ), an the capital calligraphy letters to enote higher ( 3) orer tensors (e.g. F R n4 ), with subscriptions of inices being their entries (e.g. x 1, A i, F i1 i i 3 i 4 R). A tensor is sai to be super-symmetric if its entries are invariant uner all permutations of its inices. As mentione earlier, the symmetrization mapping sym makes a given tensor to be super-symmetric, which is F sym (G) with F i1 i...i 1 Π(i 1 i... i ) π Π(i 1 i...i ) G π 1 i 1, i,..., i n, where Π(i 1 i... i ) is the set of all istinct permutations of the inices {i 1, i,..., i. The symbol represents the outer prouct of vectors or matrices. In particular, if F x x x for some x R n, then F i1 i...i for some X R n, then k1 x i k ; an if G X X X G i1 i...i k1 X i k 1 i k. Besies, enotes the supporting set of certain ranom variable, an Ω R n is the sample space of a set of ranom variables {ξ 1, ξ,..., ξ n, i.e., the space of all possible outcomes of (ξ 1, ξ,..., ξ n ) T. Finally, the following two subsets of Z n + are frequently use in the iscussion, an for given prime number q, P n k : { (p 1, p,..., p n ) T Z n + p 1 + p + + p n k, It is easy to see that P n k (q) Pn k ( ) n+k 1 k. P n k (q) : {p Pn k i (1 i n) such that q p i. 1 During the review process of this paper, Barak et al. [3] inepenently prove that it is NP-harness to compute the matrix 4 norm. 5

6 Hilbert s Ientity an -th Moments Tensor Let us start our iscussion with the famous Hilbert s ientity, which states that for any fixe positive integers an n, there always exist rational vectors b 1, b,..., b t R n such that ( ) t x ( i (b ) T x ) x (x 1, x,..., x n ) T R n. (5) For instance, when n 4 an, we have (x 1 + x + x 3 + x 4) 1 6 (x i + x ) i< 4 (x i x ) 4, (6) 1 i< 4 which is calle Liouville s ientity. It is worth mentioning that Hilbert s ientity is very well known an is a funamental result in mathematics. For example, with the help of (5), Reznick [18] manage to prove the following result: Let p(x) be -th egree homogeneous positive polynomial in x R n. Then there exists a positive integer r an vectors b 1, b,..., b r R n such that x r p(x) r ((b i ) T x) r. Reznick s result above solve Hilbert s seventeenth problem constructively (albeit only for the case p(x) being positive efinite). As another example, Hilbert [10] in 1909 solve Waring s problem: Can every positive integer be expresse as a sum of at most g(k) k-th powers of positive integers, where g(k) epens only on k, not on the number being represente? in the affirmative for all k. The key unerpinning tool in the proof is also Hilbert s ientity (5); see e.g. [7, 16] for more stories on Warning s problem an Hilbert s ientity. In fact, Hilbert s ientity can be reaily extene to a more general setting. For any given A 0, by letting y A 1 x an applying (5), one has (x T Ax) (y T y) t ( (b ) T y ) t ( ) (b ) T A 1 x, which guarantees the existence of vectors a 1, a,..., a t R n with a A 1 b for 1,,..., t such that (x T Ax) t ( (a ) T x ). (7) 6

7 The iscussion so far appears to be only concerne about ecomposing a specific polynomial function. Let us now relate Hilbert s ientity to the moments tensor. Observe that super-symmetric tensors are biectively relate to homogenous polynomial functions. In particular, if f(x) 1 i 1 i i n G i1 i...i k1 x ik is a -th egree homogenous polynomial, then its associate super-symmetric tensor F with F i1 i...i G i1 i...i / Π(i 1 i... i ) is uniquely etermine by f(x) F(x, x,..., x), an vice versa. This is the same as the one-to-one corresponence between symmetric matrices an quaratic forms. Therefore, the tensor sym (A A A ) is associate with the polynomial ( x T Ax ), an the following relationship hols immeiately. Proposition.1 For any A 0, there exit vectors a 1, a,..., a t R n such that (x T Ax) t ( (a ) T x ), i.e., sym (A A A ) t ai a i a i. This implies that tensor sym (A A A ) is a -th moments tensor if A 0. As we mentione earlier, the size of such representation from Hilbert s ientity is exponential in n. To see this, let us recall the claim of Hilbert (see [14]): Given fixe positive integers an n, there exist + 1 real numbers β 1, β,..., β +1, + 1 positive real numbers ρ 1, ρ,..., ρ +1, an a positive real number α, such that (x T x) 1 α i 1 1 i 1 i +1 1 ρ i1 ρ i... ρ i+1 (β i1 x 1 + β i x + + β i+1 x i+1 ). (8) It is obvious that the number of -powere linear terms on the right han sie of (8) is ( + 1) n, which is too lengthy for practical purposes. In the following, let us focus on how to get a polynomialsize ecomposition of Hilbert s ientity, or essentially the tensor sym (A A A ) with A 0. In light of the above iscussion, it suffices to fin a polynomial-size representation of (5). Towar this en, let us first rewrite (x T x) in terms of the expectation of a polynomial function. In particular, by efining i.i.. ranom variables ξ 1, ξ,..., ξ n with supporting set {β 1, β,..., β +1 an Prob (ξ k β i ) ρ i γ for all 1 i + 1 an 1 k n, where γ +1 β i, ientity (8) is equivalent to (x T x) γ E γ n n ξ x E ξ p x p γ n [ ] E ξ p n x p α α α. (9) p P n p P n 7

8 As a consequence, if for any n ranom variables η 1, η,..., η n satisfying n n [ ] E E p P n, (10) [ ] an E η p (x T x) γ α E η p η p E [ξ p 1 ] for all 0 < p an 1 n, then it is straightforwar to verify that [ (n ) η x ]. Notice that (10) is actually equivalent to η 1, η,..., η n being -wise uncorrelate, an we have the next result following (9) an (10). Proposition. If ξ 1, ξ,..., ξ n are i.i.. ranom [ ] variables, an η 1, η,..., η n are -wise uncorrelate, satisfying the moments constraints E η p E [ξ p 1 ] for all 0 < p an 1 n, then [ (n ) ] [ (n ) E ξ x E η x ]. We en this section with the conclusion that the key to reucing the length of representation in (5) is to construct -wise uncorrelate ranom variables satisfying certain moments conitions, such that the sample space is as small as possible, which will be the subect of our subsequent iscussions. As we will see later, the construction makes use of the structure of the support set (4). For general support sets, the techniques consiere in [13] may be useful, an it is a topic for future research. 3 Construction of k-wise Uncorrelate Ranom Variables In this section, we shall construct k-wise uncorrelate ranom variables, which are ientical an uniformly istribute on q efine by (4). The rough iea is as follows. We first generate m i.i.. ranom variables ξ 1, ξ,..., ξ m, base on which we can efine new ranom variables η 1, η,..., η n such that η i : 1 m ξc i for i 1,,..., n. Therefore, the size of sample space of {η 1, η,..., η n is boune above by q m, which yiels a polynomial-size space if we let m O(log q n). The remaining part of this section is evote to the iscussion of the property for the power inices c i s, in orer to guarantee η 1, η,..., η n to be k-wise uncorrelate, an how to fin those power inices. 3.1 k-wise Regular Sequence Let us start with some notations an efinitions for the preparation. Suppose c is a number with m igits an c[l] is the value of its l-th bit. We call c to be enowe with the base q, if c[l] {0, 1,..., q 1 for all 1 l m. In other wors, c m l1 c[l]ql 1. Now we can efine the concept of k-wise regular sequence as follows. 8

9 Definition 3.1 A sequence of m igits numbers {c 1, c,..., c n of base q is calle k-wise regular if for any p P n k (q), there exists l (1 l m) such that p c [l] 0 mo q. Why are we intereste in such regular sequences? The answer lies in the following proposition. Proposition 3. Suppose m igits numbers {c 1, c,..., c n of base q are k-wise regular, where q is a prime number, an ξ 1, ξ,..., ξ m are i.i.. ranom variables uniformly istribute on q. Then η 1, η,..., η n with are k-wise uncorrelate. η i : 1 l m ξ c i[l] l, i 1,,..., n (11) Proof. Let η 1, η,..., η n be efine as in (11). As ξ i is uniformly istribute on q for 1 i m an q is prime, we have { ] E[ξ p i [η ] E p 1 if q p, 0 otherwise for any i an any with c (0, 0,..., 0). Otherwise if c (0, 0,..., 0) for some, then E For any given p P n k, if q p i for all 1 i n, then n E E ξ p 1 c 1 [l] η p 1 l m 1 l m [ E ξ l n p c [l] l ξ p c [l] l 1 l m ] 1 n [ E... η p ]. ξ pn cn[l] l 1 l m [ ] η p 1. Otherwise, there exists some i 0 such that q p i0, implying that p P n k (q). By k-wise regularity, we can fin some l 0 satisfying [ n n p c [l 0 ] 0 mo q, implies that E ξ p ] c [l 0 ] l 0 0. Moreover, there [ exists ] some 0 such that p 0 c 0 [l 0 ] 0 mo q, i.e., q p 0 an c 0 [l 0 ] 0. This leas to E η p 0 0 0, an we have n E η p 1 l m [ E ξ n p c [l] l ] 0 n [ E η p ], an the conclusion follows. 9

10 3. A Ranomize Algorithm We shall now focus on how to fin such k-wise regular sequence {c 1, c,..., c n of base q. First, we present a ranomize process, in which c i [l] is ranomly an uniformly chosen from {0, 1,..., q 1 for all 1 i n an 1 l m. The algorithm is as follows. Algorithm RAN Input: Dimension n an m : k log q n. Output: A sequence {c 1, c,..., c n in m igits of base q. Step 0: Construct S {(0,..., 0, 0), (0,..., 0, 1),..., (q 1,..., q 1, q 1) of base q. m m m Step 1: Inepenently an uniformly take c i S for i 1,,..., n. Step : Assemble the sequence {c 1, c,..., c n an exit. Theorem 3.3 If 1 < k < n an q is a prime number, then Algorithm RAN returns a k-wise m-igit regular sequence {c 1, c,..., c n of base q with probability at least 1 (1.5)k 1 k!, which is inepenent of n an q. Proof. Since {c 1, c,..., c n is a sequence of m-igit numbers of base q, if it is not regular, then there exist p P n k, such that p c [l] 0 mo q 1 l m. Therefore, we have Prob c 1, c,..., c n is not k-wise regular Prob p c [l] 0 mo q, 1 l m. p P n k (q) For any given p P n k (q), we may without loss of generality assume that q p n. If we fix c 1, c,..., c n 1, as q is prime, then there is only one solution for c n such that n p c [l] 0 mo q, 1 l m. Combining the fact that c 1, c,..., c n are inepenently an uniformly 10

11 generate, we have Prob p c [l] 0 mo q, 1 l m Prob p c [l] 0 mo q, 1 l m c 1 1, c,..., c n 1 n 1 Prob {c 1 1, c,..., c n 1 n 1 1 q m 1,,..., n 1 S 1,,..., n 1 S Prob {c 1 1, c,..., c n 1 n 1 1 n k. (1) Finally, Prob c 1, c,..., c n is k-wise regular 1 Prob c 1, c,..., c n is not k-wise regular ( ) 1 P n k (q) 1 n k 1 Pn k 1 n + k 1 n k 1 1 k n k 1 (1.5)k 1. k! For some special q an k, in particular relating to the the simplest case of Hilbert s ientity (4-wise regular sequence of base ), the lower boun of the probability in Theorem 3.3 can be improve. Proposition 3.4 If k 4 an q, then Algorithm RAN returns a 4-wise regular sequence {c 1, c,..., c n of base with probability at least 1 1 n 1 4!. The proof is similar to that of Theorem 3.3, an thus is omitte. 3.3 Deranomization Although k-wise regular sequence always exists an can be foun with high probability, one may however wish to construct such regular sequence eterministically. In fact, this is possible if we apply Theorem 3.3 in a slightly ifferent manner, which is shown in the following algorithm. Basically, we start with a short regular sequence C, an enumerate all the remaining numbers in orer to fin c such that C {c is also regular. Upating C with C {c, we repeat this proceure until the carinality of C reaches n. Moreover, thanks to the polynomial-size sample space, this brute force approach still runs in polynomial-time. 11

12 Algorithm DET Input: Dimension n an m : k log q n. Output: A sequence {c 1, c,..., c n in m igits of base q. Step 0: Construct S {(0,..., 0, 0), (0,..., 0, 1),..., (q 1,..., q 1, q 1) of base q, an m m m a sequence C : {c 1, c,..., c k in m igits, where c i : (0,..., 0, 0, 1, 0,..., 0, 0) for k 1 i 1,,..., k. Let the inex count be τ : k. Step 1: Step : If τ n, then go to Step ; Otherwise enumerate S \ C to fin a c S \ C such that C {c is k-wise regular. Let c τ+1 : c, C : C {c τ+1 an τ : τ + 1, an return to Step 1. Assemble the sequence {c 1, c,..., c n an exit. It is obvious that the initial sequence {c 1, c,..., c k is k-wise regular. In orer for Algorithm DET to exit successfully, it remains to argue that it is always possible to expan the k-wise regular sequence by one in Step 1, as long as τ < n. Theorem 3.5 Suppose that 3 k τ < n, q is a prime number, an C with C τ is k-wise regular. If we uniformly pick c τ+1 from S, then Prob {C {c τ+1 is k-wise regular 1 (1.5)k k! ensuring that {c τ+1 S C {c τ+1 is k-wise regular. ( ) τ + 1 k, n Proof. Like in the proof of Theorem 3.3, we have Prob {C {c τ+1 is not k-wise regular p P τ+1 k (q) τ+1 Prob p c [l] 0 mo q, 1 l m. For any p P τ+1 k (q), since q is prime, by using a similar argument as of (1), we can get τ+1 Prob p c [l] 0 mo q, 1 l m 1 n k. Essentially, the argument in (1) works by conitioning on the elements in C, the selection orering in C uring the previous steps is not important. Therefore, Prob {C {c τ+1 is k-wise regular 1 P τ+1 k (q) 1 ( ) τ + k 1 n k 1 k n k 1 (1.5)k k! ( ) τ + 1 k > 0. n 1

13 By the above theorem, Step 1 of Algorithm DET guarantees to expan the k-wise regular sequence of base q before reaching the esire carinality τ n. A straightforwar computation shows that Algorithm DET requires an overall complexity of O(n k 1 log q n). 4 Polynomial-Size Representation of Moments Tensor 4.1 Polynomial-Size Representation of the Fourth Moments Tensor With the help of k-wise uncorrelate ranom variables, we are able to construct polynomial-size representation of the fourth moments tensor. In Hilbert s construction (9), the support set is too general to apply the result in Section 3. However as we mentione earlier, such ecomposition of (9) is not unique. In fact, when, we observe that ( ) 4 (x T x) x i x 4 i E ξ x, (13) where ξ 1, ξ,..., ξ n are i.i.. symmetric Bernoulli ranom variables. Applying either Algorithm RAN or Algorithm DET leas to a 4-wise regular sequence of base, base on which we can efine ranom variables η 1, η,..., η n as we i in (11). Proposition 3. guarantees that η 1, η,..., η n are 4-wise uncorrelate, an it is easy to check that E[η ] E[η 3 ] E[ξ 1 ] E[ξ1] 3 0, E[η ] E[η 4 ] E[ξ1] E[ξ1] n. [ (n ) ] [ 4 (n ) 4 Thus, by Proposition., we have E η x E ξ x ]. Moreover, the size of the sample space of {η 1, η,..., η n is at most k log q n n 4, which means the new representation has at most n + n 4 fourth powere terms. Combining with Proposition.1, we have the following main result. Theorem 4.1 Given a positive integer n, we can fin τ ( n 4 ) vectors b 1, b,..., b τ polynomial time, such that (x T x) τ x 4 ( i + (b ) T x ) 4 x R n, 3 or equivalently, sym (I I) 3 τ e i e i e i e i + b b b b, R n in where e i R n is the i-th unit vector (with the i-th entry 1 an other entries zeros). 13

14 The result can be extene to a more general setting as follows. Corollary 4. Given a positive semiefinite matrix A R n n, we can fin τ ( n 4 + n) vectors a 1, a,..., a τ R n in polynomial time, such that (x T Ax) τ ( (a i ) T x ) 4 x R n, or equivalently, sym (A A) τ a i a i a i a i. Proof. Due to the one to one corresponence between super-symmetric tensors an homogeneous polynomials, we only nee to prove the first ientity. By letting y A 1 x an applying Theorem 4.1, we can fin b 1, b,..., b τ in polynomial time with τ n 4, such that (x T Ax) (y T y) 3 yi 4 + τ ( (b ) T y ) ( 4 ( ) 1 ) 4 4 (e i ) T A 1 x + 3 τ ( (b ) T A 1 x ) 4. The conclusion follows by letting a i ( ) A 1 e i for i 1,,..., n, an a i+n A 1 b i for i 1,,..., τ. 4. Polynomial-Size Representation of Complex q-th Moments Tensor In this subsection we are going to generalize the result in Section 4.1 to q-th moments tensor. Denote I q to be the q-th orer ientity tensor, whose entry is 1 when all its inices are ientical, an is zero otherwise. We are intereste in whether I q I q I q is a q-th moments tensor or not. If it is true, then for any given positive integers q, an n, there exist vectors a 1, a,..., a t R n, such that or equivalently sym (I q I q I q ) ( ) x q i t a i a i a i, (14) q t ( (a ) T x ) q x R n. (15) Unfortunately, the above oes not hol in general, as the following counter example shows. Example 4.3 The function f(x) (x x3 ) x x3 1 x3 + x6 cannot be ecompose in the form of (15) with q 3 an, i.e., a sum of sixth powere linear terms. 14

15 This can be easily proven by contraiction. Suppose we can fin a 1, a,..., a t R n, such that x x 3 1x 3 + x 6 t (a i x 1 + b i x ) 6. (16) There must exist some (a, b ) with a b 0, since otherwise there is no monomial x 3 1 x3 in the right han sie of (16). As a consequence, the coefficient of monomial x 1 x4 in the right han sie of (16) is at least ( 6 ) a b 4 > 0, which is null on the left sie of the equation, leaing to a contraiction. In the same vein one can actually show that (15) cannot hol for any q 3. Therefore, we turn to q-th moments tensor in the complex omain, i.e., both entries of the tensor an vector a i s in (14) an (15) are now allowe to take complex values. Similar to (13), we have the following ientity: x q 1 ) ( q q ( q q x q + ( )E ) q ξ i x i, (17) where ξ 1, ξ,..., ξ n are i.i.. ranom variables uniformly istribute on q. further prove (15) for the more general complex case. Moreover, we can Proposition 4.4 For any given positive integers q, an n, there exist a 1, a,..., a τ C n such that or equivalently, ( x q i ) τ ( (a ) T x ) q sym (I q I q I q ) x C n, (18) t a i a i a i. q Proof. Due to the one to one corresponence between super-symmetric tensors an homogeneous polynomials, we only nee to prove the first ientity, whose proof is base on mathematical inuction. The case 1 is alreay guarantee by (17). Suppose that (18) is true for 1, then there exist b 1, b,..., b t C n such that ( x q i ) ( ) 1 x q i t ( (b ) T x ) 1 q By applying (17) to the above ientity, there exist c 1, c,..., c τ C t, such that ( x q i ) t ( (b ) T x ) 1 q τ t (c i ) (b ) T x q. τ ( (c i ) T B T x ) q, where B (b 1, b,..., b t ) C n t. Letting a i Bc i (1 i τ) completes the inuctive step. 15

16 The next step is to reuce the number τ in (18). Uner the conition that q is prime, we can get a k-wise regular sequence of base q using either Algorithm RAN or Algorithm DET. With the help of Theorem., we can further get a polynomial-size representation of complex Hilbert s ientity an complex q-th moments tensor, by applying a similar argument as in Theorem 4.1. Theorem ( 4.5 ) For any given positive integers q, an n with q being prime, we can fin τ O n (q) 1 vectors a 1, a,..., a τ C n in polynomial time, such that or equivalently, ( x q i ) τ ( (a i ) T x ) ( q) sym (I q I q I q ) x C n, τ a i a i a i. q 5 Shortest Representation of Hilbert s Ientity In Section 4.1, we constructe polynomial-size representation of Hilbert s ientity, in particular, the fourth moments tensor sym (I I). The number of fourth powere linear functions require (in Theorem 4.1) is n + n 4. As we shall see later, this size is in general not smallest possible. This raises the issue of how to fin the shortest representation of the fourth moments tensor. In general, we are intereste in the following quantity: { τ (n) : min b 1, b,..., b m R n, such that ( x T x ) m ( (b i ) T x ) x R n. m Z + If fact, τ (n) is closely relate to the rank of the super-symmetric tensor sym (I I I ), which is the following: { ρ (n) : min b 1, b,..., b r R n, λ R r, such that ( x T x ) r ( λ i (b i ) T x ) x R n, r Z + or in the language of tensors, the smallest r such that sym (I I I ) r λ i b i b i b i. The ifference between τ (n) an ρ (n) lies in the fact that the latter one allows negative rank-one tensors. Therefore we have τ (n) ρ (n). Computing the exact values for τ (n) an ρ (n) is not easy for general n an, an the only clear case is for 1 whereas τ (n) ρ (n) n. In this section we focus on the case, i.e., τ 4 (n) an ρ 4 (n). In fact, the lower boun for τ (n) was alreay stuie by Reznick [17]. Below we first summarize the result of Reznick [17]. 16

17 Theorem 5.1 (Theorem 8.15 of [17]) For any given positive integers an n, the number of -th powere linear terms in Hilberts ientity (5) is at least ( ) n+ 1 n 1, i.e., τ (n) ( ) n+ 1 n 1. Furthermore when, the exact values τ (n) for some specific n s are known in the literature. Proposition 5. (Proposition 9.6 of [17]) τ 4 (n) ( ) n+ 1 n n(n+1) + 1 when n 4, 5, 6. We remark that when, n(n + 1)/ is also a lower boun for the number of rank-one terms to represent sym (A A) with A 0. Besies, if ξ 1, ξ,..., ξ n are symmetric Bernoulli ranom variables, an they are 4-wise uncorrelate, then Theorem 5.1 also inicates that n(n + 1)/ is a lower boun for the size of sample space generate by {ξ 1, ξ,..., ξ n. In fact, n(n + 1)/ is also a lower boun for the rank of sym (I I), as the following theorem stipulates. Theorem 5.3 For any positive integer n, it hols that n(n + 1)/ ρ 4 (n) n. Proof. Denote the shortest representation to be x m a i x 4 l b i x where m + l ρ 4 (n). By comparing the coefficient of each monomial, we have m a4 i l b4 i 1 1 n m a i 1 a i l b i 1 b i n m a3 i 1 a i l b3 i 1 b i n m a i 1 a i a i3. l b i 1 b i b i ,, 3 n with k t if k t m a i 1 a i a i3 a i4 l b i 1 b i b i3 b i ,, 3, 4 n with k t if k t (19) Construct matrices A R m n n(n 1) m, B R, C R l n n(n 1) l an D R, where A a 11 a 1... a 1n b 11 b 1... b 1n a 1 a... a n......, C b 1 b... b n......, 4, B a m1 a m... a mn b l1 b l... b ln a 11 a 1 a 11 a a 11 a 1n a 1 a 13 a 1 a a 1 a 1n... a 1,n 1 a 1n a 1 a a 1 a 3... a 1 a n a a 3 a a 4... a a n... a,n 1 a n a m1 a m a m1 a m3... a m1 a mn a m a m3 a m a m4... a m a mn... a m,n 1 a mn 17

18 an D b 11 b 1 b 11 b b 11 b 1n b 1 b 13 b 1 b b 1 b 1n... b 1,n 1 b 1n b 1 b b 1 b 3... b 1 b n b b 3 b b 4... b b n... b,n 1 b n b l1 b l b l1 b l3... b l1 b ln b l b l3 b l b l4... b l b ln... b l,n 1 b ln By (19), it is straightforwar to verify that [ [A, B] T [A, B] [C, D] T A T A C T C [C, D] B T A D T C A T B C T D B T B D T D ] [ ] 1 3 E + 3 I O 1 O 3 I 0. Thus [A, B] T [A, B] is also positive efinite, hence full-rank. Finally, ρ 4 (n) m rank ([A, B]) rank ( [A, B] T [A, B] ) n(n + 1)/. The upper boun follows from the following ientity (formula (10.35) in [17]): x 1 (x + x k ) (x x k ) n <k <k x 4. When n 5, the coefficient 4 n 3 is negative, an so it is not a vali representation of Hilbert s ). ientity, but it is still a rank-one ecomposition for Since there are no more than n ( n x rank one terms in this expression, it yiels an upper boun of n for ρ 4 (n). Remark as ρ 4 (n) τ 4 (n), Theorem 5.3 immeiately implies Theorem 5.1 when. The following examples show that n(n + 1)/ is the exact value for ρ 4 (n) as well as τ 4 (n) when n 3 (note that the case n 1 is trivial). Example 5.4 (x 1 + x ) 1 ( ) 4 ( ) 4 x x + 1 x x x4. Example 5.5 (x 1 + x + x 3 ) 1 ( (a 4 +1) β±1 (x1 + βax ) 4 + (x + βax 3 ) 4 + (x 3 + βax 1 ) 4), where a 3± 5. We remark that the above tight representations are not unique. One may fin other representations, e.g. (8.9) an (8.30) of [17], which inclue Examples 5.4 an 5.5 as special cases. Moreover, in light of Proposition 5., Liouville s ientity (6) which involving 1 terms, is not tight for both ρ 4 (4) an τ 4 (4). The following tight example for τ 4 (4) only inclues 11 terms. 18

19 Example 5.6 ((9.7)(i) of [17]) (x 1 + x + x 3 + x 4) 1 3 (x 1 + x + x 3 + x 4 ) i< 4 4 3x i x i (1 + )(x i + x ) (1 ) 4 k i,k x k 4. This example along with Theorem 5.3 implies that 10 ρ 4 (4) 11. It remains an open problem to locate the exact value of ρ 4 (4). In general, fining the exact values (or a tighter upper boun) of τ 4 (n) an ρ 4 (n), as well as fining a succinct algorithm to construct a shorter (less than n 4 + n) representation of the fourth moments tensor sym (I I), are interesting future research questions. 6 Matrix q p Norm Problem In this section, we shall illustrate the power of polynomial-size representation of moments tensor by a specific example. In particular, we consier the problem of computing the so-calle q p (1 p, q ) norm of a matrix A, efine as follows: A q p : max x q1 Ax p. This problem can be viewe as a natural extension of several useful problems. For instance, the case p q correspons to the largest singular value of A. The case (p, q) (1, ) correspons to the bilinear optimization problem in binary variables, which is relate to the so-calle matrix cut norm an Grothenieck s constant; see Alon an Naor []. In case p q, the problem becomes the matrix p-norm problem, which has applications in scientific computing; cf. [9]. In terms of the computational complexity, three easy cases are well known: (1) q 1 an p 1 is a rational number; () p an q 1 is a rational number; (3) p q. Steinberg [19] showe that computing A q p is NP-har for general 1 p < q, an she further conecture that the above mentione three cases are the only exceptional easy cases where the matrix q p norm can be compute in polynomial time. Henrickx an Olshevsky [8] mae some progress along this line by figuring out the complexity status of the iagonal case of p q. Moreover, very recently Bhaskara an Viayaraghavan [5] prove that this problem is NP-har to approximate to any constant factor when < p q. However, the problem of etermining the complexity status for the case p > q still remains open. Here we shall show that the problem A q p is NP-har when p 4 an q. To this en, let us first present the following lemma. 19

20 Lemma 6.1 Given positive integers n, i, with 1 i < n, we can fin t ( n 4 +n+) vectors a 1, a,..., a t in polynomial time, such that x i x + (x T x) t k1 ( (a k ) T x) 4. Proof. Recall in Theorem 4.1, we can fin τ ( n 4 ) vectors a 1, a,..., a τ R n in polynomial time, such that 3 τ ( 4 x 4 l + (a l ) x) T (x T x). (0) l1 On the other han, one verifies straightforwarly that for 1 i n we have l1 1 ( (xi + x ) 4 + (x i x ) 4) + x 4 i + x l n, l i, x 4 l 6x i x + Diviing by 3 on both sies of (1) an then summing up with τ ( l1 (a l ) T x ) 4 yiels τ ( ) 4 (a l ) T 1 x + 1 ( (xi + x ) 4 + (x i x ) 4) + x 4 i + x l1 τ l1 ( ) 4 (a l ) T x + x i x + 3 x i x + (x T x), l1 x 4 l x 4 l. (1) l1 1 l n, l i, x 4 l where the last equality is ue to (0). Now we are in a position to prove the main theorem of this section. Theorem 6. Computing A 4 max x 1 Ax 4 is NP-har. Proof. The reuction is mae from computing the maximum (vertex) inepenence set of a graph. In particular, for a given graph G (V, E), Nesterov [15] showe that the following problem can be reuce from the maximum inepenence number problem: max (i,) E, i< x i x s.t. x 1, x R n, hence is NP-har. Moreover, the above is obviously equivalent to (P ) max (i,) E, i< x i x + E x 4 ) (i,) E, i< (x i x + (xt x) s.t. x 1, x R n. 0

21 By Lemma 6.1, the obective in (P ) can be expresse by no more than E (n 4 + n + ) number of fourth powere linear terms, making (P ) be an instance of A 4 (polynomial-size). The polynomial reuction is thus complete. Suppose that p an q are the conugates of p an q respectively, i.e., 1 p + 1 p 1 an 1 q + 1 q 1. By using the fact that x p max y p 1 y T x, one can prove that A q p A T p q. Therefore, Theorem 6. implies that computing A 4 is also NP-har. We remark that Theorem 6. was 3 inepenently prove by Barak et al. [3] using a similar argument, after the initial version of this paper was submitte. Acknowlegements. We woul like to thank the anonymous referees an the associate eitor for their insightful comments, which helpe to significantly improve this paper from its original version. References [1] N. Alon, L. Babai, an A. Itai, A Fast an Simple Ranomize Algorithm for the Maximal Inepenent Set Problem, Journal of Algorithms, 7, , [] N. Alon an A. Naor, Approximating the Cut-Norm via Grothenieck s Inequality, SIAM Journal on Computing, 35, , 006. [3] B. Barak, F. Branao, A. W. Harrow, J. Kelner, D. Steurer, an Y. Zhou, Hypercontractivity, Sum-of-Squares Proofs, an Their Applications, Proceeings of the 44th Annual ACM Symposium on Theory of Computing, , 01. [4] A. Barvinok, A Course in Convexity, Grauate Stuies in Mathematics, Volume 54, American Mathematical Society, 00. [5] A. Bhaskara an A. Viayaraghavan, Approximating Matrix p-norms. Proceeings of the n Annual ACM-SIAM Symposium on Discrete Algorithms, , 011. [6] C. Carathéoory, Über en Variabilitätsbereich er Fourierschen Konstanten von positiven harmonischen Funktionen, Reniconti el Circolo Matematico i Palermo, 3, , [7] W. J. Ellison, Waring s Problem, The American Mathematical Monthly, 78, 10 36, [8] J. M. Henrickx an A. Olshevsky, Matrix p-norms Are NP-Har to Approximate If p 1,,, SIAM Journal on Matrix Analysis an Applications, 31, 80 81, 010. [9] N. J. Higham, Estimating the Matrix p-norm, Numerische Mathematik, 6, ,

22 [10] D. Hilbert, Beweis für ie Darstellbarkeit er ganzen Zahlen urch eine feste Anzahl n ter Potenzen (Waringsches problem), Mathematische Annalen, 67, , [11] B. Jiang, Z. Li, an S. Zhang, On Cones of Nonnegative Quartic Forms, Technical Report, Department of Inustrial an Systems Engineering, University of Minnesota, Minneapolis, 011. [1] A. Joffe, On a Set of Almost Deterministic k-wise Inepenent Ranom Variables, Annals of Probability,, , [13] H. Karloff an Y. Mansour, On Construction of k-wise Inepenent Ranom Variables, Combinatorica, 17, , [14] M. B. Nathanson, Aitive Number Thoery: The Classical Bases, Grauate Texts in Mathematics, Volumn 164, Springer-Verlag, New York, [15] Yu. Nesterov, Ranom Walk in a Simplex an Quaratic Optimization over Convex Polytopes, CORE Discussion Paper, UCL, Louvain-la-Neuve, Belgium, 003. [16] P. Pollack, On Hilbert s Solution of Waring s Problem, Central European Journal of Mathematics, 9, , 011. [17] B. Reznick, Sums of Even Powers of Real Linear Forms, Memoirs of the American Mathematical Society, Volume 96, Number 463, 199. [18] B. Reznick, Uniform Denominators in Hilbert s Seventeenth Problem, Mathematische Zeitschrift, 0, 75 97, [19] D. Steinberg, Computation of Matrix Norms with Applications to Robust Optimization, Research Thesis, Technion Israel University of Technology, Haifa, Israel, 005.