Sufficient Conditions to Ensure Uniqueness of Best Rank One Approximation

Transcription

1 Sufficient Conditions to Ensure Uniqueness of Best Rank One Approximation Yang Qi Joint work with Pierre Comon and Lek-Heng Lim Supported by ERC grant # Decoda August 3, 2015 Yang Qi (GIPSA-Lab) Uniqueness of Best Rank One Approximation August 3, / 23

2 Overview 1 Motivation 2 Normalized Singular Vector Tuples and Eigenvectors 3 Sufficient Conditions for Uniqueness of Best Rank One Symmetric Approximations 4 Sufficient Conditions for Uniqueness of Best Rank One Approximation Yang Qi (GIPSA-Lab) Uniqueness of Best Rank One Approximation August 3, / 23

3 Overview 1 Motivation 2 Normalized Singular Vector Tuples and Eigenvectors 3 Sufficient Conditions for Uniqueness of Best Rank One Symmetric Approximations 4 Sufficient Conditions for Uniqueness of Best Rank One Approximation Yang Qi (GIPSA-Lab) Uniqueness of Best Rank One Approximation August 3, / 23

4 Motivation Let V 1,..., V d be vector spaces over a field K(= R, C) of dimensions n 1,..., n d respectively. When K = R, fix a basis for each V i, then V 1 V d is naturally endowed with a basis. Given T V 1 V d, let [T i1...i d ] denote the coordinate of T under this basis. Definition If each T i1...i d Definition 0, T is called a nonnegative tensor. The nonnegative rank, denoted by rank + (T ), of a nonnegative tensor T is the minimal integer r such that T = r u 1,i u d,i i=1 where each vector u j,i is nonnegative. Yang Qi (GIPSA-Lab) Uniqueness of Best Rank One Approximation August 3, / 23

5 Three Fundamential Questions For any T V 1 V d over K, in engineering it is important to consider a best rank-r approximation T of T, which is a tensor with rank r satisfying T T = inf T X, rank(x ) r where is the Hilbert-Schmidt (Frobenius) norm. The following are three fundamental questions need to be answered: 1 Does T have a best rank-r approximation? 2 when (under what condition) is T unique? 3 How to find this T? Proposition (Lim-Comon) The set of nonnegative tensors with nonnegative rank r is a closed subset. Yang Qi (GIPSA-Lab) Uniqueness of Best Rank One Approximation August 3, / 23

6 Matrix Case For a m n matrix T, let T = UΣV be its singular value decomposition, where U and V are unitary matrices, and Σ = diag{σ 1,..., σ k, 0,..., 0} is a non-negative diagonal m n real matrix with σ 1 σ k. Theorem (Schmidt-Eckart-Young) A best rank-r approximation of T is UΛV, where Λ = diag{σ 1,..., σ r, 0,..., 0}. Consequentially, a best rank-r approximation of T can be obtained from r successive best rank one approximations. Definition (Vannieuwenhoven-Nicaise-Vandebril-Meerbergen) Such a T is said to have a Schmidt-Eckart-Young decomposition. Yang Qi (GIPSA-Lab) Uniqueness of Best Rank One Approximation August 3, / 23

7 Tensor Case For a tensor, things are quite different from the matrix case. Theorem (Vannieuwenhoven-Nicaise-Vandebril-Meerbergen) The set of tensors that do not admit a Schmidt-Eckart-Young decomposition is of positive measure. Theorem (Stegeman-Comon) Subtracting a best rank one approximation does not necessarily decrease tensor rank. Theorem (Hillar-Lim) Finding a best rank one approximation of a tensor is a NP hard problem. Yang Qi (GIPSA-Lab) Uniqueness of Best Rank One Approximation August 3, / 23

8 Best Rank One Approximations Every tensor has a best rank one approximation. Theorem (Friedland-Ottaviani) Almost every T over R has a unique best rank one approximation. For almost every symmetric T S d V, a best rank one approximation is unique and symmetric. Lemma Given a nonnegative T. Let u 1 u d V 1 V d be a best rank one approximation of T, then u 1,..., u d can be chosen to be nonnegative. Aim of This Talk Sufficient conditions to ensure a unique best rank one approximation. Yang Qi (GIPSA-Lab) Uniqueness of Best Rank One Approximation August 3, / 23

9 Overview 1 Motivation 2 Normalized Singular Vector Tuples and Eigenvectors 3 Sufficient Conditions for Uniqueness of Best Rank One Symmetric Approximations 4 Sufficient Conditions for Uniqueness of Best Rank One Approximation Yang Qi (GIPSA-Lab) Uniqueness of Best Rank One Approximation August 3, / 23

10 Critical Points of Distance Let S n i 1 be the unit sphere in V i, then a best rank one approximation of T is a solution of the following optimization problem: Lemma (Lim) min min T α v 1 v v i S n i 1 d (1) α 0 A solution of the optimization problem (1) satisfies T, v 1 v i v d = λv i (2) for all 1 i d and some λ R, where, means contraction and means omission. Remark Equation (2) is equivalent to T p T p X, where p = v1 v d is in the cone X of the Segre variety X = Seg(PV 1 PV d ). Yang Qi (GIPSA-Lab) Uniqueness of Best Rank One Approximation August 3, / 23

11 Singular Vector Tuples and Eigenvectors Definition (Friedland-Ottaviani) For T V 1 V d over C, ([u 1 ],..., [u d ]) PV 1 PV d is called a singular vector tuple if for some λ i C, i = 1,..., d. Definition (Lim, L. Qi) T, u 1 û i u d = λ i u i (3) For a symmetric tensor T S d V, λ is called an eigenvalue, and a unit vector v V is called an eigenvector corresponding to λ if T, v d 1 = λv. (4) Yang Qi (GIPSA-Lab) Uniqueness of Best Rank One Approximation August 3, / 23

12 Number of Critical Points Theorem (Cartwright-Sturmfels) For a (general) symmetric T S d V with dim V = n, T has (d 1)n 1 d 2 (equivalence classes of) eigenpairs counted with multiplicity over C. Theorem (Friedland-Ottaviani) For a generic T V 1 V d, T has c simple singular vector tuples corresponding to nonzero singular values, where c is the coefficient of the monomial d i=1 tn i 1 i in 1 j d s n j j t n j j s j t j, s j = k j t k. Remark (Draisma-Horobeț-Ottaviani-Sturmfels-Thomas) These theorems tell us the Euclidean distance degree (ED degree) of the Veronese variety ν d (P n 1 ) is (d 1)n 1 d 2 and the ED degree of the Segre variety Seg(PV 1 PV d ) is c. Yang Qi (GIPSA-Lab) Uniqueness of Best Rank One Approximation August 3, / 23

13 Normalized Singular Pairs and Eigenpairs Definition For T V 1 V d, (λ, u 1,..., u d ) is called a normalized singular pair of T if { T, u 1 û i u d = λu i (5) u i, u i = 1 for i = 1,..., d. λ is called a normalized singular value, and (u 1,..., u d ) is called a normalized singular vector tuple corresponding to λ. Definition For T S d W over C, (λ, u) is called a normalized eigenpair of T if { T, u d 1 = λu u, u = 1 Yang Qi (GIPSA-Lab) Uniqueness of Best Rank One Approximation August 3, / 23

14 Equivalence Relations Definition Two normalized eigenpairs (λ, u) and (µ, v) of T are called equivalent if (λ, u) = (µ, v), or ( 1) d 2 λ = µ and u = v. Definition Two normalized singular pairs (λ, u 1,..., u d ) and (µ, v 1,..., v d ) of T are called equivalent if (λ, u 1,..., u d ) = (µ, v 1,..., v d ), or ( 1) k λ = µ and u j = v j for j {i 1,..., i k } and u l = v l for l / {i 1,..., i k }. Yang Qi (GIPSA-Lab) Uniqueness of Best Rank One Approximation August 3, / 23

15 Overview 1 Motivation 2 Normalized Singular Vector Tuples and Eigenvectors 3 Sufficient Conditions for Uniqueness of Best Rank One Symmetric Approximations 4 Sufficient Conditions for Uniqueness of Best Rank One Approximation Yang Qi (GIPSA-Lab) Uniqueness of Best Rank One Approximation August 3, / 23

16 Definition (Hillar-Lim) For T V 1 V d over R, let σ 1 (T ) = max{ T, u 1 u d : u 1 = = u d = 1}, and call σ 1 (T ) the spectral norm of T. σ 1 (T ) is the maximal absolute value of normalized singular values of T. Theorem (Friedland-Stawiska) For a closed semi-algebraic set C R m, let S(C) denote the set of points x R m whose best approximation in C is not unique, then S(C) is a nowhere dense semi-algebraic set and contained in a hypersurface. Proposition (Q.-Comon-Lim) Over R, the subset H σ1 S d V consisting of symmetric tensors which have non-unique equivalence classes of normalized eigenpairs corresponding to their spectral norms is indeed a hypersurface in S d V. Yang Qi (GIPSA-Lab) Uniqueness of Best Rank One Approximation August 3, / 23

17 Resultant Theory Given n + 1 homogeneous polynomials F 0,..., F n C[x 0,..., x n ] with positive total degrees d 0,..., d n, let F i = α =d i c i,α x α. For each pair of indices i, α, we introduce a variable u i,α. Given a polynomial P C[u i,α ], let P(F 0,..., F n ) denote the number obtained by replacing each variable u i,α in P with the corresponding coefficient c i,α. Theorem There is a unique polynomial Res Z[u i,α ] which has the following properties: 1 The equations F 0 = = F n = 0 have a nonzero solution over C if and only if Res(F 0,..., F n ) = 0. 2 Res(x d 0 0,..., x dn n ) = 1. 3 Res is irreducible in C[u i,α ]. Yang Qi (GIPSA-Lab) Uniqueness of Best Rank One Approximation August 3, / 23

18 Characteristic Polynomial Definition (L. Qi) Over C, for T S d V and d = 2l, let ψ T (λ) be the resultant of the equation T, u d 1 λ u, u l 1 u = 0. For d odd, let ψ T (λ) be the resultant of the equations { T, u d 1 λx d 2 u = 0 x 2. u, u = 0 ψ T (λ) is called the E-characteristic polynomial of T. Definition Let χ(t ) denote the resultant of the equations ψ T (λ) = ψ T (λ) = 0. Yang Qi (GIPSA-Lab) Uniqueness of Best Rank One Approximation August 3, / 23

19 Proposition (Cartwright-Sturmfels) A general T S d V has distinct normalized eigenvalues. Consequence χ(t ) is a nonzero polynomial on S d V, which implies χ(t ) = 0 defines a complex hypersurface H consisting of tensors T S d (V R C) which admit multiple equivalence classes of normalized eigenpairs. Yang Qi (GIPSA-Lab) Uniqueness of Best Rank One Approximation August 3, / 23

20 Example in S 3 R 2 For T = [T jkl ] S 3 R 2, ψ T (λ) = c 2 λ 6 + c 4 λ 4 + c 6 λ 2 + c 8 for some homogeneous polynomials c i of degree i in T jkl. Then χ(t ) is the determinant of some matrix in T jkl. For a generic T, ψ T (λ) = c(λ 2 γ 1 )(λ 2 γ 2 )(λ 2 γ 3 ) for some c C and distinct γ i C, and χ(t ) 0. For T H, ψ T (λ) has multiple roots, χ(t ) = 0. Yang Qi (GIPSA-Lab) Uniqueness of Best Rank One Approximation August 3, / 23

21 Example in S 3 R 2 Example Let T by T 111 = T 222 = 1 and other terms T ijk = 0, then χ(t ) = 0, which implies T has multiple eigenpairs. In fact, ψ T (λ) = (λ + 1) 2 (λ 1) 2 (2λ 2 1), there are two eigenvectors (1, 0) and (0, 1) corresponding to the eigenvalue 1, and two eigenvectors ( 1, 0) and (0, 1) corresponding to the eigenvalue 1. This example corresponds to the case that x 3 + y 3 has two best rank one approximations x 3 and y 3, where x and y are two linearly independent vectors. Yang Qi (GIPSA-Lab) Uniqueness of Best Rank One Approximation August 3, / 23

22 Conditions for Symmetric Case All real normalized eigenvalues σ 1 σ k of T are roots of ψ T (λ), and in fact for each σ i we have Proposition (Q.-Comon-Lim) 1 Over R, the subset H σi consisting of symmetric tensors which have non-unique equivalence classes of normalized eigenpairs corresponding to σ i is indeed a hypersurface in S d V. 2 These H σi form the components of the real points of H, i.e. H σ1 H σk = real points ofh 3 If χ(t ) 0, T has a unique best rank one approximation which is symmetric. Yang Qi (GIPSA-Lab) Uniqueness of Best Rank One Approximation August 3, / 23

23 Overview 1 Motivation 2 Normalized Singular Vector Tuples and Eigenvectors 3 Sufficient Conditions for Uniqueness of Best Rank One Symmetric Approximations 4 Sufficient Conditions for Uniqueness of Best Rank One Approximation Yang Qi (GIPSA-Lab) Uniqueness of Best Rank One Approximation August 3, / 23

24 Singular Characteristic Polynomial Definition Let W 1,..., W d be complex vector spaces, for T W1 W d, u i W i, and α i C, denote by φ T (λ) the resultant of the following homogeneous equations { α i T, u 1 û i u d = λ( j i α j)u i u i, u i = αi 2. (6) By the resultant theory, φ T (λ) vanishes if and only if Equations 6 has a nontrivial solution, i.e. φ T (λ) = 0 if and only if (λ, u 1,..., u d ) is a normalized singular pair of T. φ T (λ) is called the singular characteristic polynomial of T, whose roots λ are normalized singular values of T. Definition Let χ(t ) denote the resultant of the equations φ T (λ) = φ T (λ) = 0. Yang Qi (GIPSA-Lab) Uniqueness of Best Rank One Approximation August 3, / 23

25 Proposition (Q.-Comon-Lim) The subset X σ1 V1 V d consisting of tensors which have non-unique best rank one approximations forms an algebraic hypersurface. Proposition (Q.-Comon-Lim) A general T W 1 W d over C has distinct equivalence classes of normalized singular pairs. Therefore for a generic T, φ T (λ) has simple roots, which implies χ(t ) does not vanish identically. Yang Qi (GIPSA-Lab) Uniqueness of Best Rank One Approximation August 3, / 23

26 Sufficient Conditions for Uniqueness Since χ(t ) vanishes on X σ1, and does not vanish identically, then χ(t ) = 0 defines a hypersurface in V 1 V d over C. Theorem (Q.-Comon-Lim) The subset M formed by any tensor whose singular characteristic polynomial has multiple roots is a hypersurface, which is defined by χ(t ) = 0. X σ1 forms some components of the real points of M. Any real tensor T with χ(t ) 0 has a unique best rank one approximation. If this T is symmetric, then its unique best rank one approximation is also symmetric. Yang Qi (GIPSA-Lab) Uniqueness of Best Rank One Approximation August 3, / 23

27 Thank you very much for your attention! Yang Qi (GIPSA-Lab) Uniqueness of Best Rank One Approximation August 3, / 23

28 S. Banach, Über homogene polynome in (L2 ), Studia Math., 7 (1938), no. 1, pp M. Banagl, The tensor product of function semimodules, Algebra universalis, vol. 70, no. 3, pp , C. Bocci, L. Chiantini, and G. Ottaviani, Refined methods for the identifiability of tensors, Annali di Matematica Pura ed Applicata, May 2013 D. Cartwright, B. Sturmfels, The number of eigenvalues of a tensor, Linear Algebra Appl., no. 438, pp , 2013 K. C. Chang, K. Pearson, and T. Zhang, Perron Frobenius theorem for nonnegative tensors, Commun. Math. Sci., vol. 6, no. 2, pp , L. Chiantini, G. Ottaviani, and N. Vannieuwenhoven, An algorithm for generic and low-rank specific identifiability of complex tensors, SIAM J. matrix Ana. Appl., 2014, to appear. Yang Qi (GIPSA-Lab) Uniqueness of Best Rank One Approximation August 3, / 23

29 P. Comon, Tensors: a brief introduction, IEEE Sig. Proc. Magazine, vol. 31, no. 3, pp , May 2014, special issue on BSS. hal D. A. Cox, J. B. Little, and D. O Shea, Using Algebraic Geometry, ser. Graduate Texts in Mathematics, Springer, 2nd ed. 2005, Vol V. De Silva and L.-H. Lim, Tensor rank and the ill-posedness of the best low-rank approximation problem, SIAM Journal on Matrix Analysis Appl., vol. 30, no. 3, pp , S. Friedland, Best rank-1 approximation of real symmetric tensors can be chosen symmetric, Frontiers of Mathematics in China, vol. 8, no. 1, pp , S. Friedland, S. Gaubert, and L. Han, Perron Frobenius theorem for nonnegative multilinear forms and extensions, Linear Algebra Appl., vol. 438, pp , S. Friedland and G. Ottaviani, The number of singular vector tuples and uniqueness of best rank-one approximation of tensors, Mar. Yang Qi (GIPSA-Lab) Uniqueness of Best Rank One Approximation August 3, / 23

30 2014, arxiv: , to appear in Foundations of Computational Mathematics. S. Friedland and M. Stawiska, Best approximation on semialgebraic sets and k-border rank approximation of symmetric tensors, Nov. 2013, arxiv: L. D. Garcia, M. Stillman, and B. Sturmfels, Algebraic geometry of Bayesian networks, J. Symbolic Comput., 39 (2005), no. 3 4, pp I. M. Gelfand, M. M. Kapranov, and A. V. Zelevinsky, Discriminants, Resultants, and Multidimensional Determinants, ser. Mathematics: Theory & Applications, Birkhäuser Boston, W. Hackbusch, Tensor Spaces and Numerical Tensor Calculus, ser. Series in Computational Mathematics. Berlin, Heidelberg: Springer, C. J. Hillar and L.-H. Lim, Most tensor problems are NP-hard, Jour. of the ACM, vol. 60, issue 6, no. 45, Nov Yang Qi (GIPSA-Lab) Uniqueness of Best Rank One Approximation August 3, / 23

31 M. I. Jordan, Graphical models, Statistical Science, vol. 19, no. 1, pp , D. Koller and N. Friedman, Probabilistic Graphical Models. MIT Press, J. B. Kruskal, Three-way arrays: Rank and uniqueness of trilinear decompositions, Linear Algebra and Applications, vol. 18, pp , J. M. Landsberg, Tensors: Geometry and Applications, ser. Graduate Studies in Mathematics. AMS publ., 2012, vol A.-M. Li, L. Qi, and B. Zhang, E-characteristic polynomials of tensors, in Commun. Math. Sci., vol. 11, no. 1, pp , L.-H. Lim, Tensors and hypermatrices, Handbook of Linear Algebra, 2nd Ed., CRC Press, Boca Raton, FL, L.-H. Lim, Singular values and eigenvalues of tensors: a variational approach, in IEEE Int. Workshop on Comput. Adv. Multi-Sensor Adapt. Proc., Puerto Vallarta, Mexico, Dec Yang Qi (GIPSA-Lab) Uniqueness of Best Rank One Approximation August 3, / 23

32 L.-H. Lim and P. Comon, Nonnegative approximations of nonnegative tensors, Jour. Chemometrics, vol. 23, pp , Aug. 2009, hal [Online]. Available: L. Oeding and G. Ottaviani, Eigenvectors of tensors and algorithms for Waring decomposition, J. Symbolic Comput., vol. 54, pp. 9 35, 2013 L. Qi, Eigenvalues of a real spersymmetric tensor, J. Symbolic Comput., vol. 40, issue. 6, pp , Dec L. Qi, Eigenvalues and invariants of tensors, J. Math. Anal. Appl., vol. 325, issue. 2, pp , Jan A. Shashua and T. Hazan, Non-negative tensor factorization with applications to statistics and computer vision, in 22nd International Conference on Machine Learning, Bonn, 2005, pp Yang Qi (GIPSA-Lab) Uniqueness of Best Rank One Approximation August 3, / 23

33 N. D. Sidiropoulos and R. Bro, On the uniqueness of multilinear decomposition of N-way arrays, Jour. Chemo., vol. 14, pp , A. Smilde, R. Bro, and P. Geladi, Multi-Way Analysis. Chichester UK: Wiley, A. Stegeman and P. Comon, Subtracting a best rank-1 approximation does not necessarily decrease tensor rank, Linear Algebra Appl., vol. 433, no. 7, pp , Dec. 2010, hal N. Vannieuwenhoven, J. Nicaise, R. Vandebril, and K. Meerbergen, On generic nonexistence of the Schmidt Eckart Young decomposition for complex tensors, SIAM Journal on Matrix Analysis and Applications, vol. 35, no. 3, pp , M. Velasco, Linearization functors on real convex sets, SIAM Journal on Optimization, to appear. Yang Qi (GIPSA-Lab) Uniqueness of Best Rank One Approximation August 3, / 23

34 Y. Yang and Q. Yang, Further Results for Perron-Frobenius Theorem for Nonnegative Tensors, SIAM. J. Matrix Anal. & Appl., vol. 31, issue. 5, pp , P. Zhang, H. Wang, R. Plemmons, and P. Pauca, Tensor methods for hyperspectral data analysis: A space object material identification study, J. Optical Soc. Amer., vol. 25, no. 12, pp , Dec J. Zhou, A. Bhattacharya, A. Herring, and D. Dunson, Bayesian factorizations of big sparse tensors, Jun. 2013, arxiv: Yang Qi (GIPSA-Lab) Uniqueness of Best Rank One Approximation August 3, / 23