Coupled Matrix/Tensor Decompositions:

Transcription

1 Coupled Matrix/Tensor Decompositions: An Introduction Laurent Sorber Mikael Sørensen Marc Van Barel Lieven De Lathauwer KU Leuven Belgium 1

2 Canonical Polyadic Decomposition Rank: minimal number of rank-1 terms [Hitchcock, 197] Canonical Polyadic Decomposition (CPD): decomposition in minimal number of rank-1 terms [Harshman 70], [Carroll and Chang 70] c 1 c R T = b b R a 1 a R Unique under mild conditions on number of terms and differences between terms Orthogonality (triangularity,...) not required (but may be imposed)

3 Factor Analysis and Blind Source Separation Decompose a data matrix in rank-1 terms that can be interpreted E.g. statistics, telecommunication, biomedical applications, chemometrics, data analysis,... A = F G T A g 1 g g R = f 1 f f R F: mixing matrix G: source signals Decompose a data matrix in rank-1 terms that can be interpreted 9

4 What about SVD? SVD is unique... thanks to orthogonality constraints A = U S V T = R s rr u r v T r r=1 U, V orthogonal, S diagonal Whether these constraints make sense, depends on the application SVD is great for dimensionality reduction best rank-r approximation truncated SVD = A U S V T 11

5 Uniqueness: C has full column rank CPD: T = R r=1 a r b r c r C I J K T [1,;3] = (A B) C T C IJ K e.g. C-mode is sample mode Khatri-Rao product second compound matrices: U = C (A) C (B) C I(I 1) J(J 1) R(R 1) u i1 i j 1 j r 1 r = a i 1 r 1 a i r 1 a i1 r a i b j 1 r 1 b j r 1 r b j1 r b j r 1 i 1 < i I 1 j 1 < j J 1 r 1 < r R Theorem: if U and C have full column rank, then CPD is unique (proof is constructive) [Jiang and Sidiropoulos, 04], [DL 06] 15

6 Uniqueness: C has full column rank () Theorem: if U C I(I 1) then CPD is unique J(J 1) R(R 1) and C C K R have full column rank, Generic: CPD is unique for R bounded by I,J,K as in I(I 1) J(J 1) R(R 1) and K R Approximately: IJ R K R Compare to Kruskal: min(i,r) + min(j,r) R + and K R 16

7 Recent results Unifying theory Constructive proof Algorithm for Kruskal s condition (and beyond) [Domanov, DL, 1], [Domanov, DL, 13] 17

8 Canonical Polyadic Decomposition Rank: minimal number of rank-1 terms [Hitchcock, 197] Canonical Polyadic Decomposition (CPD): decomposition in minimal number of rank-1 terms [Harshman 70], [Carroll and Chang 70] c 1 c R T = b b R a 1 a R Unique under mild conditions on number of terms and differences between terms Orthogonality (triangularity,...) not required (but may be imposed) 19

9 Decomposition in rank-(l, L, 1) terms c 1 c R T = A 1 B A R B R Unique under mild conditions [DL 08] 0

10 Decomposition in rank-(r 1,R,R 3 ) terms C 1 C R T = A 1 B A R B R Unique under mild conditions Rank-1 term data atom Block term data molecule [DL 08] 1

11 Constraints Examples: orthogonality [Sørensen and DL 1] nonnegativity [Cichocki et al. 09] Vandermonde [Sørensen and DL 1] independence [De Vos et al. 1]... Not needed for uniqueness in tensor case Pro: relaxed uniqueness conditions easier interpretation no degeneracy (NN, orthogonality) higher accuracy Depending on type of constraints, lower or higher computational cost

12 5 Tensorlab Tensorlab a MATLAB toolbox for tensor decompositions esat.kuleuven.be/sista/tensorlab Elementary operations on tensors Multicore-aware and profiler tuned Tensor decompositions with structure and/or symmetry CPD, LMLRA, MLSVD, block term decompositions Global minimization of bivariate polynomials Exact line and plane search for tensor optimization Cumulants, tensor visualization, estimating a tensor s rank or multilinear rank,...

13 Coupled matrix/tensor decompositions One or more matrices and/or one or more tensors Symmetric and/or nonsymmetric One or more factors shared (or parts of factors, or generators) Constraints (orthogonal, nonnegative, exponential, constant modulus, polynomial, rational, Toeplitz, Hankel,...) Large, possibly incomplete data Multi-view data / data fusion E.g. coupled EEG-fMRI; person-location-activity and person-person and location-location...;...

14 Uniqueness: C has full column rank Coupled CPD: T (n) = R r=1 a(n) r b (n) r c r C I n Jn K Khatri-Rao product second compound matrices: U (n) U = = C (A (n) ) C (B (n) ) C I n(in 1) U (1) U (). U (N) Jn(Jn 1) R(R 1) Theorem: if U and C have full column rank, then coupled CPD is unique (proof is constructive) 3

15 Increase spatial diversity: Widely Separated Antenna Arrays R y (1) 1jk = a (r,1) h (r,1) s (r) 1 j k r=1 s (1) k h (1,1) j h (1,N) j A (1) y (1) Ijk = R r=1 a (r,1) h (r,1) s (r) I j k s (R) k h (R,N) j h (R,1) j A (N) R y (N) 1jk = a (r,n) h (r,n) s (r) 1 j k r=1 y (N) Ijk = R a (r,n) h (r,n) s (r) I j k r=1 (A biotensor example is multimodal data fusion: EEG fmri MEG ) Coupled Tensor Decompositions 11 / 19 Mikael Sørensen

16 Signal Separation from a Coupled Tensorial Perspective s (1) s (R) Y (1) = a (1,1) + + a (R,1). h (1,1) s (1) h (R,1) s (R) Y (N) = a (1,N) + + a (R,N) h (1,N) h (R,N) Coupled Tensor Decompositions 1 / 19 Mikael Sørensen

17 Extension to Incoherent Multipath with Small Delay Spread s (1) k h (1,1,1) j h (P1,1,1,1) j h (1,1,N) h (P1,N1,N) j j A (1) y (1) 1jk = R r=1 Pr,1 p=1 R Pr,1 y (1) Ijk = r=1 p=1 a (p,r,1) h (p,r,1) s (r) 1 j k a (p,r,1) I h (p,r,1) s (r) j k s (R) k h (1,R,1) j h (1,R,N) j h (PR,N,R,N) j h (P1,N,R,1) j A (N) R Pr,N y (N) 1jk = a (p,r,n) h (p,r,n) s (r) 1 j k y (N) Ijk = r=1 p=1 R Pr,N r=1 p=1 a (p,r,n) I h (p,r,n) s (r) j k Coupled Tensor Decompositions 14 / 19 Mikael Sørensen

18 Signal Separation from a Coupled Tensorial Perspective s (1) s (R) Y (1) = P 1,1 H (1,1) A (1,1)T + + P R,1 H (R,1) A (R,1)T. s (1) s (R) Y (N) = P 1,N H (1,N) A (1,N)T + + P R,N H (R,N) A (R,N)T Coupled Tensor Decompositions 15 / 19 Mikael Sørensen

19 Tensorlab v.0 Tensorlab v.0 Major upgrade which brings: I Full support for sparse and incomplete tensors I Major improvements in computational and memory efficiency I Structured data fusion Structured: choose from a large library of constraints to impose on factors (nonnegative, orthogonal, Toeplitz,... ) Data fusion: jointly factorize multiple data sets Y X Z

20 3 Applications Example 4: GPS data set Five coupled data sets: user-location-activity, user-user, location-feature, activity-activity and user-location Challenge: predict user participation in activities Solution with SDF: compute coupled tensor factorization minimize U,L,A,F,λ,μ,ν + ω + ω 4 + ω 6 ω 1 M () (U, U, λ) T () M (1) (U, L, A) T (1) F + ω 3 W (1) M (3) (L, F ) T (3) M (4) (A, A, μ) T (4) + ω 5 M (5) (U, L, ν) T (5) F F ( ) U F + L F + A F + F F + λ F + μ F + ν F F

21 Example 4: 80% missing entries in user-location-activity tensor Applications 1 True positive rate SDF CPD False positive rate 33

22 Example 4: 50 users missing in user-location-activity tensor Applications 1 True positive rate SDF False positive rate 34

23 Breaking the Curse of Dimensionality in Data Analysis 9th-order ( ) data set 10-component thermodynamic phase diagram incomplete tensor: known samples (out of ) sum of structured rank-1 terms 4

24 7 Applications Example 3: InsPyro materials data set Data set: an incomplete tensor in which each dimension represents the concentration of a metal in an alloy and the entries are the alloy s melting temperature Challenge: predict melting temperatures of different alloys Solution with SDF: use structured CPD where each factor vector is a sum of RBF kernels u (n) r u (n) b,r = 8 a exp ( (t b) /(c ) ) i=1 where a b and c are the free parameters in u (n) r

25 Applications Example 3: InsPyro materials data set 0 15 U (3) c 3 (%) 8

26 9 Applications Example 3: InsPyro materials data set,000 1, ,00,000 1, , ,00 1,400 1,600 1,400 c 3 (%) 30 1,800 1,00 1, ,600 1, c (%)

27 30 Applications Example 3: InsPyro materials data set,000 1, ,00 0 1,600 1,400 c 3 (%) 1, , ,00 1,600 1, c (%)

28 31 Applications Example 3: InsPyro materials data set,000 1, ,00 1, ,000 1,400 1,800 1, c 3 (%) , ,400 1, , c (%)

29 Conclusion Coupled matrix/tensor decompositions = important new concept Many applications Gold mine for research topics: algebra (numerical) linear algebra randomized NLA large-scale numerical optimization... 5