Coupled Matrix/Tensor Decompositions:
|
|
- Joanna Riley
- 5 years ago
- Views:
Transcription
1 Coupled Matrix/Tensor Decompositions: An Introduction Laurent Sorber Mikael Sørensen Marc Van Barel Lieven De Lathauwer KU Leuven Belgium 1
2 Canonical Polyadic Decomposition Rank: minimal number of rank-1 terms [Hitchcock, 197] Canonical Polyadic Decomposition (CPD): decomposition in minimal number of rank-1 terms [Harshman 70], [Carroll and Chang 70] c 1 c R T = b b R a 1 a R Unique under mild conditions on number of terms and differences between terms Orthogonality (triangularity,...) not required (but may be imposed)
3 Factor Analysis and Blind Source Separation Decompose a data matrix in rank-1 terms that can be interpreted E.g. statistics, telecommunication, biomedical applications, chemometrics, data analysis,... A = F G T A g 1 g g R = f 1 f f R F: mixing matrix G: source signals Decompose a data matrix in rank-1 terms that can be interpreted 9
4 What about SVD? SVD is unique... thanks to orthogonality constraints A = U S V T = R s rr u r v T r r=1 U, V orthogonal, S diagonal Whether these constraints make sense, depends on the application SVD is great for dimensionality reduction best rank-r approximation truncated SVD = A U S V T 11
5 Uniqueness: C has full column rank CPD: T = R r=1 a r b r c r C I J K T [1,;3] = (A B) C T C IJ K e.g. C-mode is sample mode Khatri-Rao product second compound matrices: U = C (A) C (B) C I(I 1) J(J 1) R(R 1) u i1 i j 1 j r 1 r = a i 1 r 1 a i r 1 a i1 r a i b j 1 r 1 b j r 1 r b j1 r b j r 1 i 1 < i I 1 j 1 < j J 1 r 1 < r R Theorem: if U and C have full column rank, then CPD is unique (proof is constructive) [Jiang and Sidiropoulos, 04], [DL 06] 15
6 Uniqueness: C has full column rank () Theorem: if U C I(I 1) then CPD is unique J(J 1) R(R 1) and C C K R have full column rank, Generic: CPD is unique for R bounded by I,J,K as in I(I 1) J(J 1) R(R 1) and K R Approximately: IJ R K R Compare to Kruskal: min(i,r) + min(j,r) R + and K R 16
7 Recent results Unifying theory Constructive proof Algorithm for Kruskal s condition (and beyond) [Domanov, DL, 1], [Domanov, DL, 13] 17
8 Canonical Polyadic Decomposition Rank: minimal number of rank-1 terms [Hitchcock, 197] Canonical Polyadic Decomposition (CPD): decomposition in minimal number of rank-1 terms [Harshman 70], [Carroll and Chang 70] c 1 c R T = b b R a 1 a R Unique under mild conditions on number of terms and differences between terms Orthogonality (triangularity,...) not required (but may be imposed) 19
9 Decomposition in rank-(l, L, 1) terms c 1 c R T = A 1 B A R B R Unique under mild conditions [DL 08] 0
10 Decomposition in rank-(r 1,R,R 3 ) terms C 1 C R T = A 1 B A R B R Unique under mild conditions Rank-1 term data atom Block term data molecule [DL 08] 1
11 Constraints Examples: orthogonality [Sørensen and DL 1] nonnegativity [Cichocki et al. 09] Vandermonde [Sørensen and DL 1] independence [De Vos et al. 1]... Not needed for uniqueness in tensor case Pro: relaxed uniqueness conditions easier interpretation no degeneracy (NN, orthogonality) higher accuracy Depending on type of constraints, lower or higher computational cost
12 5 Tensorlab Tensorlab a MATLAB toolbox for tensor decompositions esat.kuleuven.be/sista/tensorlab Elementary operations on tensors Multicore-aware and profiler tuned Tensor decompositions with structure and/or symmetry CPD, LMLRA, MLSVD, block term decompositions Global minimization of bivariate polynomials Exact line and plane search for tensor optimization Cumulants, tensor visualization, estimating a tensor s rank or multilinear rank,...
13 Coupled matrix/tensor decompositions One or more matrices and/or one or more tensors Symmetric and/or nonsymmetric One or more factors shared (or parts of factors, or generators) Constraints (orthogonal, nonnegative, exponential, constant modulus, polynomial, rational, Toeplitz, Hankel,...) Large, possibly incomplete data Multi-view data / data fusion E.g. coupled EEG-fMRI; person-location-activity and person-person and location-location...;...
14 Uniqueness: C has full column rank Coupled CPD: T (n) = R r=1 a(n) r b (n) r c r C I n Jn K Khatri-Rao product second compound matrices: U (n) U = = C (A (n) ) C (B (n) ) C I n(in 1) U (1) U (). U (N) Jn(Jn 1) R(R 1) Theorem: if U and C have full column rank, then coupled CPD is unique (proof is constructive) 3
15 Increase spatial diversity: Widely Separated Antenna Arrays R y (1) 1jk = a (r,1) h (r,1) s (r) 1 j k r=1 s (1) k h (1,1) j h (1,N) j A (1) y (1) Ijk = R r=1 a (r,1) h (r,1) s (r) I j k s (R) k h (R,N) j h (R,1) j A (N) R y (N) 1jk = a (r,n) h (r,n) s (r) 1 j k r=1 y (N) Ijk = R a (r,n) h (r,n) s (r) I j k r=1 (A biotensor example is multimodal data fusion: EEG fmri MEG ) Coupled Tensor Decompositions 11 / 19 Mikael Sørensen
16 Signal Separation from a Coupled Tensorial Perspective s (1) s (R) Y (1) = a (1,1) + + a (R,1). h (1,1) s (1) h (R,1) s (R) Y (N) = a (1,N) + + a (R,N) h (1,N) h (R,N) Coupled Tensor Decompositions 1 / 19 Mikael Sørensen
17 Extension to Incoherent Multipath with Small Delay Spread s (1) k h (1,1,1) j h (P1,1,1,1) j h (1,1,N) h (P1,N1,N) j j A (1) y (1) 1jk = R r=1 Pr,1 p=1 R Pr,1 y (1) Ijk = r=1 p=1 a (p,r,1) h (p,r,1) s (r) 1 j k a (p,r,1) I h (p,r,1) s (r) j k s (R) k h (1,R,1) j h (1,R,N) j h (PR,N,R,N) j h (P1,N,R,1) j A (N) R Pr,N y (N) 1jk = a (p,r,n) h (p,r,n) s (r) 1 j k y (N) Ijk = r=1 p=1 R Pr,N r=1 p=1 a (p,r,n) I h (p,r,n) s (r) j k Coupled Tensor Decompositions 14 / 19 Mikael Sørensen
18 Signal Separation from a Coupled Tensorial Perspective s (1) s (R) Y (1) = P 1,1 H (1,1) A (1,1)T + + P R,1 H (R,1) A (R,1)T. s (1) s (R) Y (N) = P 1,N H (1,N) A (1,N)T + + P R,N H (R,N) A (R,N)T Coupled Tensor Decompositions 15 / 19 Mikael Sørensen
19 Tensorlab v.0 Tensorlab v.0 Major upgrade which brings: I Full support for sparse and incomplete tensors I Major improvements in computational and memory efficiency I Structured data fusion Structured: choose from a large library of constraints to impose on factors (nonnegative, orthogonal, Toeplitz,... ) Data fusion: jointly factorize multiple data sets Y X Z
20 3 Applications Example 4: GPS data set Five coupled data sets: user-location-activity, user-user, location-feature, activity-activity and user-location Challenge: predict user participation in activities Solution with SDF: compute coupled tensor factorization minimize U,L,A,F,λ,μ,ν + ω + ω 4 + ω 6 ω 1 M () (U, U, λ) T () M (1) (U, L, A) T (1) F + ω 3 W (1) M (3) (L, F ) T (3) M (4) (A, A, μ) T (4) + ω 5 M (5) (U, L, ν) T (5) F F ( ) U F + L F + A F + F F + λ F + μ F + ν F F
21 Example 4: 80% missing entries in user-location-activity tensor Applications 1 True positive rate SDF CPD False positive rate 33
22 Example 4: 50 users missing in user-location-activity tensor Applications 1 True positive rate SDF False positive rate 34
23 Breaking the Curse of Dimensionality in Data Analysis 9th-order ( ) data set 10-component thermodynamic phase diagram incomplete tensor: known samples (out of ) sum of structured rank-1 terms 4
24 7 Applications Example 3: InsPyro materials data set Data set: an incomplete tensor in which each dimension represents the concentration of a metal in an alloy and the entries are the alloy s melting temperature Challenge: predict melting temperatures of different alloys Solution with SDF: use structured CPD where each factor vector is a sum of RBF kernels u (n) r u (n) b,r = 8 a exp ( (t b) /(c ) ) i=1 where a b and c are the free parameters in u (n) r
25 Applications Example 3: InsPyro materials data set 0 15 U (3) c 3 (%) 8
26 9 Applications Example 3: InsPyro materials data set,000 1, ,00,000 1, , ,00 1,400 1,600 1,400 c 3 (%) 30 1,800 1,00 1, ,600 1, c (%)
27 30 Applications Example 3: InsPyro materials data set,000 1, ,00 0 1,600 1,400 c 3 (%) 1, , ,00 1,600 1, c (%)
28 31 Applications Example 3: InsPyro materials data set,000 1, ,00 1, ,000 1,400 1,800 1, c 3 (%) , ,400 1, , c (%)
29 Conclusion Coupled matrix/tensor decompositions = important new concept Many applications Gold mine for research topics: algebra (numerical) linear algebra randomized NLA large-scale numerical optimization... 5
Decompositions of Higher-Order Tensors: Concepts and Computation
L. De Lathauwer Decompositions of Higher-Order Tensors: Concepts and Computation Lieven De Lathauwer KU Leuven Belgium Lieven.DeLathauwer@kuleuven-kulak.be 1 L. De Lathauwer Canonical Polyadic Decomposition
More informationDealing with curse and blessing of dimensionality through tensor decompositions
Dealing with curse and blessing of dimensionality through tensor decompositions Lieven De Lathauwer Joint work with Nico Vervliet, Martijn Boussé and Otto Debals June 26, 2017 2 Overview Curse of dimensionality
More informationA randomized block sampling approach to the canonical polyadic decomposition of large-scale tensors
A randomized block sampling approach to the canonical polyadic decomposition of large-scale tensors Nico Vervliet Joint work with Lieven De Lathauwer SIAM AN17, July 13, 2017 2 Classification of hazardous
More information/16/$ IEEE 1728
Extension of the Semi-Algebraic Framework for Approximate CP Decompositions via Simultaneous Matrix Diagonalization to the Efficient Calculation of Coupled CP Decompositions Kristina Naskovska and Martin
More informationThe multiple-vector tensor-vector product
I TD MTVP C KU Leuven August 29, 2013 In collaboration with: N Vanbaelen, K Meerbergen, and R Vandebril Overview I TD MTVP C 1 Introduction Inspiring example Notation 2 Tensor decompositions The CP decomposition
More informationENGG5781 Matrix Analysis and Computations Lecture 10: Non-Negative Matrix Factorization and Tensor Decomposition
ENGG5781 Matrix Analysis and Computations Lecture 10: Non-Negative Matrix Factorization and Tensor Decomposition Wing-Kin (Ken) Ma 2017 2018 Term 2 Department of Electronic Engineering The Chinese University
More informationDimitri Nion & Lieven De Lathauwer
he decomposition of a third-order tensor in block-terms of rank-l,l, Model, lgorithms, Uniqueness, Estimation of and L Dimitri Nion & Lieven De Lathauwer.U. Leuven, ortrijk campus, Belgium E-mails: Dimitri.Nion@kuleuven-kortrijk.be
More informationDownloaded 05/11/15 to Redistribution subject to SIAM license or copyright; see
SIAM J MATRIX ANAL APPL Vol 36, No 2, pp 496 522 c 2015 Society for Industrial and Applied Mathematics Downloaded 05/11/15 to 1345825357 Redistribution subject to SIAM license or copyright; see http://wwwsiamorg/journals/ojsaphp
More informationLinear Algebra (Review) Volker Tresp 2018
Linear Algebra (Review) Volker Tresp 2018 1 Vectors k, M, N are scalars A one-dimensional array c is a column vector. Thus in two dimensions, ( ) c1 c = c 2 c i is the i-th component of c c T = (c 1, c
More informationHigher-Order Singular Value Decomposition (HOSVD) for structured tensors
Higher-Order Singular Value Decomposition (HOSVD) for structured tensors Definition and applications Rémy Boyer Laboratoire des Signaux et Système (L2S) Université Paris-Sud XI GDR ISIS, January 16, 2012
More informationTensorlab. User Guide Getting started 1
Tensorlab User Guide 2014-05-07 Laurent Sorber * S Marc Van Barel * Lieven De Lathauwer S Contents 1 Getting started 1 2 Data sets: dense, incomplete and sparse tensors 5 2.1 Representation................................
More informationDecomposing a three-way dataset of TV-ratings when this is impossible. Alwin Stegeman
Decomposing a three-way dataset of TV-ratings when this is impossible Alwin Stegeman a.w.stegeman@rug.nl www.alwinstegeman.nl 1 Summarizing Data in Simple Patterns Information Technology collection of
More informationJOS M.F. TEN BERGE SIMPLICITY AND TYPICAL RANK RESULTS FOR THREE-WAY ARRAYS
PSYCHOMETRIKA VOL. 76, NO. 1, 3 12 JANUARY 2011 DOI: 10.1007/S11336-010-9193-1 SIMPLICITY AND TYPICAL RANK RESULTS FOR THREE-WAY ARRAYS JOS M.F. TEN BERGE UNIVERSITY OF GRONINGEN Matrices can be diagonalized
More informationTensor Decompositions and Applications
Tamara G. Kolda and Brett W. Bader Part I September 22, 2015 What is tensor? A N-th order tensor is an element of the tensor product of N vector spaces, each of which has its own coordinate system. a =
More informationTime-Delay Estimation via CPD-GEVD Applied to Tensor-based GNSS Arrays with Errors
Time-Delay Estimation via CPD-GEVD Applied to Tensor-based GNSS Arrays with Errors Daniel Valle de Lima 1, João Paulo C. L. da Costa 1, Felix Antreich 2, Ricardo Kerhle Miranda 1, and Giovanni Del Galdo
More informationFitting a Tensor Decomposition is a Nonlinear Optimization Problem
Fitting a Tensor Decomposition is a Nonlinear Optimization Problem Evrim Acar, Daniel M. Dunlavy, and Tamara G. Kolda* Sandia National Laboratories Sandia is a multiprogram laboratory operated by Sandia
More informationBlind Source Separation of Single Channel Mixture Using Tensorization and Tensor Diagonalization
Blind Source Separation of Single Channel Mixture Using Tensorization and Tensor Diagonalization Anh-Huy Phan 1, Petr Tichavský 2(B), and Andrzej Cichocki 1,3 1 Lab for Advanced Brain Signal Processing,
More informationCVPR A New Tensor Algebra - Tutorial. July 26, 2017
CVPR 2017 A New Tensor Algebra - Tutorial Lior Horesh lhoresh@us.ibm.com Misha Kilmer misha.kilmer@tufts.edu July 26, 2017 Outline Motivation Background and notation New t-product and associated algebraic
More informationGeneric uniqueness of a structured matrix factorization and applications in blind source separation
1 Generic uniqueness of a structured matrix factorization and applications in blind source separation Ignat Domanov and Lieven De Lathauwer, Fellow, IEEE arxiv:1601.01173v1 [math.ag] 6 Jan 016 Abstract
More informationTensor Decompositions for Signal Processing Applications
Tensor Decompositions for Signal Processing Applications From Two-way to Multiway Component Analysis A. Cichocki, D. Mandic, A-H. Phan, C. Caiafa, G. Zhou, Q. Zhao, and L. De Lathauwer Summary The widespread
More informationAuthor manuscript: the content is identical to the content of the published paper, but without the final typesetting by the publisher
Citation/eference Domanov I., De Lathauwer L., ``Canonical polyadic decomposition of thirdorder tensors: reduction to generalized eigenvalue decomposition'', SIAM Journal on Matrix Analysis and Applications
More informationModeling Parallel Wiener-Hammerstein Systems Using Tensor Decomposition of Volterra Kernels
Modeling Parallel Wiener-Hammerstein Systems Using Tensor Decomposition of Volterra Kernels Philippe Dreesen 1, David T. Westwick 2, Johan Schoukens 1, Mariya Ishteva 1 1 Vrije Universiteit Brussel (VUB),
More informationOrthogonal tensor decomposition
Orthogonal tensor decomposition Daniel Hsu Columbia University Largely based on 2012 arxiv report Tensor decompositions for learning latent variable models, with Anandkumar, Ge, Kakade, and Telgarsky.
More informationFundamentals of Multilinear Subspace Learning
Chapter 3 Fundamentals of Multilinear Subspace Learning The previous chapter covered background materials on linear subspace learning. From this chapter on, we shall proceed to multiple dimensions with
More informationScalable Tensor Factorizations with Incomplete Data
Scalable Tensor Factorizations with Incomplete Data Tamara G. Kolda & Daniel M. Dunlavy Sandia National Labs Evrim Acar Information Technologies Institute, TUBITAK-UEKAE, Turkey Morten Mørup Technical
More information(0)
Citation/eference Andrzej Cichocki, Danilo P. Mandic, Anh Huy Phan, Cesar F. Caiafa, Guoxu Zhou, Qibin Zhao, and Lieven De Lathauwer, (2015, Tensor Decompositions for Signal Processing Applications. From
More informationMulti-Way Compressed Sensing for Big Tensor Data
Multi-Way Compressed Sensing for Big Tensor Data Nikos Sidiropoulos Dept. ECE University of Minnesota MIIS, July 1, 2013 Nikos Sidiropoulos Dept. ECE University of Minnesota ()Multi-Way Compressed Sensing
More informationNEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS AND DATA PROCESSING
NEW TENSOR DECOMPOSITIONS IN NUMERICAL ANALYSIS AND DATA PROCESSING Institute of Numerical Mathematics of Russian Academy of Sciences eugene.tyrtyshnikov@gmail.com 11 October 2012 COLLABORATION MOSCOW:
More informationThe Canonical Tensor Decomposition and Its Applications to Social Network Analysis
The Canonical Tensor Decomposition and Its Applications to Social Network Analysis Evrim Acar, Tamara G. Kolda and Daniel M. Dunlavy Sandia National Labs Sandia is a multiprogram laboratory operated by
More informationLinear Algebra (Review) Volker Tresp 2017
Linear Algebra (Review) Volker Tresp 2017 1 Vectors k is a scalar (a number) c is a column vector. Thus in two dimensions, c = ( c1 c 2 ) (Advanced: More precisely, a vector is defined in a vector space.
More informationFROM BASIS COMPONENTS TO COMPLEX STRUCTURAL PATTERNS Anh Huy Phan, Andrzej Cichocki, Petr Tichavský, Rafal Zdunek and Sidney Lehky
FROM BASIS COMPONENTS TO COMPLEX STRUCTURAL PATTERNS Anh Huy Phan, Andrzej Cichocki, Petr Tichavský, Rafal Zdunek and Sidney Lehky Brain Science Institute, RIKEN, Wakoshi, Japan Institute of Information
More informationCP DECOMPOSITION AND ITS APPLICATION IN NOISE REDUCTION AND MULTIPLE SOURCES IDENTIFICATION
International Conference on Computer Science and Intelligent Communication (CSIC ) CP DECOMPOSITION AND ITS APPLICATION IN NOISE REDUCTION AND MULTIPLE SOURCES IDENTIFICATION Xuefeng LIU, Yuping FENG,
More informationIntroduction to Tensors. 8 May 2014
Introduction to Tensors 8 May 2014 Introduction to Tensors What is a tensor? Basic Operations CP Decompositions and Tensor Rank Matricization and Computing the CP Dear Tullio,! I admire the elegance of
More informationA Simpler Approach to Low-Rank Tensor Canonical Polyadic Decomposition
A Simpler Approach to Low-ank Tensor Canonical Polyadic Decomposition Daniel L. Pimentel-Alarcón University of Wisconsin-Madison Abstract In this paper we present a simple and efficient method to compute
More informationA variable projection method for block term decomposition of higher-order tensors
A variable projection method for block term decomposition of higher-order tensors Guillaume Olikier 1, P.-A. Absil 1, and Lieven De Lathauwer 2 1- Université catholique de Louvain - ICTEAM Institute B-1348
More informationThe power of low rank TENSOR Approximations in Smart Patient Monitoring
The power of low rank TENSOR Approximations in Smart Patient Monitoring Prof. Sabine Van Huffel EUSIPCO 2017: 25 th European Signal Processing Conference Kos Island, Greece August 28-September 2, 2017
More informationMath 671: Tensor Train decomposition methods
Math 671: Eduardo Corona 1 1 University of Michigan at Ann Arbor December 8, 2016 Table of Contents 1 Preliminaries and goal 2 Unfolding matrices for tensorized arrays The Tensor Train decomposition 3
More informationTENSOR LAYERS FOR COMPRESSION OF DEEP LEARNING NETWORKS. Cris Cecka Senior Research Scientist, NVIDIA GTC 2018
TENSOR LAYERS FOR COMPRESSION OF DEEP LEARNING NETWORKS Cris Cecka Senior Research Scientist, NVIDIA GTC 2018 Tensors Computations and the GPU AGENDA Tensor Networks and Decompositions Tensor Layers in
More informationc 2008 Society for Industrial and Applied Mathematics
SIAM J. MATRIX ANA. APP. Vol. 30, No. 3, pp. 1033 1066 c 2008 Society for Industrial and Applied Mathematics DECOMPOSITIONS OF A HIGHER-ORDER TENSOR IN BOCK TERMS PART II: DEFINITIONS AND UNIQUENESS IEVEN
More informationTensor Decompositions Workshop Discussion Notes American Institute of Mathematics (AIM) Palo Alto, CA
Tensor Decompositions Workshop Discussion Notes American Institute of Mathematics (AIM) Palo Alto, CA Carla D. Moravitz Martin July 9-23, 24 This work was supported by AIM. Center for Applied Mathematics,
More informationA FLEXIBLE MODELING FRAMEWORK FOR COUPLED MATRIX AND TENSOR FACTORIZATIONS
A FLEXIBLE MODELING FRAMEWORK FOR COUPLED MATRIX AND TENSOR FACTORIZATIONS Evrim Acar, Mathias Nilsson, Michael Saunders University of Copenhagen, Faculty of Science, Frederiksberg C, Denmark University
More informationNovel Alternating Least Squares Algorithm for Nonnegative Matrix and Tensor Factorizations
Novel Alternating Least Squares Algorithm for Nonnegative Matrix and Tensor Factorizations Anh Huy Phan 1, Andrzej Cichocki 1,, Rafal Zdunek 1,2,andThanhVuDinh 3 1 Lab for Advanced Brain Signal Processing,
More informationComputing and decomposing tensors
Computing and decomposing tensors Tensor rank decomposition Nick Vannieuwenhoven (FWO / KU Leuven) Sensitivity Condition numbers Tensor rank decomposition Pencil-based algorithms Alternating least squares
More informationLECTURE NOTE #11 PROF. ALAN YUILLE
LECTURE NOTE #11 PROF. ALAN YUILLE 1. NonLinear Dimension Reduction Spectral Methods. The basic idea is to assume that the data lies on a manifold/surface in D-dimensional space, see figure (1) Perform
More informationA Robust Form of Kruskal s Identifiability Theorem
A Robust Form of Kruskal s Identifiability Theorem Aditya Bhaskara (Google NYC) Joint work with Moses Charikar (Princeton University) Aravindan Vijayaraghavan (NYU Northwestern) Background: understanding
More informationc 2008 Society for Industrial and Applied Mathematics
SIAM J. MATRIX ANAL. APPL. Vol. 30, No. 3, pp. 1067 1083 c 2008 Society for Industrial and Applied Mathematics DECOMPOSITIONS OF A HIGHER-ORDER TENSOR IN BLOCK TERMS PART III: ALTERNATING LEAST SQUARES
More informationPostgraduate Course Signal Processing for Big Data (MSc)
Postgraduate Course Signal Processing for Big Data (MSc) Jesús Gustavo Cuevas del Río E-mail: gustavo.cuevas@upm.es Work Phone: +34 91 549 57 00 Ext: 4039 Course Description Instructor Information Course
More informationA concise proof of Kruskal s theorem on tensor decomposition
A concise proof of Kruskal s theorem on tensor decomposition John A. Rhodes 1 Department of Mathematics and Statistics University of Alaska Fairbanks PO Box 756660 Fairbanks, AK 99775 Abstract A theorem
More informationNonlinear Eigenvalue Problems and Contour Integrals
Nonlinear Eigenvalue Problems and Contour Integrals Marc Van Barel a,,, Peter Kravanja a a KU Leuven, Department of Computer Science, Celestijnenlaan 200A, B-300 Leuven (Heverlee), Belgium Abstract In
More informationNumerical Methods in Matrix Computations
Ake Bjorck Numerical Methods in Matrix Computations Springer Contents 1 Direct Methods for Linear Systems 1 1.1 Elements of Matrix Theory 1 1.1.1 Matrix Algebra 2 1.1.2 Vector Spaces 6 1.1.3 Submatrices
More informationRecovering Tensor Data from Incomplete Measurement via Compressive Sampling
Recovering Tensor Data from Incomplete Measurement via Compressive Sampling Jason R. Holloway hollowjr@clarkson.edu Carmeliza Navasca cnavasca@clarkson.edu Department of Electrical Engineering Clarkson
More informationLarge Scale Data Analysis Using Deep Learning
Large Scale Data Analysis Using Deep Learning Linear Algebra U Kang Seoul National University U Kang 1 In This Lecture Overview of linear algebra (but, not a comprehensive survey) Focused on the subset
More informationCombining Sparsity with Physically-Meaningful Constraints in Sparse Parameter Estimation
UIUC CSL Mar. 24 Combining Sparsity with Physically-Meaningful Constraints in Sparse Parameter Estimation Yuejie Chi Department of ECE and BMI Ohio State University Joint work with Yuxin Chen (Stanford).
More informationLecture 1: Introduction to low-rank tensor representation/approximation. Center for Uncertainty Quantification. Alexander Litvinenko
tifica Lecture 1: Introduction to low-rank tensor representation/approximation Alexander Litvinenko http://sri-uq.kaust.edu.sa/ KAUST Figure : KAUST campus, 5 years old, approx. 7000 people (include 1400
More informationMath 671: Tensor Train decomposition methods II
Math 671: Tensor Train decomposition methods II Eduardo Corona 1 1 University of Michigan at Ann Arbor December 13, 2016 Table of Contents 1 What we ve talked about so far: 2 The Tensor Train decomposition
More informationSparseness Constraints on Nonnegative Tensor Decomposition
Sparseness Constraints on Nonnegative Tensor Decomposition Na Li nali@clarksonedu Carmeliza Navasca cnavasca@clarksonedu Department of Mathematics Clarkson University Potsdam, New York 3699, USA Department
More informationProperties of Matrices and Operations on Matrices
Properties of Matrices and Operations on Matrices A common data structure for statistical analysis is a rectangular array or matris. Rows represent individual observational units, or just observations,
More informationBig Tensor Data Reduction
Big Tensor Data Reduction Nikos Sidiropoulos Dept. ECE University of Minnesota NSF/ECCS Big Data, 3/21/2013 Nikos Sidiropoulos Dept. ECE University of Minnesota () Big Tensor Data Reduction NSF/ECCS Big
More informationarxiv: v1 [math.ra] 13 Jan 2009
A CONCISE PROOF OF KRUSKAL S THEOREM ON TENSOR DECOMPOSITION arxiv:0901.1796v1 [math.ra] 13 Jan 2009 JOHN A. RHODES Abstract. A theorem of J. Kruskal from 1977, motivated by a latent-class statistical
More informationElementary Row Operations on Matrices
King Saud University September 17, 018 Table of contents 1 Definition A real matrix is a rectangular array whose entries are real numbers. These numbers are organized on rows and columns. An m n matrix
More informationTutorial on MATLAB for tensors and the Tucker decomposition
Tutorial on MATLAB for tensors and the Tucker decomposition Tamara G. Kolda and Brett W. Bader Workshop on Tensor Decomposition and Applications CIRM, Luminy, Marseille, France August 29, 2005 Sandia is
More informationNUMERICAL METHODS WITH TENSOR REPRESENTATIONS OF DATA
NUMERICAL METHODS WITH TENSOR REPRESENTATIONS OF DATA Institute of Numerical Mathematics of Russian Academy of Sciences eugene.tyrtyshnikov@gmail.com 2 June 2012 COLLABORATION MOSCOW: I.Oseledets, D.Savostyanov
More informationComputational Linear Algebra
Computational Linear Algebra PD Dr. rer. nat. habil. Ralf-Peter Mundani Computation in Engineering / BGU Scientific Computing in Computer Science / INF Winter Term 2018/19 Part 6: Some Other Stuff PD Dr.
More informationInformation Retrieval
Introduction to Information CS276: Information and Web Search Christopher Manning and Pandu Nayak Lecture 13: Latent Semantic Indexing Ch. 18 Today s topic Latent Semantic Indexing Term-document matrices
More informationPencil-based algorithms for tensor rank decomposition are unstable
Pencil-based algorithms for tensor rank decomposition are unstable September 13, 2018 Nick Vannieuwenhoven (FWO / KU Leuven) Introduction Overview 1 Introduction 2 Sensitivity 3 Pencil-based algorithms
More informationLecture 4. CP and KSVD Representations. Charles F. Van Loan
Structured Matrix Computations from Structured Tensors Lecture 4. CP and KSVD Representations Charles F. Van Loan Cornell University CIME-EMS Summer School June 22-26, 2015 Cetraro, Italy Structured Matrix
More informationNonnegative Tensor Factorization using a proximal algorithm: application to 3D fluorescence spectroscopy
Nonnegative Tensor Factorization using a proximal algorithm: application to 3D fluorescence spectroscopy Caroline Chaux Joint work with X. Vu, N. Thirion-Moreau and S. Maire (LSIS, Toulon) Aix-Marseille
More informationFaloutsos, Tong ICDE, 2009
Large Graph Mining: Patterns, Tools and Case Studies Christos Faloutsos Hanghang Tong CMU Copyright: Faloutsos, Tong (29) 2-1 Outline Part 1: Patterns Part 2: Matrix and Tensor Tools Part 3: Proximity
More informationA Brief Guide for TDALAB Ver 1.1. Guoxu Zhou and Andrzej Cichocki
A Brief Guide for TDALAB Ver 1.1 Guoxu Zhou and Andrzej Cichocki April 30, 2013 Contents 1 Preliminary 2 1.1 Highlights of TDALAB...................... 2 1.2 Install and Run TDALAB....................
More informationTENSOR APPROXIMATION TOOLS FREE OF THE CURSE OF DIMENSIONALITY
TENSOR APPROXIMATION TOOLS FREE OF THE CURSE OF DIMENSIONALITY Eugene Tyrtyshnikov Institute of Numerical Mathematics Russian Academy of Sciences (joint work with Ivan Oseledets) WHAT ARE TENSORS? Tensors
More informationTensor Analysis. Topics in Data Mining Fall Bruno Ribeiro
Tensor Analysis Topics in Data Mining Fall 2015 Bruno Ribeiro Tensor Basics But First 2 Mining Matrices 3 Singular Value Decomposition (SVD) } X(i,j) = value of user i for property j i 2 j 5 X(Alice, cholesterol)
More informationarxiv: v2 [math.sp] 23 May 2018
ON THE LARGEST MULTILINEAR SINGULAR VALUES OF HIGHER-ORDER TENSORS IGNAT DOMANOV, ALWIN STEGEMAN, AND LIEVEN DE LATHAUWER Abstract. Let σ n denote the largest mode-n multilinear singular value of an I
More informationIntroduction to Smoothing spline ANOVA models (metamodelling)
Introduction to Smoothing spline ANOVA models (metamodelling) M. Ratto DYNARE Summer School, Paris, June 215. Joint Research Centre www.jrc.ec.europa.eu Serving society Stimulating innovation Supporting
More informationIntroduction to Data Mining
Introduction to Data Mining Lecture #21: Dimensionality Reduction Seoul National University 1 In This Lecture Understand the motivation and applications of dimensionality reduction Learn the definition
More informationA BLIND SPARSE APPROACH FOR ESTIMATING CONSTRAINT MATRICES IN PARALIND DATA MODELS
2th European Signal Processing Conference (EUSIPCO 22) Bucharest, Romania, August 27-3, 22 A BLIND SPARSE APPROACH FOR ESTIMATING CONSTRAINT MATRICES IN PARALIND DATA MODELS F. Caland,2, S. Miron 2 LIMOS,
More informationWindow-based Tensor Analysis on High-dimensional and Multi-aspect Streams
Window-based Tensor Analysis on High-dimensional and Multi-aspect Streams Jimeng Sun Spiros Papadimitriou Philip S. Yu Carnegie Mellon University Pittsburgh, PA, USA IBM T.J. Watson Research Center Hawthorne,
More informationRank Determination for Low-Rank Data Completion
Journal of Machine Learning Research 18 017) 1-9 Submitted 7/17; Revised 8/17; Published 9/17 Rank Determination for Low-Rank Data Completion Morteza Ashraphijuo Columbia University New York, NY 1007,
More informationOptimal solutions to non-negative PARAFAC/multilinear NMF always exist
Optimal solutions to non-negative PARAFAC/multilinear NMF always exist Lek-Heng Lim Stanford University Workshop on Tensor Decompositions and Applications CIRM, Luminy, France August 29 September 2, 2005
More informationCSL361 Problem set 4: Basic linear algebra
CSL361 Problem set 4: Basic linear algebra February 21, 2017 [Note:] If the numerical matrix computations turn out to be tedious, you may use the function rref in Matlab. 1 Row-reduced echelon matrices
More informationEfficient time evolution of one-dimensional quantum systems
Efficient time evolution of one-dimensional quantum systems Frank Pollmann Max-Planck-Institut für komplexer Systeme, Dresden, Germany Sep. 5, 2012 Hsinchu Problems we will address... Finding ground states
More informationOn Kruskal s uniqueness condition for the Candecomp/Parafac decomposition
Linear Algebra and its Applications 420 (2007) 540 552 www.elsevier.com/locate/laa On Kruskal s uniqueness condition for the Candecomp/Parafac decomposition Alwin Stegeman a,,1, Nicholas D. Sidiropoulos
More informationFrom Matrix to Tensor. Charles F. Van Loan
From Matrix to Tensor Charles F. Van Loan Department of Computer Science January 28, 2016 From Matrix to Tensor From Tensor To Matrix 1 / 68 What is a Tensor? Instead of just A(i, j) it s A(i, j, k) or
More informationECE Department, U. of Minnesota, Minneapolis, MN. 3SPICE-ECE-UMN N. Sidiropoulos / SIAM50
( *;:
More informationPractical Linear Algebra: A Geometry Toolbox
Practical Linear Algebra: A Geometry Toolbox Third edition Chapter 12: Gauss for Linear Systems Gerald Farin & Dianne Hansford CRC Press, Taylor & Francis Group, An A K Peters Book www.farinhansford.com/books/pla
More informationSPARSE TENSORS DECOMPOSITION SOFTWARE
SPARSE TENSORS DECOMPOSITION SOFTWARE Papa S Diaw, Master s Candidate Dr. Michael W. Berry, Major Professor Department of Electrical Engineering and Computer Science University of Tennessee, Knoxville
More informationMatrix assembly by low rank tensor approximation
Matrix assembly by low rank tensor approximation Felix Scholz 13.02.2017 References Angelos Mantzaflaris, Bert Juettler, Boris Khoromskij, and Ulrich Langer. Matrix generation in isogeometric analysis
More informationLOW RANK TENSOR DECONVOLUTION
LOW RANK TENSOR DECONVOLUTION Anh-Huy Phan, Petr Tichavský, Andrze Cichocki Brain Science Institute, RIKEN, Wakoshi, apan Systems Research Institute PAS, Warsaw, Poland Institute of Information Theory
More informationPascal Lab. I3S R-PC
Pascal Lab. I3S R-PC-2004-06 Canonical Tensor Decompositions Presented at ARCC Workshop on Tensor Decompositions, American Institute of Mathematics, Palo Alto, CA, July 18 24 2004 Pierre Comon, Laboratoire
More informationDATA MINING LECTURE 8. Dimensionality Reduction PCA -- SVD
DATA MINING LECTURE 8 Dimensionality Reduction PCA -- SVD The curse of dimensionality Real data usually have thousands, or millions of dimensions E.g., web documents, where the dimensionality is the vocabulary
More informationStability of the Gram-Schmidt process
Stability of the Gram-Schmidt process Orthogonal projection We learned in multivariable calculus (or physics or elementary linear algebra) that if q is a unit vector and v is any vector then the orthogonal
More informationNon-negative Tensor Factorization Based on Alternating Large-scale Non-negativity-constrained Least Squares
Non-negative Tensor Factorization Based on Alternating Large-scale Non-negativity-constrained Least Squares Hyunsoo Kim, Haesun Park College of Computing Georgia Institute of Technology 66 Ferst Drive,
More informationWeighted Singular Value Decomposition for Folded Matrices
Weighted Singular Value Decomposition for Folded Matrices SÜHA TUNA İstanbul Technical University Informatics Institute Maslak, 34469, İstanbul TÜRKİYE (TURKEY) suha.tuna@be.itu.edu.tr N.A. BAYKARA Marmara
More informationThis work has been submitted to ChesterRep the University of Chester s online research repository.
This work has been submitted to ChesterRep the University of Chester s online research repository http://chesterrep.openrepository.com Author(s): Daniel Tock Title: Tensor decomposition and its applications
More informationAMS526: Numerical Analysis I (Numerical Linear Algebra)
AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 3: Positive-Definite Systems; Cholesky Factorization Xiangmin Jiao Stony Brook University Xiangmin Jiao Numerical Analysis I 1 / 11 Symmetric
More informationProduct of P-polynomial association schemes
Product of P-polynomial association schemes Ziqing Xiang Shanghai Jiao Tong University Nov 19, 2013 1 / 20 P-polynomial association scheme Definition 1 P-polynomial association scheme A = (X, {A i } 0
More informationSubtracting a best rank-1 approximation may increase tensor rank
Subtracting a best rank- approximation may increase tensor rank Alwin Stegeman, Pierre Comon To cite this version: Alwin Stegeman, Pierre Comon. Subtracting a best rank- approximation may increase tensor
More informationOptimization of Symmetric Tensor Computations
Optimization of Symmetric Tensor Computations Jonathon Cai Department of Computer Science, Yale University New Haven, CT 0650 Email: jonathon.cai@yale.edu Muthu Baskaran, Benoît Meister, Richard Lethin
More informationCANDECOMP/PARAFAC Decomposition of High-order Tensors Through Tensor Reshaping
CANDECOMP/PARAFAC Decomposition of High-order Tensors Through Tensor Reshaping Anh Huy Phan, Petr Tichavský and Andrzej Cichocki Abstract In general, algorithms for order-3 CANDECOMP/- PARAFAC (CP), also
More informationIntroduction to the Tensor Train Decomposition and Its Applications in Machine Learning
Introduction to the Tensor Train Decomposition and Its Applications in Machine Learning Anton Rodomanov Higher School of Economics, Russia Bayesian methods research group (http://bayesgroup.ru) 14 March
More information1776 IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 55, NO. 3, MARCH 2017
1776 IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 55, NO. 3, MARCH 2017 Matrix-Vector Nonnegative Tensor Factorization for Blind Unmixing of Hyperspectral Imagery Yuntao Qian, Member, IEEE,
More information