Enroatrso.o ay IEIR' Elmer" decomposition. I= ±Erob roer. decomposition. IE,a YoEYo. The CP decomposition. decomposition - given.

Size: px

Start display at page:

Download "Enroatrso.o ay IEIR' Elmer" decomposition. I= ±Erob roer. decomposition. IE,a YoEYo. The CP decomposition. decomposition - given."

Alexandrina Patrick
5 years ago
Views:

1 theorem undthethankofateusorllectorouterproduct En Em # ' N ) isan N of way N tensor vectors Iu HoE' no oa M with every element defined as the producer of the corresponding elements of the vectors tiniaeio?taylayy Effie f±# Eijiiaibjck

2 rank The CP decomposition The exact CP decomposition ' # an N way tensor Enroatrso o ay IEIR' I % Elmer where REIN and of has the form for all are [ n ] and re [ RT In the approximate for fixed decomposition the size R is and we're looking for the least ) given ( P error decomposition IEa YoEYo HI 'll oay For tensors now and we concentrate on 3 write way I ±Erob roer

3 ait th In In Visually the 3 way CP is [# TEf 7EFI + We can gather mode in factor tmalles setting A[ a E for IEIR ' k the vector for each we B :[ II have ] the 3 way CEEI IXR JXR K xp gd We can express the 3 CP decomposition way the slices using of I and frontal the factor matrices : Tk ADMBT where D ' diuy KC ( k :)) ie a diagonal matrix with the k now of C on its diagonal

4 Rao IEIIEEFEHEF Elate The frontal slee formulation doesn't generalize easily for more than } modes For more generalized representation weneed the KhatriRamatrixprodu : AEIRKK and BEIR product is ' ' given matrices their Khatri Rao aonyamil?ieeiiiiiihideni That 1 times is each column of B is copied and the i th copy of the K column B multiplied by uik Khatri product can be written The th more concisely using the

5 Rao Thoth? ftpadf?iaysmeai a top Notice that in Kronecker product the matrices can be of arbitrary size whereas in Khatri Rao they must have the same number of columns The Khatri Rao product of A e IR and BEIRNK can now be written as A 013 [ b? Eaten ] that is Khatri is column wise knnecker product If? Better AEIR KXR and CEIR are the factor matrices of a ( P decomposition of tensor IEIR ' k then

6 The DK Tc A ( COIDT Tea D ( CO A) T Ta CCBCOAT More generally and factor matrices A ' if I has A modes An Am Tcnj At ) (AMO Oakmont ' no OAMF To garn intuition on the Khatri Rao formulation consider the frontal slice formulation of CP : with D ' ' factor slices A TkAD 'Bt diag ( CCIKD ' appears so we can [ IT da[d' it just saume with ale frontal at isbt stuck them : E 'M

7 ftptt D I Ta Tk A ' EBT ' BtD DH BT The first row of E has the first column of B multiplied by 4 followed by the first column of B multiplied by Ca and so on E [ e Isis I Hence ge#lexoit Extending this to all rows of 't we see that ET ( COBY and hence To 't Its TEFACCOBYI The connections in the other modes can be derived analogously

8 One sometimes normalizes the columns Of the factor matrices to unit length The lengths are then stored in factors Xr lliirlliltbilltlerll collected in a vector IELRR or in a matrix ^ Then Th A^( COBY ete diagtxy e and IRRM It xee?broe A common notation for the CP decomposition is to write I [ A B CD :{ nu?oiroi or with the scaling IEI ; A BCI#xriioIriir

9 This The ALIEN formulations In AkoBT etc provide a way to solve the ( approximate ) ( P decomposition When ( and 13 are fixed ( GB is a fixed matrix and the problem becomes ; Ta and D minimizes 4Th solved as A Th using DTY ( Penrose pseudo the sample matrices find Given call it D matrix A that ADTHP can be the SVD and pseudo inverse where ( it is the Moore inverse following algorithm : random 13 and C repeat Tear A Th ( ( C 013Mt until let 13 Tea ( ( CoA + )t ) let C Te )( ( BOAFH convergence This leads to

10 The ALS algorithm requires us to compute ( COBY (COAT and R by JK R the pseudo inverses by ( BOAT IK and R which of are by 1J matrices respectively operation a full row often but This is an expensive if have which rank is thesematrices IK JK } then R< & min { 1J likely we can use the following equality as ( to B) where X * Y is the + KATA) * (BTBDYAON T # Hadamartmatrixproduct I or element wise product) ' between *4( and YEIR ' XEIR XMYM Xnzynz ftpiix#ifb)er' man Xii Yin Xis Yls

11 Similarly The proof of identity (8) is left as a homework but it involves the following identity that is also on : occasionally useful itself For XEIRN ' and YEIR ' k we have (XOYYTXOY) xtx * Yty Proof : Let X 'XtX and notice that x '*cxi yj* < ipj # > ftp for Now ) let YEYTY LEog in#oxgifixixoyxyxogd wehave and consider a single element zpne : Hate ( zpecxia@jrpex0fpilelxikprxiefe7iexikxae Fr Be > > < ipriye ' xkeyke and hence ZX' * Y 'xt * yty p

12 With equality ( * ) we can write A Ta ELCEBIJ RXJK as A Th kobj[ etc * BTBT RXR and we only have to take the pseudo inverse from a much smaller matrix This formulation can however issues with the numerical cause stability ALS is not the only possibility We Can instead use for instance gradient based methods : each row iii can he updated based on the gradient iii iii soak' nl#ijhaitcobstdi ALS is the most commonly used approach though

Introduction to Tensors. 8 May 2014

Introduction to Tensors. 8 May 2014 Introduction to Tensors 8 May 2014 Introduction to Tensors What is a tensor? Basic Operations CP Decompositions and Tensor Rank Matricization and Computing the CP Dear Tullio,! I admire the elegance of