Multiway Regularized Generalized Canonical Correlation Analysis

Transcription

1 Mutiway Reguarized Generaized Canonica Correation Anaysis Arthur Tenenhaus, Laurent Le Brusquet, Gisea Lechuga To cite this version: Arthur Tenenhaus, Laurent Le Brusquet, Gisea Lechuga. Mutiway Reguarized Generaized Canonica Correation Anaysis. 47èmes Journée de Statistique de a SFdS (JdS 2015), Jun 2015, Lie, France. <ha > HAL Id: ha Submitted on 1 Dec 2015 HAL is a muti-discipinary open access archive for the deposit and dissemination of scientific research documents, whether they are pubished or not. The documents may come from teaching and research institutions in France or abroad, or from pubic or private research centers. L archive ouverte puridiscipinaire HAL, est destinée au dépôt et à a diffusion de documents scientifiques de niveau recherche, pubiés ou non, émanant des étabissements d enseignement et de recherche français ou étrangers, des aboratoires pubics ou privés.

2 Mutiway Reguarized Generaized Canonica Correation Anaysis Arthur Tenenhaus 1,2, Laurent Le Brusquet 1, Gisea Lechuga 1. 1 Laboratoire des Signaux et Systèmes (L2S, UMR CNRS 8506) CNRS - CentraeSupeec - Université Paris-Sud - 3, rue Joiot Curie 91192, Gif-sur-Yvette, prenom.nom@centraesupeec.fr 2 Bioinformatics/Biostatistics Patform IHU-A-ICM, Brain and Spine Institute 47-83, Bd de hôpita; Paris, France Résumé. L Anayse Canonique Généraisée Réguarisée (RGCCA) permet étude des reations entre différents bocs de données. Dans ce papier, une version mutivoie de RGCCA (MGCCA) est proposée. MGCCA cherche à décrire et comprendre es reations entre tenseurs. Mots-cés. Anayse Canonique Généraisée, Données Mutivoie Abstract. Reguarized Generaized Canonica Correation Anaysis (RGCCA) is currenty geared for the anaysis two-way data matrix. In this paper, mutiway RGCCA (MGCCA) extends RGCCA to the mutiway data configuration. More specificay, MGCCA aims at studying the compex reationships between a set of three-way data tabe. Keywords. Generaized Canonica Correation, Mutiway data 1 Introduction On the one hand, mutiboc anaysis concerns the anaysis of data structured in bocs of variabes. In this framewor, a coumn partition X = [X 1,..., X,..., X L ] is considered. Each I J data matrix X is caed a boc and represents a set of J variabes observed on I individuas. The number and nature of the variabes usuay differ from one boc to another but the individuas are the same across bocs. The main aim is to investigate the reationships between bocs. It appears that Reguarized Generaized Canonica Correation Anaysis (RGCCA) [Tenenhaus and Tenenhaus, 2011], is a genera framewor for mutiboc data anaysis. On the other hand, suppose that measurements are avaaibe from, for instance, I individuas on J variabes measured at K occasions. In the itterature, such data is caed three-way data because per occasion measurements are avaiabe for the same group of individuas on the same set of variabes. These three-way data can be coected in an I J K three-way array X, with fronta sices X containing the I J data matrices 1

3 for each of the K occasions. Severa exampes of either three-way data or mutiboc data can be found in a variety of domains incuding chemometrics, psychometry, bioinformatics to name but a few. Nowadays, it frequenty occurs to encounter data combining mutiway and mutiboc structures. For instance, neuroimaging is increasingy recognised as an intermediate phenotype to understand the compex path between genetics and behavioura or cinica phenotypes. In this context, the goa is primariy to identify a set of genetic biomarers that expains some neuroimaging variabiities which impies some modifications of the behavioura. Often, the neuroimaging and the behaviora variabiities are evauated througth time. Thus, it is crucia to perform mutipe experiments (e.g. SNPs, functiona MRI across time, behavioura data across time) on a singe set of patients and the joint anaysis of dataset that gathers two-way and three-way data becomes more and more crucia. RGCCA is currenty geared for the anaysis two-way data matrices. In this paper, mutiway RGCCA (MGCCA) extends RGCCA to the mutiway data configuration. More specificay, MGCCA aims at studying the compex reationships between a set of muti-way data tabe. 2 Reguarized Generaized Canonica Correation anaysis Let us consider L data bocs X 1,..., X,..., X L. Each boc X is of dimension I J. We aso associate to each matrix X a symmetric definite positive matrix M of dimensions J J. In addition, a design matrix C = (c j ) is defined with c j = 1 if bocs X j and X are connected, and = 0 otherwise. RGCCA for muti-boc data anaysis is defined as the foowing optimization probem. maximize w 1,...,w L L j,=1:j c j g (cov(x j w j, X w )) s.t. w M w = 1, = 1,..., L (1) with g(x) a convex function of the scaar x. Typica choices of g(x) are the identity, the absoute vaue or the square function. The vector w is caed vector of weights, the vector X w is caed boc component. RGCCA is recovered through the optimization probem (1) with M = τ I J +(1 τ )X X and the shrinage constant τ varying between 0 and 1. By setting a = M 1/2 w and P = X M 1/2, the optimization probem (1) becomes: maximize a 1,...,a L L j,=1:j c j g (cov(p j a j, P a )) s.t. a a = 1, =,..., L (2) 2

4 A monotone convergent agorithm (i.e. the bounded criteria to be maximized increases at each step of the procedure) is proposed in [Tenenhaus and Tenenhaus, 2011] and in [Tenenhaus and Tenenhaus, 2014] for the resoution of optimization probem (2). 3 Mutiway RGCCA Let us consider L up to order 3 tensors X 1,..., X L. Each tensor X is of dimension I J K. The shared dimension across tensors is the row-mode. Let us consider their corresponding matricized versions X 1,..., X L. Each matrix X = [X 1,..., X,K ] is of dimension I J K and represents a the fronta sices of the tensor next to each other. We aso associate to each X a symmetric definite positive matrix M of dimensions J K J K which usuay has a boc structure. A design matrix C = (c j ) is aso defined with c j = 1 if X j and X are connected, and = 0 otherwise. Taing into account the networ of connections between the three-way data, our objective is to study the reationships between them. Using the same transforms as the ones used previousy, a = M 1/2 a and P = [P 1,..., P,K ] = [X 1,..., X,K ]M 1/2, muti-way RGCCA (MGCCA) is defined as the foowing optimization probem: L maximize f(a 1,..., a L ) = c j g ( P j a j, P a ) a 1,...,a L j,=1:j (3) s.t. a a = 1 and a = c b, = 1,..., L The soe difference between RGCCA and MGCCA reies on the ronecer constraints (structura constraints) that are appied on the outer weight vectors. These structura constraints are usua in the muti-way iterature (see for instance [Bro, 1996]). Two genera set-ups constitute the interna mechanism for the maximization of the optimization probem (3). In the first, the function is to be maximized over different parameter vectors (i.e. a 1,..., a J ), and this is approached by updating each of the parameter vectors in turn, eeping the others fixed. If each update improves the function vaue, the function wi be optimized graduay over the compete set of parameter vectors. This principe is caed boc reaxation by [De Leeuw, 1994]. The second set-up reies on iterative majorization [Hunter and Lange, 2004]. Actuay, at each boc reaxation substep the core optimization that has to be considered in MGCCA is: a s+1 = argmax P a, z s subject to a = c b. (4) a, a =1 3

5 where z s is the so-caed inner component defined by z s = < c g ( P a s, P a s+1 ) P a s+1 + c g ( P a s, P a s ) P a s (5) > The optimization probem (4) bois down to find a set of weight vectors c and b that produce a boc component y = P a with maxima scaar product with z. This probem is equivaent to: ] a s+1 = argmax (z s ) P (c b) = argmax (z s ) [P (c I J ) b [ K [ K = argmax (z s ) c P ]b = argmax c (z s ) P ]b =1 = argmax c Q b (6) where Q is a K J matrix defined by Q = [P 1 zs,..., P K z s ]. From equaity (6), we deduce that c and b, soution of the optimization probem (4), are the first eft and right singuar vectors of the matrix Q. It appears that c and b are constrained to be normaized, satisfying automaticay the unit norm constraint on a. Note that a simiar optimization is found for Mutiinear Partia Least Squares [Bro, 1996]. The entire MGCCA agorithm is described in Agorithm 1. Data: X j s, τ j s, g, ε Resut: a j s Initiaization: choose random unit norm a 0 for each = 1,..., L; s = 0 ; whie f(a s+1 1,..., a s+1 J ) f(a s 1,..., a s J ) < ε do for = 1, 2,..., L do z s = c g ( P a s, P a s+1 ) P a s+1 + c g ( P a s, P a s ) P a s < > =1 a s+1 = c s+1 b s+1 where c s+1 and b s+1 are obtained as the first eft and right singuar vectors of the matrix Q = [P 1 zs,..., P K z s ] of dimension K J. end s = s + 1 ; end Agorithm 1: RGCCA agorithm for three-way data anaysis 4

6 4 Aternative formuation and interpretations This section proposes aternative expressions of the matricized matrices that appear in the MGCCA agorithm. This reformuation aows compementary views/interpretations. New notations are introduced in this section. Let P = [P 1,..., P K ] and P = [P 1,..., P K ], be two matricized matrices of dimension I J K and I J K. Let a = c b and a = c b be the corresponding outer weight vectors. Let P ḷ. = P be the th fronta sice of P of dimension I J and P ḷ j. be the j th atera sice of P of dimension I K. Figure 4 depicts the fronta and atera sices that are handed within MGCCA. (a) Fronta sices (b) Latera sices Figure 1: Fronta (P.. ) and atera (P.j. ) sices of the tensor X that are handed within MGCCA P a = P (I K b )c = = P (c I J )b = ( J ) b j P ḷ j. c (7) j=1 ( K c P ḷ. =1 ) b (8) From equations (7) and (8), we concude that the outer component P a can be expressed as a inear combination of the coumns of either J j=1 b jp ḷ j. or K =1 c P ḷ.. It yieds that the dot product between components can be expressed as foows: P a, P a = (c b ) t P t P (c b ) [ K ] t [ K ] = b t c h P ḷ.h c h P ḳ.h b (9) = c t h=1 [ J h=1 ] t [ J ] b j P ḷ j. b j P ḳ j. c (10) j=1 5 j=1

7 From expressions (7)-(10), it appears that the matricization operations can be avoided and repaced by weigthed means of either the atera or the fronta sices of the tensors X or X. 5 Concusion We have shown that a simpe monotone convergent agorithm can be used for mutiboc/mutiway data. A ot of experiences have been gathered in muti-boc data anaysis. The fied of mutiboc/mutiway data anaysis has been ess expored. Our proposa of using RGCCA for mutiboc/mutiway data anaysis is new and opens a new fied to be expored. References [Bro, 1996] Bro, R. (1996). Mutiway caibration. Mutiinear PLS. Journa of Chemometrics, 10: [Tenenhaus and Tenenhaus, 2011] Tenenhaus, A. and Tenenhaus, M. (2011). Reguarized Generaized Canonica Correation Anaysis. Psychometria, 76: [Tenenhaus and Tenenhaus, 2014] Tenenhaus, A. and Tenenhaus, M. (2014). Reguarized Generaized Canonica Correation Anaysis for mutiboc or mutigroup data anaysis. European Journa of Operationa Research, 238: [De Leeuw, 1994] De Leeuw, J. (1994). Boc reaxation agorithms in statistics. Information Systems and Data Anaysis, Springer, Berin, Boc, H.H. Lensi W. Richter M.M. (Eds.), [Hunter and Lange, 2004] Hunter, D.R. and Lange, K. (2004). A tutoria on MM agorithms. The American Statistician, 58(1):