Estimating representational dissimilarity measures

Estimating representational dissimilarity measures lexander Walther <awalthermail@gmail.com> MRC Cognition and rain Sciences Unit University of Cambridge Institute of Cognitive Neuroscience University College London

Overview Distance measures for Representational Similarity nalysis fmri pattern noise normalization Crossvalidated dissimilarity measures Distance measures vs. pattern classifiers

What does Representational Similarity nalysis measure? General linear model True Regression Coefficients Ordinary least squares ˆ = (X T X) X T Y Estimated regression coefficients True activity patterns U Dissimilarity measure

The Euclidean distance in RS fmri activity patterns U u 2 = u 2 2, + u,2 u 2 = u 2 2, + u,2.8.5!.3.6! u u u.6,.3 u,2 u,2 d(,) u,2.5 u,2 Voxel u,.8 Representational dissimilarity matrix d d Voxel 2 u, u, Squared Euclidean distance d(,) 2 = (u u ) 2 + (u u ) 2,,,2,2 = (u u )(u u ) T

Euclidean distance & Pearson correlation distance fmri activity patterns U C 62.8.5.2! u.3.6.5! u u C Voxel 3 d(,) u u Voxel 2

Euclidean distance & Pearson correlation distance fmri activity patterns U.8.5.2! u.3.6.5! u Voxel 3 Voxel 2

Euclidean distance & Pearson correlation distance fmri activity patterns U C 62.8.5.2! u.3.6.5! u u C u * = u u Voxel 3 u * = u u Voxel 2

Euclidean distance & Pearson correlation distance fmri activity patterns U C 62.8.5.2! u.3.6.5! u u C u * = u u σ u Voxel 3 u * = u u σ u r = cos(u * u * ) u * Correlation distance d(,) = r α u * Voxel 2

Euclidean distance & Pearson correlation distance Euclidean distance is invariant to baseline shifts Correlation distance is invariant to scaling differences

Reliability of dissimilarity measures Split-half reliability analysis Stimulus 2 3 4 5 Stimulus 2 3 4 5 r Split Split 2 high low SVM LD Euclidean Correlation LDC LDt RDM split-half reliability [Pearson correlation].5 High SNR.7.35 Low SNR

Overview Distance measures for Representational Similarity nalysis fmri pattern noise normalization Crossvalidated dissimilarity measures Distance measures vs. pattern classifiers

Univariate & multivariate noise normalization General linear model.8.5!.3.6! Noise normalizations Univariate ˆb p * = ˆb p σ εp for p = P Voxel ˆΣ ε = T εt ε Multivariate û = ˆb ˆΣ 2 Voxel

From raw to noise normalized dissimilarity measures fmri beta weights ˆ.8.5!.3.6! ˆb ˆb Squared Euclidean distance d(,) 2 = ( ˆb ˆb )( ˆb ˆb ) T Multivariate noise normalization û = ˆb ˆΣ 2 Squared Mahalanobis distance d(,) 2 = (û û )(û û ) T Representational dissimilarity matrix d d

Overview Distance measures for Representational Similarity nalysis fmri pattern noise normalization Crossvalidated dissimilarity measures Distance measures vs. pattern classifiers

Conventional distances are positively biased

Crossvalidated distance estimates Run Run 2 Û ().8.5!.3.6! û () û () Û (2).7.6!.5.4! û (2) û (2) û () = u () + ε () û (2) = u (2) + ε (2) û () = u () + ε () û (2) = u (2) + ε (2) Crossvalidated distance d(,) = (û () û () )(û (2) û (2) ) T Training Testing

Conventional distances are positively biased

Conventional distances are positively biased Run Training ˆd(û,û ) = û () û () Run 2 Testing ( )( û (2) (2) û ) T Estimated activity patterns Training Testing True activity patterns û = u + ε û 2 = u 2 + ε 2 Estimated error = û ()û (2) T û ()û (2) T û ()û (2) T + û ()û (2) T Crossvalidation = ( u () + ε () )( u (2) + ε (2) ) T ( u () + ε () )( u (2) + ε (2) ) T ( u () + ε () )( u (2) + ε (2) ) T... = + ( u () + ε () )( u (2) + ε (2) ) T = u () u (2) T + u () ε (2)T + ε () u (2) T + ε () ε (2)T u () u (2) T u () ε (2)T ε () u (2) T ε () ε (2)T... = u () u (2) T u () ε (2)T... ε () u (2) T ε () ε (2)T + u () u (2) T + ε (2) u () T + ε () u (2) T + ε () ε (2)T E ( ˆd(û,û )) = u () (2) u T u () (2) u T u () (2) u T + u () (2) u T = ( u () () u )( u (2) (2) u ) T = d(u,u )

Crossvalidation schemes Crossvalidated distance ˆd(,) = (û () û () )(û (2) û (2) ) T Training Testing ˆ δ () ˆ δ (2) Split-half crossvalidation Leave-one-run-out crossvalidation Run ˆ δ () ˆ δ () ˆ δ () ˆ δ () ˆ δ () ˆ δ () ˆ δ () ˆ δ () ˆ δ () ˆ δ () Run 2 Run 3 ˆ δ (2) ˆ δ (3) ˆ δ (2) ˆ δ (3) ˆ δ (2) ˆ δ (3) = ˆ δ (2) ˆ δ (3) ˆ δ (2) ˆ δ (3) ˆ δ (2) ˆ δ (3) ˆ δ (2) ˆ δ (3) = ˆ δ (2) ˆ δ (3) ˆ δ (2) ˆ δ (3) ˆ δ (2) ˆ δ (3) Run 4 ˆ δ (4) ˆ δ (4) ˆ δ (4) ˆ δ (4) ˆ δ (4) ˆ δ (4) ˆ δ (4) ˆ δ (4) ˆ δ (4) ˆ δ (4) ˆd(,) = δ ˆ () + ˆ 3 2 δ (2)! δ ˆ (3) + ˆ 2 δ (4) T 4 ( δ ˆ () ) δ ˆ (2) + δ ˆ(3) + δ ˆ(4)! 3 T 6 ( δ ˆ () δ ˆ (2) T )!

From raw to noise normalized dissimilarity measures fmri beta weights ˆ.8.5!.3.6! ˆb ˆb Squared Euclidean distance d(,) = ( ˆb ˆb )( ˆb ˆb ) T Multivariate noise normalization û = ˆb ˆΣ 2 Squared Mahalanobis distance d(,) = (û û )(û û ) T Crossvalidation Representational dissimilarity matrix d d Crossvalidated Mahalanobis distance estimate ( Linear discriminant contrast, LDC) ˆd(,) = (û () u () )(u (2) u (2) ) T

From conventional to crossvalidated dissimilarity measures C! 62 fmri beta weights ˆ.8.5!.3.6!.5.!! C 62 u C ˆb ˆb ˆb C Crossvalidated Mahalanobis distance estimate ( Linear discriminant contrast, LDC) ˆd(i, j) = (û i () û j () )(û i (2) û j (2) ) T Representational dissimilarity matrix (FF) faces bodies inanimate Preserves ratio between distances faces inanimate bodies t test (FDR=.5) 6 3 dissimilarity [Linear discriminent contrast]

From crossvalidated Mahalanobis to Linear Discriminant t fmri beta weights ˆ.8.5!.3.6! ˆb ˆb Crossvalidated Mahalanobis distance estimate ( Linear discriminant contrast, LDC) ˆd(,) = (û () û () )(û (2) û (2) ) T Training w Testing Normalize by standard error SE LDC σ ε 2 = diag(w ˆΣ b w T ) SE LDC = σ ε 2 ( c X T ) X c T Representational dissimilarity matrix t t Linear discriminant t value, LDt ( ) ˆt(,) = (û () û () )(û (2) û (2) ) T SE LDC

Overview Distance measures for Representational Similarity nalysis fmri pattern noise normalization Crossvalidated dissimilarity measures Distance measures vs. pattern classifiers

Distances vs. linear classifiers Split-half reliability analysis Stimulus 2 3 4 5 Stimulus 2 3 4 5 r Split Split 2 high low Euclidean Correlation LDC LDt SVM LD RDM split-half reliability [Pearson correlation] High SNR.7.5.35 Multivariate noise normalization Low SNR

Walther et al (in review) Key insights Euclidean and correlation distance measure dissimilarity differently and are invariant to different pattern transformations 2 Multivariate noise normalization significantly enhances the reliability of dissimilarity measures 3 Crossvalidated dissimilarity estimates are unbiased and ratio scale (with interpretable zero point) 4 Pair-wise classification accuracy is a quantized measure of representational distinctness and significantly less reliable than continuous distance measures

Thanks!