BMI/STAT 768: Lecture 04 Correlations in Metric Spaces

Transcription

1 BMI/STAT 768: Lecture 04 Correlations in Metric Spaces Moo K. Chung February 1, 2018 The elementary statistical treatment on correlations can be found in [4]: mchung/papers/chung.2007.sage.pdf Correlations are most widely used similarity measure in imaging however, this is not a trivial concept. Here we formulate it as a metric. The mathematical setting and notations will follow [5, 7]. 1 Pearson correlations Definition 1 Consider two data vectors x = (x 1, x 2,, x n ) and y = (y 1, y 2,, y n ). The Pearson correlation coefficient ρ between two vectors x and y is de-

2 fined as ρ(x, y) = n (x i x)(y i ȳ) n (x i x) 2 n (y i ȳ), (1) 2 where x = 1 m n x i and ȳ = 1 m n y i are the sample means. Then algebraic manipulation can show that ρ(x, y) = ρ(ax + b, cy + d) for any nonzero a, c R and any b, d R. Thus, the correlation is scale and translation invariant. The correlation (1) can be factored as a vector product: ρ(x, y) = [x ] y, where x = αx + β and y = γy + δ such that α = γ = 1 n (x i x), β = x 2 n (x i x), 2 1 n (y i ȳ), δ = ȳ 2 n (y i ȳ). 2 Then trivially, correlations under any nontrivial linear transformation, i.e., a, c 0, can be represented as a vector product.

3 Among all linear transformations, following linear transformation is the most often used in relation to sparse models [5, 6]. Consider a transformation that center and scale x and y such that x i = y i = 0, x 2 = x x = y 2 = y y = 1. This projects n-dimensional vectors x, y R n onto a n-dimensional unit sphere S n 1. Thus the centering and scaling operations can be viewed as data embedding from higher Euclidean space R n onto unit sphere S n 1. Then the correlation between x and y is simply the cosine similarity often used in engineering and computer science literature: cos θ = x y, where θ is the angle between vectors x and y. 2 Relation to linear models Assuming x and y are centered and scaled, consider the following linear model

4 y = βx + e, where e is a mean zero error vector and β is unknown scalar parameter we need to estimate. The least squares estimation (LSE) of β is given by minimizing the sum of the squared residuals: e e = (y βx) (y βx). Trivially we can show that LSE of β is β = x y, the Pearson correlation. Even if x and y are not centered and scaled, there is a relationship between correlations and the regression coefficients. This is left as an exercise. 3 Computing large correlation matrices For large correlation matrices of size p p, where p > 10000, the matrix computation takes long time and requires significant memory. The built-in functions in MAT- LAB and R packages are often not written in a computationally efficient fashion. The pairwise computation of

5 correlations is not efficient. A more efficient way is to use the scaling and normalization (2) and compute the correlation matrix as a matrix product [5, 6]. Suppose we have n p data matrices X = (x ij ) and Y = (x ij ), where n is the number of images and p is the number of voxels we are interested in computing correlations. We scale and normalize along the columns such that x ij = 0, y ij = 0, x 2 ij = 1, yij 2 = 1. Then the correlation matrices of X and Y are defined as corr(x) = X X, corr(y ) = Y Y. Since the data is normalized, the covariance matrix is reduced to the correlation matrix. If n p, the correlation matrices are full rank and invertible. If not, we no longer have a well defined correlation matrices and regularizations are possibly needed. The cross-correlation matrices of X and Y are defined as corr(x, Y ) = X Y,

6 which is not symmetric. To make it a symmetric, we can perform a symmetrization: (X Y + Y X)/2. The above procedure can be implemented as function Z = corr2norm(x) [n p]=size(x); meanx=mean(x,1); meanx=repmat(meanx,n,1); X1=X-meanX; diagx1 = sqrt(diag(x1 *X1)); X1=X1./repmat(diagX1, n,1); Z=X1; Once we normalize the data matrix, the cross-correlation matrix C is simply given as the product of martrices: X= corr2norm(x); Y = corr2norm(y); C = X *Y; This is perhaps the fastest way to compute large-scale correlation matrices. Due to scaling, X *X= 1 Y *Y= 1

7 Figure 1: Schematic of computing the cross-correlation matrix in monozygotic (MZ) and same-sex dizygotic (DZ) twin images [6]. Given paired images in twins, the cross-correlation matrix will not be symmetric. Thus, it requires the symmeterization. Often, data matrix X will likely to have missing values, which are often denoted as NaN in MATLAB. The above algorithm assumes there is no missing value. It may necessary to perform missing data treatment for any entry with NaN.

8 Figure 2: fmri cross-correlation network of monozygotic (MZ) and same-sex dizygotic (DZ) twins with nodes [6]. Only positive correlations are shown.

9 4 Correlation Metric Consider a node set V = {1,..., p} and edge weights ρ = (ρ ij ), where ρ ij is the weight between nodes i and j. The edge weights measure similarity or dissimilarity between nodes. The edge weights in most brain networks are usually given by some similarity measure between nodes [7, 8, 9, 10, 12]. Weighted network X = (V, ρ) is formed by the pair of node set V and edge weights ρ. If X is a metric, network interpretation is straightforward. We will show how to construct a metric using correlations. Consider n 1 measurement vector x j = (x 1j,, x nj ) on node j. Suppose we center and rescale the measurement x j such that x j 2 = x jx j = x 2 ij = 1 and x ij = 0. Naturally, we are interested in using correlations or their simple functions as edge wights, i.e., ρ ij = x i x j or ρ ij = 1 x i x j.

10 However, not every functions of correlations are metric. Example 1 (Zhiwei Ma of University of Chicago) ρ ij = 1 x i x j is not a metric. Consider the following 3-node counter example: x i = (0, 1 2, 1 2 ), x j = ( 1 2, 0, 1 2 ), x k = ( 1 6, Then we have ρ ij > ρ ik + ρ jk. 1, 2 ). 6 6 Then interesting methodological question is to identify minimum conditions that make a function of correlations a metric. Theorem 1 For centered and scaled data x 1,, x p, let ρ ij = cos 1 (x i x j ). Then ρ ij is metric. Proof. On unit sphere S n 1, the correlation between x i and x j is the cosine angle θ ij between the two vectors, i.e., x i x j = cos θ ij.

11 The geodesic distance ρ between nodes x i and x j on the unit sphere is given by angle θ ij : ρ(x i, x j ) = cos 1 (x i x j ). For nodes x i, x j S n 1, there are two possible angles θ ij and 2π θ ij depending on if we measure the angles along the shortest arc or longest arc. We take the convention of using the smallest angle in defining θ ij. With this convention, ρ(x i, x j ) π. Given three nodes x i, x j and x k, which forms a spherical triangle, we then have spherical triangle inequality ρ(x i, x j ) ρ(x i, x k ) + ρ(x k, x j ). (2) The proof to (2) is given in [11]. Thus we proved ρ is a metric. Theorem 2 For any metric ρ ij, f(ρ ij ) is also a metric if f(0) = 0 and f(x) is increasing and concave for x > 0 The proof is given in [13]. Example 2 Any power [cos 1 (x i x j)] 1/m for m 1 is metric. When m = 1, we have the simplest possible metric ρ(x i, x j ) = cos 1 (x i x j), which obtains minimum 0 when x i x j = 1 and maximum π when x i x j = 1.

12 Theorem 3 For any x 1,, x p R n, ρ ij = [ 1 corr(x i, y j ) ] 1/2 is a metric, where corr(x i, y j ) is the Pearson correlation. This elementary proof is left as an exercise. References [1] V. Arsigny, P. Fillard, X. Pennec, and N. Ayache. Fast and simple calculus on tensors in the log- Euclidean framework. Lecture Notes in Computer Science, 3749: , [2] V. Arsigny, P. Fillard, X. Pennec, and N. Ayache. Log-Euclidean metrics for fast and simple calculus on diffusion tensors. Magnetic resonance in medicine, 56: , [3] V. Arsigny, P. Fillard, X. Pennec, and N. Ayache. Geometric means in a novel vector space structure on symmetric positive-definite matrices. SIAM journal on matrix analysis and applications, 29: , 2007.

13 [4] M.K. Chung. Correlation coefficient. The Encyclopedia of Measurement and Statistics (Editor Salkind, N.J.), Sage Publications, pages 73 74, [5] M.K. Chung, J.L. Hanson, J. Ye, R.J. Davidson, and S.D. Pollak. Persistent homology in sparse regression and its application to brain morphometry. IEEE Transactions on Medical Imaging, 34: , [6] M.K. Chung, V. Vilalta-Gil, H. Lee, P.J. Rathouz, B.B. Lahey, and D.H. Zald. Exact topological inference for paired brain networks via persistent homology. In Information Processing in Medical Imaging (IPMI), Lecture Notes in Computer Science, volume 10265, pages , [7] H. Lee, M.K. Chung, H. Kang, B.-N. Kim, and D.S. Lee. Computing the shape of brain networks using graph filtration and Gromov-Hausdorff metric. MICCAI, Lecture Notes in Computer Science, 6892: , [8] Y. Li, Y. Liu, J. Li, W. Qin, K. Li, C. Yu, and T. Jiang. Brain Anatomical Network and Intelligence. PLoS Computational Biology, 5(5):e , 2009.

14 [9] AR Mclntosh and F. Gonzalez-Lima. Structural equation modeling and its application to network analysis in functional brain imaging. Human Brain Mapping, 2:2 22, [10] M.E.J. Newman and D.J. Watts. Scaling and percolation in the small-world network model. Physical Review E, 60: , [11] M. Reid and B. Szendròi. Geometry and Topology. Cambridge University Press, [12] C. Song, S. Havlin, and H.A. Makse. Self-similarity of complex networks. Nature, 433: , [13] S. Van Dongen and A.J. Enright. Metric distances derived from cosine similarity and Pearson and Spearman correlations. arxiv preprint arxiv: , 2012.