Analyses for Test-Retest fmri Studies. Documentation

Size: px

Start display at page:

Download "Analyses for Test-Retest fmri Studies. Documentation"

Dale Oliver
6 years ago
Views:

1 University of Wisconsin - Madison Department of Psychiatry Analyses for Test-Retest fmri Studies Documentation Erin K Molloy emolloy@wisc.edu Author Rasmus M Birn rbirn@wisc.edu Supervisor Last modified: April 3, 013 Special Thanks to Rémi Patriat and Taurean Parker

2 Contents 1 Connectivity Matrices Intraclass Correlation Coefficient.1 Definition Example MATLAB Code Kendall s Coefficient of Concordance Definition Example MATLAB Code Root-Mean-Square Deviation Definition Example MATLAB Code Dice Coefficient Definition Example MATLAB Code A Analysis of Variance for Intraclass Correlations 10 A.1 Definition A. Example A.3 MATLAB Code Please cite this as Molloy, E.K. & Birn, R.M. n.d.). Analyses for Test-Retest fmri Studies: Documentation and MATLAB Code. Birn Laboratory Resources. Retrieved month day, year, from 1

3 1 Connectivity Matrices Most of the computation time in functional MRI fmri) reliability analysis comes from the creation of the connectivity matrices. An m x m connectivity matrix is generated for each fmri scan by correlating each of the m fmri signals of interest with every fmri signal of interest and converting the resulting Pearson s r into a z r via the Fisher transform equation 1) [7]. It saves time to only compute the mm 1)/ unique correlations i.e., upper right triangle of matrix). z r 1 ) 1 + r ln 1 r ezr 1 e zr + 1 r 1) The m fmri signals of interest may be selected in different ways, such as averaging the signal within m a priori regions of interest ROIs) [4] or deriving m signals from the fmri data itself via independent components analysis [14]. Depending on the selection method, other correction/processing techniques may be desired. This includes but is not limited to physiological noise correction [8] and the removal of signals from the white matter WM), cerebral spinal fluid CSF), soft tissues, and entire brain i.e., global signal) [9] [3] [] [19]. In addition, it is common to include the six rigid body motion registration parameters in the nuisance variable regression [10]. All of these correction/processing techniques involve regressing out signals of no interest i.e., non-neuronal fluctuations) from the fmri data and carrying the residuals from this regression into the next stage of analysis. One degree of freedom will be lost for each of these regressors; however, most programs will not automatically keep track of this change in the degrees of freedom. Furthermore, when time points corrupted by motion are censored out [15], the degrees of freedom may be different for each fmri scan. Keeping track of the degrees of freedom allows for determination of significance by transforming the Pearson s r with known degrees of freedom) into a t-statistic equation ) and using the cumulative distribution function cdf) to obtain a p-value [7]. t r df t r ) 1 r df + t Conversely, a significant z r threshold can found by determining the t-statistic with known degrees of freedom) into a p-value via the inverse cdf, transforming the t-statistic into an r equation ) and substituting the r into the Fisher transform equation 1). Finally, when determining the significance, correct the p-value for multiple comparisons. For example, a Bonferroni-corrected p-value is the p-value divided by the total number of comparisons, which in this case is the mm 1)/ unique connections [0]. Intraclass Correlation Coefficient The Intraclass Correlation Coefficient ICC) is a measure of the reliability with which k judges independently rate a sample of n targets [18]. In fmri, it has been used to measure the reliability with which fmri scans judges) estimate a functional connection in individuals connectivity matrices targets). Specifically, there will exist an n x k matrix for each functional connection, where n is the number of subjects and k is the number of fmri scans [17]..1 Definition ICCs are divided into three models. In the first model, a different group of k judges, who have been selected from a larger population of judges, rate each target random effects). In the second model, the same group of k judges, who have been selected from a larger population of judges, rate each target mixed effects). In the third model, the same group of k judges, who are the only judges of interest, rate each target fixed

4 effects). Each model has a unique formulation for individual ratings 1) and the mean of several ratings k) [18]. As there is nothing intrinsic about the label scan one for each of the subjects scans all scans have occurred at different times), it is appropriate use a random effects model [17]. Let X be an n x k matrix, where n is the number of targets and k the number of judges. Then let BMS denote the between targets row) mean square, W MS denote the within target row) mean square, JMS denote the between judges column) mean square, and EMS denote the residual mean square see Appendix A). Then define ICC1, 1) BMS W MS BMS + k 1)W MS ICC1, k) BMS W MS BMS 3) ICC, 1) BMS EMS BMS + k 1)EMS + kjms)/n ICC, k) BMS EMS BMS + JMS EMS)/n 4) ICC3, 1) BMS EMS BMS + k 1)EMS ICC3, k) BMS EMS BMS 5). Example Let X be the 6 x 4 matrix [18] X Then see Table, Appendix A) BMS MS br 11.4 W MS MS wr 6.6 and by definition ICC1, 1) JMS MS bc 3.49 EMS MS e ICC1, k) ) ICC, 1) ) )/6 0.9 ICC, k) )/6 0.6 ICC3, 1) ICC3, k) )

5 .3 MATLAB Code 1 function [ ICC ] ICCX, model, r a t i n g ) % 3 % I n t r a c l a s s C o r r e l a t i o n C o e f f i c i e n t 4 % 5 % [ ICC ] ICCX, model, r a t i n g ) 6 % r e t u r n s the I n t r a c l a s s C o r r e l a t i o n C o e f f i c i e n t o f X 7 % 8 % The input X i s an n x k matrix, where n i s the number o f tar 9 % g e t s and k the number o f j u d g e s. ICCs are d i v i d e d i n t o t h r e e 10 % models f o r two types o f r a t i n g s. 11 % 1 % Model Types : 13 % 1 random e f f e c t s model > a d i f f e r e n t group o f k judges, 14 % who have been s e l e c t e d from a l a r g e r p o p u l a t i o n o f jud 15 % ges, r a t e each t a r g e t 16 % mixed e f f e c t s model > the same group o f k judges, who 17 % have been s e l e c t e d from a l a r g e r p o p u l a t i o n o f judges, 18 % r a t e each t a r g e t 19 % 3 f i x e d e f f e c t s model > the same group o f k judges, who 0 % are the only j u d g e s o f i n t e r e s t, r a t e each t a r g e t s. 1 % % Rating Types : 3 % 1 i n d i v i d u a l r a t i n g 4 % k mean o f s e v e r a l r a t i n g s 5 % 6 % S c r i p t : 7 [ n, k ] size X) ; 8 t a b l e ANOVA for ICC X, o f f ) ; 9 30 BMS t a b l e { 6, 4 } ; WMS t a b l e { 7, 4 } ; 31 JMS t a b l e { 3, 4 } ; EMS t a b l e { 8, 4 } ; 3 33 i f i s e q u a l r a t i n g, 1 ) 34 i f i s e q u a l model, 1 ) 35 ICC BMS WMS) / BMS + k 1) WMS) ; 37 e l s e i f i s e q u a l model, ) 38 ICC BMS EMS) / BMS + k 1) EMS + k JMS EMS) /n ) ; 40 e l s e i f i s e q u a l model, 3 ) 41 ICC BMS EMS) /... 4 BMS + k 1) EMS) ; 43 else 44 error Model type does not e x i s t. ) ; 45 end 46 e l s e i f i s e q u a l r a t i n g, k ) 47 i f i s e q u a l model, 1 ) 48 ICC BMS WMS) / 49 BMS; 50 e l s e i f i s e q u a l model, ) 51 ICC BMS EMS) /... 5 BMS + JMS EMS) /n ) ; 53 e l s e i f i s e q u a l model, 3 ) 54 ICC BMS EMS) / 55 BMS; 56 else 57 error Model type does not e x i s t. ) ; 58 end 59 else 60 error Rating type does not e x i s t. ) ; 61 end 6 % 4

6 3 Kendall s Coefficient of Concordance Kendall s Coefficient of Concordance W) is a measure of the consistency with which p judges rank n objects. It ranges from 0 to 1, where 0 indicates no agreement among judges and 1 indicates perfect agreement of the judges [11] [13]. In fmri, it has been used to measure the agreement of ranking subjects objects) across multiple fmri scans judges) based on the strength of one specific connection i.e., there exists one matrix per connection). It has also been used to show the agreement of ranking an individual s connections objects) across multiple functional MRI scans judges) based on the strength of each connection i.e., there exists one matrix per subject). In the former use, multiple analyses with random permutations of the scan order would be appropriate as there is nothing intrinsic about the label scan one for each of the subjects; all scans have occurred at different times [17]. 3.1 Definition Let X be p n matrix, where p is the number of judges and n is the number of objects. Let K i be the total number of unique elements in row i, and let each unique element in row i be denoted by k i, where i {1,..., K i }. Then the tie correction factor for each row is defined as K i T i t 3 k i t ki ), where t ki is the total number instances of k i in row i 6) k Let X ranked be matrix X where the elements of each row have been ranked from 1 to n. If there exists a tie between two or more elements, average the possible rankings for the elements in the tie and assign this averaged rank. Then p R X ranked i,1 R p R j n S X ranked i, p ) X ranked i,n 7) 1 p n + 1) 8) R j R) 9) and by definition 3. Example Let X be the 3 x 5 matrix W p n 3 n) 1 S ) p p T i ) 10) χ p n 1) W, df n 1 11) X As row one has ties there are instances of element 14 and 3 instances of element 13), and both row two and row three have 0 ties there is exactly 1 instance of each element), the tie correction matrix is 3 ) ) 30 T 1 3 1) ) ) ) ) ) ) ) ) ) 0 5

7 In row one, element 14 could be ranked either fourth or fifth, and element 13 could be either ranked first, second, or third. Rank all instances of 13 and 14 with the average of their respective potential ranks to create X ranked Then R ) R ) 9.0 S 7.5 9) + 5 9) + 8 9) ) ) 50.5 Thus Kendall s W is 3.3 MATLAB Code )) )) χ 3 5 1) , df p Please note that X from above should be transposed i.e., the input is an n x p matrix, where n is the number of objects and p is the number of judges). 1 function [W, Chi, df, pval ] KendallsW X) % 3 % Kendall s C o r r e l a t i o n o f Concordance W) 4 % 5 % [W] KendallsW X) 6 % r e t u r n s the Kendall s W o f X 7 % [W, Chi, df, pval ] KendallsW X) 8 % r e t u r n s the Kendall s W o f X with 9 % Friedman s Chi Squared, d e g r e e s o f 10 % freedom, and p value 11 % 1 % The input X i s an n x p matrix, where n i s the number o f 13 % o b j e c t s and p i s the number o f j u d g e s i. e., ranking o c c u r s 14 % within columns ). 15 % 16 % S c r i p t : 17 [ n, p ] size X) ; % Tie C o r r e c t i o n Factors 0 for i 1 : p 1 [V, iv, i x ] unique X :, i ) ) ; i x s sort i x ) ; t hist i x s, i x s end) ) ; 3 T i ) sum t. ˆ 3 t ) ; 4 end 5 6 % Kendall s W 7 R sum t i e d r a n k X), ) ; R mean meanr) ; 8 S sum R R mean ). ˆ ) ; 9 W 1 S ) / p ˆ nˆ3 n ) ) p sumt) ) ; % Friedman s Chi Squared, d e g r e e s o f freedom, and p value 3 Chi p n 1) W; df n 1; 33 pval gammainc Chi /, df /, upper ) ; 34 % 6

8 4 Root-Mean-Square Deviation The Root-Mean-Standard Deviation is typically used to describe the difference between predicted values from a model and experimentally observed values [16]; however, in functional MRI, it has been used examine the reproducibility with which functional MRI can estimate an individual s connectivity matrix [1]. This is easily generalizable to 3-dimensions for use on Statistical Parametric Maps SPMs). While this metric is not biased by thresholding see Dice Coefficient), it is extremely important to be working with normalized connectivity matrices. In addition, this metric would be biased by head motion or anything that would systematically alter the correlations. 4.1 Definition Let A and B be m x n matrices. Then the root-mean-square deviation is defined as A ij B ij ) A B) m n m n 1) 4. Example Let A and B be the 3 x 5 matrices A B Then A B) Thus the Root-Mean-Square Deviation is functional MRI literature [1]: I originally found the root-mean-square deviation as the mean difference in correlation used by Anderson et al. 011; however, the formula given on page 549 does not square the difference. I assumed this was an error, since this definition could result in a negative under the radical. Furthermore, in the case below the mean difference in correlation would be 0 when it is clearly 1. Let A and B be the 1 x 4 matrices A ) B 6 6 ) Then A B ) ) ) m A B) m n A B) m n 7

9 4.3 MATLAB Code 1 function [RMSD] RMSDA, B) % 3 % Root Mean Square Deviation 4 % 5 % RMSD RMSDA, B) 6 % r e t u r n s the RMSD o f A and B 7 % 8 % The i n p u t s A and B are both n d i m e n s i o n a l m a t r i c e s. 9 % 10 % S c r i p t : 11 i f i s e q u a l size A), size B) ) 1 error Matrices dimensions do not match. ) ; 13 end s q d i f s A B). ˆ ; 16 RMSD sqrt sum s q d i f s : ) ) / prod size A) ) ) ; 17 % 5 Dice Coefficient The Dice Coefficient is a measure of the similarity between sets ranging from 0 to 1, where 0 indicates the sets are disjoint and 1 indicates the sets are identical [6]. In functional MRI, it has been used to assess the similarity between those connections which survive a particular threshold in two different connectivity matrices e.g., two matrices from the same individual). This method is dependent on the choice of threshold, so it is preferable to use a meaningful threshold. For example, consider connections that survive thresholding by a significant p-value Bonferroni-corrected) to be connected i.e., exist in the set) and connections that do not survive to be un-connected i.e., do not exist in the set). This is easily generalizable to 3-dimensions for use on Statistical Parametric Maps SPMs). Note that the Dice Coefficient is applied to sets; however, each connection could be considered an edge in graph. Consider the set of nodes {a, b, c, d, e, f} and graphs G 1 and G with edges {a, c), b, c), c, d), d, e), d, f)} and {a, c), b, c), d, e), d, f)}, respectively. The Dice Coefficient between these sets of edges is 0.89 very high); however, c, d) is an edge cut making graph G disconnected. The Dice Coefficient does not account for how differences between sets of edges affect the fundamental properties of their graphs. 5.1 Definition Let A and B be sets. Then the Dice Coefficient of A and B is defined as simply A B A + B 13) functional MRI literature [5]: Let X be a m x n matrix and given a value t define the function fx ij, t) { 1 if Xij > t 0 if X ij t, where i {1,..., m} and j {1,..., n} 14) If fx ij ) 1, then we say the element with indices ij fx). Similarly, if fx ij ) 0, then we say the element with indices ij / fx). Therefore, we can define Dice Coefficient of m x n matrices A and B with 8

10 respective thresholds t A and t B as m fa ij, t A )fb ij, t B ) m 5. Example fa ij, t A ) + m fb ij, t B ) m fa, t A ) fb, t B )) ij fa, t A )) ij + m 15) fb, t B )) ij Let t A t B 13, and let A and B be the 3 x 5 matrices A B Then and Thus the Dice Coefficient is 5.3 MATLAB Code fa, t A ) fb, t B ) fa, t A ) fb, t B ) function [DC] DCA, B, ta, tb, a b s o l u t e ) % 3 % Dice C o e f f i c i e n t i. e., S i m i l a r i t y Index f o r S e t s ) 4 % 5 % [DC] DCA, B, ta, tb, a b s o l u t e ) 6 % r e t u r n s the Dice C o e f f i c i e n t o f A and B 7 % 8 % The i n p u t s A and B are both n d i m e n s i o n a l m a t r i c e s ; ta and tb 9 % are the t h r e s h o l d s f o r A and B, r e s p e c t i v e l y. Absolute i s a 10 % l o g i c a l, which determines how the t h r e s h o l d i s a p p l i e d t r u e /1 11 % a b s o l u t e value g r e a t e r than t h r e s h o l d, f a l s e /0 value great 1 % e r than t h r e s h o l d ). 13 % 14 % S c r i p t : 15 i f i s e q u a l size A), size B) ) 16 error Matrix dimensions do not match. ) ; 17 end % Threshold Matrices 0 i f i s e q u a l a b s o l u t e, 1) 1 fa abs A)>tA ; fb abs B)>tB ; else 3 fa A>tA ; fb B>tB ; 4 end 5 fafb fa. fb ; 6 7 % Dice C o e f f i c i e n t 8 DC sum fafb : ) ) ) / sum fa : ) ) + sum fb : ) ) ) ; 9 % 9

11 A Analysis of Variance for Intraclass Correlations A.1 Definition Let X be the m x n matrix. Then define the following quantities [7] SS bc ) X j m ) X ij m 1) n 1) SS br m ) ) X i X ij n m 1) n 1) MS bc SS bc n 1 MS br SS br m 1 SS wc X ij 1 m X j SS wr X ij 1 n m ) X i MS wc SS wc nm 1) MS wr SS wr mn 1) SS e X ij 1 n m ) X i 1 m X j ) X ij + m 1) n 1) MS e SS e m 1)n 1) F bc SS bc MS e, where df 1 n 1 and df m 1)n 1) F br SS br MS e, where df 1 m 1 and df m 1)n 1) Table 1: ANOVA Summary Table Source SS df MS F p Columns between SS bc n 1) MS bc F bc p bc within SS wc mn 1) MS wc Rows between SS br m 1) MS br F br p br within SS wr nm 1) MS wr Residual SS e m 1)n 1) MS e 10

12 Note that all quantities can be calculated from the same four terms: B c ) X j m A m m X ij ) X ij D m 1) n 1) B r m ) X i n A. Example Let X be the 6 x 4 matrix [18] X X Then m X i ) m ) X i ) 4 1 X j X j and thus by definition B r B c A m ) X j m m X ij ) X i n

13 m ) X ij D m 1) n 1) ) m 1) n 1) SS bc B c A SS wc D S c MS bc MS 71.5 wc 4 6 1) 3.58 SS br B r A SS wr D B r MS br MS wr ) 6.6 SS e MS e ) 4 1) 1.0 F bc , where df 1 3 and df 15 p bc 9.45e 07 F br , where df 1 5 and df 15 p br 1.35e 04 Table : ANOVA Summary Table Source SS df MS F p Columns between e-7 within Rows between e-4 within Residual

14 A.3 MATLAB Code 1 function [ t a b l e ] ANOVA for ICC X, d i s p l a y o p t ) % 3 % A n a l y s i s o f Variance f o r Ratings f o r I n t r a c l a s s C o r r e l a t i o n s ) 4 % 5 % [ t a b l e ] ANOVA for ICC X, d i s p l a y o p t ) 6 % r e t u r n s the ANOVA summary t a b l e as a c e l l array 7 % 8 % The input X i s an n x k matrix, where n i s the number o f tar 9 % g e t s and k i s he number o f j u d g e s. The o ption o f d i s p l a y i n g 10 % an ANOVA summary t a b l e i s handled by d i s p l a y o p t, which can be 11 % s e t to on d e f a u l t ) or o f f. 1 % 13 % S c r i p t : 14 [m, n ] size X) ; 15 t o t a l m n ; A sumsumx. ˆ ) ) ; 18 Bc sumsumx, 1 ). ˆ ) /m; 19 Br sumsumx, ). ˆ ) /n ; 0 D sumsumx) ) ˆ/ t o t a l ; 1 SSbc Bc D; 3 SSwc A Bc ; 4 SSbr Br D; 5 SSwr A Br ; 6 SSe A Br Bc+D; 7 8 DFbc n 1; 9 DFwc n m 1) ; 30 DFbr m 1; 31 DFwr m n 1) ; 3 DFe DFbc DFbr ; MSbc SSbc/DFbc ; 35 MSwc SSwc/DFwc ; 36 MSbr SSbr /DFbr ; 37 MSwr SSwr/DFwr ; 38 MSe SSe/DFe ; Fbc MSbc/MSe ; Pbc 1 f c d f Fbc, DFbc, DFe) ; 41 Fbr MSbr/MSe ; Pbr 1 f c d f Fbr, DFbr, DFe) ; 4 43 t a b l e { Source o f Variance SS df MS F Prob>F 44 Columns [ ] [ ] [ ] [ ] [ ] 45 betweeen SSbc DFbc MSbc Fbc Pbc 46 within SSwc DFwc MSwc [ ] [ ] 47 Rows [ ] [ ] [ ] [ ] [ ] 48 between SSbr DFbr MSbr Fbr Pbr 49 within SSwr DFwr MSwr [ ] [ ] 50 R e s i d u a l SSe DFe MSe [ ] [ ] } ; 51 5 i f i s e q u a l nargin, 1) i s e q u a l d i s p l a y o p t, on ) 53 s t a t d i s p t a b l e t a b l e, ANOVA f o r ICC, ANOVA Summary Table, ) ; 55 end 56 % 13

15 References [1] Anderson J.S., Ferguson M.A., Lopez-Larson M. & Yurgelun-Todd D. 011). Reproducibility of Single- Subject Functional Connectivity Measurements. American Journal of Neuroradiology, 33), [] Anderson J.S., Druzgal T.J., Lopez-Larson M., Jeong E.K., Desai K. & Yurgelun-Todd D. 011). Network anticorrelations, global regression, and phase-shifted soft tissue correction. Human Brain Mapping, 36), [3] Behzadi Y., Restom K., Liau J. & Liu T.T. 007). A Component Based Noise Correction Method CompCor) for BOLD and Perfusion Based fmri. NeuroImage, 371), [4] Biswal B., Yetkin F.Z., Haughton V.M. & Hyde J.S. 1995). Functional connectivity in the motor cortex of resting brain using echo-planar MRI. Magnetic Resonance in Medicine, 344), [5] Craddock R.C., James G.A., Holtzheimer, III P.E., Hu X.P. & Mayber H.S. 011). A Whole Brain fmri Atlas Generated via Spatially Constrained Spectral Clustering. Human Brain Mapping, 338), [6] Dice L.R. 1945). Measures of the Amount of Ecologic Association Between Species. Ecology, 63), [7] Frank H. & Althoen S.C. 1994). Statistics: Concepts and Applications. New York, NY: Cambridge University Press, pages , 584, [8] Glover G.H., Li, T.Q. & Ress D. 000). Images-Based Method for Retrospective Correction of Physiological Motion Effects in fmri: RETROICOR. Magnetic Resonance in Medicine, 441), [9] Jo H.J., Saad Z.S., Simmons W.K., Milbury L.A. &Cox R.W. 010). Mapping sources of correlation in resting state FMRI, with artifact detection and removal. NeuroImage, 5), [10] Johnstone T., Ores Walsh K.S., Greischar L.L., Alexander A.L., Fox A.S., Davidson R.J. & Oakes T.R. 006). Motion correction and use of motion covariates in multiple-subject fmri analysis. Human Brain Mapping, 710), [11] Kendall M.G. & Smith B.B. 1939). The Problem of m Rankings. The Annals of Mathematical Statisitcs, 103), [1] LaTeX n.d.). In WikiBooks - Open books for an open world. Retrieved September 9, 01, from [13] Legendre P. 005). Species Associations: The Kendall Coefficient of Concordance Revisited. Journal of Agricultural, Biological and Environmental Statistics, 10), [14] McKeon M.J., Hansen L.K. & Sejnowk T.J. 003). Independent component analysis of functional MRI: what is signal and what is noise? Curr. Opin. Neurobiology, 13, [15] Power J.D., Barnes K.A., Snyder A.Z., Shlagger B.L. & Petersen S.E. 01). Spurious but systematic correlations in functional connectivity MRI networks arise from subject motion. NeuroImage, 593), [16] Root-Mean-Square Deviation. n.d.). In Wikipedia. Retrieved September 9, 01, from deviation. [17] Shehzad Z., Kelly A.M., Reiss P.T., Gee D.J., Gotimer K., Uddin L.Q., Lee S.H., Margulies D.S., Roy A.K., Biswal B.B., Petkova E., Castellanos F.X. & Milham M.P. 009). The resting brain: unconstrained yet reliable. Cerebral Cortex, 1910),

16 [18] Shrout P.E. & Fleiss J.L. 1979). Intraclass Correlations: Uses in Assessing Rater Reliability. Psychological Bulletin, 86), [19] Weissenbacher A., Kasess C., Gerstl F., Lanzenberger R., Moser E. & Windischberger C. 009). Correlations and anticorrelations in resting-state functional connectivity MRI: A quantitative comparison of preprocessing strategies. NeuroImage, 474), [0] Weisstein E.W. n.d.). Bonferroni Correction. MathWorld A Wolfram Web Resource. Retrieved September 9, 01, from 15

Statistical Analysis Aspects of Resting State Functional Connectivity

Statistical Analysis Aspects of Resting State Functional Connectivity Biswal s result (1995) Correlations between RS Fluctuations of left and right motor areas Why studying resting state? Human Brain =