Model-free Functional Data Analysis

Model-free Functional Data Analysis MELODIC Multivariate Exploratory Linear Optimised Decomposition into Independent Components decomposes data into a set of statistically independent spatial component maps and associated time courses fully automated (incl estimation of the number of components) inference on IC maps using alternative hypothesis testing 1

The FMRI inferential path Experiment 2

The FMRI inferential path Experiment Physiology 2

The FMRI inferential path Experiment Physiology MR Physics 2

The FMRI inferential path Experiment Physiology Analysis MR Physics 2

The FMRI inferential path Experiment Interpretation of final results Physiology Analysis MR Physics 2

Variability in FMRI Experiment Interpretation of final results suboptimal event timing, non-efficient design, etc Physiology Analysis MR Physics 3

Variability in FMRI Experiment Interpretation of final results suboptimal event timing, non-efficient design, etc Physiology secondary activation, ill-defined baseline, resting-fluctuations etc Analysis MR Physics 3

Variability in FMRI Experiment Interpretation suboptimal event timing, Physiology of final results non-efficient design, etc secondary activation, ill-defined baseline, resting-fluctuations etc filteringanalysis & sampling artefacts, design misspecification, stats & thresholding issues etc MR Physics MR noise, field inhomogeneity, MR artefacts etc 3

Model-based (GLM) analysis = β i + model each measured time-series as a linear combination of signal and noise If the design matrix does not capture every signal, we typically get wrong inferences! 4

Data Analysis Confirmatory How well does my model fit to the data? Problem Model Data Analysis Results Results depend on the model 5

Data Analysis Confirmatory How well does my model fit to the data? Exploratory Is there anything interesting in the data? Problem Data Problem Data Model Analysis Analysis Model Results Results depend on the model Results can give unexpected results 5

EDA techniques try to explain / represent the data by calculating quantities that summarise the data by extracting underlying hidden features that are interesting differ in what is considered interesting are localised in time and/or space (Clustering) explain observed data variance (PCA, FDA, FA) are maximally independent (ICA) typically are multivariate and linear 6

EDA techniques for FMRI are mostly multivariate often provide a multivariate linear decomposition: space # maps space time Scan #k FMRI data = time # maps spatial maps Data is represented as a 2D matrix and decomposed into factor matrices (or modes) 7

PCA for FMRI 8

PCA for FMRI 1 assemble all (demeaned) data as a matrix time space Scan #k FMRI data 8

PCA for FMRI 1 assemble all (demeaned) data as a matrix 2 calculate the data covariance matrix 8

PCA for FMRI # maps 1 assemble all (demeaned) data as a matrix 2 calculate the data time covariance matrix 3 calculate SVD 8

PCA for FMRI 1 assemble all (demeaned) data as a matrix 2 calculate the data covariance matrix 3 calculate SVD 4 project data onto the Eigenvectors 8

PCA decompositions: Implications 9

PCA decompositions: Implications spatial maps and time courses are orthogonal (uncorrelated) 9

PCA vs ICA 5 4 3 2 1 0 1 2 3 4 5 5 4 3 2 1 0 1 2 3 4 5 Gaussian data 10

PCA vs ICA PCA finds projections of maximum amount of variance in Gaussian data (uses 2nd order statistics only) 5 4 3 2 1 0 1 2 3 4 5 5 4 3 2 1 0 1 2 3 4 5 Gaussian data 10

PCA vs ICA PCA finds projections of maximum amount of variance in Gaussian data (uses 2nd order statistics only) 5 4 3 2 1 0 1 2 3 4 5 5 4 3 2 1 0 1 2 3 4 5 non-gaussian data 10

PCA vs ICA PCA finds projections of maximum amount of variance in Gaussian data (uses 2nd order statistics only) 5 4 3 2 1 Independent Component Analysis (ICA) finds projections of maximal independence in non- Gaussian data (using higher-order statistics) 0 1 2 3 4 5 5 4 3 2 1 0 1 2 3 4 5 non-gaussian data 10

PCA decompositions: Implications spatial maps and time courses are orthogonal (uncorrelated) 11

PCA decompositions: Implications spatial maps and time courses are orthogonal (uncorrelated) uncorrelated un-related 11

Correlation vs independece 12

Correlation vs independece de-correlated signals can still be dependent y = sin(z) 10 05 00 05 10 Plot x vs y r=0000 05 00 05 xy 10 05 00 05 10 0 100 200 300 400 500 x = cos(z) 2 2 12

Correlation vs independece de-correlated signals can still be dependent y = sin(z) 10 05 00 05 10 Plot x vs y r=0000 05 00 05 xy 10 05 00 05 10 0 100 200 300 400 500 higher-order statistics (beyond mean and variance) can reveal these dependencies Stone et al 2002 y 2 = sin(z) 2 00 05 10 15 x = cos(z) x = cos(z) high order correlations r= 0118 00 01 02 03 04 2 2 x 2 y 2 00 05 10 15 0 100 200 300 400 500 x 2 = cos(z) 2 12

PCA decompositions: Implications 13

PCA decompositions: Implications spatial maps and time courses are orthogonal (uncorrelated) uncorrelated un-related only looks at 2nd order statistics: problem with non-gaussian sources PCA is rotationally invariant 13

Rotational invariance 14

Rotational invariance consider de-correlated signals Y, ie R Y = I 14

Rotational invariance consider de-correlated signals Y, ie R Y = I consider projecting the data onto a basis R QY QY Y T Q T = QQ T Q 14

Rotational invariance consider de-correlated signals Y, ie R Y = I consider projecting the data onto a basis R QY QY Y T Q T = QQ T Q then, R QY = R Y = I for any Q st QQ T = I 14

Rotational invariance consider de-correlated signals Y, ie R Y = I consider projecting the data onto a basis R QY QY Y T Q T = QQ T Q then, R QY = R Y = I for any Q st QQ T = I QQ T = I : Q is a rotation matrix 14

The Geometry of PCA and ICA 08 08 s 2 s 2 PC 1 04 04 PC 2 s 1 0 s 1 0 04 04 08 08 08 04 0 04 08 08 04 0 04 08 2 independent, uniformly distributed sources linear mixtures of sources 15

The Geometry of PCA and ICA 08 s 1 IC 1 08 IC 2 s 2 04 04 0 0 s 1 IC 2 IC 1 04 s 2 04 08 08 08 04 0 04 08 PCA solution 08 04 0 04 08 ICA solution 15

spatial ICA for FMRI McKeown et al (1998): data is decomposed into a set of spatially independent component maps and a set of time-course space # ICs space time Scan #k FMRI data = time # ICs Components impose spatial independence McKeown et al HBM 1998 16

PCA pre-processing for ICA whitening Data uncorrelated signals 17

PCA pre-processing for ICA whitening Data uncorrelated signals 2-stage approach to ICA: PCA to obtain whitened (uncorrelated) signals using 2nd order statistics 17

PCA pre-processing for ICA whitening rotation Data uncorrelated signals 2-stage approach to ICA: independent signals PCA to obtain whitened (uncorrelated) signals using 2nd order statistics estimate rotation which minimises higher-order dependencies 17

ICA - the overfitting problem fitting a noise-free model to noisy observations: no control over signal vs noise (noninterpretable results) statistical significance testing not possible 18

ICA - the overfitting problem fitting a noise-free model to noisy observations: no control over signal vs noise (noninterpretable results) statistical significance testing not possible GLM analysis standard ICA (unconstrained) 18

Probabilistic ICA model statistical latent variables model: we observe linear mixtures of hidden sources y i = Xβ i + ɛ i for all i Y = XB + E we can estimate the model order from the Eigenspectrum of the data covariance matrix need to account for differences in voxel-wise variance (can use a pre-whitening approach (eg al, NeuroImage 14(6) 2001) Woolrich et 19

Variance-normalisation we might choose to ignore temporal autocorrelation in the EPI time-series but always need to normalise by the voxel-wise variance Voxel wise standard deviation 55 5 45 this amounts to modelling the spatial covariance matrix as a diagonal matrix V 1/2 = diag(σ 1,, σ N ) Estimated voxel-wise std deviation (logscale) from FMRI data obtained under rest condition 4 35 20

Model Order Selection The sample covariance matrix has a Wishart distribution and we can calculate the empirical distribution function for the eigenvalues Everson & Roberts, IEEE Trans Sig Proc 48(7), 2000 Empirical distribution function observed Eigenspectrum 21

Model Order Selection (PPCA) use a probabilistic PCA model and calculate (approximate) the Bayesian evidence for the model order Minka, TR 514 MIT Media Lab 2000 Laplace approximation BIC AIC 22

Conditioning the Eigenspectrum (a) (b) x2 P C 2 (c) 10 sources introduced into white noise (AR 0) 23

ICA estimation need to find an unmixing matrix such that the dependency between estimated sources is minimised need (i) a contrast (objective/cost) function which measures statistical independence and (ii) an optimisation technique: kurtosis or cumulants & gradient descent (Jade) maximum entropy & gradient descent (Infomax) neg-entropy & fixed point iteration (FastICA) 24

Non-Gaussianity is interesting 25

Non-Gaussianity is interesting mixing 25

ICA estimation 26

ICA estimation Random mixing results in more Gaussian shapes PDFs (Central Limit Theorem) 26

ICA estimation Random mixing results in more Gaussian shapes PDFs (Central Limit Theorem) conversely: if an unmixing matrix produces less Gaussian shaped PDFs this is unlikely to be a random result measure non-gaussianity can use neg-entropy J (s) = H(s gauss ) H(s) 26

Probabilistic ICA (I) Data pre-processing: voxel wise demeaning and variance normalization estimate intrinsic dimensionality (PCA) unmix source signals using negentropy as a measure of non- Gaussianity (Hyvärinen, 1999) 27

ICA and spatially correlated sources Spatial correlation: ρ(s 1, s 2 ) = s t 1s 2 N Var(s 1 ) Var(s 2 ) In the presence of noise: ρ(s 1 + η 1, s 2 + η 2 ) = s t 1s 2 N Var(s 1 ) + σ 2 Var(s 2 ) + σ 2 28

ICA and spatially correlated sources Example: (a) 2 sources correlated (ρ=05); (b) in the presence of noise, ρ<01 and (c) de-correlating the two maps preserves the structure; (d) thresholded maps represent the sources well (a) (b) (c) (d) 28

Probabilistic ICA (II) form voxel-wise Z-statistics using the estimated standard deviation of the noise x 2 PC 2 C 1 IC 1 ˆɛ i 29

Thresholding Null-hypothesis testing is invalid: right tail IC-map thresholding based on Z-transforming across the spatial domain gives wrong false-positives rate! Under the model: 1 2 3 4 5 6 left tail 1 15 2 25 B = W Y = W (XB + E) 2 1 0 1 2 3 4 5 6 7 ie the estimated spatial maps contain a linear projection of the noise term the distribution of the estimated spatial maps is a mixture distribution 30

Alternative Hypothesis Test use Gaussian/Gamma mixture model fitted to the histogram of intensity values (using EM) 31

Full PICA model MELODIC 32

Full PICA model MELODIC Beckmann and Smith IEEE TMI 2004 32

Probabilistic ICA designed to address the overfitting problem : GLM analysis standard ICA (unconstrained) 33

Probabilistic ICA designed to address the overfitting problem : GLM analysis probabilistic ICA (constrained) 33

Probabilistic ICA designed to address the overfitting problem : GLM analysis probabilistic ICA (constrained) avoids generation of spurious results 33

Probabilistic ICA designed to address the overfitting problem : GLM analysis probabilistic ICA (constrained) avoids generation of spurious results high spatial sensitivity and specificity 33

Simulated data Receiver - Operator Characteristics: PICA vs GLM at different `activation` levels and different thresholds 34

Applications EDA techniques can be useful to investigate the BOLD response estimate artefacts in the data find areas of activation which respond in a non-standard way analyse data for which no model of the BOLD response is available 35

Investigate BOLD response estimated signal time course standard hrf model Wise & Tracey 36

Artefact detection FMRI data contain a variety of source processes Artefactual sources typically have unknown spatial and temporal extent and cannot easily be modeled accurately Exploratory techniques do not require a priori knowledge of time-courses and spatial maps 38

slice drop-outs 39

gradient instability 40

EPI ghost 41

EPI ghost 42

high-frequency noise 43

head motion 44

field inhomogeneity 45

eye-related artefacts 46

eye-related artefacts Wrap around 46

spin-history effects 47

Implication for the GLM structured noise appears: in the GLM residuals and inflate variance estimates (more false negatives) in the parameter estimates (more false positives and/or false negatives) In either case lead to suboptimal estimates and wrong inference! 48

Structured noise and GLM Z-stats bias Correlations of the noise time courses with typical FMRI regressors can cause a shift in the histogram of the Z- statistics Thresholded maps will have wrong false-positive rate 49

Denoising FMRI before denoising 600 500 PE raw 350 300 Z raw Example: left vs right 500 400 400 300 300 200 200 100 100 300 250 250 200 200 150 150 100 100 50 hand finger tapping Johansen-Berg et al PNAS 2002 0!1000!500 0 500 1000 600 500 0!10!5 0 5 10 LEFT - RIGHT LEFT contrast PE clean Z clean 500 400 300 250 250 400 200 300 200 300 150 200 150 200 100 100 100 100 50 0!1000!500 0 500 1000 0!10!5 0 5 10 350 300 after denoising 50

Denoising FMRI before denoising 600 PE raw 350 Z raw Example: left vs right 500 400 300 200 100 300 250 200 150 100 50 hand finger tapping Johansen-Berg et al PNAS 2002 0!1000!500 0 500 1000 600 500 400 300 200 100 0!10!5 0 5 10 LEFT - RIGHT LEFT contrast PE clean Z clean 0!1000!500 0 500 1000 350 300 250 200 150 100 50 0!10!5 0 5 10 after denoising 50

Denoising FMRI before denoising Example: left vs right hand finger tapping LEFT - RIGHT contrast Johansen-Berg et al PNAS 2002 after denoising 50

Apparent variability McGonigle et al: 33 Sessions under motor paradigm de-noising data by regressing out noise: reduced apparent session variability 51

Example: stim correl motion GLM est z-rotation FE-map PICA est y-translation standard ICA 53

! 7!fIU!8*)#8'13!3!36214!2%!%82,B1!2$1!,%41'!%;!2$1!NKf!9$'1!8*%21 &%+*)23#+&)%! %41'!>(!(88*%8*(21!,%41'=%*41*!31'1#2%)!?aA:!! Clinical example!!!!!!!!!!!!!!!!!!!!!!!!!!!! #%)341*(5'1! (#%632#! )%31! 57! 32118! -*(41)2! 86'313! (2! 6)*132*#214! -<! 6)3$1'414! "&+=3%6)43! )1>2(5'7! (#2>(21! 2$1! (642%*7! 37321,! ()4!! %;! #%)#%,2()2! %*! 36##133>1! 32,6'(2%)3:! +)321(4! %;! >()3$)-! 2$1!! -:<!@A<!%6*!-%('!9(3!2%!62'B1!2!;%*!2$1!;6)#2%)('!)16*%,(-)-!%;!(642%):! 57! 12$1*! )2*%46#)-! (51**()2! -*(41)2! 392#$13!?C=DA<! (,8'26413! %*! 7!3(,8')-!8$73%'%-#(''7!41'(714!$1,%47)(,#!FGHI=*138%)313!2%!2$1! )2'7! 41'(714! 562! %2$1*931! 6),%4;14! -*(41)2! )%31!?MA:! N1*1<! 91!!(3!91''!(3!4(2(=4*>1)!4121#2%)!%;!,%46'(2%)3!)!2$1!FGHI!3-)('!%;!2$1! *1(4=%623!/KPG30!;*%,!2$1!-*(41)2=2*()!%;!()!"&+!86'31!31Q61)#1:! f+wzk"!d\( K-$2!>13256'(*!3#$9())%,(!P!8%32%81*(2>1!41(;)133! use EPI readout gradient noise to evoke auditory /T@=9!V"!8%32!W(4%')6,^!*1-6'(*B14!J"HGI+Xh\!8s`:E`0! responses in patients before cochlear implant surgery 1-+,)2"! >!1$+-*&$4!!@=_!*1(4=%623! ()3! 91*1!*1(4! +A! 9$#$! 9(3! %)3!/D@=aM4F0! 91''! (3!,6;;'1! '1!,%2%)3:! 9]@_M`NBP&O<! %*,('!$1(*)-! (8]CEj0!91*1! Zf+='%#('B1*3:! #())1*:! 2$*11! 4;;1*1)2! $9())%,(<! (!,! %;! (! $1*813= 33214! %;! 3'#1=!/@:E!2,13!2$1! 3!(!32,6'63!%;! da0:!t$1!3#()3! (!321(47=32(21:! ;! f"ut=413-)! 41*>(2>1:! f%*! VH0!9(3!6314:!!! JGI+f+"I!"&+!WKUI+"ST=TKU+S!n+TN!K"UI=GZT!GJ+VV+GSV!o!"R&"XT"I!UZI+TGKm!FGHI!V+WSUH!JGIZHUT+GSV\!!!"#$%&'()*!"#$%&'"()%+,!"##-(**+&,-&-"$*/0(1*)2,(1"(0+*345 <<<%#=>?%@ABCDEF=DAG%<<<%!! +J&GV+SW!&K+GK!T"J&GKUH!XGSTKU+STV!TG!IUTU=IK+c"S!&KGFUF+H+VT+X!+SI"&"SI"ST!XGJ&GS"ST!USUHmV+V! /pk"wzhuk+q"il!j"hgi+x0\! n2$)! 2$1! &+XU! ;*(,19%*k<! 132,(2%)! %;!,(O,(''7! )%)=W(633()! 3%6*#13! 3! 5(314! %)! ()! "-1)>('61! 41#%,8%32%)!%;!2$1!4(2(=#%>(*()#1=,(2*O!/KR0:!f%*!fJK+!4(2(!)!-1)1*('!()4!;%*!2$3!32647!)!8(*2#6'(*<!2$1! 3-)('!%;!)21*132! 3!(336,14!2%!>(*7!*(2$1*!3'%9'7!%>1*!2,1:!T$63<!$-$=;*1Q61)#13! 71'4!*(2$1*!)%)=8(635'1! 38(2('!,(83!()4!2,1=#%6*313<!()4!2$1*!132,(2%)!3!4381)3(5'1:!T%!;(>%6*!p3,%%2$L!2,1=#%6*313<!J"HGI+X! 9(3! *1-6'(*B14! 57! )#%*8%*(2)-! 8*)#8'13! %;! ;6)#2%)('! 4(2(! ()('733! /fiu<!?_a0\! +41)2;#(2%)! %;! )2('! "-1)>1#2%*3!9(3!21,8%*(''7!#%)32*()2!57!(!362(5'7!#$%31)!312!%;!*1-6'(*'7!38(#14!/TKPE0!#65#!F=38')1!5(33! ;6)#2%)3:! T$1*157<! *136'2)-! +XU! 2,1=#%6*313! =! 132,(214! (3! ')1(*! #%,5)(2%)3! %;! 2$1! "-1)>1#2%*3! =! 91*1! *132*#214!2%!#%)2()!8*,(*'7!'%91*!;*1Q61)#13:!f)('!&+XU!,(83!-1)1*(214!57!p%*4)(*7L!(3!91''!(3!*1-6'(*B14! J"HGI+X!91*1!2$*13$%'414!63)-!()!('21*)(2>1!$78%2$133!2132!5(314!%)!W(633()PW(,,(!,O26*1!,%41'3!?aA:!! Bartsch et al HBM 2004 54

PICA on resting data perform ICA on null data and compare spatial maps between subjects/scans ICA maps depict spatially localised and temporally coherent signal changes Example: ICA maps - 1 subject at 3 different sessions 56

Spatial characteristics (a) x=17 y=-73 z=-12 (b) x=-13 y=-61 z=6 (a) x=17 y=-73 z=-12 (b) x=-13 y=-61 z=6 R (c) L Medial visual cortex (d) z=15 (d) L y=-17 x=3 x=-4 y=-29 (e) z=33 y=-29 R (g) x=45 R (f) y=6 L y=6 R z=47 z=27 Sensori-motor system x=5 L y=-42 L R z=33 z=51 y=-21 x=1 x=5 (f) z=51 L R Auditory system x=-4 Lateral Visual Cortex L R L y=-21 x=1 L R (e) z=15 y=-17 x=3 R (c) R (h) x=-45 z=27 L y=-42 z=47 57

(c) x=-4 z=33 y=-42 x=45 (f) (h) y=6 L y=-42 R (h) z=27 Executive control x=-45 L z=47 z=27 L R z=47 y=-42 y=6 x=5 L R (g) x=5 R Visuospatial system x=45 (f) z=51 L R z=33 y=-29 L y=-21 x=1 L R (g) (d) L y=-29 x=-4 R z=15 y=-17 x=3 R (e) z=51 y=-21 x=1 L R (e) (d) Spatial characteristics R (c) z=15 y=-17 x=3 x=-45 z=47 L y=-42 z=47 R L R L R L R L Visual Stream 58

ICA Group analysis Extend single ICA to higher dimensions space # maps space time Scan #k FMRI data = time # maps spatial maps 59

ICA Group analysis subject subject space # maps space time Scan #k FMRI data = time # maps spatial maps Factors characterise processes across space, time and sessions/subjects x ijk = r a ir b jr c kr + ɛ ijk Beckmann and Smith NI 2005 59

PARAFAC 60

PARAFAC as a symmetric least-square problem this is known as PARAFAC (Parallel Factor Analysis; Harshman 1970) can be solved using Alternating Least Squares (ALS), ie by iterating least-squares solutions for X i = Bdiag(a i )C t + E i X j = Cdiag(b j )A t + E j X k = Adiag(c k )B t + E k i = 1,, I j = 1,, J k = 1,, K 60

Tensor-PICA 61

Tensor-PICA rewrite: X IK J = (C A)B t + E 61

Tensor-PICA rewrite: X IK J = (C A)B t + E can be treated as a 2-stage estimation problem: 61

Tensor-PICA rewrite: X IK J = (C A)B t + E can be treated as a 2-stage estimation problem: 1 PICA estimation of B from by estimating M c X IK J as the mixing matrix 61

Tensor-PICA rewrite: X IK J = (C A)B t + E can be treated as a 2-stage estimation problem: 1 PICA estimation of B from by estimating M c X IK J as the mixing matrix M r 2 rank-1 Eigen-decomposition of each column c, reshaped into a I K matrix, in order to find underlying factors such that M c = (C A) 61

Tensor-ICA Higher-level GLM designs are even more simplistic (eg assumption of constant within group activation strength) 10 sessions under motor paradigm (right index finger tapping) Group-level GLM results (mixed-effects) McGonigle et al NI 2000 62

tensor-pica map tensor-pica: example associated time course 0 20 40 60 80 5 normalised response over sessions 4 3 2 1 2 4 6 8 10 63

tensor-pica: example estimated de-activation associated time course 0 20 40 60 80 4 normalised response over sessions 3 2 1 0 2 4 6 8 10 64

tensor-pica: example residual stimulus correlated motion associated time course 0 20 40 60 80 3 normalised response over sessions tensor-pica map 2 1 0 1 2 2 4 6 8 10 Z-stats map (session 9) Group-GLM (ME) Z-stats map 65