Joint Brain Connectivity Estimation from Diffusion and Functional MRI Data

Transcription

1 Joint Brain Connectivity Estimation from Diffusion and Functional MRI Data Shu-Hsien Chu, Christophe Lenglet and Keshab K. Parhi University of Minnesota, Minneapolis, MN, USA ABSTRACT Estimating brain wiring patterns is critical to better understand the brain organization and function. Anatomical brain connectivity models axonal pathways, while the functional brain connectivity characterizes the statistical dependencies and correlation between the activities of various brain regions. The synchronization of brain activity can be inferred through the variation of blood-oxygen-level dependent (BOLD) signal from functional MRI (fmri) and the neural connections can be estimated using tractography from diffusion MRI (dmri). Functional connections between brain regions are supported by anatomical connections, and the synchronization of brain activities arises through sharing of information in the form of electro-chemical signals on axon pathways. Jointly modeling fmri and dmri data may improve the accuracy in constructing anatomical connectivity as well as functional connectivity. Such approach may lead to novel multimodal biomarkers potentially able to better capture functional and anatomical connectivity variations. We present a novel brain network model which jointly models the dmri and fmri data to improve the anatomical connectivity estimation and extract the anatomical subnetworks associated to specific functional modes by constraining the anatomical connections as structural supports to the functional connections. The key idea is similar to a multi-commodity flow optimization problem that minimizes the cost or maximizes the efficiency for flow configuration and simultaneously fulfills the supply-demand constraint for each commodity. In the proposed network, the nodes represent the grey matter (GM) regions providing brain functionality, and the links represent for white matter (WM) fiber bundles connecting those regions and delivering information. The commodities can be thought of as the information corresponding to brain activity patterns as obtained for instance by independent component analysis (ICA) of fmri data. The concept of information flow is introduced and used to model the propagation of information between GM areas through WM fiber bundles. The link capacity, i.e., ability to transfer information, is characterized by the relative strength of fiber bundles, e.g., fiber count gathered from the tractography of dmri data. The node information demand is considered to be proportional to the correlation between neural activity at various cortical areas involved in a particular functional mode (e.g. visual, motor, etc.). These two properties lead to the link capacity and node demand constraints in the proposed model. Moreover, the information flow of a link cannot exceed the demand from either end node. This is captured by the feasibility constraints. Two different cost functions are considered in the optimization formulation in this paper. The first cost function, the reciprocal of fiber strength represents the unit cost for information passing through the link. In the second cost function, a min-max (minimizing the maximal link load) approach is used to balance the usage of each link. Optimizing the first cost function selects the pathway with strongest fiber strength for information propagation. In the second case, the optimization procedure finds all the possible propagation pathways and allocates the flow proportionally to their strength. Additionally, a penalty term is incorporated with both the cost functions to capture the possible missing and weak anatomical connections. With this set Work partly supported by NIH grants P41 EB015894, P30 NS076408, R01 EB Data were provided in part by the Human Connectome Project, WU-Minn Consortium (Principal Investigators: David Van Essen and Kamil Ugurbil; 1U54MH091657) funded by the 16 NIH Institutes and Centers that support the NIH Blueprint for Neuroscience Research; and by the McDonnell Center for Systems Neuroscience at Washington University. Further author information: (Send correspondence to S.C.) S.C: chuxx214@umn.edu, Telephone: C.L.: clenglet@umn.edu, Telephone: K.K.P.: parhi@umn.edu, Telephone:

2 of constraints and the proposed cost functions, solving the network optimization problem recovers missing and weak anatomical connections supported by the functional information and provides the functional-associated anatomical subnetworks. Feasibility is demonstrated using realistic diffusion and functional MRI phantom data. It is shown that the proposed model recovers the maximum number of true connections, with fewest number of false connections when compared with the connectivity derived from a joint probabilistic model using the expectation-maximization (EM) algorithm presented in a prior work. 1 We also apply the proposed method to data provided by the Human Connectome Project (HCP). Keywords: Brain connectivity, functional MRI, diffusion MRI, network, tractography, information flow 1. INTRODUCTION Human brains are known be a large-scale neuron network which is complex, hierarchical and can be decomposed accordingly at different scales. Brain connectivity refers to the pattern of anatomical links (anatomical connectivity), statistical dependencies (functional connectivity) and causal interactions (effective connectivity) which can be determined by observing structural links such as fiber pathways, by representing statistical relationships measured as cross-correlations and coherence, and causal relationship, respectively, between parcels of the brain. However, the relationship between connectivity pattern and brain functionality is not well understood. Thus, understanding the cognitive mechanisms and functioning of the brain from connectivity patterns is of significant interest. 2 5 In a brain neuron system, signal is processed in the grey matter (GM) area and transmitted between GM regions through axons located mainly in white matter region (WM). Functional MRI (fmri) measures the blood oxygen level dependent (BOLD) signal that describes the contrast of oxygen consumption over time and is correlated to neural activities. 6 fmri data is often used to analyze and correlate the neuronal activity between regions, and to construct the functional connectivity pattern. Several methods have been widely used to generate and analyze the functional connectivity: 7 seed-based correlation, 8 independent component analysis (ICA) 9 and graph-theoretic network analysis. 10 Moreover, the WM microstructure can be captured non-invasively through diffusion MRI (dmri) which relies on the water molecule anisotropic diffusion along axons. Diffusion MRI has been used to estimate brain connectivity pattern in many clinical applications 11 such as Alzheimers disease, 12 neuroscience research, and the construction of the human connectome. 13 Furthermore, many neurological diseases 12, 14 result from abnormal changes of brain anatomical connectivity. However, dmri and associated analysis methods such as tractography 15 suffer from certain limitations which restrict the accuracy with which fiber pathways can be estimated, thereby possibly leading to missed connections. In the complex area, a major part of connections is missed, or weakened. Despite significant advances in multimodal imaging techniques and analysis approaches, unimodal studies are still the most common way to investigate brain changes or group differences. It has been found that the functional connectivity is significantly shaped by the anatomical structure More specifically, brain regions with high anatomical connectivity usually have high functional connectivity as well, but the converse is not necessarily true. Extracting information jointly from each both dmri and fmri may lead to more accurate 18, 19 connectivity. Recently, a joint connectivity approach under a probabilistic framework using expectationmaximization algorithm has been proposed for a group study 1 and a functional connectivity pattern estimation based on sparse Gausian Graphical Model penalized by structural connectivity has been presented. 20 In this paper, we propose a novel network flow model to improve the estimation of brain connectivity pattern based on joint analysis of dmri and fmri. In the model, the nodes represent brain functional regions, the links represent connections between regions, and the flow models the information propagation between functional regions through structural pathways. The capability for transmitting information on each link, i.e., link capacity constraint, is defined as the fiber count obtained from dmri data and tractography; the demand of information on each regions, i.e., demand constraint for each region, for each functional mode is computed from the fmri ICA result. In the ICA framework, the fmri data corresponding to a brain function is decomposed into many spatio-temporal components, i.e., functional modes, where modes relate to specific brain functions and the spatial map provides the spatial distribution for each mode associated with the temporal activities. Through the spatial

3 distribution, the amount of information needed to trigger the neural activities is estimated and considered as a demand. In the network, the active regions within a functional mode share information. However, the amount of shared information cannot exceed the demand from either end node. It is modeled by feasibility constraints. A solution to the network flow problem based on three sets of constraints leads to structural connectivity patterns which are supporting the functional network. The proposed network flow model is validated on a diffusion and functional MRI phantom data with various structural complexities, i.e., fiber crossing, to show the improvement. This paper is organized as follows. Section 2 describes the data and processing pipeline used in this paper. Section 3 presents the proposed network flow model and network analysis techniques and Section 4 presents the connectivity results derived from the proposed model.comparison of the brain connectivity with prior work is also presented in Section DATA ACQUISITION AND PROCESSING A realistic phantom, Fiber Cup, 21, 22 dmri data set with known fiber layout and synthetic fmri data that are perfectly aligned with the true connectivity pattern is first used to verify the model, demonstrate the improvement from combining fmri data and compare the performance with existing methods. For brain connectivity pattern study, two MRI data sets are used in this paper: one from the phantom and the other from the Human Connectom Project (HCP). 2.1 Phantom Data Set Figure 1. The phantom 21, 22 data set: (a) Ground truth fiber connections used in dmri scanning; (b) b0 image and the parcellation of 16 regions, P1, P2,..., and P16, centered at the seed voxels given in the 2009 MICCAI Fiber Cup contest. The color of the regions represent the true connectivity pattern. In other words, the regions sharing the same color are anatomically connected to each other. (c) ground truth anatomical connections in a network formation with regions as nodes and connections as edges (Generated using BrainNet Viewer 23 ). The phantom 21, 22 used in the FiberCup tractography challenge in the 12th International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI) in London in 2009 and made publicly available on the webpage ( of LNAO laboratory is used for validating the result and comparing performance. Ground truth for 16 fibers shown in Figure 1 and seed points which are centered at each region in Figure 1 (b) are available with the data. The phantom comprises seven distinct bundles, and contains 3 crossings, 1 kissing, and 3 bundles that split. The various crossing, splitting and kissing fiber configurations simulate a coronal section of the human brain and can be used to evaluate the quality of connectivity estimation. The network presentation of true anatomical connections in Figure 1 (c) is generated using BrainNet Viewer 23 by taking the regions as nodes and connections as edges. The network contains 5 separated sub-networks which coincide with the true connectivity pattern Diffusion-weighted MRI acquisitions Diffusion-weighted data of the phantom 21, 22 are 3mm isotropic with 64 directions uniformly distributed over the sphere. The data with b = 2000 were used in this paper.

4 2.1.2 Functional MRI simulation The fmri BOLD signal on each voxel were simulated using a modification of Generate synthetic fmri data function published on A design matrix shown in Figure 2a (upper right panel) consisting in 5 randomly generated and minimally correlated binary stimulating sequences was first generated. The sequences were assigned to the regions based on the rule that regions in each connected pattern share the same stimulating sequence. Then, the BOLD signal in Figure 2a (bottom panel) is obtained from the convolution of the design matrix with hemodynamic response function illustrated in Figure 2a (upper left panel). The BOLD signals were spread out spatially with exponential decade and additive white Gaussian noise was added to the whole fmri data set with average SNR equal to The imaging parameters for the simulated data are T R = 720ms and 1200 time points. In order to verify the largest improvement we may gain from the joint analysis, the fmri data is designed to have a fixed functional connectivity which coincides with the true anatomical connectivity. The validations are shown in Figure 2 (b) and (c). Figure 2. The fmri time courses were simulated according to the 5 true anatomical connectivity patterns. They were assigned with a stimulating pattern encoded in the design matrix, (a) upper right panel. The BOLD signal, (a) bottom panel, for each region is then given by the convolution of hemodynamic response function, (a) upper left panel, and the design matrix. Figure (b) shows simulation result at t = 9, 34, 44, 76. It demonstrates that the fmri signals are synchronized for the connected regions. It can also be verified by checking the correlation, shown in (c), of signals between each region where thicker the edge, stronger the correlation. Nodes are strongly correlated only when they are anatomically connected, i.e., in the same color, otherwise they are minimally correlated. The numbers on the links represent the correlation coefficients. 2.2 Human Connectome Project Data Set The HCP24, 25 data is publicly available on its website ( with multiple imaging modalities including diffusion imaging (dmri), resting-state fmri (R-fMRI), task-evoked fmri (TfMRI), T1- and T2-weighted MRI which can be used to characterize brain connectivity and function and their variability in healthy adults. T1-weighted MRI, diffusion-weighted MRI, and resting-state functional MRI data are used for constructing brain networks in our study. Data was acquired on a 3T Siemens scanner using the following parameters for fmri: T R = 720ms, T E = 33.1ms, F A = 52, 2 2 2mm voxels, F OV = mm, and 72 oblique axial slices alternated between phase encoding in a right to left direction in one run and phase encoding in a left to right direction in the other run Each functional run lasted min (1200 time points). Diffusion data sets were acquired with mm voxels, F OV = 210mm, 111 slices, T E = 89.50ms, T R = 5520ms, b-value= 1000, 2000, 3000s/mm2, 90 directions per b-values and 18 non-diffusion-weighted volumes. 2.3 Data Preprocessing The anatomical connectivity estimated from dmri data is referred to as the link capacity Dl and the region information requirement, Rim, is calculated from the spatial maps in ICA of the fmri data and represents how

5 significant the activity is in that region corresponding to the m th mode. Both link capacity D l and region information requirement Ri m are the inputs of the proposed model. The details regarding the model will be provided in the next section. Here, we first introduce the process for getting these input data Diffusion Data were preprocessed using the default HCP preprocessing pipeline (v.3.4), 30 which includes correction for susceptibility, motion and eddy current distortions. In the preprocessing step, the DWI data were corrected for eddy current distortions and local fiber orientations were estimated using bedpostx. 31 Probabilistic fiber tracking with 5000 samples in each seeded voxel and distance correction is performed to generate the brain tractography. The number of tracks connecting two regions, D l, was normalized by the number of samples and seeded region volume and taken as the fiber strength acting as the capacity of each link in the network model rfmri preprocessing We utilized the fmri preprocessed data released by the HCP. 30 Before processing, the first 21 time stamps of fmri data were discarded. Then, high pass filtering with a cutoff frequency 1/60 Hz, motion correction, 4mm spatial smoothing, and FILM pre-whitening were applied using FSL. 31 Spatial Independent Component Analysis (spatial ICA) The spatial independent component analysis (ICA), using FSL melodic, 32 is performed to decompose the brain fmri into spatial independent components which are composed from correlation maps and temporal series. The decomposition is illustrated in Figure 3. The observed fmri data is arranged in a time by voxel matrix Y which represents the combination of functional components. Each component is associated with a time series which is a column of T and a spatial map that is described by a row of M. Here, the maps are spatially independent and do not overlap with others. From a matrix multiplication perspective, the absolute component value in a spatial map can be considered as the weight factor corresponding to a time course for each voxel. Therefore, these weights in the i th region are used to generate Ri m as the amount of information associated to mode m. The meaningful components were selected automatically 33 based on the skewness of the histogram of the spatial map and the spectrum analysis of the corresponding temporal signal from the noise components. Furthermore, the spatial map outputs were refined by applying the existing parcellation and classifying each voxel into active or inactive status using K-means. For each selected mode, the regional mean correlation level was calculated from the absolute value from the voxels in the region. The levels are further used for identifying the activation status of the region by K-means. 34 The functional demand Ri m in the model was set to the mean correlation level for active regions, otherwise the Ri m was set to zero. Figure 3. Functional MRI data organized in a time space matrix can be decomposed into the multiplication of two matrices. One contains the mean temporal signals and the other matrix contains the spatial independent maps which illustrate the correlation between the BOLD signals in each voxel and the temporal signal. 3. BRAIN NETWORK MODEL The brain can be viewed as a multi-commodity supporting network with replicable goods. The analogies are made between: 1) GM region and commodity factories, 2) WM fiber and delivering pathways, and 3) commodity and information. The brain function is accomplished by the interconnected neurons and as a result of information

6 processing and exchange. The neuronal cell bodies are majorly responsible for information processing and mostly located in the grey matter (GM). The parcels in GM are the nodes in the brain network, similar to the factories in a commodity network. Information is the commodity and the axon fiber bundles, the links in a network, in the white matter (WM) are the exchanging pathways which are similar to the delivering pathway in a commodity network. Combining functional and structural MRI improves the estimation of anatomical connectivity by maintaining the supply-demand relationship. Assuming that 1) a neuron reacts to the received signal, 2) a neuron doesn t produce signal with no excitation, and 3) the synchronization is positively correlated to the amount of received signal and the capability to generate new signal, the underestimated fibers including missing connections can be recovered from retrieved fmri data. Symbol fl m P l Ri m D l l = (i, j) N(i) γ β ρ Table 1. Definition for variables, parameters and symbols in the information network model Description Information flow variables for mode m on link l Comprehensive anatomical network variable that captures the missing connections l in D l network The region information demand regarding to m th mode and i t h region The capacity of link l for delivering information The connection l with associated end regions i and j The collection of links associated with region i Unit conversion between information and capacity Information sharing rate Penalty parameter for P l In the proposed network flow model, the nodes represent brain functional regions and the links between the nodes represent potential fiber connections. The information flow is used to describe the information propagation. The flow of information through the links is influenced by the strength of the anatomical connectivity measured by dmri: the stronger the connection, the larger the flow can be. The demand associated with a node is modeled separately for each functional mode of the fmri. The notations used in this paper to describe the network flow formulation are summarized in Table 1. Let fl m represent the information flow through link l associated with functional mode m, the capacity of a link be denoted as D l, and the information demand associated with node i corresponding to functional mode be denoted by Ri m. The modes corresponding to the independent temporalspatial components of the fmri are obtained by independent component analysis (ICA). The proposed network model for brain connectivity is described by equations (1)-(5). s.t. Input: D l, Ri m, ρ, γ ; Output: f l m, P l [ ( L M ) ] 1 D l fl m + ρ(1 + D l )P l m=1 min f m l l N(i), P l l=1 f m l R m i m = 1, 2... M, i = 1, 2,... N (2) M m=1 f m l γ(d l + P l ) l = 1, 2,... L (3) fl m β min{ri m, Rm j } where l = (i, j), l = 1, 2,... L (4) 0 fl m, 0 P l m, l (5) (1) The objective function described in (1) minimizes the overall cost for link usage and the unit cost is defined by the reciprocal of capacity, i.e., the summation of link usage cost which is the summed information flow multiplied by the unit cost. The second term of the objective function minimizes the penalty for using

7 strength-underestimated links. Incorporating the reciprocal of link capacity as part of the cost function favors a connectivity network where the links with larger capacity (strong links) have higher probability to be selected. The constraints imposed by fmri and dmri are captured by the three constraints (2)-(4). These constraints include: information demand constraint, link capacity constraint, and feasibility constraint. Information demand constraint: This constraint, described by (2), states that for each functional mode of the fmri, the gathered information at a region from all neighboring links must exceed the information demand for that mode. The information demand can be interpreted as the signal needed to have resulting neural activity observed in fmri. Moreover, the received information is replicable. It can be further shared with other regions. Thus, the region information demand represents a lower bound on the sum of information on all links passing through the region, for a specific functional mode of the fmri. Link flow capacity constraint: This constraint, described by (3), states that the total flow through a link should not exceed its capacity obtained from the dmri. Thus, the link capacity represents an upper bound on the link information flow. In (3), the parameter γ is necessary for unit conversion between information and capacity, i.e., fiber count. Feasibility constraint: This constraint, described by (4), states that the link flow for a specific mode m on a link l must be less than the minimum of the information demand corresponding to the same mode of the two regions associated with the link. From the sharing prospective, the shared information cannot exceed the existing information at the sharing end; on the other node, the obtained information will result in the stimulus which the demand is based on, thus the shared information cannot exceed the demand at the receiving end as well. min f 1 1,f 1 2,f D 1 f D 2 f D 3 f 1 3 (6) s.t. f f 1 2 R 1 1, f f 1 3 R 1 2, f f 1 3 R 1 3 (7) f 1 1 γd 1, f 1 2 γd 2, f 1 3 γd 3 (8) f 1 1 min{r 1 1, R 1 2}, f 1 2 min{r 1 1, R 1 3}, f 1 3 min{r 1 2, R 1 3} (9) 0 f 1 1, f 1 2, f 1 3 (10) Figure 4. A simple example has one mode m = 1, three regions with R 1 1 = R 1 2 = R 1 3 = 10, and three undirected links with capacity D 1 = 10, D 2 = 100, D 3 = 100. Example: To illustrate the proposed network flow model, we consider a brain network containing three regions with input parameters shown in Figure 4. The network flow formulation for this example is given by (6)-(10). In this example, we assume P 1 = P 2 = P 3 = 0. The solution to this network flow problem is given by f 1 1 = 0, f 1 2 = 10, f 1 3 = 10 and the corresponding value of objective function is 0.2. Note that f 1 1 = 5, f 1 2 = 5, f 1 3 = 5 leads to the cost function of 0.6. Intuitively the solution favors links with higher capacity. These optimization problems can be solved easily by commercial off-the-shelf software once Rl m and D l are obtained from the ICA result of the fmri and the analysis of the dmri, respectively. The proposed network model has the following properties: the missing and weak anatomical connections can be discovered from P l ; the more complete anatomical network can be formed from D l + P l ; the anatomical subnetwork for each individual functional mode m can be extracted from fl m functional-associated anatomical subnetwork; called the the links with higher anatomical connectivity have higher probability to be selected.

8 4. RESULTS AND DISCUSSION We first present the phantom validation results for verifying the true connections and comparing the performance to an existing work. 1 Then, the brain example based on HCP data will be presented in the next section. 4.1 Phantom Validation This section presents connectivity results based on three methods: 1) anatomical connectivity defined by fiber count from probabilistic fiber tracking using PROBTRACKX2 31 and the dmri data only, 2) the proposed network flow model using dmri and fmri data jointly, and 3) a probabilistic framework with Estimation- Maximization (EM) algorithm. 1 Both separate and joint frameworks 1 are implemented and the results are compared. (a) (b) Figure 5. (a) Tractography constructed by TrackVis; 35 (b) The anatomical network generated through probabilistic tracking with 5000 samples per seed voxel. The numbers on edges represent the numbers of tracks connecting the corresponding pairs of regions. Note the false-positive links from regions P1, P2, P3, P4, and P5 to region P16 and from regions P7, P8, P10, and P11 to region P9; the missing link P5-P7; and weak connections P6-P9, P9-P12, and P11-P15. In probabilistic tracking, Monte Carlo simulations with, typically, 5000 samples are performed and the number of streamlines can be interpreted as the probability and the strength of the anatomical connectivity as shown in Figure 5 (b). This network contains false links from regions P1, P2, P3, P4, and P5 to region P16 and from regions P7, P8, P10, and P11 to region P9, since these connections are not part of the ground truth shown in Figure 1 (b). These links are therefore artifacts due to the diffusion pre-processing step. Furthermore, the connection P5-P7 is missing and connections P6-P9, P9-P12, and P11-P15 are weak. In summary, probabilistic tracking leads to some inaccurately estimated connections; therefore, without other information or refinement we can only trade off between false connections and the absence of true connections. Figure 6 shows the connectivity derived from the proposed network flow model for all 5 functional modes. Based on Ri m and D l found from fmri data and dmri data, the network flow model was solved and the link flow fl m are shown in Figure 6 where the values of the flow correspond to the thicknesses of the links. The connectivity network and the part of the network that is missing from the ground truth are shown in Figures 7 (a) and (b), respectively. Figure 7 (a) is obtained by combining the connectivity networks of 5 modes. Figure 7 (b) contains the anatomical links that are absent from Figure 5 (b) but are part of the ground truth. Note that it is very hard to find the connection P5-P7; however this connection is captured by the proposed model. Furthermore, the links P6-P9, P9-P12, and P8-P10, that are weak in Figure 5 (b), are also strengthened by the proposed model. The connectivity result of using EM algorithm based on probabilistic framework 1 is shown in Figure 8. Figures 8 (a) and (b) illustrate the anatomical connectivity obtained only from dmri and functional connectivity only from fmri, respectively, and Figures 8 (c) and (d) represent the anatomical connectivity and functional connectivity jointly estimated from dmri and fmri. Note that joint estimation helps find extra anatomical connections and eliminates the false functional connections.

9 (a) (b) (c) (d) (e) Figure 6. The information network with respect to functional modes (m = 1, 2, 3, 4, 5). The size of node represents the value Ri m and the width of link stands for fl m. The anatomical connections were selected to form 5 subnetworks that support the 5 functional modes, respectively. (a) (b) Figure 7. (a) connectivity network obtained from combining the networks for all 5 modes. (b) connections that are part of the proposed model but are missing in Figure 5 (b).

10 (a) (b) (c) (d) Figure 8. (a) and (b) represent the estimated anatomical network and functional network, respectively, by EM approach 1 obtained from dmri and fmri data individually. (c) and (d) represent the results obtained from joint estimation. By joint estimation, 5 missing links were discovered in (c) but with 10 additional false connections. In functional network (d), the misleading negative correlation between blue mode and orange mode is eliminated. Table 2. Performance Comparison on Synthetic Data. The number of false, missing, and correct anatomical connections are compared for various schemes. Scheme False Missing Correct Connections Connections Connections Data Set Ground Truth Probabilistic Tracking α dmri Separate EM 1β dmri Joint EM dmri+fmri (Proposed) Network Flow Framework dmri+fmri α The links are thresholded at 5% of the maximum. β A simplified version of joint EM 1 that estimates connectivity separately. The numbers of false connections, missing connections, and correct connections with respect to all three schemes and the ground truth (Figure 1 (b)) are summarized in Table 2. Combining fmri data helps find missing anatomical connections. Moreover, in the proposed network flow model, the false connections are the fewest, the missing connections are the least, and the correct connections are the highest. The proposed model leads to the best connectivity for the synthetic data considered in the paper. 4.2 Brain Network from the Human Connectome Project Data The network model is solved to obtain the anatomical network, D l + P l, and functional-associated networks, fl m. The anatomical networks are composed from the connections estimated by probabilistic tractography and additional connections found by the joint modeling. The functional-associated networks are the subnetworks of the anatomical network dedicated to each ICA component. Because the order of ICA components may vary, we focus on the aggregated functional-associated network, f l = m f l m, instead of each individual sub-network. Both anatomical network and aggregated functional-associated network are undirected and weighted. To explore various levels of significance, we further threshold both networks into unweighted networks of sparsity level from 1% to 30% with 1% steps according to the weights on each link. 36 For a functional-associated network, the subnetwork is picked according to the objective function (1) in the model. Moreover, since the objective function listed in the equation (1) constrains the model to select the strongest pathways, we also propose to use another min-max objective function to find the maximal portion of anatomical network related to the fmri data. M m=1 min max f l m (11) f m l l D l + P l

11 (a) (b) (c) Figure 9. The adjacent matrices of anatomical network (Left panel, presented in logarithm scale) and resting-stateassociated anatomical subnetworks (Middle panel is with objective function (1) and right panel is with objective function (11) ) obtained from an HCP subject using the proposed joint model on dmri and rfmri modalities. Color code indicates the regions which are corpus callosum, left hemisphere (frontal, parietal, occipital, temporal, and limbic and sub cortical areas), and right hemisphere use the same as previous list. The min-max objective function balances the capacity usage for each link by spreading the flow out proportionally according to the capacity of links. Therefore, we are able to obtain the largest portion of the anatomical network that are relevant to the brain function of the fmri data. 5. CONCLUSION AND FUTURE WORK This paper has presented a new network flow formulation for computing brain connectivity from the joint analysis of dmri and fmri. Using a synthetic data example, it is shown that the proposed model leads to the best connectivity network with the fewest number of false or missing connections, and the highest number of correct connections. Future work will be directed towards validating the proposed model on a larger database of fmri and dmri data such as the HCP. Future work will also be directed towards investigating the effect of alternative cost functions on the resulting network connectivity. REFERENCES [1] Venkataraman, A., Rathi, Y., Kubicki, M., Westin, C.-F., and Golland, P., Joint modeling of anatomical and functional connectivity for population studies, Medical Imaging, IEEE Transactions on 31(2), (2012). [2] Rabinovich, M. I., Afraimovich, V. S., Bick, C., and Varona, P., Information flow dynamics in the brain, Physics of life reviews 9(1), (2012). [3] Salinas, E. and Sejnowski, T. J., Correlated neuronal activity and the flow of neural information, Nature Reviews Neuroscience 2(8), (2001). [4] Kriegeskorte, N., Goebel, R., and Bandettini, P., Information-based functional brain mapping, Proceedings of the National Academy of Sciences of the United States of America 103(10), (2006). [5] Korzeniewska, A., Mańczak, M., Kamiński, M., Blinowska, K. J., and Kasicki, S., Determination of information flow direction among brain structures by a modified directed transfer function (ddtf) method, Journal of neuroscience methods 125(1), (2003). [6] Biswal, B., Zerrin Yetkin, F., Haughton, V. M., and Hyde, J. S., Functional connectivity in the motor cortex of resting human brain using echo-planar MRI, Magnetic resonance in medicine 34(4), (1995). [7] Lee, M., Smyser, C., and Shimony, J., Resting-state fmri: a review of methods and clinical applications, American Journal of Neuroradiology 34(10), (2013).

12 [8] Uddin, L. Q., Clare Kelly, A., Biswal, B. B., Xavier Castellanos, F., and Milham, M. P., Functional connectivity of default mode network components: correlation, anticorrelation, and causality, Human brain mapping 30(2), (2009). [9] Fox, M. D., Greicius, M., Fox, M., Greicius, M., et al., Clinical applications of resting state functional connectivity, Frontiers in systems neuroscience 4, 19 (2010). [10] Wang, J., Zuo, X., and He, Y., Graph-based network analysis of resting-state functional MRI, Frontiers in systems neuroscience 4 (2010). [11] Assaf, Y. and Pasternak, O., Diffusion tensor imaging (DTI)-based white matter mapping in brain research: a review, Journal of Molecular Neuroscience 34(1), (2008). [12] Bozzali, M., Falini, A., Franceschi, M., Cercignani, M., Zuffi, M., Scotti, G., Comi, G., and Filippi, M., White matter damage in alzheimer s disease assessed in vivo using diffusion tensor magnetic resonance imaging, Journal of Neurology, Neurosurgery & Psychiatry 72(6), (2002). [13] Sporns, O., The human connectome: a complex network, Annals of the New York Academy of Sciences 1224(1), (2011). [14] Griffa, A., Baumann, P. S., Thiran, J.-P., and Hagmann, P., Structural connectomics in brain diseases, Neuroimage 80, (2013). [15] Johansen-Berg, H. and Behrens, T. E., Just pretty pictures? what diffusion tractography can add in clinical neuroscience, Current opinion in neurology 19(4), 379 (2006). [16] Honey, C., Sporns, O., Cammoun, L., Gigandet, X., Thiran, J.-P., Meuli, R., and Hagmann, P., Predicting human resting-state functional connectivity from structural connectivity, Proceedings of the National Academy of Sciences 106(6), (2009). [17] Honey, C. J., Thivierge, J.-P., and Sporns, O., Can structure predict function in the human brain?, Neuroimage 52(3), (2010). [18] Damoiseaux, J. S. and Greicius, M. D., Greater than the sum of its parts: a review of studies combining structural connectivity and resting-state functional connectivity, Brain Structure and Function 213(6), (2009). [19] Zhu, D., Zhang, T., Jiang, X., Hu, X., Chen, H., Yang, N., Lv, J., Han, J., Guo, L., and Liu, T., Fusing DTI and fmri data: A survey of methods and applications, NeuroImage 102, (2013). [20] Ng, B., Varoquaux, G., Poline, J.-B., and Thirion, B., A novel sparse graphical approach for multimodal brain connectivity inference, in [Medical Image Computing and Computer-Assisted Intervention MICCAI 2012], , Springer (2012). [21] Fillard, P., Descoteaux, M., Goh, A., Gouttard, S., Jeurissen, B., Malcolm, J., Ramirez-Manzanares, A., Reisert, M., Sakaie, K., Tensaouti, F., et al., Quantitative evaluation of 10 tractography algorithms on a realistic diffusion MR phantom, Neuroimage 56(1), (2011). [22] Poupon, C., Rieul, B., Kezele, I., Perrin, M., Poupon, F., and Mangin, J.-F., New diffusion phantoms dedicated to the study and validation of high-angular-resolution diffusion imaging (HARDI) models, Magnetic Resonance in Medicine 60(6), (2008). [23] Xia, M., Wang, J., and He, Y., Brainnet viewer: a network visualization tool for human brain connectomics, PLoS One 8(7), e68910 (2013). [24] Van Essen, D. C., Ugurbil, K., Auerbach, E., Barch, D., Behrens, T., Bucholz, R., Chang, A., Chen, L., Corbetta, M., Curtiss, S. W., et al., The Human Connectome Project: a data acquisition perspective, Neuroimage 62(4), (2012). [25] Van Essen, D. C., Smith, S. M., Barch, D. M., Behrens, T. E., Yacoub, E., and Ugurbil, K., The WU-Minn human connectome project: an overview, NeuroImage 80, (2013). [26] Feinberg, D. A., Moeller, S., Smith, S. M., Auerbach, E., Ramanna, S., Glasser, M. F., Miller, K. L., Ugurbil, K., and Yacoub, E., Multiplexed echo planar imaging for sub-second whole brain fmri and fast diffusion imaging, PLoS One 5(12), e15710 (2010). [27] Moeller, S., Yacoub, E., Olman, C. A., Auerbach, E., Strupp, J., Harel, N., and Uğurbil, K., Multiband multislice GE-EPI at 7 tesla, with 16-fold acceleration using partial parallel imaging with application to high spatial and temporal whole-brain fmri, Magnetic Resonance in Medicine 63(5), (2010).

13 [28] Setsompop, K., Gagoski, B. A., Polimeni, J. R., Witzel, T., Wedeen, V. J., and Wald, L. L., Blippedcontrolled aliasing in parallel imaging for simultaneous multislice echo planar imaging with reduced g-factor penalty, Magnetic Resonance in Medicine 67(5), (2012). [29] Xu, J., Moeller, S., Strupp, J., Auerbach, E., Chen, L., Feinberg, D., Ugurbil, K., and Yacoub, E., Highly accelerated whole brain imaging using aligned-blipped-controlled-aliasing multiband EPI, in [Proceedings of the 20th Annual Meeting of ISMRM], 2306 (2012). [30] Glasser, M. F., Sotiropoulos, S. N., Wilson, J. A., Coalson, T. S., Fischl, B., Andersson, J. L., Xu, J., Jbabdi, S., Webster, M., Polimeni, J. R., et al., The minimal preprocessing pipelines for the Human Connectome Project, NeuroImage 80, (2013). [31] Jenkinson, M., Beckmann, C. F., Behrens, T. E., Woolrich, M. W., and Smith, S. M., FSL, Neuroimage 62(2), (2012). [32] Beckmann, C. F. and Smith, S. M., Probabilistic independent component analysis for functional magnetic resonance imaging, Medical Imaging, IEEE Transactions on 23(2), (2004). [33] Storti, S. F., Formaggio, E., Nordio, R., Manganotti, P., Fiaschi, A., Bertoldo, A., and Toffolo, G. M., Automatic selection of resting-state networks with functional magnetic resonance imaging, Frontiers in neuroscience 7 (2013). [34] Späth, H., [Cluster dissection and analysis: theory, FORTRAN programs, examples], Computers and their applications, Horwood (1985). [35] Wang, R., Benner, T., Sorensen, A., and Wedeen, V., Diffusion toolkit: a software package for diffusion imaging data processing and tractography, in [Proc Intl Soc Mag Reson Med], 15, 3720 (2007). [36] Sporns, O., [Networks of the Brain], MIT press (2011).