A Unified Estimation Framework for State- Related Changes in Effective Brain Connectivity

Transcription

1 Universiti Teknologi Malaysia From the SelectedWorks of Chee-Ming Ting PhD 6 A Unified Estimation Framework for State- Related Changes in Effective Brain Connectivity Chee-Ming Ting, PhD S. Balqis Samdin Hernando Ombao, University of California - Irvine Sh-Hussain Salleh Available at:

2 Citation information: DOI.9/TBME , IEEE Transactions on Biomedical Engineering IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING A Unified Estimation Framework for State-Related Changes in Effective Brain Connectivity S. Balqis Samdin, Chee-Ming Ting, Member, IEEE, Hernando Ombao and Sh-Hussain Salleh Abstract Objective: This paper addresses the critical problem of estimating time-evolving effective brain connectivity. Current approaches based on sliding window analysis or time-varying coefficient models do not simultaneously capture both slow and abrupt changes in causal interactions between different brain regions. Methods: To overcome these limitations, we develop a unified framework based on a switching vector autoregressive (SVAR) model. Here, the dynamic connectivity regimes are uniquely characterized by distinct VAR processes and allowed to switch between quasi-stationary brain states. The state evolution and the associated directed dependencies are defined by a Markov process and the SVAR parameters. We develop a three-stage estimation algorithm for the SVAR model: (.) feature extraction using time-varying VAR (TV-VAR) coefficients; (.) preliminary regime identification via clustering of the TV-VAR coefficients; (3.) refined regime segmentation by Kalman smoothing and parameter estimation via expectation-maximization (EM) algorithm under a state-space formulation, using initial estimates from the previous two stages. Results: The proposed framework is adaptive to state-related changes and gives reliable estimates of effective connectivity. Simulation results show that our method provides accurate regime change-point detection and connectivity estimates. In real applications to brain signals, the approach was able to capture directed connectivity state changes in fmri data linked with changes in stimulus conditions, and in epileptic EEGs, differentiating ictal from non-ictal periods. Conclusion: The proposed framework accurately identifies state-dependent changes in brain network and provides estimates of connectivity strength and directionality. Significance: The proposed approach is useful in neuroscience studies that investigate the dynamics of underlying brain states. Index Terms Dynamic brain connectivity, Vector autoregressive models, Regime-switching models, State-space models. I. INTRODUCTION FUNCTIONAL connectivity in brain networks is broadly defined as the cross-dependence between distinct brain regions. It is usually inferred from neuroimaging signals such as electroencephalogram (EEG) and functional magnetic resonance imaging (fmri) time series []. Effective connectivity is a more specific measure of cross-dependence in a sense that it quantifies the influence of one neuronal region over another []. Vector autoregressive (VAR) models are the commonly-used tools for analyzing effective connectivity. However, traditional analyses fit a time-invariant VAR model over the entire time course, e.g., for fmri [3] [7]. This assumes temporal stationarity of the connectivity networks which is a limitation because recent fmri studies (e.g., [8] [3]) report that functional connectivity may exhibit nonstationary behavior and can be dynamic even in a resting state. S. B. Samdin, C.-M. Ting and S.-H. Salleh are with the Center for Biomedical Engineering (CBE), Universiti Teknologi Malaysia (UTM), 83 Skudai, Johor, Malaysia ( sbalqis4@live.utm.my; cmting@utm.my). H. Ombao is with the Department of Statistics and the Department of Cognitive Sciences, University of California at Irvine, Irvine CA 9697, USA ( hombao@uci.edu). To estimate dynamic functional connectivity, these studies used the sliding-window approach where the choice of window size is crucial: a large window leads to low statistical power for detecting abrupt and highly localized changes; a small window produces noisy estimates for smooth changes. [4] recently suggested using time-varying multivariate volatility models for fmri to capture these instantaneous changes in functional connectivity. Moreover, [5] and [6] proposed a regressionbased approach to estimate both smooth and abrupt changes in task-related and resting-state connectivity. However, these approaches do not offer information about directionality. They only examined dynamic functional connectivity which is undirected and hence did not sufficiently address the question of whether one region exerts an influence over another. An alternative approach to estimating dynamic connectivity or coherence with directionality in EEG [7] [] and fmri [], is based on time-varying VAR models with coefficients that change instantaneously with time. However, recent evidence from fmri studies suggests that the dynamic undirected connectivity tends to exhibit recurring quasi-stable temporal regimes with relatively long periods, driven by distinct brain states. This reveals state-related dynamic behavior of the brain connectivity [9], [3], [4] with states being turned on and off across recording periods. This observation motivates our current work, that dynamic directed connectivity could be estimated using time-series blocks that belong to the same state. Extracting information from different blocks (or instantaneous possibly non-contiguous time) of the same state can produce estimates of VAR parameters with lower mean-squared error and hence improved estimates of both instantaneous and smooth state-related changes in effective connectivity. In this paper, we develop a unified framework for estimating dynamic effective connectivity based on switching VAR processes. It combines different procedures to allow for reliable, adaptive estimation of dynamic connectivity structures of underlying brain process with state-related changes, regardless they are slow or abrupt. The proposed framework is potentially important for a number of applications in neuroscience data such as identifying states that correspond to cognitive regime versus resting state or differentiating seizure from non-seizure states. We demonstrate the suitability of our method for various types of brain signals with dynamic behavior by applying it to both task-related fmri and epileptic seizure EEG data. In this framework, we consider both fmri and EEG data as realization of a dynamic VAR model. In particular, we consider two types of dynamic VAR processes: (.) VAR with timevarying parameters (TV-VAR); and (.) VAR with Markovian regime-switching (SVAR). The SVAR is a combination of unique VAR processes switching between a few discrete states. The TV-VAR and SVAR models capture, respectively, the Copyright (c) 6 IEEE. Personal use of this material is permitted. However, permission to use

3 Citation information: DOI.9/TBME , IEEE Transactions on Biomedical Engineering IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING dynamic connectivity with instantaneous abrupt change at each time point, and that with regime shift between quasi-stationary states. Particularly, the SVAR model is capable of detecting changes at different time scales, both smooth and abrupt, avoiding the problems associated with fixed time windowing. Neuroimaging signals are typically obscured or confounded by various noise sources (e.g., non-neuronal physiological signals and instrumental artifacts), in data with a low signalto-noise (SNR) ratio. This renders it difficult to obtain reliable estimates of the underlying dynamic effective connectivity. The SVAR formulation provides the benefit of more robust estimation of connectivity in the presence of noise by pooling together and averaging over more samples from the recurring (potentially non-contiguous) time blocks that belong to the same state. To further address the problem, we employ a state-space representation for both the TV-VAR and SVAR model. This approach captures the underlying time-evolving VAR connectivity structures as an unobserved latent process in the state equation. The observation equation can accommodate the noise by representing the observed EEG or fmri as contaminated versions of the latent process. Such representation allows the latent dynamic connectivity (as characterized by the TV-VAR or SVAR process parameters) be estimated from the observed signals and separated recursively from the noise, using the Kalman filter (KF) with refinement by the Kalman smoother (KS). The maximum-likelihood estimation can be accomplished by the expectation-maximization (EM) algorithm. We recently used TV-VAR model with EM and KF estimation [5] to estimate time-varying directed coherence from EEG data. This differs from previous connectivity studies using state-space TV-VAR [9], [], [], which depend on some sub-optimal heuristic methods for parameter estimation. We develop a three-stage estimation framework. The first stage is feature extraction, using KF to compute TV-VAR coefficients to represent the time-evolving directed connectivity. The second is initial identification of connectivity states via K-means clustering of the estimated TV-VAR connectivity features, to partition the time-varying connectivity structure of neuroimaging time series into a finite number of distinct quasi-stable states/regimes. The third is the refined regime estimation based on the SVAR model. The change-points or transition between connectivity states are refined by switching KF and switching KS (SKF and SKS) and, simultaneously, more accurate estimates of the VAR connectivity graph for each state are obtained by the EM algorithm. The SVAR model is initialized based on the estimates from the first two stages. In a related work, [9] used K-means clustering with sliding windows to estimate dynamic functional connectivity with the transition states, which only gives sub-optimal estimates. This is because the K-means algorithm provides a hard assignment of time points into states and does not account for the temporal correlation structure. In contrast, KS generates soft state-time alignment by estimating sequentially, for each time point, the probability of the occupying states based on the entire observation time course. We note that a switching linear dynamic system (SLDS) was also used to estimate effective connectivity in fmri [6]. However, a limitation of this work is the manual tuning of the initial parameters. This is timeconsuming and does not guarantee optimal estimates, as EM algorithm is known to be sensitive to initial estimates and tends to be trapped in a local maximum. Moreover, [6] developed it only for VAR models of order one which might not be optimal across various neural data sets [7]. In contrast, we develop a generalized version of state-space SVAR model with higher orders. Our proposed procedure, which is based on the K- means clustering of TV-VAR features, can automatically and data-adaptively select better initial parameters for the SVAR. The paper is organized as follows: Section II and III describe our proposed VAR models and state-space approach for estimating dynamic effective connectivity. Section IV presents the novel unified estimation framework for state-related changes. Section V reports performance evaluation based on simulations and real EEG and fmri data. Section VI draws the conclusion. II. VAR MODELS FOR EFFECTIVE CONNECTIVITY A. Stationary VAR Model One of the key ingredients in our proposed SVAR model is the distinct connectivity states, each is characterized by a unique VAR process. The stationary VAR processes are the building blocks of the SVAR model. Let Y t = [Y t,..., Y Nt ], t =,..., T be a N dimensional vector of neuroimaging time series of length T, measured from N nodes (regions or channels) in a brain connectivity network (e.g. fmri time series from voxels or multi-channel EEGs from scalp electrodes). A common approach to estimating effective connectivity in Y t is to fit a VAR model of order P, VAR(P ) Y t = Φ Y t Φ P Y t P + v t, () where v t = [v t,..., v Nt ] is a N Gaussian white noise with mean zero and covariance matrix R, v t N(, R). R is constant across all time points t. The N N matrix of VAR coefficients for lag l, denoted by Φ l = [φ lij ], i, j N, quantifies the directed connections between different brain regions at time lag l. There exists a directed influence in the Granger-causality sense with direction from region j to region i for any connection strength φ ij >. However, for fmri, this only indicates the presence of Granger causality in the hemodynamic activity. That is, the past hemodynamic activity in one region is statistically significant for explaining future hemodynamic activity in another region. Due to the time-lag of BOLD hemodynamic response relative to the neural activity and the differences in latency across the brain, we need to be cautious in the interpretation. We should not extrapolate it as a direct inference of information flows in neuronal activity. The time-invariant model () has been used to infer effective connectivity in fmri data, by fitting over the entire time-course [3], [4], [6], [7]. The VAR parameters can be estimated by formulating the model () as a multivariate linear model, Y = Uβ+E, where Y = [Y P +,..., Y T ] is the (T P ) N time series data matrix; U is a (T P ) NP matrix of previous observations U = Y P Y P... Y Y P + Y P... Y Y T Y T... Y T P Copyright (c) 6 IEEE. Personal use of this material is permitted. However, permission to use,

4 Citation information: DOI.9/TBME , IEEE Transactions on Biomedical Engineering IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING 3 β = [Φ,..., Φ P ] is a NP N matrix composed of coefficients VAR matrices of all lags and E = [v P +,..., v T ] is the collection of noise terms at all time points. The conditional least squares (LS) estimators of β and R given time points t =,..., P are defined respectively as β = (U U) U Y and R = (/(T P ))(Y U β) (Y U β). The LS estimator b = vec( β) has an asymptotic normal distribution [8] T ( b b) D N(, Γ) () The covariance matrix of the estimator can be estimated by Γ = R (U U/T ) where denotes the Kronecker product. Based on this, we test the significance of each VAR coefficient in β as being different from zero, with H : b k = against H : b k, where b k is k-th element of b. The test statistic is approximately distributed as t k = b k / Γ kk /T N(, ) when T P is sufficiently large, where Γ kk is k-th diagonal entry of Γ. A coefficient is significant if the p value < α/d with α the significance level and D = P N the number of tested coefficients, implying corrections for multiple testing by Bonferroni method. To model effective connectivity at specific frequencies, we consider frequency-domain measures constructed from the VAR parameters. Here, we use the partial directed coherence (PDC) which is a frequency-domain interpretation (analogue) of Granger causality. PDC measures the directed influence of one signal on another at specific frequency f. Let Φ(f) = I P l= Φ lexp( iπlf/f s ) be the Fourier transform of the VAR coefficient matrices, where f s is the sampling frequency. The PDC is the defined as Φ ij (f) π ij (f) = N (3) k= Φ kj(f) π ij (f) [, ] quantifies the strength of directed influence from Y jt to Y it at frequency f relative to the total influence of Y jt on all signals. A value close to one would indicate that the causal influences originating from brain region j are directed, for the most part toward region i, and a value of zero indicates no directed influences from region j to region i. B. Time-Varying VAR Model To capture the dynamic structure of effective connectivity, our first step is to extract the time-varying connectivity features by fitting a time-varying VAR (TV-VAR) model, a generalization of the VAR model () where the VAR coefficient matrix at each lag l is allowed to change with time t, as given by P Y t = Φ lt Y t l + v t (4) l= where Φ lt is the N N matrix of TV-VAR coefficients at lag l =,..., P and time t =,..., T and v t N(, R) is the N white Gaussian observational noise. Defining a t = vec([φ t,..., Φ P t ] ) the P N state vector of TV- VAR coefficients at time t, the model (4) can be represented in a state-space form, as in [9] a t = a t + w t (5) Y t = C t a t + v t (6) with C t = I N X t where I N is a N N identity matrix and X t = [Y t,..., Y t P ]. The TV-VAR process (4) is rewritten in a compact form (6) as the observation equation, with a linear mapping C t consisting of past observations. In state equation (5), the hidden state a t is assumed to follow a firstorder Gauss-Markov process where w t is white Gaussian state noise with mean zero and P N P N covariance matrix Q, w t N(, Q). Both R and Q are assumed as time-invariant. We denote by θ = (R, Q) the parameters of TV-VAR statespace model. In our proposed estimation algorithm (in Section IV), we shall fit the TV-VAR model to the brain signals to compute the estimates of the connectivity matrices which will then be fed into the clustering algorithm to obtain an initial estimates of the partitions of the connectivity regimes. C. Switching VAR Model Switching VAR (SVAR) is a quasi-stationary model consisting of a collection of K independent underlying processes, where each state is characterized completely by a unique VAR model, indexed by a hidden random indicator S t Y t = P Φ l,[st]y t l + v t (7) l= where {S t {j =,..., K}, t =,..., T } is a sequence of state variables, which varies over time and take values in a discrete space j =,..., K; and {Φ l,[j], l =,..., P } are VAR coefficient matrices for state j. The regimes of the distinct VAR structures switch according to the latent state variable S t, commonly assumed to follow a hidden Markov chain with transition matrix Z = [z ij ], i, j K where z ij = P (S t = j S t = i) (8) denotes the probability of transition from state i at time t to state j at t. Only one latent process (and hence only one VAR process) is active (or turned on) at each time point t. The remaining latent processes are turned off. One aim is to determine the temporal regimes where each state is active (or equivalently, identify the state that is active for each regime). We propose a state-space representation of the SVAR(P ) model. Here, we use the fact that any VAR(P ) process can be written in VAR() form where the equivalent VAR() has a higher dimensionality. It is an extension of the switching VAR() state-space model in [6] to the general case of SVAR(P ). This is because some types of brain signals such as EEG, typically follow higher-order autoregressive processes, where the first-order assumption is too simplistic and does not sufficiently capture the complex dependence structure between channels. The SVAR(P ) model in a switching linear Gaussian state-space form is defined as follows X t = A [St]X t + w t (9) Y t = HX t + v t () where X t = [x t, x t,..., x t P + ], w t = [η t,,..., ] is NP state noise, and A [St] is a NP NP state transition Copyright (c) 6 IEEE. Personal use of this material is permitted. However, permission to use

5 Citation information: DOI.9/TBME , IEEE Transactions on Biomedical Engineering IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING 4 matrix switching with the state variables S t, and of the form Φ Φ... Φ P Φ P I N... A = I N I N The matrix A [St] describes the effective connectivity that varies across states. The underlying SVAR(P ) signal x t now follows a (higher dimensional) latent VAR() process X t which is not directly observed since it is buried in various noise sources modeled by v t as noisy observations Y t, via the N NP mapping matrix H = [I N,,..., ]. We further assume that the observation noise and the state noise are white Gaussian processes, i.e. v t N(, R [St]) and w t N(, Q [St]), with covariance matrices R [St] and Q [St] switching with S t. Since the Gaussian distribution varies between the K regimes, the full model is the approximation of a mixture of Gaussian distributions. Note that the connectivity and the noise structure A [St], R [St] and Q [St] are piecewise constant functions of the discrete state S t. Instead of hard state assignment for each time-point t, we can evaluate the probability of activation for each state, P (S t = j Y :T ), which is termed soft-alignment. We denote all model parameters from each of the states as Θ = {Θ j = (A j, Q j, R j ) : j {,..., K}}. III. ESTIMATING TIME-EVOLVING DIRECTED CONNECTIVITY BY STATE-SPACE APPROACH In this section, we describe our algorithms for estimating the time-varying directed connectivity based on the statespace approach. Our final estimates are derived from the SVAR model. However, the initial estimates of the parameters and regimes are provided by fitting the TV-VAR model and by clustering the coefficient matrix estimates. Our procedure involves estimation of the hidden state parameters as well as the model parameters of the developed state-space models. Given the observations Y t, t =,..., T, the latent state vectors of the TV-VAR coefficients a t, t =,..., T in the TV-VAR model (5) and (6) can be estimated sequentially in time using the KF and refined by the KS (also called Rauch Tung Striebel smoother). The maximum likelihood estimate of θ is obtained by using the iterative EM algorithm (see [5], [3], [3]). Similarly for the SVAR state-space model, the aims are to estimate the underlying latent brain signals X t and the latent switch variable S t. This involves estimating sequentially in time the filtered densities or probabilities p(x t Y :t ) and P (S t Y :t ) given the noisy signal observations up to time t, Y :t = {Y,..., Y t }, and the more accurate smoothed densities p(x t Y :T ) and P (S t Y :T ) given the available entire set of observations Y :T = {Y,..., Y T }. We estimate the filtered and smoothed densities of X t given state j at time t, by the KF and the KS, respectively X j t t = E(X t Y :t, S t = j) () V j t t = Cov(X t Y :t, S t = j) () X j t T = E(X t Y :T, S t = j) (3) V j t T = Cov(X t Y :T, S t = j) (4) V j t,t T = Cov(X t, X t Y :T, S t = j) (5) where X j t t and V j t t are the mean and covariance of the filtered density p(x t Y :t, S t = j); X j t T and V j t T are the mean and covariance of the smoothed density p(x t Y :T, S t = j) is the cross-variance of joint density p(x t, X t Y :T, S t = j) The estimates of filtered and smoothed state occupancy probability of being state j at time t are also computed given state j at time t. V j t,t T M j t t = P (S t = j Y :t ) (6) M j t T = P (S t = j Y :T ) (7) The ML estimates of the model parameters Θ can be obtained by maximizing the log-likelihood L = log p(y :T Θ). We use a extension of the EM algorithm suggested by [3]. In the expectation step (E-step), the sufficient statistics are obtained from the smoothed estimates P t = E(X t X t Y :T ) = V t T + X t T X t T (8) P t,t = E(X t X t Y :T ) = V t,t T + X t T X t T (9) where X t T, V t T and V t,t T are quantities of the smoothed densities p(x t Y :T ) and p(x t, X t Y :T ), corresponding to (3) to (5) by marginalizing out the state variable j of the p(x t Y :T, S t = j) and p(x t, X t Y :T, S t = j) using Gaussian approximation. We retain the terms switching KF (SKF) and switching KS (SKS) to refer to KF/KS approach to estimating state parameters of the SVAR model, as in [3]. In the maximization step (M-step), the estimates of the model parameters for regime j are updated as follows ( T ) ( T ) Â j = W j t P t,t W j t P t () Q j = ẑ ij = t= t= ( ) ( T T t= W j W j t P t Âj t t= T t= P (S t = j, S t = i Y :T ) T t= W j t ) T W j t P t,t () t= () where the weights W j t = M j t T are computed from the smoothing step. Here, the observation noise covariance R j is initialized from a distinct stationary VAR model for each regime and not updated by the EM algorithm. IV. A FRAMEWORK FOR ESTIMATING DYNAMIC CHANGES IN CONNECTIVITY STATES We propose a unified framework for robust estimation of state-related dynamic changes in effective brain connectivity using neuroimaging time-series data, such as electrophysiological (e.g. EEG) and hemodynamic (fmri) signals. Motivated by recent works [9], [3] on functional connectivity, we shall assume that the time-varying directed connectivity also tends to cluster into quasi-stationary regimes, according to a finite number of distinct brain states. The connectivity structure is constant or slowly varying within a regime, but changes across Copyright (c) 6 IEEE. Personal use of this material is permitted. However, permission to use

6 Citation information: DOI.9/TBME , IEEE Transactions on Biomedical Engineering IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING 5 regimes as driven by the transition of the states, probably in an abrupt manner. To capture these characteristics, the proposed framework combines different procedures: () TV- VAR model the feature extractor to represent the time-evolving connectivity with instantaneous changes; () K-means clustering algorithm on the VAR coefficients in Step () to produce initial clusters where each cluster represents a quasi-stationary connectivity state (regime); and (3) SVAR model that simultaneously accomplishes a more refined detection of the temporal change-points of the distinct dynamic regimes (modeled by separate VAR processes switching according to a Markov chain) and a re-estimation of a set of directed dependencies between brain regions within each regime. These operate within a state-space formulation where the parameters are estimated using the SKS and the EM algorithm respectively. The overview of the proposed framework is shown in blockdiagram in Figure. First, we use KF to extract adaptively a sequence of TV-VAR coefficients in (5)-(6) from the neuroimaging signals as features that characterize the time-varying directed brain connectivity. Second, the signals are partitioned into segments corresponding to a finite number of discrete connectivity states. This partitioning of the signals is a result of the K-means clustering step of the TV-VAR coefficients, which produces a state sequence ŜKM t. The number of states K is assumed known or estimated from data. Finally, we employ a Markov-switching VAR model (8)-() to capture the dynamics of connectivity regimes for these brain connectivity states, via distinct state-dependent VAR models switching across time between the states, according to a hidden Markov chain. The estimates of the coefficient matrices of the SVAR model for each state, are initialized by the least-squares (LS) estimates, ΦLS-KM [j] of distinct stationary VAR models fitted separately on each regime time-segments derived from the K-means partitioning. Given the estimated SVAR model Θ, switching KF is then used to refine the regime segmentation by estimating the latent state sequence ŜSKF t through P (S t = j Y :t ) in (6) thus indicating, for each time point, the most likely active state. The time-state assignment is then further refined using the switching KS, as ŜSKS t through the estimated P (S t = j Y :T ) in (7), based on the past and future observations. Next, the SVAR parameters are reestimated from the neuroimaging signals iteratively using the EM algorithm (8)-(). We investigate two different estimation procedures for the state-dependent coefficient matrices of the SVAR model: (A.) using the EM-updated estimates in the M-step directly given Φ EM in (), denoted here by [j] and (B.) a second-stage LS estimates of distinct quasi-stationary VAR for each state, but based on the SKF and SKS-segmented regimes obtained using the EM-estimated model, denoted respectively by and Φ LS-SKS [j] Φ LS-SKF [j]. The second estimator can better capture the stationarity of each regime, compared to the direct EM updates which is an approximation relying on the average of other estimated instantaneously varying quantities such as the covariances P t. Our simulation studies indicate that (B.) provides substantially reduced mean-squared errors, relative to (A.). The proposed estimation framework is summarized in Algorithm. Algorithm : Unified Framework for Estimating State- Related Dynamic Effective Connectivity Input: N brain signals Y t = [Y t,..., Y Nt ], t =,..., T. Number of states = K, with each state j {,..., K}. Step : Feature Extraction using TV-VAR Model Given model θ (5)-(6), compute the TVAR coefficient vectors â t = E(a t Y :t ) by KF. Step : Initial Estimation of Connectivity Regimes by Clustering the TV-VAR Coefficients Apply K-means algorithm to partition {Y... Y T } into K disjoint clusters C {C,..., C K } defined by arg min C K j= Y t C j D(â t, µ j ) where µ j is the centroid of C j and D(â t, µ j ) is the distance measure between â t and µ j. Generate state sequence Ŝ KM t = arg min D(â t, µ j ), t =,..., T. j {,...,K} Φ LS-KM Compute the connectivity matrix estimate [j] by LS fitting of a stationary VAR model () to partition C j = {Y t : ŜKM t = j}, for each state j =,..., K. Step 3: Markov-SVAR Estimation using EM Algorithm Initialize the estimate Θ () at iteration r =. Iterate, for r, E-Step: Given estimates Θ (r), Compute the mean and covariance of filtered densities p(x t Y :t ) and P (S t Y :t ) using SKF. Compute the mean and covariance of smoothed densities p(x t Y :T ) and P (S t Y :T ) using SKS. Compute the expectations according to (8)-(9). M-Step: Update the estimates Θ (r+) according to ()-(). Until convergence, set Θ = Θ (r). Output: Refined state sequence given Θ Ŝt SKF = arg max P (S t Y :t ), j {,...,K} Ŝt SKS = arg max P (S t Y :T ), t =,..., T. j {,...,K} Improved connectivity estimates for j =,..., K. EM estimates Φ EM [j] from  j. Φ LS-SKF Φ LS-SKS LS estimates [j] and [j] by fitting a stationary VAR to time segments of state j, Π j = {Y t : Ŝt = j}, according to ŜSKF t and ŜSKS t. Copyright (c) 6 IEEE. Personal use of this material is permitted. However, permission to use

7 Citation information: DOI.9/TBME , IEEE Transactions on Biomedical Engineering IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING 6 Multivariate Neural Signals Feature Extraction TV-VAR coefficients K-means Clustering State sequence Signal Segmentations for Model Initialization Switching VAR Model EM & Kalman Smoother Refined states Connectivity matrices Fig.. A unified framework for estimating dynamic effective connectivity states in brain signals. The connectivity state estimation is initialized by K-means clustering of time-evolving connectivity features extracted using a TV-VAR process, and then refined by the SKF and EM algorithm based on a SVAR process. V. EXPERIMENTAL RESULTS In this section, we evaluate the proposed framework for estimating the state-related dynamics of directed brain connectivity through simulation studies and the analysis of a blockdesign motor-task fmri data and an epileptic seizure EEG data set. We compared the performance of K-means clustering as the baseline method with the EM-based Markov-SVAR estimators for () tracking the regime changes via the inferred state time-courses indicating which regime is active at each time point, and () estimating the VAR connectivity matrix associated with each regime. For the simulated and the fmri data, the correct state labels for each time point and the statedependent VAR coefficient matrices are considered known, and serve as ground-truth for evaluation. For the epileptic EEG, the change points of the seizure state are known up to a coarse approximation (based on the neurologist s assessment). A. Simulations We illustrate the application of our proposed estimation algorithm via a series of simulation studies. Careful attention was given to two implementation issues i.e. determination of the number of states and scalability of our estimation procedures to different connectivity network dimensions, using a realistic simulated multi-subject EEG data set. We shall emulate temporal changes in the directed dependence structure in multi-channel EEG signals, alternating between resting and task conditions in a block-design experimental paradigm with a multi-subject setting. We simulated a multi-subject EEG data set consisting of subjects and divided into conditiondependent time-blocks of piece-wise stationary signals, each with a fixed length of T B = 5 (total length of T = ). The data were generated from a regime-switching VAR() process with K = states, where the coefficient matrices of the two independent stationary VAR() processes characterize the effective connectivity between the EEG channels during the resting state and the task-activated state, respectively. To emulate the state-dependent recurring changes in the VAR connectivity structure, the successive time-blocks were generated according to the distinct coefficient matrices in a cyclic manner, alternating between the two connectivity states, following procedure in [] for functional connectivity. The noise covariance structures for the resting state and the active state were set respectively as R [] =.I and R [] =.5I. The entries of the VAR coefficient matrices for the subjects were drawn randomly from normal population distributions. To synthesize connectivity structures that reflect those in real EEG data, the population distributions for the VAR coefficients were estimated from the VAR fits of subjects from real EEG data sets [33] provided by Dr. S. C. Cramer (Neurology, UC Irvine): a resting-state and a motor-task data set, respectively for each connectivity state. The mean of the simulated VAR matrices over the subjects was computed as ground-truth for evaluation. We investigate the impact of increasing the network dimensions on the estimation performance, by varying the numbers of channels N from to with increments of. The locations of the channels were selected to cover regions of the motor and resting-state brain networks (N = : C3 and C4; N = 4: C3, C4, P3 and P4; N = 6: C3, C4, P3, P4, Fz and Pz; N = 8: C3, C4, P3, P4, Fz, Pz, F7 and F8; N = : C3, C4, P3, P4, Fz, Pz, F7, F8, P7 and P8). For each value of N, the multi-subject simulations were repeated times, giving a total of ( replications subjects) simulated time-courses. ) Estimation of number of states: We performed cluster analysis on the TV-VAR coefficient features to identify the discrete partitioning of effective connectivity regimes in the simulated data, generated from the SVAR model with known number of states K =. For each replication of simulations, we applied K-means clustering algorithm at the subject level to the sequence of TV-VAR coefficient vectors extracted from data of each subject using the Kalman filtering. Throughout the experiments, we used, as in [9], the L (Manhattan) distance which may be more effective for clustering high-dimensional data, compared to the L (Euclidean) distance. In practice, the number of quasi-stable states in the brain connectivity dynamics is typically unknown a priori, and needs to be determined either by the expert s judgment or estimation from data. In this simulation, we illustrate data-driven procedures to infer it via estimating the number of clusters in the TV-VAR features, using two cluster validity indexes: Copyright (c) 6 IEEE. Personal use of this material is permitted. However, permission to use

8 This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI.9/TBME , IEEE Transactions on Biomedical Engineering 7 N Variance Ratio Silhouette Width the variance ratio criterion of Calinski-Harabasz [34] and the silhouette width criterion of Kaufman-Rousseeuw [35]. [34] examined a criterion based on the ratio between the betweencluster variability to the within-cluster variability as a function of K. An alternative criterion uses the average silhouette width over the entire data, where the silhouette value of a data point measures its dissimilarity to points in its own cluster, as compared to the points of neighboring clusters. The optimal b is obtained by choosing among a range number of clusters, K of candidates K {,,..., Kmax } that maximize the stated criteria. We asses the ability of these criteria to recover the true K from the simulated data. Table I shows the estimation b over the simulated results in occurrences of selected K time courses, using the two criteria based on the subjectlevel clustering. The maximum number of states evaluated is Kmax = 8. It is clear that both criteria were capable of selecting the correct number of connectivity states K = in the data consistently for all dimensions N, as indicated by the highest occurrences of being selected (numbers in bold), though with higher precision by the variance ratio criterion. However, there is higher tendency of the criteria producing b and selecting over-fitted an over-dispersed distribution of K partitions for the lower dimensions, compared to the very high precision obtained for the higher dimensions (e.g., the b in probability of almost one to the true convergence of K K for N = 8 and N = ). This is probably due to added connectivity features arising from having more channels in the analysis that are discriminative between the two states. ) Effect of varying network dimensions: We studied the effect of different connectivity network sizes on the performance of various procedures for estimating the dynamic regimes, with the varied number of simulated channels. The K-means clustering and the Markov-SVAR model-based estimates for b [j] the state sequence Sbt and the VAR coefficient matrix Φ for the resting-state j = and the active-state j =, were computed as in the Algorithm for each value of N. Fig. (a) shows one realization of simulated EEG time courses for one subject, where we can see some form of changes over piecewise constant regimes, though rather vaguely. The estimated TV-VAR coefficients shown in Fig. (b), clearly capture the 5 5 Time Point 75 (a) TV-VAR Coefficients Criterion Selected number of states Time Point (b) True K-means States TABLE I F REQUENCY OF NUMBER OF STATES SELECTED BY VARIANCE RATIO AND SILHOUETTE WIDTH CRITERION OVER SIMULATED EEG TIME COURSES GENERATED FROM REGIME - SWITCHING VAR() PROCESSES WITH KNOWN K = STATES FOR DIFFERENT NUMBERS OF CHANNELS N. VAR Signals IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING SKF SKS 5 5 Time Point (c) Fig.. (a) One realization of simulated 6-channels of EEG data for a single subject, with VAR regime-switching between K = states at every 5 time points. (b) Sequence of TV-VAR coefficient matrices (lag one) estimated using KF (each vectorized with dimension N P = 6 = 36). (c) Ground-truth (red) and inferred state sequences by the K-means clustering bkm (black), switching KF, S bskf (blue) and of the TV-VAR coefficients, S t t bsks (light blue) based on a EM-estimated SVAR model. switching KS, S t non-stationarity of the dependence structure. Connectivity varies very slowly within a regime but changes abruptly across the regimes, as seen at every change points. Fig. (c) displays the tracking results of the regime changes via the estimated state sequence Sbt. Compared to the K-means approach, both the switching Kalman estimates follow the ground-truth state time course (red) more closely. We see that the SKS method provides accurate detection of the abrupt change points of each regime, and is able to follow the slow stationary behavior within a regime. In contrast, the K-means clustering suffers from gross change point misdetection and produces spurious abrupt regime changes in the stationary part, e.g. the short burst noises from time point t = 85 to t = 95. Moreover, the SKS using both past and future observations has natural ability to refine the SKF estimates by re-adjusting shifted change points, e.g. at t = 5 and smoothing the spurious regimes. To evaluate the connectivity regime change-point detection, we measured the accuracy or percentage of time points correctly classified to the true states for time-course of each subject in each replication. Fig. 3 plots the averages and standard deviations of the state classification accuracies by the different estimators over all subjects and all replications for increasing N. The performance of all methods improves as number of channels increases, which is similar to the results in Table I. The estimates SbtSKF and SbtSKS perform better than the K-means clustering, with substantially higher accuracy with Copyright (c) 6 IEEE. Personal use of this material is permitted. However, permission to use

9 Citation information: DOI.9/TBME , IEEE Transactions on Biomedical Engineering IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING 8 Accuracy (%) LS KM LS SKF LS SKS Number of channels Fig. 3. Accuracy of state classification of time-points obtained using K-means clustering, ŜKM t, switching KF, ŜSKF t and switching KS, ŜSKS t, as a function of number of channels N for the simulated multi-subject EEG data from a regime-switching VAR with K = states. Lines and error bars represent the averages and standard deviations over all subjects and all replications. Squared error LS KM LS SKF LS SKS EM Number of channels (a) lower standard deviations when N is small. To measure performance of the state-dependent connectivity estimators, for each repetition of the multi-subject simulations, we computed the total squared errors over all entries between the ground-truth and the estimated mean VAR connectivity matrices over subjects, Φ [j] Φ [j] F for each state j =,. H F = tr (H H) / denotes the Frobenius norm of matrix H. We compared the performance of various estimators by repeated-measures ANOVA tests, with their squared errors modeled as the outcomes of the ANOVA model. The differences between the error outcomes were tested via pairwise confidence intervals derived from the fitted ANOVA model. There is no significant difference in performance between a pair of estimators, if the computed confidence interval for the difference in their squared errors contains zero. The confidence intervals are adjusted using the Bonferroni s method for multiple comparisons at a global confidence level of 95%. Fig. 4 shows boxplots of estimation errors of various estimators of the group mean connectivity matrix over the replications, for the two states and for increasing N of the multi-subject data. The median and dispersion of the error distribution indicate respectively the accuracy (unbiasedness) and consistency of an estimator. The LS estimate based on the SKS-segmented regimes under the EM-updated SVAR model performs the best among all estimators, with significantly lower median squared errors and smaller variability of estimates for both state connectivity matrices and for all dimensions, generally. The results can be explained by the most accurate regime segmentation of using SKS as already shown in Fig. and Fig. 3. However, the SKS estimates perform comparably to the SKF for the active state, with no significant difference in errors as shown in Fig. 4(b). This is illuminated by the close state classification performance between the two, as in Fig. 3. Among the SVAR modelbased estimators, using direct EM estimates shows poorer performance, possibly because it relies on the average of timevarying statistics P t in () to approximate the time-invariant VAR parameters in each regime, which are more appropriately fitted by a stationary procedure using ordinary least-squares. Squared error LS KM LS SKF LS SKS EM Number of channels (b) Fig. 4. Boxplots of estimation errors in Frobenius norm of the K-means clustering and Markov SVAR model-based estimators for the group mean connectivity matrix Φ [j] Φ [j] F across subjects for (a) j = : restingstate and (b) j = : active-state on simulated EEG data with increasing number of channels N. The data blocks of fixed length T B = 5 was generated in a cyclic manner from two independent VAR() models inferred from real resting-state and motor-task EEG data. Each plot is based on replications of simulation. Horizontal braces and asterisks indicate significant difference in performance between pairs of estimators tested via ANOVA at confidence level of 95% with Bonferroni correction for multiple comparisons. Another reason may be the inherent asymptotic limitation of the conventional EM algorithm in handling estimation of very high-dimensional state-space models. We can also see that the estimation errors of all methods increase as N increases, due to growing number of VAR parameters to be fitted with fixed T. This trend, however, is less obvious for the resting state compared to the active state. Moreover, the errors and variances for the resting state are also higher than the active one. This biased and inconsistent estimates are probably due to the large variability in connectivity within and between subjects during rest when mental activity was unconstrained. This is in contrast to the active state with more coherent activation of the same brain regions across subjects, when induced by same motor task. Copyright (c) 6 IEEE. Personal use of this material is permitted. However, permission to use

10 States TV-VAR Coefficients fmri Series This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI.9/TBME , IEEE Transactions on Biomedical Engineering IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING 9 B. Analysis of Task-related fmri Data We further demonstrate the performance of our proposed framework in identifying the dynamic changes in effective connectivity regimes in a motor-task fmri dataset collected by Dr. S. C. Cramer and analyzed in [6]. It is based on a blockdesign experimental paradigm with alternating resting and task conditions, which is expected to induce piecewise stationary regimes of directed dependence between the fmri signals, switching between the two states. The data set has been used to study multi-subject variation in effective connectivity based on VAR in [6] which, however, has not explored the dynamic variability aspect of the connections. The data were acquired on a Philips Achieva 3.-Tesla scanner. Blood oxygenation level-dependent, BOLD functional images were obtained using a T-weighted gradient-echo planar imaging (EPI) sequence (TR/TE = /3 ms;; flip angle (FA) = 7 ; field of view (FOV) = ; voxel size = mm 3 ; number of slices = 3). The subjects performed hand grasp-release movements (active condition) with rest condition. The experiment session consisted of T = 48 fmri scans divided into four time-blocks of T B = scans, alternating between the two conditions, always starting with the rest condition. Refer to [6] for the details in the brain region parcellation and selection and data pre-processing. We analyzed directed connections between five main regions of interest (ROIs) associated with motor tasks as in [6]: primary motor cortex (M) (left & right), dorsal premotor cortex (PMd) (left & right) and a supplementary motor area (SMA). A mean signal was extracted by averaging voxel-wise fmri signals for each ROI. The VAR connectivity analysis was performed on the residuals of the signals as in [36], having regressed out drift, physiological noise and task-related changes in the mean levels of activation due to hemodynamic response, modeled by a general linear model (GLM) as the convolution of the stimulus function (known by experimental design) and a canonical HRF. For the resting-state fmri where explicit input functions are unknown, a recent blind deconvolution approach [37] using spontaneous pseudo-events to extract the HRF was applied to remove the confounding effect of hemodynamic response. Using Schwarz s Bayesian criterion, the optimal model order selected for this data is P =. Thus, we used a 5-dimensional TV-VAR() model to extract the time-varying connectivity features, and a twostate Markov SVAR() to estimate the temporal dynamics of the two brain states (rest & active) and directed dependencies between the 5 motor ROIs in each state. The fmri ROI time courses from a healthy subject, the estimated TVAR coefficients (vectorized) and state sequence by different procedures of the framework are shown in Fig. 5. In Fig. 5(b), we note that the directed connections between the ROIs are vary across pairs and also change over time. The evolving pattern shows that these changes tend to congregate into distinct piece-wise stable regimes according to states, i.e., time points t = and t = 5 36 for resting state; t = 3 4 and t = for active state. Compared to that of the task-activated state, the connections within the regimes of the resting state are sparser (fewer ROIs tend to be con- SKS True K-means SKF Time Point (a) Time Point (b) Time Point (c) Fig. 5. Estimation of temporal dynamics of effective connectivity (instantaneous and state-related changes) in a real fmri data. (a) BOLD fmri mean time-series of five motor-related ROIs from a healthy subject. (b) Sequence of TV-VAR coefficient matrices between the ROIs (lag one) estimated using KF (each vectorized with dimension N P = 5 = 5). (c) Groundtruth state sequences assumed known from the experimental designs (red) and estimates by the K-means clustering of the TV-VAR coefficients, ŜKM t (black), switching KF, Ŝt SKF (blue) and switching KS, Ŝt SKS (light blue) based on a EM-estimated SVAR() model with K = states. The true regime changepoints are indicated by vertical dotted lines. Accuracy (%) LS KM LS SKF LS SKS Time Point Fig. 6. Accuracy of state classification at each time point for fmri time-series of five motor ROIs, obtained using K-means clustering, ŜKM t, switching KF, Ŝt SKF and switching KS, ŜSKS t. The results are averages over all subjects. The ground-truth state sequence was assumed known by experimental designs with a change occurred at every time points (indicated by vertical dotted lines) alternating between the resting state and active state. nected) and have weaker strengths. Our model produces results that reconfirms the general findings of increased connectivity in the motor networks during tasks. More interestingly, our new method further reveals the temporal variability in fine structures not previously reported in [6]. The connectivity regimes shift with the transition of states. The state-dependent change points in connectivity regimes are more clearly represented by the state time course in Fig. 5(c), estimated by the proposed piecewise modeling approaches using the K-means clustering and the SVAR model. In agreement Copyright (c) 6 IEEE. Personal use of this material is permitted. However, permission to use

11 ACTIVE REST This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI.9/TBME , IEEE Transactions on Biomedical Engineering IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING TRUE LS-KM EM LS-SKF LS-SKS SMA LPMd LM RPMd RM SMA. LPMd LM. RPMd -. RM SMA LPMd LM RPMd RM (a) (a) SMA LPMd LM RPMd RM (b) SMA LPMd LM RPMd RM (c) SMA LPMd LM RPMd RM (d) SMA LPMd LM RPMd RM (e) (e) Fig. 7. Estimated mean VAR connectivity matrices between five motor brain regions from fmri data over healthy subjects during the resting and the active state, using different estimators of the proposed framework. (a) Ground truth. (b) LS estimates based on K-means regime segmentation, Φ LS-KM. (c)-(e) SVAR model-based estimates. (c) EM estimates, Φ EM. (d)-(e) LS estimates based on switching KF and KS segmentation, Φ LS-SKF and Φ LS-SKS. M: primary motor cortex (left and right), PMd: dorsal premotor cortex (left and right) and SMA: midline supplementary motor area. TABLE II COMPARISON OF ESTIMATION PERFORMANCE IN SQUARED ERRORS Φ [j] Φ [j] F BETWEEN K-MEANS CLUSTERING AND DIFFERENT SVAR MODEL-BASED ESTIMATORS ON FMRI DATA FOR DIRECTED CONNECTIVITY MATRICES Φ [R] AND Φ [A] BETWEEN FIVE MOTOR ROIS, DURING THE RESTING AND TASK-ACTIVATED BRAIN STATES. THE RESULTS ARE AVERAGES OVER HEALTHY SUBJECTS. LS-KM Markov SVAR Model EM LS-SKF LS-SKS Φ [R] Φ [A] Φ [R] Φ [A] Φ [R] Φ [A] Φ [R] Φ [A] with the simulation results, SKS performs the best compared to the SKF and K-means clustering, giving more accurate tracking of the state changes, with state assignment accuracy of more than 95%. It smooths out the spurious regimes (the blocks t = 5 7, t = 5 9 and t = 4 43 and the peak at t = 3 3) produced by the K-means clustering. It also refines the regime change points of the SKF e.g. at t = 36. Fig. 6 compares the multi-subject performance of different estimators in tracking the connectivity regime changes across time, by plotting the average state classification accuracy over the subjects, as a function of time. As expected from simulation results, the switching Kalman estimates clearly outperform the K-means clustering with the best performance by SKS, providing better tracking of both the smooth changes within stationary regimes and the abrupt changes in transition across different regimes, as indicated by the higher accuracy across time course. The K-means clustering based on the abruptly varying TV-VAR coefficients performs worse than the others with noisy estimates noticeably at the stationary regimes, despite that it gives marginally better detection than the SKF for the rapid state changes. We also note a decrease in performance of all three procedures in the vicinity of abrupt change points between two different states compared to within a stationary state, and in the active state compared to the resting state. Fig. 7 shows the estimation results of different procedures for the VAR coefficient matrices between the motor ROIs for the resting (top) and the task-activated state (bottom), averaged over subjects. We computed the least-squares estimates based on the regime boundaries provided by the experimental design as in Fig. 5(c) (top), as ground-truth for comparison (Fig. 7(a)). It is shown that the estimates based on switching KF and KS-segmented regimes with the EM-estimated SVAR model (Fig. 7(d) and Fig. 7(e)), give a better match to the true connectivity pattern overall than the other estimators, with Φ LS-SKS performing the best. On the other hand, the estimates based on the K-means segmentation (Fig. 7(b)) and the direct EM (Fig. 7(c)) have higher false positive rates during rest, incorrectly detecting connections which are actually absent. These results are well reflected by the averages of squared estimation errors over subjects in Table II, which demonstrates consistently the best performance of the estimates based on the SKS-segmented regimes using the EM-updated SVAR model. Among the SVAR-based estimators, the LS estimates relying on the switching Kalman segmentation are more accurate (with lower errors) than using the EM estimates directly. Fig. 8 shows the between-subject standard deviations of the VAR connectivity coefficients between the pairs of motor ROIs, estimated using the SKS procedure. Larger variability across subjects in effective connectivity of the motor network is observed during the active state compared to the resting state. C. Analysis of Epileptic EEG In this section, we apply our novel framework to analyzing state-related effective connectivity changes that can be associated with epileptic seizure recorded in EEG data. Seizure EEGs are realizations of a rapidly changing electrophysiological process resulting from abnormal firing behavior of network of neuronal sub-populations. These aberrations in neuronal electrical activity are expressed by fluctuating amplitudes of waveforms, changing spectral decompositions and evolving cross-dependence between channels. Growing evidence suggests network of epileptogenic regions, rather than a single focal source, contribute to the origination of the ictal state. Moreover, the onset and the duration of the seizure Copyright (c) 6 IEEE. Personal use of this material is permitted. However, permission to use

12 Citation information: DOI.9/TBME , IEEE Transactions on Biomedical Engineering IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING SMA LPMd TO LM RPMd RM SMA LPMd LM RPMd RM FROM Fig. 8. Inter-subject standard deviations of estimated connectivity coefficients between the five motor ROIs using the SKS procedure Φ LS-SKS for the restingstate (blue) and the active-state (red). are unknown a priori. Epileptogenic networks have been investigated in the aspects of functional connectivity for both static [38] and dynamic [39], and effective connectivity recently in [4] using window-based VAR. A complete analysis of the complex, synchronized ictal activity requires advanced nonstationary modeling tools. Our data-driven approach simultaneously allows for an automatic partitioning of the epileptic event in the EEG signals at different channels into distinct dynamic regimes, as well as estimation of a set of directed dependencies between the channels for each partition. We used a EEG data set recorded from a patient of Dr. Malow (neurologist at the University of Michigan) during an epileptic seizure, previously analyzed in [4]. The patient was diagnosed with left temporal lobe epilepsy. The data set consists of 9 bipolar recordings with sampling frequency Hz and time series length T = 5,. The data has been previously analyzed in [4] [43] relating to the timeevolving un-directed coherence. In this study, we analyzed the dynamic changes in the directed connectivity regimes driven by distinct brain states, between the EEG signals recorded at eight important channels over the frontal (F3, F4), central (C3, C4), parietal (P3, P4) and temporal (T3, T4) regions, before seizure onset and during an epileptic seizure. Here, we focus on single-subject analysis as the EEG monitoring for epilepsy is typically conducted in a subject-specific basis. For this data, both the variance ratio and silhouette width criterion based on the K-means clustering of TV-VAR coefficients suggested number of connectivity clusters present as K =, which are expected to correspond to the seizure and non-seizure state. We understand that there are possible micro-states even within the both states though. We employed a two-state Markov-SVAR model of order three to capture the dynamic connectivity for the normal and the seizure brain state. Use of a full observational noise covariance R may accommodate the confounding instantaneous correlation between channels induced by volume conduction [5], when analyzing scalp EEGs here. Besides, the fully-conditioned multivariate VAR model adopted here can better detect the synergetic effects in information flows than a pairwise analysis, as shown in [44] for the epileptic EEG. However, it may introduce redundancy where a group of sources share same information on a future target. A method combining the fully-conditioned with the pairwise approach as suggested by [44] could be incorporated as an extension of our framework in the future, to handle the redundancy and thus to provide a more effective analysis. Fig. 9(a) shows the TV-VAR coefficient matrices (vectorized) at different time-lags, estimated by the KF from the eight EEG channels. The results suggest that the directed connectivity between brain areas as measured from EEG also exhibits changes over time, instead of frequently-assumed stationarity [45]. The non-stationarity of coherence between these channels has also been revealed using other methods, in particular the SLEX [4], [4] and the FreSpeD method [46]. The SLEX is a tool for finding the optimal spectral representation of nonstationary signals from a library of local orthonormal Fourier waveforms. The FreSpeD is a novel approach that detects frequency specific change-points using a CUSUM-like statistic that essentially finds the time point that maximizes the contrast between adjacent time blocks. However, the TV-VAR features provide additional insights on the directionality of information flows between the brain regions. The estimates also indicate strongest connections at lag, followed by lags and 3. Despite the presence of changes in the finer time scale as shown in Fig. 9(a) (here with the resolution of. seconds), the directed connectivity tends to be clustered to quasi-stationary discrete states (normal and seizure) for long periods (e.g. the seizure event has a time-span of about one minute as in Fig. 9(c)), with relatively slow connectivity dynamics within each state. This is indicated by the slower variations in the estimated TV-VAR coefficients within each regime (normal, before seizure onset: -36sec; after seizure onset: 36-5sec), compared to the rapid transition between these regimes at 36sec. Besides, strengthened and more abrupt changes in connectivity can also be clearly seen during the seizure event with an onset around 36 sec, compared to the normal brain activity. This implies a regime shift in the directional dependence structure between the normal and the seizure state. Our result is a generalization in terms of connectivity and coherency, to that in [47] which also detected changes in spectra over locally-stationary segments in the preseizure EEG, using the auto-slex method on single-channels. Fig. 9(b) shows the inferred brain states at each point in time using different steps of the proposed framework. The data is fitted assuming two states which can be interpreted as normal (blue) and seizure (red). In consistency with the results in previous sections, both SVAR estimators, the SKF and SKS with EM updates, provide better tracking of the connectivity state dynamics, in terms of more accurate timestate assignments. In contrast, using only K-means clustering based on the TV-VAR coefficients yields noisy state estimates, Copyright (c) 6 IEEE. Personal use of this material is permitted. However, permission to use

13 This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI.9/TBME , IEEE Transactions on Biomedical Engineering IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING (a) states K-means Clustering states Switching VAR-KF states Switching VAR-KS Time (sec) (b) F3 F4 Channels C3 C4 P3 P4 T3 T Time (sec) (c) Fig. 9. Estimation of state-related dynamics of directed connectivity between eight main channels of a epileptic seizure EEG data. (a) Sequence of TVVAR coefficient matrices at different time-lags ` =,, 3 estimated using KF (vectorized with dimension P L = 8 3 = 9). (b) Inferred states (blue: non-ictal; red: ictal) at each time point by K-means clustering of the bkm (top); switching KF, S bskf (middle); and switching TV-VAR coefficients, S t t bsks (bottom). The SKF and SKS estimates are based on a EM-estimated KS, S t three-dimensional SVAR(3) model with K = states. (c) EEG data overlaid by final estimated states. particularly in the post-seizure intervals, because of a low SNR ratio in the time series of VAR coefficients. Moreover, the switching estimators are able to more precisely localize the change points in connectivity at the seizure onset, compared to the K-means clustering with a delayed detection of the onset. The improved fit can be explained by the more accurate model parameter estimates, updated iteratively by the EM algorithm, and the refined temporal boundaries of the states resulting from the SKS based on both past and future observations. Among the SVAR estimates, using the SKS improves the state estimates of SKF. This can be seen in the smoothing of some spurious estimates when using the SKF. However, the SKF detected abrupt change-points in directed connectivity which might be associated with the pre-seizure spikes, even before the seizure onset, as also reported in the coherence using the FreSpeD. Fig. 9(c) shows the EEG data overlaid by the estimated states by the SKS, which demonstrates the reasonably well detected temporal change points between states. This suggests that the seizure onset can be characterized not only by changes in channel-specific auto-spectrum and undirected coherence (as detected by the FreSpeD method), but also in the directionality of the connections between channels. Fig. shows the inferred directed connectivity matrices at three lags between the eight channels for the normal and seizure states. They are LS-fitted on the SKS regimes, using the SVAR(3) model estimated by the EM algorithm. Only connections significantly different from zeros are shown, tested based on the asymptotic normality in (), at α =.5 with Bonferroni correction. Note that the connectivity structures are substantially different between the two states, and across time lags. We found enhanced causal influences directed to T3 from the central regions (C3, C4) and the left parietal (P3) during the seizure state compared to the normal state, across lags, especially evident for lag. In particular, the strong connection of P3 and T3 is observed consistently at all lags. By detecting apparent increase of information inflows to T3 from many other regions during seizure, our procedure potentially localizes the left temporal lobe as the seizure onset zone, responsible for the generation of ictal activity. Besides, we can see that in both states the number of connections decreases at the longer lags. However, brain regions in the normal state are more densely connected due to the presence of various mental activities. In contrast, the connectivity in the seizure state are sparse (as obvious at lag and lag 3) but with stronger strength concentrated on regions affected by epilepsy. We computed the PDC from the estimated VAR coefficient matrices, and analyzed the band PDC averaged over the theta (4-7 Hz), alpha (8- Hz) and beta (- Hz) frequency band. The squared absolute PDC estimates between the eight channels at the different frequency bands for each brain state are shown in Fig.. The connectivity maps were plotted using the econnectome toolbox [48]. Contrasting with the non-ictal (normal) state, we can clearly see denser causal interactions with increase in PDC during seizure between regions which can be viewed as the epileptogenic zone, across all frequency bands. Our results are consistent with the general findings of enhanced EEG functional connectivity in epileptic brain networks accompanying the seizure onset, as already shown for alpha and beta band in [43] using the evolutionary undirected coherence analysis on the same data, and for the theta band in [49]. Our method, however, provides additional information on the directionality. Besides, it is obvious that the strongest directed connectivity occurred at the theta band, followed by the alpha and beta band. For the ictal state, it is interesting that the directed interactions between the channels at left hemisphere of the brain are stronger than that at the right hemisphere over all bands, suggesting a epileptic network ipsilateral to the seizure origin. In addition, we detected unidirected information flows at the theta band from T3 to both left and right central (T3 C3, T3 C4) and to both left and right frontals (T3 F3, T3 F4), and via F3 to the right frontal, right temporal and right parietal (F3 F4, F3 T4 and F3 P4). This suggests the rapid propagation of seizure activity across the brain hemispheres at theta oscillations from the left side to the right, with T3 and F3 respectively be the primary and secondary hub of the epileptic network. Copyright (c) 6 IEEE. Personal use of this material is permitted. However, permission to use

14 This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI.9/TBME , IEEE Transactions on Biomedical Engineering 3 Beta (- Hz) Alpha (8- Hz) Theta (4-7 Hz) IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING Fig.. Estimated VAR connectivity matrices (at three lags) between the eight epileptic EEG channels by LS fitting based on the SKS segmentation, b LS-SKS, ` =,, 3, for the non-ictal (left) and the ictal (right) brain state. Φ ` The connections shown are significantly different from zero at level α =.5 with Bonferroni correction for multiple testings. VI. C ONCLUSION We developed a novel general framework to estimate the time-evolving effective connectivity and brain state activity using the Markov-switching vector auto-regressive (SVAR) model. The SVAR model-based estimation enables us to achieve one of the important objectives in neuroscience studies, that it is ideally suited to identifying changes in connectivity structure of brain networks as characterized by the transition between distinct underlying brain states. Our framework offers a number of advantages over existing approaches. In particular, it is able to capture the state-dependent changes that are both abrupt (across different switching regimes) and smooth (within a quasi-stationary regime). Moreover, the framework is able to provide measures of dependence with directionality. We have also demonstrated the suitability of our method for analyzing different types of brain signals. In analyzing the fmri data, the method was able to capture the changes in the state which was linked to a change in the stimulus sequence, e.g., from rest to active stimulus. Thus, our proposed approach was able to estimate the state regimes (or temporal boundaries between states). Our method was also demonstrated to be suitable for analyzing epileptic seizure EEGs. It produced estimates of non-ictal and ictal periods which were consistent with previous findings [4], [4], [47]. Besides, this new method produced results on directionality of seizure propagation in an epileptic network at different frequencies based on the PDC between channels, never investigated in earlier studies. Future works will apply the proposed framework to analyzing Fig.. Topological maps of band-limited partial directed coherence between EEG channels at the theta (4-7 Hz), alpha (8- Hz) and beta (- Hz) frequency band for the non-ictal (left) and the ictal (right) brain state, as implied by the estimated VAR coefficient matrices in Fig.. Edges represents significant connections with squared absolute PDC greater than a threshold of.5, and arrows indicate the directionality of the connections. the more challenging resting-state neural data [8] [], [4], [6], where the connectivity dynamics are potentially more prominent due to the unconstrained mental activities during rest, with both unknown regime change points and the number of brain states. R EFERENCES [] B. He et al., Electrophysiological imaging of brain activity and connectivity-challenges and opportunities, IEEE Trans. Biomed. Eng., vol. 58, no. 7, pp ,. [] K. Friston, Functional and effective connectivity: A review, Brain Connectivity, vol., no., pp. 3 36,. [3] L. Harrison et al., Multivariate autoregressive modeling of fmri time series, NeuroImage, vol. 9, no. 4, pp , 3. [4] P. A. Valde s-sosa et al., Estimating brain functional connectivity with sparse multivariate autoregression, Philos. Trans. R. Soc. B, vol. 36, no. 457, pp , 5. [5] G. Deshpande et al., Assessing and compensating for zero-lag correlation effects in time-lagged Granger causality analysis of fmri, IEEE Trans. Biomed. Eng., vol. 57, no. 6, pp ,. [6] C. Gorrostieta, M. Fiecas, and H. Ombao et al., Hierarchical vector auto-regressive models and their applications to multi-subject effective connectivity, Frontiers Comput. Neurosci., vol. 7, 3. [7] C.-M. Ting et al., Estimating effective connectivity from fmri data using factor-based subspace autoregressive models, IEEE Sig. Process. Lett., vol., no. 6, pp , 4. [8] C. Chang and G. H. Glover, Time frequency dynamics of resting-state brain connectivity measured with fmri, NeuroImage, vol. 5, no., pp. 8 98,. [9] E. A. Allen et al., Tracking whole-brain connectivity dynamics in the resting state, Cerebral Cortex, p. bhs35,. [] R. M. Hutchison et al., Dynamic functional connectivity: Promise, issues, and interpretations, NeuroImage, vol. 8, pp , 3. Copyright (c) 6 IEEE. Personal use of this material is permitted. However, permission to use