Functional brain imaging: extracting temporal responses of multiple cortical areas from multi-focal visual evoked potentials. Shahram Dastmalchi

Transcription

1 Functional brain imaging: extracting temporal responses of multiple cortical areas from multi-focal visual evoked potentials. By Shahram Dastmalchi B.S. (University of California, Davis) 99 A dissertation submitted in partial satisfaction of the requirements for the degree of Doctor of Philosophy in Vision Science in the GRADUATE DIVISION of the UNIVERSITY OF CALIFORNIA, BERKELEY Committee in charge: Professor Stanley A. Klein, Chair Professor Gunilla Haegerstrom-Portnoy Professor Walter J. Freeman Dr. Thom Carney Spring 3

2 Table of Contents Introduction: Functional Brain Imaging... Technology overview... Thesis overview References... 7 Chapter : Signal-to-Noise Optimized Laplacian Derivation... 9 Introduction... Theoretical formulations Laplacian estimation Estimation of the Laplacian weighting function Matrix inversion Adaptive Laplacian-like Derivation (ALLD) Simulations Signal-to-noise ratio (SNR) calculation Simulating random electrode montages... 4 Results & discussion Signal-to-noise benefit of using SL-V i instead of SL- V Accounting for noisy electrodes Adaptive Laplacian-like Derivation (ALLD) Optimal number of electrodes in a montage Conclusion Appendix... 4

3 6. Proof of ALLD Proof of Variance of the Laplacian References Chapter : Multi-Source Analysis Simulations Introduction Theory and method Scalp potential Rotation problem Constraining the Rotation Space Penalty function: the Ribbon Metric Source analysis algorithm Simulation Results & discussion Rotation in the absence of noise Rotation in the presence of spatial and temporal white noise Rotation in the presence of model misspecification Rotation in the presence of spatially and temporally correlated noise 3.5 Summary of discussions Conclusion References... 4 Chapter 3: Source Analysis of Visual Evoked Potentials... 7 Introduction... 9 Material and methods... 33

4 . VEP data collection Source analysis Validation of dipole solutions Results and disscusion The sub-metrics The settings used in calculating sub-metrics and Ribbon Metrics Are dipoles one and two modeling V and V sources? Temporal components Conclusion Appendix References... 64

5 INTRODUCTION: FUNCTIONAL BRAIN IMAGING

6 TECHNOLOGY OVERVIEW Rapid advances in functional brain imaging technologies are paving the way for understanding neuronal mechanisms of human perception and cognition. The phrase functional brain imaging is used to distinguish between methods used to study neural correlates of brain function from those used to visualize brain anatomy. Commonly used functional brain imaging technologies fall into two categories: ) those that indirectly measure neural activity by detecting the resulting hemodynamic changes such as positron emission tomography (PET), functional magnetic resonance imaging (fmri), and transcranial optical imaging, and ) those that detect neural activity by measuring the resulting electrical and magnetic fluxes outside the head such as electroencephalography (EEG) and magnetoencephalography (MEG). fmri is today s most widely used functional brain imaging tool because the MRI magnet is ubiquitous in the clinical setting. fmri provides a 3-dimensional image of brain activity with spatial resolution as high as -3 mm (Baillet ). The drawback of fmri is its limited temporal resolution of more than second due to slow hemodynamic response. Temporal resolution in the millisecond range is needed in order to extract timing information about electrical events at different cortical regions. EEG and MEG (collectively referred to as E/MEG) have high temporal resolution. The typical E/MEG sampling on the order of 6 Hz is well above the range required to record E/MEG waveforms of interest, which are typically in the to Hz range. The challenge associated with the use of E/MEG to study neural correlates of cognition is to reliably decompose the spatio-temporal signal into its underlying source components. The E/MEG signal, which is recorded on (EEG) or above (MEG) the

7 surface of the head, provides a blurred spatial image of brain activity therefore making it difficult to discriminate between the individual underlying neural generators. Inverse solutions are used to create a spatio-temporal image of the activity inside the head. However, because of their inherent ambiguity and the present lack of accurate head models the inverse solutions can contain insidious spatial and temporal errors (Mosher 993; Supek 993; Zhang 994; Jewett 995). This unreliability in E/MEG based spatial and temporal imaging of brain activity has created a general mistrust of its use and adoption by researchers. Spatio-temporal imaging of neural generators in the brain via inverse solutions of E/MEG data requires having forward models that map the neural generator activities to the measured electromagnetic signal. Although there are various forward models of varying accuracy in use today, the common denominator for all models is the equivalent current dipole. An equivalent dipole models the electrical activities of a large pool of pyramidal neurons in the cortex. It is believed that the dendritic trunks of the pyramidal neurons, which are oriented perpendicular to the cortical surface, are the major contributors to the E/MEG signals therefore making it possible to assume that the dipoles are roughly perpendicular to the cortical surface (Scherg 99). It is standard to represent the spatio-temporal E/MEG signal as V=E(p)T. E is a matrix that maps dipole spatial-parameters, p, to the signal at every sensor. T is a time series which represents the temporal component of each dipole. Each column of E and row of T represent the contribution of a single dipole to the spatio-temporal potential/magnetic field. An inverse solution can be used to estimate p and T for every given spatio-temporal potential/magnetic field. There are two general approaches for 3

8 obtaining inverse solutions of E/MEG signals: ) the linear approach and ) the nonlinear approach. The linear approach, which is also referred to as the imaging approach or minimum norm estimation, reduces the inverse problem to a linear one by placing a dense set of dipoles evenly distributed throughout the head model. By fixing the dipole locations, which are the only non-linear parameters, the problem can be solved using minimum norm estimation (Gorodnitsky 99; Liu 998; Koles 998). One of the advantages of the linear approach is that no a priori assumption regarding the number of active brain regions needs to be made. One of the key problems with the linear approach is the existence of a large number of solutions explaining the same data. This degeneracy in the solutions is due to the equations being hugely underdetermined. The number of sensors is typically in the order of where the number of unknowns (dipole parameters) is in the order of,. Many statistical, anatomical and physiological constraints have been used in order to limit the solution space but the linear inverse solutions remain largely unreliable (Baillet ). Furthermore, the application of the linear approach in estimating the sources of visual evoked potential has been tested in simulations and found to be less reliable than the nonlinear approach (Stenbacka ). The framework of the linear approach lends itself quite well to using anatomical constraints from other imaging technologies. However, studies using fmri loci to constrain the solution space have reported mixed results regarding the reliability of the solutions (Liu 998; Gonzalez ). The fact that not all active regions contributing to E/MEG signal produce fmri activity can lead to erroneous solutions (Baillet ). There are various non-linear techniques for solving the inverse problem. The two most well known nonlinear techniques include dipole source analysis/localization and 4

9 multiple signal classification (Music). Music in spite of its suggestive name has been shown to provide erroneous solutions when there are multiple spatially and temporally correlated sources active (Zhang 994). The most widely used methodology is dipole source analysis, whose popularity benefits from commercially available software applications: BESA ( and EMSE ( Dipole source analysis involves estimating the nonlinear dipole parameters by iteratively searching for dipole parameters that minimize the sum of squared errors between measured and calculated spatio-temporal signal ( Vdata-Vmodel ). Dipole source analysis as with all other aforementioned methods suffers from crosstalk between multiple dipoles leading to insidious errors when sources are spatially close (Zhang 994). This thesis is concerned with analyzing visual evoked potentials, which consist of spatially correlated sources as dictated by the closely positioned visual cortical regions. The sources are also likely to be somewhat temporally correlated because of the existence of feedforward and feedback loops between the different cortical regions. Constraints have to be placed on the solution space in order to mitigate the crosstalk problem. Anatomically based constraints have been used in source analysis of auditory and visual evoked responses (Scherg 99; Clark 995) but no systematic attempt was made to validate the solutions; therefore it is unclear if the solutions were improved or not. 5

10 THESIS OVERVIEW The objective of this thesis is to help advance the field of E/MEG based functional brain imaging particularly as applied to the early visual cortex. The first chapter describes several ways of improving the signal-to-noise ratio of the local Laplacian derivation. The Laplacian derivation, which is a second derivative estimation of the surface potentials, has been shown to improve multi-source analysis (Klein 995). The second chapter describes and tests via simulations a new method for extracting the sources of early visual areas. The new method is based on first defining a signal subspace of interest which is then scanned for anatomically and physiologically plausible solutions. The constraints are based on continuity of solutions between neighboring dipoles within the first and second visual cortical areas (V and V). The biological constraints are built into the dipole source analysis framework; however it is possible to generalize some aspects of the constraints for use with other E/MEG analysis techniques. The third chapter applies the methodology described in the second chapter to real evoked potential data collected using multi-focal recording. It is my belief that E/MEG based approaches will in the future play an important role in multi-modal functional brain imaging. The field of human cognitive research is being fueled by the success of fmri technology, which in turn is fueling the growth of complementary technologies. The growing interests in E/MEG inverse solutions coupled with the rapid improvements in computer processing speed and consequently in the mathematical modeling of nonlinear systems will undoubtedly improve its reliability and contribute to its adoption. 6

11 3 REFERENCES Clark, V.P., Fan, S., and Hillyard, S.A. Identification of early visual evoked potential generators by retinotopic and topographic analysis. Human Brain Mapping, 995, : Gonzalez A.S., Blanke, O., Lantz, G., Thut, G., and Drave de Peralta Menendez, R. The use of function constraints for the neuroelectromagnetic inverse problem: alternatives and caveats. International Journal of Bioelectromagnetism,, 3: -6. Gorodnitsky, I.F., and Rao, B.D. Source localization in MEG using an iterative weighted minimum norm algorithm. IEEE, 99, 9: Jewett D., Zhang, Z. Multiple-generator errors are unavoidable under model misspecification. Electroencephalography Clin. Neurophysiology, 995, 95: Klein, A. S. and Carney, T. The usefulness of the Laplacian in principal component analysis and dipole source localization. Brain Topography, 995, 8: 9-8. Koles, Z.J. Trends in EEG source localization. Electroencephalography and clinical Neurophysiology, 998, 6: Liu, A.K., Belliveau, J.W., and Dale, A.M. Spatio-temporal imaging of human brain activity using functional MRI constrained MEG data: Monte Carlo simulations. Proc. Natl. Acad. Sci., 998, 95: Luck, S.J., Hillyard, S.A., Mouloua, M., Woldorff, M.G., Clark, V.P., and Hawkins, H.L. Effects of spatial cueing on luminance delectability; psychophysical and electrophysiological evidence for early selection. Journal of Exp. Psych: Human Perception and Performance, 994, : Mosher, J. C., Spencer, M. E., Leahy, R. M., and Lewis, P. S. Error bounds for MEG and EEG source localization. Electroenceph. and Clin. Neurophys. 993, 86: Nunez, P. Electric Fields of the Brain: The Neurophysics of EEG. Oxford University Press, New York, 98. Scherg, M., and, Berg, P. Use of prior knowledge in brain electromagnetic source analysis. Brain Topography, 99, 4: Stenbacka, L., Vanni, S., Kimmo, U., and Hari, R. Comparison of minimum current estimate and dipole modeling in analysis of simulated activity in human visual cortexes. NeuroImage,, 6: Supek, S., and Aine, C. J. Simulation studies of multiple dipole neuromagnetic source localization: Model order and limits of source resolution. IEEE transactions on 7

12 Biomed. Engineering, 993, 4: Zhang, Z., and Jewett, D. DSL and music under model misspecification and noiseconditions. Brain Topography, 994, 7:

13 CHAPTER : SIGNAL-TO- NOISE OPTIMIZED LAPLACIAN DERIVATION Summary Laplacian derivation is often used as a spatial filter to enhance scalp potential data. There are two distinct approaches for estimating the Laplacian of the voltage field: ) a global approach and ) a local approach. In practice, most local approaches first determine a weighting function for every local group of electrodes, and then use it to operate on the raw voltages to estimate the surface Laplacian. This chapter describes three new techniques for determining signal-to-noise optimized weighting functions. The first method involves making a subtle but important change in a previously described method of Laplacian derivation. The second method makes use of the covariance term in the Weighted Least Square procedure to minimize noise contribution by noisy electrodes. The first two techniques make the Laplacian operator more robust to noise without abandoning its symmetry constraint. The third technique is a Laplacian-like derivation 9

14 that provides higher signal-to-noise ratio than the other techniques at the expense of the symmetry constraint. Under certain electrode configurations, the third technique causes systematic error and consequently distorts the voltage field. However, for the purpose of dipole source analysis it is more important to increase the signal-to-noise ratio than to preserve the shape of the voltage field since the distortion would be accounted for by the forward solution. The third technique has also an added property that it can be tuned, much like a Wiener filter, for a given voltage model and noise distribution, hence the name Adaptive Laplacian-like Derivative (ALLD).

15 INTRODUCTION The neural activity in human body results in electrical potentials that can be measured on the skin. For example, electrocardiogram is used to measure heart activity and electroencephalography is used to measure brain activity. Electroencephalography is an important tool for studying brain function. The electrical activity on the scalp holds information about the timing and location of the underlying activities. However, potentials recorded on the scalp are low pass filtered due to the underlying conductive tissue and therefore it is useful to correct for the blurring. Increase in the spatial resolution of the measured scalp potentials can provide a better estimation of the location of the underlying neural activity than the raw potentials. One of the most commonly used filtering techniques is the Laplacian operator. The surface Laplacian improves the spatial resolution of scalp potential topography by reducing the common activities between neighboring electrodes. Furthermore, the surface Laplacian has been shown to improve dipole source analysis (SA). SA is a technique used to spatio-temporally map individual sources of brain activity. The Laplacian operation improves SA by reducing the bias caused by distant sources or volume conductor misspecification (Klein 995). The Laplacian operator, by reducing slow varying potentials, augments local sources therefore increasing their contribution to a source analysis procedure. In other words, spatially wide sources as result of far-field dipoles or model misspecification are filtered out before significantly biasing the source analysis procedure. Laplacian estimation of the EEG data falls into two general categories: ) a local Laplacian operation in which only the data from the neighboring electrodes are used in

16 the estimation (Hjorth 975 & 98, Katznelson 98, Gevins 989, Huiskamp 99, Le 994, Wang 999) and ) a global approach in which an interpolation function such as a spline is used to model the voltage topography over the whole electrode array and then a second derivative operation is applied directly to the function (Perrin 987, Nunez 989, Law 99). They each have their strengths and weaknesses. The spline Laplacian produces a current source density map that is useful for visualization purposes since it provides a continuous map. It also spatially smooths out the voltage topography making it more robust against high frequency noise contamination due to the Laplacian s inherent high-pass characteristic. However, there are two major concerns with using the global Laplacian method for dipole source analysis procedures: ) spatial smoothing may wash out the contribution from small dipole sources which one is trying to localize; ) assigning high weights to distant electrodes can result in biases from far away sources (Biggins 99; Fein 99). For these reasons, a local technique for estimating the surface Laplacian is preferred for source analysis. Local estimation of the surface Laplacian at an electrode is fairly straightforward when using an electrode montage with a symmetric center-surround configuration (Hjorth 975). For example in a five-electrode montage with the surrounding electrodes at a distance of cm from the center electrode and 9 degrees apart from the adjacent electrodes (square montage), the Laplacian is calculated by subtracting a weighted sum of the voltages at the center electrodes from the voltage at the center electrode, Laplacian = V.5( V + V + V3 + V4 ). V is the voltage at center electrode, V -4 are the voltages at the surrounding electrodes and.5 is the weighting used for calculating the sum. In a case where the square

17 montage is squeezed along in the y-axis such that the electrodes above and below the center electrodes are at a distance of cm from the center, the weighting of.5 is no longer appropriate. The quarter weighting would result in a Laplacian estimate that would be biased toward the x-axis and therefore would not be a true Laplacian. A true Laplacian estimation should be balanced in all directions. Therefore when the radial symmetry is broken, the Laplacian is often estimated by solving a system of linear equations (Gevins 989; Greer 989, Huiskamp 99, Wang 999). This chapter presents several techniques for significantly improving the signal-tonoise ratio (SNR) performance of an existing surface Laplacian derivation (Gevins 989, Greer 989, Huiskamp 99). The first technique involves an important modification in the formulation that makes the DC term a free parameter. The second technique involves using a priori knowledge of electrode noisiness to improve the SNR of the Laplacian estimation. Both techniques are in compliance with the true definition of Laplacian operator. The third technique is a novel method for estimating a Laplacian-like derivation that adaptively optimizes SNR for any given voltage generator model. This method, which called Adaptive Laplacian-like Derivation (ALLD), is suited for use with source analysis procedure. Even though ALLD is not a true Laplacian, it shares a critical characteristic that makes the local Laplacian Derivation advantageous for use in source analysis procedures. ALLD reduces the contribution of spatially slow varying potentials that are associated with distant sources. The use of the word Laplacian from here forward refers to a local surface Laplacian operator. 3

18 THEORETICAL FORMULATIONS. Laplacian estimation The Laplacian of a planar potential field V(x,y), can be calculated by using a twodimensional second derivative operator, L V V V ( x, y) = + x y =. () In the case of scalp potentials, V is a function of a three dimensional space, (X, Y, Z). Since the distances of the nearest electrodes are small in relation to the curvature of the head, a planar geometry, (x,y), can be assumed as a good approximation of the local surface. To achieve this, a local electrode montage is projected onto a best fitting plane (Wang 999) such that the center in the montage is at the origin of the planar coordinate system. The projection of a local montage is repeated at regular intervals (or at each electrode) until all electrodes are included in the coverage. The center of each local montage becomes the point at which the Laplacian is evaluated. In practice the Laplacian at an electrode can be numerically estimated by the weighted sum of potentials at that electrode and its neighboring surround. n L = w V i i () i= o L is the Laplacian at the center electrode e. The summation index i is from through n, the number of neighboring electrodes. w i is a weighting function and it can be calculated independent of the voltage field. This is important since w i can be calculated for all the electrodes in a recording cap and arranged in a matrix format. Following data collection 4

19 the Laplacian can be estimated by taking a dot product of the weighting and voltage matrixes.. Estimation of the Laplacian weighting function Given a montage of electrodes with the center electrode, e, placed at the origin surrounded with n number of electrodes, the voltage at each electrode, V i, can be estimated by a truncated two-dimensional Taylor series (Huiscamp 99; Gevins 989, Greer 989): ) ( 4 ) ( 4 e y x i i e y x i i e xy i i e y i e x i i y V x V y x y V x V y x y x V y x y V y x V x V V (3) Equation 3 is similar to the work by the investigators with the exception that the last two terms are algebraically rearranged such that the partial derivatives in the last term takes the form of the Laplacian operator. Note that the DC term,v, of the Taylor expansion is on the left side of the equation. For the purpose of simplicity, V and the partial derivatives of V in the first four terms of the above expression are replaced with the notations v i and the partial derivative in the Laplacian term is replaced with L. ) ( 4 ) ( 4 ) ( 4 3 e e e e i i i i i i i e i i y x L y x y x y x V ν ν ν ν ν (4) With n surround electrodes, equation 4 can be represented in a matrix format: 5

20 + + + o n o o n n n n n n n n V V V L y x y x y x y x y x y x y x y x y x y x y x y x ν ν ν ν ν ν ν : : 4 ) / ( 4 ) / ( : : : : : : : : : : 4 ) / ( 4 ) / ( 4 ) / ( 4 ) / ( 4 3 (5) or V Md =. (6) Once M is inverted using various methods described in the next section, the Laplacian weighting function, w i, of equation is equal to the last row of M inverse, inverse i M lastrow w =. (7) For reasons discussed later, having the DC term, v, on the left hand side of equation 4 has signal-to-noise disadvantages when using the method of Least Squares to invert M. Adding v to both sides of equation 4 results in the following matrix formulation: n n n n n n n n n V V V L y x y x y x y x y x y x y x y x y x y x y x y x... 4 / ) ( 4 / ) ( / ) ( 4 / ) ( 4 / ) ( 4 / ) ( 4 3 ν ν ν ν ν (8) Wang et al (999) also used a similar formulation but they did not point out the difference between this formulation and the other formulation given by equation 5. The Laplacian weighting function is found similarly to the way described for equation 5. The Laplacian estimations using equations 5 and 8 are referred to as SL-V i (Surface Laplacian using V ) and SL- V (Surface Laplacian using V) methods, respectively. 6

21 .3 Matrix inversion A unique solution for L requires that there are at least 5 surrounding electrodes for both SL-V i and SL- V methods. In the case of SL- V, there are five unknowns, v -4 and L, therefore five equations are needed to perform a matrix inversion. In the case of SL- V i, there are six unknowns, v -4 and L, however there is no need for additional electrodes since the center electrode makes the total number of equations equal to six. In the case with n greater than 5, where both equations are over-determined, a typical least square solution can be found by computing a pseudo-inverse of M (Seber 977), M pseudo inverse = ( M T M ) M T (9) If a priori estimation of spatial noise at every electrode is available, equation 9 can be expressed as a Weighted Least Squares (WLS) estimation (Seber 977), M pseudo inverse T T = ( M C M ) M C. () C is a covariance matrix with its diagonal describing the variance of the noise density distribution of potential data collected from each electrode, C = ii where, var( V i ) () V = V + i ideal noise i. () If the diagonal entries in C are ones and all other off-diagonal entries are zeros then equation reduces into equation 9. This formulation has the added property that it weights the solutions according to electrode noise (variance). In other words in the event that some electrodes are noisier than others then their contribution to the Laplacian estimation would be reduced. 7

22 In the underdetermined case where n is less than five, there is a family of solutions. In this case, Singular Value Decomposition (SVD) procedure can be used to find a solution, [ ] T i inverse Pseudo U diag V M ) / ( Σ =. () V and U are the decomposed row and column orthonormal matrices, respectively. Σ is a diagonal matrix of eigenvalues associated with V and U. SVD attempts to reduce the solution space by finding the true rank of M and keeping principal components with significant eigenvalues. To perform SVD an existing Matlab function was used. Detailed discussion and implementation of this technique can be found in the Numerical Recipes (Press 99)..4 Adaptive Laplacian-like Derivation (ALLD) Described here is a novel method called Adaptive Laplacian-like Derivation (ALLD) with significant signal-to-noise benefits. One of the key characteristics of ALLD is that its filtering properties can be tuned for a given voltage model, hence the word adaptive. Another key characteristic of ALLD is that it is less constrained than a true Laplacian derivation. For reasons discussed later in this chapter, we find that, under certain voltage field conditions, truncating the cross-product term, xy, and the difference term, x - y, from equation has a signal-to-noise advantage. ALLD has the following generalized formulation where f(x, y) is the voltage model for which the filter is tuned: n o n n n n V V V N y x f y x N y x f y x N y x f y x : : ), ( : : : : : : : : ), ( ), ( λ ν ν ν. () 8

23 where N is a normalization factor that forces f ( x, y) =. ALLD estimate, λ, is x=, y = calculated in the same manner as solving for L in equation 8 with the exception that the matrix in equation is invertible for a case of 4 electrodes. In a case of 5 or more surrounding electrodes a solution is reached using the least squares. It is interesting to note that for both ALLD and AL-V i methods, where the v term is included as a free parameter, there is no longer a dependency for having a center-surround electrode configuration. In fact, the Laplacian(-like) estimation can be performed for any coordinate within the montage. Estimating ALLD weightings using the Weighted Least Squares procedure described above is effectively the same as solving for constraints (see Appendix for proof): w i that satisfies the following n n n wi = i wi xi = i w y = i i i i= o, i= o, i= o, and, max imize λ w i var( noise i ), () where var( noise ) i is the variance of spatial noise at each electrode. The function that is being maximized is shown in the next section to be the description of the signal-to-noise ratio for the Laplacian estimation. An alternative implementation of ALLD, which we used to confirm our results, is to use the above constraints in an iterative minimization procedure. Note that the normalization factor, N, is ignored in the above constraints since it bears no significance in the discussion. 9

24 3 SIMULATIONS 3. Signal-to-noise ratio (SNR) calculation Signal-to-noise ratio (SNR) was used as the main criterion for evaluating the performance of the different methods. Two distinct test functions were used to perform different SNR analysis ) a two dimensional Parabola, V ( x + y ) / s dimensional Gaussian, V = e. G x P = + y, and ) a two The noise is defined as the standard deviation (square root of the variance) of the Laplacian voltage. If the noise at each electrode, noise i, is uncorrelated then the Laplacian noise is estimated by (see appendix for proof): σ Laplacian = w i var( noise i ). (3) Therefore, SNR can be expressed as, signal = Lap SNR σ Laplacian wi var( noisei ), (4) where Lap is the Laplacian(-like) estimation. 3. Simulating random electrode montages In some cases, the benefits of the Laplacian derivation modifications are not obvious until an electrode montage begins to deviate from checkerboard or radially symmetric configurations, in which case Monte-Carlo simulations were used to calculate a grand average SNR (GA-SNR) over a large number of randomly generated electrode positions. As depicted in figure, the positions of the surrounding electrodes in a given

25 montage were deviated using normally distributed random numbers with a standard deviation of.5 to cm. The SNR was calculated using equation (4) and the process was iterated a thousand times over which the SNR is averaged. SNR in this chapter, unless specified as grand average, refers to a one-time signal-to-noise calculation for a given montage using equation 4. Figure Randomly changing position of surrounding electrodes to create different electrode configurations.

26 4 RESULTS & DISCUSSION 4. Signal-to-noise benefit of using SL-V i instead of SL- V Moving the term v, which accounts for voltage at the center electrode, in equation 4 from left to the right side may at first seem to be inconsequential to the Laplacian estimation. In fact the SL- V (surface Laplacian using delta voltage) or SL-V i (surface Laplacian using raw voltage) methods referring to using equation 5 or 8, respectively, result in the same Laplacian weighting estimation when the number of surrounding electrodes, n, is equal to five or less. However the two methods start differing when n is larger than five and the method of Least Squares is used to invert the matrix in equation 5 or 8. Figure compares the weightings and signal-to-noise ratios (SNR) for a nineelectrode checkerboard montage. The weightings for A and B are calculated using SL- V and SL-V i methods, respectively. The SNR is calculated by dividing the Laplacian voltage by the standard deviation of noise (see methods). A parabolic (parabola-shaped) voltage field (see methods) was used as a test function. The true Laplacian of this test voltage field is equal to regardless of the size and shape of the electrode montage. The size and shape invariant property makes it simple to compare SNR calculations between montages of varying number of electrode and size used in other sections. Both SL- V and SL-V i, because of the quadratic terms in the Taylor expansion, result in weightings that are proportional to inverse of squared distance ( d ) from the center electrode. When using a parabolic test function, the distance dependence of the two methods is

27 offset by the quadratic increase of the test voltage as a function of distance from the center. A. five terms B. 4 4 six terms (.6) SNR: SNR: 4. Figure Laplacian weightings at each electrode in a nine-electrode montage (n=8). The weightings for A and B were calculated using the five-term and six-term methods, respectively. The signal-to-noise ratios are below each figure. The weighting for center electrode in A is not specified by the five-term method. The number in the parentheses represents the negative of the sum of all other electrode weightings. The standard deviation of noise in the Laplacian calculation has a simple relation to the Laplacian weightings, σ Laplacian N = w σ, (5) i= o i i when noise at each electrode, σ i, is uncorrelated. The SL-V i method yields significantly improved SNR over the SL- V method. This improvement in SNR can easily be demonstrated by assuming σ = in which case is equal to sum of the squared Laplacian weightings. It is easy to see that i σ Laplacian σ Laplacian would be reduced when absolute values of the weightings are close to one another, or in other words, well balanced. Alternatively, one possible intuitive explanation is that the weighting function in Figure 3

28 B takes a weighted average of the voltages at 5 electrodes and not just the center electrode. This averaging helps reduce the propagation of center electrode noise. Making the center electrode noisier than others further emphasizes this point. Figure 3 shows the percent SNR improvement of SL-V i over SL- V, as a function of center electrode noise ( σ ). The SNR s were calculated using the same weighting functions as in figures A and B. The results show that the SL-V i method for Laplacian estimation is particularly robust in a montage where the center electrode is noisy. This beneficial property of the SL-V i method comes at a cost. A well-balanced % SNR Improvement 5% % 5% % 5% % Electrode noise (sigma) Figure 3 Percent SNR improvement of SL-V i over SL- V as the center electrode noise (σ in equation 5) is increased. weighting function means all electrodes are equally susceptible to noise and therefore a noisy surround electrode can be more costly than with the SL- V method, which assigns lower weightings to the surrounding electrodes. However, there is a general solution described in the next section for reducing the effect of noisy electrodes. 4

29 4. Accounting for noisy electrodes Noise level is often not homogenous across all electrodes in a montage. In fact, some electrodes can become quite noisy in a long recording session as the recording gel dries up and the impedance increases. Using the Weighted Least Squares (WLS) formulation, this section presents a way for SNR optimization of the Laplacian by taking into account the level of noise at each electrode. WLS introduces a covariance term, C, (see methods) where the diagonal entries in C are the variance of noise at each electrode and all off-diagonal entries are assumed zero. The off-diagonal entries, which represent the covariance of the voltage field at the electrodes, are assumed zero since there is no feasible model that can be used for correlated noise. Figure 4 shows the Laplacian weightings and SNRs for two scenarios where one electrode is four times as noisy as other electrodes. In Figures 4 A and B the covariance term is used to account for the noisiness of a surround electrode where in Figures 4 C and D is not. The results show that using WLS to account for electrode noise can significantly improve SNR for both the SL- V and SL-V i methods of calculating the Laplacian. WLS reduces the contribution of a noisy electrode by simply reducing its Laplacian weighting. Figures 4 E, F, G and H are making similar comparisons except that in this scenario it is the center electrode that is four times noisier. Note that the SL- V method does not show any SNR improvement even when using WLS to account for the center electrode noise. The reason is that the DC component, v, is not used as a free parameter in the SL- V method and therefore its contribution cannot be adjusted using WLS. This further emphasizes the disadvantage of using the SL- V method and its susceptibility to error propagation from the center electrode. Furthermore, if the value of the entry into the 5

30 covariance matrix that accounts for the variance of the center electrode is zero, the SL-V i method results in the Laplacian weightings that are identical to those by the SL- V method (data not shown). This result indicates that the SL- V method implicitly assumes that the center electrode voltage measurement has zero variance, which is not a reasonable assumption under real experimental conditions. Using WLS along with the SL-V i term method results, for all practical noise distributions and montage scenarios, in better SNRs than the more commonly used SL- V method (data not shown). 6

31 A SL- V B SL-V i Ordinary Least Squares * C - SNR:.9897 (-/.66895) * D - SNR: (-/.595) * * Weighted Lease Squares (WLS) * SNR: (-/.66) - SNR: (-/.534) E F Ordinary Least Squares * G - SNR:.8974 (-/.44) - SNR:.4446 (-/.3844) *3 * H Weighted Lease Squares (WLS) SNR:.8974 (-/.44) * SNR:.8638 (-/.69838) Figure 4 Laplacian weightings and SNRs calculated using Weighted Least Squares (WLS) to optimize for electrode noise. A through D represent a scenario where the middle left electrode is 4 times noisier (filled circles). C and D make use of the Covariance matrix to optimize for noise while A and B do not. E through H represents a scenario where the center electrode is 4 times noisier (filled circles). G and H make use of the Covariance matrix to optimize for noise and E and F do not. * points out SNR improvement by using SL-V i over SL- V method. * points out improvement when using the WLS to lessen the contribution of noisy electrodes. *3 points out that using WLS for SL- V method has no signal-to-noise benefit when the center electrode is noisy. *4 points out the susceptibility of SL-- V to center electrode noise

32 4.3 Adaptive Laplacian-like Derivation (ALLD) As discussed earlier, it is advantageous to use the Laplacian of the voltage field in source analysis procedure because it reduces the biases caused by far field potentials. The Laplacian s advantage comes from the fact that it reduces the DC and slow varying components of the voltage field which is the basis behind the Adaptive Laplacian-like Derivation (ALLD). In equation 3, the zero and first order terms (first three terms) account for the DC and the slow varying components of the voltage field and the secondorder terms account for the quadratic components. In ALLD the higher order terms are replaced with a function that best models the voltage topology. As a result, ALLD has three key characteristics: ) it reduces the DC and slow varying components; ) it is adaptive in the sense that it can be optimized for a specific voltage field model; and 3) it yields better SNR than a true local surface Laplacian ALLD tuned for parabolic voltage field In order to give reader more insight into how ALLD results in better SNR than other methods described in this chapter, we use a parabola-shaped voltage field to tune the filter. We refer to this special case as ALLD-Parabola. Using a parabolic function enables us to compare ALLD directly with the SL- V and SL-V i methods. ALLD- Parabola is essentially a truncated version of the SL-V i method with the cross and difference terms omitted. Equations for SL- V and SL-V i methods contain three secondorder terms to model the voltage field, which are the Laplacian, difference and cross terms with coefficients v3, v4 and L, respectively (see equation 4). The Laplacian term has a shape of a symmetrical two-dimensional parabola and the asymmetries in the voltage field are accounted for by the difference and cross terms. 8

33 Figure 5 compares the resulting electrode weightings of a few different configurations of the four-surround montage using the different methods. In case of a square montage, all methods result in the same weightings shown in Figure 5 A. In Figures 5 B and C the square montage is symmetrically squeezed along the y-direction to one half of its original distance, referred to as the rectangle montage. In case of the rectangle montage, we can see that the SL- V and SL-V i methods (figure 5 C) produce weightings that are 4 times greater at the closer electrodes than the farther ones. It is easy to see that none of these methods is calculating a true Laplacian. The ALLD-Parabola produces (figure 5 B) positive weightings for the two closer electrodes and negative weightings for the farther ones, hence taking a weighted average of the voltages along the y direction and a second difference along the x direction. In fact figure 5 D, where the two surrounding electrodes along the y-axis are rotated clockwise by 45, further demonstrate that the ALLD-Parabola method acts as a sum of two one-dimensional second-difference operators. In this example, the rotation is not affecting the weightings, which suggests that the weighting function in this case is treating each pair of opposite electrodes independently. This property of the ALLD method, which is due to the omission of the cross and difference terms, leads to a more balanced electrode weightings than do the SL- V and SL-V i methods, thus resulting in less propagation of electrode noise. It is interesting to note that the result from SL- V and SL-V i methods in Figure 5 E is not a true Laplacian either. Both methods require at least 5 surrounding electrodes in order to find a unique solution by matrix inversion. In the case with 4 surrounding electrodes, the equations 5 and 8 are underdetermined (have 9

34 B ALLD-Parabola C SL- V & SL-V i A All methods SNR:.878 (-/.69) SNR:.699 (-/.894) D ALLD-Parabola E SL- V & SL-V i - SNR:.7889 (-/.8) SNR:.7889 (-/.8) SNR:.777 (-./.643) Figure 5 Laplacian weightings calculated using the three different methods. All four methods result in the same weightings for a square montage shown in A. fewer equations than unknowns) and are solved using Singular Value Decomposition (SVD). SVD finds a unique solution by systematically eliminating the unnecessary parameter(s). In Figure 5 B, since the electrode configuration in symmetrical in angle, the cross term and difference terms become rather unnecessary and are ignored by SVD. On the other hand, in Figure 5 E, the electrode montage does not have angular symmetry and therefore SVD s attempt to reduce the number of parameters results in an inaccurate Laplacian estimation. The Laplacian estimation in Figure 5 E is -. (the numerator of the fraction in the parenthesis next to SNR) where it should be based on the analytical equation used to generate the test function. ALLD is not a true Laplacian and therefore can sometimes distort the voltage field. However, any asymmetry in the voltage topography, caused by this method, is not 3

35 of concern for source analysis procedure. Any distortions introduced in the data would be mirrors by the forward solution and therefore its effect canceled when calculating the error function: ( forward ) ( data) = wv i ( forward ) i wiv ( fdata) i i i [ L L ] error =. L data depicts the Laplacian of the recorded voltage field. L forward is the Laplacian of a voltage field generated using a volume conductor model. Source analysis finds a solution that minimizes the sum of squared errors. There is a possible source of concern in that volume conductor misspecification may introduce unexpected systematic error. Of course, model misspecification is a general concern in source analysis and a topic well outside the scope of this chapter ALLD tuned for Gaussian voltage field It is advantageous to use a function that more realistically resembles an EEG voltage field. The trouble with using a two-dimensional parabola is that the signal quadratically increases as a function of distance from the center electrode, thus resulting in higher electrode weightings for farther electrodes. In practice, the voltage field from a single source is more like a Gaussian function where the voltage field eventually tapers off as a function of distance. For small electrode montages the quadratic property of a parabola does not cause any problem since the montage could very well be positioned within the quadratic region of the signal. However it is not suitable for larger montages where the peripheral electrodes may fall outside of the hot spot of the signal in which case they would only contribute noise. 3

36 Figure 6 shows the weighting functions produced by ALLD that are tuned for a Gaussian voltage field (see methods). Figures 6 A and B are each optimized for a voltage field generated by a Gaussian function with a standard deviation of and, respectively. The main difference between them is that Figure B spatially averages the voltage field and therefore has a higher lower frequency cutoff. The spatial averaging is as a result of the four positively weighted surrounding electrodes. The voltage field used in tuning Figure B has a narrower bandwidth than Figure A therefore spatial averaging would not diminish the signal as it would in Figure A. In fact, spatial averaging would lead to better SNR by reducing high frequency noise. This property makes Figure A tuned better for a spatially sharp signal (broad in frequency domain), produced by a shallow dipole. 3

37 A B Figure 6 Laplacian weightings calculated using ALLD. The weightings in A and B are tuned for signals generated by a Gaussian voltage model with a standard deviation of and, respectively. The weighting functions are scaled such that the center electrode weighting is.33 in order to facilitate comparisons with the SL-V i method (in Figure B). Figure 7 compares the ALLD-Gaussian, ALLD-Parabola, SL- V and SL-V i methods by analyzing their relative SNR loss to ALLD-Gaussian under different electrode configurations and voltage topography. The following are several key points that arise from the figure. First, the advantage of using the ALLD in a small montage is that it is less susceptible to asymmetric electrode configurations than the SL- V and SL- V i methods. There is great SNR loss in a nine-electrode montage as adding noise to the electrode positions results in an unbalanced weighting function. This effect is reduced in the larger montages because having a larger number of electrodes provides a better chance for balanced weightings. However, in larger montages ALLD s adaptive characteristic provides yet another advantage, which can be seen by comparing ALLD- Gaussian and ALLD-Parabola results for the 5-electrode montage. Another key point 33

38 made from figure 7 is that the previously reported SL- V method has a dependency on having a center electrode in a montage. The SL- V method does not yield a solution in the case of a 6-electrode checkerboard montage where there is no center electrode. In fact this configuration yields very robust signal-to-noise ratios when used with the ALLD method and therefore is a favorable configuration to use. Final key point made from figure 7 is that SL- V can result in no SNR loss when used with a spatially narrow signal (see eight-electrode configuration used with the Gaussian voltage field with a standard deviation of ). The reason is that the SL- V method often assigns negative weightings for the surrounding electrodes therefore resulting in a broadly tuned (in frequency) filter. The broad tuning characteristic allows all frequencies to pass without any loss of the signal however its downfall is that the signal can be contaminated with high frequency noise. The advantage of using ALLD is that the filter characteristics are optimized to cut out noise without diminishing the signal. 34

39 Electrode Configuration Votage Field Shape (Sigma) Deviation from Symmetry (Standard Deviation) Signal-to-noise Ratio Loss Compared to ALLD- Gaussian SL-DeltaV SL-Vi ALLD-Parabola ALLD-Gaussian 3 9-electrode checkerboard configuration electrode checkerboard configuration electrode checkerboard configuration Figure 7 SNR loss relative to ALLD-Gaussian (depicted by the bars) under different electrode configurations, voltage field sizes, and levels of electrode position jitter. Three different electrode configurations shown by the schematics were tested. A two-dimensional Gaussian voltage field of varying size (standard deviation shown in the second column) was used as the test function. The same test function was used to tune the ALLD-Gaussian filter. In Monte Carlo simulations, varying levels of electrode position jitter (shown in the third column) was used to test the robustness of each scenario against non-uniform electrode configurations. 35

40 4.4 Optimal number of electrodes in a montage Determining the optimal number of electrodes to be used in a montage for the Laplacian estimation can be tricky and often depends on such factors as the topology of the voltage field, level of noise and electrode spacing. Using ALLD simplifies the process by providing a simple framework for taking these factors into account. Once a proper tuning function is chosen, ALLD can help search for the optimal electrode montage by adding electrodes one by one, in the ascending order of distance from the center of the montage, until the signal to noise reaches a saturation point (dotted line in Figure 8A). Electrodes that improve SNR up to 8-9% of the saturation point are kept in the montage and the rest are eliminated. The solid line in Figure 8A shows the percent change in SNR as you add more electrodes for the SL-V i method. The test function used is a Gaussian voltage field with a standard deviation of.5. In this case, the SNR for the most part improves up to the addtion of the st electrode after which it begins to decrease. The SNR decreases because of two reasons: ) since it is optimized for a parabolic function, as the montage gets bigger it keeps increasing the area of spatial averaging, therefore starts washing out the spatially narrow signal; and ) it assigns high weightings to electrodes near the edge where it falls outside of the hot spot of a spatially narrow signal and only contribute noise. Incidentally, there is a dip after addition of the th electrode. The weighting function that results in the dip is shown in Figure 8 B. The reason for this quick downward change is that the SL-V i method is very susceptible to any breakage in symmetry, which leads to a very unbalanced weighting function. On the other hand, the ALLD method is more robust to changes in symmetry, hence resulting in a smoother 36

41 curve. The dotted curve eventually starts flattening out which suggests that the ALLD is not going to greatly benefit from including additional electrodes. The shape of the saturation curve varies from case to case mainly based on electrode spacing and voltage topography. A B 4 Percent SNR change Number of electrodes Figure 8 A shows the percent improvement in SNR over a five-electrode montage as more electrodes are added to the montage. The electrodes are arranged in a checkerboard manner with adjacent electrode spacing of. Electrodes are added as a function of increasing distance from the center of the montage. The dotted and solid lines are as a result of ALLD-Gaussian and SL-V i method, respectively, subjected to a Gaussian voltage field with a standard deviation of.5. B shows the electrode configuration and weightings that result in the dip seen in the solid line when the tenth electrode is added. In practice, the electrode positions are projected from the three-dimensional space onto a tangent plane. Therefore there is also a limit to how far you can accurately model the spherical geometry with a plane until the error as a result of the curvature becomes too big. The generally accepted range is about a radial distance of about 3 cm on 37

42 the scalp (Le 994) above which the error in distance in the radial direction on the scalp versus the planar geometry exceeds 4.5% (error=[3- *sin(3/)], where the radius of a sphere is assumed to be and the surface on the sphere to be 3). 38

43 5 CONCLUSION This chapter presents two techniques for improving the Signal-to-noise ratio (SNR) of previously described Surface Laplacian Derivation (Gevins 989, Greer 989, Huiskamp 99) by ) including the DC term as a free parameter in the Laplacian calculation and ) differentially weighting electrodes based on their noise level. These techniques are optimized for SNR without compromising the spatial symmetry of the true Laplacian. The first technique, SL-V i, yields higher SNR for montages with six or more surrounding electrodes by balancing the contribution of noise more evenly across all electrodes. The SL-V i method often yields, in addition to the center electrode, a few other positively weighted electrodes. By making some electrode weightings positive, the SL-V i method takes a weighted average of the potential. The averaging acts as a low pass filter and therefore mitigates the second derivative s inherent amplification of high frequency noise. In fact, a spatial low-pass filter has been shown to be effective in reducing the relative spatial error of the surface Laplacian (Le 994). The second technique optimizes SNR by taking into account the noisiness of an electrode in calculating the Laplacian weighting function. It optimized SNR by reducing the noise contribution of noisy electrodes by reducing their weightings. We also present a novel implementation of an adaptive Laplacian-like derivation (ALLD) that is SNR optimized for an arbitrary voltage model. This method is not a true Laplacian but it has similar properties in that it cancels out the DC and slow varying components of the scalp potential. This method was devised for the purpose of dipole source analysis. Klein et al. (995) have shown in their simulations that it is advantageous to use the Laplacian over raw data for dipole source analysis procedure. 39

44 Using the Laplacian of the potentials results in more accurate solutions by reducing the bias caused by far field potential and model misspecification. This chapter is mainly concerned with analysis of sources in the early visual areas whose depth can range roughly from to 5 centimeters from the surface. Consequently the spatial frequency components of the resulting potential fields can vary broadly as well. ALLD s tuning property can be useful in optimizing the Laplacian operator for dipoles of particular depth. ALLD optimizes SNR by also yielding in a very balanced weighting function even when the electrode montage is very asymmetric. The main disadvantage of ALLD is that it can distort the voltage field. However, the topology distortions should not cause any problem for dipole source analysis since it is accounted for by the forward solution. Calculating the optimal number of electrodes to be used in a montage is a complex issue that depends on factors such as ) Laplacian formulation, ) the shape and frequency components of the voltage model, 3) the noise distribution, and 4) electrode sampling and configuration. The SNR formulation presented in this chapter provides a simple framework for choosing the optimal number of electrodes to be used in the Laplacian calculation. 4

45 6 APPENDIX 6. Proof of ALLD Inverting matrix M from equation using Least Squares technique (equation 9 or ) results in a matrix, M psuedo-inverse, with the following property: M psuedo-inverse M = I, where I is the Identity matrix. Since ALLD weighting function is a vector, w, equal to the last row of M psuedo-inverse, replacing M psuedo-inverse with w gives: [ ] ), ( : : : ), ( ), ( = = n n n n y x f y x y x f y x y x f y x w wm ϖ ϖ. Multiplying w with the first, second and third column of M gives i n o i i w = =,, and, i i n o i i x w = = i n o i i y i w = = respectively, which proves our first three constraints from equation. In addition to the above three constraints, ALLD satisfies the following constraint, ) var( max w i noise i λ. The function being maximized is the formulation used to calculate signal-to-noise ratio (equation 4). Least squares procedure produces a weighting function that is optimal in 4

46 the sense of asymptotic mean square error, which inherently optimizes the signal-to-noise ratio for the Laplacian estimation. 4

47 6. Proof of Variance of the Laplacian The Laplacian of a voltage field is defined by equation as the dot product of the weighting function, w, and the voltage vector, V, L = w V i i i. Using the above equation, the uncertainty of the Laplacian would be: var( L ) = var( w V i i ) i. The variance of the sum is equal to the sum of the variance, var( L ) = i var( w i V i ) and applying the weighted sums rule (Bevington 99) for variance gives var( L ) = i w i var( V i ). Assuming that noise at each electrode is additive and uncorrelated, the variance of the Laplacian reduces into a simple relationship: var( L ) = i w i var( noise i ). 43

48 7 REFERENCES Ary, J.P., Klein, S. A. and Fender, D. H Location of sources of evoked scalp potential: correction for skull and scalp thickness. IEEE trans. On Biomed. Eng. 988: Babiloni, F., Babiloni, C., Fattorini, L., Carducci, F., Onorati, P., and Urbano, A. Performances of surface Laplacian estimators: astudy of simulated and real scalp potential distributions. Brain Topogr., 995,8: Bevington, P. R., Robinson, D. K. Data reduction and error analysis for the physical sciences. Second edition. WCB McGraw-Hill. 99. Biggins, C., Fein, G., Raz, J. and Amir, A. Artifactually high coherences result from using spherical spline computation of scalp current density. Electroenceph. Clin. Neurophisiol., 99, 79: Gevins, A.S., Bricket, P., Costales, B., Le, J. and Reutter, B. Beyond topographic mapping: towards functional-anatomical imaging with 4-channel EEGs and 3- DMRIs. Brain Topography, 99, 3: Gevins, A.S., Dynamic functional topography of cognitive tasks. Brain Topography, 989, : Greer D. An algorithm for reconstructing the surface of the central nervous system in Vivo. Ph.D Dissertation, Dept. of EECS, UC Berkeley, Berkeley CA, 989. Hjorth, B. Online transformation of EEG scalp potentials into orthogonal source derivations. Electroencephalography and Clinical Neurophysiology. 975, 39: Hjorth, B. Source derivation simplifies topographical EEG interpretation. Am. J. EEG Technology. 98, : -3. Huiskamp, G., Difference formulas for the surface Laplacian on a triangulated surface. Journal of Computational Physics, 99, 95: Katznelson, R. EEG recording electrode placement and aspects of generator localization. In: P. Nunez (Ed.), Electric Fields of the Brain: The Neurophysics of EEG. Oxford University Press, New York, 98. Klein, A. S. and Carney, T. The usefulness of the Laplacian in principal component analysis and dipole source localization. Brain Topography, 995, 8: 9-8. Klein, A. S. Inverting a Laplacian topography map. Brain Topogr., 993, 6:

49 Law, S. and Nunez, P. Quantitative representation of the upper surface of the human head. Brian Topogr., 99, 3: Law, S., Nunez, P. and Wijesinghe, R. High-resolution EEG using spline generated suface Laplacians on spherical and ellipsoidal surfaces. IEEE Trans. Biomed. Eng., 993, 4: Nunez, P., Estimation of large-scale neocoritical source activity with EEG surface Laplacians. Brain Topogr., 989, : Perrin, F., Bertrand, O. and Pernier, J. Scalp current density mapping: value and estimation from potential data. IEEE Trans. Biomed. Eng., 987, 34: Press W. H., Teukolsky, S. A., Vetterling, W. T., Flannery, B. P. Numerical Recipes in C: The art of scientific computing. Second Edition. 99, Cambridge University Press, Cambridge. Seber G.A.F. Linear Regression Analysis. 977, John Wiley & Sons, Inc. Srinivasan, R., Nunez, P. and Silberstein, R. Spatial filtering and neocortical dynamics: estimation of EEG coherence. IEEE Trans. Biomed. Eng., 998, 45: Thickbroom, G., Mastaglia, F., Carroll, W., and Davies, H. Source derivation: application to topographic mapping of visual evoked potentials. Electroencephalography and Clinical Neurophysiology, 984, 59: Wallin, G., and Stalberg, E. Source derivation in clinical routine EEG. Electroencephalography and Clinical Neurophysiology, 98, 5: 8-9. Wang, W., Begleiter, Henri. Local polynomial estimate of surface Laplacian. Brain Topogr. 999, :

50 CHAPTER : MULTI-SOURCE ANALYSIS SIMULATIONS Summary Electro/magneto-encephalographic (E/MEG) source analysis has the potential to provide researchers and medical practitioners with timing information about brain activity in the millisecond range. The high-resolution temporal information can be combined with the high-resolution spatial information from other imaging techniques in order to build a spatio-temporal image of neural activity in the brain. The validity of current source analysis methods have been in question for many reasons including poor signal-to-noise, volume conductor misspecification, presence of unaccounted sources, and rotation ambiguity in multi-source analysis. With the exception of rotation ambiguity, all other problem areas in source analysis, while producing errors in dipole locations and orientations, for the most part result in good temporal information. On the other hand, for areas of the brain, such as the visual cortex, where there are 46

51 simultaneously active sources in close vicinity of one another, the rotation problem leads to inaccurate temporal information. The rotation problem results from degeneracy in the solution space such that many possible linear combinations of simultaneously active sources can explain the same spatio-temporal surface potential topography. This chapter presents and examines, via simulations, a novel approach for overcoming the rotation problem caused by sources in the visual cortex areas V and V. The new approach solves the rotation ambiguity by placing anatomical and physiological constraints, specific to the visual cortex, on the solution space. 47

52 INTRODUCTION To fully understand mechanisms of brain processing requires studying the underlying pattern of neural activity. One of the most promising yet unfulfilled techniques to study the spatio-temporal pattern of brain activity has been electro/magneto-encephalographic (E/MEG) source analysis (SA). SA is a promising technique because it can potentially provide temporal information in the order of milliseconds where as functional magnetic resonance imaging (fmri), which is the most widely accepted brain imaging technique today, provides temporal information in the order of seconds. The downfall of SA has been its inaccuracy when there are multiple spatially close active sources (Mosher 993; Supek 993; Zhang 994; Jewett 995), such as found in early sensory processing areas. One of the contributing factors to the inaccuracy of SA is head model misspecification (He 989, Roth 993, Zhang 994). Model misspecification refers to inaccuracies in the models used to map electrical currents inside the brain to signals recorded at the sensory array. Zhang has shown that model misspecification can introduce insidious errors in the locations and orientation of the dipoles used to model the source activity. Developing accurate models has proven to be challenging. There are methods such as boundary/finite element modeling (B/FEM) that provide a more accurate model than commonly used multi-shell homogeneous models (Fender 99, Van Oosterom 99, Fletcher 995). However B/FEM methods are not commonly used because most researchers believe the difficulty of developing such a subject specific model is not justified by the improvement in performance. 48

53 It is not always necessary to use the latest head models. For example, if there is only one source of brain activity then the temporal waveform of the source can be accurately estimated even when the spatial solution is incorrect. Although under realistic conditions there are multiple simultaneously active sources in the brain, it is possible under certain circumstances to isolate a particular source thus accurately estimate its temporal dynamics. Some of the techniques used to isolate a source include: ) manipulation of the stimulus to generate a dominant source, ) selecting a time window when only one source is dominant (it is, however, difficult to know which time window to select), and 3) various spatio-temporal filtering methods. Another challenge faced by such best of both worlds approaches is to properly register the source loci from SA to that of fmri. The registration can be difficult because of inherent systematic error introduced by both techniques, particularly by SA. However when a given source is well isolated, gross localization by SA provides enough information for the maps to be combined. In spatially close multi-source situations, the difficulty of combining the two techniques does not stop at the registration problem. The registration problem implies that if the solutions for the corresponding sources are somehow correctly identified between the two techniques then one can take the temporal information from SA and combine it with the spatial information from fmri to create a complete map. Unfortunately, another major difficulty in getting reliable dipole solutions surfaces when there are multiple spatially close sources. A number of different combinations of dipole pair solutions, in the presence of noise, can equally model the superposition of multiple spatially close sources. In other words, there is no unique solution to this inherently ill- 49

54 posed problem. The problem caused by the described ambiguity in the solutions is called the rotation problem. The error introduced by the rotation ambiguity can be larger than the error due to model misspecification (Mosher 993). Once the rotation problem is overcome by finding the correct rotation, then the issue of model misspecification can be bypassed and the promise of combining SA and fmri becomes more of a reality for multiple sources. The last hurdle to clear before combining the two techniques would be, as in the case with the single dominant source, to solve the registration problem, which is not as daunting to solve as model misspecification. Analysis of simultaneously active sources of the first and second visual areas (V and V) suffers from the rotation problem because of their close proximity. This chapter presents a novel approach that combines anatomical and physiological constraints to find the most plausible rotation solution for sources in V and V. The approach is tested using simulation under a variety of noise and model misspecification situations. The constraints used to limit the rotation space are based on several assumptions. Firstly, the sources within every visual area evoked by isoeccentric stimulus patches have similar temporal dynamic. Secondly, the sources within every visual area and hemisphere evoked by a set of adjacent cortically scaled stimulus patches are closely positioned and similarly oriented. Thirdly, the sources within every visual area evoked by cortically scaled stimulus patches have similar strengths. The source position, orientation, and strength continuity assumptions are integrated into a metric called Ribbon Metric. The motivation to pursue the Ribbon Metric approach was provided by the fact that our research group has the technology to efficiently collect visual evoked potential (VEP) data using a dense stimulus array (Baseler 994, Slotnick 999, Slotnick ). 5

55 THEORY AND METHOD. Scalp potential The potentials recorded on the scalp can be described by a quasi-static approximation of Maxwell equations (Nunez 98). The quasi-static model, which assumes that current-flows through the head are instantaneous, simplifies the set of equations mapping the location, orientation and strength of a neural generator source to the measured potential at every electrode. The sources are modeled by equivalent dipoles, each of which represents the center-of-mass of the current sources and sinks for a sufficiently small area of neural activation. The potential at every electrode, V(e), is a function of the electrode position, e, source location, l, and source moment, m: V(e) = E(e,l) m () where E is often a complicated function that non-linearly maps the source location to a three-dimensional vector representing the three dimensional lead-field at every electrode. The source moment, m, is a three-dimensional vector that is based on the strength and orientation of a source. The Pythagorean length of m, m, represents the source strength and the normalized moment vector, m/ m, represents the orientation of the source. Finding a set of equations that accurately model the electrical distributions on the scalp has been an ongoing area of research in lead field theory. The so called forward solution is one of the most critical and yet the most elusive piece of the puzzle for estimating sources of brain activity. The myriad of different approaches to the forward solution can be divided into two very distinct approaches (Fender 99, Baillet ). 5

56 The oldest and most common approach is to represent the forward solution by an analytical formula. This approach for the most part makes the assumption that the volume conductor is spherical in shape and isotropic in conductivity and therefore badly suffers from the problems that arise from misspecification. The more recent and ambitious approach has been to numerically represent the volume conduction by more accurately modeling the shape and inhomogeneities using Boundary Element Modeling (BEM) or Finite Element Modeling (FEM). Although there have been many advances in forward models using B/FEM, the numerical forward solution approach has been slow to be adopted because the improvement in accuracy in the mind of many does not justify the difficulties in implementation. To account for the time series component of scalp potentials, equation () can be expanded such that the moment vector, m, would also be a function of time, t, V(e,t) = E (e,l) M(t). () where M(t), a matrix with dimension {three by number of time points}, represents the dipole moment at every point in time. Equation () implicitly states that a dipole s location does not change over time whereas its orientation can. It has been argued that a dipole s orientation as well its position can be assumed fixed over time since the underlying neural generators also remain fixed in their position (Scherg 99). A dipole with fixed position and orientation over time results in scalp potentials V(e,t) = E(e, l, m)t(t) (3) where m is the dipole moment and T is a vector depicting the modulation of activity over time. By normalizing T such that T = and appropriately scaling m, then T can be 5

57 thought of as a time function for a dipole of magnitude m. The benefit of normalizing T is that it can be easily compared or constraint to that of other dipoles (Slotnick 999). Source analysis methodology in this chapter assumes fixed dipoles position and orientation over time. The equations () through (3) can easily be extended for surface potentials resulting from multiple dipoles by way of superposition of surface potential V total = V d d. (4) Equivalently in case of multiple dipoles equation (3) in matrix notation would be V =ET (5) where V is a matrix with dimension {number of electrodes by number of time points}; E is a matrix with dimension {number of electrodes by number of dipoles} and T is a matrix with dimension {number of dipoles by number of time points}. 53

58 . Rotation problem One of the most daunting tasks in multi-source analysis has been to accurately disambiguate nearby sources or to avoid the so called rotation problem. To develop formulation for the rotation of signal components let us first assume that any surface potential, V, can be decomposed into a set of spatial and temporal components, S and T, respectively: V = S T. (6) S is a matrix of dimension {number of sensors by number of components} and T is a matrix of dimension {number of components by number of time points}. Note that S is different from E in equation (5) in that it is not a function of dipole parameters. Any pattern of activity on the surface of any volume conductor can be accounted for by an infinite number of possible potential distribution and time function components. The linear transformation or the so called rotation of the potential components is achieved by inserting a rotation matrix, R, and its inverse, R -, into equation (6) V = S R - R T (7) In case of two dipoles, cos( α) R = sin( β ) sin( α) cos( β ) (8) where α and β are rotation parameters (ranging from -π to π) operating on the first and second components, respectively. The above rotation matrix allows for oblique rotations which means that each component is independently rotated and can be at various angles from one another. For when α = β the rotation would be a rigid rotation which means that 54

59 the angles between the two components are preserved as the components are rotated clockwise. R Based on equations (7) and (8), the individual rotated temporal components, T and T R, can be written as T R = cos(α) T + sin(α) T (9a) and T R =-sin(β) T + cos(β) T (9b) where T and T are the first and second components (columns) of T from equation (7). The R superscript denotes that the components are rotated in respect to the originals. The rotated spatial components, S R and S R, are S R = (cos(β) S + sin(β) S ) / cos(α-β) (9c) and S R = (-sin(α)s + cos(α) S ) / cos(α-β) (9d) where S and S are the first and second components (rows) of S from equation (7). From equations (9a), (9b), (9c) and (9c), we can see that changing the values of α and β does not change the surface potential that is to mean: S T + S T = S R T R + S R T R. However the rotation property of surface potential components does not always lead to rotation ambiguity in dipole source analysis. There is no rotation ambiguity only if S and S can each be modeled by a single dipole and not their rotated counter parts. For example if there are two very shallow sources at the opposite poles of a spherical volume conductor whose temporal components are somewhat correlated. Solving for two dipole locations, orientations and temporal components would not, as one s intuition would suggest, present any rotation problem. 55

60 The reason is that each source has created a spatially narrow potential field far from the other. Therefore each of the potential fields can only be fitted by a single dipole. To further examine the practical implication of rotation ambiguity in dipole source analysis, the potential field in case of two dipole sources is written as V = (E M ) T + (E M ) T, () where dipole based spatial components, (E M ) and (E M ) from equation () are replaced with the generalized spatial components, S and S, respectively. The important point is that when two dipole sources are superimposed in which case E = E, equation () reduces to V = E (M T + M T ) () where E = E = E. By factoring the dipole position term, E, it no longer has direct dependency on its corresponding moment and temporal component. Under such conditions, there exist an infinite number of possible orientations and temporal components whose linear combinations would result in the same potential field. For this reason, source analysis of overlapping sources is impossible unless additional constraints are used. In reality, because of presence of noise, the rotation problem exists even when the underlying sources are close but not overlapping in position. As the level of noise increases the rotation ambiguity becomes more significant. 56

61 .3 Constraining the Rotation Space A subspace spanned by two source components can be parameterized by two rotation parameters, α and β, in a linear transformation matrix shown by equation (8). The rotation parameters, α and β, define the axes of a planar space called the rotation space. As discussed above, under realistic multi-source analysis conditions, sufficiently close dipoles would suffer from rotation ambiguity. This chapter presents a novel methodology for constraining the rotation space using continuity rules derived from the general anatomy of the first two visual cortical areas, V and V. The constraints, which are combined in a metric called the Ribbon Metric, limit the rotation space to regions that result in anatomically plausible solutions. Although others have used anatomical constraints in the past (Scherg 99, Clark 995), the novelty of our approach is in using a series of closely placed and independently evoked sources such that we can place piecewise continuity constraints on the solution space. The closely placed sources create a grid of 4 by 6 sources in the topographically organized visual areas..3. Common temporal dynamic constraint The inspiration behind pursuing the piecewise continuity approach comes from the work published by Slotnick et al (999). The work by Slotnick et al is relevant to this chapter in two important ways. Firstly, they developed an efficient method, using multi-stimulus m-sequence technique, for recoding visual evoked responses (VEP) in humans using an array of small visual stimulus patches. The cortically scaled dartboard shaped stimulus array used in the study were designed to evoke sources whose center-of-mass are estimated to be separated by about 7 mm along the surface of the cortex. The ability to 57

62 evoke closely spaced sources on the cortex allows a practical implementation of the work described by this chapter. Secondly, the methodology described in this chapter relies on the ability to equally rotate the temporal components of all dipole pairs corresponding to all stimulus patches. The rotation of all dipole pair components in unison requires that somehow all temporal components be tied together. Slotnick et al. developed and validated a novel method that constrained the dipoles corresponding to iso-eccentric stimulus patches (patches in the same stimulus ring) to share the same temporal component. This temporal constraint is referred to as the common time function. The surface potential formulation under the common time function assumption is V ( e, t, p) = E( e, l( d, p) m( d, p)) T ( t, d) () d where p is stimulus patch index and d is dipole index. The common time function assumption provided several benefits including signal-to-noise improvement of temporal component solutions, rescuing of weaker dipoles, and decreasing the rotation ambiguity. Beyond these benefits, the common time function provides a framework for synchronously rotating the dipole solutions corresponding to stimulus patches within a ring. Yet a challenge for this chapter is to find a way for synchronizing the temporal components between the different stimulus rings. A technique, which is referred to as the temporal yoking (described by section.5.), was devised to ensure that the temporal components corresponding to different stimulus rings are synchronized. 58

63 .3. Anatomical constraints This chapter also presents constraints based on general anatomical assumptions. Using individual anatomical maps presents many technical challenges beyond the practical issues such as cost and time of attaining such maps. As was discussed by Scherg and Berg (99), absolute spatial constraints based on anatomical maps, because of current inaccuracies of forward models, can result in systematic error. The constraints used in the Ribbon Metric only make general piecewise continuity assumptions, which should still hold true under global warping effects due to model misspecification. Misspecified head shape, ventricles, skull thickness, and CSF are likely to only bend, rotate, and/or translate the spatial maps therefore still preserving their relative position and topological relationship. This chapter shows that the rotation ambiguity caused by misspecification of shell conductivities in a spherical model can be disambiguated using the Ribbon Metric therefore providing evidence that under this type of misspecification the relative continuity is maintained..4 Penalty function: the Ribbon Metric.4. V/V lay out The first and second cortical visual areas, V and V, are topographically organized with respect to the visual field such that adjacent regions of the contralateral hemifield map correspond to adjacent cortical regions (see Figure ). Both V and V are located along the calcarine fissure in the occipital lobe. A good portion of V is tucked in the calcarine sulcus with the horizontal meridian (HM) roughly mapping to the base of the sulcus. The cortical map for V is inverted such that the upper and lower parts of the 59

64 visual field map to the lower and upper parts of the cortical region, respectively. The upper and lower edges of V, representing the lower and upper part of the vertical meridian (VM), respectively, form the borders with V. V in each hemisphere is split into two regions representing each visual quadrant. The V map is inverted with respect to V such that V/V cortical regions representing regions close to the VM are located in close proximity of one another. For both V and V, the regions closer to the posterior end of the occipital lobe represent the center of the visual field. The more anterior regions of the cortex represent the more peripheral regions of the visual field. Visual Space Medial View of Anterior Cortex V V Eccentricity UVM Parieto-occipital sulcus HM Fovea HM Occipital lobe LVM LVM UVM HM Calcarine sulcus Figure V and V anatomical layout 6

65 .4. Ribbon construction The position and orientation continuity of neighboring regions in V and V, which correspond to the cortically scaled stimulus patches shown in Figure, can be constrained to a Ribbon shaped surface. This Ribbon forms the basis for all of our constraints described in this chapter. Stimulus patches in visual space Continuity representation of corresponding patches in V and V Dipole continuity representation: Ribbon Space Upper Visual Field Lower Visual Field Upper Vertical Meridian (UVM) Horizontal Meridian (HM) Lower Vertical Meridian (LVM) V - Lower VF V - Lower VF V - Upper VF V - Upper VF Figure Ribbon Metric (RM) structure 6

66 As shown in Figure, the regions of cortical activity in V and V, evoked by a stimulus array consisting of four semi-rings with each containing 6 cortically scaled patches, could be represented by a 4 by grid. Every stimulus patch maps to two patches on the grid, one representing V and the other V. In turn, every grid patch maps to an equivalent dipole that models the center-of-mass of activated cortical patch. The ribbon points are treated as vertices of a polygon mesh connecting neighboring points thereby defining a 3-dimensional space. This polygonal mesh provides a framework for creating a group of cost functions, collectively called a Ribbon Metric. The Ribbon Metric evaluates the anatomical plausibility of dipoles solutions by operating on every ribbon vertex and its neighboring vertexes. To solve the rotation ambiguity, the two-dimensional rotation space, parameterized by α and β, is divided into a number of equal segments ( by in this case) for each of which a Ribbon Metric value is calculated (the rotation space is later described in detail). The Ribbon Metric value is projected onto the by twodimensional rotation space, forming an image. The rotation with the most anatomically and physiologically plausible dipole solution is depicted by the image pixel with the lowest value. 6

67 .4.3 Ribbon Metric estimation The Ribbon Metric is a weighted sum of several sub-metrics. The sub-metrics are based on the following continuity assumptions: ) the distance separating neighboring dipoles, as defined by the Ribbon, are within assumed limits; ) each dipole has a similar orientation as its surrounding dipoles; 3) prominent cortical folds have high curvature in one direction and low curvature in the orthogonal direction; 4) each dipole has similar magnitude as its neighboring dipoles within every visual area in the Ribbon. These piecewise smoothness assumptions between neighboring regions are not completely true but provide us with a set metrics that are reasonable given the known anatomy. The first metric, called the distance continuity metric (DCM) assumes the neighboring dipoles are separated by a certain distance, D lim. The neighbors for a given dipole are defined by the first layer of surrounding nodes connected with dashed lines in V - UVF V - UVF V - LVF V - LVF Figure 3 RM node/surround connection Figure 3. The formulation for DCM is DCM = NS N S ( n s λ λ D ) B (3) n ns lim where B is a binary operator 63

68 , B =, ( λn λns Dlim ) ( λ λ D ) n ns lim >. (4) λ and λ are the locations of a dipole at a node, n, and its surrounding dipoles, s, n ns respectively. N denotes the total number of nodes in a ribbon. S denotes the total number of surrounds for every node. The double brackets denote vector norm. Dlim is also a minimum limit on the distance between two dipoles below which no penalty is assumed. In the V region, D lim value used in this chapter is.5 cm. In the V region, D lim value is.5 cm for neighboring nodes along the Ribbon width and.5 cm along the length. DCM is evaluated over all Ribbon nodes. A very important metric related to DCM is distance continuity across the horizontal meridian (DCHM). As the name suggests DCHM is evaluated for the dipoles corresponding to the stimulus patches adjacent to the horizontal meridian. DCHM is important for identifying which dipoles correspond to the V sources and which to the V sources. Orientation continuity metric (OCM) is based on an assumption that neighboring sources on the cortex have similar orientations. This assumption is not completely true for real cortex and even for our simulated surface however it has some properties discussed later that make it very useful. OCM is estimated by OCM = NS N n S s cos m m n n m m ns ns (4) where m and m are the moment vectors for dipoles corresponding to a node and its n ns immediate surrounding nodes, respectively. OCM is evaluated over all ribbon nodes. A very important metric related to OCM is orientation continuity across the vertical 64

69 meridian (OCVM). As the name suggest OCVM is evaluated for nodes adjacent to the vertical meridian line which is also the V/V border on the ribbon metric. OCVM is important in that it indicates if a rotation results in V and V dipoles (next to the vertical meridian) that are orientated in the same direction. Furthermore, OCVM is important for disambiguating the relative sign of the V and V time functions. Gaussian curvature metric (GCM) is based on an assumption that prominent cortical folds such as the calcarine sulcus have high curvature in one direction and low curvature in the orthogonal direction. First, Gaussian curvature is estimated for a surface fitted to a set of dipole orientations corresponding to a 3-by-3 grid of neighboring nodes. The local curvature estimates are then summed over the whole ribbon. Gaussian curvature is high only when there is strong curvature along both curvature principle-axes. The x and y components of surface-normals to a homogeneous second-order surface, ( x, y) = a x + a y + a x + a y + a xy, can be estimated by taking the gradient of the f function along x and y, respectively: f x and f y = a + a3 + a 5 y = a + a4 + a5x (5) (6) 65

70 Letting x and y be positions of a set of 3 by 3 neighboring Ribbon nodes and replacing each surface-normal N (x, y) with a normalized dipole moment vector, m=[m x /m z, m y / m z, ] would result in a set of linear equations: = / / / / / / a a a a a x y x y x y y x y x y x m m m m m m m m m m m m n n n n z y z y z y zn xn z x z x Μ Μ Μ Μ Μ Μ Μ Μ Μ Μ Μ Μ (7) where x -n and y -n are coordinates for the group of nodes. The parameters a -6 are estimated using a minimum -norm solution. The Gaussian curvature of the estimated surface is a a a C = (8) Then the Gaussian curvature metric is estimated by = N n C n N GCM. (9) where n is the index for the interior Ribbon nodes and N is the total number of interior nodes. A sub-metric called magnitude continuity metric (MCM) is based on continuity of magnitude between neighboring dipoles of the same region. The formulation used to calculate MCM is The moment vectors corresponding to each set of 3 by 3 grid of nodes is first rotated such that the average of their x as well as y components vanish. 66

71 MCM = NS N n S s m n m ns, () where m n and m ns are moment vectors for a node and its surround, respectively. The subscripts n and s are indexes for the Ribbon node and its surround, respectively. N denotes the total number of nodes in a ribbon and S denotes the total number of surrounds for every node. The single and double brackets denote vector norm and absolute value, respectively. The sub-metrics topography can have very sharp ridges therefore each sub-metric is saturated (given a ceiling) at twice the standard deviation above the mean. The metric values from the singular rows are not used in the mean and standard deviation calculations. The Ribbon Metric is calculated by taking the weighted sum of normalized sub-metrics. Each sub-metric is normalized by the mean value of its non-singular areas. The weightings used for all simulation in this chapter are,,,,., and. for DCM, OCM, GCM, MCM, DCHM, and OCVM, respectively. The sub-metric weightings are determined post hoc, as will be discussed, using the following logic: those that accurately identify the correct rotation are given weightings of (DCM, GCM and MCM); one that is inaccurate is given a weighting of (OCM); those that are nonspecific in identifying a rotation but are useful in detecting dipole switching and/or orientation reversal are given the weighting. (DCHM and OCVM). DCM, OCM, GCM and MCM are also evaluated over the V and V Ribbon regions separately. The region specific maps are only used for the purpose of visual analysis. The region specific maps for V and V are referred to by the addition of D or D, respectively, to the end of every map name. 67

72 .5 Source analysis algorithm Spatio-temporal surface potential data are analyzed for two dipole sources using a three-step procedure. The first step involves estimating a pair of orthogonal basis temporal-vectors corresponding to each stimulus ring (or semi-ring). Each basis vector pair is used to define a subspace containing the two temporal components of interest. The second step involves estimating dipole locations, orientations and magnitudes for 44 rotations of the basis temporal-vectors. The third step involves using the Ribbon Metric to determine which rotation leads to the most plausible solution..5. Step one: estimating the subspace Estimating two orthogonal basis temporal-vectors for each stimulus ring, which define the subspace containing the temporal components of interests, is a two-part process: Part one involves performing a single-dipole fit to the potential data, V patchdata, using the common temporal constraint methodology described by Slotnick et al (999). Locations, orientations, magnitudes of 4 dipoles corresponding to the 4 stimulus patches and the 4 temporal component corresponding to the 4 stimulus rings are iteratively estimated by minimizing sum of squared error (SSE) between V patchdata and V model. V model is estimated using equation (). Part two involves fitting the spatio-temporal data with 48 dipole pairs using the same general minimization framework described in part one. To limit the degeneracy of the solution space, the following constraints are placed on the dipole parameters: ) each dipole pair is constrained to share the same location parameter; ) each second-dipole temporal component is made orthogonal to the first-dipole temporal component using the 68

73 Gram-Schmidt process (Hill 996); 3) each orthogonal temporal-vector pair is rotated such that the correlation between the first-dipole temporal component and the corresponding single-dipole-fit temporal component is maximized. Each of the resulting four orthogonal temporal component pairs is referred to as a basis temporal-vector pair..5. Step two: estimating location and orientation at every rotation Step two involves using a grid search of all temporal components at different rotations, which are used to localize two dipoles corresponding to every stimulus patch. First the rotation space, parameterized by α and β from equation (8), is partitioned into 44unique rotations by sampling each rotation parameter at different angles (more details regarding how the space is partitioned is described later). Two temporal components for ring (the second of the four rings), depicted by matrix T template, are estimated at every rotation, ( ) =, () Ttemplate t R T ring basis where R is a rotation matrix as a function of α and β. T ringbasis is a matrix whose rows consists of the basis temporal-vectors for stimulus ring (described by section.5.). T template is then used as a template to calculate 4 temporal component pairs for every ring. Each temporal component pair corresponding to each ring is estimated by projecting T template onto the space spanned by the corresponding basis temporal-vectors pairs, T fix = T template (T T basis T basis ), () where T basis is a matrix whose rows consist of two basis temporal-vectors corresponding to a stimulus ring. T T basis is T basis transposed. Following the projection operation, the rows of T fix are each normalized to be of unit length. 69

74 Please note it was later discovered that the rows of T fix were not normalized in the source analysis procedures of this thesis. This normalization error only affects (although insignificantly) the results in section 3.4 of this chapter and the results of Chapter 3. The effect of the normalization error was tested and found to be insignificant in relation to the final outcome. This normalization error of the temporal components manifests itself in the dipole magnitudes being slightly overestimated (as much as 5% in worse cases). The projection operator described by equation () helps synchronize the rotations of the basis temporal-vectors between the four subspaces (corresponding to the four rings). This synchronization process is referred to as the temporal yoking. T fix is used to estimate the location, orientation, and magnitude of two dipoles for every spatio-temporal potential data, V patchdata, corresponding to a stimulus patch. An iterative nonlinear least squares procedure using Marquardt algorithm is used to minimize the sum of squared error, [ min imize SSE = V patchdata ( e, t) ( Emod el ( e, l, m) T fix ( t)) ], (3) e t where E model is surface potential distribution modeled by six location and six moment parameters, represented by vectors l and m, respectively. The difference quantity in equation (3) is squared and then summed over all electrodes, e, and time points, t. The planar rotation space defined by α and β, is made limited to a square shaped area centered at {, } parameter values by letting α = γ + δ and β = γ δ, (4 & 5) where γ and δ range from -π/ to π/ in π / increments. The resulting rotation space is a by grid of different rotation possibilities. 7

75 7 A α β 8 8 Orthogonal Orthogonal Orthogonal Singular Singular B. Figure 4 Rotation Space

76 Figure 4 A shows the α and β parameter values, which are the top and bottom numbers in each grid, respectively, for 44 rotations. This square space eliminates rotations that lead to redundant dipole parameters. In fact, the rotation space wraps around itself, which is evident by the fact that the rotations along the opposite sides of the square space result in identical dipole solutions. The point δ=π/ is identical to δ= π/ because both α and β are incremented by π. The opposite sides represent temporal components that are reversed in sign, which benignly flips the orientation of the resulting dipoles, by 8 degrees and results in the identical ribbon metric value. The same argument shows that any point outside of the by grid has the identical ribbon metric as a point inside the grid. Figure 4 B shows the results of using every 44 parameter-pairs to operate on the two orthogonal unit vectors found in the center grid. As one moves horizontally the vectors rotate rigidly in 9 deg (8/) steps. As one moves vertically the two vectors rotate in opposite directions in 9 deg steps. The vertical position indicates the correlation between the resulting temporal components. At the central point the two vectors are at right angles, so at 5 steps up or down from that point the vectors are either in the same or opposite directions. In going upward from the central row, where the vectors are at right angles the correlations are:, -.3, -.59, -.8, -.95 and -.. That is, in the row in the row above the central row the two time functions have a correlation of -.3. At five rows above the central row the two time functions are anti-correlated. The vector pairs in the middle row, where α and β are equal, represent rigid rotations of the basis temporal-vectors. The rotations along the top and bottom rows are similar to the middle row rotations in that they result in orthogonal temporal components. 7

77 To create the vector components along top and bottom row, the rigid rotation of the basis vectors has to be followed by sign reversal of one resulting component. The significance of the top, middle and bottom rows is discussed in the results section. The rest of the rotations in the space are oblique rotations of the basis vectors. Other important landmarks in the space are the two rows designated as singular. These rows depict regions in the rotation space where the two temporal components become positively or negatively correlated thus leading to unstable dipole solutions. It is often best for practical reasons to avoid estimating dipole solutions at the singular regions because those solutions are unstable. Also it is highly unlikely that the V and V dipoles have identical (or reversed) time functions. The dipole solutions can be more efficiently calculated via simple linear operations on a subset of dipole solutions in the space thereby bypassing the need for computationally expensive process of iteratively estimating dipole solutions for the whole space. The blue shaded quadrants are different only in that one component of each vector pair is reversed in sign. The ribbon metric is different in the two quadrants because of the OCVM sub-metrics but the dipole locations and magnitudes are the same (except for the flipped sign). The same applies to the yellow shaded quadrants. Switching first and second components within the region of the top dotted rectangle makes the region a mirror image of the rectangle just below it. The same applies to the bottom two rectangles. In general, the whole space can be replicated by simple operation performed on one of the four rectangles. At first glance it might not be clear why switching of dipole solutions and the sign reversal of one dipole orientation/temporal component is important in source analysis. In 73

78 fact it is not necessary to keep track of such events for other source analysis procedures however the V/V continuity framework of the Ribbon Metric requires to do so. The Ribbon Metric maps one dipole to V region and the other to V region. Consequently the Ribbon Metric would result in optimal value when proper dipoles are mapped to V and V regions. It is because of this property that the Ribbon Metric solves the registration problem. The orientation continuity assumption between V/V borders in Ribbon Metric makes it sensitive to sign reversal of only one dipole component therefore making the optimal solution have properly aligned V/V regions..5.3 Step Three: Ribbon metric estimation and analysis The plausibility of dipole solutions for each rotation is evaluated using the Ribbon Metric. Ribbon Metric values are determined for all 44 rotations using the methodology described in section.4.3. The Ribbon Metric is then graphically presented as a by pixel image. The pixel with the lowest value marks the most physiologically and anatomically plausible rotation. 74

79 .6 Simulation The source analysis methodology was tested under four different simulation conditions: ) no noise; ) spatial and temporal white noise; 3) model misspecification; 4) spatially and temporally correlated noise. In every simulation condition, 4 potential maps, representing activation by each stimulus patch shown in Figure, was estimated using the equivalent dipoles model. The potential at 7 electrodes, covering roughly one third of the head model surface, was simulated using, unless otherwise noted, an excellent approximation to the four-shell spherical volume conductor model (Berg 994). A head model of radius cm is assumed so that distances can be articulated in centimeters. At least two equivalent dipoles representing sources in V and V, referred to as V and V dipole sources, respectively, were used to create each potential map. The V and V dipole sources corresponding to each stimulus patch were arranged in accordance with the general layout of V and V anatomy described in section

80 Top of the head V dipole V dipole.5.6 Calcarine Sulcus Z (CM) X (CM) Right side of the head Figure 5 Dipole source configuration 76

81 Figure 5 shows the back of the head view of V/V dipole sources corresponding to a group of six stimulus patches along the same eccentricity. The blue dots mark a line on the surface of the equivalent dipole space that is used to represent the cortical surface in and around the calcarine sulcus. Each blue dot is.5 cm away from its immediate neighbors. Consequently dipole sources, represented by open circles, being positioned on every tenth blue dot would be.5 cm away from one another. In general all V dipole sources along the direction marked by the blue dots are placed.5 cm away from other V dipole sources and.375 cm from other V dipole sources. The V dipole sources are.5 cm from one another. The arrow attached to each open circle represents the orientation of that dipole. Four sets of dipole sources shown in Figure 5 are stacked.5 cm apart to create a set of 48 dipole sources corresponding to the whole stimulus array shown in Figure. The details of dipole source parameters are found in Chart. Dipole source magnitudes are slightly modulated so that they are not completely uniform. The modulation of dipole strengths from one pair to another is designed to simulate a realistic scenario where the signal partially self-cancels due to cortical folding. The variability in source magnitudes is not required for proper functioning of the Ribbon Metric rather it is intended to make the topology of the maps more realistic. 77

82 Chart Simulation Conditions Dipole source configurations common to all simulation conditions: Unit volume conductor is used however all numbers are converted to centimeters by assuming a head with radius of cm. X, Y and Z coordinates increase in the direction of right, top, and front of the head. Surface potentials were simulate at 7 evenly spaced electrodes on the backside of the head model covering nearly /3 of the surface. The normalized temporal components for the first and second-dipoles are [.3 ]/ [.3 ] and [.3 ]/ [.3 ] ], respectively. The correlation is.55. The X and Z coordinates of first-dipole locations are in accordance to Figure 5. The Y coordinates are 6.5, -6., -5.5 & -5 cm for first and second-dipoles corresponding to stimulus semi-ring,,3 &4 (see Figure ), respectively. The X and Z coordinates of unit moment vectors ( m =) are in accordance with Figure 5. The Y coordinates are all zero. Two different cases of dipole magnitudes were used. The magnitudes determining the length of moment vectors ( m ) were either: o Even-magnitude case: one for both dipole one and two. o Dominant-magnitude case: and.33 for dipole one and two, respectively. Each magnitude was modulated by adding Gaussian noise with a standard deviation of.5 times the dipole magnitude. Case No noise: No noise was added to the potentials. Case Spatial and temporal white noise: Gaussian noise was added to every spatio-temporal data point. The average signal-to-noise ratio was about. 78

83 Case 3 Misspecification (special case of spatially and temporally correlated noise): The third-dipoles have temporal components equal to first-dipoles. The forthdipoles have temporal components equal to second-dipoles. Location of both dipoles is [-5-5 ]. The same normalized moment vector is used for both dipoles: [ ]. The magnitudes are either: o.5 for dipoles three and four for the even-magnitude case. o.33 for dipole three and. for dipole four in the dominant-magnitude case. Case 4 Spatially and temporally correlated noise (additional independent dipoles) Two more dipoles with different temporal components are added to the twodipole cases Third-dipoles The third-dipoles follow a retinotopic organization. The X coordinates are.8,.6,.4,.4,.6, and.8 cm for third-dipoles corresponding to patches.,.,.3,.4,.5, and.6, respectively. The Z coordinates are -, -.8, -.6,.6,.8, and cm for third-dipoles corresponding to patches.,.,.3,.4,.5, and.6, respectively. The Y coordinates are 6.5, -6., -5.5 & -5 cm for dipoles corresponding to stimulus semi-ring,,3 &4 (see Figure ), respectively. The orientation vectors [x,y,z] are [ ], [ ], [ ], [ ], [ ], and [ ] for third-dipoles corresponding to patches.,.,.3,.4,.5, and.6. The temporal component, [ ]/ [ ], was used for all third-dipoles. Third-Dipole strengths are equal to those of the second-dipoles before the strengths are modulated. Fourth-dipoles The forth-dipoles of the share the same location and orientation. X and Y coordinates, which are and 5 cm, respectively, are the same for all forth-dipoles. The Z coordinates are 3 and 3 for dipoles corresponding to upper and lower visual field patches, respectively. The orientation vectors are [ -] and [ ] for dipoles corresponding to upper and lower visual field patches, respectively. The temporal component, [.4.4. ]/ [.4.4. ], was used for all forth-dipoles. Forth-Dipole strengths are equal to those of the second-dipoles before the strengths are modulated. 79

84 3 RESULTS & DISCUSSION 3. Rotation in the absence of noise The Ribbon Metric is first tested without adding any stochastic noise to the simulated potential data for both even-magnitude and dominant-magnitude cases. Although there is no rotation ambiguity in the absence of noise, it serves as an ideal example for examining the effects of rotation on dipole parameters. Rotation ambiguity is not present because the residual variance space of the fitting procedure is perfectly smooth therefore the minimum is accurately determined even when the space is quite flat due to sources being close. 3.. Even-magnitude case The first simulation case evaluated consists of 4 pairs of dipoles positioned in a way that mimics the general anatomy of V and V. Each dipole pair on average has equal strength, referred to as the even-magnitudes case. A multi-shell spherical head model was used to generate simulated surface potential at 7 electrodes covering about one third of the surface. The details of each simulation can be found in Chart. Ribbon Metric (RM) values are evaluated for 44 rotations and graphically presented as a by image seen in Figure 6. Each pixel in the figure shows the value of RM calculated at a unique rotation angle. The color of each pixel in the image depicts the RM value. The smaller images on left of the dotted line in Figure 6 represents the sub-metrics used to calculate the RM. A detailed description of the topographic features seen in these images is given later in this section. The panel of images on the right of the dashed line in Figure 8

85 6 compares the resulting dipole solutions for all 44 rotations to the original dipole sources. The errors (details given later) between the optimal solutions and original sources is given above every subplot. The four explicitly written vectors below subplots P, Q, and R are the estimated and original temporal components. The top two vectors labeled TCD (R) estimate and TCD (R) estimate are the estimated temporal components shared by all first- and second-dipoles corresponding to stimulus ring. The temporal components corresponding to stimulus rings, 3, and 4 are not shown because they are nearly identical to those of stimulus ring (the differences between the temporal components, which are in radians rounded to the third decimal place, are provided to the right of the asterisks on the same figure). Please note that other analogous figures in sections 3., 3. and 3.3 also present only the temporal components for ring. The two vectors labeled TCD source and TCD source are the original source temporal components for the first- and second-dipoles, respectively (the same temporal component pairs were used for all source pairs). 8

86 A. Ribbon Metric (RM) P. TC (R) error (.3 rad) Q. TCD (R) err (.3 rad) R. TCD (R) err (.43 rad) TCD (R) estimate*: [.96.9 ] TCD (R) estimate*: [ ] TCD source: [.96.9 ] TCD source: [.9.96 ] * TCD of ring is different from rings, 3, & 4 by,, & radians. * TCD of ring is different from rings, 3, & 4 by,, & radians. B. DCM C. DCMD.5 D. DCMD E. DCHM S. loc error (.65 cm).5 T. locd err (.59 cm).5 U. locd err (.7 cm) F. OCM G. OCMD H. OCMD I. OCVM V. orien error (.4 rad) 3 W. oriend err (.34 rad) 3 X. oriend err (. rad) 3 J. GCM K. GCMD L. GCMD M. MCM N. MCMD.5 O. MCMD.5.5 Y. mag error (.4) Z. magd err AA. magd err (.6).6 (.4) Figure 6 Metrics for even-magnitudes no noise condition 8

87 The rotation that results in the most anatomically and physiologically plausible dipole solutions is found by identifying the lowest value in RM. To help the reader, a black arrow is used to point out the optimal RM position in subplot A. Arrows in subplots B to AA point to the same optimal RM position as well (not to the lowest value in each subplot). Subplot P shows that the optimal RM position accurately estimates the original temporal components. This conclusion is based on the fact that the arrow is pointing to the pixel with the darkest blue color. The pixel with the darkest blue color in P represents an error of.45 radians between the estimated and original temporal components. The error is estimated by the root mean square (RMS) of angles between temporal component of solutions and those of the corresponding two sources. It is important to note that due to the sparse sampling of the rotation space, the estimated optimal rotation can be off from the true rotation by as much as.6 radians. The dark blue diagonal regions in subplots Q and R indicate the rotations producing temporal components for dipole one and two that closely match that of the first and second sources, respectively. The Ribbon Metric s performance is furthermore validated by the fact that, as shown by Figures 6 S thru AA, the resulting dipole positions, orientations and magnitudes also closely match that of the original sources. Subplots S, T and U compare the predicted dipole locations to the originals by showing the RMS of differences in distance (in centimeters) between estimated and original locations. Subplots V, W, and X compare the predicted dipole orientations to the originals by showing the RMS of angles (in radians) separating the estimated and original dipoles. Subplots Y, Z, and AA compare the predicted dipole magnitudes to the originals via RMS of Michaelson 83

88 contrasts between estimated and original magnitudes. All error values (rounded to the third decimal place) for the optimal RM rotation can be found above each image. The optimal RM rotation results in location, orientation and magnitude errors of.65 cm,.4 radians, and.4 (4%), respectively. These errors are all due to the coarse sampling of the ribbon space. The RM and its sub-metrics share many similar topological patterns. A small detour in examining these topological features can aid in better understanding how RM works. Figure 7 shows the four lines that roughly depict the ridges consistently seen in RM and its sub-metrics. The ridges are areas in the rotation space where the metrics are exerting high penalty and hence are non-optimal. The dashed line depicts the ridge that often runs through the center of the image in the direction of constant dipole one and varying dipole two temporal components. This diagonal ridge is a result of poor dipole two fit. In the region along the dashed diagonal, the first dipole s temporal component is positioned such that dipole one nearly fits all the energy from the two sources, particularly for dipole sources that are orientated in the same direction, thus leaving the second-dipole to model any residual potential. The reason for the ridge passing though the center of the metric has to do with the way the initial temporal decomposition step is performed. The metrics are constructed such that its center pixel represents the unrotated basis temporal-vectors from the orthogonal decomposition step. In the decomposition step the basis temporal-vectors are positioned with first component closely matching that of the best single dipole fit. The alignment process stabilizes the landscape such that different source analysis runs can be compared. Furthermore, as discussed later the width Michaelson contrast between two values, a and b, would be estimated as follow: (a-b)/(a+b). 84

89 of the ridge and the shape of the adjacent valley can provide important information about the relative strength between the first- and second-sources. The two dotted lines depict ridges where the first dipole is doing a poor fit of the data. In fact the cause of the dotted ridge which is in the direction of varying dipole one temporal component is very similar to the dashed line ridge with the exception that the roles of dipole one and two are switched. It should be noted that the two dotted lines are connected and therefore represent a continuous region. The top of one continues to become the bottom of the other and vise versa. The solid lines depict ridges where the dipole solutions are unstable due to the temporal components becoming identical (positively or negatively correlated). Dipole temporal component varies Dipole temporal component varies Figure 7 Topological features 85

90 Let us now examine the individual sub-metric images. As described in the Theoretical section, the Ribbon Metric is a weighted sum of six sub-metrics (subplots B, F, J, M, E and I): distance continuity metric (DCM), orientation continuity metric (OCM), Gaussian curvature metric (GCM), magnitude continuity metric (MCM), distance continuity across horizontal meridian (DCHM) and orientation continuity about vertical meridian (OCVM). The component metrics each measure the validity of dipole solutions based on assumptions about the anatomical layout and physiology of V and V. DCM: Subplot B of Figure 6 shows the combined effect of first- and seconddipoles best fitting the layout of V and V. Note that the darkest blue region in B coincides very well with the optimal RM rotation (shown by the arrow) and more importantly with the region of lowest error in subplot S (as a reminder, S compares estimated to original dipole locations). Subplot C, labeled DCMD, shows the results of evaluating the distance continuity of the first-dipoles using continuity assumptions for the V region of the ribbon space (for details see section 3.4.3). Similarly, subplot D, labeled DCMD, shows the results of evaluating the distance continuity of the seconddipoles using continuity of assumptions for the V regions of the ribbon space. The sum of DCMD and DCMD does not perfectly equal DCM since the interactions at the V/V border is ignored by the region specific metrics. The dark blue regions running along the left side of the red diagonal regions in Subplot C represent rotations resulting in first-dipole locations that best fit V. It is important to note that the blue regions extend along the direction of varying first-dipole temporal components and constant second-dipole temporal components. It may seem counter intuitive that varying second-dipole temporal components as opposed to the first- 86

91 dipole temporal components result in major changes in first-dipole locations. The explanation for this cross interaction between the temporal component of one dipole and spatial solution of the other lies in equations 9 a-d where T R and E R are functions of α and T R and E R are functions of β. DCHM: Subplot E depicts the DCHM, which is the sub-metric that emphasizes the location continuity across the horizontal meridian. The importance of DCHM is that it can distinguish between V and V dipoles by detecting whether the first or seconddipoles show continuity across the horizontal meridian. OCM: Orientation Continuity Metric (OCM) detects deviation of dipole orientation due to source crosstalk. Note that OCM shown in Figure 6 F has similar ridges as seen in DCM. OCM makes the assumption that nearby sources on the cortex have similar orientations. This assumption is not completely true since the cortical surface is curved, however the assumption gains accuracy, as the center-of-mass of neighboring sources get closer. The assumption also gains validity based on the fact that the curvature of a surface defined by equivalent dipoles is less than the corresponding cortical surface. An equivalent dipole models the sum of many small current sources over a patch of cortex. The summation process in effect acts as a low pass filter consequently smoothing out the sharp corners. Comparing OCM to the corresponding error plot (subplot V) reveals that OCM is not performing optimally. The minimum in OCM does not map to the best orientation solution. V in the simulations is modeled to be more flat than V therefore the metric works effectively when orientation continuity is evaluated for V (OCMD) as seen in subplot H. On the other hand, the metric for orientation 87

92 continuity over V (OCMD), as seen in G, misses its mark since the correct dipole orientation is not always equal between neighboring patches. OCVM: OCVM (subplot I) is a metric that isolates the orientation continuity across horizontal meridian. The purpose of this metric is for identifying the regions (half of the whole rotation space) that result in V and V sources being properly oriented and not opposed in orientation by 8 degrees. GCM: Gaussian curvature metric (GCM) is used to test for continuity of orientation in a less stringent way than OCM and one that is more conducive to the simulated cortical surface and perhaps the real anatomy. GCM results in a high penalty when dipole orientations suggest that the cortical surface is greatly curved along both principle-axes and results in a low penalty when curvature along one of two principle axes is low. Subplots J, K, and L show GCM, GCMD (Gaussian curvature evaluated for dipole one over V ), and GCMD (Gaussian curvature evaluated for dipole two over V ), respectively. GCM exhibits similar ridges as seen in DCM and OCM with the exception of two additional prominent ridges, whose origins are not well understood, running along the direction of varying temporal component of the first dipole. The additional ridges are believed to be artifacts since their position is not consistent as simulation conditions change. GCM, because of the inherent noisiness of derivative based functions, sometimes produces unusual features. As seen in subplot J, GCM is rather devoid of a prominent minimum with the exception of a few isolated single pixel minima. Unfortunately the simulation conditions used for Figure 6 are non-ideal for demonstrating the effectiveness of GCM. Crosstalk between sources, under simulation conditions used for Figure 6, result in dipole orientations rotating in unison about a single 88

93 axis thus resulting in a flat GCM space. As shown in the next section, GCM produces prominent minima when additional sources of noise are added since poor fitting dipoles will follow noise thus creating more prominent changes in curvature. In general GCM has been a very effective component of RM in the simulations. As will be discussed in Chapter 3 GCM is too noisy when applied to real data. MCM: Magnitude continuity metric (MCM) is used as another component of the Ribbon Metric in order to avoid two dipoles from sticking together when the temporal components are highly negatively or positively correlated, hereafter simply referred to as correlated. When the two dipoles are temporally correlated, as some of the rotations dictate (five rows up or down from the central row), their orientations get correlated and consequently magnitude parameters become very large and erratic. The described instability in the parameters is a common problem faced when solving ill-conditioned least squared problems (Hansen 99). In many cases some type of regularization techniques such as Tikhonov regularization is used to place additional constraints on the solution space. MCM (shown in Figure 6) is not just used to ensure that the Ribbon Metric does not favor rotations resulting in correlated temporal components but to also identify the correct rotation. The importance of MCM is further emphasized in section 3.4 where the effects of correlated noise are explored. 3.. Dominant-magnitude case The above discussions of topological features of the RM and its sub-metrics were limited to a case where the two dipole-sources have equal strengths (even-magnitude case). The topology changes when the source strengths are uneven and recognition of such topological feature is informative particularly for analysis of real data. Figure 8 89

94 shows the Ribbon Metric and for a simulation case where the 4 source pairs on average have a V/V ratio of 3-to- (dominant-magnitude case). A. Ribbon Metric (RM) B. DCM C. DCM D D. DCM D Figure 8 Metrics for dominant-magnitude no noise condition. 9

95 The significant change in the topology, as seen by comparing the Ribbon Metrics (subplot A) in Figures 6 and 8, is that the optimal RM position in case of a dominant source is very close to the diagonal red region that passes through the middle. The reason for the optimal region (the dark blue region) being close to the non-optimal region (the red region) is that, in case of a dominant source, the first-dipole basis temporal-vectors end up being very close to the first-source temporal components. For the same reason, the dark blue region in Figure 8 C is much narrower in width than subplot B. Note that the metrics in Figure 6 having an optimal RM rotation position close to the bottom and that of Figure 8 being close to the middle is simply due to switching of dipoles one and two and has no bearing on the source analysis outcome. In the even-magnitude example of Figure 6 the initial choice of which dipole is D is arbitrary. 9

96 3. Rotation in the presence of spatial and temporal white noise The effect of spatial and temporal white noise was tested for both even-magnitude and dominant-magnitude cases however since addition of white noise had similar effects in both cased, only the even-magnitude case is presented in this section. The result of using the Ribbon Metric in presence of white noise is shown in Figure 9 (note that full sets of all metric and error plots for all simulation cases can be found at The general topology of the Ribbon Metric map does not change with addition of noise (Figure 9 a). Overall the Ribbon Metric shows its robustness to noise by identifying the correct rotation. The most notable change from the no noise conditions is that here the RM shows sharper diagonal ridges. The sharpening of the ridges has to do with the fact that when one dipole fits nearly all the energy, the other dipole ends up fitting the noise thus resulting in spurious solutions. A. Ribbon Metric (RM) B. TC (R) error (.37) C. TCD (R) err (.5) D. TCD (R) err (.) TCD (R) estimate*: [ ] TCD (R) estimate*: [.8.96 ] TCD source: [.96.9 ] TCD source: [.9.96 ] * TCD of ring is different from rings, 3, & 4 by.,., &.6 radians * TCD of ring is different from rings, 3, & 4 by.,.9, &. radians Figure 9 Metrics for even-magnitudes white noise condition. 9

97 It is important to note that the topographic features of GCM become more prominent in presence of noise. The topography of GCM changes because in presence of noise the dipoles orientations vary more strongly as temporal components are rotated. Figure compares GCM of white noise and no noise conditions. A. White noise condition B. No noise condition GCM GCM Figure White noise vs. no noise condition (even-magnitudes case) 93

98 In the absence of noise there is no rotation ambiguity if source locations are not completely superimposed. On the other hand, as spatial and temporal white noise is added to the potential data at every electrode, the rotation ambiguity can begin to pose a problem even when locations are not superimposed. In cases where the sources are correlated in location and orientation, presence of noise can lead to formation of false minimum via formation of quadrupoles. A quadrupole, which can result from dipolepairs becoming anti-correlated in orientation and temporal component, better fits the perturbations of the potential field due to noise. In cases where the sources are only correlated in location but not in orientation, a false local minimum can form by the perturbation of the flat residual variance space due to noise. In general, as more noise is added the resolving power of the source localization technique fades and therefore source pairs have to be farther apart spatially for them to be correctly modeled. The rotation ambiguity effect in presence of white noise is shown in Figure where the sum of squared errors (SSE) between model and data for every dipole pair is shown. Each pixel in the each image shows the goodness of fit of a dipole pair corresponding to a stimulus patch at a given rotation. The black arrow points out the position of true rotation, which is shown later to be also the optimal RM rotation. The white arrow points to one of four existing minima in the SSE space. If only the black arrow is visible then the white arrow is in the same position. There are four minima since, as described in detail in the Theory and Methods section, the rotation space has certain redundancies. In other words, the four minima are different only in that either one dipole s temporal component or orientation is flipped (sign reversal) or the two dipoles are switched. All these differences do not manifest themselves in the SSE estimation and therefore result in equal SSE values. The 94

99 single white arrow however shows the minimum that corresponds to the solution that is most consistent with those of its neighbors. There are two reasons for distinguishing between the four minima: ) makes it easier for the reader to see if the optimal rotation coincides with the SSE minima in which case only the black arrow would be visible ) the minimum shown by the white arrow is used for another analysis later in this section. Nearly every SSE map has a minimum that is not lined up with the true rotation. In the case of the SSE maps corresponding to patches along the vertical meridian (top and bottom row of images in Figure ) where the source-pairs are correlated in location and orientation, the false minima are due to formations of quadrupoles. Note that the minima of these SSE maps lie very close to the singular regions. In case of the other SSE maps (rows -5 of images) where the source-pairs are not highly correlated in orientation, the false minima are due to the local minima problem. In fact, the flatness of the SSE space is quite evident by looking at the range of values shown by the colorbar. On the other hand, not all SSE maps corresponding to patches away from the horizontal meridian (rows to 4 in Figure ) exhibit incorrect minima. SSE maps labeled as. and.4 (the number before and after the decimal point represent the stimulus ring and patch number within each ring, respectively) exhibit minima that are in the true rotation position. It is interesting to note that SSE maps,. and.4, correspond to stimulus ring one whose dipoles are closer to the surface of the volume conductor. In fact depth of dipole pairs also contribute to the flatness of the residual variance space and therefore affect the degree to which they are affected by the rotation problem. 95

100 VM HM VM Figure Individual SSE maps for even-magnitudes white noise condition 96

101 The fact that there is varying degree of flatness to the SSE space depending on dipole parameters allows us to overcome the rotation problem by summing over the SSEs of all patches. The resulting grand total SSE (shown in Figure ) has four minima (darkest blue pixels) one of which is at the correct rotation position. The grand total SSE is able to lessen the effect of rotation ambiguity because ) the steeper SSE maps (maps corresponding to dipole-pairs with greater separation) dominate over the flatter ones and ) the averaging process reduces the uncertainty. In fact, this is the basis for the common time function methodology used by Slotnick et al (999). Grand total SSE Figure Grand total SSE map for even-magnitudes white noise condition. 97

102 To provide intuition for what the effect of crosstalk is on the dipole locations, Figure 3 shows the dipole locations for: ) original sources, ) optimal RM solution, and 3) patchby-patch source analysis. The patch-by-patch spatio-temporal source localizations are based on the minima of SSE maps for individual patches. The white arrows in Figure mark the rotation used in acquiring the locations. Identifying the proper minimum is intended to isolate the effect of crosstalk on dipole locations and not switching of dipoles one and two. The mesh vertices of Figure 3 represent dipole locations. The polygoncoloring scheme used is to aid in identifying different regions of the Ribbon metric. The polygon color changes as one moves along the length of the Ribbon space. Optimal RM solution Solution based on SSE of individual patches Original sources 6 V LVM 4 V HM Z Z Z V UVM V X -6 Y -5 - X -8-6 Y X -6 Y -5 Figure 3 Locations for even-magnitudes white noise condition 98

103 The main point of Figure 3 is not that Ribbon Metric analysis is better than patch-by-patch source analysis for generating dipole locations that are more in accordance with those of original sources. Rather, the objective of this chapter is to solve for accurate temporal components and to a lesser extent accurate source locations. Dipole locations for the optimal RM rotation is shown in order to make a point about the Ribbon Metric being able to find the correct temporal components even when the estimated dipole locations are somewhat noisy. The next challenge may lie in determining Ribbon Metric s tolerance to noise above which the error in dipole parameters would undermine the continuity assumptions. 99

104 3.3 Rotation in the presence of model misspecification The phrase model misspecification means systematic error due to volume conductor and/or electrode position misspecifications. In the presence of model misspecification, dipole errors in multi-source analysis are often greater than the dipole errors in single source analysis. Model misspecification is a source of noise that, when unaccounted for by the fitting procedure, creates a rotation error in the dipole solutions. Since model specification can be caused by many factors such as unaccounted inhomogeneities and anisotropies of the brain, and electrode position error, it is rather difficult to recreate all the characteristics in simulations. Two types of misspecification are used to test the Ribbon Metric: ) using a different volume conductor model in the inverse solutions than is used to generate the simulated data, and ) using third- and forth-dipoles having the same temporal component as the first- and second-dipoles, respectively Misspecification based on number of shells in spherical models; tested for even-magnitude scenario The Ribbon Metric was tested using a multi-shell volume conductor model (Berg 994) in fitting simulated data generated via a single shell volume conductor model (Brody 973). The D lim (in equation (3)) used in estimating DCM were quasi-ary corrected (Ary 98) by reducing them by 3%. All other experimental conditions are identical to the no noise even-magnitude condition. Figure 4 shows that the Ribbon Metric results in accurate temporal components. The RMS error in temporal components between those of estimated and source is.9 radians.

105 A. Ribbon Metric (RM) B. TC (R) error (.5) C. TCD (R) err (.3) D. TCD (R) err (.68) 3 TCD (R) estimate*: [.96.7 ] TCD (R) estimate*: [ ] TCD source: [.96.9 ] TCD source: [.9.96 ] * TCD of ring is different from rings, 3, & 4 by,, & radians. * TCD of ring is different from rings, 3, & 4 by,, & radians. Figure 4 Metrics for even-magnitudes misspecified model shell case Optimal RM solution Solution based on SSE of individual patches Original sources.5.8 V LVM V.4.55 HM Z Z Z V UVM V X -3 - Y Y X X -6 Y Figure 5 Locations for even-magnitudes misspecified model shell case

106 Zhang et al. (994) reported that such misspecification results in uncorrectable errors. Since they only used residual variance of potentials in their minimization procedures they did not have the necessary information to disambiguate the rotation. Looking at the locations of sources in Figure 5 one can see that once the true rotation is found then the dipole locations are different from those of dipole sources by only a systematic bias towards the center of the head model. Note that there is no jitter in the estimated locations. The systematic errors in locations are correctable by moving each dipole in the radial direction away from the center by an Ary factor. The central point is that misspecification that results in global bias of the solutions can easily be overcome by the Ribbon Metric Misspecification based on additional dipoles; tested for evenmagnitude scenario The Ribbon Metric was tested in the presence of misspecification generated using two additional dipole sources placed in the opposite hemisphere. The third- and forthdipole sources have orientation in the direction of the Y-axis and magnitudes equal to /8 of first and second dipoles used in the even-magnitude case. The third and fourth dipole sources have the same temporal components as the first- and second-dipoles. The details of experimental conditions can be found in Chart. As shown by Figure 6 the RM results in accurate temporal components for the first- and second-dipoles. The estimated temporal components (of ring ) differ from the originals by an RMS error of about.5 radians. Recall that the sampling of the ribbon metric is in units of pi/=.57 radians. The landscape of the Ribbon Metric remains very similar to the white noise and no noise conditions.

107 A. Ribbon Metric (RM) B. TC (R) error (.6) C. TCD (R) err (.35) 3 D. TCD (R) err (.8) 3 TCD (R) estimate*: [.97.5 ] TCD (R) estimate*: [ ] TCD source: [.96.9 ] TCD source: [.9.96 ] * TCD of ring is different from rings, 3, & 4 by,, & radians. * TCD of ring is different from rings, 3, & 4 by,, & radians. Figure 6 Metrics for even-magnitudes misspecification dipole case 3

108 Unlike the white noise case, the misspecification causes rotation ambiguity in all 4 dipole pairs. Figure 7 shows the SSE space resulting from fitting every dipole pair to the data. As a reminder, the black arrow in every image points to the true rotation position and the white arrow point to the most relevant one of the four minima in the image. The black and white arrow do not coincide in any of the images indicating that the dipole solution based on individual SSE maps do not result in the correct rotation. The reason is that the correlated noise used to mimic misspecification has sufficient amount of energy to create false minima in the flat SSE space VM HM VM Figure 7 Individual SSE maps for even-magnitudes misspecification dipole case. 4

109 Summing over the entire individual SSE maps results in a grand total map shown in Figure 8 whose minima does not coincide with the true rotation positions. This shows that using the common temporal component assumption in the formulation used by Slotnick et al. cannot overcome the rotation problem under the present simulation conditions. Grand total SSE Figure 8 Grand total SSE map for even-magnitudes misspecification dipole case. 5

110 Looking at the resulting locations for optimal Ribbon Metric rotation compared to the original locations (Figure 9) shows that the overall distance continuity between the neighboring dipoles is sufficiently maintained even though the locations are jittered. The Ribbon Metric would at some point fail; as the error in locations, orientations and magnitudes become so large that the continuity assumptions no longer hold. Understanding the tolerance level for the Ribbon Metric is important and requires investigation using more realistic misspecification models. Optimal RM solution Solution based on SSE of individual patches Original sources.5 V LVM V.5.5 HM V UVM V Z Z Y X Z X Y X -6 Y -5 Figure 9 Locations for even-magnitudes misspecification dipole case 6

111 One of the most notable changes to the sub-metrics under misspecification condition is that the red diagonal regions in DCMD and DCMD become less continues in intensity as one moves along their lengths. Figure compare the DCM, DCMD and DCMD maps resulting from the misspecification condition to that of the no noise condition. As marked on the Figure the less intense (lighter red) regions along the diagonals are close to the top, middle and bottom rows in the rotation space where the dipoles temporal components are orthogonal. The discontinuity in the diagonal regions is a sign of misspecification. As the effect of misspecification gets larger by increasing the strength of third and forth dipole sources (the misspecification dipoles), the less intense section of the red diagonal regions lose more intensity and eventually the diagonal regions become completely discontinuous (data not shown). Less intense A. DCM B. DCM D C. DCM D More intense Misspecification case D. DCM E. DCM D F. DCM D No Noise case Figure Misspecification dipole vs. no noise case (even-magnitudes) 7

112 3.3.3 Misspecification based on additional dipoles; tested for the dominant-magnitude scenario The Ribbon Metric was tested in the presence of model misspecification for the dominant-magnitude case. Misspecification was introduced by using third- and forthdipole sources positioned in the left hemisphere with temporal components equal to those of first- and second-dipoles. The misspecification dipoles have orientations in the direction of the Y-axis. The magnitudes of the third- and forth-dipole sources are /3 of the first and second dipole magnitudes. The resulting Ribbon Metric and the temporal component (TC) error maps are shown in Figure (subplots A, F, G, and H). The resulting Ribbon Metric map is similar to the corresponding no noise case. In subplot F, the optimal RM rotation indicated by the black arrow does not coincide with the true rotation position (the darkest blue position). Looking at subplots G reveals that the black arrow falls into the dark blue valley indicating that the first dipole is correctly modeling the temporal component of the firstsource. On the other hand, looking at subplot H reveals that the second-dipole is not accurately estimating the temporal component of the second source. Incidentally, the metric does a reasonable job of getting the correct positions of the first- and secondsource as indicated by the arrows in subplots J and K pointing to the dark blue areas. The second dipole locations accurately match those of the second-sources (tight blue region around the arrow). At first glance it may seem odd that the locations of the second-source is best modeled using an incorrect temporal component. However the explanation for this observation not only provides us with a powerful insight into problems caused by 8

113 misspecification but also a possible way to improve multi-source analysis. It turns out that the second dipole s temporal component is orthogonal to that of the first-dipole since the temporal components are generated by a rotation that lies in the middle row of the rotation space. All the pixels in the Ribbon Metric along the middle row of the map result in orthogonal temporal components. When the temporal components of the first- and second-dipoles are orthogonal and the first-dipole s temporal component is correctly modeled then the second-dipole is blind to the first-dipole s misspecifications. This silencing effect provides the Ribbon Metric with a strong cue for finding the temporal component of the first-source. A. Ribbon Metric (RM) F. TC (R) error (.359) G. TCD (R) err H. TCD (R) err (.83) 3 (.5) TCD (R) estimate*: [.98. ] TCD (R) estimate*: [ ] TCD source: [.96.9 ] TCD source: [.9.96 ] * TCD of ring is different from rings, 3, & 4 by,, & radians. * TCD of ring is different from rings, 3, & 4 by,, & radians. B. DCM C. DCMD.5.5 D. DCMD E. DCHM I. loc erro (.86) J. locd err (.73) K. locd err (.) 4 3 Figure Metrics for dominant-magnitude misspecification dipole case 9

114 Because of the silencing effect of the first-source misspecification coupled with the fact that the first-source is dominant (has a magnitude three times the second-dipole), it is possible to accurately estimate the temporal component of the first-source at a much higher misspecification strength than in the even-strength case. In fact using misspecification dipoles at /3 the magnitude of the first- and second-sources in the evenmagnitude case, would result in the Ribbon Metric failing to accurately estimate the true rotation (results not shown).

115 3.4 Rotation in the presence of spatially and temporally correlated noise Spatially and temporally correlated noise present under realistic recording conditions (hereafter referred to as correlated noise) often contributes to source analysis error. Many possible sources of correlated noise include undesirable and unaccounted sources of brain activity, alpha waives, muscle noise and heart pulsation. Many of such sources of noise are not time-locked to the stimulus and are sufficiently reduced by the stimulus-response cross-correlation used in estimating the evoked response. The discussion in this section is limited to dealing with sources of correlated noise that are time locked to the stimulus thereby contaminating the m-sequence based VEP data. The activity could either come from nearby cortical areas such as V3 or far away cortical regions such as the parietal cortex. Addition of correlated noise was tested for both the even-magnitude and dominant-magnitude cases. Luckily, with m-sequence simultaneous stimulation of multiple patches (6 patches for the data of Chapter 3) only brain regions with fine-grained retinotopy will produce a significant VEP Even-magnitude case Correlated noise was generated in the simulated data by addition of two more dipole sources (third and forth dipoles) with magnitudes equal to those of the evenmagnitude scenario. The third source was positioned where V3 source is expected to be which is either immediately above or below V for a source corresponding to the lower and upper visual field, respectively. The fourth source was meant to roughly mimic the activity from the parietal regions therefore was positioned farther away from the first-

116 and second-sources than were the third-sources. This is a worst case scenario, since it is expected that after V the source strength decreases substantially for two reasons: ) Neural receptive fields become larger so that the responses to individual patches become weaker. ) The responses to the high spatial frequency and high temporal frequency alternating checkerboards are reduced in the higher visual areas. Figure shows the resulting Ribbon Metric and the four pairs of temporal components. Each pair is representing the temporal components of the first- and seconddipoles corresponding to each stimulus ring. The resulting temporal components are different for each ring therefore four sets of temporal component error plots are presented by subplots B thru E3 to the right of which the actual temporal component values are presented. The original source temporal components for all four dipoles are found just below subplot A. All eight resulting temporal components have errors of.57 radians or higher (the error values are found above every error subplots B, B3, C, C3, D, D3, E and E3) which indicates that the optimal RM rotation badly fails in estimating the temporal components of the first- and second-dipoles. There are two main causes of the failure. Firstly, the decomposition of the sources into two orthogonal components is systematically biased by presence of other sources of equal strength (strong correlated noise). Therefore, the temporal components of the original V/V sources would not lie within the space spanned by the temporal component of the basis temporal-vectors. In other words, there exist no point on the rotation space that would result in the correct temporal components. This can be seen by the fact that the minima in the error plots have values significantly higher than zero. The second cause of the failure is that the spatial dipole parameters are badly influenced by the correlated noise and therefore the optimal

117 RM position does not result in the true rotation. This can be seen by the fact that the arrows pointing to the optimal RM position do not coincide with the trough of the error plots. A. Ribbon Metric (RM) 4.5 TCD source: [.96.9 ] TCD source: [.9.96 ] TCD3 source: [ ] TCD4 source: [ ] B. TC error:.89 B. TCD error:. Ring.5 C. TC error:.6 C. TCD error:.57.5 Ring.5.5 D. TC error:.9 D. TCD error:.5 Ring 3.5 E. TC error:.8 E. TCD error:.96 Ring B3. TCD error:.75 C3. TCD error: D3. TCD error:.77 E3. TCD error: TCD estimated: [ ] TCD estimated: [ ] TCD estimated: [ ] TCD estimated: [ ] TCD estimated: [ ] TCD estimated: [ ] TCD estimated: [ ] TCD estimated: [ ] Figure Metrics for even-magnitude correlated noise case 3

118 It is also important to note why the four temporal component pairs are different from one another while the sources for all stimulus rings had identical temporal components. The implications of this problem could be that our temporal yoking mechanism described in the methods section could result in artifacts. The reason behind the differences has to do with the source magnitudes being slightly modulated from one dipole to the next, consequently the subspace defined by the basis temporal-vector can be slightly different from one ring to another. For example, the subspace in case of one ring could pick up more of the third source than in other rings because the average strength of the third sources corresponding to a stimulus ring happens to be more than that of other rings. The fact that in simulations only six stimulus patches per ring are used versus patches used in real data would suggest that this particular problem would be worse in simulations. The more patches the better since the averaging process evens out the effect of sources with aberrant strengths. It is important in case of the real data to pay particular attention to temporal components from different stimulus rings to make sure that the differences are systematic as one moves from the inner to outer rings. For example seeing a systematic latency change in the temporal components as one moves from the inner ring to the outer ring would not be a cause for concern since the temporal yoking mechanism was designed with that very physiologically plausible effect in mind Dominant-magnitude case The dominant-magnitude case was tested in presence of correlated noise. The average magnitudes of the third and forth sources were equal to the average of the second source magnitudes. The spatial and temporal parameters of the third and forth sources were identical to the above simulation. Figure 3 shows the resulting Ribbon Metric, 4

119 temporal component error plots, and temporal component values. The four first-dipole temporal components corresponding to the four stimulus rings were very close to the original source temporal components. They contained errors ranging from. to.5 radians (values above subplots B, C, D, and E), which is within the error size of.6 radians caused by the sparse sampling of the rotation space. On the other hand the second-dipole temporal components contain large errors ranging from.6 to.63 radians. One of the reasons why the first-dipole temporal components being reasonably accurate is that each subspace spanned by the basis temporal-vectors is nearby the corresponding first-source temporal components. This can be seen by the fact that the minima in TCD error plots (subplots B, C, D, and E) have a value close to zero. However, the second-source temporal components are very distant from the subspace spanned by the basis temporal-vectors. Therefore there exist no rotation within the defined subspace that would result in accurate temporal components for both dipoles. 5

120 A. Ribbon Metric (RM) 4.5 TCD source: [.96.9 ] TCD source: [.9.96 ] TCD3 source: [ ] TCD4 source: [ ] Ring Ring Ring 3 B. TC error:.44 C. TC error:.43 D. TC error:.45 B. TCD error: C. TCD error:. 3 D. TCD error: B3. TCD error:.6.5 TCD estimated: [ ].5 TCD estimated: [ ] C3. TCD error:.6.5 TCD estimated: [ ].5 TCD estimated: [ ] D3. TCD error:.6.5 TCD estimated: [ ].5 TCD estimated: [ ] Ring 4 E. TC error:.46 E. TCD error: E3. TCD error: TCD estimated: [.95 TCD estimated: [ ].9 ] Figure 3 Metrics for dominant-magnitude correlated noise case 6

121 One of the interesting occurrences with the sub-metrics in presence of correlated noise is that the DCM map minimum is found near the singular regions and not at the optimal RM rotation position. In the dominant-magnitude case, this occurrence provides important insight, which is also talked about in Chapter 3, regarding the robustness of first-dipole solutions. The white arrows in Figure 4 subplots point to the position of DCM minimum (subplot B) whereas the black arrows show the optimal RM rotation position. The optimal RM position does not coincide with the DCM minimum because MCM (subplot F) exerts a high penalty in and around the singular regions. In fact, MCM has a minimum that correctly points out the correct rotation. This demonstrates the importance of MCM in presence of correlated noise. The locations of the first-dipoles are not very different when using the rotation pointed to by the white arrow versus using the optimal RM rotation (data not shown). Both of these rotations lie closely along the valley seen in DCMD map (subplot C) where the locations of the first dipoles are nearly optimal. When there are additional sources (correlated noise) affecting the locations of the seconddipoles thereby disrupting their relative continuity, DCM finds the rotation near the singular regions optimal since the second-dipoles maintain their relative continuity by simply sticking to the first-dipoles. In this case, MCM plays a very important role in estimating a rotation that mitigates the contamination of first-dipole solutions by extrastriate sources and likewise the contamination of second-dipole solutions by the V source. 7

122 A. Ribbon Metric (RM) B. DCM C. DCMD.5 D. DCMD E. DCHM 4 I. loc error (.3).5 J. locd err (.53) K. locd err (.88).5 F. MCM F. MCMD.5 H. MCMD L. mag error (.357) M. magd err (.84) N. magd err (.73) Figure 4 8

123 3.5 Summary of discussions Spatio-temporal source analysis (SA) based on the Ribbon Metric was tested under various noise conditions. These tests not only provided information about the reliability of the procedure under different noise conditions but also provided valuable insights into how the Ribbon Metric and sub-metrics function. The following summarizes the important points from the above results and discussions. A critical part of the Ribbon Metric based source analysis is the decomposition of the basis temporal-vectors. Since the basis temporal-vectors are used to define the subspace containing the temporal solutions, their accuracy directly impacts the outcome. It was shown by the correlated noise simulations that the subspace used to model the V/V sources can be greatly biased by other extrastriate sources. In the even-magnitude case, the correlated noise adversely affects the solutions of the first- and second-dipoles. In the dominant-magnitude case, the solutions for the first-dipoles are potentially reliable but not that of the second-dipoles. In this case the dominating strength of the first-dipole forced the subspaces to be sufficiently near the first-dipole temporal components thus resulting in accurate temporal solutions. One way to overcome the problem caused by correlated noise is to extend the rotation space to a three-dimensional space parameterized by six rotation parameters. Then similar search strategies as described by this chapter can be used in estimating the correct rotation in the six-dimensional Ribbon Metric map. Extending to three basis temporal-vectors would likely allow for the V source to be isolated if its strength is dominating relative to other extra-striate sources. Alternatively, selective filtering 9

124 approaches can be used to isolate the sources of interests. For example using the Laplacian of the potential field can potentially reduce the effect of deep or far away sources. In case of the correlated noise simulations used here, using the Laplacian of the potential field may reduce the effect of the fourth dipole-source, which was used to mimic a relatively faraway parietal source. Another critical part of the methodology is how the different dipole sets are grouped to share temporal components. This chapter groups the dipoles corresponding to the 6 stimulus patches within a semi-ring to share the same pair of temporal components. A group consisting of only 6 dipole-pairs may too few since, as shown by section 3.4, the variability in source strengths may result in inconsistencies between the 4 different subspaces. To over come this problem, all 4 dipoles could be grouped together however since this chapter was meant as a prelude to using the real data, it was decided not to do so. Slotnick (998) has shown that the temporal responses are likely to be different at different eccentricities. In chapter three, dipoles corresponding to stimulus patches are grouped which should lessen the inconsistencies between the subspaces corresponding to different rings. Another important part of the SA methodology is the assumptions made by the sub-metrics. The simulations have show that OCM results in a minimum that is not in agreement with the true rotation. OCM fails because it is too constrained. The cortical folding is likely to result in neighboring sources that are not fully correlated in orientation. For this reason OCM s weighting was set to zero. However it should be noted that OCVM, which is a component of OCM, is a very critical part of the Ribbon Metric. OCVM, which assures orientation continuity across the vertical meridian, is used to

125 determine the relative sign of the direction of the V and V sources. DCM, GCM, MCM, and DCHM were determined to be key components of the Ribbon Metric calculation. It is important to note that the Ribbon Metric works because of the continuity in spatial solutions. However, the Ribbon Metric can tolerate some deteriation in spatial continuity. In other word, the Ribbon Metric is able to accurately estimate the temporal components even when the spatial solutions are somewhat noisy. Another important discussion is how SA deals with model misspecification. The errors caused by the misspecification in multi-source analysis can be attributed to two types of effects. The first effect, referred to as the direct effect, causes errors in the spatial solutions because the surface potentials are incorrectly mapped to the sources. The errors due to the direct effect would exist even if the solutions are constraint to the correct temporal components. The second effect, referred to as the indirect effect, has to do with the misallocation of variance (or crosstalk) between several dipoles fitting several sources. The misallocation is due to the fact that each dipole would never completely fit the potential field of the corresponding source therefore leaving some residual variance. This residual variance due to each fit can interact with other dipoles resulting in spatial errors that are much larger than if each source were to be isolated and fitted by a single dipole. This crosstalk or the misallocation of variance can also result in large error in temporal solutions. The Ribbon Metric, in the absence of correlated noise, can easily overcome the indirect effect of model misspecification as long as the direct effect does not undermine the continuity assumption. In a sense, the Ribbon Metric side steps one of the major problems caused by model misspecification. The model shell misspecification case

126 provided a perfect example where the errors due to the direct effect are very systematic therefore the Ribbon Metric easily overcame the errors due to the indirect effect. However in general, the errors due to the direct effect are not as symmetric and systematic as those caused by the model shell misspecification case and therefore could present some problems for the Ribbon Metric. Understanding the direct effect of model misspecification is therefore an important part of the future development for the Ribbon Metric methodology and would require further investigation.

127 4 CONCLUSION The source analysis (SA) methodology described in this chapter paves the way for accurate estimation of dipole solutions in V and V. The dipole solutions for V and V can be, due to the rotation problem, very inaccurate when using conventional source analysis methods. The simulations show that the SA method is able to accurately decompose the spatio-temporal signal into V/V sources components under noise and model misspecification conditions. The presence of correlated noise due to extra-striate sources can lead to incorrect temporal component estimates. However it was shown that if the V sources are dominating in strength, its temporal components can be reliably estimated even in the presence of correlated noise. 3

128 5 REFERENCES Ary, J.P., Klein, S. A. and Fender, D. H. Location of sources of evoked scalp potential: correction for skull and scalp thickness. IEEE trans. On Biomed. Eng. 988: Baillet, S., Mosher, J.C., and Leahy, R.M. Electromagnetic brain mapping. IEEE signal processing magazine,, 4-3. Baseler, H.A., Sutter E.E., Klein, S.A. and Carney, T. The topography of visual evoked response properties across the visual field. Electroencephalography Clin. Neurophysiology, 994, 9: Berg, P., and Scherg, M., A fast method for forward computation of multiple-shell spherical head models. Electroencephalography and clinical Neruophysiology, 994, 9: Bevington, P. R., Robinson, D. K. Data reduction and error analysis for the physical sciences. Second edition. WCB McGraw-Hill. 99. Brody, D.A., Terry, F.H., Ideker, R.E. Eccentric dipole in a spherical medium: generalized expression for surface potential. IEEE Trans. Biomed. Engineering. 973, : Clark, V.P., Fan, S., and Hillyard, S.A. Identification of early visual evoked potential generators by retinotopic topographic analyses. Human Brain Mapping, 995, : Engel, S.A., Glover, G.H., and Wandell, B.A. Retinotopic organization in human visual cortex and the spatial precision of fmri. Cerebral Cortex, 997, 7: 8-9. Fender, D. H. Models of the human brain and the surrounding media: their influence on the reliability of source localization. 99, Gevins, A.S. Dynamic functional topography of cognitive tasks. Brain Topography, 989, : Gevins, A.S., Bricket, P., Costales, B., Le, J. and Reutter, B. Beyond topographic mapping: towards functional-anatomical imaging with 4-channel EEGs and 3- DMRIs. Brain Topography, 99, 3: Hansen P.C. Regularization Tools: A Matlab package for analysis and solution of discrete ill-posed problems. Technical University of Denmark

129 He, B., and Musha, T. Effects of cavities on EEG dipole localization and their relations with surface electrode positions, Int. J. Biomed. Comput. 989, 4: Hill, R. Elementary linear algebra with applications. Third edition. Saunders College Publishing, 996. Hjorth, B. Online transformation of EEG scalp potentials into orthogonal source derivations. Electroencephalography and Clinical Neurophysiology. 975, 39: Hjorth, B. Source derivation simplifies topographical EEG interpretation. Am. J. EEG Technology. 98, : -3. Horton, J.C., and Hoyt, W.F. The representation of the visual field in human striate cortex. Arch Ophthalmology, 99, 9: Jewett D., Zhang, Z. Multiple-generator errors are unavoidable under model misspecification. Electroencephalography Clin. Neurophysiology, 995, 95: Katznelson, R. EEG recording electrode placement and aspects of generator localization. In: P. Nunez (Ed.), Electric Fields of the Brain: The Neurophysics of EEG. Oxford University Press, New York, 98. Klein, A. S. and Carney, T. The usefulness of the Laplacian in principal component analysis and dipole source localization. Brain Topography, 995, 8: 9-8. Law, S. and Nunez, P. Quantitative representation of the upper surface of the human head. Brian Topogr., 99, 3: Mosher, J. C., Spencer, M. E., Leahy, R. M., and Lewis, P. S. Error bounds for MEG and EEG source localization. Electroenceph. and Clin. Neurophys. 993, 86: Nunez, P.L. Electrical field of the brain. Oxford University Press: New York, 98. Roth, B., Balish, A., Gorbach, A. and Sato, S. How well does a three-sphere model predict positions of dipoles in a realistically shaped head? Electroencephalography Clin. Neurophysiology, 993, 87: Scherg, M. Fundamentals of dipole source analysis. In Grandori, F., Hoke, M., and Roman G.L. (eds): Auditory evoked magnetic fields and electric potentials. Basel, Karger. 99, pp Scherg, M., and Berg, P. Use of prior knowledge in brain electromagnetic source analysis. Brain Topography, 99, 4: Slotnick, S.D., Klein, S.A., Carney, T., and Sutter, E. Electrophysiological estimate of 5

130 human cortical magnification. Clinical Neurophysiology,, : Slotnick, S.D., Klein, S.A., Carney, T., Sutter, E., and Dastmalchi, S. Using multistimulus VEP source localizations to obtain a retinotopic map of human primary visual cortex. Clinical Neurophysiology, 999, : Supek, S., and Aine, C. J. Simulation studies of multiple dipole neuromagnetic source localization: Model order and limits of source resolution. IEEE transactions on Biomed. Engineering, 993, 4: Van Oosterom, A., History and evolution of method for solving the inverse problem. Journal of Clinical Neurophysiology, 99, Zhang, Z., and Jewett, Don. DSL and music under model misspecification and noiseconditions. Brain Topography, 994, 7: 5-6. Zhang, Z., and Jewett, Don., and Goodwill, G. Insidious errors in dipole parameters due to shell model misspecification using multiple time points. Brain Topography, 994, 6:

131 CHAPTER 3 : SOURCE ANALYSIS OF VISUAL EVOKED POTENTIALS Summary Accurate estimation of temporal maps for human visual cortical activation has proven to be a problem. Brain imaging techniques, such as fmri and PET, that rely on hemodynamics have poor temporal resolution. E/MEG based temporal mapping techniques have thus far proven to contain insidious errors. The neural generators of early cortical areas are spatially and temporally correlated making inverse solution for sources of E/MEG signal ill conditioned. This chapter uses a novel method described in Chapter, which places biological constraints in order to reduce crosstalk between close by sources, to decompose pattern reversal evoked potentials into two dipole components. The pattern-reversal evoked potentials were collected using multi-stimulus m-sequence technology. The stimulus was a cortically scaled dartboard array of 48 patches. 48 dipole pairs corresponding to each stimulus patch was used to model the data. First dipoles were 7

132 found to be dominating in strength and of V origin. Second dipoles were found to model a mixture of V and higher extra-striate sources. 8

133 INTRODUCTION Studying the mechanisms by which the human visual cortex deciphers and encodes visual information necessitates the need for non-invasive methods for spatiotemporal mapping of cortical activity. Imaging techniques such as Positron Emission Topography (PET) and function Magnetic Resonance Imaging have been used to spatially map the early visual areas including V and V (Corbetta 99, Zeki 99, Watson 993, Sereno 995, Deyoe 996, Engel 997). However temporal mapping of cortical activity in the visual areas has proven to be very difficult. PET and fmri rely on hemodynamic response therefore limiting their temporal sensitivity to within seconds. The need for accurate timing of activation has led many vision researchers (Heinze 994, Simpson 995, Baillet, Martinez ) to combine spatial maps with temporal information from other techniques such as Electro/Magneto-encephalography (E/MEG). E/MEG is able to record brain activity in the millisecond range, however, accurate decomposition of the signal into separate sources has been difficult. As was discussed by Chapter, the visual cortex has several closely positioned and simultaneously active cortical areas whose signal superposition can lead to solution ambiguity (Clark 995). Nevertheless, several researchers have collected timing information by assuming that during certain time windows, determined by the shape characteristics of the signal, only one or two cortical areas are predominantly active (Clark 995). This assumption, which reduces multi-source solution ambiguity, allows for limited timing estimation of cortical activation. An example of such study is the combined use of fmri and EEG by Martinez () to study the effects of attention modulation of activity in V. In spite of 9

134 attempts to characterize and validate the origins of such M/EEG signal epochs, it is unlikely that other sources are sufficiently silent during such an epoch. Moreover, segmenting M/EEG signal into smaller epochs can only provide course sequencing of events and not complete temporal maps. Two main issues must be resolved before complete temporal activation maps of visual cortex using M/EEG can be attained and combined with fmri spatial maps. The main issue involves solving the ambiguity caused by the superposition of multiple sources. The problem posed by the multi-source ambiguity, referred to as the rotation problem, has been shown in simulation studies (Mosher 993, Zhang 994) to be the most significant source of errors in source analysis. If the error due to the rotation problem were overcome, the next challenge would be to accurately register each source component with its respective fmri locus. The registration of EEG sources with those of fmri is challenging because of the false matches due to the close proximity of visual areas. Even when the rotation problem is overcome, model misspecification can still result in global warping of location and orientation of dipole sources making it difficult to identify corresponding fmri source loci. The model misspecification refers to the inaccuracy inherent in today s forward models. The most comprehensive EEG based spatio-temporal mapping of V to date has been performed by Slotnick (Slotnick 998; Slotnick 999). Unlike previous attempts using only 8 or fewer number of stimuli presented one at a time (Gevins 996, Aine 996), Slotnick used a dense stimulus array in order to enforce a powerful constraint in fitting a single source to the response evoked by every patch. The constraint used was to assume a common temporal component for the source dipoles evoked by iso-eccentric 3

135 patches. The common time constraint has two main benefits. First, it rescues dipoles that are weak due to signal cancellation around cortical folds. Second, it allows the dipole sources in V that are farther from the corresponding V sources, hence suffering to a lesser degree from rotation problem, to help disambiguate others that are closer. Dipole sources evoked by patches around the vertical meridian are particularly close to one another therefore suffer most severely from rotation ambiguity. The spatial maps found using this method were retinotopic and in agreement with the general anatomy of Vand in agreement with fmri based spatial maps of the same subjects (Baker 3). The temporal maps from the six stimulus annuli (group of patches at the same eccentricity) cannot be validated by other techniques but it is important to note that they were consistent from one stimulus annulus to another. The temporal map consistency from one stimulus annulus to another would suggest that all source components were in a similar rotation state hence the solution space was not badly degenerate. However, simulated source analysis using this technique (results not published) revealed that the common time constraint alone does not completely disambiguate the source components and the spatio-temporal maps of Slotnick did suffer from the rotation problem to a limited degree since the EEG signal was dominated by a V source. However second dipole solutions using standard methods to fit V sources, would be adversely affected by V source contamination. This chapter describes and tests, using evoked potential data from two subjects, the framework for solving the rotation and registration problem encountered when solving for sources of V and V. The described methodology, which has already been 3

136 validated by simulations (Chapter ), places additional anatomical and physiological constraints to disambiguate the rotation problem. 3

137 MATERIAL AND METHODS. VEP data collection The visual evoked data used in this chapter is the same used by Slotnick et al. (999). VEP data was collected from two healthy adults (T.C. and H.B.) with best corrected visual acuity of / or better. An array of 43 and 48 electrodes for T.C. and H.B., respectively, were placed on the back of the head. The full electrode array, shown in Figure, is a denser version of the posterior - positions (Jasper, 958). Electrode czp was used as a reference. All electrode impedances where kept below 5 kω during recording. The evoked potentials sampled at 6 Hz were band-pass filtered at a setting of.5- Hz. Figure Electrode array From Slotnick

138 Subjects fixated on the center of a dartboard shaped visual stimulus with sixty checkerboard patches (Figure ). All 6 cortically scaled patches were simultaneously modulated at 75 Hz using shifted versions of the same m-sequence (Baseler 994, Slotnick 999). An m-sequence is a pseudo-random sequence of s and s representing the two pattern reversal states. A commercial software application, Veris ( was used to calculate the second order kernel responses of every electrode to every stimulus patch. The second order kernel, which is calculated by crosscorrelating the response of an electrode with the m-sequence of a patch (Sutter 99), is analogous to the pattern reversal VEP (Baselar 997). An m-sequence of length 6 (65,536) was presented in 6 segments, each lasting 55 seconds followed by a minute long relaxation period. The VEP data for each subject is a grand average of four to six separate recoding sessions. Figure Stimulus From Slotnick

139 . Source analysis The source analysis methodology used in this chapter is identical to that of Chapter with the exceptions that spatial (number of electrodes) and temporal (number of time points) dimensions of data are different. The number of electrodes is 43 and 48 for T.C and H.B., respectively. The electrode positions were determined by the radial projection of the measured electrode positions on to a unit sphere (Slotnick 999). The number of time points is limited to the first samples (about 333 ms) of the second order kernel. Source analysis was performed on the potentials evoked by the 48 patches in the outer four rings of the stimulus. A summary of the source analysis methodology is as follow: Step one: Subspace estimation Step two: Dipole locations, orientations and magnitudes estimation for every pair of temporal components in the rotation space Step three: Ribbon Metric and sub-metric estimation Step four: Optimal temporal component estimation.3 Validation of dipole solutions The difficulty in validating the E/MEG source analysis results from this chapter lies in the lack of temporal maps from another imaging modality. The method of choice for validating source analysis data has historically been to compare the resulting spatial maps to the fmri loci. However the spatial maps from E/MEG and to a lesser extent the fmri maps contain systematic errors therefore making the comparison difficult. Furthermore, results in Chapter two suggested that our EEG source analysis could result 35

140 in accurate temporal maps even when the spatial maps are not very accurate therefore undermining the motivation for comparing spatial maps. Short of using invasive measurement methods, the only option left for validation is to closely support every source analysis step with simulations. 36

141 3 RESULTS AND DISSCUSION 3. The sub-metrics This section assumes the reader has read Chapter and has general familiarity with the aforesaid rotation space, Ribbon Metric (RM) and its sub-metrics. The Ribbon Metric and sub-metrics were estimated for each subject s (T.C. and H.B.) left and right hemispheres independently. Figures 3 and 4 show the metrics for T.C. and H.B., respectively. In each Figure, the left and middle panels of images consist of metrics for the left and right hemispheres, respectively and the right panel consists of combined metrics, which are estimated by averaging the corresponding metrics for each hemisphere. Discussions about the Ribbon Metrics shown in subplots A, A and A3 (Figures 3 and 4) are deferred until later. I start by pointing out a few observations and their significance about the sub-metrics. 37

142 LH RH LH+RH A. RMLH 3 A. RMRH 3 A3. Ribbon Metric (RM) B. DCM B. DCHM.. B3. DCM...8 B4. DCHM.. B5. DCM.3 B6. DCHM.5... C. OCM. C. OCVM 3 C3. OCM..8 C4. OCVM 3 C5. OCM..8 C6. OCVM 3 D. GCM D. GCM D3. GCM.9 E. MCM.5 E. MCM.55 E3. MCM Figure 3 Metrics for Subject T.C. 38

143 LH RH LH+RH A. RMLH A. RMRH A3. Ribbon Metric (RM) B. DCM B. DCHM B3. DCM B4. DCHM B5. DCM C. OCM C. OCVM C3. OCM C4. OCVM C5. OCM B6. DCHM C6. OCVM.. 3 D. GCM D. GCM D3. GCM E. MCM E. MCM E3. MCM Figure 4 Metrics for subject H.B. 39

144 First Observation: Distance continuity metric (DCM), orientation continuity metric (OCM), magnitude continuity metric (MCM), distance continuity across horizontal meridian (DCHM) and orientation continuity across the vertical meridian (OCVM) have prominent structures (subplots B, B, B3, B4, C, C, C3, C4, E and E). In fact, all general topological features including the ridges and valleys have been reproduced in Chapter simulations, which suggest that these sub-metrics are able to measure the intended continuity effects in the real data. On the other hand, Gaussian curvature metrics (GCM) do not clearly present prominent features and are very noisy (subplot D and D). One possible explanation for the noisiness of GCM is the inherent noisiness of derivative based operations. It is also possible that the sparseness of the evoked sources on the cortical surface undermines the continuity assumption and the assumption that Gaussian curvature of the estimated surface can only be high along one curvature principle-axis. This is unfortunate because, as discussed in chapter, OCM is also thought to be too rigid in its assumptions therefore making it an unreliable metric. Having a reliable metric that captures orientation continuity between neighboring dipoles is very useful since dipole orientation is more reliably estimated than dipole location. Henceforth, submetrics excludes GCM. Second observation: Comparing the sub-metrics from the left hemisphere to the respective sub-metrics from the right hemispheres reveals that they have in general similar topologies. The position of red diagonal regions in GCM, OCM, and DCHM are fairly consistent between the left and right hemispheres which indicates that the temporal components that result in good single dipole fit of the left and right hemispheres are found in the same rotation position for both hemispheres. This finding further validates 4

145 our common temporal component assumption for sources corresponding to iso-eccentric patches. Note that the red diagonal regions are generally wider than those of simulations, which can be explained by high levels of misspecification (Chapter section 4.3) or by a greater variability in the magnitude strength across patches. Furthermore it looks as though, particularly in subplot B3, each diagonal region consists of two close ridges. The double ridges are likely caused by systematic variations between the sources of different eccentricities (of the visual space). It should be noted that the double ridge effect is an indicator of possible uncertainty in the optimal RM rotation position. Third observation: The minima of OCM and DCM maps (the dark blue shades) for all four hemispheres are found in the immediate vicinity of the upper singular row, where the temporal components are nearly correlated. This effect is very similar to one of the simulation cases in section 4.4 of Chapter where there are two additional sources, one mimicking a V3 source and the other a more distance source. The simulated V and V source temporal components had a positive correlation of.55 but DCM, OCM and GCM incorrectly estimated the temporal components to be (anti-) correlated. The reason for this effect in the simulations, and likely for the real data, is that a nearby third source contaminates dipole one and two estimates however dipole one fitting a weaker V source gets more adversely effected by the crosstalk. Therefore, the rotation that results in the most spatially continuous solutions is one in which the second dipole nearly superimposes the first one and their orientation and temporal components are nearly anticorrelated. Two nearly superimposed dipoles with anti-correlated orientation and temporal components can form a quadrupole, which is a multi-polar current source that can fit a potential field resulting from several sources. A quadrupole often forms when 4

146 the dipoles used in the fitting procedure, due to misspecification or other sources of noise, cannot effectively model the potential field. The resulting superposition of first and second dipoles is referred to as folded ribbon. The folded ribbon can have better spatial continuity only when the first source is dominant in which case it maintains continuity even when there is cross talk with other sources. On the other hand, contamination of the second dipole by additional sources of noise results in solutions with poor spatial continuity unless it is temporally rotated in such a way that folds back on the first dipole thus benefiting from its spatial continuity. The way the Ribbon Metric prevents the folded ribbon scenario is via MCM. MCM exerts a heavy penalty in and around the singular regions, which result in positive and negatively correlated temporal components. Further discussion regarding the optimal weighting of MCM and its effect on the dipole estimates takes place later. 3. The settings used in calculating sub-metrics and Ribbon Metrics 3.. Assumed distance between neighboring sources Reliability of DCM depends on an accurate estimation of D lim (of equation 3 in Chapter ) which is the assumed distance separating the neighboring sources (DCM = sum{ separation of neighboring pairs - D lim }/total number of pairs). The average cortical distance between the neighboring sources based on a magnification factor with E=.75 is about.7 cm. However the linear distance between the neighboring sources can be much smaller because of cortical folding. Also the surface area of early visual areas can vary by a factor of two from subject to subject. Simulation 4

147 has shown that it is better to underestimate the value of D lim than to overestimate it (data not shown). Overestimation has a scattering effect on the topological features whereas underestimation has more of a sharpening effect on the area around the minima. Several settings between. and.7 cm were tested with the real data. A value of.3 cm was the highest level that results in a relatively smooth DCM for subject T.C. However data from H.B. did not show much difference even with as high a setting as.7 cm. To be consistent, a final D lim value of.3 was used for both subjects. 3.. Sub-metric weightings In order to calculate the Ribbon Metric (RM), first the RMLH and RMRH, which are metrics for left and right hemispheres, respectively, are calculated by taking the weighted sum of normalized sub-metrics of each hemisphere. The normalization process involves first placing a ceiling (saturation) on each map at the mean plus twice the standard deviation. The metric values at the singular regions, which are the 6 th and 6 th rows of the rotation space, are excluded from the mean and standard deviation calculations. Then each map is normalized to the mean of its nonsingular regions. The sub-metric weightings used are identical to those of Chapter (the weighting does not include the normalization factor). The weightings of Chapter were determined after examining the effectiveness of each sub-metric in determining the optimal rotation. To avoid unnecessary complications, the post hoc analysis and the subsequent determination of weightings consisted of a simple logic: If a sub-metric s minimum was determined to be an accurate marker for the correct rotation (recall that in Chapter the correct rotation is known), it was given a weighting of. If the sub-metric was determined to be nonspecific, it was given a weighting of.. If a metric was determined to be inaccurate it 43

148 was given a weighting of. Thus, the resulting weightings from Chapter were for DCM, GCM and MCM;. for DCHM and OCHM; for OCM. For the sake of objectivity, it is best not make drastic changes to weightings from Chapter. However no choice was left but to zero the weighting for GCM because of its noisiness. All other weightings are kept at the same level as in Chapter. Subplots A and A (in Figures 3 and 4) show the RMLH and RMRH, respectively, calculated using the described weightings Combining metrics from left and right hemispheres The Ribbon Metric (RM) is an average of RMLH and RMRH for each subject. Figure 5 shows the Ribbon Metric (RM), RMLH and RMRH for each subject. These maps are the same as those found in Figures 3 and 4, except that a different colormap is used in order to make it easier to see the minimum in each map. The black arrows in every subplot points to the optimal RM position, determined by the minimum of the corresponding RM (for both hemispheres combined). Note that each RMLH and RMRH maps has several dark blue pixels making the minima not easily identifiable. The closely valued pixels of each map add uncertainty to where the minimum may lie since any small fluctuation can change the location of the minima. Stochastic noise in the dipole spatial solutions and/or systematic noise due to local effects of misspecification are the likely sources of variability between RMLH and RMRH. The averaging of the maps from the left and right hemispheres should mitigate the effect of stochastic noise. The dark blue regions of the RMLH and RMRH for each subject generally coincide. However, there are some discrepancies between the maps from left and righthemispheres. ) The optimal RM position (where black arrow is pointing) does not 44

149 coincide with any of the dark blue pixels of subject T.C. s RMLH. This difference between the two maps means that optimal rotations based on each map would be a little different (temporal components based independently on the RMLH and RMRH are found in the appendix.). ) T.C. s RMRH map has a few dark blue pixels around the upper and lower right side of the image that are not seen in RMLH. The resulting temporal components depicted by each of the pixels are different from those found near the optimal RM position by a sign reversal of only one temporal component. The OCVM metric is used primarily to avoid such reversals but unfortunately in this case the OCVM s weighting of. was not enough to overcome the problem. However it should be noted that the sign reversal of one temporal component and the subsequent sign reversal of the corresponding dipole orientation does not significantly impact the spatial solutions. 3) Both subjects RMLH have dark blue pixels above and below the singular regions. The dark blue regions above and below the singular regions are generally different in that dipoles one and two are switched. The switching of dipoles only impacts the registration problem in that the mapping of the first-dipole to V and second dipole to V would be incorrect. The DCHM s purpose is to identify which solution is more anatomically correct. However the DCHM is based on locations of only 8 dipoles, which makes it susceptible to noise. Overall the optimal regions in RMLH and RMRH are remarkably similar (with both subjects). If the minimums in quadrants other than the one in which the black arrows reside are ignored, the resulting minimums based on RMLH and RMRH are only different by a single pixel in both subjects (a single pixel represents a rotation of.5 radians (π /)). As discussed earlier, the optimal regions in other quadrants are simply 45

150 due to either dipoles one and two switching places or a single dipole sign reversal, which do not bear much significance in the source analysis procedure. Although the difference of only one pixel may be fortuitous, (the resulting temporal components based on the described minimums are shown in Figures 4 and 5 of the appendix), the minimums are likely no more than two to three pixels apart. The remarkable similarities of the optimal rotations based on the individual left- and right-hemisphere maps validate the continuity assumptions of the Ribbon Metric. A. RMLH 3 A. RMRH 3.8 A3. Ribbon Metric (RM) 3.8 Subject T.C B. RMLH 3.5 B. RMRH B3. Ribbon Metric (RM) Subject H.B Figure 5 Comparing maps from the left and right hemispheres. 46

151 3.3 Are dipoles one and two modeling V and V sources? Slotnick et al. (999) performing a one-dipole fit of the same data for T.C. and H.B. concluded that the 6 dipoles corresponding to the 6 stimulus patches are modeling the V sources. Their finding was based on the waveform polarity reversals, the dipole retinotopic organization, and continuity across the horizontal meridian. The dipole locations corresponding to the 48 stimulus patches in the four outer rings were further validated by Baker (3) to be of V origin by comparing the dipole locations to fmri loci from the same subjects. The single-dipole fit of the data by Slotnick et al. accounted for about 5% of the variance in each subject indicating the signal may be dominated by one source particularly since the percent variance across the four averaged data sets for subject H.B. was about % (the percent variance is available only for subject H.B.). This chapter further validates Slotnick s finding regarding the dominance of a single dipole. The two-dipole fit accounts for 8% and 75% of the variance for subjects T.C. and H.B., respectively (percent residual maps as a function of rotation can be found in the appendix). The majority of the energy fitted by the two-dipole fit is attributed to the firstdipoles since on the average the ratio of first-dipole to second-dipole magnitudes based on optimal RM solutions for T.C. and H.B. were found to be.4 and.48, respectively. Of course one has to be careful about putting too much weight on arguments based on the percent variance of fit since it is possible that the each dipole is fitting several sources. Whether each dipole is fitting one source or several sources is further investigated later in the chapter. An important reason that so much of the variance can be accounted for is that the m-sequence stimulation elicits responses only from the early retinotopic areas with 47

152 relatively small receptive fields. Later visual areas would not respond well since the rapidly flickering checkerboards would only be able to stimulate early visual areas. Figure 4 compares the resulting first-dipole locations and orientation using optimal RM rotation for both subjects to the locations reported by Slotnick. Each cone s location (of its base) and orientation represents the location and orientation of a dipole. The dipoles are viewed from the back of the head (projected on the XZ plane). The locations of dipoles corresponding to the outer three rings are shifted in the X direction so that they do not lie on top of each other. The lines connect the neighboring dipoles belonging to the same stimulus ring. The RM based dipole solutions though a little different are generally similar to Slotnick s solutions in their relative position and orientation. Based on Slotnick and Baker s conclusions that these dipoles are of V origin, it is very likely that the origin of dipole one in this study is V as well. 48

153 Subject T.C. Dipole one of optimal RM solution Locs d thom Shift Shift Subject T.C. One-dipole fit Subject H.B. Dipole one of optimal RM solution Subject H.B. One-dipole fit Figure 6 First-dipole and single-dipole fit locations (cm) and orientations 49

154 How much are V spatial maps estimated by optimal RM rotation influenced by extrastriate sources? One possible way for dipole one to be relatively free of influence from extra-striate sources would be if there were only two sources in which case the RM solution would be able to effectively isolate them. However the fact that the DCM minima for all four hemispheres lies very close to the singular regions suggest that there are more than two sources significantly contributing to the signal (Chapter Section 4.4). Another possible way for the first-dipoles to be relatively free of influence from extrastriate sources would be if the V sources were dominant in strength. Given that there are several sources that contribute to the signal, then the only way for dipoles identified by Slonick (from a single dipole fit) and the first-dipoles estimated by the Ribbon Metric methodology to be retinotopically organized would be if the first-sources are dominating. The average dipoles one to two magnitude-ratios based on optimal RM solutions for T.C. and H.B. are.4 and.48, respectively. These ratios are likely to depict the average strength ratio of the V sources to a mixture of extra-striate sources. In Chapter Section 4.4, a scenario containing a dominant source in the presence of three weaker sources was tested using simulations and it was shown that it is possible for dipole one to primarily fit the dominant source and the second dipole fitting a mixture of weaker sources. To further validate that the first-dipole solutions based on the optimal RM rotation are not significantly contaminated by the extra-striate sources, the effect of MCM weighting on the locations of the first-dipoles were tested. By gradually increasing MCM weighting from to, first- and second-dipole temporal components break away from being anti-correlated (become more correlated) thus their locations would move away from one another. Furthermore because of the idiosyncratic layout of the visual 5

155 cortex, dipoles one and two would move in a predictable way therefore making it possible to distinguish if a dipole is fitting a V source or extra-striate sources. The visual cortex is laid out such that the upper visual field (UVF) is represented by the ventral side of occipital cortex and the lower visual field (LVF) by the dorsal side. V is continuous across the horizontal meridian; most other early visual cortex regions are split at the horizontal meridian into the dorsal and ventral regions. Because of this anatomical layout, if a dipole fitting V is contaminating with extra-striate sources its location is expected to shift away from the cortical area representing the horizontal meridian. In the same way, if a dipole fitting V becomes less influenced by V, its location would shift away from the V region. When MCM weighting is zero, both dipoles one and two converge on the same location as shown schematically on the left hand side of Figure 7. On the other hand, as MCM weighting is increased, dipoles one and two move away from each other Dipoles and fitting V and V (or extra-striate) Dipole one fitting V Dipole two fitting V Dipole two fitting extra-striate sources Increase MCM weighting 5 5 HM 3 4 HM or 3 4 HM Figure 7 Ribbon folding effect 5

156 in a predictable manner as shown by the right hand side of Figure 7. If the dipole two is fitting primarily V then one would also expect there to be strong continuity in location of neighboring dipoles, if other sources are intruding then the location continuity may be reduced. To measure the effect of MCM weighting on dipole locations, the distance between dipoles labeled as and 6, and 5, and 3 and 4 were averaged over all firstdipoles. Similar distances were calculated for all second dipoles. Figure 8 shows the average distances for every one the four hemispheres as the MCM weighting is changed from to in. increments. The asterisks and circles mark the resulting change in distance of dipoles one and two, respectively. As one would expect, based on the anatomy, the average distances between the second-dipole locations increase. This increase strongly suggests that the second dipole is increasingly fitting extra-striate sources as MCM weighting is increased. The distances between the first-dipole locations do not change as much as for the second-dipoles. One would expect that the distances between first-dipoles to decrease to some degree, as they would be less contaminated by extra-striate sources with increasing MCM weighting. However since the dominant firstdipoles even when contaminated by other sources do not show a significant dorsal/ventral gap, the change-in-distance effect for the first-dipoles is less. A significant decrease only happens in one hemisphere whereas the other three show a small increase. 5

157 LH RH.6.5 Subject T.C. Distance (cm).4 Distance (cm).5 Distance (cm)..5.5 MCM weighting MCM weighting LH RH Subject H.B. Distance (cm).8.6 Distance (cm) MCM weighting MCM weighing Figure 8 Effect of MCM weighting change 53

158 Figure 9 shows the second-dipole locations and orientations for both subjects resulting from the optimal RM rotation. Similar to Figure 6, the view is from the back of the head and the dipole locations are fanned out along the X dimension. It is very difficult to judge by visual inspection whether dipole two locations are retinotopic. There must be some spatial continuity by tautology since the Ribbon Metric chooses solutions with best continuity Subject T.C. 4.5 Dipole two of optimal RM solution Shift Shift Subject H.B. Dipole two of optimal RM solution Figure 9 Second-dipole location (cm) and orientations 54

159 3.4 Temporal components The central objective of this chapter (also thesis) is to extract temporal components that are largely free of contamination by other sources. Thus far, analysis of dipole locations for real data has suggested that the two-dipole decomposition of the VEP data for both subjects have likely resulted in first-dipole solutions representing predominantly V activations. The second-dipole solutions are likely modeling a mixture of extrastriate sources and therefore cannot be attributed to only V. It is important to note that the first-dipole spatial solutions of the two-dipole fit being similar to the singledipole fit does not mean that their temporal maps would be similar. In fact, the first basis temporal-vectors are rotated by the Ribbon Metric to a greater extent than the second basis temporal-vectors in order to reduce the V intrusion on the second-dipole spatial solutions. This cross interaction of second-dipole spatial solutions and the first-dipole temporal components is due to the nature of the equations 9a thru 9d. Figures and show temporal components for subjects T. C. and H.B. respectively. The resulting temporal components for dipole one and two are depicted by blue and green colored curves, respectively. The temporal components of the singledipole fits are depicted by the black curves. Note that there are systematic changes in shapes and latencies of the temporal components as one moves from rings to 4. The fact that the changes are systematic and not erratic suggest that the temporal yoking mechanism for synchronizing the temporal components of each ring is working. The latency and shape differences are likely due to timing differences in activation for different eccentricities. 55

160 Subject T.C. Ring one TCD TCD TC one-dipole fit Ring two Ring three Ring four Time (ms) Figure Temporal components (normalized) for Subject T.C. 56

161 Subject H.B..5 Ring one TCD TCD TC one-dipole fit Ring two Ring three Ring four Time (ms) Figure Temporal components (normalized) for subject H.B. 57

162 4 CONCLUSION The overall goal of this project has been to provide a method to improve estimation of the temporal response of multiple visual areas. The Chapter simulations demonstrated the potential of using an anatomically based constraint to minimize the difficult problem of spatially close sources having temporal functions that rotate into, or interfering with, each other. This final chapter applies the anatomical constraint and provides an initial validation of the method using human VEP data. Two lines of evidence support the validation of the source analysis methodology. First, the spatial solutions were consistent with the anatomy of the early visual cortices. Although the first-dipole locations are retinotopically organized, these local spatial continuities cannot be used as a validation since local continuity is part of the Ribbon Metric constraint. The anatomical evidence is provided by the fact that as the temporal components rotate, as a result of increasing the MCM weighting, the gap between the second-dipole locations corresponding to the dorsal and ventral regions is increased. This effect is consistent with the fact that early visual cortices with the exception of V are split into roughly dorsal and ventral regions. Therefore since the first-dipole is shown to be of V origin, the second-dipoles as a result of fitting V or a mixture of extra-striate sources would be pulled apart into dorsal and ventral regions. This effect is also dependent on the fact that the first-sources are dominant in strength, which is shown to be the case for the evoked potentials used by this study. The second line of evidence providing validation for the methodology is that independent application of the Ribbon Metric of left and right hemispheres for each subject results in very similar rotations for the two-dipole solutions. While this is 58

163 preliminary and the error estimates of the optimal rotation have yet to be determined, the optimal rotation in this particular case provides for very similar inter hemispheric temporal functions as shown in Figures 4 and 5 of the appendix (see appendix for details). The temporal functions for left and right hemispheres of an individual are expected to be the same, since the RM resulted in similar inter hemispheric temporal functions the method has achieved physiological validation, an exciting development in mapping the spatio-temporal response of the human brain. 59

164 5 APPENDIX Total residual variance (.) Residual variance patch by patch Figure Residual variance maps for Subject T.C. (Residual variance = sum of squared errors / sum of squared potentials) (The left and right 4 columns correspond to the left and right hemispheres.) 6

165 Total Residual variance (.5) Residual variance patch by patch Figure 3 Residual variance maps for Subject H.B. (Residual variance = sum of squared errors / sum of squared potentials) (The left and right 4 columns correspond to the left and right hemispheres.) 6