Brain Topogr (2012) 25:27–38 DOI 10.1007/s10548-011-0187-9
ORIGINAL PAPER
Incorporating fMRI Functional Networks in EEG Source Imaging: A Bayesian Model Comparison Approach Xu Lei • Jiehui Hu • Dezhong Yao
Received: 28 March 2011 / Accepted: 21 April 2011 / Published online: 6 May 2011 Springer Science+Business Media, LLC 2011
Abstract Brain functional networks extracted from fMRI can improve the accuracy of EEG source localization. However, the coupling between EEG and fMRI remains poorly understood, i.e., whether fMRI networks provide information about the magnitude of neural activity, and whether neural sources demonstrate temporal correlations within each network. In this paper, we present an improved version of the NEtwork-based SOurce Imaging method (iNESOI) through Bayesian model comparison. Different models correspond to various matching between EEG and fMRI, and the appropriate one is selected by data with the model evidence. Synthetic and real data tests show that iNESOI has potential to select the appropriate fMRI priors to reach a better source reconstruction than some other typical approaches. Keywords EEG source imaging Parametric Empirical Bayesian EEG fMRI
X. Lei J. Hu D. Yao (&) The Key Laboratory for NeuroInformation of Ministry of Education, School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu 610054, China e-mail:
[email protected] J. Hu School of Foreign Languages, University of Electronic Science and Technology of China, Chengdu 610054, China
Introduction Electroencephalography (EEG) and functional Magnetic Resonance Imaging (fMRI) are complementary functional neuroimaging techniques due to their respective strengths in spatial and temporal resolution. EEG directly measures neuroelectric potentials with millisecond resolution, but localizing these potentials within the brain is an ill-posed problem (Helmholtz 1853; Michel et al. 2004; Lei et al. 2009). In contrast, fMRI normally relying on the Blood Oxygenation Level-Dependent (BOLD) signal has the spatial resolution of millimeters, but takes several seconds to rise. Therefore, integrating EEG and fMRI may provide a combined imaging technique with high spatiotemporal resolution. There are currently three broad potential approaches to EEG/fMRI integration: (i) ‘‘symmetric fusion’’ constructs a common generative model to explain both modalities (Aubert and Costalat 2002; Daunizeau et al. 2007); (ii) ‘‘temporal prediction’’ models fMRI data with certain EEG features (Debener et al. 2006; Britz et al. 2010); and (iii) ‘‘spatial constraint’’ uses the spatial information from fMRI for EEG source reconstruction (Liu et al. 1998; TrujilloBarreto et al. 2001). Implementing (ii) and (iii) in parallel leads to a unified Spatial–Temporal EEG/FMRI Fusion (STEFF) (Lei et al. 2010). We adopt the ‘‘spatial constraint’’ approaches in this work. Previous studies use Statistical Parametric Map (SPM) activity to constrain the spatial locations of the likely EEG sources (Liu et al. 1998), or to initially seed dipoles for further dipole fittings (Ahlfors et al. 1999). Another way is to use the Parametric Empirical Bayesian (PEB) framework to employ SPM information as priors to relax the direct correspondence between EEG and fMRI sources (Phillips et al. 2002; Phillips et al. 2005). The PEB
123
28
framework has been proved to be a promising tool for reliable estimation of EEG sources, because various priors can be used for source reconstruction (Mattout et al. 2006). Based on the PEB framework, we proposed a source imaging method based on multiple priors derived from functional connectivity analysis of fMRI data, termed NEtwork-based SOurce Imaging (NESOI) (Lei et al. 2011). Different from its SPM competitor, the prior generated by independent component analysis (ICA) are free from all assumption about the time courses contributing to signal changes (Jacobs et al. 2008). In addition, transiently taskrelated and non-task-related components extracted by ICA (Calhoun et al. 2001b) can also facilitate the source imaging. Brain functional networks can improve the accuracy of EEG source localization (Lei et al. 2011). However, the coupling between modalities for EEG source imaging remains poorly understood. In fact, fMRI spatial information can be projected onto EEG solution space in different ways (Henson et al. 2010): (i) whether fMRI priors provide quantitative information about magnitude of neural activity, or only spatial information; (ii) whether EEG sources have similar time courses within each fMRI prior (network). These differences correspond to different Covariance Components (CC) in the PEB framework: binary versus continuous, and variance versus covariance CC. Fortunately, PEB framework provides an approximation to the ‘‘model evidence (ME)’’, which can not only determine the relative weight of the valid priors, but also eliminate invalid priors. Here, a specific question we are interested in is the influence of the matching between modalities, i.e., which structure of CC is appropriate in practice. Henson et al. (2010) have shown that ‘‘binary’’ and ‘‘variance’’ priors extracted by SPM provide the best ME for real data. We will refer this improved method based on model selection as ‘‘iSPM’’. In this work, we present an improved NESOI (iNESOI) to discuss the valuable information from fMRI functional networks. Moreover, as the fMRI reflect different functional organizations of the brain, our comparison can reveal how the hemodynamic and neuroelectric signals relate to each other during different states of brain activities. In ‘‘Materials and Methods’’ section, we outline the theoretical considerations behind our method. We first review the PEB approach briefly and then present the scheme to convert fMRI functional networks to multiple spatial priors. In ‘‘Simulations’’ and ‘‘Real Data Test’’ sections, synthetic and real data tests are conducted, respectively to show that certain associations of the matching between modalities and the structures of CC have crucial effect on source localization. We conclude in ‘‘Discussion’’ section by discussing future extensions of our method.
123
Brain Topogr (2012) 25:27–38
Materials and Methods Parametric Empirical Bayesian Parametric Empirical Bayesian model (Phillips et al. 2002; Mattout et al. 2006; Lei et al. 2011) is used for EEG source imaging: Y ¼ Lh þ e1
e1 Nð0; T; C1 Þ
h ¼ 0 þ e2
e2 Nð0; T; C2 Þ;
ð1Þ
where Y 2 Rns is the EEG recording with n sensors and s samples. L 2 Rnd is the known lead-field matrix, and h 2 Rds is the unknown source dynamics for d dipoles. N(l, T, C) denotes a multivariate Gaussian distribution on a matrix, namely e Nðl; T; CÞ , vecðeÞ Nðl; T CÞ; with mean l and covariance T C: vec denotes the column-stacking operator, and is the Kronecker tensor product. The terms e1 and e2 represent random fluctuations in sensor and source spaces, respectively. The temporal correlations are denoted by T, which is assumed to be fixed and known for simplicity. The spatial covariances of e1 and e2 are mixtures of CC at each level. In sensor space, we assume C1 = a-1In to encode the covariance of sensor noise, where In is the n-by-n identity matrix. In source space, we express this in a covariance component form, C2 ¼
k X
ci V i ;
ð2Þ
i¼1
where c = [c1, c2, …, ck]T is a vector of k non-negative hyperparameters that control the relative contribution of each covariance basis matrix, Vi. The components set, V = {V1, V2, …, Vk}, is assumed to be fixed and known. Such a formulation is extremely flexible, because a rich variety of candidate covariance bases can be easily combined in such a PEB framework using Eq. 2 (Mattout et al. 2006). The hyperparameters c are unknown, and they are akin to the standard regularization parameters in ill-posed problems. These hyperparameters can be estimated using Restricted Maximum Likelihood (ReML) algorithm (Friston et al. 2007), which generates Maximal A Posteriori (MAP) estimates of the source distribution simultaneously. In Bayesian inference, the objective function maximized in ReML can be approximated by the variational freeenergy (Friston et al. 2007). For models of the form of Eq. 1, the free-energy provides a lower bound on the model’s ‘‘log-evidence’’, ln p(y|M). Model M is defined by lead-field matrix L and components set V. The log-evidence increases with the accuracy but decreases with complexity (Friston et al. 2008), yielding a parsimonious model. We use ME below to evaluate the usefulness of fMRI priors. We compared the forward models with different components set V, which reflects different types of CC. Further
Brain Topogr (2012) 25:27–38
29
mathematical details of this PEB approach can be found in (Friston et al. 2007). The ReML algorithm is performed in the academic software package SPM8, a free MATLAB toolbox (http://www.fil.ion.ucl.ac.uk/spm/). Deriving CC from Anatomic Information In the classical Minimum Norm Estimation (MNE), CC is simply an identity matrix, that is, V1 = Id (Phillips et al. 2005). Considering local anatomic coherence, the Green function (Harrison et al. 2007) is useful, G ¼ expð0:6AÞ ¼ ½q1 ; q2 ; . . .qd ;
ð3Þ
where A is the adjacency matrix to encode the neighboring relationships among vertexes of the cortical mesh in the solution space. If j is the adjacent vertex connected to i, then Aij = 1; otherwise, Aij = 0. The value 0.6 is a smoothness parameter, which propagates spatial dependences over neighboring vertexes about 6 mm apart (Friston et al. 2008; Lei et al. 2011). Here the d vertexes are approximately uniformly distributed over the cortex surface. The ith column of G, qi, defines a subset of neighboring vertexes, weighted by their distances to the centre, the ith vertex. In a recent approach (Friston et al. 2008), multiple sparse priors (MSP) from the columns of G are uniformly sampled to produce several hundred CC at the source level, V ¼ fq1 qT1 ; q2 qT2 ; . . .; qk qTk g. The iNESOI An improved version of fMRI iNESOI is illustrated in Fig. 1. This process is extended from PEB framework (Henson et al. 2010) to fMRI functional networks, thus providing an optimal priors search strategy to find the best covariance structure. Functional networks extracted by ICA (Fig. 1a) are used to form connected subsets or clusters (Fig. 1b). Each cluster is then independently projected on the surface mesh using nearest-neighbor interpolation (Fig. 1c), leading to the definition of the same number of CC (Fig. 1d). Different choices of CC are compared using Bayesian model selection, and then EEG sources are reconstructed using the fMRI prior with the appropriate structure of CC. These steps (a–d) are detailed in the sections below. Deriving Priors from Functional Networks Independent component analysis is used to group brain areas that share similar response patterns (McKeown et al. 1998; Chen et al. 2003) to extract the fMRI functional networks. After data preprocessing, fMRI data is decomposed into n independent components (ICs) and time courses by spatial ICA. The number of components n is
usually determined by minimum description length criteria (Li et al. 2007). Each spatial IC expresses the intensity distribution over all voxels and corresponds to a temporally coherent network. To measure the relative contribution of a voxel in a particular IC, the intensity values in each map (spatial component) are scaled to z scores (D’Argembeau et al. 2005). The group-wise decomposition can be considered in this model for group studies (Calhoun et al. 2001b). Note that various other schemes can be used to extract temporal correlate network. For example, the cross-correlation approaches base on region-of-interest (ROI) (Hampson et al. 2004). However, as the result is ROI dependent, we adopted the much simpler and more convenient ICA approach to extract these priors in this work. Defining the Clusters For a functional network (IC), voxels with absolute z scores larger than a certain threshold are preserved for further analysis. Negative z scores indicate that the BOLD signals are modulated oppositely to the network’s waveform (McKeown et al. 1998). The clusters are grouped as spatially contiguous voxels (Haralick and Shapiro 1992). Previous work (Henson et al. 2010) has proved that defining each spatially connected cluster as a single prior can benefit EEG source imaging, because neural activity in each fMRI cluster may have its specific dynamic time course. Small clusters less than certain number of voxels (eight in this work) are removed from further processing, as the typical area of neural activity associated with detectable changes in scalp EEG is estimated as 6 cm2 (Nunez and Silberstein 2000). Indeed, the threshold value and the size of clusters could be optimized by ME. However, empirical values are employed here for simplicity. The previous thresholding and labeling procedures produce fMRI cluster matrix U 2 Rvk corresponding to the k clusters that have been detected automatically, where v refers to the number of voxels. Note that the number of clusters (NoC) k is larger than the number of components n, because most ICs contain more than one cluster (see Figs. 1 and 5 for an instance). Projecting Cluster onto Cortical Surface The clusters in the 3D fMRI space are projected onto the 2D cortical surface with nearest-neighbor interpolation (Henson et al. 2010). Each dipole in EEG solution (or source) space is assigned the z scores of its nearestneighbor voxels, and the mean value is employed if a dipole has several nearest-neighbor voxels. This process translates the fMRI cluster matrix U into W 2 Rdk in solution space, where d is the number of dipoles and k is
123
30
Brain Topogr (2012) 25:27–38
Fig. 1 Schematic representation of the improved network-based source imaging (iNESOI). a Brain functional networks are extracted from fMRI by spatial independent component decomposition; b each spatial pattern (or IC) is thresholded to form spatial connected clusters; c Each cluster is then independently projected onto the surface mesh, leading to the definition of the covariance components; d Covariance components have four different structures from two
interacting factors. Up the covariance matrices created with or without off-diagonal terms (using a strong assumption that EEG sources in a fMRI cluster have coherent time course, Eqs. 4a and 4c, or not, Eqs. 4b and 4d); Left continuous or binary structures (using fMRI statistical quantities as the magnitude of neuroelectric activity, Eqs. 4a and 4b, or the information about location, Eqs. 4c and 4d)
the NoC. A detail description can be found in the literature (Henson et al. 2010), and several other methods for projecting a 3D volume onto a cortical surface (Grova et al. 2006; Operto et al. 2008).
ViCovBin ¼ binaðViCovCon Þ
Deriving CC from Clusters A simple way to construct a covariance component from the ith cluster is to assign the diagonal terms with values from the ith column of W 2 Rdk , and assign the other terms with zero (Liu et al. 1998). If we consider the local coherence in the solution space and provide greater robustness to mis-registration errors between EEG and fMRI, the covariance component Vi can be formed as (Lei et al. 2011): ViCovCon ¼
1 GWi WiT G; ni
ð4aÞ
where ni ¼ WiT Wi is the sum of the square of amplitudes in ith cluster, and Wi is the ith column of W. G is the Green function that encode anatomic coherence and it is defined in ‘‘Deriving CC from Anatomic Information’’ section. If one is unwilling to assume that the time courses of EEG sources within an fMRI cluster are correlated, one might prefer to set the off-diagonal term as zero, yielding a ‘‘variance’’ component (Henson et al. 2010): ViVarCon ¼ diagðViCovCon Þ
ð4bÞ
where the diag() operator retains only the leading diagonal of a matrix. If one does not know the precise mapping between BOLD and the generators of EEG, one might prefer to binarize Eqs. 4a and 4b to:
123
ð4cÞ
and ViVarBin ¼ binaðViVarCon Þ
ð4dÞ
where the bina() operator give an element a value of ‘‘1’’ if its absolute value is larger than zero, otherwise, give a value of ‘‘0’’. This translation considers fMRI patterns as information about locations, rather than quantitative information about the magnitude of neural activities. A caricature of these four structures is given in Fig. 1d.
Simulations In this section, simulations are used to prove that the coupling relation between EEG and fMRI has critical effect on EEG source imaging, and that the iNESOI can select the appropriate covariance components structure. The influence of complete uncoupling between EEG and fMRI is not discussed here, because this influence is similar to the effect of invalid priors, which has been investigated in our previous study (Lei et al. 2011). Moreover, discussing the appropriate structure of covariance components in this condition is meaningless. The results of our simulation are compared in terms of four evaluation metrics (Friston et al. 2008; Lei et al. 2011): localization error (LE), temporal accuracy (TA), explained variance (EV) and ME. LE is defined as the mean geodesic distance between the assumed most active dipole and the estimated dipole with the largest conditional expectation within the same time bins; TA is defined as the squared correlation between the true time course of the assumed most active dipole and the conditional estimate
Brain Topogr (2012) 25:27–38
used in assessing LE; EV is the fitted part of the data over all sensors and time bins; ME is defined as the log of the marginal likelihood, and is approximated by free-energy (Friston et al. 2007). Synthetic Data Forward model is derived from a high-density canonical cortical mesh. These meshes were obtained by warping a template mesh to the T1-weighted structural anatomy of an individual subject, as described in (Mattout et al. 2007). This warping is the inverse of the transformation derived for the spatial normalization of the subject’s structural MRI image. The template mesh was generated by Fieldtrip (http://fieldtrip.fcdonders.nl/download.php), and was extracted from a structural MRI of a neurotypical male. The wrapping procedure provided a high-density mesh with 33,001 vertexes, which was uniformly distributed on the gray-white matter interface. The mesh was further down-sampled to 6,144 vertexes to reduce the computational load. Each vertex was assumed to have one dipole, oriented perpendicular to the surface. The sensors from the ‘‘multi-modal face study’’ (see ‘‘Real Data Test’’ section) were registered to the scalp surface, and the lead-field matrix was calculated within SPM8. Using the real EEG data from the ‘‘multi-modal face study’’ (Henson et al. 2003), we performed singular value decomposition in sensor space to identify its principal time courses over the time window of 800 ms, starting 200 ms before stimulus onset. We kept the first five singular vectors, Th 2 R5821 and deployed them over five distributed sources
31
(clusters). Based on the fMRI spatial ICA results of the ‘‘multi-modal face study’’ (see Fig. 5a), twenty-five clusters were extracted from fMRI data, yielding a ‘‘6,144 9 25’’ matrix W. For each simulation, the five clusters in source space, Sh 2 R61445 ; were randomly sampled from columns of W. The assumed sources were projected through the leadfields to the sensor space as the simulated EEG signals by the transformation LShTh. Temporal smoothed Gaussian noise was scaled to one tenth of the L1-norm of the simulated signal, providing a signal-to-noise ratio about 10. The signal and noise were then mixed to generate simulated data. Schematic illustration of the construction of synthetic data can also be found in (Lei et al. 2011). The procedure was repeated 256 times to produce the average and standard deviation of the estimation metrics described in ‘‘Complete Coupling Between EEG and fMRI’’ and Partial Coupling Between EEG and fMRI’’ sections. Complete Coupling Between EEG and fMRI In order to evaluate the methods in a condition that only part of the spatial priors derived from fMRI are completely matched (or valid) with the EEG sources (Sh), we assumed that among the five spatial priors (i.e., W has five columns), three were consistent with those in Sh. In other words, 60% of the fMRI priors were valid and they were completely coupling with the EEG sources. Four models from two interacting factors (Variance vs. Covariance; Binary vs. Continuous) were implemented over the 256 realizations of simulation data. The results of statistical analyses are shown in Fig. 2.
Fig. 2 Mean value and standard deviation of the four evaluation metrics for the four models in complete coupling condition. Var, Variance; Cov, Covariance; Bin, Binary; Con, Continuous. Upper panels localization error (left) and TA (right); lower panels explained variance (left) and ME (right)
123
32
Figure 2 illustrates that the estimations of the four models are stable with LE less than 20 mm and TA larger than 80%. The first interesting comparison is the effect of using continuous priors. Continuous components (both for variance, VarCon, and covariance, CovCon) locate the sources with mean spatial error of 7.68 mm, much smaller than that of binary components (14.28 mm) (VarBin, CovBin). Continuous components produce TA with 86.10%, EV with 86.87% and ME with 1,398. In contrast, the corresponding binary components are 83.85% (TA), 78.46% (EV), and 1395 (ME). Regardless of invalid priors (two of the five), fMRI sources (valid priors, three of the five) and EEG sources were completely matched in our simulation, i.e., the statistical values of fMRI were directly interpreted as EEG source intensity in our simulation. In this case, detailed hypotheses (priors) from fMRI could be explained by continuous covariance components that covariance components assume the brain regions have similar time courses meanwhile continuous components assume the precise mapping between hemodynamic and neuroelectric activities. As a result, the corresponding model (i.e., ‘‘CovCon’’ in Fig. 2) performed best in all four metrics, with LE: 6.26 mm, TA: 87.57%, EV: 86.84%, and ME: 1400. Specifically, it had a significantly lower LE and significantly higher TA than all the other models (paired t-test, q \ 0.01).
Brain Topogr (2012) 25:27–38
surface. Due to this operation, valid fMRI priors only partially overlapped with the EEG sources. Other settings were similar to the above section, and the statistical results are shown in Fig. 3. Figure 3 illustrates the effect of using different structures of CC in partial coupling condition. Different from the simulation in ‘‘Complete Coupling Between EEG and fMRI’’ section, continuous components produced larger LE (20.82 mm) than binary components (14.12 mm), despite the distinct differences between covariance and variance. Variance components had better performance than covariance components in all evaluation metrics. Variance components produced LE with 14.19 mm, TA with 82.21%, EV with 82.02%, and ME with 1383, and the corresponding covariance components were, LE: 20.76 mm, TA: 80.65%, EV: 60.52%, and ME: 1372. For variance component, binary version performed better than continuous one. Combination of variance and binary performed the best with mean LE 11.23 mm (significantly lower than any of the other models, paired t-test, q \ 0.01), TA 82.26%, EV 84.35% (significantly higher with q \ 0.01) and ME 1386. Considering that the quantitative information about the magnitude of neural activities was shuffled in this simulation by the spatial shift operation, ‘‘binary’’ priors are acceptable models because they treat fMRI data as information about location, rather than precise mapping between BOLD and EEG sources.
Partial Coupling Between EEG and fMRI Different from the simulation in the above section, the three valid priors were moved along the mesh surface with the geodesic distance about 10 mm. By moving each fMRI cluster in random direction, and then projecting onto cortical Fig. 3 Mean value and standard deviation of the four evaluation metrics for the four models in partial coupling condition. Var, Variance; Cov, Covariance; Bin, Binary; Con, Continuous. Upper panels localization error (left) and TA (right); lower panels explained variance (left) and ME (right)
123
Real Data Test We used real data to provide some provisional validation of the iNESOI through model comparison. This dataset
Brain Topogr (2012) 25:27–38
33
Fig. 4 The differential ERPs between faces and scrambled faces stimulations. Spatial distributions are shown at 100, 175 and 250 ms after stimulus onset. Waveform of the ERPs masked by the dashed rectangle depicts the baseline. The scalp measurements exhibited a peak at 175 ms after stimulus onset
contains EEG and fMRI data from the same subject within the same paradigm, allowing a comparison between normal face images and scrambled face stimuli (for a detailed description of the paradigm, see Henson et al. (2003)).
then baseline-corrected from -200 to 0 ms. Resulting ERP and its spatial distribution are shown in Fig. 4. The ERP exhibited an activity peak at 175 ms following stimulus onset, in accord with a face specific ‘‘N170’’ (Henson et al. 2003).
Data Acquisition and Forward Head Model Extracting fMRI Functional Networks Electroencephalography and fMRI were acquired separately in this study. A T1-weighted structural MRI was acquired on a 1.5 T Siemens Sonata via an MDEFT sequence with resolution of 1 mm3. An EEG forward head model was established with the approach described in ‘‘Synthetic Data’’ section (by matching the T1-weighted structural anatomy of the subject to the template). The T2-weighted fMRI data was acquired using a TrajectoryBased Reconstruction (TBR) gradient-echo EPI sequence. There were 32 slices of 3 9 3 9 3 mm3 pixels, which were acquired in a sequential descending order with a TR of 2.88 s. EEG data were acquired with a 128-sensor Active Two System with a sampling rate of 1024 Hz. Two sensors on the left and right earlobe, as well as two other sensors were used to measure HEOG and VEOG. Preprocessing of EEG The preprocessing of the EEG data included a re-referencing to the average (Yao 2001), and artifact rejection. Artifacts were defined as time-points that exceeded an absolute threshold of 120 lV (these were found primarily in the VEOG). Eighty of the 344 trials in total were rejected due to artifacts. The differential event-related potentials (ERP) between faces and scrambled faces were
SPM8 was used for pre-processing fMRI data. Functional image time series data were first corrected for differences in slice acquisition times, then realigned with T1 volumes, warped into standard Talairach anatomical space, and smoothed with an isotropic 8-mm full-width-at-half-maximum Gaussian kernel. After dimension reduction using principal component analysis, spatial independent ICs and their waveforms were estimated for the fMRI dataset using the deflation approach of the FastICA algorithm (http:// www.cis.hut.fi/projects/ica/fastica/). The intensities of each spatial IC map were transformed to z scores. Those voxels with absolute z scores higher than three were considered to be IC activated voxels. Defining the cluster usually generates more than one cluster for an IC. We sorted the ICs with the NoC in ascending order, and applied additional thresholds on the NoC to remove large NoC components related to artifacts or noises. In this work, the components with NoC larger than ten were canceled in the following discussion, and all the reserved nine ICs in Fig. 5a were used in EEG source imaging. Both task- and nontask-related components are presented in Fig. 5. IC16 exhibited a cluster around the orbitofrontal and frontopolar cortex. IC3 showed two clusters around the right lateral fusiform and visual
123
34
Brain Topogr (2012) 25:27–38
Fig. 5 Spatial priors extracted by ICA and the fMRI SPM result. Sagittal, coronal and axial views of different clusters are shown in a maximum intensity projection format. a IC sorted by NoC. Nine components are presented including task-related ICs and
nontask-related ICs; b Top SPM returns three NoC: orbitofrontal, right fusiform and right mid-temporal gyrus; bottom SPM design matrix. ICA: independent component analysis; SPM: Statistical Parametric Map; IC: Independent component; NoC: number of clusters
association cortex. IC16 and IC3 were very similar to the positive activation areas in SPM (see Fig. 5b). IC5 involved a bilateral pair of the medial fusiform gyrus, which is heavily involved in face perception. There were some components that resemble several resting state networks. IC15 involved two clusters around the bilateral subcentral and superior temporal gyrus. These areas are widely considered to represent the auditory cortex. IC11 showed activities in the ventral anterior and dorsal posterior cingulate cortex. These areas appeared to be part of the default-mode network shown in IC9. IC9 showed six clusters consisting of the prefrontal, anterior cingulate, inferior temporal gyrus, and superior parietal regions. This pattern of brain regions is known as the default-mode network, as described by (Raichle et al. 2001). In total, 25 clusters were extracted as fMRI priors for EEG source imaging. Because ReML can automatically select meaningful components and discard unrelated ones (Phillips et al. 2005; Mattout et al. 2006), we skipped manual selection by inspecting for spatial structure and
time course (Calhoun et al. 2001a). The following results also confirmed this benefit of ReML. As a contrast, priors for iSPM (Henson et al. 2010) are also extracted by general linear model. Stimulus onsets were convolved with the standard canonical hemodynamic response function. Six parameters for spatial realignment were included to model the effects of head motion. These procedures generated a SPM design matrices as illustrated in Fig. 5b. Data were high-pass (drift removal) filtered by entering sinusoidal functions (up to a frequency of 1/128 s as covariates) into the model. The activated areas that responded differentially between faces and scrambled faces were calculated using statistical t-tests above an absolute t value of 2.62 (P \ 0.05, uncorrected). This produced three clusters shown in the top panel in Fig. 5b.
123
Model Comparison After extracting fMRI priors, the four different styles of CC were compared using the differential ERP between faces
Brain Topogr (2012) 25:27–38
35
Fig. 6 Model comparison among MNE, MSP, iSPM and iNESOI. a Model evidence (arbitrary units); b Explained variance (%). c–f The inflated cortex projections of the spatial activities obtained by MNE, MSP, and iSPM (with VarBin) and iNESOI (with CovCon). Var, Variance; Cov, Covariance; Bin, Binary; Con, Continuous
and scrambled faces in iNESOI. Meanwhile, we employed two single modality approaches (MNE (Phillips et al. 2005) and MSP (Friston et al. 2008)) and a multimodalities method iSPM (Henson et al. 2010) for model comparison. The results are shown in Fig. 6. Obviously, iNESOI and iSPM prefer different structures of CC (Fig. 6a, b). iSPM prefer variance components, in which continuous priors produced the highest ME (-391.5) and EV (73.29%) and binary priors produced a little lower ME (-391.6) and EV (73.77%). iNESOI, in contrast, prefer
covariance components. For covariance priors, continuous priors produced better results than binary priors, with ME (-386.4) and EV (88.85%). Meanwhile, the results confirmed that MSP outperformed MNE, and this performance is in accordance with the previous study (Friston et al. 2008). For MSP, there was a marked change in ME as the number of components increased from 32 to 256. MSP reached the maximum ME (-403.0) with 128 components per hemisphere (the white bar in Fig. 6a, b). Compared with other inversion approaches, iNESOI with continuous CC
123
36
produced the best ME (as EV may prefer complex model), although the performance difference was relatively small. Figure 6c–f showed 512 dipoles with the greatest activities occurring at 175 ms. Only sources of the best model selected by Bayesian ME are illustrated for each inversion approach. The reconstructed profile of MNE was substantially more superficial and dispersed. The activations revealed by MNE were mainly located in the bilateral striate and extrastriate visual cortex of Brodmann’s area (BA) 17 and 18. MSP reconstructed activities in bilateral middle and superior temporal gyrus (BA 21/38) with 128 sparse priors. iSPM produced three activated areas in the right middle temporal gyrus with Talairach coordinates (58.3, -54.2, -1.7 mm), the right fusiform (41.0, -56.5, -19.0 mm) and the left medial frontal gyrus (-9.0, 38.7, -14.8 mm). This result is consistent with the fMRI SPM result shown in Fig. 5b. However, it is difficult to relate the right fusiform source to the negative area in the left occipital region (see topography at 175 ms after stimulus onset in Fig. 4). The highest ME and five clusters were produced by iNESOI, which employed CC in continuous version. The most active three sources are listed with Talairach coordinates in Fig. 6f. Both lateral (52.4, -58.1, -20.3 mm) and medial (32.1, -39.8, -18.0 mm) mid-fusiform were reconstructed in the right temporal lobe. A weak source was also found in the left fusiform gyrus (-34.1, -40.9, -17.4 mm). This area might be the generator of the negative left occipital potentials in the EEG topography (Fig. 4). iSPM did not reveal activation in this area. In summary, compared with the other methods, iNESOI indicated activations in the bilateral fusiform gyri. These results are consistent with both the scalp potential map (Fig. 4) and previous studies of brain activations measured in a group of eighteen subjects using fMRI SPM (Henson et al. 2003).
Discussion In our previous work, we have introduced fMRI functional connectivity analyses into EEG source imaging for the first time (Lei et al. 2011). In this work, we proposed an improved version for NESOI, not only separating component into small clusters, but also including various CC to consider the coupling level between modalities. The influence of different structures of CC is systematically investigated with synthetic data to reveal the effect of structures of CC in real condition. And then the performance of iNESOI is further discussed with data from the face-processing experiment (Henson et al. 2003). Our results suggest that iNESOI may be more informative for source imaging.
123
Brain Topogr (2012) 25:27–38
Based on the above results, the main conclusions include: First, the difference between covariance and variance priors relates to the temporal correlation of different sources within an fMRI cluster. Activities of the dipoles covary over time within each covariance prior, and their similarities depend on the value of off-diagonal term. Constraining sources in this way may not improve the ME if the sources within the same cluster have different time courses. For example, a prior actually encompasses multiple functionally distinct regions. Though specific spatial architectures are utilized in our simulation, the temporal correlation is implicitly considered. In complete coupling condition, EEG time-series demonstrate temporal correlations within a cluster. In the case of partial coupling, as the fMRI priors only partially overlapped with the EEG sources, we simulated the condition that EEG signals are uncorrelated while fMRI is correlated. Variance priors may help if there are biases in the fMRI priors, because covariance may lead to inaccuracies by the additional constraint. Then one may ask whether the coherent fMRI sources covary in EEG domain. An interesting result of our real data shows that the ‘‘covariance’’ has higher ME for priors extracted by ICA than that extracted by SPM. This may be related to the different temporal assumptions in SPM and ICA (Hu et al. 2005). A straight answer to this question is related to neurovascular coupling (Logothetis 2008), which is beyond the topic of current investigation. Second, the influence of continuous and binary priors may vary in different conditions. Continuous priors assume more detailed hypotheses about the coupling between hemodynamic and neuroelectric activities. In contrast, binary priors treat fMRI data as information about location, rather than quantitative information about the magnitude of neural activities. As the precise mapping between neuron source and BOLD is unclear, treating fMRI as only location information is a reliable scheme (Henson et al. 2010). Here, we recommend using Bayesian model selection to find the right structure in practice. In our simulation, continuous priors were selected in the ‘‘complete coupling’’ condition, and binary priors in ‘‘partial coupling’’. In general, previous works prove that fMRI priors extracted by both ICA and SPM can produce physiologically reasonable results (Henson et al. 2010; Lei et al. 2011). The main benefits of iNESOI are related to the datadriven ICA. ICA does not require a priori specification of activation waveforms, and can identify similar loci of taskrelated activation (Calhoun et al. 2009), and can include more flexible spatial priors. These advantages are illustrated by our real data test, in which bilateral fusiform gyri in IC5 carries additional information about face processing (Fig. 5). As mentioned previously, priors extracted by ICA prefer ‘‘covariance’’ components, and their ME and EV are
Brain Topogr (2012) 25:27–38
higher than their activity counterparts. SPM requires a priori knowledge about the signal changes. Sources are extracted because their increased BOLD signals are associated with one experimental condition relative to another. In contrast, ICA is a data-driven method based on the assumption that the causes of responses are statistically independent. Brain regions are extracted because they exhibit temporally coherent fluctuations. Notice that we do not want to refer the fMRI priors extracted by ICA as ‘‘connectivity’’ priors, because both the intrinsic coupling and the extrinsic inputs may cause temporal coherence. The priors caused by extrinsic inputs may also be the ‘‘activity’’ priors. Our results reveal that priors extracted by ICA may carry more information about hemodynamic and neuroelectric dynamics. In simultaneous EEG-fMRI acquisition, task-related fMRI priors can be identified using temporal correlation. And this information would be useful when it’s hard to determine which priors were likely to be of neurophysiological origin. Any attempt at combining information of EEG and fMRI inevitably encounters the problem that EEG and fMRI measure different physiological processes. However, several previous studies have shown that the BOLD signals and neural activities are correlated. The BOLD signals have been shown to be proportional to local field potentials (Shmuel et al. 2006). Moreover, it has been shown that the BOLD signals are roughly linearly proportional to the neural activities with fixed frequency (Heeger et al. 2000; Shmuel et al. 2006). Since the current sources (and local field potentials) are shown to be proportional to the neural activities, it is reasonable to expect the sources of the EEG signals to be proportional to the BOLD signals. This provides the logic basis for our inverse scheme, in which the priors obtained from fMRI recordings are added as covariance of current densities to ensure that they are proportional to the BOLD signals. It should be noted, however, that covariance only define the dipole’s probability to be an active source. Dipoles with the same covariance can have different mean values, and their direction can be very different from each other. Because the coupling-uncoupling between EEG and fMRI is unknown and experiment-dependent, model selection based on evidence maximization is recommended for CC definition. An important issue here is that the model-evidence used in current study is approximated by free-energy, which may mislead the result because it does not constitute a rigorous bound on ME (Wipf and Nagarajan 2009). Though previous empirical reports (Friston et al. 2008; Lei et al. 2011) indicate that the Laplace approximation can be very informative in practice, it should be noted that the current approximation could potentially distort the results of the model selection.
37 Acknowledgments This project was funded by grants from the National Nature Science Foundation of China #60736029, the 863 Project 2009AA02Z301 and the 973 project 2011CB707803. The authors are grateful to the FIL methods group (http://www.fil. ion.ucl.ac.uk) for providing the data.
References Ahlfors SP, Simpson GV, Dale AM et al (1999) Spatiotemporal activity of a cortical network for processing visual motion revealed by MEG and fMRI. J Neurophysiol 82(5):2545–2555 Aubert A, Costalat R (2002) A model of the coupling between brain electrical activity, metabolism, and hemodynamics: application to the interpretation of functional neuroimaging. Neuroimage 17(3):1162–1181 Britz J, Van De Ville D, Michel CM (2010) BOLD correlates of EEG topography reveal rapid resting-state network dynamics. Neuroimage 52(4):1162–1170 Calhoun VD, Adali T, McGinty VB, Pekar JJ, Watson TD, Pearlson GD (2001a) fMRI activation in a visual-perception task: network of areas detected using the general linear model and independent components analysis. Neuroimage 14(5):1080–1088 Calhoun VD, Adali T, Pearlson GD, Pekar JJ (2001b) A method for making group inferences from functional MRI data using independent component analysis. Hum Brain Mapp 14(3): 140–151 Calhoun VD, Liu J, Adali T (2009) A review of group ICA for fMRI data and ICA for joint inference of imaging, genetic, and ERP data. Neuroimage 45(1 Suppl):S163–S172 Chen H, Yao D, Zhuo Y, Chen L (2003) Analysis of fMRI data by blind separation of data in a tiny spatial domain into independent temporal component. Brain Topogr 15(4):223–232 D’Argembeau A, Collette F, Van der Linden M, Laureys S, Del Fiore G, Degueldre C, Luxen A, Salmon E (2005) Self-referential reflective activity and its relationship with rest: a PET study. Neuroimage 25(2):616–624 Daunizeau J, Grova C, Marrelec G, Mattout J, Jbabdi S, PelegriniIssac M, Lina JM, Benali H (2007) Symmetrical event-related EEG/fMRI information fusion in a variational Bayesian framework. Neuroimage 36(1):69–87 Debener S, Ullsperger M, Siegel M, Engel AK (2006) Single-trial EEG-fMRI reveals the dynamics of cognitive function. Trends Cogn Sci 10(12):558–563 Friston K, Mattout J, Trujillo-Barreto N, Ashburner J, Penny W (2007) Variational free energy and the Laplace approximation. Neuroimage 34(1):220–234 Friston K, Harrison L, Daunizeau J, Kiebel S, Phillips C, TrujilloBarreto N, Henson R, Flandin G, Mattout J (2008) Multiple sparse priors for the M/EEG inverse problem. Neuroimage 39(3):1104–1120 Grova C, Makni S, Flandin G, Ciuciu P, Gotman J, Poline JB (2006) Anatomically informed interpolation of fMRI data on the cortical surface. Neuroimage 31(4):1475–1486 Hampson M, Olson IR, Leung HC, Skudlarski P, Gore JC (2004) Changes in functional connectivity of human MT/V5 with visual motion input. Neuroreport 15(8):1315–1319 Haralick R, Shapiro L (1992) Computer and robot vision, vol I. Addison-Wesley Longman Publishing Co., Boston, pp 28–48 Harrison LM, Penny W, Ashburner J, Trujillo-Barreto N, Friston KJ (2007) Diffusion-based spatial priors for imaging. Neuroimage 38(4):677–695 Heeger DJ, Huk AC, Geisler WS, Albrecht DG (2000) Spikes versus BOLD: what does neuroimaging tell us about neuronal activity? Nat Neurosci 3(7):631–633
123
38 Helmholtz H (1853) Ueber einige Gesetze der Vertheilung elektrischer Stro¨me in ko¨rperlichen Leitern mit Anwendung auf die thierischelektrischen Versuche. Ann Phys Chem 165(6):211–233 Henson RN, Goshen-Gottstein Y, Ganel T, Otten LJ, Quayle A, Rugg MD (2003) Electrophysiological and haemodynamic correlates of face perception, recognition and priming. Cereb Cortex 13(7):793–805 Henson RN, Flandin G, Friston KJ, Mattout J (2010) A parametric empirical bayesian framework for fMRI-constrained MEG/EEG source reconstruction. Hum Brain Mapp 31(10):1512–1531 Hu D, Yan L, Liu Y, Zhou Z, Friston KJ, Tan C, Wu D (2005) Unified SPM-ICA for fMRI analysis. Neuroimage 25(3):746–755 Jacobs J, Hawco C, Kobayashi E, Boor R, LeVan P, Stephani U, Siniatchkin M, Gotman J (2008) Variability of the hemodynamic response as a function of age and frequency of epileptic discharge in children with epilepsy. Neuroimage 40(2):601–614 Lei X, Xu P, Chen A, Yao D (2009) Gaussian source model based iterative algorithm for EEG source imaging. Comput Biol Med 39(11):978–988 Lei X, Qiu C, Xu P, Yao D (2010) A parallel framework for simultaneous EEG/fMRI analysis: Methodology and simulation. Neuroimage 52(3):1123–1134 Lei X, Xu P, Luo C, Zhao J, Zhou D, Yao D (2011) fMRI functional networks for EEG source imaging. Hum Brain Mapp. doi: 10.1002/hbm.21098 Li YO, Adali T, Calhoun VD (2007) Estimating the number of independent components for functional magnetic resonance imaging data. Hum Brain Mapp 28(11):1251–1266 Liu AK, Belliveau JW, Dale AM (1998) Spatiotemporal imaging of human brain activity using functional MRI constrained magnetoencephalography data: Monte Carlo simulations. Proc Natl Acad Sci USA 95(15):8945–8950 Logothetis NK (2008) What we can do and what we cannot do with fMRI. Nature 453(7197):869–878 Mattout J, Phillips C, Penny WD, Rugg MD, Friston KJ (2006) MEG source localization under multiple constraints: an extended Bayesian framework. Neuroimage 30(3):753–767
123
Brain Topogr (2012) 25:27–38 Mattout J, Henson RN, Friston KJ (2007) Canonical source reconstruction for MEG. Comput Intell Neurosci:Article 67613 McKeown MJ, Makeig S, Brown GG, Jung TP, Kindermann SS, Bell AJ, Sejnowski TJ (1998) Analysis of fMRI data by blind separation into independent spatial components. Hum Brain Mapp 6(3):160–188 Michel CM, Murray MM, Lantz G, Gonzalez S, Spinelli L, Grave de Peralta R (2004) EEG source imaging. Clin Neurophysiol 115(10):2195–2222 Nunez PL, Silberstein RB (2000) On the relationship of synaptic activity to macroscopic measurements: does co-registration of EEG with fMRI make sense? Brain Topogr 13(2):79–96 Operto G, Bulot R, Anton JL, Coulon O (2008) Projection of fMRI data onto the cortical surface using anatomically-informed convolution kernels. Neuroimage 39(1):127–135 Phillips C, Rugg MD, Fristont KJ (2002) Systematic regularization of linear inverse solutions of the EEG source localization problem. Neuroimage 17(1):287–301 Phillips C, Mattout J, Rugg MD, Maquet P, Friston KJ (2005) An empirical Bayesian solution to the source reconstruction problem in EEG. Neuroimage 24(4):997–1011 Raichle ME, MacLeod AM, Snyder AZ, Powers WJ, Gusnard DA, Shulman GL (2001) A default mode of brain function. Proc Natl Acad Sci USA 98(2):676–682 Shmuel A, Augath M, Oeltermann A, Logothetis NK (2006) Negative functional MRI response correlates with decreases in neuronal activity in monkey visual area V1. Nat Neurosci 9(4):569–577 Trujillo-Barreto N, Martinez-Montes E, Melie-Garcia L, Valdes-Sosa P (2001) A symmetrical Bayesian model for fMRI and EEG/ MEG neuroimage fusion. Int J Bioelectromagn 3(1):1 Wipf D, Nagarajan S (2009) A unified Bayesian framework for MEG/ EEG source imaging. Neuroimage 44(3):947–966 Yao D (2001) A method to standardize a reference of scalp EEG recordings to a point at infinity. Physiol Meas 22(4):693–711
