Mapping and Tracking the Flow of Brain Activations

Mapping and Tracking the Flow of Brain Activations using MEG/EEG: Hypothesis and Methods Julien Lefèvre and Sylvain Baillet Cognitive Neuroscience & Brain Imaging Laboratory CNRS UPR640–LENA, Université Pierre & Marie Curie, Paris6; Hôpital de la Salpêtrière, Paris, F-75013, France [email protected], [email protected] Abstract— We introduce a mathematical tool for the exploration of spatiotemporal dynamics of brain activations as revealed by time-resolved brain mapping techniques. In that respect, Magneto (MEG) and Electroencephalography (EEG) source imaging has been considerably maturing in terms of leading access to the dynamics of brain activity. Here we suggest that the local analysis in space and time of the resulting source measures might be performed through the computation of a cortical displacement field within the optical flow framework. The theoretical principles of the method are briefly introduced and are subsequently illustrated by simulations. Finally, this technique was applied on brain image sequences from a ball-catching paradigm, in which significant structures of the resulting optical flow during the early brain response that accompanied the fall of the ball could be revealed.

I. I NTRODUCTION The exquisite time resolution of Magnetoencephalography (MEG) and Electroencephalography (EEG) provides access to the quasi-continuous investigation of mass-neural processes. Electromagnetic brain mapping through imaging techniques of MEG/EEG generators have been considerably improving recently [1], insomuch we are able to track the dynamical evolution of cortical surface currents within the millisecond range. With these imaging tools in hands however, neuroscientists are facing the emerging problem of finding significance in the plethora of MEEG spatiotemporal measures. Several approaches have been proposed in this respect which include the microstate concept and related techniques [9]. This latter aims at partitioning time series of EEG scalp topographies in terms of low-dimensional episodes with large spatial covariance. Here we propose an extended framework to the investigation of the spatiotemporal dynamics of brain activations in terms of the estimation and analysis of their displacement field. In a large number of applications, analysis of dynamical phenomena through the computation of a velocity vector field has contributed to the description and extraction of informational contents about the processes involved (see e.g. [2],[4]). Such approaches have barely been suggested for the analysis of structured patterns within brain signals and related neuroimaging sequences with high temporal resolution (e.g. using MEG/EEG or optical imaging). In [7] however, estimation of velocity fields was restricted to 2D images of narrow-band scalp EEG measures, with limited

accompanying quantitative analysis. We first introduce in Section II the general concept of cortical flow and work at the related theoretical aspects of the computation of optical flow on a 2-dimensional surface. In Section III, we evaluate the consistency of the estimates and run realistic numerical simulations. Finally, the method is illustrated from experimental data in the context of a ballcatching experimental investigation with MEG. II. V ELOCITY FIELD OF NEURAL ACTIVITY Our approach to the computation of the velocity vector field descends from optical flow techniques as introduced originally by Horn & Schunk [6]. These techniques have been demonstrating remarkable efficiency in the analysis of video sequences during the last two decades (see e.g. [3], which reviews a selection of computational methods associated with sound performance evaluation). The computation of vector flow is generally driven by basic assumptions which postulate conservation of the brightness of the moving objects. These restrictive hypotheses may not fit rigourously the exact nature of phenomena but have proven to yield commensurate estimations of vector fields provided they are valid locally in time and space [3]. The excellent timeresolution of MEG/EEG images is generally compatible with these assumptions, as brain responses unfold with substantial spatiotemporal smoothness to a large extent. In the context of brain imaging though, we are facing the issue of distributed intensity variations in 3D. Detection of activation may be restricted to the cortical surface as a first approach though, hence recent surface flattening tools could be applied [12] prior to using classical 2D flow estimators. However, such a transform entails multiple limitations: the required topological cuts in the closed surface of the brain induce linking problems at boundaries; moreover, local distortions of angles and distances are problematic when it comes to estimating the local orientation of the flow. Here we introduce a formalism based on differential geometry to extend the computation of optical flow equations on non-flat surfaces (see [5] for an introduction), hence on the folded geometry of the brain. A. Optical flow on a non-flat domain Let us define M, a 2-Riemannian manifold representing the imaging support (i.e. scalp or cortex), parameterized by

the local coordinate system φ : p ∈ M 7→ (x1 , x2 ) ∈ R2 . We introduce a scalar quantity defined in time on a 2-dimensional surface — e.g. brain activity from scalp MEG/EEG topographies or estimated cortical activation maps — as a function I(p, t) ∈ M, where (p, t) ∈ M × R. We note eα = ∂xα p, the canonical basis of the tangent space S Tp M at a point p of the manifold, and T M = p Tp M the tangent bundle of M. M is equipped with the canonical Euclidian metric :

Moreover E(V) can be simplified from (2) as a combination of the following linear and bilinear forms: Z ­ ® f (U) = − U, ∇M I ∂t I dµM , Z M ­ ®­ ® a(U, V) = U, ∇M I V, ∇M I dµM M Z + λ Tr(t ∇U∇V) dµM .

< . >: Tp M × Tp M → R, ³­ ®´ Note that the matrix eα , eβ is not identity because

Minimizing E(V) on Γ1 (M) is then equivalent to the following problem : ¡ ¢ a(V, V) − 2f (V) . (4) min 1

α,β

of the geometric curvature of the surface. As in classical computation approaches to optical flow, we now assume that the activity of a point moving on a curve c(t) in M is constant along time, hence : ­ ® ∂t I + V, ∇M I = 0.

(1)

Here, only the component of the flow V in the direction of the gradient is accessible for estimation. This corresponds to the well-known aperture problem [6], which requires additional constraints on the flow to yield a unique solution. This approach classically reduces to minimizing an energy functional such as [6]: ¶ Z µ Z ® 2 ∂I ­ E(V) = + V, ∇M I dµM +λ C(V)dµM , ∂t M M (2) The first term is a measure of fit of the optical flow model to the data, while the second term acts as a spatial regularizer of the flow. The scalar parameter λ tunes the respective contribution of these two terms in the net energy cost E(V). Here we use the smoothness term from [6], which can be expressed as a Frobenius norm: C(V) = Tr(t ∇V.∇V).

(3)

This constraint will tend to generate a regularized vector field with small spatial derivatives, that is a field with weak local variations and heterogeneities. B. Variational formulation Variational formulation of 2D-optical flow equation has been first proposed by Schnörr in [10], which advantages are twofold: 1) theoretically, it ensures the problem is wellposed; that is there exists a unique solution in a specific and convenient function space; 2) numerically, it allows to solve the problem on discrete irregular surface tessellations and to yield discrete solutions belonging to the chosen function space. We introduced in [8] a working space of vector fields called Γ1 (M) (typically the vector space of continuous piecewise affine vector fields on a tessellation) and an associated scalar product on which the functional E(V) can be minimized.

M

V∈Γ (M)

Lax-Milgram theorem ensures unicity of the solution V to (4) which satisfies: a(V, U) = f (U), ∀ U ∈ Γ1 (M).

(5)

C. Computation of the optical flow The variational formulation yields a solution to the regularized optical flow problem that may be sampled on a c with N edges through the finite elements tessellation M method (F.E.M.). Let us consider the vector space of continuous piecewise c which belong to the tangent space affine vector fields on M at each node of the tessellation. A convenient basis is: © ª wα,i = w(i)eα (i) for 1 ≤ i ≤ N , α ∈ 1, 2 , where w(i) stands for the continuous piecewise affine function which is 1 at node i and 0 at all other triangle nodes, and eα (i) is a basis of tangent space at node i. The variational formulation in (5) yields a classical linear system: N X 2 © ª X ∀j ∀β ∈ 1, 2 , a(wα,i , wβ,j )Vα,i = f (wβ,j ), i=1 α=1

(6) where Vα,i are the components of the velocity field V in the basis wα,i . Note that a(wα,i , wβ,j ) and f (wβ,j ) can be explicitly computed with first-order finite elements by estimating the integrals on each triangle T of the tessellation and summing the different contributions. III. S IMULATIONS We addressed the quantitative and qualitative evaluation of optical flow with simple and illustrative simulations on a selection of surfaces. Two types of synthetic data were created, hereby yielding typical situations encountered in MEG/EEG images, with measures either evolving on head or the cortical surface. The first set of simulations illustrates the emergence and fading of activity within a single brain region (see Fig. 1). Even though this situation infringes the hypothesis of intensity conservation across time, the radial structure of optical flow gives an indication on how the system is evolving in

Fig. 1. Simulating local emergence and waning of brain activity with resulting flow. Top row illustrates the progressive emergence followed by fading of a 33 cm2 region in the posterior medio-temporal brain area. The entire process unfolds within 100 time samples. Bottom row displays the corresponding distributions of vector flow; initially diverging from (left) then converging to (right) the center of the activation zone.

time at the regional scale, namely in successions of recurrent unfolding and collapsing activation patterns. The second kind of synthetic data simulates propagations of activity across distant brain regions (Fig. 2). An approx-

S1 , . . . , Sp which can be written as M = AS, where A is the gain matrix from the forward model. Electromagnetic brain mapping consists of the estimation of MEG/EEG sources S from scalp measures M . However, this inverse problem is underdetermined since there are far more possible cortical sources than sensors. s is typically on the order of a few hundreds at each time instant, while p amounts to about 10000 elemental sources constrained onto the surface of the cortical manifold, which was extracted from MRI image sequences (http://brainvisa.info/). Inverse modelling may therefore be approached as in many other image reconstruction applications, i.e. though the introduction of priors in addition to data. Here we used a weighted-minimum norm estimate (WMNE) of source amplitudes and its implementation in the BrainStorm software (http://neuroimage.usc.edu/brainstorm). B. Evaluation on experimental data We applied optical flow computation to magnetic evokedfields in a ball-catching paradigm [11]. The subjects were asked to catch a free-falling tennis ball which fall was initiated at time t = 0 ms (‘catch’ condition). The second experimental condition (‘no-catch’) consisted for the subject in only looking at the ball falling without catching it. Results from a single subject are shown for illustration of the proofof-concept.

Fig. 2. Simulation of brain activation propagating at the surface of the cortex. Top row, from left to right: displacement field of an activation patch is translating along a predefined path during 100 steps of time. Bottom row: the mean vector field and true displacement are shown in green and blue, respectively, with indication of instantaneous angular error. For clarity purposes, only one brain hemisphere is shown within the scalp surface.

imation of a Gaussian patch propagates from a rather flat domain of the cortical manifold and travels down into a sulcal fold. We represent the velocity of the patch centroid and the mean optical flow projected on a plane containing the true displacement. IV. A PPLICATION TO THE INVESTIGATION OF SPATIOTEMPORAL DYNAMICS OF MEG SIGNALS A. The electromagnetic brain imaging problem Magneto and electroencephalography stem from similar physical principles since they are directly related to the electromagnetical activity of neurons. Magnetic fields (and similarly electric potentials) are sampled on s sensors, M1 , . . . , Ms , as a linear combination of p sources signals

Fig. 3. a) MEG activities at 59 ms (No catch condition), superior view and 75%-rear view; b) Temporal evolution of the maximum of ||V || in the catch (res. no-catch) condition. The circle marker (res. square) indicates two temporal instant of interest at 54 ms (res. 48 ms) and 65 ms (res. 59 ms); c) Display of activities in red-yellow and optical flow in green at the four previous temporal instants.

Fig. 3 shows the resulting optical flow during an early response about 60 ms. The two curves represent at each instant t the quantity maxp∈M ||V(p, t)|| in each experimental condition. The MEG activations are shown when the optical flow reaches a its maximum magnitude (65 ms in catch, 59ms in no-catch) and 11 ms before. We observe similar emerging patterns of activity in the right occipital area (V1) and on both sides of the right occipito-parietal sulcus. The evolution suggests a progressive activation of the MT/V5 area. Besides in both conditions, we also note that the organization of cortical flow is similar with a diverging structure on the V1 pattern and a diverging/propagating pattern from the occipito-parietal sulcus propagating to the dorsal pathway. This link between the directionality of the displacement field and the succession of spatial patterns seems qualitatively clear and motivates the usage of a ‘cortical flow’ concept. V. C ONCLUSION We have introduced a new method for quantifying the displacement field in the spatiotemporal data measured with MEG and EEG and in the corresponding cortical source patterns. This method extends a classical tool in computer vision – the optical flow – to surfaces. Results from simulations indicate the flow has satisfactory behavior in terms of spatial structure and angular errors. We have illustrated this new measure with MEG cortical source imaging from experimental data paradigm. We reported that the optical flow could help reveal some additional valuable information about the directionality of mass-neural dynamics at the regional scale. Ongoing developments consist in using this so-called cortical flow in data mining applications and in relating this general behavior of cortical currents to its physiological origins. R EFERENCES [1] S. Baillet, J.C. Mosher, and Leahy R.M. Electromagnetic brain mapping. IEEE Signal Processing Magazine, november 2001. [2] J.L. Barron and A. Liptay. Measuring 3-d plant growth using optical flow. Bioimaging, 5:82–86, 1997. [3] S.S. Beauchemin and J.L. Barron. The computation of optical flow. ACM Computing Surveys, 27(3):433–467, 1995. [4] T. Corpetti, D. Heitz, G. Arroyo, E. Mémin, and A. Santa-Cruz. Fluid experimental flow estimation based on an optical-flow scheme. Experiments in fluids, 40(1):80–97, 2006. [5] M.P. Do Carmo. Riemannian Geometry. Birkhäuser, 1993. [6] B.K.P. Horn and B.G. Schunck. Determining optical flow. Artificial Intelligence, 17:185–204, 1981. [7] T. Inouye, K. Shinosaki, S. Toi, Y. Matsumoto, and N. Hosaka. Potential flow of alpha- activity in the human electroencephalogram. Neurosci Lett, 187:29–32, 1995. [8] J. Lefèvre, G. Obosinski, and S. Baillet. Imaging brain activation streams from optical flow computation on 2-riemannian manifold. Journal of Royal Society Interface, submitted. [9] R.D. Pascual-Marqui, C.M. Michel, and D. Lehmann. Segmentation of brain electrical activity into microstates: Model estimation and validation. IEEE Trans. Biomed. Eng., 42:658–665, 1995. [10] C. Schnörr. Determining optical flow for irregular domains by minimizing quadratic functionals of a certain class. Int. J. Computer Vision, 6(1):25–38, 1991. [11] P. Senot, M. Zago, F. Lacquaniti, and J. McIntyre. Anticipating the effects of gravity when intercepting moving objects: Differentiating up and down based on nonvisual cues. J Neurophysiol, 94:4471–4480, 2005. [12] D.C. Van Essen, H.A. Drury, S. Joshi, and M.I. Miller. Functional and structural mapping of human cerebral cortex: Solutions are in the surfaces. Proc Natl. Acad. Sci. USA, 95:788–795, 1998.