NeuroImage 52 (2010) 326–335
Contents lists available at ScienceDirect
NeuroImage j o u r n a l h o m e p a g e : w w w. e l s e v i e r. c o m / l o c a t e / y n i m g
Theta–gamma phase synchronization during memory matching in visual working memory Elisa Mira Holz a, Mark Glennon a,c, Karen Prendergast a,c, Paul Sauseng a,b,⁎ a b c
Department of Physiological Psychology, University of Salzburg, Austria Brain Imaging and Neurostimulation Lab, Department of Neurology, University Hospital Hamburg-Eppendorf, Germany School of Psychology, National University of Ireland, Galway, Ireland
a r t i c l e
i n f o
Article history: Received 2 September 2009 Revised 23 March 2010 Accepted 2 April 2010 Available online 9 April 2010
a b s t r a c t In most cases, object recognition is related to the matching of internal memory contents and bottom-up external sensory stimulation. The aim of this study was to investigate the electrophysiological correlates of memory matching based on EEG oscillatory phase synchronization analysis. Healthy subjects completed a delayed-match to sample task in which items stored in visual-spatial short-term memory had to be compared with a matching or non-matching probe. The results show that memory matching appears as transient phase-synchronization over parieto-occipital regions between theta (4–8 Hz) and high gamma (50–70 Hz) oscillations, 150–200 ms post probe presentation. When memory representation and visual information match, phase-synchronization is stronger in the right hemisphere; conversely, when they do not match, stronger phase synchronization is observed in the left hemisphere. The present results reveal the integrative role of oscillatory activity in the memory matching process. © 2010 Elsevier Inc. All rights reserved.
Introduction Perception or recognition of visual stimuli often relies on the comparison of incoming bottom-up and top-down information stored in memory. Recognition of familiar visual information can be approached when incoming input matches with our memory contents. Bottom-up processes such as feature binding can be functionally linked to fast oscillations such as gamma frequency (above 30 Hz; Engel et al., 1999; Singer, 1999; Tallon-Baudry and Bertrand, 1999). Gamma responds to object features, e.g., size, eccentricity and duration of visual stimuli (in particular evoked gamma; Busch et al., 2004, 2006; Lenz et al., 2008; for a review see Herrmann et al., 2004b) and has been discussed to play an active role in the coding and maintenance of object properties in working memory (Tallon-Baudry et al., 1998; Howard et al., 2003; Kaiser and Lutzenberger, 2003). Top-down processes correlate with a wide range of brain oscillations. Slow oscillations, such as theta (around 6 Hz), however, seem to play a core role in top-down cognitive processes. Theta activity has consistently been found in relation to working memory processes, in particular the executive control of working memory functions (Sauseng et al., 2004, 2005; Sauseng and Klimesch, 2008), and has been related to working memory load (Gevins and Smith, 2000; Jensen and Tesche, 2002). The aforementioned findings imply that the matching of new sensory and ⁎ Corresponding author. Brain Imaging and NeuroStimulation Lab, Department of Neurology, UKE University Hospital Hamburg-Eppendorf, Martinistr. 52, D-20246 Hamburg, Germany. E-mail address:
[email protected] (P. Sauseng). 1053-8119/$ – see front matter © 2010 Elsevier Inc. All rights reserved. doi:10.1016/j.neuroimage.2010.04.003
short-term stored information should involve theta and gamma oscillations or even an interaction of these oscillations. Herrmann et al. (2004b) proposed the matching-and-utilization model (MUM) predicting that the memory matching process is correlated with early (before 150 ms) evoked gamma activity. In the context of object recognition, an enhanced evoked gamma response is evident when sensory input and memory content match in comparison to non-match conditions. After this comparison the brain utilizes matched information for memory updating or the reallocation of attention which is supposed to be associated with the late (post 200 ms) induced gamma response. Thus, the model proclaims that stimuli with a (long-term) memory representation modulate gamma responses (Herrmann et al., 2004a,b). The MUM can well account for matching between long-term memory contents and sensory input. However, it is challenged when applied to matching between short-term memory information and sensory information. For instance, in a visual classification task Kanizsa figures which were identified as targets elicited stronger evoked gamma activity than standard Kanizsa figures (Herrmann et al., 1999; Herrmann and Mecklinger, 2000) and also in an auditory working memory task (oddball task) in which a memory template had to be maintained (or preactivated) throughout the experiment higher evoked gamma activity was found for deviant compared to standard tones (Debener et al., 2003). On the other hand in a visual task a target-related increase of only late induced gamma activity was found (Kranczioch et al., 2006). Moreover, studies in animals (Jeschke et al., 2008) and humans (Lenz et al., 2008) using auditory target stimuli in working memory tasks fail to find effects in the early evoked gamma response during memory matching, as does
E.M. Holz et al. / NeuroImage 52 (2010) 326–335
research on multimodal matching in working memory (Senkowski et al., 2009). Leiberg et al. (2006) also report differences between matching and non-matching stimuli in an auditory working memory task. However, they could only find effects on induced gamma activity with higher amplitudes even for non-match trials. Thus, matching processes in short-term memory might not only rely on evoked gamma activity but may at least partly be reflected by different electrophysiological correlates. An increasing number of findings suggest that theta–gamma coupling plays an important role in working memory processes (Schack et al., 2002; Demiralp et al., 2007; Canolty et al., 2006; Sauseng et al., 2009). Studies focussing on an interaction between theta and gamma oscillations during memory matching however are rather sparse. In a study of Sauseng et al. (2008) subjects performed a visual-spatial attention paradigm, in which they had to differentiate between two targets in an attended or not attended visual field. It was found that target presentation in the cued visual field results in enhanced theta– gamma phase coupling in the contralateral as compared to the ipsilateral hemisphere. This effect was not visible when the target was presented in the unattended visual field indicating that when top-down (attentional) information meets visual sensory bottom-up input increased phase interaction between theta and gamma activity in early visual brain areas is elicited. These findings indicate that theta to gamma synchronization is a correlate of matching of a memory component (very likely reactivated in the episodic buffer of working memory) with incoming sensory information which is modulated by top-down attentional control process. It is of note that in short-term memory tasks as well as in the visuospatial task used by Sauseng et al. (2008) mental representations of memory items are persistently retained or reactivated from long-term memory. It is suggested that maintenance of information in working memory is associated with distributed oscillatory activity at theta frequency (see e.g., Sarnthein et al., 1998; Raghavachari et al., 2001; Sauseng et al., 2010). This might be the reason why evoked gamma response as correlate for memory matching in short-term memory has not been found consistently (Jeschke et al., 2008; Lenz et al., 2008; Senkowski et al., 2009), but could be rather reflected by an interaction between theta and gamma oscillations (Sauseng et al., 2008). On the other hand, when memory information is not preactivated and persistently retained in a short-term memory store but instead activated in long-term memory during the matching process evoked gamma activity might be the adequate mechanism reflecting memory matching (Herrmann et al., 2004b). However, in general, it is not completely clear yet under which circumstances there will be increased evoked or induced gamma activity or even cross-frequency interaction during the matching of bottom-up sensory and top-down memory information in working memory. Therefore, this study was designed to find out whether memory matching in a visuospatial short-term memory task is reflected by evoked or induced gamma activity or by cross-frequency interaction between theta and gamma frequencies. In contrast to the findings by Sauseng et al. (2008), who reported cross-frequency interaction between theta and lower gamma frequencies we expected synchronization between theta and higher gamma activity beyond 50 Hz in the present study, as it has been suggested by Sauseng et al. (2009) that higher gamma frequency is coupled to theta activity during visuospatial working memory processes. EEG was recorded in 23 subjects while performing a visuospatial delayed match to sample task. Subjects had to encode five sequentially presented locations within a 5 × 5 matrix. After a retention period, a probe, showing either five positions matching with the positions shown before or not matching, was presented, and subjects had to decide whether the probe locations were identical with those previously encoded or not. Our main focus was to find interaction effects of bottom-up and top-down processes in theta–gamma phase coupling. As suggested by Sauseng et al. (2008), memory contents are stored within a distributed theta network, which is matched with
327
incoming sensory input at posterior brain areas. Therefore, we expected to find interaction effects between theta and gamma oscillations at parietal and occipital regions. As suggested by several neuroimaging studies (Jonides et al., 1993; D'Esposito et al., 1998; Berryhill and Olson, 2008; van der Ham et al., 2009) visuospatial working memory is lateralized with stronger right hemispheric activation. Thus, it was hypothesized that memory matching-related theta–gamma phase coupling should be stronger at right compared to left posterior recording sites. In addition local amplitude estimates (induced and evoked EEG activity) were assessed in order to test the validity of the MUM as suggested by Herrmann et al. (2004b) for visual delayed match to sample tasks. Moreover, we calculated interregional phase coherence to investigate the role of large-scale interaction in particular of theta activity, which is suggested to be relevant for maintenance of visual information in short-term memory, during memory matching and scanning. Methods Participants Twenty-three volunteers participated in this experiment. EEG data of five participants were excluded from analysis due to heavy contamination by blink artifacts. The remaining sample of eighteen subjects (5 men and 13 women) had a mean age of 23.67 (SD = 2.99). All subjects were right-handed, had normal or corrected to normal vision and gave informed consent prior to the experiment. The study was conducted in accordance with the declaration of Helsinki. Experimental Design EEG was recorded while participants performed a visuospatial delayed match to sample task (see Fig. 1). On a computer monitor a matrix consisting of 5 × 5 squares was presented (covering 6.9°*6.9° visual angle). In every trial 28 positions flashed up sequentially (with a duration of 167 ms). Five positions were indicated in green, two in red, the other positions flashed up in gray (the three colors were isoluminat; neural basis of encoding in this experiment will be addressed in another paper; thus, encoding processes are beyond the scope of this manuscript). The participants were asked to retain only the green positions, whereas the red and gray ones were to be ignored. The temporal order of item presentation was identical in each trial, i.e., green target items always were separated by three distracter items. 2000 ms after presentation of the last spatial position – in which the empty matrix was shown – a probe item displaying five green positions was presented for 2000 ms (retrieval; therefore possible afterimages could only occur after the analyzed segment). The subjects had to decide by button-press whether the probe matched with the encoded spatial positions. In half the trials the probe matched (match condition) with the previously encoded positions, in the other half of trials it did not match, i.e., that means four identical positions and one random new position were presented (non-match condition). The whole experiment consisted of 200 trials. Before the experiment a training block consisting of 10 trials was run. EEG recording and analysis EEG was recorded between 0.16 and 70 Hz with a notch filter at 50 Hz and a sampling frequency of 1000 Hz from 60 Ag-AgCl scalp electrodes using a BrainAmp amplifier (Brain Products, Germany). Electrodes were positioned according to the 10–20 system and recorded against a common reference placed on the nose. In addition, vertical and horizontal EOG were recorded. Impedance was kept below 8 kΩ. First, data were re-referenced to digitally linked earlobes. A low-cutoff filter (FIR filter with 48 dB/oct) at 0.5 Hz and ocular correction (Gratton et al., 1983) was applied. To attenuate spurious distributed EEG
328
E.M. Holz et al. / NeuroImage 52 (2010) 326–335
Fig. 1. Experimental paradigm. Spatial locations within a 5 × 5 matrix were displayed sequentially in red, gray and green. The green positions had to be encoded. After retention of 2 s a probe (either matching or non-matching respectively) was presented. In response to the probe participants had to indicate whether the retained locations were congruent (match) or incongruent (non-match) with the ones of the probe.
activity due to volume conduction (in particular for coherence analysis) laplacian current source density (CSD) was calculated (order of splines: 4; max. deg. of Legendre polynomials: 10; lambda: 1 e−5). According to Melloni et al. (2009) CSD transformation will also prevent posterior recording sites from being affected by artifacts coming from micro saccadic eye movements which can potentially influence also crossfrequency synchronization estimates (Kramer et al., 2008). Remaining artifacts due to eye-blinks or muscle activity were rejected semi-automatically. Data were segmented into 2000 ms intervals starting 1000 ms prior to probe item onset. Only correctly responded trials were used for analysis. On average 70.28 segments in the match and 65.56 segments in the non-match condition remained for further analysis. To obtain cross-frequency phase synchronization between theta and gamma oscillations Gabor expansion was applied to single trials using Matlab 7.0.1 (Mathworks). This resulted in time and frequency information of EEG phase and amplitude for 70 frequency bins between 1 and 70 Hz (with a 1 Hz resolution). As frequency bands of interest theta (4–8 Hz), and a lower (30–50 Hz) and higher gamma band (50– 70 Hz) were selected. Sauseng et al. (2008) reported increased theta to lower gamma phase coupling during matching of top-down and bottom-up information in a visual attention task. However, a recent study suggests theta to high gamma phase synchronization as a correlate of visual working memory (Sauseng et al., 2009). For crossfrequency phase synchronization the center frequency for each band was used (theta: 6 Hz, low gamma: 40 Hz, high gamma: 60 Hz). Then for each trial and time frame instantaneous phase difference between two frequencies fm and fn (representing theta and gamma) was calculated following Eq. (1). k
k
m·fn = n·fm ΔΦk ð fn ; fm ; tÞ≅ðn·Φ1 ðfm ; tÞ−m·Φ1 ðfn ; tÞÞ modulus 2π
ð1Þ
Thereby Ф denotes the instantaneous phase angle, k the kth trial and t the tth sampling point. For each sampling point t a cross-frequency phase synchronization index was calculated by estimating the consistency of phase differences derived from Eq. (1) over trials. This was done following Eq. (2). ΓˆΦ ðfn ; fm ; tÞ =
∣〈e
j·ΔΦk ð fn ; fm ; tÞ
〉∣; j =
pffiffiffiffiffiffiffiffi −1
ð2Þ
This estimate can range from 0 to 1, with 1 denoting highest phase synchronization (for details on the method see Schack et al., 2005; Schack and Weiss, 2005; Sauseng et al., 2008). For statistical analysis cross-frequency phase coupling estimates were averaged over time (11 time windows: reference: −200–0 ms, t0: 0– 50 ms, t1: 50–100 ms, t2: 100–150 ms, t3: 150–200 ms, t4: 200–250 ms, t5: 250–300 ms, t6: 300–350 ms, t7: 350–400 ms, t8: 400–450 ms, t9: 450–500 ms) and over frequency ranges (for the combinations of center frequencies theta (6 Hz) to low gamma (40 Hz), and theta (6 Hz) to high gamma (60 Hz)) for each condition (match vs. non-match) separately and at each of the 58 electrode sites. Then the difference of crossfrequency coupling estimates between the two conditions match and non-match was calculated and pooled for two regions of interest (ROI): left (PO3, PO7, O1) and right parieto-occipital (PO4, PO8, O2). Parietooccipital regions were selected because it was hypothesized that matching processes should occur in early visual areas. As the effects were expected strongest in the right hemisphere left and right posterior ROIs were compared. Three-way ANOVAs with the factors TIME (11 levels), HEMISPHERE (left parieto-occipital vs. right parieto-occipital) and CONDITION (match vs. non-match) were run for each frequency pair (theta-low gamma, theta-high gamma) separately. To exclude that cross-frequency phase synchronization was only elicited due to simultaneous phase resetting of theta and gamma in response to probe presentation, the same cross-frequency analyses as described above were also carried out on a data set where theta was shifted for one trial in reference to gamma (i.e., phase differences between theta from trial 2 and gamma from trial 1, theta from trial 3 and gamma from trial 2, etc. were used for estimating cross-frequency synchronization; this was done only for theta-high gamma because significant effects were exclusively found for this frequency pair). Therefore, spurious effects of theta–gamma phase synchronization which rely on evoked responses should occur at a fixed latency in each trial and consequently should be unchanged compared to real data. Real effects independent of phase-locking to stimulus onset on the other hand should be higher in the real than in surrogate data. To further control for the possibility of spurious cross-frequency phase synchronization due to evoked responses, we calculated a phase locking index for theta and gamma separately (PLI; for details see Schack and Klimesch, 2002). This index estimates the consistency of phaselocking to stimulus onset within single frequency bins. A three-way ANOVA with the factors CONDITION (match vs. non-match), TIME (11
E.M. Holz et al. / NeuroImage 52 (2010) 326–335
steps) and ROI (left vs. right) was run for each frequency band separately. Post hoc t-tests were run to compare PLI-values of the reference interval (−200–0 ms) with all other time windows in order to quantify changes of phase-locking as a function of time. For cross-frequency phase synchronization and PLI all measures were normally distributed as tested using Kolmogorov–Smirnov tests. Greenhouse–Geisser correction was applied where necessary. In order to investigate interregional oscillatory activity contributing to matching processes, a phase locking value (PLV) was computed for all possible electrode pairs, as implemented in BESA 5.1 (MEGIS Software Inc). PLV is a measure to obtain the intertrial variability of the phase differences between two signals recorded from separate electrodes (for details see Lauchaux et al., 1999). A PLV close to 1 indicates that the phase differences show consistency across trials, and
329
thus it reflects high connectivity between the two recording sites, whereas a PLV close to 0 means that phase differences vary over trials. To compare PLV values between conditions, we calculated Wilcoxon tests for all 1770 electrode pairs and 100 ms time windows. To correct for multiple comparisons a bootstrapping procedure was conducted. Data were permuted and Wilcoxon tests were run on the randomized data; this procedure was repeated 1000 times. Then, from each of the 1000 runs, the distribution of p-values was considered and the p-value representing the 1st percentile was entered in further analysis. From these so obtained 1000 p-values the critical p-value was defined on the criterion where again only 1% of p-values was smaller. Only those electrode pairs for which the difference of PLV between match and non-match condition yielded a Wilcoxon test (on the original data) with a p-value smaller than the critical p-value determined by the
Fig. 2. Theta-high gamma phase synchronization difference (match - non-match) averaged over left (PO3, PO7, O1) and right (PO4, PO8, O2) parieto-occipital recording sites for real (A) and surrogate (B) data. Positive values depict higher cross-frequency phase synchronization for the match than non-match condition and negative values depict higher synchronization for the non-match condition. Scalp maps show posterior topography of theta: high gamma cross-frequency phase synchronization after probe presentation (for the time intervals: −200–0 ms, 50–100 ms, 150–200 ms, 250–300 ms, 350–400 ms, from left to right). Red denotes higher synchronization for the match than non-match, blue denotes higher synchronization in the non-match than match condition.
330
E.M. Holz et al. / NeuroImage 52 (2010) 326–335
bootstrapping procedure were considered as relevant. This analysis was carried out for the frequencies theta (4–8 Hz), low gamma (30– 50 Hz) and high gamma (50–70 Hz) and for the time intervals reference: −200–0 ms, t0: 0–100 ms, t1: 100–200 ms, t2: 200–300 ms, t3: 300–400 ms and t4: 400–500 ms, separately. McNemar tests (chi²tests for repeated measures) were used to assess changes in the number of electrode pairs showing significant PLV differences between conditions from the reference interval to post-stimulus time windows. The results from McNemar tests were Bonferroni corrected. Through averaging of single trials, event-related potentials (ERPs) were obtained. For analysis of local amplitude differences between conditions, current source density transformed single trials were filtered using bandpass filtering (FIR filter; 48 dB/oct) in the above defined theta, low and high gamma frequency bands. For evoked power, filtered segments were averaged and subsequently rectified, for each condition (match vs. non-match) separately. For total power, filtered segments were first rectified and then averaged for each condition (match vs. non-match) separately. Three-way ANOVAs using evoked/ total power with factors CONDITION (match vs. non-match), TIME (11 steps; same as used for PLI) and HEMISPHERE (left vs. right) were run for each frequency band. Post hoc t-tests were run to compare reference interval power (−200–0 ms) with all other time windows for each hemisphere. As revealed by Kolmogorov–Smirnov tests, all measures showed normal distribution except total power values within the theta band for the 400–450 ms period, in the right hemisphere (p = .034). However, as this might be a false positive result obtained by multiple Kolmogorov–Smirnov tests, for simplicity, parametric statistics were also applied for analysis of local amplitude estimates. Greenhouse– Geisser correction was applied where necessary. Results Behavioral performance 79.3% (SD = 9.72) of responses in the match condition and 73.5% (SD = 14.86) of the responses in the non-match condition were correct. This difference was not significant (t17 = 1.71, p = .12). However, although no speeded responses were required in the task, response times differed between conditions. In the match condition (1005.3 ms, SD = 230.15) reaction times were significantly shorter (t17 = −6.04, p b .001) than in the non-match condition (1129.5 ms, SD = 234.12). Cross-frequency phase synchronization A three-way interaction between the factors CONDITION, HEMISPHERE and TIME was significant (F10,170 = 2.39; p b .05), indicating largest theta-high gamma phase synchronization difference between the two conditions 150–200 ms after probe presentation in parietooccipital cortex. Whereas theta-high gamma phase synchronization in the right hemisphere was higher for match than for non-match, the left hemisphere showed higher synchronization for non-match than match (see Figs. 2 and 3). There was no significant effect for crossfrequency phase synchronization between theta and low gamma. Cross-frequency phase synchronization on surrogate data The analysis of theta-high gamma on surrogate data revealed a significant interaction between the factors CONDITION, HEMISPHERE and TIME, (F10,170 = 2.31, p b .05). Similar to real data in the left hemisphere there was a trend for higher theta-high gamma phase synchronization for the non-match compared to match condition. For the right hemisphere also higher theta-high gamma phase synchronization was found for non-match than match within the time window from 150–200 ms (see Fig. 2). This finding is in sharp contrast to the effects reported for real data.
Phase locking index (PLI) PLI was analyzed for theta and high gamma in order to control for spurious cross-frequency phase synchronization due to simultaneous phase-locking to stimulus onset of theta and high gamma oscillations. For theta we found a significant main effect for the factor TIME (F10,170 = 71.52, p b .001), indicating a significant increase in theta phase locking after probe presentation (see Fig. 3). Post-stimulus PLI estimates were significantly higher as compared to a reference interval (p b .001) with the exception of t9 (450–500 ms; t17 = −1.7, p = .11). There were no other significant main effects (HEMISPHERE or CONDITION) or interactions. For high gamma there was also a main effect for the factor TIME (F10,170 = 4.52, p = .01). However, in contrast to theta, gamma showed a decrease in PLI after stimulus onset. Compared to the pre stimulus interval, gamma PLI changed significantly 100–150 ms after probe presentation – all post hoc comparisons with the reference interval were significant, p b .05, except for t0 = 0–50 ms (p = .55), t1 = 50–100 ms (p = .30) and t8 = 400–450 ms (p = .11). Interregional phase synchronization (PLV) We found significantly stronger interregional synchronization, especially long-range connections, for non-match than match conditions in the theta as well as low and high gamma bands. Compared with the reference interval, the number of electrode pairs showing stronger PLV in the non-match than in the match condition was higher for theta 300–400 ms (χ² = 6.61, p b .01) and 400–500 ms (χ² = 25.81, p b .01) post-stimulus, for low gamma 300–400 ms (χ² = 9.75, p b .01) and 400– 500 ms (χ² = 13.83, p b .01), and high gamma 200–300 ms (χ² = 11.22, p b .01) post-stimulus (see Fig. 4). Instances where electrode pairs exhibited stronger interregional phase synchronization for match than non-match conditions – in any frequency band – were rare. There was no significant increase or decrease in number of electrode pairs with higher PLV in match than non-match conditions in the post-stimulus time windows as compared to the reference interval. Total and evoked power The comparison of total theta power revealed a significant interaction between the factors CONDITION, HEMISPHERE and TIME (F10,170 = 2.89, p b .05). There was stronger total theta power in nonmatch than in match for the left hemisphere, 200–250 ms (t17 = 2.38, p b .05) and 300–350 ms (t17 = 2.12, p b .05) post-stimulus, as well as in the right hemisphere, 0–50 ms (t17 = 2.40, p b .05), 50–100 ms (t17 = 2.18, p b .05) and 100–150 ms (t17 = 2.32, p b .05; compared to a reference interval). There were no significant effects obtained neither for total gamma nor for evoked theta or gamma power (see Fig. 5). Discussion Cross-frequency phase synchronization The purpose of the present study was to investigate the electrophysiological correlates of memory matching process in visuospatial working memory. Subjects performed a delayed-match to sample task by comparing a mental representation of visual spatial locations retained in working memory with either matching or non-matching sensory input. Our findings suggest that a comparison of new (bottom up) and expected (top-down) information is associated with crossfrequency theta-high gamma phase synchronization in parieto-occipital brain areas in an early time window, 150–200 ms after probe presentation. This is well in line with assumptions of Herrmann et al. (2004b) and also with findings of Sauseng et al. (2008), suggesting that an interaction of bottom-up and top-down information takes place in an early time window. Recent findings indicate that different oscillations, e.g., alpha-gamma, become phase synchronized in response to working
E.M. Holz et al. / NeuroImage 52 (2010) 326–335
331
Fig. 3. Cross-frequency phase synchronization (A), ERPs (B) and PLI (C) for theta and high gamma for match (red line) and non-match (blue line) condition separately for the left and right hemisphere.
332
E.M. Holz et al. / NeuroImage 52 (2010) 326–335
Fig. 4. Interregional phase synchronization for theta (A), low (B) and high (C) gamma. Temporal evolution of the number of electrode pairs which show significantly higher phase coupling in the non-match compared with match condition is depicted. Note that differences between conditions increase in later time windows. Scalp maps for time windows showing significantly more electrode pairs with a reliable difference between match and non-match after probe presentation (dark gray bars) as compared to pre-stimulus (red bars) are depicted. Blue lines indicate higher interregional phase coupling for the non-match than match, red lines indicate higher interregional phase synchronization for the match than non-match condition.
memory demands (Palva et al., 2005). To investigate the frequency specificity of theta-high gamma phase synchronization in the current study, we also analyzed alpha (8–12 Hz with a center frequency of 10 Hz) to lower and higher gamma band cross-frequency phase synchronization (the same analysis strategy and statistics as used for theta–gamma coupling were applied). However, none of these frequency combinations showed any significant effect. Interestingly, matching led to enhanced theta-high gamma phase coupling in the right hemisphere. Non-matching on the other hand
surprisingly elicited stronger theta-high gamma phase coupling in the left hemisphere. Initially it was hypothesized that stronger matchingrelated theta–gamma phase coupling would be obtained for the right hemisphere. However, we did not expect the reversed pattern, i.e., stronger theta-high gamma phase synchronization for non-match than match, at left posterior recording sites. The posterior parietal/occipital cortex seems to play a core role in visual working memory. A large body of findings stress the importance of especially the right parietal cortex for processing of visual-spatial
E.M. Holz et al. / NeuroImage 52 (2010) 326–335
333
Fig. 5. Time frequency plots of total (A) and evoked (B) power. Mean values for regions of interest (ROI) of the left and right hemisphere; upper row shows the match, lower row the non-match condition.
334
E.M. Holz et al. / NeuroImage 52 (2010) 326–335
working memory (Jonides et al., 1993; Berryhill and Olson 2008). Moreover, evidence indicates that there is hemispheric specialization regarding the processing of visual features. Findings from behavioral (Andersen and Marsolek, 2005; Marsolek and Deason, 2007) as well as brain imaging studies (Koutstaal et al., 2001) suggest specialization of the two hemispheres in processing of visual information. Holistic processing operates more effectively in the right hemisphere, whereas detail-based processing is carried out more successfully by the left hemisphere (for a review see Marsolek et al., 1996). In studies on global/ local selective attention tasks subjects have to identify either the global or local letter of a hierarchical stimulus (e.g., a large, global H made up of small, local Ss). Results indicate more pronounced right hemispheric activity for global targets and even for cues indicating global targets, and more pronounced left hemispheric activity for local targets as well as cues indicating local processing, respectively (Volberg and Hübner, 2004; Weissman and Woldorff, 2005; Volberg et al., 2009). Inferred from these findings we suggest that memory matching strains global and local processing with differential extent of involvement that causes the hemispheric lateralization effects. Thereafter a match probe involves global processing (implemented by the right hemisphere) to a higher degree than a non-match probe, on the other hand a non-match probe strains local processing (implemented by the left hemisphere) stronger than a match probe. In this context global processing means that in the case of a match probe the similarity of the probe and the retained items can be recognized by the “shape” or structure formed by the items. Otherwise a non-match probe entails a rather detailed scanning of the local components or single items to ensure which item has changed comparing the retained figure of the items with the items of the probe. Therefore, higher right-hemispheric theta–gamma phase synchronization might indicate a more global and very efficient early matching process. Increased cross-frequency coupling in the non-match condition at left parietal sites might indicate the detection of a discrepancy between mental representations and a probe item. This left lateralized activation might in turn trigger further detailed processing of incoming information. Phase resetting The aforementioned results on PLI show that theta might undergo phase resetting after stimulus onset indicated by strong phase locking. We therefore assume that the reset of theta phase enables theta–gamma phase synchronization. These findings are well in line with electrophysiological studies in animals (Givens, 1996; Williams and Givens, 2003), as well as in humans (Tesche and Karhu, 2000; Sauseng et al., 2008), showing that theta phase resetting is important in working memory processes. Furthermore, recent findings indicate that lower frequencies such as theta can modulate gamma oscillations. In monkeys (Lakatos et al., 2005) as well as in humans (Canolty et al., 2006; Demiralp et al., 2007) it was found that gamma amplitude is modulated by theta phase. The circumstance that in our data gamma does not exhibit simultaneous phase-locking to stimulus onset as theta does suggests that the transient increase in cross-frequency phase synchronization is no artifact caused by phase concentration due to an evoked response (see also Sauseng et al., 2008). This is also underpinned by the cross-frequency phase synchronization control analysis on surrogate data showing unspecific or even oppositional results to the original data for the right hemisphere. This implies that the effect on theta-high gamma phase synchronization is not an artifact due to an evoked response. If this increase of phase synchronization between theta and high gamma 150–200 ms after probe presentation was generated through an evoked response then the equivalent effect should have been found also for the surrogate data. Interregional phase synchrony When sensory input does not match with the expected stimulus, here spatial locations, it is probable that higher cognitive effort is
required. We expected that non-matching should strain cognitive resources to a higher degree than matching processes, involving in particular later time windows, even after a first comparison with a negative result around 150 ms has been performed. Indeed we found that non-matching leads to stronger interregional, very distributed and rather unspecific synchronization compared to matching. Coupling was pronounced in the theta (4–8 Hz) as well as both gamma (30–50 Hz and 50–70 Hz) bands and was strongest in later time windows, 300–500 ms post-stimulus. Long-range theta phase synchronization was found in context with executive control of encoding and retrieval processes (Sauseng et al., 2005, 2006; see Sauseng and Klimesch, 2008, for a review). It seems plausible to argue that non-match requires monitoring or scanning processes that are controlled by the central executive. Longrange gamma synchrony has been correlated with (conscious) perception (Melloni et al., 2007) and the maintenance of a coherent object representation (for a review see Tallon-Baudry, 2003). In addition it was proposed that long-range gamma phase synchrony serves as a mechanism of large scale cognitive integration rather than local feature binding processes (Rodriguez et al., 1999). As suggested by Herrmann et al. (2004b), utilization process such as redirecting of attention or coordinating behavioral performance should be reflected by large-scale gamma coherence. It seems plausible that non-matching strains longlasting interregional phase coupling to a higher degree for operating with new input and the short-term memory information necessary for decision making. Local amplitude Furthermore we found that theta power in bilateral parietal regions was higher for processing of non-match probes, possibly indicating active memory retrieval attempts (Raghavachari et al., 2006). First right parietal, then left parietal regions were involved. Theta power therefore does not show the same activation pattern as theta-high gamma phase synchronization does. Because one would rather expect memory matching correlates at gamma frequency than at theta, we interpret the theta power results to rather reflect active, less automatic memory scanning processes. This also suggests, that amplitude and phase information can indicate different cognitive processes (Palva et al., 2005; Palva and Palva, 2007). Although there is a large body of findings suggesting the role of evoked gamma oscillations in memory matching process (for a review see Herrmann et al., 2004b), we were not able to find significant effects for the evoked gamma response in the current experiment. In most of the studies investigating memory matching process, an object or sensory information had to be compared with an existing representation stored in short-term (Herrmann et al., 1999; Herrmann and Mecklinger, 2000; Debener et al., 2003) or long-term memory (Herrmann et al., 2004a). The paradigm of the present study differed in the way that contents had to be actively held in working memory and to be compared with presented probe information. Our results underpin the assumption that this process is reflected in an interaction of theta and gamma oscillations. It is conjecturable that evoked gamma rather appears to play a central role in matching representations stored in short-term (e.g., target information) or long-term memory (e.g., object properties) with sensory information. But due to inconsistent findings (Jeschke et al., 2008; Lenz et al., 2008) and due to the possibility that the stimulus paradigm does not comprise of the reasonable features to pronounce a salient effect in evoked gamma (Busch et al., 2006) this aspect still remains unclear. Conclusion The results of the present study demonstrate that to compare incoming visual input with the stored contents of working memory, neuronal networks oscillating at theta (around 6 Hz) and high gamma (around 60 Hz) frequency become phase synchronized at parieto-
E.M. Holz et al. / NeuroImage 52 (2010) 326–335
occipital recording sites in an early time window. The findings indicate that working memory contents retained in a theta network are matched with perceptual input constituted within a gamma network at posterior brain sites. This synchronization cannot be explained by common phase resetting of theta and gamma oscillations or evoked response but rather through phase modulation at theta frequency. References Andersen, D.R., Marsolek, C.J., 2005. Does a causal relation exist between the functional hemispheric asymmetries of visual processing subsystems? Brain Cogn. 59, 135–144. Berryhill, M.E., Olson, I.R., 2008. The right parietal lobe is critical for visual working memory. Neuropsychologia 46, 1767–1774. Busch, N.A., Debener, S., Kranczioch, C., Engel, A.K., Herrmann, C.S., 2004. Size matters: effects of stimulus size, duration and eccentricity on the visual gamma-band response. Clin. Neurophysiol. 115, 1810–1820. Busch, N.A., Schadow, J., Fründ, I., Herrmann, C.S., 2006. Time-frequency analysis of target detection reveals an early interface between bottom-up and top-down in the gamma-band. NeuroImage 29, 1106–1116. Canolty, R.T., Edwards, E., Dalal, S.S., Soltani, M., Nagarajan, S.S., Kirsch, H.E., Berger, M.S., Barbare, N.M., Knight, R.T., 2006. High gamma power is phase-locked to theta oscillations in human neocortex. Science 313, 1626–1628. Debener, S., Herrmann, C.S., Kranczioch, C., Gembris, D., Engel, A.K., 2003. Top-down attentional processing enhances auditory evoked gamma band activity. NeuroReport 14, 683–686. Demiralp, T., Bayraktaroglu, Z., Lenz, D., Junge, S., Busch, N.A., Maess, B., Ergen, M., Herrmann, C.S., 2007. Gamma amplitudes are coupled to theta phase in human EEG. Int. J. Psychophysiol. 64, 24–30. D'Esposito, M., Aguirre, G.K., Zarahn, E., Ballard, D., Shin, R.K., Lease, L., 1998. Functional MRI studies of spatial and non-spatial working memory. Cogn. Brain Res. 7, 1–13. Engel, A.K., Fries, P., König, P., Brecht, M., Singer, W., 1999. Temporal binding, bin-ocular rivalry, and consciousness. Conscious. Cogn. 8, 128–151. Givens, B., 1996. Stimulus-evoked resetting of the dentate theta rhythm: relation to working memory. NeuroReport 8, 159–163. Gevins, A., Smith, M.E., 2000. Neurophysiological measures of working memory and individual differences in cognitive ability and cognitive style. Cereb. Cortex 10, 829–839. Gratton, G., Coles, M.G.H., Donchin, E., 1983. A new method for off-line removal of ocular artifact. Electroencephalogr. Clin. Neurophysiol. 55, 468–484. Herrmann, C.S., Mecklinger, A., 2000. Magnetoencephalographic responses to illusory figures early evoked gamma is affected by processing of stimulus features. Int. J. Psychophysiol. 38, 265–281. Herrmann, C.S., Mecklinger, A., Pfeifer, E., 1999. Gamma responses and ERPs in a visual classification task. Clin. Neurophysiol. 110, 636–642. Herrmann, C.S., Lenz, D., Junge, S., Busch, N.A., Maess, B., 2004a. Memory-matches evoke human gamma-responses. BMC Neurosci. 5, 13. Herrmann, C.S., Munk, M.H.J., Engel, A.K., 2004b. Cognitive functions of gamma-band activity: memory match and utilization. Trends Cogn. Sci. 8, 347–355. Howard, M.W., Rizzuto, D.S., Caplan, J.B., Madsen, J.R., Lisman, J., AschenbrennerScheibe, R., Schulze-Bonhage, A., Kahana, M.J., 2003. Gamma oscillations correlate with working memory load in humans. Cereb. Cortex 13, 1369–1374. Jensen, O., Tesche, C.D., 2002. Frontal theta activity in humans increases with memory load in a working memory task. Eur. J. Neurosci. 15, 1395–1399. Jeschke, M., Lenz, D., Budinger, E., Herrmann, C.S., Ohl, F.W., 2008. Gamma oscillations in gerbil auditory cortex during a target-discrimination task reflect matches with short-term memory. Brain Res. 1220, 70–80. Jonides, J., Smith, E.E., Koeppe, R.A., Awh, E., Minoshima, S., Mintun, M.A., 1993. Spatial working memory in humans as revealed by PET. Nature 363, 623–625. Kaiser, J., Lutzenberger, W., 2003. Induced gamma-band activity and human brain function. Neuroscientist 9, 475–484. Koutstaal, W., Wagner, A.D., Rotte, M., Maril, A., Buckner, R.L., Schacter, D.L., 2001. Perceptual specificity in visual object priming: functional magnetic resonance imaging evidence for a laterality difference in fusiform cortex. Neuropsychologia 39, 184–199. Kramer, M.A., Tort, A.B., Kopell, N.J., 2008. Sharp edge artifacts and spurious coupling in EEG frequency comodulation measures. J. Neurosci. Methods 170, 352–357. Kranczioch, C., Debener, S., Herrmann, C.S., Engel, A.K., 2006. EEG gamma-band activity in rapid serial visual presentation. Exp. Brain Res. 169, 246–254. Lakatos, P., Shah, A.S., Knuth, K.H., Ulbert, I., Karmos, G., Schroeder, C.E., 2005. An oscillatory hierarchy controlling neuronal excitability and stimulus processing in the auditory cortex. J. Neurophysiol. 94, 1904–1911. Lauchaux, J.P., Rodriguez, E., Martinerie, J., Varela, F.J., 1999. Measuring phase synchrony in brain signals. Hum. Brain Mapp. 8, 194–208. Leiberg, S., Kaiser, J., Lutzenberger, W., 2003. Gamma-band activity dissociates between matching and nonmatching stimulus pairs in an auditory delayed matching-tosample task. NeuroImage 30, 1357–1364. Lenz, D., Jeschke, M., Schadow, J., Naue, N., Ohl, F.W., Herrmann, C.S., 2008. Human EEG very high frequency oscillations reflect the number of matches with a template in auditory short-term memory. Brain Res. 1220, 81–92.
335
Marsolek, C.J., Deason, R.G., 2007. Hemispheric asymmetries in visual word-form processing: progress, conflict, and evaluating theories. Brain Lang. 103, 304–307. Marsolek, C.J., Schacter, D.L., Nicholas, C.D., 1996. Form-specific visual priming for new associations in the right cerebral hemisphere. Mem. Cognit. 24, 539–556. Melloni, L., Molina, C., Pena, M., Torres, D., Singer, W., Rodriguez, E., 2007. Synchronization of neural activity across cortical areas correlates with conscious perception. J. Neurosci. 27, 2858–2865. Melloni, L., Schwiedrzik, C.M., Rodriguez, E., Singer, W., 2009. (Micro)Saccades, corollary activity and cortical oscillations. Trends Cogn. Sci. 13, 239–245. Palva, S., Palva, J.M., 2007. New vistas for α-frequency band oscillations. Trends Neurosci. 30, 150–158. Palva, J.M., Palva, S., Kaila, K., 2005. Phase synchrony among neuronal oscillations in the human cortex. J. Neurosci. 25, 3962–3972. Raghavachari, S., Kahana, M.J., Rizzuto, D.S., Caplan, J.B., Kirschen, M.P., Bourgeois, B., Madsen, J.R., Lisman, J.E., 2001. Gating of human theta oscillations by a working memory task. J. Neurosci. 21, 3175–3183. Raghavachari, S., Lisman, J.E., Tully, M., Madsen, J.R., Bromfield, E.B., Kahana, M.J., 2006. Theta oscillations in human cortex during a working-memory task: evidence for local generators. J. Neurophysiol. 95, 1630–1638. Rodriguez, E., George, N., Lachaux, J.P., Martinerie, J., Renault, B., Varela, F.J., 1999. Perception's shadow: long-distance synchronization of human brain activity. Nature 397, 430–433. Sarnthein, J., Petsche, H., Rappelsberger, P., Shaw, G.L., von Stein, A., 1998. Synchronization between prefrontal and posterior association cortex during human working memory. Proc. Natl. Acad. Sci. U. S. A. 95, 7092–7096. Sauseng, P., Klimesch, W., 2008. What does phase information of oscillatory brain activity tell us about cognitive processes? Neurosci. Biobehav. Rev. 32, 1001–1013. Sauseng, P., Klimesch, W., Doppelmayr, M., Hanslmayr, S., Schabus, M., Gruber, W.R., 2004. Theta coupling in the human electroencephalogram during a working memory task. Neurosci. Lett. 354, 123–126. Sauseng, P., Klimesch, W., Schabus, M., Doppelmayr, M., 2005. Fronto-parietal EEG coherence in theta and upper alpha reflect central executive functions of working memory. Int. J. Psychophysiol. 57, 97–103. Sauseng, P., Klimesch, W., Freunberger, R., Doppelmayr, M., 2006. Relevance of EEG alpha and theta oscillations during task switching. Exp. Brain Res. 170, 295–301. Sauseng, P., Klimesch, W., Gruber, W.R., Birbaumer, N., 2008. Cross-frequency phase synchronization: a brain mechanism of memory matching and attention. NeuroImage 40, 308–317. Sauseng, P., Klimesch, W., Heise, K., Gruber, W.R., Holz, E.M., Glennon, M., Karim, A., Gerloff, C., Birbaumer, N., Hummel, F., 2009. Human brain oscillations dissociate memory storage from information selection processes and predict short-term memory capacity. Curr. Biol. 19, 1846–1852. Sauseng, P., Griesmayr, B., Freunberger, R., Klimesch, W., 2010. Control Mechanisms in Working Memory: A Possible Function of EEG Theta Oscillations. Neurosci. Biobehav. Rev. 34, 1015–1022. Schack, B., Klimesch, W., 2002. Frequency characteristics of evoked and oscillatory electroencephalic activity in a human memory scanning task. Neurosci. Lett. 331, 107–110. Schack, B., Weiss, S., 2005. Quantification of phase synchronization phenomena and their importance for verbal memory processes. Biol. Cybern. 92, 275–287. Schack, B., Vath, N., Petsche, H., Geissler, H.G., Möller, E., 2002. Phase-coupling of thetagamma EEG rhythms during short-term memory processing. International Journal of Psychophysiology 44, 143–163. Schack, B., Klimesch, W., Sauseng, P., 2005. Phase synchronization between theta and upper alpha oscillations in a working memory task. Int. J. Psychphysiol. 57, 105–114. Senkowski, D., Schneider, T.R., Tandler, F., Engel, A.K., 2009. Gamma-band activity reflects multisensory matching in working memory. Exp. Brain Res. 198, 363–372. Singer, W., 1999. Neuronal synchrony: a versatile code for the definition of relations? Neuron 24, 49–65. Tallon-Baudry, C., 2003. Oscillatory synchrony and human visual cognition. J. Physiol. 97, 355–363. Tallon-Baudry, C., Bertrand, O., 1999. Oscillatory gamma activity in humans and its role in object representation. Trends Cogn. Sci. 3, 151–162. Tallon-Baudry, C., Bertrand, O., Peronnet, F., Pernier, J., 1998. Induced gamma-band activity during the delay of a visual short-term memory task in humans. J. Neurosci. 18, 4244–4254. Tesche, C.D., Karhu, J., 2000. Theta oscillations index human hippocampal activation during a working memory task. Proc. Natl. Acad. Sci. U. S. A. 97, 919–924. van der Ham, I.J.M., Raemaekers, M., van Wezel, R.J.A., Oleksiak, A., Postma, A., 2009. Categorical and coordinate spatial relations in working memory: and fMRI study. Brain Res. 1297, 70–79. Volberg, G., Hübner, R., 2004. On the role of response conflicts and stimulus position for hemispheric differences in global/local processing: an ERP study. Neuropsychologia 42, 1805–1813. Volberg, G., Kliegl, K., Hanslmayr, S., Greenlee, M.W., 2009. EEG alpha oscillations in the preparation for global and local processing predict behavioural performance. Hum. Brain Mapp. 30, 2173–2183. Weissman, D.H., Woldorff, M.G., 2005. Hemispheric asymmetries for different components of global/local attention occur in distinct temporo-parietal loci. Cereb. Cortex 15, 870–876. Williams, J.M., Givens, B., 2003. Stimulation-induced reset of hippocampal theta in the freely performing rat. Hippocampus 13, 109–116.
