Exp Brain Res DOI 10.1007/s00221-007-1052-7
R ES EA R C H A R TI CLE
Classical conditioned learning using transcranial magnetic stimulation B. Luber · P. Balsam · T. Nguyen · M. Gross · S. H. Lisanby
Received: 1 December 2006 / Accepted: 25 June 2007 © Springer-Verlag 2007
Abstract This study examined whether brain responses to transcranial magnetic stimulation (TMS) would be amenable to classical conditioning. Motor cortex in human participants was stimulated with TMS pulses, which elicited a peripheral motor response in the form of a motor evoked potential (MEP). The TMS pulses were paired with audiovisual cues that served as conditioned stimuli. Over the course of training, MEPs following the conditioned stimuli decreased in amplitude. Two experiments demonstrated that the attenuated response only occurred when the TMS was preceded by the conditioned stimulus. Unsignaled TMS and TMS preceded by a cue that was not previously paired did not attenuate the response. The experiments demonstrate that the modulation of the motor response depended on the prior pairings of the conditioned stimuli and TMS and that the eVects were stimulus speciWc. Thus we demonstrate here, for the Wrst time, that TMS can serve as the unconditioned stimulus in Pavlovian conditioning. Keywords Classical conditioning · TMS · Motor cortex · Conditioned compensatory response
B. Luber (&) · P. Balsam · T. Nguyen · M. Gross · S. H. Lisanby Brain Stimulation and Therapeutic Modulation Division, New York State Psychiatric Institute and Department of Psychiatry, Columbia University College of Physicians and Surgeons, 1051 Riverside Drive, Unit 21, New York, NY, USA e-mail:
[email protected] P. Balsam Department of Psychology, Barnard College, New York, NY, USA
Introduction Transcranial magnetic stimulation (TMS), applied using various paradigms, has been found to acutely enhance performance in a number of tasks, including choice reaction time (Evers et al. 2001), picture naming (Topper et al. 1998), mental rotation of 3D objects (Klimesch et al. 2003), backward masking (Grosbras and Paus 2003), Stroop (Hayward et al. 2004), recognition memory (Kohler et al. 2004), analogical reasoning (Boroojerdi et al. 2001), and working memory (Luber et al. 2007). However, these beneWcial eVects of TMS have not persisted long after the stimulation period ends. For example, improved performance in a picture naming task with 20 Hz repetitive TMS (rTMS) was shown to last only for a period of 2 min beyond the end of the pulse train (Sparing et al. 2001). Thus, while TMS has been shown to change cortical processing over time periods that range from milliseconds up to half an hour, there has been little evidence of long-lasting TMS-induced changes in motor or cognitive skills resulting from individual TMS applications. One strategy that has been pursued to prolong the behavioral eVects of TMS has been to repeatedly apply it on a daily basis for up to 6 weeks or more, as has been done with various degrees of success in the treatment of depression and schizophrenia (Lisanby et al. 2000). While promising, this approach is also quite time consuming and workforce intensive. Any means of prolonging beneWcial eVects of TMS, or delivering them more eYciently, could be of value in sustaining improvements in situations where it may not be practical to repeatedly apply TMS on a frequent basis. Were repeated TMS to be susceptible to classical conditioning, knowledge of conditioned eVects could be exploited in the design of optimal TMS paradigms to maximize desired outcomes. For example, if brain responses to
123
Exp Brain Res
TMS could be triggered by other stimuli (such as an auditory or visual stimulus) via classical conditioning, this could reduce the need to administer actual TMS. Likewise, if the application of TMS becomes inadvertently paired with stimuli that elicit a conditioned compensatory response that opposes the direct eVects of TMS (producing a net attenuation of the behavioral response to TMS), then avoiding the development of opponent responses could amplify and potentially prolong the eVects of TMS. In the usual TMS experiment, subjects are exposed to repeated applications of TMS over many trials, often at regular intervals or frequencies. As a result, participants may come to anticipate the TMS pulses or trains. Cues that regularly precede the TMS such as the movement of technicians, presentation of a neuropsychological task, or sounds of equipment might become associated with the TMS. The cues themselves may come to evoke conditioned responses (CRs) that might elicit their own changes in brain activity or might modulate the eVects of the subsequent TMS, either of which might either be therapeutic or counter-therapeutic. No prior reports to our knowledge have examined whether brain responses to TMS would be amenable to classical conditioning. Here we directly examined whether TMS could be used as an unconditioned stimulus (US) within the framework of classical conditioning. This examination was predicated by prior work in animals using direct cortical stimulation as the US. In these past studies an experimental paradigm was developed in which electrodes were implanted in the motor cortex of animals, such that direct electrical stimulation (the US) produced muscle Xexions, usually of the leg (the unconditioned response: UR). The US was paired with another stimulus, often an audible tone (the conditioned stimulus: CS), and tests were done to see if the CS alone would produce conditioned motor Xexions. While initial attempts to establish that direct electrical stimulation of motor cortex could function as an US were not successful (e.g., Loucks 1935), later investigators were able to produce conditioned Xexions by using direct stimulation to a cortical region (e.g., visual cortex) that did not produce the UR as the CS; after conditioning with the direct stimulation US, stimulation to the CS area alone could produce the CR (Doty 1969; Doty and Giurgea 1961). Subsequently, investigators were able to demonstrate successful conditioning using auditory CSs (Wagner et al. 1967; Kandel and Benevento 1973) and direct stimulation of the motor cortex as the US. In both of those studies, tests using the CS alone began producing conditioned motor Xexions after about 100 US/CS pairings. While no further animal work using this paradigm has been reported since 1973, the accumulated research established that a conditioned response can be developed in animals using electrical stimulation of motor cortex as the unconditioned stimulus.
123
Given the evidence from animal research that the eVects of electrical cortical stimulation could be conditioned, and recent evidence that TMS can facilitate performance of a variety of cognitive and motor tasks, it would be useful to know whether these behavioral eVects of TMS could be conditioned to occur. Were this approach viable, then performance enhancements could be produced without needing to apply TMS itself after the initial training period. As a proof-of-concept study, we examined the case of the motor response elicited by single pulse TMS of the primary motor cortex. In the present experiment, stimulation of the motor cortex with TMS pulses (US) elicited a peripheral motor response (UR) in the form of a motor evoked potential (MEP). The TMS pulses were paired with audio-visual cues that served as the CS. Two possible types of CRs might occur in this context: an excitatory case, in which the CR would be similar to the UR, and an inhibitory case, in which a CR develops that opposes the UR. We did not know a priori which type of CR might arise in our study, so diVerent test trials were developed to allow observation of either type. We anticipated that an excitatory CR would be a motor response as was the case in the earlier animal studies in which direct stimulation of the motor cortex served as the US. On the other hand, there are quite a few examples of CRs that are antagonistic to the UR. A well studied example is the conditioned response to opiate injection. Siegel (cf. 2005) has demonstrated numerous times that the conditioned response to environmental cues associated with drug administration is a withdrawal response. This conditioned compensatory response (CCR) blunts the eVects of the drug and functions as a basis for drug tolerance and for drug craving when no drug is available. CCR’s have been described for drugs such as morphine, heroin, alcohol, barbiturates and nicotine (for a review see Siegel and Ramos 2002) as well as in heart rate conditioning (Marchand and Kamper 2000). On this basis it is possible that we would not directly observe an excitatory motor CR on CS alone test trials. However, if there is a CCR then there should be an attenuation of the TMS elicited response when it is preceded by the CS. Consequently, in addition to looking for a CR that resembles the UR we also examined the ability of the CS to modulate the UR.
Methods Subjects Eighteen healthy subjects (eight female) with a mean age of 29.1 § 7.6 (SD) years were recruited and signed written consent for the study. The study was approved by the Columbia University/New York State Psychiatric Institute
Exp Brain Res
IRB. Subjects were required to have normal or correctedto-normal vision and to be right-handed. Potential subjects were excluded if they had a history of current or past Axis I psychiatric disorder including substance abuse/dependence as determined by the Structured Clinical Interview for DSM-IV Axis I Disorders (SCID-NP) or a history of neurological disease. All subjects were screened with physical and neurological examinations, urine drug screens, and pregnancy tests for women of childbearing capacity. Experimental stimuli
for TMS. Coil placement was guided using Brainsight, a computerized frameless stereotaxic system (Rogue Research, Montreal, Canada). This system uses an infrared camera to monitor the positions of reXective markers attached to the coil and the subject, so that relative positions of the coil to the head could be tracked, allowing precise positioning of the coil at the optimal site. It also allowed precise monitoring of deviations from the target site caused by subject movement: deviations of even a few millimeters during experimental sessions could be observed and corrected online. EMG data acquisition and analysis
In any given block of trials, two types of stimuli were presented, only one of which was the conditioned stimulus (CS+) paired with a TMS pulse. The TMS pulse, which lasted about 300 ms, was applied at the end of the CS+, to the eVect that the CS and US were co-terminating. The second stimulus, presented alone (CS¡), acted as a control. Stimuli were audio-visual: a 2-s tone (two diVerent frequencies for conditioned and control stimuli) occurring simultaneously with a brightly colored visual pattern (two diVerent colored patterns, for conditioned and control stimuli). Compound cues were used because multimodal stimulation has been shown to facilitate conditioned learning (Bahrick et al. 2004; Rescorla and Wagner 1972). Four auditory cue pairs were used: 1 and 3 kHz, 2 and 0.5 kHz, 3.5 and 7 kHz, and 1.5 and 2.5 kHz, digitally generated by computer and presented with a speaker facing the subject. Eight diVerent color visual stimuli were used, consisting of a colored geometric shape centered in a contrasting-colored rectangular background. Visual stimuli measured 3.1° £ 3.9° of visual angle. Any single subject experienced a single pair of auditory cues and a single pair of visual cues. One auditory cue and one visual cue were presented as the CS+ and the remaining audio-visual compound served as the CS¡. Once chosen for each subject, the particular stimulus pairs were consistently used throughout the learning and testing periods. Pairs of visual stimuli and auditory stimuli were counterbalanced across subjects.
Electromyographic (EMG) data was recorded continuously throughout each block of trials using electrodes attached to the right hand in a standard belly-tendon montage for readings of FDI contraction. A ground electrode was attached to the back of the same hand. The EMG signal was ampliWed through an external ampliWer (S.A. Instruments, La Jolla, CA, USA), band-passed between 30 and 1,000 Hz, and digitized at 4,000 samples/s using Snapmaster acquisition software on an IBM-compatible PC. We quantiWed the EMG response by measuring the largest and smallest EMG amplitudes in the 50 ms period after the oVset of each audio-visual stimulus. In trials with TMS pulses, this corresponded to the peak-to-peak amplitude of the MEP resulting from the pulse. In non-TMS trials, this was expected to document any changes in EMG resulting from excitatory CRs. For clarity, this measure on both TMS and non-TMS trials will be referred to as the MEP. In addition, we quantiWed EMG activity by measuring power (V2) in the 30–250 Hz frequency band using a discrete Fourier transform. This was done in 0.125 s epochs, beginning 2 s before the onset to 2 s after the oVset of the audio-visual stimulus. In both Experiments I and II, no changes in EMG power reXecting the eVects of training could be observed in CS¡ or CS+ alone trials, and will not be further reported here.
TMS application
Experiment I procedure
Transcranial magnetic stimulation was applied using a Figure 8 coil (9 cm diameter) powered by a Magstim 200 stimulator (Magstim Co., Whitland, South West Wales, UK). TMS stimulus intensity was set at 150% of resting motor threshold of the left hemisphere, which was deWned as the lowest intensity needed to evoke motor potentials of at least 50 V recorded from the Wrst dorsal interosseus muscle (FDI) in at least 5/10 stimulations (Rossini et al. 1994). For each subject, the scalp region that produced the largest MEP amplitude was marked as the optimal FDI stimulation site, and used in all experimental sessions as the target site
Eleven subjects participated in the Wrst experiment. Five received eight conditioning sessions on eight diVerent days, while the remaining six subjects had two sessions on two successive days. Subjects were seated comfortably in front of a computer monitor, with their right hands in a relaxed position. The TMS coil was placed over the optimal site for FDI stimulation, tangentially to the skull, handle pointing backwards, and rotated 45° to the midline of the head. The optimal site was recorded using the Brainsight system, and a mechanical frame held the coil in position on the scalp. The coil was kept within 3 mm of the optimal site through
123
Exp Brain Res
continuous monitoring with Brainsight. Before conditioning trials began, MEPs from Wve TMS pulses was obtained as a baseline measure. The conditioning session consisted of 90 trials, which included 40 CS+ cues paired with TMS, and 45 CS¡ stimuli with no TMS administration. The remaining Wve trials presented the CS+ stimuli without TMS administration, to test for a conditioned EMG response. The order for these three trial types was pseudorandom, with no more than three CS+ or CS¡ trials in a row, and with at least ten trials between test stimuli. The intertrial interval (ITI) was variable with a mean of 20 § 10 s, and subjects received the 90 trials without interruption. There were 120 trials in the last session: 90 conditioning trials as just described, followed without a break by 30 more trials, in order to test for excitatory and inhibitory conditioned eVects. There were three types of test trials, presented in random order: CS+ without a TMS pulse (to test for an excitatory CR), CS¡, and TMS pulses without a prior audio-visual stimulus (to test for habituation to the unsignaled TMS during training). These trials were run with the same 20 § 10 s ITI. Subjects were instructed to pay attention to the audio-visual stimuli when they occurred. Between trials there was no speciWc instruction about what they should do.
paired with TMS, 45 CS¡ stimuli with no TMS administration, and Wve trials of CS+ stimuli without TMS, in random order. Then, instead of CS+ without TMS, CS¡, and TMSalone pulses as test trials as in Experiment I, the test trials were simply a set of ten trials that formed the end of the conditioning block of the second conditioning session. There were Wve CS¡ stimuli paired with TMS pulses (the test stimuli for an inhibitory conditioning eVect) and Wve CS+ stimuli presented without an accompanying TMS pulse (the test for an excitatory conditioning eVect). These ten trials were presented in random order. In addition to changing the test for an inhibitory eVect from TMS-alone to CS¡ signaled TMS, using two test stimuli at the end of the trial block instead of three (as was the case in Experiment I) allowed the average inter-TMS pulse interval of the test trials to remain the same as it had been during the preceding conditioning trials. This was done to rule out the possible inXuence of longer average inter-TMS pulse intervals on MEPs.
Experiment II procedure
Peak-to-peak MEP amplitudes for the baseline and each set of ten consecutive CS+ trials in the Wnal conditioning session were averaged into Wve bins. We tested whether the subjects trained for eight sessions diVered from those trained for two with a two-way analysis of variance with amount of training (two vs eight sessions) and bin (one through Wve) as factors. The analysis indicated that there was no signiWcant eVect of the number of conditioning sessions (F1,9 < 1, ns), nor was there an interaction between the amount of training and bin (F4,36 < 1, ns). No evidence of an excitatory CR could be observed in the MEP amplitudes of CS+ alone trials. However, there were indications of a CCR in the decrease in average MEP response in CS+ trials over training. As a Wrst step, in order to look at acquisition of the CR minimally aVected by any within-session habituation we examined the Wrst bin of conditioning trials on successive days. Since all subjects in the experiment had a minimum of two days of conditioning we compared baseline to the Wrst bin of trials on days 1 and 2. We had complete data for all three periods in eight subjects. As Fig. 1 shows, baseline and the Wrst bin of conditioning trials did not diVer in the strength of the MEP [t(7) = 0.71, ns]. However, after the 40 pairings of session 1, the MEP response during the Wrst bin of trials in session 2 was signiWcantly below Baseline [t(7) = 2.79, P < 0.05). This provides evidence that the decrease in signaled MEP was a result of the training experience.
As Experiment I was performed, it became clear that there was a decrease in average MEP response following the CS+ as trial number increased (results presented below). With presentation of un-signaled TMS pulses at the end of the training, there was a signiWcant increase of the MEP. To test if this trend was actually due to conditioning to a speciWc stimulus, Experiment II was designed. Following pairings of a CS+ cue with TMS subjects were also tested with CS¡ cues which were now paired with TMS. This was done to see if the attenuation of the TMS elicited response during training was due to the prior pairings of the CS+ with TMS or if any cue prior to TMS would reduce the strength of the UR. If conditioning did occur, then the UR attenuation would be stimulus speciWc: only the CS+ would be eVective. In this case, the presentation of the CS¡ followed by TMS should produce an MEP similar to the one elicited by TMS alone. Four of the subjects from Experiment I, along with seven additional subjects, participated in two conditioning sessions in the second experiment. The procedures used were essentially the same as Experiment I, except for the test trials at the end of the second session. Subjects were seated comfortably in front of a computer monitor. The TMS coil was placed over the optimal site for stimulating the FDI muscle. Baseline trials were run at the beginning of the Wrst session, followed by 90 conditioning trials: 40 CS+ cues
123
Results Experiment I
Exp Brain Res 3500
a
3000
3000
2500
2500
2000
MEP (uV)
MEP (uV)
3500
1500
2000 1500
1000 1000
500 500
0 Baseline
Day 1
Day 2
Fig. 1 Mean peak-to-peak amplitudes (and SE) of MEP recordings from the FDI muscle of eight subjects as collected over the Wrst ten trials of CS+ cue signaled TMS in the Wrst and second conditioning sessions, as compared to the mean baseline peak-to-peak amplitudes
0 Baseline
1
2
3
4
Bin
b
3000
Average MEP to TMS pulses decreased from baseline levels through the end of the conditioning session (F4,36 = 2.74, P < 0.05). This can be seen in Fig. 2 for an individual subject and in Fig. 3a for the whole group. The attenuated MEP to TMS is not solely due to habituation as responses to TMS-alone test pulses at the end of the last session showed a recovery in amplitude (Figs. 2, 3b). A similar ANOVA indicated that there was no eVect of the amount of prior conditioning but that amplitudes of the unsignaled TMS are signiWcantly greater than those in the Wnal bin of CS+ responses (F1,9 = 9.40, P < 0.05; Fig. 3b). Lastly, the strength of responding to TMS alone in the Wnal test did not diVer from the initial baseline level of responding (all F⬘s < 1, ns).
MEP (uV)
2500 2000 1500 1000 500 0 CS+ Signaled Unsignaled TMS CS+ (No TMS) TMS
CS- (No TMS)
Fig. 3 (Experiment I) a Mean peak-to-peak amplitudes (and SE) of MEP recordings from the FDI muscle of 11 subjects as collected over four 10-trial bins of CS+ cue signaled TMS. b Mean peak-to-peak amplitudes (and SE) of MEPs of the last 10 trials (4th bin) of CS+ signaled TMS as compared to the amplitudes of MEPs of unsignaled TMS, CS+ test trials without TMS, and CS- trials (no TMS)
Experiment II 1000 800 600
Baseline 4th Bin Test
Mean MEP (uV)
400 200 0 -200 -400 -600 -800 -1000
10 ms
-1200
Fig. 2 Mean MEPs for the baseline trials, last ten trials (4th bin) of CS+ signaled TMS, and the TMS alone test trials in Experiment I for a single subject. The arrow indicates the point the TMS pulse was applied
We tested to see if participation in Experiment I had an impact on the performance in the second experiment with a two-way ANOVA with experimental history (experienced or naïve subject) and bin as factors. There was no eVect of prior history nor did this factor interact with bin (F⬘s < 1, ns). The decrease in MEP amplitude over conditioning found in Experiment I was replicated. (Fig. 4a). A repeated measures ANOVA of MEP across bins showed a signiWcant eVect of bin (F4,36 = 3.99, P < 0.05). Additionally, the MEP to TMS following the presentation of a CS¡ cue was signiWcantly greater than the response to TMS following the CS+ cue during the Wnal bin of conditioning (F1,9 = 8.52, P < 0.05; Fig. 4b). The mean magnitude of MEP elicited by TMS following the presentation of a CS¡ cue was smaller than the mean baseline level by 538 V. However, this diVerence was not signiWcant (F1,9 = 2.23, P < 0.20).
123
Exp Brain Res
a
3500 3000
MEP (uV)
2500 2000 1500 1000 500 0 Baseline
1
2
3
4
Bin
b
3000
MEP (uV)
2500 2000 1500 1000 500 0 CS+ Signaled TMS
CS- Signaled TMS
CS+ (No TMS)
Fig. 4 (Experiment II) a Mean peak-to-peak amplitudes (and SE) of MEP recordings from the FDI muscle of 11 subjects as collected over four 10-trial bins of CS+ cue signaled TMS. b Mean peak-to-peak amplitudes (and SE) of MEPs of the last 10 trials (4th bin) of CS+ signaled TMS as compared to the amplitudes of MEPs of CS-signaled TMS and CS+ test trials without TMS
Discussion Pairing an audio-visual stimulus with TMS resulted in a CR. In Experiment I, tests for both an excitatory CR that was similar to the UR and one that was antagonistic to it suggested the presence of the latter: a conditioned compensatory response (CCR) in which the muscle response was attenuated by the CS. Over the course of training the response to TMS following the CS+ decreased. This attenuation of the TMS elicited response might have been simply due to habituation. However, at the end of training when subjects were given test trials with unsignaled TMS the elicited response was just as strong as it was at the start of conditioning. Thus the CS+ inhibited the strength of the UR, and the attenuation of the TMS elicited response could not be explained by habituation. In Experiment II, further evidence that the attenuated response to TMS was a CR was obtained by showing that the attenuation of the UR was speciWc to the CS+ previously paired with the US. Presentation of the CS¡ cue which subjects had experienced just as frequently as the CS+ but which had never been paired
123
with TMS failed to attenuate the UR when it was presented prior to TMS. Taken together the experiments demonstrate that the modulation of the UR depended on the prior pairings of the CS and US and that the eVects were stimulus speciWc. Thus we demonstrate here, for the Wrst time, that TMS can serve as the US in Pavlovian conditioning. In the past, conditioning experiments using stimulation of the motor cortex of animals as the US have succeeded in demonstrating conditioning where the CR resembled the UR (e.g., Wagner et al. 1967; Kandel and Benevento 1973), while in the present study, TMS of motor cortex in humans did not. One possible explanation lies in the diVerences between direct electrical stimulation of motor cortex and stimulation from current induction via changing magnetic Welds from outside the head. The direct electrical stimulation used in the animal studies was of greater intensity and focality than the TMS of the present study. The changing magnetic Weld induces eVective transmembrane currents in selected portions of stimulated neurons, primarily bends in axons and axon terminations, while direct electric currents which are radially oriented (as opposed to the tangential orientation of TMS induced currents) do so throughout neurons (Epstein et al. 1990; Kobayashi and Pascual-Leone 2003). As a result, the TMS would not have stimulated corticospinal motor neurons directly (as direct electric stimulation can), but only indirectly via excitatory interneurons (Di Lazzaro et al. 2004). In addition, compared with Wxed implanted electrodes, TMS application was somewhat less spatially precise, as subtle subject movements could shift the TMS coil slightly away from the optimal stimulation location (this may have been a negligible eVect, as the online use of the frameless stereotactic instrument and the coregistration program mitigated the impact of subject movement and subject/coil variation was kept within 3 mm throughout conditioning sessions). Any of these reasons may have resulted in a magnetic stimulation that was not as eYcacious a US as direct electrical stimulation from implanted electrodes has been shown to be. It is possible that higher stimulation intensities, or diVerent stimulation protocols, might have resulted in an excitatory CR. In general, diVerent parameters of stimulation and diVerent target areas might also have produced diVerent CRs. The conditioned response that was obtained in the present study was antagonistic to the UR: MEP amplitude decreased as trial number increased as a result of stimulation paired with the CS+ stimuli. Decrements in MEPs produced by rTMS to motor cortex have been observed previously (Chen et al. 1997; Siebner et al. 1999; Muellbacher et al. 2000; Gilio et al. 2003; Plewnia et al. 2003; Heide et al. 2006). These decrements have been frequencyspeciWc: 1 Hz rTMS applied for 15–30 min to motor cortex typically reduced MEP amplitude while pulses repeated at a 0.1 Hz rate did not result in MEP amplitude decreases
Exp Brain Res
(Chen et al. 1997). These results appear to indicate that motor cortex recovers from the eVects of a TMS pulse between 1 and 10 s, and have led to the suggestion that 5– 10 s be allowed to elapse between pulses in procedures such as motor threshold determination to avoid carry-over eVects of one pulse to the next. In the present study, MEP amplitude decreases were found consistently across subjects with single pulse TMS that had a rate of repetition across trials that, in the mean, roughly corresponded to a frequency of 0.025 Hz (given an average 40 s between pulses in both experiments: actual pulse separation in this condition ranged from 12 s minimum to 122 s maximum). The attenuated TMS response only occurred for TMS pulses preceded by the CS+. TMS pulses alone (Experiment I) or TMS pulses preceded by the CS¡ (Experiment II) with the same average 0.025 Hz frequency across trials did not demonstrate any MEP decrement, consistent with the Wnding of Chen et al. (1997). This observation lends further support to the argument that the attenuation of responding observed in the current experiment is not likely to be the result of habituation. On the other hand, as can be seen by comparing Figures 3a with 3b and 4a with 4b, the MEP amplitudes to test TMS pulses were slightly less than baseline amplitudes in both experiments. While the diVerence was not signiWcant in either case, there may be a possibility that an incomplete return to baseline amplitudes in response to test TMS pulses indicated an additional eVect of habituation. Such an eVect at the low mean rate of stimulation used here has not been previously observed. Overall however, our results suggest that TMS-induced attenuation of MEPs can be obtained with single pulse TMS with an average repetition rate that is well below 1 Hz through the exploitation of classical conditioning. By extension, it is possible that the strength of attenuation with 1 Hz rTMS might be intensiWed or prolonged through conditioning. It should be noted that decrements in MEPs have been produced by single pulse TMS to motor cortex when the pulse has been preceded by an auditory stimulus (Furubayashi et al. 2000; Fisher et al. 2004). Such modulation of motor cortical excitability with unexpected auditory stimuli is linked to the auditory startle reXex, arising from interactions of the motor cortex and brainstem reticular structures. While there were some similarities in the experimental paradigm used to examine those interactions and that of the present study, there are two diVerences that argue against attribution of the eVects seen here to startle-induced suppression of motor cortex activity. First, the suppression eVects found in Furubayashi et al. and Fisher et al. operate on a diVerent time scale. Attenuation of TMS-induced MEPs in both of those studies occurred when the time between the onset of the auditory stimulus and the TMS pulse was between 30 and 50 ms, but not when it was increased to 70 and 80 ms. In the present study, the time
diVerence between the CS onset and the TMS pulse was 2 s, well beyond the latencies where startle induced suppression has been observed. Second, as demonstrated in the second experiment, the attenuation of MEPs was stimulus speciWc, occurring with CS+ but not CS¡ pairings with TMS pulses. This would not be the case if the suppression was due to startle eVects of unexpected auditory stimuli, as the onset of CS¡ stimuli were as unpredictable as CS+. Instead, the stimulus speciWcity of the attenuation suggests a learned phenomenon. Beyond the speciWc phenomenon of auditory startle, perhaps any change in trial content at the end of a block of trials might cause general disinhibition in the MEPs. As this study was designed to test for both excitatory and inhibitory learned eVects, CS+ unpaired with TMS and either TMS alone (Experiment I) or CS¡ signaled TMS (Experiment II) in random order were used. To test for the attenuation eVect alone, a randomly-ordered test sequence of CS¡ signaled TMS responses, unsignaled TMS responses, and CS+ signaled TMS responses might be preferable in future studies. Nonetheless, any sequence of test trials diVerent than what a subject has recently experienced is open to a dishabituation critique, and it is not clear that one could design an experiment that was wholly free from this problem. It has been suggested that the lasting MEP decrements caused by 1 Hz rTMS are due to changes in the excitability of excitatory interneurons locally in motor cortex (Heide et al. 2006). This was unlikely to be the mechanism behind the changes seen in the present study, since such a mechanism would have resulted in MEP decrements in the test pulses as well. Here, the decrement was caused by learning that most likely occurred outside of the motor cortex, or at least involved an interaction of motor cortex with other brain regions. With the former possibility, one possible mechanism could be that conditioning of Wnger movement, like conditioning of the eye blink reXex, might take place in the cerebellum (Fanselow and Poulos 2005; Thompson 2005). After learning takes place, the audio-visual CS input to the cerebellum could result in output that inhibited descending motor signals from the cerebral cortex caused by TMS. With the latter possibility, it should be noted that motor cortex appears to be involved at a relatively high hierarchical level in sensory-motor learning and adaptation, as demonstrated in animal work (e.g., Wise et al. 1998; Paz et al. 2003) and in humans with TMS (Muellbacher et al. 2001). The CCR in the present study occurred in the presence of the US (the TMS pulse). While it is true that many CSs evoke measurable CRs in the absence of the US, some CRs are only evident in how they modulate other responses. In fact, measures of response modulation are often employed because they do provide sensitive measures of learned associations, especially in animal research. For example, in fear
123
Exp Brain Res
conditioning in animals, measuring disruption of ongoing behavior such as bar pressing provides a very sensitive assay of the conditioned fear and anxiety. The same applies to research on classical conditioning in humans. In many studies, the conditioning eVect of a CS on a CR is demonstrated when the CS and US were still paired. For example, in a study on how alcohol-predictive cues enhance tolerance to alcohol (McKusker and Brown 1990), one group of subjects was given alcohol in a familiar context (beer in a simulated bar: the beer-bar group), and another group was administered the same dose of alcohol in an unusual form and context (alcohol mixed in carbonated water and consumed in an oYce setting: the alcohol-oYce group). Subjects in the beer-bar group were less impaired on cognitive and motor tasks than were the subjects in the alcohol-oYce group. Similarly, heroin addicts are less likely to experience an overdose if they take the drug in the usual environment than if they take it in an environment not previously paired with the drug (Gutiérrez-Cebollada et al. 1994). In both of these cases with human subjects, the demonstration of a CCR was analogous to our own. The sort of compensatory conditioning found in the present study has been studied extensively as a model for drug tolerance, (Siegel 1975; Siegel and Ramos 2002). Siegel proposed that the CCR caused the development of stimuli-speciWc tolerance by counteracting the drug aVect. TMS is being explored as a treatment for psychiatric disorders including depression (e.g., Lisanby et al. 2000). The ready formation of CCR in most subjects in the present study suggests that a learned “tolerance” to TMS treatments may limit its potential eVectiveness. The environmental cues associated with TMS treatment might need to be controlled to minimize CCR eVects. The current study is the Wrst demonstration of conditioning responses using TMS. The results support the possibility of conditioning TMS eVects. Should future studies establish the eYcacy of TMS as a therapeutic approach, the idea that TMS responses can be conditioned will be important. If anticipation of TMS reduces the unconditioned response then optimum treatment eYcacy might be compromised by these CRs. Alternatively, if excitatory conditioned responses can be evoked, they might augment treatment eVects or even allow TMS eVects to be sustained outside the hospital setting at a much reduced cost. More generally, our results caution us to remember that anytime a treatment is administered repetitively in a stereotyped fashion there is the opportunity for conditioning. It is possible that some therapeutic eVects represent a blend of the treatment and the response the body makes in anticipation of that treatment. If the anticipatory CR is synergistic, treatments may be enhanced, but if the response is a CCR, treatment eVects may be attenuated by the Pavlovian CR.
123
Acknowledgments This research was supported by a grant from the Defense Advanced Research Projects Agency (DARPA) and by grant MH068073 to PB. Approved for public release, distribution unlimited. Dr Lisanby has received support from Magstim Company, Neuronetics and Cyberonics.
References Bahrick LE, Lickliter R, Flom R (2004) Intersensory redundancy guides the development of selective attention, perception, and cognition in infancy. Curr Dir Psychol Sci 13:99–102 Boroojerdi B, Phipps M, Kopylev L, Wharton CM, Cohen LG, Grafman J (2001) Enhancing analogic reasoning with rTMS over the left prefrontal cortex. Neurology 56:526–528 Chen R, Classen J, GerloV C, Celnik P, Wassermann E, Hallett M, Cohen L (1997) Depression of motor cortex excitability by low-frequency transcranial magnetic stimulation, Neurology 48:1398–1403 Di Lazzaro V, Oliviero A, Pilato F, Saturno E, Dileone M, Mazzone P, Insola A, Tonali PA, Rothwell JC (2004) The physiological basis of transcranial motor cortex stimulation in conscious humans. Clin Neurophysiol 115:255–266 Doty RW (1969) Electrical stimulation of the brain in behavioral context. Annu Rev Psychol 20:289–320 Doty RW, Giurgea C (1961) Conditioned reXexes established by coupling electrical excitation of two cortical areas. In: Delafresnaye JF (ed) Brain mechanisms and learning. Blackwell ScientiWc Publications, Oxford Epstein CM, Schwartzberg DG, Davey KR, Sudderth DB (1990) Localizing the site of magnetic brain stimulation in humans. Neurology 40:666–670 Evers S, Bockermann I, Nyhuis PW (2001) The impact of transcranial magnetic stimulation on cognitive processing: an event-related potential study. NeuroReport 12:2915–2918 Fanselow MS, Poulos AM (2005) The neuroscience of mammalian associative learning. Annu Rev Psychol 56:207–234 Fisher RJ, Sharott A, Kuhn AA, Brown P (2004) EVects of combined cortical and acoustic stimuli on muscle activity. Exp Brain Res 157:1–9 Furubayashi T, Ugawa Y, Terao Y, Hanajima R, Sakai K, Machii K, Mochizuki H, Shiio Y, Uesugi H, Enomoto H, Kanazawa I (2000) The human hand motor area is transiently suppressed by an unexpected auditory stimulus. Clin Neurophysiol 111:178–183 Gilio F, Rizzo V, Siebner HR, Rothwell JC (2003) EVects on the right motor hand area excitability produced by low-frequency rTMS over human contralateral homologous cortex. J Physiol 509:607– 618 Grosbras M-H, Paus T (2003) Transcranial magnetic stimulation of the human frontal eye Weld facilitates visual awareness. Eur J Neurosci 18:3121–3126 Gutiérrez-Cebollada J, de la Torre R, Ortuno J, Garces JM, Cami J (1994) Psychotropic drug consumption and other factors associated with heroin overdose. Drug Alcohol Depend 35:169– 174 Hayward G, Goodwin GM, Harmer CJ (2004) The role of the anterior cingulate cortex in the counting Stroop task. Exp Brain Res 154:355–358 Heide G, Witte OW, Ziemann U (2006) Physiology of modulation of motor cortex excitability by low frequency suprathreshold repetitive transcranial magnetic stimulation. Exp Brain Res 171:26–34 Kandel GL, Benevento LA (1973) Classically conditioned limb reXexes reinforced by motor cortex stimulation. Physiol Behav 11(4):481–488
Exp Brain Res Klimesch W, Sauseng P, GerloV C (2003) Enhancing cognitive performance with repetitive transcranial magnetic stimulation at human individual alpha frequency. Eur J Neurosci 17:1129–1133 Kobayashi M, Pascual-Leone A (2003) Transcranial magnetic stimulation in neurology. Lancet Neurology 2:145–156 Kohler S, Paus T, Buckner RL, Milner B (2004) EVects of left inferior prefrontal stimulation on episodic memory formation: a two-stage fMRI-rTMS study. J Cogn Neurosci 16:178–188 Lisanby SH, Luber B, Sackeim HA (2000) Transcranial magnetic stimulation: applications in basic neuroscience and neuropsychopharmacology. Int J Neuropsychopharmacol 2:259–273 Loucks RB (1935) The experimental delimitation of neural structures essential for learning. The attempt to condition striped muscle responses with faradization of the sigmoid gyri. J Psychol 1:5–44 Luber B, Kinnunen LH, Rakitin BC, Ellsasser R, Stern Y, Lisanby SH (2007) Facilitation of performance in a working memory task with rTMS stimulation of the precuneus: frequency and timedependent eVects. Brain Res 1128:120–129 Marchand AR, Kamper E (2000) Time course of cardiac conditioned responses in restrained rats as a function of the trace CS-US interval. J Exp Psychol Anim Behav Process 26:385–398 McKusker CG, Brown K (1990) Alcohol-predictive cues enhance tolerance to and precipitate “craving” for alcohol in social drinkers. J Stud Alcohol 51:494–499 Muellbacher W, Ziemann U, Boroojerdi B, Cohen L, Hallett M (2001) Role of the human cortex in rapid motor learning. Exp Brain Res 136:431–438 Muellbacher W, Ziemann U, Boroojerdi B, Hallett M (2000) EVects of low-frequency transcranial magnetic stimulation on motor excitability and basic motor behavior. Clin Neurophysiol 111:1002– 1007 Paz R, Boraud T, Natan C, Bergman H, Vaadia E (2003) Preparatory activity in motor cortex reXects learning of local visuomotor skills. Nature Neurosci 6:882–890 Plewnia C, Lotze M, GerloV C (2003) Disinhibition of the contralateral motor cortex by low frequency rTMS. Neuroreport 14:609–612 Rescorla RA, Wagner AR (1972) A theory of Pavlovian conditioning: variations in the eVectiveness of reinforcement and nonreinforce-
ment. In: Black A, Prokasy WF (eds) Classical conditioning II: current research and theory. Appleton-Century-Crofts, New York, pp 64–99 Rossini PM, Barker AT, Berardelli A, Caramia MD, Caruso G, Cracco RQ, Dimitrijevic MR, Hallett M, Katayama Y, Lucking CH, Maertens de Noordhout AL, Marsden CD, Murray NMF, Rothwell JC, Swash M, Tomberg C (1994) Non-invasive electrical and magnetic stimulation of the brain, spinal cord and roots: basic principles and procedures for routine clinical application. Report of an IFCN committee. Electroencephalogr Clin Neurophysiol 91:79–92 Siebner HR, Auer C, Conrad B (1999) Abnormal increase in the corticomotor output to the aVected hand during repetitive transcranial magnetic stimulation of the primary motor cortex in patients with writer’s cramp. Neurosci Lett 262:133–136 Siegel S (1975) Evidence from rats that morphine tolerance is a learned response. J Comp Physiol Psychol 89:498–506 Siegel S (2005) Two views of the addiction elephant: comment on McSweeney, Murphy, and Kowal (2005). Exp Clin Psychopharmacol 13:190–193 Siegel S, Ramos B (2002) Applying laboratory research: drug anticipation and the treatment of drug addiction. Exp Clin Psychopharmacol 10:162–183 Sparing R, Mottaghy FM, Hungs M, Brugmann M, Foltys H, Huber W, Topper R (2001) Repetitive transcranial magnetic stimulation eVects on language function depend on the stimulation parameters. J Clin Neurophysiol 18:326–330 Thompson RF (2005) In search of memory traces. Annu Rev Psychol 56:1–24 Topper R, Mottaghy FM, Brugmann M, Noth J, Huber W (1998) Facilitation of picture naming by focal transcranial magnetic stimulation of Wernicke’s area. Exp Brain Res 121:371–378 Wagner AR, Thomas E, Norton T (1967) Conditioning with electrical stimulation of motor cortex: evidence of a possible source of motivation. J Comp Physiol Psychol 64:191–199 Wise SP, Moody SL, Blomstrom KJ, Mitz AR (1998) Changes in motor cortical activity during visuomotor adaptation. Exp Brain Res 121:285–299
123
