Connectionist simulations with a dual route model of fear conditioning Paul den Dulk, Bas Rokers, and R. Hans Phaf Psychonomics Department, Faculty of Psychology, University of Amsterdam, Roetersstraat 15, 1018 WB Amsterdam, The Netherlands
[email protected] Abstract The dual route model of fear conditioning, developed by LeDoux (1986, 1996), was investigated in four simulations within a connectionist framework using CALM Maps, a competitive learning procedure. We adapted the model and replicated the simulations by Armony, Servan-Schreiber, Cohen, and LeDoux (1995), compared the two models and finally focused on functional differences between the two routes. Extinction due to interference by intervening material could be found in the model. Latent inhibition was also simulated in this model. The relative contribution of both pathways to the behavior of the model was examined in several lesion experiments. It is argued that the computational model may need further extensions to fully implement the functional differences between the two routes in the theoretical model.
Introduction Amongst all psychological phenomena that have been subject to connectionist modeling, affective processes seem relatively neglected. Neurobiological, and particularly animal, research has provided some conceptual models that may be excellent starting points for neural network modeling. The dual route model of LeDoux (1986, 1996), in particular, has already served as inspiration for a neural network model by Armony et al. (1995). This work, however, focused on one emotion (i.e., fear) and animal behavior (i.e., conditioning). Though we eventually aim at simulating human behavior (e.g., affective priming; see Murphy & Zajonc, 1993) in this type of model now we first replicated and extended the work by Armony et al. (1995). Their network model was rebuilt in another competitive learning framework (i.e., CALM; see Murre, Phaf, & Wolters, 1992; Phaf, Tijsseling, & Lebert, Submitted) which may have some advantages over the learning procedure used by Armony et al. (1995). We also wanted to investigate the functional differences between the two pathways in the model more explicitly, for instance, by separately lesioning the pathways. In classical conditioning an affectively neutral conditioned stimulus (CS) (e.g., a tone of a particular frequency) and a fear evoking unconditioned stimulus (US) (e.g., an electric shock) are presented together, causing presentation, thereafter, of the CS alone to evoke a fear response. The ability to evoke a fear response decreases with repeated presentation of the CS without an US. Relapse of the initial fear response, however, can occur in a number of situations, indicating that memory of the conditioned CS-
US pair remains after extinction in some part of the system (e.g., LeDoux, 1996). Neurobiological research, mainly lesion studies on rats, has shed some light on the neural systems involved. LeDoux, Sakaguchi, and Reis (1984) showed that rats can still be conditioned to auditory stimuli even after total ablation of the auditory cortex, but lesions to the thalamus and the midbrain totally prevented conditioning. Apparently, some subcortical path, sprouting from the thalamus was sufficient for conditioning. Anatomical tracing techniques, revealed a direct connection from the thalamus to the amygdala, which may be the central structure for fearrelated processing (LeDoux, 1996). Lesion studies of this direct route, however, did not fully eliminate conditioning (Romanski & LeDoux, 1992). The path along which the CS was transmitted thus branches at the level of the thalamus. The direct route between thalamus and amygdala seems more important for learning relatively crude CS-US associations, whereas an indirect route (from thalamus through the cortex to the amygdala) seems necessary for learning more detailed and specific associations.
The Armony et al. (1995) Model The Armony et al. (1995) model (the ‘ASCL’ model) consisted of four modules representing the amygdala, the cortex, and two substructures of the thalamus involved in auditory processing: the medial division of the medial geniculate body combined with the posterior intralaminar nucleus (MGm/PIN) and the ventral division of medial geniculate body (MGv). MGm/PIN has direct connections to the amygdala, but it also has connections to the cortex. MGv is part of the indirect path and only has connections to the cortex. The cortex module, in turn, was connected to the amygdala module. There was full (unidirectional) connectivity between modules. The extent of processing in the two pathways is reflected by the respective sizes of the modules: 8 nodes in MGv and 8 nodes in the cortex (i.e., the indirect path), and 3 nodes in MGm/PIN (i.e., the direct path). The amygdala consisted of 3 nodes. A modification of the competitive learning algorithm by Rumelhart and Zipser (1985), incorporating continuous values for the activations, was used. Within-module competition was implemented by inhibition between nodes in a module. Competitive learning leads to the spontaneous emergence of receptive fields in a module in response to any set of input patterns. In these simulations a series of pure tones of contiguous frequencies (and equal intensities) in an arbitrary scale served as input patterns to MGv and MGm/PIN. The specificity of cell responses in all modules
was recorded after a familiarization phase of all input patterns without an US. The US was represented as an external binary input to all nodes of MGm/PIN and amygdala modules, so an equal amount of activation would be sent to all nodes in these modules. After coupling the US to one pure tone, the total activation of all nodes in the amygdala showed a clear increase for the selected tone, indicating successful conditioning. Changes in receptive fields similar to those observed experimentally in animals could, moreover, be observed in the corresponding modules of the network model.
auditory pathways of many animals emerges from the learning rules used in CALM Maps. Cortex
A E
Amygdala A E
Indirect route Thalamus MGv
Direct route
MGm/PIN A E
A E
A Dual Route Model With CALM Maps The architecture of our network model mirrored the ASCL model, except that we used CALM Maps as the competitive learning algorithm. The models had the same connection scheme (i.e., connections between modules were again all-to-all and unidirectional, see Figure 1). CALM Maps are modifications of CALM Modules (Murre, Phaf, & Wolters, 1992), which are able to self-organize topologically along the dimension that has the largest range of variation in the set of input patterns. Learning is achieved by adjusting intermodular connection weights. All intramodular connections remain constant. The CALM procedure is a competitive learning procedure which has a novelty-dependent elaboration learning mechanism. Novel input patterns generally evoke much competition in a module which leads to the activation of an Arousal-node. The Arousal-node activation is then conveyed to the External (E) node, which is assumed to be some external center involved in the regulation of plasticity. It also drives a stochastic mechanism involved in the search for new representations. The activation of the E-node is used as a modifier on the learning parameter of CALM, in such a way that learning is enhanced for novel patterns. In the brain this role might be fulfilled by a neuromodulator. In CALM excitatory and inhibitory nodes are separated explicitly and each representation is, therefore, formed by two nodes, an excitatory Representation-node (R-node) and an inhibitory Veto-node (V-node). By implementing the ASCL model with CALM Maps, each Representation-node was replaced by such a pair. Parameters in the model are mostly the fixed intramodular connections (see Appendix), which were not fitted to the present experiments but were taken from previous simulation work (see Phaf et al., Submitted; Van Immerzeel, 1996). No effort was spent here to adjust parameters to obtain optimal model behavior. In CALM Maps a (Gaussian) gradient of inhibitory connection weights ensures the topological ordering of input patterns. The width of the Gaussian distribution (σ) for inhibitory weights of the four CALM Maps was 2, 2, 3 and 3 nodes for the MGm/PIN, Amygdala, MGm and cortex, respectively. The first and the last nodes in a module were connected so that a circular arrangement (a ring) was created. Consequently all nodes had the same number of neighbors. In the Armony et al. (1995) simulation neighboring frequencies were probably not represented on neighboring nodes in the module, because their competitive learning scheme did not allow for its development. Such a tonotopical ordering, which can be found in the
Nociceptive Input (US) Auditory input (CS)
Figure 1: The architecture of the dual route model with CALM Maps. Rectangles represent CALM Maps, with Arousal-nodes indicated externally of the module. CALM Maps have a ring arrangement. Activity bubbles around a winning node are depicted.
Replication of Conditioning In the first phase of the simple conditioning experiment, all patterns were familiarized. Those input patterns (potential CSs) represented 15 contiguous frequencies. Every frequency-pattern had an overlap of one active node with a pattern to either side. The right- and left-most frequencies, only had overlap to one side. In the second phase, one tone was coupled to an US. We choose the same CS as Armony et al. (1995) (i.e., frequency number 5). After both phases the receptive fields in all modules and the total activity in the amygdala was registered per pattern. The CALM Map probably provides a more suitable way of applying the unconditioned stimulus than the Rumelhart and Zipser (1995) type of competitive learning. Novelty is often associated with fear responses. The novelty detection in CALM can be seen as one way of evoking fear responses. Direct activation of the Arousal-node should, therefore, lead to a fear response. The US-input (with activation 1.0) was fed directly to the A-nodes of the two modules (MGm/PIN and Amygdala) which also received the US in the ASCL model. In the familiarization phase all 15 patterns were presented 150 times for 20 iterations (a cycle of calculating all activations and weights) each. In the conditioning phase frequency 5 was coupled to the US, thus making this pattern the CS. The US-CS pair was fed to the network for 10 presentations. Every test of the performance of the network was averaged over five equal repetitions, because CALM Maps contain a (state-dependent) stochastic factor (see Phaf et al., Submitted). To examine the effects of conditioning we compared the receptive fields of individual nodes before and after conditioning. Because amygdala activity is assumed to result (via the hypothalamus, e.g.,
0.6
pre-conditioning post-conditioning
0.5
Activation
LeDoux, 1996) in diverse autonomic and endocrine reactions, total amygdala activity can be seen as a measure of autonomic activity. The summed activation in the amygdala was, therefore, registered as a function of the frequencies presented, both before and after conditioning.
0.4 0.3 0.2
Development of tonotopical organization. 0.1
The CALM Maps produced a good tonotopical ordering in all modules (e.g., see Figure 2). Such organization was probably not obtained in the ASCL model.
0 1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
7 6
Figure 3: Pre- and post-conditioning receptive fields in MGm/PIN of the node most responsive to the CS before conditioning.
5 4 3 2 1 0 1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
Frequency
Figure 2: Tonotopical organization in MGv. Because the CALM Map has a ‘ring’ structure the starting node in the map is arbitrary.
Shift in Receptive Fields. The results are quite similar to those of Armony et al. (1995). Because all modules have less nodes than there are input patterns, a receptive field generally contains more than one pattern. As a consequence of conditioning the receptive field of the relevant MGm/PIN node (Figure 3) sharpens and shifts toward the frequency of the conditioned stimulus (input frequency 5). The frequencyspecific changes, moreover, occurred only for cells in which the CS evoked a non-zero response before conditioning. The receptive fields of the cortex and the amygdala showed similar shifts. All three modules showed a substantial increase in their response to the CS. Only the representations in those modules that received input both from the CS and US showed these changes. In MGv there was no clear effect of conditioning. This is expected because MGv received no activation, direct or indirect, from the US. The cortex received the converging activation of US and CS only through MGm/PIN. The change of receptive field in MGm/PIN, therefore, led to a change of receptive field in the cortex. The amygdala received US activation in three ways, direct from the US to its Arousalnode, indirect activation from MGm/PIN to the amygdala, and indirect activation via the cortex to the amygdala. The architecture of this model is such that the effect of conditioning focuses on the amygdala.
Behavioral Response. After conditioning, the frequency of the CS produced the highest total amygdala activation. As the distance of the frequency from the CS increased, moreover, the quasiautonomic response decreased and response was smaller than before conditioning (Figure 4). Again, these results were similar to the simulation results found by Armony et al. (1995) and to experimental results obtained in fear conditioning experiments with animals. In sum, the actual competitive learning procedure used does not seem critical for obtaining these conditioning results. They seem to arise primarily from the neurobiologically inspired network architecture. A possible advantage of CALM Maps may be that it is prepared for administering unconditioned stimuli. Activation of the Arousal-node appears to be a suitable way of eliciting fear responses. At any rate, with this type of US, classical conditioning can be simulated successfully. 0.2
Total amygdala activation
Node number in MGV
Frequency
pre-conditioning post-conditioning
0.18 0.16 0.14 0.12 0.1 0.08 0.06 0.04 0.02 0 1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
Frequency
Figure 4: Summed activation of all three nodes in the Amygdala as function of input frequency.
Extinction In this simulation we investigated whether repeated presentation of the CS, without the US, could lead to extinction of the conditioned response to the CS. Repeated presentation of a single stimulus within a competitive learning framework will generally lead to an improved representation of that stimulus and consequently higher activation
0.16
non-conditioned stimulus
0.16 0.14 0.12 0.1 0.08 0.06 0.04 0.02 0 1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
Frequency
0.14 0.12 0.1
0.2
0.08 0.06 0.04 0.02 0 1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
Prensentation
Figure 5: Amygdala activation of most responsive node as a function of presentation during extinction.
Total amygdala activation
Activation
conditioned stimulus
pre-conditioning post-conditioning
0.18
Figure 6: Total amygdala activation as a function of frequency in the latent inhibition simulation. Frequency 4 has been conditioned, after it was familiarized in 100 presentations.
0.2 0.18
0.2
Total amygdala activation
throughout the modules of the network. In order to simulate extinction within this framework, we assumed it was caused by interference due to learning of intervening material (e.g., noises in the environment). For this purpose we presented all other frequencies together with the CS during extinction. The other frequencies were presented twice as often as the CS to ensure sufficient interference. The network state that resulted from conditioning in the previous simulation was exposed 15 times (for 20 iterations) to a randomized pattern batch consisting of the CS and two instances of all other frequencies. For comparison, we performed the same extinction procedure on the network that resulted from the familiarization phase (i.e., before conditioning).
pre-conditioning post-conditioning
0.18 0.16 0.14 0.12 0.1 0.08 0.06 0.04 0.02 0 1
The conditioned frequency showed a decrease in fear response over repeated presentations (Figure 5), whereas, the control stimulus lacked such a decrease, in fact it increased slightly. After 15 presentations the activation levels were about equal. Though extinction can be interpreted as caused by interference, a more active inhibition processing seems more plausible. In animal research the conditioned response to the CS can be restored after extinction by lesioning the indirect route (LeDoux, 1996). The conditioning information in the model is, however, permanently lost by the interference. An adequate simulation of extinction probably also requires an active top-down control process, which can be eliminated by disrupting the indirect route.
Latent Inhibition Latent inhibition can be described as the lower susceptibility to conditioning of a familiar stimulus than of an unfamiliar one. The model presents a good opportunity for simulating this, because both novelty and the US lead to fear responses. With relatively novel stimuli the fear response during conditioning would be larger than with familiar stimuli. To reflect this difference in experience, one half of the frequencies was presented 100 times, and the other half 200 times in the familiarization phase. Stimuli from both groups were alternated. In one condition we chose a CS that was previously presented 100 times (frequency 4) and in the other condition a CS which had been previously presented 200 times (frequency 5). Conditioning again consisted of 10 presentations of the CS-US pair.
2
3
4
5
6
7
8
9
10
11
12
13
14
15
Frequency
Figure 7: Total amygdala activation as a function of frequency in the latent inhibition simulation. Frequency 5 has been conditioned, after it was familiarized in 200 presentations. A latent inhibition effect was found in the total amygdala activation, because conditioning was larger for low familiar (LF) stimuli than for high familiar (HF) stimuli (Figures 6 and 7). It should be noted that latent inhibition was found despite the fact that LF stimuli were represented less strongly in the network than HF stimuli. Without elaboration learning conditioning would probably be smaller for the LF than for the HF stimulus. A similar novelty detection is not present in the ASCL model. The application of CALM Maps in this model, therefore, extends the simulation opportunities to include latent inhibition of conditioning.
Lesioning the Routes To investigate the contributions of the individual routes to conditioning, we lesioned the direct and indirect routes of the network both before and after conditioning. The lesions to the indirect path were implemented by disabling the connections from the cortex to the amygdala. In the direct path the connections between MGm/PIN and amygdala
0.2
Total amygdala activation
were disabled. Apart from the lesions, all features of this simulation were identical to the first simulation. The simulation of lesions to the indirect route before conditioning, is essentially a simulation of the kind of cortical lesion studies by LeDoux et al. (1984), which led him to conclude that the subcortical pathway alone, was also sufficient for conditioning. Our model could also be conditioned without the cortical route (see Figure 8). The conditioning effect had about the same size as the effect found with both routes intact.
post, indirect lesion
0.18
post, direct lesion pre-conditioning
0.16 0.14 0.12 0.1 0.08 0.06 0.04 0.02 0 1
Total amygdala activation
0.2 0.18
pre-conditioning
0.16
po st-conditioning
3
4
5
6
7
8
9
10
11
12
13
14
15
Frequency
Figure 10: Total pre- and post-conditioning amygdala activation as a function of frequency, after lesioning the indirect route or the direct route after conditioning.
0.14 0.12 0.1 0.08 0.06 0.04 0.02 0 1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
Frequency
Figure 8: Total pre- and post-conditioning amygdala activation as function of frequency, after lesioning the indirect route before conditioning. 0.2
Total amygdala activation
2
0.18
post-conditioning
0.16
pre-conditioning
Discussion
0.14 0.12 0.1 0.08 0.06 0.04 0.02 0 1
2
3
4
5
6
7
8
9
10
11
12
13
14
Lesioning the indirect path after conditioning revealed that almost the entire conditioning effect remained, whereas it almost completely disappeared after lesioning the direct path (see Figure 10). This again strengthens the idea that the direct route in the model is more important for conditioning than the indirect route. It is not clear whether the ASCL model shows similar unevenly balanced conditioning effects along the two routes, but that would not conform to the narrow biological theory that formed the inspiration for it.
15
Frequency
Figure 9: Total pre- and post-conditioning activation as a function of frequency, after lesioning the direct route before conditioning. Lesioning the direct pathway before conditioning resulted in a large decrease in autonomic response, though a slight conditioning effect remained (see Figure 9). This seems at odds with the experimental finding that, without the direct path, conditioning is still possible. Because there is an additional layer in the indirect pathway, activations transported to the amygdala through this pathway are attenuated. Conditioning effects may have been smaller as a result of this attenuation. Contrary to experimental findings, the contribution of the indirect pathway to conditioning in the model seems much smaller than the contribution of the direct pathway.
The CALM Map implementation of the dual route model replicated the simulations of Armony et al. (1995) successfully. Characteristic changes in receptive fields and behavioral response were obtained with the model after conditioning. Latent inhibition of conditioning followed quite naturally from the novelty detection property of CALM Maps. Exctinction could be simulated in the model if it was considered an interference phenomenon. Lesion simulations showed that either pathway alone may support conditioning effects, but the direct pathway was the main contributor to those effects. There were no clear indications that the direct pathway in this model is involved in more crude processing than the indirect pathway. The replacement of competitive learning in the ASCL model by CALM Maps showed a number of advantages. Due to the gradient inhibition a tonotopic ordering of input frequencies arose automatically in all modules. Another advantage was that the network could easily be extended to include feedback connections between modules as has been done in other multimodular models with CALM (Murre et al., 1992; Phaf et al., Submitted). The theoretical model (e.g., LeDoux, 1996) assumes bidirectional (but not symmetric) connections between amygdala and cortex. The most important advantage, however, is that CALM Maps are prepared for fear responses. In the standard CALM Maps, novelty automatically leads to activation of the Arousal-node which can be seen as a fear response. The assumption that the US directly activates all nodes in the amygdala and the MGm/PIN can be avoided in this way. The combined operation of novelty and US, moreover,
leads to a latent inhibition effect, which probably does not occur in the ASCL model. Some improvements to the computational model enabling a better correspondence with the theoretical model also present themselves from these simulations. A clear problem is that the contribution to conditioning of the indirect pathway seems much smaller than of the direct pathway. The additional layer between input and amygdala in the indirect pathway attenuates its activation. This renders it difficult to judge the conditioning effects along the indirect pathway. If the cortex activation would be larger, the indirect pathway might well show a finer discrimination of stimuli than the direct pathway. One way to obtain larger activations in the cortex may be to incorporate bidirectional connections between thalamus and cortex, so that positive feedback enhances the representations in the cortex. The model may also be incomplete at a conceptual level. Although the two pathways of our model differ in speed (and intensity) at which signals are processed (the indirect route is slower and weaker), as well as in the processing capacity of the two routes (the indirect route has more nodes), there is no further distinction made as for the explicit function of either pathway. LeDoux (1986, 1996), for instance, has suggested that the direct pathway may be more important for fast reactions in unexpected dangerous situations, whereas the indirect path may be involved in more higher order behavior, such as control processes. The proper way to model extinction of conditioning would probably be to incorporate such control processes. Extinction appears to arise from the learning of regulatory control in the indirect route, but not from interference per se. Appropriate modeling of the function of the indirect route would require implementation of this regulatory control. Though there are few ideas on how to implement (sequential) control processes in a connectionist framework, Sequentially Recurrent Networks (SRNs), which also show a working memory function (see Phaf, Mul, & Wolters, 1994), provide an opportunity for learning and executing sequential operations on items activated in working memory. The disruption of (the influence of) these control processes after extinction, for instance by lesioning the indirect route, could restore the initial conditioning. The full implementation of the dual route model would offer the possibility to simulate many modern experimental results with human subjects in the field of emotion (e.g., LeDoux, 1996). Affective priming, for instance, is the remarkable phenomenon that affective influences (both positive and negative) on the evaluation of neutral stimulus objects can be stronger when the affective prime is not consciously perceived than when it is (Murphy & Zajonc, 1993). There are even indications that the priming effect reverses in conscious conditions. This is completely at odds with non-affective priming results where conscious influences are, generally, larger than non-conscious ones. Such results may emphasize the importance of emotions (and emotion research) for human behavior, because it indicates that the human organism has been evolutionary prepared to perform direct emotional reactions, above which a good deal of regulatory control has developed. Such a view contradicts with the idea that subjective (conscious) report may provide a good means of researching
emotions (see LeDoux, 1996). Consciously experienced emotions probably only give a ‘controlled’ and biased picture of emotions. The best way to further our knowledge about emotions, which may be a central part of human behavior, would be to do both neurobiological and human experimental research and to bridge the gap between the two with connectionist models.
References Armony, J.L., Servan-Schreiber, D., Cohen, J. D., & LeDoux, J.E. (1995). An anatomically constrained neural network model of fear conditioning. Behavioral Neuroscience, 109, 1--12. LeDoux, J.E. (1986). Sensory systems and emotion: A model of affective processing. Integrative Psychiatry, 4, 237-248. LeDoux, J.E. (1996). The Emotional Brain. New York: Simon & Schuster. LeDoux, J.E., Sakaguchi, A., & Reis, D. J. (1984). Subcortical efferent projections of the medial geniculate nucleus mediate emotional responses conditioned by acoustic stimuli. Journal of Neuroscience, 4, 683--698. Murphy, S.T., & Zajonc, R.B. (1993). Affect, cognition, and awareness: Affective priming with optimal and suboptimal stimulus exposures. Journal of Personality and Social Psychology, 64, 723--739. Murre, J.M.J., Phaf, R.H., & Wolters, G. (1992). CALM: Categorizing and learning module. Neural Networks, 5, 55-82. Phaf, R.H., Mul, N.H., & Wolters, G. (1994). A connectionist view on dissociations. In C. Umiltà & M. Moscovitch (Eds.), Attention and performance XV. Cambridge, MA: MIT Press. (pp. 725--751). Phaf, R.H., Tijsseling, A.G., & Lebert, E. (Submitted). Self-organizing CALM Maps. Romanski, L.M., & LeDoux, J.E. (1992). Equipotentiality of thalamo-amygdala and thalamo-cortico-amygdala circuits in auditory fear conditioning. Journal of Neuroscience, 12, 4501--4509. Rumelhart, D.E., & Zipser, D. (1985). Feature discovery by competitive learning. Cognitive Science, 9, 75--112. Van Immerzeel, M. (1996). Simulations with a connectionist model for implicit and explicit memory tasks. Unpublished Master’s thesis. University of Amsterdam, the Netherlands.
Appendix The parameters used in the simulations (for explanation, see Murre et. al., 1992, Phaf et al., Submitted; Van Immerzeel, 1996) were the following: Connection weights: Up-weight 0.5, Down-weight -1.2, Cross-weight -10.0, Flat-weight -1.0, High-weight -2.0, Low-weight 0.4, AEweight 1.0, ER-weight (Strange) 0.1, Initial learning weight value 0.5. Up-weight in input module 1.0. Activation and learning rule parameters: Reset value µ 0.005, Grossberg K-parameter 1.0, Grossberg L-parameter 2.0, Base rate learning 0.005, Virtual weight from the E-node to µ 0.0005. Parameters for the Gaussian inhibition gra-
dient: A 8.8, B 10.0, σ was; 3.0 for MGv, 3.0 for Cortex, 2.0 for MGm/PIN, 2.0 for Amygdala.
