A Computational Model for Development of Post-Traumatic Stress Disorders by Hebbian Learning Sebastien Naze and Jan Treur VU University Amsterdam, Agent Systems Research Group De Boelelaan 1081, 1081 HV, Amsterdam, The Netherlands Email:
[email protected],
[email protected] Abstract This paper contributes a computational model for developing a Post-Traumatic Stress Disorder (PTSD), based on insights from the neurological literature. A number of simulations are presented that show how under specific circumstances the model develops PTSD-phenomena such as re-experiencing, dissociation and flashback episodes.
1
Introduction
Post-Traumatic Stress Disorders (PTSD) may develop in response to a traumatic event. Two primary symptoms are re-experiencing and dissociation. Re-experiencing means that a strong emotional feeling occurs, often with flashbacks, similar to the feeling experienced during the traumatic event. Dissociation is an emotional withdrawal (due to the emotional load triggered by a stimulus) involving loss of body perception or so-called out-of-body experience. Central characteristics of PTSD are altered connectivity between default network and the amygdala, hippocampus and insula. Dissociation may also involve alterations in the relation between the default network and subregions serving cognitive abilities. The computational model for development of PTSD presented here is based on neurological studies. It reflects the understanding of brain functions and reactions observed in reality. The development of the disorder involves the representation of stimuli having some association to the traumatic event (but which by themselves may be neutral), automatic preparation of emotional (fear) responses, feeling the emotions strongly and fear regulation processes. The regulation processes perform inhibition of over-reacting to such stimuli. Indeed in [1] and [2], it is suggested that it may engage top-down reflexive or effortful emotion regulation that, from [3], seems to be impaired in PTSD. In Section 2 the detailed computational model is introduced. Section 3 illustrates different simulation scenarios and their outcomes. Section 4 is dedicated to a discussion on the repercussions of this work and global insights over the topic.
2
Description of the Computational Model
As put forward in [4], recent findings lead to the hypothesis that the increased prefrontal activity in dissociative PTSD reflects stronger emotional regulation and inhibition of limbic emotional networks, including the amygdala. Thus in [4] it is concluded that dissociation is a strategic and controlled regulatory process invoked by extreme arousal to reduce the experience of aversive emotions. This same study shows that thalamic activity is increased in dissociation, which supports the theory that a higher sensory transmission mediates bottom-up excitatory processes. This is also claimed by Oathes in [5] who put forward that dissociative patients show faster emotion labeling. To summarize, it has been shown in the recent literature that PTSD patients suffer from an impaired emotion regulation process combined with a higher sensitivity to emotional stimuli. There exists two main ways of dealing with a memory recall of a traumatic event, each patient usually reacts automatically with only one of these responses. Flashback patients are over-reacting 1
and fall into a strong re-experience of the trauma accompanied with visual recall. Dissociative patients react to traumatic emotion recalls by suppressing body and emotional affects and appraisals. As put forward in [6], the development of the reaction can be assimilated both during the trauma and while recalling the memory: ‘In an event of great arousal and threat, only one trial may be necessary for a conditioned response to be established.’ ([6], p. 84) ‘As the kindled cycle of PTSD continues and becomes chronic, avoidance and withdrawal become increasingly prominent, often with subsidence of symptoms of arousal, hypervigilence and phobia.’ ([6], p. 86)
The assimilation of the trauma was modeled by Hebbian learning. Hebbian learning is based on the principle that connected neurons that are frequently activated simultaneously strengthen their connecting synapse. The principle goes back to Hebb (1949), but has recently gained enhanced interest by more extensive empirical support (e.g., [7]), and more advanced mathematical formulations (e.g., [8]). Not every person exposed to a traumatic episode will develop PTSD. For example in [9], only a small part of Vietnam soldiers are affected by the disorder, although a larger part of this population encompassed shocking experiences. Indeed, there seems to be some physiological dispositions that increase the chance for the disorder to establish, as put forward in [10]: ‘(…) post-traumatic stress disorder (PTSD), a disorder that is characterized by a failure to extinguish traumatic memories and which may develop more frequently in people who exhibit a deficit in extinction of conditioned fear responses before trauma exposure, is associated with abnormally low plasma and urine cortisol levels and increased negative feedback of the hypothalamic–pituitary–adrenal axis. Hence, low cortisol may contribute to the persistence of PTSD symptoms by impairing extinction. Interestingly, daily administration of low doses of hydrocortisone to PTSD patients has been reported to reduce symptoms including reexperiencing, hyperarousal, and avoidance, perhaps in part through a facilitation of extinction.’ ([10] p. 135)
The computational model introduced here uses sensory representation states for external stimuli and body states, and preparation states for emotional responses and regulation actions to turn away from stimuli that lead to high, disturbing levels of arousal; for an overview, see Figure 1.
Fig. 1: Overview of the computational model
Moreover, a control state is used that detects disturbing levels of arousal, and in turn can activate suppressing or regulating processes. In line with [11] and [12], it is assumed that emotional response preparations affect sensory representations of related body states (body 2
maps), both by an internal as-if body loop and an external body loop. These body maps are considered the basis of feeling the emotion. Moreover, it is assumed that this feeling in turn has a strengthening effect on the emotional preparation state, so that a cyclic process occurs, in line with [13], pp. 91-92. In Figure 1, s2 is the stimulus that causes the traumatic experience, and s1 is a more neutral stimulus that has some association to the situation of the trauma. For example, s2 is the visual image of a fire while being inside a burning house, while s1 is the image of the house from outside while it is not burning. It is assumed here that during the event that involves the traumatic experience, the person receives both stimuli. Moreover, the emotional response and feeling are assumed to relate to the preparation and sensory representation of a body state indicated by b. In Table 1 an overview is given of the states (indicating real numbers between 0 and 1) used in the model, and in Table 2 the connection strengths. Note that the strengths for inhibiting links are taken negative; this applies to ω7 , ω8 , ω11 , ω14 (see also Table 3 for example values). Table 1: Overview of the states used notation wsS ssW srsW epb beb csb taps1 taes1
explanation World state for stimulus S (S is stimulus s1 or s2) Sensor state for W (W is stimulus s1, stimulus s2, or body state b) Sensory representation state for W (W is stimulus s1, stimulus s2, or body state b) Emotional response preparation state for b Body effector state for b Control state for b Turn away preparation state for s1 Turn away effector state for s1 Table 2: Overview of connections and weights From state
sss1 epb, sss2 ssb , epb , csb srsb , srss2 , srss1 srss1 , srss2 , srsb , csb csb epb, csb taps1 beb csb , wss1 wss2
To state
srss1 srss2 srsb csb epb taps1 beb taes1 ssb sss1 sss2
Weights ω1 ω16, ω1
LP LP1
ω2 , ω4 , ω11 ω6 , ω17 , ω18
LP2 LP3
ω5 , ω15 , ω3 , ω7
LP4
ω9
LP5
ω12 , ω8 ω10
LP6 LP7
ω13 ω14, ω0
LP8
ω0
LP9
Below, the dynamics following the connections between the states in Figure 1 are described in more detail. This is done for each state by a dynamic (temporally) Local Property (LP) specifying how the activation value for this state is updated (after a time step of t) based on the activation values of the states connected to it (the incoming arrows in Fig. 1). In these update specifications a function f is used. This function is the identity function f(X) = X for LP8 and LP9 for the sensor state of s2. For the other dynamic properties f is defined as a combination function as follows: f(X1, .., Xk) = th(σ, τ, X1+ …+ Xk)
with th(σ, τ, W) = [1 / (1+e – σ (W – τ)) - 1 / (1+e σ τ)](1+e – σ τ)
a logistic threshold function, where σ is the steepness and τ is the threshold; this function f is applied to LP1 for s1 , LP2 to LP7 and LP9 for s1. Parameter is an update speed factor.
3
LP1 Sensory representation of stimulus s1 and s2 The process of generating sensory representations for stimuli s 1 and s2 are described by: d srss1 /dt = [ f(ω1 sss1 ) – srss1 ]
d srss2 /dt = [ f(ω16 epb , ω1 sss2 ) – srss2 ]
LP2 Sensory representation of a body state (body map) The body map is maintained taking into account the suppressing effect of the control state: d srsb/dt = [ f(ω2 ssb, ω4 epb, ω11 csb) – srsb ]
LP3 Control state for a sensory representation of a body state The control state is generated based on the sensory representation of b (feeling the emotion) and the considered stimuli. d csb/dt = [ f(ω6 srsb , ω17 srss2 , ω18 srss1 ) – csb]
LP4 Emotional preparation for a body state The preparation for an emotional response is generated, depending on stimuli and the feeling. Here also a suppressing effect of the control state is incorporated. d epb/dt = [ f(ω3 srsb , ω5 srss1 , ω7 csb , ω15 srss2 ) – epb ]
LP5 Turn-away preparation One type of emotion regulation concerns antecedent-focused regulation (situation selection/modification, attentional deployment); cf. [1] [14]. This has been modeled in the form of an avoidance reaction for the stimulus, indicated by ‘turn-away’: d taps1 /dt = [ f(ω9 csb) – taps1 ]
LP6 Body change A body state is changed based on the preparation for it, but possibly suppressed by the control state. d beb/dt = [ f(ω12 epb , ω8 csb) – beb ]
LP7 Turn-away action A prepared turn-away action is performed as follows:
d taes1 /dt = [ f(ω10 taps1 ) – taes1 ]
LP8 Sensing a body state Sensing a body state is described in a straightforward manner: d ssb/dt = [ f(ω13 beb) – ssb]
LP9 Sensing stimulus s1 or s2 Sensing stimulus s1 does not only depend on the actual world state, but also on whether a turn-away action has been performed; on the other hand, sensing stimulus s2 does only depend on the actual world state: d sss1 /dt = [ f(ω14 taes1 , ω0 wss1) – sss1 ] d sss2 /dt = [ f(ω0 wss2 ) – sss2 ]
Hebbian learning rules The learning rules to achieve such an adaptation process is based on the Hebbian learning principle that connected neurons that are frequently activated simultaneously strengthen their connecting synapse e.g. [4]. The change in strength for the connection ωij between nodes i, j is determined by a function g(yi, yj, ωij) as follows: d ωij(t) /dt = g(yi , yj , ωij)
with g(yi , yj , ωij) = ωyi(t)yj(t)(ωs – ωij(t))
+ ω (ωw – ωij(t))
Here yx(t) are levels of node yx at time point t, ωs and ωware the strongest and the weakest possible connection strengths, respectively. Moreover, ωis the learning rate and ωthe extinction rate of the connection.
4
3
Simulation Experiments
Based on the model described above, simulation experiments have been performed using the LEADSTO software environment (cf [15]), and the jHepWork data-analysis framework [16] to plot data from the generated traces. This section presents some example traces involving the assimilation of the traumatic event that set the disorder. The parameter values used in the simulation are shown in Table 3. Simulations are discussed for 3 scenarios: healthy, flashback and dissociative subjects. They involve first a learning process that corresponds to the assimilation of the traumatic experience by Hebbian Learning as specified in Section 2. The person reacts to neutral and frightful stimuli s1 and s2 at the same time in the form of emotional involvement and body responses, which stimulate specific neural circuits. Table 3: Parameter values for the three scenarios Threshold and steepness values sensory representation of s1 sensory representation of body sensory representation of s2 emotional preparation control state turn away preparation turn away effector body effector
Healthy τ σ 0.2 0.2 0.2 0.6 0.5 0.2 0.2 0.4
Flashback τ σ
4 4 4 4 4 4 4 4
0.2 0.2 0.2 0.2 0.5 0.2 0.2 0.4
Dissociation τ σ
4 4 4 4 4 4 4 4
0.2 0.2 0.2 0.2 0.3 0.2 0.2 0.4
4 4 4 4 4 4 4 4
values Healthy Flashback Dissociation ω0 ω1 ω2 ω3 ω5 ω6 ω4 ω7 ω8 ω9 ω11 ω12 ω10 ω13 ω14 ω15 ω16 ω17 ω18
1 1 1 0.6 0 0.6 0.2 -0.4 -0.4 0.8 0 0.7 0.6 1 -1 0.8 0 0.5 0
1 1 1 0.6 0.6 0.6 0.7 -0.4 -0.4 0.8 -0.2 0.7 0.6 1 -1 0.8 0.6 0.5 0.3
1 1 1 0.6 0.6 0.6 0.7 -0.7 -0.4 0.8 -0.8 0.7 0.6 1 -1 0.8 0.2 0.5 0.3
rates ω4 ω5 ω7 ω11 ω16 ω17 ω18
Healthy
Flashback
Dissociation
0.3 0.3 0.3 0.1 0.3 0.3 0.3
0.05 0.05 0.005 0.001 0.05 0.001 0.05
0.3 0.3 0.3 0.1 0.3 0.3 0.3
0.005 0.001 0.005 0.001 0.005 0.001 0.005
0.3 0.3 0.3 0.1 0.3 0.3 0.3
0.005 0.001 0.005 0.001 0.005 0.001 0.005
Secondly, the neutral stimulus s1 occurs at some later times after a period of rest. In subjects predisposed to the disorder, the stimulus has the effect of re-activating circuits that have fired during the traumatic event, conveying a flashback or dissociative re-experience. In healthy subjects (who do not have predisposition to PTSD), those circuits are not re-activated due to a proper extinction mechanism as put forward in Section 2. Simulations for healthy persons are shown in part a), flashback assimilation and re-experience are shown in part b) and part c) concerns dissociative assimilation. All cases describe a situation where subjects encompass a traumatic event from time points 0 to 60. In predisposed subject, it brings PTSD flashback (b) or dissociative (c) symptoms, while in healthy subjects (not predisposed to PTSD) the disorder does not develop as connections extinguish efficiently. The person faces a traumatic situation represented by stimulus s2 (such as the vision of mutilated bodies on a car accident) that also contains some neutral element (like the car itself). This vision is accompanied by a very strong emotional load that the neurobiological 5
system is not used to deal with. The presence of the stimuli and the emotional involvement at the same time links both stimuli to an emotional preparation which itself is associated to the feeling. Parameters values used are the same than in Table 3. The person starts with healthy subject values and evolves towards flashback (b) or dissociation (c). The simulations are presented in Figures 2, 3 and 4. They have in common the following four phases: 1. The external stimulus s2 occurs with s1 in parallel, that triggers their representations srss2 and srss1 respectively. Moreover, srss2 generates a high emotional preparation epb via the strong ω3 connection. 2. This preparation involves the feeling of the emotion (body representation srs b) through ω4 that is so intense that it increases it from 0.1 to 0.6 between times 5 and 20, and also affects the body response (body effector beb, such as facial expressions) via ω12. 3. The emotion feeling srsb is very strong (as it is a traumatism) and the emotional regulation system represented by csb is not able to handle such a high intensity. 4. This emotional involvement (high srsb at time 20) in parallel with the neutral stimulus s1 makes the association between this stimulus s1 and the preparation of emotion epb through ω5, which did not exist before and is the principal source of symptoms re-experience at a later time. The same process happens with the control state csb that is associated to srss1 via ω18. Also the strong representation srss2 of stimulus s2 along with the high intensity of emotional regulation csb leads to the association between both states via ω17. The last phase of the development (or recovery) of the disorder is specific for each of the cases discussed in this paper. Healthy, non-predisposed subject 5. Connections ω5 , ω4 and ω16 are properly extinguished when the traumatic event finishes (from time point 60). Thus, no emotional preparation is triggered from the neutral stimulus after this spontaneous rehabilitation.
Fig. 2: Simulated trauma recovery in Healthy scenario (first graph shows external states; second graph shows internal states; third graph shows connection strengths). Only relevant connections are displayed.
6
In the simulation, it can be noticed that connections ω5 , ω4 and ω16 reach complete extinction at time point 120. This is due to their extinction rate, i.e. ω5 = ω16 = ω4 = 0.05, assumed higher in healthy, non-predisposed subjects. Flashback response 5. The strong emotional preparation ep b is associated to the representation srss2 of the traumatic stimulus s2, by connection ω16. It is through this connection that the flashback symptom occurs while re-presenting the neutral stimulus s1 to the subject at later time points 140 and 210. In the simulation it can be seen that when the stimuli end the state of the person takes more time to get back to stable (i.e. s1 and s2 stops at time 60 but the person is still perturbed until time 100); a person who undergoes a traumatic experience takes some time to calm down. It is also seen that the assimilation of the mechanism continues to be learnt during the re-experience (around time 150, flashback episode leads to the evolution of some connection strengths), this relates to the fact that the traumatic event may be only a first trigger to the development of the disorder; later growth of symptoms may occur from the re-experiences.
Fig. 3: Simulated Assimilation of Flashback PTSD scenario (first graph shows external states; second graph shows internal states; third graph shows connection strengths). Only relevant connections are displayed
Dissociative response 5. The very strong activation of the control state csb involves hyper-inhibition of emotions, which is expressed by the growth of connection ω11 and ω7 to srsb and epb, respectively, that results in the feeling of dissociation. It is through those connections that the dissociative symptom occurs while re-presenting the neutral stimulus s1 to the subject at later time steps 140 and 210. In the simulation, it can be seen that during the traumatic event the person has a first dissociation experience, i.e. srss2 gets lower while stimuli s1 and s2 are still perceived between time step 20 and 60. As for flashback, it is also seen that the assimilation of the mechanism continues to be learnt during the re-experience (post-trauma dissociative episodes may lead to the evolution of some connection strengths), this again relates to the fact that the traumatic event may be only a first trigger to the development of the disorder; later growth of symptoms may occur from the dissociative re-experiences. 7
Fig. 4: Simulated Assimilation of Dissociative PTSD scenario (first graph shows external states; second graph shows internal states; third graph shows connection strengths). Only relevant connections are displayed.
4
Discussion
The computational model presented in this paper was designed using principles from the neurobiological literature on Post-Traumatic Stress Disorders. It was shown that when it is assumed that some connections are altered, patterns of feelings and behaviors are obtained similar to those described in literature. The assumption is that not only the traumatic experience has an impact on these connections, but also some neurophysiological predispositions make those connection changes effective for longer time periods (depending mostly of the other experiences that the subject endured in the past). The presented model follows findings in the neurophysiology of PTSD; as example, it fits into the animal model presented in [17], including features from stress-based and mechanism-based models. However, it differs substantially from the model presented in [18] and [19], as this associative memory model focuses mainly on memory formation and does not consider the neurological modifications due to extreme emotions in addition to memory recall. Indeed, it rather explores sights in which PTSD symptoms appears due to memory processes that have impact on emotions. It has been demonstrated that it seems to work also the other way, i.e. strong emotions directly intervene during the memory formation, and this association might disrupt consciousness in a way that differs from regular memory formation. The obtained computational model can be used in education of therapists to develop simulationbased applications to train and get insights in the processes in action in some types of PTSD patients. Other types of applications can rise, such as a system for agent-based virtual stories in which, for example, persons with PTSD play a role which can be based on the presented model during a fear extinction therapy, to force to unlearn a specific conditioned response.
8
References 1. Gross, J.J.: Antecedent- and response-focused emotion regulation: divergent consequences for experience, expression, and physiology. Journal of Personality and Social Psychology 74, 224–237 (1998) 2. Lanius, R.A., Williamson, P.C., Bluhm, R.L., Densmore, M., Boksman, K., Neufeld, R.W.J., Gati, J.S., and Menon, R.S.: Functional Connectivity of Dissociative Responses in Posttraumatic Stress Disorder: A Functional Magnetic Resonance Imaging Investigation. Biol Psychiatry 57, 873-884 (2005) 3. Lanius R.A., Bluhm R.L., Coupland N.J., Hegadoren K.M., Rowe B., Théberge J., Neufeld R.W.J., Williamson P.C., Brimson M.: Default network connectivity as a predictor of post-traumatic stress disorder symptom severity in acutely traumatized subjects. Acta Psychiatrica Scandinavica 121, 33-40 (2010) 4. Felmingham K., Kemp A.H., Williams L., Falconer E., Olivieri G., Peduto A., Bryant R.: Dissociative response to conscious and non-conscious fear impact underlying brain function in Post-Traumatic Stress Disorder. Psychological Medicine 38, 1771-1780 (2008) 5. Oathes Desmond J., Ray William J.: Dissociative Tendencies and Facilitated Emotional Processing. Emotion 8, 653–661 (2008) 6. Scaer R.C.: The Neurophysiology of Dissociation and Chronic Disease. Applied Psychophysiology and Biofeedback 26, 73-91 (2001) 7. Bi, G., Poo, M.: Synaptic modification by correlated activity: Hebb’s postulate revisited. Annual Review of Neuroscience 24, 139-66 (2001) 8. Gerstner, W., Kistler, W.: Mathematical Formulations of Hebbian Learning. Biological Cybernetics 87, 404-415 (2002) 9. Emery, V.O., Emery, P.E., Shama, D.K., Quiana, N.A., Jassani, A.K.: Predisposing Variables in PTSD Patients. Journal of Traumatic Stress 4, 325-343 (1991) 10. Myers, K.M., Davis, M : Mechanisms of fear extinction. Molecular Psychiatry 12, 120-150 (2007) 11. Damasio, A.R.: The Feeling of What Happens: Body and Emotion in the Making of Consciousness. New York: Harcourt Brace (1999) 12. Lanius, R.A., Williamson, P.C., Bluhm, R.L., Densmore, M., Boksman, K., Neufeld, R.W.J., Gati, J.S., and Menon, R.S.: Functional Connectivity of Dissociative Responses in Posttraumatic Stress Disorder: A Functional Magnetic Resonance Imaging Investigation. Biol Psychiatry 57, 873-884 (2005) 13. Damasio, A.R.: Looking for Spinoza: Joy, Sorrow, and the Feeling Brain. Vintage Books, London (2003) 14. Goldin, P.R., McRae, K., Ramel,W., Gross, J.J.: The neural bases of emotion regulation: Reappraisal and Supression of Negative Emotion. Biological Psychiatry 63, 577-586 (2008) 15. Bosse, T., Jonker, C.M., Meij, L. van der, and Treur, J., A Language and Environment for Analysis of Dynamics by Simulation. International Journal of Artificial Intelligence Tools 16, 435-464 (2007) 16. Chekanov, S.: Scientific Data analysis using Jython Scripting and Java. Springer Verlag (2010) 17. Cohen, H., Richter-Levin, G.: Toward Animal Models of Post-Traumatic Stress Disorder. In: Shiromani, P.J., Keane, T.M. and LeDoux, J.E. (eds.), Post-Traumatic Stress Disorder: Basic Science and Clinical Practice, pp. 133-149. Springer Verlag (2009) 18. Tryon, W.W.: A Neural Network Explanation of Posttraumatic Stress Disorder. Journal of Anxiety Disorders 12, 373-385 (1998) 19. Tryon, W.W.: A Bidirectional Associative Memory Explanation of Posttraumatic Stress Disorder. Clinical Psychology Review 19, 789–818 (1999)
9
