The neural circuitry of reward and its relevance to ...

The Neural Circuitry of Reward and Its Relevance to Psychiatric Disorders David T. Chau, PhD, Robert M. Roth, PhD, and Alan I. Green, MD*

Address *Department of Psychiatry, Dartmouth Medical School, Dartmouth Health and Medical Center, One Medical Center Drive, Lebanon, NH 03756, USA. E-mail: [email protected] Current Psychiatry Reports 2004, 6:391–399 Current Science Inc. ISSN 1523-3812 Copyright © 2004 by Current Science Inc.

Scientific interest in how the brain processes reward has burgeoned during the past 50 years since the discovery that rats will do tasks such as pressing a lever to obtain electrical stimulation of the brain. This interest was additionally encouraged by the observation of an association between reward and dopamine activity in the mesocorticolimbic system. In this article, we will discuss the complex nature of reward processing and recent animal studies and human functional neuroimaging studies to elucidate the current understanding of the neural substrates of reward processing and its components. Lastly, we will review recent theoretical and empirical work investigating the role of brain reward circuitry in several psychiatric disorders, including substance use disorders, schizophrenia, pathologic gambling, major depressive disorder, and attention-deficit/ hyperactivity disorder.

Introduction Broadly defined, rewards are environmental incentives that are approached and revisited after they have been experienced [1]. Rewards can lead to positive emotional experiences, such as feelings of pleasure, and they can facilitate the establishment of learned associations and ways of responding to stimuli in the environment. Rewards also can promote the maintenance of behaviors (or an increase in the intensity and frequency of behaviors) that are associated with obtaining reward [1,2]. Responding appropriately to natural (primary) rewards, such as food and sex, is part of our evolutionary heritage and contributes to species survival and reproduction [3]. There also is a class of rewards that involve abstract cognitive representations, which are indirectly associated with the ability to obtain primary rewards. These “cognitive” rewards include things such as money, beauty, and power [2]. Studies from disciplines such as psychology and economics have begun to show the complexity of human

reward processing. For example, it has been observed that people generally overestimate how much they will enjoy anticipated events or situations, such as being wealthy. They discount the value of future rewards relative to more immediate ones, and choose a less desirable reward to avoid selecting among two more desirable rewards [4–6]. Such findings not only show a significant cognitive component to reward processing, but they also show that the abstract representation of rewards can influence cognitive processes such as decision making. Such complexity of reward processing has been shown in animals and humans. For example, studies have found that animals also tend to prefer smaller immediate rewards to larger delayed rewards [7]. A recent study showed that capuchin monkeys will refuse a piece of cucumber (a less desirable reward) when they see another monkey receive a grape (a more desirable reward) [8]. Although the precise mechanism motivating the behavior of these animals is debatable [9], it is evident that there is a mediating process at work beyond simple stimulusresponse associations for obtaining reward. It is increasingly evident that reward is not a unitary concept. The term “reward” subsumes multiple psychologic and physiologic components subservient to the motivational, learning, and emotional or hedonic aspects of reward processing [2,10••]. The fractionation of the components of reward is receiving increasing support from animal and human investigations showing distinct neural circuitry and neurochemical substrates for particular components of reward processing. We will provide an overview of this burgeoning area of research.

Functioning of Brain Reward Circuitry: Animal Studies Fifty years ago, it was reported that animals will work to obtain electrical stimulation in specific brain regions including the medial forebrain bundle, ventral tegmental area, and hypothalamus [11,12]. Subsequent research established that such brain stimulation activates dopamine neurons within the ventral tegmental area that project to the nucleus accumbens (the so-called mesolimbic pathway), and that this neural pathway serves to reinforce responses for food and drugs of abuse [13]. More recently, several other brain regions have been shown to play a role in electrical or chemical selfstimulation, including the orbitofrontal cortex, medial frontal cortex, amygdala, hippocampus, accumbens, ventral pal-

392

Neuropsychiatric Disorders Figure 1. Brain reward circuitry. Reward information is processed in prefrontal-temporal limbic regions, thalamus, and hypothalamic and basal forebrain nuclei. Nucleus accumbens serves as a gateway for limbic information to access motor pathways. Information through the accumbens is gated by dopamine. Thick arrows indicate limbic cortical-striatal-thalamic pathways providing limbic-to-motor interface. AC–anterior cingulate; BLA–basal lateral amygdala; BNST–bed nucleus of stria terminalis; CeA–central nucleus of amygdala; DA–dopamine; DLPFC–dorsolateral prefrontal cortex; LA–lateral amygdala; LH–lateral hypothalamus; MD–mediodorsal; MPFC–medial prefrontal cortex; OPFC–orbital prefrontal cortex; PPT–pedunculopontine tegmental nucleus; SLEA–sublenticular extended amygdala; SN–substansia nigra; VP–ventral pallidum.

lidum, and mediodorsal thalamus [12]. Many of these regions lie within the limbic cortico-striatal-thalamic circuit (Fig. 1) that interacts extensively with the mesolimbic dopamine pathway [14••,15]. Additional research using behavioral probes of hedonic reaction and studies using intracranial electrophysiologic recording, have begun to identify specific brain regions and pathways underlying aspects of reward processing including motivation (goal planning, incentive salience or “wanting”), learning (reward expectancy, stimulus-reward association), and emotional reactions, such as the experience of pleasure or “liking.” We will focus on several key brain reward regions as they relate to reward processing in animals.

Nucleus Accumbens The nucleus accumbens is a major part of the ventral striatum and a central node within the limbic system [16]. The ventral striatum with the extended amygdala (central nucleus of the amygdala and bed nucleus of stria terminalis) mediates reward-based drive and motivation [16,17••,18]. The accumbens receives input from all major limbic regions, including medial and orbital prefrontal cortex, amygdala, and hippocampus (Fig. 1) [18]. It is divided into two functionally distinct subregions: the phylogenetically old ventromedial shell linked to primarily limbic structures; and the central and lateral core linked to structures responsible for cognition and motor control [19]. Limbic information in the shell influences activity in the core through interaction with the substantia nigra, or by indirect connection with the dorsolateral prefrontal cortex, a region involved in executive

function such as motor planning [14••]. In turn, the accumbens core interacts with extrapyramidal motor circuits. Therefore, the accumbens may serve as an interface between limbic and motor circuits, translating emotion into action [20]. The accumbens shell plays an important role in motivation and affect, mediating Pavlovian conditioning [21], reaction to novelty [22], control of feeding [23], and taste hedonics [10,24]. Additionally, the shell seems to mediate the rewarding impact of drugs of abuse. Animals will selfinfuse various drugs of abuse directly into the shell but not the core [15]. The core, by contrast, plays a critical role in the learning and expression of behavior driven by the incentive value of expected rewards [23,25,26]. The core also seems essential for adaptive behavior, promoting response for delayed rewards [25].

Frontal Lobe Reward processing in the frontal lobe occurs mainly in orbital and medial subregions [27]. The orbitofrontal cortex receives and processes modality-specific sensory inputs and emotional and motivational information through reciprocal connections with other limbic regions. These diverse inputs allow orbitofrontal neurons to discriminate different primary rewards and to vary their activity according to the immediate motivational value of the respective rewards [28,29••]. For example, these neurons respond to the taste of food in a hungry animal, but not in a satiated one. The role of the orbitofrontal cortex in reward processing does not seem unitary. Some neurons respond to reward-predicting cues, whereas others follow delivery of a

The Neural Circuitry of Reward and Its Relevance to Psychiatric Disorders • Chau et al.

393

reward. Orbitofrontal neurons also interact with the basolateral amygdala to code ongoing changes in stimulusreward associations [29••,30]. The medial frontal cortex is involved in reward-based action selection and evaluation of action-outcome contingencies, but it is not a site for stimulus-reward association [25,31–33]. For example, monkeys trained to discriminate multiple response-reward pairs fail to select among correct responses for the respective reward after lesions are created in the medial frontal cortex [32]. The medial frontal cortex has extensive connection with the lateral frontal lobe, a region that also contributes to action selection and execution. However, the medial region is involved in the internal evaluation of the consequences of performance, whereas the lateral frontal cortex uses external cues to do this function [33]. In this context, subregions of the medial frontal cortex (monkey areas 25 and 32) heavily innervate the shell of the nucleus accumbens [14••], which mediates the “liking” of rewards [10••,24].

behavior, and for inhibiting inappropriate appetitive behavior [25]. The basolateral nucleus likely drives appetitive behavior through its connection with the nucleus accumbens core and the lateral hypothalamus [25,39]. In addition to its role in Pavlovian conditioning, the central nucleus of the amygdala mediates orienting responses to reward through its widespread projections to the hypothalamus and to the chemically defined diffuse neurotransmitter systems in the reticular formation, namely the dopaminergic ventral tegmental area, the noradrenergic locus coeruleus, the serotonergic raphe nucleus, and the forebrain cholinergic systems [25,38]. The central nucleus of the amygdala and the bed nucleus of stria terminalis also influence affect, incentive motivation, and visceral responses through its projections to the shell of the nucleus accumbens and the ventral tegmental area. These regions of the extended amygdala are involved in sexual and appetitive behavior, and may be central to various psychiatric conditions, especially those related to stress [17••].

Amygdala and Extended Amygdala

Mesocorticolimbic Dopamine System

The amygdala is a heterogeneous nuclear group, divided into three major subregions: the basolateral, central, and lateral nuclei [34,35]. The basolateral nucleus is connected with medial and orbital frontal cortex, and also can access olfactory, visual, gustatory, and multimodal information from sensory and association cortices; it projects to the central nucleus, the shell and core of the nucleus accumbens, and the lateral hypothalamus. In contrast, the central nucleus derives its afferent input primarily from the lateral hypothalamus and brainstem, areas involved in gustatory and visceral responses, and it sends widespread projections throughout the brainstem and diencephalic areas that control orienting behavior and autonomic and neuroendocrine responses. The central nucleus of amygdala is connected to the bed nucleus of the stria terminalis and the sublenticular region. These three regions comprise the extended amygdala. The extended amygdala is placed strategically in the basal forebrain, around the internal capsule. It coordinates input from the limbic cortex and basolateral amygdala and output to motor and autonomic effectors in the ventral striatum, hypothalamus and brainstem [16]. The basolateral and central nuclei of the amygdala seem to have specific roles in reward-based emotional learning and attention [36–38]. The basolateral amygdala is required for appetitive instrumental behavior, whereas the central nucleus mediates simple Pavlovian response. Fundamental to the operation of basolateral amygdala is its ability to create linked associations. For example, intact rats trained to associate light with food, and then light with tone, will seek food when the tone is presented later. However, lesions of the basolateral nucleus prevent rats from responding to tone, but not to light. Therefore, the basolateral nucleus seems to be important for making higher order associations between multiple stimuli and rewards, for facilitating instrument

Midbrain dopamine neurons are an integral part of reward circuitry. Dopamine neurons in the ventral tegmental area are regulated by inputs from all major limbic regions including the frontal lobe, amygdala (especially the central nucleus), extended amygdala, and the pedunculopontine tegmental nucleus, an obligatory site for lateral hypothalamic stimulation reward and a relay nucleus for taste and visceral information from the hypothalamus and brainstem [40–42]. Dopamine neurons in the ventral tegmental area give rise to the mesolimbic and mesocortical pathways. Although the mesolimbic system has been proposed as an important common pathway for the positive reinforcing effects of natural rewards and drugs of abuse [43], the mesocortical pathway does not produce concrete behavioral output, per se, but may modulate reward processing by influencing frontal lobe function [42]. Additionally, dopamine in the prefrontal cortex regulates subcortical dopamine release under highly stimulating conditions such as stress or when presented with rewarding stimuli [42]. Dopamine traditionally has been assigned an important role in behavioral reinforcement [44•]. Recent work has provided alternative views of dopamine function, including a central role in reward learning [26, 45], in switching behavior and attention to biologically important events [46], and in attributing incentive motivational and salience (wanting) to neural representations of reward stimuli [47,48]. Dopamine neurons seem to respond preferentially to reward-predicting stimuli and to unanticipated reward delivery [45]. They become less active when reward expectation is not met or during exposure to aversive stimuli [49]. However, neurons in the ventral tegmental area also may be stimulated under certain stressful conditions [42]. These paradoxical responses suggest that the ventral tegmental area may serve a broader role such as mediating incentive arousal, facilitating

394

Neuropsychiatric Disorders

learning or that it comprises a functionally heterogeneous neuronal population.

Functioning of Brain Reward Circuitry: Human Neuroimaging Studies Positron emission tomography (PET) and functional magnetic resonance imaging (fMRI) provide in vivo identification of brain regions involved in cognitive, motor and sensory processes in the healthy and disordered brain [50– 52]. A growing body of research has used these functional neuroimaging techniques to help elucidate the neural substrates of reward processing in humans. Human PET and fMRI studies have most consistently used money as the reward for correct responses in stimulus-response type tasks. Results generally have been consistent in revealing activation in several brain regions linked to the processing of reward including the nucleus accumbens, caudate nucleus, orbitofrontal cortex and medial frontal cortex [53,54]. Activations also have been noted in the amygdala, cingulate gyrus and other frontal cortical subregions, but less consistently. This pattern of activations generally is consistent with that observed in animal studies that have used primary rewards such as food. Discrepancies among human neuroimaging studies likely are attributable in part to methodological differences including varying task demands, the amount of monetary reward provided, and the analytic strategies used. Some of the variability in findings has been clarified through studies designed to identify whether, like in animals, the neural circuitry subservient to different components of reward can be dissociated in humans. Several recent fMRI studies have focused on identifying regions activated during the anticipation and consumption components of reward processing. Results have tended to show that anticipation activates the nucleus accumbens, caudate nucleus, thalamus, and amygdala [55]. Reward consumption seems to preferentially activate several subregions of the prefrontal cortex, particularly the orbital and medial frontal cortex, with less dramatic activation seen in the caudate, accumbens and amygdala [55–57]. The specific regions activated during the phases of anticipation and consumption of reward seem to depend partly on the magnitude of the reward expected and received, with greater activation observed in the medial and orbital frontal cortex, accumbens and amygdala with an increasing amount of monetary reward [53,58]. In addition, greater activation is seen for the accumbens and orbitofrontal cortex when the receipt of reward is unpredictable [59], and for the accumbens after failure to receive a reward at the predicted time [60•]. The latter finding is consistent with recent animal studies indicating that striatal dopamine may be involved in the processing of errors of reward prediction [45]. Positron emission tomography and fMRI studies also have shown that brain reward circuitry in healthy humans is activated to various primary and secondary rewards

including food such as chocolate [61], listening to enjoyable music [62], looking at beautiful faces [63] or humorous cartoons [64], and sexual activity [65]. These findings suggest that numerous types of stimuli may have an effect on cognition, emotion and behavior through their impact on reward circuitry. Several recent functional neuroimaging studies have used [11 C] raclopride binding as an index of dopamine release in the study of reward circuitry in humans. Increased dopamine release has been observed in the ventral striatum in healthy humans when they received monetary reward [66], particularly if presented on a variable rather than fixed schedule [67]. Increased dopamine release in the nucleus accumbens also has been reported in patients with Parkinson’s disease after administration of apomorphine (a dopamine agonist that can improve motor symptoms) or placebo (saline) injections [68]. Patients who subjectively experienced improved motor symptoms after placebo administration showed increased dopamine release. This finding was interpreted as suggesting that dopamine release in the accumbens is related to the expectation of reward rather than to its receipt. Overall, neuroimaging findings may be interpreted as suggesting that frontal subregions are involved in tracking and representing the value, magnitude and emotional intensity of rewards, and that subcortical regions play a primary role in the motivational and emotion generating aspects of rewards. However, recent studies have raised the possibility that the nucleus accumbens also may be involved in the processing of aversive stimuli such as painful thermal or electric stimulation [69,70]. This had led some authors to hypothesize that the accumbens responds more generally to the salience of stimuli rather than reward [71].

Integration of Data from Animal and Human Studies It is evident from the animal studies discussed previously that there are different neurons within given regions, such as the orbitofrontal cortex, that respond to disparate phases of reward processing. However, the limited spatial resolution of functional neuroimaging techniques used in the current literature on reward processing in humans makes fine discriminations of proximate within-region activations difficult. Yet, MRI scanners with greater magnetic fields are becoming increasingly available, and have been shown to provide improved spatial resolution of fMRI activations [72]. Use of such scanners will facilitate the delineation of the specific circuitry subservient to different reward processes. Despite the caveat regarding spatial resolution, findings from animal and human studies are remarkably consistent. Both areas of research indicate that mesial and ventral frontal regions and the striatum are particularly important for reward processing. These studies also are consistent in showing that aspects of reward processing (expectancy, consumption) are mediated by at least partially discrete neural


395

circuitry. Lastly, it is increasingly evident that the role of dopamine is more complex than once was thought. Its specific role in motivation and reward will continue to be a topic of ongoing debate and investigation.

rewards such as money, but not to less salient rewards such as positive verbal feedback [86]. These findings generally are consistent with the hypothesized reward circuitry abnormality proposed by Volkow et al. [76].

Brain Reward Circuitry in Psychiatric Disorders

Schizophrenia

Several psychiatric disorders have attracted theoretical and empirical attention as potentially involving abnormal reward circuitry. These include substance use disorders, schizophrenia, pathologic gambling, major depressive disorder, and attention-deficit hyperactivity disorder.

Substance Use Disorders Several authors have proposed that brain reward circuitry abnormality may be central to both primary and secondary substance use disorders [73••,74–76]. For example, Volkow et al. [76] proposed that the reward value of substances of abuse and their associated cues (odors, paraphernalia) are increased, whereas the value of other rewards is decreased in patients with substance use disorders. This purportedly occurs as a result of conditioned learning and elevation of reward thresholds induced by the high level of stimulation provided by these substances. This is consistent with evidence that substances of abuse produce up to a five times greater increase in striatal dopamine than natural rewards. The result is that when abusers are exposed to substances of abuse or related cues, reward circuitry is overactivated, while activation of the neural circuitry involved in inhibitory control of the drive to seek and consume substances is diminished. A large body of research done in animals has confirmed that substances of abuse increase activity of dopaminergic neurons in the ventral tegmental area and promote the release of dopamine and several other neurochemicals implicated in the processing of reward, such as endogenous opioids [77]. The effects of substances of abuse on reward circuitry also have been shown in neuroimaging studies of humans. Consumption of methamphetamine [78] or alcohol [79] increases reward circuitry activation and dopamine release in the nucleus accumbens as measured by [11C] raclopride binding [80], in healthy adults. Exposure to alcohol or drug cues, or drug administration, results in heightened activation of reward circuitry in abusers or populations at risk for becoming abusers [81–83]. The latter finding is important because it is consistent with research showing that rats bred for alcohol preference, but without prior alcohol exposure, have a deficit in the mesolimbic pathway [84], but once exposed to alcohol, the mesolimbic pathway becomes hyperactive [85]. Therefore, abnormal reward circuitry may be a risk factor for substance use disorders rather than simply a consequence of use. Lastly, patients with substance use disorders, in contrast with patients without substance use disorders, show significant activation in reward circuitry to salient

Green et al. [75] have proposed, based on animal studies, that patients with schizophrenia have abnormal dopamine-mediated responses to rewarding stimuli. This is purported to arise secondary to neuropathologic abnormalities in reward circuitry including impaired signaling in the dopamine-mediated mesocorticolimbic pathways. This formulation additionally suggests that these patients may use substances of abuse because the substances transiently ameliorate the dysfunction in the reward circuitry, and thereby briefly increase the ability of these individuals to experience normal feelings of pleasure and satisfaction. Unfortunately, such substance use also substantially worsens the course of schizophrenia. Several areas of research have provided indirect evidence for a reward circuitry abnormality in schizophrenia. Patients with schizophrenia do not show the normal increase in P300 event-related potential amplitude to stimuli associated with monetary reward [87]. Structural neuroimaging studies have shown that patients with schizophrenia have abnormal volumes of the orbital and medial frontal cortex, anterior cingulate gyrus, nucleus accumbens, caudate nucleus, and amygdala. These are regions implicated in reward processing [88,89]. In addition, patients with schizophrenia who do not take antipsychotic medications have been observed to rate a pleasant odor as significantly less pleasurable than healthy adults, while not differing in their ratings of an unpleasant odor. An association between these patients’ subjective ratings of olfactory stimuli and an abnormal reward circuitry activation has been noted with PET [90]. This finding is in line with the known overlap in brain regions subserving olfactory and reward processing, such as the orbitofrontal cortex and nucleus accumbens [91]. Lastly, treatment of the symptoms of schizophrenia with clozapine, a medication that has been proposed to act in part by ameliorating a dysfunction in the mesocorticolimbic circuitry, significantly reduces alcohol and substance use in this population [92–94].

Pathologic Gambling Pathologic gambling is a disorder for which an abnormality of reward circuitry has clear face validity. Research suggests that dysregulated reward circuitry is important in this disorder. First, anticipation of winning money is associated with heightened arousal [95] and winning itself is highly rewarding to most people. Second, patients with focal ventromedial frontal cortex lesions show enhanced discounting of large delayed rewards in favor of smaller immediate rewards on an experimental decision making task that involves

396


gambling [96]. The same exaggerated decision-making strategy has been reported in patients with pathologic gambling [97], and it has been argued that the ventromedial frontal cortex and related structures affect decision making and risk assessment in these people through preferential responding to rewarding stimuli [98]. Third, there is evidence of a relationship between dopamine receptor gene polymorphisms associated with reward circuitry and pathologic gambling [99]. In addition to these empirical findings, evolutionary mismatch theory has been applied to help explain pathologic gambling [98]. This theory posits that there are behaviors and physiologic mechanisms that evolved through natural selection to facilitate obtaining the rewards necessary for survival of our species (food, sex), which can become maladaptive in the context of the recent and dramatic changes in our environment and lifestyle. The more persistent response shown by animals when reward is provided on a variable rather than fixed interval schedule may reflect such an adaptive mechanism in that it facilitates the learning of accurate predictors of reward [100]. Dopaminergic neuron activity is substantially increased when the receipt of reward is uncertain [100]. The risky, unpredictable nature of obtaining reward during gambling may result in heightened dopaminergic neuron activity that reinforces maladaptive risk-taking behavior in gambling [100].

Major Depressive Disorder Several authors have proposed that depression may involve abnormal reward circuitry [101–103], through decreased motivation to seek reward [101] or a reduced ability to experience reward [102]. In accordance with these hypotheses, animal models have been developed that use reduced response to obtain reward as an analogue of anhedonia in human depression [104]. Several animal studies have found that neonatal maternal separation in the rat can lead to behavioral changes in adulthood that resemble anhedonia, and to reduced intracranial electrical self-stimulation of reward circuitry [104]. Such studies suggest the possibility of early environmentally influenced changes in brain reward circuitry in human major depressive disorder. One study has directly investigated the integrity of brain reward circuitry in patients with major depressive disorder. A recent PET study showed that patients with major depression show underactivation of the caudate nucleus and ventromedial orbitofrontal cortex to positive and negative feedback during performance of a cognitive task, relative to healthy comparison subjects [105]. These findings were interpreted as suggesting that patients with depression have reduced expectation of receiving reward (caudate abnormality) and impaired ability to make stimulus-reward associations (orbitofrontal cortex abnormality). Additional indirect evidence for abnormal reward circuitry in depression is found in PET studies showing decreased metabolism in the subgenual cingulate region of the frontal cortex [106], an

area noted to be extensively connected to the ventral striatum (a region involved in motivation and affect).

Attention-Deficit/Hyperactivity Disorder Attention-deficit/hyperactivity disorder (ADHD) is being increasingly recognized as involving executive dysfunction, with an apparent abnormality of its underlying frontostriatal circuitry [52]. In particular, impaired inhibitory control has been argued to contribute to many of the myriad disturbances in cognition, behavior and emotional control seen in the disorder [107]. However, others have hypothesized that ADHD is attributable to impairment in reward processes such as delay aversion [108,109]. Specifically, individuals with ADHD have an abnormally high rate of discounting of delayed rewards relative to more immediate rewards. Considerable behavioral evidence has accumulated indicating that children with ADHD have difficulty waiting for rewards, and that this problem can be dissociated from inhibitory control deficits [109]. However, no study to date has directly evaluated neural circuitry activation during reward processing in ADHD.

Conclusions Understanding of the neural circuitry of reward has grown considerably since the first brain stimulation studies in rodents. Multiple brain regions and associated circuitry are involved, beyond the original emphasis on midbrain dopaminergic neurons. In addition, animal and human studies strongly support the idea that reward processing involves multiple components (anticipation, consumption) with at least partly distinct underlying neural circuitry. Additional investigations using more refined behavioral paradigms in animals and functional neuroimaging studies in humans will lead to more precise and detailed models relating subprocesses of reward to specific brain regions and circuitry. The growth in knowledge pertaining to reward circuitry has facilitated incorporation of this information into etiologic models of several psychiatric disorders. The empirical evidence for reward circuitry dysfunction in humans with psychiatric disorder is limited, and mostly indirect, at this time. However, the evidence available suggests that additional study is likely to significantly improve our understanding of the cause and treatment these disorders. In particular, several human and animal studies have raised the possibility that genetic and environmental factors may contribute to reward processing abnormalities in certain psychiatric disorders. Therefore, including evaluation of potential environmental contributors to reward circuitry dysfunction and genetic markers and functional neuroimaging of reward circuitry in developmental studies of individuals at high risk for psychiatric disorders, such as the offspring of parents with major depressive disorder or schizophrenia, likely will prove fruitful.


References and Recommended Reading

Papers of particular interest, published recently, have been highlighted as: • Of importance •• Of major importance Wise RA: Brain reward circuitry: insights from unsensed incentives. Neuron 2002, 36:229–240. 2. Schultz W: Multiple reward signals in the brain. Nature Rev Neurosci 2000, 1:199–207. 3. Kelley AE, Berridge KC: The neuroscience of natural rewards: relevance to addictive drugs. J Neurosci 2002, 22:3306–3311. 4. Gilbert DT, Driver-Linn E, Wilson TD: The trouble with Vronsky: impact bias in the forecasting of future affective states. In The Wisdom of Feelings. Edited by Feldman-Barrett L, Salovey P. New York: Guilford; 2002. 5. Laibson D: Golden eggs and hyperbolic discounting. Q J Econ 1997, 112:443–477. 6. Tversky A, Shafir E: Choice under conflict: the dynamics of deferred decision. Psychol Sci 1992, 3:358–361. 7. Fehr E: The economics of impatience. Nature 2002, 415:269–272. 8. Brosnan SF, De Waal FB: Monkeys reject unequal pay. Nature 2003, 425:297–299. 9. Wynne CD: Animal behaviour: fair refusal by capuchin monkeys. Nature 2004, 428:140. 10.•• Berridge KC, Robinson TE: Parsing reward. Trends Neurosci 2003, 26:507–513. The author defines positive affect and makes a distinction between conscious pleasure or “liking” and unconscious affective reaction. Methods for measuring the latter in animals are discussed and brain regions identified. 11. Olds J, Milner PM: Positive reinforcement produced by electrical stimulation of spetal area and other regions of rat brain. J Comp Physiol Psychol 1954, 47:419–427. 12. Wise RA, Rompre PP: Brain dopamine and reward. Annu Rev Psychol 1989, 40:191–225. 13. Wise RA: Addictive drugs and brain stimulation reward. Annu Rev Neurosci 1996, 19:319–340. 14.•• Haber SN: The primate basal ganglia: parallel and integrative networks. J Chem Neuroanat 2003, 26:317–330. The author provides a review of the anatomic organization of primate cortical-striatal pathways responsible for emotion and motivation, and goal-directed behavior. 15. McBride WJ, Murphy JM, Ikemoto S: Localization of brain reinforcement mechanisms: intracranial self-administration and intracranial place-conditioning studies. Behav Brain Res 1999, 101:129–152. 16. Heimer L: A new anatomical framework for neuropsychiatric disorders and drug abuse. Am J Psychiatry 2003, 160:1726–1739. 17.•• McGinty JF: Advancing from the ventral striatum to the extended amygdale. Ann N Y Acad Sci 1999, 877: 12–15. This volume provides an outstanding review of the anatomic organization of the ventral striatum and extended amygdala and their roles in behavior and psychiatric disorders. 18. Haber SN, McFarland NR: The concept of the ventral striatum in nonhuman primates. Ann N Y Acad Sci 1999, 877:33–48. 19. Deutch AY, Bourdelais AJ, Zahm DS: The nucleus accumbens core and shell: accumbal compartments and their functional attributes. In Limbic Motor Circuits and Neuropsychiatry. Edited by Kalivas PW, Barnes CD. Boca Raton, FL: CRC Press, Inc.; 1993:45–88. 20. Mogenson GJ, Jones DL, Yim CY: From motivation to action: functional interface between the limbic system and the motor system. Prog Neurobiol 1980, 14:69–97. 21. Corbit LH, Muir JL, Balleine BW: The role of the nucleus accumbens in instrumental conditioning: Evidence of a functional dissociation between accumbens core and shell. J Neurosci 2001, 21:3251–3260. 1.

397

Rebec GV, Grabner CP, Johnson M, et al.: Transient increases in catecholaminergic activity in medial prefrontal cortex and nucleus accumbens shell during novelty. Neuroscience 1997, 76:707–714. 23. Kelley AE: Ventral striatal control of appetitive motivation: role in ingestive behavior and reward-related learning. Neurosci Biobehav Rev 2004, 27:765–776. 24. Kelley AE, Bakshi VP, Haber SN, et al.: Opioid modulation of taste hedonics within the ventral striatum. Physiol Behav 2002, 76:365–377. 25. Cardinal RN, Parkinson JA, Hall J, et al.: Emotion and motivation: the role of the amygdala, ventral striatum, and prefrontal cortex. Neurosci Biobehav Rev 2002, 26:321–352. 26. Di Chiara G: Nucleus accumbens shell and core dopamine: differential role in behavior and addiction. Behav Brain Res 2002, 137:75–114. 27. Ongur D, Price JL: The organization of networks within the orbital and medial prefrontal cortex of rats, monkeys and humans. Cerebral Cortex 2000, 10:206–219. 28. Hikosaka K, Watanabe M: Delay activity of orbital and lateral prefrontal neurons of the monkey varying with different rewards. Cerebral Cortex 2000, 10:263–271. 29.•• Rolls ET: The Brain and Emotion. Oxford: Oxford University Press; 1999. (AU: Can chapter title, editor, and page range be provided?) This book is an exceptional source of information on research pertaining to reward processing in humans and animals, as well as an important theoretical framework for interpreting the myriad sources of evidence. 30. Holland PC, Gallagher M: Amygdala-frontal interactions and reward expectancy. Curr Opin Neurobiol 2004, 14:148–155. 31. Balleine BW, Dickinson A: Goal-directed instrumental action: contingency and incentive learning and their cortical substrates. Neuropharmacology 1998, 37:407–419. 32. Hadland KA, Rushworth MF, Gaffan D, et al.: The anterior cingulate and reward-guided selection of actions. J Neurophysiol 2003, 89:1161–1164. 33. Matsumoto K, Tanaka K: The role of the medial prefrontal cortex in achieving goals. Curr Opin Neurobiol 2004, 14:178–185. 34. Amaral DG, Price JL, Pitkanen A, et al.: Anatomical organization of the primate amygdaloid complex. In The Amygdala: Neurobiological Aspects of Emotion, Memory, and Mental Dysfunction. Edited by Aggleton JP. New York, NY: Wiley-Liss, Inc.; 1992:1–66. 35. Pitkanen A: Connectivity of the rat amygdaloid complex. In The Amygdala. Edited by Aggleton JP. Oxford: Oxford University Press; 2000:31–115. 36. Baxter MG, Murray EA: The amygdala and reward. Nature Rev Neurosci 2002, 3:563–573. 37. Everitt BJ, Cardinal RN, Parkinson JA, et al.: Appetitive behavior: impact of amygdala-dependent mechanisms of emotional learning. Ann N Y Acad Sci 2003, 985:233–250. 38. Gallagher M: The amygdala and associative learning. The Amygdala. Edited by Aggleton JP. Oxford: Oxford University Press; 2000:311–329. 39. Petrovich GD, Gallagher M: Amygdala subsystems and control of feeding behavior by learned cues. Ann N Y Acad Sci 2003, 985:251–262. 40. Domesick VB: Neuroanatomical organization of dopamine neurons in the ventral tegmental area. Ann N Y Acad Sci 1988, 537:10–26. 41. Fudge JL, Haber SN: Bed nucleus of the stria terminalis and extended amygdala inputs to dopamine subpopulations in primates. Neuroscience 2001, 104:807–827. 42. Tzschentke TM: Pharmacology and behavioral pharmacology of the mesocortical dopamine system. Prog Neurobiol 2001, 63:241–320. 43. Wise RA: Neurobiology of addiction. Curr Opin Neurobiol 1996, 6:243–251. 22.

398


44.• Spanagel R, Weiss F: The dopamine hypothesis of reward: past and current status. Trends in Neurosci 1999, 22:521–527. The authors reviewed current theories regarding the function of the dopamine-mediated mesolimbic pathway, and modulation of this pathway by drugs of abuse. 45. Schultz W: Getting formal with dopamine and reward. Neuron 2002, 36:241–263. 46. Redgrave P, Prescott TJ, Gurney K: Is the short-latency dopamine response too short to signal reward error? Trends Neurosci 1999, 22:146–51. 47. Berridge KC, Robinson TE: What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience? Brain Res Brain Res Rev 1998, 28:309–369. 48. Salamone JD, Correa M: Motivational views of reinforcement: implications for understanding the behavioral functions of nucleus accumbens dopamine. Behav Brain Res 2002, 137:3–25. 49. Ungless MA, Magill PJ, Bolam JP: Uniform inhibition of dopamine neurons in the ventral tegmental area by aversive stimuli. Science 2004, 303:2040–2042. 50. Cabeza R, Nyberg L: Imaging cognition II: An empirical review of 275 PET and fMRI studies. J Cog Neurosci 2000, 12:1–47. 51. McAllister TW, Sparling MB, Flashman LA, et al.: Neuroimaging findings in mild traumatic brain injury. J Clin Exp Neuropsychol 2001, 23:775–791. 52. Roth RM, Saykin AJ: Executive dysfunction in attentiondeficit/hyperactivity disorder: cognitive and neuroimaging findings. Psychiatr Clin North Am 2004, 27:83–96. 53. Delgado MR, Locke HM, Stenger VA, et al.: Dorsal striatum responses to reward and punishment: effects of valence and magnitude manipulations. Cog Affect Behav Neurosci 2003, 3:27–38. 54. Knutson B, Fong GW, Bennett SM, et al.: A region of mesial prefrontal cortex tracks monetarily rewarding outcomes: characterization with rapid event-related fMRI. Neuroimage 2003, 18:263–272. 55. Hommer DW, Knutson B, Fong GW, et al.: Amygdalar recruitment during anticipation of monetary rewards: an eventrelated fMRI study. Ann NY Acad Sci 2003, 985:476–478. 56. Knutson B, Fong GW, Adams CM, et al.: Dissociation of reward anticipation and outcome with event-related fMRI. Neuroreport 2001, 12:3683–3687. 57. Rogers RD, Ramnani N, Mackay C, et al.: Distinct portions of anterior cingulate cortex and mediual prefrontal cortex are activated by reward processing in seperable phases of decisionmaking cognition. Biol Psychiatry 2004, 55:594–602. 58. O'Doherty J, Kringelbach ML, Rolls ET, et al.: Abstract reward and punishment representations in the human orbitofrontal cortex. Nature Neurosci 2001, 4:95–102. 59. Berns GS, McClure SM, Pagnoni G, et al.: Predictability modulates human brain response to reward. J Neurosci 2001, 21:2793–2798. 60.• Pagnoni G, Zink CF, Montague PR, et al.: Activity in human ventral striatum locked to errors of reward prediction. Nature Neurosci 2002, 5:97–98. This study used fMRI to specify the role of ventral striatum in reward processing. It is the first human functional neuroimaging study to show that the ventral striatum specifically responds to errors in predicting receipt of reward, consistent with that observed in animal studies. 61. Small DM, Zatorre RJ, Dagher A, et al.: Changes in brain activity related to eating chocolate: from pleasure to aversion. Brain 2001, 124:1720–1733. 62. Blood AJ, Zatorre RJ: Intensely pleasurable responses to music correlate with activity in brain regions implicated in reward and emotion. Proc Natl Acad Sci USA 2001, 98:11818–11823. 63. Aharon I, Etcoff N, Ariely D, et al.: Beautiful faces have variable reward value: fMRI and behavioral evidence. Neuron 2001, 32:537–551. 64. Mobbs D, Greicius MD, Abdel-Azim E, et al.: Humor modulates the mesolimbic reward centers. Neuron 2003, 40:1041–1048. 65. Holstege G, Georgiadis JR, Paans AM, et al.: Brain activation during human male ejaculation. J Neurosci 2003, 23:9185–9193.

Pappata S, Dehaene S, Poline JB, et al.: In vivo detection of striatal dopamine release during reward: a PET study with [(11)C]raclopride and a single dynamic scan approach. Neuroimage 2002, 16:1015–1027. 67. Zald DH, Boileau I, El-Dearedy W, et al.: Dopamine transmission in the human striatum during monetary reward tasks. J Neurosci 2004, 24:4105–4112. 68. de la Fuente R, Phillips AG, Zamburlini M, et al.: Dopamine release in human ventral striatum and expectation of reward. Behav Brain Res 2002, 136:359–363. 69. Becerra L, Breiter HC, Wise R, et al.: Reward circuitry activation by noxious thermal stimuli. Neuron 2001, 32:927–946. 70. Jensen J, McIntosh AR, Crawley AP, et al.: Direct activation of the ventral striatum in anticipation of aversive stimuli. Neuron 2003, 40:1251–1257. 71. Horvitz JC: Mesolimbocortical and nigrostriatal dopamine responses to salient non-reward events. Neuroscience 2000, 96:651–656. 72. Pfeuffer J, van de Moortele PF, Yacoub E, et al.: Zoomed functional imaging in the human brain at 7 Tesla with simultaneous high spatial and high temporal resolution. Neuroimage 2002, 17:272–286. 73.•• Blum K, Braverman ER, Holder JM, et al.: Reward deficiency syndrome: a biogenetic model for the diagnosis and treatment of impulsive, addictive, and compulsive behaviors. J Psychoactive Drugs 2000, 32:1–112. This paper provides an excellent outline the theoretical importance of abnormal reward circuitry for several psychiatric disorders first proposed several years ago. 74. Chambers RA, Krystal JH, Self DW: A neurobiological basis for substance abuse comorbidity in schizophrenia. Biol Psychiatry 2001, 50:71–83. 75. Green AI, Zimmet SV, Strous RD, et al.: Clozapine for comorbid substance use disorder and schizophrenia: do patients with schizophrenia have a reward-deficiency syndrome that can be ameliorated by clozapine? Harvard Rev Psychiatry 1999, 6:287–296. 76. Volkow ND, Fowler JS, Wang GJ: The addicted human brain: insights from imaging studies. J Clin Invest 2003, 111:1444–1451. 77. Wise RA: Drug-activation of brain reward pathways. Drug Alcohol Depend 1998, 51:13–22. 78. Vollm BA, De Araujo IE, Rolls ET, et al.: Methamphetamine activates reward circuitry in drug naive human subjects. Neuropsychopharmacology 2004, In press. 79. Ingvar M, Ghatan PH, Wirsen-Meurling A, et al.: Alcohol activates the cerebral reward system in man. J Stud Alcohol 1998, 59:258–269. 80. Boileau I, Assaad JM, Pihl RO, et al.: Alcohol promotes dopamine release in the human nucleus accumbens. Synapse 2003, 49:226–231. 81. Breiter HC, Gollub RL, Weisskoff RM, et al.: Acute effects of cocaine on human brain activity and emotion. Neuron 1997, 19:591–611. 82. Kareken DA, Claus ED, Sabri M, et al.: Alcohol-related olfactory cues activate the nucleus accumbens and ventral tegmental area in high-risk drinkers: Preliminary findings. Alcohol Clin Exp Res 2004, 28:550–557. 83. Tapert SF, Cheung EH, Brown GG, et al.: Neural response to alcohol stimuli in adolescents with alcohol use disorder. Arch Gen Psychiatry 2003, 60:727–735. 84. Zhou FC, Zhang JK, Lumeng L, et al.: Mesolimbic dopamine system in alcohol-preferring rats. Alcohol 1995, 12:403–412. 85. Thielen RJ, Engleman EA, Rodd ZA, et al.: Ethanol drinking and deprivation alter dopaminergic and serotonergic function in the nucleus accumbens of alcohol-preferring rats. J Pharmacol Exp Ther 2004, 309:216–225. 86. Martin-Soelch C, Chevalley AF, Kunig G, et al.: Changes in rewardinduced brain activation in opiate addicts. Eur J Neurosci 2001, 14:1360–1368. 87. Brecher M, Begleiter H: Event-related brain potentials to highincentive stimuli in unmedicated schizophrenic patients. Biol Psychiatry 1983, 18:661–674. 66.


88. 89. 90.

91. 92. 93. 94. 95. 96. 97. 98. 99.

Lauer M, Senitz D, Beckmann H: Increased volume of the nucleus accumbens in schizophrenia. J Neural Transm 2001, 108:645–660. Shenton ME, Dickey CC, Frumin M, et al.: A review of MRI findings in schizophrenia. Schizophr Res 2001, 49:1–52. Crespo-Facorro B, Paradiso S, Andreasen NC, et al.: Neural mechanisms of anhedonia in schizophrenia: a PET study of response to unpleasant and pleasant odors. JAMA 2001, 286:427–435. Serby MJ, Chobor KL: The Science of Olfaction. New York, Springer, 1992. Drake RE, Xie H, McHugo GJ, et al.: The effects of clozapine on alcohol and drug use disorders among patients with schizophrenia. Schizophr Bull 2000, 26:441–449. Green AI, Salomon MS, Brenner MJ, et al.: Treatment of schizophrenia and comorbid substance use disorder. Curr Drug Targ 2002, 1:129–139. Green AI, Burgess ES, Dawson R, et al.: Alcohol and cannabis use in schizophrenia: effects of clozapine vs. risperidone. Schizophr Res 2003, 60:81–85. Ladouceur R, Sevigny S, Blaszczynski A, et al.: Video lottery: winning expectancies and arousal. Addiction 2003, 98:733–738. Bechara A: Risky business: emotion, decision-making, and addiction. J Gamb Stud 2003, 19:23–51. Petry NM: Pathological gamblers, with and without substance use disorders, discount delayed rewards at high rates. J Abnorm Psychol 2001, 110:482–487. Spinella M: Evolutionary mismatch, neural reward circuits, and pathological gambling. Int J Neurosci 2003, 113:503–512. Ibanez A, Blanco C, de Castro IP, et al.: Genetics of pathological gambling. J Gamb Stud 2003, 19:11–22.

399

100. Fiorillo CD, Tobler PN, Schultz W: Discrete coding of reward probability and incertainty by dopamine neurons. Science 2003, 299:1898–1902. 101. Layne C: Motivational deficits in depression: people's expectations x outcomes' impacts. J Clin Psychol 1980, 36:647–652. 102. Lewinsohn PM, Youngren MA, Grosscup SJ: Reinforcement and depression. In The Psychobiology of Depressive Disorders. Edited by Depue RA. New York: Academic Press; 1979:291–316. 103. Naranjo CA, Tremblay LK, Busto UE: The role of the brain reward system in depression. Prog Neuropsychopharmacol Biol Psych 2001, 25:781–823. 104. Matthews K, Robbins TW: Early experience as a determinant of adult behavioural responses to reward: the effects of repeated maternal separation in the rat. Neurosci Biobehav Rev 2003, 27:45–55. 105. Elliott R, Sahakian BJ, Michael A, et al.: Abnormal neural response to feedback on planning and guessing tasks in patients with unipolar depression. Psychol Med 1998, 28:559–571. 106. Drevets WC: Neuroimaging and neuropathological studies of depression: implications for the cognitive-emotional features of mood disorders. Curr Opin Neurobiol 2001, 11:240–249. 107. Barkley RA: Behavioral inhibition, sustained attention, and executive functions: constructing a unifying theory of ADHD. Psychol Bull 1997, 121:65–94. 108. Johansen EB, Aase H, Meyer A, et al.: Attention-deficit/hyperactivity disorder (ADHD) behaviour explained by dysfunctioning reinforcement and extinction processes. Behav Brain Res 2002, 130:37–45. 109. Sonuga-Barke EJ: The dual pathway model of AD/HD: an elaboration of neuro-developmental characteristics. Neurosci Biobehav Rev 2003, 27:593–604.