news and views
The value of competition in the rat race Mark E Walton & Mathieu Baudonnat
According to a poll published in a British newspaper in 2009, the average person in the UK will spend nearly 6 months of his or her life standing in a queue. Such ordered queuing appears to be a common human solution to the problem of allocating scarce resources to avoid unnecessary competition and aggression. In the absence of equivalent rules of etiquette, animals daily face decisions about when it is worth vying with other individuals to procure their needs. These choices require an assessment based on not just the potential future benefits, but also the likely energetic and physical costs. Although several studies have investigated the neural underpinnings of cost-benefit decisions in isolated individuals 1–4, little is known about this fundamental question of how the brain represents competitive effort—the costs of competing with others for a limited resource. Into this gap comes a study by Hillman and Bilkey5 in this month’s Nature Neuroscience showing that competing with another rat for food modulates a rat’s choices and the activity of neurons in the anterior cingulate cortex (ACC) in the frontal lobe (Fig. 1a). The basic task design was pleasingly ecological and straightforward (Fig. 1b). On every trial, a rat could choose between two options in a maze. Each arm led to a reward zone containing some food and another rat confined behind a wire-mesh barrier. If a reward zone was noncompetitive, the rats were kept apart and each received an equal amount of food. If, however, it was competitive, a small partition was raised, allowing the animals to forage head to head. By varying the size of the reward and the degree of competition in the two arms of the maze both across and within sessions, Hillman and Bilkey5 found that their rats were sensitive to the cost of competitive effort. Of particular interest, when the costbenefit parameters were objectively equal, the rats would adjust their choices to take
Mark E. Walton and Mathieu Baudonnat are in the Department of Experimental Psychology, University of Oxford, Oxford, UK. e-mail:
[email protected]
1182
into consideration the dominance or degree of hunger of a competitor. Such patterns of behavior fit a wealth of ecological studies on competitive foraging. What sets this study apart is that the authors also monitored neuronal activity in the ACC as a function of the types of decisions that the animals faced and the options that they selected. Several previous studies have suggested that the ACC might help to enable animals to choose to overcome effort constraints and persist with a course of action to achieve a future goal1,6. Hillman and Bilkey5 extend this definition to include the effort involved in social competition. They found that most ACC cells reflected the net value of a competitive course of action (that is, the
benefits of gaining the reward discounted by the cost of obtaining it), with higher mean firing rates during selection and anticipation of preferred competitive options and lower firing rates for rejected ones. Moreover, in all of these conditions, the difference in average firing rates between the two options positively correlated with the likelihood of choosing the competitive option, suggesting a link between relative ACC activity and foraging decisions. Even in such a straightforward task, the key issue is what these signals might reflect. ACC in rats and primates has connections with limbic, motoric and autonomic structures7,8, so cell activity could conceivably be related to the value of the chosen action, the specific
a ACC
b Competitive
Noncompetitive
To vie or not to vie... Marina Corral
npg
© 2012 Nature America, Inc. All rights reserved.
Animals often must vie with others for scarce resources, such as food, water and mates. Deciding when to engage and when to avoid such contests might critically depend on the activity of anterior cingulate cortex neurons.
Figure 1 Rodent ACC and social competition. (a) Sagittal view of a rat brain, depicting the location of the ACC, from which neuronal recordings took place. (b) Task design. In the standard sessions, the experimental rat had to choose between competing with another rat for a large amount of food and gaining a smaller amount of food without the need to compete. The amount of food, requirement to compete and status of the competitor rat were then systematically varied across different test sessions while the activity of ACC cells was monitored.
volume 15 | number 9 | september 2012 nature neuroscience
npg
© 2012 Nature America, Inc. All rights reserved.
news and views motor program or the arousal associated with the social encounter. The last two of these possibilities respectively can be ruled out because the ACC firing rate tracked the net value as preference switched from the left to right goal arms and as it switched from the competitive to the noncompetitive option. However, the ACC neurons also do not simply represent the subjective value of the alternative actions, as there were no differences when neither choice led to a competitive interaction. This is similar to results of the authors’ previous study involving scaling barriers to gain reward, where, again, most ACC neurons only signaled net value when the barrier was present in at least one of the two goal arms9. Hillman and Bilkey5 therefore conclude that the ACC signals the value of a chosen option if and only if a choice requires integration of effort expenditure and eventual reward. This hypothesis leads to the clear prediction that disruption to this region should impair decision making in any condition in which at least one of the options requires this type of cost-benefit integration. At first glance, this does not seem to fit the available data. ACC lesioned rats do not exhibit any noticeable deficit in a differentially rewarded T-maze task in conditions in which rats have to climb over a barrier to gain either reward1. However, there is an important difference between the lesion studies (for example, ref. 1) and the tasks used by Hillman and Bilkey5,9. The former investigated animals trained with a particular cost-benefit ratio before ACC disruption, whereas the authors only introduced their manipulations on the first day of recording and took care to alter the competition and reward contingencies from day to day. Thus, one possibility is that the ACC may have a transitory role in effort-related decisions at times at which there has been a change in the environment that might require an update in an animal’s decision policy10,11. This is consistent with at least some of the lesion literature, which shows that ACC damage has its most marked effect when it is necessary to reassess costbenefit contingencies1. Such a role for ACC might also help to explain why firing rate changes were found not only in the middle arm of the maze, before a decision, but also while approaching and occupying the reward zone and even sometimes while returning to the middle
arm, well after an action plan had been implemented. Although the variability in firing locations across cells may reflect distinct populations of ACC neurons performing different computations, it might be that some of the same ACC cells detect changes in costbenefit contingencies and then bias animals to gain information about the environment to further update these estimates (compare ref. 12). It is notable that, at least in some conditions, the firing rates at the reward zone seem to reflect the new net value faster than those in the middle arm. Although there is an evident connection between ACC neuronal activity and the value of social competition, the precise cost in this study is surprisingly difficult to pin down. On the basis of a range of evidence, Hillman and Bilkey5 suggest that the key factor relates to effort, with competitive foraging, like barrier climbing, requiring more energy expenditure and therefore being costly. Although this makes intuitive sense, the exact parameters that define what is effortful are not clear. For instance, how much energy, in terms of calories expended, does it take to scale a barrier or compete with another rat as compared with running along an unoccupied arm? Presumably moving around a maze always involves some energy, yet ACC net value effort encoding disappeared when the rats did not need to consider competition or barrier climbing costs. ACC has been connected to both the physical effort and mental effort13 associated with actions, but is this related to brain energetics or to a more abstract similarity based around both mental and physical work requiring persistence through multiple steps to achieve a goal? Given that ACC has previously been linked with social valuation, as well as with effortrelated decisions14, there is a natural synergy between the ideas of competitive foraging and effort. Nonetheless, it is not clear whether the cost in this study relates directly to the social nature of the competition or merely to the uncertainty of the food yield. Evidence in monkeys and humans suggests specialization within the ACC, with information about the value of one’s own actions being more strongly represented in dorsal ACC, in the prominent sulcus present in primates, whereas the value of social information often appears to be represented in a more ventral ACC gyral region15. Whether any similar division is present in the rodent
nature neuroscience volume 15 | number 9 | september 2012
brain, which lacks an ACC sulcus and possibly some equivalent motor regions found in primate dorsal ACC, remains unknown. Such information might not only inform our understanding of the homologies between rodent and primate ACC, but might also give important clues about their decisionmaking strategies. There may be a fundamental difference between making choices based solely on an assessment of the utility of one’s own actions, with utility defined by, among other factors, effort and social competition, and making choices on the basis of separate assessments of the utility of one’s own choices relative to the choices of others in your environment, as primates seem to be able to do. These are all questions for future studies. What is exciting about the approach taken here and in several other recent studies10,11 is the demonstration that fundamental elements of social foraging behavior can now be linked directly to brain activity. Researchers will be queuing up to tackle these issues. ACKNOWLEDGMENTS This work was supported by the Wellcome Trust. We thank K. Hillman and M. Rushworth for discussions. COMPETING FINANCIAL INTERESTS The authors declare no competing financial interests. 1. Rudebeck, P.H., Walton, M.E., Smyth, A.N., Bannerman, D.M. & Rushworth, M.F. Nat. Neurosci. 9, 1161–1168 (2006). 2. Gan, J.O., Walton, M.E. & Phillips, P.E. Nat. Neurosci. 13, 25–27 (2010). 3. Amemori, K. & Graybiel, A.M. Nat. Neurosci. 15, 776–785 (2012). 4. Prévost, C., Pessiglione, M., Metereau, E., Clery-Melin, M.L. & Dreher, J.C. J. Neurosci. 30, 14080–14090 (2010). 5. Hillman, K.L. & Bilkey, D.K. Nat. Neurosci. 15, 1290–1297 (2012). 6. Walton, M.E., Kennerley, S.W., Bannerman, D.M., Phillips, P.E. & Rushworth, M.F. Neural Netw. 19, 1302–1314 (2006). 7. Gabbott, P.L., Warner, T.A., Jays, P.R., Salway, P. & Busby, S.J. J. Comp. Neurol. 492, 145–177 (2005). 8. Van Hoesen, G.W., Morecraft, R.J. & Vogt, B.A. in Neurobiology of Cingulate Cortex and Limbic Thalamus: a Comprehensive Handbook (eds. Vogt, B.A. & Gabriel, M.) 249–283 (Birkhauser, Boston, 1993). 9. Hillman, K.L. & Bilkey, D.K. J. Neurosci. 30, 7705–7713 (2010). 10. Kolling, N., Behrens, T.E., Mars, R.B. & Rushworth, M.F. Science 336, 95–98 (2012). 11. Hayden, B.Y., Pearson, J.M. & Platt, M.L. Nat. Neurosci. 14, 933–939 (2011). 12. Kennerley, S.W., Behrens, T.E. & Wallis, J.D. Nat. Neurosci. 14, 1581–1589 (2011). 13. Botvinick, M.M., Huffstetler, S. & McGuire, J.T. Cogn. Affect. Behav. Neurosci. 9, 16–27 (2009). 14. Rudebeck, P.H., Bannerman, D.M. & Rushworth, M.F. Cogn. Affect. Behav. Neurosci. 8, 485–497 (2008). 15. Behrens, T.E., Hunt, L.T. & Rushworth, M.F. Science 324, 1160–1164 (2009).
1183
