Separate Valuation Subsystems for Delay and Effort Decision Costs

Decision making consists of choosing among available options on the basis of a valuation of their potential costs and benefits. Most theoretical models of decision making in behavioral economics, psychology, and computer science propose that the desirability of outcomes expected from alternative options can be quantified by utility functions. These utility functions allow a decision maker to assign subjective values to each option under consideration by weighting the likely benefits and costs resulting from an action and to select the one with the highest subjective value. Here, we used model-based neuroimaging to test whether the human brain uses separate valuation systems for rewards (erotic stimuli) associated with different types of costs, namely, delay and effort. We show that humans devalue rewards associated with physical effort in a strikingly similar fashion to those they devalue that are associated with delays, and that a single computational model derived from economics theory can account for the behavior observed in both delay discounting and effort discounting. However, our neuroimaging data reveal that the human brain uses distinct valuation subsystems for different types of costs, reflecting in opposite fashion delayed reward and future energetic expenses. The ventral striatum and the ventromedial prefrontal cortex represent the increasing subjective value of delayed rewards, whereas a distinct network, composed of the anterior cingulate cortex and the anterior insula, represent the decreasing value of the effortful option, coding the expected expense of energy. Together, these data demonstrate that the valuation processes underlying different types of costs can be fractionated at the cerebral level.

[1]  G. Loewenstein,et al.  Time Discounting and Time Preference: A Critical Review , 2002 .

[2]  G. Ainslie Specious reward: a behavioral theory of impulsiveness and impulse control. , 1975, Psychological bulletin.

[3]  T. Shallice,et al.  Human cingulate cortex and autonomic control: converging neuroimaging and clinical evidence. , 2003, Brain : a journal of neurology.

[4]  Brian Knutson,et al.  Dissociable neural representations of future reward magnitude and delay during temporal discounting , 2009, NeuroImage.

[5]  Kenji Doya,et al.  Humans Can Adopt Optimal Discounting Strategy under Real-Time Constraints , 2006, PLoS Comput. Biol..

[6]  Philippe Adam,et al.  Les comportements sexuels en France: Edited by A. Spira, N. Bajos and le groupe ACSF. La documentation française, Paris, 1993, 352 pp., FF 100. ISBN 2-11-002904-8 , 1995 .

[7]  M. Walton,et al.  Separate neural pathways process different decision costs , 2006, Nature Neuroscience.

[8]  Matthew F. S. Rushworth,et al.  Weighing up the benefits of work: Behavioral and neural analyses of effort-related decision making , 2006, Neural Networks.

[9]  Timothy Edward John Behrens,et al.  Effort-Based Cost–Benefit Valuation and the Human Brain , 2009, The Journal of Neuroscience.

[10]  W. Schultz,et al.  Short-Term Temporal Discounting of Reward Value in Human Ventral Striatum , 2009, Journal of neurophysiology.

[11]  Colin Camerer,et al.  Self-control in decision-making involves modulation of the vmPFC valuation system , 2009, NeuroImage.

[12]  Matthew T. Kaufman,et al.  Distributed Neural Representation of Expected Value , 2005, The Journal of Neuroscience.

[13]  C. Büchel,et al.  Overlapping and Distinct Neural Systems Code for Subjective Value during Intertemporal and Risky Decision Making , 2009, The Journal of Neuroscience.

[14]  J. Deakin,et al.  Effects of lesions of the orbitofrontal cortex on sensitivity to delayed and probabilistic reinforcement , 2002, Psychopharmacology.

[15]  Benjamin Y Hayden,et al.  Economic principles motivating social attention in humans , 2007, Proceedings of the Royal Society B: Biological Sciences.

[16]  Brian Knutson,et al.  Dissociation of reward anticipation and outcome with event-related fMRI , 2001, Neuroreport.

[17]  Faruk Gul,et al.  The Case for Mindless Economics , 2005 .

[18]  Timothy Edward John Behrens,et al.  Contrasting roles for cingulate and orbitofrontal cortex in decisions and social behaviour , 2007, Trends in Cognitive Sciences.

[19]  C. Büchel,et al.  Neural representations of subjective reward value , 2010, Behavioural Brain Research.

[20]  A. Ǻgmo,et al.  Sexual motivation—an inquiry into events determining the occurrence of sexual behavior , 1999, Behavioural Brain Research.

[21]  Karl J. Friston,et al.  Encoding of Marginal Utility across Time in the Human Brain , 2009, The Journal of Neuroscience.

[22]  Samuel M. McClure,et al.  Separate Neural Systems Value Immediate and Delayed Monetary Rewards , 2004, Science.

[23]  S. Floresco,et al.  Dopaminergic and Glutamatergic Regulation of Effort- and Delay-Based Decision Making , 2008, Neuropsychopharmacology.

[24]  Saori C. Tanaka,et al.  Prediction of immediate and future rewards differentially recruits cortico-basal ganglia loops , 2004, Nature Neuroscience.

[25]  T. Robbins,et al.  Impulsive Choice Induced in Rats by Lesions of the Nucleus Accumbens Core , 2001, Science.

[26]  Colin Camerer,et al.  A framework for studying the neurobiology of value-based decision making , 2008, Nature Reviews Neuroscience.

[27]  M. Rushworth,et al.  Behavioral / Systems / Cognitive Connectivity-Based Parcellation of Human Cingulate Cortex and Its Relation to Functional Specialization , 2008 .

[28]  B. Kolachana,et al.  Variation in dopamine genes influences responsivity of the human reward system , 2009, Proceedings of the National Academy of Sciences.

[29]  B. Richmond,et al.  Anterior Cingulate: Single Neuronal Signals Related to Degree of Reward Expectancy , 2002, Science.

[30]  J. Gläscher,et al.  Dissociable Systems for Gain- and Loss-Related Value Predictions and Errors of Prediction in the Human Brain , 2006, The Journal of Neuroscience.

[31]  J. Muir,et al.  The cerebral cortex of the rat and visual attentional function: dissociable effects of mediofrontal, cingulate, anterior dorsolateral, and parietal cortex lesions on a five-choice serial reaction time task. , 1996, Cerebral cortex.

[32]  A. Tversky,et al.  Prospect theory: an analysis of decision under risk — Source link , 2007 .

[33]  S. Hamann,et al.  Men and women differ in amygdala response to visual sexual stimuli , 2004, Nature Neuroscience.

[34]  K. Berman,et al.  Cerebral Cortex doi:10.1093/cercor/bhj004 Neural Coding of Distinct Statistical Properties of Reward Information in Humans , 2005 .

[35]  Martin P Paulus,et al.  Decision-Making Dysfunctions in Psychiatry—Altered Homeostatic Processing? , 2007, Science.

[36]  J. Dreher Decomposing brain signals involved in value-based decision making , 2009 .

[37]  T. Robbins,et al.  Disconnection of the anterior cingulate cortex and nucleus accumbens core impairs Pavlovian approach behavior: further evidence for limbic cortical-ventral striatopallidal systems. , 2000, Behavioral neuroscience.

[38]  T. Robbins,et al.  Contrasting Roles of Basolateral Amygdala and Orbitofrontal Cortex in Impulsive Choice , 2004, The Journal of Neuroscience.

[39]  S. Floresco,et al.  Amygdala-prefrontal cortical circuitry regulates effort-based decision making. , 2006, Cerebral cortex.

[40]  Camelia M. Kuhnen,et al.  The Neural Basis of Financial Risk Taking , 2005, Neuron.

[41]  G. McCarthy,et al.  Decisions under Uncertainty: Probabilistic Context Influences Activation of Prefrontal and Parietal Cortices , 2005, The Journal of Neuroscience.

[42]  A. Meyer-Lindenberg,et al.  Age-related changes in midbrain dopaminergic regulation of the human reward system , 2008, Proceedings of the National Academy of Sciences.

[43]  P. Glimcher,et al.  The neural correlates of subjective value during intertemporal choice , 2007, Nature Neuroscience.

[44]  B. Hayden,et al.  Distinct Value Signals in Anterior and Posterior Ventromedial Prefrontal Cortex , 2010, The Journal of Neuroscience.

[45]  J. Patton,et al.  Factor structure of the Barratt impulsiveness scale. , 1995, Journal of clinical psychology.

[46]  S. Quartz,et al.  Neural Differentiation of Expected Reward and Risk in Human Subcortical Structures , 2006, Neuron.

[47]  Timothy E. J. Behrens,et al.  Frontal Cortex Subregions Play Distinct Roles in Choices between Actions and Stimuli , 2008, The Journal of Neuroscience.

[48]  Martin P. Paulus,et al.  Time and decision making: differential contribution of the posterior insular cortex and the striatum during a delay discounting task , 2007, Experimental Brain Research.

[49]  T. Bussey,et al.  Dissociable effects of cingulate and medial frontal cortex lesions on stimulus-reward learning using a novel Pavlovian autoshaping procedure for the rat: implications for the neurobiology of emotion. , 1997, Behavioral neuroscience.

[50]  Nikolaus Weiskopf,et al.  Optimal EPI parameters for reduction of susceptibility-induced BOLD sensitivity losses: A whole-brain analysis at 3 T and 1.5 T , 2006, NeuroImage.

[51]  M. Walton,et al.  Comparing the role of the anterior cingulate cortex and 6‐hydroxydopamine nucleus accumbens lesions on operant effort‐based decision making , 2009, The European journal of neuroscience.

[52]  Daniella J. Furman,et al.  Menstrual cycle phase modulates reward-related neural function in women , 2007, Proceedings of the National Academy of Sciences.

[53]  Samuel M. McClure,et al.  Time Discounting for Primary Rewards , 2007, The Journal of Neuroscience.

[54]  J. Dreher,et al.  The Architecture of Reward Value Coding in the Human Orbitofrontal Cortex , 2010, The Journal of Neuroscience.

[55]  Mark W Woolrich,et al.  Associative learning of social value , 2008, Nature.

[56]  Alfredo Braunstein,et al.  In the works , 1991, Nature.

[57]  J. Hirsch,et al.  A Neural Representation of Categorization Uncertainty in the Human Brain , 2006, Neuron.

[58]  S. Floresco,et al.  Fundamental Contribution by the Basolateral Amygdala to Different Forms of Decision Making , 2009, The Journal of Neuroscience.

[59]  J. O'Doherty,et al.  Orbitofrontal Cortex Encodes Willingness to Pay in Everyday Economic Transactions , 2007, The Journal of Neuroscience.

[60]  Emily Franck Hoon,et al.  Sexual Arousability Inventory (SAI) and Sexual Arousability Inventory - Expanded (SAI-E) 1986 , 2007 .

[61]  Jonathan D. Wallis,et al.  Neurons in the Frontal Lobe Encode the Value of Multiple Decision Variables , 2009, Journal of Cognitive Neuroscience.

[62]  Jeffrey R. Stevens,et al.  Will Travel for Food: Spatial Discounting in Two New World Monkeys , 2005, Current Biology.

[63]  Matthew F S Rushworth,et al.  Functional Specialization within Medial Frontal Cortex of the Anterior Cingulate for Evaluating Effort-Related Decisions , 2003, The Journal of Neuroscience.

[64]  M. Walton,et al.  Dissociable cost and benefit encoding of future rewards by mesolimbic dopamine , 2009, Nature Neuroscience.

[65]  M. Yücel,et al.  Dopamine modulates neural networks involved in effort-based decision-making , 2009, Neuroscience & Biobehavioral Reviews.

[66]  P. Dayan,et al.  Tonic dopamine: opportunity costs and the control of response vigor , 2007, Psychopharmacology.