Effort-Based Cost–Benefit Valuation and the Human Brain

In both the wild and the laboratory, animals' preferences for one course of action over another reflect not just reward expectations but also the cost in terms of effort that must be invested in pursuing the course of action. The ventral striatum and dorsal anterior cingulate cortex (ACCd) are implicated in the making of cost–benefit decisions in the rat, but there is little information about how effort costs are processed and influence calculations of expected net value in other mammals including humans. We performed a functional magnetic resonance imaging study to determine whether and where activity in the human brain was available to guide effort-based cost–benefit valuation. Subjects were scanned while they performed a series of effortful actions to obtain secondary reinforcers. At the beginning of each trial, subjects were presented with one of eight different visual cues that they had learned indicated how much effort the course of action would entail and how much reward could be expected at its completion. Cue-locked activity in the ventral striatum and midbrain reflected the net value of the course of action, signaling the expected amount of reward discounted by the amount of effort to be invested. Activity in ACCd also reflected the interaction of both expected reward and effort costs. Posterior orbitofrontal and insular activity, however, only reflected the expected reward magnitude. The ventral striatum and anterior cingulate cortex may be the substrate of effort-based cost–benefit valuation in primates as well as in rats.

[1]  L. S. Kogan Review of Principles of Behavior. , 1943 .

[2]  B. Skinner,et al.  Principles of Behavior , 1944 .

[3]  E. Charnov Optimal foraging, the marginal value theorem. , 1976, Theoretical population biology.

[4]  Alex Kacelnik,et al.  Optimal foraging and timing processes in the starling, Sturnus vulgaris: effect of inter-capture interval , 1992, Animal Behaviour.

[5]  Alan C. Evans,et al.  A Three-Dimensional Statistical Analysis for CBF Activation Studies in Human Brain , 1992, Journal of cerebral blood flow and metabolism : official journal of the International Society of Cerebral Blood Flow and Metabolism.

[6]  B. Vogt,et al.  Connections of the Monkey Cingulate Cortex , 1993 .

[7]  S. Haber,et al.  Primate cingulostriatal projection: Limbic striatal versus sensorimotor striatal input , 1994, The Journal of comparative neurology.

[8]  D. Collins,et al.  Automatic 3D Intersubject Registration of MR Volumetric Data in Standardized Talairach Space , 1994, Journal of computer assisted tomography.

[9]  J. Salamone,et al.  Anhedonia or anergia? Effects of haloperidol and nucleus accumbens dopamine depletion on instrumental response selection in a T-maze cost/benefit procedure , 1994, Behavioural Brain Research.

[10]  T. Preuss Do Rats Have Prefrontal Cortex? The Rose-Woolsey-Akert Program Reconsidered , 1995, Journal of Cognitive Neuroscience.

[11]  Michael S. Gazzaniga,et al.  On Neural Circuits and Cognition , 1995, Neural Computation.

[12]  A. Kacelnik,et al.  Accuracy of memory for amount in the foraging starling, Sturnus vulgaris , 1995, Animal Behaviour.

[13]  J Tanji,et al.  Overlapping corticostriatal projections from the supplementary motor area and the primary motor cortex in the macaque monkey: An anterograde double labeling study , 1996, The Journal of comparative neurology.

[14]  B. Richmond,et al.  Neuronal Signals in the Monkey Ventral Striatum Related to Progress through a Predictable Series of Trials , 1998, The Journal of Neuroscience.

[15]  M. D’Esposito,et al.  The Inferential Impact of Global Signal Covariates in Functional Neuroimaging Analyses , 1998, NeuroImage.

[16]  P S Goldman-Rakic,et al.  Widespread origin of the primate mesofrontal dopamine system. , 1998, Cerebral cortex.

[17]  J. Price,et al.  Prefrontal cortical projections to the striatum in macaque monkeys: Evidence for an organization related to prefrontal networks , 2000, The Journal of comparative neurology.

[18]  Karl J. Friston,et al.  Dissociable Neural Responses in Human Reward Systems , 2000, The Journal of Neuroscience.

[19]  Stephen M. Smith,et al.  A global optimisation method for robust affine registration of brain images , 2001, Medical Image Anal..

[20]  A. Kacelnik,et al.  To walk or to fly? How birds choose among foraging modes. , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[21]  T. Robbins,et al.  Impulsive Choice Induced in Rats by Lesions of the Nucleus Accumbens Core , 2001, Science.

[22]  Stephen M. Smith,et al.  Temporal Autocorrelation in Univariate Linear Modeling of FMRI Data , 2001, NeuroImage.

[23]  Michael Brady,et al.  Improved Optimization for the Robust and Accurate Linear Registration and Motion Correction of Brain Images , 2002, NeuroImage.

[24]  M. Walton,et al.  The Role of Rat Medial Frontal Cortex in Effort-Based Decision Making , 2002, The Journal of Neuroscience.

[25]  Stephen M Smith,et al.  Fast robust automated brain extraction , 2002, Human brain mapping.

[26]  J. Deakin,et al.  Effects of quinolinic acid-induced lesions of the orbital prefrontal cortex on inter-temporal choice: a quantitative analysis , 2002, Psychopharmacology.

[27]  B. Richmond,et al.  Anterior Cingulate: Single Neuronal Signals Related to Degree of Reward Expectancy , 2002, Science.

[28]  R Turner,et al.  Optimized EPI for fMRI studies of the orbitofrontal cortex , 2003, NeuroImage.

[29]  J. Salamone,et al.  Nucleus Accumbens Dopamine and the Regulation of Effort in Food-Seeking Behavior: Implications for Studies of Natural Motivation, Psychiatry, and Drug Abuse , 2003, Journal of Pharmacology and Experimental Therapeutics.

[30]  Stephen M. Smith,et al.  General multilevel linear modeling for group analysis in FMRI , 2003, NeuroImage.

[31]  Matthew F S Rushworth,et al.  Functional Specialization within Medial Frontal Cortex of the Anterior Cingulate for Evaluating Effort-Related Decisions , 2003, The Journal of Neuroscience.

[32]  H. Groenewegen,et al.  The medial prefrontal cortex in the rat: evidence for a dorso-ventral distinction based upon functional and anatomical characteristics , 2003, Neuroscience & Biobehavioral Reviews.

[33]  Saori C. Tanaka,et al.  Prediction of immediate and future rewards differentially recruits cortico-basal ganglia loops , 2004, Nature Neuroscience.

[34]  Mark W. Woolrich,et al.  Advances in functional and structural MR image analysis and implementation as FSL , 2004, NeuroImage.

[35]  Jonathan D. Cohen,et al.  Conflict monitoring and anterior cingulate cortex: an update , 2004, Trends in Cognitive Sciences.

[36]  Mark W. Woolrich,et al.  Multilevel linear modelling for FMRI group analysis using Bayesian inference , 2004, NeuroImage.

[37]  Stephen M. Smith,et al.  Probabilistic independent component analysis for functional magnetic resonance imaging , 2004, IEEE Transactions on Medical Imaging.

[38]  M. Walton,et al.  Differential involvement of serotonin and dopamine systems in cost-benefit decisions about delay or effort , 2005, Psychopharmacology.

[39]  William J Powers,et al.  Activation of human medial prefrontal cortex during autonomic responses to hypoglycemia. , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[40]  S. Lehéricy,et al.  3-D diffusion tensor axonal tracking shows distinct SMA and pre-SMA projections to the human striatum. , 2004, Cerebral cortex.

[41]  T. Robbins,et al.  Contrasting Roles of Basolateral Amygdala and Orbitofrontal Cortex in Impulsive Choice , 2004, The Journal of Neuroscience.

[42]  Timothy Edward John Behrens,et al.  Quantitative Investigation of Connections of the Prefrontal Cortex in the Human and Macaque using Probabilistic Diffusion Tractography , 2005, The Journal of Neuroscience.

[43]  Wolfgang Hauber,et al.  Involvement of the rat anterior cingulate cortex in control of instrumental responses guided by reward expectancy. , 2005, Learning & memory.

[44]  Jeffrey R. Stevens,et al.  Will Travel for Food: Spatial Discounting in Two New World Monkeys , 2005, Current Biology.

[45]  Matthew T. Kaufman,et al.  Distributed Neural Representation of Expected Value , 2005, The Journal of Neuroscience.

[46]  J. Salamone,et al.  Ratio and time requirements on operant schedules: effort‐related effects of nucleus accumbens dopamine depletions , 2005, The European journal of neuroscience.

[47]  M. Roesch,et al.  Orbitofrontal Cortex, Associative Learning, and Expectancies , 2005, Neuron.

[48]  Paul Leonard Gabbott,et al.  Prefrontal cortex in the rat: Projections to subcortical autonomic, motor, and limbic centers , 2005, The Journal of comparative neurology.

[49]  P. Dayan,et al.  Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control , 2005, Nature Neuroscience.

[50]  Naoya Aoki,et al.  Neural correlates of the proximity and quantity of anticipated food rewards in the ventral striatum of domestic chicks , 2005, The European journal of neuroscience.

[51]  T. Matsushima,et al.  Localized lesions of ventral striatum, but not arcopallium, enhanced impulsiveness in choices based on anticipated spatial proximity of food rewards in domestic chicks , 2006, Behavioural Brain Research.

[52]  S. Haber,et al.  Reward-Related Cortical Inputs Define a Large Striatal Region in Primates That Interface with Associative Cortical Connections, Providing a Substrate for Incentive-Based Learning , 2006, The Journal of Neuroscience.

[53]  Timothy E. J. Behrens,et al.  Optimal decision making and the anterior cingulate cortex , 2006, Nature Neuroscience.

[54]  T. Matsushima,et al.  Localized lesions of arcopallium intermedium of the lateral forebrain caused a handling‐cost aversion in the domestic chick performing a binary choice task , 2006, The European journal of neuroscience.

[55]  Matthew F. S. Rushworth,et al.  Weighing up the benefits of work: Behavioral and neural analyses of effort-related decision making , 2006, Neural Networks.

[56]  M. Walton,et al.  Separate neural pathways process different decision costs , 2006, Nature Neuroscience.

[57]  B. Richmond,et al.  Dopamine neuronal responses in monkeys performing visually cued reward schedules , 2006, The European journal of neuroscience.

[58]  R. Dolan,et al.  Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans , 2006, Nature.

[59]  M. Roesch,et al.  Encoding of Time-Discounted Rewards in Orbitofrontal Cortex Is Independent of Value Representation , 2006, Neuron.

[60]  J. O'Doherty,et al.  Reward Value Coding Distinct From Risk Attitude-Related Uncertainty Coding in Human Reward Systems , 2006, Journal of neurophysiology.

[61]  David W. McNeal,et al.  Amygdala interconnections with the cingulate motor cortex in the rhesus monkey , 2007, The Journal of comparative neurology.

[62]  Hugo J Spiers,et al.  A navigational guidance system in the human brain , 2007, Hippocampus.

[63]  J. O'Doherty,et al.  Orbitofrontal Cortex Encodes Willingness to Pay in Everyday Economic Transactions , 2007, The Journal of Neuroscience.

[64]  M. Botvinick Conflict monitoring and decision making: Reconciling two perspectives on anterior cingulate function , 2007, Cognitive, affective & behavioral neuroscience.

[65]  M. Roesch,et al.  Dopamine neurons encode the better option in rats deciding between differently delayed or sized rewards , 2007, Nature Neuroscience.

[66]  S. Floresco,et al.  Amygdala-prefrontal cortical circuitry regulates effort-based decision making. , 2006, Cerebral cortex.

[67]  P. Glimcher,et al.  The neural correlates of subjective value during intertemporal choice , 2007, Nature Neuroscience.

[68]  Colin Camerer,et al.  Dissociating the Role of the Orbitofrontal Cortex and the Striatum in the Computation of Goal Values and Prediction Errors , 2008, The Journal of Neuroscience.

[69]  Steven P. Wise,et al.  Forward frontal fields: phylogeny and fundamental function , 2008, Trends in Neurosciences.

[70]  Timothy E. J. Behrens,et al.  Frontal Cortex Subregions Play Distinct Roles in Choices between Actions and Stimuli , 2008, The Journal of Neuroscience.

[71]  S. Quartz,et al.  Human Insula Activation Reflects Risk Prediction Errors As Well As Risk , 2008, The Journal of Neuroscience.

[72]  Samuel M. McClure,et al.  BOLD Responses Reflecting Dopaminergic Signals in the Human Ventral Tegmental Area , 2008, Science.

[73]  M. Rushworth,et al.  Behavioral / Systems / Cognitive Connectivity-Based Parcellation of Human Cingulate Cortex and Its Relation to Functional Specialization , 2008 .

[74]  Joseph T. McGuire,et al.  Effort discounting in human nucleus accumbens , 2009, Cognitive, affective & behavioral neuroscience.

[75]  Jonathan D. Wallis,et al.  Neurons in the Frontal Lobe Encode the Value of Multiple Decision Variables , 2009, Journal of Cognitive Neuroscience.