Weighing up the benefits of work: Behavioral and neural analyses of effort-related decision making

How we decide whether a course of action is worth undertaking is largely unknown. Recently, neuroscientists have been turning to ecological approaches to address this issue, examining how animals evaluate the costs and benefits of different options. We present here evidence from rodents and monkeys that demonstrate the degree to which they take into account work and energetic requirements when deciding what responses to make. These calculations appear to be critically mediated by the anterior cingulate cortex (ACC) and mesolimbic dopamine (DA) pathways, with damage to either causing a bias towards options that are easily obtained but yield relatively smaller reward rather than alternatives that require more work but result in greater reward. The evaluation of such decisions appears to be carried out in systems independent of those involved in delay-discounting. We suggest that top-down signals from ACC to nucleus accumbens (NAc) and/or midbrain DA cells may be vital for overcoming effort-related response costs.

[1]  H. de Wit,et al.  Determination of discount functions in rats with an adjusting-amount procedure. , 1997, Journal of the experimental analysis of behavior.

[2]  J. Salamone,et al.  Nucleus accumbens dopamine depletions alter relative response allocation in a T-maze cost/benefit task , 1996, Behavioural Brain Research.

[3]  Trevor W Robbins,et al.  Interactions between Serotonin and Dopamine in the Control of Impulsive Choice in Rats: Therapeutic Implications for Impulse Control Disorders , 2005, Neuropsychopharmacology.

[4]  T. Zentall,et al.  “work ethic” in pigeons: Reward value is directly related to the effort or time required to obtain the reward , 2000, Psychonomic bulletin & review.

[5]  J. Salamone,et al.  Haloperidol and nucleus accumbens dopamine depletion suppress lever pressing for food but increase free food consumption in a novel food choice procedure , 2005, Psychopharmacology.

[6]  J. Price,et al.  Prefrontal cortical projections to the hypothalamus in Macaque monkeys , 1998, The Journal of comparative neurology.

[7]  Stephen C. Fowler,et al.  Haloperidol produces within-session increments in operant response duration in rats , 1990, Pharmacology Biochemistry and Behavior.

[8]  U. Ungerstedt Adipsia and aphagia after 6-hydroxydopamine induced degeneration of the nigro-striatal dopamine system. , 1971, Acta physiologica Scandinavica. Supplementum.

[9]  J. Mayhew,et al.  How Visual Stimuli Activate Dopaminergic Neurons at Short Latency , 2005, Science.

[10]  T. Preuss Do Rats Have Prefrontal Cortex? The Rose-Woolsey-Akert Program Reconsidered , 1995, Journal of Cognitive Neuroscience.

[11]  J. Monterosso,et al.  Beyond discounting: possible experimental models of impulse control , 1999, Psychopharmacology.

[12]  Keiji Tanaka,et al.  Neuronal Correlates of Goal-Based Motor Selection in the Prefrontal Cortex , 2003, Science.

[13]  John D. Salamone,et al.  Nucleus accumbens dopamine depletions in rats affect relative response allocation in a novel cost/benefit procedure , 1994, Pharmacology Biochemistry and Behavior.

[14]  J. Salamone,et al.  Nucleus accumbens dopamine depletions make rats more sensitive to high ratio requirements but do not impair primary food reinforcement , 1999, Neuroscience.

[15]  S. Ikemoto,et al.  Dissociations between appetitive and consummatory responses by pharmacological manipulations of reward-relevant brain regions. , 1996, Behavioral neuroscience.

[16]  A. Kacelnik Normative and descriptive models of decision making: time discounting and risk sensitivity. , 2007, Ciba Foundation symposium.

[17]  H. Evans The Study of Instinct , 1952 .

[18]  M. Walton,et al.  The Role of Rat Medial Frontal Cortex in Effort-Based Decision Making , 2002, The Journal of Neuroscience.

[19]  E. Miller,et al.  An integrative theory of prefrontal cortex function. , 2001, Annual review of neuroscience.

[20]  S. Mobini,et al.  Theory and method in the quantitative analysis of ”impulsive choice” behaviour: implications for psychopharmacology , 1999, Psychopharmacology.

[21]  Matthew F S Rushworth,et al.  Functional Specialization within Medial Frontal Cortex of the Anterior Cingulate for Evaluating Effort-Related Decisions , 2003, The Journal of Neuroscience.

[22]  T. Robbins,et al.  Effects of unilateral 6-Hydroxydopamine lesions of the caudate-putamen on skilled forepaw use in the rat , 1984, Behavioural Brain Research.

[23]  R Bandler,et al.  Orbitomedial prefrontal cortical projections to hypothalamus in the rat , 2001, The Journal of comparative neurology.

[24]  R. Wightman,et al.  Subsecond dopamine release promotes cocaine seeking , 2003, Nature.

[25]  M. Walton,et al.  Interactions between decision making and performance monitoring within prefrontal cortex , 2004, Nature Neuroscience.

[26]  T. Robbins,et al.  Impulsive Choice Induced in Rats by Lesions of the Nucleus Accumbens Core , 2001, Science.

[27]  Anders Björklund,et al.  Organization of catecholamine neurons projecting to the frontal cortex in the rat , 1978, Brain Research.

[28]  Kenji Doya,et al.  Metalearning and neuromodulation , 2002, Neural Networks.

[29]  J. Salamone,et al.  Motivational views of reinforcement: implications for understanding the behavioral functions of nucleus accumbens dopamine , 2002, Behavioural Brain Research.

[30]  K. Berridge,et al.  What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience? , 1998, Brain Research Reviews.

[31]  M. Walton,et al.  The mesocortical dopamine projection to anterior cingulate cortex plays no role in guiding effort-related decisions. , 2005, Behavioral neuroscience.

[32]  HC Fibiger,et al.  Electrical stimulation of the prefrontal cortex increases dopamine release in the nucleus accumbens of the rat: modulation by metabotropic glutamate receptors , 1995, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[33]  F. Gonon,et al.  Prefrontal cortex regulates burst firing and transmitter release in rat mesolimbic dopamine neurons studied in vivo , 1993, Neuroscience Letters.

[34]  J. Evenden,et al.  The pharmacology of impulsive behaviour in rats: the effects of drugs on response choice with varying delays of reinforcement , 1996, Psychopharmacology.

[35]  T. Shallice,et al.  Human cingulate cortex and autonomic control: converging neuroimaging and clinical evidence. , 2003, Brain : a journal of neurology.

[36]  J. Salamone,et al.  Anhedonia or anergia? Effects of haloperidol and nucleus accumbens dopamine depletion on instrumental response selection in a T-maze cost/benefit procedure , 1994, Behavioural Brain Research.

[37]  Jonathan D. Cohen,et al.  Conflict monitoring and anterior cingulate cortex: an update , 2004, Trends in Cognitive Sciences.

[38]  S. Ikemoto,et al.  The role of nucleus accumbens dopamine in motivated behavior: a unifying interpretation with special reference to reward-seeking , 1999, Brain Research Reviews.

[39]  C. Janson Experimental evidence for spatial memory in foraging wild capuchin monkeys, Cebus apella , 1998, Animal Behaviour.

[40]  P. Montague,et al.  Neural Economics and the Biological Substrates of Valuation , 2002, Neuron.

[41]  J. Price,et al.  Prefrontal cortical projections to longitudinal columns in the midbrain periaqueductal gray in Macaque monkeys , 1998, The Journal of comparative neurology.

[42]  J. E. Mazur An adjusting procedure for studying delayed reinforcement. , 1987 .

[43]  S. Mizumori,et al.  Neurons in rat medial prefrontal cortex show anticipatory rate changes to predictable differential rewards in a spatial memory task , 2001, Behavioural Brain Research.

[44]  Matthew T. Kaufman,et al.  Distributed Neural Representation of Expected Value , 2005, The Journal of Neuroscience.

[45]  P. Groves,et al.  Burst firing induced in midbrain dopamine neurons by stimulation of the medial prefrontal and anterior cingulate cortices , 1988, Brain Research.

[46]  R. Roth,et al.  Topographical organization of the efferent projections of the medial prefrontal cortex in the rat: An anterograde tract‐tracing study with Phaseolus vulgaris leucoagglutinin , 1989, The Journal of comparative neurology.

[47]  B. Vogt,et al.  Form and distribution of neurons in rat cingulate cortex: Areas 32, 24, and 29 , 1981, The Journal of comparative neurology.

[48]  D. Signorini,et al.  Neural networks , 1995, The Lancet.

[50]  B. Richmond,et al.  Anterior Cingulate: Single Neuronal Signals Related to Degree of Reward Expectancy , 2002, Science.

[51]  Wolfgang Hauber,et al.  Involvement of the rat anterior cingulate cortex in control of instrumental responses guided by reward expectancy. , 2005, Learning & memory.

[52]  B. Berger,et al.  Catecholamine innervation of the human cerebral cortex as revealed by comparative immunohistochemistry of tyrosine hydroxylase and dopamine‐beta‐hydroxylase , 1989, The Journal of comparative neurology.

[53]  Peter Dayan,et al.  How fast to work: Response vigor, motivation and tonic dopamine , 2005, NIPS.

[54]  William Rowan,et al.  The Study of Instinct , 1953 .

[55]  H. de Wit,et al.  Effects of methamphetamine on the adjusting amount procedure, a model of impulsive behavior in rats , 1999, Psychopharmacology.

[56]  Samuel M. McClure,et al.  A computational substrate for incentive salience , 2003, Trends in Neurosciences.

[57]  J. N. P. Rawlins,et al.  The effects of delaying reward on choice preference in rats with hippocampal or selective septal lesions , 1985, Behavioural Brain Research.

[58]  B. Berger,et al.  Dopaminergic innervation of the frontal cerebral cortex. Evolutionary trends and functional implications. , 1992, Advances in neurology.

[59]  R. Dias,et al.  Effects of selective excitotoxic prefrontal lesions on acquisition of nonmatching‐ and matching‐to‐place in the T‐maze in the rat: differential involvement of the prelimbic–infralimbic and anterior cingulate cortices in providing behavioural flexibility , 2000, The European journal of neuroscience.

[60]  Paul Glimcher,et al.  Review Decisions, Decisions, Decisions: Choosing a Biological Science of Choice Nally Designed...the Theory Requires That the Values of Different Outcomes (for Example, Financial Rewards, the Risks of Death and the Pleasures of a Clear Conscience) , 2022 .

[61]  H. Uylings,et al.  Qualitative and quantitative comparison of the prefrontal cortex in rat and in primates, including humans. , 1990, Progress in brain research.

[62]  T. R. Wade,et al.  Effects of dopaminergic drugs on delayed reward as a measure of impulsive behavior in rats , 2000, Psychopharmacology.

[63]  K. R. Ridderinkhof,et al.  The Role of the Medial Frontal Cortex in Cognitive Control , 2004, Science.

[64]  M. Gabriel,et al.  Effects of cingulate cortical lesions on avoidance learning and training-induced unit activity in rabbits , 2004, Experimental Brain Research.

[65]  T. Matsushima,et al.  Localized lesions of arcopallium intermedium of the lateral forebrain caused a handling‐cost aversion in the domestic chick performing a binary choice task , 2006, The European journal of neuroscience.

[66]  K. Nakano,et al.  Efferent projections of infralimbic and prelimbic areas of the medial prefrontal cortex in the Japanese monkey, Macaca fuscata , 2001, Brain Research.

[67]  R. Wightman,et al.  Dopamine Operates as a Subsecond Modulator of Food Seeking , 2004, The Journal of Neuroscience.

[68]  Aaron Ettenberg,et al.  Haloperidol blocks the response-reinstating effects of food reward: A methodology for separating neuroleptic effects on reinforcement and motor processes , 1988, Pharmacology Biochemistry and Behavior.

[69]  A. Kacelnik,et al.  To walk or to fly? How birds choose among foraging modes. , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[70]  J. Salamone,et al.  Accumbens dopamine and the regulation of effort in food-seeking behavior: modulation of work output by different ratio or force requirements , 2004, Behavioural Brain Research.

[71]  G F Koob,et al.  Effects of mesolimbic dopamine depletion on responding maintained by cocaine and food. , 1994, Journal of the experimental analysis of behavior.

[72]  M. Platt,et al.  Neural correlates of decisions , 2002, Current Opinion in Neurobiology.

[73]  R Bandler,et al.  Orbitomedial prefrontal cortical projections to distinct longitudinal columns of the periaqueductal gray in the rat , 2000, The Journal of comparative neurology.

[74]  J P de Bruin,et al.  Comparison of the effects of neonatal and adult medial prefrontal cortex lesions on food hoarding and spatial delayed alternation. , 1991, Behavioural brain research.

[75]  R. Wise,et al.  Brain dopamine and reward. , 1989, Annual review of psychology.

[76]  C. M. Bradshaw,et al.  Comparison of the effects of clozapine, haloperidol, chlorpromazine and d-amphetamine on performance on a time-constrained progressive ratio schedule and on locomotor behaviour in the rat , 2000, Psychopharmacology.

[77]  D. Tranel,et al.  Acquired Personality Disturbances Associated With Bilateral Damage to the Ventromedial Prefrontal Region , 2000, Developmental neuropsychology.

[78]  M. Walton,et al.  Action sets and decisions in the medial frontal cortex , 2004, Trends in Cognitive Sciences.

[79]  Jeffrey R. Stevens,et al.  Will Travel for Food: Spatial Discounting in Two New World Monkeys , 2005, Current Biology.

[80]  A. Kacelnik,et al.  Cost can increase preference in starlings , 2002, Animal Behaviour.

[81]  J. Schall,et al.  Neural selection and control of visually guided eye movements. , 1999, Annual review of neuroscience.

[82]  P. Bailey The neurobiology of the nucleus accumbens R. B. Chronister and J. F. de France (Eds). Haer Institute for Electrophysiological Research (1981). 388 pp , 1982, Neuroscience.

[83]  M. Walton,et al.  Differential involvement of serotonin and dopamine systems in cost-benefit decisions about delay or effort , 2005, Psychopharmacology.

[84]  T. Robbins,et al.  The effects of d-amphetamine, chlordiazepoxide, α-flupenthixol and behavioural manipulations on choice of signalled and unsignalled delayed reinforcement in rats , 2000, Psychopharmacology.

[85]  J. Seamans,et al.  Functional differences between the prelimbic and anterior cingulate regions of the rat prefrontal cortex. , 1995, Behavioral neuroscience.

[86]  Jack E. Henningfield,et al.  Preference among research cigarettes with varying nicotine yields , 1990, Pharmacology Biochemistry and Behavior.

[87]  E. Procyk,et al.  Anterior cingulate activity during routine and non-routine sequential behaviors in macaques , 2000, Nature Neuroscience.

[88]  C. Bradshaw,et al.  Choice Between Delayed Reinforcers in a Discrete-Trials Schedule: The Effect of Deprivation Level , 1992, The Quarterly journal of experimental psychology. B, Comparative and physiological psychology.

[89]  W. Schultz Getting Formal with Dopamine and Reward , 2002, Neuron.

[90]  J. Deakin,et al.  Effects of orbital prefrontal cortex dopamine depletion on inter-temporal choice: a quantitative analysis , 2004, Psychopharmacology.

[91]  Jaak Panksepp,et al.  Dissociations between appetitive and consummatory responses by pharmacological manipulations of reward-relevant brain regions. , 1996 .

[92]  G. Shulman,et al.  Persistence and brain circuitry , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[93]  I. Weiner,et al.  The effects of excitotoxic lesion of the medial prefrontal cortex on latent inhibition, prepulse inhibition, food hoarding, elevated plus maze, active avoidance and locomotor activity in the rat , 1998, Neuroscience.

[94]  J. Price,et al.  The organization of networks within the orbital and medial prefrontal cortex of rats, monkeys and humans. , 2000, Cerebral cortex.

[95]  P. Overton,et al.  Stimulation of the prefrontal cortex in the rat induces patterns of activity in midbrain dopaminergic neurons which resemble natural burst events , 1996, Synapse.