Arbitration between controlled and impulsive choices

The impulse to act for immediate reward often conflicts with more deliberate evaluations that support long-term benefit. The neural architecture that negotiates this conflict remains unclear. One account proposes a single neural circuit that evaluates both immediate and delayed outcomes, while another outlines separate impulsive and patient systems that compete for behavioral control. Here we designed a task in which a complex payout structure divorces the immediate value of acting from the overall long-term value, within the same outcome modality. Using model-based fMRI in humans, we demonstrate separate neural representations of immediate and long-term values, with the former tracked in the anterior caudate (AC) and the latter in the ventromedial prefrontal cortex (vmPFC). Crucially, when subjects' choices were compatible with long-run consequences, value signals in AC were down-weighted and those in vmPFC were enhanced, while the opposite occurred when choice was impulsive. Thus, our data implicate a trade-off in value representation between AC and vmPFC as underlying controlled versus impulsive choice.

[1]  B. Balleine,et al.  Calculating Consequences: Brain Systems That Encode the Causal Effects of Actions , 2008, The Journal of Neuroscience.

[2]  P. Dayan,et al.  Goals and Habits in the Brain , 2013, Neuron.

[3]  T. Robbins,et al.  Inhibition and the right inferior frontal cortex , 2004, Trends in Cognitive Sciences.

[4]  E. Barratt,et al.  Psychiatric aspects of impulsivity. , 2001, The American journal of psychiatry.

[5]  B. Balleine,et al.  Human and Rodent Homologies in Action Control: Corticostriatal Determinants of Goal-Directed and Habitual Action , 2010, Neuropsychopharmacology.

[6]  H. Damasio,et al.  Characterization of the decision-making deficit of patients with ventromedial prefrontal cortex lesions. , 2000, Brain : a journal of neurology.

[7]  Raymond J. Dolan,et al.  Disentangling the Roles of Approach, Activation and Valence in Instrumental and Pavlovian Responding , 2011, PLoS Comput. Biol..

[8]  Oliver Gruber,et al.  When Desire Collides with Reason: Functional Interactions between Anteroventral Prefrontal Cortex and Nucleus Accumbens Underlie the Human Ability to Resist Impulsive Desires , 2009, The Journal of Neuroscience.

[9]  Oliver Speck,et al.  The impact of physiological noise correction on fMRI at 7 T , 2011, NeuroImage.

[10]  Saori C. Tanaka,et al.  Prediction of immediate and future rewards differentially recruits cortico-basal ganglia loops , 2004, Nature Neuroscience.

[11]  M. Delgado,et al.  Modulation of Caudate Activity by Action Contingency , 2004, Neuron.

[12]  R. Baumeister,et al.  The Physiology of Willpower: Linking Blood Glucose to Self-Control , 2007, Personality and social psychology review : an official journal of the Society for Personality and Social Psychology, Inc.

[13]  P. Dayan,et al.  Behavioral/systems/cognitive Action Dominates Valence in Anticipatory Representations in the Human Striatum and Dopaminergic Midbrain , 2010 .

[14]  Karl J. Friston,et al.  Action-Specific Value Signals in Reward-Related Regions of the Human Brain , 2012, The Journal of Neuroscience.

[15]  A. Rangel Regulation of dietary choice by the decision-making circuitry , 2013, Nature Neuroscience.

[16]  A. Rangel,et al.  Activity in dlPFC and its effective connectivity to vmPFC are associated with temporal discounting , 2014, Front. Neurosci..

[17]  Y. Niv,et al.  Ventral Striatum and Orbitofrontal Cortex Are Both Required for Model-Based, But Not Model-Free, Reinforcement Learning , 2011, The Journal of Neuroscience.

[18]  B. Balleine Neural bases of food-seeking: Affect, arousal and reward in corticostriatolimbic circuits , 2005, Physiology & Behavior.

[19]  P. Dayan,et al.  Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control , 2005, Nature Neuroscience.

[20]  Andrea Brovelli,et al.  Differential roles of caudate nucleus and putamen during instrumental learning , 2011, NeuroImage.

[21]  K. Doya,et al.  Representation of Action-Specific Reward Values in the Striatum , 2005, Science.

[22]  David R. Cox The analysis of binary data , 1970 .

[23]  J. Malmaud,et al.  Focusing Attention on the Health Aspects of Foods Changes Value Signals in vmPFC and Improves Dietary Choice , 2011, The Journal of Neuroscience.

[24]  R. Baumeister,et al.  Ego depletion: is the active self a limited resource? , 1998, Journal of personality and social psychology.

[25]  Ilya E Monosov,et al.  Regionally Distinct Processing of Rewards and Punishments by the Primate Ventromedial Prefrontal Cortex , 2012, The Journal of Neuroscience.

[26]  Geoffrey Schoenbaum,et al.  Neural Estimates of Imagined Outcomes in the Orbitofrontal Cortex Drive Behavior and Learning , 2013, Neuron.

[27]  Vivian V. Valentin,et al.  Determining the Neural Substrates of Goal-Directed Learning in the Human Brain , 2007, The Journal of Neuroscience.

[28]  Deborah L. Harrington,et al.  Motor Functions of the Basal Ganglia , 2009 .

[29]  Joshua L. Jones,et al.  Orbitofrontal Cortex Supports Behavior and Learning Using Inferred But Not Cached Values , 2012, Science.

[30]  Raymond J. Dolan,et al.  Go and no-go learning in reward and punishment: Interactions between affect and effect , 2012, NeuroImage.

[31]  P. Glimcher,et al.  The neural correlates of subjective value during intertemporal choice , 2007, Nature Neuroscience.

[32]  P. Glimcher,et al.  An "as soon as possible" effect in human intertemporal decision making: behavioral evidence and neural mechanisms. , 2010, Journal of neurophysiology.

[33]  Samuel M. McClure,et al.  Separate Neural Systems Value Immediate and Delayed Monetary Rewards , 2004, Science.

[34]  P. Dayan,et al.  Mapping value based planning and extensively trained choice in the human brain , 2012, Nature Neuroscience.

[35]  Colin Camerer,et al.  Self-control in decision-making involves modulation of the vmPFC valuation system , 2009, NeuroImage.

[36]  M. Roesch,et al.  Ventral Striatal Neurons Encode the Value of the Chosen Action in Rats Deciding between Differently Delayed or Sized Rewards , 2009, The Journal of Neuroscience.

[37]  F. Strack,et al.  Impulse and Self-Control From a Dual-Systems Perspective , 2009, Perspectives on psychological science : a journal of the Association for Psychological Science.

[38]  Z. Kurth-Nelson,et al.  Anterior Cingulate Cortex Instigates Adaptive Switches in Choice by Integrating Immediate and Delayed Components of Value in Ventromedial Prefrontal Cortex , 2014, The Journal of Neuroscience.

[39]  Mark W. Woolrich,et al.  Trial-Type Dependent Frames of Reference for Value Comparison , 2013, PLoS Comput. Biol..

[40]  P. Dayan,et al.  Action versus valence in decision making , 2014, Trends in Cognitive Sciences.

[41]  D. Cox,et al.  Analysis of Binary Data (2nd ed.). , 1990 .

[42]  Sara E. Berger,et al.  Parceling Human Accumbens into Putative Core and Shell Dissociates Encoding of Values for Reward and Pain , 2013, The Journal of Neuroscience.

[43]  B. Balleine,et al.  Goal-directed instrumental action: contingency and incentive learning and their cortical substrates , 1998, Neuropharmacology.

[44]  Wilhelm Hofmann,et al.  Desire The New Hot Spot in Self-Control Research , 2012 .

[45]  B. Balleine,et al.  The role of the dorsomedial striatum in instrumental conditioning , 2005, The European journal of neuroscience.

[46]  J. O'Doherty,et al.  The Role of the Ventromedial Prefrontal Cortex in Abstract State-Based Inference during Decision Making in Humans , 2006, The Journal of Neuroscience.

[47]  Jonathan D. Cohen,et al.  Anterior Cingulate Conflict Monitoring and Adjustments in Control , 2004, Science.