Action controls dopaminergic enhancement of reward representations

Dopamine is widely observed to signal anticipation of future rewards and thus thought to be a key contributor to affectively charged decision making. However, the experiments supporting this view have not dissociated rewards from the actions that lead to, or are occasioned by, them. Here, we manipulated dopamine pharmacologically and examined the effect on a task that explicitly dissociates action and reward value. We show that dopamine enhanced the neural representation of rewarding actions, without significantly affecting the representation of reward value as such. Thus, increasing dopamine levels with levodopa selectively boosted striatal and substantia nigra/ventral tegmental representations associated with actions leading to reward, but not with actions leading to the avoidance of punishment. These findings highlight a key role for dopamine in the generation of appetitively motivated actions.

[1]  O. Mowrer.  On the dual nature of learning—a re-interpretation of "conditioning" and "problem-solving." , 1947 .

[2]  U. Ungerstedt.  Adipsia and aphagia after 6-hydroxydopamine induced degeneration of the nigro-striatal dopamine system. , 1971, Acta physiologica Scandinavica. Supplementum.

[3]  J. Gray,et al.  Précis of The neuropsychology of anxiety: An enquiry into the functions of the septo-hippocampal system , 1982, Behavioral and Brain Sciences.

[4]  P. Soubrié.  Reconciling the role of central serotonin neurons in human and animal behavior , 1986, Behavioral and Brain Sciences.

[5]  A. Grace.  Phasic versus tonic dopamine release and the modulation of dopamine system responsivity: A hypothesis for the etiology of schizophrenia , 1991, Neuroscience.

[6]  W. Schultz,et al.  Preferential activation of midbrain dopamine neurons by appetitive rather than aversive stimuli , 1996, Nature.

[7]  P. Dayan,et al.  A framework for mesencephalic dopamine systems based on predictive Hebbian learning , 1996, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[8]  Peter Dayan,et al.  A Neural Substrate of Prediction and Reward , 1997, Science.

[9]  K. Berridge,et al.  What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience? , 1998, Brain Research Reviews.

[10]  P. Redgrave,et al.  Is the short-latency dopamine response too short to signal reward error? , 1999, Trends in Neurosciences.

[11]  L. Nystrom,et al.  Tracking the hemodynamic responses to reward and punishment in the striatum. , 2000, Journal of neurophysiology.

[12]  Brian Knutson,et al.  Anticipation of Increasing Monetary Reward Selectively Recruits Nucleus Accumbens , 2001, The Journal of Neuroscience.

[13]  Sham M. Kakade,et al.  Opponent interactions between serotonin and dopamine , 2002, Neural Networks.

[14]  Samuel M. McClure,et al.  Temporal Prediction Errors in a Passive Learning Task Activate Human Striatum , 2003, Neuron.

[15]  Karl J. Friston,et al.  Temporal Difference Models and Reward-Related Learning in the Human Brain , 2003, Neuron.

[16]  W. Schultz,et al.  Discrete Coding of Reward Probability and Uncertainty by Dopamine Neurons , 2003, Science.

[17]  R. Palmiter,et al.  Dopamine controls the firing pattern of dopamine neurons via a network feedback mechanism , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[18]  J. Bolam,et al.  Uniform Inhibition of Dopamine Neurons in the Ventral Tegmental Area by Aversive Stimuli , 2004, Science.

[19]  Karl J. Friston,et al.  Dissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning , 2004, Science.

[20]  Michael J. Frank,et al.  By Carrot or by Stick: Cognitive Reinforcement Learning in Parkinsonism , 2004, Science.

[21]  Rebecca Elliott,et al.  Instrumental responding for rewards is associated with enhanced neuronal response in subcortical reward systems , 2004, NeuroImage.

[22]  M. Delgado,et al.  Modulation of Caudate Activity by Action Contingency , 2004, Neuron.

[23]  J. Salamone,et al.  Beyond the reward hypothesis: alternative functions of nucleus accumbens dopamine. , 2005, Current opinion in pharmacology.

[24]  W. Schultz,et al.  Adaptive Coding of Reward Value by Dopamine Neurons , 2005, Science.

[25]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[26]  P. Glimcher,et al.  Midbrain Dopamine Neurons Encode a Quantitative Reward Prediction Error Signal , 2005, Neuron.

[27]  R. Dolan,et al.  Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans , 2006, Nature.

[28]  S. Knecht,et al.  Tonic Dopaminergic Stimulation Impairs Associative Learning in Healthy Subjects , 2006, Neuropsychopharmacology.

[29]  P. Dayan,et al.  Tonic dopamine: opportunity costs and the control of response vigor , 2007, Psychopharmacology.

[30]  Michael X. Cohen,et al.  Different neural systems adjust motor behavior in response to reward and punishment , 2007, NeuroImage.

[31]  P. Dayan,et al.  A temporal difference account of avoidance learning , 2008, Network.

[32]  N. Logothetis.  What we can do and what we cannot do with fMRI , 2008, Nature.

[33]  Samuel M. McClure,et al.  BOLD Responses Reflecting Dopaminergic Signals in the Human Ventral Tegmental Area , 2008, Science.

[34]  R. Palmiter.  Dopamine Signaling in the Dorsal Striatum Is Essential for Motivated Behaviors , 2008, Annals of the New York Academy of Sciences.

[35]  S. Huettel,et al.  Activation in the VTA and Nucleus Accumbens Increases in Anticipation of Both Gains and Losses , 2009, Front. Behav. Neurosci..

[36]  O. Hikosaka,et al.  Two types of dopamine neuron distinctly convey positive and negative motivational signals , 2009, Nature.

[37]  M. Ungless,et al.  Phasic excitation of dopamine neurons in ventral VTA by noxious stimuli , 2009, Proceedings of the National Academy of Sciences.

[38]  P. Dayan,et al.  Serotonin in affective control. , 2009, Annual review of neuroscience.

[39]  T. Robbins,et al.  Reconciling the Role of Serotonin in Behavioral Inhibition and Aversion: Acute Tryptophan Depletion Abolishes Punishment-Induced Inhibition in Humans , 2009, The Journal of Neuroscience.

[40]  P. Tobler,et al.  Functional imaging of the human dopaminergic midbrain , 2009, Trends in Neurosciences.

[41]  Jean-Luc Anton,et al.  Region of interest analysis using an SPM toolbox , 2010 .

[42]  T. Maia.  Two-factor theory, the actor-critic model, and conditioned avoidance , 2010, Learning & behavior.

[43]  Ethan S. Bromberg-Martin,et al.  Dopamine in Motivational Control: Rewarding, Aversive, and Alerting , 2010, Neuron.

[44]  T. Robbins,et al.  Serotonin Modulates Sensitivity to Reward and Negative Feedback in a Probabilistic Reversal Learning Task in Rats , 2010, Neuropsychopharmacology.

[45]  G. Gründer,et al.  Serotonergic modulation of response inhibition and re‐engagement? Results of a study in healthy human volunteers , 2010, Human psychopharmacology.

[46]  S. Haber,et al.  The Reward Circuit: Linking Primate Anatomy and Human Imaging , 2010, Neuropsychopharmacology.

[47]  N. Daw,et al.  Serotonin and Dopamine: Unifying Affective, Activational, and Decision Functions , 2011, Neuropsychopharmacology.

[48]  P. Dayan,et al.  Behavioral/systems/cognitive Action Dominates Valence in Anticipatory Representations in the Human Striatum and Dopaminergic Midbrain , 2010 .

[49]  P. Dayan,et al.  Opponency Revisited: Competition and Cooperation Between Dopamine and Serotonin , 2010, Neuropsychopharmacology.