Brief optogenetic inhibition of dopamine neurons mimics endogenous negative reward prediction errors

Correlative studies have strongly linked phasic changes in dopamine activity with reward prediction error signaling. But causal evidence that these brief changes in firing actually serve as error signals to drive associative learning is more tenuous. Although there is direct evidence that brief increases can substitute for positive prediction errors, there is no comparable evidence that similarly brief pauses can substitute for negative prediction errors. In the absence of such evidence, the effect of increases in firing could reflect novelty or salience, variables also correlated with dopamine activity. Here we provide evidence in support of the proposed linkage, showing in a modified Pavlovian over-expectation task that brief pauses in the firing of dopamine neurons in rat ventral tegmental area at the time of reward are sufficient to mimic the effects of endogenous negative prediction errors. These results support the proposal that brief changes in the firing of dopamine neurons serve as full-fledged bidirectional prediction error signals.

[1]  R. Rescorla Reduction in the effectiveness of reinforcement after prior excitatory conditioning , 1970 .

[2]  R. Rescorla,et al.  A theory of Pavlovian conditioning : Variations in the effectiveness of reinforcement and nonreinforcement , 1972 .

[3]  W. Schultz,et al.  Importance of unpredictability for reward responses in primate dopamine neurons. , 1994, Journal of neurophysiology.

[4]  Peter Dayan,et al.  A Neural Substrate of Prediction and Reward , 1997, Science.

[5]  K. Berridge,et al.  What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience? , 1998, Brain Research Reviews.

[6]  W. Schultz,et al.  Dopamine responses comply with basic assumptions of formal learning theory , 2001, Nature.

[7]  Peter Dayan,et al.  Dopamine: generalization and bonuses , 2002, Neural Networks.

[8]  W. Pan,et al.  Dopamine Cells Respond to Predicted Events during Classical Conditioning: Evidence for Eligibility Traces in the Reward-Learning Network , 2005, The Journal of Neuroscience.

[9]  Richard S. Sutton,et al.  Learning to predict by the methods of temporal differences , 1988, Machine Learning.

[10]  R. Rescorla,et al.  Spontaneous recovery from overexpectation , 2006, Learning & behavior.

[11]  P. Dayan,et al.  Tonic dopamine: opportunity costs and the control of response vigor , 2007, Psychopharmacology.

[12]  R. Rescorla Renewal after overexpectation , 2007, Learning & behavior.

[13]  P. Glimcher,et al.  Statistics of midbrain dopamine neuron spike trains in the awake primate. , 2007, Journal of neurophysiology.

[14]  P. Shepard,et al.  Lateral Habenula Stimulation Inhibits Rat Midbrain Dopamine Neurons through a GABAA Receptor-Mediated Mechanism , 2007, The Journal of Neuroscience.

[15]  Michael J. Frank,et al.  Genetic triple dissociation reveals multiple roles for dopamine in reinforcement learning , 2007, Proceedings of the National Academy of Sciences.

[16]  O. Hikosaka,et al.  Lateral habenula as a source of negative reward signals in dopamine neurons , 2007, Nature.

[17]  M. Roesch,et al.  Dopamine neurons encode the better option in rats deciding between differently delayed or sized rewards , 2007, Nature Neuroscience.

[18]  R. Wightman,et al.  Associative learning mediates dynamic shifts in dopamine signaling in the nucleus accumbens , 2007, Nature Neuroscience.

[19]  P. Redgrave,et al.  What is reinforced by phasic dopamine signals? , 2008, Brain Research Reviews.

[20]  Samuel M. McClure,et al.  BOLD Responses Reflecting Dopaminergic Signals in the Human Ventral Tegmental Area , 2008, Science.

[21]  R. Palmiter,et al.  Disruption of NMDAR-dependent burst firing by dopamine neurons provides selective assessment of phasic dopamine-dependent behavior , 2009, Proceedings of the National Academy of Sciences.

[22]  O. Hikosaka,et al.  Two types of dopamine neuron distinctly convey positive and negative motivational signals , 2009, Nature.

[23]  K. Deisseroth,et al.  Phasic Firing in Dopaminergic Neurons Is Sufficient for Behavioral Conditioning , 2009, Science.

[24]  M. Roesch,et al.  The Orbitofrontal Cortex and Ventral Tegmental Area Are Necessary for Learning from Unexpected Outcomes , 2009, Neuron.

[25]  J. Tepper,et al.  Glutamatergic Signaling by Mesolimbic Dopamine Neurons in the Nucleus Accumbens , 2010, The Journal of Neuroscience.

[26]  G. Stuber,et al.  Dopaminergic Terminals in the Nucleus Accumbens But Not the Dorsal Striatum Corelease Glutamate , 2010, The Journal of Neuroscience.

[27]  F. Ohl,et al.  Differential Neuromodulation of Acquisition and Retrieval of Avoidance Learning by the Lateral Habenula and Ventral Tegmental Area , 2010, The Journal of Neuroscience.

[28]  Ilana B. Witten,et al.  Recombinase-Driver Rat Lines: Tools, Techniques, and Optogenetic Application to Dopamine-Mediated Reinforcement , 2011, Neuron.

[29]  P. Glimcher Understanding dopamine and reinforcement learning: The dopamine reward prediction error hypothesis , 2011, Proceedings of the National Academy of Sciences.

[30]  Simon Hong,et al.  Negative Reward Signals from the Lateral Habenula to Dopamine Neurons Are Mediated by Rostromedial Tegmental Nucleus in Primates , 2011, The Journal of Neuroscience.

[31]  Guillem R. Esber,et al.  Reconciling the influence of predictiveness and uncertainty on stimulus salience: a model of attention in associative learning , 2011, Proceedings of the Royal Society B: Biological Sciences.

[32]  M. Morales,et al.  Duration of Inhibition of Ventral Tegmental Area Dopamine Neurons Encodes a Level of Conditioned Fear , 2011, The Journal of Neuroscience.

[33]  J. Salamone,et al.  The Mysterious Motivational Functions of Mesolimbic Dopamine , 2012, Neuron.

[34]  Heterogeneous composition of dopamine neurons of the rat A10 region: molecular evidence for diverse signaling properties , 2013, Brain Structure and Function.

[35]  Alice M Stamatakis,et al.  Activation of lateral habenula inputs to the ventral midbrain promotes behavioral avoidance , 2012, Nature Neuroscience.

[36]  C. Fiorillo,et al.  Optogenetic Mimicry of the Transient Activation of Dopamine Neurons by Natural Reward Is Sufficient for Operant Reinforcement , 2012, PloS one.

[37]  G. Elmer,et al.  The habenula governs the attribution of incentive salience to reward predictive cues , 2013, Front. Hum. Neurosci..

[38]  Geoffrey Schoenbaum,et al.  Neural Estimates of Imagined Outcomes in the Orbitofrontal Cortex Drive Behavior and Learning , 2013, Neuron.

[39]  Karl Deisseroth,et al.  A Unique Population of Ventral Tegmental Area Neurons Inhibits the Lateral Habenula to Promote Reward , 2013, Neuron.

[40]  Josiah R. Boivin,et al.  A Causal Link Between Prediction Errors, Dopamine Neurons and Learning , 2013, Nature Neuroscience.

[41]  P. Glimcher,et al.  Phasic Dopamine Release in the Rat Nucleus Accumbens Symmetrically Encodes a Reward Prediction Error Term , 2014, The Journal of Neuroscience.

[42]  S. Ikemoto,et al.  Similar Roles of Substantia Nigra and Ventral Tegmental Dopamine Neurons in Reward and Aversion , 2014, The Journal of Neuroscience.

[43]  S. Floresco,et al.  Overriding Phasic Dopamine Signals Redirects Action Selection during Risk/Reward Decision Making , 2014, Neuron.

[44]  Alice M Stamatakis,et al.  Considerations When Using Cre-Driver Rodent Lines for Studying Ventral Tegmental Area Circuitry , 2015, Neuron.

[45]  Shiliang Zhang,et al.  Glutamatergic and dopaminergic neurons in the mouse ventral tegmental area , 2015, The European journal of neuroscience.

[46]  Liqun Luo,et al.  Diversity of Transgenic Mouse Models for Selective Targeting of Midbrain Dopamine Neurons , 2015, Neuron.

[47]  Satoshi Ikemoto,et al.  Basal ganglia circuit loops, dopamine and motivation: A review and enquiry , 2015, Behavioural Brain Research.

[48]  A. Bonci,et al.  Dopaminergic and glutamatergic microdomains within a subset of rodent mesoaccumbens axons , 2016 .

[49]  D. H. Root,et al.  Norepinephrine Activates Dopamine D4 Receptors in the Rat Lateral Habenula , 2015, The Journal of Neuroscience.