A Causal Link Between Prediction Errors, Dopamine Neurons and Learning

Situations in which rewards are unexpectedly obtained or withheld represent opportunities for new learning. Often, this learning includes identifying cues that predict reward availability. Unexpected rewards strongly activate midbrain dopamine neurons. This phasic signal is proposed to support learning about antecedent cues by signaling discrepancies between actual and expected outcomes, termed a reward prediction error. However, it is unknown whether dopamine neuron prediction error signaling and cue-reward learning are causally linked. To test this hypothesis, we manipulated dopamine neuron activity in rats in two behavioral procedures, associative blocking and extinction, that illustrate the essential function of prediction errors in learning. We observed that optogenetic activation of dopamine neurons concurrent with reward delivery, mimicking a prediction error, was sufficient to cause long-lasting increases in cue-elicited reward-seeking behavior. Our findings establish a causal role for temporally precise dopamine neuron signaling in cue-reward learning, bridging a critical gap between experimental evidence and influential theoretical frameworks.

[1]  L. Kamin Predictability, surprise, attention, and conditioning , 1967 .

[2]  L. Kamin Attention-like processes in classical conditioning , 1967 .

[3]  R. Rescorla,et al.  A theory of Pavlovian conditioning : Variations in the effectiveness of reinforcement and nonreinforcement , 1972 .

[4]  W. F. Prokasy,et al.  Classical conditioning II: Current research and theory. , 1972 .

[5]  W. Nauta,et al.  Efferent connections of the substantia nigra and ventral tegmental area in the rat , 1979, Brain Research.

[6]  A G Barto,et al.  Toward a modern theory of adaptive networks: expectation and prediction. , 1981, Psychological review.

[7]  L. Swanson,et al.  The projections of the ventral tegmental area and adjacent regions: A combined fluorescent retrograde tracer and immunofluorescence study in the rat , 1982, Brain Research Bulletin.

[8]  P. Holland Unblocking in Pavlovian appetitive conditioning. , 1984, Journal of experimental psychology. Animal behavior processes.

[9]  W. Schultz,et al.  Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task , 1993, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[10]  W. Schultz,et al.  Importance of unpredictability for reward responses in primate dopamine neurons. , 1994, Journal of neurophysiology.

[11]  P. Dayan,et al.  A framework for mesencephalic dopamine systems based on predictive Hebbian learning , 1996, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[12]  Peter Dayan,et al.  A Neural Substrate of Prediction and Reward , 1997, Science.

[13]  J. Hollerman,et al.  Dopamine neurons report an error in the temporal prediction of reward during learning , 1998, Nature Neuroscience.

[14]  K. Berridge,et al.  What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience? , 1998, Brain Research Reviews.

[15]  J. Becker Gender Differences in Dopaminergic Function in Striatum and Nucleus Accumbens , 1999, Pharmacology Biochemistry and Behavior.

[16]  A. Dickinson,et al.  Neuronal coding of prediction errors. , 2000, Annual review of neuroscience.

[17]  J. Wickens,et al.  A cellular mechanism of reward-related learning , 2001, Nature.

[18]  W. Schultz,et al.  Dopamine responses comply with basic assumptions of formal learning theory , 2001, Nature.

[19]  Sham M. Kakade,et al.  Opponent interactions between serotonin and dopamine , 2002, Neural Networks.

[20]  John N. J. Reynolds,et al.  Dopamine-dependent plasticity of corticostriatal synapses , 2002, Neural Networks.

[21]  Roland E. Suri,et al.  TD models of reward predictive responses in dopamine neurons , 2002, Neural Networks.

[22]  C. Salum,et al.  The effect of amphetamine on Kamin blocking and overshadowing , 2003, Behavioural pharmacology.

[23]  D. Huso,et al.  Estrous cycle and ovarian changes in a rat mammary carcinogenesis model after irradiation, tamoxifen chemoprevention, and aging. , 2003, Comparative medicine.

[24]  K. Deisseroth,et al.  Millisecond-timescale, genetically targeted optical control of neural activity , 2005, Nature Neuroscience.

[25]  Mihaela D Iordanova,et al.  Dopamine activity in the nucleus accumbens modulates blocking in fear conditioning , 2006, The European journal of neuroscience.

[26]  Feng Zhang,et al.  Channelrhodopsin-2 and optical control of excitable cells , 2006, Nature Methods.

[27]  M. Roesch,et al.  Dopamine neurons encode the better option in rats deciding between differently delayed or sized rewards , 2007, Nature Neuroscience.

[28]  J. Horvitz,et al.  Dopaminergic Mechanisms in Actions and Habits , 2007, The Journal of Neuroscience.

[29]  Elyssa B. Margolis,et al.  Ventral tegmental area neurons in learned appetitive behavior and positive reinforcement. , 2007, Annual review of neuroscience.

[30]  R. Wightman,et al.  Associative learning mediates dynamic shifts in dopamine signaling in the nucleus accumbens , 2007, Nature Neuroscience.

[31]  R. Joosten,et al.  Reward-Predictive Cues Enhance Excitatory Synaptic Strength onto Midbrain Dopamine Neurons , 2008, Science.

[32]  Geoffrey Schoenbaum,et al.  The role of the orbitofrontal cortex in the pursuit of happiness and more specific rewards , 2008, Nature.

[33]  R. Palmiter,et al.  Disruption of NMDAR-dependent burst firing by dopamine neurons provides selective assessment of phasic dopamine-dependent behavior , 2009, Proceedings of the National Academy of Sciences.

[34]  J. Peters,et al.  Extinction circuits for fear and addiction overlap in prefrontal cortex. , 2009, Learning & memory.

[35]  K. Deisseroth,et al.  Phasic Firing in Dopaminergic Neurons Is Sufficient for Behavioral Conditioning , 2009, Science.

[36]  M. Roesch,et al.  The Orbitofrontal Cortex and Ventral Tegmental Area Are Necessary for Learning from Unexpected Outcomes , 2009, Neuron.

[37]  K. Tye,et al.  Methylphenidate facilitates learning-induced amygdala plasticity , 2010, Nature Neuroscience.

[38]  X. Zhuang,et al.  Faculty Opinions recommendation of A selective role for dopamine in stimulus-reward learning. , 2010 .

[39]  S. B. Evans,et al.  Absence of NMDA receptors in dopamine neurons attenuates dopamine release but not conditioned approach during Pavlovian conditioning , 2010, Proceedings of the National Academy of Sciences.

[40]  K. Deisseroth,et al.  Drug-Driven AMPA Receptor Redistribution Mimicked by Selective Dopamine Neuron Stimulation , 2010, PloS one.

[41]  Ana I. Domingos,et al.  Leptin regulates the reward value of nutrient , 2011, Nature Neuroscience.

[42]  B. Balleine,et al.  Differential dependence of Pavlovian incentive motivation and instrumental incentive learning processes on dopamine signaling. , 2011, Learning & memory.

[43]  Ilana B. Witten,et al.  Recombinase-Driver Rat Lines: Tools, Techniques, and Optogenetic Application to Dopamine-Mediated Reinforcement , 2011, Neuron.

[44]  A. Cooper,et al.  Predictive Reward Signal of Dopamine Neurons , 2011 .

[45]  P. Glimcher Understanding dopamine and reinforcement learning: The dopamine reward prediction error hypothesis , 2011, Proceedings of the National Academy of Sciences.

[46]  K. Deisseroth,et al.  Optogenetic Interrogation of Dopaminergic Modulation of the Multiple Phases of Reward-Seeking Behavior , 2011, The Journal of Neuroscience.

[47]  C. Gerfen,et al.  Modulation of striatal projection systems by dopamine. , 2011, Annual review of neuroscience.

[48]  Anne E Carpenter,et al.  Neuron-type specific signals for reward and punishment in the ventral tegmental area , 2011, Nature.

[49]  Ann Allergy,et al.  O R I G I N a L a R T I C L E S , 2022 .