Dopamine transients are sufficient and necessary for acquisition of model-based associations

In the version of this article initially published, the laser activation at the start of cue X in experiment 1 was described in the first paragraph of the Results and in the third paragraph of the Experiment 1 section of the Methods as lasting 2 s; in fact, it lasted only 1 s. The error has been corrected in the HTML and PDF versions of the article.

[1]  W. Brogden.  Sensory pre-conditioning. , 1939 .

[2]  E. Tolman.  Cognitive maps in rats and men. , 1948, Psychological review.

[3]  R. Rescorla,et al.  Associations in second-order conditioning and sensory preconditioning. , 1972, Journal of comparative and physiological psychology.

[4]  N. Mackintosh.  A Theory of Attention: Variations in the Associability of Stimuli with Reinforcement , 1975 .

[5]  R. Rescorla,et al.  The effect of two ways of devaluing the unconditioned stimulus after first- and second-order appetitive conditioning. , 1975, Journal of experimental psychology. Animal behavior processes.

[6]  P. Holland.  Conditioned stimulus as a determinant of the form of the Pavlovian conditioned response. , 1977, Journal of experimental psychology. Animal behavior processes.

[7]  J. Pearce,et al.  A model for Pavlovian learning: Variations in the effectiveness of conditioned but not of unconditioned stimuli. , 1980 .

[8]  P. Holland,et al.  Effects of amygdala central nucleus lesions on blocking and unblocking. , 1993, Behavioral neuroscience.

[9]  R. Colwill.  An Associative Analysis of Instrumental Learning , 1993 .

[10]  B. Balleine,et al.  Motivational control of goal-directed action , 1994 .

[11]  J. Horvitz,et al.  Burst activity of ventral tegmental dopamine neurons is elicited by sensory stimuli in the awake cat , 1997, Brain Research.

[12]  Peter Dayan,et al.  A Neural Substrate of Prediction and Reward , 1997, Science.

[13]  W. Schultz.  Dopamine neurons and their role in reward mechanisms , 1997, Current Opinion in Neurobiology.

[14]  J. Hollerman,et al.  Dopamine neurons report an error in the temporal prediction of reward during learning , 1998, Nature Neuroscience.

[15]  Peter Dayan,et al.  Dopamine: generalization and bonuses , 2002, Neural Networks.

[16]  W. Schultz,et al.  Coding of Predicted Reward Omission by Dopamine Neurons in a Conditioned Inhibition Paradigm , 2003, The Journal of Neuroscience.

[17]  G. Hall,et al.  Preserved Sensitivity to Outcome Value after Lesions of the Basolateral Amygdala , 2003, The Journal of Neuroscience.

[18]  P. Holland.  Relations between Pavlovian-instrumental transfer and reinforcer devaluation. , 2004, Journal of experimental psychology. Animal behavior processes.

[19]  W. Pan,et al.  Dopamine Cells Respond to Predicted Events during Classical Conditioning: Evidence for Eligibility Traces in the Reward-Learning Network , 2005, The Journal of Neuroscience.

[20]  P. Holland,et al.  Variations in unconditioned stimulus processing in unblocking. , 2005, Journal of experimental psychology. Animal behavior processes.

[21]  P. Dayan,et al.  Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control , 2005, Nature Neuroscience.

[22]  R. Wightman,et al.  Associative learning mediates dynamic shifts in dopamine signaling in the nucleus accumbens , 2007, Nature Neuroscience.

[23]  S. Lammel,et al.  Unique Properties of Mesoprefrontal Neurons within a Dual Mesocorticolimbic Dopamine System , 2008, Neuron.

[24]  J. Paul Bolam,et al.  Faculty Opinions recommendation of Unique properties of mesoprefrontal neurons within a dual mesocorticolimbic dopamine system. , 2008 .

[25]  Geoffrey Schoenbaum,et al.  The role of the orbitofrontal cortex in the pursuit of happiness and more specific rewards , 2008, Nature.

[26]  Samuel M. McClure,et al.  BOLD Responses Reflecting Dopaminergic Signals in the Human Ventral Tegmental Area , 2008, Science.

[27]  Adam Johnson,et al.  Looking for cognition in the structure within the noise , 2009, Trends in Cognitive Sciences.

[28]  K. Deisseroth,et al.  Phasic Firing in Dopaminergic Neurons Is Sufficient for Behavioral Conditioning , 2009, Science.

[29]  M. Roesch,et al.  The Orbitofrontal Cortex and Ventral Tegmental Area Are Necessary for Learning from Unexpected Outcomes , 2009, Neuron.

[30]  P. Dayan,et al.  States versus Rewards: Dissociable Neural Prediction Error Signals Underlying Model-Based and Model-Free Reinforcement Learning , 2010, Neuron.

[31]  Simon Hong,et al.  A pallidus-habenula-dopamine pathway signals inferred stimulus values. , 2010, Journal of neurophysiology.

[32]  Ilana B. Witten,et al.  Recombinase-Driver Rat Lines: Tools, Techniques, and Optogenetic Application to Dopamine-Mediated Reinforcement , 2011, Neuron.

[33]  K. Deisseroth,et al.  Optogenetic Interrogation of Dopaminergic Modulation of the Multiple Phases of Reward-Seeking Behavior , 2011, The Journal of Neuroscience.

[34]  P. Dayan,et al.  Model-based influences on humans’ choices and striatal prediction errors , 2011, Neuron.

[35]  Y. Niv,et al.  Ventral Striatum and Orbitofrontal Cortex Are Both Required for Model-Based, But Not Model-Free, Reinforcement Learning , 2011, The Journal of Neuroscience.

[36]  Guillem R. Esber,et al.  Reconciling the influence of predictiveness and uncertainty on stimulus salience: a model of attention in associative learning , 2011, Proceedings of the Royal Society B: Biological Sciences.

[37]  Anne E Carpenter,et al.  Neuron-type specific signals for reward and punishment in the ventral tegmental area , 2011, Nature.

[38]  D. Shohamy,et al.  Preference by Association: How Memory Mechanisms in the Hippocampus Bias Decisions , 2012, Science.

[39]  Joshua L. Jones,et al.  Orbitofrontal Cortex Supports Behavior and Learning Using Inferred But Not Cached Values , 2012, Science.

[40]  Josiah R. Boivin,et al.  A Causal Link Between Prediction Errors, Dopamine Neurons and Learning , 2013, Nature Neuroscience.

[41]  S. Ikemoto,et al.  Similar Roles of Substantia Nigra and Ventral Tegmental Dopamine Neurons in Reward and Aversion , 2014, The Journal of Neuroscience.

[42]  S. Floresco,et al.  Overriding Phasic Dopamine Signals Redirects Action Selection during Risk/Reward Decision Making , 2014, Neuron.

[43]  H. Nakahara.  Multiplexing signals in reinforcement learning with internal models and dopamine , 2014, Current Opinion in Neurobiology.

[44]  S. Killcross,et al.  The prelimbic cortex contributes to the down-regulation of attention toward redundant cues. , 2014, Cerebral cortex.

[45]  S. Robinson,et al.  Chemogenetic Silencing of Neurons in Retrosplenial Cortex Disrupts Sensory Preconditioning , 2014, The Journal of Neuroscience.

[46]  R. Dolan,et al.  Ventral striatal dopamine reflects behavioral and neural signatures of model-based control during sequential decision making , 2015, Proceedings of the National Academy of Sciences.

[47]  Naoshige Uchida,et al.  Erratum: Arithmetic and local circuitry underlying dopamine prediction errors , 2015, Nature.

[48]  Ilana B. Witten,et al.  Reward and choice encoding in terminals of midbrain dopamine neurons depends on striatal target , 2016, Nature Neuroscience.

[49]  M. Poo,et al.  Phasic dopamine release in the medial prefrontal cortex enhances stimulus discrimination , 2016, Proceedings of the National Academy of Sciences.

[50]  Guillem R. Esber,et al.  Brief optogenetic inhibition of dopamine neurons mimics endogenous negative reward prediction errors , 2015, Nature Neuroscience.

[51]  K. Wassum,et al.  Nucleus accumbens core dopamine signaling tracks the need‐based motivational value of food‐paired cues , 2016, Journal of neurochemistry.

[52]  Geoffrey Schoenbaum,et al.  Midbrain dopamine neurons compute inferred and cached value prediction errors in a common framework , 2016, eLife.

[53]  N. Uchida,et al.  Dopamine neurons share common response function for reward prediction error , 2016, Nature Neuroscience.

[54]  G. Stuber,et al.  Physiological state gates acquisition and expression of mesolimbic reward prediction signals , 2016, Proceedings of the National Academy of Sciences.