Dopamine neurons encode the better option in rats deciding between differently delayed or sized rewards

The dopamine system is thought to be involved in making decisions about reward. Here we recorded from the ventral tegmental area in rats learning to choose between differently delayed and sized rewards. As expected, the activity of many putative dopamine neurons reflected reward prediction errors, changing when the value of the reward increased or decreased unexpectedly. During learning, neural responses to reward in these neurons waned and responses to cues that predicted reward emerged. Notably, this cue-evoked activity varied with size and delay. Moreover, when rats were given a choice between two differently valued outcomes, the activity of the neurons initially reflected the more valuable option, even when it was not subsequently selected.

[1]  J. R.,et al.  Quantitative analysis , 1892, Nature.

[2]  R J HERRNSTEIN,et al.  Relative and absolute strength of response as a function of frequency of reinforcement. , 1961, Journal of the experimental analysis of behavior.

[3]  R. Roth,et al.  Comparison of effects of L-dopa, amphetamine and apomorphine on firing rate of rat dopaminergic neurones. , 1973, Nature: New biology.

[4]  A. Grace,et al.  Dopamine auto- and postsynaptic receptors: electrophysiological evidence for differential sensitivity to dopamine agonists. , 1979, Science.

[5]  R. Thaler Some empirical evidence on dynamic inconsistency , 1981 .

[6]  J. Elster,et al.  Choice Over Time , 1992 .

[7]  W. Schultz,et al.  Importance of unpredictability for reward responses in primate dopamine neurons. , 1994, Journal of neurophysiology.

[8]  P. Dayan,et al.  A framework for mesencephalic dopamine systems based on predictive Hebbian learning , 1996, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[9]  Jennifer A. Mangels,et al.  A Neostriatal Habit Learning System in Humans , 1996, Science.

[10]  P. Holland,et al.  Neurotoxic Lesions of Basolateral, But Not Central, Amygdala Interfere with Pavlovian Second-Order Conditioning and Reinforcer Devaluation Effects , 1996, The Journal of Neuroscience.

[11]  J. Evenden,et al.  The pharmacology of impulsive behaviour in rats: the effects of drugs on response choice with varying delays of reinforcement , 1996, Psychopharmacology.

[12]  J. Hollerman,et al.  Dopamine neurons report an error in the temporal prediction of reward during learning , 1998, Nature Neuroscience.

[13]  G. Rebec,et al.  Heterogeneity of ventral tegmental area neurons: Single-unit recording and iontophoresis in awake, unrestrained rats , 1998, Neuroscience.

[14]  G. Schoenbaum,et al.  Orbitofrontal Cortex and Representation of Incentive Value in Associative Learning , 1999, The Journal of Neuroscience.

[15]  S. Mobini,et al.  Theory and method in the quantitative analysis of ”impulsive choice” behaviour: implications for psychopharmacology , 1999, Psychopharmacology.

[16]  A. Tversky,et al.  Choices, Values, and Frames , 2000 .

[17]  B. Everitt,et al.  Limbic cortical-ventral striatal systems underlying appetitive conditioning. , 2000, Progress in brain research.

[18]  E. Murray,et al.  Control of Response Selection by Reinforcer Value Requires Interaction of Amygdala and Orbital Prefrontal Cortex , 2000, The Journal of Neuroscience.

[19]  T. Robbins,et al.  The effects of d-amphetamine, chlordiazepoxide, α-flupenthixol and behavioural manipulations on choice of signalled and unsignalled delayed reinforcement in rats , 2000, Psychopharmacology.

[20]  D. Joel,et al.  The connections of the dopaminergic system with the striatum in rats and primates: an analysis with respect to the functional and compartmental organization of the striatum , 2000, Neuroscience.

[21]  Nikolaus R. McFarland,et al.  Striatonigrostriatal Pathways in Primates Form an Ascending Spiral from the Shell to the Dorsolateral Striatum , 2000, The Journal of Neuroscience.

[22]  T. R. Wade,et al.  Effects of dopaminergic drugs on delayed reward as a measure of impulsive behavior in rats , 2000, Psychopharmacology.

[23]  W. Schultz,et al.  Dopamine responses comply with basic assumptions of formal learning theory , 2001, Nature.

[24]  T. Robbins,et al.  Impulsive Choice Induced in Rats by Lesions of the Nucleus Accumbens Core , 2001, Science.

[25]  W. Schultz Getting Formal with Dopamine and Reward , 2002, Neuron.

[26]  P. Dayan,et al.  Reward, Motivation, and Reinforcement Learning , 2002, Neuron.

[27]  J. Deakin,et al.  Effects of lesions of the orbitofrontal cortex on sensitivity to delayed and probabilistic reinforcement , 2002, Psychopharmacology.

[28]  W. Schultz,et al.  Coding of Predicted Reward Omission by Dopamine Neurons in a Conditioned Inhibition Paradigm , 2003, The Journal of Neuroscience.

[29]  J. O'Doherty,et al.  Encoding Predictive Reward Value in Human Amygdala and Orbitofrontal Cortex , 2003, Science.

[30]  W. Schultz,et al.  Discrete Coding of Reward Probability and Uncertainty by Dopamine Neurons , 2003, Science.

[31]  R. Wise Dopamine, learning and motivation , 2004, Nature Reviews Neuroscience.

[32]  J. Deakin,et al.  Effects of orbital prefrontal cortex dopamine depletion on inter-temporal choice: a quantitative analysis , 2004, Psychopharmacology.

[33]  O. Hikosaka,et al.  Dopamine Neurons Can Represent Context-Dependent Prediction Error , 2004, Neuron.

[34]  Karl J. Friston,et al.  Dissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning , 2004, Science.

[35]  B. Balleine,et al.  Lesions of dorsolateral striatum preserve outcome expectancy but disrupt habit formation in instrumental learning , 2004, The European journal of neuroscience.

[36]  O. Hikosaka,et al.  Reward-predicting activity of dopamine and caudate neurons--a possible mechanism of motivational control of saccadic eye movement. , 2004, Journal of neurophysiology.

[37]  T. Robbins,et al.  Limbic Corticostriatal Systems and Delayed Reinforcement , 2004, Annals of the New York Academy of Sciences.

[38]  T. Robbins,et al.  Contrasting Roles of Basolateral Amygdala and Orbitofrontal Cortex in Impulsive Choice , 2004, The Journal of Neuroscience.

[39]  Y. Shaham,et al.  Central amygdala ERK signaling pathway is critical to incubation of cocaine craving , 2005, Nature Neuroscience.

[40]  W. Pan,et al.  Dopamine Cells Respond to Predicted Events during Classical Conditioning: Evidence for Eligibility Traces in the Reward-Learning Network , 2005, The Journal of Neuroscience.

[41]  W. Schultz,et al.  Adaptive Coding of Reward Value by Dopamine Neurons , 2005, Science.

[42]  P. Glimcher,et al.  Midbrain Dopamine Neurons Encode a Quantitative Reward Prediction Error Signal , 2005, Neuron.

[43]  T. Kalenscher,et al.  Single Units in the Pigeon Brain Integrate Reward Amount and Time-to-Reward in an Impulsive Choice Task , 2005, Current Biology.

[44]  P. Dayan,et al.  Choice values , 2006, Nature Neuroscience.

[45]  E. Vaadia,et al.  Midbrain dopamine neurons encode decisions for future action , 2006, Nature Neuroscience.

[46]  M. Roesch,et al.  Encoding of Time-Discounted Rewards in Orbitofrontal Cortex Is Independent of Value Representation , 2006, Neuron.

[47]  R. Wightman,et al.  Associative learning mediates dynamic shifts in dopamine signaling in the nucleus accumbens , 2007, Nature Neuroscience.

[48]  M. Roesch,et al.  Previous Cocaine Exposure Makes Rats Hypersensitive to Both Delay and Reward Magnitude , 2007, The Journal of Neuroscience.

[49]  R. K. Simpson Nature Neuroscience , 2022 .