Expectancy-related changes in firing of dopamine neurons depend on orbitofrontal cortex

The orbitofrontal cortex has been hypothesized to carry information regarding the value of expected rewards. Such information is essential for associative learning, which relies on comparisons between expected and obtained reward for generating instructive error signals. These error signals are thought to be conveyed by dopamine neurons. To test whether orbitofrontal cortex contributes to these error signals, we recorded from dopamine neurons in orbitofrontal-lesioned rats performing a reward learning task. Lesions caused marked changes in dopaminergic error signaling. However, the effect of lesions was not consistent with a simple loss of information regarding expected value. Instead, without orbitofrontal input, dopaminergic error signals failed to reflect internal information about the impending response that distinguished externally similar states leading to differently valued future rewards. These results are consistent with current conceptualizations of orbitofrontal cortex as supporting model-based behavior and suggest an unexpected role for this information in dopaminergic error signaling.

[1]  W. F. Prokasy,et al.  Classical conditioning II: Current research and theory. , 1972 .

[2]  J. Pearce,et al.  A model for Pavlovian learning: variations in the effectiveness of conditioned but not of unconditioned stimuli. , 1980, Psychological review.

[3]  A. Grace,et al.  The control of firing pattern in nigral dopamine neurons: burst firing , 1984, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[4]  Peter Dayan,et al.  A Neural Substrate of Prediction and Reward , 1997, Science.

[5]  J. Hollerman,et al.  Dopamine neurons report an error in the temporal prediction of reward during learning , 1998, Nature Neuroscience.

[6]  H Eichenbaum,et al.  Neural Correlates of Olfactory Recognition Memory in the Rat Orbitofrontal Cortex , 2000, The Journal of Neuroscience.

[7]  J. O'Doherty,et al.  Neural Responses during Anticipation of a Primary Taste Reward , 2002, Neuron.

[8]  M. Farah,et al.  Ventromedial frontal cortex mediates affective shifting in humans: evidence from a reversal learning paradigm. , 2003, Brain : a journal of neurology.

[9]  J. O'Doherty,et al.  Encoding Predictive Reward Value in Human Amygdala and Orbitofrontal Cortex , 2003, Science.

[10]  T. Robbins,et al.  Dissociable Contributions of the Orbitofrontal and Infralimbic Cortex to Pavlovian Autoshaping and Discrimination Reversal Learning: Further Evidence for the Functional Heterogeneity of the Rodent Frontal Cortex , 2003, The Journal of Neuroscience.

[11]  Geoffrey Schoenbaum,et al.  Different Roles for Orbitofrontal Cortex and Basolateral Amygdala in a Reinforcer Devaluation Task , 2003, The Journal of Neuroscience.

[12]  Karl J. Friston,et al.  Dissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning , 2004, Science.

[13]  E. Murray,et al.  Bilateral Orbital Prefrontal Cortex Lesions in Rhesus Monkeys Disrupt Choices Guided by Both Reward Value and Reward Contingency , 2004, The Journal of Neuroscience.

[14]  T. Robbins,et al.  Putting a spin on the dorsal–ventral divide of the striatum , 2004, Trends in Neurosciences.

[15]  W. Pan,et al.  Dopamine Cells Respond to Predicted Events during Classical Conditioning: Evidence for Eligibility Traces in the Reward-Learning Network , 2005, The Journal of Neuroscience.

[16]  P. Dayan,et al.  Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control , 2005, Nature Neuroscience.

[17]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[18]  P. Glimcher,et al.  Midbrain Dopamine Neurons Encode a Quantitative Reward Prediction Error Signal , 2005, Neuron.

[19]  N. Waller.  Carving nature at its joints: Paul Meehl's development of taxometrics. , 2006, Journal of abnormal psychology.

[20]  P. Dayan,et al.  Choice values , 2006, Nature Neuroscience.

[21]  C. Padoa-Schioppa,et al.  Neurons in the orbitofrontal cortex encode economic value , 2006, Nature.

[22]  M. Quirk,et al.  Representation of Spatial Goals in Rat Orbitofrontal Cortex , 2006, Neuron.

[23]  J. O'Doherty,et al.  The Role of the Ventromedial Prefrontal Cortex in Abstract State-Based Inference during Decision Making in Humans , 2006, The Journal of Neuroscience.

[24]  Elyssa B. Margolis,et al.  The ventral tegmental area revisited: is there an electrophysiological marker for dopaminergic neurons? , 2006, The Journal of physiology.

[25]  E. Vaadia,et al.  Midbrain dopamine neurons encode decisions for future action , 2006, Nature Neuroscience.

[26]  M. Roesch,et al.  Encoding of Time-Discounted Rewards in Orbitofrontal Cortex Is Independent of Value Representation , 2006, Neuron.

[27]  Jadin C. Jackson,et al.  Reconciling reinforcement learning models with behavioral extinction and renewal: implications for addiction, relapse, and problem gambling. , 2007, Psychological review.

[28]  B. Balleine,et al.  Orbitofrontal Cortex Mediates Outcome Encoding in Pavlovian But Not Instrumental Conditioning , 2007, The Journal of Neuroscience.

[29]  M. Roesch,et al.  Dopamine neurons encode the better option in rats deciding between differently delayed or sized rewards , 2007, Nature Neuroscience.

[30]  C. Pennartz,et al.  Population Coding of Reward Magnitude in the Orbitofrontal Cortex of the Rat , 2008, The Journal of Neuroscience.

[31]  Geoffrey Schoenbaum,et al.  The role of the orbitofrontal cortex in the pursuit of happiness and more specific rewards , 2008, Nature.

[32]  Tomoyuki Furuyashiki,et al.  Rat Orbitofrontal Cortex Separately Encodes Response and Outcome Information during Performance of Goal-Directed Behavior , 2008, The Journal of Neuroscience.

[33]  F. Artigas,et al.  Pyramidal Neurons in Rat Prefrontal Cortex Projecting to Ventral Tegmental Area and Dorsal Raphe Nucleus Express 5-HT2A Receptors , 2008, Cerebral cortex.

[34]  Aldo Genovesio,et al.  Monkey Orbitofrontal Cortex Encodes Response Choices Near Feedback Time , 2009, The Journal of Neuroscience.

[35]  C. Pennartz,et al.  Single-Cell and Population Coding of Expected Reward Probability in the Orbitofrontal Cortex of the Rat , 2009, The Journal of Neuroscience.

[36]  M. Roesch,et al.  The Orbitofrontal Cortex and Ventral Tegmental Area Are Necessary for Learning from Unexpected Outcomes , 2009, Neuron.

[37]  M. Roesch,et al.  A new perspective on the role of the orbitofrontal cortex in adaptive behaviour , 2009, Nature Reviews Neuroscience.

[38]  D. Blei,et al.  Context, learning, and extinction. , 2010, Psychological review.

[39]  Y. Niv,et al.  Learning latent structure: carving nature at its joints , 2010, Current Opinion in Neurobiology.

[40]  P. Dayan,et al.  States versus Rewards: Dissociable Neural Prediction Error Signals Underlying Model-Based and Model-Free Reinforcement Learning , 2010, Neuron.

[41]  L. Fellows,et al.  Beyond Reversal: A Critical Role for Human Orbitofrontal Cortex in Flexible Learning from Probabilistic Feedback , 2010, The Journal of Neuroscience.

[42]  Simon Hong,et al.  A pallidus-habenula-dopamine pathway signals inferred stimulus values. , 2010, Journal of neurophysiology.

[43]  Jung Hoon Sul,et al.  Distinct Roles of Rodent Orbitofrontal and Medial Prefrontal Cortex in Decision Making , 2010, Neuron.

[44]  Xin Jin,et al.  Start/stop signals emerge in nigrostriatal circuits during sequence learning , 2010, Nature.

[45]  Timothy Edward John Behrens,et al.  Separable Learning Systems in the Macaque Brain and the Role of Orbitofrontal Cortex in Contingent Learning , 2010, Neuron.

[46]  D. Lodge.  The Medial Prefrontal and Orbitofrontal Cortices Differentially Regulate Dopamine System Function , 2011, Neuropsychopharmacology.

[47]  Dylan A. Simon,et al.  Neural Correlates of Forward Planning in a Spatial Decision Task in Humans , 2011, The Journal of Neuroscience.

[48]  P. Dayan,et al.  Model-based influences on humans’ choices and striatal prediction errors , 2011, Neuron.

[49]  Y. Niv,et al.  Ventral Striatum and Orbitofrontal Cortex Are Both Required for Model-Based, But Not Model-Free, Reinforcement Learning , 2011, The Journal of Neuroscience.

[50]  Daeyeol Lee,et al.  Distributed Coding of Actual and Hypothetical Outcomes in the Orbital and Dorsolateral Prefrontal Cortex , 2011, Neuron.

[51]  Michael O'Rourke,et al.  Carving Nature at its Joints , 2011 .

[52]  M. Shapiro,et al.  Dynamic Coding of Goal-Directed Paths by Orbital Prefrontal Cortex , 2011, The Journal of Neuroscience.