Dopamine Neurons Encoding Long-Term Memory of Object Value for Habitual Behavior

Dopamine neurons promote learning by processing recent changes in reward values, such that reward may be maximized. However, such a flexible signal is not suitable for habitual behaviors that are sustained regardless of recent changes in reward outcome. We discovered a type of dopamine neuron in the monkey substantia nigra pars compacta (SNc) that retains past learned reward values stably. After reward values of visual objects are learned, these neurons continue to respond differentially to the objects, even when reward is not expected. Responses are strengthened by repeated learning and are evoked upon presentation of the objects long after learning is completed. These "sustain-type" dopamine neurons are confined to the caudal-lateral SNc and project to the caudate tail, which encodes long-term value memories of visual objects and guides gaze automatically to stably valued objects. This population of dopamine neurons thus selectively promotes learning and retention of habitual behavior.

[1]  A. Graybiel Habits, rituals, and the evaluative brain. , 2008, Annual review of neuroscience.

[2]  Peter Dayan,et al.  A Neural Substrate of Prediction and Reward , 1997, Science.

[3]  Okihide Hikosaka,et al.  Positive and negative modulation of motor response in primate superior colliculus by reward expectation. , 2007, Journal of neurophysiology.

[4]  K. Doya,et al.  Parallel neural networks for learning sequential procedures , 1999, Trends in Neurosciences.

[5]  W. Schultz Responses of midbrain dopamine neurons to behavioral trigger stimuli in the monkey. , 1986, Journal of neurophysiology.

[6]  H. Bergman,et al.  Goal-directed and habitual control in the basal ganglia: implications for Parkinson's disease , 2010, Nature Reviews Neuroscience.

[7]  Anne E Carpenter,et al.  Neuron-type specific signals for reward and punishment in the ventral tegmental area , 2011, Nature.

[8]  Carol A. Seger,et al.  Systems Neuroscience , 2022 .

[9]  T. Robbins,et al.  6-Hydroxydopamine lesions of the nucleus accumbens, but not of the caudate nucleus, attenuate enhanced responding with reward-related stimuli produced by intra-accumbens d-amphetamine , 2004, Psychopharmacology.

[10]  H. Yin,et al.  The role of the basal ganglia in habit formation , 2006, Nature Reviews Neuroscience.

[11]  C. Gerfen The neostriatal mosaic: multiple levels of compartmental organization , 1992, Trends in Neurosciences.

[12]  B. Knowlton,et al.  Concurrent discrimination learning in Parkinson's disease. , 2010, Behavioral neuroscience.

[13]  Ilya E. Monosov,et al.  What and Where Information in the Caudate Tail Guides Saccades to Visual Objects , 2012, The Journal of Neuroscience.

[14]  O. Hikosaka,et al.  Functional territories in primate substantia nigra pars reticulata separately signaling stable and flexible values. , 2015, Journal of neurophysiology.

[15]  Peter Redgrave,et al.  A direct projection from superior colliculus to substantia nigra for detecting salient visual events , 2003, Nature Neuroscience.

[16]  O. Hikosaka,et al.  Functional properties of monkey caudate neurons. III. Activities related to expectation of target and reward. , 1989, Journal of neurophysiology.

[17]  JM Tepper,et al.  GABAA receptor-mediated inhibition of rat substantia nigra dopaminergic neurons by pars reticulata projection neurons , 1995, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[18]  D. Surmeier,et al.  D1 and D2 dopamine-receptor modulation of striatal glutamatergic signaling in striatal medium spiny neurons , 2007, Trends in Neurosciences.

[19]  O. Hikosaka,et al.  Two types of dopamine neuron distinctly convey positive and negative motivational signals , 2009, Nature.

[20]  Richard S. Sutton,et al.  Learning to predict by the methods of temporal differences , 1988, Machine Learning.

[21]  Talia N. Lerner,et al.  Intact-Brain Analyses Reveal Distinct Information Carried by SNc Dopamine Subcircuits , 2015, Cell.

[22]  W. Schultz,et al.  Adaptive Coding of Reward Value by Dopamine Neurons , 2005, Science.

[23]  O. Hikosaka,et al.  Functional properties of monkey caudate neurons. I. Activities related to saccadic eye movements. , 1989, Journal of neurophysiology.

[24]  S. Goto,et al.  Subdivisional involvement of nigrostriatal loop in idiopathic parkinson's disease and striatonigral degeneration , 1989, Annals of neurology.

[25]  Hyoung F. Kim,et al.  Distinct Basal Ganglia Circuits Controlling Behaviors Guided by Flexible and Stable Values , 2013, Neuron.

[26]  M. Ungless,et al.  Phasic excitation of dopamine neurons in ventral VTA by noxious stimuli , 2009, Proceedings of the National Academy of Sciences.

[27]  Rita Z. Goldstein,et al.  The Neurocircuitry of Impaired Insight in Drug Addiction , 2009, Trends in Cognitive Sciences.

[28]  L. Squire,et al.  Robust habit learning in the absence of awareness and independent of the medial temporal lobe , 2005, Nature.

[29]  J. Doyon,et al.  Distinct basal ganglia territories are engaged in early and advanced motor sequence learning. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[30]  J. Deniau,et al.  Neuronal interactions in the substantia nigra pars reticulata through axon collaterals of the projection neurons , 1982, Experimental Brain Research.

[31]  Okihide Hikosaka,et al.  Reward-Dependent Gain and Bias of Visual Responses in Primate Superior Colliculus , 2003, Neuron.

[32]  R. Wurtz,et al.  Activity of superior colliculus in behaving monkey. I. Visual receptive fields of single neurons. , 1972, Journal of neurophysiology.

[33]  O. Hikosaka,et al.  Robust Representation of Stable Object Values in the Oculomotor Basal Ganglia , 2012, The Journal of Neuroscience.

[34]  M. Mishkin,et al.  Visual habit formation in monkeys with neurotoxic lesions of the ventrocaudal neostriatum , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[35]  Hyoung F. Kim,et al.  Parallel basal ganglia circuits for voluntary and automatic behaviour to reach rewards. , 2015, Brain : a journal of neurology.

[36]  Hyoung F. Kim,et al.  Why skill matters , 2013, Trends in Cognitive Sciences.

[37]  Anthony A Grace,et al.  Opposite Influences of Endogenous Dopamine D1 and D2 Receptor Activation on Activity States and Electrophysiological Properties of Striatal Neurons: Studies CombiningIn Vivo Intracellular Recordings and Reverse Microdialysis , 2002, The Journal of Neuroscience.

[38]  A. Grace,et al.  Intracellular and extracellular electrophysiology of nigral dopaminergic neurons—1. Identification and characterization , 1983, Neuroscience.

[39]  O Hikosaka,et al.  Functional properties of monkey caudate neurons. II. Visual and auditory responses. , 1989, Journal of neurophysiology.

[40]  B. Balleine,et al.  Goal-directed instrumental action: contingency and incentive learning and their cortical substrates , 1998, Neuropharmacology.

[41]  E. Richfield,et al.  Comparative distribution of dopamine D‐1 and D‐2 receptors in the basal ganglia of turtles, pigeons, rats, cats, and monkeys , 1987, The Journal of comparative neurology.

[42]  Samuel M. McClure,et al.  BOLD Responses Reflecting Dopaminergic Signals in the Human Ventral Tegmental Area , 2008, Science.

[43]  A. Cooper,et al.  Predictive Reward Signal of Dopamine Neurons , 2011 .

[44]  Shinya Yamamoto,et al.  Reward Value-Contingent Changes of Visual Responses in the Primate Caudate Tail Associated with a Visuomotor Skill , 2013, The Journal of Neuroscience.

[45]  B. Balleine,et al.  Human and Rodent Homologies in Action Control: Corticostriatal Determinants of Goal-Directed and Habitual Action , 2010, Neuropsychopharmacology.

[46]  Hyoung F. Kim,et al.  Separate groups of dopamine neurons innervate caudate head and tail encoding flexible and stable value memories , 2014, Front. Neuroanat..