Separate neural substrates for skill learning and performance in the ventral and dorsal striatum

It is widely accepted that the striatum of the basal ganglia is a primary substrate for the learning and performance of skills. We provide evidence that two regions of the rat striatum, ventral and dorsal, play distinct roles in instrumental conditioning (skill learning), with the ventral striatum being critical for learning and the dorsal striatum being important for performance but, notably, not for learning. This implies an actor (dorsal) versus director (ventral) division of labor, which is a new variant of the widely discussed actor-critic architecture. Our results also imply that the successful performance of a skill can ultimately result in its establishment as a habit outside the basal ganglia.

[1]  Stanley J. Watson,et al.  The rat brain in stereotaxic coordinates (2nd edn) by George Paxinos and Charles Watson, Academic Press, 1986. £40.00/$80.00 (264 pages) ISBN 012 547 6213 , 1987, Trends in Neurosciences.

[2]  M. Packard,et al.  Differential effects of fornix and caudate nucleus lesions on two radial maze tasks: evidence for multiple memory systems , 1989, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[3]  G. E. Alexander,et al.  Basal ganglia-thalamocortical circuits: parallel substrates for motor, oculomotor, "prefrontal" and "limbic" functions. , 1990, Progress in brain research.

[4]  J. D. McGaugh,et al.  Double dissociation of fornix and caudate nucleus lesions on acquisition of two water maze tasks: further evidence for multiple memory systems. , 1992, Behavioral neuroscience.

[5]  B. Balleine,et al.  Motivational control of goal-directed action , 1994 .

[6]  D. Joel,et al.  The organization of the basal ganglia-thalamocortical circuits: Open interconnected rather than closed segregated , 1994, Neuroscience.

[7]  R. Boakes,et al.  Motivational control after extended instrumental training , 1995 .

[8]  S P Wise,et al.  Distributed modular architectures linking basal ganglia, cerebellum, and cerebral cortex: their role in planning and controlling action. , 1995, Cerebral cortex.

[9]  Jennifer A. Mangels,et al.  A Neostriatal Habit Learning System in Humans , 1996, Science.

[10]  P. Holland,et al.  The Role of an Amygdalo-Nigrostriatal Pathway in Associative Learning , 1997, The Journal of Neuroscience.

[11]  J. Hollerman,et al.  Dopamine neurons report an error in the temporal prediction of reward during learning , 1998, Nature Neuroscience.

[12]  Carol A. Seger,et al.  Striatal activation during acquisition of a cognitive skill. , 1999, Neuropsychology.

[13]  D. Joel,et al.  The connections of the dopaminergic system with the striatum in rats and primates: an analysis with respect to the functional and compartmental organization of the striatum , 2000, Neuroscience.

[14]  Nikolaus R. McFarland,et al.  Striatonigrostriatal Pathways in Primates Form an Ascending Spiral from the Shell to the Dorsolateral Striatum , 2000, The Journal of Neuroscience.

[15]  H. Eichenbaum,et al.  Cognitive task performance after lidocaine-induced inactivation of different sites within the basolateral amygdala and dorsal striatum. , 2001, Behavioral neuroscience.

[16]  B. Balleine,et al.  The Role of the Nucleus Accumbens in Instrumental Conditioning: Evidence of a Functional Dissociation between Accumbens Core and Shell , 2001, The Journal of Neuroscience.

[17]  O. Hikosaka,et al.  Differential activation of monkey striatal neurons in the early and late stages of procedural learning , 2002, Experimental Brain Research.

[18]  W. Schultz Getting Formal with Dopamine and Reward , 2002, Neuron.

[19]  P. Dayan,et al.  Reward, Motivation, and Reinforcement Learning , 2002, Neuron.

[20]  A. Kelley,et al.  Early consolidation of instrumental learning requires protein synthesis in the nucleus accumbens , 2002, Nature Neuroscience.

[21]  R. Reep,et al.  The associative striatum: Organization of cortical projections to the dorsocentral striatum in rats , 2003, The Journal of comparative neurology.

[22]  Karl J. Friston,et al.  Temporal Difference Models and Reward-Related Learning in the Human Brain , 2003, Neuron.

[23]  P. Holland Relations between Pavlovian-instrumental transfer and reinforcer devaluation. , 2004, Journal of experimental psychology. Animal behavior processes.

[24]  A. Dickinson,et al.  Prediction Error during Retrospective Revaluation of Causal Associations in Humans fMRI Evidence in Favor of an Associative Model of Learning , 2004, Neuron.

[25]  R. J. McDonald,et al.  Dorsal striatum and stimulus–response learning: lesions of the dorsolateral, but not dorsomedial, striatum impair acquisition of a simple discrimination task , 2004, Behavioural Brain Research.

[26]  Karl J. Friston,et al.  Dissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning , 2004, Science.

[27]  Michael J. Frank,et al.  Hippocampus, cortex, and basal ganglia: Insights from computational models of complementary learning systems , 2004, Neurobiology of Learning and Memory.

[28]  B. Balleine,et al.  Lesions of dorsolateral striatum preserve outcome expectancy but disrupt habit formation in instrumental learning , 2004, The European journal of neuroscience.

[29]  A. Kelley,et al.  Central amygdalar and dorsal striatal NMDA receptor involvement in instrumental learning and spontaneous behavior. , 2004, Behavioral neuroscience.

[30]  E. Miller,et al.  Different time courses of learning-related activity in the prefrontal cortex and striatum , 2005, Nature.

[31]  Carol A. Seger,et al.  The Roles of the Caudate Nucleus in Human Classification Learning , 2005, The Journal of Neuroscience.

[32]  B. Balleine,et al.  Double Dissociation of Basolateral and Central Amygdala Lesions on the General and Outcome-Specific Forms of Pavlovian-Instrumental Transfer , 2005, The Journal of Neuroscience.

[33]  Michael J. Frank,et al.  Dynamic Dopamine Modulation in the Basal Ganglia: A Neurocomputational Account of Cognitive Deficits in Medicated and Nonmedicated Parkinsonism , 2005, Journal of Cognitive Neuroscience.

[34]  B. Balleine Neural bases of food-seeking: Affect, arousal and reward in corticostriatolimbic circuits , 2005, Physiology & Behavior.

[35]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[36]  B. Balleine,et al.  The role of the dorsomedial striatum in instrumental conditioning , 2005, The European journal of neuroscience.

[37]  R. Reep,et al.  Overlap and interdigitation of cortical and thalamic afferents to dorsocentral striatum in the rat , 2005, Brain Research.

[38]  M. Kawato,et al.  Different neural correlates of reward expectation and reward expectation error in the putamen and caudate nucleus during stimulus-action-reward association learning. , 2006, Journal of neurophysiology.

[39]  S. Haber,et al.  Reward-Related Cortical Inputs Define a Large Striatal Region in Primates That Interface with Associative Cortical Connections, Providing a Substrate for Incentive-Based Learning , 2006, The Journal of Neuroscience.

[40]  A. Kelley,et al.  Dynamic shifts in corticostriatal expression patterns of the immediate early genes Homer 1a and Zif268 during early and late phases of instrumental training. , 2006, Learning & memory.

[41]  Ziv M. Williams,et al.  Selective enhancement of associative learning by microstimulation of the anterior caudate , 2006, Nature Neuroscience.

[42]  Michael J. Frank,et al.  Making Working Memory Work: A Computational Model of Learning in the Prefrontal Cortex and Basal Ganglia , 2006, Neural Computation.

[43]  R. Poldrack,et al.  Ventral–striatal/nucleus–accumbens sensitivity to prediction errors during classification learning , 2006, Human brain mapping.

[44]  R. Dolan,et al.  Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans , 2006, Nature.

[45]  Kae Nakamura,et al.  Role of Dopamine in the Primate Caudate Nucleus in Reward Modulation of Saccades , 2006, The Journal of Neuroscience.

[46]  M. Frank,et al.  Anatomy of a decision: striato-orbitofrontal interactions in reinforcement learning, decision making, and reversal. , 2006, Psychological review.

[47]  R. K. Simpson Nature Neuroscience , 2022 .