The striatum: where skills and habits meet.

After more than a century of work concentrating on the motor functions of the basal ganglia, new ideas have emerged, suggesting that the basal ganglia also have major functions in relation to learning habits and acquiring motor skills. We review the evidence supporting the role of the striatum in optimizing behavior by refining action selection and in shaping habits and skills as a modulator of motor repertoires. These findings challenge the notion that striatal learning processes are limited to the motor domain. The learning mechanisms supported by striatal circuitry generalize to other domains, including cognitive skills and emotion-related patterns of action.

[1]  G. A. Miller.  THE PSYCHOLOGICAL REVIEW THE MAGICAL NUMBER SEVEN, PLUS OR MINUS TWO: SOME LIMITS ON OUR CAPACITY FOR PROCESSING INFORMATION 1 , 1956 .

[2]  N. A. Bernshteĭn.  The co-ordination and regulation of movements , 1967 .

[3]  A. Dickinson.  Actions and habits: the development of behavioural autonomy , 1985 .

[4]  O Hikosaka,et al.  Functional properties of monkey caudate neurons. II. Visual and auditory responses. , 1989, Journal of neurophysiology.

[5]  O. Hikosaka,et al.  Functional properties of monkey caudate neurons. III. Activities related to expectation of target and reward. , 1989, Journal of neurophysiology.

[6]  Leslie G. Ungerleider,et al.  Functional MRI evidence for adult motor cortex plasticity during motor skill learning , 1995, Nature.

[7]  A. D. Smith,et al.  The substantia nigra as a site of synaptic integration of functionally diverse information arising from the ventral pallidum and the globus pallidus in the rat , 1996, Neuroscience.

[8]  A. Graybiel.  The basal ganglia and cognitive pattern generators. , 1997, Schizophrenia bulletin.

[9]  O. Hikosaka,et al.  Differential roles of monkey striatum in learning of sequential hand movement , 1997, Experimental Brain Research.

[10]  Andrew G. Barto,et al.  Reinforcement learning , 1998 .

[11]  A. Graybiel.  The Basal Ganglia and Chunking of Action Repertoires , 1998, Neurobiology of Learning and Memory.

[12]  B. Balleine,et al.  Goal-directed instrumental action: contingency and incentive learning and their cortical substrates , 1998, Neuropharmacology.

[13]  C. I. Connolly,et al.  Building neural representations of habits. , 1999, Science.

[14]  K. Doya,et al.  Parallel neural networks for learning sequential procedures , 1999, Trends in Neurosciences.

[15]  M. Kawato,et al.  Change in neuronal firing patterns in the process of motor command generation for the ocular following response. , 2001, Journal of neurophysiology.

[16]  O. Hikosaka,et al.  Differential activation of monkey striatal neurons in the early and late stages of procedural learning , 2002, Experimental Brain Research.

[17]  W. Schultz.  Getting Formal with Dopamine and Reward , 2002, Neuron.

[18]  Michael I. Jordan,et al.  Optimal feedback control as a theory of motor coordination , 2002, Nature Neuroscience.

[19]  A. Graybiel,et al.  Representation of Action Sequence Boundaries by Macaque Prefrontal Cortical Neurons , 2003, Science.

[20]  A. Graybiel,et al.  Activity of striatal neurons reflects dynamic encoding and recoding of procedural memories , 2005, Nature.

[21]  J. Doyon,et al.  Distinct basal ganglia territories are engaged in early and advanced motor sequence learning. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[22]  M. Ann.  The Basal Ganglia and Cognitive Pattern Generators , 2005 .

[23]  H. Yin,et al.  The role of the basal ganglia in habit formation , 2006, Nature Reviews Neuroscience.

[24]  Ivan Toni,et al.  Cerebral Changes during Performance of Overlearned Arbitrary Visuomotor Associations , 2006, The Journal of Neuroscience.

[25]  M. Lévesque,et al.  Raclopride-induced motor consolidation impairment in primates: role of the dopamine type-2 receptor in movement chunking into integrated sequences , 2007, Experimental Brain Research.

[26]  A. Graybiel.  Habits, rituals, and the evaluative brain. , 2008, Annual review of neuroscience.

[27]  A. Graybiel,et al.  Stable encoding of task structure coexists with flexible coding of task events in sensorimotor striatum. , 2009, Journal of neurophysiology.

[28]  J. Doyon,et al.  Contributions of the basal ganglia and functionally related brain structures to motor learning , 2009, Behavioural Brain Research.

[29]  Maxime Levesque,et al.  Motor sequence learning in primate: Role of the D2 receptor in movement chunking during consolidation , 2009, Behavioural Brain Research.

[30]  O. Hikosaka,et al.  Two types of dopamine neuron distinctly convey positive and negative motivational signals , 2009, Nature.

[31]  A. Dickinson,et al.  Parallel and interactive learning processes within the basal ganglia: Relevance for the understanding of addiction , 2009, Behavioural Brain Research.

[32]  Noah D. Goodman,et al.  Optimal habits can develop spontaneously through sensitivity to local cost , 2010, Proceedings of the National Academy of Sciences.

[33]  Joseph T. McGuire,et al.  Decision making and the avoidance of cognitive demand. , 2010, Journal of experimental psychology. General.

[34]  Xin Jin,et al.  Start/stop signals emerge in nigrostriatal circuits during sequence learning , 2010, Nature.

[35]  A. Graybiel,et al.  Differential Dynamics of Activity Changes in Dorsolateral and Dorsomedial Striatal Loops during Learning , 2010, Neuron.

[36]  Michel Desmurget,et al.  Motor Sequences and the Basal Ganglia: Kinematics, Not Habits , 2010, The Journal of Neuroscience.

[37]  Maxime J Parent,et al.  Movement chunking during sequence learning is a dopamine-dependant process: a study conducted in Parkinson’s disease , 2010, Experimental Brain Research.

[38]  M. Frank,et al.  Neurogenetics and Pharmacology of Learning, Motivation, and Cognition , 2011, Neuropsychopharmacology.

[39]  N. Volkow,et al.  Addiction: Beyond dopamine reward circuitry , 2011, Proceedings of the National Academy of Sciences.

[40]  N. Daw,et al.  Signals in Human Striatum Are Appropriate for Policy Update Rather than Value Prediction , 2011, The Journal of Neuroscience.

[41]  Catherine Stamoulis,et al.  Advance cueing produces enhanced action-boundary patterns of spike activity in the sensorimotor striatum. , 2011, Journal of neurophysiology.

[42]  Scott T. Grafton,et al.  Differential Recruitment of the Sensorimotor Putamen and Frontoparietal Cortex during Motor Chunking in Humans , 2012, Neuron.

[43]  L. Epstein,et al.  Multilocus Genetic Composite Reflecting Dopamine Signaling Capacity Predicts Reward Circuitry Responsivity , 2012, The Journal of Neuroscience.

[44]  Kelly R. Tan,et al.  Ventral tegmental area GABA projections pause accumbal cholinergic interneurons to enhance associative learning , 2012, Nature.

[45]  T. Bliss,et al.  Distributed synergistic plasticity and cerebellar learning , 2012 .

[46]  Kyle S. Smith,et al.  Reversible online control of habitual behavior by optogenetic perturbation of medial prefrontal cortex , 2012, Proceedings of the National Academy of Sciences.

[47]  Aymar de Rugy,et al.  Muscle Coordination Is Habitual Rather than Optimal , 2012, The Journal of Neuroscience.

[48]  Jane R. Garrison,et al.  Prediction error in reinforcement learning: A meta-analysis of neuroimaging studies , 2013, Neuroscience & Biobehavioral Reviews.

[49]  W. Schultz,et al.  Activity of striatal neurons reflects social action and own reward , 2013, Proceedings of the National Academy of Sciences.

[50]  M. Egan,et al.  Dopaminergic therapy removal differentially effects learning in schizophrenia and Parkinson’s disease , 2013, Schizophrenia Research.

[51]  Carlos Diuk,et al.  Hierarchical Learning Induces Two Simultaneous, But Separable, Prediction Errors in Human Basal Ganglia , 2013, The Journal of Neuroscience.

[52]  M. Kimura,et al.  Coding of the long-term value of multiple future rewards in the primate striatum. , 2013, Journal of neurophysiology.

[53]  T. Robbins,et al.  Neuroscience and Biobehavioral Reviews Review from the Ventral to the Dorsal Striatum: Devolving Views of Their Roles in Drug Addiction , 2022 .

[54]  Kyle S. Smith,et al.  A dual operator view of habitual behavior reflecting cortical and striatal dynamics. , 2013, Neuron.

[55]  A. Graybiel,et al.  Prolonged Dopamine Signalling in Striatum Signals Proximity and Value of Distant Rewards , 2013, Nature.

[56]  Jörn Diedrichsen,et al.  Skill learning strengthens cortical representations of motor sequences , 2013, eLife.

[57]  Ann M Graybiel,et al.  Selective Effects of Dopamine Depletion and L-DOPA Therapy on Learning-Related Firing Dynamics of Striatal Neurons , 2013, The Journal of Neuroscience.

[58]  N. Volkow,et al.  Stimulant-induced dopamine increases are markedly blunted in active cocaine abusers , 2014, Molecular Psychiatry.

[59]  Stefan Pollmann,et al.  A universal role of the ventral striatum in reward-based learning: Evidence from human studies , 2014, Neurobiology of Learning and Memory.

[60]  P. Apicella,et al.  Cortical and Thalamic Excitation Mediate the Multiphasic Responses of Striatal Cholinergic Interneurons to Motivationally Salient Stimuli , 2014, The Journal of Neuroscience.

[61]  A. Graybiel,et al.  Neurons in the Ventral Striatum Exhibit Cell-Type-Specific Representations of Outcome during Learning , 2014, Neuron.

[62]  Frank Telang,et al.  Decreased dopamine brain reactivity in marijuana abusers is associated with negative emotionality and addiction severity , 2014, Proceedings of the National Academy of Sciences.

[63]  J. Haynes,et al.  Encoding of sequence boundaries in the subthalamic nucleus of patients with Parkinson's disease. , 2014, Brain : a journal of neurology.

[64]  Scott T. Grafton,et al.  Multi-faceted aspects of chunking enable robust algorithms 1 Running head : ALGORITHM FOR CHUNKING INFERENCE Multi-faceted aspects of chunking enable robust algorithms , 2014 .

[65]  Vincent P Ferrera,et al.  Representation of Outcome Risk and Action in the Anterior Caudate Nucleus , 2014, The Journal of Neuroscience.

[66]  Dopamine function and the efficiency of human movement , 2014 .

[67]  C. A. French,et al.  Humanized Foxp2 accelerates learning by enhancing transitions from declarative to procedural performance , 2014, Proceedings of the National Academy of Sciences.

[68]  Danielle S Bassett,et al.  Learning-induced autonomy of sensorimotor systems , 2014, Nature Neuroscience.

[69]  Scott T Grafton,et al.  The Human Motor System Supports Sequence-Specific Representations over Multiple Training-Dependent Timescales. , 2015, Cerebral cortex.

[70]  Peter Stone,et al.  Reinforcement learning , 2019, Scholarpedia.