Cerebellar damage limits reinforcement learning.

This scientific commentary refers to ‘Effective reinforcement learning following cerebellar damage requires a balance between exploration and motor noise’, by Therrien et al. (doi:10.1093/brain/awv329) . An exciting challenge for research in motor learning is to disentangle the multiple processes involved, and to tie these down to distinct neural systems. About 17 years ago, Doya proposed that the cerebellum, basal ganglia and cerebral cortex were separately responsible for supervised learning, reinforcement learning, and unsupervised learning, respectively (Doya, 1999, 2000). Supervised learning is driven, unsurprisingly, by signals provided by a ‘supervisor’ and is typically equated with error-based learning: after an action, an error in performance is processed, and subsequent actions are adjusted to try to minimize the error. Reinforcement learning is driven by rewards and punishments: exploratory actions are tried out and each action’s outcome is evaluated; learning aims to maximize the value of future action choices. Unsupervised learning occurs in the face of repeated experience of the environment, and generates a mapping of its statistical regularities: it can be driven by Hebbian learning so that, for example, similar sensory events become associated with one another. In the motor domain this can equate to yet another form of learning, use-dependent learning, where there is a bias to produce actions more similar to previous ones. For many years, these learning processes were thought of as functionally and anatomically independent. However, huge efforts are now being made to understand how these various processes interact. In this issue of Brain , Therrien, Wolpert and Bastian have added to these efforts by testing and modelling how patients with cerebellar ataxia differ from healthy controls in performing error- and reinforcement-learning tasks (Therrien et al. , 2015). Using a mechanistic model, they show that optimal learning with reinforcement feedback requires subjects to balance the variability …

[1]  Kenji Doya,et al.  What are the computations of the cerebellum, the basal ganglia and the cerebral cortex? , 1999, Neural Networks.

[2]  K. Doya.  Complementary roles of basal ganglia and cerebellum in learning and motor control , 2000, Current Opinion in Neurobiology.

[3]  D. Selkoe.  Alzheimer's Disease Is a Synaptic Failure , 2002, Science.

[4]  B. Pakkenberg,et al.  Aging and the human neocortex , 2003, Experimental Gerontology.

[5]  J. Krakauer,et al.  Sensory prediction errors drive cerebellum-dependent adaptation of reaching. , 2007, Journal of neurophysiology.

[6]  John D. Lambris,et al.  The Classical Complement Cascade Mediates CNS Synapse Elimination , 2007, Cell.

[7]  L. Christensen,et al.  University of Birmingham Disruption of state estimation in the human lateral cerebellum , 2007 .

[8]  J. DeLuca,et al.  Cognitive impairment in multiple sclerosis , 2008, The Lancet Neurology.

[9]  M. Feller,et al.  Mechanisms underlying development of visual maps and receptive fields. , 2008, Annual review of neuroscience.

[10]  Peter L. Strick,et al.  The Cerebellum and Basal Ganglia are Interconnected , 2010, Neuropsychology Review.

[11]  B. Trapp,et al.  Demyelination causes synaptic alterations in hippocampi from multiple sclerosis patients , 2011, Annals of neurology.

[12]  Reza Shadmehr,et al.  Learning from Sensory and Reward Prediction Errors during Motor Adaptation , 2011, PLoS Comput. Biol..

[13]  Raymond J. Delnicki,et al.  Overcoming Motor “Forgetting” Through Reinforcement Of Learned Actions , 2012, The Journal of Neuroscience.

[14]  Chiara Romualdi,et al.  The changing clinical course of multiple sclerosis: A matter of gray matter , 2013, Annals of neurology.

[15]  Manuel A. Friese,et al.  Mechanisms of neurodegeneration and axonal dysfunction in multiple sclerosis , 2014, Nature Reviews Neurology.

[16]  J. Rothwell,et al.  The dissociable effects of punishment and reward on motor learning , 2015, Nature Neuroscience.

[17]  Manuel A. Friese,et al.  Immunopathology of multiple sclerosis , 2015, Nature Reviews Immunology.

[18]  F. Baas,et al.  Complement C1q‐C3–associated synaptic changes in multiple sclerosis hippocampus , 2015, Annals of neurology.

[19]  Sarah E. Pekny,et al.  Reward-Dependent Modulation of Movement Variability , 2015, The Journal of Neuroscience.

[20]  W. Brück,et al.  Reconstruction of single cortical projection neurons reveals primary spine loss in multiple sclerosis. , 2016, Brain : a journal of neurology.

[21]  D. Wolpert,et al.  Effective reinforcement learning following cerebellar damage requires a balance between exploration and motor noise , 2015, Brain : a journal of neurology.