Effective reinforcement learning following cerebellar damage requires a balance between exploration and motor noise

See Miall and Galea (doi: 10.1093/awv343 ) for a scientific commentary on this article. Cerebellar lesions impair both coordination and motor learning. Therrien et al. show that affected individuals can learn using a reinforcement mechanism despite a deficit in error-based motor learning. They also identify a critical feature of cerebellar patients’ movements (motor noise), which determines the effectiveness of learning under reinforcement.

[1]  Hermann von Helmholtz,et al.  Treatise on Physiological Optics , 1962 .

[2]  M. Hallett,et al.  Adaptation to lateral displacement of vision in patients with lesions of the central nervous system , 1983, Neurology.

[3]  W. Campbell,et al.  DeJong's The neurologic examination. , 1992 .

[4]  F A Mussa-Ivaldi,et al.  Adaptive representation of dynamics during learning of a motor task , 1994, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[5]  W. T. Thach,et al.  Cerebellar ataxia: abnormal control of interaction torques across multiple joints. , 1996, Journal of neurophysiology.

[6]  W. T. Thach,et al.  Throwing while looking through prisms. I. Focal olivocerebellar lesions impair adaptation. , 1996, Brain : a journal of neurology.

[7]  Jennifer A. Mangels,et al.  A Neostriatal Habit Learning System in Humans , 1996, Science.

[8]  M. Hallett,et al.  International Cooperative Ataxia Rating Scale for pharmacological assessment of the cerebellar syndrome , 1997, Journal of the Neurological Sciences.

[9]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[10]  C Ghez,et al.  Learning of Visuomotor Transformations for Vectorial Planning of Reaching Trajectories , 2000, The Journal of Neuroscience.

[11]  Eric R. Ziegel,et al.  Generalized Linear Models , 2002, Technometrics.

[12]  T. Ebner,et al.  Hereditary cerebellar ataxia progressively impairs force adaptation during goal-directed arm movements. , 2004, Journal of neurophysiology.

[13]  T. Kalenscher,et al.  Adaptive Motor Behavior of Cerebellar Patients During Exposure to Unfamiliar External Forces , 2004, Journal of motor behavior.

[14]  R. Shadmehr,et al.  Intact ability to learn internal models of arm dynamics in Huntington's disease but not cerebellar degeneration. , 2005, Journal of neurophysiology.

[15]  S. M. Morton,et al.  Cerebellar Contributions to Locomotor Adaptations during Splitbelt Treadmill Walking , 2006, The Journal of Neuroscience.

[16]  W. Schultz.  Behavioral theories and the neurophysiology of reward. , 2006, Annual review of psychology.

[17]  J. Krakauer,et al.  Sensory prediction errors drive cerebellum-dependent adaptation of reaching. , 2007, Journal of neurophysiology.

[18]  A. Doucet,et al.  A Tutorial on Particle Filtering and Smoothing: Fifteen years later , 2008 .

[19]  W. Schultz,et al.  Behavioral theories and the neurophysiology of reward. , 2006, Annual review of psychology.

[20]  P. Thier,et al.  The Cerebellum Updates Predictions about the Visual Consequences of One's Behavior , 2008, Current Biology.

[21]  Eduardo F. Morales,et al.  An Introduction to Reinforcement Learning , 2011 .

[22]  Reza Shadmehr,et al.  Learning from Sensory and Reward Prediction Errors during Motor Adaptation , 2011, PLoS Comput. Biol..

[23]  Raymond J. Delnicki,et al.  Overcoming Motor “Forgetting” Through Reinforcement Of Learned Actions , 2012, The Journal of Neuroscience.

[24]  H. Seo,et al.  Neural basis of reinforcement learning and decision making. , 2012, Annual review of neuroscience.

[25]  Sarah E. Criscimagna-Hemminger,et al.  Cerebellar Contributions to Reach Adaptation and Learning Sensory Consequences of Action , 2012, The Journal of Neuroscience.

[26]  John E. Schlerf,et al.  Individuals with cerebellar degeneration show similar adaptation deficits with large and small visuomotor errors. , 2013, Journal of neurophysiology.

[27]  A. Haith,et al.  Model-based and model-free mechanisms of human motor learning. , 2013, Advances in experimental medicine and biology.

[28]  Allison M Okamura,et al.  Cerebellar motor learning: are environment dynamics more important than error size? , 2013, Journal of neurophysiology.

[29]  Nasir H. Bhanpuri,et al.  Predictive Modeling by the Cerebellum Improves Proprioception , 2013, The Journal of Neuroscience.

[30]  Konrad Kording,et al.  Credit Assignment during Movement Reinforcement Learning , 2013, PloS one.

[31]  Yohsuke R. Miyamoto,et al.  Temporal structure of motor variability is dynamically regulated and predicts motor learning ability , 2014, Nature Neuroscience.

[32]  Eero P. Simoncelli,et al.  Partitioning neuronal variability , 2014, Nature Neuroscience.

[33]  Michael J Kahana,et al.  Microstimulation of the Human Substantia Nigra Alters Reinforcement Learning , 2014, The Journal of Neuroscience.

[34]  R. Ivry,et al.  Cerebellar and prefrontal cortex contributions to adaptation, strategies, and reinforcement learning. , 2014, Progress in brain research.

[35]  J. Rothwell,et al.  The dissociable effects of punishment and reward on motor learning , 2015, Nature Neuroscience.

[36]  Alaa A. Ahmed,et al.  Reward feedback accelerates motor learning. , 2015, Journal of neurophysiology.