Exploration of joint redundancy but not task space variability facilitates supervised motor learning

Significance Theories of reinforcement learning claim that motor variability helps in motor learning and are supported by recent experimental work. In contrast, theories of motor control propose that variability is noise that needs to be suppressed. We attempt to reconcile these apparent contradictory positions. Using the formulation of the unconstrained manifold hypothesis, we show that motor variability has two components—a part arising out of the redundancy that does not affect task-space and another component related to task-space variability. We show that the motor variability component resulting from the redundancy determines both dynamic and kinematic learning ability across subjects without affecting task-space variability. The number of joints and muscles in a human arm is more than what is required for reaching to a desired point in 3D space. Although previous studies have emphasized how such redundancy and the associated flexibility may play an important role in path planning, control of noise, and optimization of motion, whether and how redundancy might promote motor learning has not been investigated. In this work, we quantify redundancy space and investigate its significance and effect on motor learning. We propose that a larger redundancy space leads to faster learning across subjects. We observed this pattern in subjects learning novel kinematics (visuomotor adaptation) and dynamics (force-field adaptation). Interestingly, we also observed differences in the redundancy space between the dominant hand and nondominant hand that explained differences in the learning of dynamics. Taken together, these results provide support for the hypothesis that redundancy aids in motor learning and that the redundant component of motor variability is not noise.

[1]  N. A. Bernshteĭn,et al.  Human motor actions : Bernstein reassessed , 1984 .

[2]  A. Longoni,et al.  Problems in the Assessment of Hand Preference , 1985, Cortex.

[3]  K. J. Cole,et al.  Coordination of three-joint digit movements for rapid finger-thumb grasp. , 1986, Journal of neurophysiology.

[4]  F A Mussa-Ivaldi,et al.  Adaptive representation of dynamics during learning of a motor task , 1994, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[5]  Andrew G. Barto,et al.  Reinforcement learning control , 1994, Current Opinion in Neurobiology.

[6]  D H Brainard,et al.  The Psychophysics Toolbox. , 1997, Spatial vision.

[7]  Daniel M. Wolpert,et al.  Making smooth moves , 2022 .

[8]  Gregor Schöner,et al.  The uncontrolled manifold concept: identifying control variables for a functional task , 1999, Experimental Brain Research.

[9]  John W. Krakauer,et al.  Independent learning of internal models for kinematic and dynamic control of reaching , 1999, Nature Neuroscience.

[10]  C Ghez,et al.  Learning of Visuomotor Transformations for Vectorial Planning of Reaching Trajectories , 2000, The Journal of Neuroscience.

[11]  R. Sainburg Evidence for a dynamic-dominance hypothesis of handedness , 2001, Experimental Brain Research.

[12]  M. Latash,et al.  Motor Control Strategies Revealed in the Structure of Motor Variability , 2002, Exercise and sport sciences reviews.

[13]  Michael I. Jordan,et al.  Optimal feedback control as a theory of motor coordination , 2002, Nature Neuroscience.

[14]  Michael I. Jordan,et al.  A Minimal Intervention Principle for Coordinated Movement , 2002, NIPS.

[15]  Gregor Schöner,et al.  Goal-equivalent joint coordination in pointing: affect of vision and arm dominance. , 2002, Motor control.

[16]  Reza Shadmehr,et al.  Learned dynamics of reaching movements generalize from dominant to nondominant arm. , 2003, Journal of neurophysiology.

[17]  E. Todorov Optimality principles in sensorimotor control , 2004, Nature Neuroscience.

[18]  Patrick Haggard,et al.  Patterns of coordinated multi-joint movement , 2004, Experimental Brain Research.

[19]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[20]  Mark L. Latash,et al.  The role of kinematic redundancy in adaptation of reaching , 2006, Experimental Brain Research.

[21]  J. Krakauer,et al.  An Implicit Plan Overrides an Explicit Strategy during Visuomotor Adaptation , 2006, The Journal of Neuroscience.

[22]  R. Shadmehr,et al.  Interacting Adaptive Processes with Different Timescales Underlie Short-Term Motor Learning , 2006, PLoS biology.

[23]  M. Brainard,et al.  Performance variability enables adaptive plasticity of ‘crystallized’ adult birdsong , 2007, Nature.

[24]  Emanuel Todorov,et al.  Evidence for the Flexible Sensorimotor Strategies Predicted by Optimal Feedback Control , 2007, The Journal of Neuroscience.

[25]  Gregor Schöner,et al.  Toward a new theory of motor synergies. , 2007, Motor control.

[26]  Beth A. Smith,et al.  Uncontrolled manifold analysis of segmental angle variability during walking: preadolescents with and without Down syndrome , 2007, Experimental Brain Research.

[27]  R. Ivry,et al.  The coordination of movement: optimal feedback control and beyond , 2010, Trends in Cognitive Sciences.

[28]  J. Krakauer,et al.  Error correction, sensory prediction, and adaptation in motor control. , 2010, Annual review of neuroscience.

[29]  M. Latash,et al.  Changes in multifinger interaction and coordination in Parkinson's disease. , 2012, Journal of neurophysiology.

[30]  Yohsuke R. Miyamoto,et al.  Temporal structure of motor variability is dynamically regulated and predicts motor learning ability , 2014, Nature Neuroscience.

[31]  Reza Shadmehr,et al.  Motor variability is not noise, but grist for the learning mill , 2014, Nature Neuroscience.

[32]  Joshua G. A. Cashaback,et al.  The human motor system alters its reaching movement plan for task-irrelevant, positional forces. , 2015, Journal of neurophysiology.

[33]  Kang He,et al.  The Statistical Determinants of the Speed of Motor Learning , 2016, PLoS Comput. Biol..

[34]  D. Wolpert,et al.  Effective reinforcement learning following cerebellar damage requires a balance between exploration and motor noise , 2015, Brain : a journal of neurology.