Motor adaptation as a greedy optimization of error and effort.

Motor adaptation to a novel dynamic environment is primarily thought of as a process in which the nervous system learns to anticipate the environmental forces to eliminate kinematic error. Here we show that motor adaptation can more generally be modeled as a process in which the motor system greedily minimizes a cost function that is the weighted sum of kinematic error and effort. The learning dynamics predicted by this minimization process are a linear, auto-regressive equation with only one state, which has been identified previously as providing a good fit to data from force-field-type experiments. Thus we provide a new theoretical result that shows how these previously identified learning dynamics can be viewed as arising from an optimization of error and effort. We also show that the coefficients of the learning dynamics must fall within a specific range for the optimization model to be valid and verify with experimental data from walking in a force field that they indeed fall in this range. Finally, we attempted to falsify the model by performing experiments in two conditions (repeated exposure to a force field, exposure to force fields of different strengths) for which the single-state, auto-regressive equation might be expected to not fit the data well. We found however that the equation adequately captured the pattern of errors and thus conclude that motor adaptation to a force field can be approximated as an optimization of effort and error for a range of experimental conditions.

[1]  J. Lackner,et al.  Motor adaptation to Coriolis force perturbations of reaching movements: endpoint but not trajectory adaptation transfers to the nonexposed arm. , 1995, Journal of neurophysiology.

[2]  D. Wolpert,et al.  Temporal and amplitude generalization in motor learning. , 1998, Journal of neurophysiology.

[3]  Emery N. Brown,et al.  Faculty Opinions recommendation of Interacting adaptive processes with different timescales underlie short-term motor learning. , 2006 .

[4]  R A Scheidt,et al.  Learning to move amid uncertainty. , 2001, Journal of neurophysiology.

[5]  D M Wolpert,et al.  Multiple paired forward and inverse models for motor control , 1998, Neural Networks.

[6]  Otmar Bock,et al.  Early stages of load compensation in human aimed arm movements , 1993, Behavioural Brain Research.

[7]  R. Shadmehr,et al.  Interacting Adaptive Processes with Different Timescales Underlie Short-Term Motor Learning , 2006, PLoS biology.

[8]  J. Lackner,et al.  Gravitoinertial force background level affects adaptation to coriolis force perturbations of reaching movements. , 1998, Journal of neurophysiology.

[9]  Daniel M. Wolpert,et al.  Making smooth moves , 2022 .

[10]  R A Scheidt,et al.  Impedance control and internal model formation when reaching in a randomly varying dynamical environment. , 2001, Journal of neurophysiology.

[11]  R L Sainburg,et al.  Intersegmental dynamics are controlled by sequential anticipatory, error correction, and postural mechanisms. , 1999, Journal of neurophysiology.

[12]  T. Milner,et al.  Compensation for mechanically unstable loading in voluntary wrist movement , 2004, Experimental Brain Research.

[13]  T. Brashers-Krug,et al.  Functional Stages in the Formation of Human Long-Term Motor Memory , 1997, The Journal of Neuroscience.

[14]  M. Kawato,et al.  A hierarchical neural-network model for control and learning of voluntary movement , 2004, Biological Cybernetics.

[15]  O. I. Fukson,et al.  [Biomechanical characteristics of the wiping reflex cycle]. , 1984, Biofizika.

[16]  R Shadmehr,et al.  Spatial Generalization from Learning Dynamics of Reaching Movements , 2000, The Journal of Neuroscience.

[17]  F A Mussa-Ivaldi,et al.  Central representation of time during motor learning. , 1999, Proceedings of the National Academy of Sciences of the United States of America.

[18]  D. Wolpert,et al.  Failure to Consolidate the Consolidation Theory of Learning for Sensorimotor Adaptation Tasks , 2004, The Journal of Neuroscience.

[19]  E. Bizzi,et al.  Consolidation in human motor memory , 1996, Nature.

[20]  E Bizzi,et al.  Motor learning by field approximation. , 1996, Proceedings of the National Academy of Sciences of the United States of America.

[21]  D.J. Reinkensmeyer,et al.  Robot-enhanced motor learning: accelerating internal model formation during locomotion by transient dynamic amplification , 2005, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[22]  Neville Hogan,et al.  Impedance Control: An Approach to Manipulation: Part I—Theory , 1985 .

[23]  M. Kawato,et al.  The cerebellum and VOR/OKR learning models , 1992, Trends in Neurosciences.

[24]  Daniel M. Wolpert,et al.  Forward Models for Physiological Motor Control , 1996, Neural Networks.

[25]  A. Bahill,et al.  Determining ideal baseball bat weights using muscle force-velocity relationships , 1989, Biological Cybernetics.

[26]  H. Gomi,et al.  Task-Dependent Viscoelasticity of Human Multijoint Arm and Its Spatial Characteristics for Interaction with Environments , 1998, The Journal of Neuroscience.

[27]  D. Reinkensmeyer,et al.  Human-robot cooperative movement training: Learning a novel sensory motor transformation during walking with robotic assistance-as-needed , 2007, Journal of NeuroEngineering and Rehabilitation.

[28]  F A Mussa-Ivaldi,et al.  Adaptive representation of dynamics during learning of a motor task , 1994, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[29]  John W. Krakauer,et al.  Independent learning of internal models for kinematic and dynamic control of reaching , 1999, Nature Neuroscience.

[30]  E. Todorov Optimality principles in sensorimotor control , 2004, Nature Neuroscience.

[31]  M. Pandy,et al.  Dynamic optimization of human walking. , 2001, Journal of biomechanical engineering.

[32]  E. J. Lai,et al.  Influence of interaction force levels on degree of motor adaptation in a stable dynamic force field , 2003, Experimental Brain Research.

[33]  David E. Meyer,et al.  Optimization and Learning in Neural Networks for Formation and Control of Coordinated Movement , 1993 .

[34]  C Ghez,et al.  Proprioceptive control of interjoint coordination. , 1995, Canadian journal of physiology and pharmacology.

[35]  Michael I. Jordan,et al.  Are arm trajectories planned in kinematic or dynamic coordinates? An adaptation study , 1995, Experimental Brain Research.

[36]  N. A. Bernshteĭn The co-ordination and regulation of movements , 1967 .

[37]  J. Lackner,et al.  Rapid adaptation to Coriolis force perturbations of arm trajectory. , 1994, Journal of neurophysiology.

[38]  J. Liu,et al.  Motor adaptation as an optimal combination of computational strategies , 2004, The 26th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[39]  E. Fetz,et al.  Functional classes of primate corticomotoneuronal cells and their relation to active force. , 1980, Journal of neurophysiology.

[40]  P. Crago,et al.  Effects of voluntary force generation on the elastic components of endpoint stiffness , 2001, Experimental Brain Research.

[41]  宇野 洋二,et al.  Formation and control of optimal trajectory in human multijoint arm movement : minimum torque-change model , 1988 .

[42]  S. Haykin,et al.  Adaptive Filter Theory , 1986 .

[43]  G. Amico,et al.  Ecology: Mistletoe seed dispersal by a marsupial , 2000, Nature.

[44]  M. Kawato,et al.  Formation and control of optimal trajectory in human multijoint arm movement , 1989, Biological Cybernetics.

[45]  E. Bizzi,et al.  Kinematic strategies and sensorimotor transformations in the wiping movements of frogs. , 1989, Journal of neurophysiology.

[46]  Reza Shadmehr,et al.  Learning of action through adaptive combination of motor primitives , 2000, Nature.

[47]  M. Kawato,et al.  Acquisition and contextual switching of multiple internal models for different viscous force fields , 2003, Neuroscience Research.

[48]  R A Scheidt,et al.  Control strategies for the transition from multijoint to single-joint arm movements studied using a simple mechanical constraint. , 2000, Journal of neurophysiology.

[49]  Michael I. Jordan Computational aspects of motor control and motor learning , 2008 .

[50]  Elizabeth A. Peck,et al.  Introduction to Linear Regression Analysis , 2001 .

[51]  R. Kram,et al.  Biomechanics: Penguin waddling is not wasteful , 2000, Nature.

[52]  D. Ostry,et al.  Three-dimensional kinematic analysis of frog hindlimb movement in reflex wiping , 2004, Experimental Brain Research.

[53]  M. Kawato Optimization and learning in neural networks for formation and control of coordinated movement , 1993 .

[54]  David J Reinkensmeyer,et al.  Effect of muscle fatigue on internal model formation and retention during reaching with the arm. , 2006, Journal of applied physiology.

[55]  Rieko Osu,et al.  The central nervous system stabilizes unstable dynamics by learning optimal impedance , 2001, Nature.

[56]  David J. Reinkensmeyer,et al.  A robotic device for manipulating human stepping , 2006, IEEE Transactions on Robotics.

[57]  Michael I. Jordan,et al.  Optimal feedback control as a theory of motor coordination , 2002, Nature Neuroscience.

[58]  Reza Shadmehr,et al.  Quantifying Generalization from Trial-by-Trial Behavior of Adaptive Systems that Learn with Basis Functions: Theory and Experiments in Human Motor Control , 2003, The Journal of Neuroscience.

[59]  Daniel M Wolpert,et al.  Computational principles of sensorimotor control that minimize uncertainty and variability , 2007, The Journal of physiology.

[60]  F. Mussa-Ivaldi,et al.  The motor system does not learn the dynamics of the arm by rote memorization of past experience. , 1997, Journal of neurophysiology.

[61]  M. A. Arbib,et al.  A Model of the Effects of Speed, Accuracy, and Perturbation on Visually Guided Reaching , 1992 .

[62]  N. Hogan An organizing principle for a class of voluntary movements , 1984, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[63]  M I Lipshits,et al.  Measurements of human force control during a constrained arm motion using a force-actuated joystick. , 1995, Journal of neurophysiology.

[64]  Y Uno,et al.  Quantitative examinations of internal representations for arm trajectory planning: minimum commanded torque change model. , 1999, Journal of neurophysiology.

[65]  C Ghez,et al.  Roles of proprioceptive input in the programming of arm trajectories. , 1990, Cold Spring Harbor symposia on quantitative biology.

[66]  R. Happee Goal-directed arm movements. III: Feedback and adaptation in response to inertia perturbations. , 1993, Journal of electromyography and kinesiology : official journal of the International Society of Electrophysiological Kinesiology.

[67]  R A Scheidt,et al.  Persistence of motor adaptation during constrained, multi-joint, arm movements. , 2000, Journal of neurophysiology.