Modeling of Endpoint Feedback Learning Implemented Through Point-to-Point Learning Control

In the last decade, several experiments were conducted to investigate human motor control behavior for the task of arm reaching, using only visual feedback of the final hand position at the end of each reaching motion. Current computational frameworks have yet to model that the humans learn to complete such a task by feedforward action based on the feedback of a displacement error at the end of past reaching motions. This paper demonstrates how such learning can be formulated as an optimization problem. By designing a cost function which weighs the tracking of the target and the smoothness of human motion, the constructed framework, implemented in the form of point-to-point learning control, inherently embeds the feedforward control and enables learning over repeated trials using only the available feedback from past observations, here the endpoint errors of a reaching motion trajectory. The proposed framework is able to reproduce the human learning behavior observed in experiments.

[1]  R. Miall,et al.  The role of proprioception and attention in a visuomotor adaptation task , 2000, Experimental Brain Research.

[2]  M. A. Goodale,et al.  The role of visual feedback of hand position in the control of manual prehension , 1999, Experimental Brain Research.

[3]  Ufuk Topcu,et al.  Optimization-Based Constrained Iterative Learning Control , 2011, IEEE Transactions on Control Systems Technology.

[4]  Rieko Osu,et al.  CNS Learns Stable, Accurate, and Efficient Movements Using a Simple Algorithm , 2008, The Journal of Neuroscience.

[5]  Konrad Paul Kording,et al.  Estimating the sources of motor errors for adaptation and generalization , 2008, Nature Neuroscience.

[6]  Oussama Khatib,et al.  A unified approach for motion and force control of robot manipulators: The operational space formulation , 1987, IEEE J. Robotics Autom..

[7]  Stephan Riek,et al.  Real-time error detection but not error correction drives automatic visuomotor adaptation , 2010, Experimental Brain Research.

[8]  Ying Tan,et al.  Effect of sensory experience on motor learning strategy. , 2015, Journal of neurophysiology.

[9]  R. J. Beers,et al.  Motor Learning Is Optimally Tuned to the Properties of Motor Noise , 2009, Neuron.

[10]  Reza Shadmehr,et al.  Learning of action through adaptive combination of motor primitives , 2000, Nature.

[11]  Etienne Burdet,et al.  3DOM: A 3 Degree of Freedom Manipulandum to Investigate Redundant Motor Control , 2014, IEEE Transactions on Haptics.

[12]  J. Krakauer Motor learning: its relevance to stroke recovery and neurorehabilitation. , 2006, Current opinion in neurology.

[13]  E. Todorov,et al.  Inverse Optimality Design for Biological Movement Systems , 2011 .

[14]  Iven M. Y. Mareels,et al.  Stability and motor adaptation in human arm movements , 2005, Biological Cybernetics.

[15]  Ying Tan,et al.  Iterative Learning Control With Mixed Constraints for Point-to-Point Tracking , 2013, IEEE Transactions on Control Systems Technology.

[16]  M. Kawato,et al.  Visual Feedback Is Not Necessary for the Learning of Novel Dynamics , 2007, PloS one.

[17]  Philip N. Sabes,et al.  Flexible strategies for sensory integration during motor planning , 2005, Nature Neuroscience.

[18]  Keng Peng Tee,et al.  Concurrent adaptation of force and impedance in the redundant muscle system , 2010, Biological Cybernetics.

[19]  G. Loeb,et al.  Mathematical models of proprioceptors. I. Control and transduction in the muscle spindle. , 2006, Journal of neurophysiology.

[20]  Konrad Paul Kording,et al.  Review TRENDS in Cognitive Sciences Vol.10 No.7 July 2006 Special Issue: Probabilistic models of cognition Bayesian decision theory in sensorimotor control , 2022 .

[21]  Jean-Paul Gauthier,et al.  The Inactivation Principle: Mathematical Solutions Minimizing the Absolute Work and Biological Implications for the Planning of Arm Movements , 2008, PLoS Comput. Biol..

[22]  Ferdinando A Mussa-Ivaldi,et al.  Interaction of visual and proprioceptive feedback during adaptation of human reaching movements. , 2005, Journal of neurophysiology.

[23]  Anastasia Sylaidi,et al.  Principles of sensorimotor control and learning in complex motor tasks , 2015 .

[24]  J. Houk,et al.  Sampling of total muscle force by tendon organs. , 1982, Journal of neurophysiology.

[25]  J. Krakauer,et al.  Sensory prediction errors drive cerebellum-dependent adaptation of reaching. , 2007, Journal of neurophysiology.

[26]  Francesco Nori,et al.  Evidence for Composite Cost Functions in Arm Movement Planning: An Inverse Optimal Control Approach , 2011, PLoS Comput. Biol..

[27]  Etienne Burdet,et al.  Human Robotics: Neuromechanics and Motor Control , 2013 .

[28]  Ying Tan,et al.  Modeling Individual Human Motor Behavior Through Model Reference Iterative Learning Control , 2012, IEEE Transactions on Biomedical Engineering.

[29]  Reza Shadmehr,et al.  Motor Adaptation as a Process of Reoptimization , 2008, The Journal of Neuroscience.

[30]  Ying Tan,et al.  Point-to-point learning in human motor systems , 2013, 2013 American Control Conference.

[31]  Matthew Heath,et al.  The control of memory-guided reaching movements in peripersonal space. , 2004, Motor control.

[32]  Emanuel Todorov,et al.  Iterative Linear Quadratic Regulator Design for Nonlinear Biological Movement Systems , 2004, ICINCO.

[33]  T. Flash,et al.  The coordination of arm movements: an experimentally confirmed mathematical model , 1985, The Journal of neuroscience : the official journal of the Society for Neuroscience.