Learning control in robot-assisted rehabilitation of motor skills - a review

The key idea in iterative learning control is captured by the intuition of ‘practice makes perfect’. The underlying learning is based on a gradient descent algorithm iteratively optimising an appropriate input–output measured criterion. How this paradigm is used to model quantitatively, at an input/output level, the learning that happens in the context of human motor skill learning is discussed in this note. Experimental studies of human motor learning, in robotically controlled environments, indicate that a model consisting of a classical (iterative) learning control augmented with an appropriate kinematic model of human motor motion fits the observed human learning behaviour well. In the context of the rehabilitation of motor skills, such models promise better human–machine interfaces that extend the capability and capacity of rehabilitation clinicians by creating effective robot–patient–clinician feedback loops. The economic promise of robot-assisted rehabilitation is to greatly extend the intervention...

[1]  E Bizzi,et al.  Motor learning through the combination of primitives. , 2000, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[2]  Shou-Han Zhou Human learning in visuomotor field: modeling and experimental studies , 2015 .

[3]  Diego Torricelli,et al.  Quantitative assessment based on kinematic measures of functional impairments during upper extremity movements: A review. , 2014, Clinical biomechanics.

[4]  T. Platz,et al.  Electromechanical and robot-assisted arm training for improving generic activities of daily living, arm function, and arm muscle strength after stroke. , 2012, The Cochrane database of systematic reviews.

[5]  T. Platz,et al.  Electromechanical and robot-assisted arm training for improving activities of daily living, arm function, and arm muscle strength after stroke. , 2015, The Cochrane database of systematic reviews.

[6]  Ying Tan,et al.  Point-to-point learning in human motor systems , 2013, 2013 American Control Conference.

[7]  Jiping He,et al.  Determining natural arm configuration along a reaching trajectory , 2005, Experimental Brain Research.

[8]  Norbert Wiener,et al.  Cybernetics: Control and Communication in the Animal and the Machine. , 1949 .

[9]  Brian D. O. Anderson,et al.  Iterative identification and two step control design for partially unknown unstable plants , 2001 .

[10]  Emanuel Todorov,et al.  Iterative Linear Quadratic Regulator Design for Nonlinear Biological Movement Systems , 2004, ICINCO.

[11]  S. Leonhardt,et al.  A survey on robotic devices for upper limb rehabilitation , 2014, Journal of NeuroEngineering and Rehabilitation.

[12]  S. Wolf,et al.  Forced use of hemiplegic upper extremities to reverse the effect of learned nonuse among chronic stroke and head-injured patients , 1989, Experimental Neurology.

[13]  B. Wünsche,et al.  Assessment of movement quality in robot- assisted upper limb rehabilitation after stroke: a review , 2014, Journal of NeuroEngineering and Rehabilitation.

[14]  M. Kawato,et al.  Visual Feedback Is Not Necessary for the Learning of Novel Dynamics , 2007, PloS one.

[15]  Kenji Doya,et al.  Reinforcement Learning in Continuous Time and Space , 2000, Neural Computation.

[16]  Bruno Dehez,et al.  A robotic device as a sensitive quantitative tool to assess upper limb impairments in stroke patients: a preliminary prospective cohort study. , 2012, Journal of rehabilitation medicine.

[17]  Grant D. Huang,et al.  Robot-assisted therapy for long-term upper-limb impairment after stroke. , 2010, The New England journal of medicine.

[18]  E. Strickland,et al.  Good-bye, wheelchair , 2012, IEEE Spectrum.

[19]  M. Kawato,et al.  Functional significance of stiffness in adaptation of multijoint arm movements to stable and unstable dynamics , 2003, Experimental Brain Research.

[20]  P. Fitts The information capacity of the human motor system in controlling the amplitude of movement. , 1954, Journal of experimental psychology.

[21]  Jianxin Xu,et al.  Linear and Nonlinear Iterative Learning Control , 2003 .

[22]  D.J. Reinkensmeyer,et al.  Robotic movement training as an optimization problem: designing a controller that assists only as needed , 2005, 9th International Conference on Rehabilitation Robotics, 2005. ICORR 2005..

[23]  Gideon F. Inbar,et al.  Human motor control: learning to control a time-varying, nonlinear, many-to-one system , 2000, IEEE Trans. Syst. Man Cybern. Part C.

[24]  F. H. Adler Cybernetics, or Control and Communication in the Animal and the Machine. , 1949 .

[25]  J. Krakauer,et al.  Adaptation to Visuomotor Transformations: Consolidation, Interference, and Forgetting , 2005, The Journal of Neuroscience.

[26]  A.G. Alleyne,et al.  A survey of iterative learning control , 2006, IEEE Control Systems.

[27]  J. P. Friconneau,et al.  ABLE, an innovative transparent exoskeleton for the upper-limb , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[28]  Suguru Arimoto,et al.  Bettering operation of Robots by learning , 1984, J. Field Robotics.

[29]  Suguru Arimoto,et al.  A natural redundancy-resolution for 3-D multi-joint reaching under the gravity effect , 2005, J. Field Robotics.

[30]  C Ghez,et al.  Learning of Visuomotor Transformations for Vectorial Planning of Reaching Trajectories , 2000, The Journal of Neuroscience.

[31]  Rieko Osu,et al.  The central nervous system stabilizes unstable dynamics by learning optimal impedance , 2001, Nature.

[32]  Rieko Osu,et al.  Different mechanisms involved in adaptation to stable and unstable dynamics. , 2003, Journal of neurophysiology.

[33]  D. H. Owens Stability of linear multipass processes , 1977 .

[34]  H. F. Machiel van der Loos,et al.  Development of robots for rehabilitation therapy: the Palo Alto VA/Stanford experience. , 2000, Journal of rehabilitation research and development.

[35]  Bruno Siciliano,et al.  euRobotics - Shaping the future of European robotics , 2010, ISR/ROBOTIK.

[36]  M. Levin,et al.  Effect of Trunk Restraint on the Recovery of Reaching Movements in Hemiparetic Patients , 2001, Stroke.

[37]  Michael J. Grimble,et al.  Iterative Learning Control for Deterministic Systems , 1992 .

[38]  Zhiwei Luo,et al.  On Control Mechanism of Human-Like Reaching Movements with Musculo-Skeletal Redundancy , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[39]  Robert J. van Beers,et al.  How Does Our Motor System Determine Its Learning Rate? , 2012, PloS one.

[40]  Konrad Paul Kording,et al.  Bayesian integration in sensorimotor learning , 2004, Nature.

[41]  N. Filatov,et al.  Survey of adaptive dual control methods , 2000 .

[42]  Ying Tan,et al.  An investigation into the reliability of upper-limb robotic exoskeleton measurements for clinical evaluation in neurorehabilitation , 2015, 2015 7th International IEEE/EMBS Conference on Neural Engineering (NER).

[43]  Farshid Amirabdollahian,et al.  Training modalities in robot-mediated upper limb rehabilitation in stroke: a framework for classification based on a systematic review , 2014, Journal of NeuroEngineering and Rehabilitation.

[44]  Robert Riener,et al.  A novel paradigm for patient-cooperative control of upper-limb rehabilitation robots , 2007, Adv. Robotics.

[45]  G. Thielman Rehabilitation of Reaching Poststroke: A Randomized Pilot Investigation of Tactile Versus Auditory Feedback for Trunk Control , 2010, Journal of neurologic physical therapy : JNPT.

[46]  T. Flash,et al.  The coordination of arm movements: an experimentally confirmed mathematical model , 1985, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[47]  Ying Tan,et al.  Effect of sensory experience on motor learning strategy. , 2015, Journal of neurophysiology.

[48]  Petre Stoica,et al.  Decentralized Control , 2018, The Control Systems Handbook.

[49]  Masaru Uchiyama,et al.  Formation of High-Speed Motion Pattern of a Mechanical Arm by Trial , 1978 .

[50]  Hermano Igo Krebs,et al.  MIT-MANUS: a workstation for manual therapy and training. I , 1992, [1992] Proceedings IEEE International Workshop on Robot and Human Communication.

[51]  M. Kawato,et al.  Adaptation to Stable and Unstable Dynamics Achieved By Combined Impedance Control and Inverse Dynamics Model , 2003 .

[52]  S. Arimoto,et al.  Robustness of P-type learning control with a forgetting factor for robotic motions , 1990, 29th IEEE Conference on Decision and Control.

[53]  Zhong-Ping Jiang,et al.  Adaptive dynamic programming as a theory of sensorimotor control , 2012, 2012 IEEE Signal Processing in Medicine and Biology Symposium (SPMB).

[54]  M. Ernst,et al.  Humans integrate visual and haptic information in a statistically optimal fashion , 2002, Nature.

[55]  S. Arimoto Mathematical Theory of Learning with Applications to Robot Control , 1986 .

[56]  J. Doyle,et al.  Essentials of Robust Control , 1997 .

[57]  Ying Tan,et al.  Modeling Individual Human Motor Behavior Through Model Reference Iterative Learning Control , 2012, IEEE Transactions on Biomedical Engineering.

[58]  S. Schaal,et al.  Computational motor control in humans and robots , 2005, Current Opinion in Neurobiology.

[59]  Zhong-Ping Jiang,et al.  A robust adaptive dynamic programming principle for sensorimotor control with signal-dependent noise , 2015, J. Syst. Sci. Complex..

[60]  Y. Loewenstein,et al.  Reinforcement learning and human behavior , 2014, Current Opinion in Neurobiology.

[61]  Jiping He,et al.  Design and Control of RUPERT: A Device for Robotic Upper Extremity Repetitive Therapy , 2007, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[62]  Stefan Schaal,et al.  2008 Special Issue: Reinforcement learning of motor skills with policy gradients , 2008 .

[63]  Konrad Paul Kording,et al.  Review TRENDS in Cognitive Sciences Vol.10 No.7 July 2006 Special Issue: Probabilistic models of cognition Bayesian decision theory in sensorimotor control , 2022 .

[64]  Kevin L. Moore,et al.  Iterative Learning Control: Brief Survey and Categorization , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[65]  Ruggero Frezza,et al.  Nonlinear control by a finite set of motion primitives , 2004 .

[66]  Suguru Arimoto Robustness of learning control for robot manipulators , 1990, Proceedings., IEEE International Conference on Robotics and Automation.

[67]  F A Mussa-Ivaldi,et al.  Adaptive representation of dynamics during learning of a motor task , 1994, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[68]  E. Todorov Optimality principles in sensorimotor control , 2004, Nature Neuroscience.

[69]  Michael S Landy,et al.  Motor control is decision-making , 2012, Current Opinion in Neurobiology.

[70]  Eric Rogers,et al.  FES based rehabilitation of the upper limb using input/output linearization and ILC , 2012, 2012 American Control Conference (ACC).

[71]  A. Haith,et al.  Model-based and model-free mechanisms of human motor learning. , 2013, Advances in experimental medicine and biology.

[72]  V. Sanguineti,et al.  Learning, Retention, and Slacking: A Model of the Dynamics of Recovery in Robot Therapy , 2012, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[73]  S. W. On Some Key Issues in the Windsurfer Approach to Adaptive Robust Control , .

[74]  Britne A. Shabbott,et al.  Learning a visuomotor rotation: simultaneous visual and proprioceptive information is crucial for visuomotor remapping , 2010, Experimental Brain Research.

[75]  C. Chisari,et al.  Training and assessment of upper limb motor function with a robotic exoskeleton after stroke , 2012, 2012 4th IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob).

[76]  M. Kawato,et al.  Formation and control of optimal trajectory in human multijoint arm movement , 1989, Biological Cybernetics.

[77]  V. Dietz,et al.  Three-dimensional, task-specific robot therapy of the arm after stroke: a multicentre, parallel-group randomised trial , 2014, The Lancet Neurology.

[78]  Peter W Battaglia,et al.  Humans Trade Off Viewing Time and Movement Duration to Improve Visuomotor Accuracy in a Fast Reaching Task , 2007, The Journal of Neuroscience.

[79]  Ying Tan,et al.  A Tool to Address Movement Quality Outcomes of Post-stroke Patients , 2014 .

[80]  E. Todorov,et al.  A generalized iterative LQG method for locally-optimal feedback control of constrained nonlinear stochastic systems , 2005, Proceedings of the 2005, American Control Conference, 2005..

[81]  S. Saab Stochastic P-type/D-type iterative learning control algorithms , 2003 .

[82]  Y Uno,et al.  Quantitative examinations of internal representations for arm trajectory planning: minimum commanded torque change model. , 1999, Journal of neurophysiology.

[83]  Michael I. Jordan,et al.  Optimal feedback control as a theory of motor coordination , 2002, Nature Neuroscience.

[84]  Zhong-Ping Jiang,et al.  Movement Duration, Fitts's Law, and an Infinite-Horizon Optimal Feedback Control Model for Biological Motor Systems , 2013, Neural Computation.

[85]  Biao Huang,et al.  System Identification , 2000, Control Theory for Physicists.

[86]  C. Burgar,et al.  Robot-assisted movement training compared with conventional therapy techniques for the rehabilitation of upper-limb motor function after stroke. , 2002, Archives of physical medicine and rehabilitation.

[87]  Daniel M Wolpert,et al.  Adaptation to a visuomotor shift depends on the starting posture. , 2002, Journal of neurophysiology.

[88]  M. Spong,et al.  Robot Modeling and Control , 2005 .

[89]  Emmanuel Guigon,et al.  Computational Motor Control : Redundancy and Invariance , 2007 .

[90]  Norbert Wiener,et al.  Cybernetics, or control and communication in the animal and the machine, 2nd ed. , 1961 .

[91]  A. Fugl-Meyer,et al.  The post-stroke hemiplegic patient. 1. a method for evaluation of physical performance. , 1975, Scandinavian journal of rehabilitation medicine.

[92]  Dejan B. Popovic,et al.  A method for assessing the arm movement performance: probability tube , 2013, Medical & Biological Engineering & Computing.

[93]  Iven M. Y. Mareels,et al.  Stability and motor adaptation in human arm movements , 2005, Biological Cybernetics.

[94]  R. J. Beers,et al.  Motor Learning Is Optimally Tuned to the Properties of Motor Noise , 2009, Neuron.

[95]  E Burdet,et al.  A method for measuring endpoint stiffness during multi-joint arm movements. , 2000, Journal of biomechanics.

[96]  Konrad Paul Kording,et al.  Estimating the sources of motor errors for adaptation and generalization , 2008, Nature Neuroscience.

[97]  Robert Riener,et al.  ARMin: a robot for patient-cooperative arm therapy , 2007, Medical & Biological Engineering & Computing.

[98]  Vincent S. Huang,et al.  Robotic neurorehabilitation: a computational motor learning perspective , 2009, Journal of NeuroEngineering and Rehabilitation.

[99]  J. Krakauer,et al.  A computational neuroanatomy for motor control , 2008, Experimental Brain Research.