DMP-Based Motion Generation for a Walking Exoskeleton Robot Using Reinforcement Learning

For the purpose of the assistance for human walking, this paper describes a novel coupled movement sequences planning and motion adaption based on dynamic movement primitives (DMPs) for a walking exoskeleton robot. The developed exoskeleton robot has eight degrees of freedom (DOFs). The hip and knee of each artificial leg can provide two electric-powered DOFs to flexion or extension, two passive-installed DOFs of the ankle are to achieve the motion of inversion/eversion and plantarflexion/dorsiflexion, and two passive DOFs of the hip are to achieve the motion of roll or yaw. A novel trajectory-learning scheme based on reinforcement learning (RL) combined with DMPs is presented for a lower limb exoskeleton robot, aiming to give assistance to human walking. In the proposed strategy, a two-level planning is designed. In the first level, the inverted pendulum approximation under the consideration of the locomotion parameters is utilized to guarantee the zero-moment point within the ankle joint of the support leg in the phase of single support. In the second level, the joint trajectories are modeled and learned by DMPs. Meanwhile, the RL is adopted to learn the trajectories for eliminating the effects of uncertainties in joint space. The experiment involving four subjects based on a lower limb exoskeleton robot demonstrates that the proposed scheme can effectively suppress the disturbances and uncertainties.

[1]  Hongchul Kim,et al.  Design and locomotion control of a hydraulic lower extremity exoskeleton for mobility augmentation , 2017 .

[2]  Robert Babuska,et al.  Unified Modeling and Control of Walking and Running on the Spring-Loaded Inverted Pendulum , 2016, IEEE Transactions on Robotics.

[3]  Shahid Hussain,et al.  Robust Nonlinear Control of an Intrinsically Compliant Robotic Gait Training Orthosis , 2013, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[4]  Jun Nakanishi,et al.  Learning Movement Primitives , 2005, ISRR.

[5]  Jessy W. Grizzle,et al.  The Spring Loaded Inverted Pendulum as the Hybrid Zero Dynamics of an Asymmetric Hopper , 2009, IEEE Transactions on Automatic Control.

[6]  Umashankar Nagarajan,et al.  Integral admittance shaping: A unified framework for active exoskeleton control , 2016, Robotics Auton. Syst..

[7]  Jean-Jacques E. Slotine,et al.  On Contraction Analysis for Non-linear Systems , 1998, Autom..

[8]  Zhijun Li,et al.  The development of a high-speed lower-limb robotic exoskeleton , 2018, Science China Information Sciences.

[9]  Dong Liu,et al.  A brain-controlled exoskeleton with cascaded event-related desynchronization classifiers , 2017, Robotics Auton. Syst..

[10]  Ayush Agrawal,et al.  First Steps Towards Translating HZD Control of Bipedal Robots to Decentralized Control of Exoskeletons , 2017, IEEE Access.

[11]  Keijiro Yamamoto,et al.  Development of Power Assisting Suit for Assisting Nurse Labor , 2002 .

[12]  C. L. Philip Chen,et al.  A survey of human-centered intelligent robots: issues and challenges , 2017, IEEE/CAA Journal of Automatica Sinica.

[13]  Junji Hirai,et al.  Development of a Biofeedback Therapeutic-Exercise-Supporting Manipulator , 2009, IEEE Transactions on Industrial Electronics.

[14]  Y. Sankai,et al.  Power assist method based on phase sequence driven by interaction between human and robot suit , 2004, RO-MAN 2004. 13th IEEE International Workshop on Robot and Human Interactive Communication (IEEE Catalog No.04TH8759).

[15]  Jean-Jacques E. Slotine,et al.  Motion Primitives for Robotic Flight Control , 2006, ArXiv.

[16]  Winfried Stefan Lohmiller,et al.  Contraction analysis of nonlinear systems , 1999 .

[17]  R. Riener,et al.  Journal of Neuroengineering and Rehabilitation Open Access Biofeedback for Robotic Gait Rehabilitation , 2022 .

[18]  Jian Huang,et al.  Design and Evaluation of the RUPERT Wearable Upper Extremity Exoskeleton Robot for Clinical and In-Home Therapies , 2016, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[19]  Yoshiyuki Sankai,et al.  Power Assist System HAL-3 for Gait Disorder Person , 2002, ICCHP.

[20]  Yoshiyuki Sankai,et al.  Power assist method for HAL-3 estimating operator's intention based on motion information , 2003, The 12th IEEE International Workshop on Robot and Human Interactive Communication, 2003. Proceedings. ROMAN 2003..

[21]  Barbara Savini,et al.  Path Planning for Autonomous Vehicles by Trajectory Smoothing Using Motion Primitives , 2008, IEEE Transactions on Control Systems Technology.

[22]  Chenguang Yang,et al.  Physical Human–Robot Interaction of a Robotic Exoskeleton By Admittance Control , 2018, IEEE Transactions on Industrial Electronics.

[23]  Stefan Schaal,et al.  Reinforcement Learning With Sequences of Motion Primitives for Robust Manipulation , 2012, IEEE Transactions on Robotics.

[24]  Stefan Schaal,et al.  A Generalized Path Integral Control Approach to Reinforcement Learning , 2010, J. Mach. Learn. Res..

[25]  Qiang Huang,et al.  Gait Planning of Omnidirectional Walk on Inclined Ground for Biped Robots , 2016, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[26]  Kemalettin Erbatur,et al.  Natural ZMP Trajectories for Biped Robot Reference Generation , 2009, IEEE Transactions on Industrial Electronics.

[27]  Shuuji Kajita,et al.  Study of dynamic biped locomotion on rugged terrain-derivation and application of the linear inverted pendulum mode , 1991, Proceedings. 1991 IEEE International Conference on Robotics and Automation.

[28]  Taku Komura,et al.  Simulating pathological gait using the enhanced linear inverted pendulum model , 2005, IEEE Transactions on Biomedical Engineering.

[29]  Richard P. Paul,et al.  An On-Line Dynamic Trajectory Generator , 1984 .

[30]  Maarouf Saad,et al.  Cartesian Trajectory Tracking of a 7-DOF Exoskeleton Robot Based on Human Inverse Kinematics , 2019, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[31]  Antonio Bicchi,et al.  Asymmetric Bimanual Control of Dual-Arm Exoskeletons for Human-Cooperative Manipulations , 2018, IEEE Transactions on Robotics.

[32]  M. Goldfarb,et al.  Preliminary Evaluation of a Powered Lower Limb Orthosis to Aid Walking in Paraplegic Individuals , 2011, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[33]  Toshio Fukuda,et al.  Reinforcement Learning of Manipulation and Grasping Using Dynamical Movement Primitives for a Humanoidlike Mobile Manipulator , 2017, IEEE/ASME Transactions on Mechatronics.