Noninvasive Human-Prosthesis Interfaces for Locomotion Intent Recognition: A Review

The lower-limb robotic prostheses can provide assistance for amputees’ daily activities by restoring the biomechanical functions of missing limb(s). To set proper control strategies and develop the corresponding controller for robotic prosthesis, a prosthesis user’s intent must be acquired in time, which is still a major challenge and has attracted intensive attentions. This work focuses on the robotic prosthesis user’s locomotion intent recognition based on the noninvasive sensing methods from the recognition task perspective (locomotion mode recognition, gait event detection, and continuous gait phase estimation) and reviews the state-ofthe-art intent recognition techniques in a lower-limb prosthesis scope. The current research status, including recognition approach, progress, challenges, and future prospects in the human’s intent recognition, has been reviewed. In particular for the recognition approach, the paper analyzes the recent studies and discusses the role of each element in locomotion intent recognition. This work summarizes the existing research results and problems and contributes a general framework for the intent recognition based on lower-limb prosthesis.

[1]  Andrea Parri,et al.  Gait Phase Estimation Based on Noncontact Capacitive Sensing and Adaptive Oscillators , 2017, IEEE Transactions on Biomedical Engineering.

[2]  Enhao Zheng,et al.  Noncontact Capacitive Sensing-Based Locomotion Transition Recognition for Amputees With Robotic Transtibial Prostheses , 2017, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[3]  Michael Goldfarb,et al.  Volitional Control of a Prosthetic Knee Using Surface Electromyography , 2011, IEEE Transactions on Biomedical Engineering.

[4]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[5]  Elissa D. Ledoux Inertial Sensing for Gait Event Detection and Transfemoral Prosthesis Control Strategy , 2018, IEEE Transactions on Biomedical Engineering.

[6]  D. Winter,et al.  Biomechanics of below-knee amputee gait. , 1988, Journal of biomechanics.

[7]  Long Wang,et al.  A Noncontact Capacitive Sensing System for Recognizing Locomotion Modes of Transtibial Amputees , 2014, IEEE Transactions on Biomedical Engineering.

[8]  Hugh M. Herr,et al.  Powered Ankle--Foot Prosthesis Improves Walking Metabolic Economy , 2009, IEEE Transactions on Robotics.

[9]  Yifan David Li,et al.  Gait mode recognition and control for a portable-powered ankle-foot orthosis , 2013, 2013 IEEE 13th International Conference on Rehabilitation Robotics (ICORR).

[10]  Long Wang,et al.  Locomotion Mode Classification Using a Wearable Capacitive Sensing System , 2013, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[11]  Qining Wang,et al.  On-board Training Strategy for IMU-Based Real-Time Locomotion Recognition of Transtibial Amputees With Robotic Prostheses , 2020, Frontiers in Neurorobotics.

[12]  J. Czerniecki,et al.  Comparison of the Power Knee and C-Leg during step-up and sit-to-stand tasks. , 2013, Gait & Posture.

[13]  Baojun Chen,et al.  A Locomotion Intent Prediction System Based on Multi-Sensor Fusion , 2014, Sensors.

[14]  Luigi Carro,et al.  Analog reconfigurable technologies for EMG signal processing , 2007 .

[15]  Haibo He,et al.  Improving the performance of a neural-machine interface for prosthetic legs using adaptive pattern classifiers , 2013, 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[16]  Glenn K. Klute,et al.  Turn Intent Detection For Control of a Lower Limb Prosthesis , 2018, IEEE Transactions on Biomedical Engineering.

[17]  P. Cavanagh,et al.  Electromechanical delay in human skeletal muscle under concentric and eccentric contractions , 1979, European Journal of Applied Physiology and Occupational Physiology.

[18]  Aaron J. Young,et al.  Analysis of using EMG and mechanical sensors to enhance intent recognition in powered lower limb prostheses , 2014, Journal of neural engineering.

[19]  Radford M. Neal Pattern Recognition and Machine Learning , 2007, Technometrics.

[20]  Nicholas P. Fey,et al.  Classifying the intent of novel users during human locomotion using powered lower limb prostheses , 2013, 2013 6th International IEEE/EMBS Conference on Neural Engineering (NER).

[21]  S. Scott Optimal feedback control and the neural basis of volitional motor control , 2004, Nature Reviews Neuroscience.

[22]  K. Englehart,et al.  Classification of the myoelectric signal using time-frequency based representations. , 1999, Medical engineering & physics.

[23]  Robert D. Lipschutz,et al.  Robotic leg control with EMG decoding in an amputee with nerve transfers. , 2013, The New England journal of medicine.

[24]  Eugenijus Kaniusas,et al.  Relying on more sense for enhancing lower limb prostheses control: a review , 2020, Journal of NeuroEngineering and Rehabilitation.

[25]  Shiv Dutt Joshi,et al.  Force Myography Based Novel Strategy for Locomotion Classification , 2018, IEEE Transactions on Human-Machine Systems.

[26]  Hugh M. Herr,et al.  Powered ankle-foot prosthesis to assist level-ground and stair-descent gaits , 2008, Neural Networks.

[27]  Levi J. Hargrove,et al.  A Classification Method for User-Independent Intent Recognition for Transfemoral Amputees Using Powered Lower Limb Prostheses , 2016, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[28]  H. Hermens,et al.  European recommendations for surface electromyography: Results of the SENIAM Project , 1999 .

[29]  P. Geethanjali,et al.  Identification of motion from multi-channel EMG signals for control of prosthetic hand , 2011, Australasian Physical & Engineering Sciences in Medicine.

[30]  Ann M. Simon,et al.  Across-user adaptation for a powered lower limb prosthesis , 2017, 2017 International Conference on Rehabilitation Robotics (ICORR).

[31]  Robert Riener,et al.  A survey of sensor fusion methods in wearable robotics , 2015, Robotics Auton. Syst..

[32]  Levi J. Hargrove,et al.  Neuromechanical sensor fusion yields highest accuracies in predicting ambulation mode transitions for trans-tibial amputees , 2013, 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[33]  Raja Ariffin Raja Ghazilla,et al.  Reviews on Various Inertial Measurement Unit (IMU) Sensor Applications , 2013, SiPS 2013.

[34]  Angkoon Phinyomark,et al.  EMG feature evaluation for improving myoelectric pattern recognition robustness , 2013, Expert Syst. Appl..

[35]  Qining Wang,et al.  Real-Time On-Board Recognition of Continuous Locomotion Modes for Amputees With Robotic Transtibial Prostheses , 2018, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[36]  Suzanne B. Finucane,et al.  Online adaptive neural control of a robotic lower limb prosthesis , 2018, Journal of neural engineering.

[37]  Xian Huang,et al.  Capacitive Epidermal Electronics for Electrically Safe, Long‐Term Electrophysiological Measurements , 2014, Advanced healthcare materials.

[38]  Ann M. Simon,et al.  Preliminary results for an adaptive pattern recognition system for novel users using a powered lower limb prosthesis , 2016, 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[39]  Long Wang,et al.  Adaptive Slope Walking With a Robotic Transtibial Prosthesis Based on Volitional EMG Control , 2015, IEEE/ASME Transactions on Mechatronics.

[40]  Qing-Song Xu,et al.  Uncorrelated linear discriminant analysis (ULDA): A powerful tool for exploration of metabolomics data , 2008 .

[41]  Long Wang,et al.  A new strategy for parameter optimization to improve phase-dependent locomotion mode recognition , 2015, Neurocomputing.

[42]  Kevin B. Fite,et al.  The Design and Initial Experimental Validation of an Active Myoelectric Transfemoral Prosthesis , 2012 .

[43]  Nitish Thatte,et al.  Robust and Adaptive Lower Limb Prosthesis Stance Control via Extended Kalman Filter-Based Gait Phase Estimation , 2019, IEEE Robotics and Automation Letters.

[44]  Sabri Koçer,et al.  Use of Support Vector Machines and Neural Network in Diagnosis of Neuromuscular Disorders , 2005, Journal of Medical Systems.

[45]  Andrew Kusiak,et al.  Rough set theory: a data mining tool for semiconductor manufacturing , 2001 .

[46]  Qining Wang,et al.  Performance analysis of hardware acceleration for locomotion mode recognition in robotic prosthetic control , 2018, 2018 IEEE International Conference on Cyborg and Bionic Systems (CBS).

[47]  Zhaohui Wu,et al.  Accelerometer-Based Gait Recognition by Sparse Representation of Signature Points With Clusters , 2015, IEEE Transactions on Cybernetics.

[48]  Thomas G. Sugar,et al.  A novel control algorithm for wearable robotics using phase plane invariants , 2009, 2009 IEEE International Conference on Robotics and Automation.

[49]  Todd A. Kuiken,et al.  A Decision-Based Velocity Ramp for Minimizing the Effect of Misclassifications During Real-Time Pattern Recognition Control , 2011, IEEE Transactions on Biomedical Engineering.

[50]  D Graupe,et al.  Multifunctional prosthesis and orthosis control via microcomputer identification of temporal pattern differences in single-site myoelectric signals. , 1982, Journal of biomedical engineering.

[51]  Levi J. Hargrove,et al.  A strategy for labeling data for the neural adaptation of a powered lower limb prosthesis , 2014, 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[52]  Rohit Gupta,et al.  Continuous human locomotion identification for lower limb prosthesis control , 2018, CSI Transactions on ICT.

[53]  Fan Zhang,et al.  On Design and Implementation of Neural-Machine Interface for Artificial Legs , 2012, IEEE Transactions on Industrial Informatics.

[54]  Guanglin Li,et al.  Principal Components Analysis Preprocessing for Improved Classification Accuracies in Pattern-Recognition-Based Myoelectric Control , 2009, IEEE Transactions on Biomedical Engineering.

[55]  R.N. Scott,et al.  A new strategy for multifunction myoelectric control , 1993, IEEE Transactions on Biomedical Engineering.

[56]  Qinghua Huang,et al.  Continuous Monitoring of Sonomyography, Electromyography and Torque Generated by Normal Upper Arm Muscles During Isometric Contraction: Sonomyography Assessment for Arm Muscles , 2008, IEEE Transactions on Biomedical Engineering.

[57]  Michael Goldfarb,et al.  Upslope Walking With a Powered Knee and Ankle Prosthesis: Initial Results With an Amputee Subject , 2011, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[58]  He Huang,et al.  A Strategy for Identifying Locomotion Modes Using Surface Electromyography , 2009, IEEE Transactions on Biomedical Engineering.

[59]  R. Riener,et al.  Stair ascent and descent at different inclinations. , 2002, Gait & posture.

[60]  Hugh M. Herr,et al.  Powered ankle-foot prosthesis , 2008, IEEE Robotics & Automation Magazine.

[61]  Xiaodong Zhang,et al.  Surface EMG based continuous estimation of human lower limb joint angles by using deep belief networks , 2018, Biomed. Signal Process. Control..

[62]  NovakDomen,et al.  A survey of methods for data fusion and system adaptation using autonomic nervous system responses in physiological computing , 2012 .

[63]  Robert D. Gregg,et al.  A Robust Parameterization of Human Gait Patterns Across Phase-Shifting Perturbations , 2017, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[64]  A. McIntosh,et al.  Gait dynamics on an inclined walkway. , 2006, Journal of biomechanics.

[65]  Fan Zhang,et al.  Design of a robust EMG sensing interface for pattern classification , 2010, Journal of neural engineering.

[66]  Nicholas P. Fey,et al.  Intent Recognition in a Powered Lower Limb Prosthesis Using Time History Information , 2013, Annals of Biomedical Engineering.

[67]  Qining Wang,et al.  A strain gauge based locomotion mode recognition method using convolutional neural network , 2019, Adv. Robotics.

[68]  Blair A. Lock,et al.  Adaptive Pattern Recognition of Myoelectric Signals: Exploration of Conceptual Framework and Practical Algorithms , 2009, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[69]  Tommaso Lenzi,et al.  Powered knee and ankle prosthesis with indirect volitional swing control enables level-ground walking and crossing over obstacles , 2020, Science Robotics.

[70]  M. Goldfarb,et al.  Control of Stair Ascent and Descent With a Powered Transfemoral Prosthesis , 2013, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[71]  D. Lefeber,et al.  Cognitive performance and brain dynamics during walking with a novel bionic foot: A pilot study , 2019, PloS one.

[72]  Kevin B. Englehart,et al.  A wavelet-based continuous classification scheme for multifunction myoelectric control , 2001, IEEE Transactions on Biomedical Engineering.

[73]  Qining Wang,et al.  Combining Push-Off Power and Nonlinear Damping Behaviors for a Lightweight Motor-Driven Transtibial Prosthesis , 2017, IEEE/ASME Transactions on Mechatronics.

[74]  Fan Zhang,et al.  Source Selection for Real-Time User Intent Recognition Toward Volitional Control of Artificial Legs , 2013, IEEE Journal of Biomedical and Health Informatics.

[75]  Michael Goldfarb,et al.  A Phase Variable Approach for IMU-Based Locomotion Activity Recognition , 2018, IEEE Transactions on Biomedical Engineering.

[76]  Adrian D. C. Chan,et al.  Continuous myoelectric control for powered prostheses using hidden Markov models , 2005, IEEE Transactions on Biomedical Engineering.

[77]  Qining Wang,et al.  Maximum Dorsiflexion Detection Based on an On-Board Adaptive Algorithm for Transtibial Amputees With Robotic Prostheses , 2020 .

[78]  Michael Goldfarb,et al.  Multiclass Real-Time Intent Recognition of a Powered Lower Limb Prosthesis , 2010, IEEE Transactions on Biomedical Engineering.

[79]  Barbara Caputo,et al.  Improving Control of Dexterous Hand Prostheses Using Adaptive Learning , 2013, IEEE Transactions on Robotics.

[80]  Deok-Hwan Kim,et al.  Real‐Time Locomotion Mode Recognition Employing Correlation Feature Analysis Using EMG Pattern , 2014 .

[81]  Tingfang Yan,et al.  Review of assistive strategies in powered lower-limb orthoses and exoskeletons , 2015, Robotics Auton. Syst..

[82]  Junliang Huang,et al.  EEG-Based Volitional Control of Prosthetic Legs for Walking in Different Terrains , 2021, IEEE Transactions on Automation Science and Engineering.

[83]  Muhammad Zahak Jamal Signal Acquisition Using Surface EMG and Circuit Design Considerations for Robotic Prosthesis , 2012 .

[84]  Fan Zhang,et al.  An Adaptive Classification Strategy for Reliable Locomotion Mode Recognition , 2017, Sensors.

[85]  Brendan J. Frey,et al.  Graphical Models for Machine Learning and Digital Communication , 1998 .

[86]  Ping Zhou,et al.  Real time ECG artifact removal for myoelectric prosthesis control , 2007, Physiological measurement.

[87]  Long Wang,et al.  On the Design of a Powered Transtibial Prosthesis With Stiffness Adaptable Ankle and Toe Joints , 2014, IEEE Transactions on Industrial Electronics.

[88]  Fan Zhang,et al.  Continuous Locomotion-Mode Identification for Prosthetic Legs Based on Neuromuscular–Mechanical Fusion , 2011, IEEE Transactions on Biomedical Engineering.

[89]  Jun Morimoto,et al.  Learning and adaptation of a Stylistic Myoelectric Interface: EMG-based robotic control with individual user differences , 2011, 2011 IEEE International Conference on Robotics and Biomimetics.

[90]  Aboelmagd Noureldin,et al.  A Survey on Approaches of Motion Mode Recognition Using Sensors , 2017, IEEE Transactions on Intelligent Transportation Systems.

[91]  Long Wang,et al.  Fuzzy-Logic-Based Terrain Identification with Multisensor Fusion for Transtibial Amputees , 2015, IEEE/ASME Transactions on Mechatronics.

[92]  Michael Goldfarb,et al.  Design, Control, and Preliminary Assessment of a Multifunctional Semipowered Ankle Prosthesis , 2019, IEEE/ASME Transactions on Mechatronics.

[93]  Erik J. Scheme,et al.  Selective Classification for Improved Robustness of Myoelectric Control Under Nonideal Conditions , 2011, IEEE Transactions on Biomedical Engineering.

[94]  Simona Crea,et al.  Online Estimation of Continuous Gait Phase for Robotic Transtibial Prostheses Based on Adaptive Oscillators , 2020, 2020 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM).

[95]  Robert Riener,et al.  Control strategies for active lower extremity prosthetics and orthotics: a review , 2015, Journal of NeuroEngineering and Rehabilitation.

[96]  David W. Aha,et al.  A Comparative Evaluation of Sequential Feature Selection Algorithms , 1995, AISTATS.

[97]  Kevin B. Englehart,et al.  A robust, real-time control scheme for multifunction myoelectric control , 2003, IEEE Transactions on Biomedical Engineering.

[98]  Timothy Bretl,et al.  Large-area MRI-compatible epidermal electronic interfaces for prosthetic control and cognitive monitoring , 2019, Nature Biomedical Engineering.

[99]  Qining Wang,et al.  Using One Strain Gauge Bridge to Detect Gait Events for a Robotic Prosthesis , 2019, Robotica.

[100]  Uriel Martinez-Hernandez,et al.  Adaptive Bayesian inference system for recognition of walking activities and prediction of gait events using wearable sensors , 2018, Neural Networks.

[101]  Todd A. Kuiken,et al.  The Effect of ECG Interference on Pattern-Recognition-Based Myoelectric Control for Targeted Muscle Reinnervated Patients , 2009, IEEE Transactions on Biomedical Engineering.

[102]  T. Kuiken,et al.  Quantifying Pattern Recognition—Based Myoelectric Control of Multifunctional Transradial Prostheses , 2010, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[103]  Deepak Joshi,et al.  High energy spectrogram with integrated prior knowledge for EMG-based locomotion classification. , 2015, Medical engineering & physics.

[104]  Philipp Beckerle,et al.  Active lower limb prosthetics: a systematic review of design issues and solutions , 2016, BioMedical Engineering OnLine.

[105]  Dennis C. Tkach,et al.  Study of stability of time-domain features for electromyographic pattern recognition , 2010, Journal of NeuroEngineering and Rehabilitation.

[106]  Long Wang,et al.  Walk the Walk: A Lightweight Active Transtibial Prosthesis , 2015, IEEE Robotics & Automation Magazine.

[107]  Michael Goldfarb,et al.  Design and Control of a Powered Transfemoral Prosthesis , 2008, Int. J. Robotics Res..

[108]  Robert D. Gregg,et al.  Continuous-Phase Control of a Powered Knee–Ankle Prosthesis: Amputee Experiments Across Speeds and Inclines , 2018, IEEE Transactions on Robotics.

[109]  Marko Munih,et al.  A survey of methods for data fusion and system adaptation using autonomic nervous system responses in physiological computing , 2012, Interact. Comput..

[110]  J. Rafiee,et al.  Feature extraction of forearm EMG signals for prosthetics , 2011, Expert Syst. Appl..

[111]  Simona Crea,et al.  Capacitive Sensing-Based Continuous Gait Phase Estimation in Robotic Transtibial Prostheses , 2020, 2020 8th IEEE RAS/EMBS International Conference for Biomedical Robotics and Biomechatronics (BioRob).

[112]  Ming Liu,et al.  Effects of Locomotion Mode Recognition Errors on Volitional Control of Powered Above-Knee Prostheses , 2015, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[113]  Qining Wang,et al.  Implementing a SoC-FPGA Based Acceleration System for On-Board SVM Training for Robotic Transtibial Prostheses , 2018, 2018 IEEE International Conference on Real-time Computing and Robotics (RCAR).

[114]  Ann M. Simon,et al.  A Training Method for Locomotion Mode Prediction Using Powered Lower Limb Prostheses , 2014, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[115]  King-Sun Fu,et al.  Sequential Methods in Pattern Recognition and Machine Learning , 2012 .