Multi-grip classification-based prosthesis control with two EMG-IMU sensors

In the field of upper-limb myoelectric prosthesis control, the use of statistical and machine learning methods has been long proposed as a means of enabling intuitive grip selection and actuation. Recently, this paradigm has found its way toward commercial adoption. Machine learning-based prosthesis control typically relies on the use of a large number of electrodes. Here, we propose an end-to-end strategy for multi-grip, classification-based prosthesis control using only two sensors, comprising electromyography (EMG) electrodes and inertial measurement units (IMUs). We emphasize the importance of accurately estimating posterior class probabilities and rejecting predictions made with low confidence, so as to minimize the rate of unintended prosthesis activations. To that end, we propose a confidence-based error rejection strategy using grip-specific thresholds. We evaluate the efficacy of the proposed system with real-time pick and place experiments using a commercial multi-articulated prosthetic hand and involving 12 able-bodied and two transradial (i.e., below-elbow) amputee participants. Results promise the potential for deploying intuitive, classification-based multi-grip control in existing upper-limb prosthetic systems subject to small modifications.

[1]  J. Friedman Regularized Discriminant Analysis , 1989 .

[2]  K. Englehart,et al.  Classification of the myoelectric signal using time-frequency based representations. , 1999, Medical engineering & physics.

[3]  Rieko Osu,et al.  The central nervous system stabilizes unstable dynamics by learning optimal impedance , 2001, Nature.

[4]  Kevin B. Englehart,et al.  A robust, real-time control scheme for multifunction myoelectric control , 2003, IEEE Transactions on Biomedical Engineering.

[5]  B Hudgins,et al.  Myoelectric signal processing for control of powered limb prostheses. , 2006, Journal of electromyography and kinesiology : official journal of the International Society of Electrophysiological Kinesiology.

[6]  Mithat Gonen,et al.  Analyzing Receiver Operating Characteristic Curves with SAS , 2007 .

[7]  Levi J. Hargrove,et al.  A Comparison of Surface and Intramuscular Myoelectric Signal Classification , 2007, IEEE Transactions on Biomedical Engineering.

[8]  D. Hall Analyzing Receiver Operating Characteristic Curves With SAS , 2008 .

[9]  He Huang,et al.  An Analysis of EMG Electrode Configuration for Targeted Muscle Reinnervation Based Neural Machine Interface , 2008, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[10]  Nitish V. Thakor,et al.  Decoding of Individuated Finger Movements Using Surface Electromyography , 2009, IEEE Transactions on Biomedical Engineering.

[11]  Giulio Sandini,et al.  Fine detection of grasp force and posture by amputees via surface electromyography , 2009, Journal of Physiology-Paris.

[12]  K.B. Englehart,et al.  Multiple Binary Classifications via Linear Discriminant Analysis for Improved Controllability of a Powered Prosthesis , 2010, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[13]  T. Kuiken,et al.  Quantifying Pattern Recognition—Based Myoelectric Control of Multifunctional Transradial Prostheses , 2010, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[14]  Erik J. Scheme,et al.  Selective Classification for Improved Robustness of Myoelectric Control Under Nonideal Conditions , 2011, IEEE Transactions on Biomedical Engineering.

[15]  Todd A Kuiken,et al.  Target Achievement Control Test: evaluating real-time myoelectric pattern-recognition control of multifunctional upper-limb prostheses. , 2011, Journal of rehabilitation research and development.

[16]  Pedro M. Domingos A few useful things to know about machine learning , 2012, Commun. ACM.

[17]  Jasper Snoek,et al.  Practical Bayesian Optimization of Machine Learning Algorithms , 2012, NIPS.

[18]  Ilja Kuzborskij,et al.  On the challenge of classifying 52 hand movements from surface electromyography , 2012, 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[19]  Nathan E. Bunderson,et al.  Quantification of Feature Space Changes With Experience During Electromyogram Pattern Recognition Control , 2012, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[20]  Holger Urbanek,et al.  Relation between object properties and EMG during reaching to grasp. , 2013, Journal of electromyography and kinesiology : official journal of the International Society of Electrophysiological Kinesiology.

[21]  Erik J. Scheme,et al.  Confidence-Based Rejection for Improved Pattern Recognition Myoelectric Control , 2013, IEEE Transactions on Biomedical Engineering.

[22]  K. Englehart,et al.  Electromyogram Whitening for Improved Classification Accuracy in Upper Limb Prosthesis Control , 2013, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[23]  Erik J. Scheme,et al.  Validation of a Selective Ensemble-Based Classification Scheme for Myoelectric Control Using a Three-Dimensional Fitts' Law Test , 2013, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[24]  Angkoon Phinyomark,et al.  EMG feature evaluation for improving myoelectric pattern recognition robustness , 2013, Expert Syst. Appl..

[25]  Guido Bugmann,et al.  Classification of Finger Movements for the Dexterous Hand Prosthesis Control With Surface Electromyography , 2013, IEEE Journal of Biomedical and Health Informatics.

[26]  Øyvind Stavdahl,et al.  System training and assessment in simultaneous proportional myoelectric prosthesis control , 2013, Journal of NeuroEngineering and Rehabilitation.

[27]  Dario Farina,et al.  Extracting Signals Robust to Electrode Number and Shift for Online Simultaneous and Proportional Myoelectric Control by Factorization Algorithms , 2014, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[28]  Nitish V. Thakor,et al.  User Training for Pattern Recognition-Based Myoelectric Prostheses: Improving Phantom Limb Movement Consistency and Distinguishability , 2014, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[29]  Dario Farina,et al.  Self-Correcting Pattern Recognition System of Surface EMG Signals for Upper Limb Prosthesis Control , 2014, IEEE Transactions on Biomedical Engineering.

[30]  Klaus-Robert Müller,et al.  Channel selection for simultaneous and proportional myoelectric prosthesis control of multiple degrees-of-freedom , 2014, Journal of neural engineering.

[31]  Yuan-Ting Zhang,et al.  A novel channel selection method for multiple motion classification using high-density electromyography , 2014, BioMedical Engineering OnLine.

[32]  K. Englehart,et al.  On the Suitability of Integrating Accelerometry Data with Electromyography Signals for Resolving the Effect of Changes in Limb Position during Dynamic Limb Movement , 2014 .

[33]  Dario Farina,et al.  The Extraction of Neural Information from the Surface EMG for the Control of Upper-Limb Prostheses: Emerging Avenues and Challenges , 2014, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[34]  Max Ortiz-Catalan,et al.  Real-Time and Simultaneous Control of Artificial Limbs Based on Pattern Recognition Algorithms , 2014, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[35]  Max Ortiz-Catalan,et al.  Offline accuracy: A potentially misleading metric in myoelectric pattern recognition for prosthetic control , 2015, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[36]  E Scheme,et al.  A comparison of classification based confidence metrics for use in the design of myoelectric control systems , 2015, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[37]  John J. Soraghan,et al.  Automatic misclassification rejection for LDA classifier using ROC curves , 2015, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[38]  Weidong Geng,et al.  Gesture recognition by instantaneous surface EMG images , 2016, Scientific Reports.

[39]  Todd A. Kuiken,et al.  An Analysis of Intrinsic and Extrinsic Hand Muscle EMG for Improved Pattern Recognition Control , 2016, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[40]  G. Naik,et al.  Transradial Amputee Gesture Classification Using an Optimal Number of sEMG Sensors: An Approach Using ICA Clustering , 2016, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[41]  E. Clancy,et al.  Two degrees of freedom quasi-static EMG-force at the wrist using a minimum number of electrodes. , 2017, Journal of electromyography and kinesiology : official journal of the International Society of Electrophysiological Kinesiology.

[42]  Dario Farina,et al.  Translating Research on Myoelectric Control into Clinics—Are the Performance Assessment Methods Adequate? , 2017, Front. Neurorobot..

[43]  Adel Al-Jumaily,et al.  A Framework of Temporal-Spatial Descriptors-Based Feature Extraction for Improved Myoelectric Pattern Recognition , 2017, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[44]  Sethu Vijayakumar,et al.  Improved prosthetic hand control with concurrent use of myoelectric and inertial measurements , 2017, Journal of NeuroEngineering and Rehabilitation.

[45]  Marco Santello,et al.  Proof of Concept of an Online EMG-Based Decoding of Hand Postures and Individual Digit Forces for Prosthetic Hand Control , 2017, Front. Neurol..

[46]  Sethu Vijayakumar,et al.  Use of regularized discriminant analysis improves myoelectric hand movement classification , 2017, 2017 8th International IEEE/EMBS Conference on Neural Engineering (NER).

[47]  Enzo Mastinu,et al.  An Alternative Myoelectric Pattern Recognition Approach for the Control of Hand Prostheses: A Case Study of Use in Daily Life by a Dysmelia Subject , 2018, IEEE Journal of Translational Engineering in Health and Medicine.

[48]  Aude Billard,et al.  Decoding the grasping intention from electromyography during reaching motions , 2018, Journal of NeuroEngineering and Rehabilitation.

[49]  Ali Samadani EMG Channel Selection for Improved Hand Gesture Classification , 2018, 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[50]  Erik J. Scheme,et al.  Effects of Confidence-Based Rejection on Usability and Error in Pattern Recognition-Based Myoelectric Control , 2019, IEEE Journal of Biomedical and Health Informatics.

[51]  Xinjun Sheng,et al.  Spatial Information Enhances Myoelectric Control Performance With Only Two Channels , 2019, IEEE Transactions on Industrial Informatics.