Myoelectric digit action decoding with multi-label, multi-class classification: an offline analysis

The ultimate goal of machine learning-based myoelectric control is simultaneous and independent control of multiple degrees of freedom (DOFs), including wrist and digit artificial joints. For prosthetic finger control, regression-based methods are typically used to reconstruct position/velocity trajectories from surface electromyogram (EMG) signals. Although such methods have produced highly-accurate results in offline analyses, their success in real-time prosthesis control settings has been rather limited. In this work, we propose action decoding, a paradigm-shifting approach for independent, multi-digit movement intent decoding based on multi-label, multi-class classification. At each moment in time, our algorithm classifies movement action for each available DOF into one of three categories: open, close, or stall (i.e., no movement). Despite using a classifier as the decoder, arbitrary hand postures are possible with our approach. We analyse a public dataset previously recorded and published by us, comprising measurements from 10 able-bodied and two transradial amputee participants. We demonstrate the feasibility of using our proposed action decoding paradigm to predict movement action for all five digits as well as rotation of the thumb. We perform a systematic offline analysis by investigating the effect of various algorithmic parameters on decoding performance, such as feature selection and choice of classification algorithm and multi-output strategy. The outcomes of the offline analysis presented in this study will be used to inform the real-time implementation of our algorithm. In the future, we will further evaluate its efficacy with real-time control experiments involving upper-limb amputees.

[1]  Constantinos Gavriel,et al.  Gaussian Process Regression for accurate prediction of prosthetic limb movements from the natural kinematics of intact limbs , 2015, 2015 7th International IEEE/EMBS Conference on Neural Engineering (NER).

[2]  Bruce C. Wheeler,et al.  EMG feature evaluation for movement control of upper extremity prostheses , 1995 .

[3]  Øyvind Stavdahl,et al.  System training and assessment in simultaneous proportional myoelectric prosthesis control , 2013, Journal of NeuroEngineering and Rehabilitation.

[4]  Eyke Hüllermeier,et al.  An Analysis of Chaining in Multi-Label Classification , 2012, ECAI.

[5]  Marco Santello,et al.  Proof of Concept of an Online EMG-Based Decoding of Hand Postures and Individual Digit Forces for Prosthetic Hand Control , 2017, Front. Neurol..

[6]  D. Farina,et al.  Linear and Nonlinear Regression Techniques for Simultaneous and Proportional Myoelectric Control , 2014, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[7]  Max Ortiz-Catalan,et al.  Real-Time and Simultaneous Control of Artificial Limbs Based on Pattern Recognition Algorithms , 2014, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[8]  Sethu Vijayakumar,et al.  Evaluation of regression methods for the continuous decoding of finger movement from surface EMG and accelerometry , 2015, 2015 7th International IEEE/EMBS Conference on Neural Engineering (NER).

[9]  B. Hjorth EEG analysis based on time domain properties. , 1970, Electroencephalography and clinical neurophysiology.

[10]  Pornchai Phukpattaranont,et al.  Feature reduction and selection for EMG signal classification , 2012, Expert Syst. Appl..

[11]  Antonio Frisoli,et al.  Online Finger Control Using High-Density EMG and Minimal Training Data for Robotic Applications , 2019, IEEE Robotics and Automation Letters.

[12]  Kianoush Nazarpour,et al.  Effect of User Practice on Prosthetic Finger Control With an Intuitive Myoelectric Decoder , 2019, Front. Neurosci..

[13]  Dario Farina,et al.  Is Accurate Mapping of EMG Signals on Kinematics Needed for Precise Online Myoelectric Control? , 2014, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[14]  Erik Scheme,et al.  Electromyogram pattern recognition for control of powered upper-limb prostheses: state of the art and challenges for clinical use. , 2011, Journal of rehabilitation research and development.

[15]  Josef Kittler,et al.  Floating search methods in feature selection , 1994, Pattern Recognit. Lett..

[16]  Sethu Vijayakumar,et al.  Towards low-dimensionsal proportional myoelectric control , 2015, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[17]  Grigorios Tsoumakas,et al.  Multi-Label Classification: An Overview , 2007, Int. J. Data Warehous. Min..

[18]  Dario Farina,et al.  Myoelectric Control of Artificial Limbs¿Is There a Need to Change Focus? [In the Spotlight] , 2012, IEEE Signal Process. Mag..

[19]  E. Biddiss,et al.  Upper limb prosthesis use and abandonment: A survey of the last 25 years , 2007, Prosthetics and orthotics international.

[20]  Dario Farina,et al.  Clinical Perspectives in Upper Limb Prostheses: An Update , 2019, Current Surgery Reports.

[21]  Sunita Sarawagi,et al.  Discriminative Methods for Multi-labeled Classification , 2004, PAKDD.

[22]  Dario Farina,et al.  Translating Research on Myoelectric Control into Clinics—Are the Performance Assessment Methods Adequate? , 2017, Front. Neurorobot..

[23]  Levi J Hargrove,et al.  A real-time comparison between direct control, sequential pattern recognition control and simultaneous pattern recognition control using a Fitts’ law style assessment procedure , 2014, Journal of NeuroEngineering and Rehabilitation.

[24]  Kianoush Nazarpour,et al.  Discrete action control for prosthetic digits , 2020, bioRxiv.

[25]  Christian Antfolk,et al.  Extraction of Multi-Labelled Movement Information from the Raw HD-sEMG Image with Time-Domain Depth , 2019, Scientific Reports.

[26]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[27]  Tomohiro Shibata,et al.  Continuous and simultaneous estimation of finger kinematics using inputs from an EMG-to-muscle activation model , 2014, Journal of NeuroEngineering and Rehabilitation.

[28]  Tjerk P. Straatsma,et al.  NWChem: A comprehensive and scalable open-source solution for large scale molecular simulations , 2010, Comput. Phys. Commun..

[29]  M Controzzi,et al.  Online Myoelectric Control of a Dexterous Hand Prosthesis by Transradial Amputees , 2011, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[30]  Barbara Caputo,et al.  Stable myoelectric control of a hand prosthesis using non-linear incremental learning , 2014, Front. Neurorobot..

[31]  Nitish V. Thakor,et al.  Continuous decoding of finger position from surface EMG signals for the control of powered prostheses , 2008, 2008 30th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[32]  Sethu Vijayakumar,et al.  Continuous Versus Discrete Simultaneous Control of Prosthetic Fingers , 2018, 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[33]  Geoff Holmes,et al.  Classifier chains for multi-label classification , 2009, Machine Learning.

[34]  R.N. Scott,et al.  A new strategy for multifunction myoelectric control , 1993, IEEE Transactions on Biomedical Engineering.

[35]  Mohammad S. Sorower A Literature Survey on Algorithms for Multi-label Learning , 2010 .

[36]  Mohammad Hassan Moradi,et al.  Evaluation of the forearm EMG signal features for the control of a prosthetic hand. , 2003, Physiological measurement.

[37]  Raphael Vallat,et al.  Pingouin: statistics in Python , 2018, J. Open Source Softw..

[38]  Dario Farina,et al.  New developments in prosthetic arm systems , 2016, Orthopedic research and reviews.

[39]  Sethu Vijayakumar,et al.  Use of regularized discriminant analysis improves myoelectric hand movement classification , 2017, 2017 8th International IEEE/EMBS Conference on Neural Engineering (NER).

[40]  Dario Farina,et al.  Simultaneous control of multiple functions of bionic hand prostheses: Performance and robustness in end users , 2018, Science Robotics.

[41]  Levi J. Hargrove,et al.  Classification of Simultaneous Movements Using Surface EMG Pattern Recognition , 2013, IEEE Transactions on Biomedical Engineering.

[42]  Todd A Kuiken,et al.  Real-time simultaneous and proportional myoelectric control using intramuscular EMG , 2014, Journal of neural engineering.

[43]  Adel Al-Jumaily,et al.  Orthogonal Fuzzy Neighborhood Discriminant Analysis for Multifunction Myoelectric Hand Control , 2010, IEEE Transactions on Biomedical Engineering.

[44]  Ahmad R. Sharafat,et al.  Application of Higher Order Statistics to Surface Electromyogram Signal Classification , 2007, IEEE Transactions on Biomedical Engineering.

[45]  Patrick van der Smagt,et al.  Surface EMG in advanced hand prosthetics , 2008, Biological Cybernetics.

[46]  Constantinos Gavriel,et al.  Gaussian Process Autoregression for Simultaneous Proportional Multi-Modal Prosthetic Control With Natural Hand Kinematics , 2017, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[47]  Dario Farina,et al.  Extracting Signals Robust to Electrode Number and Shift for Online Simultaneous and Proportional Myoelectric Control by Factorization Algorithms , 2014, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[48]  Nicolas Sommer,et al.  Shared human–robot proportional control of a dexterous myoelectric prosthesis , 2019, Nature Machine Intelligence.

[49]  Max Ortiz-Catalan,et al.  Offline accuracy: A potentially misleading metric in myoelectric pattern recognition for prosthetic control , 2015, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[50]  Lauren H Smith,et al.  A comparison of the real-time controllability of pattern recognition to conventional myoelectric control for discrete and simultaneous movements , 2012, Journal of NeuroEngineering and Rehabilitation.