Myoelectric digit action decoding with multi-output, multi-class classification: an offline analysis

The ultimate goal of machine learning-based myoelectric control is simultaneous and independent control of multiple degrees of freedom (DOFs), including wrist and digit artificial joints. For prosthetic finger control, regression-based methods are typically used to reconstruct position/velocity trajectories from surface electromyogram (EMG) signals. Unfortunately, such methods have thus far met with limited success. In this work, we propose action decoding, a paradigm-shifting approach for independent, multi-digit movement intent prediction based on multi-output, multi-class classification. At each moment in time, our algorithm decodes movement intent for each available DOF into one of three classes: open, close, or stall (i.e., no movement). Despite using a classifier as the decoder, arbitrary hand postures are possible with our approach. We analyse a public dataset previously recorded and published by us, comprising measurements from 10 able-bodied and two transradial amputee participants. We demonstrate the feasibility of using our proposed action decoding paradigm to predict movement action for all five digits as well as rotation of the thumb. We perform a systematic offline analysis by investigating the effect of various algorithmic parameters on decoding performance, such as feature selection and choice of classification algorithm and multi-output strategy. The outcomes of the offline analysis presented in this study will be used to inform the real-time implementation of our algorithm. In the future, we will further evaluate its efficacy with real-time control experiments involving upper-limb amputees.

[1]  K. Englehart,et al.  Resolving the Limb Position Effect in Myoelectric Pattern Recognition , 2011, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[2]  Mohammad Hassan Moradi,et al.  Evaluation of the forearm EMG signal features for the control of a prosthetic hand. , 2003, Physiological measurement.

[3]  Bruce C. Wheeler,et al.  EMG feature evaluation for movement control of upper extremity prostheses , 1995 .

[4]  Kianoush Nazarpour,et al.  Combined influence of forearm orientation and muscular contraction on EMG pattern recognition , 2016, Expert Syst. Appl..

[5]  Erik Scheme,et al.  Electromyogram pattern recognition for control of powered upper-limb prostheses: state of the art and challenges for clinical use. , 2011, Journal of rehabilitation research and development.

[6]  Josef Kittler,et al.  Floating search methods in feature selection , 1994, Pattern Recognit. Lett..

[7]  Sethu Vijayakumar,et al.  Evaluation of regression methods for the continuous decoding of finger movement from surface EMG and accelerometry , 2015, 2015 7th International IEEE/EMBS Conference on Neural Engineering (NER).

[8]  Pornchai Phukpattaranont,et al.  Feature reduction and selection for EMG signal classification , 2012, Expert Syst. Appl..

[9]  Raphael Vallat,et al.  Pingouin: statistics in Python , 2018, J. Open Source Softw..

[10]  Antonio Frisoli,et al.  Online Finger Control Using High-Density EMG and Minimal Training Data for Robotic Applications , 2019, IEEE Robotics and Automation Letters.

[11]  Kianoush Nazarpour,et al.  Effect of User Practice on Prosthetic Finger Control With an Intuitive Myoelectric Decoder , 2019, Front. Neurosci..

[12]  Dario Farina,et al.  New developments in prosthetic arm systems , 2016, Orthopedic research and reviews.

[13]  Sethu Vijayakumar,et al.  Use of regularized discriminant analysis improves myoelectric hand movement classification , 2017, 2017 8th International IEEE/EMBS Conference on Neural Engineering (NER).

[14]  Dario Farina,et al.  Simultaneous control of multiple functions of bionic hand prostheses: Performance and robustness in end users , 2018, Science Robotics.

[15]  Sethu Vijayakumar,et al.  Continuous Versus Discrete Simultaneous Control of Prosthetic Fingers , 2018, 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[16]  Paul Burch,et al.  Commercial fishing patterns influence odontocete whale-longline interactions in the Southern Ocean , 2019, Scientific Reports.

[17]  Dario Farina,et al.  Is Accurate Mapping of EMG Signals on Kinematics Needed for Precise Online Myoelectric Control? , 2014, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[18]  M Controzzi,et al.  Online Myoelectric Control of a Dexterous Hand Prosthesis by Transradial Amputees , 2011, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[19]  Sethu Vijayakumar,et al.  Towards low-dimensionsal proportional myoelectric control , 2015, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[20]  Grigorios Tsoumakas,et al.  Multi-Label Classification: An Overview , 2007, Int. J. Data Warehous. Min..

[21]  Todd A Kuiken,et al.  Real-time simultaneous and proportional myoelectric control using intramuscular EMG , 2014, Journal of neural engineering.

[22]  Adel Al-Jumaily,et al.  Orthogonal Fuzzy Neighborhood Discriminant Analysis for Multifunction Myoelectric Hand Control , 2010, IEEE Transactions on Biomedical Engineering.

[23]  Ahmad R. Sharafat,et al.  Application of Higher Order Statistics to Surface Electromyogram Signal Classification , 2007, IEEE Transactions on Biomedical Engineering.

[24]  Levi J Hargrove,et al.  A real-time comparison between direct control, sequential pattern recognition control and simultaneous pattern recognition control using a Fitts’ law style assessment procedure , 2014, Journal of NeuroEngineering and Rehabilitation.

[25]  Øyvind Stavdahl,et al.  System training and assessment in simultaneous proportional myoelectric prosthesis control , 2013, Journal of NeuroEngineering and Rehabilitation.

[26]  Eyke Hüllermeier,et al.  An Analysis of Chaining in Multi-Label Classification , 2012, ECAI.

[27]  Sunita Sarawagi,et al.  Discriminative Methods for Multi-labeled Classification , 2004, PAKDD.

[28]  B. Hjorth EEG analysis based on time domain properties. , 1970, Electroencephalography and clinical neurophysiology.

[29]  D. Farina,et al.  Linear and Nonlinear Regression Techniques for Simultaneous and Proportional Myoelectric Control , 2014, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[30]  Dario Farina,et al.  Myoelectric Control of Artificial Limbs¿Is There a Need to Change Focus? [In the Spotlight] , 2012, IEEE Signal Process. Mag..

[31]  E. Biddiss,et al.  Upper limb prosthesis use and abandonment: A survey of the last 25 years , 2007, Prosthetics and orthotics international.

[32]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[33]  Constantinos Gavriel,et al.  Gaussian Process Regression for accurate prediction of prosthetic limb movements from the natural kinematics of intact limbs , 2015, 2015 7th International IEEE/EMBS Conference on Neural Engineering (NER).

[34]  Marco Santello,et al.  Proof of Concept of an Online EMG-Based Decoding of Hand Postures and Individual Digit Forces for Prosthetic Hand Control , 2017, Front. Neurol..

[35]  Max Ortiz-Catalan,et al.  Real-Time and Simultaneous Control of Artificial Limbs Based on Pattern Recognition Algorithms , 2014, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[36]  Dario Farina,et al.  The Extraction of Neural Information from the Surface EMG for the Control of Upper-Limb Prostheses: Emerging Avenues and Challenges , 2014, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[37]  R.N. Scott,et al.  A new strategy for multifunction myoelectric control , 1993, IEEE Transactions on Biomedical Engineering.

[38]  Dario Farina,et al.  Clinical Perspectives in Upper Limb Prostheses: An Update , 2019, Current Surgery Reports.

[39]  Dario Farina,et al.  Translating Research on Myoelectric Control into Clinics—Are the Performance Assessment Methods Adequate? , 2017, Front. Neurorobot..

[40]  Levi J. Hargrove,et al.  Classification of Simultaneous Movements Using Surface EMG Pattern Recognition , 2013, IEEE Transactions on Biomedical Engineering.

[41]  Barbara Caputo,et al.  Stable myoelectric control of a hand prosthesis using non-linear incremental learning , 2014, Front. Neurorobot..

[42]  Nitish V. Thakor,et al.  Continuous decoding of finger position from surface EMG signals for the control of powered prostheses , 2008, 2008 30th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[43]  Geoff Holmes,et al.  Classifier chains for multi-label classification , 2009, Machine Learning.

[44]  Tomohiro Shibata,et al.  Continuous and simultaneous estimation of finger kinematics using inputs from an EMG-to-muscle activation model , 2014, Journal of NeuroEngineering and Rehabilitation.

[45]  Patrick van der Smagt,et al.  Surface EMG in advanced hand prosthetics , 2008, Biological Cybernetics.

[46]  Constantinos Gavriel,et al.  Gaussian Process Autoregression for Simultaneous Proportional Multi-Modal Prosthetic Control With Natural Hand Kinematics , 2017, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[47]  Dario Farina,et al.  Extracting Signals Robust to Electrode Number and Shift for Online Simultaneous and Proportional Myoelectric Control by Factorization Algorithms , 2014, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[48]  Kianoush Nazarpour,et al.  Discrete action control for prosthetic digits , 2020, bioRxiv.

[49]  Christian Antfolk,et al.  Extraction of Multi-Labelled Movement Information from the Raw HD-sEMG Image with Time-Domain Depth , 2019, Scientific Reports.

[50]  Nicolas Sommer,et al.  Shared human–robot proportional control of a dexterous myoelectric prosthesis , 2019, Nature Machine Intelligence.

[51]  Max Ortiz-Catalan,et al.  Offline accuracy: A potentially misleading metric in myoelectric pattern recognition for prosthetic control , 2015, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[52]  Lauren H Smith,et al.  A comparison of the real-time controllability of pattern recognition to conventional myoelectric control for discrete and simultaneous movements , 2012, Journal of NeuroEngineering and Rehabilitation.