Multiday EMG-Based Classification of Hand Motions with Deep Learning Techniques

Pattern recognition of electromyography (EMG) signals can potentially improve the performance of myoelectric control for upper limb prostheses with respect to current clinical approaches based on direct control. However, the choice of features for classification is challenging and impacts long-term performance. Here, we propose the use of EMG raw signals as direct inputs to deep networks with intrinsic feature extraction capabilities recorded over multiple days. Seven able-bodied subjects performed six active motions (plus rest), and EMG signals were recorded for 15 consecutive days with two sessions per day using the MYO armband (MYB, a wearable EMG sensor). The classification was performed by a convolutional neural network (CNN) with raw bipolar EMG samples as the inputs, and the performance was compared with linear discriminant analysis (LDA) and stacked sparse autoencoders with features (SSAE-f) and raw samples (SSAE-r) as inputs. CNN outperformed (lower classification error) both LDA and SSAE-r in the within-session, between sessions on same day, between the pair of days, and leave-out one-day evaluation (p < 0.001) analyses. However, no significant difference was found between CNN and SSAE-f. These results demonstrated that CNN significantly improved performance and increased robustness over time compared with standard LDA with associated handcrafted features. This data-driven features extraction approach may overcome the problem of the feature calibration and selection in myoelectric control.

[1]  Max Ortiz-Catalan,et al.  Cardinality as a highly descriptive feature in myoelectric pattern recognition for decoding motor volition , 2015, Front. Neurosci..

[2]  Xinjun Sheng,et al.  User adaptation in long-term, open-loop myoelectric training: implications for EMG pattern recognition in prosthesis control , 2015, Journal of neural engineering.

[3]  Purushothaman Geethanjali,et al.  Myoelectric control of prosthetic hands: state-of-the-art review , 2016, Medical devices.

[4]  Ernest Nlandu Kamavuako,et al.  A novel approach for classification of hand movements using surface EMG signals , 2017, 2017 IEEE International Symposium on Signal Processing and Information Technology (ISSPIT).

[5]  Angkoon Phinyomark,et al.  Feature Extraction and Selection for Myoelectric Control Based on Wearable EMG Sensors , 2018, Sensors.

[6]  Harris Drucker,et al.  Comparison of learning algorithms for handwritten digit recognition , 1995 .

[7]  Erik Scheme,et al.  A feature extraction issue for myoelectric control based on wearable EMG sensors , 2018, 2018 IEEE Sensors Applications Symposium (SAS).

[8]  Beth Jelfs,et al.  Self-Recalibrating Surface EMG Pattern Recognition for Neuroprosthesis Control Based on Convolutional Neural Network , 2017, Front. Neurosci..

[9]  Marek Kurzynski,et al.  Control of Dexterous Hand Via Recognition of EMG Signals Using Combination of Decision-Tree and Sequential Classifier , 2008, Computer Recognition Systems 2.

[10]  Seong-Whan Lee,et al.  Movement intention decoding based on deep learning for multiuser myoelectric interfaces , 2016, 2016 4th International Winter Conference on Brain-Computer Interface (BCI).

[11]  Erik Scheme,et al.  Electromyogram pattern recognition for control of powered upper-limb prostheses: state of the art and challenges for clinical use. , 2011, Journal of rehabilitation research and development.

[12]  Pornchai Phukpattaranont,et al.  Feature reduction and selection for EMG signal classification , 2012, Expert Syst. Appl..

[13]  K. Englehart,et al.  Investigation of Optimum Pattern Recognition Methods for Robust Myoelectric Control During Dynamic Limb Movement , 2017 .

[14]  Nisheena V. Iqbal,et al.  A Review on Upper-Limb Myoelectric Prosthetic Control , 2018 .

[15]  Martin Fodslette Møller,et al.  A scaled conjugate gradient algorithm for fast supervised learning , 1993, Neural Networks.

[16]  Adrian D. C. Chan,et al.  Myoelectric Control Development Toolbox , 2007 .

[17]  Quoc V. Le A Tutorial on Deep Learning Part 2: Autoencoders, Convolutional Neural Networks and Recurrent Neural Networks , 2015 .

[18]  Clément Gosselin,et al.  Deep Learning for Electromyographic Hand Gesture Signal Classification by Leveraging Transfer Learning , 2018, ArXiv.

[19]  Levi J. Hargrove,et al.  Comparison of surface and intramuscular EMG pattern recognition for simultaneous wrist/hand motion classification , 2013, 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[20]  Michael L. Walters,et al.  Application of support vector machines to detect hand and wrist gestures using a myoelectric armband , 2017 .

[21]  Xinjun Sheng,et al.  Towards zero training for myoelectric control based on a wearable wireless sEMG armband , 2015, 2015 IEEE International Conference on Advanced Intelligent Mechatronics (AIM).

[22]  Huosheng Hu,et al.  Support Vector Machine-Based Classification Scheme for Myoelectric Control Applied to Upper Limb , 2008, IEEE Transactions on Biomedical Engineering.

[23]  Mads Jochumsen,et al.  Performance of Combined Surface and Intramuscular EMG for Classification of Hand Movements , 2018, 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[24]  R. N. Scott Myoelectric Control Of Prostheses: A Brief History , 1992 .

[25]  Marc'Aurelio Ranzato,et al.  Large Scale Distributed Deep Networks , 2012, NIPS.

[26]  Manfredo Atzori,et al.  Deep Learning with Convolutional Neural Networks Applied to Electromyography Data: A Resource for the Classification of Movements for Prosthetic Hands , 2016, Front. Neurorobot..

[27]  Todd A. Kuiken,et al.  Evaluating EMG Feature and Classifier Selection for Application to Partial-Hand Prosthesis Control , 2016, Front. Neurorobot..

[28]  R.N. Scott,et al.  A new strategy for multifunction myoelectric control , 1993, IEEE Transactions on Biomedical Engineering.

[29]  Manfredo Atzori,et al.  Electromyography data for non-invasive naturally-controlled robotic hand prostheses , 2014, Scientific Data.

[30]  Yinghong Peng,et al.  EMG‐Based Estimation of Limb Movement Using Deep Learning With Recurrent Convolutional Neural Networks , 2018, Artificial organs.

[31]  Angkoon Phinyomark,et al.  EMG feature evaluation for improving myoelectric pattern recognition robustness , 2013, Expert Syst. Appl..

[32]  Yoshua Bengio,et al.  Greedy Layer-Wise Training of Deep Networks , 2006, NIPS.

[33]  Hong Liu,et al.  Wearing-independent hand gesture recognition method based on EMG armband , 2018, Personal and Ubiquitous Computing.

[34]  Byunghan Lee,et al.  Deep learning in bioinformatics , 2016, Briefings Bioinform..

[35]  Osamu Matsumoto,et al.  Hand posture and gesture recognition using MYO armband and spectral collaborative representation based classification , 2015, 2015 IEEE 4th Global Conference on Consumer Electronics (GCCE).

[36]  Eros Gian Alessandro Pasero,et al.  EEG Based Eye State Classification using Deep Belief Network and Stacked AutoEncoder , 2016 .

[37]  Weidong Geng,et al.  Gesture recognition by instantaneous surface EMG images , 2016, Scientific Reports.

[38]  Tianming Liu,et al.  Predicting eye fixations using convolutional neural networks , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  Elisabeth André,et al.  EMG-based hand gesture recognition for realtime biosignal interfacing , 2008, IUI '08.

[40]  Manfredo Atzori,et al.  Building the Ninapro database: A resource for the biorobotics community , 2012, 2012 4th IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob).

[41]  Yu Hu,et al.  Surface EMG-Based Inter-Session Gesture Recognition Enhanced by Deep Domain Adaptation , 2017, Sensors.

[42]  Jason Weston,et al.  A unified architecture for natural language processing: deep neural networks with multitask learning , 2008, ICML '08.

[43]  Ernest Nlandu Kamavuako,et al.  Variability of hand motions quantified using EMG root mean square and mean frequency , 2018 .

[44]  Mohan S. Kankanhalli,et al.  Semi-Supervised Learning for Surface EMG-based Gesture Recognition , 2017, IJCAI.

[45]  I. A. Sulistijono,et al.  Comparison of five time series EMG features extractions using Myo Armband , 2015, 2015 International Electronics Symposium (IES).

[46]  Mikhail A. Lebedev,et al.  Recognition of Handwriting from Electromyography , 2009, PloS one.

[47]  Loredana Zollo,et al.  NLR, MLP, SVM, and LDA: a comparative analysis on EMG data from people with trans-radial amputation , 2017, Journal of NeuroEngineering and Rehabilitation.

[48]  G. Lundborg,et al.  Refined myoelectric control in below-elbow amputees using artificial neural networks and a data glove. , 2005, The Journal of hand surgery.

[49]  K. Englehart,et al.  Determination of optimum threshold values for EMG time domain features; a multi-dataset investigation. , 2016, Journal of neural engineering.

[50]  Kiyoshi Kotani,et al.  A Novel Unsupervised Adaptive Learning Method for Long-Term Electromyography (EMG) Pattern Recognition , 2017, Sensors.

[51]  Reza Langari,et al.  Subject-independent hand gesture recognition using normalization and machine learning algorithms , 2018, J. Comput. Sci..

[52]  U. Rajendra Acharya,et al.  Application of deep convolutional neural network for automated detection of myocardial infarction using ECG signals , 2017, Inf. Sci..

[53]  Yoshua Bengio,et al.  Attention-Based Models for Speech Recognition , 2015, NIPS.

[54]  J. Muñoz,et al.  Muscle fatigue detection through wearable sensors: a comparative study using the myo armband , 2017, Interacción.

[55]  Erik Scheme,et al.  Navigating features: a topologically informed chart of electromyographic features space , 2017, Journal of The Royal Society Interface.

[56]  T. Kuiken,et al.  Quantifying Pattern Recognition—Based Myoelectric Control of Multifunctional Transradial Prostheses , 2010, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[57]  Ernest Nlandu Kamavuako,et al.  Combined surface and intramuscular EMG for improved real-time myoelectric control performance , 2014, Biomed. Signal Process. Control..

[58]  Ernest Nlandu Kamavuako,et al.  Evaluation of the Myo armband for the classification of hand motions , 2017, 2017 International Conference on Rehabilitation Robotics (ICORR).

[59]  Dario Farina,et al.  User adaptation in Myoelectric Man-Machine Interfaces , 2017, Scientific Reports.

[60]  Luca Benini,et al.  A Versatile Embedded Platform for EMG Acquisition and Gesture Recognition , 2015, IEEE Transactions on Biomedical Circuits and Systems.

[61]  Mohan S. Kankanhalli,et al.  A multi-stream convolutional neural network for sEMG-based gesture recognition in muscle-computer interface , 2017, Pattern Recognit. Lett..

[62]  Kongqiao Wang,et al.  Hand Gesture Recognition Research Based on Surface EMG Sensors and 2D-accelerometers , 2007, 2007 11th IEEE International Symposium on Wearable Computers.

[63]  Pornchai Phukpattaranont,et al.  A Novel Feature Extraction for Robust EMG Pattern Recognition , 2009, ArXiv.

[64]  Mary L McHugh,et al.  Multiple comparison analysis testing in ANOVA. , 2011, Biochemia medica.

[65]  K. K. Ray,et al.  EMG based man-machine interaction - A pattern recognition research platform , 2014, Robotics Auton. Syst..

[66]  Dario Farina,et al.  Stacked Sparse Autoencoders for EMG-Based Classification of Hand Motions: A Comparative Multi Day Analyses between Surface and Intramuscular EMG , 2018, Applied Sciences.

[67]  Clément Gosselin,et al.  Transfer learning for sEMG hand gestures recognition using convolutional neural networks , 2017, 2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[68]  Antonio Jimeno-Yepes,et al.  Decoding EEG and LFP signals using deep learning: heading TrueNorth , 2016, Conf. Computing Frontiers.

[69]  Dennis C. Tkach,et al.  Study of stability of time-domain features for electromyographic pattern recognition , 2010, Journal of NeuroEngineering and Rehabilitation.

[70]  Dario Farina,et al.  Myoelectric Control of Artificial Limbs¿Is There a Need to Change Focus? [In the Spotlight] , 2012, IEEE Signal Process. Mag..

[71]  Manfredo Atzori,et al.  Comparison of six electromyography acquisition setups on hand movement classification tasks , 2017, PloS one.