Aalborg Universitet Stacked sparse autoencoders for EMG-based classification of hand motions

Advances in myoelectric interfaces have increased the use of wearable prosthetics including robotic arms. Although promising results have been achieved with pattern recognition-based control schemes, control robustness requires improvement to increase user acceptance of prosthetic hands. The aim of this study was to quantify the performance of stacked sparse autoencoders (SSAE), an emerging deep learning technique used to improve myoelectric control and to compare multiday surface electromyography (sEMG) and intramuscular (iEMG) recordings. Ten able-bodied and six amputee subjects with average ages of 24.5 and 34.5 years, respectively, were evaluated using offline classification error as the performance matric. Surface and intramuscular EMG were concurrently recorded while each subject performed 11 hand motions. Performance of SSAE was compared with that of linear discriminant analysis (LDA) classifier. Within-day analysis showed that SSAE (1.38 ± 1.38%) outperformed LDA (8.09 ± 4.53%) using both the sEMG and iEMG data from both able-bodied and amputee subjects (p < 0.001). In the between-day analysis, SSAE outperformed LDA (7.19 ± 9.55% vs. 22.25 ± 11.09%) using both sEMG and iEMG data from both able-bodied and amputee subjects. No significant difference in performance was observed for within-day and pairs of days with eight-fold validation when using iEMG and sEMG with SSAE, whereas sEMG outperformed iEMG (p < 0.001) in between-day analysis both with two-fold and seven-fold validation schemes. The results obtained in this study imply that SSAE can significantly improve the performance of pattern recognition-based myoelectric control scheme and has the strength to extract deep information hidden in the EMG data.

[1]  Khaled A. Harras,et al.  Multimodal Deep Learning Approach for Joint EEG-EMG Data Compression and Classification , 2017, 2017 IEEE Wireless Communications and Networking Conference (WCNC).

[2]  Weidong Geng,et al.  Gesture recognition by instantaneous surface EMG images , 2016, Scientific Reports.

[3]  Ernest Nlandu Kamavuako,et al.  A novel approach for classification of hand movements using surface EMG signals , 2017, 2017 IEEE International Symposium on Signal Processing and Information Technology (ISSPIT).

[4]  Quoc V. Le,et al.  On optimization methods for deep learning , 2011, ICML.

[5]  K. Hashtrudi-Zaad,et al.  A Method for Online Estimation of Human Arm Dynamics , 2006, 2006 International Conference of the IEEE Engineering in Medicine and Biology Society.

[6]  Ahmet Alkan,et al.  Identification of EMG signals using discriminant analysis and SVM classifier , 2012, Expert Syst. Appl..

[7]  Panagiotis K. Artemiadis,et al.  Proportional Myoelectric Control of Robots: Muscle Synergy Development Drives Performance Enhancement, Retainment, and Generalization , 2015, IEEE Transactions on Robotics.

[8]  Lauren H Smith,et al.  A comparison of the real-time controllability of pattern recognition to conventional myoelectric control for discrete and simultaneous movements , 2012, Journal of NeuroEngineering and Rehabilitation.

[9]  Huy Phan,et al.  Early prediction of future hand movements using sEMG data , 2017, 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[10]  José Manuel Fonseca,et al.  Feature Transformation Based on Stacked Sparse Autoencoders for Sleep Stage Classification , 2017, DoCEIS.

[11]  Keleş Selami,et al.  Classification Of Emg Signals Using Decision Tree Methods , 2012 .

[12]  O. Fukuda,et al.  FPGA implementation of a probabilistic neural network for a bioelectric human interface , 2004, The 2004 47th Midwest Symposium on Circuits and Systems, 2004. MWSCAS '04..

[13]  Levi J. Hargrove,et al.  Comparison of surface and intramuscular EMG pattern recognition for simultaneous wrist/hand motion classification , 2013, 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[14]  David J. Field,et al.  Sparse coding with an overcomplete basis set: A strategy employed by V1? , 1997, Vision Research.

[15]  Martin Fodslette Møller,et al.  A scaled conjugate gradient algorithm for fast supervised learning , 1993, Neural Networks.

[16]  Jun Xu,et al.  Fusing Heterogeneous Features From Stacked Sparse Autoencoder for Histopathological Image Analysis , 2016, IEEE Journal of Biomedical and Health Informatics.

[17]  Yang Bai,et al.  A novel electrocardiogram arrhythmia classification method based on stacked sparse auto-encoders and softmax regression , 2017, International Journal of Machine Learning and Cybernetics.

[18]  Yoshua Bengio,et al.  Greedy Layer-Wise Training of Deep Networks , 2006, NIPS.

[19]  N. Shoylev,et al.  Neural Networks for Online Classification of Hand and Finger Movements Using Surface EMG signals , 2006, 2006 8th Seminar on Neural Network Applications in Electrical Engineering.

[20]  P. Herberts,et al.  Hand prosthesis control via myoelectric patterns. , 1973, Acta orthopaedica Scandinavica.

[21]  Winnie Jensen,et al.  The effect of time on EMG classification of hand motions in able-bodied and transradial amputees. , 2018, Journal of electromyography and kinesiology : official journal of the International Society of Electrophysiological Kinesiology.

[22]  P. Geethanjali,et al.  Identification of motion from multi-channel EMG signals for control of prosthetic hand , 2011, Australasian Physical & Engineering Sciences in Medicine.

[23]  Nishio Takayuki,et al.  Deep Learning Tutorial , 2018 .

[24]  Yike Guo,et al.  Automatic Sleep Stage Scoring Using Time-Frequency Analysis and Stacked Sparse Autoencoders , 2015, Annals of Biomedical Engineering.

[25]  Ki Hwan Eom,et al.  EMG pattern classification using spectral estimation and neural network , 2007, SICE Annual Conference 2007.

[26]  Toshio Tsuji,et al.  An EMG controlled pointing device using a neural network , 1999, IEEE SMC'99 Conference Proceedings. 1999 IEEE International Conference on Systems, Man, and Cybernetics (Cat. No.99CH37028).

[27]  Toshio Tsuji,et al.  An EMG-controlled omnidirectional pointing device using a HMM-based neural network , 2003, Proceedings of the International Joint Conference on Neural Networks, 2003..

[28]  K. Englehart,et al.  Wrist torque estimation during simultaneous and continuously changing movements: surface vs. untargeted intramuscular EMG. , 2013, Journal of neurophysiology.

[29]  Martin Spüler,et al.  Extracting Muscle Synergy Patterns from EMG Data Using Autoencoders , 2016, ICANN.

[30]  Mads Jochumsen,et al.  The effect of arm position on classification of hand gestures with intramuscular EMG , 2018, Biomed. Signal Process. Control..

[31]  Quoc V. Le.  A Tutorial on Deep Learning Part 2: Autoencoders, Convolutional Neural Networks and Recurrent Neural Networks , 2015 .

[32]  Mohan S. Kankanhalli,et al.  Semi-Supervised Learning for Surface EMG-based Gesture Recognition , 2017, IJCAI.

[33]  Lei Wang,et al.  Deep learning-based classification of massive electrocardiography data , 2016, 2016 IEEE Advanced Information Management, Communicates, Electronic and Automation Control Conference (IMCEC).

[34]  Yu Hu,et al.  Surface EMG-Based Inter-Session Gesture Recognition Enhanced by Deep Domain Adaptation , 2017, Sensors.

[35]  Richard F. Weir,et al.  A Comparison of the Effects of Electrode Implantation and Targeting on Pattern Classification Accuracy for Prosthesis Control , 2008, IEEE Transactions on Biomedical Engineering.

[36]  Levi J. Hargrove,et al.  A Comparison of Surface and Intramuscular Myoelectric Signal Classification , 2007, IEEE Transactions on Biomedical Engineering.

[37]  R. B. Knapp,et al.  Real-time computer control using pattern recognition of the electromyogram , 1993, Proceedings of the 15th Annual International Conference of the IEEE Engineering in Medicine and Biology Societ.

[38]  Manfredo Atzori,et al.  Deep learning with convolutional neural networks: a resource for the control of robotic prosthetic hands via electromyography , 2016 .

[39]  E.M. El-Daydamony,et al.  A computerized system for SEMG signals analysis and classifieation , 2008, 2008 National Radio Science Conference.

[40]  Yoshua Bengio,et al.  Attention-Based Models for Speech Recognition , 2015, NIPS.

[41]  Fuzhen Zhuang,et al.  Transfer Learning with Multiple Sources via Consensus Regularized Autoencoders , 2014, ECML/PKDD.

[42]  Elisabeth André,et al.  EMG-based hand gesture recognition for realtime biosignal interfacing , 2008, IUI '08.

[43]  Ernest Nlandu Kamavuako,et al.  Effect of threshold values on the combination of EMG time domain features: Surface versus intramuscular EMG , 2018, Biomed. Signal Process. Control..

[44]  Yinghong Peng,et al.  EMG‐Based Estimation of Limb Movement Using Deep Learning With Recurrent Convolutional Neural Networks , 2018, Artificial organs.

[45]  Tianming Liu,et al.  Predicting eye fixations using convolutional neural networks , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[46]  Byunghan Lee,et al.  Deep learning in bioinformatics , 2016, Briefings Bioinform..

[47]  Purushothaman Geethanjali,et al.  Myoelectric control of prosthetic hands: state-of-the-art review , 2016, Medical devices.

[48]  Dario Farina,et al.  Online mapping of EMG signals into kinematics by autoencoding , 2018, Journal of NeuroEngineering and Rehabilitation.

[49]  Winnie Jensen,et al.  Relationship between grasping force and features of single-channel intramuscular EMG signals , 2009, Journal of Neuroscience Methods.

[50]  Todd A Kuiken,et al.  Target Achievement Control Test: evaluating real-time myoelectric pattern-recognition control of multifunctional upper-limb prostheses. , 2011, Journal of rehabilitation research and development.

[51]  Qin Lin,et al.  Classification of Epileptic EEG Signals with Stacked Sparse Autoencoder Based on Deep Learning , 2016, ICIC.

[52]  Daniel Graupe,et al.  Functional Separation of EMG Signals via ARMA Identification Methods for Prosthesis Control Purposes , 1975, IEEE Transactions on Systems, Man, and Cybernetics.

[53]  Robert Rosenberg.  The biofeedback pointer: EMG control of a two dimensional pointer , 1998, Digest of Papers. Second International Symposium on Wearable Computers (Cat. No.98EX215).

[54]  R.N. Scott,et al.  A new strategy for multifunction myoelectric control , 1993, IEEE Transactions on Biomedical Engineering.

[55]  Yoshua. Bengio,et al.  Learning Deep Architectures for AI , 2007, Found. Trends Mach. Learn..

[56]  Kongqiao Wang,et al.  Hand Gesture Recognition Research Based on Surface EMG Sensors and 2D-accelerometers , 2007, 2007 11th IEEE International Symposium on Wearable Computers.

[57]  Loredana Zollo,et al.  NLR, MLP, SVM, and LDA: a comparative analysis on EMG data from people with trans-radial amputation , 2017, Journal of NeuroEngineering and Rehabilitation.

[58]  Lei Wang,et al.  Automated atrial fibrillation detection based on deep learning network , 2016, 2016 IEEE International Conference on Information and Automation (ICIA).

[59]  Ernest Nlandu Kamavuako,et al.  Combined surface and intramuscular EMG for improved real-time myoelectric control performance , 2014, Biomed. Signal Process. Control..

[60]  Winnie Jensen,et al.  Surface Versus Untargeted Intramuscular EMG Based Classification of Simultaneous and Dynamically Changing Movements , 2013, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[61]  Mohan S. Kankanhalli,et al.  A multi-stream convolutional neural network for sEMG-based gesture recognition in muscle-computer interface , 2017, Pattern Recognit. Lett..

[62]  Pornchai Phukpattaranont,et al.  Feature reduction and selection for EMG signal classification , 2012, Expert Syst. Appl..

[63]  Manfredo Atzori,et al.  The Ninapro database: A resource for sEMG naturally controlled robotic hand prosthetics , 2015, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[64]  Seong-Whan Lee,et al.  Movement intention decoding based on deep learning for multiuser myoelectric interfaces , 2016, 2016 4th International Winter Conference on Brain-Computer Interface (BCI).

[65]  Adrian D. C. Chan,et al.  Myoelectric Control Development Toolbox , 2007 .