Recurrent Neural Network for electromyographic gesture recognition in transhumeral amputees

Abstract Gesture recognition is a key aspect of myoelectric control of upper-limb prostheses and is rather complex to achieve for transhumeral amputees. The prosthesis control of upper arm movements must rely only on the arm muscles, which were not involved in these gestures before the amputation. For decades, machine learning has been used in research for upper-limb gesture recognition. However, reported classification accuracies for transhumeral amputees have not improved significantly since the 1990s. Latest developments in deep learning suggest it can outperform classical machine learning both in accuracy and processing time. This study aims to determine if a deep learning approach, specifically a Recurrent Neural Network (RNN), could better recognize the movement intents in transhumeral amputees. To do so, the classification accuracy and the processing time of the RNN were measured and compared to two state-of-the-art approaches that use a linear discriminant analysis (LDA) and a multilayer perceptron (MLP) respectively. All three approaches were used to classify the signals of five transhumeral amputees between 6 upper-limb gestures. For subjects 1, 3 and 5, the classification accuracy was significantly higher (p = 0.0002) for the RNN (79.7%) compared to the LDA (67,1%) and the MLP (74,1%). Additionally, the RNN had a much smaller processing time, under 7 ms, compared to 385 ms and 377 ms for the LDA and the MLP respectively. Consequently, the RNN is better suited for a real-time prosthesis control that occurs between 100–250 ms. Results suggest deep learning as a viable solution for gesture recognition in transhumeral amputees.

[1]  Seong-Whan Lee,et al.  Movement intention decoding based on deep learning for multiuser myoelectric interfaces , 2016, 2016 4th International Winter Conference on Brain-Computer Interface (BCI).

[2]  Elaine Biddiss,et al.  Consumer design priorities for upper limb prosthetics , 2007, Disability and rehabilitation. Assistive technology.

[3]  Erik Scheme,et al.  Electromyogram pattern recognition for control of powered upper-limb prostheses: state of the art and challenges for clinical use. , 2011, Journal of rehabilitation research and development.

[4]  Dianhui Wang,et al.  Randomness in neural networks: an overview , 2017, WIREs Data Mining Knowl. Discov..

[5]  A. M. Hager,et al.  Effect of clinical parameters on the control of myoelectric robotic prosthetic hands. , 2016, Journal of rehabilitation research and development.

[6]  Huy Phan,et al.  Recurrent Neural Network Based Early Prediction of Future Hand Movements , 2018, 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[7]  B Hudgins,et al.  Myoelectric signal processing for control of powered limb prostheses. , 2006, Journal of electromyography and kinesiology : official journal of the International Society of Electrophysiological Kinesiology.

[8]  Jimeng Sun,et al.  Using recurrent neural network models for early detection of heart failure onset , 2016, J. Am. Medical Informatics Assoc..

[9]  Yu Hu,et al.  Surface EMG-Based Inter-Session Gesture Recognition Enhanced by Deep Domain Adaptation , 2017, Sensors.

[10]  P. Dijkstra,et al.  Phantom pain and phantom sensations in upper limb amputees: an epidemiological study , 2000, Pain.

[11]  Erik Scheme,et al.  Regression convolutional neural network for improved simultaneous EMG control , 2019, Journal of neural engineering.

[12]  C. Nicol,et al.  Classification of Phantom Finger, Hand, Wrist, and Elbow Voluntary Gestures in Transhumeral Amputees With sEMG , 2017, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[13]  C. Nicol,et al.  Characteristics of phantom upper limb mobility encourage phantom-mobility-based prosthesis control , 2018, Scientific Reports.

[14]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[15]  Erik J. Scheme,et al.  Effects of Confidence-Based Rejection on Usability and Error in Pattern Recognition-Based Myoelectric Control , 2019, IEEE Journal of Biomedical and Health Informatics.

[16]  S. Bandinelli,et al.  Motor reorganization after upper limb amputation in man. A study with focal magnetic stimulation. , 1991, Brain : a journal of neurology.

[17]  Filipe R. Cordeiro,et al.  A convolutional neural network with feature fusion for real-time hand posture recognition , 2018, Appl. Soft Comput..

[18]  Dario Farina,et al.  Multiday EMG-Based Classification of Hand Motions with Deep Learning Techniques , 2018, Sensors.

[19]  Nathanaël Jarrassé,et al.  Phantom-Mobility-Based Prosthesis Control in Transhumeral Amputees Without Surgical Reinnervation: A Preliminary Study , 2018, Front. Bioeng. Biotechnol..

[20]  Clément Gosselin,et al.  Deep Learning for Electromyographic Hand Gesture Signal Classification Using Transfer Learning , 2018, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[21]  Huosheng Hu,et al.  Myoelectric control systems - A survey , 2007, Biomed. Signal Process. Control..

[22]  Erik Scheme,et al.  Real-time, simultaneous myoelectric control using a convolutional neural network , 2018, PloS one.

[23]  Blair A. Lock,et al.  Determining the Optimal Window Length for Pattern Recognition-Based Myoelectric Control: Balancing the Competing Effects of Classification Error and Controller Delay , 2011, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[24]  Angkoon Phinyomark,et al.  EMG feature evaluation for improving myoelectric pattern recognition robustness , 2013, Expert Syst. Appl..

[25]  A.D.C. Chan,et al.  Examining the adverse effects of limb position on pattern recognition based myoelectric control , 2010, 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology.

[26]  Manfredo Atzori,et al.  Deep Learning with Convolutional Neural Networks Applied to Electromyography Data: A Resource for the Classification of Movements for Prosthetic Hands , 2016, Front. Neurorobot..

[27]  R.F. Weir,et al.  The Optimal Controller Delay for Myoelectric Prostheses , 2007, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[28]  Erik Scheme,et al.  EMG Pattern Recognition in the Era of Big Data and Deep Learning , 2018, Big Data Cogn. Comput..

[29]  Pascal Vincent,et al.  Representation Learning: A Review and New Perspectives , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  K. Reilly,et al.  Persistent hand motor commands in the amputees' brain. , 2006, Brain : a journal of neurology.

[31]  Dario Farina,et al.  Extracting Signals Robust to Electrode Number and Shift for Online Simultaneous and Proportional Myoelectric Control by Factorization Algorithms , 2014, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[32]  Clément Gosselin,et al.  A convolutional neural network for robotic arm guidance using sEMG based frequency-features , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[33]  Gerald Penn,et al.  Convolutional Neural Networks for Speech Recognition , 2014, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[34]  Blair A. Lock,et al.  A Real-Time Pattern Recognition Based Myoelectric Control Usability Study Implemented in a Virtual Environment , 2007, 2007 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[35]  Rita Amado Laezza,et al.  Deep neural networks for myoelectric pattern recognition - An implementation for multifunctional control , 2018 .

[36]  Joris M. Lambrecht,et al.  Electromyogram-based neural network control of transhumeral prostheses. , 2011, Journal of rehabilitation research and development.

[37]  Razvan Pascanu,et al.  On the difficulty of training recurrent neural networks , 2012, ICML.

[38]  Sofiane Achiche,et al.  Classification of Upper limb phantom movements in transhumeral amputees using electromyographic and kinematic features , 2018, Eng. Appl. Artif. Intell..

[39]  Madhura Purnaprajna,et al.  Recurrent Neural Networks: An Embedded Computing Perspective , 2019, IEEE Access.

[40]  T. Hortobágyi,et al.  Teager–Kaiser energy operator signal conditioning improves EMG onset detection , 2010, European Journal of Applied Physiology.

[41]  Pornchai Phukpattaranont,et al.  Feature reduction and selection for EMG signal classification , 2012, Expert Syst. Appl..

[42]  Kevin B. Englehart,et al.  A wavelet-based continuous classification scheme for multifunction myoelectric control , 2001, IEEE Transactions on Biomedical Engineering.

[43]  R.N. Scott,et al.  A new strategy for multifunction myoelectric control , 1993, IEEE Transactions on Biomedical Engineering.

[44]  Ganesh R. Naik,et al.  Nonlinear multiscale Maximal Lyapunov Exponent for accurate myoelectric signal classification , 2015, Appl. Soft Comput..

[45]  S. Achiche,et al.  Intra- and Intersession Reliability of Surface Electromyography on Muscles Actuating the Forearm During Maximum Voluntary Contractions. , 2016, Journal of applied biomechanics.

[46]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[47]  K. Reilly,et al.  The moving phantom: Motor execution or motor imagery? , 2012, Cortex.

[48]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[49]  Clément Gosselin,et al.  Real-Time Hand Motion Recognition Using sEMG Patterns Classification , 2018, 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[50]  Radu-Emil Precup,et al.  Recurrent dynamic neural network model for myoelectric-based control of a prosthetic hand , 2016, 2016 Annual IEEE Systems Conference (SysCon).

[51]  Ernest Nlandu Kamavuako,et al.  Phantom movements from physiologically inappropriate muscles: A case study with a high transhumeral amputee , 2015, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).