Locomo-Net: A Low -Complex Deep Learning Framework for sEMG-Based Hand Movement Recognition for Prosthetic Control

Background: The enhancement in the performance of the myoelectric pattern recognition techniques based on deep learning algorithm possess computationally expensive and exhibit extensive memory behavior. Therefore, in this paper we report a deep learning framework named ‘Low-Complex Movement recognition-Net’ (LoCoMo-Net) built with convolution neural network (CNN) for recognition of wrist and finger flexion movements; grasping and functional movements; and force pattern from single channel surface electromyography (sEMG) recording. The network consists of a two-stage pipeline: 1) input data compression; 2) data-driven weight sharing. Methods: The proposed framework was validated on two different datasets- our own dataset (DS1) and publicly available NinaPro dataset (DS2) for 16 movements and 50 movements respectively. Further, we have prototyped the proposed LoCoMo-Net on Virtex-7 Xilinx field-programmable gate array (FPGA) platform and validated for 15 movements from DS1 to demonstrate its feasibility for real-time execution. Results: The effectiveness of the proposed LoCoMo-Net was verified by a comparative analysis against the benchmarked models using the same datasets wherein our proposed model outperformed Twin- Support Vector Machine (SVM) and existing CNN based model by an average classification accuracy of 8.5 % and 16.0 % respectively. In addition, hardware complexity analysis is done to reveal the advantages of the two-stage pipeline where approximately 27 %, 49 %, 50 %, 23 %, and 43 % savings achieved in lookup tables (LUT’s), registers, memory, power consumption and computational time respectively. Conclusion: The clinical significance of such sEMG based accurate and low-complex movement recognition system can be favorable for the potential improvement in quality of life of an amputated persons.

[1]  Dario Farina,et al.  EMG Biofeedback for online predictive control of grasping force in a myoelectric prosthesis , 2015, Journal of NeuroEngineering and Rehabilitation.

[2]  Seong-Whan Lee,et al.  Movement intention decoding based on deep learning for multiuser myoelectric interfaces , 2016, 2016 4th International Winter Conference on Brain-Computer Interface (BCI).

[3]  Koushik Maharatna,et al.  Rehab-Net: Deep Learning Framework for Arm Movement Classification Using Wearable Sensors for Stroke Rehabilitation , 2019, IEEE Transactions on Biomedical Engineering.

[4]  Anna T. Winslow,et al.  Evaluation of EMG pattern recognition for upper limb prosthesis control: a case study in comparison with direct myoelectric control , 2018, Journal of NeuroEngineering and Rehabilitation.

[5]  Clément Gosselin,et al.  Transfer learning for sEMG hand gestures recognition using convolutional neural networks , 2017, 2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[6]  Gavin C. Cawley,et al.  On Over-fitting in Model Selection and Subsequent Selection Bias in Performance Evaluation , 2010, J. Mach. Learn. Res..

[7]  Øyvind Stavdahl,et al.  A multi-modal approach for hand motion classification using surface EMG and accelerometers , 2011, 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[8]  Sam L. Phillips,et al.  EMG Pattern Recognition Control of the DEKA Arm: Impact on User Ratings of Satisfaction and Usability , 2018, IEEE Journal of Translational Engineering in Health and Medicine.

[9]  Mario Konijnenburg,et al.  BiometricNet: Deep Learning based Biometric Identification using Wrist-Worn PPG , 2018, 2018 IEEE International Symposium on Circuits and Systems (ISCAS).

[10]  Yongkang Wong,et al.  A novel attention-based hybrid CNN-RNN architecture for sEMG-based gesture recognition , 2018, PloS one.

[11]  Kevin B. Englehart,et al.  A robust, real-time control scheme for multifunction myoelectric control , 2003, IEEE Transactions on Biomedical Engineering.

[12]  Rene de Jesus Romero-Troncoso,et al.  Empirical Mode Decomposition and Neural Networks on FPGA for Fault Diagnosis in Induction Motors , 2014, TheScientificWorldJournal.

[13]  Dinesh Kant Kumar,et al.  Improved sEMG signal classification using the Twin SVM , 2016, 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[14]  Arvind Gautam,et al.  MyoNet: A Transfer-Learning-Based LRCN for Lower Limb Movement Recognition and Knee Joint Angle Prediction for Remote Monitoring of Rehabilitation Progress From sEMG , 2020, IEEE Journal of Translational Engineering in Health and Medicine.

[15]  Angkoon Phinyomark,et al.  An Investigation of Temporally Inspired Time Domain Features for Electromyographic Pattern Recognition , 2018, 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[16]  Beth Jelfs,et al.  Self-Recalibrating Surface EMG Pattern Recognition for Neuroprosthesis Control Based on Convolutional Neural Network , 2017, Front. Neurosci..

[17]  J. Sola,et al.  Importance of input data normalization for the application of neural networks to complex industrial problems , 1997 .

[18]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[19]  Dinesh K Kumar,et al.  Selection of suitable hand gestures for reliable myoelectric human computer interface , 2015, Biomedical engineering online.

[20]  Yoshua Bengio,et al.  Convolutional networks for images, speech, and time series , 1998 .

[21]  Song Han,et al.  Deep Compression: Compressing Deep Neural Network with Pruning, Trained Quantization and Huffman Coding , 2015, ICLR.

[22]  G. Naik,et al.  Transradial Amputee Gesture Classification Using an Optimal Number of sEMG Sensors: An Approach Using ICA Clustering , 2016, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[23]  Dong Yu,et al.  Investigation of full-sequence training of deep belief networks for speech recognition , 2010, INTERSPEECH.

[24]  Blair A. Lock,et al.  Determining the Optimal Window Length for Pattern Recognition-Based Myoelectric Control: Balancing the Competing Effects of Classification Error and Controller Delay , 2011, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[25]  Richard F. Weir,et al.  A Comparison of the Effects of Electrode Implantation and Targeting on Pattern Classification Accuracy for Prosthesis Control , 2008, IEEE Transactions on Biomedical Engineering.

[26]  Ying-Yi Hong,et al.  FPGA Implementation for Real-Time Empirical Mode Decomposition , 2012, IEEE Transactions on Instrumentation and Measurement.

[27]  Christian Antfolk,et al.  Automatic discovery of resource-restricted Convolutional Neural Network topologies for myoelectric pattern recognition , 2020, Comput. Biol. Medicine.

[28]  B Hudgins,et al.  Myoelectric signal processing for control of powered limb prostheses. , 2006, Journal of electromyography and kinesiology : official journal of the International Society of Electrophysiological Kinesiology.

[29]  Nur Azah Hamzaid,et al.  Sensory Systems in Micro-Processor Controlled Prosthetic Leg: A Review , 2020, IEEE Sensors Journal.

[30]  Guido Bugmann,et al.  Classification of Finger Movements for the Dexterous Hand Prosthesis Control With Surface Electromyography , 2013, IEEE Journal of Biomedical and Health Informatics.

[31]  Shin-Ki Kim,et al.  A Supervised Feature-Projection-Based Real-Time EMG Pattern Recognition for Multifunction Myoelectric Hand Control , 2007, IEEE/ASME Transactions on Mechatronics.

[32]  Bo Yu,et al.  Convolutional Neural Networks for human activity recognition using mobile sensors , 2014, 6th International Conference on Mobile Computing, Applications and Services.

[33]  Ganesh R. Naik,et al.  Twin SVM for Gesture Classification Using the Surface Electromyogram , 2010, IEEE Transactions on Information Technology in Biomedicine.

[34]  Philip R. Troyk,et al.  Implantable Myoelectric Sensors (IMESs) for Intramuscular Electromyogram Recording , 2009, IEEE Transactions on Biomedical Engineering.

[35]  Po-Lei Lee,et al.  Hardware Implementation of EMD Using DSP and FPGA for Online Signal Processing , 2011, IEEE Transactions on Industrial Electronics.

[36]  Gilles Louppe,et al.  Independent consultant , 2013 .

[37]  Manfredo Atzori,et al.  Deep Learning with Convolutional Neural Networks Applied to Electromyography Data: A Resource for the Classification of Movements for Prosthetic Hands , 2016, Front. Neurorobot..

[38]  Ganesh R. Naik,et al.  CNN based approach for activity recognition using a wrist-worn accelerometer , 2017, 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[39]  Dinesh Kant Kumar,et al.  Decoding subtle forearm flexions using fractal features of surface electromyogram from single and multiple sensors , 2010, Journal of NeuroEngineering and Rehabilitation.

[40]  Erik Scheme,et al.  EMG Pattern Recognition in the Era of Big Data and Deep Learning , 2018, Big Data Cogn. Comput..

[41]  Faruk Kazi,et al.  Hand Motion Recognition from Single Channel Surface EMG Using Wavelet & Artificial Neural Network☆ , 2015 .

[42]  Sethu Vijayakumar,et al.  Improved prosthetic hand control with concurrent use of myoelectric and inertial measurements , 2017, Journal of NeuroEngineering and Rehabilitation.