Inferring Static Hand Poses from a Low-Cost Non-Intrusive sEMG Sensor

Every year, a significant number of people lose a body part in an accident, through sickness or in high-risk manual jobs. Several studies and research works have tried to reduce the constraints and risks in their lives through the use of technology. This work proposes a learning-based approach that performs gesture recognition using a surface electromyography-based device, the Myo Armband released by Thalmic Labs, which is a commercial device and has eight non-intrusive low-cost sensors. With 35 able-bodied subjects, and using the Myo Armband device, which is able to record data at about 200 MHz, we collected a dataset that includes six dissimilar hand gestures. We used a gated recurrent unit network to train a system that, as input, takes raw signals extracted from the surface electromyography sensors. The proposed approach obtained a 99.90% training accuracy and 99.75% validation accuracy. We also evaluated the proposed system on a test set (new subjects) obtaining an accuracy of 77.85%. In addition, we showed the test prediction results for each gesture separately and analyzed which gestures for the Myo armband with our suggested network can be difficult to distinguish accurately. Moreover, we studied for first time the gated recurrent unit network capability in gesture recognition approaches. Finally, we integrated our method in a system that is able to classify live hand gestures.

[1]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[2]  Mohammad Hassan Moradi,et al.  Evaluation of the forearm EMG signal features for the control of a prosthetic hand. , 2003, Physiological measurement.

[3]  Desney S. Tan,et al.  Demonstrating the feasibility of using forearm electromyography for muscle-computer interfaces , 2008, CHI.

[4]  Zhijun Yang,et al.  Development of an EMG-Controlled Mobile Robot , 2018, Robotics.

[5]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[6]  Joshua Burns,et al.  Prospective study of muscle cramps in Charcot‐Marie‐Tooth disease , 2015, Muscle & nerve.

[7]  I. Kingma,et al.  Towards optimal multi-channel EMG electrode configurations in muscle force estimation: a high density EMG study. , 2005, Journal of electromyography and kinesiology : official journal of the International Society of Electrophysiological Kinesiology.

[8]  Ping Zhou,et al.  High-Density Myoelectric Pattern Recognition Toward Improved Stroke Rehabilitation , 2012, IEEE Transactions on Biomedical Engineering.

[9]  Gea Drost,et al.  A high-density multichannel surface electromyography system for the characterization of single motor units , 2002 .

[10]  Juan Antonio Holgado Terriza,et al.  Hand gesture recognition based on sEMG signals using Support Vector Machines , 2016, 2016 IEEE 6th International Conference on Consumer Electronics - Berlin (ICCE-Berlin).

[11]  Manfredo Atzori,et al.  Movement Error Rate for Evaluation of Machine Learning Methods for sEMG-Based Hand Movement Classification , 2014, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[12]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[13]  Zhi-Hong Mao,et al.  EXGbuds: Universal Wearable Assistive Device for Disabled People to Interact with the Environment Seamlessly , 2018, HRI.

[14]  D. Weber,et al.  The role of assistive robotics in the lives of persons with disability. , 2010, American journal of physical medicine & rehabilitation.

[15]  Brendan Z. Allison,et al.  Comparison of Dry and Gel Based Electrodes for P300 Brain–Computer Interfaces , 2012, Front. Neurosci..

[16]  Vicente Julián,et al.  PHAROS—PHysical Assistant RObot System , 2018, Sensors.

[17]  Gea Drost,et al.  Clinical applications of high-density surface EMG: a systematic review. , 2006, Journal of electromyography and kinesiology : official journal of the International Society of Electrophysiological Kinesiology.

[18]  T. Kuiken,et al.  The effect of subcutaneous fat on myoelectric signal amplitude and cross-talk , 2003, Prosthetics and orthotics international.

[19]  Weidong Geng,et al.  Gesture recognition by instantaneous surface EMG images , 2016, Scientific Reports.

[20]  Sebastian Ruder,et al.  An overview of gradient descent optimization algorithms , 2016, Vestnik komp'iuternykh i informatsionnykh tekhnologii.

[21]  Chun-Yi Su,et al.  Teleoperated robot writing using EMG signals , 2015, 2015 IEEE International Conference on Information and Automation.

[22]  Dario Farina,et al.  Influence of anatomical, physical, and detection-system parameters on surface EMG , 2002, Biological Cybernetics.

[23]  Dario Farina,et al.  Self-Correcting Pattern Recognition System of Surface EMG Signals for Upper Limb Prosthesis Control , 2014, IEEE Transactions on Biomedical Engineering.

[24]  Manfredo Atzori,et al.  Comparison of six electromyography acquisition setups on hand movement classification tasks , 2017, PloS one.

[25]  Jean Scholtz,et al.  Human-Robot Interactions: Creating Synergistic Cyber Forces , 2002 .

[26]  Clément Gosselin,et al.  A convolutional neural network for robotic arm guidance using sEMG based frequency-features , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[27]  Wenpeng Yin,et al.  Comparative Study of CNN and RNN for Natural Language Processing , 2017, ArXiv.

[28]  F Stegeman Dick,et al.  High-density Surface EMG: Techniques and Applications at a Motor Unit Level , 2012 .

[29]  Othman Omran Khalifa,et al.  Advances in Electromyogram Signal Classification to Improve the Quality of Life for the Disabled and Aged People , 2010 .

[30]  Yoshua Bengio,et al.  Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling , 2014, ArXiv.

[31]  Monica Rojas-Martínez,et al.  High-density surface EMG maps from upper-arm and forearm muscles , 2012, Journal of NeuroEngineering and Rehabilitation.

[32]  Manfredo Atzori,et al.  Electromyography data for non-invasive naturally-controlled robotic hand prostheses , 2014, Scientific Data.

[33]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[34]  Z. Li,et al.  Teleoperation control of Baxter robot using body motion tracking , 2014, 2014 International Conference on Multisensor Fusion and Information Integration for Intelligent Systems (MFI).

[35]  Lijun Zhao,et al.  Development of a hybrid motion capture method using MYO armband with application to teleoperation , 2016, 2016 IEEE International Conference on Mechatronics and Automation.

[36]  Tanja Schultz,et al.  Advancing Muscle-Computer Interfaces with High-Density Electromyography , 2015, CHI.

[37]  Mohan S. Kankanhalli,et al.  Semi-Supervised Learning for Surface EMG-based Gesture Recognition , 2017, IJCAI.

[38]  Huosheng Hu,et al.  Myoelectric control systems - A survey , 2007, Biomed. Signal Process. Control..

[39]  S. Krishnan,et al.  Real-Time Classification of Forearm Electromyographic Signals Corresponding to User-Selected Intentional Movements for Multifunction Prosthesis Control , 2007, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[40]  Albert M. Cook,et al.  Essentials of Assistive Technologies , 2012 .

[41]  Lukás Burget,et al.  Recurrent neural network based language model , 2010, INTERSPEECH.

[42]  Clément Gosselin,et al.  Deep Learning for Electromyographic Hand Gesture Signal Classification by Leveraging Transfer Learning , 2018, ArXiv.

[43]  J. F. Alonso,et al.  Identification of isometric contractions based on High Density EMG maps. , 2013, Journal of electromyography and kinesiology : official journal of the International Society of Electrophysiological Kinesiology.

[44]  Clément Gosselin,et al.  Transfer learning for sEMG hand gestures recognition using convolutional neural networks , 2017, 2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[45]  Manfredo Atzori,et al.  Deep Learning with Convolutional Neural Networks Applied to Electromyography Data: A Resource for the Classification of Movements for Prosthetic Hands , 2016, Front. Neurorobot..

[46]  Audrey Watters,et al.  The Three Laws of Robotics , 2015 .