Synergistic Myoelectrical Activities of Forearm Muscles Improving Robust Recognition of Multi-Fingered Gestures

Currently, surface electromyography (sEMG) features of the forearm multi-tendon muscles are widely used in gesture recognition, however, there are few investigations on the inherent physiological mechanism of muscle synergies. We aimed to study whether the muscle synergies could be used for gesture recognition. Five healthy participants executed five gestures of daily life (pinch, fist, open hand, grip, and extension) and the sEMG activity was acquired from six forearm muscles. A non-negative matrix factorization (NMF) algorithm was employed to decompose the pre-treated six-channel sEMG data to obtain the muscle synergy matrixes, in which the weights of each muscle channel determined the feature set for hand gesture classification. The results showed that the synergistic features of forearm muscles could be successfully clustered in the feature space, which enabled hand gestures to be recognized with high efficiency. By augmenting the number of participants, the mean recognition rate remained at more than 96% and reflected high robustness. We showed that muscle synergies can be well applied to gesture recognition.

[1]  S. Sasaki,et al.  Direct and indirect cortico-motoneuronal pathways and control of hand/arm movements. , 2007, Physiology.

[2]  Dingguo Zhang,et al.  A discriminant bispectrum feature for surface electromyogram signal classification. , 2010, Medical engineering & physics.

[3]  Rodina Binti Ahmad,et al.  A systematic literature review on vision based gesture recognition techniques , 2018, Multimedia Tools and Applications.

[4]  Francesco Lacquaniti,et al.  Control of Fast-Reaching Movements by Muscle Synergy Combinations , 2006, The Journal of Neuroscience.

[5]  Gerry Leisman,et al.  Direction Modulation of Muscle Synergies in a Hand-Reaching Task , 2017, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[6]  Dick F Stegeman,et al.  Understanding the constraints of finger motor control. , 2018, Journal of electromyography and kinesiology : official journal of the International Society of Electrophysiological Kinesiology.

[7]  Agnes Huebner,et al.  Alteration of Surface EMG amplitude levels of five major trunk muscles by defined electrode location displacement. , 2015, Journal of electromyography and kinesiology : official journal of the International Society of Electrophysiological Kinesiology.

[8]  Francesco Lacquaniti,et al.  Superposition and modulation of muscle synergies for reaching in response to a change in target location. , 2011, Journal of neurophysiology.

[9]  Huosheng Hu,et al.  Support Vector Machine-Based Classification Scheme for Myoelectric Control Applied to Upper Limb , 2008, IEEE Transactions on Biomedical Engineering.

[10]  Gionata Salvietti,et al.  Replicating Human Hand Synergies Onto Robotic Hands: A Review on Software and Hardware Strategies , 2018, Front. Neurorobot..

[11]  A.D.C. Chan,et al.  Examining the adverse effects of limb position on pattern recognition based myoelectric control , 2010, 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology.

[12]  Kongqiao Wang,et al.  A Framework for Hand Gesture Recognition Based on Accelerometer and EMG Sensors , 2011, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[13]  Patrick van der Smagt,et al.  Surface EMG in advanced hand prosthetics , 2008, Biological Cybernetics.

[14]  Siti Anom Ahmad,et al.  Characterization of surface electromyography using time domain features for determining hand motion and stages of contraction , 2014, Australasian Physical & Engineering Sciences in Medicine.

[15]  E. Bizzi,et al.  Muscle synergy patterns as physiological markers of motor cortical damage , 2012, Proceedings of the National Academy of Sciences.

[16]  M. Schwartz,et al.  Electromyography Data Processing Impacts Muscle Synergies during Gait for Unimpaired Children and Children with Cerebral Palsy , 2017, Front. Comput. Neurosci..

[17]  Todd A. Kuiken,et al.  The Effects of Electrode Size and Orientation on the Sensitivity of Myoelectric Pattern Recognition Systems to Electrode Shift , 2011, IEEE Transactions on Biomedical Engineering.

[18]  M. Goldfarb,et al.  A Method for the Control of Multigrasp Myoelectric Prosthetic Hands , 2012, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[19]  Yao Zhang,et al.  Non-Uniform Sample Assignment in Training Set Improving Recognition of Hand Gestures Dominated with Similar Muscle Activities , 2018, Front. Neurorobot..

[20]  Dario Farina,et al.  Intuitive, Online, Simultaneous, and Proportional Myoelectric Control Over Two Degrees-of-Freedom in Upper Limb Amputees , 2014, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[21]  Robert D. Lipschutz,et al.  Targeted muscle reinnervation for real-time myoelectric control of multifunction artificial arms. , 2009, JAMA.

[22]  Ning Jiang,et al.  Extracting Simultaneous and Proportional Neural Control Information for Multiple-DOF Prostheses From the Surface Electromyographic Signal , 2009, IEEE Transactions on Biomedical Engineering.

[23]  Yuan-Ting Zhang,et al.  A novel channel selection method for multiple motion classification using high-density electromyography , 2014, BioMedical Engineering OnLine.

[24]  W. Rymer,et al.  Alterations in upper limb muscle synergy structure in chronic stroke survivors. , 2013, Journal of neurophysiology.

[25]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[26]  Huosheng Hu,et al.  PCA and deep learning based myoelectric grasping control of a prosthetic hand , 2018, BioMedical Engineering OnLine.

[27]  Guanglin Li,et al.  Principal Components Analysis Preprocessing for Improved Classification Accuracies in Pattern-Recognition-Based Myoelectric Control , 2009, IEEE Transactions on Biomedical Engineering.

[28]  Alessandra Pedrocchi,et al.  Robustness and Reliability of Synergy-Based Myocontrol of a Multiple Degree of Freedom Robotic Arm , 2016, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[29]  Zhizhong Wang,et al.  The application of mutual information-based feature selection and fuzzy LS-SVM-based classifier in motion classification , 2008, Comput. Methods Programs Biomed..

[30]  Adel Al-Jumaily,et al.  Evolutionary fuzzy discriminant analysis feature projection technique in myoelectric control , 2009, Pattern Recognit. Lett..

[31]  Andrea d'Avella,et al.  Modularity in the motor system: decomposition of muscle patterns as combinations of time-varying synergies , 2001, NIPS.

[32]  Elisabeth André,et al.  EMG-based hand gesture recognition for realtime biosignal interfacing , 2008, IUI '08.

[33]  Huosheng Hu,et al.  Myoelectric control systems - A survey , 2007, Biomed. Signal Process. Control..

[34]  Aiping Liu,et al.  A Novel Phonology- and Radical-Coded Chinese Sign Language Recognition Framework Using Accelerometer and Surface Electromyography Sensors , 2015, Sensors.

[35]  Xu Zhang,et al.  Muscle synergy analysis in children with cerebral palsy , 2015, Journal of neural engineering.

[36]  M. Latash,et al.  Effects of a novel method of acute tryptophan depletion on plasma tryptophan and cognitive performance in healthy volunteers , 2004, Psychopharmacology.

[37]  Sarajane Marques Peres,et al.  Classification of electromyography signals using relevance vector machines and fractal dimension , 2015, Neural Computing and Applications.

[38]  Xun Chen,et al.  Pattern recognition of number gestures based on a wireless surface EMG system , 2013, Biomed. Signal Process. Control..

[39]  Antonio Frisoli,et al.  An EMG-Controlled Robotic Hand Exoskeleton for Bilateral Rehabilitation , 2015, IEEE Transactions on Haptics.

[40]  Xiang Cao,et al.  Chinese Sign Language Recognition Based on an Optimized Tree-Structure Framework , 2017, IEEE Journal of Biomedical and Health Informatics.

[41]  J. Kalaska,et al.  Muscle synergies during locomotion in the cat: a model for motor cortex control , 2008, The Journal of physiology.

[42]  Emilio Bizzi,et al.  Combinations of muscle synergies in the construction of a natural motor behavior , 2003, Nature Neuroscience.

[43]  Bin Xie,et al.  RGB-D static gesture recognition based on convolutional neural network , 2018 .

[44]  Ankit Chaudhary.  Robust Hand Gesture Recognition for Robotic Hand Control , 2017 .

[45]  Mehran Jahed,et al.  Real-time intelligent pattern recognition algorithm for surface EMG signals , 2007, Biomedical engineering online.

[46]  Richard R Neptune,et al.  Merging of healthy motor modules predicts reduced locomotor performance and muscle coordination complexity post-stroke. , 2010, Journal of neurophysiology.

[47]  Kevin B. Englehart,et al.  A robust, real-time control scheme for multifunction myoelectric control , 2003, IEEE Transactions on Biomedical Engineering.

[48]  Fumitoshi Matsuno,et al.  Hand and Wrist Movement Control of Myoelectric Prosthesis Based on Synergy , 2015, IEEE Transactions on Human-Machine Systems.

[49]  Ankit Chaudhary,et al.  Controlling a remotely located robot using hand gestures in real time: A DSP implementation , 2016, 2016 5th International Conference on Wireless Networks and Embedded Systems (WECON).

[50]  Dario Farina,et al.  The Extraction of Neural Information from the Surface EMG for the Control of Upper-Limb Prostheses: Emerging Avenues and Challenges , 2014, IEEE Transactions on Neural Systems and Rehabilitation Engineering.