sEMG-Based Continuous Estimation of Finger Kinematics via Large-Scale Temporal Convolutional Network

Since continuous motion control can provide a more natural, fast and accurate man–machine interface than that of discrete motion control, it has been widely used in human–robot cooperation (HRC). Among various biological signals, the surface electromyogram (sEMG)—the signal of actions potential superimposed on the surface of the skin containing the temporal and spatial information—is one of the best signals with which to extract human motion intentions. However, most of the current sEMG control methods can only perform discrete motion estimation, and thus fail to meet the requirements of continuous motion estimation. In this paper, we propose a novel method that applies a temporal convolutional network (TCN) to sEMG-based continuous estimation. After analyzing the relationship between the convolutional kernel’s size and the lengths of atomic segments (defined in this paper), we propose a large-scale temporal convolutional network (LS-TCN) to overcome the TCN’s problem: that it is difficult to fully extract the sEMG’s temporal features. When applying our proposed LS-TCN with a convolutional kernel size of 1 × 31 to continuously estimate the angles of the 10 main joints of fingers (based on the public dataset Ninapro), it can achieve a precision rate of 71.6%. Compared with TCN (kernel size of 1 × 3), LS-TCN (kernel size of 1 × 31) improves the precision rate by 6.6%.

[1]  Tamara Grujic Supuk,et al.  Design, Development and Testing of a Low-Cost sEMG System and Its Use in Recording Muscle Activity in Human Gait , 2014, Sensors.

[2]  Xiangyu Zhang,et al.  Large Kernel Matters — Improve Semantic Segmentation by Global Convolutional Network , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Marek Kaminski,et al.  Natural interface for robotic arm controlling based on inertial motion capture , 2016, 2016 9th International Conference on Human System Interactions (HSI).

[4]  H. Cotta.  [On the physiology of joints]. , 1966, Langenbecks Archiv fur Chirurgie.

[5]  S Micera,et al.  Control of Hand Prostheses Using Peripheral Information , 2010, IEEE Reviews in Biomedical Engineering.

[6]  Othman Omran Khalifa,et al.  EMG signal classification for human computer interaction: a review , 2009 .

[7]  Li-Cheng Chen,et al.  Investigating an innovative service with hospitality robots , 2017 .

[8]  Nitish V. Thakor,et al.  Continuous decoding of finger position from surface EMG signals for the control of powered prostheses , 2008, 2008 30th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[9]  Manfredo Atzori,et al.  The Ninapro database: A resource for sEMG naturally controlled robotic hand prosthetics , 2015, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[10]  Clément Gosselin,et al.  Deep Learning for Electromyographic Hand Gesture Signal Classification Using Transfer Learning , 2018, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[11]  Rodolfo E. Haber,et al.  The fuzzy Kalman filter: Improving its implementation by reformulating uncertainty representation , 2021, Fuzzy Sets Syst..

[12]  Yinghong Peng,et al.  EMG‐Based Estimation of Limb Movement Using Deep Learning With Recurrent Convolutional Neural Networks , 2018, Artificial organs.

[13]  Luca Benini,et al.  Robust Real-Time Embedded EMG Recognition Framework Using Temporal Convolutional Networks on a Multicore IoT Processor , 2019, IEEE Transactions on Biomedical Circuits and Systems.

[14]  Gerardo Beruvides,et al.  A Simple Multi-Objective Optimization Based on the Cross-Entropy Method , 2017, IEEE Access.

[15]  Nikhil Ketkar,et al.  Introduction to PyTorch , 2021, Deep Learning with Python.

[16]  Gerhard Nahler,et al.  Pearson Correlation Coefficient , 2020, Definitions.

[17]  Rodolfo E. Haber,et al.  A neural network-based model for the prediction of cutting force in milling process. A progress study on a real case , 2000, Proceedings of the 2000 IEEE International Symposium on Intelligent Control. Held jointly with the 8th IEEE Mediterranean Conference on Control and Automation (Cat. No.00CH37147).

[18]  Dario Farina,et al.  Predicting wrist kinematics from motor unit discharge timings for the control of active prostheses , 2019, Journal of NeuroEngineering and Rehabilitation.

[19]  Ning Jiang,et al.  Extracting Simultaneous and Proportional Neural Control Information for Multiple-DOF Prostheses From the Surface Electromyographic Signal , 2009, IEEE Transactions on Biomedical Engineering.

[20]  D. Farina,et al.  Simultaneous and Proportional Estimation of Hand Kinematics From EMG During Mirrored Movements at Multiple Degrees-of-Freedom , 2012, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[21]  Ravinder Agarwal,et al.  SEMG signal analysis at acupressure points for elbow movement. , 2011, Journal of electromyography and kinesiology : official journal of the International Society of Electrophysiological Kinesiology.

[22]  Aurel Gontean,et al.  Controlling a robotic arm in the 3D space with stereo vision , 2013, 2013 21st Telecommunications Forum Telfor (TELFOR).

[23]  The Physiology of the Joints. Vol. 1, Upper Limb , 1971 .

[24]  Dario Farina,et al.  Myoelectric Control of Artificial Limbs¿Is There a Need to Change Focus? [In the Spotlight] , 2012, IEEE Signal Process. Mag..

[25]  Angkoon Phinyomark,et al.  EMG feature evaluation for improving myoelectric pattern recognition robustness , 2013, Expert Syst. Appl..

[26]  Alexandra-Bianca Borlea,et al.  Evolving Fuzzy Models for Prosthetic Hand Myoelectric-Based Control , 2020, IEEE Transactions on Instrumentation and Measurement.

[27]  C. Willmott,et al.  Advantages of the mean absolute error (MAE) over the root mean square error (RMSE) in assessing average model performance , 2005 .

[28]  Vladlen Koltun,et al.  An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling , 2018, ArXiv.

[29]  Manfredo Atzori,et al.  Electromyography data for non-invasive naturally-controlled robotic hand prostheses , 2014, Scientific Data.

[30]  Allison M. Okamura,et al.  Design of a Compact Actuation and Control System for Flexible Medical Robots , 2017, IEEE Robotics and Automation Letters.

[31]  Weiguang Chen,et al.  Accelerating hybrid and compact neural networks targeting perception and control domains with coarse-grained dataflow reconfiguration , 2020, Journal of Semiconductors.

[32]  Changcheng Huang,et al.  sEMG-based continuous estimation of grasp movements by long-short term memory network , 2020, Biomed. Signal Process. Control..

[33]  Ronald C. Arkin,et al.  International Governance of Autonomous Military Robots , 2010 .

[34]  Pu Liu,et al.  Identification of Constant-Posture EMG–Torque Relationship About the Elbow Using Nonlinear Dynamic Models , 2012, IEEE Transactions on Biomedical Engineering.

[35]  Joel Emer,et al.  Eyeriss: an Energy-efficient Reconfigurable Accelerator for Deep Convolutional Neural Networks Accessed Terms of Use , 2022 .

[36]  Ying Wu,et al.  Modeling the constraints of human hand motion , 2000, Proceedings Workshop on Human Motion.

[37]  Cuntai Guan,et al.  A review on EMG-based motor intention prediction of continuous human upper limb motion for human-robot collaboration , 2019, Biomed. Signal Process. Control..

[38]  Constantinos Gavriel,et al.  Gaussian Process Autoregression for Simultaneous Proportional Multi-Modal Prosthetic Control With Natural Hand Kinematics , 2017, IEEE Transactions on Neural Systems and Rehabilitation Engineering.