Deep Learning with Convolutional Neural Network for Proportional Control of Finger Movements from surface EMG Recordings

The control of robotic prosthetic hands (RPHs) for upper limb amputees is far from optimal. Simultaneous and proportional finger control of a RPH based on EMG signals is still challenging. Based on EMG and kinematics recordings of subjects following a pre-defined sequence of single and multi-fingers movements, we aimed at predicting finger flexion and thumb opposition angles. We compared two deep learning (DL) based approaches, the first one using the raw EMG signals and the second one using the spectrogram of the signal as input, with the standard state of the art decoding technique (STD) for finger angle regression. Using a genetic algorithm for hyper-parameter optimization, we obtained an optimized model architecture (and set of features in the case of STD) for each condition on one recording session. Then, we evaluated the best model of each condition on the eleven EMG and finger kinematics recordings available from four subjects. The two DL approaches based on convolutional neural networks predicted finger angles with a similar mean squared error loss but both of them outperformed the standard approach for the regression of simultaneous single-finger angles. This proposed decoding strategy and hyperparameter optimization framework provides a basis to further improve single finger proportional control for RPHs.

[1]  S. Micera,et al.  Current Solutions and Future Trends for Robotic Prosthetic Hands , 2021, Annu. Rev. Control. Robotics Auton. Syst..

[2]  Xun Chen,et al.  A Regression-Based Framework for Quantitative Assessment of Muscle Spasticity Using Combined EMG and Inertial Data From Wearable Sensors , 2019, Front. Neurosci..

[3]  Therese M Meyer,et al.  Psychological aspects of mutilating hand injuries. , 2003, Hand clinics.

[4]  Ausif Mahmood,et al.  Review of Deep Learning Algorithms and Architectures , 2019, IEEE Access.

[5]  Sean R. Anderson,et al.  Simultaneous Force Regression and Movement Classification of Fingers via Surface EMG within a Unified Bayesian Framework , 2018, Front. Bioeng. Biotechnol..

[6]  Broderick Crawford,et al.  Automating Configuration of Convolutional Neural Network Hyperparameters Using Genetic Algorithm , 2020, IEEE Access.

[7]  Yinghong Peng,et al.  EMG‐Based Estimation of Limb Movement Using Deep Learning With Recurrent Convolutional Neural Networks , 2018, Artificial organs.

[8]  D. Farina,et al.  Simultaneous and Proportional Estimation of Hand Kinematics From EMG During Mirrored Movements at Multiple Degrees-of-Freedom , 2012, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[9]  Vijayan K. Asari,et al.  The History Began from AlexNet: A Comprehensive Survey on Deep Learning Approaches , 2018, ArXiv.

[10]  Pornchai Phukpattaranont,et al.  Feature reduction and selection for EMG signal classification , 2012, Expert Syst. Appl..

[11]  Hong Liu,et al.  EMG dataset augmentation approaches for improving the multi-DOF wrist movement regression accuracy and robustness , 2018, 2018 IEEE International Conference on Robotics and Biomimetics (ROBIO).

[12]  Clément Gosselin,et al.  A convolutional neural network for robotic arm guidance using sEMG based frequency-features , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[13]  Ilja Kuzborskij,et al.  Characterization of a Benchmark Database for Myoelectric Movement Classification , 2015, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[14]  Erik Scheme,et al.  Regression convolutional neural network for improved simultaneous EMG control , 2019, Journal of neural engineering.

[15]  Yu Hu,et al.  Surface EMG-Based Inter-Session Gesture Recognition Enhanced by Deep Domain Adaptation , 2017, Sensors.

[16]  Erik Scheme,et al.  EMG Pattern Recognition in the Era of Big Data and Deep Learning , 2018, Big Data Cogn. Comput..

[17]  Erik Scheme,et al.  A Deep Transfer Learning Approach to Reducing the Effect of Electrode Shift in EMG Pattern Recognition-Based Control , 2019, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[18]  Nicolas Sommer,et al.  Shared human–robot proportional control of a dexterous myoelectric prosthesis , 2019, Nature Machine Intelligence.

[19]  Purushothaman Geethanjali,et al.  Myoelectric control of prosthetic hands: state-of-the-art review , 2016, Medical devices.

[20]  H. Kawasaki,et al.  Estimation of Finger Joint Angles from sEMG Using a Neural Network Including Time Delay Factor and Recurrent Structure , 2012 .

[21]  Manfredo Atzori,et al.  Building the Ninapro database: A resource for the biorobotics community , 2012, 2012 4th IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob).

[22]  Yongkang Wong,et al.  A novel attention-based hybrid CNN-RNN architecture for sEMG-based gesture recognition , 2018, PloS one.

[23]  Kevin B. Englehart,et al.  A robust, real-time control scheme for multifunction myoelectric control , 2003, IEEE Transactions on Biomedical Engineering.

[24]  P. Herberts,et al.  Hand prosthesis control via myoelectric patterns. , 1973, Acta orthopaedica Scandinavica.

[25]  Prachi Mukherji,et al.  Electromyogram (EMG) based fingers movement recognition using sparse filtering of wavelet packet coefficients , 2019, Sādhanā.

[26]  Manfredo Atzori,et al.  Deep Learning with Convolutional Neural Networks Applied to Electromyography Data: A Resource for the Classification of Movements for Prosthetic Hands , 2016, Front. Neurorobot..

[27]  Seong-Whan Lee,et al.  Movement intention decoding based on deep learning for multiuser myoelectric interfaces , 2016, 2016 4th International Winter Conference on Brain-Computer Interface (BCI).

[28]  Neena Aloysius,et al.  A review on deep convolutional neural networks , 2017, 2017 International Conference on Communication and Signal Processing (ICCSP).