Prospect of data science and artificial intelligence for patient-specific neuroprostheses

Abstract Machine learning and its subfield deep learning have recently gained interest in scientific research community due to their ability to analyze and learn from big data. In this chapter, we discuss the capabilities, limitations, and current applications of unsupervised and supervised machine learning methods in addition to more recent deep learning techniques for the design and control of patient-specific neuroprostheses. Furthermore, we speculate on what they could promise for future applications.

[1]  Yoav Freund,et al.  Experiments with a New Boosting Algorithm , 1996, ICML.

[2]  D. N. Tibarewala,et al.  EEG driven model predictive position control of an artificial limb using neural net , 2012, 2012 Third International Conference on Computing, Communication and Networking Technologies (ICCCNT'12).

[3]  Riley Booth,et al.  A Wrist-Worn Piezoelectric Sensor Array for Gesture Input , 2018 .

[4]  Dario Farina,et al.  Multiday EMG-Based Classification of Hand Motions with Deep Learning Techniques , 2018, Sensors.

[5]  V. Rodriguez-Galiano,et al.  Feature selection approaches for predictive modelling of groundwater nitrate pollution: An evaluation of filters, embedded and wrapper methods. , 2018, The Science of the total environment.

[6]  Badong Chen,et al.  Surface EMG Decoding for Hand Gestures Based on Spectrogram and CNN-LSTM , 2019, 2019 2nd China Symposium on Cognitive Computing and Hybrid Intelligence (CCHI).

[7]  Shaikh Anowarul Fattah,et al.  Hand movement recognition based on singular value decomposition of surface EMG signal , 2017, 2017 IEEE Region 10 Humanitarian Technology Conference (R10-HTC).

[8]  Xuegong Zhang,et al.  Recursive SVM feature selection and sample classification for mass-spectrometry and microarray data , 2006, BMC Bioinformatics.

[9]  S. N. Nagananda,et al.  Design and Development of Real Time Bionic Hand Control Using EMG Signal , 2018, 2018 IEEE International Conference on Electronics, Computing and Communication Technologies (CONECCT).

[10]  Christian Antfolk,et al.  Extraction of Multi-Labelled Movement Information from the Raw HD-sEMG Image with Time-Domain Depth , 2019, Scientific Reports.

[11]  Ali Farhadi,et al.  AJILE Movement Prediction: Multimodal Deep Learning for Natural Human Neural Recordings and Video , 2017, AAAI.

[12]  Amit Konar,et al.  A Two-fold classification for composite decision about localized arm movement from EEG by SVM and QDA techniques , 2011, The 2011 International Joint Conference on Neural Networks.

[13]  Honghai Liu,et al.  Robust sEMG electrodes configuration for pattern recognition based prosthesis control , 2014, 2014 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[14]  Rich Caruana,et al.  Greedy Attribute Selection , 1994, ICML.

[15]  Graham Morgan,et al.  Deep learning-based artificial vision for grasp classification in myoelectric hands , 2017, Journal of neural engineering.

[16]  P. Geethanjali,et al.  Actuation of prosthetic drive using EMG signal , 2009, TENCON 2009 - 2009 IEEE Region 10 Conference.

[17]  Yahya Slimani,et al.  A Novel RFE-SVM-based Feature Selection Approach for Classification , 2012 .

[18]  Sofiane Achiche,et al.  Classification of Upper limb phantom movements in transhumeral amputees using electromyographic and kinematic features , 2018, Eng. Appl. Artif. Intell..

[19]  Yongkang Wong,et al.  Surface-Electromyography-Based Gesture Recognition by Multi-View Deep Learning , 2019, IEEE Transactions on Biomedical Engineering.

[20]  Mauridhi Hery Purnomo,et al.  An Improved Performance of Deep Learning Based on Convolution Neural Network to Classify the Hand Motion by Evaluating Hyper Parameter , 2020, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[21]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[22]  Manfredo Atzori,et al.  Deep learning with convolutional neural networks: a resource for the control of robotic prosthetic hands via electromyography , 2016 .

[23]  Chi-Woong Mun,et al.  Comparison of k-nearest neighbor, quadratic discriminant and linear discriminant analysis in classification of electromyogram signals based on the wrist-motion directions , 2011 .

[24]  D. N. Tibarewala,et al.  Performance analysis of left and right lower limb movement classification from EEG , 2016, 2016 3rd International Conference on Signal Processing and Integrated Networks (SPIN).

[25]  J. C. Sanchez,et al.  Control of a center-out reaching task using a reinforcement learning Brain-Machine Interface , 2011, 2011 5th International IEEE/EMBS Conference on Neural Engineering.

[26]  Gavin Brown,et al.  Ensemble Learning , 2010, Encyclopedia of Machine Learning and Data Mining.

[27]  Xun Chen,et al.  Feasibility Study of Advanced Neural Networks Applied to sEMG-Based Force Estimation , 2018, Sensors.

[28]  Yazhou Wang,et al.  Exploration of Cervical Myelopathy Location From Somatosensory Evoked Potentials Using Random Forests Classification , 2019, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[29]  Mariska J Vansteensel,et al.  Preservation of hand movement representation in the sensorimotor areas of amputees , 2017, Brain : a journal of neurology.

[30]  Beth Jelfs,et al.  Self-Recalibrating Surface EMG Pattern Recognition for Neuroprosthesis Control Based on Convolutional Neural Network , 2017, Front. Neurosci..

[31]  Ugur Halici,et al.  A novel deep learning approach for classification of EEG motor imagery signals , 2017, Journal of neural engineering.

[32]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[33]  Ravi Vaidyanathan,et al.  Subject-Independent Data Pooling in Classification of Gait Intent Using Mechanomyography on a Transtibial Amputee , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[34]  P.E. Crago,et al.  Functional restoration of elbow extension after spinal-cord injury using a neural network-based synergistic FES controller , 2005, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[35]  D. Cox.  The Regression Analysis of Binary Sequences , 1958 .

[36]  Ming Liu,et al.  Development of an Environment-Aware Locomotion Mode Recognition System for Powered Lower Limb Prostheses , 2016, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[37]  Andrzej Wolczowski,et al.  Hand Prosthesis Control - Software Tool for EMG Signal Analysis , 2010, ICINCO.

[38]  Benoit Gosselin,et al.  A Fully Embedded Adaptive Real-Time Hand Gesture Classifier Leveraging HD-sEMG and Deep Learning , 2019, IEEE Transactions on Biomedical Circuits and Systems.

[39]  D. N. Tibarewala,et al.  Classification of lower limb motor imagery using K Nearest Neighbor and Naïve-Bayesian classifier , 2016, 2016 3rd International Conference on Recent Advances in Information Technology (RAIT).

[40]  Masato Inoue,et al.  High Spatiotemporal Resolution ECoG Recording of Somatosensory Evoked Potentials with Flexible Micro-Electrode Arrays , 2017, Front. Neural Circuits.

[41]  Philip S. Thomas,et al.  Training an Actor-Critic Reinforcement Learning Controller for Arm Movement Using Human-Generated Rewards , 2017, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[42]  Robert F. Kirsch,et al.  Combined feedforward and feedback control of a redundant, nonlinear, dynamic musculoskeletal system , 2009, Medical & Biological Engineering & Computing.

[43]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[44]  Zhi-Hong Mao,et al.  Sub-optimally solving actuator redundancy in a hybrid neuroprosthetic system with a multi-layer neural network structure , 2019, International Journal of Intelligent Robotics and Applications.

[45]  Simona Ferrante,et al.  Artificial Neural-Network EMG Classifier for Hand Movements Prediction , 2016 .

[46]  Caihua Xiong,et al.  Simultaneous and Continuous Estimation of Shoulder and Elbow Kinematics from Surface EMG Signals , 2017, Frontiers in neuroscience.

[47]  Shouqian Sun,et al.  Single-trial EEG classification of motor imagery using deep convolutional neural networks , 2017 .

[48]  Alastair J. Loutit,et al.  Dorsal Column Nuclei Neural Signal Features Permit Robust Machine-Learning of Natural Tactile- and Proprioception-Dominated Stimuli , 2020, Frontiers in Systems Neuroscience.

[49]  Xiang Chen,et al.  Hand Gesture Recognition based on Surface Electromyography using Convolutional Neural Network with Transfer Learning Method , 2020, IEEE Journal of Biomedical and Health Informatics.

[50]  Hiroshi Yokoi,et al.  Development of an upper-limb neuroprosthesis to voluntarily control elbow and hand , 2018, Adv. Robotics.

[51]  Erik Scheme,et al.  Real-time, simultaneous myoelectric control using a convolutional neural network , 2018, PloS one.

[52]  Syed Ali Raza Zaidi,et al.  A CNN-LSTM Hybrid Framework for Wrist Kinematics Estimation Using Surface Electromyography , 2019 .

[53]  Paul Burch,et al.  Commercial fishing patterns influence odontocete whale-longline interactions in the Southern Ocean , 2019, Scientific Reports.

[54]  Shih-Ching Yeh,et al.  Classification of multichannel surface-electromyography signals based on convolutional neural networks , 2019, J. Ind. Inf. Integr..

[55]  Sotiris B. Kotsiantis,et al.  Decision trees: a recent overview , 2011, Artificial Intelligence Review.

[56]  Yongkang Wong,et al.  A novel attention-based hybrid CNN-RNN architecture for sEMG-based gesture recognition , 2018, PloS one.

[57]  June Sic Kim,et al.  Prediction of motor and somatosensory function from human ECoG , 2018, 2018 6th International Conference on Brain-Computer Interface (BCI).

[58]  Zhan Li,et al.  Muscle Fatigue Tracking with Evoked EMG via Recurrent Neural Network: Toward Personalized Neuroprosthetics , 2014, IEEE Computational Intelligence Magazine.

[59]  Honghai Liu,et al.  Learn the Temporal-Spatial Feature of sEMG via Dual-Flow Network , 2019, Int. J. Humanoid Robotics.

[60]  Toshio Tsuji,et al.  EMG-Based Motion Discrimination Using a Novel Recurrent Neural Network , 2003, Journal of Intelligent Information Systems.

[61]  Christian Cipriani,et al.  Discrimination of Object Curvature Based on a Sparse Tactile Sensor Array , 2020, Micromachines.

[62]  Jennie Si,et al.  Offline Policy Iteration Based Reinforcement Learning Controller for Online Robotic Knee Prosthesis Parameter Tuning , 2019, 2019 International Conference on Robotics and Automation (ICRA).

[63]  Glenn K. Klute,et al.  Turn Intent Detection For Control of a Lower Limb Prosthesis , 2018, IEEE Transactions on Biomedical Engineering.

[64]  Alexandre Balbinot,et al.  Novel method to characterize upper-limb movements based on paraconsistent logic and myoelectric signals , 2016, 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[65]  Robert Tibold,et al.  Prediction of muscle activity during loaded movements of the upper limb , 2014, Journal of NeuroEngineering and Rehabilitation.

[66]  Erik J. Scheme,et al.  Support Vector Regression for Improved Real-Time, Simultaneous Myoelectric Control , 2014, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[67]  Clément Gosselin,et al.  Deep Learning for Electromyographic Hand Gesture Signal Classification Using Transfer Learning , 2018, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[68]  Vladimir Vapnik,et al.  Support-vector networks , 2004, Machine Learning.

[69]  Kunihiko Fukushima,et al.  Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position , 1980, Biological Cybernetics.

[70]  Kiran Marri,et al.  Classification of Muscle Fatigue in Dynamic Contraction Using Surface Electromyography Signals and Multifractal Singularity Spectral Analysis , 2016 .

[71]  Ganesh R. Naik,et al.  Nonnegative Matrix Factorization for the Identification of EMG Finger Movements: Evaluation Using Matrix Analysis , 2015, IEEE Journal of Biomedical and Health Informatics.

[72]  Marcello Restelli,et al.  Does Reinforcement Learning outperform PID in the control of FES-induced elbow flex-extension? , 2018, 2018 IEEE International Symposium on Medical Measurements and Applications (MeMeA).

[73]  Lorenzo Grazi,et al.  EMG-Based Detection of User’s Intentions for Human-Machine Shared Control of an Assistive Upper-Limb Exoskeleton , 2017 .

[74]  Syed Omer Gilani,et al.  Performance Evaluation of Convolutional Neural Network for Hand Gesture Recognition Using EMG , 2020, Sensors.

[75]  Ahmad Patooghy,et al.  An Embedded System for Collection and Real-Time Classification of a Tactile Dataset , 2020, IEEE Access.

[76]  Milan Simic,et al.  Classification of left and right foot kinaesthetic motor imagery using common spatial pattern , 2019, Biomedical physics & engineering express.

[77]  Lee E Miller,et al.  A comprehensive model-based framework for optimal design of biomimetic patterns of electrical stimulation for prosthetic sensation , 2020, Journal of neural engineering.

[78]  Abhishek Prasad,et al.  EEG-controlled functional electrical stimulation for hand opening and closing in chronic complete cervical spinal cord injury , 2018, Biomedical Physics & Engineering Express.

[79]  William W. Lytton,et al.  Cortical Spiking Network Interfaced with Virtual Musculoskeletal Arm and Robotic Arm , 2015, Front. Neurorobot..

[80]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[81]  Nitish V Thakor,et al.  Stable Responsive EMG Sequence Prediction and Adaptive Reinforcement With Temporal Convolutional Networks , 2020, IEEE Transactions on Biomedical Engineering.

[82]  Xu Cui,et al.  Speeded Near Infrared Spectroscopy (NIRS) Response Detection , 2010, PloS one.

[83]  R.F. Kirsch,et al.  Feasibility of EMG-Based Neural Network Controller for an Upper Extremity Neuroprosthesis , 2009, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[84]  Dustin J. Tyler,et al.  Stable, three degree-of-freedom myoelectric prosthetic control via chronic bipolar intramuscular electrodes: a case study , 2019, Journal of NeuroEngineering and Rehabilitation.

[85]  Giulio Sandini,et al.  Fine detection of grasp force and posture by amputees via surface electromyography , 2009, Journal of Physiology-Paris.

[86]  Erhan Akdoğan,et al.  Comparison of Classification Algorithms for Detecting Patient Posture in Expandable Tumor Prostheses , 2020 .

[87]  Haoran Wang,et al.  A Novel Combination Model of Convolutional Neural Network and Long Short-Term Memory Network for Upper Limb Evaluation Using Kinect-Based System , 2019, IEEE Access.

[88]  Theocharis Kyriacou,et al.  Feasibility of using combined EMG and kinematic signals for prosthesis control: A simulation study using a virtual reality environment , 2016, Journal of electromyography and kinesiology : official journal of the International Society of Electrophysiological Kinesiology.

[89]  Dapeng Yang,et al.  EMG Pattern Recognition Using Convolutional Neural Network with Different Scale Signal/Spectra Input , 2019, Int. J. Humanoid Robotics.

[90]  W S McCulloch,et al.  A logical calculus of the ideas immanent in nervous activity , 1990, The Philosophy of Artificial Intelligence.

[91]  Pedro Larrañaga,et al.  A review of feature selection techniques in bioinformatics , 2007, Bioinform..

[92]  Raviraj Nataraj,et al.  Trunk acceleration for neuroprosthetic control of standing: a pilot study. , 2012, Journal of applied biomechanics.

[93]  Wolfram Burgard,et al.  Deep learning with convolutional neural networks for EEG decoding and visualization , 2017, Human brain mapping.

[94]  Robert Riener,et al.  Control strategies for active lower extremity prosthetics and orthotics: a review , 2015, Journal of NeuroEngineering and Rehabilitation.

[95]  Hiroshi Okumura,et al.  Surface EMG Pattern Recognition Using Long Short-Term Memory Combined with Multilayer Perceptron , 2018, 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[96]  Sherin Youssef,et al.  Hybrid Brain Computer Interface for Movement Control of Upper Limb Prostheses , 2018, 2018 International Conference on Biomedical Engineering and Applications (ICBEA).

[97]  Nitish V. Thakor,et al.  Cortical decoding of individual finger and wrist kinematics for an upper-limb neuroprosthesis , 2009, 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[98]  Amit Konar,et al.  Motor imagery, P300 and error-related EEG-based robot arm movement control for rehabilitation purpose , 2014, Medical & Biological Engineering & Computing.

[99]  Chang Liu,et al.  Relative Entropy Regularized TDLAS Tomography for Robust Temperature Imaging , 2020, IEEE Transactions on Instrumentation and Measurement.

[100]  Amparo Alonso-Betanzos,et al.  Filter Methods for Feature Selection - A Comparative Study , 2007, IDEAL.

[101]  Miguel Nicolelis,et al.  Decoding Movements from Cortical Ensemble Activity Using a Long Short-Term Memory Recurrent Network , 2019, Neural Computation.

[102]  Erkan Kaplanoglu,et al.  Comparison of EMG Based Finger Motion Classification Algorithms , 2019, 2019 27th Signal Processing and Communications Applications Conference (SIU).

[103]  Silvestro Micera,et al.  Deep learning-based BCI for gait decoding from EEG with LSTM recurrent neural network , 2020, Journal of neural engineering.

[104]  Lars Kai Hansen,et al.  Neural Network Ensembles , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[105]  Seong-Whan Lee,et al.  Movement intention decoding based on deep learning for multiuser myoelectric interfaces , 2016, 2016 4th International Winter Conference on Brain-Computer Interface (BCI).

[106]  Chunming Xia,et al.  STUDY OF GAIT PATTERN RECOGNITION BASED ON FUSION OF MECHANOMYOGRAPHY AND ATTITUDE ANGLE SIGNAL , 2020 .

[107]  Dario Farina,et al.  Self-Correcting Pattern Recognition System of Surface EMG Signals for Upper Limb Prosthesis Control , 2014, IEEE Transactions on Biomedical Engineering.

[108]  Manfredo Atzori,et al.  Comparison of six electromyography acquisition setups on hand movement classification tasks , 2017, PloS one.

[109]  Xiang Zhang,et al.  Clustering Neural Patterns in Kernel Reinforcement Learning Assists Fast Brain Control in Brain-Machine Interfaces , 2019, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[110]  Johan Wessberg,et al.  Brain decoding of texture processing using independent component analysis and support vector machines , 2009, 2009 IEEE International Symposium on Intelligent Signal Processing.

[111]  Micael S. Couceiro,et al.  Stepping-stones to Transhumanism: An EMG-controlled Low-cost Prosthetic Hand for Academia , 2018, 2018 International Conference on Intelligent Systems (IS).

[112]  Marek Kurzynski,et al.  Control of Hand Prosthesis Using Fusion of Biosignals and Information from Prosthesis Sensors , 2015, Computational Intelligence and Efficiency in Engineering Systems.

[113]  Mehran Jahed,et al.  An exploratory study to design a novel hand movement identification system , 2009, Comput. Biol. Medicine.

[114]  Shaikh Anowarul Fattah,et al.  Basic hand action classification based on surface EMG using autoregressive reflection coefficient , 2017, TENCON 2017 - 2017 IEEE Region 10 Conference.

[115]  William W. Lytton,et al.  Virtual musculoskeletal arm and robotic arm driven by a biomimetic model of sensorimotor cortex with reinforcement learning , 2013, 2013 IEEE Signal Processing in Medicine and Biology Symposium (SPMB).

[116]  Max Ortiz-Catalan,et al.  Evaluation of surface EMG-based recognition algorithms for decoding hand movements , 2019, Medical & Biological Engineering & Computing.

[117]  Lin Chen,et al.  Hand Gesture Recognition Using Compact CNN via Surface Electromyography Signals , 2020, Sensors.

[118]  N.V. Thakor,et al.  Decoding Individuated Finger Movements Using Volume-Constrained Neuronal Ensembles in the M1 Hand Area , 2008, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[119]  Masashi Sekiya,et al.  Linear Logistic Regression for Estimation of Lower Limb Muscle Activations , 2019, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[120]  Mukhtaj Khan,et al.  Amputee walking mode recognition based on mel frequency cepstral coefficients using surface electromyography sensor , 2020, Int. J. Sens. Networks.

[121]  Marco D. Santambrogio,et al.  Robustness of Surface EMG Classifiers with Fixed-Point Decomposition on Reconfigurable Architecture , 2018, 2018 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW).

[122]  Panagiotis Artemiadis,et al.  User-Independent Hand Motion Classification With Electromyography , 2013 .

[123]  Ilja Kuzborskij,et al.  Characterization of a Benchmark Database for Myoelectric Movement Classification , 2015, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[124]  F. K. Lam,et al.  Fuzzy EMG classification for prosthesis control. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[125]  Jun Zhong,et al.  Pattern recognition and bionic manipulator driving by surface electromyography signals using convolutional neural network , 2018, International Journal of Advanced Robotic Systems.