Fusion of EEG and EMG signals for classification of unilateral foot movements

Abstract Introduction The study of motor cortex represents the presence of functional activity around mesial surface during all lower limb movements. Due to this, the problem of classification for intricate lower limb movements is particularly challenging with existing non-invasive technologies, such as electroencephalography (EEG). The other bio-signal used for detection i.e. electromyography (EMG), underlines the factors such as muscular fatigue and spasm as possible hindrance for efficient task based classification. Methods This work aims to explore the fusion of both, EEG and EMG, sensing modules to identify unilateral lower limb movements. Four channel sets were formed with optimal selection of EEG and EMG channels. The processed bio-signals were analyzed for parallel as well as cascaded classification of five tasks. The performance has been assessed using two parameters, prediction accuracy (PA) and computational time (CT). Results The approach successfully classified the five tasks with maximum PA of (96.58 ± 2.37)% and CT of (51.89 ± 1.15)ms for cascaded scheme. The optimal performance has been achieved with PA of (90.06 ± 9.71)% and (89.81 ± 9.41)% for channel-set (Ch) i.e. 7-Ch and 3-Ch, respectively. The resulting CT of (52.82 ± 3.56)ms and (65.38 ± 3.36)ms have been obtained for 7-Ch and 3-Ch, respectively. The parallel scheme resulted in PA of (85.88 ± 3.92)% and (86.16 ± 3.97)% along with CT of (33.23 ± 6.74)ms and (34.80 ± 10.42)ms for 7-Ch and 3-Ch, respectively. Conclusion The obtained results showed a higher PA for the case of cascaded classification compared to the parallel scheme. Promising results have been achieved, for healthy participants and can be used for future applications of robotic device control and rehabilitation.

[1]  Daniel P. Ferris,et al.  Learning to walk with an adaptive gain proportional myoelectric controller for a robotic ankle exoskeleton , 2015, Journal of NeuroEngineering and Rehabilitation.

[2]  Jose L. Contreras-Vidal,et al.  Design and Optimization of an EEG-Based Brain Machine Interface (BMI) to an Upper-Limb Exoskeleton for Stroke Survivors , 2016, Front. Neurosci..

[3]  Niels Birbaumer,et al.  A hybrid brain-machine interface based on EEG and EMG activity for the motor rehabilitation of stroke patients , 2017, 2017 International Conference on Rehabilitation Robotics (ICORR).

[4]  Alberto Borboni,et al.  EEG-Based Control for Upper and Lower Limb Exoskeletons and Prostheses: A Systematic Review , 2018, Sensors.

[5]  Fan Zhang,et al.  Continuous Locomotion-Mode Identification for Prosthetic Legs Based on Neuromuscular–Mechanical Fusion , 2011, IEEE Transactions on Biomedical Engineering.

[6]  Andrew N Meltzoff,et al.  Neural correlates of action observation and execution in 14-month-old infants: an event-related EEG desynchronization study. , 2011, Developmental science.

[7]  Angkoon Phinyomark,et al.  Feature extraction of the first difference of EMG time series for EMG pattern recognition , 2014, Comput. Methods Programs Biomed..

[8]  Klaus-Robert Müller,et al.  Enhanced Performance by a Hybrid Nirs–eeg Brain Computer Interface , 2022 .

[9]  Yasuharu Koike,et al.  Reconstruction of flexor and extensor muscle activities from electroencephalography cortical currents , 2012, NeuroImage.

[10]  Masa-aki Sato,et al.  Reconstruction of two-dimensional movement trajectories from selected magnetoencephalography cortical currents by combined sparse Bayesian methods , 2011, NeuroImage.

[11]  C. Braun,et al.  Movement related cortical potentials change after EEG-BMI rehabilitation in chronic stroke , 2013, 2013 6th International IEEE/EMBS Conference on Neural Engineering (NER).

[12]  Chao Li,et al.  A Brain-Machine Interface Based on ERD/ERS for an Upper-Limb Exoskeleton Control , 2016, Sensors.

[13]  L. Montesano,et al.  Detecting intention to walk in stroke patients from pre-movement EEG correlates , 2015, Journal of NeuroEngineering and Rehabilitation.

[14]  Junliang Huang,et al.  Hybrid Brain/Muscle Signals Powered Wearable Walking Exoskeleton Enhancing Motor Ability in Climbing Stairs Activity , 2019, IEEE Transactions on Medical Robotics and Bionics.

[15]  Gong Chen,et al.  A review of lower extremity assistive robotic exoskeletons in rehabilitation therapy. , 2013, Critical reviews in biomedical engineering.

[16]  Andreea Ioana Sburlea,et al.  EEG neural correlates of goal-directed movement intention , 2017, NeuroImage.

[17]  Nicolas Y. Masse,et al.  Reach and grasp by people with tetraplegia using a neurally controlled robotic arm , 2012, Nature.

[18]  Huosheng Hu,et al.  Bio-signal based control in assistive robots: a survey , 2015, Digit. Commun. Networks.

[19]  G. Pfurtscheller,et al.  Event-related beta EEG-changes during passive and attempted foot movements in paraplegic patients , 2007, Brain Research.

[20]  T. Hortobágyi,et al.  Teager–Kaiser energy operator signal conditioning improves EMG onset detection , 2010, European Journal of Applied Physiology.

[21]  Sherin Youssef,et al.  Hybrid Brain Computer Interface for Movement Control of Upper Limb Prostheses , 2018, 2018 International Conference on Biomedical Engineering and Applications (ICBEA).

[22]  Artur Polinski,et al.  The role of EMG module in hybrid interface of prosthetic arm , 2017, 2017 10th International Conference on Human System Interactions (HSI).

[23]  Arne D. Ekstrom,et al.  Single-Neuron Responses in Humans during Execution and Observation of Actions , 2010, Current Biology.

[24]  Ping Zhou,et al.  Wavelet Packet Feature Assessment for High-Density Myoelectric Pattern Recognition and Channel Selection toward Stroke Rehabilitation , 2016, Front. Neurol..

[25]  Nurhazimah Nazmi,et al.  Walking gait event detection based on electromyography signals using artificial neural network , 2019, Biomed. Signal Process. Control..

[26]  Min-Chun Pan,et al.  Lower-limb motion classification for hemiparetic patients through IMU and EMG signal processing , 2016, 2016 International Conference on Biomedical Engineering (BME-HUST).

[27]  Yuan-Ting Zhang,et al.  A novel channel selection method for multiple motion classification using high-density electromyography , 2014, BioMedical Engineering OnLine.

[28]  Clemens Brunner,et al.  Mu rhythm (de)synchronization and EEG single-trial classification of different motor imagery tasks , 2006, NeuroImage.

[29]  J L Pons,et al.  Detection of the onset of upper-limb movements based on the combined analysis of changes in the sensorimotor rhythms and slow cortical potentials , 2014, Journal of neural engineering.

[30]  Angel Rodríguez-Liñán,et al.  Hybrid BCI approach to control an artificial tibio-femoral joint , 2016, 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[31]  Siti Anom Ahmad,et al.  Classification of ankle joint movements based on surface electromyography signals for rehabilitation robot applications , 2015, Medical & Biological Engineering & Computing.

[32]  Pornchai Phukpattaranont,et al.  Feature reduction and selection for EMG signal classification , 2012, Expert Syst. Appl..

[33]  Masa-aki Sato,et al.  Evaluation of hierarchical Bayesian method through retinotopic brain activities reconstruction from fMRI and MEG signals , 2008, NeuroImage.

[34]  N. Ward,et al.  Assessment of cortical reorganisation for hand function after stroke , 2011, The Journal of physiology.

[35]  Neelesh Kumar,et al.  Scrutinizing different EEG-Based Mechanisms for Motor Control and Rehabilitation of Lower Limb Disabilities , 2017 .

[36]  Yoshiaki Hayashi,et al.  Towards Hybrid EEG-EMG-Based Control Approaches to be Used in Bio-robotics Applications: Current Status, Challenges and Future Directions , 2013, Paladyn J. Behav. Robotics.

[37]  Antonio Frisoli,et al.  Online Finger Control Using High-Density EMG and Minimal Training Data for Robotic Applications , 2019, IEEE Robotics and Automation Letters.

[38]  Dong Liu,et al.  EEG-Based Lower-Limb Movement Onset Decoding: Continuous Classification and Asynchronous Detection , 2018, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[39]  Oluwarotimi Williams Samuel,et al.  A motion-classification strategy based on sEMG-EEG signal combination for upper-limb amputees , 2017, Journal of NeuroEngineering and Rehabilitation.

[40]  N. Bolognini,et al.  Grasping with the foot: Goal and motor expertise in action observation , 2014, Human brain mapping.

[41]  Yi Wu,et al.  Neurophysiological substrates of stroke patients with motor imagery-based brain-computer interface training , 2014, The International journal of neuroscience.

[42]  Jerald D. Kralik,et al.  Real-time prediction of hand trajectory by ensembles of cortical neurons in primates , 2000, Nature.

[43]  Guanglin Li,et al.  New Control Strategies for Multifunctional Prostheses that Combine Electromyographic and Speech Signals , 2015, IEEE Intelligent Systems.

[44]  Ricardo Chavarriaga,et al.  A hybrid brain–computer interface based on the fusion of electroencephalographic and electromyographic activities , 2011, Journal of neural engineering.

[45]  S. Kastner,et al.  Complex organization of human primary motor cortex: a high-resolution fMRI study. , 2008, Journal of neurophysiology.

[46]  Sangtae Ahn,et al.  Multi-Modal Integration of EEG-fNIRS for Brain-Computer Interfaces – Current Limitations and Future Directions , 2017, Front. Hum. Neurosci..

[47]  Yasuharu Koike,et al.  Decoding finger movement in humans using synergy of EEG cortical current signals , 2017, Scientific Reports.

[48]  Ganesh R. Naik,et al.  Nonnegative Matrix Factorization for the Identification of EMG Finger Movements: Evaluation Using Matrix Analysis , 2015, IEEE Journal of Biomedical and Health Informatics.

[49]  Dennis C. Tkach,et al.  Study of stability of time-domain features for electromyographic pattern recognition , 2010, Journal of NeuroEngineering and Rehabilitation.

[50]  Neelesh Kumar,et al.  A Novel Approach for Real-Time Gait Events Detection Using Developed Wireless Foot Sensor Module , 2019, IEEE Sensors Letters.

[51]  Miguel A. L. Nicolelis,et al.  Extracting Kinematic Parameters for Monkey Bipedal Walking from Cortical Neuronal Ensemble Activity , 2009, Front. Integr. Neurosci..

[52]  Wellington Pinheiro dos Santos,et al.  Electromyography-controlled car: A proof of concept based on surface electromyography, Extreme Learning Machines and low-cost open hardware , 2019, Comput. Electr. Eng..

[53]  Dario Farina,et al.  The Extraction of Neural Information from the Surface EMG for the Control of Upper-Limb Prostheses: Emerging Avenues and Challenges , 2014, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[54]  Yasuharu Koike,et al.  Decoding of Ankle Flexion and Extension from Cortical Current Sources Estimated from Non-invasive Brain Activity Recording Methods , 2018, Front. Neurosci..

[55]  Hiroshi Yokoi,et al.  Training in Use of Brain–Machine Interface-Controlled Robotic Hand Improves Accuracy Decoding Two Types of Hand Movements , 2018, Front. Neurosci..

[56]  Jinghui Cao,et al.  Reviewing high-level control techniques on robot-assisted upper-limb rehabilitation , 2018, Adv. Robotics.

[57]  G. Pfurtscheller,et al.  Foot and hand area mu rhythms. , 1997, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[58]  G. Pfurtscheller,et al.  Functional dissociation of lower and upper frequency mu rhythms in relation to voluntary limb movement , 2000, Clinical Neurophysiology.

[59]  Yunfa Fu,et al.  A hybrid BCI based on EEG and fNIRS signals improves the performance of decoding motor imagery of both force and speed of hand clenching , 2015, Journal of neural engineering.