Decoding of Grasp Motions from EEG Signals Based on a Novel Data Augmentation Strategy

Electroencephalogram (EEG) based braincomputer interface (BCI) systems are useful tools for clinical purposes like neural prostheses. In this study, we collected EEG signals related to grasp motions. Five healthy subjects participated in this experiment. They executed and imagined five sustained-grasp actions. We proposed a novel data augmentation method that increases the amount of training data using labels obtained from electromyogram (EMG) signals analysis. For implementation, we recorded EEG and EMG simultaneously. The data augmentation over the original EEG data concluded higher classification accuracy than other competitors. As a result, we obtained the average classification accuracy of 52.49(±8.74)% for motor execution (ME) and 40.36(±3.39)% for motor imagery (MI). These are 9.30% and 6.19% higher, respectively than the result of the comparable methods. Moreover, the proposed method could minimize the need for the calibration session, which reduces the practicality of most BCIs. This result is encouraging, and the proposed method could potentially be used in future applications such as a BCI-driven robot control for handling various daily use objects.

[1]  Liu Yun,et al.  A Hand Gesture Recognition Method Based on Multi-Feature Fusion and Template Matching , 2012 .

[2]  Klaus-Robert Müller,et al.  A convolutional neural network for steady state visual evoked potential classification under ambulatory environment , 2017, PloS one.

[3]  J. Pereira,et al.  Decoding natural reach-and-grasp actions from human EEG , 2018, Journal of neural engineering.

[4]  Lorenzo Grazi,et al.  Detection of movement onset using EMG signals for upper-limb exoskeletons in reaching tasks , 2019, Journal of NeuroEngineering and Rehabilitation.

[5]  Jianjun Meng,et al.  Combining Motor Imagery With Selective Sensation Toward a Hybrid-Modality BCI , 2014, IEEE Transactions on Biomedical Engineering.

[6]  Cuntai Guan,et al.  EEG-Based Strategies to Detect Motor Imagery for Control and Rehabilitation , 2017, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[7]  David Lee,et al.  Filter Bank Regularized Common Spatial Pattern Ensemble for Small Sample Motor Imagery Classification , 2018, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[8]  Vacius Jusas,et al.  Application of Convolutional Neural Networks to Four-Class Motor Imagery Classification Problem , 2017, Inf. Technol. Control..

[9]  Andrew Y. Paek,et al.  Global cortical activity predicts shape of hand during grasping , 2015, Front. Neurosci..

[10]  Ugur Halici,et al.  A novel deep learning approach for classification of EEG motor imagery signals , 2017, Journal of neural engineering.

[11]  Xingyu Wang,et al.  Temporally Constrained Sparse Group Spatial Patterns for Motor Imagery BCI , 2019, IEEE Transactions on Cybernetics.

[12]  Oluwarotimi Williams Samuel,et al.  A motion-classification strategy based on sEMG-EEG signal combination for upper-limb amputees , 2017, Journal of NeuroEngineering and Rehabilitation.

[13]  Ji-Hoon Jeong,et al.  Brain-Controlled Robotic Arm System Based on Multi-Directional CNN-BiLSTM Network Using EEG Signals , 2020, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[14]  Klaus-Robert Müller,et al.  A lower limb exoskeleton control system based on steady state visual evoked potentials , 2015, Journal of neural engineering.

[15]  Dinggang Shen,et al.  Canonical feature selection for joint regression and multi-class identification in Alzheimer’s disease diagnosis , 2015, Brain Imaging and Behavior.

[16]  Ji-Hoon Jeong,et al.  Decoding Movement-Related Cortical Potentials Based on Subject-Dependent and Section-Wise Spectral Filtering , 2020, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[17]  Ji-Hoon Jeong,et al.  Decoding of Multi-directional Reaching Movements for EEG-Based Robot Arm Control , 2018, 2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[18]  Cuntai Guan,et al.  Filter Bank Common Spatial Pattern Algorithm on BCI Competition IV Datasets 2a and 2b , 2012, Front. Neurosci..

[19]  Feng Duan,et al.  A Novel Deep Learning Approach With Data Augmentation to Classify Motor Imagery Signals , 2019, IEEE Access.

[20]  Klaus-Robert Müller,et al.  Predicting BCI Subject Performance Using Probabilistic Spatio-Temporal Filters , 2014, PloS one.

[21]  Seong-Whan Lee,et al.  Commanding a Brain-Controlled Wheelchair Using Steady-State Somatosensory Evoked Potentials , 2018, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[22]  G. Pfurtscheller,et al.  Optimal spatial filtering of single trial EEG during imagined hand movement. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[23]  K. Müller,et al.  Effect of higher frequency on the classification of steady-state visual evoked potentials , 2016, Journal of neural engineering.

[24]  Ji-Hoon Jeong,et al.  Trajectory Decoding of Arm Reaching Movement Imageries for Brain-Controlled Robot Arm System , 2019, 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[25]  Hyoung Joong Kim,et al.  A High-Security EEG-Based Login System with RSVP Stimuli and Dry Electrodes , 2016, IEEE Transactions on Information Forensics and Security.

[26]  Medhat A. Moussa,et al.  Modeling Grasp Motor Imagery Through Deep Conditional Generative Models , 2017, IEEE Robotics and Automation Letters.

[27]  Gordon Cheng,et al.  Validating Deep Neural Networks for Online Decoding of Motor Imagery Movements from EEG Signals , 2018, Sensors.

[28]  K. Lafleur,et al.  Quadcopter control in three-dimensional space using a noninvasive motor imagery-based brain–computer interface , 2013, Journal of neural engineering.

[29]  Gernot R Müller-Putz,et al.  Upper limb movements can be decoded from the time-domain of low-frequency EEG , 2017, PloS one.

[30]  Heung-Il Suk,et al.  Subject and class specific frequency bands selection for multiclass motor imagery classification , 2011, Int. J. Imaging Syst. Technol..

[31]  Seong-Whan Lee,et al.  Changes of Functional and Effective Connectivity in Smoking Replenishment on Deprived Heavy Smokers: A Resting-State fMRI Study , 2013, PloS one.