EEG Motor Imagery Classification With Sparse Spectrotemporal Decomposition and Deep Learning

Classification of electroencephalogram-based motor imagery (MI-EEG) tasks raises a big challenge in the design and development of brain–computer interfaces (BCIs). In view of the characteristics of nonstationarity, time-variability, and individual diversity of EEG signals, a deep learning framework termed SSD-SE-convolutional neural network (CNN) is proposed for MI-EEG classification. The framework consists of three parts: 1) the sparse spectrotemporal decomposition (SSD) algorithm is proposed for feature extraction, overcoming the drawbacks of conventional time–frequency analysis methods and enhancing the robustness to noise; 2) a CNN is constructed to fully exploit the time–frequency features, thus outperforming traditional classification methods both in terms of accuracy and kappa value; and 3) the squeeze-and-excitation (SE) blocks are adopted to adaptively recalibrate channelwise feature responses, which further improves the overall performance and offers a compelling classification solution for MI-EEG applications. Experimental results on two datasets reveal that the proposed framework outperforms state-of-the-art methods in terms of both classification quality and robustness. The advantages of SSD-SE-CNN include high accuracy, high efficiency, and robustness to cross-trial and cross-session variations, making it an ideal candidate for long-term MI-EEG applications. Note to Practitioners—Motor imagery-based brain–computer interfaces (MI-BCIs) are widely used to allow a user to control a device using only his or her neural activity. This article proposed a new framework to classify two-class MI tasks based on electroencephalography (EEG) signals. In this framework, a new sparse spectrotemporal decomposition method is used to extract time–frequency features from EEG signals. A convolutional neural network with squeeze-and-excitation blocks is then constructed to classify the MI tasks. We show the superiority of our method on two datasets and prove its feasibility for long-term MI-BCI applications.

[1]  Rongrong Fu,et al.  Improvement Motor Imagery EEG Classification Based on Regularized Linear Discriminant Analysis , 2019, Journal of Medical Systems.

[2]  J. Conradt,et al.  Decoding EEG Brain Signals using Recurrent Neural Networks , 2017 .

[3]  Yan Li,et al.  Improving the Separability of Motor Imagery EEG Signals Using a Cross Correlation-Based Least Square Support Vector Machine for Brain–Computer Interface , 2012, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[4]  B. Xiaoping,et al.  The offline feature extraction of four-class motor imagery EEG based on ICA and Wavelet-CSP , 2014, Proceedings of the 33rd Chinese Control Conference.

[5]  Lawrence D. Jackel,et al.  Handwritten Digit Recognition with a Back-Propagation Network , 1989, NIPS.

[6]  Chong Li,et al.  Group Sparse Optimization via lp, q Regularization , 2016, J. Mach. Learn. Res..

[7]  Shang-Lin Wu,et al.  Common spatial pattern and linear discriminant analysis for motor imagery classification , 2013, 2013 IEEE Symposium on Computational Intelligence, Cognitive Algorithms, Mind, and Brain (CCMB).

[8]  Piotr Szachewicz,et al.  CLASSIFICATION OF MOTOR IMAGERY FOR BRAIN-COMPUTER INTERFACES , 2013 .

[9]  Nicholas V. Annetta,et al.  Restoring cortical control of functional movement in a human with quadriplegia , 2016, Nature.

[10]  Demba Ba,et al.  Robust spectrotemporal decomposition by iteratively reweighted least squares , 2014, Proceedings of the National Academy of Sciences.

[11]  Nikhil Ketkar,et al.  Introduction to PyTorch , 2021, Deep Learning with Python.

[12]  Wenfeng Zhao,et al.  Training-free compressed sensing for wireless neural recording using analysis model and group weighted ℓ1-minimization , 2017, 2016 IEEE Biomedical Circuits and Systems Conference (BioCAS).

[13]  S. Frick,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.

[14]  Hong Zeng,et al.  Discriminative spatial-frequency-temporal feature extraction and classification of motor imagery EEG: An sparse regression and Weighted Naïve Bayesian Classifier-based approach , 2017, Journal of Neuroscience Methods.

[15]  Francis R. Willett,et al.  High performance communication by people with paralysis using an intracortical brain-computer interface , 2017, eLife.

[16]  Serafeim Perdikis,et al.  Brain-Machine Interfaces: A Tale of Two Learners , 2020, IEEE Systems, Man, and Cybernetics Magazine.

[17]  K.-R. Muller,et al.  Optimizing Spatial filters for Robust EEG Single-Trial Analysis , 2008, IEEE Signal Processing Magazine.

[18]  Ugur Halici,et al.  A novel deep learning approach for classification of EEG motor imagery signals , 2017, Journal of neural engineering.

[19]  E. Donchin,et al.  Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials. , 1988, Electroencephalography and clinical neurophysiology.

[20]  Mohammed Yeasin,et al.  Learning Representations from EEG with Deep Recurrent-Convolutional Neural Networks , 2015, ICLR.

[21]  Motoaki Kawanabe,et al.  Invariant Common Spatial Patterns: Alleviating Nonstationarities in Brain-Computer Interfacing , 2007, NIPS.

[22]  Kwee-Bo Sim,et al.  Analysis the effect of PCA for feature reduction in non-stationary EEG based motor imagery of BCI system , 2014 .

[23]  Ad Aertsen,et al.  Review of the BCI Competition IV , 2012, Front. Neurosci..

[24]  Guangming Shi,et al.  Development of a Human–Robot Hybrid Intelligent System Based on Brain Teleoperation and Deep Learning SLAM , 2019, IEEE Transactions on Automation Science and Engineering.

[25]  Aiguo Song,et al.  Wavelet Transform Time-Frequency Image and Convolutional Network-Based Motor Imagery EEG Classification , 2019, IEEE Access.

[26]  Laxmidhar Behera,et al.  HJB-Equation-Based Optimal Learning Scheme for Neural Networks With Applications in Brain–Computer Interface , 2020, IEEE Transactions on Emerging Topics in Computational Intelligence.

[27]  Cuntai Guan,et al.  Filter Bank Common Spatial Pattern Algorithm on BCI Competition IV Datasets 2a and 2b , 2012, Front. Neurosci..

[28]  Lianghua He,et al.  A Deep Learning Method for Classification of EEG Data Based on Motor Imagery , 2014, ICIC.

[29]  Todd P. Coleman,et al.  A Modularized Efficient Framework for Non-Markov Time Series Estimation , 2017, IEEE Transactions on Signal Processing.

[30]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[31]  J. Decety.  Do imagined and executed actions share the same neural substrate? , 1996, Brain research. Cognitive brain research.

[32]  Jayadeva,et al.  High performance EEG signal classification using classifiability and the Twin SVM , 2015, Appl. Soft Comput..

[33]  Haixian Wang,et al.  Local temporal common spatial patterns modulated with phase locking value , 2020, Biomed. Signal Process. Control..

[34]  Stephen M. Gordon,et al.  EEGNet: A Compact Convolutional Neural Network for EEG-based Brain-Computer Interfaces , 2021 .

[35]  Enhua Wu,et al.  Squeeze-and-Excitation Networks , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  Tiago H. Falk,et al.  Deep learning-based electroencephalography analysis: a systematic review , 2019, Journal of neural engineering.

[37]  H. Flor,et al.  A spelling device for the paralysed , 1999, Nature.

[38]  G. Pfurtscheller,et al.  Designing optimal spatial filters for single-trial EEG classification in a movement task , 1999, Clinical Neurophysiology.

[39]  Florian Yger,et al.  Towards Adaptive Classification using Riemannian Geometry approaches in Brain-Computer Interfaces , 2019, 2019 7th International Winter Conference on Brain-Computer Interface (BCI).

[40]  Bangyan Zhou,et al.  How many channels are suitable for independent component analysis in motor imagery brain-computer interface , 2019, Biomed. Signal Process. Control..

[41]  Laxmidhar Behera,et al.  EEG Based Motor Imagery Classification Using Instantaneous Phase Difference Sequence , 2018, 2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[42]  Wonzoo Chung,et al.  Filter-bank local region complex-valued common spatial pattern for Motor imagery classification , 2020, 2020 8th International Winter Conference on Brain-Computer Interface (BCI).

[43]  G. Pfurtscheller,et al.  EEG-based discrimination between imagination of right and left hand movement. , 1997, Electroencephalography and clinical neurophysiology.

[44]  M Congedo,et al.  A review of classification algorithms for EEG-based brain–computer interfaces: a 10 year update , 2018, Journal of neural engineering.

[45]  Cuntai Guan,et al.  On the use of convolutional neural networks and augmented CSP features for multi-class motor imagery of EEG signals classification , 2015, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[46]  Po-Lei Lee,et al.  Frequency recognition in an SSVEP-based brain computer interface using empirical mode decomposition and refined generalized zero-crossing , 2011, Journal of Neuroscience Methods.

[47]  Laxmidhar Behera,et al.  Formulating Divergence Framework for Multiclass Motor Imagery EEG Brain Computer Interface , 2020, ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[48]  Christopher C. Cline,et al.  Noninvasive neuroimaging enhances continuous neural tracking for robotic device control , 2019, Science Robotics.

[49]  Pasin Israsena,et al.  EEG-Based Emotion Recognition Using Deep Learning Network with Principal Component Based Covariate Shift Adaptation , 2014, TheScientificWorldJournal.

[50]  Wonzoo Chung,et al.  Frequency-Optimized Local Region Common Spatial Pattern Approach for Motor Imagery Classification , 2019, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[51]  Nuri Firat Ince,et al.  Classification of single trial motor imagery EEG recordings with subject adapted non-dyadic arbitrary time–frequency tilings , 2006, Journal of neural engineering.