A Multi-Branch 3D Convolutional Neural Network for EEG-Based Motor Imagery Classification

One of the challenges in motor imagery (MI) classification tasks is finding an easy-handled electroencephalogram (EEG) representation method which can preserve not only temporal features but also spatial ones. To fully utilize the features on various dimensions of EEG, a novel MI classification framework is first introduced in this paper, including a new 3D representation of EEG, a multi-branch 3D convolutional neural network (3D CNN) and the corresponding classification strategy. The 3D representation is generated by transforming EEG signals into a sequence of 2D array which preserves spatial distribution of sampling electrodes. The multi-branch 3D CNN and classification strategy are designed accordingly for the 3D representation. Experimental evaluation reveals that the proposed framework reaches state-of-the-art classification kappa value level and significantly outperforms other algorithms by 50% decrease in standard deviation of different subjects, which shows good performance and excellent robustness on different subjects. The framework also shows great performance with only nine sampling electrodes, which can significantly enhance its practicality. Moreover, the multi-branch structure exhibits its low latency and a strong ability in mitigating overfitting issues which often occur in MI classification because of the small training dataset.

[1]  G Pfurtscheller,et al.  Seperability of four-class motor imagery data using independent components analysis , 2006, Journal of neural engineering.

[2]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Gordon Cheng,et al.  Validating Deep Neural Networks for Online Decoding of Motor Imagery Movements from EEG Signals , 2018, Sensors.

[4]  Xiaofeng Xie,et al.  Motor Imagery Classification Based on Bilinear Sub-Manifold Learning of Symmetric Positive-Definite Matrices , 2017, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[5]  Xiang Zhang,et al.  OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks , 2013, ICLR.

[6]  Wei-Yen Hsu,et al.  EEG-based motor imagery analysis using weighted wavelet transform features , 2009, Journal of Neuroscience Methods.

[7]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[8]  Mohammed Yeasin,et al.  Learning Representations from EEG with Deep Recurrent-Convolutional Neural Networks , 2015, ICLR.

[9]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[10]  Jose L. Contreras-Vidal,et al.  Sitting and standing intention can be decoded from scalp EEG recorded prior to movement execution , 2014, Front. Neurosci..

[11]  David A. Patterson,et al.  In-datacenter performance analysis of a tensor processing unit , 2017, 2017 ACM/IEEE 44th Annual International Symposium on Computer Architecture (ISCA).

[12]  Jacob Cohen,et al.  The Equivalence of Weighted Kappa and the Intraclass Correlation Coefficient as Measures of Reliability , 1973 .

[13]  Ming Yang,et al.  3D Convolutional Neural Networks for Human Action Recognition , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Andrew Y. Paek,et al.  Global cortical activity predicts shape of hand during grasping , 2015, Front. Neurosci..

[15]  Ugur Halici,et al.  A novel deep learning approach for classification of EEG motor imagery signals , 2017, Journal of neural engineering.

[16]  Ad Aertsen,et al.  Review of the BCI Competition IV , 2012, Front. Neurosci..

[17]  David G. Stork,et al.  Pattern Classification (2nd ed.) , 1999 .

[18]  Vladimir Bostanov,et al.  BCI competition 2003-data sets Ib and IIb: feature extraction from event-related brain potentials with the continuous wavelet transform and the t-value scalogram , 2004, IEEE Transactions on Biomedical Engineering.

[19]  Changle Zhou,et al.  Exploring spatial-frequency-sequential relationships for motor imagery classification with recurrent neural network , 2018, BMC Bioinformatics.

[20]  Wolfram Burgard,et al.  Deep learning with convolutional neural networks for EEG decoding and visualization , 2017, Human brain mapping.

[21]  Shuicheng Yan,et al.  Learning Temporal Information for Brain-Computer Interface Using Convolutional Neural Networks , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[22]  Wei He,et al.  Performance of Motor Imagery Brain-Computer Interface Based on Anodal Transcranial Direct Current Stimulation Modulation , 2013, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[23]  Chin-Teng Lin,et al.  Multimodal Fuzzy Fusion for Enhancing the Motor-Imagery-Based Brain Computer Interface , 2019, IEEE Computational Intelligence Magazine.

[24]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[25]  P. de Chazal,et al.  A parametric feature extraction and classification strategy for brain-computer interfacing , 2005, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[26]  G. Pfurtscheller,et al.  Optimal spatial filtering of single trial EEG during imagined hand movement. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[27]  Cuntai Guan,et al.  Filter Bank Common Spatial Pattern (FBCSP) in Brain-Computer Interface , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[28]  F. L. D. Silva,et al.  Event-related EEG/MEG synchronization and desynchronization: basic principles , 1999, Clinical Neurophysiology.

[29]  C.R. Hema,et al.  Recognition of motor imagery of hand movements for a BMI using PCA features , 2008, 2008 International Conference on Electronic Design.

[30]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[31]  Klaus-Robert Müller,et al.  Subject independent EEG-based BCI decoding , 2009, NIPS.

[32]  Shuai Wang,et al.  EEG Classification of Motor Imagery Using a Novel Deep Learning Framework , 2019, Sensors.

[33]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[34]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[35]  Peter E. Hart,et al.  Nearest neighbor pattern classification , 1967, IEEE Trans. Inf. Theory.

[36]  Shang-Lin Wu,et al.  Fuzzy Integral With Particle Swarm Optimization for a Motor-Imagery-Based Brain–Computer Interface , 2017, IEEE Transactions on Fuzzy Systems.

[37]  Yangsong Zhang,et al.  Z-Score Linear Discriminant Analysis for EEG Based Brain-Computer Interfaces , 2013, PloS one.

[38]  Yuanqing Li,et al.  Control of a Wheelchair in an Indoor Environment Based on a Brain–Computer Interface and Automated Navigation , 2016, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[39]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[40]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[41]  Ping Xue,et al.  Sub-band Common Spatial Pattern (SBCSP) for Brain-Computer Interface , 2007, 2007 3rd International IEEE/EMBS Conference on Neural Engineering.

[42]  Dario Farina,et al.  Estimation of Neuromuscular Primitives from EEG Slow Cortical Potentials in Incomplete Spinal Cord Injury Individuals for a New Class of Brain-Machine Interfaces , 2018, Front. Comput. Neurosci..

[43]  Cuntai Guan,et al.  On the use of convolutional neural networks and augmented CSP features for multi-class motor imagery of EEG signals classification , 2015, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[44]  Chao Li,et al.  A Brain-Machine Interface Based on ERD/ERS for an Upper-Limb Exoskeleton Control , 2016, Sensors.

[45]  G. Pfurtscheller,et al.  Information transfer rate in a five-classes brain-computer interface , 2001, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[46]  Djamel Bouchaffra,et al.  Random Forest and Filter Bank Common Spatial Patterns for EEG-Based Motor Imagery Classification , 2014, 2014 5th International Conference on Intelligent Systems, Modelling and Simulation.

[47]  Heung-Il Suk,et al.  Subject and class specific frequency bands selection for multiclass motor imagery classification , 2011, Int. J. Imaging Syst. Technol..