HS-CNN: a CNN with hybrid convolution scale for EEG motor imagery classification

OBJECTIVE The EEG motor imagery classification has been widely used in healthcare applications such as mobile asisstive robots and post-stroke rehabilitation. Recently, CNN-based EEG motor imagery classification methods have been proposed and achieve relatively high classification accuracy. However, these methods use single convolution scale in the CNN, while the best convolution scale differs from subject to subject. This limits the classification accuracy. Another issue is that the classification accuracy degrades when the training data is limited. APPROACH To address these issues, we have proposed a hybrid-scale CNN architecture with a data augmentation method for EEG motor imagery classification. MAIN RESULTS Compared with several state-of-the-art methods, the proposed method achieve an average classification accuracy of 87.6% with 0.2% deviation, which outperforms several state-of-the-art EEG motor imagery classification methods. SIGNIFICANCE The proposed method effectively addressed the issues of existing CNN-based EEG motor imagery classification methods and improved the classification accuracy.

[1]  Lianghua He,et al.  A Deep Learning Method for Classification of EEG Data Based on Motor Imagery , 2014, ICIC.

[2]  Cuntai Guan,et al.  Brain-Computer Interface in Stroke Rehabilitation , 2013, J. Comput. Sci. Eng..

[3]  Anton Nijholt,et al.  Experiencing BCI Control in a Popular Computer Game , 2013, IEEE Transactions on Computational Intelligence and AI in Games.

[4]  Klaus-Robert Müller,et al.  Playing Pinball with non-invasive BCI , 2008, NIPS.

[5]  Bin He,et al.  Noninvasive Brain-Computer Interfaces Based on Sensorimotor Rhythms , 2015, Proceedings of the IEEE.

[6]  Haizhou Li,et al.  A Spiking Neural Network System for Robust Sequence Recognition , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[7]  Brent Lance,et al.  EEGNet: a compact convolutional neural network for EEG-based brain–computer interfaces , 2016, Journal of neural engineering.

[8]  Na Lu,et al.  A Deep Learning Scheme for Motor Imagery Classification based on Restricted Boltzmann Machines , 2017, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[9]  Sepp Hochreiter,et al.  Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs) , 2015, ICLR.

[10]  Yi-Hung Liu,et al.  Analysis of Electroencephalography Event-Related Desynchronisation and Synchronisation Induced by Lower-Limb Stepping Motor Imagery , 2019 .

[11]  ET-Tahir Zemouri,et al.  Adaptive Time Window for EEG-based Motor Imagery Classification , 2015, IPAC.

[12]  Jose L. Contreras-Vidal,et al.  Design and Optimization of an EEG-Based Brain Machine Interface (BMI) to an Upper-Limb Exoskeleton for Stroke Survivors , 2016, Front. Neurosci..

[13]  Jun Zhang,et al.  Dynamic frequency feature selection based approach for classification of motor imageries , 2016, Comput. Biol. Medicine.

[14]  C. Calautti,et al.  Functional Neuroimaging Studies of Motor Recovery After Stroke in Adults: A Review , 2003, Stroke.

[15]  B He,et al.  Combined rTMS and virtual reality brain–computer interface training for motor recovery after stroke , 2018, Journal of neural engineering.

[16]  Urbano Nunes,et al.  Playing Tetris with non-invasive BCI , 2011, 2011 IEEE 1st International Conference on Serious Games and Applications for Health (SeGAH).

[17]  Müjdat Çetin,et al.  A latent discriminative model-based approach for classification of imaginary motor tasks from EEG data , 2012, Journal of neural engineering.

[18]  Jin-fu Yang,et al.  Adaptive Feature Extraction of Motor Imagery EEG with Optimal Wavelet Packets and SE-Isomap , 2017 .

[19]  Zied Tayeb,et al.  Decoding of motor imagery movements from EEG signals using SpiNNaker neuromorphic hardware , 2017, 2017 8th International IEEE/EMBS Conference on Neural Engineering (NER).

[20]  A. Zabidi,et al.  Short-time Fourier Transform analysis of EEG signal generated during imagined writing , 2012, 2012 International Conference on System Engineering and Technology (ICSET).

[21]  Cuntai Guan,et al.  Transcranial direct current stimulation and EEG-based motor imagery BCI for upper limb stroke rehabilitation , 2012, 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[22]  Bin He,et al.  EEG Source Imaging Enhances the Decoding of Complex Right-Hand Motor Imagery Tasks , 2016, IEEE Transactions on Biomedical Engineering.

[23]  Hui Wang,et al.  A multi-class EEG-based BCI classification using multivariate empirical mode decomposition based filtering and Riemannian geometry , 2018, Expert Syst. Appl..

[24]  Girijesh Prasad,et al.  Mu and beta rhythm modulations in motor imagery related post-stroke EEG: a study under BCI framework for post-stroke rehabilitation , 2010, BMC Neuroscience.

[25]  Isabelle Bloch,et al.  Time-frequency selection in two bipolar channels for improving the classification of motor imagery EEG , 2012, 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[26]  He-Zuo Lü,et al.  Cyclosporin A increases recovery after spinal cord injury but does not improve myelination by oligodendrocyte progenitor cell transplantation , 2010, BMC Neuroscience.

[27]  A Naga Niranjani,et al.  Motor imagery signal classification using spiking neural network , 2017, 2017 International Conference on Intelligent Sustainable Systems (ICISS).

[28]  Pheng-Ann Heng,et al.  Robust Support Matrix Machine for Single Trial EEG Classification , 2018, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[29]  Ridha Djemal,et al.  Three-Class EEG-Based Motor Imagery Classification Using Phase-Space Reconstruction Technique , 2016, Brain sciences.

[30]  Romain Tavenard,et al.  Data Augmentation for Time Series Classification using Convolutional Neural Networks , 2016 .

[31]  Cuntai Guan,et al.  Filter Bank Common Spatial Pattern (FBCSP) in Brain-Computer Interface , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[32]  M. Doppelmayr,et al.  Kinesthetic motor imagery training modulates frontal midline theta during imagination of a dart throw. , 2016, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[33]  J. Wolpaw,et al.  Mu and Beta Rhythm Topographies During Motor Imagery and Actual Movements , 2004, Brain Topography.

[34]  Bijaya K. Panigrahi,et al.  A comparative study of wavelet families for EEG signal classification , 2011, Neurocomputing.

[35]  Rana Fayyaz Ahmad,et al.  Feature extraction and classification for EEG signals using wavelet transform and machine learning techniques , 2015, Australasian Physical & Engineering Sciences in Medicine.

[36]  Cuntai Guan,et al.  Regularizing Common Spatial Patterns to Improve BCI Designs: Unified Theory and New Algorithms , 2011, IEEE Transactions on Biomedical Engineering.

[37]  Mohammed Yeasin,et al.  Learning Representations from EEG with Deep Recurrent-Convolutional Neural Networks , 2015, ICLR.

[38]  Cuntai Guan,et al.  Filter Bank Common Spatial Pattern Algorithm on BCI Competition IV Datasets 2a and 2b , 2012, Front. Neurosci..

[39]  Fabien Lotte,et al.  Signal Processing Approaches to Minimize or Suppress Calibration Time in Oscillatory Activity-Based Brain–Computer Interfaces , 2015, Proceedings of the IEEE.

[40]  Girijesh Prasad,et al.  Bispectrum-based feature extraction technique for devising a practical brain–computer interface , 2011, Journal of neural engineering.

[41]  Ronan Boulic,et al.  A Deep Learning Approach for Classification of Reaching Targets from EEG Images , 2017, 2017 30th SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI).

[42]  Youjun Li,et al.  Human Emotion Recognition with Electroencephalographic Multidimensional Features by Hybrid Deep Neural Networks , 2017 .

[43]  Shuicheng Yan,et al.  Parallel convolutional-linear neural network for motor imagery classification , 2015, 2015 23rd European Signal Processing Conference (EUSIPCO).

[44]  Cuntai Guan,et al.  Motor imagery BCI for upper limb stroke rehabilitation: An evaluation of the EEG recordings using coherence analysis , 2013, 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[45]  Cuntai Guan,et al.  Mutual information-based selection of optimal spatial-temporal patterns for single-trial EEG-based BCIs , 2012, Pattern Recognit..

[46]  Frank Kirchner,et al.  Intrinsic interactive reinforcement learning – Using error-related potentials for real world human-robot interaction , 2017, Scientific Reports.

[47]  Bin He,et al.  Noninvasive Electroencephalogram Based Control of a Robotic Arm for Reach and Grasp Tasks , 2016, Scientific Reports.

[48]  Anton Nijholt,et al.  Affective Pacman: A Frustrating Game for Brain-Computer Interface Experiments , 2009, INTETAIN.

[49]  Hui Wang,et al.  An empirical mode decomposition based filtering method for classification of motor-imagery EEG signals for enhancing brain-computer interface , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[50]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[51]  Ugur Halici,et al.  A novel deep learning approach for classification of EEG motor imagery signals , 2017, Journal of neural engineering.

[52]  Byunghan Lee,et al.  Deep learning in bioinformatics , 2016, Briefings Bioinform..

[53]  Lina Yao,et al.  Converting Your Thoughts to Texts: Enabling Brain Typing via Deep Feature Learning of EEG Signals , 2017, 2018 IEEE International Conference on Pervasive Computing and Communications (PerCom).

[54]  Yan Liu,et al.  Data Augmentation for EEG-Based Emotion Recognition with Deep Convolutional Neural Networks , 2018, MMM.

[55]  Trieu Phat Luu,et al.  Brain–machine interfaces for controlling lower-limb powered robotic systems , 2018, Journal of neural engineering.

[56]  Wolfram Burgard,et al.  Deep learning with convolutional neural networks for EEG decoding and visualization , 2017, Human brain mapping.

[57]  Girijesh Prasad,et al.  Adaptive learning with covariate shift-detection for motor imagery-based brain–computer interface , 2015, Soft Computing.

[58]  Kay Chen Tan,et al.  A brain-inspired spiking neural network model with temporal encoding and learning , 2014, Neurocomputing.

[59]  Robert Riener,et al.  The Cybathlon promotes the development of assistive technology for people with physical disabilities , 2016, Journal of NeuroEngineering and Rehabilitation.

[60]  Huosheng Hu,et al.  A Self-Paced Motor Imagery Based Brain-Computer Interface for Robotic Wheelchair Control , 2011, Clinical EEG and neuroscience.

[61]  A. Prochazka,et al.  Wavelet transform use for feature extraction and EEG signal segments classification , 2008, 2008 3rd International Symposium on Communications, Control and Signal Processing.

[62]  Lina Yao,et al.  Intent Recognition in Smart Living Through Deep Recurrent Neural Networks , 2017, ICONIP.

[63]  Danny Wee-Kiat Ng,et al.  Development of an Autonomous BCI Wheelchair , 2014, 2014 IEEE Symposium on Computational Intelligence in Brain Computer Interfaces (CIBCI).

[64]  Lina Yao,et al.  Cascade and Parallel Convolutional Recurrent Neural Networks on EEG-based Intention Recognition for Brain Computer Interface , 2017, AAAI.

[65]  John A. Rogers,et al.  Inorganic Materials and Assembly Techniques for Flexible and Stretchable Electronics , 2015, Proceedings of the IEEE.