MsCNN: A Deep Learning Framework for P300-Based Brain–Computer Interface Speller

In this paper, a novel multiscale convolutional neural network (MsCNN) architecture is proposed for P300 based BCI speller. Major limitation of BCI system is that it requires a large number of data to train the system. However, collection of large amount of data from a single subject makes the task tedious for the subject and this will affect the quality of the acquired electroencephalogram (EEG) signal. To cope with this issues, a MsCNN model with transfer learning (MsCNN-TL) technique is proposed in this work for improvement of the P300 based character recognition performance with limited amount of training data. The multi-resolution deep features are extracted from the fully connected layer of the trained MsCNN-TL architecture. These deep features are optimized using Fisher ratio (F-ratio) based feature selection technique and the optimal features are applied to the ensemble of support vector machines (ESVMs) for P300 detection. The proposed method is tested on BCI competition datasets and it achieves better performance compared to the other state-of-the-art techniques for limited training dataset.

[1]  M. Shamsollahi,et al.  Higher order spectral regression discriminant analysis (HOSRDA): A tensor feature reduction method for ERP detection , 2017, Pattern Recognit..

[2]  Ronghua Shang,et al.  Densely Based Multi-Scale and Multi-Modal Fully Convolutional Networks for High-Resolution Remote-Sensing Image Semantic Segmentation , 2019, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[3]  Guy Dove,et al.  Linking Brainwaves to the Brain: An ERP Primer , 2005, Developmental neuropsychology.

[4]  Murat Akcakaya,et al.  Noninvasive Brain–Computer Interfaces for Augmentative and Alternative Communication , 2014, IEEE Reviews in Biomedical Engineering.

[5]  G. Pfurtscheller,et al.  The BCI competition III: validating alternative approaches to actual BCI problems , 2006, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[6]  Rui Zhang,et al.  A BCI-Based Environmental Control System for Patients With Severe Spinal Cord Injuries , 2017, IEEE Transactions on Biomedical Engineering.

[7]  Kiseon Kim,et al.  Multiple kernel learning based on three discriminant features for a P300 speller BCI , 2017, Neurocomputing.

[8]  Francisco Herrera,et al.  A Review on Ensembles for the Class Imbalance Problem: Bagging-, Boosting-, and Hybrid-Based Approaches , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[9]  Wei Wu,et al.  Deep learning based on Batch Normalization for P300 signal detection , 2018, Neurocomputing.

[10]  Bernhard Schölkopf,et al.  Transfer Learning in Brain-Computer Interfaces , 2015, IEEE Computational Intelligence Magazine.

[11]  Alain Rakotomamonjy,et al.  BCI Competition III: Dataset II- Ensemble of SVMs for BCI P300 Speller , 2008, IEEE Transactions on Biomedical Engineering.

[12]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[13]  E. Donchin,et al.  Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials. , 1988, Electroencephalography and clinical neurophysiology.

[14]  Fabien Lotte,et al.  Signal Processing Approaches to Minimize or Suppress Calibration Time in Oscillatory Activity-Based Brain–Computer Interfaces , 2015, Proceedings of the IEEE.

[15]  Brendan Z. Allison,et al.  P300 brain computer interface: current challenges and emerging trends , 2012, Front. Neuroeng..

[16]  Yuanqing Li,et al.  An Online Semi-supervised Brain–Computer Interface , 2013, IEEE Transactions on Biomedical Engineering.

[17]  Sourav Kundu,et al.  P300 based character recognition using convolutional neural network and support vector machine , 2020, Biomed. Signal Process. Control..

[18]  D. Friedman,et al.  The novelty P3: an event-related brain potential (ERP) sign of the brain's evaluation of novelty , 2001, Neuroscience & Biobehavioral Reviews.

[19]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[20]  Klaus-Robert Müller,et al.  The BCI competition 2003: progress and perspectives in detection and discrimination of EEG single trials , 2004, IEEE Transactions on Biomedical Engineering.

[21]  Shuicheng Yan,et al.  Learning Temporal Information for Brain-Computer Interface Using Convolutional Neural Networks , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[22]  Hubert Cecotti,et al.  Convolutional Neural Networks for P300 Detection with Application to Brain-Computer Interfaces , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Ghanahshyam B Kshirsagar,et al.  Improving Performance of Devanagari Script Input-Based P300 Speller Using Deep Learning , 2019, IEEE Transactions on Biomedical Engineering.

[24]  S. Coyle,et al.  Brain–computer interfaces: a review , 2003 .

[25]  Yuanqing Li,et al.  Grouped Automatic Relevance Determination and Its Application in Channel Selection for P300 BCIs , 2015, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[26]  Trevor Darrell,et al.  DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition , 2013, ICML.

[27]  Helge J. Ritter,et al.  BCI competition 2003-data set IIb: support vector machines for the P300 speller paradigm , 2004, IEEE Transactions on Biomedical Engineering.

[28]  Athanasios V. Vasilakos,et al.  Brain computer interface: control signals review , 2017, Neurocomputing.

[29]  Samit Ari,et al.  P300 Detection with Brain–Computer Interface Application Using PCA and Ensemble of Weighted SVMs , 2018 .

[30]  Sourav Kundu,et al.  P300 based character recognition using sparse autoencoder with ensemble of SVMs , 2019, Biocybernetics and Biomedical Engineering.

[31]  Dinggang Shen,et al.  Multi-Channel 3D Deep Feature Learning for Survival Time Prediction of Brain Tumor Patients Using Multi-Modal Neuroimages , 2019, Scientific Reports.

[32]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[33]  Ugur Halici,et al.  A novel deep learning approach for classification of EEG motor imagery signals , 2017, Journal of neural engineering.

[34]  Weihua Li,et al.  Multisensor Feature Fusion for Bearing Fault Diagnosis Using Sparse Autoencoder and Deep Belief Network , 2017, IEEE Transactions on Instrumentation and Measurement.

[35]  Vladimir Bostanov,et al.  BCI competition 2003-data sets Ib and IIb: feature extraction from event-related brain potentials with the continuous wavelet transform and the t-value scalogram , 2004, IEEE Transactions on Biomedical Engineering.