Deep Learning Models for EEG-based Rapid Serial Visual Presentation Event Classification

We consider deep learning (DL) for event classification using electroencephalogram (EEG) measurements of brain activities. We proposed HDNN or hierarchical deep neural network, and CNN4EEG, a new convolution neural network (CNN). Both DL models are designed to improve the representation of spatial and local temporal correlations inherent in EEG data. These models were tested for image target prediction in a time-locked rapid serial visual presentation (RSVP) experiment. The performances were compared with the state-of-the-art RSVP classification algorithms including HDCA and XDAWN and with other popular machine learning algorithms. The results show that global spatial local temporal CNN (CNN4EEG) achieved a 13% improvement over the best competing non-DL algorithm and a 9% improvement over canonical CNN for image processing, and a 6% over deep neural network (DNN). Our results suggest that the unique design of temporal and spatial filters in CNN4EEG can improve the representation of EEG data, hence the prediction performance.

[1]  Yann LeCun,et al.  Comparing SVM and convolutional networks for epileptic seizure prediction from intracranial EEG , 2008, 2008 IEEE Workshop on Machine Learning for Signal Processing.

[2]  Geoffrey E. Hinton,et al.  On the importance of initialization and momentum in deep learning , 2013, ICML.

[3]  Tara N. Sainath,et al.  Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research Groups , 2012, IEEE Signal Processing Magazine.

[4]  Claus Bahlmann,et al.  In a Blink of an Eye and a Switch of a Transistor: Cortically Coupled Computer Vision , 2010, Proceedings of the IEEE.

[5]  Geoffrey Zweig,et al.  The microsoft 2016 conversational speech recognition system , 2016, 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[6]  P. Sajda,et al.  Cortically coupled computer vision for rapid image search , 2006, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[7]  N. Bigdely-Shamlo,et al.  Brain Activity-Based Image Classification From Rapid Serial Visual Presentation , 2008, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[8]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Sebastian Stober,et al.  Classifying EEG Recordings of Rhythm Perception , 2014, ISMIR.

[10]  Yoshua. Bengio,et al.  Learning Deep Architectures for AI , 2007, Found. Trends Mach. Learn..

[11]  Ming Yang,et al.  DeepFace: Closing the Gap to Human-Level Performance in Face Verification , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[13]  Dong Yu,et al.  Context-Dependent Pre-Trained Deep Neural Networks for Large-Vocabulary Speech Recognition , 2012, IEEE Transactions on Audio, Speech, and Language Processing.

[14]  Anthony J. Ries,et al.  Stimulus and Response-Locked P3 Activity in a Dynamic Rapid Serial Visual Presentation (RSVP) Task , 2013 .

[15]  Mervyn V. M. Yeo,et al.  Can SVM be used for automatic EEG detection of drowsiness during car driving , 2009 .

[16]  Miguel P. Eckstein,et al.  Single-Trial Classification of Event-Related Potentials in Rapid Serial Visual Presentation Tasks Using Supervised Spatial Filtering , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[17]  Lutz Prechelt,et al.  Automatic early stopping using cross validation: quantifying the criteria , 1998, Neural Networks.

[18]  E Donchin,et al.  Brain-computer interface technology: a review of the first international meeting. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[19]  Yufei Huang,et al.  Characterization and Robust Classification of EEG Signal from Image RSVP Events with Independent Time-Frequency Features , 2012, PloS one.

[20]  Kyungmin Su,et al.  The PREP pipeline: standardized preprocessing for large-scale EEG analysis , 2015, Front. Neuroinform..

[21]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[22]  Guillaume Gibert,et al.  xDAWN Algorithm to Enhance Evoked Potentials: Application to Brain–Computer Interface , 2009, IEEE Transactions on Biomedical Engineering.

[23]  Yi Zheng,et al.  Time Series Classification Using Multi-Channels Deep Convolutional Neural Networks , 2014, WAIM.

[24]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[25]  Sebastian Stober,et al.  Using Convolutional Neural Networks to Recognize Rhythm Stimuli from Electroencephalography Recordings , 2014, NIPS.

[26]  Yufei Huang,et al.  Classification of Imperfectly Time-Locked Image RSVP Events with EEG Device , 2013, Neuroinformatics.

[27]  Yoshua Bengio,et al.  Deep Sparse Rectifier Neural Networks , 2011, AISTATS.