A time-frequency convolutional neural network for the offline classification of steady-state visual evoked potential responses

A new convolutional neural network architecture is presented. It includes the fast Fourier transform between two hidden layers to switch the signal analysis from the time domain to the frequency domain inside the network. This technique allows the signal classification without any special pre-processing and uses knowledge from the problem in the network topology. The first step allows the creation of different spatial and time filters. The second step is dedicated to the signal transformation in the frequency domain. The last step is the classification. The system is tested offline on the classification of EEG signals that contain steady-state visual evoked potential (SSVEP) responses. The mean recognition rate of the classification of five different types of SSVEP response is 95.61% on a time segment length of 1s. The proposed strategy outperforms other classical neural network architecures.

[1]  Alain Rakotomamonjy,et al.  BCI Competition III: Dataset II- Ensemble of SVMs for BCI P300 Speller , 2008, IEEE Transactions on Biomedical Engineering.

[2]  Ivan Volosyak,et al.  Multiple Channel Detection of Steady-State Visual Evoked Potentials for Brain-Computer Interfaces , 2007, IEEE Transactions on Biomedical Engineering.

[3]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[4]  Clemens Brunner,et al.  Spatial filtering and selection of optimized components in four class motor imagery EEG data using independent components analysis , 2007, Pattern Recognit. Lett..

[5]  Reinhold Scherer,et al.  Steady-state visual evoked potential (SSVEP)-based communication: impact of harmonic frequency components , 2005, Journal of neural engineering.

[6]  Armando Barreto,et al.  Neural network classification of spatio-temporal EEG readiness potentials , 1996, Proceedings of the 1996 Fifteenth Southern Biomedical Engineering Conference.

[7]  Y. LeCun,et al.  Learning methods for generic object recognition with invariance to pose and lighting , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[8]  Terrence J. Sejnowski,et al.  Toward Brain-Computer Interfacing (Neural Information Processing) , 2007 .

[9]  Andrzej Cichocki,et al.  Fully Online Multicommand Brain-Computer Interface with Visual Neurofeedback Using SSVEP Paradigm , 2007, Comput. Intell. Neurosci..

[10]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[11]  Grgoire Montavon,et al.  Neural Networks: Tricks of the Trade , 2012, Lecture Notes in Computer Science.

[12]  Shangkai Gao,et al.  A practical VEP-based brain-computer interface. , 2006, IEEE transactions on neural systems and rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[13]  R. Silberstein,et al.  Steady-state visual evoked potentials and travelling waves , 2000, Clinical Neurophysiology.

[14]  Hubert Cecotti,et al.  Convolutional Neural Network with embedded Fourier Transform for EEG classification , 2008, 2008 19th International Conference on Pattern Recognition.

[15]  Klaus-Robert Müller,et al.  Classifying Single Trial EEG: Towards Brain Computer Interfacing , 2001, NIPS.

[16]  Gernot R. Müller-Putz,et al.  Control of an Electrical Prosthesis With an SSVEP-Based BCI , 2008, IEEE Transactions on Biomedical Engineering.

[17]  Joydeep Ghosh,et al.  HMMs and Coupled HMMs for multi-channel EEG classification , 2002, Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290).

[18]  Motoaki Kawanabe,et al.  Invariant Common Spatial Patterns: Alleviating Nonstationarities in Brain-Computer Interfacing , 2007, NIPS.

[19]  A. Graser,et al.  Low level control in a semi-autonomous rehabilitation robotic system via a Brain-Computer Interface , 2007, 2007 IEEE 10th International Conference on Rehabilitation Robotics.

[20]  Charles W. Anderson,et al.  Determining Mental State from EEG Signals Using Parallel Implementations of Neural Networks , 1995, Sci. Program..

[21]  Jirí Benes,et al.  On neural networks , 1990, Kybernetika.

[22]  Kazuyuki Aihara,et al.  Adapting Spatial Filter Methods for Nonstationary BCIs , 2006 .

[23]  T. Felzer,et al.  Analyzing EEG signals using the probability estimating guarded neural classifier , 2003, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[24]  G. Pfurtscheller,et al.  Brain-Computer Interfaces for Communication and Control. , 2011, Communications of the ACM.

[25]  L.J. Trejo,et al.  Brain-computer interfaces for 1-D and 2-D cursor control: designs using volitional control of the EEG spectrum or steady-state visual evoked potentials , 2006, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[26]  Gert Pfurtscheller,et al.  EEG-based Brain-Computer Interface Using Subject-Specific Spatial Filters , 1999, IWANN.

[27]  M Congedo,et al.  A review of classification algorithms for EEG-based brain–computer interfaces , 2007, Journal of neural engineering.

[28]  G Pfurtscheller,et al.  Using time-dependent neural networks for EEG classification. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[29]  Ettore Lettich,et al.  Ten Percent Electrode System for Topographic Studies of Spontaneous and Evoked EEG Activities , 1985 .

[30]  Klaus-Robert Müller,et al.  Efficient BackProp , 2012, Neural Networks: Tricks of the Trade.

[31]  Christa Neuper,et al.  Hidden Markov models for online classification of single trial EEG data , 2001, Pattern Recognit. Lett..

[32]  K.-R. Muller,et al.  The Berlin brain-computer interface: EEG-based communication without subject training , 2006, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[33]  Yoshua Bengio,et al.  Greedy Layer-Wise Training of Deep Networks , 2006, NIPS.

[34]  Gert Pfurtscheller,et al.  Neural network-based predictions of hand movements using simulated and real EEG data , 1995, Neurocomputing.

[35]  Patrice Y. Simard,et al.  Best practices for convolutional neural networks applied to visual document analysis , 2003, Seventh International Conference on Document Analysis and Recognition, 2003. Proceedings..

[36]  K.-R. Muller,et al.  Optimizing Spatial filters for Robust EEG Single-Trial Analysis , 2008, IEEE Signal Processing Magazine.

[37]  Klaus-Robert Müller,et al.  Machine learning for real-time single-trial EEG-analysis: From brain–computer interfacing to mental state monitoring , 2008, Journal of Neuroscience Methods.

[38]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.