Convolutional Neural Networks for P300 Detection with Application to Brain-Computer Interfaces

A Brain-Computer Interface (BCI) is a specific type of human-computer interface that enables the direct communication between human and computers by analyzing brain measurements. Oddball paradigms are used in BCI to generate event-related potentials (ERPs), like the P300 wave, on targets selected by the user. A P300 speller is based on this principle, where the detection of P300 waves allows the user to write characters. The P300 speller is composed of two classification problems. The first classification is to detect the presence of a P300 in the electroencephalogram (EEG). The second one corresponds to the combination of different P300 responses for determining the right character to spell. A new method for the detection of P300 waves is presented. This model is based on a convolutional neural network (CNN). The topology of the network is adapted to the detection of P300 waves in the time domain. Seven classifiers based on the CNN are proposed: four single classifiers with different features set and three multiclassifiers. These models are tested and compared on the Data set II of the third BCI competition. The best result is obtained with a multiclassifier solution with a recognition rate of 95.5 percent, without channel selection before the classification. The proposed approach provides also a new way for analyzing brain activities due to the receptive field of the CNN models.

[1]  U. Hoffmann,et al.  Application of the evidence framework to brain-computer interfaces , 2004, The 26th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[2]  E Donchin,et al.  The mental prosthesis: assessing the speed of a P300-based brain-computer interface. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[3]  Y. LeCun,et al.  Learning methods for generic object recognition with invariance to pose and lighting , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[4]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[5]  M Congedo,et al.  A review of classification algorithms for EEG-based brain–computer interfaces , 2007, Journal of neural engineering.

[6]  G Pfurtscheller,et al.  Using time-dependent neural networks for EEG classification. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[7]  E. W. Sellers,et al.  Toward enhanced P300 speller performance , 2008, Journal of Neuroscience Methods.

[8]  Alain Rakotomamonjy,et al.  BCI Competition III: Dataset II- Ensemble of SVMs for BCI P300 Speller , 2008, IEEE Transactions on Biomedical Engineering.

[9]  Yoshua Bengio,et al.  Scaling learning algorithms towards AI , 2007 .

[10]  Joydeep Ghosh,et al.  HMMs and Coupled HMMs for multi-channel EEG classification , 2002, Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290).

[11]  J. van Leeuwen,et al.  Neural Networks: Tricks of the Trade , 2002, Lecture Notes in Computer Science.

[12]  Patrice Y. Simard,et al.  High Performance Convolutional Neural Networks for Document Processing , 2006 .

[13]  Charles W. Anderson,et al.  Determining Mental State from EEG Signals Using Parallel Implementations of Neural Networks , 1995, Sci. Program..

[14]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[15]  T. Felzer,et al.  Analyzing EEG signals using the probability estimating guarded neural classifier , 2003, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[16]  G. Pfurtscheller,et al.  Brain-Computer Interfaces for Communication and Control. , 2011, Communications of the ACM.

[17]  G.F. Inbar,et al.  An improved P300-based brain-computer interface , 2005, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[18]  Christa Neuper,et al.  Hidden Markov models for online classification of single trial EEG data , 2001, Pattern Recognit. Lett..

[19]  Klaus-Robert Müller,et al.  Machine learning for real-time single-trial EEG-analysis: From brain–computer interfacing to mental state monitoring , 2008, Journal of Neuroscience Methods.

[20]  H. Flor,et al.  A spelling device for the paralysed , 1999, Nature.

[21]  Klaus-Robert Müller,et al.  Efficient BackProp , 2012, Neural Networks: Tricks of the Trade.

[22]  Hubert Cecotti,et al.  Time Delay Neural Network with Fourier transform for multiple channel detection of Steady-State Visual Evoked Potentials for Brain-Computer Interfaces , 2008, 2008 16th European Signal Processing Conference.

[23]  Gert Pfurtscheller,et al.  Neural network based classification of single-trial EEG data , 1993, Artif. Intell. Medicine.

[24]  Laurent Bougrain,et al.  Averaging techniques for single-trial analysis of oddball event-related potentials , 2008 .

[25]  Gabriel Curio,et al.  MACHINE LEARNING TECHNIQUES FOR BRAIN-COMPUTER INTERFACES , 2004 .

[26]  H. Lüders,et al.  American Electroencephalographic Society Guidelines for Standard Electrode Position Nomenclature , 1991, Journal of clinical neurophysiology : official publication of the American Electroencephalographic Society.

[27]  Jonathan R Wolpaw,et al.  Brain–computer interface systems: progress and prospects , 2007, Expert review of medical devices.

[28]  L. Cohen,et al.  Brain–computer interfaces: communication and restoration of movement in paralysis , 2007, The Journal of physiology.

[29]  N. Birbaumer,et al.  BCI2000: a general-purpose brain-computer interface (BCI) system , 2004, IEEE Transactions on Biomedical Engineering.

[30]  A Kostov,et al.  Parallel man-machine training in development of EEG-based cursor control. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[31]  K.-R. Muller,et al.  The Berlin brain-computer interface: EEG-based communication without subject training , 2006, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[32]  Klaus-Robert Müller,et al.  The BCI competition 2003: progress and perspectives in detection and discrimination of EEG single trials , 2004, IEEE Transactions on Biomedical Engineering.

[33]  Somsak Sukittanon,et al.  Convolutional networks for speech detection , 2004, INTERSPEECH.

[34]  Gert Pfurtscheller,et al.  Neural network-based predictions of hand movements using simulated and real EEG data , 1995, Neurocomputing.

[35]  Jiri Matas,et al.  On Combining Classifiers , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[36]  G. Pfurtscheller,et al.  The BCI competition III: validating alternative approaches to actual BCI problems , 2006, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[37]  Klaus-Robert Müller,et al.  Classifying Single Trial EEG: Towards Brain Computer Interfacing , 2001, NIPS.

[38]  J. Wolpaw,et al.  Toward enhanced P 300 speller performance , 2007 .

[39]  Patrice Y. Simard,et al.  Best practices for convolutional neural networks applied to visual document analysis , 2003, Seventh International Conference on Document Analysis and Recognition, 2003. Proceedings..

[40]  K. Shimohara,et al.  EEG topography recognition by neural networks , 1990, IEEE Engineering in Medicine and Biology Magazine.

[41]  U. Hoffmann,et al.  A Boosting Approach to P300 Detection with Application to Brain-Computer Interfaces , 2005, Conference Proceedings. 2nd International IEEE EMBS Conference on Neural Engineering, 2005..

[42]  C. E. SHANNON,et al.  A mathematical theory of communication , 1948, MOCO.

[43]  E. Donchin,et al.  Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials. , 1988, Electroencephalography and clinical neurophysiology.

[44]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.