A review of feature extraction and classification algorithms for image RSVP based BCI

In this chapter, we introduce an architecture for rapid serial visual presentation (RSVP)-based brain-computer interface (BCI) systems that use electroencephalography (EEG). Hereafter, we will refer to the coupling of the RSVP protocol with EEG to support a target-search BCI as RSVP-EEG. Our focus in this chapter is on a review of feature extraction and classification algorithms applied in RSVP-EEG development. We briefly present the commonly deployed algorithms and describe their properties based on the literature. We conclude with a discussion on the future trajectory of this exciting branch of BCI research.

[1]  Nikos Komodakis,et al.  Learning to compare image patches via convolutional neural networks , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Joachim M. Buhmann,et al.  The Balanced Accuracy and Its Posterior Distribution , 2010, 2010 20th International Conference on Pattern Recognition.

[3]  Alan F. Smeaton,et al.  Exploring EEG for Object Detection and Retrieval , 2015, ICMR.

[4]  Assaf B. Spanier,et al.  Spatiotemporal Representations of Rapid Visual Target Detection: A Single-Trial EEG Classification Algorithm , 2014, IEEE Transactions on Biomedical Engineering.

[5]  Sebastian Ruder,et al.  An overview of gradient descent optimization algorithms , 2016, Vestnik komp'iuternykh i informatsionnykh tekhnologii.

[6]  Touradj Ebrahimi,et al.  An efficient P300-based brain–computer interface for disabled subjects , 2008, Journal of Neuroscience Methods.

[7]  Yufei Huang,et al.  A Deep Learning method for classification of images RSVP events with EEG data , 2013, 2013 IEEE Global Conference on Signal and Information Processing.

[8]  D. Erdogmus,et al.  Boosting Linear Logistic Regression for Single Trial ERP Detection in Rapid Serial Visual Presentation Tasks , 2006, 2006 International Conference of the IEEE Engineering in Medicine and Biology Society.

[9]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[10]  T. Sejnowski,et al.  Removing electroencephalographic artifacts by blind source separation. , 2000, Psychophysiology.

[11]  Scott Makeig,et al.  ERP Features and EEG Dynamics : An ICA Perspective , 2009 .

[12]  Federico Girosi,et al.  Support Vector Machines: Training and Applications , 1997 .

[13]  Brent Lance,et al.  EEGNet: a compact convolutional neural network for EEG-based brain–computer interfaces , 2016, Journal of neural engineering.

[14]  Charles W. Anderson,et al.  Classification of EEG during imagined mental tasks by forecasting with Elman Recurrent Neural Networks , 2011, The 2011 International Joint Conference on Neural Networks.

[15]  Miguel P. Eckstein,et al.  Single-Trial Classification of Event-Related Potentials in Rapid Serial Visual Presentation Tasks Using Supervised Spatial Filtering , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[16]  P. Sajda,et al.  Spatiotemporal Linear Decoding of Brain State , 2008, IEEE Signal Processing Magazine.

[17]  Hecht-Nielsen.  Theory of the backpropagation neural network , 1989 .

[18]  Jonathon Shlens,et al.  A Tutorial on Principal Component Analysis , 2014, ArXiv.

[19]  Erkki Oja,et al.  Independent component analysis: algorithms and applications , 2000, Neural Networks.

[20]  Alan F. Smeaton,et al.  Overview of NTCIR-13 NAILS Task , 2017, NTCIR.

[21]  Guillaume Gibert,et al.  xDAWN Algorithm to Enhance Evoked Potentials: Application to Brain–Computer Interface , 2009, IEEE Transactions on Biomedical Engineering.

[22]  Elif Derya Übeyli,et al.  Recurrent neural networks employing Lyapunov exponents for EEG signals classification , 2005, Expert Syst. Appl..

[23]  S. Puthusserypady,et al.  Multilayer perceptrons for the classification of brain computer interface data , 2005, Proceedings of the IEEE 31st Annual Northeast Bioengineering Conference, 2005..

[24]  Amelia J. Solon,et al.  Deep Learning Approaches for P300 Classification in Image Triage: Applications to the NAILS Task , 2017, NTCIR.

[25]  J. Friedman.  Regularized Discriminant Analysis , 1989 .

[26]  Xiaoping Li,et al.  Common Spatio-Temporal Pattern for Single-Trial Detection of Event-Related Potential in Rapid Serial Visual Presentation Triage , 2011, IEEE Transactions on Biomedical Engineering.

[27]  Anil K. Jain,et al.  Statistical Pattern Recognition: A Review , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[28]  Lucas C. Parra,et al.  High-throughput image search via single-trial event detection in a rapid serial visual presentation task , 2003, First International IEEE EMBS Conference on Neural Engineering, 2003. Conference Proceedings..

[29]  P. König,et al.  Combining EEG and eye tracking: identification, characterization, and correction of eye movement artifacts in electroencephalographic data , 2012, Front. Hum. Neurosci..

[30]  Clemens Brunner,et al.  Dimensionality Reduction and Channel Selection of Motor Imagery Electroencephalographic Data , 2009, Comput. Intell. Neurosci..

[31]  J. Polich.  Updating P300: An integrative theory of P3a and P3b , 2007, Clinical Neurophysiology.

[32]  Alan F. Smeaton,et al.  Optimising the number of channels in EEG-augmented image search , 2011, BCS HCI.

[33]  Klaus-Robert Müller,et al.  The LDA beamformer: Optimal estimation of ERP source time series using linear discriminant analysis , 2016, NeuroImage.

[34]  D. M. Titterington,et al.  Do unbalanced data have a negative effect on LDA? , 2008, Pattern Recognit..

[35]  Marina Schmid,et al.  An Introduction To The Event Related Potential Technique , 2016 .

[36]  Sepideh Hajipour Sardouie,et al.  Denoising of interictal EEG signals using ICA and Time Varying AR modeling , 2014, 2014 21th Iranian Conference on Biomedical Engineering (ICBME).

[37]  Ah Chung Tsoi,et al.  Face recognition: a convolutional neural-network approach , 1997, IEEE Trans. Neural Networks.

[38]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.

[39]  Yoshua Bengio,et al.  Generative Adversarial Networks , 2014, ArXiv.

[40]  M Congedo,et al.  A review of classification algorithms for EEG-based brain–computer interfaces , 2007, Journal of neural engineering.

[41]  Heesung Kwon,et al.  Single-trial EEG RSVP classification using convolutional neural networks , 2016, Defense + Security.

[42]  Justin M. Ales,et al.  The steady-state visual evoked potential in vision research: A review. , 2015, Journal of vision.

[43]  Yufei Huang,et al.  Characterization and Robust Classification of EEG Signal from Image RSVP Events with Independent Time-Frequency Features , 2012, PloS one.

[44]  David G. Stork,et al.  Pattern Classification (2nd ed.) , 1999 .

[45]  Lucas C. Parra,et al.  Single-trial analysis of EEG during rapid visual discrimination: enabling cortically-coupled computer vision , 2007 .

[46]  K.-R. Muller,et al.  Optimizing Spatial filters for Robust EEG Single-Trial Analysis , 2008, IEEE Signal Processing Magazine.

[47]  Govinda R. Poudel,et al.  Comparison of beamformers for EEG source signal reconstruction , 2014, Biomed. Signal Process. Control..

[48]  Geoffrey E. Hinton.  A Practical Guide to Training Restricted Boltzmann Machines , 2012, Neural Networks: Tricks of the Trade.

[49]  Alan F. Smeaton,et al.  An investigation of triggering approaches for the rapid serial visual presentation paradigm in brain computer interfacing , 2016, 2016 27th Irish Signals and Systems Conference (ISSC).

[50]  Xiaolin Hu,et al.  Recurrent convolutional neural network for object recognition , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[51]  C. J. Tierra-Criollo,et al.  Optimum principal components for spatial filtering of EEG to detect imaginary movement by coherence , 2010, 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology.

[52]  Brendan Z. Allison,et al.  P300 brain computer interface: current challenges and emerging trends , 2012, Front. Neuroeng..

[53]  Charles Elkan,et al.  The Foundations of Cost-Sensitive Learning , 2001, IJCAI.

[54]  Anthony J. Ries,et al.  Best practice for single-trial detection of event-related potentials: Application to brain-computer interfaces. , 2017, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[55]  J. Hanley,et al.  The meaning and use of the area under a receiver operating characteristic (ROC) curve. , 1982, Radiology.

[56]  Elif Derya Übeyli.  Analysis of EEG signals by implementing eigenvector methods/recurrent neural networks , 2009, Digit. Signal Process..

[57]  Geoffrey E. Hinton.  Deep belief networks , 2009, Scholarpedia.

[58]  Henri Begleiter,et al.  Evoked Brain Potentials and Behavior , 1979, The Downstate Series of Research in Psychiatry and Psychology.

[59]  Gabriel Curio,et al.  MACHINE LEARNING TECHNIQUES FOR BRAIN-COMPUTER INTERFACES , 2004 .

[60]  Yijun Wang,et al.  Common Spatial Pattern Method for Channel Selelction in Motor Imagery Based Brain-computer Interface , 2005, 2005 IEEE Engineering in Medicine and Biology 27th Annual Conference.

[61]  Benjamin Blankertz,et al.  Towards a Cure for BCI Illiteracy , 2009, Brain Topography.

[62]  Ran Manor,et al.  Convolutional Neural Network for Multi-Category Rapid Serial Visual Presentation BCI , 2015, Front. Comput. Neurosci..

[63]  Misha Pavel,et al.  A framework for rapid visual image search using single-trial brain evoked responses , 2011, Neurocomputing.

[64]  Lukás Burget,et al.  Recurrent neural network based language model , 2010, INTERSPEECH.

[65]  Michael X Cohen,et al.  Analyzing Neural Time Series Data: Theory and Practice , 2014 .

[66]  Ronald A. Cohen,et al.  The N2-P3 complex of the evoked potential and human performance , 1988 .

[67]  Jianfeng Gao,et al.  Deep Learning for Web Search and Natural Language Processing , 2015 .

[68]  P. Sajda,et al.  Cortically coupled computer vision for rapid image search , 2006, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[69]  Hang Li,et al.  Convolutional Neural Network Architectures for Matching Natural Language Sentences , 2014, NIPS.

[70]  David J. C. MacKay,et al.  Bayesian Interpolation , 1992, Neural Computation.

[71]  Gustavo Carneiro,et al.  The Segmentation of the Left Ventricle of the Heart From Ultrasound Data Using Deep Learning Architectures and Derivative-Based Search Methods , 2012, IEEE Transactions on Image Processing.

[72]  N. Bigdely-Shamlo,et al.  Brain Activity-Based Image Classification From Rapid Serial Visual Presentation , 2008, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[73]  Alan F. Smeaton,et al.  An EEG Image-search dataset: a first-of-its-kind in IR/IIR. NAILS: neurally augmented image labelling strategies , 2017 .

[74]  Albert Nigrin,et al.  Neural networks for pattern recognition , 1993 .