Classifying four-category visual objects using multiple ERP components in single-trial ERP

Object categorization using single-trial electroencephalography (EEG) data measured while participants view images has been studied intensively. In previous studies, multiple event-related potential (ERP) components (e.g., P1, N1, P2, and P3) were used to improve the performance of object categorization of visual stimuli. In this study, we introduce a novel method that uses multiple-kernel support vector machine to fuse multiple ERP component features. We investigate whether fusing the potential complementary information of different ERP components (e.g., P1, N1, P2a, and P2b) can improve the performance of four-category visual object classification in single-trial EEGs. We also compare the classification accuracy of different ERP component fusion methods. Our experimental results indicate that the classification accuracy increases through multiple ERP fusion. Additional comparative analyses indicate that the multiple-kernel fusion method can achieve a mean classification accuracy higher than 72 %, which is substantially better than that achieved with any single ERP component feature (55.07 % for the best single ERP component, N1). We compare the classification results with those of other fusion methods and determine that the accuracy of the multiple-kernel fusion method is 5.47, 4.06, and 16.90 % higher than those of feature concatenation, feature extraction, and decision fusion, respectively. Our study shows that our multiple-kernel fusion method outperforms other fusion methods and thus provides a means to improve the classification performance of single-trial ERPs in brain–computer interface research.

[1]  P. Sajda,et al.  Temporal characterization of the neural correlates of perceptual decision making in the human brain. , 2006, Cerebral cortex.

[2]  R. Ratcliff,et al.  Neural Representation of Task Difficulty and Decision Making during Perceptual Categorization: A Timing Diagram , 2006, The Journal of Neuroscience.

[3]  Guillaume Thierry,et al.  Is the N170 sensitive to the human face or to several intertwined perceptual and conceptual factors? , 2007, Nature Neuroscience.

[4]  N. Bigdely-Shamlo,et al.  Brain Activity-Based Image Classification From Rapid Serial Visual Presentation , 2008, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[5]  Desney S. Tan,et al.  Human-aided computing: utilizing implicit human processing to classify images , 2008, CHI.

[6]  Robert Oostenveld,et al.  Identifying Object Categories from Event-Related EEG: Toward Decoding of Conceptual Representations , 2010, PloS one.

[7]  C. Tenke,et al.  Optimizing PCA methodology for ERP component identification and measurement: theoretical rationale and empirical evaluation , 2003, Clinical Neurophysiology.

[8]  Jürgen Kurths,et al.  Order patterns recurrence plots in the analysis of ERP data , 2007, Cognitive Neurodynamics.

[9]  Marcel van Gerven,et al.  Unsupervised Feature Learning Improves Prediction of Human Brain Activity in Response to Natural Images , 2014, PLoS Comput. Biol..

[10]  Daoqiang Zhang,et al.  Multimodal classification of Alzheimer's disease and mild cognitive impairment , 2011, NeuroImage.

[11]  Jian Yang,et al.  Feature fusion: parallel strategy vs. serial strategy , 2003, Pattern Recognit..

[12]  Li Han,et al.  Improving N1 classification by grouping EEG trials with phases of pre-stimulus EEG oscillations , 2015, Cognitive Neurodynamics.

[13]  Tim Curran,et al.  Investigation of changes in EEG complexity during memory retrieval: the effect of midazolam , 2012, Cognitive Neurodynamics.

[14]  Robi Polikar,et al.  An ensemble based data fusion approach for early diagnosis of Alzheimer's disease , 2008, Inf. Fusion.

[15]  Jiacai Zhang,et al.  Bayesian reconstruction of multiscale local contrast images from brain activity , 2013, Journal of Neuroscience Methods.

[16]  Lorenzo Bruzzone,et al.  Distinguishing Concept Categories from Single-Trial Electrophysiological Activity , 2008 .

[17]  Vikas Singh,et al.  MKL for Robust Multi-modality AD Classification , 2009, MICCAI.

[18]  Li Yao,et al.  Combining features from ERP components in single-trial EEG for discriminating four-category visual objects , 2012, Journal of neural engineering.

[19]  Clara D. Martin,et al.  Controlling for interstimulus perceptual variance abolishes N170 face selectivity , 2007, Nature Neuroscience.

[20]  J. Gallant,et al.  Identifying natural images from human brain activity , 2008, Nature.

[21]  Shravan Vasishth,et al.  Towards dynamical system models of language-related brain potentials , 2008, Cognitive Neurodynamics.

[22]  Hans A. Kestler,et al.  Proceedings of Reisensburg 2010 , 2013, Comput. Stat..

[23]  Mingdi Xu,et al.  Dissociation of category versus item priming in face processing: an event-related potential study , 2012, Cognitive Neurodynamics.

[24]  Ke Yu,et al.  The Analytic Bilinear Discrimination of Single-Trial EEG Signals in Rapid Image Triage , 2014, PloS one.

[25]  Zengrong Liu,et al.  Bifurcation dynamics and determination of alternate cell fates in bipotent progenitor cells , 2015, Cognitive Neurodynamics.

[26]  Günther Palm,et al.  On the discovery of events in EEG data utilizing information fusion , 2013, Comput. Stat..

[27]  Claus Bahlmann,et al.  In a Blink of an Eye and a Switch of a Transistor: Cortically Coupled Computer Vision , 2010, Proceedings of the IEEE.

[28]  L. Deouell,et al.  STRUCTURAL ENCODING AND IDENTIFICATION IN FACE PROCESSING: ERP EVIDENCE FOR SEPARATE MECHANISMS , 2000, Cognitive neuropsychology.

[29]  Stefan Haufe,et al.  Single-trial analysis and classification of ERP components — A tutorial , 2011, NeuroImage.

[30]  Masa-aki Sato,et al.  Visual Image Reconstruction from Human Brain Activity using a Combination of Multiscale Local Image Decoders , 2008, Neuron.