Single-trial analysis of EEG during rapid visual discrimination: enabling cortically-coupled computer vision

We describe our work using linear discrimination of multi-channel electroencephalography for single-trial detection of neural signatures of visual recognition events. We demonstrate the approach as a methodology for relating neural variability to response variability, describing studies for response accuracy and response latency during visual target detection. We then show how the approach can be utilized to construct a novel type of brain-computer interface, which we term cortically-coupled computer vision. In this application, a large database of images is triaged using the detected neural signatures. We show how ‘corticaltriaging’ improves image search over a strictly behavioral response.

[1]  Helge J. Ritter,et al.  BCI competition 2003-data set IIb: support vector machines for the P300 speller paradigm , 2004, IEEE Transactions on Biomedical Engineering.

[2]  D J McFarland,et al.  An EEG-based brain-computer interface for cursor control. , 1991, Electroencephalography and clinical neurophysiology.

[3]  Lucas C. Parra,et al.  Recipes for the linear analysis of EEG , 2005, NeuroImage.

[4]  J. Movshon,et al.  The analysis of visual motion: a comparison of neuronal and psychophysical performance , 1992, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[5]  D. Ruchkin,et al.  P300 in patients with unilateral temporal lobectomies: the effects of reduced stimulus quality. , 1991, Psychophysiology.

[6]  K. H. Britten,et al.  A relationship between behavioral choice and the visual responses of neurons in macaque MT , 1996, Visual Neuroscience.

[7]  N. Kanwisher,et al.  Stages of processing in face perception: an MEG study , 2002, Nature Neuroscience.

[8]  R. Quick A vector-magnitude model of contrast detection , 2004, Kybernetik.

[9]  K. H. Britten,et al.  Neuronal correlates of a perceptual decision , 1989, Nature.

[10]  T. Allison,et al.  Differential Sensitivity of Human Visual Cortex to Faces, Letterstrings, and Textures: A Functional Magnetic Resonance Imaging Study , 1996, The Journal of Neuroscience.

[11]  D. M. Green,et al.  Signal detection theory and psychophysics , 1966 .

[12]  Bruno A Olshausen,et al.  Timecourse of neural signatures of object recognition. , 2003, Journal of vision.

[13]  Talma Hendler,et al.  Eccentricity Bias as an Organizing Principle for Human High-Order Object Areas , 2002, Neuron.

[14]  Yehoshua Y. Zeevi,et al.  Quasi Maximum Likelihood Blind Deconvolution of Images Using Optimal Sparse Representations , 2003 .

[15]  Paul Sajda,et al.  Learning discrimination trajectories in EEG sensor space: application to inferring task difficulty. , 2006, Journal of neural engineering.

[16]  Xiaorong Gao,et al.  Design and implementation of a brain-computer interface with high transfer rates , 2002, IEEE Transactions on Biomedical Engineering.

[17]  G. Pfurtscheller Functional Topography During Sensorimotor Activation Studied with Event‐Related Desynchronization Mapping , 1989, Journal of clinical neurophysiology : official publication of the American Electroencephalographic Society.

[18]  Gert Pfurtscheller,et al.  Motor imagery and direct brain-computer communication , 2001, Proc. IEEE.

[19]  Leslie G. Ungerleider,et al.  A general mechanism for perceptual decision-making in the human brain , 2004, Nature.

[20]  P. Sajda,et al.  Temporal characterization of the neural correlates of perceptual decision making in the human brain. , 2006, Cerebral cortex.

[21]  John J. Foxe,et al.  Visual spatial attention tracking using high-density SSVEP data for independent brain-computer communication , 2005, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[22]  Lucas C. Parra,et al.  Cortical origins of response time variability during rapid discrimination of visual objects , 2005, NeuroImage.

[23]  Bruno Rossion,et al.  Early lateralization and orientation tuning for face, word, and object processing in the visual cortex , 2003, NeuroImage.

[24]  S. Makeig,et al.  EEG changes accompanying learned regulation of 12-Hz EEG activity , 2003, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[25]  P. Fldik,et al.  The Speed of Sight , 2001, Journal of Cognitive Neuroscience.

[26]  S. Thorpe,et al.  The Time Course of Visual Processing: From Early Perception to Decision-Making , 2001, Journal of Cognitive Neuroscience.

[27]  R. Hess,et al.  What causes non-monotonic tuning of fMRI response to noisy images? , 2002, Current Biology.

[28]  Klaus-Robert Müller,et al.  Spatio-spectral filters for improving the classification of single trial EEG , 2005, IEEE Transactions on Biomedical Engineering.

[29]  Kuno Kirschfeld,et al.  Cortical visual processing is temporally dispersed by luminance in human subjects , 1999, Neuroscience Letters.

[30]  Denis Fize,et al.  Speed of processing in the human visual system , 1996, Nature.

[31]  Jonathan R Wolpaw,et al.  Control of a two-dimensional movement signal by a noninvasive brain-computer interface in humans. , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[32]  T. Sejnowski,et al.  Electroencephalographic brain dynamics following visual targets requiring manual responses , 2022 .