Combining EEG and Eye Tracking: Using Fixation-Locked Potentials in Visual Search

Visual search is a complex task that involves many neural pathways to identify relevant areas of interest within a scene. Humans remain a critical component in visual search tasks, as they can effectively perceive anomalies within complex scenes. However, this task can be challenging, particularly under time pressure. In order to improve visual search training and performance, an objective, process-based measure is needed. Eye tracking technology can be used to drive real-time parsing of EEG recordings, providing an indication of the analysis process. In the current study, eye fixations were used to generate ERPs during a visual search task. Clear differences were observed following performance, suggesting that neurophysiological signatures could be developed to prevent errors in visual search tasks.

[1]  Qiang Ji,et al.  In the Eye of the Beholder: A Survey of Models for Eyes and Gaze , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Leslie G. Ungerleider,et al.  Mechanisms of visual attention in the human cortex. , 2000, Annual review of neuroscience.

[3]  M. Carrasco,et al.  Attention trades off spatial acuity , 2009, Vision Research.

[4]  T. Albright.  On the Perception of Probable Things: Neural Substrates of Associative Memory, Imagery, and Perception , 2012, Neuron.

[5]  Klaus-Robert Müller,et al.  Boosting bit rates in noninvasive EEG single-trial classifications by feature combination and multiclass paradigms , 2004, IEEE Transactions on Biomedical Engineering.

[6]  A. Jacobs,et al.  Coregistration of eye movements and EEG in natural reading: analyses and review. , 2011, Journal of experimental psychology. General.

[7]  Ramesh Srinivasan,et al.  Estimating the spatial Nyquist of the human EEG , 1998 .

[8]  Mariano Sigman,et al.  Fixation-related potentials in visual search: a combined EEG and eye tracking study. , 2012, Journal of vision.

[9]  Neil A. Macmillan,et al.  Detection theory: A user's guide, 2nd ed. , 2005 .

[10]  P. König,et al.  Combining EEG and eye tracking: identification, characterization, and correction of eye movement artifacts in electroencephalographic data , 2012, Front. Hum. Neurosci..

[11]  F W Sharbrough,et al.  Long-term electroencephalographic monitoring for diagnosis and management of seizures. , 1996, Mayo Clinic proceedings.

[12]  A. Treisman,et al.  A feature-integration theory of attention , 1980, Cognitive Psychology.

[13]  Marcus Nyström,et al.  Sampling frequency and eye-tracking measures: how speed affects durations, latencies, and more , 2010 .

[14]  T. Picton,et al.  The N1 wave of the human electric and magnetic response to sound: a review and an analysis of the component structure. , 1987, Psychophysiology.

[15]  J. Theeuwes,et al.  Unconscious attentional orienting to exogenous cues: A review of the literature. , 2010, Acta psychologica.

[16]  Thomas D Albright,et al.  On the Perception of Probable Things: Neural Substrates of Associative Memory, Imagery, and Perception , 2012, Neuron.

[17]  Jukka Heikkonen,et al.  A local neural classifier for the recognition of EEG patterns associated to mental tasks , 2002, IEEE Trans. Neural Networks.

[18]  J. G. Hollands,et al.  Engineering Psychology and Human Performance , 1984 .

[19]  J. Wolfe,et al.  Varying Target Prevalence Reveals Two Dissociable Decision Criteria in Visual Search , 2010, Current Biology.

[20]  S. Hillyard,et al.  Identification of early visual evoked potential generators by retinotopic and topographic analyses , 1994 .

[21]  E. Vogel,et al.  Sensory gain control (amplification) as a mechanism of selective attention: electrophysiological and neuroimaging evidence. , 1998, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[22]  S Ullman,et al.  Shifts in selective visual attention: towards the underlying neural circuitry. , 1985, Human neurobiology.

[23]  R. Brenner,et al.  Increasing the Yield of EEG , 2006, Journal of clinical neurophysiology : official publication of the American Electroencephalographic Society.

[24]  Measurement of olfactory threshold using an evoked potential technique. , 1999, Rhinology.

[25]  Joseph H. Goldberg,et al.  Identifying fixations and saccades in eye-tracking protocols , 2000, ETRA.

[26]  Neil A. Macmillan,et al.  Detection Theory: A User's Guide , 1991 .

[27]  M. Posner,et al.  The attention system of the human brain: 20 years after. , 2012, Annual review of neuroscience.

[28]  Arnaud Delorme,et al.  EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis , 2004, Journal of Neuroscience Methods.

[29]  Jeremy M. Wolfe,et al.  26.5 brief comms NEW , 2005 .

[30]  Chris Berka,et al.  Real-Time Analysis of EEG Indexes of Alertness, Cognition, and Memory Acquired With a Wireless EEG Headset , 2004, Int. J. Hum. Comput. Interact..

[31]  Andrew T Duchowski,et al.  A breadth-first survey of eye-tracking applications , 2002, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[32]  J. Hobson,et al.  Visual Discrimination Task Improvement: A Multi-Step Process Occurring During Sleep , 2000, Journal of Cognitive Neuroscience.

[33]  Jordan J. Louviere,et al.  Consumer neuroscience: Assessing the brain response to marketing stimuli using electroencephalogram (EEG) and eye tracking , 2013, Expert Syst. Appl..

[34]  A. Treisman.  How the deployment of attention determines what we see , 2006, Visual cognition.

[35]  An event-related potential study on visual perceptual learning under short-term and long-term training conditions , 2002, Neuroreport.

[36]  H. Müller,et al.  Visual search and selective attention , 2006 .

[37]  Richard D. Jones,et al.  EEG-Based Lapse Detection With High Temporal Resolution , 2007, IEEE Transactions on Biomedical Engineering.

[38]  Dov Sagi,et al.  Common mechanisms of human perceptual and motor learning , 2012, Nature Reviews Neuroscience.

[39]  Jaakko Malmivuo,et al.  Effect of measurement noise and electrode density on the spatial resolution of cortical potential distribution with different resistivity values for the skull , 2006, IEEE Transactions on Biomedical Engineering.

[40]  E. John,et al.  Evoked-Potential Correlates of Stimulus Uncertainty , 1965, Science.

[41]  A. Cowey,et al.  The role of the parietal cortex in visual attention—hemispheric asymmetries and the effects of learning: a magnetic stimulation study , 1998, Neuropsychologia.

[42]  Ruxandra Sireteanu,et al.  Perceptual learning in visual search generalizes over tasks, locations, and eyes , 2000, Vision Research.

[43]  I. Colrain.  Sleep and the Brain , 2011, Neuropsychology Review.

[44]  H. Landolt.  Genetic determination of sleep EEG profiles in healthy humans. , 2011, Progress in brain research.