Brain–Computer Interfaces for Detection and Localization of Targets in Aerial Images

Objective. The N2pc event-related potential (ERP) appears on the opposite side of the scalp with respect to the visual hemisphere where an object of interest is located. We explored the feasibility of using it to extract information on the spatial location of targets in aerial images shown by means of a rapid serial visual presentation (RSVP) protocol using single-trial classification. Methods. Images were shown to 11 participants at a presentation rate of 5 Hz while recording electroencephalographic signals. With the resulting ERPs, we trained linear classifiers for single-trial detection of target presence and location. We analyzed the classifiers’ decisions and their raw output scores on independent test sets as well as the averages and voltage distributions of the ERPs. Results. The N2pc is elicited in RSVP presentation of complex images and can be recognized in single trials (the median area under the receiver operating characteristic curve was 0.76 for left versus right classification). Moreover, the peak amplitude of this ERP correlates with the horizontal position of the target within an image. The N2pc varies significantly depending on handedness, and these differences can be used for discriminating participants in terms of their preferred hand. Conclusion and Significance. The N2pc is elicited during RSVP presentation of real complex images and contains analogue information that can be used to roughly infer the horizontal position of targets. Furthermore, differences in the N2pc due to handedness should be taken into account when creating collaborative brain–computer interfaces.

[1]  Roberto Togneri,et al.  Quantifying target spotting performances with complex geoscientific imagery using ERP P300 responses , 2014, Int. J. Hum. Comput. Stud..

[2]  Marina Schmid,et al.  An Introduction To The Event Related Potential Technique , 2016 .

[3]  H. B. Mann,et al.  On a Test of Whether one of Two Random Variables is Stochastically Larger than the Other , 1947 .

[4]  J. Theeuwes,et al.  Electrophysiological Evidence of the Capture of Visual Attention , 2006, Journal of Cognitive Neuroscience.

[5]  Jonathan R. Wolpaw,et al.  Brain-computer interfaces (BCIs) for communication and control , 2007, Assets '07.

[6]  Lucas C. Parra,et al.  Cortical origins of response time variability during rapid discrimination of visual objects , 2005, NeuroImage.

[7]  Kenneth I. Forster,et al.  Visual perception of rapidly presented word sequences of varying complexity , 1970 .

[8]  J. Régis,et al.  Effects of handedness and sex on the morphology of the corpus callosum: A study with brain magnetic resonance imaging , 1991, Brain and Cognition.

[9]  M. Nuttin,et al.  A brain-actuated wheelchair: Asynchronous and non-invasive Brain–computer interfaces for continuous control of robots , 2008, Clinical Neurophysiology.

[10]  Wai Yie Leong,et al.  An EEG-based approach for left-handedness detection , 2014, Biomed. Signal Process. Control..

[11]  J A Swets,et al.  Measuring the accuracy of diagnostic systems. , 1988, Science.

[12]  E Donchin,et al.  A metric for thought: a comparison of P300 latency and reaction time. , 1981, Science.

[13]  Claus Bahlmann,et al.  In a Blink of an Eye and a Switch of a Transistor: Cortically Coupled Computer Vision , 2010, Proceedings of the IEEE.

[14]  Riccardo Poli,et al.  Towards cooperative brain-computer interfaces for space navigation , 2013, IUI '13.

[15]  J. Hanley,et al.  The meaning and use of the area under a receiver operating characteristic (ROC) curve. , 1982, Radiology.

[16]  Steven J. Luck,et al.  Electrophysiological Correlates of the Focusing of Attention within Complex Visual Scenes: N2pc and Related ERP Components , 2011 .

[17]  David Windridge,et al.  High throughput screening for mammography using a human-computer interface with rapid serial visual presentation (RSVP) , 2013, Medical Imaging.

[18]  Ellen S. Wilschut,et al.  Brain–machine interfaces in space: Using spontaneous rather than intentionally generated brain signals , 2010 .

[19]  N. Birbaumer,et al.  Brain-computer communication: self-regulation of slow cortical potentials for verbal communication. , 2001, Archives of physical medicine and rehabilitation.

[20]  N. Squires,et al.  Two varieties of long-latency positive waves evoked by unpredictable auditory stimuli in man. , 1975, Electroencephalography and clinical neurophysiology.

[21]  Anthony J. Ries,et al.  Optimization of Single-Trial Detection of Event-Related Potentials Through Artificial Trials , 2015, IEEE Transactions on Biomedical Engineering.

[22]  J. Polich,et al.  P3a, perceptual distinctiveness, and stimulus modality. , 1998, Brain research. Cognitive brain research.

[23]  C. Cinel,et al.  Multi-brain fusion and applications to intelligence analysis , 2013, Defense, Security, and Sensing.

[24]  P. Sajda,et al.  Spatiotemporal Linear Decoding of Brain State , 2008, IEEE Signal Processing Magazine.

[25]  A. Schlogl,et al.  Information transfer of an EEG-based brain computer interface , 2003, First International IEEE EMBS Conference on Neural Engineering, 2003. Conference Proceedings..

[26]  Christa Neuper,et al.  An asynchronously controlled EEG-based virtual keyboard: improvement of the spelling rate , 2004, IEEE Transactions on Biomedical Engineering.

[27]  P. Sajda,et al.  Cortically coupled computer vision for rapid image search , 2006, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[28]  Brent Lance,et al.  Confidence Metrics Improve Human-Autonomy Integration , 2014, 2014 9th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[29]  Erich Schröger,et al.  Filter Effects and Filter Artifacts in the Analysis of Electrophysiological Data , 2012, Front. Psychology.

[30]  S. F. Witelson.  The brain connection: the corpus callosum is larger in left-handers. , 1985, Science.

[31]  Ian Daly,et al.  Brain computer interface control via functional connectivity dynamics , 2012, Pattern Recognit..

[32]  M S Treder,et al.  Gaze-independent brain–computer interfaces based on covert attention and feature attention , 2011, Journal of neural engineering.

[33]  Andrew P. Bradley,et al.  The use of the area under the ROC curve in the evaluation of machine learning algorithms , 1997, Pattern Recognit..

[34]  C. Cinel,et al.  P300-Based BCI Mouse With Genetically-Optimized Analogue Control , 2008, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[35]  Riccardo Poli,et al.  Collaborative brain-computer interfaces for the automatic classification of images , 2013, 2013 6th International IEEE/EMBS Conference on Neural Engineering (NER).

[36]  H. Roediger,et al.  Handedness is related to memory via hemispheric interaction: evidence from paired associate recall and source memory tasks. , 2008, Neuropsychology.

[37]  Luca Citi,et al.  Possible sources of perceptual errors in P300-based speller paradigm , 2004 .

[38]  Anthony J. Ries,et al.  The effect of target and non-target similarity on neural classification performance: a boost from confidence , 2015, Front. Neurosci..

[39]  Misha Pavel,et al.  A framework for rapid visual image search using single-trial brain evoked responses , 2011, Neurocomputing.

[40]  S J Luck,et al.  Spatial filtering during visual search: evidence from human electrophysiology. , 1994, Journal of experimental psychology. Human perception and performance.

[41]  Stefan Haufe,et al.  Optimizing event-related potential based brain-computer interfaces: a systematic evaluation of dynamic stopping methods. , 2013, Journal of neural engineering.

[42]  W. Kruskal,et al.  Use of Ranks in One-Criterion Variance Analysis , 1952 .

[43]  Benjamin Blankertz,et al.  Gaze-independent BCI-spelling using rapid serial visual presentation (RSVP) , 2013, Clinical Neurophysiology.

[44]  Stephen Umunna,et al.  Towards a brain computer interface based on the N2pc event-related potential , 2013, 2013 6th International IEEE/EMBS Conference on Neural Engineering (NER).

[45]  M. Eimer.  The N2pc component as an indicator of attentional selectivity. , 1996, Electroencephalography and clinical neurophysiology.

[46]  Francesco Piccione,et al.  User adaptive BCIs: SSVEP and P300 based interfaces , 2003, PsychNology J..

[47]  Riccardo Poli,et al.  Collaborative brain-computer interfaces for target localisation in rapid serial visual presentation , 2014, 2014 6th Computer Science and Electronic Engineering Conference (CEEC).

[48]  Andrés Úbeda,et al.  Visual evoked potential-based brain-machine interface applications to assist disabled people , 2012, Expert Syst. Appl..

[49]  E. Donchin,et al.  Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials. , 1988, Electroencephalography and clinical neurophysiology.

[50]  R. C. Oldfield.  The assessment and analysis of handedness: the Edinburgh inventory. , 1971, Neuropsychologia.

[51]  Martin Eimer,et al.  The N2pc component and its links to attention shifts and spatially selective visual processing. , 2008, Psychophysiology.

[52]  C. Koch.  Strategies and Models of Selective Attention , 2010 .

[53]  Riccardo Poli,et al.  On the moments of the sampling distribution of particle swarm optimisers , 2007, GECCO '07.

[54]  J. Polich.  Updating P300: An integrative theory of P3a and P3b , 2007, Clinical Neurophysiology.

[55]  E. Sellers,et al.  How many people are able to control a P300-based brain–computer interface (BCI)? , 2009, Neuroscience Letters.

[56]  Jon Touryan,et al.  Estimating endogenous changes in task performance from EEG , 2014, Front. Neurosci..

[57]  Jacques Montplaisir,et al.  Interhemispheric EEG coherence during sleep and wakefulness in left- and right-handed subjects , 1990, Brain and Cognition.

[58]  S Ullman,et al.  Shifts in selective visual attention: towards the underlying neural circuitry. , 1985, Human neurobiology.