Speed of Rapid Serial Visual Presentation of Pictures, Numbers and Words Affects Event-Related Potential-Based Detection Accuracy

Rapid serial visual presentation (RSVP) based brain-computer interfaces (BCIs) can detect target images among a continuous stream of rapidly presented images, by classifying a viewer’s event related potentials (ERPs) associated with the target and non-targets images. Whilst the majority of RSVP-BCI studies to date have concentrated on the identification of a single type of image, namely <italic>pictures</italic>, here we study the capability of RSVP-BCI to detect three different target image types: <italic>pictures, numbers</italic> and <italic>words</italic>. The impact of presentation duration (speed) i.e., 100–200ms (5–10Hz), 200–300ms (3.3–5Hz) or 300–400ms (2.5–3.3Hz), is also investigated. 2-way repeated measure ANOVA on accuracies of detecting targets from non-target stimuli (ratio 1:9) measured via area under the receiver operator characteristics curve (AUC) for <inline-formula> <tex-math notation="LaTeX">${N}={15}$ </tex-math></inline-formula> subjects revealed a significant effect of factor Stimulus-Type (<italic>pictures, numbers, words</italic>) (F (2,28) = 7.243, <inline-formula> <tex-math notation="LaTeX">${p} = {0.003}$ </tex-math></inline-formula>) and for Stimulus-Duration (F (2,28) = 5.591, p = 0.011). Furthermore, there is an interaction between stimulus type and duration: F (4,56) = 4.419, <inline-formula> <tex-math notation="LaTeX">${p} = {0.004}$ </tex-math></inline-formula>). The results indicate that when designing RSVP-BCI paradigms, the content of the images and the rate at which images are presented impact on the accuracy of detection and hence these parameters are key experimental variables in protocol design and applications, which apply RSVP for multimodal image datasets.

[1]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[2]  Jonathan Winawer,et al.  A Brain Area for Visual Numerals , 2013, The Journal of Neuroscience.

[3]  Yonghong Huang,et al.  A Fusion Approach for Image Triage using Single Trial ERP Detection , 2007, 2007 3rd International IEEE/EMBS Conference on Neural Engineering.

[4]  Jon Touryan,et al.  Progress Toward Adaptive Integration and Optimization of Automated and Neural Processing Systems: Establishing Neural and Behavioral Benchmarks of Optimized Performance , 2014 .

[5]  Miguel P. Eckstein,et al.  Single-Trial Classification of Event-Related Potentials in Rapid Serial Visual Presentation Tasks Using Supervised Spatial Filtering , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[6]  Lucas C. Parra,et al.  Evoked Neural Responses to Events in Video , 2014, IEEE Journal of Selected Topics in Signal Processing.

[7]  E Donchin,et al.  A metric for thought: a comparison of P300 latency and reaction time. , 1981, Science.

[8]  Claus Bahlmann,et al.  In a Blink of an Eye and a Switch of a Transistor: Cortically Coupled Computer Vision , 2010, Proceedings of the IEEE.

[9]  Dean V Buonomano,et al.  Influence of the interstimulus interval on temporal processing and learning: testing the state-dependent network model , 2009, Philosophical Transactions of the Royal Society B: Biological Sciences.

[10]  Misha Pavel,et al.  Rapid image analysis using neural signals , 2008, CHI Extended Abstracts.

[11]  A. Cichocki,et al.  A novel BCI based on ERP components sensitive to configural processing of human faces , 2012, Journal of neural engineering.

[12]  M. Potter,et al.  Pictorial and conceptual representation of glimpsed pictures. , 2004, Journal of experimental psychology. Human perception and performance.

[13]  Lucas C. Parra,et al.  High-throughput image search via single-trial event detection in a rapid serial visual presentation task , 2003, First International IEEE EMBS Conference on Neural Engineering, 2003. Conference Proceedings..

[14]  Kenneth I. Forster,et al.  Visual perception of rapidly presented word sequences of varying complexity , 1970 .

[15]  Anthony J. Ries,et al.  The effect of target and non-target similarity on neural classification performance: a boost from confidence , 2015, Front. Neurosci..

[16]  Jason Farquhar,et al.  Interactions Between Pre-Processing and Classification Methods for Event-Related-Potential Classification , 2012, Neuroinformatics.

[17]  Damien Coyle,et al.  A Review of Rapid Serial Visual Presentation-based Brain-1 Computer Interfaces 2 , 2017 .

[18]  J. Duncan,et al.  Effects of similarity, difficulty, and nontarget presentation on the time course of visual attention , 1997, Perception & psychophysics.

[19]  Lucas C. Parra,et al.  Cortical origins of response time variability during rapid discrimination of visual objects , 2005, NeuroImage.

[20]  Axel Schäfer,et al.  An event-related potential study on exposure therapy for patients suffering from spider phobia , 2009, Biological Psychology.

[21]  Klaus-Robert Müller,et al.  Shifting stimuli for brain computer interface based on rapid serial visual presentation , 2017, 2017 5th International Winter Conference on Brain-Computer Interface (BCI).

[22]  Shih-Fu Chang,et al.  Brain state decoding for rapid image retrieval , 2009, ACM Multimedia.

[23]  Gustav Öquist,et al.  Adaptive Rapid Serial Visual Presentation , 2001 .

[24]  Miguel P. Eckstein,et al.  Effects of performing two visual tasks on single-trial detection of event-related potentials , 2012, 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[25]  E Peli,et al.  Scrolled and rapid serial visual presentation texts are read at similar rates by the visually impaired. , 1995, Journal of the Optical Society of America. A, Optics, image science, and vision.

[26]  V. Lamme,et al.  The distinct modes of vision offered by feedforward and recurrent processing , 2000, Trends in Neurosciences.

[27]  Chi Zhang,et al.  Multirapid Serial Visual Presentation Framework for EEG-Based Target Detection , 2017, BioMed research international.

[28]  Brad Wyble,et al.  Detecting meaning in RSVP at 13 ms per picture , 2013, Attention, perception & psychophysics.

[29]  K. Arnell,et al.  Attention and the processing of emotional words: Dissociating effects of arousal , 2007, Psychonomic Bulletin & Review.

[30]  M. Potter,et al.  A two-stage model for multiple target detection in rapid serial visual presentation. , 1995, Journal of experimental psychology. Human perception and performance.

[31]  A. Engel,et al.  Event-related potential correlates of the attentional blink phenomenon. , 2003, Brain research. Cognitive brain research.

[32]  G. Woodman A brief introduction to the use of event-related potentials in studies of perception and attention. , 2010, Attention, perception & psychophysics.

[33]  J. Polich Updating P300: An integrative theory of P3a and P3b , 2007, Clinical Neurophysiology.

[34]  J. Wolpaw,et al.  Brain-Computer Interfaces: Principles and Practice , 2012 .

[35]  Christian Mühl,et al.  EEG-based workload estimation across affective contexts , 2014, Front. Neurosci..

[36]  Mark Witkowski,et al.  Rapid Serial Visual Presentation: Design for Cognition , 2013 .

[37]  S Bonnet,et al.  Efficient mental workload estimation using task-independent EEG features , 2016, Journal of neural engineering.

[38]  Monika Althaus,et al.  The effects of memory load and stimulus relevance on the EEG during a visual selective memory search task: An ERP and ERD/ERS study , 2006, Clinical Neurophysiology.

[39]  Miguel P. Eckstein,et al.  Impact of target probability on single-trial EEG target detection in a difficult rapid serial visual presentation task , 2011, 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[40]  J. Touryan,et al.  Real-Time Measurement of Face Recognition in Rapid Serial Visual Presentation , 2011, Front. Psychology.

[41]  Riccardo Poli,et al.  Collaborative brain-computer interfaces for target localisation in rapid serial visual presentation , 2014, 2014 6th Computer Science and Electronic Engineering Conference (CEEC).

[42]  Yiwen Wang,et al.  A rapid face recognition BCI system using single-trial ERP , 2013, 2013 6th International IEEE/EMBS Conference on Neural Engineering (NER).

[43]  Dong Ming,et al.  Time-locked and phase-locked features of P300 event-related potentials (ERPs) for brain-computer interface speller , 2010, Biomed. Signal Process. Control..

[44]  N. Bigdely-Shamlo,et al.  Brain Activity-Based Image Classification From Rapid Serial Visual Presentation , 2008, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[45]  John F Maguire,et al.  Failure to detect meaning in RSVP at 27 ms per picture , 2016, Attention, perception & psychophysics.

[46]  Salil H. Patel,et al.  Characterization of N200 and P300: Selected Studies of the Event-Related Potential , 2005, International journal of medical sciences.

[47]  K L Shapiro,et al.  Temporary suppression of visual processing in an RSVP task: an attentional blink? . , 1992, Journal of experimental psychology. Human perception and performance.

[48]  M Congedo,et al.  A review of classification algorithms for EEG-based brain–computer interfaces , 2007, Journal of neural engineering.

[49]  G. Woodman,et al.  Event-related potential studies of attention , 2000, Trends in Cognitive Sciences.

[50]  E Donchin,et al.  P300 and the word frequency effect. , 1988, Electroencephalography and clinical neurophysiology.

[51]  Don M. Tucker,et al.  A single-trial analytic framework for EEG analysis and its application to target detection and classification , 2008, NeuroImage.

[52]  J. Sweller COGNITIVE LOAD THEORY, LEARNING DIFFICULTY, AND INSTRUCTIONAL DESIGN , 1994 .

[53]  Misha Pavel,et al.  A framework for rapid visual image search using single-trial brain evoked responses , 2011, Neurocomputing.