Tactile, Visual, and Bimodal P300s: Could Bimodal P300s Boost BCI Performance?

The P300 is a positive peak in EEG occurring after presentation of a target stimulus. For brain-computer interfaces (BCIs), eliciting P300s by tactile stimuli would have specific advantages; the display can be hidden under clothes and keeps the user’s gaze free. In addition, robust classification is especially important for BCIs. This motivated us to investigate P300s in response to tactile and visual stimuli unimodally and bimodally. Tactile stimuli were delivered by tactors around the participant’s waist. Visual stimuli were flashed circles on a monitor, schematically representing the tactors. Participants attended to the vibrations and/or flashes of a “target” presented in a stream of standards. The P300 amplitude for the different modalities was comparable in size and depended on electrode location. Classification accuracy was highest in the bimodal condition. We conclude that bimodal stimuli could enhance classification results within a BCI context compared to unimodal presentations.

[1]  Yasoichi Nakajima,et al.  Relationships between attention effects and intensity effects on the cognitive N140 and P300 components of somatosensory ERPs , 2000, Clinical Neurophysiology.

[2]  Brendan Z Allison,et al.  Effects of SOA and flash pattern manipulations on ERPs, performance, and preference: implications for a BCI system. , 2006, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[3]  N. Birbaumer.  Breaking the silence: brain-computer interfaces (BCI) for communication and motor control. , 2006, Psychophysiology.

[4]  B. Blankertz,et al.  (C)overt attention and visual speller design in an ERP-based brain-computer interface , 2010, Behavioral and Brain Functions.

[5]  J. Polich,et al.  On P300 measurement stability: habituation, intra-trial block variation, and ultradian rhythms , 1999, Biological Psychology.

[6]  J. Wolpaw,et al.  A P300 event-related potential brain–computer interface (BCI): The effects of matrix size and inter stimulus interval on performance , 2006, Biological Psychology.

[7]  John Polich,et al.  P3(00) habituation from auditory and visual stimuli , 1996, Physiology & Behavior.

[8]  J.D. Bayliss,et al.  Use of the evoked potential P3 component for control in a virtual apartment , 2003, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[9]  K. Spencer,et al.  Poststimulus EEG spectral analysis and P300: attention, task, and probability. , 1999, Psychophysiology.

[10]  Dana H. Ballard,et al.  Single trial P3 epoch recognition in a virtual environment , 2000, Neurocomputing.

[11]  Jan B. F. van Erp,et al.  A Tactile P300 Brain-Computer Interface , 2010, Front. Neurosci..

[12]  J. Polich,et al.  P3a and P3b from typical auditory and visual stimuli , 1999, Clinical Neurophysiology.

[13]  D.J. McFarland,et al.  The wadsworth BCI research and development program: at home with BCI , 2006, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[14]  J. Polich,et al.  P300 and alpha event-related desynchronization (ERD). , 2001, Psychophysiology.

[15]  T. Stanford,et al.  Superadditivity in multisensory integration: putting the computation in context. , 2007, Neuroreport.

[16]  P. Sajda,et al.  Cortically coupled computer vision for rapid image search , 2006, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[17]  Helge J. Ritter,et al.  BCI competition 2003-data set IIb: support vector machines for the P300 speller paradigm , 2004, IEEE Transactions on Biomedical Engineering.

[18]  John J. Foxe,et al.  Multisensory auditory-visual interactions during early sensory processing in humans: a high-density electrical mapping study. , 2002, Brain research. Cognitive brain research.

[19]  C. Gonsalvez,et al.  Target-to-target interval versus probability effects on P300 in one- and two-tone tasks. , 2003, Psychophysiology.

[20]  J Polich,et al.  Comparison of P300 from passive and active tasks for auditory and visual stimuli. , 1999, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[21]  J A Veltman,et al.  Tactile cueing effects on performance in simulated aerial combat with high acceleration. , 2007, Aviation, space, and environmental medicine.

[22]  Haihong Zhang,et al.  A Brain-Controlled Wheelchair Based on P300 and Path Guidance , 2006, The First IEEE/RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics, 2006. BioRob 2006..

[23]  J. Polich,et al.  P300 from auditory and somatosensory stimuli: probability and inter-stimulus interval. , 1991, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[24]  B. Kotchoubey,et al.  Event-related potentials in an auditory semantic oddball task in humans , 2001, Neuroscience Letters.

[25]  Ben H. Jansen,et al.  An exploratory study of factors affecting single trial P300 detection , 2004, IEEE Transactions on Biomedical Engineering.

[26]  Hendrik A. H. C. van Veen,et al.  Waypoint navigation with a vibrotactile waist belt , 2005, TAP.

[27]  M. Woldorff,et al.  Selective attention and audiovisual integration: is attending to both modalities a prerequisite for early integration? , 2006, Cerebral cortex.

[28]  R S Nickerson,et al.  Intersensory facilitation of reaction time: energy summation or preparation enhancement? , 1973, Psychological review.

[29]  S. Hillyard,et al.  Intra-modal and cross-modal spatial attention to auditory and visual stimuli. An event-related brain potential study. , 1999, Brain research. Cognitive brain research.

[30]  E. Schröger,et al.  Speeded responses to audiovisual signal changes result from bimodal integration. , 1998, Psychophysiology.

[31]  Pieter Padmos,et al.  Image parameters for driving with indirect viewing systems , 2003, Ergonomics.

[32]  T. Sejnowski,et al.  Analysis and visualization of single‐trial event‐related potentials , 2001, Human brain mapping.

[33]  F. Babiloni,et al.  Multimodal stimulation for a P300-based BCI , 2007 .

[34]  J. Polich,et al.  P300 topography and modality effects from a single-stimulus paradigm. , 1996, Psychophysiology.

[35]  Hendrik A. H. C. van Veen,et al.  Touch down: The effect of artificial touch cues on orientation in microgravity , 2006, Neuroscience Letters.

[36]  E. Donchin,et al.  A P300-based brain–computer interface: Initial tests by ALS patients , 2006, Clinical Neurophysiology.

[37]  J. Polich.  Updating P300: An integrative theory of P3a and P3b , 2007, Clinical Neurophysiology.

[38]  Jan B. F. van Erp,et al.  Presenting directions with a vibrotactile torso display. , 2005 .

[39]  G. Pfurtscheller,et al.  Brain-Computer Interfaces for Communication and Control. , 2011, Communications of the ACM.

[40]  B. Kotchoubey,et al.  Brain responses to number sequences with and without active task requirement , 2002, Clinical Neurophysiology.

[41]  G.F. Inbar,et al.  An improved P300-based brain-computer interface , 2005, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[42]  Jan B. F. van Erp,et al.  Multisensory temporal numerosity judgment , 2008, Brain Research.

[43]  John J. Foxe,et al.  Multisensory auditory-somatosensory interactions in early cortical processing revealed by high-density electrical mapping. , 2000, Brain research. Cognitive brain research.

[44]  E. Donchin,et al.  Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials. , 1988, Electroencephalography and clinical neurophysiology.

[45]  Cuntai Guan,et al.  High performance P300 speller for brain-computer interface , 2004, IEEE International Workshop on Biomedical Circuits and Systems, 2004..

[46]  F. Piccione,et al.  P300-based brain computer interface: Reliability and performance in healthy and paralysed participants , 2006, Clinical Neurophysiology.

[47]  M. Scherg,et al.  Localizing P300 Generators in Visual Target and Distractor Processing: A Combined Event-Related Potential and Functional Magnetic Resonance Imaging Study , 2004, The Journal of Neuroscience.

[48]  H. Jasper.  Report of the committee on methods of clinical examination in electroencephalography , 1958 .

[49]  John J. Foxe,et al.  The case for feedforward multisensory convergence during early cortical processing , 2005, Neuroreport.

[50]  Narayanan Srinivasan,et al.  Cognitive neuroscience of creativity: EEG based approaches. , 2007, Methods.

[51]  J Peter Rosenfeld,et al.  P300-based detection of concealed autobiographical versus incidentally acquired information in target and non-target paradigms. , 2006, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.