Toward a hybrid brain-computer interface based on repetitive visual stimuli with missing events

BackgroundSteady-state visually evoked potentials (SSVEPs) can be elicited by repetitive stimuli and extracted in the frequency domain with satisfied performance. However, the temporal information of such stimulus is often ignored. In this study, we utilized repetitive visual stimuli with missing events to present a novel hybrid BCI paradigm based on SSVEP and omitted stimulus potential (OSP).MethodsFour discs flickering from black to white with missing flickers served as visual stimulators to simultaneously elicit subject’s SSVEPs and OSPs. Key parameters in the new paradigm, including flicker frequency, optimal electrodes, missing flicker duration and intervals of missing events were qualitatively discussed with offline data. Two omitted flicker patterns including missing black/white disc were proposed and compared. Averaging times were optimized with Information Transfer Rate (ITR) in online experiments, where SSVEPs and OSPs were identified using Canonical Correlation Analysis in the frequency domain and Support Vector Machine (SVM)-Bayes fusion in the time domain, respectively.Results and conclusionsThe online accuracy and ITR (mean ± standard deviation) over nine healthy subjects were 79.29 ± 18.14 % and 19.45 ± 11.99 bits/min with missing black disc pattern, and 86.82 ± 12.91 % and 24.06 ± 10.95 bits/min with missing white disc pattern, respectively. The proposed BCI paradigm, for the first time, demonstrated that SSVEPs and OSPs can be simultaneously elicited in single visual stimulus pattern and recognized in real-time with satisfied performance. Besides the frequency features such as SSVEP elicited by repetitive stimuli, we found a new feature (OSP) in the time domain to design a novel hybrid BCI paradigm by adding missing events in repetitive stimuli.

[1]  Deniz Erdogmus,et al.  A comparison of temporal windowing schemes for single-trial ERP detection , 2009, 2009 4th International IEEE/EMBS Conference on Neural Engineering.

[2]  S. Jones AEPs at the onset and offset of repetitive sound modulation, due to mismatch with the contents of an auditory sensory store. , 1992, Electroencephalography and clinical neurophysiology.

[3]  Robert Oostenveld,et al.  FieldTrip: Open Source Software for Advanced Analysis of MEG, EEG, and Invasive Electrophysiological Data , 2010, Comput. Intell. Neurosci..

[4]  Michael J. Berry,et al.  Detection and prediction of periodic patterns by the retina , 2007, Nature Neuroscience.

[5]  Fanglin Chen,et al.  A novel hybrid BCI speller based on the incorporation of SSVEP into the P300 paradigm , 2013, Journal of neural engineering.

[6]  D. Linden The P300: Where in the Brain Is It Produced and What Does It Tell Us? , 2005, The Neuroscientist : a review journal bringing neurobiology, neurology and psychiatry.

[7]  Xingyu Wang,et al.  Aggregation of Sparse Linear Discriminant analyses for Event-Related potential Classification in Brain-Computer Interface , 2014, Int. J. Neural Syst..

[8]  Rami Saab,et al.  A Hybrid Brain–Computer Interface Based on the Fusion of P300 and SSVEP Scores , 2015, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[9]  Xiaorong Gao,et al.  An online multi-channel SSVEP-based brain–computer interface using a canonical correlation analysis method , 2009, Journal of neural engineering.

[10]  Michael Bach,et al.  Adaptation characteristics of steady-state motion visual evoked potentials , 2003, Clinical Neurophysiology.

[11]  Andrzej Cichocki,et al.  L1-Regularized Multiway Canonical Correlation Analysis for SSVEP-Based BCI , 2013, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[12]  Jianjun Meng,et al.  Combining Motor Imagery With Selective Sensation Toward a Hybrid-Modality BCI , 2014, IEEE Transactions on Biomedical Engineering.

[13]  Yijun Wang,et al.  A high-speed BCI based on code modulation VEP , 2011, Journal of neural engineering.

[14]  Pablo F. Diez,et al.  Asynchronous BCI control using high-frequency SSVEP , 2011, Journal of NeuroEngineering and Rehabilitation.

[15]  Günter Edlinger,et al.  A Hybrid Brain-Computer Interface for Smart Home Control , 2011, HCI.

[16]  Wolfgang Rosenstiel,et al.  Online Adaptation of a c-VEP Brain-Computer Interface(BCI) Based on Error-Related Potentials and Unsupervised Learning , 2012, PloS one.

[17]  T H Bullock,et al.  Event-related potentials in the retina and optic tectum of fish. , 1990, Journal of neurophysiology.

[18]  H. Gray,et al.  P300 as an index of attention to self-relevant stimuli , 2004 .

[19]  Xingyu Wang,et al.  Frequency Recognition in SSVEP-Based BCI using Multiset Canonical Correlation Analysis , 2013, Int. J. Neural Syst..

[20]  T H Bullock,et al.  Dynamic properties of human visual evoked and omitted stimulus potentials. , 1994, Electroencephalography and clinical neurophysiology.

[21]  T H Bullock,et al.  Event-related potentials to omitted visual stimuli in a reptile. , 1994, Electroencephalography and clinical neurophysiology.

[22]  Alexandre Balbinot,et al.  Executed Movement Using EEG Signals through a Naive Bayes Classifier , 2014, Micromachines.

[23]  D G Pelli,et al.  The VideoToolbox software for visual psychophysics: transforming numbers into movies. , 1997, Spatial vision.

[24]  M. Vogel-Sprott,et al.  THE OMITTED STIMULUS POTENTIAL IS RELATED TO THE COGNITIVE COMPONENT OF REACTION TIME , 2008, The International journal of neuroscience.

[25]  Touradj Ebrahimi,et al.  An efficient P300-based brain–computer interface for disabled subjects , 2008, Journal of Neuroscience Methods.

[26]  Ying Sun,et al.  Asynchronous P300 BCI: SSVEP-based control state detection , 2010, 2010 18th European Signal Processing Conference.

[27]  T. Jung,et al.  Assessing the feasibility of online SSVEP decoding in human walking using a consumer EEG headset , 2014, Journal of NeuroEngineering and Rehabilitation.

[28]  Yuanjin Zheng,et al.  Coherent Photoacoustic-Ultrasound Correlation and Imaging , 2014, IEEE Transactions on Biomedical Engineering.

[29]  Xingyu Wang,et al.  Multiway Canonical Correlation Analysis for Frequency Components Recognition in SSVEP-Based BCIs , 2011, ICONIP.

[30]  Daniel S. Yeung,et al.  Weighted Mahalanobis Distance Kernels for Support Vector Machines , 2007, IEEE Transactions on Neural Networks.

[31]  Ina M. Tarkka,et al.  Source Localization of P300 from Oddball, Single Stimulus, and Omitted-Stimulus Paradigms , 2004, Brain Topography.

[32]  K. Spencer,et al.  Poststimulus EEG spectral analysis and P300: attention, task, and probability. , 1999, Psychophysiology.

[33]  José del R. Millán,et al.  Error-Related EEG Potentials Generated During Simulated Brain–Computer Interaction , 2008, IEEE Transactions on Biomedical Engineering.

[34]  Jing Wang,et al.  Steady-State Motion Visual Evoked Potentials Produced by Oscillating Newton's Rings: Implications for Brain-Computer Interfaces , 2012, PloS one.

[35]  Wei Wu,et al.  Frequency recognition based on canonical correlation analysis for SSVEP-based BCIs , 2007, IEEE Transactions on Biomedical Engineering.

[36]  Dennis J. McFarland,et al.  Brain–computer interfaces for communication and control , 2002, Clinical Neurophysiology.

[37]  Yuanqing Li,et al.  A Hybrid BCI System Combining P300 and SSVEP and Its Application to Wheelchair Control , 2013, IEEE Transactions on Biomedical Engineering.

[38]  F. Cincotti,et al.  Attention and P300-based BCI performance in people with amyotrophic lateral sclerosis , 2013, Front. Hum. Neurosci..

[39]  G Pfurtscheller,et al.  Toward a hybrid brain–computer interface based on imagined movement and visual attention , 2010, Journal of neural engineering.

[40]  Brendan Z. Allison,et al.  Improved signal processing approaches in an offline simulation of a hybrid brain–computer interface , 2010, Journal of Neuroscience Methods.

[41]  Yijun Wang,et al.  Visual and Auditory Brain–Computer Interfaces , 2014, IEEE Transactions on Biomedical Engineering.

[42]  Dong Ming,et al.  A hybrid BCI speller paradigm combining P300 potential and the SSVEP blocking feature , 2013, Journal of neural engineering.

[43]  D H Brainard,et al.  The Psychophysics Toolbox. , 1997, Spatial vision.

[44]  Xingyu Wang,et al.  Sparse Bayesian Classification of EEG for Brain–Computer Interface , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[45]  H. Hotelling Relations Between Two Sets of Variates , 1936 .

[46]  P Finkenzeller,et al.  Evoked responses as a function of external and stored information. , 1968, Electroencephalography and clinical neurophysiology.

[47]  A. Cichocki,et al.  Steady-state visually evoked potentials: Focus on essential paradigms and future perspectives , 2010, Progress in Neurobiology.

[48]  Stefan Haufe,et al.  Single-trial analysis and classification of ERP components — A tutorial , 2011, NeuroImage.

[49]  Rabab K Ward,et al.  Automatic artefact removal in a self-paced hybrid brain- computer interface system , 2012, Journal of NeuroEngineering and Rehabilitation.

[50]  Xiaorong Gao,et al.  A BCI-based environmental controller for the motion-disabled. , 2003, IEEE transactions on neural systems and rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[51]  Brendan Z. Allison,et al.  The Hybrid BCI , 2010, Frontiers in Neuroscience.

[52]  Marc M. Van Hulle,et al.  Simultaneous Detection of P300 and Steady-State Visually Evoked Potentials for Hybrid Brain-Computer Interface , 2015, PloS one.

[53]  S. Heinrich A primer on motion visual evoked potentials , 2007, Documenta Ophthalmologica.

[54]  F. Musiek,et al.  Comparisons of P300s from standard oddball and omitted paradigms: implications to exogenous/endogenous contributions. , 2009, Journal of the American Academy of Audiology.

[55]  A. Kübler,et al.  Toward brain-computer interface based wheelchair control utilizing tactually-evoked event-related potentials , 2014, Journal of NeuroEngineering and Rehabilitation.

[56]  Zoran Nenadic,et al.  Brain-computer interface controlled robotic gait orthosis , 2012, Journal of NeuroEngineering and Rehabilitation.

[57]  Ali Bahramisharif,et al.  Brain-computer interfacing using modulations of alpha activity induced by covert shifts of attention , 2011, Journal of NeuroEngineering and Rehabilitation.

[58]  Misha Pavel,et al.  A framework for rapid visual image search using single-trial brain evoked responses , 2011, Neurocomputing.

[59]  G Calhoun,et al.  Brain-computer interfaces based on the steady-state visual-evoked response. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[60]  Xingyu Wang,et al.  SSVEP recognition using common feature analysis in brain–computer interface , 2015, Journal of Neuroscience Methods.

[61]  Yuanqing Li,et al.  Target Selection With Hybrid Feature for BCI-Based 2-D Cursor Control , 2012, IEEE Transactions on Biomedical Engineering.