A Dual Stimuli Approach Combined with Convolutional Neural Network to Improve Information Transfer Rate of Event-Related Potential-Based Brain-Computer Interface

Increasing command generation rate of an event-related potential-based brain-robot system is challenging, because of limited information transfer rate of a brain-computer interface system. To improve the rate, we propose a dual stimuli approach that is flashing a robot image and is scanning another robot image simultaneously. Two kinds of event-related potentials, N200 and P300 potentials, evoked in this dual stimuli condition are decoded by a convolutional neural network. Compared with the traditional approaches, this proposed approach significantly improves the online information transfer rate from 23.0 or 17.8 to 39.1 bits/min at an accuracy of 91.7%. These results suggest that combining multiple types of stimuli to evoke distinguishable ERPs might be a promising direction to improve the command generation rate in the brain-computer interface.

[1]  W. Pritchard.  Psychophysiology of P300. , 1981, Psychological bulletin.

[2]  E. Donchin,et al.  Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials. , 1988, Electroencephalography and clinical neurophysiology.

[3]  J. Cohen,et al.  P300, stimulus intensity, modality, and probability. , 1996, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[4]  E Donchin,et al.  Brain-computer interface technology: a review of the first international meeting. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[5]  T. Sejnowski,et al.  Analysis and visualization of single‐trial event‐related potentials , 2001, Human brain mapping.

[6]  W. A. Sarnacki,et al.  Brain–computer interface (BCI) operation: optimizing information transfer rates , 2003, Biological Psychology.

[7]  Ramaswamy Palaniappan,et al.  Neural network classification of autoregressive features from electroencephalogram signals for brain–computer interface design , 2004, Journal of neural engineering.

[8]  Salil H. Patel,et al.  Characterization of N200 and P300: Selected Studies of the Event-Related Potential , 2005, International journal of medical sciences.

[9]  S. Heinrich.  A primer on motion visual evoked potentials , 2007, Documenta Ophthalmologica.

[10]  Rabab K Ward,et al.  A survey of signal processing algorithms in brain–computer interfaces based on electrical brain signals , 2007, Journal of neural engineering.

[11]  C. Gonsalvez,et al.  Target-to-target interval, intensity, and P300 from an auditory single-stimulus task. , 2007, Psychophysiology.

[12]  Alain Rakotomamonjy,et al.  BCI Competition III: Dataset II- Ensemble of SVMs for BCI P300 Speller , 2008, IEEE Transactions on Biomedical Engineering.

[13]  Tao Liu,et al.  N200-speller using motion-onset visual response , 2009, Clinical Neurophysiology.

[14]  E. Sellers,et al.  How many people are able to control a P300-based brain–computer interface (BCI)? , 2009, Neuroscience Letters.

[15]  Yijun Wang,et al.  VEP-based brain-computer interfaces: time, frequency, and code modulations [Research Frontier] , 2009, IEEE Computational Intelligence Magazine.

[16]  Xingyu Wang,et al.  A new P300 stimulus presentation pattern for EEG-based spelling systems , 2010, Biomedizinische Technik. Biomedical engineering.

[17]  Shangkai Gao,et al.  An online brain–computer interface using non-flashing visual evoked potentials , 2010, Journal of neural engineering.

[18]  Hubert Cecotti,et al.  Convolutional Neural Networks for P300 Detection with Application to Brain-Computer Interfaces , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Dennis J. McFarland,et al.  The P300-based brain–computer interface (BCI): Effects of stimulus rate , 2011, Clinical Neurophysiology.

[20]  Andrew D. Engell,et al.  The relationship of γ oscillations and face-specific ERPs recorded subdurally from occipitotemporal cortex. , 2011, Cerebral cortex.

[21]  Xingyu Wang,et al.  A combined brain–computer interface based on P300 potentials and motion-onset visual evoked potentials , 2012, Journal of Neuroscience Methods.

[22]  Dong Ming,et al.  A hybrid BCI speller paradigm combining P300 potential and the SSVEP blocking feature , 2013, Journal of neural engineering.

[23]  Yili Liu,et al.  EEG-Based Brain-Controlled Mobile Robots: A Survey , 2013, IEEE Transactions on Human-Machine Systems.

[24]  Ming Yang,et al.  3D Convolutional Neural Networks for Human Action Recognition , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Genshe Chen,et al.  A P300 Model for Cerebot - A Mind-Controlled Humanoid Robot , 2013, RiTA.

[26]  Fanglin Chen,et al.  A novel hybrid BCI speller based on the incorporation of SSVEP into the P300 paradigm , 2013, Journal of neural engineering.

[27]  Gernot R. Müller-Putz,et al.  A Single-Switch BCI Based on Passive and imagined movements: toward Restoring Communication in Minimally Conscious patients , 2013, Int. J. Neural Syst..

[28]  H. Adeli,et al.  Brain-computer interface technologies: from signal to action , 2013, Reviews in the neurosciences.

[29]  A. Cichocki,et al.  An optimized ERP brain–computer interface based on facial expression changes , 2014, Journal of neural engineering.

[30]  Jie Li,et al.  Evaluation and Application of a Hybrid Brain Computer Interface for Real Wheelchair Parallel Control with Multi-Degree of Freedom , 2014, Int. J. Neural Syst..

[31]  Tzyy-Ping Jung,et al.  A High-Speed Brain Speller using steady-State Visual evoked potentials , 2014, Int. J. Neural Syst..

[32]  J. Buford,et al.  Combined corticospinal and reticulospinal effects on upper limb muscles , 2014, Neuroscience Letters.

[33]  J. Buford,et al.  Brain–Computer Interface after Nervous System Injury , 2014, The Neuroscientist : a review journal bringing neurobiology, neurology and psychiatry.

[34]  Xiaohui Xie,et al.  Handwritten Hangul recognition using deep convolutional neural networks , 2014, International Journal on Document Analysis and Recognition (IJDAR).

[35]  J. Buford,et al.  Wavelet methodology to improve single unit isolation in primary motor cortex cells , 2015, Journal of Neuroscience Methods.

[36]  Wei Li,et al.  Control of humanoid robot via motion-onset visual evoked potentials , 2015, Front. Syst. Neurosci..

[37]  Bo Liu,et al.  Image segmentation with pulse-coupled neural network and Canny operators , 2015, Comput. Electr. Eng..

[38]  Richard Kempter,et al.  State-dependencies of learning across brain scales , 2015, Front. Comput. Neurosci..

[39]  Xingyu Wang,et al.  A P300 Brain-Computer Interface Based on a Modification of the Mismatch Negativity Paradigm , 2015, Int. J. Neural Syst..

[40]  Tzyy-Ping Jung,et al.  High-speed spelling with a noninvasive brain–computer interface , 2015, Proceedings of the National Academy of Sciences.

[41]  Anca Radulescu,et al.  Neural Network Spectral Robustness under Perturbations of the Underlying Graph , 2016, Neural Computation.

[42]  Jinling Liang,et al.  Multistability of complex-valued neural networks with distributed delays , 2016, Neural Computing and Applications.

[43]  Long Chen,et al.  Incorporation of Inter-Subject Information to Improve the Accuracy of Subject-Specific P300 Classifiers , 2016, Int. J. Neural Syst..

[44]  Andrés Úbeda,et al.  EEG-Based Detection of Starting and Stopping During Gait Cycle , 2016, Int. J. Neural Syst..

[45]  Marc M. Van Hulle,et al.  Faster P300 Classifier Training Using Spatiotemporal Beamforming , 2016, Int. J. Neural Syst..

[46]  Andrés Ortiz,et al.  Ensembles of Deep Learning Architectures for the Early Diagnosis of the Alzheimer's Disease , 2016, Int. J. Neural Syst..

[47]  Andrés Bustillo,et al.  Interpreting tree-based prediction models and their data in machining processes , 2016, Integr. Comput. Aided Eng..

[48]  Helge Ritter,et al.  Using a cVEP-Based Brain-Computer Interface to Control a Virtual Agent , 2016, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[49]  Chun-Hsiang Chuang,et al.  An EEG-Based Fatigue Detection and Mitigation System , 2016, Int. J. Neural Syst..

[50]  Wei Li,et al.  Increasing N200 Potentials Via Visual Stimulus Depicting Humanoid Robot Behavior , 2016, Int. J. Neural Syst..

[51]  Christoph Pokorny,et al.  A hybrid three-class brain-computer interface system utilizing SSSEPs and transient ERPs. , 2016, Journal of neural engineering.

[52]  Rami Saab,et al.  An Auditory-Tactile Visual Saccade-Independent P300 Brain-Computer Interface , 2016, Int. J. Neural Syst..

[53]  Qi Wu,et al.  Using Fractal and Local Binary Pattern Features for Classification of ECOG Motor Imagery Tasks Obtained from the Right Brain Hemisphere , 2016, Int. J. Neural Syst..

[54]  Tao Jiang,et al.  Design and implementation of membrane controllers for trajectory tracking of nonholonomic wheeled mobile robots , 2015, Integr. Comput. Aided Eng..

[55]  Jing Zhao,et al.  Behavior-Based SSVEP Hierarchical Architecture for Telepresence Control of Humanoid Robot to Achieve Full-Body Movement , 2017, IEEE Transactions on Cognitive and Developmental Systems.

[56]  Oral Büyüköztürk,et al.  Deep Learning‐Based Crack Damage Detection Using Convolutional Neural Networks , 2017, Comput. Aided Civ. Infrastructure Eng..

[57]  Leonardo Franco,et al.  Layer multiplexing FPGA implementation for deep back-propagation learning , 2017, Integr. Comput. Aided Eng..

[58]  Yi-Zhou Lin,et al.  Structural Damage Detection with Automatic Feature‐Extraction through Deep Learning , 2017, Comput. Aided Civ. Infrastructure Eng..

[59]  Ren Long,et al.  iRSpot-EL: identify recombination spots with an ensemble learning approach , 2017, Bioinform..

[60]  Benjamin Wittevrongel,et al.  Code-modulated visual evoked potentials using fast stimulus presentation and spatiotemporal beamformer decoding , 2017, Scientific Reports.

[61]  Hojjat Adeli,et al.  A novel machine learning‐based algorithm to detect damage in high‐rise building structures , 2017 .

[62]  Hongzhe Dai,et al.  A Wavelet Support Vector Machine‐Based Neural Network Metamodel for Structural Reliability Assessment , 2017, Comput. Aided Civ. Infrastructure Eng..

[63]  Wei Zhang,et al.  A Unified Framework for Street-View Panorama Stitching , 2016, Sensors.

[64]  Boguslaw Cyganek,et al.  Image recognition with deep neural networks in presence of noise - Dealing with and taking advantage of distortions , 2017, Integr. Comput. Aided Eng..

[65]  Tom Chau,et al.  Online EEG Classification of Covert Speech for Brain-Computer Interfacing , 2017, Int. J. Neural Syst..

[66]  Nitish V. Thakor,et al.  EEG Classification with a Sequential Decision-Making Method in Motor Imagery BCI , 2017, Int. J. Neural Syst..

[67]  R. Doerge,et al.  Novel Resampling Improves Statistical Power for Multiple-Trait QTL Mapping , 2017, G3: Genes, Genomes, Genetics.

[68]  Alok Sharma,et al.  CSP-TSM: Optimizing the performance of Riemannian tangent space mapping using common spatial pattern for MI-BCI , 2017, Comput. Biol. Medicine.

[69]  Dario Farina,et al.  A Real-Time Method for Decoding the Neural Drive to Muscles Using Single-Channel Intra-Muscular EMG Recordings , 2017, Int. J. Neural Syst..

[70]  Francesco Carlo Morabito,et al.  Deep Learning Representation from Electroencephalography of Early-Stage Creutzfeldt-Jakob Disease and Features for Differentiation from Rapidly Progressive Dementia , 2017, Int. J. Neural Syst..

[71]  Ping Zhou,et al.  Real-Time Control of an Exoskeleton Hand Robot with Myoelectric Pattern Recognition , 2017, Int. J. Neural Syst..

[72]  Yang Liu,et al.  Automated Pixel‐Level Pavement Crack Detection on 3D Asphalt Surfaces Using a Deep‐Learning Network , 2017, Comput. Aided Civ. Infrastructure Eng..

[73]  M Congedo,et al.  A review of classification algorithms for EEG-based brain–computer interfaces: a 10 year update , 2018, Journal of neural engineering.

[74]  Ming Zhu,et al.  Geometry based three-dimensional image processing method for electronic cluster eye , 2018, Integr. Comput. Aided Eng..

[75]  Hojjat Adeli,et al.  A novel unsupervised deep learning model for global and local health condition assessment of structures , 2018 .