A Deep Neural Network for SSVEP-based Brain Computer Interfaces

The target identification in brain-computer interface (BCI) speller systems refers to the multi-channel electroencephalogram (EEG) classification for predicting the target character that the user intends to spell. The EEG in such systems is known to include the steady-state visually evoked potentials (SSVEP) signal, which is the brain response when the user concentrates on the target while being visually presented a matrix of certain alphanumeric each of which flickers at a unique frequency. The SSVEP in this setting is characteristically dominated at varying degrees by the harmonics of the stimulation frequency; hence, a pattern analysis of the SSVEP can solve for the mentioned multi-class classification problem. To this end, we propose a novel deep neural network (DNN) architecture for the target identification in BCI SSVEP spellers. The proposed DNN is an end-to-end system: it receives the multi-channel SSVEP signal, proceeds with convolutions across the sub-bands of the harmonics, channels and time, and classifies at the fully connected layer. Our experiments are on two publicly available (the benchmark and the BETA) datasets consisting of in total 105 subjects with 40 characters. We train in two stages. The first stage obtains a global perspective into the whole SSVEP data by exploiting the commonalities, and transfers the global model to the second stage that fine tunes it down to each subject separately by exploiting the individual statistics. In our extensive comparisons, our DNN is demonstrated to significantly outperform the state-of-the-art on the both two datasets, by achieving the information transfer rates (ITR) 265.23 bits/min and 196.59 bits/min, respectively. To the best of our knowledge, our ITRs are the highest ever reported performance results on these datasets. The code, and the proposed DNN model are available at this https URL.

[1]  Xiaorong Gao,et al.  Enhancing the classification accuracy of steady-state visual evoked potential-based brain–computer interfaces using phase constrained canonical correlation analysis , 2011, Journal of neural engineering.

[2]  Wei Wu,et al.  Probabilistic Common Spatial Patterns for Multichannel EEG Analysis , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Anton Nijholt,et al.  Brain–Computer Interface Software: A Review and Discussion , 2020, IEEE Transactions on Human-Machine Systems.

[4]  Wei Wu,et al.  Frequency recognition based on canonical correlation analysis for SSVEP-based BCIs , 2007, IEEE Transactions on Biomedical Engineering.

[5]  Natasha M. Maurits,et al.  Stimuli and Feature Extraction Algorithms for Brain-Computer Interfaces: A Systematic Comparison , 2018, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[6]  Erik Cambria,et al.  Recent Trends in Deep Learning Based Natural Language Processing , 2017, IEEE Comput. Intell. Mag..

[7]  Hubert Cecotti,et al.  Convolutional Neural Networks for P300 Detection with Application to Brain-Computer Interfaces , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Arnaud Delorme,et al.  EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis , 2004, Journal of Neuroscience Methods.

[9]  Ivan Volosyak,et al.  Multiple Channel Detection of Steady-State Visual Evoked Potentials for Brain-Computer Interfaces , 2007, IEEE Transactions on Biomedical Engineering.

[10]  Xingyu Wang,et al.  Multiway Canonical Correlation Analysis for Frequency Components Recognition in SSVEP-Based BCIs , 2011, ICONIP.

[11]  Tzyy-Ping Jung,et al.  A High-Speed Brain Speller using steady-State Visual evoked potentials , 2014, Int. J. Neural Syst..

[12]  Heung-Il Suk,et al.  A Novel Bayesian Framework for Discriminative Feature Extraction in Brain-Computer Interfaces , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Justin M. Ales,et al.  The steady-state visual evoked potential in vision research: A review. , 2015, Journal of vision.

[14]  Mihaly Benda,et al.  Brain–Computer Interface Spellers: A Review , 2018, Brain sciences.

[15]  Xiaogang Chen,et al.  Filter bank canonical correlation analysis for implementing a high-speed SSVEP-based brain–computer interface , 2015, Journal of neural engineering.

[16]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[17]  J. Donoghue,et al.  Sensors for brain-computer interfaces , 2006, IEEE Engineering in Medicine and Biology Magazine.

[18]  John Thomas,et al.  Deep learning-based classification for brain-computer interfaces , 2017, 2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[19]  Jaeseung Jeong,et al.  Toward Brain-Actuated Humanoid Robots: Asynchronous Direct Control Using an EEG-Based BCI , 2012, IEEE Transactions on Robotics.

[20]  Yu-Te Wang,et al.  A Comparison Study of Canonical Correlation Analysis Based Methods for Detecting Steady-State Visual Evoked Potentials , 2015, PloS one.

[21]  Toshihisa Tanaka,et al.  Two-Stage Frequency Recognition Method Based on Correlated Component Analysis for SSVEP-Based BCI , 2018, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[22]  Saeid Nahavandi,et al.  A time domain classification of steady-state visual evoked potentials using deep recurrent-convolutional neural networks , 2018, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[23]  Xingyu Wang,et al.  Frequency Recognition in SSVEP-Based BCI using Multiset Canonical Correlation Analysis , 2013, Int. J. Neural Syst..

[24]  Xiaogang Chen,et al.  A Benchmark Dataset for SSVEP-Based Brain–Computer Interfaces , 2017, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[25]  Andrzej Materka,et al.  Cluster analysis of CCA coefficients for robust detection of the asynchronous SSVEPs in brain-computer interfaces , 2014, Biomed. Signal Process. Control..

[26]  Paul Sajda,et al.  Compact convolutional neural networks for classification of asynchronous steady-state visual evoked potentials , 2018, Journal of neural engineering.

[27]  Toby P. Breckon,et al.  On the Relative Contribution of Deep Convolutional Neural Networks for SSVEP-Based Bio-Signal Decoding in BCI Speller Applications , 2019, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[28]  Klaus-Robert Müller,et al.  A convolutional neural network for steady state visual evoked potential classification under ambulatory environment , 2017, PloS one.

[29]  Yijun Wang,et al.  BETA: A Large Benchmark Database Toward SSVEP-BCI Application , 2019, Frontiers in Neuroscience.

[30]  Klaus-Robert Müller,et al.  Towards Zero Training for Brain-Computer Interfacing , 2008, PloS one.

[31]  A. Norcia,et al.  Measuring Integration Processes in Visual Symmetry with Frequency-Tagged EEG , 2017, Scientific Reports.

[32]  Gernot R. Müller-Putz,et al.  Single Versus Multiple Events Error Potential Detection in a BCI-Controlled Car Game With Continuous and Discrete Feedback , 2016, IEEE Transactions on Biomedical Engineering.

[33]  Feng Wan,et al.  Learning across multi-stimulus enhances target recognition methods in SSVEP-based BCIs , 2020, Journal of neural engineering.

[34]  Ramesh Srinivasan,et al.  Steady-State Visual Evoked Potentials: Distributed Local Sources and Wave-Like Dynamics Are Sensitive to Flicker Frequency , 2006, Brain Topography.

[35]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[36]  Gao Xiaorong,et al.  Brain-computer interface based on the high-frequency steady-state visual evoked potential , 2005, Proceedings. 2005 First International Conference on Neural Interface and Control, 2005..

[37]  R Zerafa,et al.  To train or not to train? A survey on training of feature extraction methods for SSVEP-based BCIs , 2018, Journal of neural engineering.

[38]  Andrzej Cichocki,et al.  L1-Regularized Multiway Canonical Correlation Analysis for SSVEP-Based BCI , 2013, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[39]  Yijun Wang,et al.  A high-speed BCI based on code modulation VEP , 2011, Journal of neural engineering.

[40]  Yijun Wang,et al.  Enhancing Detection of SSVEPs for a High-Speed Brain Speller Using Task-Related Component Analysis , 2018, IEEE Transactions on Biomedical Engineering.

[41]  Feng Wan,et al.  A comparison of minimum energy combination and canonical correlation analysis for SSVEP detection , 2011, 2011 5th International IEEE/EMBS Conference on Neural Engineering.

[42]  Yu Zhang,et al.  Hierarchical feature fusion framework for frequency recognition in SSVEP-based BCIs , 2018, Neural Networks.

[43]  Dennis J. McFarland,et al.  Brain–computer interfaces for communication and control , 2002, Clinical Neurophysiology.

[44]  Naotsugu Tsuchiya,et al.  From intermodulation components to visual perception and cognition-a review , 2019, NeuroImage.

[45]  Tzyy-Ping Jung,et al.  SNR analysis of high-frequency steady-state visual evoked potentials from the foveal and extrafoveal regions of Human Retina , 2012, 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[46]  Cuntai Guan,et al.  Brain–Computer Interface for Neurorehabilitation of Upper Limb After Stroke , 2015, Proceedings of the IEEE.

[47]  Tzyy-Ping Jung,et al.  High-speed spelling with a noninvasive brain–computer interface , 2015, Proceedings of the National Academy of Sciences.

[48]  Hiroki Sato,et al.  Task-related component analysis for functional neuroimaging and application to near-infrared spectroscopy data , 2013, NeuroImage.

[49]  Aravind Ravi,et al.  Comparing user-dependent and user-independent training of CNN for SSVEP BCI , 2020, Journal of neural engineering.

[50]  Yijun Wang,et al.  Enhancing detection of steady-state visual evoked potentials using individual training data , 2014, 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.