Deep Representation-Based Domain Adaptation for Nonstationary EEG Classification

In the context of motor imagery, electroencephalography (EEG) data vary from subject to subject such that the performance of a classifier trained on data of multiple subjects from a specific domain typically degrades when applied to a different subject. While collecting enough samples from each subject would address this issue, it is often too time-consuming and impractical. To tackle this problem, we propose a novel end-to-end deep domain adaptation method to improve the classification performance on a single subject (target domain) by taking the useful information from multiple subjects (source domain) into consideration. Especially, the proposed method jointly optimizes three modules, including a feature extractor, a classifier, and a domain discriminator. The feature extractor learns the discriminative latent features by mapping the raw EEG signals into a deep representation space. A center loss is further employed to constrain an invariant feature space and reduce the intrasubject nonstationarity. Furthermore, the domain discriminator matches the feature distribution shift between source and target domains by an adversarial learning strategy. Finally, based on the consistent deep features from both domains, the classifier is able to leverage the information from the source domain and accurately predict the label in the target domain at the test time. To evaluate our method, we have conducted extensive experiments on two real public EEG data sets, data set IIa, and data set IIb of brain-computer interface (BCI) Competition IV. The experimental results validate the efficacy of our method. Therefore, our method is promising to reduce the calibration time for the use of BCI and promote the development of BCI.

[1]  Xiaomu Song,et al.  Adaptive Common Spatial Pattern for single-trial EEG classification in multisubject BCI , 2013, 2013 6th International IEEE/EMBS Conference on Neural Engineering (NER).

[2]  Pascal Fua,et al.  Beyond Sharing Weights for Deep Domain Adaptation , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Tatsuhiko Tsunoda,et al.  A Deep Learning Approach for Motor Imagery EEG Signal Classification , 2016, 2016 3rd Asia-Pacific World Congress on Computer Science and Engineering (APWC on CSE).

[4]  Cuntai Guan,et al.  On the use of convolutional neural networks and augmented CSP features for multi-class motor imagery of EEG signals classification , 2015, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[5]  Klaus-Robert Müller,et al.  Combined Optimization of Spatial and Temporal Filters for Improving Brain-Computer Interfacing , 2006, IEEE Transactions on Biomedical Engineering.

[6]  Yu Qiao,et al.  A Discriminative Feature Learning Approach for Deep Face Recognition , 2016, ECCV.

[7]  Zhihua Zhang,et al.  Support Matrix Machines , 2015, ICML.

[8]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[9]  Shouqian Sun,et al.  Single-trial EEG classification of motor imagery using deep convolutional neural networks , 2017 .

[10]  Tong Zhang,et al.  A Novel Neural Network Model based on Cerebral Hemispheric Asymmetry for EEG Emotion Recognition , 2018, IJCAI.

[11]  Kerwin J. F. Olfers,et al.  Game-based training of flexibility and attention improves task-switch performance: near and far transfer of cognitive training in an EEG study , 2017, Psychological Research.

[12]  Lexin Li,et al.  Regularized matrix regression , 2012, Journal of the Royal Statistical Society. Series B, Statistical methodology.

[13]  Sadasivan Puthusserypady,et al.  An end-to-end deep learning approach to MI-EEG signal classification for BCIs , 2018, Expert Syst. Appl..

[14]  M Congedo,et al.  A review of classification algorithms for EEG-based brain–computer interfaces , 2007, Journal of neural engineering.

[15]  Chiew Tong Lau,et al.  A New Discriminative Common Spatial Pattern Method for Motor Imagery Brain–Computer Interfaces , 2009, IEEE Transactions on Biomedical Engineering.

[16]  Cuntai Guan,et al.  Convolutional neural network-based transfer learning and knowledge distillation using multi-subject data in motor imagery BCI , 2017, 2017 8th International IEEE/EMBS Conference on Neural Engineering (NER).

[17]  Mohammed Yeasin,et al.  Learning Representations from EEG with Deep Recurrent-Convolutional Neural Networks , 2015, ICLR.

[18]  Yongtian He,et al.  Deep learning for electroencephalogram (EEG) classification tasks: a review , 2019, Journal of neural engineering.

[19]  Cuntai Guan,et al.  Bayesian Learning for Spatial Filtering in an EEG-Based Brain–Computer Interface , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[20]  Cuntai Guan,et al.  A Large Clinical Study on the Ability of Stroke Patients to Use an EEG-Based Motor Imagery Brain-Computer Interface , 2011, Clinical EEG and neuroscience.

[21]  Klaus-Robert Müller,et al.  Spatio-spectral filters for improving the classification of single trial EEG , 2005, IEEE Transactions on Biomedical Engineering.

[22]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[23]  L. Cohen,et al.  Brain–computer interfaces: communication and restoration of movement in paralysis , 2007, The Journal of physiology.

[24]  Bo Hu,et al.  A hierarchical semi-supervised extreme learning machine method for EEG recognition , 2018, Medical & Biological Engineering & Computing.

[25]  Seungjin Choi,et al.  Composite Common Spatial Pattern for Subject-to-Subject Transfer , 2009, IEEE Signal Processing Letters.

[26]  Rabab K Ward,et al.  User-customized brain computer interfaces using Bayesian optimization , 2016, Journal of neural engineering.

[27]  Leon A. Gatys,et al.  Image Style Transfer Using Convolutional Neural Networks , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Dan Liu,et al.  A Fast, Efficient Domain Adaptation Technique for Cross-Domain Electroencephalography(EEG)-Based Emotion Recognition , 2017, Sensors.

[29]  Ugur Halici,et al.  A novel deep learning approach for classification of EEG motor imagery signals , 2017, Journal of neural engineering.

[30]  Cuntai Guan,et al.  Filter Bank Common Spatial Pattern (FBCSP) in Brain-Computer Interface , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[31]  Cuntai Guan,et al.  Filter Bank Common Spatial Pattern Algorithm on BCI Competition IV Datasets 2a and 2b , 2012, Front. Neurosci..

[32]  Raymond Y. K. Lau,et al.  Least Squares Generative Adversarial Networks , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[33]  Lei Cao,et al.  Short time Fourier transformation and deep neural networks for motor imagery brain computer interface recognition , 2018, Concurr. Comput. Pract. Exp..

[34]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  Wolfram Burgard,et al.  Deep learning with convolutional neural networks for EEG decoding and visualization , 2017, Human brain mapping.

[36]  Shuicheng Yan,et al.  Learning Temporal Information for Brain-Computer Interface Using Convolutional Neural Networks , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[37]  Erik Cambria,et al.  Recent Trends in Deep Learning Based Natural Language Processing , 2017, IEEE Comput. Intell. Mag..

[38]  Mahnaz Arvaneh,et al.  Weighted Transfer Learning for Improving Motor Imagery-Based Brain–Computer Interface , 2019, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[39]  Lucas C. Parra,et al.  Bilinear Discriminant Component Analysis , 2007, J. Mach. Learn. Res..

[40]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[41]  Ad Aertsen,et al.  Review of the BCI Competition IV , 2012, Front. Neurosci..

[42]  Girijesh Prasad,et al.  Adaptive learning with covariate shift-detection for motor imagery-based brain–computer interface , 2015, Soft Computing.

[43]  Pheng-Ann Heng,et al.  Sparse Support Matrix Machine , 2018, Pattern Recognit..

[44]  Dongrui Wu,et al.  Transfer Learning for Brain–Computer Interfaces: A Euclidean Space Data Alignment Approach , 2018, IEEE Transactions on Biomedical Engineering.

[45]  Wojciech Samek,et al.  Transferring Subspaces Between Subjects in Brain--Computer Interfacing , 2012, IEEE Transactions on Biomedical Engineering.

[46]  Victor S. Lempitsky,et al.  Unsupervised Domain Adaptation by Backpropagation , 2014, ICML.

[47]  Heung-Il Suk,et al.  Domain Adaptation with Source Selection for Motor-Imagery based BCI , 2019, 2019 7th International Winter Conference on Brain-Computer Interface (BCI).