Toward Subject Invariant and Class Disentangled Representation in BCI via Cross-Domain Mutual Information Estimator

In recent years, deep learning-based feature representation methods have shown a promising impact in electroencephalography (EEG)-based brain-computer interface (BCI). Nonetheless, owing to high intra- and inter-subject variabilities, many studies on decoding EEG were designed in a subject-specific manner by using calibration samples, with no concern of its practical use, hampered by time-consuming steps and a large data requirement. To this end, recent studies adopted a transfer learning strategy, especially domain adaptation techniques. Among those, to our knowledge, an adversarial learning has shown its potential in BCIs. In the meantime, it is known that adversarial learning-based domain adaptation methods are prone to negative transfer that disrupts learning generalized feature representations, applicable to diverse domains, e.g., subjects or sessions in BCIs. In this paper, we propose a novel framework that learns class-relevant and subject-invariant feature representations in an information-theoretic manner, without using adversarial learning. To be specific, we devise two operational components in a deep network that explicitly estimate mutual information between feature representations; (1) to decompose features in an intermediate layer into class-relevant and class-irrelevant ones, (2) to enrich class-discriminative feature representation. On two large EEG datasets, we validated the effectiveness of our proposed framework by comparing with several comparative methods in performance. Further, we conducted rigorous analyses by performing an ablation study in regard to the components in our network, explaining our model's decision on input EEG signals via layer-wise relevance propagation, and visualizing the distribution of learned features via t-SNE.

[1]  Alexander Binder,et al.  Explaining nonlinear classification decisions with deep Taylor decomposition , 2015, Pattern Recognit..

[2]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[3]  Kate Saenko,et al.  Domain Agnostic Learning with Disentangled Representations , 2019, ICML.

[4]  John Williamson,et al.  EEG dataset and OpenBMI toolbox for three BCI paradigms: an investigation into BCI illiteracy , 2019, GigaScience.

[5]  Seungjin Choi,et al.  Composite Common Spatial Pattern for Subject-to-Subject Transfer , 2009, IEEE Signal Processing Letters.

[6]  Michael I. Jordan,et al.  Transferable Adversarial Training: A General Approach to Adapting Deep Classifiers , 2019, ICML.

[7]  Liyuan Liu,et al.  On the Variance of the Adaptive Learning Rate and Beyond , 2019, ICLR.

[8]  Heung-Il Suk,et al.  A Plug-in Method for Representation Factorization , 2019 .

[9]  Heung-Il Suk,et al.  A Novel Bayesian Framework for Discriminative Feature Extraction in Brain-Computer Interfaces , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Ye Wang,et al.  Transfer Learning in Brain-Computer Interfaces with Adversarial Variational Autoencoders , 2019, 2019 9th International IEEE/EMBS Conference on Neural Engineering (NER).

[11]  Heung-Il Suk,et al.  Domain Adaptation with Source Selection for Motor-Imagery based BCI , 2019, 2019 7th International Winter Conference on Brain-Computer Interface (BCI).

[12]  Yoshua Bengio,et al.  Mutual Information Neural Estimation , 2018, ICML.

[13]  Joaquin Vanschoren,et al.  Meta-Learning: A Survey , 2018, Automated Machine Learning.

[14]  Dong Xu,et al.  Collaborative and Adversarial Network for Unsupervised Domain Adaptation , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[15]  Sebastian Nowozin,et al.  f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization , 2016, NIPS.

[16]  José M. F. Moura,et al.  Adversarial Multiple Source Domain Adaptation , 2018, NeurIPS.

[17]  François Laviolette,et al.  Domain-Adversarial Training of Neural Networks , 2015, J. Mach. Learn. Res..

[18]  Ye Wang,et al.  Learning Invariant Representations From EEG via Adversarial Inference , 2020, IEEE Access.

[19]  Yuan-Pin Lin,et al.  A subject-transfer framework for obviating inter- and intra-subject variability in EEG-based drowsiness detection , 2018, NeuroImage.

[20]  Minkyu Ahn,et al.  Journal of Neuroscience Methods , 2015 .

[21]  Li Sun,et al.  Disentangling Latent Space for VAE by Label Relevant/Irrelevant Dimensions , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Yoshua Bengio,et al.  Learning deep representations by mutual information estimation and maximization , 2018, ICLR.

[23]  Klaus-Robert Müller,et al.  Integrating dynamic stopping, transfer learning and language models in an adaptive zero-training ERP speller , 2014, Journal of neural engineering.

[24]  Wolfram Burgard,et al.  Deep learning with convolutional neural networks for EEG decoding and visualization , 2017, Human brain mapping.

[25]  François Chollet,et al.  Xception: Deep Learning with Depthwise Separable Convolutions , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Michael C. Mozer,et al.  Learning Deep Disentangled Embeddings with the F-Statistic Loss , 2018, NeurIPS.

[27]  Brendan Z. Allison,et al.  Brain-Computer Interfaces: A Gentle Introduction , 2009 .

[28]  Dumitru Erhan,et al.  Unsupervised Pixel-Level Domain Adaptation with Generative Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Heung-Il Suk,et al.  Deep recurrent spatio-temporal neural network for motor imagery based BCI , 2018, 2018 6th International Conference on Brain-Computer Interface (BCI).

[30]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[31]  Sung Chan Jun,et al.  EEG datasets for motor imagery brain–computer interface , 2017, GigaScience.

[32]  Bernhard Schölkopf,et al.  Transfer Learning in Brain-Computer Interfaces , 2015, IEEE Computational Intelligence Magazine.

[33]  Motoaki Kawanabe,et al.  Toward Unsupervised Adaptation of LDA for Brain–Computer Interfaces , 2011, IEEE Transactions on Biomedical Engineering.

[34]  Qisong Wang,et al.  Unsupervised domain adaptation techniques based on auto-encoder for non-stationary EEG-based emotion recognition , 2016, Comput. Biol. Medicine.

[35]  Mei Wang,et al.  Deep Visual Domain Adaptation: A Survey , 2018, Neurocomputing.

[36]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[37]  Liang Lin,et al.  Deep Cocktail Network: Multi-source Unsupervised Domain Adaptation with Category Shift , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[38]  Mahnaz Arvaneh,et al.  Weighted Transfer Learning for Improving Motor Imagery-Based Brain–Computer Interface , 2019, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[39]  Trevor Darrell,et al.  Adversarial Discriminative Domain Adaptation , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[40]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[41]  Cuntai Guan,et al.  Regularizing Common Spatial Patterns to Improve BCI Designs: Unified Theory and New Algorithms , 2011, IEEE Transactions on Biomedical Engineering.

[42]  Shuicheng Yan,et al.  Learning Temporal Information for Brain-Computer Interface Using Convolutional Neural Networks , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[43]  Jianmin Wang,et al.  Transferable Attention for Domain Adaptation , 2019, AAAI.

[44]  S. Varadhan,et al.  Asymptotic evaluation of certain Markov process expectations for large time , 1975 .

[45]  Sheng-De Wang,et al.  Detach and Adapt: Learning Cross-Domain Disentangled Deep Representation , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[46]  T. Jung,et al.  Improving EEG-Based Emotion Classification Using Conditional Transfer Learning , 2017, Front. Hum. Neurosci..

[47]  Roger B. Grosse,et al.  Isolating Sources of Disentanglement in Variational Autoencoders , 2018, NeurIPS.

[48]  Alexander Binder,et al.  On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation , 2015, PloS one.

[49]  P. Welch The use of fast Fourier transform for the estimation of power spectra: A method based on time averaging over short, modified periodograms , 1967 .

[50]  G. Pfurtscheller,et al.  Optimal spatial filtering of single trial EEG during imagined hand movement. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[51]  Cuntai Guan,et al.  Filter Bank Common Spatial Pattern (FBCSP) in Brain-Computer Interface , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[52]  Michael I. Jordan,et al.  Learning Transferable Features with Deep Adaptation Networks , 2015, ICML.