Deep Neural Network with Joint Distribution Matching for Cross-Subject Motor Imagery Brain-Computer Interfaces

Motor imagery brain-computer interfaces (BCIs) have demonstrated great potential and attract world-spread attentions. Due to the nonstationary character of the motor imagery signals, costly and boring calibration sessions must be proceeded before use. This prevents them from going into our realistic life. In this paper, the source subject's data are explored to perform calibration for target subjects. Model trained on source subjects is transferred to work for target subjects, in which the critical problem to handle is the distribution shift. It is found that the performance of classification would be bad when only the marginal distributions of source and target are made closer, since the discriminative directions of the source and target domains may still be much different. In order to solve the problem, our idea comes that joint distribution adaptation is indispensable. It makes the classifier trained in the source domain perform well in the target domain. Specifically, a measure for joint distribution discrepancy (JDD) between the source and target is proposed. Experiments demonstrate that it can align source and target data according to the class they belong to. It has a direct relationship with classification accuracy and works well for transferring. Secondly, a deep neural network with joint distribution matching for zero-training motor imagery BCI is proposed. It explores both marginal and joint distribution adaptation to alleviate distribution discrepancy across subjects and obtain effective and generalized features in an aligned common space. Visualizations of intermediate layers illustrate how and why the network works well. Experiments on the two datasets prove the effectiveness and strength compared to outstanding counterparts.

[1]  Ugur Halici,et al.  A novel deep learning approach for classification of EEG motor imagery signals , 2017, Journal of neural engineering.

[2]  N. Birbaumer,et al.  Brain–computer interfaces for communication and rehabilitation , 2016, Nature Reviews Neurology.

[3]  Jian Shen,et al.  Adversarial Representation Learning for Domain Adaptation , 2017, ArXiv.

[4]  Ivor W. Tsang,et al.  Domain Adaptation via Transfer Component Analysis , 2009, IEEE Transactions on Neural Networks.

[5]  Yan Wu,et al.  Convolutional deep belief networks for feature extraction of EEG signal , 2014, 2014 International Joint Conference on Neural Networks (IJCNN).

[6]  Fuchun Sun,et al.  Deep Transfer Learning for EEG-Based Brain Computer Interface , 2018, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[7]  Christian Jutten,et al.  Transfer Learning: A Riemannian Geometry Framework With Applications to Brain–Computer Interfaces , 2018, IEEE Transactions on Biomedical Engineering.

[8]  Wolfram Burgard,et al.  Deep learning with convolutional neural networks for EEG decoding and visualization , 2017, Human brain mapping.

[9]  Jieyu Zhao,et al.  Transferring Common Spatial Filters With Semi-Supervised Learning for Zero-Training Motor Imagery Brain-Computer Interface , 2019, IEEE Access.

[10]  Wojciech Samek,et al.  Transferring Subspaces Between Subjects in Brain--Computer Interfacing , 2012, IEEE Transactions on Biomedical Engineering.

[11]  Abdellah Adib,et al.  Cross-Subject EEG Signal Classification with Deep Neural Networks Applied to Motor Imagery , 2019, MSPN.

[12]  Yuchen Zhang,et al.  Bridging Theory and Algorithm for Domain Adaptation , 2019, ICML.

[13]  Feng Duan,et al.  A Novel Deep Learning Approach With Data Augmentation to Classify Motor Imagery Signals , 2019, IEEE Access.

[14]  Shuicheng Yan,et al.  Learning Temporal Information for Brain-Computer Interface Using Convolutional Neural Networks , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[15]  Sung Chan Jun,et al.  EEG datasets for motor imagery brain–computer interface , 2017, GigaScience.

[16]  Yufei Huang,et al.  A Deep Learning method for classification of images RSVP events with EEG data , 2013, 2013 IEEE Global Conference on Signal and Information Processing.

[17]  Gunnar Rätsch,et al.  Kernel PCA and De-Noising in Feature Spaces , 1998, NIPS.

[18]  Sergio Cruces,et al.  EEG Signal Processing in MI-BCI Applications With Improved Covariance Matrix Estimators , 2019, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[19]  Bernhard Schölkopf,et al.  Transfer Learning in Brain-Computer Interfaces , 2015, IEEE Computational Intelligence Magazine.

[20]  Justin A. Blanco,et al.  Modeling electroencephalography waveforms with semi-supervised deep belief nets: fast classification and anomaly measurement , 2011, Journal of neural engineering.

[21]  Aimin Jiang,et al.  LSTM-Based EEG Classification in Motor Imagery Tasks , 2018, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[22]  Omid Dehzangi,et al.  Portable Brain-Computer Interface for the Intensive Care Unit Patient Communication Using Subject-Dependent SSVEP Identification , 2018, BioMed research international.

[23]  Philip S. Yu,et al.  Adaptation Regularization: A General Framework for Transfer Learning , 2014, IEEE Transactions on Knowledge and Data Engineering.

[24]  Klaus-Robert Müller,et al.  The non-invasive Berlin Brain–Computer Interface: Fast acquisition of effective performance in untrained subjects , 2007, NeuroImage.

[25]  Xiaomu Song,et al.  Improving brain-computer interface classification using adaptive common spatial patterns , 2015, Comput. Biol. Medicine.

[26]  Sang-Hoon Park,et al.  Small Sample Setting and Frequency Band Selection Problem Solving Using Subband Regularized Common Spatial Pattern , 2017, IEEE Sensors Journal.

[27]  Hans-Peter Kriegel,et al.  Integrating structured biological data by Kernel Maximum Mean Discrepancy , 2006, ISMB.

[28]  Yue Cao,et al.  Transferable Representation Learning with Deep Adaptation Networks , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Yoshua Bengio,et al.  Adversarial Domain Adaptation for Stable Brain-Machine Interfaces , 2018, ICLR.

[30]  Shiliang Sun,et al.  A subject transfer framework for EEG classification , 2012, Neurocomputing.

[31]  Christian Jutten,et al.  Riemannian Procrustes Analysis: Transfer Learning for Brain–Computer Interfaces , 2019, IEEE Transactions on Biomedical Engineering.

[32]  Korris Fu-Lai Chung,et al.  On minimum distribution discrepancy support vector machine for domain adaptation , 2012, Pattern Recognit..

[33]  Na Lu,et al.  A Deep Learning Scheme for Motor Imagery Classification based on Restricted Boltzmann Machines , 2017, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[34]  Trevor Darrell,et al.  Deep Domain Confusion: Maximizing for Domain Invariance , 2014, CVPR 2014.

[35]  Mohammed Yeasin,et al.  Learning Representations from EEG with Deep Recurrent-Convolutional Neural Networks , 2015, ICLR.

[36]  Hubert Cecotti,et al.  Convolutional Neural Networks for P300 Detection with Application to Brain-Computer Interfaces , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  Dean J. Krusienski,et al.  Chapter 11 Brain–Computer Interface Research at the Wadsworth Center , 2009 .

[38]  K.-R. Muller,et al.  Optimizing Spatial filters for Robust EEG Single-Trial Analysis , 2008, IEEE Signal Processing Magazine.

[39]  Cuntai Guan,et al.  Regularizing Common Spatial Patterns to Improve BCI Designs: Unified Theory and New Algorithms , 2011, IEEE Transactions on Biomedical Engineering.

[40]  Cuntai Guan,et al.  Inter-subject transfer learning with an end-to-end deep convolutional neural network for EEG-based BCI , 2019, Journal of neural engineering.

[41]  Yoshua Bengio,et al.  How transferable are features in deep neural networks? , 2014, NIPS.

[42]  Shouqian Sun,et al.  Single-trial EEG classification of motor imagery using deep convolutional neural networks , 2017 .

[43]  David Lee,et al.  Filter Bank Regularized Common Spatial Pattern Ensemble for Small Sample Motor Imagery Classification , 2018, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[44]  Sadasivan Puthusserypady,et al.  An end-to-end deep learning approach to MI-EEG signal classification for BCIs , 2018, Expert Syst. Appl..

[45]  R. Leeb,et al.  BCI Competition 2008 { Graz data set B , 2008 .

[46]  Javaid Iqbal,et al.  Design of Embedded System for Multivariate Classification of Finger and Thumb Movements Using EEG Signals for Control of Upper Limb Prosthesis , 2018, BioMed research international.

[47]  Brendan Z. Allison,et al.  P300 brain computer interface: current challenges and emerging trends , 2012, Front. Neuroeng..