MIN2Net: End-to-End Multi-Task Learning for Subject-Independent Motor Imagery EEG Classification

\textit{Objective:} Advances in the motor imagery (MI)-based brain-computer interfaces (BCIs) allow control of several applications by decoding neurophysiological phenomena, which are usually recorded by electroencephalography (EEG) using a non-invasive technique. Despite significant advances in MI-based BCI, EEG rhythms are specific to a subject and various changes over time. These issues point to significant challenges to enhance the classification performance, especially in a subject-independent manner. \textit{Methods:} To overcome these challenges, we propose MIN2Net, a novel end-to-end multi-task learning to tackle this task. We integrate deep metric learning into a multi-task autoencoder to learn a compact and discriminative latent representation from EEG and perform classification simultaneously. \textit{Results:} This approach reduces the complexity in pre-processing, results in significant performance improvement on EEG classification. Experimental results in a subject-independent manner show that MIN2Net outperforms the state-of-the-art techniques, achieving an F1-score improvement of 6.72\%, and 2.23\% on the SMR-BCI, and OpenBMI datasets, respectively. \textit{Conclusion:} We demonstrate that MIN2Net improves discriminative information in the latent representation. \textit{Significance:} This study indicates the possibility and practicality of using this model to develop MI-based BCI applications for new users without calibration.

[1]  James Philbin,et al.  FaceNet: A unified embedding for face recognition and clustering , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Amit Konar,et al.  A Generic Transferable EEG Decoder for Online Detection of Error Potential in Target Selection , 2017, Front. Neurosci..

[3]  F. L. D. Silva,et al.  Event-related EEG/MEG synchronization and desynchronization: basic principles , 1999, Clinical Neurophysiology.

[4]  Yu Zhang,et al.  Sparse Group Representation Model for Motor Imagery EEG Classification , 2019, IEEE Journal of Biomedical and Health Informatics.

[5]  Ad Aertsen,et al.  Review of the BCI Competition IV , 2012, Front. Neurosci..

[6]  Ekapol Chuangsuwanich,et al.  Universal Joint Feature Extraction for P300 EEG Classification Using Multi-Task Autoencoder , 2018, IEEE Access.

[7]  Pascal Vincent,et al.  Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion , 2010, J. Mach. Learn. Res..

[8]  Weidong Zhou,et al.  Denoising Sparse Autoencoder-Based Ictal EEG Classification , 2018, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[9]  Reinhold Scherer,et al.  Motor Imagery Brain-Computer Interfaces: Random Forests vs Regularized LDA - Non-linear Beats Linear , 2014 .

[10]  Shuicheng Yan,et al.  Learning Temporal Information for Brain-Computer Interface Using Convolutional Neural Networks , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[11]  P. Tonella,et al.  EEG data compression techniques , 1997, IEEE Transactions on Biomedical Engineering.

[12]  G. Pfurtscheller,et al.  Optimal spatial filtering of single trial EEG during imagined hand movement. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[13]  Yang Hua,et al.  Ranked List Loss for Deep Metric Learning , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Martin Luessi,et al.  MEG and EEG data analysis with MNE-Python , 2013, Front. Neuroinform..

[15]  Weilin Huang,et al.  Deep Metric Learning with Hierarchical Triplet Loss , 2018, ECCV.

[16]  Seong-Whan Lee,et al.  Adaptive transfer learning for EEG motor imagery classification with deep Convolutional Neural Network , 2020, Neural Networks.

[17]  Miguel P. Eckstein,et al.  Single-Trial Classification of Event-Related Potentials in Rapid Serial Visual Presentation Tasks Using Supervised Spatial Filtering , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[18]  Wei Chen,et al.  MetaSleepLearner: A Pilot Study on Fast Adaptation of Bio-Signals-Based Sleep Stage Classifier to New Individual Subject Using Meta-Learning , 2020, IEEE Journal of Biomedical and Health Informatics.

[19]  Ji-Hoon Jeong,et al.  Decoding Movement-Related Cortical Potentials Based on Subject-Dependent and Section-Wise Spectral Filtering , 2020, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[20]  Heung-Il Suk,et al.  A Novel Bayesian Framework for Discriminative Feature Extraction in Brain-Computer Interfaces , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Wolfram Burgard,et al.  Deep learning with convolutional neural networks for EEG decoding and visualization , 2017, Human brain mapping.

[22]  Yunhao Liu,et al.  Making Sense of Spatio-Temporal Preserving Representations for EEG-Based Human Intention Recognition , 2020, IEEE Transactions on Cybernetics.

[23]  S. Keene,et al.  A learned embedding space for EEG signal clustering , 2017, 2017 IEEE Signal Processing in Medicine and Biology Symposium (SPMB).

[24]  Lina Yao,et al.  Motor Imagery Classification via Temporal Attention Cues of Graph Embedded EEG Signals , 2020, IEEE Journal of Biomedical and Health Informatics.

[25]  Yaohua Wang,et al.  Hierarchical Clustering With Hard-Batch Triplet Loss for Person Re-Identification , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Xinbo Gao,et al.  Deep Convolutional Neural Networks for mental load classification based on EEG data , 2018, Pattern Recognit..

[27]  Junfeng Chen,et al.  Learning Spatial–Spectral–Temporal EEG Features With Recurrent 3D Convolutional Neural Networks for Cross-Task Mental Workload Assessment , 2019, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[28]  Kaiqi Huang,et al.  Beyond Triplet Loss: A Deep Quadruplet Network for Person Re-identification , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Stephen M. Gordon,et al.  EEGNet: A Compact Convolutional Neural Network for EEG-based Brain-Computer Interfaces , 2021 .

[30]  Lina Yao,et al.  Multi-View Multi-Scale Optimization of Feature Representation for EEG Classification Improvement , 2020, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[31]  Andrzej Cichocki,et al.  Improving EEG Decoding via Clustering-Based Multitask Feature Learning , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[32]  Theerawit Wilaiprasitporn,et al.  Consumer Grade EEG Measuring Sensors as Research Tools: A Review , 2020, IEEE Sensors Journal.

[33]  Cuntai Guan,et al.  EEG-Based Strategies to Detect Motor Imagery for Control and Rehabilitation , 2017, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[34]  Poramate Manoonpong,et al.  A Single-Channel Consumer-Grade EEG Device for Brain–Computer Interface: Enhancing Detection of SSVEP and Its Amplitude Modulation , 2018, IEEE Sensors Journal.

[35]  Dana Kulic,et al.  Data augmentation of wearable sensor data for parkinson’s disease monitoring using convolutional neural networks , 2017, ICMI.

[36]  Ce Liu,et al.  Supervised Contrastive Learning , 2020, NeurIPS.

[37]  Theerawit Wilaiprasitporn,et al.  Affective EEG-Based Person Identification Using the Deep Learning Approach , 2018, IEEE Transactions on Cognitive and Developmental Systems.

[38]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[39]  Jiankun Hu,et al.  Deep Gaussian Mixture-Hidden Markov Model for Classification of EEG Signals , 2018, IEEE Transactions on Emerging Topics in Computational Intelligence.

[40]  Roozbeh Jafari,et al.  Automatic Identification of Artifact-Related Independent Components for Artifact Removal in EEG Recordings , 2016, IEEE Journal of Biomedical and Health Informatics.

[41]  Seong-Whan Lee,et al.  Subject-Independent Brain–Computer Interfaces Based on Deep Convolutional Neural Networks , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[42]  Quoc V. Le,et al.  Self-Training With Noisy Student Improves ImageNet Classification , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[43]  Mohd Zuki Yusoff,et al.  Motor Imagery Classification for Brain Computer Interface Using Deep Metric Learning , 2020, IEEE Access.

[44]  Cuntai Guan,et al.  Regularizing Common Spatial Patterns to Improve BCI Designs: Unified Theory and New Algorithms , 2011, IEEE Transactions on Biomedical Engineering.

[45]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[46]  D.J. McFarland,et al.  Sensorimotor rhythm-based brain-computer interface (BCI): feature selection by regression improves performance , 2005, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[47]  Mahnaz Arvaneh,et al.  Weighted Transfer Learning for Improving Motor Imagery-Based Brain–Computer Interface , 2019, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[48]  Kihyuk Sohn,et al.  Improved Deep Metric Learning with Multi-class N-pair Loss Objective , 2016, NIPS.

[49]  Rifai Chai,et al.  Brain–Computer Interface Classifier for Wheelchair Commands Using Neural Network With Fuzzy Particle Swarm Optimization , 2014, IEEE Journal of Biomedical and Health Informatics.

[50]  Gert Pfurtscheller,et al.  Motor imagery and direct brain-computer communication , 2001, Proc. IEEE.

[51]  John Williamson,et al.  EEG dataset and OpenBMI toolbox for three BCI paradigms: an investigation into BCI illiteracy , 2019, GigaScience.

[52]  Rabab Kreidieh Ward,et al.  Semi-supervised Stacked Label Consistent Autoencoder for Reconstruction and Analysis of Biomedical Signals , 2017, IEEE Transactions on Biomedical Engineering.

[53]  Cuntai Guan,et al.  Filter Bank Common Spatial Pattern (FBCSP) in Brain-Computer Interface , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[54]  Cuntai Guan,et al.  Filter Bank Common Spatial Pattern Algorithm on BCI Competition IV Datasets 2a and 2b , 2012, Front. Neurosci..

[55]  J R Wolpaw,et al.  EEG-Based Brain-Computer Interfaces. , 2017, Current opinion in biomedical engineering.

[56]  Vicenç Gómez,et al.  Adaptive Multiclass Classification for Brain Computer Interfaces , 2014, Neural Computation.

[57]  Klaus-Robert Müller,et al.  Spatio-spectral filters for improving the classification of single trial EEG , 2005, IEEE Transactions on Biomedical Engineering.

[58]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .