Transfer Learning for Brain-Computer Interfaces: A Complete Pipeline

Transfer learning (TL) has been widely used in electroencephalogram (EEG) based brain-computer interfaces (BCIs) to reduce the calibration effort for a new subject, and demonstrated promising performance. After EEG signal acquisition, a closed-loop EEG-based BCI system also includes signal processing, feature engineering, and classification/regression blocks before sending out the control signal, whereas previous approaches only considered TL in one or two such components. This paper proposes that TL could be considered in all three components (signal processing, feature engineering, and classification/regression). Furthermore, it is also very important to specifically add a data alignment component before signal processing to make the data from different subjects more consistent, and hence to facilitate subsequential TL. Offline calibration experiments on two MI datasets verified our proposal. Especially, integrating data alignment and sophisticated TL approaches can significantly improve the classification performance, and hence greatly reduce the calibration effort.

[1]  T. Lagerlund,et al.  Spatial filtering of multichannel electroencephalographic recordings through principal component analysis by singular value decomposition. , 1997, Journal of clinical neurophysiology : official publication of the American Electroencephalographic Society.

[2]  Dongrui Wu,et al.  Different Set Domain Adaptation for Brain-Computer Interfaces: A Label Alignment Approach , 2020, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[3]  Dongrui Wu,et al.  Online and Offline Domain Adaptation for Reducing BCI Calibration Effort , 2017, IEEE Transactions on Human-Machine Systems.

[4]  G. Pfurtscheller,et al.  Optimal spatial filtering of single trial EEG during imagined hand movement. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[5]  Fabien Lotte,et al.  Brain-Computer Interfaces: Beyond Medical Applications , 2012, Computer.

[6]  Bernhard Schölkopf,et al.  Transfer Learning in Brain-Computer Interfaces , 2015, IEEE Computational Intelligence Magazine.

[7]  Dezhi Zheng,et al.  Transfer Kernel Common Spatial Patterns for Motor Imagery Brain-Computer Interface Classification , 2018, Comput. Math. Methods Medicine.

[8]  Xiaomu Song,et al.  A study of kernel CSP-based motor imagery brain computer interface classification , 2012, 2012 IEEE Signal Processing in Medicine and Biology Symposium (SPMB).

[9]  Christian Jutten,et al.  Riemannian Procrustes Analysis: Transfer Learning for Brain–Computer Interfaces , 2019, IEEE Transactions on Biomedical Engineering.

[10]  Pierre Jallon,et al.  A comparison of ERP spatial filtering methods for optimal mental workload estimation , 2015, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[11]  M. Teplan.  FUNDAMENTALS OF EEG MEASUREMENT , 2002 .

[12]  Haiping Lu,et al.  Regularized Common Spatial Pattern With Aggregation for EEG Classification in Small-Sample Setting , 2010, IEEE Transactions on Biomedical Engineering.

[13]  Arnaud Delorme,et al.  EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis , 2004, Journal of Neuroscience Methods.

[14]  Philip S. Yu,et al.  Adaptation Regularization: A General Framework for Transfer Learning , 2014, IEEE Transactions on Knowledge and Data Engineering.

[15]  Simanto Saha,et al.  Evidence of Variabilities in EEG Dynamics During Motor Imagery-Based Multiclass Brain–Computer Interface , 2018, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[16]  Guillaume Gibert,et al.  xDAWN Algorithm to Enhance Evoked Potentials: Application to Brain–Computer Interface , 2009, IEEE Transactions on Biomedical Engineering.

[17]  Brent Lance,et al.  Switching EEG Headsets Made Easy: Reducing Offline Calibration Effort Using Active Weighted Adaptation Regularization , 2016, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[18]  Heng Tao Shen,et al.  Principal Component Analysis , 2009, Encyclopedia of Biometrics.

[19]  Yifan Xu,et al.  Transfer Learning for EEG-Based Brain–Computer Interfaces: A Review of Progress Made Since 2016 , 2020, IEEE Transactions on Cognitive and Developmental Systems.

[20]  Klaus-Robert Müller,et al.  The non-invasive Berlin Brain–Computer Interface: Fast acquisition of effective performance in untrained subjects , 2007, NeuroImage.

[21]  S. Coyle,et al.  Brain–computer interfaces: a review , 2003 .

[22]  Wolfram Burgard,et al.  Deep learning with convolutional neural networks for EEG decoding and visualization , 2017, Human brain mapping.

[23]  Dennis J. McFarland,et al.  Brain–computer interfaces for communication and control , 2002, Clinical Neurophysiology.

[24]  Scott E. Kerick,et al.  Brain–Computer Interface Technologies in the Coming Decades , 2012, Proceedings of the IEEE.

[25]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[26]  Ao Zhang,et al.  Cross-subject driver status detection from physiological signals based on hybrid feature selection and transfer learning , 2019, Expert Syst. Appl..

[27]  M Congedo,et al.  A review of classification algorithms for EEG-based brain–computer interfaces: a 10 year update , 2018, Journal of neural engineering.

[28]  Igor Kononenko,et al.  Estimating Attributes: Analysis and Extensions of RELIEF , 1994, ECML.

[29]  Philip S. Yu,et al.  Domain Invariant Transfer Kernel Learning , 2015, IEEE Transactions on Knowledge and Data Engineering.

[30]  Fabien Lotte,et al.  Signal Processing Approaches to Minimize or Suppress Calibration Time in Oscillatory Activity-Based Brain–Computer Interfaces , 2015, Proceedings of the IEEE.

[31]  Christa Neuper,et al.  Rehabilitation with Brain-Computer Interface Systems , 2008, Computer.

[32]  Dongrui Wu,et al.  Manifold Embedded Knowledge Transfer for Brain-Computer Interfaces , 2020, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[33]  K.-R. Muller,et al.  Optimizing Spatial filters for Robust EEG Single-Trial Analysis , 2008, IEEE Signal Processing Magazine.

[34]  Dongrui Wu,et al.  Transfer Learning for Brain–Computer Interfaces: A Euclidean Space Data Alignment Approach , 2018, IEEE Transactions on Biomedical Engineering.

[35]  Tzyy-Ping Jung,et al.  Spatial Filtering for EEG-Based Regression Problems in Brain–Computer Interface (BCI) , 2017, IEEE Transactions on Fuzzy Systems.

[36]  Christian Jutten,et al.  Transfer Learning: A Riemannian Geometry Framework With Applications to Brain–Computer Interfaces , 2018, IEEE Transactions on Biomedical Engineering.